Caching overview

A cacheable response is an HTTP response that Cloud CDN can store and quickly retrieve, thus allowing for faster load times. Not all HTTP responses are cacheable. Cloud CDN uses HTTP response headers that your origin server sends to determine how to cache content at the edge. You can use standard Cache-Control headers to manage caching universally, or use targeted CDN-Cache-Control headers to apply specific caching rules to Cloud CDN without affecting how end-user browsers cache your content.

Cache modes

With cache modes, you can control the factors that determine whether Cloud CDN caches your content.

Cloud CDN offers three cache modes, which define how responses are cached, whether Cloud CDN respects cache directives sent by the origin, and how cache TTLs are applied.

The available cache modes are shown in the following table:

Error responses may be cached even in the absence of valid cache directives.

Before you set the cache mode to FORCE_CACHE_ALL, consider the following behaviors:

• For signed URLs or signed cookies, FORCE_CACHE_ALL overrides the maximum age specified through the Cache entry maximum age setting in the Google Cloud console or the gcloud --signed-url-cache-max-age option.

• FORCE_CACHE_ALL changes the time to live (TTL) of any previously cached content. This change can cause some entries that were previously considered fresh (due to having longer TTLs from origin headers) to be considered stale, and it can cause some entries that were previously considered stale to be considered fresh.

• FORCE_CACHE_ALL overrides cache directives (Cache-Control and Expires) but does not override other origin response headers. In particular, a Vary header might suppress caching even if the cache mode is FORCE_CACHE_ALL. For more information, see Vary headers.

For setup instructions, see Setting the cache mode.

Static content

Static content is content that is always the same, even when accessed by different users. The CSS that you use to style your site, JavaScript to provide interactivity, video, and image content typically don't change for each user for a given URL (cache key), and thus benefit from being cached across Cloud CDN's global edge network.

When you set the cache mode to CACHE_ALL_STATIC, and a response does not have explicit caching directives in Cache-Control or Expires headers, Cloud CDN automatically caches that response for the following:

• Web Assets, including CSS (text/css), JavaScript (application/javascript) and all web fonts, including WOFF2 (font/woff2)
• Images, including JPEG (image/jpg) and PNG (image/png)
• Videos, including H.264, H.265, and MP4 (video/mp4)
• Audio files, including MP3 (audio/mpeg) and MP4 (audio/mp4)
• Formatted documents, including PDF (application/pdf)

The following table provides a summary.

Cloud CDN inspects the Content-Type HTTP response header, which reflects the MIME type of the content being served.

Note the following:

• Your origin's web server software must set the Content-Type for each response. Many web servers automatically set the Content-Type header, including NGINX, Varnish, and Apache.

• Cloud Storage sets the Content-Type header automatically when you use the Google Cloud console or the Google Cloud CLI to upload content.

• Cloud Storage always provides a Cache-Control header to Cloud CDN. If no value is explicitly chosen, it sends a default value. As a result, all successful Cloud Storage responses are cached according to Cloud Storage default values, unless you explicitly adjust the cache control metadata for objects in Cloud Storage or use FORCE_CACHE_ALL mode to override the values sent by Cloud Storage.

• If you want to cache text/html and application/json content types, you must set explicit Cache-Control headers in the response, being careful not to accidentally cache one user's data and serve it to all users.

If a response is cacheable based on its MIME type but has a Cache-Control response header of private or no-store, or a Set-Cookie header, it isn't cached. To learn more, see cacheability rules.

Other content types, such as HTML (text/html) and JSON (application/json), are not cached by default for successful responses. These types of responses are typically dynamic (per user). Examples include shopping carts, product pages with user personalization, and authenticated API responses. Negative caching, if enabled, can still cause them to be cached for certain status codes, however.

Cloud CDN doesn't use file extensions in the URL path to determine whether a response is cacheable because many valid cacheable responses aren't reflected in URLs.

Cache policy configuration methods

Depending on how much control you need over caching behavior, you can configure caching behavior for Cloud CDN at the backend service, backend bucket, or at a more granular level in URL maps.

Backend service or backend bucket cache policy

Configure a cache policy on a backend service or backend bucket to apply a single caching policy to all requests that are routed to that backend.

Cache policies in URL maps

You can configure Cloud CDN cache policies at various levels of the URL map. This provides granular control over caching policies based on criteria like hostname, URL path, HTTP headers, and query parameters for specific routes.

Setting cache policies at different levels of the URL map, such as root, path matchers, path rules, and route rules, gives you granular control over caching different types of content served by the same backend.

For example, you can configure path rules to:

• Cache static images in the /images/* path for 24 hours.
• Cache HTML pages in the /pages/* path for 5 minutes.

You can configure cache policies in URL maps when:

• a single backend serves different types of content
• different paths require different caching behavior
• you want to enable caching for specific routes only

For details on how to configure cache policies in URL map, see Configure a Cloud CDN cache policy.

Default values for caching

For caching parameters, Cloud CDN uses the following default values:

GKE controller feature support

The following table compares the availability of specific Cloud CDN features when managed through the GKE Ingress controller versus the GKE Gateway controller.

Cacheable content

Cloud CDN caches responses that meet all of the requirements in this section. Some of these requirements are specified by RFC 7234, and others are specific to Cloud CDN.

Cloud CDN may periodically change the exact set of conditions under which it caches content. If you want to explicitly prevent Cloud CDN from caching your content, follow the guidelines in RFC 7234 to determine how to specify a guaranteed-uncacheable response. See also the non-cacheable content based on origin headers section.

Cloud CDN stores responses in cache if all of the following are true.

For Cloud Storage backend buckets, follow these additional suggestions:

• Make your bucket publicly readable. This is the approach that we recommend for public content. With this setting, anyone on the internet can view and list your objects and their metadata, excluding ACLs. The recommended practice is to dedicate specific buckets for public objects.

• Use managed folders to make a portion of your bucket publicly readable.

• Make individual objects publicly readable. We don't recommend this approach, because it uses a legacy, Cloud Storage-specific permissioning system.

• Don't store the object in a bucket that has Requester Pays enabled or resides within a Virtual Private Cloud service perimeter.

• Don't encrypt the object by using customer-managed encryption keys or customer-supplied encryption keys.

By default, when an object is public and doesn't specify Cache-Control metadata, Cloud Storage assigns a Cache-Control: public, max-age=3600 header to the object. You can set different values by using Cache-Control metadata.

For an example that shows how to configure an external Application Load Balancer with a backend bucket, see Setting up Cloud CDN with a backend bucket.

Maximum size

Cloud CDN enforces a maximum size for each response. Any response with a body larger than the maximum size is not cached but is still delivered to the client.

The maximum size varies depending on whether the origin server supports byte range requests.

Nearly all modern web servers (including NGINX, Apache, and Varnish) support byte range requests.

Non-cacheable content based on origin headers

There are checks that block caching of responses. Cloud CDN may periodically change the exact set of conditions under which it caches content, so if you want to explicitly prevent Cloud CDN from caching your content, follow the guidelines in the standard (RFC 7234) to determine how to specify a guaranteed-uncacheable response.

Cloud CDN doesn't cache a response if it does not meet the requirements for Cacheable content, or if any of the following is true.

If Cache-Control: no-store or private is present, but the content is still being cached, this is due to one of the following:

• URL signing is configured.
• The Cloud CDN cache mode is set to force caching of all responses.

Prevent caching

To prevent private information from being cached in Cloud CDN caches, do the following:

1. Make sure that Cloud CDN cache mode isn't set to the FORCE_CACHE_ALL mode, which unconditionally caches all successful responses.
2. Include a Cache-Control: private header in responses that shouldn't be stored in Cloud CDN caches, or a Cache-Control: no-store header in responses that shouldn't be stored in any cache, even a web browser's cache.
3. Don't sign URLs that provide access to private information. When content is accessed by using a signed URL, it is potentially eligible for caching regardless of any Cache-Control directives in the response.
4. For origin (cache fill) requests that include the Authorization request header, Cloud CDN only caches responses that include the public, must-revalidate, or s-maxage cache control directives when the cache mode is set to USE_ORIGIN_HEADERS or CACHE_ALL_STATIC. This prevents accidentally caching per-user content and content that requires authentication. The FORCE_CACHE_ALL cache mode does not have this restriction.

Custom response headers

With custom response headers, you can specify headers that the classic Application Load Balancer adds to proxied responses. Custom response headers let you reflect the cache status to your clients, client geographic data, and your own static response headers.

For instructions, see Configure custom response headers.

Cache keys

Each cache entry in a Cloud CDN cache is identified by a cache key. When a request comes into the cache, the cache converts the URI of the request into a cache key, and then compares it with keys of cached entries. If it finds a match, the cache returns the object associated with that key.

For backend services, Cloud CDN defaults to using the complete request URI as the cache key. For example, https://example.com/images/cat.jpg is the complete URI for a particular request for the cat.jpg object. This string is used as the default cache key. Only requests with this exact string match. Requests for http://example.com/images/cat.jpg or https://example.com/images/cat.jpg?user=user1 don't match.

For backend buckets, the default is for the cache key to consist of the URI without the protocol or host. By default, only query parameters that are known to Cloud Storage are included as part of the cache key (for example, "generation").

Thus, for a given backend bucket, the following URIs resolve to the same cached object:

• http://example.com/images/cat.jpg
• https://example.com/images/cat.jpg
• https://example.com/images/cat.jpg?user=user1
• http://example.com/images/cat.jpg?user=user1
• https://example.com/images/cat.jpg?user=user2
• https://media.example.com/images/cat.jpg
• https://www.example.com/images/cat.jpg

You can change which parts of the URI are used in the cache key. While the filename and path must always be part of the key, you can include or omit any combination of protocol, host, or query string when customizing your cache key. Using cache keys describes how to customize your cache keys.

In addition to including or omitting the entire query string, you can use portions of the query string by using include lists and exclude lists.

Query string include list

You can selectively control which query string parameters Cloud CDN incorporates into cache keys. For example, if you create an include list of user, thenhttps://example.com/images/cat.jpg?user=user1&color=blue creates a cache key of https://example.com/images/cat.jpg?user=user1 that also matches https://example.com/images/cat.jpg?user=user1&color=red.

To use this option, you must include the query string, specify a non-empty include list, and not specify an exclude list.

Query string include list for Cloud Storage cache keys

Including URL query parameters in cache keys for Cloud Storage buckets helps support cache busting. Cache busting lets a user retrieve a new version of the file that has been uploaded, even if the earlier version is still validly cached based on the TTL setting.

You can use an include list with query string parameters in the cache key used for serving responses from a backend bucket. Although Cloud Storage does not serve different content or route based on query parameters, you can choose to include parameters that allow you to cache-bust static content stored in Cloud Storage buckets.

For example, you can append a ?version=VERSION or ?hash=HASH query parameter that is based on the underlying content. This limits the need to proactively invalidate content and aligns with modern web development workflows, where web frameworks and URLs use a hash of the content to avoid serving stale objects across deployments.

Because including query parameters in the cache key is opt-in only, Cloud CDN doesn't support excluding query parameters from a cache key to a backend bucket.

Query string exclude list

You can selectively control which query string parameters Cloud CDN ignores by using an exclude list. For example, if you create an exclude list of user, all query string parameters except user are used in the cache key.

With the exclude list configured and an input of https://example.com/images/cat.jpg?user=user1&color=blue, Cloud CDN creates a cache key of https://example.com/images/cat.jpg?color=blue that also matches https://example.com/images/cat.jpg?user=user2&color=blue but not https://example.com/images/cat.jpg?user=user1&color=red.

To use this option, you must include the query string, specify a non-empty exclude list, and not specify an include list.

Query parameter order

The generated cache key doesn't depend on the order of the query parameters.

For example, the following query parameters generate the same cache key:

• info=123&variant=13e&geography=US
• geography=US&variant=13e&info=123

HTTP headers and HTTP cookies settings

You can improve cache hit rates and origin offload with the following cache key configuration settings:

• For backend services and buckets: Use HTTP headers as part of cache keys by including named headers in the cache key configuration.
• For backend services only: Use named HTTP cookies as cache keys, such as for A/B (multivariate) testing, canarying, and similar scenarios.

Cache requests that include additional HTTP headers or HTTP cookies in the request are cached on the third request in a cache location for that cache key. This reduces the impact of high-cardinality header or cookie values on your cache eviction rates. Under normal circumstances and user traffic conditions, this shouldn't be noticeable and helps ensure that popular content remains cached.

Include request headers

To cache additional variations of a response, you can include additional request headers in the cache key.

Some headers are not allowed in cache keys because they are typically very high cardinality. In most cases, the values of these headers are either unique per user (Cookie,Authorization) or have thousands of likely values (Referer, User-Agent, Accept). For example, the User-Agent header can have over 5,000 unique values given the large variety of browsers, user-devices, and operating systems. These types of headers would have a severe negative impact on cache hit rates.

Only valid HTTP header field names are accepted per RFC 7230. Header field names are case-insensitive, and duplicates are rejected.

You may optionally configure your origin server to include configured cache-key request headers in the Vary response. It is not required for Cloud CDN, but can be helpful for downstream caches. For more information, see Vary headers.

If an x-http-method-override header is present in the request, and specifies a method that doesn't match the underlying GET or HEAD method, the header will automatically be added to the cache key. Origin servers are unpredictable in how they respond to this header, so Cloud CDN treats the presence of the header conservatively and caches separate responses for different header values.

Cloud CDN doesn't allow the following headers to be included in the list of headers:

• Accept
• Accept-Encoding
• Authority, because this is controlled by configuration (cdnPolicy.includeHost)
• Authorization, typically per-user as in OAuth Bearer tokens
• CDN-Loop
• Connection
• Content-MD5
• Content-Type
• Cookie
• Date
• Forwarded, often per-client or per-proxy
• From
• Host, because this is controlled by configuration (cdnPolicy.includeHost)
• If-Match, If-Modified-Since, or If-None-Match
• Origin
• Proxy-Authorization
• Range
• Referer (or Referrer)
• User-Agent
• Want-Digest
• X-CSRFToken and X-CSRF-Token as used by Django and Ruby on Rails
• X-Forwarded-For, often per-client or per-proxy
• X-User-IP
• Any header starting with the following:
  • Access-Control-, such as Access-Control-Request-Headers and Access-Control-Request-Method
  • Sec-Fetch-
  • Sec-GFE-
  • Sec-Google-
  • X-Amz-
  • X-GFE-
  • X-Goog-
  • X-Google-

Use custom variables with request headers

Cache keys are helpful when you need to serve content differently based on each user's device and location. For example, you can enable a responsive website to serve the appropriate images for users that are viewing content based on their device type or set a helpful default language based on their location. You can define cache keys by using custom request headers and custom variables.

To use custom variables with request headers, do the following:

1. Define a custom request header for your backend service. Include one or more variables for the custom request header value.
2. Update the cache key to use the custom request header.

For Cloud CDN, you can use only the following variables when defining headers that are both custom request headers and cache key headers:

• device_request_type
• user_agent_family
• client_region
• client_region_subdivision

Cloud CDN limits the variables to help maintain cache performance. This is similar to limits on the headers that can be used as cache keys.

For example, if you could specify X-Lat-Long:{client_city_lat_long} as a custom request header and then add X-Lat-Long to your set of cache key headers, Cloud CDN would attempt to cache one copy of the response for each value of client_city_lat_long. This would ultimately lead to cache overuse, unnecessary content flushing, and reduced opportunity to return cache hits.

For these reasons, variables that have high cardinality aren't included in the list of variables that are used to define custom request headers and subsequently cache keys.

Same headers with different values

Suppose the user sends multiple same-named headers with different header values—for example:

My-Header: Value1
My-Header: Value2

In this case, Cloud CDN modifies the request by assuming that the header must follow the standard convention that allows some headers to have multiple values. Cloud CDN collapses them into a comma-separated list to send to the backend, so it's as if the client sent the following:

My-Header: Value1, Value2

Include named cookies

An HTTP cookie is a name=value pairing, and a request can include multiple HTTP cookies, either separated by a semicolon on the same line, or as discrete Cookie request headers with one cookie per header.

You can provide a list of up to five cookie names.

User agents (such as web browsers) often limit the number of cookies stored per domain to 4 KB; make sure not to send too many (or too large) cookies, as the user agent might not send all cookies in a request. This can impact whether a user receives a specific cached response.

If you are serving your static content from a different hostname from which you issue cookies, ensure that the Domain attribute of the cookie (and the Path) attribute allows the cookie to be sent along with requests for static content.

If a request includes multiple instances of the same cookie name, only the first one is honored.

Cache control header precedence

Origin servers often return multiple cache control headers in a single response to target different caching layers. When evaluating an origin response, Cloud CDN uses a strict order of precedence to select one header to define its caching behavior.

The precedence order, from highest to lowest priority, is: