HTTP caching

Types of caches

In the HTTP Caching specthere are two main types of caches: private caches and shared caches.

Private caches

A private cache is a cache tied to a specific client — typically a browser cache. Since the stored response is not shared with other clientsa private cache can store a personalized response for that user.

On the other handif personalized contents are stored in a cache other than a private cachethen other users may be able to retrieve those contents — which may cause unintentional information leakage.

If a response contains personalized content and you want to store the response only in the private cacheyou must specify a private directive.

Cache-Control: private

Personalized contents are usually controlled by cookiesbut the presence of a cookie does not always indicate that it is privateand thus a cookie alone does not make the response private.

Shared cache

The shared cache is located between the client and the server and can store responses that can be shared among users. And shared caches can be further sub-classified into proxy caches and managed caches.

Proxy caches

In addition to the function of access controlsome proxies implement caching to reduce traffic out of the network. This is usually not managed by the service developerso it must be controlled by appropriate HTTP headers and so on. Howeverin the pastoutdated proxy-cache implementations — such as implementations that do not properly understand the HTTP Caching standard — have often caused problems for developers.

Kitchen-sink headers like the following are used to try to work around "old and not updated proxy cache" implementations that do not understand current HTTP Caching spec directives like no-store.

Cache-Control: no-storeno-cachemax-age=0must-revalidateproxy-revalidate

Howeverin recent yearsas HTTPS has become more common and client/server communication has become encryptedproxy caches in the path can only tunnel a response and can't behave as a cachein many cases. So in that scenariothere is no need to worry about outdated proxy cache implementations that cannot even see the response.

On the other handif a TLS bridge proxy decrypts all communications in a person-in-the-middle manner by installing a certificate from a CA (certificate authority) managed by the organization on the PCand performs access controletc. — it is possible to see the contents of the response and cache it. Howeversince CT (certificate transparency) has become widespread in recent yearsand some browsers only allow certificates issued with an SCT (signed certificate timestamp)this method requires the application of an enterprise policy. In such a controlled environmentthere is no need to worry about the proxy cache being "out of date and not updated".

Managed caches

Managed caches are explicitly deployed by service developers to offload the origin server and to deliver content efficiently. Examples include reverse proxiesCDNsand service workers in combination with the Cache API.

The characteristics of managed caches vary depending on the product deployed. In most casesyou can control the cache's behavior through the Cache-Control header and your own configuration files or dashboards.

For examplethe HTTP Caching specification essentially does not define a way to explicitly delete a cache — but with a managed cachethe stored response can be deleted at any time through dashboard operationsAPI callsrestartsand so on. That allows for a more proactive caching strategy.

It is also possible to ignore the standard HTTP Caching spec protocols in favor of explicit manipulation. For examplethe following can be specified to opt-out of a private cache or proxy cachewhile using your own strategy to cache only in a managed cache.

Cache-Control: no-store

For exampleVarnish Cache uses VCL (Varnish Configuration Languagea type of DSL) logic to handle cache storagewhile service workers in combination with the Cache API allow you to create that logic in JavaScript.

That means if a managed cache intentionally ignores a no-store directivethere is no need to perceive it as being "non-compliant" with the standard. What you should do isavoid using kitchen-sink headersbut carefully read the documentation of whatever managed-cache mechanism you're usingand ensure you're controlling the cache properly in the ways provided by the mechanism you've chosen to use.

Note that some CDNs provide their own headers that are effective only for that CDN (for exampleSurrogate-Control). Currentlywork is underway to define a CDN-Cache-Control header to standardize those.

Heuristic caching

HTTP is designed to cache as much as possibleso even if no Cache-Control is givenresponses will get stored and reused if certain conditions are met. This is called heuristic caching.

For exampletake the following response. This response was last updated 1 year ago.

HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 1024
Date: Tue22 Feb 2022 22:22:22 GMT
Last-Modified: Tue22 Feb 2021 22:22:22 GMT

<!doctype html>
…

It is heuristically known that content which has not been updated for a full year will not be updated for some time after that. Thereforethe client stores this response (despite the lack of max-age) and reuses it for a while. How long to reuse is up to the implementationbut the specification recommends about 10% (in this case 0.1 year) of the time after storing.

Heuristic caching is a workaround that came before Cache-Control support became widely adoptedand basically all responses should explicitly specify a Cache-Control header.

Fresh and stale based on age

Stored HTTP responses have two states: fresh and stale. The fresh state usually indicates that the response is still valid and can be reusedwhile the stale state means that the cached response has already expired.

The criterion for determining when a response is fresh and when it is stale is age. In HTTPage is the time elapsed since the response was generated. This is similar to the TTL in other caching mechanisms.

Take the following example response (604800 seconds is one week):

HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 1024
Date: Tue22 Feb 2022 22:22:22 GMT
Cache-Control: max-age=604800

<!doctype html>
…

The cache that stored the example response calculates the time elapsed since the response was generated and uses the result as the response's age.

For the example responsethe meaning of max-age is the following:

• If the age of the response is less than one weekthe response is fresh.
• If the age of the response is more than one weekthe response is stale.

As long as the stored response remains freshit will be used to fulfill client requests.

When a response is stored in a shared cacheit is possible to tell the client the age of the response. Continuing with the exampleif the shared cache stored the response for one daythe shared cache would send the following response to subsequent client requests.

HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 1024
Date: Tue22 Feb 2022 22:22:22 GMT
Cache-Control: max-age=604800
Age: 86400

<!doctype html>
…

The client which receives that response will find it to be fresh for the remaining 518400 secondsthe difference between the response's max-age and Age.

Expires or max-age

In HTTP/1.0freshness used to be specified by the Expires header.

The Expires header specifies the lifetime of the cache using an explicit time rather than by specifying an elapsed time.

Expires: Tue28 Feb 2022 22:22:22 GMT

Howeverthe time format is difficult to parsemany implementation bugs were foundand it is possible to induce problems by intentionally shifting the system clock; thereforemax-age — for specifying an elapsed time — was adopted for Cache-Control in HTTP/1.1.

If both Expires and Cache-Control: max-age are availablemax-age is defined to be preferred. So it is not necessary to provide Expires now that HTTP/1.1 is widely used.

Vary

The way that responses are distinguished from one another is essentially based on their URLs:

But the contents of responses are not always the sameeven if they have the same URL. Especially when content negotiation is performedthe response from the server can depend on the values of the AcceptAccept-Languageand Accept-Encoding request headers.

For examplefor English content returned with an Accept-Language: en header and cachedit is undesirable to then reuse that cached response for requests that have an Accept-Language: ja request header. In this caseyou can cause the responses to be cached separately — based on language — by adding Accept-Language to the value of the Vary header.

Vary: Accept-Language

That causes the cache to be keyed based on a composite of the response URL and the Accept-Language request header — rather than being based just on the response URL.

Alsoif you are providing content optimization (for examplefor responsive design) based on the user agentyou may be tempted to include User-Agent in the value of the Vary header. Howeverthe User-Agent request header generally has a very large number of variationswhich drastically reduces the chance that the cache will be reused. So if possibleinstead consider a way to vary behavior based on feature detection rather than based on the User-Agent request header.

For applications that employ cookies to prevent others from reusing cached personalized contentyou should specify Cache-Control: private instead of specifying a cookie for Vary.

Validation

Stale responses are not immediately discarded. HTTP has a mechanism to transform a stale response into a fresh one by asking the origin server. This is called validationor sometimesrevalidation.

Validation is done by using a conditional request that includes an If-Modified-Since or If-None-Match request header.

If-Modified-Since

The following response was generated at 22:22:22 and has a max-age of 1 hourso you know that it is fresh until 23:22:22.

HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 1024
Date: Tue22 Feb 2022 22:22:22 GMT
Last-Modified: Tue22 Feb 2022 22:00:00 GMT
Cache-Control: max-age=3600

<!doctype html>
…

At 23:22:22the response becomes stale and the cache cannot be reused. So the request below shows a client sending a request with an If-Modified-Since request headerto ask the server if there have been any changes made since the specified time.

GET /index.html HTTP/1.1
Host: example.com
Accept: text/html
If-Modified-Since: Tue22 Feb 2022 22:00:00 GMT

The server will respond with 304 Not Modified if the content has not changed since the specified time.

Since this response only indicates "no change"there is no response body — there's just a status code — so the transfer size is extremely small.

HTTP/1.1 304 Not Modified
Content-Type: text/html
Date: Tue22 Feb 2022 23:22:22 GMT
Last-Modified: Tue22 Feb 2022 22:00:00 GMT
Cache-Control: max-age=3600

Upon receiving that responsethe client reverts the stored stale response back to being fresh and can reuse it during the remaining 1 hour.

The server can obtain the modification time from the operating-system file systemwhich is relatively easy to do for the case of serving static files. Howeverthere are some problems; for examplethe time format is complex and difficult to parseand distributed servers have difficulty synchronizing file-update times.

To solve such problemsthe ETag response header was standardized as an alternative.

ETag/If-None-Match

The value of the ETag response header is an arbitrary value generated by the server. There are no restrictions on how the server must generate the valueso servers are free to set the value based on whatever means they choose — such as a hash of the body contents or a version number.

As an exampleif a hash value is used for the ETag header and the hash value of the index.html resource is 33a64df5the response will be as follows:

HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 1024
Date: Tue22 Feb 2022 22:22:22 GMT
ETag: "33a64df5"
Cache-Control: max-age=3600

<!doctype html>
…

If that response is stalethe client takes the value of the ETag response header for the cached responseand puts it into the If-None-Match request headerto ask the server if the resource has been modified:

GET /index.html HTTP/1.1
Host: example.com
Accept: text/html
If-None-Match: "33a64df5"

The server will return 304 Not Modified if the value of the ETag header it determines for the requested resource is the same as the If-None-Match value in the request.

But if the server determines the requested resource should now have a different ETag valuethe server will instead respond with a 200 OK and the latest version of the resource.

Force Revalidation

If you do not want a response to be reusedbut instead want to always fetch the latest content from the serveryou can use the no-cache directive to force validation.

By adding Cache-Control: no-cache to the response along with Last-Modified and ETag — as shown below — the client will receive a 200 OK response if the requested resource has been updatedor will otherwise receive a 304 Not Modified response if the requested resource has not been updated.

HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 1024
Date: Tue22 Feb 2022 22:22:22 GMT
Last-Modified: Tue22 Feb 2022 22:00:00 GMT
ETag: "deadbeef"
Cache-Control: no-cache

<!doctype html>
…

It is often stated that the combination of max-age=0 and must-revalidate has the same meaning as no-cache.

Cache-Control: max-age=0must-revalidate

max-age=0 means that the response is immediately staleand must-revalidate means that it must not be reused without revalidation once it is stale — soin combinationthe semantics seem to be the same as no-cache.

Howeverthat usage of max-age=0 is a remnant of the fact that many implementations prior to HTTP/1.1 were unable to handle the no-cache directive — and so to deal with that limitationmax-age=0 was used as a workaround.

But now that HTTP/1.1-conformant servers are widely deployedthere's no reason to ever use that max-age=0 and must-revalidate combination — you should instead just use no-cache.

Don't cache

The no-cache directive does not prevent the storing of responses but instead prevents the reuse of responses without revalidation.

If you don't want a response stored in any cacheuse no-store.

Cache-Control: no-store

Howeverin generala "do not cache" requirement in practice amounts to the following set of circumstances:

• Don't want the response stored by anyone other than the specific clientfor privacy reasons.
• Want to provide up-to-date information always.
• Don't know what could happen in outdated implementations.

Under that set of circumstancesno-store is not always the most-appropriate directive.

The following sections look at the circumstances in more detail.

Do not share with others

It would be problematic if a response with personalized content is unexpectedly visible to other users of a cache.

In such a caseusing the private directive will cause the personalized response to only be stored with the specific client and not be leaked to any other user of the cache.

Cache-Control: private

In such a caseeven if no-store is givenprivate must also be given.

Provide up-to-date content every time

The no-store directive prevents a response from being storedbut does not delete any already-stored response for the same URL.

In other wordsif there is an old response already stored for a particular URLreturning no-store will not prevent the old response from being reused.

Howevera no-cache directive will force the client to send a validation request before reusing any stored response.

Cache-Control: no-cache

If the server does not support conditional requestsyou can force the client to access the server every time and always get the latest response with 200 OK.

Dealing with outdated implementations

As a workaround for outdated implementations that ignore no-storeyou may see kitchen-sink headers such as the following being used.

Cache-Control: no-storeno-cachemax-age=0must-revalidateproxy-revalidate

It is recommended to use no-cache as an alternative for dealing with such outdated implementationsand it is not a problem if no-cache is given from the beginningsince the server will always receive the request.

If it is the shared cache that you are concerned aboutyou can make sure to prevent unintended caching by also adding private:

Cache-Control: no-cacheprivate

What's lost by no-store

You may think adding no-store would be the right way to opt-out of caching.

Howeverit's not recommended to grant no-store liberallybecause you lose many advantages that HTTP and browsers haveincluding the browser's back/forward cache.

Thereforeto get the advantages of the full feature set of the web platformprefer the use of no-cache in combination with private.

Reload and force reload

Validation can be performed for requests as well as responses.

The reload and force reload actions are common examples of validation performed from the browser side.

Reload

For recovering from window corruption or updating to the latest version of the resourcebrowsers provide a reload function for users.

A simplified view of the HTTP request sent during a browser reload looks as follows:

GET / HTTP/1.1
Host: example.com
Cache-Control: max-age=0
If-None-Match: "deadbeef"
If-Modified-Since: Tue22 Feb 2022 20:20:20 GMT

(The requests from ChromeEdgeand Firefox look very much like the above; the requests from Safari will look a bit different.)

The max-age=0 directive in the request specifies "reuse of responses with an age of 0 or less" — soin effectintermediately stored responses are not reused.

As a resulta request is validated by If-None-Match and If-Modified-Since.

That behavior is also defined in the Fetch standard and can be reproduced in JavaScript by calling fetch() with the cache mode set to no-cache (note that reload is not the right mode for this case):

// Note: "reload" is not the right mode for a normal reload; "no-cache" is
fetch("/"{ cache: "no-cache" });

Force reload

Browsers use max-age=0 during reloads for backward-compatibility reasons — because many outdated implementations prior to HTTP/1.1 did not understand no-cache. But no-cache is fine now in this use caseand force reload is an additional way to bypass cached responses.

The HTTP Request during a browser force reload looks as follows:

GET / HTTP/1.1
Host: example.com
Pragma: no-cache
Cache-Control: no-cache

(The requests from ChromeEdgeand Firefox look very much like the above; the requests from Safari will look a bit different.)

Since that's not a conditional request with no-cacheyou can be sure you'll get a 200 OK from the origin server.

That behavior is also defined in the Fetch standard and can be reproduced in JavaScript by calling fetch() with the cache mode set to reload (note that it's not force-reload):

// Note: "reload" — rather than "no-cache" — is the right mode for a "force reload"
fetch("/"{ cache: "reload" });

Avoiding revalidation

Content that never changes should be given a long max-age by using cache busting — that isby including a version numberhash valueetc.in the request URL.

Howeverwhen the user reloadsa revalidation request is sent even though the server knows that the content is immutable.

To prevent thatthe immutable directive can be used to explicitly indicate that revalidation is not required because the content never changes.

Cache-Control: max-age=31536000immutable

That prevents unnecessary revalidation during reloads.

Note thatinstead of implementing that directiveChrome has changed its implementation so that revalidation is not performed during reloads for subresources.

Deleting stored responses

There is no way to delete responses on an intermediate server that have been stored with a long max-age.

Imagine that the following response from https://example.com/ was stored.

HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 1024
Cache-Control: max-age=31536000

<!doctype html>
…

You may want to overwrite that response once it expired on the serverbut there is nothing the server can do once the response is stored — since no more requests reach the server due to caching.

One of the methods mentioned in the specification is to send a request for the same URL with an unsafe method such as POSTbut for many clients that is difficult to do.

The Clear-Site-Data: cache header and directive value can be used to clear browser caches — but has no effect on intermediate caches. Otherwise responses will remain in the browser cache until max-age expiresunless the user manually performs a reloadforce-reloador clear-history action.

Caching reduces access to the serverwhich means that the server loses control of that URL. If the server does not want to lose control of a URL — for examplein the case that a resource is frequently updated — you should add no-cache so that the server will always receive requests and send the intended responses.

Request collapse

The shared cache is primarily located before the origin server and is intended to reduce traffic to the origin server.

Thusif multiple identical requests arrive at a shared cache at the same timethe intermediate cache will forward a single request on behalf of itself to the originwhich can then reuse the result for all clients. This is called request collapse.

Request collapse occurs when requests are arriving at the same timeso even if max-age=0 or no-cache is given in the responseit will be reused.

If the response is personalized to a particular user and you do not want it to be shared in collapseyou should add the private directive:

Common caching patterns

There are many directives in the Cache-Control specand it may be difficult to understand all of them. But most websites can be covered by a combination of a handful of patterns.

This section describes the common patterns in designing caches.

Default settings

As mentioned abovethe default behavior for caching (that isfor a response without Cache-Control) is not simply "don't cache" but implicit caching according to so-called "heuristic caching".

To avoid that heuristic cachingit's preferable to explicitly give all responses a default Cache-Control header.

To ensure that by default the latest versions of resources will always be transferredit's common practice to make the default Cache-Control value include no-cache:

Cache-Control: no-cache

In additionif the service implements cookies or other login methodsand the content is personalized for each userprivate must be given tooto prevent sharing with other users:

Cache-Control: no-cacheprivate

Cache Busting

The resources that work best with caching are static immutable files whose contents never change. And for resources that do changeit is a common best practice to change the URL each time the content changesso that the URL unit can be cached for a longer period.

As an exampleconsider the following HTML:

<script src="bundle."></script>
<link rel="sheet" href="build.css" />
<body>
  hello
</body>

In modern web developmentJavaScript and CSS resources are frequently updated as development progresses. Alsoif the versions of JavaScript and CSS resources that a client uses are out of syncthe display will break.

So the HTML above makes it difficult to cache bundle. and build.css with max-age.

Thereforeyou can serve the JavaScript and CSS with URLs that include a changing part based on a version number or hash value. Some of the ways to do that are shown below.

# version in filename
bundle.v123.

# version in query
bundle.?v=123

# hash in filename
bundle.YsAIAAAA-QG4G6kCMAMBAAAAAAAoK.

# hash in query
bundle.?v=YsAIAAAA-QG4G6kCMAMBAAAAAAAoK

Since the cache distinguishes resources from one another based on their URLsthe cache will not be reused again if the URL changes when a resource is updated.

<script src="bundle.v123."></script>
<link rel="sheet" href="build.v123.css" />
<body>
  hello
</body>

With that designboth JavaScript and CSS resources can be cached for a long time. So how long should max-age be set to? The QPACK specification provides an answer to that question.

QPACK is a standard for compressing HTTP header fieldswith tables of commonly-used field values defined.

Some commonly-used cache-header values are shown below.

36 cache-control max-age=0
37 cache-control max-age=604800
38 cache-control max-age=2592000
39 cache-control no-cache
40 cache-control no-store
41 cache-control publicmax-age=31536000

If you select one of those numbered optionsyou can compress values in 1 byte when transferred via HTTP3.

Numbers 3738and 41 are for periods of one weekone monthand one year.

Because the cache removes old entries when new entries are savedthe probability that a stored response still exists after one week is not that high — even if max-age is set to 1 week. Thereforein practiceit does not make much difference which one you choose.

Note that number 41 has the longest max-age (1 year)but with public.

The public value has the effect of making the response storable even if the Authorization header is present.

So if the response is personalized with basic authenticationthe presence of public may cause problems. If you are concerned about thatyou can choose the second-longest value38 (1 month).

# response for bundle.v123.

# If you never personalize responses via Authorization
Cache-Control: publicmax-age=31536000

# If you can't be certain
Cache-Control: max-age=2592000

Validation

Don't forget to set the Last-Modified and ETag headersso that you don't have to re-transmit a resource when reloading. It's easy to generate those headers for pre-built static files.

The ETag value here may be a hash of the file.

# response for bundle.v123.
Last-Modified: Tue22 Feb 2022 20:20:20 GMT
ETag: "YsAIAAAA-QG4G6kCMAMBAAAAAAAoK"

In additionimmutable can be added to prevent validation on reload.

The combined result is shown below.

# bundle.v123.
HTTP/1.1 200 OK
Content-Type: text/javascript
Content-Length: 1024
Cache-Control: publicmax-age=31536000immutable
Last-Modified: Tue22 Feb 2022 20:20:20 GMT
ETag: "YsAIAAAA-QG4G6kCMAMBAAAAAAAoK"

Cache busting is a technique to make a response cacheable over a long period by changing the URL when the content changes. The technique can be applied to all subresourcessuch as images.

Cache-Control: publicmax-age=31536000
Cache-Control: immutable

Main resources

Unlike subresourcesmain resources cannot be cache busted because their URLs can't be decorated in the same way that subresource URLs can be.

If the following HTML itself is storedthe latest version cannot be displayed even if the content is updated on the server side.

<script src="bundle.v123."></script>
<link rel="sheet" href="build.v123.css" />
<body>
  hello
</body>

For that caseno-cache would be appropriate — rather than no-store — since we don't want to store HTMLbut instead just want it to always be up-to-date.

Furthermoreadding Last-Modified and ETag will allow clients to send conditional requestsand a 304 Not Modified can be returned if there have been no updates to the HTML:

HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 1024
Cache-Control: no-cache
Last-Modified: Tue22 Feb 2022 20:20:20 GMT
ETag: "AAPuIbAOdvAGEETbgAAAAAAABAAE"

That setting is appropriate for non-personalized HTMLbut for a response that gets personalized using cookies — for exampleafter a login — don't forget to also specify private:

HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 1024
Cache-Control: no-cacheprivate
Last-Modified: Tue22 Feb 2022 20:20:20 GMT
ETag: "AAPuIbAOdvAGEETbgAAAAAAABAAE"
Set-Cookie: __Host-SID=AHNtAyt3fvJrUL5g5tnGwER; Secure; Path=/; HttpOnly

The same can be used for favicon.icomanifest.on.well-knownand API endpoints whose URLs cannot be changed using cache busting.

Most web content can be covered by a combination of the two patterns described above.

More about managed caches

With the method described in previous sectionssubresources can be cached for a long time by using cache bustingbut main resources (which are usually HTML documents) can't be.

Caching main resources is difficult becauseusing just standard directives from the HTTP Caching specificationthere's no way to actively delete cache contents when content is updated on the server.

Howeverit is possible by deploying a managed cache such as a CDN or service worker.

For examplea CDN that allows cache purging via an API or dashboard operation would allow for a more aggressive caching strategy by storing the main resource and explicitly purging the relevant cache only when an update occurs on the server.

A service worker could do the same if it could delete the contents in the Cache API when an update occurs on the server.

For more informationsee the documentation for your CDNand consult the service worker documentation.

See also

• RFC 9111: Hypertext Transfer Protocol (HTTP/1.1): Caching
• Caching Tutorial - Mark Nottingham