Skip to content

Conversation

@syphar
Copy link
Member

@syphar syphar commented Dec 2, 2025

The quickfix middelware based surrogate solution was more a quickfix / hack from the start, which worked fine until I can build the actual solution.

I would love any design feedback, there are places I don't really like reading, perhaps you have better ideas.

We have three different kinds of cache policies we serve:

  • two that forbid caching with some variance
  • one that allows caching forever, everywhere (when you have hashed filenames)
  • and two where we only allow the CDN to cache, and not the browser (mostly).

The last one is quite valueable mostly because documentation is static, but when there is a build / rebuild, we need to update the content. So we need to be able to ping the CDN that it has to refetch certain content.

In cloudfront this was possible using path prefixes / patterns (we used /{krate} and /crate/{krate}), but also took ages and had concurrency limits.

In fastly, this relies on surrogate keys which are (kind of) tags on the content.

So when we serve the page to fastly, we also return the tags. Fastly stores these tags next to the content in the caches. And when a purge-request comes, it can quickly find tagged content.

Now to this PR:
I believe when handlers choose certain cache policies, they have to be force to provide the needed tags too..

The tags don't matter when we don't cache, and the tags don't matter when we cache forever (static assets most of the time).

Also I'm adding a all surrogate key that we'll add to all doc pages, so we can also purge the whole service with one tag.

( there is an advantage of using the all surrogate key for purging, compared to a normal "full purge": you can use soft purges, which I want to introduce at some point

@syphar syphar requested a review from a team as a code owner December 2, 2025 14:35
@github-actions github-actions bot added the S-waiting-on-review Status: This pull request has been implemented and needs to be reviewed label Dec 2, 2025
@syphar syphar self-assigned this Dec 2, 2025
@syphar syphar force-pushed the explicit-surrogate-key-2 branch from 715cfdb to 776d588 Compare December 2, 2025 17:16
@syphar syphar force-pushed the explicit-surrogate-key-2 branch from 776d588 to c342321 Compare December 3, 2025 05:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S-waiting-on-review Status: This pull request has been implemented and needs to be reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant