fastly-invalidation: use explicit surrogate keys everywhere, remove magic #3060
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The quickfix middelware based surrogate solution was more a quickfix / hack from the start, which worked fine until I can build the actual solution.
I would love any design feedback, there are places I don't really like reading, perhaps you have better ideas.
We have three different kinds of cache policies we serve:
The last one is quite valueable mostly because documentation is static, but when there is a build / rebuild, we need to update the content. So we need to be able to ping the CDN that it has to refetch certain content.
In cloudfront this was possible using path prefixes / patterns (we used
/{krate}and/crate/{krate}), but also took ages and had concurrency limits.In fastly, this relies on surrogate keys which are (kind of) tags on the content.
So when we serve the page to fastly, we also return the tags. Fastly stores these tags next to the content in the caches. And when a purge-request comes, it can quickly find tagged content.
Now to this PR:
I believe when handlers choose certain cache policies, they have to be force to provide the needed tags too..
The tags don't matter when we don't cache, and the tags don't matter when we cache forever (static assets most of the time).
Also I'm adding a
allsurrogate key that we'll add to all doc pages, so we can also purge the whole service with one tag.( there is an advantage of using the
allsurrogate key for purging, compared to a normal "full purge": you can use soft purges, which I want to introduce at some point