My 404 response are being cached by section.io by default, why is this?
One really easy way to attack a server is to send a huge number of requests for pages that do not exist. Although delivering a single 404 (Not Found) to a client is pretty light work for a server relative to requests for images and HTML documents, multiplying that by 100 or 1000 per second can significantly slow a server down or even cause it to crash. The section.io platform caches 404s by default to protect against this kind of attack. With 404 caching enabled, our proxy servers can deliver the cached 404s and your origin never even sees the barrage of requests.
Is there an easy way to check whether 404 pages are being cached?
On @shaun’s recommendation, we’ve added a rule to our
default.vcl to cache 404 pages, but I can’t see the usual
section-io-cache: Hit header showing up for them.
The reason you are not seeing
section-io-cache: Hit or
section-io-cache: Miss is because you’ve implemented a custom 404 page using section.io’s custom error feature feature(https://www.section.io/docs/reference/http-error-messages/#custom-error-messages)
Custom error pages are served from the Edge proxy in section.io, not the Varnish cache proxy. This is why it doesn’t contain any Varnish headers.
You can tell if 404s are being cached using the HTTP logs. Just filter by
status:404 and look at the
I’ve had a look in the logs, and I’m seeing
varnish_handling: miss on all 404 requests (in staging – we haven’t got this on production yet) – I take it that means they’re not being cached?
I’m a bit unsure because our own server logs seem to show the caching working; reloading multiple times only creates one request at the origin.
Keep in mind that the HTTP logs are shared with your production environment.
Make sure when you are looking at the logs that you filter by the
environmentId of your staging environment.
Alternatively, you can exclude the production
environmentId from your search.
Never mind – on further investigation it turns out I was filtering to
varnish_handling: miss. I’ve removed that filter and I can now see both hits and misses showing up, which is the information I needed.