Home About News Products Blog

Why does section.io cache 404s by default?


#1

My 404 response are being cached by section.io by default, why is this?


#2

One really easy way to attack a server is to send a huge number of requests for pages that do not exist. Although delivering a single 404 (Not Found) to a client is pretty light work for a server relative to requests for images and HTML documents, multiplying that by 100 or 1000 per second can significantly slow a server down or even cause it to crash. The section.io platform caches 404s by default to protect against this kind of attack. With 404 caching enabled, our proxy servers can deliver the cached 404s and your origin never even sees the barrage of requests.


#3

Is there an easy way to check whether 404 pages are being cached?

On @shaun’s recommendation, we’ve added a rule to our default.vcl to cache 404 pages, but I can’t see the usual section-io-cache: Hit header showing up for them.


#4

Hi Edward

The reason you are not seeing section-io-cache: Hit or section-io-cache: Miss is because you’ve implemented a custom 404 page using section.io’s custom error feature feature(https://www.section.io/docs/reference/http-error-messages/#custom-error-messages)

Custom error pages are served from the Edge proxy in section.io, not the Varnish cache proxy. This is why it doesn’t contain any Varnish headers.

You can tell if 404s are being cached using the HTTP logs. Just filter by status:404 and look at the varnish_handling field.


#5

Thanks Shaun.

I’ve had a look in the logs, and I’m seeing varnish_handling: miss on all 404 requests (in staging – we haven’t got this on production yet) – I take it that means they’re not being cached?

I’m a bit unsure because our own server logs seem to show the caching working; reloading multiple times only creates one request at the origin.


#6

Hi Edward

Keep in mind that the HTTP logs are shared with your production environment.
Make sure when you are looking at the logs that you filter by the environmentId of your staging environment.
Alternatively, you can exclude the production environmentId from your search.


#7

Never mind – on further investigation it turns out I was filtering to varnish_handling: miss. I’ve removed that filter and I can now see both hits and misses showing up, which is the information I needed.