The site has been stable in the new configuration for a bit now, so we will consider this maintenance closed!
Posted Jul 25, 2018 - 15:23 UTC
As part of the upgrades yesterday the DNS record for www.pool.ntp.org temporarily (and mistakenly, obviously) routed around the Fastly CDN.
For the "real" traffic to the pool system that's totally fine, but there's also a regular background load of HTTP requests from slightly broken/slow clients (mostly in China) that, I am guessing, tries to set the time over HTTP. The new setup is missing a piece to "hold" the slow requests when the CDN isn't active.
There are just a couple cron jobs to be moved before the server migration is complete.
Posted Jul 15, 2018 - 19:17 UTC
The website has now been migrated to the Kubernetes cluster! There are very occasional timeouts happening on IPv4 requests to the website; the cause is understood. Fix underway.
On the old servers there was a set of Varnish servers in front of the application. Most things are cached by the CDN so Varnish didn't have much to do, but it did change the request flow in subtle ways that turned out to be meaningful.