yum problems
Not an obvious solution
So, going back and fixing up auth on our few remaining older systems (centos 5, not internet facing) came across the error below. Solution was beautifully non obvious, so it goes here in the external memory pack.
yum --enablerepo=my-repo-x86_64 list updates
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: a.centos.mirror
* epel: another.centos.mirror
* extras: and.a.further.centos.mirror
* updates: centos-updates.co.uk
my-repo-x86_64 | 2.9 kB 00:00
my-repo-x86_64/primary_db | 7.1 kB 00:00
http://myreposerver.internal.domain/my-repo/repodata/7d1016c9fcac64ee6c0fe9b5b\
58ed1e791dae601b1b0be13ea8af523761fbabd-primary.sqlite.bz2: [Errno -3] \
Error performing checksum
Trying other mirror.
my-repo-x86_64/primary_db | 7.1 kB 00:00
Error: failure: repodata/7d1016c9fcac64ee6c0fe9b5b58ed1e791dae601b1b0be13ea8af5\
23761fbabd-primary.sqlite.bz2 from my-repo-x86_64: [Errno 256] \
No more mirrors to try.
Do not pass go. Do not collect your new sssd binaries.
php annoyance multiple file uploads with $_FILES
This caused me to waste 1/2 an hour or so this morning. It also needed some quite precise google-fu to uncover the solution. I'm bound to forget this and am likely to need it again so it's going here. I'm not sure how most of the zillions of "upload multiple files with php" tutorials out there work, because none of them seem to mention this.
I was doing multiple file uploads in php, and finding that although you submit multiple files with a multipart/form-data encoding type (and can see multiple files being submitted in chrome devtools), php was only reporting the last one in the list of uploaded files in $_FILES. Very annoying.
latency and the buffer_head slabcache
We were tracing a problem with occasional latency spikes on one of our systems the other day. The cause was rather unusual.
The write load was not high, being a few megabytes per second at peak on an otherwise completely idle disk. We checked for the usual hardware issues; cache batteries, failing disks as well as other programs writing to that disk, but came up empty. Fairly regularly we saw a write latency of up to 40ms. Occasionally we'd see more spikes at an increased frequency of occurrence. We discovered that sometimes the latency spikes clustered - so it looked like there was bimodal behaviour.
Our investigation showed we were waiting on writes to the page_cache whilst allocating buffer_head structures. Here's what we found out.