Atp's external memory

php annoyance multiple file uploads with $_FILES

This caused me to waste 1/2 an hour or so this morning. It also needed some quite precise google-fu to uncover the solution. I'm bound to forget this and am likely to need it again so it's going here. I'm not sure how most of the zillions of "upload multiple files with php" tutorials out there work, because none of them seem to mention this.

I was doing multiple file uploads in php, and finding that although you submit multiple files with a multipart/form-data encoding type (and can see multiple files being submitted in chrome devtools), php was only reporting the last one in the list of uploaded files in $_FILES. Very annoying.

Written by atp

Monday 08 September 2014 at 12:09 pm

Posted in Annoyances, Linux

latency and the buffer_head slabcache

We were tracing a problem with occasional latency spikes on one of our systems the other day. The cause was rather unusual.

The write load was not high, being a few megabytes per second at peak on an otherwise completely idle disk. We checked for the usual hardware issues; cache batteries, failing disks as well as other programs writing to that disk, but came up empty.  Fairly regularly we saw a write latency of up to 40ms. Occasionally we'd see more spikes at an increased frequency of occurrence. We discovered that sometimes the latency spikes clustered - so it looked like there was bimodal behaviour.

Our investigation showed we were waiting on writes to the page_cache whilst allocating buffer_head structures. Here's what we found out.

Written by atp

Monday 01 September 2014 at 1:26 pm

Posted in Linux

php api revisited.

In 2011 I put out an example php class that could be used to talk to the LMAX API protocol. It was incomplete and for illustrative purposes only. It remains that way. Over the years however there have been some improvements and it can now be used for placing, closing and cancelling orders. I've made the latest version available on github, just in case the handful of people who have contacted me over the years about it find it useful to have the latest version. 

Here is the lmax-php-api github repo. As well as the original ticker app, there's a small example web app that shows how to get and handle position state events, as well as the example that forms the rest of the post. 

Its not under active development. Its not supported, and should not be used for real trading - its purely an educational tool. However if you send me a patch I'll probably add it in.

Written by atp

Wednesday 14 May 2014 at 2:42 pm

Posted in Default, Linux

leading zeros - builtins and FFSL

I took the time to start using Mike's HDR histogram port to C the other day. It was pretty painless to use, once I'd overcome a couple of initial problems. One of which is that the C compiler I have on CentOS 6.4 was too old. So I came up with a workaround for older compilers/machines. 

HDRHistogram is Azul's high dynamic range histogram. The readme on github explains what it is. Suffice it to say that I'm in the process of dumping the various implementations of histogram code I've evolved over the years in favour of this for latency and performance management.

Written by atp

Wednesday 26 March 2014 at 2:09 pm

Posted in Default, Linux

debugging a crash in haproxy

Horrors uncovered in kerberos

Last week we had several reports of disconnections to some of our haproxy fronted services after we upgraded to a longer/stronger SSL certificate. As the version of haproxy we are running is quite old, we tried an upgrade to the latest (dev21). This fixed the disconnections, but after a few packets, it reliably crashed on the production systems. However

we could not repeat the problem on any test or development systems, even through kickstart and puppet ensured they were identical to production. 

This post details how we found the cause of the problem, so I have a record/teaching aid.

Written by atp

Friday 31 January 2014 at 09:47 am

Posted in Default

gdb useful tidbits/bookmarks

Some of these are hard to google for and I've used more than once now, so to save time I'm bookmarking them here.

  1. Setting watch points in gdb
  2. Displaying the whole string in gdb
  3. Setting gdb to break at a particular file/line
  4. Setting the coredump limit of a running process to catch a core

None of this is rocket science, but still useful.

Written by atp

Thursday 12 December 2013 at 08:32 am

Posted in Default, Linux

Printing and redhat/centos 5.x

Another in the series of this is harder than it should be

This should be a simple enough job - after all 'it works on ubuntu out of the box' (tm).  Take the USB printer and move it onto the home server running Centos 5.9, so that I can share it with the mrs' macbook and my desktop/laptops etc.

Here's the hurdles encountered and the fixes, so my future self won't have to waste time on this again.

Written by atp

Sunday 01 September 2013 at 11:53 am

Posted in Linux


some quick bookmarks

These are really interesting/useful for explaining how the code that executes isn't the code you wrote as well as making you think about performance. A good example is the last one. As Mr Sutter explains, the swap from 80's architectures such as the VAX to modern systems was accompanied by a real change in speed of processor but without a large change in memory latency. As a result "elegant" algorithms that were right for 70s and 80s hardware are totally wrong for modern cache laden systems.

I stumbled over them a while ago, and then spent ages trying to find them again for a colleague. For some reason my google-fu was sadly deficient at work, but once sat back at home I found them within 15 minutes of trying. 

C++ and Beyond 2012: Herb Sutter - atomic<> Weapons, 1 of 2

C++ and Beyond 2012: Herb Sutter - atomic<> Weapons, 2 of 2

Herb Sutter @ NWCPP: Machine Architecture: Things Your Programming Language Neve

C++ and Beyond 2011: Herb Sutter - Why C++?

Bjarne Stroustrup: Why you should avoid Linked Lists - YouTube

Written by atp

Sunday 04 August 2013 at 3:52 pm

Posted in Default