atp

Some times its hard to find a way of explaining to the technically less literate the power that the Unix shell hands you. The original ideas behing the combination wordprocessor and programmers work bench combined with the power of pipes make text processing easy.

 So to today's problem - analysing GC logs for latency spikes. As we run the jvm with the following properties; 

-XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCApplicationStoppedTime \
-XX:+PrintTenuringDistribution -Xloggc:/path/to/gc.logfile

I settled on piwigo for a web photo gallery, after the demise of ZangZing. I looked at the others and I'm aware that most people use twitbook for sharing family photos, but I don't really trust them with my data in the long term. Plus I'm just too much of a DIY dinosaur to use any of the modern trendy sites like candygram or whatever.  

So here's a short list of pitfalls I encountered whilst getting piwigo up and running, including video.

Tinola was off the air for a couple of days due to a double fault. By which I mean I lost the primary and backup systems in the same powercut. Arguably they should have been in different datacentres. Nonetheless its unusual to get that unlucky, and now I'm considering how to move forward on the hosting front.  

The new server is smaller than the old one, and after restoring from backup, getting Zimbra up and running required some manual tweaks. So this what I found out myself and from various other blogs and web pages on how to make Zimbra run on a low memory system.

It is hard to let things go sometimes. 

Porting Linux to the VAX architecture taught me a lot about the detail of how Linux works, as well as how computer hardware and operating systems interact. It both removed the air of mystery about the kernel, and opened my eyes to the complexity of the GCC compiler and toolchain.

The VAX architecture is simple and easy to understand for a beginner - so much so, that when I started there were several good computer science text books which used it as a teaching system/example. I still have one or two of them on my bookshelf.  

All in all it was an ideal first computer to learn operating system systems programming with, in the same way that the  Talbot Horizon I had at the same time taught me about how engines and cars work - and that rust does not make an effective conductor.

Some aspects of linux have the reputation of being hard. Traffic control via queueing disciplines for bandwidth management for example. Even the title is enough to strike fear into the heart of a seasoned system admin.

Which is a pity really, as the things outlined in chapter 9 of the lartc are very useful in practise.  The problem is the documentation is very descriptive - which is good once you know roughly what you're doing - but which has quite a steep learning curve if you don't. In fact it's pretty vertical if you don't already know quite a lot about networking. A few more worked examples would help over and above those in the cookbook.

Instead, like most people in a rush, I have relied on attempting to bash together snippets of code that are on random blogs to make /sbin/tc do what I want it to do, without really understanding what is going on. 

This time, when presented with a problem for which this is the exact tool, I found I needed to dive deeper, and actually understand it, as none of the precanned recipes worked. It was a case of "if all else fails try the manual".

So now I think I've got a vague handle on what is going on, I'm documenting what I ended up doing because I'm sure I will need a worked example when I come back to this in the future. If its useful to you too, so much the better.

We had to clean up some left over cgroups after another set of experiments with LXC.The guys doing it encountered problems, as the logic is the opposite of what you expect from your experience on a normal unix filesystem.

Specifically the problem happens when you have a nested cgroup - for example /cgroup/foo/bar/

While setting up rhoda's blog on tinola I came across a couple of hurdles converting from one site to multiple sites.

The pivotx manual helps with a section on running multiple websites. But not everything works that smoothly. So, here's what I needed to do to make it work.

Like most people I've got an amazon ec2 cloud instance for all sorts of things. Mine is used as a DNS secondary.

I'm currently fed up with the gyrations that running a proper dns secondary makes me go through. It was not complex until I introduced views. At that point you find that the replication of DNS zone data starts becoming mixed up with the acls on the zone views themselves, forcing you to use multiple IP addresses to replicate both internal and external views of the DNS. Thats painful, and hard to maintain.

When I added the EC2 instance, I also had the problem of dealing with replicating across the internet, for which the usual remedy is to use a VPN. However I found it far simpler to use rsync and make each one believe it is the master for the zones. Its easiest to rsync as the named user, or root itself. However, you don't really want to let privileged users accept stuff from the internet directly, and turning PermitRootLogins on is anathema.

So this post is about how to use ssh forced-command-only and the from options to lock down rsync backups,  as I had to assemble this from bits of blog posts and man pages, and I'll probably need it again in the future.