Continuous Integration, Delivery and Devops Consulting
(+44) 207 183 0323
hello@build-doctor.com
@builddoctor
Best. DevOpsDays. Ever. Just found my talk video. Much love to Tom and Patrick.
devopsdays Goteborg 2011 – Julian Simpson , Everything I learned about Continuous Integration , I learned from System Administra from devopsdays on Vimeo.
Continuous Integration matters. Configuration Management matters. What happens when you mix the two?
Let’s find out. Chris Read and I are speaking at the BCS Configuration Management group meeting on November 1. We’ll be doing our talk from Agile 2011, which I hope will spill over into a massive discussion over nibbles and wine.
The venue will be the BCS offices in Southampton Street, London. You need to register for security and catering purposes. Chris and I will be talking from 18:30. We’ll be preceded by the CMSG AGM, which non-members are welcome to attend (though you can’t vote unless you’re a member)
[link]
Atlassian kicked off the Bamboo Task Master Challenge today. I am honoured to be on the judging panel.
What’s it all about? A Continuous Integration server becomes the heart of your development infrastructure. CI servers need to know how to talk to the rest of your systems.
So Atlassian are encouraging you to add to the Bamboo ecosystem by offering the winner of the competition:
The first 10 entries get an Atlassian T-shirt and the Bamboo Keyboard and Mouse.
I think it’s a clever move. Plugins made Hudson/Jenkins more attractive to it’s users, and gave everyone the incentive to encourage more plugin authors (see Crossing the Chasm for more on this)
Here’s an idea to kick things off: I’d like to see someone use face-detection and face-recognition libraries to detect the face of the person who broke the build.
An targeting system for a Nerf Vulcan wouldn’t go amiss, either.
[link]
I was honoured to give a talk at DevOpsDays Gothenburg on Saturday. Much love to Patrick Debois for encouraging me to submit the talk, and huge props to my erstwhile colleague Tom Sulston for agreeing to co-present. The whole thing was a blast.
2004 will let you keep Subversion a little while longer.
Hello from Gothenburg. Here is the news.
… this is a repository. I’ve been experimenting with Amazon S3 as a Maven repository. I threw the results up on GitHub [link].
Why? I’ve been working with some nice people who use binary dependencies. In order to scale their CI system past one node, they need a repository manager, to temporarily store built artifacts on.
They actually have a couple of repositories, but not geographically close to where I’m building out their new CI system (on EC2). It’s important to keep the feedback loop fast; so deploying a repository close to(or indeed on) on the CI server is desirable.
The plan was to deploy Archiva onto an EC2 instance via Puppet. Then I had a better idea: use an S3 bucket as a highly scalable, low latency and (mostly) highly reliable repository.
Turns out that there’s support for S3 in their build tool (Maven). So it turned out to be easy. In the github repo, I publish and artifact using an S3 client, and then retrieve it using plain HTTP.
As long as your EC2-hosted Continuous Integration server is in the same availability zone as the S3 bucket, you’re not going to be liable for a high traffic bill.
They may want to use a different configuration for the real repository, where they serve artifacts to the other consumers. This approach is all about speeding up and scaling out their build.
Update: Oliver Lamy asks: ‘what is wrong with Archiva’?
Nothing. I’d cheerfully use it. It has a permissive license and cheerful absence of public spats with other repo managers.
In this case I wanted some very fast and robust, which led me to S3. The alternative was to deploy Archiva on an EC2 instance: I could have done that, but then I would have had a dependency on a single host.
IMO, this approach is more suitable for the context (caching intermediate build artifacts, on AWS).
Where the hell did 2011 go? Here’s where to find me for the rest of the year:
It looks like a lot when you write it down.
From my email inbox, here’s the gossip:
I announce all CI and build tool releases or news. If people tell me.
(Thanks to Jason Sankey of Pulse for this Guest Post)
On behalf of the Zutubi team, I’m excited to announce the latest release of our continuous integration server, Pulse 2.4! Before I go into details, I’d like to touch briefly on one of the areas we’ve been working on for this release that doesn’t get a lot of attention because, frankly, it’s not very “sexy”. It is, however, an important reality for anyone that has to maintain a build server. I’m referring to the termination of runaway builds.
A runaway build occurs when not all processes created by the build exit cleanly. Catching and killing all processes created by a complex build can be difficult. Processes that last beyond their intended lifetime – zombies – may hang the build, or simply stay around in the background waiting to wreak havoc. Zombies are a source of multiple headaches:
The pain of zombies us something we do our best to eliminate with Pulse. Pulse has always had the ability to kill builds after a timeout, but not all builds cooperate. So in this latest release we’ve added some new process terminationl logic, leveraging platform-specific code and tools where we can.
First, I’d like to give a plug to a little library called javasysmon, from Jez Humble (our friendly Thoughtworks competition). Although incomplete and dormant for some time, this library provides a starting point for richer process discovery and manipulation through Java APIs. We’ve employed it as one prong in our attack against zombie processes. Given the Java platform’s anemic support for process control, we’re hoping to build on (and contribute to) this effort over time.
The combination of Windows and Java provides extra challenges for process control. Windows doesn’t have true process trees, rather process groups are the preferred way to manipulate related processes. However, Java APIs provide no way to create or manipulate process groups. On the upside, Microsoft ship a handy utility by the name of taskkill with all recent Windows versions. Pulse has no aversion to employing external platform tools where they can help, so our zombie razing toolbox includes use of taskkill when it is available.
If a good flailing via Java APIs, amputation with javasysmon and evisceration with taskkill all fail to stop the zombie hordes, Pulse 2.4 also offers a shotgun-to-the-head fallback: the kill build action. This new action is the kill -9 of the Pulse world: it cuts the process loose and wraps up the build state in Pulse immediately. We’d prefer it never came to this, and will continue enhancing our automated weaponry to avoid it, but a last-ditch way to restore order is better than taking your build server down.
Zero zombie tolerance is just one of the improvements in Pulse 2.4. Other significant updates include:
If you’d like to learn more about Pulse 2.4, check out the new in 2.4 page on our website for details and a few screenshots. Or you can join the war on zombies by downloading and trying Pulse 2.4 for free today.
Disclosure: Jason’s competitor sponsors this blog.
The Build Doctor Ltd, Suite 17189, Lower Ground Floor, 145-157 St John Street, London, EC1V 4PW
© Build Doctor 2007-2012
