build

CITCON Europe 2008. Tom Sulston proposes a session for CI developers and vendors to show off their wares. The Continuous Integration Cage Fight is born. Seven geeks. One prize of … nothing.

Your humble narrator was given the job of recording the battle, which he did by juggling an Asus Eeepc, and a Nokia N73. Given all that, it’s not a shock that the video quality isn’t good. For CITCON Europe 2009 (registration is open) he’s packing a Flip Ultra.

This is a list of all the write-ups, with video. The fighters were:

• Ivan Moore (build-o-matic)
• Paul Julius (CruiseControl)
• Eric Lefevre (Hudson)
• Chris Read( Cruise)
• Jason Sankey (Pulse)
• Yegor Yarko (Team City)
• The man from IBM (BuildForge)

The winner was .. everyone. You’re all using CI. Think of those poor souls who aren’t.

Image by army.mil

This is the final (and very late) installment in the CI cage fight series. The last speaker was Leuwie from IBM, discussing IBM Rational Build Forge.

By this stage in the talk I was done with trying to take notes (curse you, cute eeepc that caught Tom’s eye!) and film at the same time. Things got a little sketchy. Here’s what I managed to note down:

The licensing model is by concurrent user. As it truly is an enterprise product, you can’t just go quoting fixed prices ( as different clients have all sorts of deals for buying their gear). We were a tough audience in that respect, as most of us come from a small product/small team background.

BuildForge understands environments. My notes said ‘environments are mapped’: that was probably something profound six months ago (on the bright side, it’s less than six months to the next CITCON!).

Build agents have manifests that describe their capabilities. Pretty standard. It will parse build output and fail on if a given string is detected. It also supports LDAP. Good.

BuildForge runs commands as lowest common denominator for integration. That includes VCS access.

It has wide platform support. I think Leuwie mentioned something about Nintendo support as well - which means NetBSD support. Officially, it supports:

Windows, AIX, Solaris, HP-UX, Linux, Mac OS X, z/OS, i5/OS

This is possibly the best agent support that I’ve ever seen in a CI server and that’s what I’d say a uniqe feature is. If you’re already drinking the IBM Kool-aid, then it’s probably a very good fit for you. Which brings me to my main point; I’m probably not the guy to be commenting on this stuff.

I might have worked with WebSphere, DB2 and AIX. I’ve even installed OS/2. Still don’t really understand the culture - even though I apprecate what IBM’s research has brought to our industry. So all I can do is say thanks to Leuwie for fighting in the cage, and a huge thanks to IBM Nederland for hosting a great weekend in Amsterdam.

I’m happy to say that I’m going to be at three conferences this year. First, I’ll be speaking on Continuous Integration at the JAOO Developer Day in Copenhagen, Denmark. March 4th.

I’ll be doing a similar talk a few days later at QCon London, March 11-13.

Finally, I’ll be at at CITCON Europe in Paris on September 18, naturellement.

It’s a great honour to be invited to speak at the first two, and to be helping organise the latter. After a quiet year for conferences in 2008, 2009 is looking good already.

Great news: we have a date for CITCON Europe - and it’s in Paris!

What is CITCON? It’s a free Continuous Integration and Testing conference.

• If you’re a developer interested in how to better craft your code with tests, you’ll fit right in.
• If you’re a tester looking to automate tests more effectively, welcome.
• If you run a Continuous Integration system or just happen to like CI, this is for you.

It’s an Open Space conference: there’s no keynote and no timetable. You help choose and organise the sessions. You get a say in how it’s run. Did I mention that it’s a free conference?

Last year I only went to one conference. This one. I paid for the travel and accommodation myself. CITCON Europe. Because it’s worth it.

Update:

CITCON is also coming to North America in April, and Australia in July.

Image taken from Al Ianni’s Photostream

Second to last in the talk was Team City, presented by Yegor Yarko of JetBrains.

They’ve done a very good job of the tool and used their position as developers of IntelliJ and Resharper to promote this product very cleverly: so I don’t think I need to do much introduction to the tool. Here’s the notes I managed to scratch down:

• Free for 20 developers and 20 projects
• Enterprise license $3000, more build agents (you get 2 or 3 free) are $300 each
• Configuration is via xml file or web app
• Build agents are secured: you have to grant them access
• Unique feature: Gives failed test feedback before build finishes.
• Unique feature: IDE plugins (IntelliJ, Eclipse, Visual Studio)
• Unique feature: auto commit (it runs your tedious build and then commits to your repo - while you dine)
• It has static code analysis features
• It is very developer friendly
• Unique feature: stop a build [though I think others can do that]
• Very good .NET support
• They might open source some of the plugins

Yegor also did a great job photographing the conference. Link

Jason Sankey of Yutubi presented Pulse. Pulse is a closed source (though they will let you peek! see below) CI server. Jason and his team set out to make something easy and flexible, with a config GUI and an easy install.

What I like is that it scratches a very painful itch: duplication in the config. It uses a template system so that you only ever need to specify something once. With some other servers, you’ll need to specify the same facts (yes, Ant is in /usr/bin, just like the last time). Not so with Pulse.

Pulse also supports:

• Distributed, multiplatform builds
• Private builds
• Role based security

It’s reasonably priced: free for open source, or up to 2 users and 2 projects, up to 4000 USD for the enterprise license which I believe gives you access to the source. Jason told us over dinner that they can easily accommodate towards firms that need escrow or source access. Apparently there’s a decent plugin API. The guys just released Pulse 2 a few days ago.

Your build files probably need some TLC. Sorry. It’s true. If you’re lucky enough to work on a project with clean code (and let’s face it, most codebases aren’t too pretty), then you’re even luckier to have a nice clean build. Wait! Don’t go getting upset, I can help you.

I’ll be introducing some build refactorings in this series. You can refactor code, so why not your build files? I wrote an article about this in the ThoughtWorks Anthology. The Pragmatic Programmers own that now so I’m writing some short articles for this blog. Please consider using the Amazon link above if you want to buy a copy of the book - it will help the cause.

Today’s refactoring is extract target. This example is written in Apache Ant syntax. It holds equally true for NAnt and even other build tools. Consider the following build file:

<project name="unrefactored" default="compile_and_configure_web_app">

	<target name="compile_and_configure_web_app">
		<delete dir="build" />
		<mkdir dir="build/classes" />
		<javac srcdir="src" destdir="build/classes"/>
		<!-- the config file  -->
		<copy file="src/web.xml" todir="build/web.xml">
			<filterset>
				<filter token="DB_HOST" value="oradvdb17"/>
			</filterset>
		</copy>
		<mkdir dir="build/WEB-INF/lib" />
		<!-- build the jar file -->
		<jar  jarfile="build/WEB-INF/lib/webapp.jar"
				basedir="build/classes"
				manifest="src/manifest.mf">
		</jar>
		<!-- and the war file  -->
		<war file="build/webapp.war"
					webxml="build/web.xml"
					manifest="src/manifest.mf"
					basedir="build">
			<include name="**/*.jar" />
		</war>
	</target>

</project>

It’s a bit obtuse, isn’t it? It’s doing several things at once and it’s difficult to see what’s going on.
Let’s start by teasing it apart into it’s constituent parts:

<project name="refactored" default="default">

	<target name="clean">
		<delete dir="build" includeemptydirs="true"/>
		<mkdir dir="build/classes" />
	</target>

	<target name="web.xml">
		<copy file="src/web.xml" todir="build/web.xml">
			<filterset>
				<filter token="DB_HOST" value="oradvdb17"/>
			</filterset>
		</copy>
	</target>

	<target name="webapp.jar">
		<javac srcdir="src" destdir="build/classes"/>
		<mkdir dir="build/WEB-INF/lib" />
		<jar  jarfile="build/WEB-INF/lib/webapp.jar"
				basedir="build/classes"
				manifest="src/manifest.mf">
		</jar>
	</target>

	<target name="webapp.war" depends="webapp.jar,web.xml">
		<war file="build/webapp.war"
					manifest="src/manifest.mf"
					webxml="build/web.xml"

					basedir="build">
			<include name="**/*.jar" />
		</war>
	</target>

	<target name="default" depends="clean, webapp.war" />

</project>

I think that’s much nicer. You don’t need XML comments in the middle of your target line because each smaller target can have a descriptive name.

When you make a minor change, you’re likely only to change one target. You can contain your changes, and if you’re debugging you can run just one target if you need to. And most importantly, it’s readable. I find that I tend to naturally do this as I evaluate a complex build. Tool support would be nice.

Note: I’m not going to even pretend to be unbiased about this one. I used to work for these people. I was involved in hiring Chris, and I was involved in some early discussions about previous incarnations of this product.

Chris Read presented Cruise, from ThoughtWorks Studios. Loads of people think that this product is the payware version of CruiseControl. If Cruise is a relative of CruiseControl, they are third cousins, once removed. And they don’t look alike. It’s almost completely a from scratch rewrite, and by the time I write this, the last bit of CruiseControl code should be gone from it.

Cruise has all the features that you’d come to expect in a commercial CI server these days: distributed build facility, whizzy UI, repository of built artifacts, etc. There’s two features that I’d like to talk about: the Build Pipeline and the movies.

When your CI build starts to get uncomfortably long, what do you do? Assuming you can’t shorten it, you can break it up into stages: one for developers so they know that their unit tests passed, another to demonstrate that functional tests pass. If you want to get really fancy, you can map to each stage of the QA process. This whole idea has been dubbed “The Build Pipeline” by ThoughtWorks and it seems like a reasonable implementation.

A couple of the key aspects: you can retreive built artifacts from the first build to use in later builds. So no need to rebuild artifacts or build your own repository. Also, the tool tracks all the builds so you can see that one developer checkin spawned all these builds. That’s mighty useful.

The other useful feature is that it records Flash movies of your Selenium tests. That may not sound like much, but have you ever tried to diagnose a broken Selenium build when it’s not on your PC? I have sat there eyeballing the thing waiting for it to fail. It’s not the kind of feature that you CIO will froth over, but your developers will love it.

Cruise costs money and is closed source software. Two build agents are free. There’s a bunch of other features. LDAP integration is always a good ‘un.

(Image taken from bortescristian’s photostream)

Update: I had some nice feedback from @jtf @nasrat and @cread, so I have pushed the source files to a git repository.

What’s the equivalent command to [some command in your usual build tool] in [some build tool that you're using now]? I found myself asking that question too often earlier this year, so I set out to make a tool that would help. How does it work? The tables of commands are stored in a Markdown file, which is rendered into HTML and PDF with the sweet Maruku ruby library. It’s built via CruiseControl and published to the Apache docroot if it generates successfully.

Link