Outsourcing Continuous Integration

by Julian Simpson on October 8, 2009

Updates: Added Atlassian on December 17

A deal has been made

Outsourcing Continuous Integration isn’t a new idea, but we’re seeing more and more traction in the space. The headline news is:

  • It’s certainly not for everyone.
  • The space is going to get more and more interesting as cloud services increase.

Not for everyone

If you’re at all paranoid about security, you’re unlikely to want outsourced CI.

You might fall at the first hurdle: will you be able to justify outsourcing the build of your most valuable asset to an auditor? If you can’t address the (low) risk of your code being subverted, it might be game over. Perhaps you can prove that no code built at your outsourced service is used. It also raises the question of your version control system. Is it inside your firewall? Do you need to provide secured access to your outsourcing partner? Perhaps you outsource version control elsewhere. Can those parties talk? Could there be a man-in-the-middle attack?

What about some more practical reasons for keeping it in house? You might depend on internal services for your build. What’s your internet connection like? Do you mind if you lose your connection to the Internet, and therefore the outsourced continuous integration server?

Cooler tools

Can your IT department supply you with Linux, XP, Vista, and now Windows 7 with a host of different browsers? Of course not. They aren’t there to deliver a glittering array of choice in operating systems. Your friendly local IT department is there to drive down the cost of computing by stamping a uniform operating system onto all your computers. Your helpful IT vendor is there to help said IT department, being the guys who pay Bill’s bill.

It’s a good thing Amazon branched out from selling dead trees. The Amazon Web Services tool-set is amazing. Want somewhere to keep all those built artifacts? Then how about S3? Need a few dozen build agents? EC2 is your friend. We’re really just getting started here. One of the most obvious uses for the cloud is in allowing you to test all those pesky client configurations: those permutations of Windows, IE and Firefox, for example. I predict that Continuous Integration vendors will quickly reach feature parity on this, because it’s so darn useful.

Such services will become more specialised as more service models evolve. Need to test with your enterprise stack? I imagine you’ll be able to piece together some of those components as well. Will there be an API for submitting builds to any build farm? I certainly hope so.

In the medium term, I’m not convinced that many CI servers will end up fully hosted on the cloud. What’s more likely is that many enterprises will end up with:

  • One big, hand-rolled build machine, hosted at the firm.
  • Lots of nodes in the cloud.
  • A really freaking big Amazon EC2 bill.

This works, because you get to assume that you’re protecting the your assets, and just giving your built code a workout out there in cloud-land (I also predict the rise of compromised cloud servers, FWIW). You still need to deploy the app somewhere and fire up nodes to test against it, but you are limiting the opportunities to inject malicious code at build time. This allows you to keep built artifacts (be they in a Maven-style repo, or just spat out from an Ant build) on the inside of your network (ironically where you probably face the most realistic risks of attack – by disgruntled or financially compromised employees).

Perhaps some of the cloud vendors will acquire enough security certifications to convince auditors that it’s safe to use. And maybe, enough organisations will start thinking of operating systems and middleware as bigger code objects to play with via an API or toolset, rather than infrastructure to manage with a meatcloud.

Some vendors

So who actually provides outsourced Continuous Integration? This is by no means an exhaustive list. Tweet me if you have suggestions for the list. Thanks.

  • Collabnet offer Team Forge, which looks like it used to be SourceForge Enterprise Edition. Remember that? I worked at a bank that used it. Happy times. [mainly due to NPR and Peet's Coffee. Though SFEE did work reasonably well for a large programme of work]
  • Run Code Run – have built off the back of GitHub with a sweet little model – they consume hooks from GitHub, and trigger from those to build your Java apps. They are branching out from Ruby projects to include Java as well, and will rent you a private CI system by the month.
  • CI in a Box is an Amazon EC2-based solution. I’m not sure who’s making money off of this one apart from Amazon – the house always wins. Looks like low cost and scalable Hudson implementation, anyway.
  • Mike CI contacted me the other day – they have a new service – operated out of the UK, but available everywhere, of course. They are pre-launch, but they seem to be in a similar space to Run Code Run – allowing developers to easily adopt CI. They support Java but might also offer .NET. I’ll try and get something more in-depth, and pounce on them for an interview if they come to London. They seem really nice.
  • Atlassian just joined the game with JIRA Studio, their outsourced suite of tools. This is a good play from them: they have a strong brand in JIRA, and they are leveraging it.
  • Electric Cloud offer a tool that can be fully or partly cloud hosted.  It’s not clear who offers this as a managed service or not.  I’ll ask them.
  • CI Foundry is also new, and in super-alpha. This is a bespoke service, so aimed at companies who want things done for them, or in situations where the standardised offerings don’t fit. DISCLAIMER: I’m behind this one. I’m going to be open about this. Compromising my editorial integrity would feel dirty. I’ll even try and get someone else to do reviews is there’s a problem.

Are you using outsourced Continuous Integration? Do you want to share your experiences? Tweet me!

Image thanks to Orin Zebest

Share with the group:
  • Digg
  • del.icio.us
  • Facebook
  • DZone
  • LinkedIn
  • Slashdot
  • StumbleUpon
ThoughtWorks (UK) Ltd is looking for a Configuration Manager @ ThoughtWorks UK
Looking for a job? Got a position to fill? Check out the Job Board.

Related posts:

  1. Continuous Integration in the cloud: good idea? Continuous Integration can be tricky to provision. It’s IO...
  2. Sun enters the Continuous Integration business Sun sell a Continuous Integration server. We should have...
  3. The quest for a decent Ruby Continuous Integration tool Git has become very popular in the Ruby community....
  4. Continuous Integration and Release wisdom Can’t recommend these enough. Item one: this book. Release...
  5. Every Continuous Integration server that supports Ruby + Git As I posted, I’m on a quest for the...

Related posts brought to you by Yet Another Related Posts Plugin.

Tagged as: outsourcing

  • My take on this is that hosted CI in a common virtualized environment such as EC2 won't work. A CI or a build server, unlike the rest of the applications, needs all four components of a build box, CPU, RAM, disk and network I/O. The industry wisdom says applications that are subject of virtualization may demand maximum two. Sure you can run a build in EC2, but you will have to sacrifice build speed, and that's usually the last thing you want to do. If you want fast builds, you have to run in the opposite direction, towards a dedicated, big fat box hosted locally.

    Viewtier has been hosting Continuous Integration for open source projects for five years, and our experiences shows that even builds on a dedicated build box begin to slow down if the number of long-running builds exceeds a double of number of CPUs. Actually, we observe a trend towards farms of build machines hosted locally.
  • @douglassquirrel: Okay. I guess that the elasticity of price depends on the size of organisation and the domain. Certainly what you just said is about 100% true for companies that do financial services.
  • No, no! Cost is _not_ the first issue for security of cloud CI. Steps have to be:
    1. convince techie that cloud CI is a good idea. (Existing providers do this.)
    2. convince techie that cloud CI will pass audit. (Need reference customer. Chicken and egg.)
    3. convince auditor to pass cloud CI. (Requires pen and the ability to tick boxes all day. Your first customer will give you a sample list - how did you get that first customer again?)
    Only consider cost once you've solved 2. and 3. The costs of in-house CI are so high that once you've done this you can almost certainly find someone willing to cover your bills plus a decent margin.
  • simpsonjulian
    @douglassquirrel, thanks for the insightful comments. I've pondered the security a lot. I guess the problem is going to be cost for security. Nail that one and you're done. I think VPN's are going to resolve a big chunk of those issues, and proving that there's no theoretical risks to be had at the other end. For example, I could outsource someone's CI services to a dedicated machine in a secure enclosure in a datacenter, It wouldn't be cheap, but I could do it. Finding clever ways to resolve that problem would be the challenge.

    Winning some Ebay auctions to test out ancient operating systems would be a laugh. The cost would be securing Windows 2000 and 98 from the outside world, probably with an old-fashioned Cisco firewall to block traffic from anywhere but the CI systems. You're right in that someone would have to scale up to make it worth while. Or the other approach is to try and run them all in emulators, or Wine, but that might invalidate some testing.
  • Another thought: you are right that all vendors may soon offer a wide variety of modern OSes on their build agents. A way to differentiate would be to offer _old_ OSes. Imagine the pain of a company who has important clients still on Windows 2000 or 98 (this is not unheard of, believe me). You can't even buy those systems any more except maybe on ebay or something, and forget maintaining them (even in a VM they must be a nightmare). Dev teams in this situation may have no other way to run CI other than to outsource to someone who can afford the setup cost.
  • Very interesting stuff! Nice to see longer post from you.

    Sometimes it happens that in a busy space like this, someone hits on the magic combination of features and cost and service and takes over all of a sudden. Google and iPod are easy examples. Wonder if cracking the security conundrum is what will do it here. It is certainly one of the very few barriers we have to adopting cloud CI.
blog comments powered by Disqus

Previous post: Video: Paul Julius Interview

Next post: Video: Jeffrey Fredrick Interview