Inside the Sausage Factory: PART 23 (Detour…derailed by CIFactory)

Well, its been quite some time (23 days!) since my last post on this project on October 20 and I’ve been on quite the journey in the mean time.  It all started innocently enough: with the project now having a unit test or two worth running, I figured the next step would be to setup a Continuous Integration (CI) environment for the SkillPortal project.  Like many .NET-based developers interested in CI, I turned to the old stand-by, CruiseControl.NET (CCNET).

CCNet and CIFactory

I have some experience in using CCNET to setup CI platforms, but this time I figured I’d try something a bit more powerful as part of this working experiment.  My eye turned to something that I had first been introduced to way back when on a DotNetRocks podcast a year or more ago: CIFactory.

CIFactory is the brainchild of Jay Flowers who long-ago realized that when one sets up a CI environment for themselves, there is a large number of repeating tasks that one needs to perform.  If one structures all their projects in a consistent manner, then the tasks needed to setup a project for CI can be codified and formalized.  Anything that can be codified and formalized can be automated and anything that can be automated can be programmed into a computer that can do it better/faster than a person can.  Thus, CIFactory.

As its name suggests, CIFactory is a tool for rapidly building CI environments for your projects.  An open source freeware project, I leave it to the reader to follow the links to get more information (and to download it and use it), but suffice it to say that CIFactory’s raison-d’etre (that’s French for reason-for-being smile_wink ) is that is provides an excellent jump-start on setting up A LOT of the plumbing CCNET expects and offers a wide array of plugin configurations (called ‘packages’ in CIFactory parlance — there’s the French again~!) that enable the speedy leveraging of all kinds of tools like NDepend, NCover. UnitTest runners, and others.  It uses (wraps) CCNET under the hood as its actual engine for your CI environment, but its pre-configurations handle much of the complexity of setting up CCNET and configuring it to work with other tools.

So I started by downloading CIFactory.

CIFactory stumbling block number 1: Assumptions about local working conditions

The first challenge I had with CIFactory was that it makes a large number of assumptions about my working environment that just weren’t true.  To its credit, CIFactory makes extensive use of NANT and NANT properties to enable adopters to override a wide range of defaults, but finding all the places where these defaults existed and experimenting with how and where to override them to get them to match my working environment was non-trivial.  Even with the CIFactory newsgroup on google, blog postings from any number of prior adopters, and other online resources, a lot is left to the imagination/trial-and-error.  Whether the assumption that my local source code is all located under C:\Projects\ or the assumption that my source control control (SCC) system is Subversion, there are a whole host of assumptions that aren’t true of my environment.

Setting about changing them all was quite the journey of trial and error, but eventually I puzzled my way through the complexity of one NANT build script after another with multiple externally-referenced nested build scripts.  Along the way, I gained two things:

  1. A healthy respect for some of the really clever things that Jay has managed to do using NANT with CIFactory to support concepts like extension-through-packages, the abstraction of its use of a source control system that allows anyone to swap-out the baked-in subversion support with one’s own SCC provider, and the same abstraction concepts applied to support someone swapping out its as-delivered support for MbUnit with NUnit, CSUnit, or any other unit test framework (not req’d since we love MbUnit, the default for CIFactory)
  2. A much more significant working knowledge of NANT.  Previously my NANT skills were much weaker and consisted of what anyone would have to consider pretty simple build scripts; as part of my reconfiguration of CIFactory to support my working environment, I had to rapidly improve my NANT-script-writing skills.

Since  my company tends to do consulting, our pathing for our projects tends to be more along the lines of C:\Projects\<clientname>\<projectname> instead of the CIFactory assumption of C:\projects\<projectname> but once that was changed and the several modifications needed to switch our SCC provider from Subversion to SourceGear Vault were completed, I was ready to ask CIFactory to create the CI environment for the SkillPortal project.

At least that’s what I thought.

CIFactory stumbling block number 2: Assumptions about my Vault

At the end of the build process where CIFactory is creating the CI environment for you based on all the settings you need to override to get it to match your working environment, the last ‘step’ CIFactory goes through is to bulk-load all the CI stuff it just created into your SCC repository.  At the end of this process, I got a very friendly-sounding "Build Succeeded" message.

Flush with my success, I inspected the Vault repository to see the results of my labor.  Imagine my surprise to discover that the CIFactory process had just dumped the whole CI environment — build server, build scripts, and all — right into the root of my repository.  That’s right, the whole thing ended up right under $/ in Vault.  Oops.

Obviously, I need to be able to store more than one project in my repository and to do that I need to be able to get CIFactory to create its content under something that looks more like my local folder structure (C:\Projects\<ClientName>\<ProjectName>), $/<ClientName>/<Projectname>/.  Its not at all clear to me how anyone using Vault could accept this behavior from the Vault ‘package’ in CIFactory but I will have to assume that at least some people use a separate Vault repository for every one of their projects and this package must work great for them.

But for me, this was a non-starter.  We rely on things like Vault’s ability to share files and folders from one project to the next in order to keep project dependencies in-sync (with utility and library projects) and this only works if both the sharer and the shared are in the same repository.

Open Source Software to the Rescue

It turns out that a quick posting of this issue to the CIFactory discussion group resulted in some very useful information:

  1. The behavior I was experiencing with Vault had nothing to do with CIFactory per-se; it was more a consequence of the way the Vault ‘package’ had been developed for CIFactory (it was pointed out to me that Subversion supports more than one project per repository in CIFactory, so clearly CIFactory could support this)
  2. The Vault package was contributed by another CIFactory user and Jay didn’t really have much to do with its development or its specific behavior
  3. Jay was very open to anyone offering to extend the Vault Package for CIFactory to support the flexibility I needed from it

Armed with this information, I offered to Jay that I would adapt and extend the Vault package for CIFactory to meet my needs (and by extension presumably the needs of others too).

I was extremely pleased that Jay was not only interested in my doing this, but also willing to assist in providing me any information I might need to help me get over any hurdles, etc. that might come up as I worked to integrate my changes into CIFactory’s build scripts.  Believe it or not, at one time he actually offered to setup a WebEx pair-programming session with me to work through a specific issue I was having — now THAT’S how to build a community around your project! smile_regular

In the end, I was able to adapt the Vault Package scripts for CIFactory to support the notion of there being a specified base path in the repository for each project ($/<folder1>/<folder2>/<folder3>/<ProjectName>), sent the updated package back to Jay, and now its found its way into the next release of CIFactory.  I am extremely pleased to have been able to offer the update back to the project for others to benefit from as well.

This was my first exposure to trying to contribute something to an OSS project and I have to say it was a very pleasant experience; Jay should be commended for his assistance and encouragement along my way.

With CIFactory able to easily create our needed build environment for the SkillPortal project, I was on my way to return to the work on the project.  Or so I thought. smile_sad

More on that next time~!