Principle #1 - Builds should be repeatable (Part 3)

This is the last in a series of posts discussing repeatable builds, which is one of The Five Principles of Software Deployment. This posts picks up where the previous one left off, presenting more advice on achieving repeatable builds.

4. Automate your build

According to the Joel Test, from respected author Joel Spolsky, one of the 12 steps to better code is being able to answer “yes” to question #2 “Can you make a build in one step?”.

I agree with this wholeheartedly, and I work toward achieving this goal on all my projects - large or small.

Depending on the languages and technologies used to build an application, there are a number of tools available to automate builds.

For most languages, the venerable make tool can be used to automate a build. General purpose build tools like scons are also available, which provide improvements over make.

There are also tools that are more specific to a particular language or technology like Ant, Maven, and Buildr (for Java), NAnt (for .NET), rake (for Ruby).

These tools can be used to provide automated, “one step” builds.

An automated build system should provide the following:

• A command line method for building. Building from an IDE is not recommended for achieving repeatable builds.
• A way to build dependencies as part of the build. Some tools like Maven, handle dependencies, while other tools have inspired third-party approaches to dependency managment (Ivy for Ant)
• A consistent method for building applications on a project. In one project that I worked on, we ended up using Ant to call “make” for the C/C++ portions of the project, so that every project could be built with a single ‘ant’ command.
• A way to use Continuous Integration servers. In a future post, I’ll discuss Continous Integration, but having an automated build is a necessary first step to using a CI server.

For repeatability, having an automated build system is key and worth the effort it takes to set up.

5. Create a separate build user

This is one of the most important practices for achieving a repeatable build.

As developers work on a project, their local environments invariably end up collecting files and settings that allow them, but no one else, to build an application properly. Sometimes these local differences are subtle, and an application will build fine for other team members, but will not work in exactly the same way. These local differences make it impossible to achieve a truly repeatable build.

Creating a separate build user and applying strict control over this user provides a neutral third-party to settle these “build differences”. The approach is to create a separate user both on the host operating system and in the version control system. On UNIX, this can be accomplish with a shared user that team members can ‘su‘ into.

Building a release is then done as follows:

1. Login or switch to the build user.
2. Delete all files from the previous build.
3. Checkout all files from version control that are needed for the build.
4. Build the release.

By removing all previous build files and checking out a fresh set of files, you ensure that all files have been checked in and that no local differences will impact the build. After a successful build, this “build user” would then label or tag the files in version control.

On one of my previous consulting jobs, this approach was taken a step further. In this organization, a separate team was created to perform builds that were released into production. As developers, we would make sure that everything was checked into version control and would then notify this group that we were ready for a release build. At times, this process seemed like overkill, but in the end, I think the advantages outweighed the disadvantages.

6. Document your build process

No matter how simple or complex your build process is, the steps need to be documented. It is a fact of life that people come and go over the course of a project. Without good project documentation, it is difficult for new team members to learn the correct way to build and deploy applications. On a project with a complex build process, this can be a serious problem.

Over the years, I’ve encountered a variety of approaches to documenting the build process. These include:

• Storytelling (project elders passing on the “lore of the build”)
• Document management systems
• Team or organization-wide wikis
• Project documents in version control

Each of these has particular advantages and disadvantages (although I would be hard pressed to find too many advantages to the Storytelling approach).

I tend to favor an approach that stores documentation inside the version control system. This documentation can then be labeled or tagged when a release is done. If the build process and its documentation changes later, the state of the documentation is still preserved at the time of the release.

I also tend to favor plain text documentation over binary formats like Word. Plain text files can be opened by any text editor, from vi to Eclipse. Text files are easier to diff and easier to search. Even for larger documents, I prefer text-based formats like Markdown, XHTML/CSS, and even Docbook.

Regardless of the tools used to document the build process, have good documentation for the build process is essential to achieving repeatable builds over time.

Next up: Principle #2 of the Five Principles of Software Development - Environments should be consistent.