Principle #2 - Environments should be consistent (Part 1)

This post continues the series The Five Principles of Software Deployment. It covers Principle #2, Environments should be consistent.

An environment includes all the components needed to build and run your application. As an example, take a typical database-driven Java web application, which relies on the following components on the server side:

• Operating system (e.g. Solaris)
• System libraries (e.g. OpenSSL)
• Service providers (e.g. Apache and Oracle)
• Runtime environments (e.g. JDK or JRE)
• Application containers (e.g. Tomcat)
• Application frameworks and libraries (e.g Struts, Spring)

One of the keys to successful deployment is keeping components like these consistent throughout the environments used to build and run your application.

This post discusses consistency in build environments. The next post will look at maintaining consistency across deployment environments.

Having a consistent build environment means two things. First, a build environment should use the same components used to run the application in production. Second, a build environment needs to be consistent between developers.

In an ideal world, each developer would use the same versions of every component involved in building and running an application. Looking at the list above, this means running the same versions of everything from the underlying operating system to the application libraries.

This is not always possible or practical. For example, a developer may use a Windows-based workstation for development, while deploying to a Linux or UNIX-based environment. In this case, consistency can still be maintained further up the technology stack, by running the same version of the JDK, application server, and application libraries.

The following are some practical tips for achieving consistent build environments:

1. Document your application dependencies - The first step to consistency is knowing exactly what is needed to build and run your application. This may seem obvious, but good project documentation is often overlooked. This documentation is especially important for new members that join the team.
2. Version control as many components as possible - This tip is related into the advice presented in previous blog entries about achieving repeatable builds. At a minimum, check in application frameworks and libraries like Struts and Spring and update your build scripts when the versions of these components change. This will ensure that all developers are building with the same libraries.
3. Provide a repository for component downloads - For those components that are not checked into version control, it is useful to have a shared drive or web server where team members can retrieve standard component versions. For example, this repository could contain the standard version of Eclipse used by the team.
4. Develop a strategy for running multiple versions of tools - It is sometimes necessary to switch versions of tools during a project. One example is moving to a newer version of Java - say from 1.5 to 1.6. To make this change cleanly, it is best to have both versions of the component available to allow new development to occur with the new version, while keeping the old version available for running and debugging an older version of the application. Some components, like the Java JDK, allow side-by-side installation, while others are not as easy to set up this way. Our BundleWorks product is designed to allow multiple versions of components to run side-by-side.
5. Build open-source components from source - The thought of building a component from source may make some people cringe, and although it can sometimes be painful,  there are benefits to doing this. The first is that you know exactly what version of a component you are running across all environments, since you are not relying on the version provided by the OS vendor. An example of this is the Apache web server. Different operating system versions include different versions of Apache. Building from source gives you a known, consistent version. Second, building from source allows you to install multiple versions of a component side-by-side. This is done by passing a –prefix argument to the configure script when you build the component (e.g. –prefix=/opt/apache/2.2.6). If you decide to do this, check the original source and the build script (with configure options) into your source code repository.
6. Provide common scripts for setting up a build environment - This tip applies more to UNIX-based operating systems than to Windows. When a user logs into a UNIX-based machine, startup scripts are run based on the user’s choice of shell. In a team environment, these startup scripts should rely on a common script to set up the build environment. This common script should set the standard versions of any tools needed (compilers, runtime environments, etc). Any group scripts should also be checked into version control, as they can change over time.

I’d like to make one final point about consistent build environments. This point involves Integrated Development Environments, or IDEs. I’ve used a number of IDEs over the course of my career and have found them to be helpful. Developers have differing opinions about which IDE, if any, is best. Deciding on a standard IDE for a team is not as important as standardizing versions of other build components. As long as the underlying components are consistent, a developer using vi and Ant can achieve the same end result as a developer using the latest and greatest version of IDEA.