Principle #4 - Releases should be easy to do and undo

After a bit of a break to meet some client deadlines, I’d like to continue the series The Five Principles of Software Deployment. This entry discusses the fourth principle in that series: Releases should be easy to do and undo.

Doing a software release can be as simple as installing a new version of a single component on a single machine, or it can involve upgrading multiple components across multiple machines along with configuration and database changes. Undoing a software release, i.e. rolling back if the newly released software doesn’t work properly, can be a challenging, if not impossible, task.

This blog entry provides tips on how to make your software releases easier to do and undo.

Know your environments

The most important part of a software release is knowing your environments. You should be able to answer the following questions:

• What versions of the software (if any) are running in the target environments?
• Are the correct versions of all dependencies installed in the target environments?
• What OS versions are running on the machines and is the release compatible with these versions?

If you keep your environments consistent, it is easier to answer these questions.

Have proper backups

This point is especially important for being able to undo a release. You should ensure backups for the application itself, its configuration, any runtime files produced by the application, and any databases used by the application. Be skeptical of regular backups that are performed. I’ve seen cases where these backups were not being done properly or could not be restored. It is better to take a snapshot of all components at the time of release.

Document all release steps

Good documentation is essential to a good software release. Even if a release is completely automated, it is important to document the changes that will take place. This documentation must also include rollback steps. In a perfect world, rollback steps would not be necessary, but the world of software development is far from perfect.

Here are some of the things to include in release documentation:

1. List of machines included in the release.
2. List of components, with versions, to be released on each machine.
3. All steps necessary to do the release.
4. For larger releases, an hour-by-hour plan with time estimates and checkpoints
5. All steps necessary to undo the release.
6. List of tests to be performed to verify the release.
7. List of people who need to sign off on the release.
8. List of emergency contact names and phone numbers.

Automate the release to an appropriate level

Automation can help avoid human error during a release. I’m a big fan of automation. But there are some important points to remember about automation:

• Any code that automates a release is still code. This automation code must be tested thoroughly, including all use cases - installation, upgrade, and rollback.
• Automation is not a substitute for good documentation or for understanding exactly what will happen during a release. Blind reliance on automation can make diagnosing problems more difficult during a release.
• There is always a trade-off between the benefits gained from automation and the time spent to automate. An example would be writing and testing sophisticated database rollback code when a simple database dump and restore would do as well.

Test the release

You should have a staging environment where release steps can be tested before the final production release. Also, you should get someone else to test the release. Having another person follow your release steps is a great way to make sure that the release is documented properly.

Use BundleWorks

This tip is a shameless plug for our BundleWorks product. BundleWorks has features that help to make software releases easier to do and undo, including:

• Allowing multiple versions of an application to co-exist. This allows you to keep older versions around while you upgrading to newer versions, making it easier to roll back.
• Allowing multiple environments to co-exist on the same machine. This allows you to easily set up a staging environment for testing a release.
• Application management functions which allow you to see what versions are installed where.
• Standard hooks and a library of functions to help automate installing, upgrading, and rolling back applications.

If you want to give BundleWorks a try, you can download the latest version.

The last of the The Five Principles of Software Deployment, Failures should be obvious, will be covered in the next blog entry. Stay tuned.