The best definition of continuous integration comes from Martin Flower: “Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly.”
The data.fao.org implementation
Considering the fact we have a globally dispersed development team that checks code into a version-controlled repository on a daily basis, the continuous integration methodology is the best solution for the development of data.fao.org. The process begins by developers writing their projects based on some Maven Archetypes that I customize.
On a separate virtual server, more specifically a Jenkins-CI server, the Subversion (SVN) repository is polled, runs an automated build, and sends feedback to the development team via email. Every time the code is changed, the following steps are performed on the Jenkins-CI server:
- Checkout code from the SVN.
- Compile source code.
- Run tests.
- Run inspections.
- Jenkins stores the produced .war files in Artifactory.
- Jenkins deploys the produced .war file on the remote server.
- Jenkins runs a downstream project, if present, that performs functional or black box texts.
- Integrate database deployment software in Artifactory and then in the test environment.
Each night, the Jenkins-CI server runs a job to execute smoke tests, or black box tests, against the test environment. Each build is stored on our Artifactory and is available for our developers.
Jenkins and the branching strategy
When a project come closer to the deadline, and the build is stable, we make an SVN branch usually with name “RC-#version_number”. This strategy gives developers the freedom to continue adding new features in the /trunk directory and, in the meantime, fix the bugs that affect only the release candidate on the branch. When the builds on the RC branch are stable according to the metrics and tests we run with Jenkins-CI, we make an SVN tag from the branch. So for each project, we have a job for /trunk directory that deploys in the development environment and one for the branch that deploys in the test/qa environment.p>
On the day of the release, the job that builds from the RC branch is used to build from the tag and deployment is in a production environment.
To handle builds in multiple environments while managing a large amount of resources that can occasionally vary, we need a standardized structure. Thanks to the Archetype and the Maven Profiles, we solved the issue of organizing information in only a few places. Our archetypes contain a default Maven Profile named env-dev. This profile is for developers and it is where they can put the resources used by applications such as the database connection.
There are two other profiles, contained in two different settings one for test/qa and the other for production. These settings.xml files that contains profiles for these environment (named env-test and env-prod). These settings.xml are not contained into archetypes but stored on a secure path.
When Jenkins-CI builds the package it uses the profile according to the environment where the deployment is finished. There are four environments: development, test, QA, and production.
Matter of practice
Continuous integration is a good investment for the data.fao.org development team due to the fact that it takes care of all the tasks that make life hard for developers. Also because behind all technology, including the most viable and promising, there are always people.
Author: Federico Paolantoni