17.5. Continuous Integration

As part of the build migration from Buckminster to Maven new Jenkins jobs have been created.

17.5.1. History - the Buckminster jobs

Currently we have two main types of jobs for GDA: Junit tests and product builds. Junit test jobs are used for verifying Gerrit changes and also run by the timer. Product jobs are usually just run nightly by the timer. There are a few more jobs but they are not commonly run or are irrelevant.

All of these jobs are configured as Jenkins free style jobs. This is the traditional Jenkins job where build steps are created in a web ui with menus, dropdowns, checkboxes etc.

The Jenkins config for these jobs is actually fairly lightweight, and uses the following components:

  • Jenkins global parameter definitions

  • Build triggers (from Gerrit events, or by the timer)

  • Execute shell script

  • Post build actions e.g archive artifacts

  • Trigger another job

The logic for materialize, build and test is invoked via an Execute shell script task that calls a myriad of scripts from the diamond-releng repository. These are mostly shell scripts but one or two key components are large python scripts. It is good that the scripts are in a git repository (rather than directly stored in the Jenkins config) however this repository is somewhat inaccessible/unknown to developers since the diamond-releng repo is not part of a standard GDA checkout. It is also not clear which scripts are used/called/sourced and in what order.

17.5.2. Maven jobs as they stand today

The new jobs are running as Pipeline jobs using declarative syntax. The reference for this can be found here.

This has the following advantages over other approaches:

  • The CI is code in a file that lives in the project’s source repository

  • version controlled alongside the code

  • code review as if it was code

  • more transparent than having CI logic hidden with a Jenkins job or a repository we don’t have in the workspace

  • This approach is used by Github and Gitlab’s CI systems (.travis.yml and .gitlab-ci.yml)

  • Multi branch pipeline allows automatic branch discovery hence we don’t need to manually create new jobs for releases

The build is defined by the Jenkinsfile in the root of the daq-aggregator repository - link to Jenkinsfile.

17.5.3. Key components of the build

17.5.3.1. Clean workspace before

Jenkins workspaces are reused by subsequent runs of the same job, this allows for caching of dependencies, save time by not having to re clone every repository etc. However still need to be careful that certain things from the previous build are deleted before we start.

17.5.3.2. Identify changes to test (incl notify to Gerrit at Start)

For GDA and Dawn this is highly non trivial due to having multiple git repositories. The task boils down to starting with a Gerrit change number or a Gerrit topic name and from that determining related changes across the topic and ancestor changes, checking they are valid (destination branch) and then returning a list of git references to be checked out. Can only have one reference per repository but, of course, this will include parent changes if applicable.

17.5.3.3. Clone repositories

The build needs a clone of all the git repositories required for the build checked out at the appropriate branch (which needs to be up to date). Any repositories with changes to be tested need to have this change present - this procedure for this is to check out the change reference (e.g. refs/changes/75/26475/2) and then merge origin/master (or other branch). This “merge before build” step is important to ensure a change will build against the current HEAD of the repo regardless of where in the history the changes was based off.

17.5.3.4. Build and Test

The actual build needs to be run. This Jenkinsfile stage for this is relatively straightforward:

stage ("Build") {
    steps {
        wrap([$class: 'Xvfb', additionalOptions: '', assignedLabels: '', autoDisplayName: true, displayNameOffset: 0, installationName: 'default', parallelBuild: true, screen: '']) {
            sh "workspace_git/daq-aggregator.git/releng/scripts/ci-build.sh"
        }
    }
}

The Xvfb tool ensures that a unique virtual display is available for the script. The logic is delegated to ci-build.sh. The purpose of this script is to run maven with the correct system properties and maven profiles set.

17.5.3.5. Deploy

An optional step to perform some kind of deployment after a successful build. For example deploy an unzipped product to /dls_sw/.

17.5.3.6. Post build actions

This consists of reading the test results, recording compiler (maven) warnings, archiving any test artifacts and reporting to Gerrit the outcome of the test (if applicable).

17.5.4. The Jenkins Jobs

All of the jobs can be found under the GDA Maven view on Jenkins. The test job is called GDA-maven-tests, the other jobs are all “product” jobs for each beamline. There are also some jobs to handle triggering from Gerrit.