Develop Code

Last update: 28 Sep 2020 [History] [Edit]

Create a Topic Branch

Before you start development create a space that will hold your code changes in an easily identifiable way. In git this is done by creating a new branch:

git fetch upstream
git checkout -b master-my-topic upstream/[parent_branch] --no-track

Why git fetch upstream first? This is to make sure that your local repository is completely up to date with the main repository before you start work on your patch.

What is [parent_branch]? This is the release for which you are developing your code: it could be the current Tier-0, Trigger or development branch (currently these would be branches 21.0, 21.1 and master respectively). Giving upstream/[parent_branch] as the starting point ensures that you are starting from the most recent version of that code. In this tutorial we use upstream/master as this branch will always exist and be functional.

The -b option to checkout creates a branch and then checks it out, ready to work on. (It’s a shortcut for git branch ... plus git checkout ....)

git fetch upstream
git checkout -b master-my-topic upstream/master --no-track

Alert We use --no-track to avoid git thinking you want to push and pull directly from upstream (this is more or less harmless, but can be confusing). If a parent branch were not given then the parent would be the current branch - you can use this form of branching when you want to use branches locally for tests or speculative development (it’s shown in every git tutorial under the sun).

Your new branch can be called a bugfix branch, a feature branch or a topic branch depending on it’s purpose (and which documentation you are reading). In this tutorial we’ll generally use the term topic branch.

The name of the topic branch can be anything, however, as in the normal workflow it’s visible to others when you make your merge request we recommend choosing a descriptive name, e.g., master-trk-optimise-selection. It’s no bad thing to put the name of the branch you will merge into as a prefix to remind you what the target branch is.

Good Branching in git is very fast and cheap. Take advantage of this and use branches a lot to organise your developments.

Tip You can check all of the origin and upstream branches git knows about with git branch -a. This allows you to see which target branches their are on the main repository (or you can also just browse the branches with GitLab).

If you started development before you made your topic branch…

It’s easy to do, but as long as you did not stage or commit your code, then git will keep your working copy changes in your checkout when you create your topic branch.

Warning If you started editing a piece of code on the wrong branch then things are usually more difficult. Git will not allow you to change branches if a modified file is different in each branch:

$ git checkout master-my-topic
error: Your local changes to the following files would be
 overwritten by checkout:
	Tools/PyJobTransforms/python/trfExe.py
Please commit your changes or stash them before you
 switch branches.
Aborting

Using a git stash can be useful way to try to recover if you made significant changes you don’t want to loose.

Development

Making changes to the code you can now do in your favourite editor or IDE. Do remember that you have the power of a local git repository at your fingertips:

  • You can commit code locally anytime, to make it easy to rollback to a known development snapshot
  • You can even do more speculative development on a branch from your main topic branch - merge it in if it works, throw it away if it doesn’t

It’s recommended to commit early, commit often for every coherent piece of the larger change you are working on.

Setting up to compile and test code for the tutorial

You now need to setup a base athena release on which to add your code changes. So setup a suitable release like this in a new build area:

mkdir ../build && cd ../build  # Assuming you start in the root
                               # of your git clone, but in fact
                               # you can do this anywhere outside
                               # the source area
asetup master,latest,Athena

The asetup command will setup a runtime environment for the master branch build of Athena that was built most recently. There is more information on working with nightly builds here including how to check which releases are actually available at the time you do the tutorial.

Here we are picking the runtime for the branch we want to work with, which is of course the same as our target branch.

If you ran the normal cmake; make cycle you would rebuild the entire offline release - this would take a long time and it’s almost certainly not what you want. Instead you want to only rebuild part of the release, which is a sparse build.

To do that take a local copy of Projects/WorkDir/package_filters.txt and edit it to list only the packages you want to build. Then run CMake using the options shown below:

cp ../athena/Projects/WorkDir/package_filters_example.txt ../package_filters.txt
EDITOR ../package_filters.txt
cmake -DATLAS_PACKAGE_FILTER_FILE=../package_filters.txt ../athena/Projects/WorkDir
make -j

Tip The syntax of package_filters.txt is very simple:

  • +PATH_REGEXP adds these PATH entries to the build
  • -PATH_REGEXP removes these PATH entries from the build
    The first match wins.

Tip As in the above example keeping your package_filters.txt in a different directory from the build is wise, so you can rm -fr * your build area. Just don’t keep it in the git repository as there is a danger you would commit it by mistake.

Bad Using a sparse build is extremely flexible, but it’s an extra step. The alternative sparse checkout is be easier in most cases.

Local testing and code quality

As with all ATLAS CMake builds you will need to source your build area setup to run properly. e.g.,

mkdir ../run && cd ../run
source ../build/x86_64-slc6-gcc49-opt/setup.sh
Reco_tf.py --AMI q431  # Or your own favourite test!

Though note that your architecture (x86_64-slc6-gcc49-opt) may be different.

Good developers test their code during development and we are sure that you are no different. For new features try and develop unit tests if you can.

Making a Local Commit

Git commits code in two phases:

  1. You add changed files to a staging area with git add
  2. You commit the staging area with git commit
$ cd ../athena
$ git status  # Always a good idea to see where you are...
branch master-my-topic
Changes not staged for commit:
 (use "git add <file>..." to update what will be committed)
 (use "git checkout -- <file>..." to discard changes in working directory)

 modified:   python/trfArgClasses.py
 modified:   python/trfExe.py

no changes added to commit (use "git add" and/or "git commit -a")
$ git diff    # ...and what you changed

$ git add python/trfArgClasses.py python/trfExe.py
$ git status
On branch master-my-topic
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

	modified:   python/trfArgClasses.py
	modified:   python/trfExe.py

$ git commit
... # Editor should open and you write a commit message
    # see next section...

git add can take wildcards and it’s possible to add all changed files and new files automatically with git add -A, but be very careful not to add unwanted files to the repository. Definitely use git status to check what will be/had been staged. Note that git status also helpfully tells you what to do if you want to rollback changes to a file or unstage files.

Tip git reset will unstage changes for you if you did an add of something in error.
git checkout FILE will revert FILE to the original version.
git reset --hard will unstage and discard all local changes.

Tip git status is your friend - consult it often!

Commit messages

Commit messages should be informative yet concise. It is suggested that you write a short one line explanation, followed by a blank line, then some additional longer paragraphs explaining the patch. Use less than 72 characters per line - commit logs get read at the terminal. Reference any bugs you are fixing in the usual Jira notation (e.g., ATLASRECTS-3430) and GitLab will post a comment on the Jira ticket, which is useful in correlating a bug with its fix.

The Linux kernel recommendations state:

For these reasons, the “summary” must be no more than 70-75 characters, and it must describe both what the patch changes, as well as why the patch might be necessary. It is challenging to be both succinct and descriptive, but that is what a well-written summary should do.

Further guidelines are in How to Write a Git Commit Message and 5 Useful Tips For A Better Commit Message.

An example of a good commit message would be:

Fixed uninitialised value and leak in alignment tool (ATLASRECTS-98765)

It was possible to have the m_alignmentOffset variable
uninitialised in certain situations outlined in Jira.
This patch corrects for that and changes the behaviour
to return StatusCode::FAILURE immediately if the
setup is inconsistent at initialise.

The memory leak from failing to delete the pointer returned
from the foo tool was fixed.

The commit message should say why something was changed, not what was changed (so it’s not necessary to list changes by filename).

Tip With git it’s extremely easy to see the code changes alongside the commit message:

$ git log --oneline path/to/mypackage # List only commits
                                      # that changed this path
... # Identify the COMMIT_ID you are interested in
$ git show COMMIT_ID

Alert ATLAS does not use ChangeLog files anymore. The commit message is the new change log. This is why it’s really important that you write it well. If you make a mistake use

git commit --amend

to edit it before you push anything. Software review shifters will be instructed to reject merges with bad commit messages.

Push Your Changes

Once your development is finished and tested, copy it from your local repository back to your GitLab fork:

git push --set-upstream origin master-my-topic

The --set-upstream (or just -u) associates your topic branch with the copy on your fork. So subsequent pushes can be a plain git push.

Your topic branch is now available on GitLab and now you can make a merge request.

Working with a nightly build

Usually it’s sufficient to work with a branch of Athena, as discussed above:

git fetch upstream
git checkout -b master-my-topic upstream/[parent_branch] --no-track

but sometimes you want to work with exactly the same code version as in the release you’re working with. In this case you can checkout the nightly tag - the code as it was when the nightly was built.

asetup master,Athena,2020-09-27T2101
git checkout "nightly/master/2020-09-27T2101"
#or alternatively, using the generic environment variables:
asetup master,latest,Athena
git checkout "$AtlasReleaseType/$AtlasBuildBranch/$AtlasBuildStamp"
#For a release build it's different, since the nightly tags are archived after a while.
#There is always a stable tag pointing to the release
asetup 22.0.15,Athena
git checkout release/22.0.15
###make changes
git checkout -b my_topic_branch_from_a_nightly_tag
#You can now make a merge request as usual

This is particularly useful if you don’t intend to make changes but just see the code as it is in a nightly/release build, or if your code is likely to be affected by changes in other packages that you don’t compile.