How to update to a newer Analysis Release (advanced/optional)

Last update: 01 Jul 2021 [History] [Edit]

It is likely that in the course of your analysis development you will need to update to use a newer version of the Analysis Release. It can be very easy to do this (of course this comes with caveats, especially if you have manually checked out specific package tags, and/or made modifications to those packages).

If you want to update to a newer Analysis Release, you should start a new shell on the machine (e.g. by logging out and logging back in), so that you get a clean environment. Before you setup a new release you need to stop and think a bit ….

  • You may have checked out additional packages on top of the previous Analysis Release you were using (not including your analysis package), as we may have done below:
    • Do you still need a local setup of those additional packages?
    • Did you make any local changes to those packages?
    • How will the version of those packages be different in the newer Analysis Release?

  • Other issues to be aware of is the possibility the tools you are using in your analysis code have changed (if the packages are different between the old and new release).

  • Also, before you follow the instructions below: Did you store any files in the build directory that you may want to back up first? As mentioned before, you shouldn’t really store anything in the build directory that you want to keep, but better double check to be safe.

  • Did you pass any extra parameters to cmake when you created this build directory, e.g. -DCMAKE_BUILD_TYPE=XYZ or -GNinja? If so, you probably want to add them to the cmake call in the instructions below as well.

  • A good piece of advice is to join the atlas-sw-pat-releaseannounce@cern.ch mailing list, where new Analysis Release’s are announced, along with the relevant updates to that release.

Second, assuming the points above are not issues, you can simply remove and re-create the build area as usual, find packages, and re-compile. If you wanted to update to AnalysisBase 9.9.99 (which doesn’t actually exist), you would do:

cd ..
rm -rf build
mkdir build
cd build
asetup AnalysisBase,9.9.99
cmake ../source/
source x86_64-*/setup.sh
make

As mentioned before, some users like to keep multiple build directories around so they can test the new release before getting rid of the old build directory, in which case you’d probably name your build directory something like build-9.9.99 to indicate the release used.

In the past we had instructions that re-used the build directory when changing releases, but there are a number of things that can go wrong with it and if something goes wrong the first piece of advice will usually be to remove and recreate the build directory. Actually we may also give you these instructions for a number of other possible build problems. Since there are very few upsides to not re-creating the build directory when changing releases you may just as well do that by default.

Changing/Recompiling individual packages (optional/advanced)

Sometimes you will need to change the versions of individual packages to a version that is not yet in a release. But before you do that you should double check whether there is a nightly build that already has that version of the package installed. A nightly build is like a numbered release, except that it will be deleted after about a month. Still it is preferable to checking out and compiling release packages locally, and there will be a numbered release with that package before the nightly gets deleted.

If there is no nightly you can use, you first need to clone the repository:

cd ../source
git clone [...].git athena

The [...] here should be replaced with the actual URL for that repository. If you don’t know the URL go to GitLab, which has a field that you can copy and paste into your shell. You may also have to go into the repository to checkout the specific branch you need (e.g. 21.2).

Next you need to create a source/package_filters.txt file that lists the packages you want to keep:

+ athena/PhysicsAnalysis/D3PDTools/EventLoop
- athena/.*
+ .*

tip If you followed the offline git tutorial it will tell you that you should not put the package_filters.txt file inside your athena repository so that you don’t commit it by accident. However since in this case our case the source directory is not the same as the athena repository it is safe to put it there. Indeed in the analysis git tutorial we will show you how to use submodules to control the exact version of the athena repository that gets checked out and then it is handy to have the pre-made package_filters.txt inside your source directory to select the exact packages you want from it.

Now you have to run cmake somewhat differently to tell it which packages to compile (otherwise it will just compile everything in the athena repository which would not only be very slow, but also fail):

cd ../build
cmake ../source

tip If you followed the offline git tutorial you may think this ought to be

cmake -DATLAS_PACKAGE_FILTER_FILE=../source/package_filters.txt ../source

And indeed the two do exactly the same. If you don’t specify a location for the package_filters.txt file and it finds one inside your source directory it will use that instead.