Release Setup

Last update: 19 Jun 2022 [History] [Edit]

For this tutorial we will work on lxplus, so let’s log in to it:

ssh -Y lxplus.cern.ch

For your own work you are by no means required to work on lxplus (indeed it is recommended to work on your own machine). And even this tutorial can be on a different machine, but especially for beginners it is simpler to do the tutorial on lxplus instead of copying all required files to their own machine.

tip From a Linux laptop you should instead use -X in the ssh command.

tip If your user name on lxplus is different you will have to use

ssh -Y cern-user-name@lxplus.cern.ch

There are ways to avoid this, by e.g. adding your CERN user name to your ssh configuration, but if you don’t do those, you need to specify the user name here.

Let’s setup the ATLAS software environment:

setupATLAS

This alias is already defined for you when you log in to lxplus. After you type it you will see a list of commands you can type to setup various ATLAS computing tools.

If you are not working on lxplus: you will need to define these variables: 

export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
alias setupATLAS='source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh'
setupATLAS

We will use the ATLAS Analysis Release to setup our environment and our software in the correct version (such as the versions recommended by the CP groups). You can use the Analysis Release if you have access to cvmfs (lxplus, local clusters, machines, and the Grid).

It is also possible to checkout all software in a given Analysis Release and manually compile them on your working machine. This alternative setup is described below. But for this tutorial we recommend you just use the release from cvmfs on lxplus.

Let’s create a new directory for this analysis from where we will work. Let’s create our analysis working directory and change directory there:

mkdir ROOTAnalysisTutorial
cd ROOTAnalysisTutorial
mkdir source
mkdir build
mkdir run

With cmake we have several directories we work with:

  • source This is where all of our source files go. asetup will also add a couple of administrative files there, but the build system will generally not place any files in there. If you worked with RootCore or cmt before, this used to be the work-area, but has now been moved to a sub-directory.

  • build This is where all the files created by the build system go. If you ever want to start over your build process you can just remove and recreate the directory to start over. As such you shouldn’t put any of your own files into this directory. If you worked with RootCore before, this used to be the RootCoreBin directory. Note that some advanced users also have several build directories (e.g. one for AthAnalysis and one for AnalysisBase, or starting a new one whenever you switch releases). If you are unclear why one would want to have multiple build directories, don’t worry about it, a lot of people work happily with a single build directory.

  • run This is where you actually run your programs, collect your output files, etc. The organization of this directory can be in any way you want, you can create multiple directories, subdirectories, etc.

tip Note that by and large your source directory should correspond to a git repository (or at least some version controlled area). That way if you break your code you have a history of what you have changed. Since the mechanics of version control are largely orthogonal to compiling and running the code, we do not include them in this tutorial, but instead have a separate tutorial for them. However, for any practical work you should put your code into version control or you may run into issues.

tip Note that the naming of these directories is merely a suggestion, none of these names is actually hard-coded in the atlas software. In particular if the source directory is a git repository it is by and large preferable to name it after the git repository instead. And if you have multiple build or run directories you ought to give them names that describe more clearly their different purposes.

Inside the source directory we need a file CMakeLists.txt which will essentially tell cmake that this is a “normal” ATLAS work area. Let’s create this file as source/CMakeLists.txt:

#
# Project configuration for UserAnalysis.
#

# Set the minimum required CMake version:
cmake_minimum_required( VERSION 3.7 FATAL_ERROR )

# Set the name of your project and its version.
# Please note that recent analysis releases _require_ a version to be set,
# otherwise configuring will give an error.
project(UserAnalysis VERSION 1.0.0)

# Try to figure out what project is our parent. Just using a hard-coded list
# of possible project names. Basically the names of all the other
# sub-directories inside the Projects/ directory in the repository.
set( _parentProjectNames Athena AthenaP1 AnalysisBase AthAnalysis
   AthSimulation AthDerivation AnalysisTop )
set( _defaultParentProject AnalysisBase )
foreach( _pp ${_parentProjectNames} )
   if( NOT "$ENV{${_pp}_DIR}" STREQUAL "" )
      set( _defaultParentProject ${_pp} )
      break()
   endif()
endforeach()

# Set the parent project name based on the previous findings:
set( ATLAS_PROJECT ${_defaultParentProject}
   CACHE STRING "The name of the parent project to build against" )

# Clean up:
unset( _parentProjectNames )
unset( _defaultParentProject )

# Find the AnalysisBase project. This is what, amongst other things, pulls
# in the definition of all of the "atlas_" prefixed functions/macros.
find_package( ${ATLAS_PROJECT} REQUIRED )

# Set up CTest. This makes sure that per-package build log files can be
# created if the user so chooses.
atlas_ctest_setup()

# Set up the GitAnalysisTutorial project. With this CMake will look for "packages"
# in the current repository and all of its submodules, respecting the
# "package_filters.txt" file, and set up the build of those packages.
atlas_project( UserAnalysis 1.0.0
   USE ${ATLAS_PROJECT} ${${ATLAS_PROJECT}_VERSION} )

# Set up the runtime environment setup script. This makes sure that the
# project's "setup.sh" script can set up a fully functional runtime environment,
# including all the externals that the project uses.
lcg_generate_env( SH_FILE ${CMAKE_BINARY_DIR}/${ATLAS_PLATFORM}/env_setup.sh )
install( FILES ${CMAKE_BINARY_DIR}/${ATLAS_PLATFORM}/env_setup.sh
   DESTINATION . )

# Set up CPack. This call makes sure that an RPM or TGZ file can be created
# from the built project. Used by Panda to send the project to the grid worker
# nodes.
atlas_cpack_setup()

Now let’s setup the Analysis Release and compile it. We will use a ‘flavor’ called AnalysisBase (which is the general purpose one maintained by ASG. Other version and series are documented on the AnalysisRelease page:

cd build/
asetup AnalysisBase 22.2.76
cmake ../source/
make

tip Note that the asetup command creates a .asetup.save file in the current directory, which you can use by calling asetup --restore to set up the same release again. If you plan to use multiple build directories with different releases, asetup should be called in the build directory. If you do not intend to use multiple releases, it may be better to call asetup in the parent directory to preserve .asetup.save in case you need to delete the contents of build.

tip Note that the setup is virtually the same for AnalysisBase and AthAnalysis (i.e. the standalone vs. the athena based releases), all you have to do is replace AnalysisBase with AthAnalysis. Indeed a lot of this tutorial can be equally applied to either AnalysisBase or AthAnalysis. If you follow the Athena path within the tutorial the above line should be:

asetup AthAnalysis 22.2.76

This tutorial works equally well for both AnalysisBase and AthAnalysis, but to keep things simple you should pick one of them to work through the tutorial. Experience shows that it is very easy to confuse yourself otherwise. You are more than welcome to come back later on and try the tutorial with the other release series, but if you do, do so in a fresh shell (i.e. log in again) and with a new build directory (either remove the old one or create a new one under a different name).

If you are doing this for the first time and as part of the tutorial week you are strongly encouraged to do this tutorial with the AnalysisBase release. This is the only time we work with AnalysisBase in the tutorial week and we will do a separate AthAnalysis tutorial tomorrow.

tip You also have the option to check out the latest version of the analysis release instead of a numbered release:

asetup AnalysisBase 21.2 latest

This is useful if you really need the latest and greatest version of the analysis software, but by and large it is better to stick to a numbered release as it is otherwise difficult to document which version of the software you are using, and thereby for others to reproduce what you have done.

Recompiling

Note that the above instructions are also what you need to do to recompile. We will repeat those instructions a lot:

cd ../build/
make

Or alternatively you can also use the --build option of cmake:

cmake --build ../build/

This will internally go to the build directory and call make there.

warning Sometimes we tell you to run cmake explicitly (typically because we added files or packages). In that case you really have to go to the build directory and call cmake. Calling cmake --build will sometimes not be sufficient.

Building your own release (optional & advanced)

In some situations it can be helpful to build your own release, most commonly because you are working on a machine without access to cvmfs. Actually building a local release can take a while, but it is very straightforward to do (assuming you have cmake installed):

RELEASE_DIR=/...
mkdir release_build
cd release_build
git clone https://:@gitlab.cern.ch:8443/atlas/athena.git
cd athena
git checkout release/22.2.76
cd ..
./athena/Projects/AnalysisBase/build_externals.sh -c
./athena/Projects/AnalysisBase/build.sh -a -c -m -i
mkdir $RELEASE_DIR
mv ./build/install/* $RELEASE_DIR/

tip Building a release is a lot faster if you do it on a local disk instead of a network disk, as the release consists of a large number of small files. On most machines /tmp/ will be a local disk even if all the user disks are located on file servers.

tip The instructions above will install a numbered release, which should be your default to work with. If you want to use the absolutely latest, bleeding edge version you can replace release/22.2.76 with 21.2.

warning Note that with the newer versions of macOS (with the latest version of Xcode installed) you’ll have problems compiling older releases. And unfortunately some numbered releases (like 21.2.36 for instance) don’t build on macOS at all.

To setup this new release just do:

source $RELEASE_DIR/AnalysisBase/*/InstallArea/x86_64-*/setup.sh
cd ../build
source x86_64-*/setup.sh

Note that you don’t actually need a variable $RELEASE_DIR I just used that in this example to make sure we use the same directory everywhere.

If you want to build another (updated) release, you don’t need to checkout the repository again (which is rather slow). Instead it is sufficient to do instead:

cd athena
git pull
cd ..

Use a Docker release image (optional & advanced)

Another option to compile/run your code is within a docker image containing that release. There are a lot of pros and cons to this, which we won’t discuss here. If you are doing this as part of the software tutorial week there will be a dedicated session for Docker as well. This is just the brief/short how-to for using them.

You normally start out by downloading the docker image for the release you want (not strictly necessary, but good practice). The docker images get managed by docker behind the scenes (details in the docker session), so you don’t have to worry about that yourself:

docker pull atlas/analysisbase:22.2.76

tip These instructions will use a numbered release, which should be your default to work with. If you want to use the absolutely latest, bleeding edge version you can replace atlas/analysisbase:22.2.76 with atlas/analysisbase or atlas/analysisbase/latest.

Next you should start up a container from that image, using a command like this:

docker run --rm -it -v $PWD/source:/home/atlas/source -v $PWD/build:/home/atlas/build atlas/analysisbase:22.2.76 bash

There are a lot of things to this command, but just to point out the most important ones:

  • -it ... bash starts an interactive session, i.e. one in which you can actually type commands into a shell.

  • -v .../... makes directories on your machine available inside the container.

  • These instructions assume you are in the directory containing source and build. If you are in a different directory replace $PWD with the absolute path to that directory (i.e. starting with /).

  • We map both the source and the build directory into the container. You should normally keep your source directory outside of the container, but you can keep the build directory completely in the container. All you have to do is leave out the second -v option, and then create the build directory when you get into the image, and rerun cmake every time.

Then inside the container you need to setup your release and build directory:

source release_setup.sh
cd build
cmake ../source
source x86_64-*/setup.sh 
make

Or if you already initialized the build area just do:

source release_setup.sh
source build/x86_64-*/setup.sh

warning You should keep your build directory separate between docker and native builds, and you should also create a new build directory for each release. Or, as indicated above, you may just opt for recreating the build directory each time, which avoids such problems.