In addition to the usual centrally produced releases and nightlies it is of course possible to build the release yourself. And for certain developments, this is probably the best way to go — you build the entire Athena code stack yourself, make your changes, compile and run your tests. Of course this requires a powerful machine (don’t try this on lxplus!) and even so will take several hours the first time (subsequent rebuilds should usually be much faster). It will also require a significant amount of space - how much exactly will depend on what you do, but for e.g. a full rebuild of Gaudi, atlasexternals and Athena, you should probably allow about 100 GB.
To start with, please make sure you have read the guide about the releases and the nightlies, so you understand a bit how it all fits together.
Be aware that we depend on the LCG releases, so we’re assuming that you have
cvmfs installed. If not, it’s still possible to install from e.g. RPMs but this is not yet covered in these instructions.
For ATLAS people, you can also check the instructions on the reconstruction software twiki.
Note that these instructions refer to building the projects out of the master branch of athena. Building releases out of other/older branches can be a little different. (Though the overall setup for a full build is generally kept stable.)
Normally you won’t want to compile exactly what was already in a nightly, but rather a version with modifications. So make sure to have a look at building the externals and building athena below.
For the nightlies, the central build machines do the following:
asetup none,gcc112,cmakesetup ./athena/Projects/Athena/build_externals.sh ./athena/Projects/Athena/build.sh
i.e. they use asetup to get a compiler, then call a script to checkout, build, install and package the externals, and finally another script to build, install and package Athena.
You can have a look at the general build scripts under Build/AtlasBuildScripts, and the Project specific scripts are found in the Projects directory. For example, the Athena scripts are under Projects/Athena. The main scripts that matter here are build_externals.sh (which checks out and builds the AthenaExternals project out of the appropriate version of the atlasexternals repository according to Projects/Athena/externals.txt), and build.sh which builds Athena.
The scripts come with help messages about their numerous command line options. At the time of writing these are as follows:
[bash][atspot01]:atlassoftwaredocs > ./athena/Projects/Athena/build_externals.sh -h Script building AthenaExternals. Usage: build_externals.sh [options] Options: -t: Build type, by default set to 'RelWithDebInfo' -b: 'Main build directory' to use for the code checkout and build -f: Force rebuild of externals from scratch, otherwise if script finds an external build present it will only do an incremental build -c: Build the externals for the continuous integration (CI) system, skipping the build of the externals RPMs. -x: Extra cmake argument(s) to provide for the build(configuration) of all externals needed by Athena. -k: Extra 'make tool' argument(s) to provide to the build tool. If a build directory is not given, the default is '../build' relative to the athena source directory. [bash][atspot01]:atlassoftwaredocs >
[bash][atspot01]:atlassoftwaredocs > ./athena/Projects/Athena/build.sh -h Script building Athena. Usage: build.sh [options] Options: General flags: -t: The (optional) CMake build type to use. (default: RelWithDebInfo) -b: The (optional) build directory to use. -x: Custom argument(s) to pass to the CMake configuration. -k: Custom argument(s) to pass to the 'make tool'. -a: Abort on error. Build step selection: -c: Execute the CMake step. -m: Execute the make/build step. -i: Execute the install step. -p: Execute the CPack step. If none of the c, m, i or p options are set then the script will do all steps necessary for a nightly build. Otherwise only the enabled steps are run - it's your reponsibility to ensure that precusors are in a good shape. If a build directory is not given, the default is '../build' relative to the athena/ source directory. [bash][atspot01]:atlassoftwaredocs >
To build externals and Athena (without building any RPM packages) on an “lxplus-like machine” you can go to a fresh directory and do the following:
setupATLAS lsetup git git clone https//gitlab.cern.ch/atlas/athena.git asetup none,gcc112,cmakesetup ./athena/Projects/Athena/build_externals.sh -c ./athena/Projects/Athena/build.sh -acmi
At the time of writing the compiler we use in production is GCC 11.2. You have to be explicit with setting it up,
gcc11would set up the latest GCC 11.X version, which at the time of writing is GCC 11.3. Which is not fully compatible with all of our code at the moment.
For older gcc62-based releases, you should instead do:
asetup none,gcc62,cmakesetup --gccversion=6.2.0binutils
In some cases, you may need to provide a specific cmake version:
asetup none,gcc83,cmakesetup --cmakeversion=3.14.7
You should refer to the list of command line options for the meaning of the recommended options given above. These turn off the build of RPM packages, and make sure that the build would stop right away on errors. (As opposed to how we perform “nightly builds”, which continue on errors as long as they can.)
A full Athena build needs access to CVMFS, AFS and EOS at the time of writing.
You can find the appropriate path for the nightly/branch that you want to build locally, like:
[bash][lxplus722]:~ > asetup Athena,master,latest Using Athena/23.0.8 [cmake] with platform x86_64-centos7-gcc11-opt at /cvmfs/atlas-nightlies.cern.ch/repo/sw/master_Athena_x86_64-centos7-gcc11-opt/2022-10-17T2101 Unchanged: COOL_ORA_ENABLE_ADAPTIVE_OPT=Y [bash][lxplus722]:~ > echo $G4PATH /cvmfs/atlas-nightlies.cern.ch/repo/sw/master_Athena_x86_64-centos7-gcc11-opt/Geant4 [bash][lxplus722]:~ >
On appropriate systems you may need to explicitly mount EOS, like:
export EOS_MGM_URL=root://eosatlas.cern.ch /usr/bin/eos -b fuse mount eos
Depending on your EOS installation, after
kinit and mounting EOS, you may
need to also do:
CMake and the build scripts will make use of GNU Make for the build by default. In order to use some other build system generator, you can pass extra arguments to CMake through the build scripts.
The most useful alternative build system to use is Ninja, a lightweight piece of software, which is much more efficient at dealing with the hundreds of thousands of files and tens of thousands of build targets that we use while building Athena.
To use it, first make sure that the
ninja executable is available in your
runtime environment. In an “lxplus-like environment” you should do this with:
With it set up, you can tell the Athena (or AthenaExternals) build to use Ninja with:
-x "-G Ninja"
If you use the full build for “iterative development”, you should definitely make use of Ninja. And once the first build is done, perform the additional builds like:
./athena/Projects/Athena/build.sh -ami -x "-G Ninja"
In order to use the above, in a fresh terminal you would do:
setupATLAS asetup Athena,<VERSION> --releasepath=<YOURBUILDAREA> --siteroot=/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_Athena_x86_64-centos7-gcc11-opt
If needed, you can help Athena find additional files that may be missing from your local build (as discussed above) like:
To run the unit tests, use the following environment setup:
setupATLAS asetup AthenaExternals,<VERSION> --releasepath=<YOURBUILDAREA> --siteroot=/cvmfs/atlas-nightlies.cern.ch/repo/sw/master_Athena_x86_64-centos7-gcc11-opt cd <YOURBUILDAREA>/build/Athena/ ctest
If you want to test a new version of an external (like Gaudi, Geant4, Acts, etc.), it is often enough to only build a select number of packages on top of an existing nightly, instead of embarking on a full project build. In such cases it is generally enough to only build the external in question, and not the full AthenaExternals project.
The following will specifically talk about building an alternative version of Gaudi, but the same techniques can be used for all other externals as well.
Note that to build externals without using atlasexternals is also possible, but is not in the scope of this page. ATLAS people can for instance refer to this twiki for how to develop Gaudi code like that.
In case the new/alternative version of the external does not need modified
build instructions, you can make use of the
./athena/Projects/Athena/build_externals.sh -b $PWD/gaudi-build -x -DATLAS_GAUDI_TAG="vXXrY.ZZZ" -k Package_Gaudi
You need to check exactly which CMake cache variable(s) you should override in the appropriate package in atlasexternals/External to build the version of the external that you want.
In case you need to modify the build recipe of the external in question, please refer to the instructions given in the atlasexternals repository. But in general, you will be able to build your (modified) package like:
asetup none,gcc112,cmakesetup cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -S ./atlasexternals/Projects/AthenaExternals/ -B ./gaudi-build cmake --build ./gaudi-build --target Package_Gaudi DESTDIR=./gaudi-install cmake --install ./gaudi-build
Once you’re happy with your modifications to atlasexternals, you can test your feature branch’s build using the build scripts of the athena repository with:
export AtlasExternals_URL=https://gitlab.cern.ch/<username>/atlasexternals.git export AtlasExternals_REF=origin/<branch_name> ./athena/Projects/Athena/build_externals.sh -b $PWD/gaudi-build -k Package_Gaudi
To now build a select number of packages on top of a full nightly build, but using your locally installed external(s), you can do the following:
setupATLAS asetup Athena,master,latest mkdir build cd build/ export CMAKE_PREFIX_PATH=<EXTERNAL_INSTALL_DIR>:$CMAKE_PREFIX_PATH cmake -DATLAS_PACKAGE_FILTER_FILE=../package_filters.txt ../athena/Projects/WorkDir/
Note that the directory that you need to point to with
depends on whether you used
build_externals.sh, or you built the external
yourself by hand.
build_externals.sh, you need to do the following wrt. to the “build directory” specified for the script:
Once the externals are built correctly, or in case you just want to use the externals from an existing nightly / release, you still have some options for how to perform a full Athena build.
To build Athena on top of the externals from the nightly, you can use the same recipe as shown earlier, as long as you are in the correct environment. Like:
setupATLAS asetup AthenaExternals,master,latest ./athena/Projects/Athena/build.sh -acm -x "-G Ninja"
Note that the installation step was skipped in this case, since most of the time you don’t actually need it when building Athena. Also note that for incremental builds you should later on just execute:
I.e. you shouldn’t re-run the CMake configuration on every build attempt. Not unless your code modifications explicitly require it.
An even better setup can be to just build Athena “completely by hand”. For interactive code development this results in a much better user experience.
In this setup you would set up your build like:
setupATLAS asetup AthenaExternals,master,latest mkdir build cd build/ cmake -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_EXPORT_COMPILE_COMMANDS=TRUE ../athena/Projects/Athena/ cmake --build .
-DCMAKE_EXPORT_COMPILE_COMMANDS=TRUE argument. If you use it,
code editors like [VS Code]((/guides/vscode)
will be able to find all headers when looking at C++ sources.
If you want to build Athena “by hand” (directly executing the build commands
as shown above), and want to do it on top of a full local build of the
externals performed by
build_externals.sh, then the most convenient
setup to use is:
setupATLAS asetup none,gcc112,cmakesetup source ./athena/Projects/Athena/build_env.sh -b <EXTERNAL_INSTALL_DIR> mkdir build cd build/ cmake -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_EXPORT_COMPILE_COMMANDS=TRUE ../athena/Projects/Athena/ cmake --build .