In addition to the usual centrally produced releases and nightlies it is of course possible to build the release yourself. And for certain developments, this is probably the best way to go — you build the entire Athena code stack yourself, make your changes, compile and run your tests. Of course this requires a powerful machine (don’t try this on lxplus!) and even so will take several hours the first time (subsequent rebuilds should usually be much faster). It will also require a significant amount of space - how much exactly will depend on what you do, but for e.g. a full rebuild of Gaudi, atlasexternals and Athena, you should probably allow about 100 GB.
To start with, please make sure you have read the information on releases and nightly builds, so you understand a bit how it all fits together.
Be aware that we depend on the LCG releases, so we’re assuming that you have cvmfs
installed. If not, it’s still possible to install from e.g. RPMs but this is not yet covered in these instructions.
For ATLAS people, you can also check the instructions on the reconstruction software twiki.
Note that these instructions refer to building the projects out of the main
branch of athena. Building releases out
of other/older branches can be a little different. (Though the overall setup
for a full build is generally kept stable.)
Normally you won’t want to compile exactly what was already in a nightly, but rather a version with modifications. So make sure to have a look at building the externals and building athena below.
For the nightlies, the central build machines do the following:
asetup none,gcc131,cmakesetup
./athena/Projects/Athena/build_externals.sh
./athena/Projects/Athena/build.sh
i.e. they use asetup to get a compiler, then call a script to checkout, build, install and package the externals, and finally another script to build, install and package Athena.
You can have a look at the general build scripts under Build/AtlasBuildScripts, and the Project specific scripts are found in the Projects directory. For example, the Athena scripts are under Projects/Athena. The main scripts that matter here are build_externals.sh (which checks out and builds the AthenaExternals project out of the appropriate version of the atlasexternals repository according to Projects/Athena/externals.txt), and build.sh which builds Athena.
The scripts come with extensive help messages (see --help
) about their numerous command line options.
To build externals and Athena (without building any RPM packages) on an “lxplus-like machine” you can go to a fresh directory and do the following:
setupATLAS
lsetup git
git clone https://gitlab.cern.ch/atlas/athena.git
asetup none,gcc131,cmakesetup
./athena/Projects/Athena/build_externals.sh -c
./athena/Projects/Athena/build.sh -acmi
For older gcc62-based releases, you should instead do:
asetup none,gcc62,cmakesetup --gccversion=6.2.0binutils
In some cases, you may need to provide a specific cmake version:
asetup none,gcc83,cmakesetup --cmakeversion=3.14.7
You should refer to the list of command line options for the meaning of the recommended options given above. These turn off the build of RPM packages, and make sure that the build would stop right away on errors. (As opposed to how we perform “nightly builds”, which continue on errors as long as they can.)
A full Athena build needs access to CVMFS, AFS and EOS at the time of writing.
export G4PATH=/cvmfs/atlas-nightlies.cern.ch/repo/sw/main_Athena_x86_64-el9-gcc13-opt/Geant4
You can find the appropriate path for the nightly/branch that you want to build locally, like:
asetup Athena,main,latest
echo $G4PATH
/cvmfs/atlas-nightlies.cern.ch/repo/sw/main_Athena_x86_64-el9-gcc13-opt/Geant4
On appropriate systems you may need to explicitly mount EOS, like:
export EOS_MGM_URL=root://eosatlas.cern.ch
/usr/bin/eos -b fuse mount eos
Depending on your EOS installation, after kinit
and mounting EOS, you may
need to also do:
eosfusebind -g
CMake and the build scripts will make use of GNU Make for the build by default. In order to use some other build system generator, you can pass extra arguments to CMake through the build scripts.
The most useful alternative build system to use is Ninja, a lightweight piece of software, which is much more efficient at dealing with the hundreds of thousands of files and tens of thousands of build targets that we use while building Athena.
To use it, first make sure that the ninja
executable is available in your
runtime environment. In an “lxplus-like environment” you should do this with:
export PATH=/cvmfs/sft.cern.ch/lcg/contrib/ninja/1.11.1/Linux-x86_64/bin:$PATH
With it set up, you can tell the Athena (or AthenaExternals) build to use Ninja with:
-x "-G Ninja"
If you use the full build for “iterative development”, you should definitely make use of Ninja. And once the first build is done, perform the additional builds like:
./athena/Projects/Athena/build.sh -ami -x "-G Ninja"
In order to use the above, in a fresh terminal you would do:
setupATLAS
asetup Athena,<VERSION> --releasepath=<YOURBUILDAREA> --siteroot=/cvmfs/atlas-nightlies.cern.ch/repo/sw/main_Athena_x86_64-el9-gcc13-opt
If needed, you can help Athena find additional files that may be missing from your local build (as discussed above) like:
export DATAPATH=$DATAPATH:/cvmfs/atlas-nightlies.cern.ch/repo/sw/main_Athena_x86_64-el9-gcc13-opt/atlas/offline/ReleaseData/v20
To run the unit tests, use the following environment setup:
setupATLAS
asetup Athena,<VERSION> --releasepath=<YOURBUILDAREA> --siteroot=/cvmfs/atlas-nightlies.cern.ch/repo/sw/main_Athena_x86_64-el9-gcc13-opt
cd <YOURBUILDAREA>/build/Athena/
ctest
If you want to test a new version of an external (like Gaudi, Geant4, Acts, etc.), it is often enough to only build a select number of packages on top of an existing nightly, instead of embarking on a full project build. In such cases it is generally enough to only build the external in question, and not the full AthenaExternals project.
The following will specifically talk about building an alternative version of Gaudi, but the same techniques can be used for all other externals as well. Specific instructions for building against a locally modified version of Gaudi can be found in this section.
In case the new/alternative version of the external does not need modified
build instructions, you can make use of the build_externals.sh
script
directly. Like:
./athena/Projects/Athena/build_externals.sh -b $PWD/gaudi-build -x -DATLAS_GAUDI_TAG="vXXrY.ZZZ" -k Package_Gaudi
You need to check exactly which CMake cache variable(s) you should override in the appropriate package in atlasexternals/External to build the version of the external that you want.
In case you need to modify the build recipe of the external in question, please refer to the instructions given in the atlasexternals repository. But in general, you will be able to build your (modified) package like:
asetup none,gcc131,cmakesetup
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -S ./atlasexternals/Projects/AthenaExternals/ -B ./gaudi-build
cmake --build ./gaudi-build --target Package_Gaudi
DESTDIR=./gaudi-install cmake --install ./gaudi-build
Once you’re happy with your modifications to atlasexternals, you can test your feature branch’s build using the build scripts of the athena repository with:
export AtlasExternals_URL=https://gitlab.cern.ch/<username>/atlasexternals.git
export AtlasExternals_REF=origin/<branch_name>
./athena/Projects/Athena/build_externals.sh -b $PWD/gaudi-build -k Package_Gaudi
To now build a select number of packages on top of a full nightly build, but using your locally installed external(s), you can do the following:
setupATLAS
asetup Athena,main,latest
mkdir build
cd build/
export CMAKE_PREFIX_PATH=<EXTERNAL_INSTALL_DIR>:$CMAKE_PREFIX_PATH
cmake -DATLAS_PACKAGE_FILTER_FILE=../package_filters.txt ../athena/Projects/WorkDir/
Note that the directory that you need to point to with CMAKE_PREFIX_PATH
depends on whether you used build_externals.sh
, or you built the external
yourself by hand.
build_externals.sh
, you need to do the following wrt. to the
“build directory” specified for the script:export CMAKE_PREFIX_PATH=$PWD/gaudi-build/install/AthenaExternals/<VERSION>/InstallArea/<ARCHITECTURE>:$CMAKE_PREFIX_PATH
export CMAKE_PREFIX_PATH=$PWD/gaudi-install/AthenaExternals/<VERSION>/InstallArea/<ARCHITECTURE>:$CMAKE_PREFIX_PATH
Once the externals are built correctly, or in case you just want to use the externals from an existing nightly / release, you still have some options for how to perform a full Athena build.
To build Athena on top of the externals from the nightly, you can use the same recipe as shown earlier, as long as you are in the correct environment. Like:
setupATLAS
asetup AthenaExternals,main,latest
./athena/Projects/Athena/build.sh -acm -x "-G Ninja"
Note that the installation step was skipped in this case, since most of the time you don’t actually need it when building Athena. Also note that for incremental builds you should later on just execute:
./athena/Projects/Athena/build.sh -am
I.e. you shouldn’t re-run the CMake configuration on every build attempt. Not unless your code modifications explicitly require it.
An even better setup can be to just build Athena “completely by hand”. For interactive code development this results in a much better user experience.
In this setup you would set up your build like:
setupATLAS
asetup AthenaExternals,main,latest
mkdir build
cd build/
cmake -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_EXPORT_COMPILE_COMMANDS=TRUE ../athena/Projects/Athena/
cmake --build .
Note the -DCMAKE_EXPORT_COMPILE_COMMANDS=TRUE
argument. If you use it,
code editors like VS Code
will be able to find all headers when looking at C++ sources.
If you want to build Athena “by hand” (directly executing the build commands
as shown above), and want to do it on top of a full local build of the
externals performed by build_externals.sh
, then the most convenient
setup to use is:
setupATLAS
asetup none,gcc131,cmakesetup
source ./athena/Projects/Athena/build_env.sh -b <EXTERNAL_INSTALL_DIR>
mkdir build
cd build/
cmake -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_EXPORT_COMPILE_COMMANDS=TRUE ../athena/Projects/Athena/
cmake --build .
Sometimes it is necessary to build a subset of Athena packages on top of a locally modified Gaudi version.
The build needs to be done in two separate SSH sessions (to avoid mixing Gaudi/Athena environments) and you will end up creating a directory hierarchy like the following in your working area:
- ${GAUDI_INSTALL}
- Gaudi # Gaudi sources
- build # Gaudi build area
- inst # installed Gaudi libraries
- athena # Athena sources
- build # Athena build area
The first step is to find the currently used version of Gaudi and LCG. In case of building against the main
branch check Projects/Athena/build_externals.sh where
you will find e.g.:
ATLAS_EXTRA_CMAKE_ARGS=(-DLCG_VERSION_NUMBER=104
-DLCG_VERSION_POSTFIX="d_ATLAS_19"
-DATLAS_GAUDI_SOURCE="URL;https://gitlab.cern.ch/atlas/Gaudi/-/archive/v38r2.000/Gaudi-v38r2.000.tar.gz;URL_MD5;739b777252581eb13dd159532ea19985"
Then use this information to configure a standalone build of Gaudi. Note that a few components need to be disabled in the Gaudi build
(taken from ExternalProject_Add
in the official externals build configuration):
# for convenience
export GAUDI_VERSION=v38r2.000
export LCG_VERSION=LCG_104d_ATLAS_19
# setup environment (choose release as appropriate)
setupATLAS && lsetup git
asetup AthenaExternals,main,latest
# create build aread and checkout Gaudi
mkdir ${GAUDI_INSTALL} && cd ${GAUDI_INSTALL}
mkdir build inst
git clone https://gitlab.cern.ch/atlas/Gaudi.git -b ${GAUDI_VERSION}
cd build
# build
export CMAKE_PREFIX_PATH=${CMAKE_PREFIX_PATH}:/cvmfs/sft.cern.ch/lcg/views/${LCG_VERSION}/${LCG_PLATFORM}
cmake ../Gaudi -DCMAKE_INSTALL_PREFIX=../inst/${LCG_PLATFORM} -DGAUDI_USE_GPERFTOOLS:BOOL=FALSE -DGAUDI_USE_DOXYGEN:BOOL=FALSE -DGAUDI_USE_INTELAMPLIFIER:BOOL=FALSE -DGAUDI_USE_JEMALLOC:BOOL=FALSE -DBUILD_TESTING:BOOL=FALSE
make install
The above uses the ATLAS mirror (atlas/Gaudi) of the official Gaudi repository (gaudi/Gaudi). It contains the official Gaudi tags (with postfix
.000
) and optionally tags with back-ports of Gaudi patches (e.g..001
). You can also use the above recipe to build against theHEAD
of the official Gaudi repository by replacing of the URL in thegit clone
.
In a new shell, you can now use this build of Gaudi in a regular sparse build of Athena:
setupATLAS
asetup Athena,main,latest
export Gaudi_DIR=${GAUDI_INSTALL}/inst/${LCG_PLATFORM}
mkdir build && cd build
# edit your package_filters.txt
cmake -DATLAS_PACKAGE_FILTER_FILE=package_filters.txt ../athena/Projects/WorkDir
make
source ./${LCG_PLATFORM}/setup.sh