Athena on Docker containers

Generic Docker images for HEP software development

An example docker image is available that allows you to access ATLAS software over a local installation of CVMFS. To download the image do:

docker pull atlas/atlas_external_cvmfs

Then run a docker container with that image with the following command:

docker run -i -t -v /cvmfs:/cvmfs -v $HOME:$HOME atlas/atlas_external_cvmfs

Important: refer to the configuration sections for Mac and Ubuntu for arguments to docker run which are specific to these platforms.

Once the container is running, you will be presented with a prompt where you can setup atlas software as usual:

-bash-4.1$ setupATLAS
-bash-4.1$ asetup main,latest,Athena
Using Athena/24.0.6 [cmake] with platform x86_64-centos7-gcc11-opt
        at /cvmfs/atlas-nightlies.cern.ch/repo/sw/main/2023-07-04T2101

You can now start a ROOT session or run a test job in the container:

mkdir run && cd run
Reco_tf.py --AMI q449 --outputESDFile myESD

You can checkout Athena code from Git and compile it in the docker container. Clone the Athena git repository via https:

git clone  https://[YOUR_USER_NAME]@gitlab.cern.ch/[YOUR_USER_NAME]/athena.git

Alternatively, you can use Kerberos authentication:

kinit yourlxplususername@CERN.CH
git clone https://:@gitlab.cern.ch:8443/yourlxplususername/athena.git

Then you can develop code as you would normally do in lxplus. You can find a full Git tutorial in ATLAS Software Git Workflow. Instructions for saving your changes to an image are found in Save a Docker image.

Full Athena releases in Docker images

Docker images with a complete ATLAS offline installation can be readily produced using a Dockerfile, which constitutes a set of sequential instructions for the docker build command. An example Dockerfile to build a full Athena release can be found in the docker git repository available at gitlab.cern.ch/atlas-sit/docker. To build the image follow the steps outlined below:

Clone the docker repository

git clone https://:@gitlab.cern.ch:8443/atlas-sit/docker.git

Move to the sub-directory named slc6-athena and build the image of an official release:
```
docker build -t athena:21.0.38 --build-arg RELEASE=21.0.38 .
```
The period at the end instructs docker build to use the Dockerfile found in that sub-directory. This specific Dockerfile installs an Athena release. Alternatively, a nightly release can be installed with the following command:
```
docker build -t athena:21.0.39_2017-10-01T2151 --build-arg RELEASE=21.0.39 --build-arg TIMESTAMP=2017-10-01T2151 .
```
The specific nightly for a given release number is set with the TIMESTAMP option.
Run the image
```
docker run -i -t -v /cvmfs:/cvmfs -v $HOME:$HOME athena:21.0.38
```
Refer to the Mac or Ubuntu setup sections for arguments to docker run which are specific to these platforms. For interactive processes (like a shell), you must use -i -t together in order to allocate a TTY for the container process. -i -t is often written -it. To automatically remove the container after you exit, use the extra option --rm.

When installing the main branch of Athena, it may be necessary to patch the base installation for large RPMs. If this becomes necessary, you would need to add the following commands (with the appropriate syntax) to the Dockerfile:

wget http://atlas-software-dist-eos.web.cern.ch/atlas-software-dist-eos/RPMs/rpm/4.8.0_patch/rpm-libs-4.8.0-55.slc6.UNSUPPORTED.x86_64.rpm
rpm2cpio rpm-libs-4.8.0-55.slc6.UNSUPPORTED.x86_64.rpm | cpio -idmv
export LD_LIBRARY_PATH=$PWD/usr/lib64:$LD_LIBRARY_PATH

Useful Docker commands

Configure a container for remote file access

If you have your grid credentials in a sub directory of your local home area, add the following additional option to the parameters of docker run:

-v $HOME/.globus:/home/atlas/.globus

when you setup Rucio with lsetup rucio set the RUCIO_ACCOUNT to the user associated with the credentials.

Additional shells

To open additional shells of a running container use the docker exec command:

docker exec -i -t <CONTAINER ID> /bin/bash

where the hash CONTAINER ID is the container ID of the docker machine. This hash is typically displayed as the hostname in the shell; however, if it is not visible, type:

hostname

or open a new shell and type:

docker ps

This last command prints a list of all running containers with their CONTAINER ID.

Save a Docker image

You can save a docker image locally and restart it using the following commands:

Stop the docker session with the exit command
Save the session with: docker commit <CONTAINER ID> athena:21.0.38, where the hash CONTAINER ID is the hostname or container ID of the docker machine. If the hash is not visible, the following command will print a list of existing containers: docker ps -a
Restart the saved docker image with: docker run -i -t -v /cvmfs:/cvmfs -v $HOME:$HOME athena:21.0.38

Upload a Docker image

If you want to share a docker image with other users, you can create a local snapshot and upload it to Docker Hub. Remember to remove your grid credentials from the image if you had copy them to it.

Create a Docker Hub account at: hub.docker.com
Make a snapshot after exiting the docker machine with, e.g.,
```
docker commit <CONTAINER ID> athena:21.0.38
```
The meaning of CONTAINER ID was explained in the previous section.
Get the IMAGE ID hash with:
```
docker images
```
Note: this is different from the container ID!
Rename/tag the image to match your docker account name and use the previously listed docker image ID hash
```
docker tag <IMAGE ID> username/athena:21.0.38
```
(Replace username with your Docker Hub account name)
Provide Docker Hub credentials with
```
docker login
```
Push the tagged image to Docker Hub with:
```
docker push username/athena:21.0.38
```