git-atlas is the go-to git extension to handle the ATLAS git repositories. It is meant to be a wrapper around some git commands but not to replace your usage of standard git commands.
It includes several commands: init-config, init-workdir, addpkg and rmpkg, and finally list-pkg.
Setup
To use the git-atlas script, you simply need to run
lsetup git
which will also provide you with a more recent version of git than the default for most distributions.
In case you want to try bleeding edge features, you’ll need to add the following to your *rc file.
source /afs/cern.ch/user/k/karolos/public/git-tools/setup.sh
Options
> git atlas
Usage: git atlas init-config [CERN login] [--apply]
or: git atlas init-workdir [-p pkgList [--]] [-b branch] [-g group] baseRepo [localPath]
pkgList can be a file containing package paths, or a list of packages
To save disk space, baseRepo may be a local workdir of the same project
Options:
-b, --branch Branch you want to check out (for init-workdir)
-g, --group GitLab group to use instead of 'atlas'
-p Packages for sparse-checkout, provided via either
file, or list terminated by option, -- or repo source (baseRepo)
The following flavors are availble from within a clone/checkout:
or: git atlas list-pkg [--all]
or: git atlas addpkg|rmpkg PackageName
Global options:
-h, --help This help
-v, --verbose Verbose output (level 2)
-V, --version Display version
-q, --quiet Quiet output (verbosity level 0)
-a, --all Set 'all' flag for the commands who need it
-n, --dry-run Dry run: do not execute any command
init-config
This is the command that will set you up for using git within ATLAS. It allows setting up settings such as user.name and user.email (these are necessary to be able to commit to a repository) as well as some ATLAS-specific items such as athena: as a shortcut to https://:@gitlab.cern.ch:8443/atlas/athena.git.
To get started, simply run
git atlas init-config
The script will then lookup your email and name in the CERN LDAP database (credentials needed outside CERN/lxplus), assuming your CERN user name is the same as that of your system. If it’s not, you may add it as the first argument: git atlas init-config karolos. Below is the output you’d get by default.
# [git-atlas init-config] No CERN login provided: assuming karolos
# [git-atlas init-config] Fetching ATLAS configuration items
# [git-atlas init-config] The following commands will be executed when setting the --apply option:
git config --global atlas.user karolos
git config --global url."https://:@gitlab.cern.ch:8443/atlas/athena.git".insteadOf athena:
git config --global user.name "Karolos Potamianos"
git config --global user.email "karolos.potamianos@cern.ch"
Note that you may have less options in case you’ve already setup some of these options.
Now let’s call the script with the --apply option:
git atlas init-config karolos --apply
This will result in the relevant items being added to your git configuration file.
# [git-atlas init-config] Fetching ATLAS configuration items
# [git-atlas init-config] Applying configuration settings
+ git config --global atlas.user karolos
+ git config --global url.https://:@gitlab.cern.ch:8443/atlas/athena.git.insteadOf athena:
+ git config --global user.name 'Karolos Potamianos'
+ git config --global user.email karolos.potamianos@cern.ch
Note: if you see the following message in the output
# [git-atlas init-config] Your (global) git user email is not in the .cern.ch domain.
Assuming you did that on purpose. But I'll use your CERN email when checking out an ATLAS repository
don’t worry. This means that you’ve already setup your name and email in git. We’ll leave it like that but we’ll set the committer email for your clone to your CERN email address.
Checking that everything is OK
After you’ve run with the --apply option, you can confirm that all is at it should be using the following command
git atlas init-config
When you’ve already executed the script (or did the adjustments manually), you should then see the following output:
# [git-atlas init-config] No CERN login provided: assuming karolos
# [git-atlas init-config] Fetching ATLAS configuration items
# [git-atlas init-config] You're all set: nothing to add to git config
init-workdir
The init-workdir command is used to setup your working copy of the ATLAS Offline Git repository. Your first workdir will be a clone from a remote repository, https://:@gitlab.cern.ch:8443/atlas/athena.git, and will contain a full copy of all the repository contents. Your subsequent working directories should start off your first workdir in order to save disk space, so as to point to it using symbolic links. This section describes how to achieve this.
To perform your first clone of athena, you can run
git atlas init-workdir https://:@gitlab.cern.ch:8443/atlas/athena.git atlasoff-base -p SCT_Cabling
OR, in case you are not on lxplus and have no Kerberos credentials
git atlas init-workdir https://:@gitlab.cern.ch:8443/atlas/athena.git atlasoff-base -p SCT_Cabling
whose full output at the time of writing is
# [git-atlas init-workdir] Creating new working directoy at atlasoff-base:
* switching to branch main of https://:@gitlab.cern.ch:8443/atlas/athena.git
* using your fork at https://:@gitlab.cern.ch:8443/karolos/athena.git as the repository to push to
* you didn't request any package to be checked out
Initialized empty Git repository in /tmp/karolos/atlasoff-base/.git/
remote: Counting objects: 235721, done.
remote: Compressing objects: 100% (99737/99737), done.
remote: Total 235721 (delta 128385), reused 235561 (delta 128339)
Receiving objects: 100% (235721/235721), 161.19 MiB | 31.87 MiB/s, done.
Resolving deltas: 100% (128385/128385), done.
From https://gitlab.cern.ch:8443/atlas/athena
* [new branch] 19.2 -> origin/19.2
[...]
* [new tag] release/21.0.8 -> release/21.0.8
# [git-atlas init-workdir] Adding some packages needed by CMake: AtlasCMake, Athena
Note: this requirement will be removed in the future.
Branch main set up to track remote branch main from origin.
Already on 'main'
# [git-atlas manage-pkg] Called with action 'addpkg' and package name 'SCT_Cabling'
InnerDetector/InDetDetDescr/SCT_Cabling/ added to sparse checkout
Already on 'main'
Your branch is up-to-date with 'origin/main'.
This cloned a copy of https://:@gitlab.cern.ch:8443/atlas/athena.git (using your Kerberos credentials) into atlasoff-base as a sparse checkout of the main branch, only checking out the package SCT_Cabling (but you’ll still have a complet copy of the repository inside atlasoff-base/.git).
You can see this running
find atlasoff-base -name CMakeLists.txt
and seeing that you only have the file atlasoff-base/InnerDetector/InDetDetDescr/SCT_Cabling/CMakeLists.txt.
Working directory without specifying packages
In case you don’t know what package you want to checkout yet (or want to use list-pkg to get a full list of items), you can run
git atlas init-workdir https://:@gitlab.cern.ch:8443/atlas/athena.git atlasoff-base
whose output is as above, except the end
# [git-atlas init-workdir] You haven't specified a list of packages:
* use git atlas addpkg from your checkout to add some
Initial checkout at /tmp/karolos/atlasoff-base
Use git atlas addpkg to add a package.
You then need to cd to atlasoff-base and run git atlas addpkg PackageName.
Using repository aliases
You can use a shortcut notation (made available to you by init-config) to clone a repository:
git atlas init-workdir athena: atlasoff-base main -p SCT_Cabling
You can create custom aliases using the following command:
git config --global url."FULL_URL".insteadOf SHORT_NAME:
and replacing FULL_URL and SHORT_NAME as appropriate. Note that the colon is compulsory. You can then run
git clone SHORT_NAME:
or
git atlas init-workdir SHORT_NAME: LOCAL_FOLDER
Adding more working copies
Now let’s make another workdir using another package:
git atlas init-workdir atlasoff-base atlasoff-beamspot -p InDetBeamSpotExample
This time, the command ran much faster and you got an output like
# [git-atlas init-workdir] Creating new working directoy at atlasoff-beamspot:
* switching to branch main of https://:@gitlab.cern.ch:8443/atlas/athena.git
* BUT using the .git local repository from atlasoff-base
* using your fork at https://:@gitlab.cern.ch:8443/karolos/athena.git as the repository to push to
* checking out the following packages: InDetBeamSpotExample
Already on 'main'
Your branch is up-to-date with 'origin/main'.
# [git-atlas init-workdir] Adding some packages needed by CMake: AtlasCMake, Athena
Note: this requirement will be removed in the future.
Already on 'main'
Your branch is up-to-date with 'origin/main'.
# [git-atlas manage-pkg] Called with action 'addpkg' and package name 'InDetBeamSpotExample'
InnerDetector/InDetExample/InDetBeamSpotExample/ added to sparse checkout
Already on 'main'
Your branch is up-to-date with 'origin/main'.
Initial checkout at /tmp/karolos/atlasoff-beamspot
Use git atlas addpkg to add a package.
You can note that the script detected that atlasoff-base was already a checkout and told you it will be using .git from atlasoff-base, but that the remotes have been set to https://:@gitlab.cern.ch:8443/atlas/athena.git (pull and push) and https://:@gitlab.cern.ch:8443/atlas/athena.git (pull only).
Do not remove the
atlasoff-basefolder as subsequent working directories point to it
Let’s create yet another working copy
git atlas init-workdir atlasoff-beamspot atlasoff-PixelGeoModel -p PixelGeoModel SCT_Cabling
Which yields
# [git-atlas init-workdir] Detected that atlasoff-beamspot is not a base checkout (i.e. .git is already symlinked)
Using the source of the link (in case you delete the working copy): /tmp/karolos/atlasoff-base/
# [git-atlas init-workdir] Creating new working directoy at atlasoff-PixelGeoModel:
* switching to branch main of https://:@gitlab.cern.ch:8443/atlas/athena.git
* BUT using the .git local repository from /tmp/karolos/atlasoff-base/
* using your fork at https://:@gitlab.cern.ch:8443/karolos/athena.git as the repository to push to
* checking out the following packages: PixelGeoModel SCT_Cabling
Already on 'main'
Your branch is up-to-date with 'origin/main'.
# [git-atlas init-workdir] Adding some packages needed by CMake: AtlasCMake, Athena
Note: this requirement will be removed in the future.
Already on 'main'
Your branch is up-to-date with 'origin/main'.
# [git-atlas manage-pkg] Called with action 'addpkg' and package name 'PixelGeoModel'
InnerDetector/InDetDetDescr/PixelGeoModel/ added to sparse checkout
# [git-atlas manage-pkg] Called with action 'addpkg' and package name 'SCT_Cabling'
InnerDetector/InDetDetDescr/SCT_Cabling/ added to sparse checkout
Already on 'main'
Your branch is up-to-date with 'origin/main'.
Initial checkout at /tmp/karolos/atlasoff-PixelGeoModel
Use git atlas addpkg to add a package.
The command understood that atlasoff-beamspot is not a base clone, thus it did use the its base atlasoff-base. We also provided two packages to add: PixelGeoModel and SCT_Cabling, which were both added.
You are free to remove
atlasoff-beamspotbecause it is not a base checkout, but you should still not removeatlasoff-base.
addpkg, rmpkg and list-pkg
Once you’ve set your workdir up using init-workdir, you can also edit the packages to be checked out using
git atlas addpkg PackageName
where /PackageName/ is going to be searched for in the list of available files, i.e. PixelCabiling will checkout InnerDetector/InDetDetDescr/PixelGeoModel/ but InnerDetector will checkout all packages under InnerDetector/.
To remove a package, use
git atlas rmpkg SCT_Cabling
To list the packages you have checked out, use
git atlas list-pkg
If you don’t remember a package name, you can use
git atlas list-pkg --all PackageName
where PackageName is the regular expression that is going to be provided to grep -i. For example,
> git atlas list-pkg Pixel
# [git-atlas list-pkg] Listing all known ATLAS packages (i.e. paths with CMakeLists.txt) matching 'Pixel'
InnerDetector/InDetCalibAlgs/PixelCalibAlgs
InnerDetector/InDetConditions/PixelConditionsData
InnerDetector/InDetConditions/PixelConditionsServices
InnerDetector/InDetConditions/PixelConditionsTools
InnerDetector/InDetConditions/PixelCoralClientUtils
InnerDetector/InDetDetDescr/SCT_Cabling
InnerDetector/InDetDetDescr/PixelGeoModel
InnerDetector/InDetDigitization/PixelDigitization
InnerDetector/InDetEventCnv/PixelRawDataByteStreamCnv
InnerDetector/InDetG4/PixelG4_SD
InnerDetector/InDetMonitoring/PixelMonitoring
InnerDetector/InDetRecTools/InDetTestPixelLayer
InnerDetector/InDetRecTools/PixelToTPIDTool
InnerDetector/InDetValidation/PixelRTT
You can then addpkg the package you want. Note that using regular expression on addpkg was not added on purpose. We might reconsider this for the future. In the meanwhile, you can maybe checkout the parent folder in case you need a collection of packages.
Advanced usage
For more complex setups, you can provide a text file with the list of packages (absolute path form the root) and pass that as an argument to init-workdir instead of a package name.
> cat myPkgList.txt
InnerDetector/InDetDigitization/PixelDigitization
InnerDetector/InDetEventCnv/PixelRawDataByteStreamCnv
InnerDetector/InDetG4/PixelG4_SD
> git atlas init-workdir atlasoff-beamspot atlasoff-MyPkgList -p myPkgList.txt
# [git-atlas init-workdir] Detected that atlasoff-beamspot is not a base checkout (i.e. .git is already symlinked)
Using the source of the link (in case you delete the working copy): /tmp/karolos/atlasoff-base/
# [git-atlas init-workdir] Creating new working directoy at atlasoff-MyPkgList:
* switching to branch main of https://:@gitlab.cern.ch:8443/atlas/athena.git
* BUT using the .git local repository from /tmp/karolos/atlasoff-base/
* using your fork at https://:@gitlab.cern.ch:8443/karolos/athena.git as the repository to push to
* cheking out the items listed in myPkgList.txt:
InnerDetector/InDetDigitization/PixelDigitization
InnerDetector/InDetEventCnv/PixelRawDataByteStreamCnv
InnerDetector/InDetG4/PixelG4_SD
Already on 'main'
Your branch is up-to-date with 'origin/main'.
# [git-atlas init-workdir] Adding some packages needed by CMake: AtlasCMake, Athena
Note: this requirement will be removed in the future.
Already on 'main'
Your branch is up-to-date with 'origin/main'.
Already on 'main'
Your branch is up-to-date with 'origin/main'.
Initial checkout at /tmp/karolos/atlasoff-MyPkgList
Use git atlas addpkg to add a package.
> find atlasoff-MyPkgList/ -name CMakeLists.txt
atlasoff-MyPkgList//InnerDetector/InDetDigitization/PixelDigitization/CMakeLists.txt
atlasoff-MyPkgList//InnerDetector/InDetEventCnv/PixelRawDataByteStreamCnv/CMakeLists.txt
atlasoff-MyPkgList//InnerDetector/InDetG4/PixelG4_SD/CMakeLists.txt
This mode also allows using regular expressions in the sparse-checkout file.