Using Grid Credentials

Last update: 03 Dec 2024 [History] [Edit]

In order to access data, submit your analysis jobs to the Grid, and access some restricted web pages, you will need to:

  • Have a Grid certificate
  • Be registered with the Atlas VO
  • Have the certificate installed in two places:
    • in a web browser (Firefox is recommended)
    • in the ~/.globus directory of every machine you use. The ~/.globus/userkey.pem and ~/.globus/usercert.pem files must exist at the end of these instructions.

Most of these steps were already covered in the previous sections. We will now cover installing your grid certificate on your computer for use with the Grid.

Export Certificate From Browser

If your certificate is in your browser, you need to export (backup) the certificate. The procedure for this depends on your OS and your browser. As an example, if you are using Firefox, the procedure is:

Preferences (or Tools) → Advanced → Encryption → View Certificates → Your Certificates → Backup

Export the certificate as myCertificate.p12 (or something similar).

tip If you are using MacOS, do not export it from keychain. Instead, download the p12 file from the link when you create it. (The issue is that MacOS keychain exports a legacy version of p12.)

tip Windows users should use the downloaded p12 file mentioned here instead of exporting it from the browser. (The issue is that the encryption is unsupported.)

More details can be found here.

Install Certificate on lxplus or a Tier-3

Make sure you have a directory called .globus in your home directory on lxplus or whichever remote machine you are using.

tip Note that .globus is a hidden directory because it begins with a period and therefore normally isn’t listed when you call the ls command. To see hidden files and directories, use the -a option (for “all”): ls -a.

If .globus does not exist in your home directory, create it using:

mkdir ~/.globus

Use scp (or similar) to copy myCertificate.p12 from your personal computer to your .globus directory on lxplus:

scp myCertificate.p12 <username>@lxplus.cern.ch:~/.globus/.

After you have copied it to lxplus, log into lxplus for the remaining steps.

First, remove any old .pem files in your .globus directory:

rm ~/.globus/userkey.pem
rm ~/.globus/usercert.pem

Next, check the permissions of myCertificate.p12. They should be preserved when copying to lxplus, but this may not be the case on other remote machines. To be safe, you can set the correct permissions using:

chmod 400 ~/.globus/myCertificate.p12

Then create new .pem files from myCertificate.p12 using:

cd ~/.globus
openssl pkcs12 -nocerts -in ./myCertificate.p12 -out ./userkey.pem
openssl pkcs12 -clcerts -nokeys -in ./myCertificate.p12 -out ./usercert_noText.pem
openssl x509 -in ./usercert_noText.pem -text > ./usercert.pem
rm ./usercert_noText.pem
chmod 444 ./usercert.pem
chmod 400 ./userkey.pem

tip Note that you need to use the file names userkey.pem and usercert.pem exactly for this to work.

tip When executing openssl pkcs12 -nocerts -in ./myCertificate.p12 -out ./userkey.pem, you must enter a PEM pass phrase or it could lead to problems. Make sure to remember this pass phrase in addition to the one you used for the original certificate. These can be the same pass phrase.

tip For MacOS users who exported their certificate from Keychain in .p12 file the encryption standard used might be problematic in LXPLUS (EL9) machines and the conversions to the .pem files might fail. This can be resolved by adding the -legacy option to the openssl commands.

tip You will need to request and install a new certificate annually and it is useful to keep previous certificates in case you have any issues with your newest one. One way to do this is to keep year directories within .globus and copy your new .pem files to the corresponding year. Make sure to keep a copy of your latest version directly in .globus.

Repeat this procedure on any other remote machines you wish to work on or you can copy your .globus directory from lxplus to your home directory on other remote machines. Do not copy it from another remote machine onto lxplus.

From your remote machine, make sure ~/.globus exists and then use the following commands:

cd ~
scp -r <username>@lxplus.cern.ch:~/.globus/./

If you wish, you can remove the original certificate file myCertificate.p12 from lxplus or your Tier-3 machine when you are done. It is a good idea to keep a backup of the file somewhere for safe keeping.

Test Credentials

After your registration with LCG for the Atlas VO has been approved and also your voms roles approved, you can then check that everything is working by doing:

# NOTE: This should be done on lxplus or the Tier-3 machine on which you 
# will submit grid jobs, not on your local computer / laptop
setupATLAS
diagnostics
gridCert

and follow the instructions regarding protections.

All tests must pass as described at the end of the gridCert command.

tip It may take up to 6 hours for the IAM authentication server to sync, so you may see something like “2/4 OK for voms authentication”. Make sure to try again later and it must pass within 24 hours, otherwise your setup is not complete.

tip It may take an hour or so for your grid credentials to propagate to rucio. If you have set everything up correctly but are unable to use rucio, try again in an hour or two.

warning If any problems arise when running gridCert, send the complete file that is produced to the experts. This will make it much easier for them to help you quickly resolve the problem(s).