CVMFS Client Installation

Last update: 03 Dec 2024 [History] [Edit]

Introduction

This is the client installation documentation for CVMFS which requires root privileges. Documentation and official description of CVMFS are available and the CVMFS software can also be downloaded from CERN for many OS flavors.

This document describes its installation and configuration for ATLAS users; where possible, either the OSG or EGI installations should be followed as they have Cloudflare support.

Requirements and Preparations

Local Squids (Optional)

Tip Users on the CERN network can use the available squids: http://ca-proxy-atlas.cern.ch:3128;http://ca-proxy.cern.ch:3128

If there are many computers accessing CVMFS like at a Tier1, Tier2 or large Tier3, it will be beneficial to run a local squid server.

Instructions to do this are the same as that for the Frontier-squid server and the same squid to serve both CVMFS and Frontier.

Note the syntax to enter this information in local configuration file:

  • CVMFS_HTTP_PROXY="localsquid"
    • if there is only one local squid
  • CVMFS_HTTP_PROXY="localsquid1\|localsquid2;remotesquid3"
    • This “load-balancing” of the two local squids allows for removing them consecutively for maintenance tasks.
    • If it is a DNS round robin, define the DNS alias instead of defining each proxy. The alias will then be automatically resolved into a load balancing group by CVMFS.
  • CVMFS_HTTP_PROXY="squid1;squid2"
    • This will use squid2 if squid1 becomes unavailable.

Local Cache Requirements

Note that a local cache is needed on a machine for CVMFS clients; it is recommended that this cache be on a local disk so that subsequent access of CVMFS files will be fast and not network limited. Recommendations:

  • The default for Linux /var/lib/cvmfs;
    • if /var is not a separate partition, use another location to avoid filling up the root (/) partition.
  • a cache of 25-50 GB is recommended.
    • only 80% will be usable because of internal db files.
  • if SELinux is enabled, and a different cache dir is defined, then do 
    chcon -R -t cvmfs_cache_t <path to cache dir>

Limit Space Usage

Although 25-50 GB of space is highly recommended, you can set a limit on how much cache disk is used if you have much less space. In the local configuration file /etc/cvmfs/default.local, set the variable CVMFS_QUOTA_LIMIT. You must set the value no matter how much space you have. Remember to set it to only 80% of the available space (the other 20% will be taken up by hidden SQLite files). Consider using an alien cache if the nodes are diskless or have small disks.

Local Configuration File Example

This file will override the CVMFS defaults; you will need root privileges to create and edit it. Create a file /etc/cvmfs/default.local like this example:

# List of ATLAS cvmfs repos to mount and probe
CVMFS_REPOSITORIES=atlas.cern.ch,atlas-condb.cern.ch,atlas-nightlies.cern.ch,sft.cern.ch,sft-nightlies.cern.ch,unpacked.cern.ch

# where the cache files will live; don't use a root (/) partition
#  default is /var/lib/cvmfs so change this only if needed.
#  25-50 GB cache is recommended.
#  if SELinux is enabled, do also
#    chcon -R -t cvmfs_cache_t /home/cvmfs
CVMFS_CACHE_BASE=/home/cvmfs

# This is 80% of the space allocated to CVMFS_CACHE_BASE in MB.
CVMFS_QUOTA_LIMIT=10000

# your local proxies to use if any.  Don't define if no proxy.
#  This is what is recommended for machines on cern.ch domain
#   from https://twiki.cern.ch/twiki/bin/view/CvmFS/ClientSetupCERN
CVMFS_HTTP_PROXY='http://ca-proxy-atlas.cern.ch:3128;http://ca-proxy.cern.ch:3128'

Installations

Linux

This section describes how to install CVMFS client for supported ATLAS Linux OS.

If you are installing at a Tier1, Tier2 or large Tier3 site, you are encouraged to follow the OSG or EGI instructions. These OSG/EGI instructions will configure CVMFS fallback proxies. This is important for the WLCG squid operations team and the stratum 1 operators because it keeps failovers (when local squids aren’t working) from going directly to the CVMFS stratum 1s.

The OSG/EGI configuration also redirects those who set a DIRECT proxy to use Cloudflare for CVMFS stratum 1s; this is a good reason for everyone else to follow these instructions instead of the default ones.

However note that the OSG/EGI may install CVMFS versions that are slightly older than that of the default ones.

OSG Installation

(Recommended for OSG and US sites / users)

Please follow the instructions.

EGI Installations

(Recommended for everyone)

Please follow the instructions. In summary:

yum install https://ecsft.cern.ch/dist/cvmfs/cvmfs-contrib-release/cvmfs-contrib-release-latest.noarch.rpm
cp /etc/yum.repos.d/cvmfs-contrib.repo /tmp
mv -f /tmp/cvmfs-contrib.repo /etc/yum.repos.d
yum-config-manager --enable cvmfs-contrib-egi >/dev/null
yum install egi-cvmfs
yum install cvmfs
echo "/cvmfs /etc/auto.cvmfs" > /etc/auto.master.d/cvmfs.autofs
systemctl enable autofs
systemctl restart autofs
reboot  # for a clean check

Default CERN Installations

(Recommended to use OSG or EGI instead if possible.)

yum install https://ecsft.cern.ch/dist/cvmfs/cvmfs-release/cvmfs-release-latest.noarch.rpm
yum install cvmfs
echo "/cvmfs /etc/auto.cvmfs" > /etc/auto.master.d/cvmfs.autofs
systemctl enable autofs
systemctl restart autofs
reboot # for a	clean check

(Obsolete) NFS export

If anyone needs or are still doing nfs export of cvmfs, please get in touch with ATLAS ADC. There should not be a use case for this.

If you already have a shared file system, the solution to this would be atlas-cvmfsexec and a relocated cvmfs.

MacOS

Install Fuse

  • Download and install FUSE for MacOS.
    • Open “System Preferences” and then “Privacy” and ‘“Allow updates” to software from Benjamin Fleischer.
    • Reboot.
    • Apple Silicon: Use the Startup Security Utility to allow non-Apple Kernel extensions. Entering recovery mode has also Startup Security Utility changed.

Install CVMFS Client

  • Download the latest CVMFS client image as instructed in the link for MacOS.
    • Click on the download; if it fails, you may have to do this:
      • Open System Preferences; Select Security & Privacy; General Tab, Click on “Open Anyway”.
  • Allow macFUSE (“software from Benjamin Fleischer”) to open.
    • This may show up as a fail to mount CVMFS after the reboot.
  • Reboot.

Create Mount Points

The installer will mount CVMFS on /User/shared/cvmfs with a firm link to /cvmfs after reboot. Create the mount points as shown below:

mkdir -p /Users/Shared/cvmfs/cvmfs-config.cern.ch
mkdir -p /Users/Shared/cvmfs/atlas.cern.ch
mkdir -p /Users/Shared/cvmfs/atlas-condb.cern.ch
mkdir -p /Users/Shared/cvmfs/atlas-nightlies.cern.ch
mkdir -p /Users/Shared/cvmfs/sft.cern.ch
mkdir -p /Users/Shared/cvmfs/sft-nightlies.cern.ch
mkdir -p /Users/Shared/cvmfs/unpacked.cern.ch

Create mount / unmount scripts

You can create scripts mount.sh and umount.sh as shown below: Put them somewhere you can run as root.

#! /bin/bash
# e.g. of mount.sh script; needs to run as superuser.  (sudo <path>/mount.sh)
mount -t cvmfs cvmfs-config.cern.ch /cvmfs/cvmfs-config.cern.ch
mount -t cvmfs atlas.cern.ch /cvmfs/atlas.cern.ch
mount -t cvmfs atlas-condb.cern.ch /cvmfs/atlas-condb.cern.ch
mount -t cvmfs atlas-nightlies.cern.ch /cvmfs/atlas-nightlies.cern.ch
mount -t cvmfs sft.cern.ch /cvmfs/sft.cern.ch
mount -t cvmfs sft-nightlies.cern.ch /cvmfs/sft-nightlies.cern.ch
mount -t cvmfs unpacked.cern.ch /cvmfs/unpacked.cern.ch

#! /bin/bash
# e.g. of unmount.sh script; needs to run as superuser.  (sudo <path>/unmount.sh)
umount /cvmfs/cvmfs-config.cern.ch
umount  /cvmfs/atlas.cern.ch
umount  /cvmfs/atlas-condb.cern.ch
umount  /cvmfs/atlas-nightlies.cern.ch
umount  /cvmfs/sft.cern.ch
umount  /cvmfs/sft-nightlies.cern.ch
umount  /cvmfs/unpacked.cern.ch

Make sure those scripts are executable.

chmod +x mount.sh
chmod +x unmount.sh

Now you can mount and unmount them from a terminal by doing

sudo <path>/mount.sh   # to mount
sudo <path>/unmount.sh # to unmount

Checking and Troubleshooting

  • cvmfs_config probe: probe each repo defined in default.local
  • cvmfs_config chksetup: check the configurations
  • cvmfs_config status: check the status
  • cvmfs_config stat -v: show details of stratum 1 and proxies
  • cvmfs_config showconfig: show the configurations
  • cvmfs_config chksetup: check the setup (requires root privileges)

In addition, cvmfs_talk -h and cvmfs_config -h offer much more.

Usage

If setupATLAS is not defined for you, do this in your login scripts. You can type it after a fresh login. (Don’t define this on lxplus; it is already defined for you.)

if [ -z $ATLAS_LOCAL_ROOT_BASE ]; then
    export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
fi

# setupATLAS is defined as a function here
setupATLAS ()
{
   if [ -d $ATLAS_LOCAL_ROOT_BASE ]; then
      source $ATLAS_LOCAL_ROOT_BASE/user/atlasLocalSetup.sh;
      return $?;
   else
      echo "Error: /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase is unavailable" 1>&2;
      return 64;
   fi
}

Getting Help and Mailing Lists