Using Systematics Handles

Last update: 16 Aug 2024 [History] [Edit]

For the common CP algorithms we developed a set of data handles to take care of handling data access and managing systematics. If you haven’t already, please work through the CP algorithm tutorial first.

The systematics handles are similar in concept to the data handles in AthenaMT, so if you are familiar with those you may recognize some of the concepts, but if not don’t worry, the information below should be enough to get you going.

Generally the systematics handles do three things:

  • They take care of the actual read/write access to the event store (TEvent, TStore or StoreGate), meaning you don’t directly talk to the event store anymore, but call the systematics handle instead.

  • They do the actual systematics management, i.e. they know what object in the event store to access for which systematic. One implication of this is that if the input to your algorithm is affected by systematics, then you need to use systematics handles even if your algorithm itself isn’t directly affected by systematics.

  • They add all the properties to the algorithm that are needed to configure their behavior. The default values of those properties are usually passed in as constructor arguments to the handle. There are actually special configuration helpers to take care of setting these properties correctly, and it is usually advisable to rely on those instead of setting them directly. They are still described here, but that is (mostly) meant for experts.

All the systematics handles are defined in the package SystematicsHandles. The way they are written assumes that you rely on C++11 in-class initialization of all handles (as illustrated below). They also try to rely on other modern C++ techniques as appropriate.

For the naming of systematics-varied containers, the handles usually employ a string like MyMuon_%SYS%, in which %SYS% gets replaced with the name of the systematic (NOSYS for nominal). It is generally fine to specify a name without %SYS%, but the implication is that there are no systematics variations on the container. Technically it also doesn’t have to be a container, though it normally is.

Note that we only ever tested systematics handles with AnaAlgorithm, it should probably work with other athena algorithms, but probably not with tools. Overall, you probably have the best experience if you stick with AnaAlgorithm.

SysListHandle: List of Systematics

The SysListHandle takes care of the actual list of systematics to process, and to filter it for the list of systematics actually affecting this algorithm. As such every systematics-aware algorithm needs this handle, and every other systematics handle needs to connect to it (which is also the reason we currently (14 Jan 19) don’t support systematics handles on tools).

The full list of systematics is placed into the event store by the SysListLoaderAlg, as such that algorithm has to run before any other systematics-aware algorithm. For now (14 Jan 19) it is putting that information into the event store on every execute, but at some point we may change it to put the information into the meta-data store instead.

C++ Interface

The declaration in the class header is usually very straightforward:

CP::SysListHandle m_systematicsList {this};

However, you will then have to do some initialization work in the initialize function:

// if you have a systematics data handle, register it here
ANA_CHECK (m_sysDataHandle.initialize (m_systematicsList));

// if you have an ISystematicsTool, retrieve and register it
ANA_CHECK (m_systematicsTool.retrieve());
ANA_CHECK (m_systematicsList.addSystematics (m_systematicsTool));

// initialize the systematics list
ANA_CHECK (m_systematicsList.initialize());

Retrieving the tool and adding the list of affecting systematics does three things:

  • it ensures that the tool actually exists
  • it gives the tool a chance to enter its systematics into the systematics registry
  • it lets the systematics list handle know what systematics we are adding, so it can detect certain kinds of configuration errors

Then in the execute you need to add a loop over all systematics. This uses a C++11 lambda function, but apart from that it ought to be very straight-forward:

StatusCode MyAlgorithm ::
execute ()
{
  return m_systematicsList.foreach ([&] (const CP::SystematicSet& sys) -> StatusCode {
    // if you have an ISystematicsTool, set the systematic we use
    ANA_CHECK (m_systematicsTool->applySystematicVariation (sys));

    // do actual algorithm work

    return StatusCode::SUCCESS;
  });
}

Configuration Interface (experts only)

The handle defines two properties:

  • systematics: The name of the systematics list in the event store. The name can be changed through a constructor parameter, but there is generally no need for that.
  • systematicsRegex: The regular expression describing the list of systematics that affect this algorithm directly (as opposed to affecting it only indirectly through its inputs). This name is derived from the name of the main property.

You may have noticed that we set the list of affecting systematics twice, once in initialize based on what the tool reports and then again as a property from the configuration side. There are two reasons for that:

  • It allows us to double-check that the configuration really knows which systematics affect this algorithm. Should the configured expression not match all the systematics reported by the tool it will be considered an error.

  • It allows us to configure the algorithm to run for more systematics than strictly needed, which could e.g. be used to optimize away some shallow copies which may otherwise be necessary.

This configured list of regular expressions is combined with the systematics affecting any of the inputs, so generally it is neither necessary nor advised to include a systematic just because it affects the input. However, it doesn’t hurt either.

There is a pending feature request to allow not reading the list of systematics from the event store, but instead assuming we only want to process the nominal systematics. This is mostly meant for running CP algorithms the derivations, where we don’t care about systematics and don’t want to run the SysListLoaderAlg. However, that hasn’t been implemented yet (14 Jan 19).

There has also been some discussion to put the systematics list into the meta-data store instead of the event store, as the list of systematics shouldn’t really change event-by-event. Besides some possibly efficiency improvements this would also allow to store the list of systematics in the output file. However, that hasn’t been implemented yet (14 Jan 19) and there could also be potential problems with such a design.

SysReadHandle: Reading Objects

The SysReadHandle takes care of reading objects from the event store that you intent to neither modify nor decorate. If you want to do that, see the SysCopyHandle below. As such it only allows const access to the object retrieved.

C++ Interface

The declaration in the class header is usually very straightforward:

CP::SysReadHandle<xAOD::MuonContainer> m_inputHandle {
  this, "propertyName", "defaultPropertyValue", "property description"};

In the initialize function, it then needs to be added to the SysListHandle (see above):

m_systematicsList.addHandle (m_inputHandle);

And then in the systematics loop it can be accessed similar to how you would call retrieve on the event store:

const xAOD::MuonContainer *input = nullptr;
ANA_CHECK (m_inputHandle.retrieve (input, sys));

Configuration Interface (experts only)

The handle defines two properties:

  • propertyName: The name of the input container.

  • propertyNameRegex: The regular expression describing the list of systematics that affect the input container.

Note that there is no check that the regular expression specified is correct. If it is too loose there will be an error because the corresponding container can’t be found. If it is too tight, systematic variations will be silently dropped. So, if you configure this manually, you better get it right.

Just as for the global systematics list, there has also been some thoughts whether the systematics affecting a container could be stored as meta-data, but nothing has come of it yet (14 Jan 19).

SysReadHandleArray: Reading Multiple Containers

There is an array version of the read handle, that allows to read a user configured number of containers at once. This was meant for MET or Overlap Removal to take as input just a list of IParticleContainer objects instead of having a separate property for each object type. However, currently (14 Jan 19) none of the common CP algorithms is using this.

It mostly works like the regular SysReadHandle, except there is a size() member to check the number of objects to read, and the retrieve method has an extra argument to indicate the index of the object to be read. And on the configuration side the properties are changed from strings to arrays of strings.

Note that despite the name this is not an array of SysReadHandle objects, and the two classes are not related.

SysCopyHandle: Modifying/Copying Containers

The goal of SysCopyHandle is to allow you to retrieve a container and modify/decorate it. As such it tends to be the more common for CP algorithms than read or write handles, as most CP algorithms just take a single input container and do something with/to it.

When dealing with systematics it is quite common that you want to operate on a copy of the input container, instead of modifying the input container itself. Basically if you want to apply multiple systematic variations to the same input container you need to make multiple copies so that those variations don’t overwrite one another. As such, the handle can be configured to make a (shallow) copy of its input container (or not to do so).

C++ Interface

The declaration in the class header is usually very straightforward:

CP::SysCopyHandle<xAOD::MuonContainer> m_copyHandle {
  this, "propertyName", "defaultPropertyValue", "property description"};

In the initialize function, it then needs to be added to the SysListHandle (see above):

m_systematicsList.addHandle (m_copyHandle);

And then in the systematics loop it can be accessed similar to how you would call retrieve on the event store:

xAOD::MuonContainer *container = nullptr;
ANA_CHECK (m_copyHandle.getCopy (container, sys));

Configuration Interface (experts only)

The handle defines three properties:

  • propertyName: The name of the input container.

  • propertyNameRegex: The regular expression describing the list of systematics that affect the input container.

  • propertyNameOut: The name under which to store copies of the container, or the empty string to perform no copies.

All the caveats for SysReadHandle also apply for SysCopyHandle.

SysWriteHandle: Writing New Containers

The goal of SysWriteHandle is to allow you to retrieve a newly created container. As such it doesn’t get used a lot, as in most situations you have some input container that you want to modify/decorate instead.

Due to the nature of the xAOD container, the SysWriteHandle has a second (optional) template argument. This is the associated aux-store for your object. Those generally go together and if you want to add an xAOD container to the store, you need to add the aux-store at the same time. If your object doesn’t have an associated aux-store object you can just omit the second template argument.

C++ Interface

The declaration in the class header is usually very straightforward:

SysWriteHandle<xAOD::MissingETContainer,xAOD::MissingETAuxContainer> m_writeHandle {
  this, "propertyName", "defaultPropertyValue", "property description"};

In the initialize function, it then needs to be added to the SysListHandle (see above):

m_systematicsList.addHandle (m_writeHandle);

And then in the systematics loop it can be accessed similar to how you would call record on the event store, except you do one record call instead of two:

auto met = std::make_unique<xAOD::MissingETContainer> ();
auto aux = std::make_unique<xAOD::MissingETAuxContainer> ();
met->setStore (aux.get());
ANA_CHECK (m_writeHandle.record (std::move (met), std::move (aux), sys));

Configuration Interface (experts only)

The handle defines three properties:

  • propertyName: The name of the output container.

Note that in this case we do not need to configure the list of systematics (and there is no property for it), as that list depends solely on the list of systematics the algorithm ran on, i.e. there is exactly one copy per systematic.

Helper Algorithms

There are a couple of helper algorithms that are specifically designed to help with systematics handling and associated tasks.

SysListLoaderAlg: Load List of Systematics

This algorithm is needed in any systematics-aware job to load the list of systematics into the event-store, so that subsequent algorithms can pick it up and use it. As such it has to be loaded before all other systematics-aware algorithms.

The default way of using this algorithm is to pick up the list of systematics from the systematics registry. As CP tools get created they register their systematics into the registry, making it very easy to retrieve the complete list of systematics and run on all of them:

sysLoader = createAlgorithm ('CP::SysListLoaderAlg', 'SysLoaderAlg')
sysLoader.sigmaRecommended = 1

Or to run all the 2-sigma variations instead of the 1-sigma variation you can just change the associated property:

sysLoader.sigmaRecommended = 2

While this is fine in some situations, you may want either more control over which systematics are run, or may simply want to have the certainty that the list of systematics doesn’t change as you reconfigure your tools or algorithms. In that case you can specify the list of systematics directly (and leave out sigmaRecommended):

sysLoader.systematicsList = [...]

This algorithm will also print out the list of systematics that are contained in the systematics registry on job startup, so if you quickly want to check the list of systematics that’s one place you can take it from.