Monitored Chains

• Introduction
• Implementation of monGroups in the Trigger menu
• Storage of monGroups
• Access to monGroups
  • Python access
  • TriggerToolWeb access

Introduction

In Run 3 menu-aware monitoring is facilitated through “monitoring groups”, monGroups for short. MonGroups are properties that can be assigned to an HLT chain in the Trigger Menu defining the signature group which monitors the HLT chain via a signature flag and the monitoring level at which the HLT chain is monitored.

Signature tags:

Monitoring levels:

• For offline/t0 monitoring:
  • shifter: highest level; chain is always monitored and is used to fill the histograms shown in the SHIFTER folder, chain is monitored both in T0 and validation jobs
  • t0: second-highest level, includes shifter level; chain is always monitored at T0 and can be used to fill histograms in the EXPERT folder, chain is monitored in both T0 and validation jobs
• For online monitoring:
  • online: chain is monitored online

Offline/T0 monitored chain histograms are viewed through the validation web-displays and the Run DQ signoff web displays, where experts of all trigger signatures review the performance of the triggers and main reconstruction algorithms for each data-taking run and MC/data validation reprocessing.

The production of DQ signoff displays in T0 for each data stream can take from a few hours to a few days, depending on the data rates for each stream. Because of the high prescales and low rates, the express stream will usually finish processing shortly after the run is stopped, and is used for the first DQ sign-off. The complete list of DQ web displays is available in the DQ Monitoring dashboard. The choice of histograms and their location in the DQ web display is controlled by the han configuration files in DataQualityConfigurations/config/HLT. The complete set of histograms is available from the HIST file corresponding to each run and stream, retrievable through RUCIO.

Online monitored histograms are actively monitored at P1 primarily by the Trigger and DQ shifters, through the DQMD and OHP applications. Similar to the DQ web displays, the location and choice of histograms to be shown in the applications is controlled through standalone configuration files, in this case in the format of XML files. A mirror of the current P1 configuration is available in the operation_tools_configurations repository. Any changes to the configurations can be prepared by the signature experts in a fork of that repository, and will then be copied to P1 by the DQ experts. More information on the update procedure can be found here. Both online monitoring applications can also be accessed through web simplified interfaces: web DQMD, and web OHP.

Additional signature tags and monitoring levels can be added depending on necessity.

In the following section, a brief introduction to monGroups will be provided which should be sufficient for most developers. For more detailed information see the HLT Monitoring Groups for Run 3 TWiki.

Implementation of monGroups in the Trigger menu

MonGroups can be added as ChainProp to individual chains in the respective menu where the chain is defined. The syntax is as follows:

monGroups=['<signature tag>:<monitoring level>']

Several monGroups, for both different signatures and monitoring levels, can be appended:

monGroups=['<signature tag 1>:<monitoring level A>', '<signature tag 1>:<monitoring level B>', '<signature tag 2>:<monitoring level A>']

Example:

ChainProp(name='HLT_j45_pf_ftf_preselj20_L1jJ40', l1SeedThresholds=['FSNOSEED'], stream=[PhysicsStream,'express'], groups=SingleJetGroup+SupportPhIGroup, monGroups=['idMon:shifter','jetMon:t0','jetMon:online']),

In this example the HLT chain HLT_j45_pf_ftf_preselj20_L1jJ40 is marked as being used for monitoring by the HLT ID software group, idMon, at online level, and the jet signature group, jetMon, at t0 and online level.

If a chain is used for monitoring at both shifter and t0 level, only the first shall be added as chain property.

Adding monGroups to the chain properties of an HLT chain does not automatically mean that monitoring histograms are created and available for online and offline monitoring. The actual monitoring logic still needs to be implemented in the respective signature code, see next section.

For more information about how HLT chains, see Individual Chain Construction in this guide.

Storage of monGroups

Information about monGroups for each HLT chains are stored as dictionary in a json file, HLTMonitoring.json, when the HLT menu is created. The HLTMonitoring.json can be inserted in the Trigger database when a new SMK is created, but can also be uploaded separately and linked to an existing SMK. Thus, multiple HLTMonitoring.json files can be attached to a single SMK. It is important to note only one of them will be used

Access to monGroups

Python access

This information can be accessed via two functions:

• monitoringDict(): returns full dictionary of for all monGroups
• monitoredChains(signatures="", monLevels="", wildcard=""): returns chains for given signatures, monitoring levels and matched to the wildcard

Example 1: will return full dictionary

from TrigConfigSvc.TriggerConfigAccess import getHLTMonitoringAccess
moniAccess=getHLTMonitoringAccess(ConfigFlags)

moniAccess.monitoringDict()

Example 2: will return only chains with the selected signature group and/or monitoring levels and/or matching to the wildcard

from TrigConfigSvc.TriggerConfigAccess import getHLTMonitoringAccess
moniAccess=getHLTMonitoringAccess(ConfigFlags)

moniAccess.monitoredChains(signature="idMon", monLevels="t0", wildcard="msonly")

Implementations like the above examples can be used in the respective signature monitoring code to create and fill histograms only for the selected subset of chains. Regarding information about how to create monitoring histograms, see Online Monitoring in this guide.

TriggerToolWeb access

The HLTMonitoring.json files stored in the Trigger database can be viewed and downloaded via TriggerToolWeb.
or more information on TriggerToolWeb, see TriggerToolWeb Documentation.