JobOption Filters

Last update: 15 Nov 2022 [History] [Edit]

Introduction to filters

Typically when producing MC with different final states, a separate JOs is used for each final state. This is used to ensure sufficient statistics in each final state, especially when they have small branching ratios. Furthermore, it avoids generating events that are not useful in your analysis.

In most cases when using MadGraph, a particular final state can be defined simply through the generate expression or the options for the parton shower tool if the decay is performed there. Other generators may not provide this option.

In some instances, however, it is not possible to select a particular final state in this way. In these cases, generator filters can be used to keep only events that meet certain criteria.

The generator filter logic is applied after event generation and decays are done. Any events that fail the filters are discarded and are not passed through the computationally expensive detector simulation and reconstruction steps in MC production. This means that events that fail the filter requirements count towards nevents but not towards maxEvents.

Often a combination of generator commands and filters can be used together to optimize MC sample production. Try to avoid relying too heavily on filters with very low efficiencies. This can be unnecessarily expensive in CPU hours when a less computationally expensive solution exists.

In this tutorial, you will be using a suboptimal combination of generator commands and filters to be able to easily see how to use filters. In an analysis scenario, the dielectron and dimuon final states would be created as separate samples using generator commands and no filter would be used.

A variety of generator filters are available for use. Most filters have names that intuitively describe their purpose. You can also use the header files to read their purpose and the corresponding source files to see the configurable properties and how the logic is implemented.

The existing filters provide significant coverage of the possible use-cases that you might encounter. It may take some clever combination of filters to get your desired outcome. If there is no way to use the existing filters to produce your final state with a reasonably high efficiency, you can create a new filter following the examples of the existing filters.

Making inclusive JOs

If you have not already done so, change the generate expression in your JOs to

generate g g > lepton lepton q q

This will give you inclusive leptoquark decays to electrons and muons.

Implementing filters

In order to select events that have either two electrons or two muons we will use MultiElectronFilter and MultiMuonFilter since these allow a selection on electrons and muons instead of the total number of charged leptons.

There can be leptons in the generated events that come from sources other than the leptoquark decays that we do not want to count for the filters. These typically have low transverse momenta, so we will require the filters to use leptons with transverse momenta of at least 10 GeV. This also serves the purpose of cutting away any signal events that have such low object pTs that they would not be accepted by the analysis selection.

To implement the filters, add the following lines to the bottom of the JOs:

# Apply filters to get the decays we want
# We want either two electrons or two muons with pt > 100 GeV

# Import the filters and add them to the filter sequence
from GeneratorFilters.GeneratorFiltersConf import xAODMultiElectronFilter
filtSeq += xAODMultiElectronFilter("eeFilter")

from GeneratorFilters.GeneratorFiltersConf import xAODMultiMuonFilter
filtSeq += xAODMultiMuonFilter("mumuFilter")

# This is necessary for xAOD filters
include ("GeneratorFilters/CreatexAODSlimContainers.py")
createxAODSlimmedContainer("TruthElectrons",prefiltSeq)
createxAODSlimmedContainer("TruthMuons",prefiltSeq)
prefiltSeq.xAODCnv.AODContainerName = 'GEN_EVENT'

# Set the number of leptons
filtSeq.eeFilter.nElectrons = 2 # Note that this will change to NElectrons in 22.6.23
filtSeq.mumuFilter.NMuons = 2

# Set a pt cut of 100 GeV
# Note that the units are in MeV
filtSeq.eeFilter.ptcut = 100000. # This will change to Ptcut in 22.6.23
filtSeq.mumuFilter.Ptcut = 100000.

# Set the filter message levels to DEBUG so we can see what is happening
filtSeq.eeFilter.OutputLevel = DEBUG
filtSeq.mumuFilter.OutputLevel = DEBUG

# Implement the logic for passing the set of filters
filtSeq.Expression = "eeFilter or mumuFilter"

The filtSeq.Expression line provides the logic for using multiple filters.

The messaging level of the filters is set to DEBUG to provide information in log.generate to help show how the decisions are made.

Run the event generation again using the same Gen_tf.py command.

If implemented correctly, the event generation should fail with the error:

Pythia8 ERROR Exceeded the max number of consecutive event failures.

Debugging filter errors

The failure you have encountered is one of the most common issues when implementing a generator filter. Look through log.generate to find the cause of this crash.

With a bit of searching, you should find a line that says:

PYTHIA Abort from Pythia::next: reached end of Les Houches Events File

As events are processed, messages such as the following are printed to indicate the progress:

AthenaEventLoopMgr INFO ===»> start processing event #25, run #100000 24 events processed so far «<=== … AthenaEventLoopMgr INFO ===»> done processing event #25, run #100000 25 events processed so far «<===

The final line before the error sheds light onto the issue:

AthenaEventLoopMgr INFO ===»> start processing event #111, run #100000 110 events processed so far «<===

MadGraph has generated 110 Les Houches events (from nevents and the safety factor of 1.1), but clearly this is not sufficient. To fix this, we need to increase the safety factor to account for the filter efficiency. An arbitrarily large safety factor is computationally wasteful since the Les Houches events are generated before the filters are applied. Use information in log.generate to find the optimal safety factor.

Search for a message from Py:EvgenFilterSeq that includes:

INFO Filter Efficiency

From this, you should see a filter efficiency of approximately 50%. From this, it can be inferred that the safety factor should be around 2.0. Due to possible fluctuations in the event generation, it is wise to increase the safety factor somewhat. A value of 2.2 should be sufficient for this sample.

Update the safety factor and run the same Gen_tf.py command again as well as the derivation and validation steps to make sure that the filters are correctly implemented.