The production Athena jobs are steering using so called Job Transforms.
Examples of those are
Sim_tf.py and more. They can usually
be recognised by ending with
Each transform can consist on multiple substeps, also depending on the configuration.
Some arguments are “substep-aware”, which means that one can set the name of the step as a prefix,
--CA 'Overlay:True' will only run
ComponentAccumulator-based configuration for overlay.
There are a few helpers available:
all: applies a specific argument to all substeps (
default: applies a specific argument unless overridden explicitly by another argument using a substep name (
first: only apply this argument to the first substep
There are many ways on how job transforms can be steered. A
can be used to get a list of all available properties (e.g.
In production most of those properties are steered by AMI tags.
Any AMI tag can also be evaluated from the command line e.g. using
Reco_tf.py --AMIConfig q442.
Note that usually inputs and outputs need to be specified explicitly, but in case of this
q-tag they are also present in the definition.
While higher-level job configuration can be steered using different arguments there is also a possibility to affect it lower-level. The following arguments are used for that
preExec: Can call arbitrary python on
ConfigFlags(only this is exposed).
preInclude: Calls a function that accepts ConfigFlags as the only argument. Passed as an “import string” of the form
<package>.<function>if aliased in
postExec: Can call arbitrary python but only
ConfigFlagsand the top-level CA called
postInclude: Calls a function that accepts
ConfigFlagsand the main CA instance. If a function with only one mandatory argument (the flags) is passed, it is assumed to be a configuration fragment and is merged with the top-level CA.
Any of the described arguments can be applied to all trans
The order of execution can be transform dependent but it is usually like: