Tools and tips

Last update: 24 Feb 2021 [History] [Edit]

Browsing and comparing configurations

The configuration stored in ComponentAccumulator can be saved in the file like this:

acc = ComponentAccumulator()
...
# at the end of config file
with open("config.pkl", "wb") as file:
    acc.store(f)

The format of the file is binary however it can be browsed with the confTool.py and iconfTool tools:

> confTool.py -p config.pkl
# or for a more interactive and visual browsing
> iconfTool -i config.pkl

The confTool.py has multiple options to constrain or expand the amount of details in the output (see -h option). For action available in the iconfTool hit the keyword h.

Both tools can be used for comparing the configuration.

> confTool.py --diff old-config.pkl config.pkl

The output will indicate missing as well as different settings.

For comparisons with old configuration the athena option --config-only="old-config.pkl" can be added to Athena command line. With this the job does not actually run but the configuration is saved in the file that can be browsed or compared as above.

Discovering properties of the component

The component properties are also inherited from base classes. A quickest way to discover all of them is by instantiating the python configuration class and list the class description as in the following example:

# in the terminal with atlas s/w set-up
> python
>>> from AthenaConfiguration.ComponentFactory import CompFactory
>>> algClass = CompFactory.HelloAlg # an example algorithm in this case
>>> help(algClass)

This will result in the output like below with the details of all properties enlisted:

Help on class HelloAlg in module GaudiConfig2.Configurables:

class HelloAlg(GaudiConfig2._configurables.Configurable)
 |  HelloAlg(name=None, **kwargs)
 |
 |  Properties
 |  ----------
 |  - ExtraInputs: std::unordered_set<DataObjID,DataObjID_Hasher,std::equal_to<DataObjID>,std::allocator<DataObjID> > ([])
 |     [DataHandleHolderBase<PropertyHolder<CommonMessaging<implements<IAlgorithm,IDataHandleHolder,IProperty,IStateful> > > >]
 |
 |  - ExtraOutputs: std::unordered_set<DataObjID,DataObjID_Hasher,std::equal_to<DataObjID>,std::allocator<DataObjID> > ([])
 |     [DataHandleHolderBase<PropertyHolder<CommonMessaging<implements<IAlgorithm,IDataHandleHolder,IProperty,IStateful> > > >]
 |
 |  - OutputLevel: int (0)
 |    output level [Gaudi::Algorithm]
 |
 |  - Enable: bool (True)
 |    should the algorithm be executed or not [Gaudi::Algorithm]
 |
 |  - ErrorMax: unsigned int (1)
 |    [[deprecated]] max number of errors [Gaudi::Algorithm]
 .....

In the configuration objects (not classes!) can be queried for:

  • properties that are already set with comp._properties that returns dictionary with property name and value,
  • all properties with comp._descriptors returning dictionary with property name and descriptor object holding info about C++ type, default value and semantics,
  • complete component class name/instance name comp.getFullJobOptName() (e.g. MyNameSpace::MyTool/ToolA),
  • short name (without the class name) comp.name
  • component type comp.getGaudiType() that returns one of strings “Algorithm”, “Tool” or “Service”.

Debugging the configuration process

Sometimes it is necessary to interrupt the configuration process and be able to enter interactive mode somewhere deep in the calls stack. The code python module can be used for this.

In place where you intend the configuration process to be interrupted add the following:

import code; code.interact(banner="me debugging >>> ", local=locals())

With this the configuration process will be interrupted and a control given to you. You can inspect local scope variables as needed. With Control-D that session will be ended and configuration process resumed.

Another option could be to use the python debugger pdb. Where you want to abort the configuration, add the following:

import pdb; pdb.set_trace()

Finding more about the conflicts

When using the CA the merge conflicts can arise. Since it is sometimes difficult to discover how the conflicting component is configured in such situation. Therefore the collection of context information can be enabled by setting: ComponentAccumulator.debugMode=True in the main script. With this, the merging conflict exception is extended by an information as to where the conflicting components are defined. That may look like this:

Traceback (most recent call last):
 File "/srv/build/x86_64-centos7-gcc8-opt/python/AthenaConfiguration/ComponentAccumulatorTest.py", line 98, in test_conflict_in_cond_alg
    acc.addCondAlgo(ExampleAlg1("Cond1", MyInt=8))
.....
ValueError: conflicting settings for property MyInt of ExampleAlg1: cannot merge values 8 and 7
with the context
 Adding Conditions Algorithm to the ComponentAccumulator that has instance of it already added in this context
 >>> runpy.py:193(_run_module_as_main) case.py:624(run) ComponentAccumulatorTest.py:57(setUp)

The last line contains an abbreviated sequence of calls that lead to the creation of the component conflicting with the one attempted to be added.

If the context collection is not enabled the hint message like below is displayed:

...
ValueError: conflicting settings for property MyInt of Cond1: cannot merge values 8 and 7
with the context
 Unknown (enable it with ComponentAccumulator.debugMode = True)

Beware that the context collection is not enabled by default because it slows down the configuration process measurably.