Identifiable Containers

• Introduction
• Implementation details
• Filling methods
  • 1. Fill via IDC_WriteHandle Lock
  • 2. Fill via IDC addOrDelete
• Retrieving items from container
  • Iterate over all pointers
  • Iterate over all hash and pointers
  • Random access
  • Random access then iterator
• Creating a Cache

Introduction

The identifiable container is a template container class for collections of detector data mapped to a hash (an unsigned integer typically starting from 0). This has been redesigned to meet the following requirements:

• To be backwards compatible with the large code base that already uses this class.
• To allow concurrent filling of the container in different views allowing the mitigation/elimination of work duplication.
• To be used transparently in both online and offline reconstruction as in accordance with the design principles of “views”.
• To allow fast wait-free retrieval of the collections.
• To recognise when a hash has been “aborted” i.e. contains no data across every view after it started to be constructed.

Implementation details

The container is split into two classes the cache and the container. The cache contains the actual collections and is designed to be concurrently accessible across threads. The container provides access to the cache and includes a mask to identify the collections visible from the current view.

The container has been updated to have multiple modes, online and offline mode. When in offline mode the container owns its collections and will delete them on destruction. The container can be initialised in different modes to efficiently function in a given use-case.

The mode is determined on creation of the container, constructing it with the maximum hash will place it in offlineLowMemory mode, while presenting an external cache will construct it in online mode. You can also pass the enumerator EventContainers::Mode::OfflineFast to enable the container in offline mode but providing faster random access, this mode requires more memory however. An example:

// Split the methods to have one where we use the cache and one where we just setup the container
const bool externalCacheRDO = !m_rdoContainerCacheKey.key().empty();
if(!externalCacheRDO){
  ATH_CHECK( rdoContainerHandle.record(std::make_unique<CscRawDataContainer>( m_muonMgr->cscIdHelper()->module_hash_max(), EventContainers::Mode::OfflineFast)));
  ATH_MSG_DEBUG( "Created CSCRawDataContainer" );
}
else{
  SG::UpdateHandle<CscRawDataCollection_Cache> update(m_rdoContainerCacheKey, ctx);
  ATH_CHECK(update.isValid());
  ATH_CHECK(rdoContainerHandle.record (std::make_unique<CscRawDataContainer>( update.ptr() )));
  ATH_MSG_DEBUG("Created container using cache for " << m_rdoContainerCacheKey.key());
}

Filling methods

There are two ways one can fill an identifiable container from multiple threads:

1. Fill via IDC_WriteHandle Lock

The recommended way of filling the container is to do the following:

PixelClusterContainer::IDC_WriteHandle lock = clusterContainer->getWriteHandle(listOfPixIds[i]);
if( lock.alreadyPresent() ) continue;

auto clusterCollection = DoWork();
if (clusterCollection && !clusterCollection->empty()){
  ATH_CHECK(lock.addOrDelete( std::move(clusterCollection) ));
}else{
  ATH_MSG_DEBUG("No PixelClusterCollection to write");
}

In this example a IDC_WriteHandle is obtained and from this you can determine if the item is already present, in which case you should skip making it - the mask in the container is automatically adjusted to make it visible in this view. If it is not present the collection should be constructed and added to the container via the lock. If the collection is not added by the time the lock goes out of scope the collection is globally set as ABORTED, this means no other view can construct it for the current event. If this behaviour is not desired you should use method 2 instead.

If a thread tries to access an item that is still under construction, the thread will wait until the item is finished or the lock object falls out of scope.

2. Fill via IDC addOrDelete

if( clusterContainer->tryAddFromCache(hash) ) continue;
auto clusterCollection = DoWork();
if (clusterCollection && !clusterCollection->empty()){
  ATH_CHECK(clusterContainer->addOrDelete( std::move(clusterCollection), hash ));
}else{
  ATH_MSG_DEBUG("No PixelClusterCollection to write");
}

When tryAddFromCache is called, if the collection already exists in another view the mask is adjusted to make the collection visible in this view and returns true allowing you to skip it.

Since this method doesn’t lock on the hash, there is no guarantee work is not duplicated. In the event two threads try to commit the same item the first one is accepted and all subsequent items are deleted within addOrDelete.

Retrieving items from container

Although all other methods present in the class are still compatible, the best way to retrieve an item depends on the mode the container is initialised in. But the following guidelines should be used for optimal speed in most situations.

Iterate over all pointers

for(auto ptr : container){
  DoSomething(ptr);
}

Iterate over all hash and pointers

for(const auto &[hashId, ptr] : container->GetAllHashPtrPair()){
  DoSomething(hashId, ptr);
}

Random access

If the container is created in OfflineLowMemory mode then this will give slow performance

auto collection = container->indexFindPtr(hash);
DoSomething(hashId);

If the item is not present (or not visible in your view) it will return a nullptr. If the item is still being produced the thread will wait until it is ready and the item retrieved.

Random access then iterator

This method should only be used if you intend to iterate further as it offers poor performance for random access

for(auto iterator = container->indexFind(hash); iterator != container->end(); ++iterator){
   DoSomething(*iterator);
}

Creating a Cache

Cache creation must occur in an algorithm run outside of the views and before the views are executed. The algorithm need to simply create a the cache object and give it to storegate by a non-const method (caches are then accessed within views by update handles). This algorithm must then be scheduled to run outside the view appropriately.

    template<typename T>
    StatusCode CacheCreator::createContainer(const SG::WriteHandleKey<T>& containerKey, long unsigned int size, const EventContext& ctx) const{
        if(containerKey.key().empty()){
            ATH_MSG_DEBUG( "Creation of container "<< containerKey.key() << " is disabled (no name specified)");
            return StatusCode::SUCCESS;
        }
        SG::WriteHandle<T> ContainerCacheKey(containerKey, ctx);
        ATH_CHECK( ContainerCacheKey.recordNonConst ( std::make_unique<T>(IdentifierHash(size), nullptr) ));
        ATH_MSG_DEBUG( "Container "<< containerKey.key() << " created to hold " << size );
        return StatusCode::SUCCESS;
    }