Algorithm data flow interface
1 / 11

Algorithm / Data-flow Interface - PowerPoint PPT Presentation

  • Uploaded on

John Baines. Algorithm / Data-flow Interface. Contents: PESA Software Requirements: Algorithms Interface Layer Interfaces Data Descriptor Data Preparation Implementation issues Summary.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Algorithm / Data-flow Interface' - katalin-molnar

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Algorithm data flow interface

John Baines

Algorithm / Data-flow Interface


PESA Software Requirements:


Interface Layer


Data Descriptor

Data Preparation

Implementation issues


Pesa software requirements
PESA software Requirements

  • PESA software Requirements (currently being reviewed):

    Section 4.8 lists requirements for “Interface to Data Collection”

  • Assumptions made in that document:

    • Algorithms running in LVL2 processors will request data by specifying an abstract “Region” and the required object-type for the returned data.

    • An interface layer is required between PESA algorithms and data collection, fulfilling requirements and constraints from both sides.

    • The ROS may have the capability to perform some data preparation tasks.

    • The current requirements specify a higher degree of flexibility than may be required in the final system because at this stage of the project we need to evaluate various scenarios for ROS-DC-HLT e.g. to evaluate data preparation in the LVL2 processor compared to the same data preparation performed in the ROS.

  • Assumed constraints:

    • Data collection never looks inside the ROS data.

    • Granularity used by the readout system may be coarser than that algorithms would ideally use to select their data, i.e. additional data to that requested may be returned.

    • Data from RODs may vary between predefined formats from event to evente.g. different levels of compression. Full readout / zero suppressed readout etc.

Algorithm requirements
Algorithm Requirements

  • Want flexible way to specify requested data in order to :

    • Avoid inefficiency due to part of trajectory outside simple RoI

      (e.g. due to track curvature, vertex spread etc.)

    • Specify data in the most natural way for a given detector.

    • Minimise the amount of “extra” data in order to :

      • Reduce data-flow

      • Simplify pattern recognition task

  • Want data returned as a collection of objects of specified type i.e. unpacked with headers removed and converted to appropriate form => data preparation.

  • Ideally want same interface for all algorithms:

    => Can evaluate same algorithm in different parts of the system in development phase

    Might also be beneficial to have this flexibility in the final system.

  • Types of algorithm:

    • Data pre-selection : e.g. select subset of hits from detectors inside specified region

    • Data preparation : e.g. : SCT & pixel cluster-formation, SCT stereo-association,

    • Conversion of hit information to position in space, LAr calibration, etc.

    • Feature Extraction : e.g. Calorimeter cluster finding, muon track finding etc.

used to be called “pre-processing”

Interface layer requirements
Interface Layer Requirements

The interface layer should:

  • Provide a uniform style of data access regardless of detector or level of data preparation requested.

  • Provide data unpacked into objects with a different Class for each detector. Also different classes for different formats (e.g. SCT hits, SCT clusters). Requested Class determines the detector and level of data-preparation.

  • If data preparation / pre-selection is required this is initiated directly by the interface layer in the LVL2 processor or the request is passed on to be initiated at the ROS, this may require some parameters passed to the data pre-selection algorithm at the ROS.

  • The data descriptor must provide for a range of different pre-defined ways to specify the geometrical region for the requested data.

  • The interface layer must convert the geometrical region to a list of RoBs to be requested from data collection.

  • It should be possible to specify a level of granularity below RoB e.g. list of modules. This list would form the parameters used as input to a data pre-selection algorithm running either in the LVL2 processor or at the ROS.


FEX Algorithm

Data Preparation or Pre-Selection Algorithm

Data Preparation or Pre-Selection Algorithm

DataDescriptorType region;

DataType data;

ErrorCode rc = dataService->get (region, data)

Interface Layer

raw data

raw data

prepared data

prepared data


Data Collection



rc = FexAlgorithm -> execute(seed)

data as collection of objects of specified type

region, data type


Geometry info

ROB map

Event ID, List of ROB IDs

Optional data preparation params.

raw or prepared data

Raw data

Data descriptor
Data Descriptor


h - Dh

h + Dh



  • Trigger algorithms use seeds to derive the region of the detector from which data is required. Seeds can be, e.g.:

    • LVL1 RoI

    • LVL2 cluster

    • LVL2 track

  • Can use LVL1 RoI to specify a cone (h, Dh, f, Df), but this is not sufficient/appropriate in all cases, e.g.:

    • It is crucial to take vertex spread into account for the pixels and SCT

      =>h, Dh, f, Df, Dz(or hmin, hmax, fmin, fmax, zmin, zmax)

  • h, f not natural for all systems, e.g. natural to use x-y range for (FCAL)

  • Ability to additionally specify layers can reduce data-flow e.g. by afactor 2using layer-by-layer sequential selection in calorimeter (ATL-DAQ-2000-042)

Data descriptor cont
Data Descriptor (cont.)

  • For LVL2 seeds knowledge of pT and charge sign could significantly reduce data-volume to be read out, especially at low pT,

    e.g. confirmation of muon in TileCal and ID:data volume reduced w.r.t. simple RoI by allowing data to be specified in a road about the trajectory measured in the muon system.

Data descriptor implimentation
Data Descriptor Implimentation

Abstract base class :

class Region {


virtual bool

select (const ROBDescriptor& desc) = 0;


Implementation of specific region description:

class EtaPhiRegion : public Region {

virtual bool

select(const ROBDescriptor& desc){

// return true if desc in eta/phi region



ROB Descriptor:

class ROBDescriptor {

// Properties e.g. eta, phi, layer etc.


  • It is the task of the Interface Layer to determine which ROBs contain detectors inside the region described by the data descriptor.

  • Suggestion from Reiner Hauser for an implementation of the data descriptor:

    • Region is abstract base class with several derived classes, one for each distinct way of specifying the data.

    • Region has a select method which returns true of false according to whether or not a given ROB contains detectors inside the region.

    • To change the way a region is specified, e.g. if a more complicated selection is required, then a new class is defined derived from the Region base class.

    • The RoB Descriptor contains whatever information is necessary to determine whether the RoB is in the Region, e.g. h and f range covered by the RoB, layers, z range etc.

  • Region could also have a method used by a data preparation algorithm to say whether a given detector is inside the region.

Data preparation
Data Preparation

  • Data pre-selection might be performed at the ROS in order to reduce data flow. e.g.

    • Zero suppression of TRT data could reduce the data volume by a factor ~5 at low luminosity (ATLAS DAQ Note 66) i.e. 128 kByte 28 kByte assuming 3% occupancy.

    • Selecting the subset of SCT and pixel modules which lie wholly or partially in the RoI gives factor ~10 data reduction i.e. ~25 kByte / RoI ~2.5 kByte / RoI at L=1034 cm-2s-1.

      • For an optimised ROB mapping there are an average of ~1.5 Pixel ROBs and ~2 SCT ROBs per (small) RoI (Dh x Df = 0.1 x 0.1) i.e. an average of 72 Pixel detectors and 192 SCT detectors read out per RoI, of which only a small subset of modules lie inside the RoI.

  • Data preparation at Interface Layer in order to simplify the task for the algorithm and provide a uniform style of data access and hide details of the raw data format:

    • Conversion to format convenient for FEX algorithm, e.g.

      • Unpacking & creation of objects

      • SCT and Pixel clustering

      • SCT stereo association

      • LAr Calibration

      • Space-point formation : conversion from address to position in space.

    • Possibly further data pre-selection e.g. select sub-set of space-points lying within RoI

Implementation issues
Implementation Issues

  • In the case of re-arrangement of data (e.g. unpacking) copying of data could be avoided if the algorithm receives a pointer to a collection of objects and can iterate over these. The details of the raw data are hidden e.g. whether the hits are in separate ROS fragments.

  • The conversion from Raw data to offline-type classes could use the code being developed for the Event Builder, provided LVL2 and EB uses the same Classes.

  • Could data pre-selection and simple data preparation (e.g. SCT clustering) be performed on demand to avoid copying? e.g.:

    • pre-selection: skip over hits outside Region

    • SCT clustering: find centre of sequence of adjacent hits


The different requirements of Data Collection and HLT Algorithms could be met as


  • Algorithmsspecify their data request in the form of :

    • A Data Descriptor (Region)

    • The Class for the returned data.

  • An Interface Layer is provided in order to:

    • Hide details of the Raw Data and readout architecture (ROS, ROB) from the algorithm

    • Hide details of geometrical regions and offline-type Classes from Data Collection.

  • TheInterface Layerconverts the data request to a list of ROBs

  • Datapreparationand/orpre-selectionare performed at the interface layer as required, determined by Region and Class i.e.the interface layer converts the Raw data to a collection of objects of the requested Class Type lying within the requested region.

  • Additionally pre-selection or data preparation may be requested at the ROS. In this case, the interface layer provides the necessary parameters to be passed to the pre-selection or data preparation algorithm at the ROS.