1 / 15

@ With TANGO S. Poirier – Data management group - PowerPoint PPT Presentation

  • Uploaded on

@ With TANGO S. Poirier – Data management group. Short introduction Devices servers GUIs. Introduction. SOLEIL requirements for data recording A system able to record any kind of data Generic (independent from beamlines and experiments types)

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' @ With TANGO S. Poirier – Data management group' - hadley-franco

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript



S. Poirier – Data management group


  • SOLEIL requirements for data recording

    • A system able to record any kind of data

    • Generic (independent from beamlines and experiments types)

    • That can easely configurated (even by users!)

  • The SOLEIL choice for data format: NeXus

    • Based on HDF (Hierarchical Data Format) and, recently, XML

  • Why ?

    • Logical organisation rather than physical organisation

    • Ability to store any kind of data

    • Self-describing format

    • Efficiency (binary, compression…)

    • Used in other facilities: DIAMOND, APS, LLB, ISIS, SNS, PSI, ESRF?


  • a NeXus file is a tree structure, like a filesystem.

    • Datasets are recorded inside typed data groups (NXentry, NXsample, NXdata, NXsource, …)

    • Experimental data is recorded in NXdata groups

    • Metadata is recordedIn the other groups

    • Big datasets may becompressed

In this sample, you show the data for 3 experiments

Devices servers
Devices servers

  • DataRecorder

    • Writes experimental data and associated metadata in NeXus files

  • AuthServer

    • Users enter login, password & project code. Then the ds generates a ‘key’ allowing DataRecorder to write data for a particular project in the right place in the storage facility.

  • NeXusReader

    • This device reads NeXus files and expose datasets, on demand, as dynamics attributes.

Devices servers1
Devices servers

  • TechnicalData

    • Device that store two lists of ds containing data about the technical environment:

      • The DataRecorder reads theses lists and records the corresponding data at the begining and at the end of a recording session (i.e for a NXentry group)

  • SampleData

    • sample data collector device. The aim of this ds is to provide a unified interface to sample information.

Datarecorder how it works
DataRecorder : how it works ?

  • It’s based a on shared library: libDataStorage (written in C++).

    • This library don’t know Tango

    • But it knows how to record data in the NeXus format

  • Experimental data is collected through a plug-ins mechanism: data collectors

    • 2 collectors at this moment: &

  • A tool can be used to export data from NeXus file into ASCII or binary based format (e.g. EDF) for users convenience

Data production diagram
Data production diagram

  • From the DataRecorder point of view

Data collectors


  • Bolero

    • Edition of the devices servers recording configuration


  • TechnicalData DS lists edition:


  • Data Storage Control Center

    • Manage and monitor storage devices


  • Baladi

    • A graphical tool to explore Nexus files

The twist project
The TWIST project

  • Web portal for experimental data retreiving

  • Written in java and use JNLP technology

  • This not a pure TANGO project but, in order to allow browsing inside NeXus files, client app uses ATK components and exchange data with a NeXusReader device

Other software tools
Other software tools

  • NXextractor: a tool able to export data from a NeXus file into a file in a arbitrary format (ASCII or binary). It uses scripts to describe theses formats.

  • IGOR extension (XOP) for reading NeXus files

  • Next steps: adapt post-process software. To be planned.