1 / 17

Integration of the ICAT API at Institut Laue-Langevin (ILL) (Grenoble, France)

ICAT developper Workshop (25/26 August 2009). Integration of the ICAT API at Institut Laue-Langevin (ILL) (Grenoble, France). Agenda. Overview (ILL - Project) Data(file) import (ILL data import into ICAT) User Interface (web-based user Frontend) Conclusions (Problems, Questions).

taniel
Download Presentation

Integration of the ICAT API at Institut Laue-Langevin (ILL) (Grenoble, France)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ICAT developper Workshop(25/26 August 2009) Integration of the ICAT APIat Institut Laue-Langevin (ILL)(Grenoble, France) Jean-François Perrin

  2. Agenda Overview (ILL - Project) Data(file) import (ILL data import into ICAT) User Interface (web-based user Frontend) Conclusions (Problems, Questions)

  3. IT Service ILL Besides two other IT services at ILL: SCI (Instrument Control) and CS (Computing for science) Responsibilities (groups): Infrastructure (network and phone) DB Usersupport Developpement Administration ICAT Team (Developpement group): Holger Gebhard Najor Cruz-Cruz (since July)

  4. ILL-ICAT-Project Current state at ILL (IDA application): text and script based system, user must know how to retrieve date (not a search) 3 steps to integrate ICAT at ILL replace existing (IDA) functionalities public, more comfortable access for a larger group of users Extend meta-data base and search functionalities Practical point of viewInstallation and exploration data-import GUI (Web App)

  5. Data Import 1 General: Since 1973 all ILL data are available online about 40 active instruments (60 overall) Filesystem organised by cycle – instrumentstored on NFS (sometimes CIFS) Total number of files: 29.145.595 (148 cycles) Average size: 229 kb max ~ 40 mb (compressed) ILL formats: Historical ILL Formats (ascii) Nexus (2 Instruments) edf (proprietary binary format) Impacts for ICAT integration at ILL relevant parameters are specific to (groups) of instrument(s).  Need for expert advice

  6. Data Import 2 Example: Import first cycle of 2006 for instrument d22 No relation to proposals IDA replacement ! One specific parser class per (group) of instrument(s) Analyse directory level Analyse file level …/061/d22/ 015673.Z … 015713.Z …. 020967.Z … 021354.Z …/061/d22/ 015673.Z … 021354.Z • Integrate found • Data files • Data sets • Investigations • into Database Read parameters and metadata Map files to dataset

  7. Data Import 3 DATASET DATAFILE DATAFILE_PARAMETER INVESTIGATION ICAT_AUTHORIZATION: (user_id = any, role=downloader) PARAMETER (ex. wavelength, pressure, voltage) Used (filled) tables: • DATAFILE_FORMAT • ILL Nexus, ILL ASCII • INSTRUMENT (# 110, 61 activ) • FACILITY_CYCLE (#142) • INVESTIGATION_TYPE • DATASET_STATUS (complete) • DATASET_TYPE(experiment_raw) • USER_ROLES (some testusers)

  8. ILL User Interface Available in a stand-alone application and/or integrated into intra-/internet As described in documentation: Compile WSDL to generate remote proxy and classes for business delegates Call remote proxy methods to interact with ICAT API

  9. Feedback Tests and developments done so far worked fine for us ICAT Installation: straightforward Database Scripts, ICAT API Test with a dummy prototype Certificate handling and user authentication during installation No performance critical tests

  10. ILL’s open issues ICAT integration: Mapping between data files and proposal data  no proper notion of "ICAT investigation" at ILL Specimen data (and others) are stored with proposal Control of data format and parameter variety General ILL Data Policy data ownership delay until public availability Unification of data file formats Competence and responsibility Complexity

  11. Some open issues Search by parameter in ICAT Future prospect: Opening ICAT’s Authentication mechanism to ILL’s SSO/LDAP Future prospect: Search between ICAT facilities repositories

  12. Thanks Comments? Questions?

  13. Technical envorinment Hardware and Database Virtual Server (Suse SLES 10.2) for ICAT Oracle 10g (Cluster RAC) as DB Development Environment: Java 1.6, Glassfish 1.4, Ant 1.7.1 ICAT version: 3.3.3

  14. Question details searchable parameters Parameter table: ‘Wavelength’ , ‘Å’, ‘Amstrong’ , Y, Y, N, Y, Y, ‘Desc Wavelenght’, Y, 999, GEBHARD, 31/07/09 02:20, 31/07/09 02:20, ‘NAJOR’, Y, N Also: Voltage and Pressure Join to tables: data_file_parameter and data_set_parameter Known searches: SearchByAdvanced SearchByKeywords SearchByRunNumber SearchByUserId SearchByUserSurName SearchDatasetsBySample Advanced Search Fields private java.lang.String datafileName; private java.util.Calendar dateRangeEnd; private java.util.Calendar dateRangeStart; private java.lang.String experimentNumber; private java.lang.Long grantId; private java.lang.String[] instruments; private java.lang.String investigationAbstract; private uk.icat3.client.InvestigationInclude investigationInclude; private java.lang.String investigationName; private java.lang.String investigationType; private java.lang.String[] investigators; private java.lang.String[] keywords; private java.lang.Double runEnd; private java.lang.Double runStart; private java.lang.String sampleName; private java.lang.String visitId;

  15. ILL-ICAT-Project 2 From the practical point of view ICAT installation and exploration Data Import (File metadata to ICAT, ~ 10 of 40 instruments) Webbased User Interface (Simple version, planned for end of the year) Extension of metadata and userinterfaceAutomated, continious data import(all intruments, all cycles)

  16. Data File example RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR 068762 0 4 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA 80 1 Inst User L.C. Date Time D19 tf sam 14-May-09 10:16:11 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA 80 1 Title Scantype backgr om-mono IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII 31 4 nvers ntype kctrl manip nbang nkmes npdone jcode ipara ianal imode itgv iregul ivolt naxe npstart ilast1 isa flgkif ih ik nbsqs nb_det nbdata icdesc1 icdesc2 icdesc3 icdesc4 icdesc5 icdesc6 4 2 4 8 1 21 21 1 1 0 0 0 1 0 1 0 0 0 0 0 0 0 1 163840 8 0 0 0 0 0 FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF 50 10 H (Hmin) K (Kmin) L (Lmin) phi chi omega 2theta (gamma) psi ub(1,1) ub(1,2) ub(1,3) ub(2,1) ub(2,2) ub(2,3) ub(3,1) ub(3,2) ub(3,3) wavelength dmonochromator danalyser energy Hmax Kmax Lmax DeltaH -0.10000000E+01 0.50000000E+00 0.50000000E+00 -0.49880001E+02 0.15721400E+03 0.23701000E+02 0.64004000E+02 0.00000000E+00 0.12238280E+00 0.53997743E-01 -0.64825200E-02 -0.53893240E-01 0.12252440E+00 0.37250000E-02 0.69433160E-02 -0.74291900E-03 0.14317410E+00 0.24220001E+01 0.10000000E+01 0.10000000E+01 0.00000000E+00 0.00000000E+00 0.00000000E+00 0.00000000E+00 0.00000000E+00

More Related