1 / 14

HPS Web 2.0 OR Web Apps and Databases (Oh My!)

HPS Web 2.0 OR Web Apps and Databases (Oh My!). Jeremy McCormick (SLAC). OVERVIEW. Database Server Conditions Database Run Database Web Server Run Web Data Catalog Online DQM. WHY?. Databases Central facility for accessing shared information Functional replacement for text files

beaudry
Download Presentation

HPS Web 2.0 OR Web Apps and Databases (Oh My!)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. HPS Web 2.0ORWeb Apps and Databases(Oh My!) Jeremy McCormick (SLAC)

  2. OVERVIEW • Database Server • Conditions Database • Run Database • Web Server • Run Web • Data Catalog • Online DQM

  3. WHY? • Databases • Central facility for accessing shared information • Functional replacement for text files • Provides backend for other applications like recon and web apps • Change management and authentication (user accounts) • Web applications • View from “anywhere” without needing any special applications (such as ROOT) • Easily share important information such as DQM plots • See what’s inside data sources (databases, files, etc.) • Because Javascript is magic!

  4. DATABASE SERVER • Dedicated HPS MYSQL server • hpsdb.jlab.org • Hosts conditions, run and datacat databases • Full access from inside the JLAB firewall from ‘jlab’ hosts • Grants for write operations depend on the account • Several users have admin privileges (myself and Maurik) • Read only access via an external replica • External non-jlab hosts cannot see the primary db directly. • Non-production copy at SLAC for Hudson builds • Accessible in hps-java through APIs built on JDBC

  5. CONDITIONS DATABASE • Contains detector conditions data from Test Run to present. • Each type of condition has its own table with sets of rows assigned to a collection. • Collections are assigned valid run number ranges. • Sets of collections can be tagged for reproducibility. • Data is accessible in reconstruction via an extensive Java API. • The ‘hpsuser’ account has a public password and is used for read-only mode. • The ‘hpscalibrations’ account can be used to insert, update or delete existing data.

  6. RUN DATABASE • Database of run metadata extracted from crawling the EVIO data • Run start and end times • Number of events • etc. • EPICS data • Scaler data • TI Trigger Time Offset • Java API through RunManager • Can easily add additional information to DB and API!

  7. RUN LOG • Run database currently != run log • Run database is currently populated from crawling the data after it is copied to MSS and extracting metadata from EVIO. • Run log would be updated via hooks in the DAQ so that it happens concurrently with END and PRESTART events. • Implementing requires some coordination with Hall B staff. • Need to add C code into CODA • Determine which values to include (have gotten some good feedback here) • Some values also could be filled interactively by pop-ups, GUI interfaces, etc.

  8. WEB SERVER • http://hpsweb.jlab.org • RHEL 7, dual core, 4 GB VM • Tomcat 7 • Standard Java Servlet container • Admin manager only accessible behind JLAB firewall • Apache Proxy • Specific application URLs mapped to Tomcat • Managed by IT Division / CNI Group • Terminal access only allowed with crytocard

  9. RUN WEB • https://hpsweb.jlab.org/hps-run-webapp/ • Shows information from run database on the web. • Sortable run table • Run page • Summary • EPICS data • Scaler data • EVIO datacat link • DQM datacat link

  10. DATACAT • https://hpsweb.jlab.org/datacat/display/browser/HPS • New data catalog server and REST API from SLAC • Query and update via HTTP requests • Built-in browser interface • MySQL backend (runs at JLAB) • Java client API • Wrapper for URL based requests • Add, update, delete, search • Crawler for finding data, extracting meta data and adding to the catalog

  11. EXAMPLE DATACAT PATHS • Display Browser • /display/browser/HPS • Show a path • /r/path/HPS • Show a dataset • /r/path.json/HPS/test/dataset01;s=SLAC • Filter • /r/search.json/HPS/dataset01?filter=runMin+>+1000 • Recursive Search • /r/search.json/HPS/**?filter=runMin+eq+5772

  12. DQM ONLINE • Example DQM Plots (not production yet!) • AIDA TLD library of JSP tags (SLAC) • Allows easily showing plot files in a web page • Can read from AIDA or ROOT files on the file system • At JLAB /work can be mounted by server read-only. • Export individual plots to graphics files (PNG, etc.)

  13. TODO • Finish up JLAB web config • Enable authentication via scicomp comp certificate • Get DQM server working • Run database • Consolidate multiple sources of run data into database • Code for extraction of run (meta)data should be integrated into the crawler module • Data catalog • Command line interface • Integration with reconstruction and Auger jobs

  14. NOW FOR SOMETHING DIFFERENT! • What’s new in SLIC these days? • Current versions of Geant4 and other dependencies • Support for 3D magnetic field maps • Displaced vertices in StdHep files now supported (and maybe even tested) • Conversion from StdHep event records into LCIO MCParticle collection uses a standard/shared LCIO class • Code for translation of Geant4 tracks into MCParticle output collection rewritten

More Related