1 / 16

New logging and notification system for the Offline DropBox

New logging and notification system for the Offline DropBox. AlCaDB/ PdmV meeting. Thursday, 26 April 2012 . Martynas Mitrulevi č ius. Vilnius University, Lithuania. Outline. What is Offline DropBox Motivation Internship goals Authentication SQLite upload Meta data upload

jaclyn
Download Presentation

New logging and notification system for the Offline DropBox

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. New logging and notification system for the Offline DropBox AlCaDB/PdmVmeeting Thursday, 26 April 2012 MartynasMitrulevičius Vilnius University, Lithuania

  2. Outline • What is Offline DropBox • Motivation • Internship goals • Authentication • SQLite upload • Meta data upload • Meta data structure • Upload workflow • Upload script • Offline DropBox DB schema • Online DropBox DB schema • Workflowin Online DropBox • References Martynas Mitrulevičius

  3. What is Offline DropBox The Offline DropBox service: • is implemented using web application and supports the automaticexport of calibration constants. • simplifies tasksfor end-users in case they need to upload calibration resulting from an Offline analysis intothe condition databases. • is used to facilitate the automation ofalignment and calibration workflows. Martynas Mitrulevičius

  4. Motivation • The current application is working reliably and smoothly, allowing for calibration constants to be exported into production DB, but some consolidation from the development and the operation team is needed, and some features were also requested by end-users • In the Offline part of the DropBox the notification system for the exportation of calibration constants is not implemented. • In the online process, only the logging coming from the CMS Framework libraries (known as PopCon monitoring) is implemented. • Tails of the log file are exported for checking by experts • http://cms-conddb.cern.ch/popcon Martynas Mitrulevičius

  5. Internship goals • Getacquainted to technologiesand frameworks used at CERN andinthe CMS experiment. • Create PdmV database. • Create Offline DropBox server with new logging and notification system. • Create a Web pageallowing for calibration dataset upload. • Create CLI for the automatic exportation of calibration constants. Martynas Mitrulevičius

  6. Authentication • Authentication using CERN username and password • Only for members of cms-cond-dropbox • CERN Authentication SOAP WebServices for user authentication Martynas Mitrulevičius

  7. SQLite upload • Necessary tables in SQLite file: • ORA_DB • ORA_NAMING_SERVICE • ORA_C_COND_IOVSEQUENCE • ORA_C_COND_IOVSEQU_A0 Martynas Mitrulevičius

  8. Meta data upload Martynas Mitrulevičius

  9. Meta data structure destDBconnectionstring tagtagname inputtaginputtagname sincebegin TimetypetimeSpec IOVCheckchecker DuplicateTagHLThltTag DuplicateTagEXPRESSexpressTagDuplicateTagPROMPTpromptTag DuplicateTagPCLpclTag usertextuserlog checksum sqlitechecksum username user Martynas Mitrulevičius

  10. Upload workflow Server tasks: Authenticates user. Gets SQLite file. Checks if SQLite is not corrupted and has required content. Prepares pre-filled form. Gets and validates metadata from the user. Creates Tarball from SQLite and meta data. Movesthe file into Offline DropBox repository and adds new record to DB. Martynas Mitrulevičius

  11. Upload script • Command: • python dropbox_upl.py -u username -s sqlitefile.db -t metadata.txt • How it works: • Server authenticatestheuser. • Script uploads SQLite file. • Server checks if the file is not corrupted and has required content. • Server returns the form. • Script reads meta data file, fills the form and upload it. • Moves file into Offline DropBox repository andacknowledges the file transfer to the user and to thenotification system. Martynas Mitrulevičius

  12. Offline DropBox DB schema Martynas Mitrulevičius

  13. Online DropBox DB schema Martynas Mitrulevičius

  14. Workflow in Online DropBox • Downloads Tarball file and extract it. • Checks the SQLite file checksum. • Moves to the DropBox directory. • Process the file. • Exports calibration constants with synchronization if needed. • Duplicates payloads if requested. All these steps are logged in the logging schema. Martynas Mitrulevičius

  15. References • Upload file: • http://m-dropbox.cern.ch/uploadSQLiteFile • Get tarball list: • http://m-dropbox.cern.ch/getTarList • Get tarball file: • http://m-dropbox.cern.ch/getTarFile?file=FILENAME • Project files: • https://svnweb.cern.ch/cern/wsvn/cmsconddbweb/OfflineDropBox • Documentation: • https://twiki.cern.ch/twiki/bin/edit/CMS/OfflineDropBox Martynas Mitrulevičius

  16. Thank you Martynas Mitrulevičius

More Related