Database reports and the ioc crawler
1 / 13

Database Reports and the IOC Crawler - PowerPoint PPT Presentation

  • Uploaded on

Database Reports and the IOC Crawler. Presented by Katia Danilova 09/01/2005. Please. Hold the questions and comments, write down your questions and suggestions! discussion is planned after the presentation we can use your notes to address the problems later Thank you!.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Database Reports and the IOC Crawler' - ova

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Database reports and the ioc crawler

Database Reports and the IOC Crawler

Presented by Katia Danilova



  • Hold the questions and comments, write down your questions and suggestions!

    • discussion is planned after the presentation

    • we can use your notes to address the problems later

      • Thank you!

Why to discuss reports and crawler together
Why to discuss Reports and Crawler together:

  • They are “complimentary” programs providing access to IOC configuration info for the client (user)

  • Crawler takes IOC data from EPICS servers and puts them into the database

  • Web-based reports take this data from the database for the end user

    • ROCS system is under development

    • Reports by request (in the form of spreadsheet or flat file) are possible


  • Crawler is a computer program which “crawls” through the IOC configuration related files on EPICS boot servers, collects the information, constructs a profile for each IOC, and automatically saves this data in a relational database.

  • Language used:

    • Initially: Java (Jeff Patton)

    • Currently: Perl (Don Dohan, Jeff Patton, Greg Lawson)

  • Relational Database used:

    • mySQL (Argonne)

    • Oracle (SNS)


  • Participants: Argonne and SNS

    • Don Dohan (Argonne)

    • Coles Sibley, Jeff Patton, Greg Lawson (SNS)

  • Crawler is a part of IRMIS (Integrated Relational Model of Installed Systems )

  • Basic package is made in Argonne

  • Other labs can add the modules to adjust the program to their realities – this is what Greg Lawson does for Control Systems

Examples of files parsed
Examples of files parsed:

  • Startup.cmd

  • Bootline

  • St.cmd

  • iocInfo/

    • bspVersion

    • epicsVersion

    • vxVersion…

    • pvList

  • db, dbd template files

Why controls systems care about the crawler
Why Controls Systems care about the Crawler:

  • the latest information (about each IOC, or how many people use this or that version, or software changes…) is easily accessible through reports and viewing programs

  • Some problems are already identified:

    • Mistakes/typos in PV names

    • Can tell when somebody is not following the naming standards

  • In a case of disaster recovery: can look at the last configuration and reconstruct IOC from the DB

  • If a bug in a certain version of driver found: can find all IOC that require manual changes

In perspective
In perspective

  • Rather than pulling info from IOC, info can be pushed into IOC from the database automatically

    • If global changes are required: all IOC can be pushed at once by making changes in one place in the DB and can be loaded automatically instead of restoring everything manually

    • If a bug found: changes to all IOC that use the same driver can be done at the same time

  • If in addition to writing to a disk all IOC info goes to DB, then it can be used for logging almost real time

  • DB applications that talk directly to IOC can be created

Current issues to solve for developers
Current Issues to solve for developers:

  • Crawler is under development =>

  • Every time Argonne makes changes to improve the program, SNS has to make adjustments (Jeff Patton has to adopt all the changes to Oracle)

  • Need to develop GUI viewing programs (like Archiver and ROCS)

  • Currently, when IOC engineers make changes in a file structure, or just move the files, crawler needs to be updated to be able to find info

For the ioc engineers
For the IOC engineers:

  • Keep the standard organization of the directories, otherwise crawler does not know were to look for the files. Order is important!

  • Keep the standard structure of the files the program parses, otherwise crawler can not find data. Order is important!

  • If there are no concrete standards yet, probably it’s the time to develop them?

My sources
My sources:

  • IRMIS collaboration meeting

  • Relational Database Collaboration @ Argonne National Laboratory.ppt

  • Argonne National Laboratory: IRMIS PV Crawler.ppt

  • Coles Sibley. SNS Requirements.ppt

  • Jeff Patton. ORNL EPICS RDB Tools.ppt

  • Jeff Patton: Extending RDB Core.ppt

  • An overview of IRMIS

  • Interviews with J. Patton, G. Lawson, C. Sibley, E. Williams

Crawler animation
Crawler Animation:

  • is a student project completed by Katia Danilova for Flash class at UT, spring 2005

  • Requirement: develop an animated web-based instructional/ educational unit (3-5 min length, funny creatures designed)

  • This unit:

    • shows a little of how the IOC boots up

    • shows how the crawler gathers IOC configuration info from the EPICS IOC boot server

    • shows how a client, for example a web based report, gets an updated info from a DB

      Crawler Animation