IRMIS Components Overview

1. IRMIS Components Overview Gabriele Carcassi 18 April 2011

2. IRMIS3 Modules • IRMIS has been conceptually divided into independent modules • Components (installation/cabling/lightweight inventory), Lattice, eLog, PV, … • Motivation: • Easier to have independent development • User can pick and choose • This talk is about the Components module

3. IRMIS3 Data model

4. IRMIS3 Data Model - Components Where is it? How is it powered? How is it controlled? How is it connected?

5. IRMIS3 Data Model - Components • Distinguishes between • Component instance -physical component with inventory information associated with it • Component installation - set of instructions of how a particular component is installed (housed, powered, controlled, cabled, …) • Independent concepts (can track an instance in the inventory without having it installed; can track an installation without caring which instance)

6. IRMIS3 Data Model - Components • Each component is of a particular type • The type defines attributes: manufacturer, ports, which parents/children can have, generic key/value pairs (e.g. pointers to documentation, maximum drawn current, local contact, …) • Each component installation • Lives in multiple hierarchies: control, housing and power • Can be cabled • Can be associated with a Field Name (i.e. ioc name, site naming convention) and a set key/value pairs (i.e. MAC address, geolocation)

7. IRMIS3 Data Model - Components • Instances of each type can be tracked in 3 ways: • NONE: components of that type are not tracked • COUNT: only the number of components is tracked • INSTANCES: each instance is tracked individually • is identified by a serial number which has to be unique within the same type (“N/A” is only non-unique allowed): use barcode if available • can have multiple key/value pairs (i.e. last inspection date, purchase order number, …)

8. IRMIS3 Data Model - Components • Cable type defines what type of cable (copper, fiber, diameter, …) and how is it used (high power, BPM, …) • Each cable connects two ports of two component instances • Support for cable bundle/pull • Each bundle/pull includes a set of cables • Each bundle/pull can be associated with a path of different components (i.e. this bundle goes from this horizontal tray, to this vertical tray, …)

9. IRMIS3 Service architecture

10. Project infrastructure Build and test serverAutomated tests SourceForge project (mailing list, bug tracking, wiki).Monthly release cycle. Production server since January 2009 IE or firefox on Windows/Linux/Mac supported “alpha” - AuthZ and AuthN only designed

11. IRMIS3 Architecture

12. Software Stack 3rd party Java apps Jython scripts 3rd party Perl/Pyton scripts Integration with external tools (i.e. physcs) Few database utilities: backup, consistency check, etc… Web applications Client JavaScript bridge Applets and Widgets Java Client API XML protocol (REST style WS) Data Service layer Server Database layer

13. Getting data <componentType id="90" description="Oscilloscope: 4 Channel @ 500MS/s; 500MHz BW" name="HP54540A"> <manufacturer id="5" name="Agilent (HP)"/> <formFactor id="5" description="Freestanding"/> <functions> <function id="46" description="CCMS"/> <function id="8" description="Instrument"/> </functions> <requires> <interface id="143" description="GPIB_Slave" relType="control"/> <interface id="5" description="Freestanding" relType="housing"/> <interface id="78" description="120VAC" relType="power"/> </requires> <presents> <interface id="234" description="Port" relType="control"/> </presents> <ports> <port name="Chnl 1"> <portType id="13" name="BNC-F" group="RF Connectors"> <pinDesignator id="303" name="1"/> </portType> <pin usage="Chnl 1" pinDesignatorId="303"> <signalType id="1" direction="IN"/> </pin> </port> <port name="Chnl 2">...</port> <port name="Chnl 3">...</port> <port name="Chnl 4">...</port> <port name="RS232">...</port> ... </ports> </componentType> XML returnedREST style (data service) De-normalized data Id references This is an example of what the data service might deliver

14. Writing data XML describing a transaction sent through a POST <transaction xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xmlns='http://xml.bnl.gov/schema/irmis' xsi:schemaLocation='http://xml.bnl.gov/schema/irmis irmis.xsd'> <create> <manufacturer name="Acopian"/> </create> <update> <componentType id="2" name="Site" description="Overall facility. Root of housing hierarchy."> <manufacturer id="1" name="None"/> <formFactor id="1" description="Virtual"/> <properties/> <functions/> <requires/> <presents/> <ports/> </componentType> </update> <delete> <formFactor id="4" description="IndustryPack"/> </delete> <connect> <cable color="blue" label="AA" portAId="123" portBId="321"/> <conductor pinAId="123" pinBId="321"/> <conductor pinAId="124" pinBId="322"/> </connect> </transaction> different independent “commands” no lock “FOR UPDATE” idempotent actions (error or same result) allows remote/disconnected work

15. Implementation Java SE 6 Java EE 5Glassfish 3.x Stream out XML as soon as possibleto reduce delay Servlet Read Business logic JPA SAX MySQL JAXB Write JAXB to convert transaction requestto Java objects JPA vs JDBC: simpler queries to maintain6% performance loss

16. Performance • Dell Precision M4300 laptop, Intel Core Duo T7500, 2.20 GHz - APS dataset • Entire component data set streamed in about 5 seconds (31338 components, 63911 relationships) • Most other queries under 100 ms (manufacturers 88ms, interfaces 95ms) • Roughly half of the time is spent in DB access and half in XML generation • Streaming starts right after DB access • Only tuning done is on the database query (was “good enough”) • Other areas: hardware/os tuning, xml generation improvements, profiling, caching, …

17. IRMIS3 Client API

18. Java API: design principles • The Java API implement the XML protocol and provides objects that represent the data • All reads are cached: no need to keep caches in your code. • All reads are synchronized • you can have reads on multiple threads on the same data and will not trigger multiple reads on the server (extremely useful for GUIs)

19. Java API design principles • The read access is object oriented • Data is exposed through interfaces • Component, ComponentType, Manufacturer • Some classes function as “finders” for the classes containing the data • Components.getById(123), FormFactors.getByName(“Rack”) • You can follow references from one object to the other • component.getComponentType().getManufacturer()

20. Java API design principles • The write access is protocol oriented • You create a transaction • You call methods to add elements in the transaction • Manufacturers.createManufacturer(transaction, “ACME”) • You save the transaction, which may fail • The read access is refreshed only after the data is saved

21. Importing components from Excel (using Jython) from gov.bnl.irmis.api import * from java.net import URI from java.io import File from jxl import * workbook = Workbook.getWorkbook(File(“components.xls")) sheet = workbook.getSheet(0) nRows = sheet.getRows() nColumns = sheet.getColumns() irmisDB = IrmisDB.getInstance(); irmisDB.setURI(URI("http://cs-build.nsls2.bnl.gov:8080/IRMISDataService/data")) transaction = irmisDB.createTransaction(); for row in range(1, nRows): componentTypeName = sheet.getCell(0, row).getContents(); componentType = ComponentTypes.getByName(componentTypeName); serialNumber = sheet.getCell(1, row).getContents(); Components.createComponent(transaction, componentType, serialNumber) transaction.save(); print "Operation successful"

22. Importing from Excel (Jython) from gov.bnl.irmis.api import * from java.net import URI from java.io import File from jxl import * workbook = Workbook.getWorkbook(File(“components.xls")) sheet = workbook.getSheet(0) nRows = sheet.getRows() nColumns = sheet.getColumns() irmisDB = IrmisDB.getInstance(); irmisDB.setURI(URI("http://cs-build.nsls2.bnl.gov:8080/IRMISDataService/data")) transaction = irmisDB.createTransaction(); for row in range(1, nRows): componentTypeName = sheet.getCell(0, row).getContents(); componentType = ComponentTypes.getByName(componentTypeName); serialNumber = sheet.getCell(1, row).getContents(); Components.createComponent(transaction, componentType, serialNumber) transaction.save(); print "Operation successful" Load excel file

23. Importing from Excel (Jython) from gov.bnl.irmis.api import * from java.net import URI from java.io import File from jxl import * workbook = Workbook.getWorkbook(File(“components.xls")) sheet = workbook.getSheet(0) nRows = sheet.getRows() nColumns = sheet.getColumns() irmisDB = IrmisDB.getInstance(); irmisDB.setURI(URI("http://cs-build.nsls2.bnl.gov:8080/IRMISDataService/data")) transaction = irmisDB.createTransaction(); for row in range(1, nRows): componentTypeName = sheet.getCell(0, row).getContents(); componentType = ComponentTypes.getByName(componentTypeName); serialNumber = sheet.getCell(1, row).getContents(); Components.createComponent(transaction, componentType, serialNumber) transaction.save(); print "Operation successful" Connect to the service

24. Importing from Excel (Jython) from gov.bnl.irmis.api import * from java.net import URI from java.io import File from jxl import * workbook = Workbook.getWorkbook(File(“components.xls")) sheet = workbook.getSheet(0) nRows = sheet.getRows() nColumns = sheet.getColumns() irmisDB = IrmisDB.getInstance(); irmisDB.setURI(URI("http://cs-build.nsls2.bnl.gov:8080/IRMISDataService/data")) transaction = irmisDB.createTransaction(); for row in range(1, nRows): componentTypeName = sheet.getCell(0, row).getContents(); componentType = ComponentTypes.getByName(componentTypeName); serialNumber = sheet.getCell(1, row).getContents(); Components.createComponent(transaction, componentType, serialNumber) transaction.save(); print "Operation successful" Loop over the rows

25. Importing from Excel (Jython) from gov.bnl.irmis.api import * from java.net import URI from java.io import File from jxl import * workbook = Workbook.getWorkbook(File(“components.xls")) sheet = workbook.getSheet(0) nRows = sheet.getRows() nColumns = sheet.getColumns() irmisDB = IrmisDB.getInstance(); irmisDB.setURI(URI("http://cs-build.nsls2.bnl.gov:8080/IRMISDataService/data")) transaction = irmisDB.createTransaction(); for row in range(1, nRows): componentTypeName = sheet.getCell(0, row).getContents(); componentType = ComponentTypes.getByName(componentTypeName); serialNumber = sheet.getCell(1, row).getContents(); Components.createComponent(transaction, componentType, serialNumber) transaction.save(); print "Operation successful" Get data outCreate components

26. Fluent API • Navigating through hierarchies can be tedious • On top of the “beans” API there is a fluent API (internal domain specific language) Search API Builders Change Set API

27. IRMIS3 User interface

28. UI design • Web interface: uses applets and Web Start (both greatly improved with Java 6u10) • Uses Swing standard components plus open source (SwingX, Flamingo, JIDE, …) • UI elements can optionally be used in custom pages and integrated with javascript

29. UI elements Component hierarchy tree widget (filtering, sorting, search, …) Component pathwidget

30. Component applications

31. Component inventory

32. Cabling applications

33. Status • Component and Component types applications is considered production level • Cabling UI is considered beta while the API and service support is considered production