James N. Bellinger University of Wisconsin-Madison CMS Week June 2010

1. Coordination: Things to do James N. Bellinger University of Wisconsin-Madison CMS Week June 2010 James N. Bellinger CMS week 2010

2. James N. Bellinger CMSWeek June 2010 Decide on Data Location • Computer cluster for central analysis • Analysis can be done elsewhere (eg. the Barrel fit), but need collection point • We all need access • Disk location • Subdivided by project • Input for subproject • Output from subproject • Communications area • Final output area • Inspection, debugging, etc Can we decide today? HOME LINK ENDCAP BARREL CODE IN OUT WORK CODE Who can make this available?

3. James N. Bellinger CMSWeek June 2010 Understand DB Location • CERN DB for final results • Barrel uses its own DB for all phases of processing • Grouped by project • Input for subproject • Output from subproject Spell this out today.

4. James N. Bellinger CMSWeek June 2010 Define Signaling needs • When is data available-from/agreed-on by all subgroups? • When is the Link fit finished? • When is the Z-calculator finished? • When is the Transfer Line fit finished? • When are the SLM fits finished? • When is the Barrel fit finished? • Do we need to iterate with the Barrel? • Do we have a complete collection? • Is the process complete? Is this complete?

5. James N. Bellinger CMSWeek June 2010 Signaling needs: Breakdown 1 • When is data available from/agreed-on by all subgroups? • We don’t have this process automated, nor do we have a clear naming convention • “Available” means the inputs to Cocoa are ready • When done, start the processing • When is the Link fit finished? • Link is fast and first • Transfer Line and Z-calculator can begin immediately afterwards, using MAB info. Barrel too, though that’s a design question • When is the Z-calculator finished? • This doesn’t exist yet (all by-hand!) • When done, info goes to SLM models

6. James N. Bellinger CMSWeek June 2010 Signaling needs: Breakdown 2 • When is the Transfer Line fit finished? • This took only about 10 minutes • When done, info about Transfer Plate positions has to migrate to the SLM model • Barrel model could be revised to use MAB DCOPS constraints • When are the SLM fits finished? • Takes about an hour • After this, recover the fit CSC chamber positions and interpolate/fit the rest: part of our deliverables

7. James N. Bellinger CMSWeek June 2010 Signaling needs: Breakdown 3 • When is the Barrel fit finished? • Takes over 24 hours • Writes to local database: need to transfer info • Do we need to iterate with the Barrel? • Design question. If Barrel fit has large shifts or doesn’t agree with Transfer Line constraints, may want to iterate; redo Transfer Line et seq. • Do we have a complete collection? • We could have a tentative complete collection even while iterating • Is the process complete? • Write to the DB and set up testing

8. James N. Bellinger CMSWeek June 2010 Testing the fits • Compare with previous and reference • Need estimates of range expected • Count excursions, flag if above some level • Time plots of selected fit quantities? • Human eyeballs needed at first • Data monitoring is a different animal We need to spell out what each group is doing right now

9. James N. Bellinger CMSWeek June 2010 Working Details • CMSSW versions are ephemeral • Need automated “build me a new release” script • Work from different areas, different machines (cocoa files overwrite each other) • Want robust inter-machine communication and file transfer • If all on the same afs/nfs cluster, no problem: semaphore files • Don’t want to monkey with socket programming • Mother process watches for semaphore, starts jobs • Can be tricky

10. James N. Bellinger CMSWeek June 2010 Processing cycle: Given an event time or range • Collect data from subsystems for event and massage to fit • Run Link job • Watch for done on Link • Rewrite HSLM models w/ Link fits, also Z-calculator and Transfer Line: provide info to Barrel • Run Z-calculator, Transfer Line, Barrel • Watch for done on Z-calculator and Transfer Line • Rewrite SLM models, write info for Barrel • Run SLM models • Watch for done on SLM • Fetch and interpolate chamber info from various models • Collect and present • Watch for done on Barrel • Fetch and interpolate chamber info • Collect everything and check against reference • At each step be ready to abort if fails or inconsistent

11. James N. Bellinger CMSWeek June 2010 Devil in the Details:Step 1: Collect data • Specify event by time interval? • First event in the interval if more than one? • What naming convention for events? • Start time? I like seconds in epoch… • All event files, both temporary and permanent, should have the same timing ID somewhere in the name • Easy to get DCOPS if online, scripts exist to do all this stuff: but we probably want a program, unless “mother” is also a script • Not sure how to get Endcap analog. Some jitter: do we want to use an average? • Does Barrel read data directly from the DB? • Need “don’t use this” flags for known bad readings; each system with its own flags and definitions and lookup file or DB • Do we have good automatic sanity checks? I still eyeball plots.

12. James N. Bellinger CMSWeek June 2010 Devil in the Details:Step 2: Run Link • Spawn a process? Needs to be in the right directory, with the right arguments. Can do it, but spawning needs careful monitoring. Using a script as the mother might be better. • Endcap and Transfer Lines wants • MAB fit X/Y/Z and rx/ry/rz; estimated errors would be nice • LD fit positions • What format is good? Simple text files are easy to read • Barrel wants • MAB fit X/Y/Z and rx/ry/rz • ? • What format is good for Barrel? DB?

13. James N. Bellinger CMSWeek June 2010 Devil in the Details:Step 4: Rewrite models • DCOPS uses text SDF files. We can create include files like MinusLD_1276014332.include and use a soft link from the current one to MinusLD.include Thanks to the internal structure of the Transfer Line SDF there would be a lot of these for the MABs. • Since the Z-calculator doesn’t exist yet we can define any input we like. Simpler is better • If I understood correctly the Barrel looks to a database for input for everything, so the rewrite needs to write to a database also. Scripts can do this also.

14. James N. Bellinger CMSWeek June 2010 Devil in the Details:Step 10: Interpolate data • Sometimes a simple fit matches the fit chamber positions (or angles) obviously well, and sometimes it doesn’t and I use an interpolation. Or, as when something doesn’t fit at all, I use the disk position and orientation and apply that to the photogrammetry. • How does our automatic procedure know which to use?

15. James N. Bellinger CMSWeek June 2010 Devil in the Details:Architecture • Need to check for failures at each step • Need to check for timeout failures (hangs, reboots, etc) • Need to have appropriate cleanup procedures at each step • If writing to DB, may need to roll-back changes on failure? Or at any rate flag the entries as BAD

16. James N. Bellinger CMSWeek June 2010 Documentation • DAQ • DCS general controls • Expert-only controls • Data handling • Fitting procedures • What to do with changes All I have is #2 and #3 for DCOPS

17. James N. Bellinger CMSWeek June 2010 Additional Material Sample scripts DAQ Still TODO

18. James N. Bellinger CMSWeek June 2010 Sample scripts 1 • getframeALL.awk • http://www.hep.wisc.edu/~jnb/cms/tools/getframeALL.awk • If report.out was generated using the correct flags and Samir’s code modification, this retrieves the positions and angles for each component in the coordinate system of each of its mother volumes in the hierarchy. • CMS CMS/yep1/slm_p12 x dx y dy z dz rx drx ry dry z drz • Errors are only valid when using the immediate mother volume • In my jargon the output is a “frame” file

19. James N. Bellinger CMSWeek June 2010 Sample scripts 2 • makeNewTPAwkFile.com • http://www.hep.wisc.edu/~jnb/cms/tools/makeNewTPAwkFile.com • If the report.out file was generated by a Transfer Line fit and you created a “frame” file using getframeALL.awk, then • This generates a new awk script whose name incorporates the framework file name • You can then use the new awk script to process an ideal SLM’s SDF and create one with Transfer Plate positions as found by the Transfer Line fit. • I call this re-writing, but that’s misleading: you make a new file with certain parts changed

20. James N. Bellinger CMSWeek June 2010 Sample scripts 3 • unpdbloose.awk • http://www.hep.wisc.edu/~jnb/cms/tools/unpdbloose.awk • This takes a text file containing the row data from an event in the DCOPS database and creates a Cocoa text input data file from it • Refitting using the root histograms gives quality info unfortunately lost in the summary stored into the database, but this works • Since not all insanities are flagged, I edit the file to increase the errors on profiles I know to be bad but which pass the simple quality cuts. This script needs to be replaced by a program which reads in a “known-bad” list.

21. James N. Bellinger CMSWeek June 2010 DCOPS DAQ TODO

22. James N. Bellinger CMSWeek June 2010 DCOPS DAQ • Phoenix DAQ • DCS • Data Quality Monitoring • Data transfer to offline • Transforming selected event (not really DAQ)

23. James N. Bellinger CMSWeek June 2010 Phoenix DAQ • Write out every 60’th event as root? • 1/day, 5MB/event • Gives full plots, more details from fit if required • Need to move root files offline to permanent area • Read Oracle password from protected file • Automatic start at boot time • Can hack this with a cron job and avoid being tied to a single machine • Tools to remotely kill/restart? • Not sure if ssh permissions allow this

24. James N. Bellinger CMSWeek June 2010 DCOPS DCS • Fix fake error bug • Cleanup user interface

25. James N. Bellinger CMSWeek June 2010 Data Quality Monitoring • Not sure how to integrate into overall DQM • Simple job (cron?) can collect raw data for day/week/month and flag excursions in a temperature plot • 1’st question: is the DAQ still running? • Need database table (file at first) with the known bad readings flagged: 504∗4 possible • Need tool for experts to manipulate aforementioned table • Need tool to make diagnostics available • Not keen on reinventing the wheel

26. James N. Bellinger CMSWeek June 2010 DCOPS Data to Offline • Data put in Online DB, never finished job of moving it offline • Move root histogram files (if we want them)

27. James N. Bellinger CMSWeek June 2010 DCOPS Event selection • Easy to create a database query and rewrite the results into a Cocoa-text file • Pieces exist, combine • Need a “bad profile” reference file • Add communication details and locations and naming conventions • Partition into different input files • HSLM files are special, using analog and Link data also • This has to be coordinated with the rest of the group

28. James N. Bellinger CMSWeek June 2010 Link DAQ TODO Barrel DAQ TODO I haven’t a clue. You tell us.