PanDA Monitor Development - PowerPoint PPT Presentation

eudora
panda monitor development n.
Skip this Video
Loading SlideShow in 5 Seconds..
PanDA Monitor Development PowerPoint Presentation
Download Presentation
PanDA Monitor Development

play fullscreen
1 / 15
Download Presentation
PanDA Monitor Development
83 Views
Download Presentation

PanDA Monitor Development

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. PanDA Monitor Development ATLAS S&C Workshop by V.Fine (BNL)

  2. ATLAS User driven development • Maintenance: • Implementing the new “Feature Requests” • Fixing the “known” bugs Development: • high priority, short term developments to address high priority needs of operations and analysis users • plotting system: “flot”-based plots (after investigating options) to add requested plotting features • ”enhanced” job info and statistics pages • jQueryintegrated and used as standard basis for implementing 'client intelligence' and highquality UI • ”json” introduced as standard basis for server->client data export • modular, OO design used in new developments for greater maintainability, reliability, flexibility, and easier reuse PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011

  3. ADC Monitoring Feb 16, 2011: http://indico.cern.ch/conferenceDisplay.py?confId=127099 “User request driven” approach DONE DONE DONE DONE DONE DONE DONE DONE

  4. Monitor: Before and After Panda Monitor Web Server URL QUERY DB SQL QUERY HTML Page Generators Panda Monitor Web Server URL QUERY DB SQL QUERY flot histograms json html/json Where: JQuery “generic” code Page Generators PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011

  5. Jaroslava’s feedback https://indico.cern.ch/getFile.py/access?contribId=2&resId=0&materialId=slides&confId=130719 Slide 5 PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011

  6. Related Savannah tickets PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011

  7. Related Savannah tickets There were 72 Savannah tickets and a bunch of the E-mail communications Trend: the number of “Bug Report” declined , the number of the “Feature request” grew and right now it is on the level 2.5 request per week. There was no complain about quality /performance of the “plotting” approach and its implementation. The requests are coming from users, managers, shifters, PanDA developers Among them: • Reprocessing jobs’ time distribution • Analysis “merging” jobs monitoring • Flexible statistic against of any imaginable filtering per user / date / site / cloud region/ processingType … in the other words against of any job parameter the Panda DB records. There was no new “plotting”-related request recently. There is the growing number of requests for the new “Monitoring” tables (requests (#79437 - “High priority tasks” , #79618 - “Page with list of multicloud sites”) . We are taking in account this trend to set our development priorities. DONE DONE DONE PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011

  8. Site performance issue analysis https://savannah.cern.ch/bugs/index.php?80368 PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011

  9. reco performance: Intel vs. AMD https://indico.cern.ch/getFile.py/access?contribId=10&sessionId=1&resId=0&materialId=slides&confId=119169 From: AlexandreVaniachine <Sacha.Vaniachine@cern.ch> Date: March 18, 2011 8:21:03 PM GMT+03:00 To: davidrousseau <rousseau@lal.in2p3.fr> Cc: Wensheng Deng <wdeng@bnl.gov> Subject: reco performance: Intel vs. AMD ….we observed a difference in HI reconstruction performance on Intel vs. AMD that is larger than the difference in the GHz.Do you think it would be interesting to investigate if there is such a difference on the level of particular algorithms? PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011

  10. DOE/NSF Bi-weekly Operations Meeting PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011

  11. ATLAS User Analysis in the US Cloud http://indico.cern.ch/getFile.py/access?contribId=6&resId=0&materialId=slides&confId=119945 PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011

  12. Panda Monitor / Dashboard Panda Monitor / Dashboard cross references PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011

  13. Output Merging for Analysis Jobs https://indico.cern.ch/getFile.py/access?contribId=121&sessionId=13&resId=0&materialId=slides&confId=119169 PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011

  14. Summary After a month of the development followed by one monthof use Monitor is free of the “known” bug and has been enhanced with about 2 dozens of the new features requested by ATLAS community via Savannah system. It includes the graphical representation of the user selected information. • Supporting large Monitor user communities in operations and analysis • Quick response to up to 4-6 requests/week from ops and analysis users • Incrementally upgrading monitor design/function in the context of the production monitor Near term developments to come: • more work to identify common effort and common code with other monitoring activities • improve flexibility of data summary histograms • client side intelligence, ajax to improve response time and minimize DB load • server side modular python framework for easier support and integration with other monitoring PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011

  15. Questions? PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011