1 / 18

Continuous Integration for sPHENIX

Continuous Integration for sPHENIX. Jin Huang (BNL). sPHENIX Software Repo. Hardware Design Decisions. Physics obs. Projection. Detector Design. → G4 Simulation → Digitization Reconstruction Real Data. Open source @

elvirab
Download Presentation

Continuous Integration for sPHENIX

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Continuous Integration for sPHENIX Jin Huang (BNL)

  2. sPHENIX Software Repo. Hardware Design Decisions Physics obs. Projection Detector Design → G4 Simulation → Digitization Reconstruction Real Data Open source @ https://github.com/sPHENIX-Collaboration/ Core software: 200k line of code Analysis: 59k line of code Jin Huang <jhuang@bnl.gov>

  3. Questions and solutions Jin Huang <jhuang@bnl.gov> You might had these questions: When did jet energy scale changed? Did the code run slower/faster recently? Would my pull request break the sPHENIX build? I build this new module, may I have some suggestion? Our new solution: Implementing continuous integration for sPHENIX software Automatically run compile, QA, and code analysis for new GitHub pull request. Regularly test/archive default setups

  4. Pull request checks https://github.com/sPHENIX-Collaboration/coresoftware/pulls On-going checks Passed checks Report available Jin Huang <jhuang@bnl.gov>

  5. Would my pull request break the sPHENIX build? https://github.com/sPHENIX-Collaboration/coresoftware/pull/509#issuecomment-452144033 Reports automatically post to pull request Also sent to email list: https://lists.bnl.gov/mailman/listinfo/sphenix-github-l ← Passed tests Failed tests → Jin Huang <jhuang@bnl.gov>

  6. Behind the scene https://github.com/sPHENIX-Collaboration/coresoftware/pull/509#issuecomment-452144033 • Run pipelines of tests • On a dedicated 32-core server • Reproduce RCF env with sPHENIX Singularity container https://github.com/sPHENIX-Collaboration/singularity Jin Huang <jhuang@bnl.gov>

  7. Did the code run slower recently? https://github.com/sPHENIX-Collaboration/coresoftware/pull/509#issuecomment-452144033 https://web.racf.bnl.gov/jenkins-sphenix/job/sPHENIX/job/test-calo-single-qa/ Jin Huang <jhuang@bnl.gov>

  8. When did jet energy scale changed? https://github.com/sPHENIX-Collaboration/coresoftware/pull/509#issuecomment-452144033 • https://github.com/sPHENIX-Collaboration/coresoftware/pull/104 • https://github.com/sPHENIX-Collaboration/coresoftware/pull/105 • https://github.com/sPHENIX-Collaboration/coresoftware/pull/107 Jin Huang <jhuang@bnl.gov>

  9. When did jet energy scale changed? https://web.racf.bnl.gov/jenkins-sphenix/job/sPHENIX/job/test-calo-single-qa/13//QA_20Report/ Jin Huang <jhuang@bnl.gov>

  10. When did jet energy scale changed? https://web.racf.bnl.gov/jenkins-sphenix/job/sPHENIX/job/test-calo-single-qa/13//QA_20Report/ Jin Huang <jhuang@bnl.gov>

  11. May I have some suggestion on my code? https://github.com/sPHENIX-Collaboration/coresoftware/pull/538#issuecomment-452992916 Jin Huang <jhuang@bnl.gov>

  12. More checks … https://github.com/sPHENIX-Collaboration/coresoftware/pull/538#issuecomment-452992916 Jin Huang <jhuang@bnl.gov>

  13. Jenkins dashboard https://web.racf.bnl.gov/jenkins-sphenix/view/sPHENIX%20Pipelines/ Jin Huang <jhuang@bnl.gov>

  14. Need your help Jin Huang <jhuang@bnl.gov> • Tracking QA modules → plots • Jet samples to run/archive jet QA • Summarize QA results, e.g. KS-tests • Add more checks • Pull request check (~100 CPU-hr) • Weekly checks (~1000 CPU-hr) • Suggestions welcomed • At Tue sim./software meeting: https://indico.bnl.gov/category/88/ • At email list:  sphenix-software-l@lists.bnl.gov.

  15. Extra information Jin Huang <jhuang@bnl.gov>

  16. Portability and containers https://singularity.lbl.gov/ Jin Huang <jhuang@bnl.gov> • sPHENIX software is built and distributed on RCF only. But there are needs to use it offsite • Use computing resource at grid or user institutes • Development code offline from RCF • RCF has a container for sPHENIX already • A light layer of software environment wrapping for sPHENIX computing environment • Based on singularity container (non-SU version of Docker). • Promising to solving software compatibility reproducibility issue. • Called for volunteer testing in last collaboration meeting

  17. Update from the last meeting Ross Corliss (MIT) Ron Soltz (LLNL) sPHENIX in container @ BNL TPC lab sPHENIX in container @ LLNL Fe-55 → GEM →SAMPA → FELIX Jin Huang <jhuang@bnl.gov> RCF authorized us to allow distribution to sPHENIX collaborators Simple macros setup for download and sync with nightly builds: https://github.com/sPHENIX-Collaboration/Singularity Thanks to our 1st users for trying it out: Ross Corliss (MIT) and Ron Soltz (LLNL) Possible sim production @ LLNL super computer Expect coming version of container updated to use CVMFS, reduce downloads

  18. Wish list (a few as example) EMCal tower occupancy A test Jenkins CI server DELPHES 3 [arXiv:1307.6346 ] Zero suppression Jin Huang <jhuang@bnl.gov> Simulation and evaluation of SiPM noise and zero-suppression20 GeV energy in EMCal just from SiPM noise and zero suppression Fast simulation: General packages e.g. DELPHES? Continuous Integration for sPHENIX softwareTesting a Jenkins for automatic software verification Containers, virtual machines: Distribute sPHENIX software to your laptop

More Related