1 / 15

Geant 4 Hadron Validation Framework

Introduction Scope Workflow Select Execute Compare Store Publish Authentication Deployment Summary Next Steps. Geant 4 Hadron Validation Framework. Intro: Motivation and Benefits. Motivation: Standardize testing among hadron model developers Benefits:

sandro
Download Presentation

Geant 4 Hadron Validation Framework

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Introduction Scope Workflow Select Execute Compare Store Publish Authentication Deployment Summary Next Steps Geant 4 Hadron Validation Framework

  2. Intro: Motivation and Benefits • Motivation: • Standardize testing among hadron model developers • Benefits: • Improve consistency of tests • Quality assurance of test procedures • Access to the results in a central location • Share the tools and resources • Share the comparison references

  3. Intro: Team • Authors: • S.Baerjee – Geant 4 Hadron Model developer (FNAL) • S.Gysin – Software Professional (FNAL) • J. Yarba – Geant 4 developer (FNAL) • Reviewers: • J.Apostolakis CERN/PH • G.Folger CERN/PH • D.Elviria FNAL/CD • C.Green FNAL/CD • V.Ivantchenko CERN/PH • A.Ribon CERN/PH • D.Wright SLAC

  4. Intro: Requirement Document • Describes ‘What’ needs to be done • Does not include ‘How’ it will be done • Will be used • to limit the scope and hence the cost • to choose technologies and resources for the design • as an agreement between the user and developer on the functionality to be developed • The most costly errors in software development are made at the requirements stage. • If we get this one wrong we will build the wrong thing

  5. G4 Validation FrameworkScope: • Hadron models only • Regression tests • compare release to release • Validation Tests • compare to physics output • Metric for comparison • MC/data for validation • Chi-squared for regression • Unit tests and system tests are not included

  6. Workflow – the 5 step program • Selecting and building tests and group of tests • Execute tests • Compare the results • Store the results and tests • Publish the results

  7. Selecting Tests and Test Groups • A test is: • Executable + starting condition • Creating a test means • selecting the 2 parts and giving them a name for reference. • Creating test groups means • Making a collection of tests and giving them a name for reference • Detailed requirements about what on pg. 4

  8. Executing Tests • The 3 magic numbers (much debated): • Design to 40,000 test per release • Currently 20,000 • Design to 200,000 CPU hours • Currently 100,000 • Completion time: 3 weeks • Storing results for 2 months after execution • Access to results by all authenticated users • Interrupt option • Must work for small and large tests • Executed on a central server • Web interface

  9. Comparisons • Experimental data • Reference • Storing the output of the comparisons • Quality Measures: MC/Data and Chi-squared • Centralized • Web interface

  10. Storing Results • Save as a reference • Reference store: 10 years • Short Term store: 2 weeks • Control Access by test owner • Centralized • Web Interface • Web Browsing

  11. Publishing • What is published: • Comparisons (charts) • Tests • References • Create and organize hierarchies • requires admin permission • Web interface for browsing • Load: 20 simultaneous users • Central server

  12. Authentication • Authentication • are you who you say you are? • Authorization • do you have permission to do this? • No Authentication needed to view published results • Editing, comparing and publishing restricted to test Owner

  13. Deployment • Central server • Operations: G4 designated admin

  14. Summary • The G4 Validation is categorized into 5 steps • The most resource intensive steps are: • Executing tests • Publishing/browsing the results • Recommend to design these two areas first. • They can be done in parallel and are mostly independent. • They can be used to estimate the work for the framework

  15. Next Steps • Experts review the requirements and give us their honest opinion whether we are on the right track • Finalize the requirements • Allocate resources to the design • Design ‘executing’ and ‘publishing’ • Estimate and plan the implementation

More Related