1 / 25

EMI Quality Assurance Tools

EMI Quality Assurance Tools. Lorenzo Dini (CERN) SA2.4 Task Leader. Outline. Task Goals Initial situation Build, Integration, Packaging Build and Test Infrastructure Software Repositories Continuous Testing Quality Assurance Metrics Workflow QA Reports. SA2.4 - EMI QA Tools.

psikes
Download Presentation

EMI Quality Assurance Tools

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. EMI Quality Assurance Tools Lorenzo Dini (CERN) SA2.4 Task Leader

  2. Outline • Task Goals • Initial situation • Build, Integration, Packaging • Build and Test Infrastructure • Software Repositories • Continuous Testing • Quality Assurance • Metrics Workflow • QA Reports EGI User Forum - EMI Technical Forum - April 11-14 Vilnius

  3. SA2.4 - EMI QA Tools • EMI QA tools not only for Quality Assurance • Single tool-chain and workflow for • Build, integration, Packaging, Testing, QA • Unified project builds in a single box • Single release tool • Single dependency management • Uniform and compliant packaging • Tests stored together with configuration • QA metrics created in builds EGI User Forum - EMI Technical Forum - April 11-14 Vilnius

  4. Initial Situation EGI User Forum - EMI Technical Forum - April 11-14 Vilnius

  5. Survey Results 1 EGI User Forum - EMI Technical Forum - April 11-14 Vilnius

  6. Survey Results 2 EGI User Forum - EMI Technical Forum - April 11-14 Vilnius

  7. Survey Results 3 EGI User Forum - EMI Technical Forum - April 11-14 Vilnius

  8. Challenges • Differentcultures, environments, requirements and priorities led to different tools difficult to merge • Uniform the build environment to provide a common software engineering workflow without affect developer efficiency • Provide a comprehensive infrastructure able to satisfy the totality of needs • Introduce testing, QA metrics generation and reporting in the same tool-chain • Support everybody on a new system EGI User Forum - EMI Technical Forum - April 11-14 Vilnius

  9. Build, Integration and Packaging • Taken the versatility of the gLite ETICS system as a general framework • Adopted the ARC dependency management with EPEL and packaging with Mock (Debian with PBuilder) • Configured minimal worker nodes as dCache with packages installed as privileged user at build start and build in user space • Kept Maven to uniformly and effectively build Java components as UNICORE EGI User Forum - EMI Technical Forum - April 11-14 Vilnius

  10. Build, Integration and Packaging • Every EMI software component is configured in a single ETICS project and organized in releases • The whole release is built from source in a single minimal clean node 4 times a day • ETICS checks-out the source code, installs the dependencies using YUM/APT and triggers the build systems (Autotools, Ant, Maven) in user space • ETICS creates missing SPEC files and produces RPMs and DEBs • ETICS triggers Mock/PBuilder in a pristine environment for the packaging EGI User Forum - EMI Technical Forum - April 11-14 Vilnius

  11. Build, Integration and Packaging EGI User Forum - EMI Technical Forum - April 11-14 Vilnius

  12. Build, Integration and Packaging EGI User Forum - EMI Technical Forum - April 11-14 Vilnius

  13. Build and Test Infrastructure • Virtual infrastructure based on CERN Microsoft HyperV and Condor • >50 VMs in >10HVs available at any time for builds and tests • VMs are automatically scratched after jobs to ensure clean environments • Some high performance nodes have been dedicated to project builds to minimize the build cycle EGI User Forum - EMI Technical Forum - April 11-14 Vilnius

  14. Build and Test Infrastructure EGI User Forum - EMI Technical Forum - April 11-14 Vilnius

  15. Software Repositories • EMI software repositories are provided for each release and release candidate • These repositories are automatically populated after builds • An EMI external repository is also provided for packages not available in the OS (SL, EPEL, Debian) • Automatic ETICS repositories are also generated with the contents of the builds EGI User Forum - EMI Technical Forum - April 11-14 Vilnius

  16. Continuous Testing “The gLite Data Management Continuous Integration and Testing Process” – 14:30 EGI User Forum - EMI Technical Forum - April 11-14 Vilnius

  17. Quality Assurance • The goal of Process Metrics is to identify improvement opportunities in the way the software is developed and tested, from a process point of view. In our context the main sources of process metrics are the Defect Tracking tools used in EMI. • Product (or Internal) Metrics measure certain characteristics of the product like complexity, changeability and testability. At the moment we rely on static code analyzers to measure these characteristics. • ”Software Metrics Defines, Reports and Analysis in EMI” – 12:00 EGI User Forum - EMI Technical Forum - April 11-14 Vilnius

  18. QA Metrics Workflow Process http://gridka-school.scc.kit.edu/img/dCache_logo.gif Product

  19. Bug Tracker exporters • The middleware distributions decided not to merge their tracking systems • Agreements on fields, states, workflow and interfaces to track the process • Exporters query each system and provide a standard XML document with the data • This documents are stored and backed up on servers provided by distributions “Software Quality Assurance in EMI” – 11:00 EGI User Forum - EMI Technical Forum - April 11-14 Vilnius

  20. QA Plug-ins • QA Plug-ins are ETICS modules that run during the build and trigger common static analysis tools for QA • SLOCCount, FindBugs, PMD, Checkstyle, PyLint, RPMLint, CPPCheck, etc... • Product QA Metrics are produced and stored permanently in the ETICS repository EGI User Forum - EMI Technical Forum - April 11-14 Vilnius

  21. QA Report Generator • Queries the XML for the process metrics and the ETICS repository for the product metrics creating aggregated metrics. • Generates charts, plots and tables with QA information and fills in QA templates with the dynamic information. • Reports can be produced daily for Product Teams and Release Manager • Produces drafts for EU QA Deliverables EGI User Forum - EMI Technical Forum - April 11-14 Vilnius

  22. QA Reports: Process Metrics EGI User Forum - EMI Technical Forum - April 11-14 Vilnius

  23. QA Reports: Product Metrics EGI User Forum - EMI Technical Forum - April 11-14 Vilnius

  24. Conclusions • A very heterogeneous starting point • Uniform software engineering tool-chain • Successfully integrated EMI 0 and EMI 1 • New integrated testing and QA process • All performed improvements benefit all teams at once. • Specific solution can make a single team happier but are globally more expensive and not maintainable. EGI User Forum - EMI Technical Forum - April 11-14 Vilnius

  25. Thank you EMI is partially funded by the European Commission under Grant Agreement INFSO-RI-261611 EGI User Forum - EMI Technical Forum - April 11-14 Vilnius

More Related