A Quality Certification Model for Grid Research Projects the ETICS feasibility Study

1. A Quality Certification Model for Grid Research Projectsthe ETICS feasibility Study Adriano Rippa (adriano.rippa@eng.it) - Engineering Ingegneria Informatica s.p.a. On behalf of the ETICS Project

2. Summary • Grid Quality Certification Model (GQCM) - UPDATE • Comparing GQCM and other QA standards • Timeline • Gotten feedbacks • Publications • A possible first implementation ETICS All Hands Meeting - Geneva 21-23/11/2007

3. In Madison we left here… • The Grid Quality Assurance Certification Process (GQACP) was organized in three perspectives looking at Code, Platforms and Standards quality aspects. • Now it asses the quality against the same macro-aspects but… Code Standards Platforms ETICS All Hands Meeting - Geneva 21-23/11/2007

4. …Now we are here • GQCM has been reorganized as follows: • A new name! • GQCM has been described according to several ISO standards (e.g. ISO/IEC 25000, 14598) • GQCM has been restructured according to ISO 25041 • Quality attributes has been named using the same terminology as ISO 9126 • More metrics has been added including bounds and target values ETICS All Hands Meeting - Geneva 21-23/11/2007

5. GQCM: the structure • GQCM is structured in Evaluation Modules (EM) according to ISO 25041 • The set of evaluation techniques are grouped in families. Every family is an EM • 5 EMs: • EM: Static analysis • EM: Coding style • EM: Structural testing • EM: Functional testing • EM: Standards compliance ETICS All Hands Meeting - Geneva 21-23/11/2007

6. Evaluation Modules 1/5 • EM Static analysis • Quality characteristics: • Reliability – maturity • Maintainability – analysability • Maintainability – changeability • Maintainability – testability • Technique: • Static analysis of classes. Statistics on measures are used as predictor of quality characteristics. • Applicability: • Object oriented programming languages. • Input: • source code, each class of the code is analysed ETICS All Hands Meeting - Geneva 21-23/11/2007

7. Evaluation Modules 2/5 • EM Coding style • Quality characteristics: • Maintainability – analysability • Technique: • Static analysis of the source code. • Applicability: • Most programming language • Input: • source code ETICS All Hands Meeting - Geneva 21-23/11/2007

8. Evaluation Modules 3/5 • EM Structural testing • Quality characteristics: • Functionality – accuracy • Reliability – maturity • Technique: • Structural testing. The intention of this EM is to test specific classes that are identified by static measures as being statistically more likely to have many errors. • Applicability: • Object oriented programming languages • Input: • source code ETICS All Hands Meeting - Geneva 21-23/11/2007

9. Evaluation Modules 4/5 • EM Functional testing • Quality characteristics: • Functionality – accuracy • Functionality – interoperability • Reliability – maturity • Portability – adaptability • Portability - installability • Technique: • Functional testing • The purpose of this EM is both to check platform compliance and to check to functional abilities of the software product. • Applicability: • General • Input: • compiled code • user documentation ETICS All Hands Meeting - Geneva 21-23/11/2007

10. Evaluation Modules 5/5 • EM Standards compliance • Quality characteristics: • Functionality – standards compliance • Technique: • Functional testing • This EM has a good potential for automating the test. • Applicability: • Standard specific • Input: • compiled code ETICS All Hands Meeting - Geneva 21-23/11/2007

11. GQCM: Final Score • Final score should be provided according to the following schema. The items which should be available for the users are: • A table summarizing the results • A list of passed and non passed tests • All the important information as: • Software product (e.g. name, version, executable code, documentation..) • Platform (name, version, date) • Quality characteristics (name, evaluation result, evaluation module identification) • Standard compliance (for each standard: name, version, date) • Identification of evaluation report (organization, report number, date) • Identification of certification body (organization, contact information) • Certification data (dates, certification number) • Electronic signature of certification record ETICS All Hands Meeting - Geneva 21-23/11/2007

12. GQCM: Summarizing Table example G = Good M = Medium P = Poor ETICS All Hands Meeting - Geneva 21-23/11/2007

13. GQCM integration in an ISO 9126 certified organization • Facts: • ISO 9126 define quality attributes • GQCM support the measure and evaluation of quality metrics • Why they are compatible each other? • Many ISO9126 aspects can be evaluated by GQCM GQCM apply what ISO 9126 asks • Any tool implementing GQCM and can be integrated as supporting ISO9126 adoption • So • GQCM can be both an independent quality model and a “tool” to be easily compliant with ISO 9126 ETICS All Hands Meeting - Geneva 21-23/11/2007

14. GQCM integration in a CMMi certified organization • Facts • CMM is process oriented (it gives guidelines). • GQCM is product oriented. • Hypothesis: • CMM2 • asks to “control and value” • GQCM proposes the measures and metrics to be collected, hence controlling and evaluating specific sw related items • CMM3 • asks to “use internal standards and have a reference model” • GQCM proposes code and implementation conventions (how to organise and structure code, how to define tests, etc.) ETICS All Hands Meeting - Geneva 21-23/11/2007

15. GQCM integration in a CMM certified organization • CMM4 • asks to “have measured and controlled processes using quantitative and statistical techniques” • GQCM proposes the continuous collection of quality data to perform trends analysis. • CMM5 • asks to have an “improvement process and quality based on measurement” • GQCM provides numerical reference to set the improvement processes. • Our Thesis • GQCM is not in contrast with CMMI and can be integrated in organization as a tool to support CMMI adoption ETICS All Hands Meeting - Geneva 21-23/11/2007

16. The timeline EGEE ’07 1-5 October Budapest OGF 21 15-19 October Seattle OGF 20/EGEE UF 7-11 May Manchester Now EELA 3° Conference 3-5 December Catania AHM 23-25 May Madison OCTOBER MAY DECEMBER FEBRUARY QUALIPSO Conference 16-17 January 2008 Rome ECHOGRID/EUChinagrid Conference 24-25 April Beijing Belief Conference 25-28 June Rio de Janeiro ETICS All Hands Meeting - Geneva 21-23/11/2007

17. GQCM: publications • 2 papers submitted • 2 papers accepted • When/Where • EELA third conference • 3-5 December 2007 • Catania/Italy • QUALIPSO first conference • 16-17 January 2008 • Rome/Italy • The first quality oriented conference! ETICS All Hands Meeting - Geneva 21-23/11/2007

18. GQCM: feedbacks • The model has got good feedbacks and acknowledgements • People are asking for more information • Some issues about the name of the model (solved) • One issue about ISO9000 compliance (outside the scope?) • Requests for metrics about the process  outside the scope? ETICS All Hands Meeting - Geneva 21-23/11/2007

19. GQCM: future and open issues • Within ETICS 2 • Implementation • Improvements • Pilot projects to test it • ISO standardisation • Open issue • First implementation? ETICS All Hands Meeting - Geneva 21-23/11/2007

20. What we have now? • Programming languages  Java/Phyton • Others? • JDepend can calculate • Number of classes indicator of the extensibility of the package • Afferent Couplings indicator of the package’s responsibility • Efferent Couplings … independence • Abstractness 0/1  package concrete – abstract • Instability 0/1 • Distance  indicator of the package bilance between abstractness and stability • PMD scans Java source code and looks for potential problems like: • Possible bugs - empty try/catch/finally/switch statements • Dead code - unused local variables, parameters and private methods • Suboptimal code - wasteful String/StringBuffer usage • Overcomplicated expressions - unnecessary if statements, for loops that could be while loops • Duplicate code - copied/pasted code means copied/pasted bugs • Cobertura/Emma • Coverage: class, method, lines, block • FindBugs reports • high/medium/low priority bugs for java • JUnit • Java unit tests • PyUnit • Python unit tests • PyCyclomaticComplexity • Python Cyclomatic Complexity ETICS All Hands Meeting - Geneva 21-23/11/2007

21. A possible first “implementation”? • We can implement partially three evaluation modules • EM: Static Analysis (Only one measure..) • WMC: looks at the complexity of classes • EM: Coding Style • Only some measurement but we can have a realistic value from it • EM: Structural Testing • it is expected that a small part of the code contain a relative large number of the defects in the code. Some metrics are used to identify potential problematic classes. • Use other test cases than already used for statement coverage. • The first implementation will aim at the programming languages Java and Python. […] ETICS All Hands Meeting - Geneva 21-23/11/2007

22. Q&A http://www.eu-etics.org ETICS All Hands Meeting - Geneva 21-23/11/2007