1 / 22

Toward a Quality Certification Model for Grid Research Projects: an ETICS feasibility Study

Toward a Quality Certification Model for Grid Research Projects: an ETICS feasibility Study. Andrea Manieri Adriano Rippa - Engineering Ingegneria Informatica s.p.a. On behalf of ETICS Project. The starting point. Current quality assurance standards are useful but: are process oriented

winka
Download Presentation

Toward a Quality Certification Model for Grid Research Projects: an ETICS feasibility Study

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Toward a Quality Certification Model for Grid Research Projects: an ETICS feasibility Study Andrea Manieri Adriano Rippa - Engineering Ingegneria Informatica s.p.a. On behalf of ETICS Project

  2. The starting point • Current quality assurance standards are useful but: • are process oriented • provide theoretical guidelines on the organization • Theproblem: • Guidelines can be applied but it’s hard to systematically verify goodness of results • Oursolution: • A quality assurance model • fully automatable in measuring and verifying activities • not subjective ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

  3. The timeline OGF 21 15-19 Oct Now ALL HANDS MEETING 23-25 May OGF 20/EGEE UF 7-11 May FIRST BETA VERSION FINAL BETA VERSION FIRST OFFICIAL VERSION FEBRUARY MAY OCTOBER DECEMBER ECHOGRID/EUChinagrid Conf. Beijing 24-25 April QUALIPSO Conference Dec 2007 Belief Conference RIO – BRASIL 25-28 June ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

  4. Terminology • Measure: a value (number or category) assigned to an attribute of an entity (e.g. software component) • Measurement: The act or process of assigning a number or category to an entity to describe an attribute of that entity. • Metric: A function whose inputs are software measures and whose output is a single numerical value that can be interpreted as the degree to which software possesses a given attribute that affects its quality. • Perspective: A set of homogenous software attributes that lead to some quality assumptions ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

  5. Gird Quality Assurance Certification Model (GQACM ): a 3D QA Model • Let’s suppose A, B, C to be different perspectives • Most of Common QA Standards (e.g. CMMI) are Waterfall-like • GQACM uses 3 independent perspectives to provide client-friendly results • Users can be interested in some attributes and less in others.. • So-made perspectives can have sublevels… A B C A C B ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

  6. GQACM: Preliminary remarks • GQACM is independent from the ETICS tools • However, any GQACM implementation must use automation – the most useful characteristic of the model is that it can be automatically applied! • GQACM has three perspectives (I.e. independent points of view) to derive the quality degree of a software : • Code and test • Platforms • Standards ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

  7. Perspectives composition • Code Test Perspective: to qualify the goodness of the code and of the test • Coding Conventions • Code Commenting • Other QA metrics related to the corresponding programming language • Unit and functional tests • Platform perspective: to qualify B&T requirements on different platforms • The set of GQACM available platforms • Standard perspective: to qualify the compliance with standards (e.g. IPV6, BES, JSDL, SRM, WS-I) • The set of GQACM available standards ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

  8. Test execution metrics • Cyclomatic complexity • CCT (method level): • if C is the cyclomatic complexity (of the method) and T is the number of executed tests then CCT = C/T • RFCT (class level): • if CK RFC is the the number of methods which can be invoked from an instance of the class target to answer a message ) and T is the number of executed tests on it then RFCT = RFC/T • ET (class level): • if E is the number of events which interact with the class and T is the number of executed tests then ET = E/T • NOCT (class level): • if CK NOC is the Number Of Children of a class and T is the number of executed tests on the class then NOCT = NOC/T ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

  9. Code comment metrics • FC: • if F is the number of functions and C is the number of comment paragraphs then FC = F/C • AD: • Automatical documentation programs use: e.g. doxygen ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

  10. Coding convention metrics 1/2 • Only one class definition per header file • Filename = Classname • Filename in lowercase • No cycles in “include” operations • No C instruction (in C++ programming) • Coherent Programming Convention • Curly Brackets positioning • First character in variables names • Lower case in macros names • Coherent use of new lines • Semicolons only at the end of a line ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

  11. Coding convention metrics 2/2 • No Never used functions • No Not reachable code paths • No Not defined prototypes • No Never used constants • No Never used macros • No Never used types definitions • No Never used variables • Too frequent “breaks” use • No absence of variables de-alloaction • No Null pointers • Management of one instruction long “if” or one instruction long “cycle” • Shared resources management ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

  12. other metrics • Other software related metrics • (e.g. Chidamber and Kemerer (CK) Metrics for OO software) ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

  13. Final Score1/2 • For every perspective must be genrated: • a score between 0 and 100 • a list of passed and not passed tests • The way to calculate the score is different for every perspective. • For code perspective • The score is simply calculated using this mathematical formula: (X/N)*100 where X is the number of successes, N the number of used metrics ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

  14. Final Score 2/2 • For platform perspective • If • DP is the number of declared compliant platforms, • CP the number of really compliant platforms, • TP the number of GQACM available platforms , • CTP the total number of compliant platforms , • then the score can be calculated with this mathematical formula: (CP/DP)*50+(CTP/TP)*50 • For standard perspective • If • DS is the number of declared compliant standards, • CS is the number of really compliant standards, • TS is the number of available standards, • CTS is the total number of compliant standards (declared + not declared), • then score can be calculated with this mathematical formula: (CS/DS)*50+(CTS/TS)*50 ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

  15. Score visualization • The graphical view of the score is related to the used test tool • In general, all tools have to use at least the “classical version”: • Final score: <total>/<max possible score> ([<perspective><score>]) • Where • Perspective can be: Code (C), Platforms (P), Standards (S) • Score: 0-100 (where 0 is 0% compliant, 100 is 100% compliant) • Total score: sum of the scores • E.g. component X: GQACM score = 151/300 (C90-P51-S10) • ETICS could complement the standard basic visualization using a 3d graph to give the user a better feeling interface ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

  16. GQACM integration in an ISO 9126 certified organization • Facts: • ISO 9126 define quality attributes . • GQACM support the measure and evaluation of quality metrics. • Hypothesis: • ISO9126 Portability: adaptability, installability • ISO9126 asks OS adaptability and installability for software of certified organizations. • GQACM may test deployment of SW on different platforms (HW+OS) • ISO9126 Maintainability: all. • ISO9126 asks to develop a software that is easy to analyse, easy to modify, Stable • GQACM metrics (such as the CK for OO or coding convetions metrics) helps to verify in concrete these requirements ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

  17. GQACM integration in an ISO 9126 certified organization • ISO9126 Functionality: accuracy. • ISO9126 asks that a software have to give results according with requirements. • GQACM certification can run functional test (or plug-in external test suits) of declared compliances. • ISO9126 Functionality: adequacy. • ISO9126 asks for correct functions with specific goals. • GQACM may check not dead functions. • Our thesis: • Many ISO9126 quality metrics can be measured and evaluated based on GQACM • Any tool implementing it and can be integrated as supporting ISO9126 adoption ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

  18. GQACM integration in a CMMi certified organization • Facts • CMM is process oriented (it gives guidelines). • GQACM is product oriented. • Hypotesis: • CMM2 • asks to “control and value” • GQACM propose the measures and metrics to be collected, hence controlling and evaluating specific sw related items • CMM3 • asks to “use internal standards and have a reference model” • GQACM proposes code and implementation conventions (how to organise and structure code, how to define tests, etc.) ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

  19. GQACM integration in a CMM certified organization • CMM4 • asks to “have measured and controlled processes using quantitative and statistical techniques” • GQACM propose the continuous collection of quality data to perform trends analysis. • CMM5 • asks to have an “improvement process and quality based on measurement” • GQACM provide numerical reference to set the improvement processes!! • Our Thesis • GQACM is not in contrast with CMMI and can be integrated in organization as a tool to support CMMI adoption ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

  20. ETICS B&T tool and GQACM • ETICS: • Could perform many measurements and calculate metrics (e.g. JPmdPlugin and JFindbugsPlugin) Code and test perspective • Can build and test software on several distinct platforms  Platform perspective • Could test the software compliance with standards  Standard perspective • Can automate all the processes (build, deployment, test) ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

  21. Conclusions • GQACM is a certification model • Automatable • 3D • Not in contrast with classical standards • Free • GQACM doesn’t require big efforts to be used ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

  22. Thanks Questions? Feedbacks? http://www.eu-etics.org ETICS 3rd All-Hands Meeting - Universiti of Wisconsin, Madison 23-25/05/2007

More Related