1 / 16

Iterative Development of a Domain-Specific Fault Classification An Industrial Case Study

Iterative Development of a Domain-Specific Fault Classification An Industrial Case Study. Philip Preissing and Jan Schulte 2009-06-19. Table of Contents. Introduction RUAG Context Related Work & Quality Characteristics Development Process Case Study Discussion Summary.

tamah
Download Presentation

Iterative Development of a Domain-Specific Fault Classification An Industrial Case Study

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Iterative Development of a Domain-Specific Fault Classification An Industrial Case Study Philip Preissing and Jan Schulte2009-06-19

  2. Table of Contents WEMSE Workshop- 2009-06-19 • Introduction • RUAG • Context • Related Work & Quality Characteristics • Development Process • Case Study • Discussion • Summary

  3. RUAG Aerospace Sweden AB • Headquarter in Göteborg • 360 employees • Formerly SAAB Aerospace • Highly reliable Satellite Equipment • Computers • Antennas • Microwave Systems • Projects • Ariane 5 • Herschel/Planck • Galileo WEMSE Workshop- 2009-06-19

  4. Problems at RUAG High dependability & reliability ECSS Standards Late faults several times more expensive  Software Testing accounts for 60% of Development Time Master thesis to develop an optimization framework for the Verification & Validation Activities (VAs) WEMSE Workshop- 2009-06-19

  5. Goals • Measure defects Gain insight into VAs Fault-slippage between VAs Overlap in VAs  Fault classification(FC) to group similar types of defects • Analyze problems in the process on a high level • Simplify the measurement process WEMSE Workshop- 2009-06-19

  6. Example WEMSE Workshop- 2009-06-19

  7. Related Work WEMSE Workshop- 2009-06-19 • Schemes exist • Orthogonal Defect Classification (IBM) • Standard Classification for SW Anomalies (IEEE) • Origins, Types & Modes (HP) • FC in use at Ericsson • Classifications need to be adapted (Case studies) • Development processes • Expert opinion • Commit comments  Development process needed

  8. Quality characteristics • Classes should… … be at most 5-10 … describe the fault type … be orthogonal … be consistent … be complete … be applicable to every software artifact WEMSE Workshop- 2009-06-19

  9. Development Process • Initial Fault Classification • Iterative Refinement • Selection • Classification • Comparison & Discussion • Quality Review WEMSE Workshop- 2009-06-19

  10. Case Study – Initial Fault Classification WEMSE Workshop- 2009-06-19

  11. Case Study – Iterative Refinement • Select data source • Code Inspection sheets • Classification of faults • Completely sure • Uncertain • Don‘t know • Analysis Several iterations WEMSE Workshop- 2009-06-19

  12. Case Study – Quality Review • Agreement Factor (Consistency) • Initial: 0.30 • Final: 0.71 • Workshop • Project Manager, Designer, Developer, Tester • Only minor changes WEMSE Workshop- 2009-06-19

  13. Advantages & Drawbacks • Analysis of all stages • Low developer involvement • Participants have limited understanding • Validation first performed at the end WEMSE Workshop- 2009-06-19

  14. Lessons learned • A posteriori analysis difficult • Documentation important • Alternative data sources difficult • Checkin comments • Observation of developers • Interviews WEMSE Workshop- 2009-06-19

  15. Summary WEMSE Workshop- 2009-06-19 • Software Testing very expensive • FC necessary to analyse & improve process • Process to develop domain-specific FC • Incorporating existing schemes • Iterative refinement by analyzing all steps • Quality review • Validation in industrial case study at RUAG

  16. WEMSE Workshop- 2009-06-19 Thank you very much! Merci beaucoup ! Tack så mycket! Muchasgracias! Muitoobrigado! Vielen Dank! Moltes gràcies! Millegrazie!

More Related