The Value of Managing the Review Process Mike Chapman
Intro • Metrics Data Program • NASA Glenn Research Center • JPL • Michael Fagan
Definitions • Inspection – In software engineering this refers to peer review of any work product to look for defects. • Might also be called a Fagan Inspection • Common review practice • The goal is to identify and repair defects • Code Review – a kind of inspection that identifies defects in source code • Peer Review • Software walkthroughs • Software inspections
Elements of a Peer Review • Structured review process • Defined roles of participants • Forms and reports
The Standard Process Review is scheduled Reviewers fill in comments or defects 1. 2. 3. Results are summarized Reviewers convene, discuss defects 4. (Time can be tracked throughout the process) 5. End Review
The Process (cont) • Instantiation • Who • What • When • Where • Preparation • The key to success • Review • Resolution • Enter defects into tracking system • Assign action items
The Centrality of the Review Process • Affects every team member at some point • Impacts schedule • Impacts costs • Prevention of downstream costs • Cost of reviews • Impacts morale
Defect Removal Efficiency Low Median High Low Median High No design inspections 30% 40% 50% Formal design inspections 95% 99% 99.99% No code inspections Formal code inspections No quality assurance Formal quality assurance No formal testing Formal testing Formal design inspections 43% 60% 68% Formal code inspections 43% 57% 66% And nothing else and nothing else Formal quality assurance 32% 45% 55% Formal testing 37% 53% 60% And nothing else and nothing else Formal design inspections 70% 85% 90% Formal quality assurance 50% 65% 75% Formal code inspections Formal testing Formal design inspections 85% 97% 99% Formal code inspections 75% 87% 93% Formal code inspections Formal quality assurance Formal testing Formal testing Software Quality by Capers Jones
Relative Cost to Fix Defects by Phase 1000 500 Larger software projects 200 100 Relative cost to fix defect 50 20 Smaller software projects 10 5 2 1 Requirements Design Code Development Acceptance Operation test test Phase in which defect was fixed
Software Defect Repair Rates Non-test Defect Removal Average 2.5 Post-Release Defect Removal Average 9.57
Savings from Performing Inspections C error rates are very well documented: 13.7/KLOC (Open bracket to Close bracket) 100,000 line project – 1370 Errors Defect removal Cost and Savings per 100,000 LOC by identification activity Do this. Save this!
So let’s run out and start doing inspections! While the value of doing inspections has been proven, there is no consensus on a process.
The Politics of the Internal Review Senior Management SQA Software Lead Project Management Software Team
Appropriate Processes • Reviews have a cost • Stay lean • Meeting size (Software lead, Group lead, developer, interfaces, QA,CM) • Meeting length (a maximum of 2 hours, major issues only)* • Schedule (stay absolutely stringent) • Automate the process • No one process is right for everyone • Be careful of dogma • Not-Invented-Here • Some groups need to own the process • * The number of defects identified was increased by 33% if there was a meeting. • ``An Experiment to Assess the Cost-Benefit of Code Inspections in Large Scale Software Development", A.Porter, H.Siy, C.Toman, • and L.Votta, accepted for Third Symposium on the Foundations of Software Engineering, October 1995.
The Automation of the Review Process • Saves time and reduces frustration for the team • Helps limit costs • Archives results for future retrieval • Really its essential if you are going to take advantage • of the quantitative output (CMMI Level 4)
Contact Info Integrated Software Metrics, Inc. www.ismwv.com Mike Chapman email@example.com 304-657-7187