Formal Technical Reviews. Annette Tetmeyer Fall 2009. Outline. Overview of FTR and relationship to software quality improvement History of software quality improvement Impact of quality on software products The FTR process Beyond FTR Discussion and questions. Formal Technical Review.
What is Formal Technical Review (FTR)?
Definition (Philip Johnson)
A method involving a structured encounter in which a group of technical personnel analyzes or improves the quality of the original work product as well as the quality of the method.
quality of the original work product quality of the method
requirements design coding testing
Improve the quality of the method
An inspection is ‘a visual examination of a software product to detect and identify software anomalies, including errors and deviations from standards and specifications’
A walkthrough is ‘a static analysis technique in which a designer or programmer leads members of the development team and other interested parties through a software product, and the participants ask questions and make comments about possible errors, violation of development standards, and other problems’
A review is ‘a process or meeting during which a software product is presented to project personnel, managers, users, customers, user representatives, or other interested parties for comment or approval’
Source: IEEE Std. 1028-1997
Source: Johnson, P. M. (1996). Introduction to formal technical reviews.
In reality, there is also a middle ground between informal and formal techniques
What is the impact of the annual cost of software defects in the US?
Source: NIST, The Economic Impact of Inadequate Infrastructure for Software Testing, May 2002
Why are software inspections not widely used?
From class website: sw-inspections.pdf (Parnas)
Can steps be skipped or combined?
How many people hours are typically involved?
Preparation is crucial to effective reviews
Who should not be involved and why?
Refer to sample forms distributed in class
Source: Ciolkowski, M., Laitenberger, O., & Biffl, S. (2003). Software reviews, the state of the practice. Software, IEEE, 20(6), 46-51.
Full references handout provided in class
Ackerman, A. F., Buchwald, L. S., & Lewski, F. H. (1989). Software inspections: an effective verification process. Software, IEEE, 6(3), 31-36.
Aurum, A., Petersson, H., & Wohlin, C. (2002). State-of-the-art: software inspections after 25 years. Software Testing, Verification and Reliability, 12(3), 133-154.
Boehm, B., & Basili, V. R. (2001). Top 10 list [software development]. Computer, 34(1), 135-137.
Ciolkowski, M., Laitenberger, O., & Biffl, S. (2003). Software reviews, the state of the practice. Software, IEEE, 20(6), 46-51.
D'Astous, P., Détienne, F., Visser, W., & Robillard, P. N. (2004). Changing our view on design evaluation meetings methodology: a study of software technical review meetings. Design Studies, 25(6), 625-655.
Denger, C., & Shull, F. (2007). A Practical Approach for Quality-Driven Inspections. Software, IEEE, 24(2), 79-86.
Fagan, M. E. (1976). Design and code inspections to reduce errors in program development. IBM Systems Journal, 15(3), 182-211.
Freedman, D. P., & Weinberg, G. M. (2000). Handbook of Walkthroughs, Inspections, and Technical Reviews: Evaluating Programs, Projects, and Products: Dorset House Publishing Co., Inc.
IEEE Standard for Software Reviews and Audits (2008). IEEE STD 1028-2008, 1-52.
Johnson, P. M. (1996). Introduction to formal technical reviews, from http://www.ccs.neu.edu/home/lieber/com3205/f02/lectures/Reviews.ppt
Johnson, P. M. (1998). Reengineering inspection. Commun. ACM, 41(2), 49-52.
Johnson, P. M. (2001), You can’t even ask them to push a button: Toward ubiquitous, developer-centric, empirical software engineering, Retrieved from http://www.itrd.gov/subcommittee/sdp/vanderbilt/position_papers/philip_johnson_you_cant_even_ask.pdf, Accessed on October 7, 2009.
Johnson, P. M., & Tjahjono, D. (1998). Does every inspection really need a meeting? Empirical Software Engineering, 3(1), 9-35.
Neville-Neil, G. V. (2009). Kode Vicious<br />Kode reviews 101. Commun. ACM, 52(10), 28-29.
NIST (May 2002). The economic impact of inadequate infrastructure for software testing. Retrieved October 8, 2009. from http://www.nist.gov/public_affairs/releases/n02-10.htm.
Parnas, D. L., & Weiss, D. M. (1987). Active design reviews: principles and practices. J. Syst. Softw., 7(4), 259-265.
Porter, A., Siy, H., & Votta, L. (1995). A review of software inspections: University of Maryland at College Park.
Porter, A. A., & Johnson, P. M. (1997). Assessing software review meetings: results of a comparative analysis of two experimental studies. Software Engineering, IEEE Transactions on, 23(3), 129-145.
Rombach, D., Ciolkowski, M., Jeffery, R., Laitenberger, O., McGarry, F., & Shull, F. (2008). Impact of research on practice in the field of inspections, reviews and walkthroughs: learning from successful industrial uses. SIGSOFT Softw. Eng. Notes, 33(6), 26-35.
Votta, L. G. (1993). Does every inspection need a meeting? SIGSOFT Softw. Eng. Notes, 18(5), 107-114.
Gilb: Extreme Inspection