1 / 11

What we will do / Structure

Getting Business Value out of (Test) Metrics Richard Terry (UK) richard.terry@sogeti.com Rob Baarda (NL). What we will do / Structure. Subjects: Test Metric goals / stakeholders Which metrics; some demo Process around metrics How to apply metrics / pittfalls Wrap-up

lacy-maddox
Download Presentation

What we will do / Structure

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Getting Business Value out of (Test) MetricsRichard Terry (UK)richard.terry@sogeti.comRob Baarda(NL)

  2. What we will do / Structure • Subjects: • Test Metric goals / stakeholders • Which metrics; some demo • Process around metrics • How to apply metrics / pittfalls • Wrap-up • For each subject: intro, discussion • Discussion • hand raise • hand raise with comments • open

  3. Metric Goals / Stakeholders • Hand raise only • Who is using project metrics? • Who is using test metrics? • Who would like to use test metrics? • Some opinions please! • Goals / Stakeholders for test metrics? OPEN! • Your votes please

  4. Which test metrics? Size test basis Test Process # test cases Specifying test cases / scripts # defects in test basis Test cases/ scripts Production Size test object # defects in test object # defects in production # repair rounds For each process: # hours effort lead time # = number of Test basis Test object Test Execution Defects Repair

  5. Deductible metrics • Effort = Size /# hours • Productivity = # test cases / # hours • Efficiency = # defects / (# hours or # test cases) • Specification • Test execution • Retest of repaired defects • DDP Defect Detection Percentage (Europe)DRE Defect Removal Efficiency (USA) • Defect injection rate for rework • Damage prevented? • More metrics: • Availability of test environments • OPEN

  6. Dutch test metrics experiences • Dutch initiative to gather test metrics • Parties involved • NESMANetherlands Software Metrics Association • Testnet, including SogetiDutch Testing community • LaQuSOLaboratory for Software QualityUniversities Eindhoven & Nijmegen Some slides with data from the database will follow in the presentation, not in the hand-out, due to a lack of statistical evidence

  7. Processes around metrics • Collection in a project • Embedded in daily work • Weekly summarisation • Sanity checks • Cost: about 2% project budget • Distribution • For a benchmark on the level of: • Project releases • Organisation • Country • International: ISBSGInternational Software Benchmarking Standards Group

  8. To what extent? • OPEN Discussion about • Accuracy of definitions • Number of types of defects • Is a batch test case the same as an online test case? • Only testing of functionality or also security, performance, usability • How to include regression testing? • Predictive valueaverage (mean), median, standard deviation, correlations with?Prediction model needed?

  9. To apply test metrics • Project • Estimating Testing New development • Estimating Testing Maintenance release • Controlling the test project • Benchmark • Measure the effectiveness of the testing, Improve inside / outside the test process? • PITTFALLS?

  10. Make a judgement based with metrics? • Of person production/ quality of work • OPEN Discussion • Some Dutch effects • Of Service Level Agreement • What to measure and how to judge

  11. Wrap-up • Business value out of metricsHand raise • Who would like to use test metrics? • Changed opinions • Comments please (-  + ,+  -) • Reading: TMap Next • Chapter 13 Goal-Question-Metrics for test • Chapter 11 for data • End of 2008: www.isbsg.orgParticipate? Leave your business card

More Related