readiness index is your application ready for production l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Readiness Index – Is your application ready for Production? PowerPoint Presentation
Download Presentation
Readiness Index – Is your application ready for Production?

Loading in 2 Seconds...

play fullscreen
1 / 26

Readiness Index – Is your application ready for Production? - PowerPoint PPT Presentation


  • 96 Views
  • Uploaded on

Readiness Index – Is your application ready for Production?. Jeff Tatelman SQuAD October 2008. Agenda. Building a Metrics Program Typical Test Organizations Readiness for Change Metrics and Productivity Production Readiness Introduction & Objective Four Key Metrics Reporting.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Readiness Index – Is your application ready for Production?' - orien


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
readiness index is your application ready for production

Readiness Index – Is your application ready for Production?

Jeff Tatelman

SQuAD October 2008

agenda
Agenda
  • Building a Metrics Program
    • Typical Test Organizations
    • Readiness for Change
    • Metrics and Productivity
  • Production Readiness
    • Introduction & Objective
    • Four Key Metrics
    • Reporting
typical test organizations
Typical Test Organizations
  • No metrics collected
  • No collection of requirements
  • Limited formal reporting on project status
  • Some central test repositories but not dash boarding results
  • Minimal effort towards process improvement
dimensions of readiness
Dimensions Of Readiness
  • Motivation
  • Investment
  • Skills
  • Education
  • Culture
  • Support Staff
  • Aids/Maturity
  • Process Maturity
getting started
Getting Started
  • Set Measurement Objectives
  • Select Measures
  • Develop Measurement Program
  • How to Use the Results
set measurement objectives
Set Measurement Objectives
  • Determine Approach
    • Which measures will be included as a priority
    • Focus on a few to start with to get ‘quick wins’
  • Determine Scope
    • One application?
    • One project?
  • Determine Method
    • Identify best method to track and analyze agreed metrics
  • Determine Timescale
    • Full project lifecycle?
    • Time boxed approach, eg. 1, 3 or 6 months
  • Determine Deliverables
    • Report, audience, next steps
metrics and productivity
Metrics and Productivity
  • Strategic
      • Is it to increase customer satisfaction?
      • Is it to achieve CMM level 2?
      • Is it to achieve industry standards?
  • Tactical
      • Is it to improve vendor delivery?
      • Is it to increase timeliness of system delivery?
  • Operational
      • Is it to reduce defects found in testing?
      • Is it to improve requirements definition?
measuring unit examples
Measuring Unit Examples
  • Strategic
    • Quality
    • Customer satisfaction
    • Timeliness (delivery)
    • Personnel
    • Industry benchmarking
  • Tactical
    • Estimating and planning
    • System quality
    • System delivery cost/time
    • Budget
    • Productivity
    • Cost of quality
  • Operational
    • Schedule tracking
    • Effort tracking
    • Defects
    • Problem resolution
    • System availability
develop measurement program
Develop Measurement Program
  • Establish Specific Measurement Objectives
  • Define a Critical Metrics Set
  • Select Measures to Support the Metrics Set
  • Put Collection Mechanisms into Place
  • Determine Timing of Data Collection
  • Establish Mechanism for EvaluatingResults
  • Communicate Results
  • Establish Process for Future Planning Based on Metrics
how to use the results
How To Use The Results
  • Conduct root cause analysis of the data
  • Identify areas which would have the biggest impact and look to how to improve
  • Implement procedures to continually improve the processes
  • Revisit original list and determine what to address next
key points to remember
Key Points to Remember
  • Don’t try to measure too much
  • Understand the goals of your team before you determine what to measure
  • Determine which metrics support these goals
  • Don’t let your metrics define the behavior of the team
    • Change or choose a set of relative metrics that can not be manipulated
  • Monitor and identify trends , define areas for improvement

Allan Page Measurement that Matter, Better Software October 2005

production readiness

Production Readiness

Based On Mike Ennis’ Presentation at

Star West 2006

how do you know when a product is ready to ship
How do you know when a product is ready to ship?
  • Quality Metrics
  • Customer Commitment
  • Release Dates Are Preset
  • Indefinite Testing
  • Adequate Test Coverage
  • Time & Resources
  • Release Criteria
release criteria
Release Criteria
  • No open critical or high defects
  • Minimal number of medium & low defects that have been approved by Senior Management
  • Product is able to run for 72 consecutive hours
  • No open installation or configuration issues
  • All pre-defined performance criteria has been met
determining what to measure
Determining What to Measure
  • Test Case Execution Percentage
    • The percentage of tests that have been attempted during the test cycle
  • Test Case Success Percentage
    • The percentage of tests that have passed during the test cycle
  • Number of Unresolved Defects
    • The number of open defects that are currently logged against the product
  • Defect Arrival Rate
    • The number of defects found in a given day, week or build
readiness metrics objective
Readiness Metrics Objective
  • Provide management an overall picture to assess if the project is ready to be placed in production.
  • Evaluate an application go / no-go production readiness status by
    • Tracking test case execution and defect metrics over time
    • Calculating the Production Readiness Index based on the above metrics
setting range for each metric
Setting Range For Each Metric
  • Test Case Execution Percentage
    • 10 = 100%
    • 9 = 90-99%
    • 8 = 80-89%
    • 7 = 70-79%
    • … etc
  • Defect Arrival Rate This Week
    • 10 = 0
    • 9 = 1-2
    • 8 = 3-4
    • … etc
creating and analyzing readiness example
Creating and Analyzing Readiness Example

RATING RANGES

Defects Arrival Rate 9 10 = 0, 9 = 1-2, 8 = 3-4, 7 = 4-5, 6 = 5-6, 5=6-7, 4=…

Total Open Defects 9 10 = 0-5, 9 = 5-10, 8 = 11-15, 7 = 16-20, 6 = 20-25 …

Test Success % 7 10 = 100%, 9 = 99-97%, 8 = 96-94%, 7 = 93-90%...

Test Completion % 6 10 = 100-95%, 9 = 94-90%, 8 = 89-85%, 7 = 84-80%..

TOTAL RATING 31 Green = 35-40, Yellow = 29-34 Red < 28

final thoughts
Final Thoughts
  • Managing the risks
    • Understand the relationships between the metrics
    • Learn to anticipate and minimize the risks before they happen
    • Always know the information behind the data
  • Are you ready to release?
    • Redefine your Release Criteria using the individual/overall rating scale
    • Use colors for presentation & effectiveness
    • Let the data speak for itself
key attributes of a good measure
Key Attributes Of a Good Measure
  • Usefulness
    • Does it tell us what’s happening, wrong or need to know?
  • Trustworthiness
    • Does not provide false indications
  • Timeliness
    • Does it indicate the health of a system...advanced warning?
  • Simplicity
    • Too difficult to read, it will get abandoned
  • Accessibility
    • Must be visible, easy to get to
questions
Questions

jeffcolorado@juno.com