Improving Testbed Identification,
Download
1 / 41

Improving Testbed Identification, Creation, Check-out and manipulation. - PowerPoint PPT Presentation


  • 95 Views
  • Uploaded on

Improving Testbed Identification, Creation, Check-out and manipulation. Global Initiatives Team – June 12, 2006. Topics. Project Team Team Charter - Business Case, Problem Statement, Scope DMAIC Define – SIPOC and Critical to Quality (CTQ) Measure – Data Collected Appendix.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Improving Testbed Identification, Creation, Check-out and manipulation. ' - ellery


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Improving Testbed Identification,

Creation, Check-out and manipulation.

Global Initiatives Team – June 12, 2006


Topics
Topics

  • Project Team

  • Team Charter - Business Case, Problem Statement, Scope

  • DMAIC

    • Define – SIPOC and Critical to Quality (CTQ)

    • Measure – Data Collected

  • Appendix


Project team
Project Team

Add Hoc Team

HTS: Emilo Sanfalice

TBC (TBD) TBC (TBD

ChampionGreg Geppert

Process Owner Kim Simmons

LeadChris Columbia

Core Team

PPD: Chris Whyde

Mary Gerred

Alla Crowder

Lata Walsh

Tony Rumilla

Toby Brooks

Scott Brubaker

Ci Hye Kim

P&M: Pat Fabian

Dave Jarvis

BPG Richard Mitchum


Team charter business case
Team Charter - Business Case

  • It is important that the testbeds for the release be developed in the shortest time frame and at the lowest cost possible. the customer wants and needs prior to project scheduling, task planning, and resource allocation.

  • Assumptions, misinterpretations, changes in customer wants and needs after scheduling, result in increased costs (rework), risks (schedule and quality), and negative affects on morale (unscheduled overtime, perceived wasted effort, shifting priorities and violation of the function-quality-schedule relationship).


Team charter problem statement
Team Charter- Problem Statement

  • With the addition of MortgageServ testing in November to the quarterly release, the need to identify, select and protect test cases loaded to the test region for use as testbeds has become a critical piece of the preparation.

  • Since August 2005 excluding STAR WARS, 27 projects required a total of 82 PCR’s, of which 39 (47%) were associated with business requirements and 18 (22%) resulted in functionality that was not delivered with the release. The STAR WARS project included 94 PCR’s of which 27% were associated with business requirements.

  • For projects delivered in 2006, more than half of the HTS customer survey responses indicate that the time planned to complete the BRD was not appropriate (56%) and that the BRD did not contain the necessary detail to develop the system according to what the customer (end user) expected to see in the final product. (61%).


  • Over 1800 testbeds were created for the May release. The test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation.

  • The CS and Coll Cross functional teams actually stopped testing for periods of time to assist GL and Ins Svc. In testbed preparation. Over 40% of the testbeds provided to GL had to be reworked due to lack of detail in the initial request.Over 40% of the testbed request from Ins Svc required additional investigation by the QD team before that could be completed.


Team charter project scope
Team Charter - Project Scope test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation.

In

Out

Effectively leveraging analysis and requirements documentation from IR and the initiation phase

All aspects of the Requirements and analysis phase including Business, PMO, Compliance, QD and HTS practices and participation as it relates to the BRD development

PCR and Requirements quality measures

Initiation Phase and IR development

PCR creation and change management after BRD

Functional and technical specifications

Test execution practices

Testing defect management

Measuring employee satisfaction with the BRD process


Dmaic
DMAIC test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation.

Define

March 23

(End)

Measure

April 19

(End)

Analyze

May 5

(End)

May 15

(Solutions -End)

October 31

(Implementation -End)

Improve

November 1

(Begin)

Control

September 1


D maic define sipoc
D test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. MAIC – Define (SIPOC)


D maic define critical to quality
D test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. MAIC - Define (Critical to Quality)

Level 2 Requirement

Level 1 Requirement

Output/Need

Accurate- includes end user needs

Quality

Scope – clear definition of desired vs. must

Easy to Understand

Business process flow-what it will look like

BRD Draft

Time allocation for research & analysis

Consulting- right amount of time for analysis

Creation – right amount of time to create

Collaboration – decisions by consensus

Key resources

Commitment and participation (available and active)

The right HTS SME’s are involved

Knowledge – systems and business – assumptions validated

Primary Customer: HTS


D maic define critical to quality1
D test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. MAIC - Define (Critical to Quality)

Level 2 Requirement

Level 1 Requirement

Output/Need

Compliant –All compliance requirements included

Quality

Completeness – nothing missing

Accurate – without errors

Thoroughness- no impact to other areas overlooked

BRD Draft

Easy to Understand

Visual – flow charts at right level of detail

Plain English – written at right level of language

Concise – No irrelevant information

Efficiency

Turn around time

Cost – no re-work of same analysis or reviews

Primary Customer: Business

Cost – no HTS scope or cost changes


D m aic measure what our customers told us hts
D test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. MAIC - Measure What our Customers Told Us (HTS)

More than 61% are not satisfied

The BRD provided HTS all details needed to develop the system according to how the end user expected to see it in the final product

More than 55% are not satisfied

The correct amount of time was planned and used for the BRD development avoiding follow-up meetings and requirements changes after BRD sign off.


D m aic measure what our customers told us qd
D test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. MAIC - Measure What our Customers Told Us (QD)

All business requirements, including compliance and legal requirements have been included in the BRD avoiding additional requirements gathering meetings and changes to the requirements after final sign-off.

Business tells us that they have higher levels of satisfaction than QD

After the BRD sign-off there were no HTS scope or cost increases due to missing or unclear requirements in the BRD.


D m aic measure pcr analysis
D test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. MAIC - Measure PCR Analysis

Almost 50% of PCR’s on projects delivered in 2006 were associated with BRD quality

The larger the project the more likely we see PCR’s


D m aic measure brd risk and milestone performance
D test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. MAIC - Measure BRD Risk and Milestone Performance

Most of the risk is associated with larger sized projects, however it does not appear to show any significant correlation project size

Analysis: The PPD database contains 127 projects of which 20 were in yellow and/or red status (16%) prior to BRD completion.

No Correlation To Project Size


D m aic measure brd revisions and ir inputs
D test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. MAIC - Measure BRD Revisions and IR Inputs

A correlation was not found between use of an IR and number of BRD revisions

We have seen 35% improvement in use of IR’s


Dm a ic analyze y f x
DM test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. AIC – Analyze Y = f(x)

Micro Problem Statement 1Y= Why do we not have the correct SME and decision makers in the BRD process (lack of proper knowledge and skills?)X= Business, QD, and HTS expectations for the BRD have not been developed or approved (no common understanding of what is expected to be delivered)

X= No established process to identify SME's(Tasks and timeframes, training, reference materials, Communication plan, roles and responsibilities)

Micro Problem Statement 2Y= Why do we have lack of commitment from required SME's and decision makers in the BRD process?X = Lack of a planning process with established roles and responsibilities to define and validate the WBS and BRD and establish a accurate resource and time estimate

X = SME resource availability is not validated

Micro Problem Statement 3

Y = Why do 38% of the HTS customers and 18% of the QD customers tell us that they are not satisfied with how easy the BRD is to understand while only 4% of the business is unsatisfied?X = Business, QD, and HTS expectations for the BRD have not been developed or

approved


Dm a ic analyze y f x1
DM test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. AIC – AnalyzeY = f(x)

Micro Problem Statement 4Y = Why are 56% of HTS customers unsatisfied with scheduled time for BRD consulting and creation while only 4% of the business is unsatisfied?X = Measurements are not available to identify the cost (budget, cycle time, quality, morale) associated with failure to follow the scheduling processMeasurements needed to move from a culture that relies on a History of making it happen and MBO goals tied to unplanned dependencies on new functionalityX = Lack of a planning process with established roles and responsibilities to define and validate the WBS and BRD and establish a accurate resource and time estimateX = SME resource availability is not validated

Micro Problem Statement 5

Y = Why do 61% of the HTS customer survey responses and 18% of the QD responses tell us that the BRD does not meet their expectations for necessary detail, while only 9% of the business responses feel we have not met their expectation? X =Business, QD, and HTS expectations for the BRD have not been developed or approvedX= Lack of measures to show impact of BRD quality


Dma i c improve process solution 1 establish brd standards
DMA test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. IC – ImproveProcessSolution 1:Establish BRD Standards

  • Define the level of detail and format expected from each customer group (HTS, QD and Business).

  • Project criteria used to allow expectations to be defined for projects which include new product and product changes as well as those which have minimum impact on workflow.

  • Implement a process to “tailor” and clearly establish and validate the BRD expectations at project kick off.


Dma i c improve process solution 2 sme validation process
DMA test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. IC – ImproveProcess Solution 2:SME Validation Process

  • Skills and time necessary to complete BRD tasks clearly defined and validated with SME and SME management. Confirmation of resource availability at kick off.

  • Process established to update and maintain departmental resource inventory and capacity across projects.

  • Management and control process which includes re-validation, reporting, and escalation related resource commitments.

  • Establish a formalized process to track missed requirements to disengaged resources (impact and repercussions associated with commitment failures)


Dma i c improve process solution 3 brd planning process
DMA test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. IC – Improve ProcessSolution 3:BRD Planning Process

  • Establish a framework for consistent planning all BRD tasks. Includes guidelines, templates and task definitions.

  • Planning framework uses project criteria (size, complexity, user group impact etc.) to establish standard lists of tasks and the expected time for completion.

  • Common and repeated task definitions are defined with objectives, roles and responsibilities and entrance criteria.

  • The appropriate level of iterative development is planned.


Dma i c improve measurement solution brd quality scorecard
DMA test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. IC – ImproveMeasurement Solution:BRD Quality Scorecard

  • Utilize Customer Satisfaction Survey questions to establish baseline measures of satisfaction with Business Requirements CTQ’s for HTS, QD and Business.

  • Create a BRD traceability quality measure derived from an objective assessment of gaps between the Initial Request, Business Requirements and Functional Specifications.

  • Establish a process for using clear operational definitions of PCR root cause to establish measures of “good” and “bad” PCRs.


Dma i c improve leveraging best practices
DMA test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. IC – Improve Leveraging Best Practices

Process Improvement

Best Practices


Dma i c improve goals
DMA test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. IC – Improve Goals

  • 25% reduction from baseline in unscheduled PCRs associated with business requirements defects (gaps, scope, inaccuracies, misinterpretations).

  • Achieve an increase of xx% in BRD quality

    (as measured by the functional specification

    traceability measurement)

  • Improve Customer Satisfaction Scores for HTS, QD and the Business:

    • Achieve a release level aggregate HTS customer satisfaction rating of 3 that the BRD provided all the necessary details to develop the system according to how the end user expected to see the final product and that the correct amount of time was used to develop the BRD (12% improvement)

    • Achieve a release level aggregate QD customer satisfaction rating of 2.5 for BRD quality (completeness, accuracy, consensus and ease of understanding) and BRD efficiency (turn around time and cost of re-work) (10% improvement)

    • Achieve a release level aggregate Business customer satisfaction rating of 2.1 for BRD quality (completeness, accuracy, consensus and ease of understanding) and BRD efficiency (turn around time and cost of re-work) (17 % improvement)


Dma i c improve implementation timelines
DMA test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. IC – Improve Implementation Timelines

* See appendix for solution step details


Dmai c control
DMAI test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. C – Control


Dmai c control brd quality dashboard
DMAI test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. C – Control BRD Quality Dashboard


Appendix
Appendix test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation.

  • Process Flow Chart

  • Data Collection Plan

  • Fish Bone Root Cause

  • Scoring Solutions

  • Create BRD Standards Solution Steps

  • SME Validation Solution Steps

  • Project Plan


Appendix dmaic define process flow
Appendix test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. DMAIC- Define (Process Flow)


Appendix dmaic define process flow1
Appendix test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. DMAIC- Define (Process Flow)


Appendix dmaic measure data collection plan quality
Appendix test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. DMAIC - Measure Data Collection Plan (Quality)


Appendix dmaic measure data collection plan easy to understand
Appendix test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. DMAIC - Measure Data Collection Plan (Easy to Understand)


Appendix dmaic measure data collection plan efficiency
Appendix test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. DMAIC - Measure Data Collection Plan (Efficiency)


Appendix dmaic measure data collection plan best practices
Appendix test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. DMAIC - Measure Data Collection Plan (Best Practices)


Appendix dmaic analysis fishbone root cause
Appendix test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. DMAIC - Analysis (Fishbone Root Cause)


Appendix dmaic analysis fishbone root cause1
Appendix test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. DMAIC - Analysis (Fishbone Root Cause)


Appendix dmaic improve scoring solutions must want criteria
Appendix test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. DMAIC - Improve Scoring Solutions –Must/Want Criteria


Appendix dmaic improve scoring solutions must want criteria1
Appendix test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. DMAIC - Improve Scoring Solutions –Must/Want Criteria

Smaller numbers have greater importance


Appendix dmaic improve create brd standards steps
Appendix test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. DMAIC - Improve Create BRD Standards (Steps)

Step 1

  • Define the level of detail and format expected for each customer group (HTS, QD, and Business.

  • Implement a BRD template which meets these expectations within a single BRD.

  • Project criteria will be established to allow expectations to be defined for projects which include new product and product changes as well as those which have minimum impact on workflow.

    Step 2

  • Implement a process to tailor the BRD expectations at project kick off

  • Create a process that uses project criteria to “tailor” and clearly establish the BRD expectations at project kick off.

  • It involves establishing a communication and agreement process with appropriate documentation related to expectations to ensure that the process remains both flexible and consistent..


Appendix dmaic improve sme validation steps
Appendix test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. DMAIC - Improve SME Validation (Steps)

  • Step 1

    • Prior to kick off meeting implement a process to identify the proper resource needs and gain commitment to this understanding

  • Step 2

    • Create a process to confirm resource availability at kickoff (Both SME manager and the SME) Note: This confirmation process will expand to include the improved estimates developed from implementation of Step 4.

  • Step 3

    • Create a process to update and maintain an inventory of resources by department and departmental resource capacity across projects

  • Step 4

    • Prior to Kick off implement a process to include pre-kick off estimate of time required and when, define roles and responsibilities before assignment of SME, clear definition of skills needed and validation of correct SME

  • Step 5

    • Create a management and control process which includes re-validation, communication, and escalation related to resource commitments

  • Step 6

    • Establish repercussions for lack of commitment- formalized process to track missed requirements to disengaged resources


Appendix dmaic improve project plan
Appendix test Coordinators on the GIT team spent as much as 3 hour out of each day during testing was focused on testbed research, selection and manipulation. DMAIC - Improve Project Plan


ad