Why is Test Planning so Hard

Why is Test Planning so Hard PowerPoint PPT Presentation


  • 93 Views
  • Uploaded on
  • Presentation posted in: General

Download Presentation

Why is Test Planning so Hard

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


1. Why is Test Planning so Hard? Elizabeth D’Angelo, PhD CGI Group Inc.

2. 2 Confidential Agenda Background Test Planning: Three Issues Test Planning: Three Solutions Practical Tips to Help you in Test Planning Conclusions Q&A

3. 3 Confidential Reasons Why Test Planning is Difficult

4. 4 Confidential Test Planning Issue 1: Knowledge (1) Right Mix is Critical! Requirements Requirements: Implicit vs. explicit knowledge (people vs. docs) Expertise Thorough knowledge of applications and flows Breath vs. depth

5. 5 Confidential Test Planning Issue 2: Project Management (1) Test Planning is Project Management within Project Management! It is NOT optional Be prepared to articulate WHY is needed Plan – Prepare – Execute motto Take time to do proper planning “Vanilla flavour” plan is useless

6. 6 Confidential Test Planning Issue 2: Project Management (2)

7. 7 Confidential Test Planning Issue 3: The “Gruyere” paradigm How many holes are in this cheese? What is actually delivered vs. what was planned to be delivered? Are all the features truly complete?

8. 8 Confidential Test Planning Solution 1: The Expert (1) KNOW the WHAT… Industry Knowledge Application/Environment Knowledge Testing Knowledge

9. 9 Confidential Test Planning Solution 1: The Expert (2) Become the testing guru Help your team to become the testing experts Some Ideas: Formal training/certification Attend conferences But there are also cheaper ways: Lunch and Learn sessions Organize workshops with presentations Test Planning Risk-based approaches Test terminology Test case design Metrics Test tools Create a testing library

10. 10 Confidential Test Planning Solution 1: The Expert (3) Build your credibility by becoming the industry/application expert Read ALL documentation Review the requirements, design, architecture Talk to users, developers Don’t be afraid of asking questions Ensure to communicate your status properly Assume responsibility when needed

11. 11 Confidential Test Planning Solution 1: The Expert (4) And there is more… Soft Skills! Communications (defect reporting) Detective Negative thinking or “What can go wrong” Time management Discipline and Tenacity Creativity

12. 12 Confidential Test Planning Solution 1: The Planner (1) KNOW the HOW Testing scope Discuss under/over testing Who will be doing what What needs to be in place before test starts Test case design guidelines Tool usage Metrics to be collected Entry/Exit criteria …

13. 13 Confidential Test Planning Solution 1: The Planner (2) You will reap what you sow Planning is the most essential activity! PLAN, PLAN, PLAN …. You cannot prepare what you have not planned for You cannot execute what you have not prepared for!

14. 14 Confidential Test Planning Solution 1: The Planner (3) Make your Master Test Plan the Bible Document ALL your assumptions Think of ALL risks Use ALL information available to you Establish in/out scope INCLUDING Test boundaries Brainstorm and define what will be tested when! Phased approach Don’t forget Environment Test Data Access/Security

15. 15 Confidential Test Planning Solution 3: The Weather-Man (1) Look at signs to forecast quality issues Previous history (application, business function, etc.) Get to know the team Collect and store metrics ? you will need them! Clearly communicate deviations from plan based on unexpected quality issues

16. 16 Confidential Test Planning Solution 3: The Weather-Man (2) Establish and Communicate Forecast for your Plan: Failure rate SLAs for defect fixes Environment availability

17. Confidential Practical Tips to Help you in Test Planning

18. 18 Confidential ALWAYS Use Risk-Based Approach! (1) Spend your effort where is worth it,…Unless you have unlimited time/budget/resources ? Risk-based approach will allow you to test More comprehensibly what really matters First what is more important If needed, know where to cut testing NOT an isolated task, you need to socialize approach and get agreement on risk areas

19. 19 Confidential ALWAYS Use Risk-Based Approach! (2) Technical risks assessment should include: New and immature technology, Brand new system or enhancement New and difficult to understand business requirements, Complexity of business logic, Complexity of technology, Size of the project, Skill mix on the project, Number of users (including customer facing or back office considerations) historical data when appropriated (the risk is higher for modified functions that have frequently failed in the past, i.e., Do you have any historical data of defects by area?)

20. 20 Confidential ALWAYS Use Risk-Based Approach! (3) Business Impact assessment should include: Importance of features from client/customer perspective Criticality of features from the business perspective Frequency of use

21. 21 Confidential ALWAYS Use Risk-Based Approach! (4) As a rule of thumb: High Impact: Calculation Validation Legal Front end (end user) High frequency, High volume of users Medium Impact: Data changes Wrong information displayed Medium frequency and groups of users Low Impact: Display Visual appearance Used by a few users Back office

22. 22 Confidential Risk-Based Approach Process Organize test risk assessment workshop with business and technical stakeholders Identify and list requirements, business functions Evaluate technical and business impact (rate 1-3) Evaluate probability of failure (rate 1-3)

23. 23 Confidential Risk-Based Approach: Business/Requirements Map

24. 24 Confidential Cover all aspects of testing When planning… make sure to cover the overall process:

25. 25 Confidential Schedule Templates Detailing All Activities (1) Have an overall schedule template that you can easily fill with specific data

26. 26 Confidential Schedule Templates Detailing All Activities (2)

27. 27 Confidential

28. 28 Confidential Forecasting with Metrics – Failure Rate/Env Document what is the expected failure rate Existing application/delivery team: use previous data Be as granular as possible By Functional Area By Application If you don’t have data, rule of thumb is around 30% (i.e., 30% of test cases will fail during test execution) Incorporate retesting activities to schedule

29. 29 Confidential Forecasting with Metrics – Defect Age SLAs Negotiate and document SLAs for defect fixing, then measure How long defects remain in New state (i.e., not assigned?) How long defects remain in Assigned state (Are too many defects assigned to the same resource and therefore there is no progress? How long defects remain in Fixed state (i.e., before being closed by Test Team)? (Is testing team following up on defect fix verification? Are there competitive priorities (execution vs. verification)? A high number of defects in Need Info state could indicate that there are concerns with the requirements, lack of details on the defect (to reproduce).

30. 30 Confidential Conclusions DO PLAN, PLAN, PLAN …. AND THEN PLAN! Get buy-in into risk-based approach Create templates to help you be efficient Be systematic Use historical information/metrics to predict future Share your documents, status and metrics so stakeholders understand challenges Hope for the best, but plan and prepare for the worst! DON’T Work in isolation Over complicate your methodology Think you don’t need to understand domain Do ad-hoc testing (unless it is planned) Be inconsistent Keep issues, departures from assumptions to yourself Be naďve and plan for the best!

31. THANKS!! Questions??

32. 32 Confidential Contact Information

  • Login