Fusion testing
This presentation is the property of its rightful owner.
Sponsored Links
1 / 21

Fusion Testing PowerPoint PPT Presentation


  • 65 Views
  • Uploaded on
  • Presentation posted in: General

Fusion Testing. Maximizing Test Execution. By: James Tischart. About Me. Director - Product Delivery SaaS at McAfee 15+ years experience in testing and engineering Multiple certifications in Agile and Testing Passionate about Testing as an eng discipline, a science and an art

Download Presentation

Fusion Testing

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Fusion testing

Fusion Testing

Maximizing Test Execution

By: James Tischart


About me

About Me

  • Director - Product Delivery SaaS at McAfee

  • 15+ years experience in testing and engineering

  • Multiple certifications in Agile and Testing

  • Passionate about Testing as an eng discipline, a science and an art

  • Continue to challenge the status quo, test new approaches and always strive to improve

  • Support the legitimacy of the testing professional in the broader engineering world


What is fusion testing

What is Fusion Testing?

  • An occurrence that involves the production of a union

    • Organization of structured testing

    • Freedom of exploratory testing

    • Rigors of automated testing

    • Combined into one test methodology

  • To achieve

    • Maximize code execution

    • Increased test coverage

    • Reduced test artifact and documentation

    • Higher quality for users

    • Improved data for organization


Fusion

Fusion

Focus – start your day with 15 minutes of thought

Usage – how will your users work with the system

Scope – decide on the scope of everything

Initiate – just go and explore

Organize – create a plan & be ready to deviate from it

Note – keep track of your exploration to retrace steps


Fusion testing guidance

Fusion Testing Guidance

Maximize test execution with Fusion by:

  • Identify Goals for Testing

  • Choose the right mix of methodologies

  • Utilize Tests Lists to guide exploration

  • Automate the Right Things

  • Document at the Right Detail


Fusion implementation

Fusion Implementation

Guide your team through the change

Create a Change Message

Mentor your team members

Train the team in testing and engineering

Create Metrics to measure success & failure


Fusion planning

Fusion & Planning

  • Release Test Plan

    • Set the goals for the overall release

    • Plan the test environment

    • Organize the metrics and performance

    • Plan a budget, resources and a time frame

    • Plan your automation and tool set

    • Plan at a high level

  • Iterative Test Plan

    • Very effective in Agile implementations

    • Break up the testing into iterations

    • Low level details for the time frame

      • Keep to two weeks or less

      • Detail the testing to accomplish

      • Detail the performance tests

      • List what will not be tested

      • Identify risks to the plan

      • Plan what Regression Testing

      • Identify what test-lists need to be written

  • Daily Plan

    • Individually created each morning

    • Provides guidance and goals for the day

    • Details the execution plan

  • Test Lists

    • A grouping of test ideas

    • No specifics in the test list

    • Priority, variations & guide for exploration


The power of many

The Power of Many

  • Utilize different users of your system to bring fresh perspectives

  • - Support personnel- Sales staff

  • - Developers- Training

  • - User groups- Product Management

  • Help guide the testing by providing checklists, environments and goals

  • Don’t give exact details or specific steps this minimizes innovation

  • Make the event fun and you will have many people continuing to help

  • Have testing experts available to help the volunteers for problems


Identify defect traits

Identify Defect Traits

  • Next analyze Sev 1 internally found and fixed issues and determine the traits

    • Inform the developers & testers of the traits

  • Create test lists or test ideas to cover these traits

  • Repeat the process with each severity level you use

  • For new products use historical traits from similar projects

  • Analyze Severity 1 Customer reported incidents

  • Determine the trends to those defects in coding & testing

  • Constrain the review to a calendar year of defects


Fusion automation

Fusion & Automation

  • Determine your layers for automation:

  • Server

    • Databases, server processes

    • Machine interaction points

    • Process interaction with other layers

  • Middleware/API layers

  • User Interface

  • Optimize your test artifacts

    • Don’t duplicate test-cases

    • Use automation for traceability.


Test results metrics

Test Results & Metrics

Since you can’t test everything, here are some ideas of results to report:

  • # of Test Ideas Executed based on Priority

  • # of Test Cases passed/failed versus total coverage that the tests represent

  • # of Automated Tests Passed/Failed by # of executed times

  • Performance/Benchmark comparisons by build/iteration/release

  • Team Quality Satisfaction Rating – get the gut feeling of the team

  • Open defects to highlight potential issues that your customers may find

  • Go to fusiontesting.blogspot.com for more details on these metrics

There will always be interest in the test data no matter the test structure.

Q: How can you accurately assess quality when the testing combinations

exceed the particles of the universe?

A: You can’t! We need to present what was tested, what has not been tested and support the assessment used to make this prioritization.


Traceability matrices

Traceability Matrices


Implementation challenges

Implementation Challenges

  • There are challenges to implementation centralized in 5 areas:

    • Management

    • Product Management

    • Engineering Teams

    • Regulated Projects

    • Outsourcing


Challenge management

Challenge: Management

  • Identify the Challenges:

  • Reliance on historical metrics

  • Understand the current processes and practices

  • Decision-making timelines

  • Need traceability to feel confident of data

  • Respond to the Challenges:

  • Provide better metrics

  • Show how new process improves quality

  • Prove how decision can be made faster

  • Review traceability needs and support them


Challenge pmo

Challenge: PMO

  • Identify the Challenges:

  • Reliance on historical metrics

  • Struggle with the details of testing

  • Need predictability for costs and schedules

  • Require improved time to market

  • Respond to the Challenges:

  • Debunk misleading metrics

  • Show how more testing provides better data

  • Prove reduced costs with efficiency gains

  • Track lost opportunity cost on test artifacts


Challenge engineering

Challenge: Engineering

  • Identify the Challenges:

  • Takes time away from coding

  • Testing can be tedious

  • Not their specialization

  • Rely on a serial approach to testing

  • Respond to the Challenges:

  • Test first design increases new coding time

  • Automated & exploratory tests are less tedious

  • Understanding testing improves code writing

  • Less defects will be logged with up-front tests


Challenge regulation

Challenge: Regulation

  • Identify the Challenges:

  • Documentation Requirements

  • Formal or Standards Approval

  • Full traceability

  • Rigorous Automation & detailed test results

  • Respond to the Challenges:

  • Lean towards more structured testing

  • Use exploratory but document results

  • Plan for and work in shorter iterations

  • Automate more, document more


Challenge outsourcing

Challenge: Outsourcing

  • Identify the Challenges:

  • Require more instruction and detail

  • Lack expertise in the product and markets

  • Multi-cultural communication

  • Formal process & document to support work

  • Respond to the Challenges:

  • Phase in Fusion and train on exploratory

  • Provide more details in documentation

  • Implement multiple communication paths

  • Support and train new processes


Ensuring success

Ensuring Success

Implement Monitor

Adjust

Implement in stages

Measure the impact to customers (before & after)

Collect metrics for a project for test execution time before implementation

Introduce Fusion to cover about 25% of testing

Increase Fusion usage to 50% to further improve execution time

Continue to phase in until 75 to 90% of testing is Fusion

Monitor

Increases or decreases in execution time

Review documentation size closely reviewing the test artifacts

Measure the amount of direct automation

Review with the team the amount of time spent on maintenance

Continuously try to improve exploration time and increase efficiency

Adjust based on the needs of the project

One project may require more precise documentation

Another project may require less structure

Identify these requirements at the start of projects and adjust

Reduce the amount of time spent on artifact management

Inform people of the time impacts of data collection


Five keys to fusion

Five Keys to Fusion

5

5

5

  • Detail – the right level of detail at the right time

  • Planning – consistently plan, execute and adjust

  • Automate - spend time on specifics in automation

  • Report – measure what truly represents the team and customer

  • POM – get more people involved to expand the breadth of test

5

5


Conclusion

Conclusion

Rick Craig

James Bach

Thank You

My family, friends, colleagues

who have supported me!

QA&Test

Hung Nyugen

Cem Kaner


  • Login