Testistanbul 2013 conference future of testing new techniques and methodologies
This presentation is the property of its rightful owner.
Sponsored Links
1 / 43

TestIstanbul 2013 Conference “Future Of Testing: New Techniques and Methodologies” PowerPoint PPT Presentation


  • 87 Views
  • Uploaded on
  • Presentation posted in: General

TestIstanbul 2013 Conference “Future Of Testing: New Techniques and Methodologies”. The Future Of Test Management “Expect The Unexpected” Gerlof Hoekstra. Introduction Who am I, the world I come from. Gerlof Hoekstra Started as software engineer Early 90’s: Testing !

Download Presentation

TestIstanbul 2013 Conference “Future Of Testing: New Techniques and Methodologies”

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Testistanbul 2013 conference future of testing new techniques and methodologies

TestIstanbul 2013 Conference“Future Of Testing: New Techniques and Methodologies”

The Future Of Test Management

“Expect The Unexpected”

Gerlof Hoekstra


Introduction who am i the world i come from

Introduction Who am I, the world I come from

  • Gerlof Hoekstra

  • Started as software engineer

  • Early 90’s: Testing !

  • Current job: Test consultant/test manager/solution consultant at Atos NL

  • From software testing to large scale business process integration tests

    • Complex application chains

    • Many stakeholders

    • Many software suppliers


Intended audience

Intended audience

  • Do you sometimes

    • Feel that your message is not heard?

    • Feel that nobody cares about your test plans and test cases?

  • Does your customer

    • Complain that testing takes too long, asks you:

      • ‘When you’re done?’, ‘Can’t you go faster?’, ‘You are delaying the project’

    • Ignore your good advice and still go live with a flawed system?

  • Do you sometimes feel frustrated because

    • You only get half the time that you require for a proper test

    • Many of the defects you found are still not solved?

  • Do you worry

    • If your job still exists in a few years, when all projects are ‘Agile’?


Presentation subject

Presentation subject

  • Being successful as a test manager often is very much dependent of the way you behave, set expectations, use words & symbols etc.

  • Very often (a combination of) subtle things

  • Some test managers I met and saw at work

    • are worried about the future

    • have been frustrated, burned out, replaced (yes, it can be a challenging job)

      WHICH QUALITIES CAN HELP TESTMANAGERS TO DELIVER MORE VALUE TO THEIR CUSTOMERS

      AND IN THE MEANTIME HAVE A MORE FUN JOB AND BE PREPARED FOR THE FUTURE


Agenda

Agenda

Introduction

Mission

Test planning & specification

Showing test results

The future of test management

Closure


The test manager s mission

The Test Manager’s Mission

How the first 20 minutes of your (to be) assignment can be crucial


Job interview 1

Job Interview 1

‘…. write a test plan, make sure the acceptance criteria are SMART, make test cases, have all requirements covered, get the test cases reviewed and formally accepted, execute the test cases, have blocking defects get solved, perform re-tests, regression testing, deliver test report and provide a go/no go advice’.


Job interview 2

Job Interview 2

‘I expect you, the future test manager, to assure the quality and decide whether the system can go live, how will you do that?’

‘What the …?!??’

‘I don’t…….’


My mission where i stand for and not

My mission: Where I stand for (and not)

  • I am here to serve, to assist the stakeholders

  • My job is to provide USEFUL INFORMATION

    • that the stakeholders need

    • to make their own informed decisions

  • I am the weather man !

  • I am NOT the quality police, nor the judge

  • I do NOT ensure product quality either


Then what do i ensure

Then what DO I ‘ensure’ ?

  • My observations are

    • Objective, facts, no opinions

    • Made by skilled & trained investigators

  • You will receive forecasts that you will learn to appreciate and are

    • Pretty reliable

    • Early in time (!)

  • The information you will get from me is very clear, in terms that the stakeholders understand, no explaining or advice needed, enables them to make their own decisions!

  • The information gathering process is efficient

I PROMISE YOU, IN CASE YOUR PROJECT IS HEADING INTO THE WRONG DIRECTION, I WILL BE THE FIRST TO TELL YOU, LOUD AND CLEAR !


Two basic values

Two ‘basic values’

  • ‘Expect The Unexpected’

    • Projects are subject of constant change and movement

    • One cannot possibly plan all activities ahead

    • Test strategy should be able to cope with change

  • Management style

    • Most testers are highly skilled and trained professionals

    • Test manager should treat them appropriate

    • Leave room for personal initiative and creativity


Agenda1

Agenda

Introduction

Mission

Test planning & specification

Showing test results

The future of test management

Closure


Discussion with an old school test manager

Discussion with an ‘old school test manager’

  • How to we know your test set is complete?

  • Your test cases are not very detailed.

  • You did not define in the test plan the criteria when a defect is considered blocking.

  • You do not have certified testers in your team

  • You did not define acceptance criteria (coverage%, #test cases passed, #open defects)

  • You should complete the system test before you start the acceptance test

  • The most important deliverable (go/no go advice) is missing in your plan

  • ---

  • ---

Review test plan v0.5


Testistanbul 2013 conference future of testing new techniques and methodologies

Let me introduce you to ‘The Doctor’You don’t have to be a control freak to get missions accomplished


How to prepare for a football match

How to prepare for a football match?


Our first holiday abroad

Our first holiday abroad


Detailed preparation versus flexibility

Detailed preparation versus Flexibility

  • Have a simple but clear plan, not too detailed

  • Have attention to details by the time when really needed, be prepared !

  • Prepare / train your team so that they are able to respond when the Unexpected happens


Writing a test plan the theory

Writing a Test Plan – The Theory

The creation of the test plan involves the following activities:

  • Establishing the assignment

  • Understanding the assignment

  • Determining the test basis

  • Analysing the product risks

  • Determining the test strategy

  • Estimating the effort

  • Determining the planning

  • Allocating test units and test techniques

  • Defining the test products

  • Defining the organisation

  • Defining the infrastructure

  • Organising the management

  • Determining the test project risks and countermeasures

  • Feedbackand consolidation of the plan    


Writing a test plan the theory1

Writing a Test Plan – The Theory

The creation of the test plan involves the following activities:

  • Establishing the assignment

  • Understanding the assignment

  • Determining the test basis

  • Analysing the product risks

  • Determining the test strategy

  • Estimating the effort

  • Determining the planning

  • Allocating test units and test techniques

  • Defining the test products

  • Defining the organisation

  • Defining the infrastructure

  • Organising the management

  • Determining the test project risks and countermeasures

  • Feedbackand consolidation of the plan    

Now look at this !?!


What i prefer instant continuous feedback and collaboration

What I preferInstant, continuous feedback and collaboration


Test plan example

Test Plan Example


Some additional thoughts

Some additional thoughts

  • You can not write a test plan on your own, hiding in your office

  • Received no review comments? The reviewer did not read/understand the plan or is not interested. Don’t consider the plan done!

  • Strip to the bone, minimize #words, avoid meaningless phrases, use images & tables

  • A test plan is NOT a contract!

  • Expect the plan to change


Coverage maps a great way to plan your test

Coverage MapsA great way to plan your test

Tip: you can use James Bach’s San Francisco DePOT acronym


Test specification

Test specification

  • How many test cases & scripts do we really need?

  • How detailed should they be?

  • Scripted vs Exploratory testing?

IT IS ALL ABOUT BALANCE

My observation:

  • Many test educations (and as a result, many certified testers) heavily over-rate the value of specifying a lot of detailed test scripts

  • Often, you can get away with less test scripts than you might think


Testistanbul 2013 conference future of testing new techniques and methodologies

Example, a limited number of formal test cases, based on a decision table , completed with test charters


Agenda2

Agenda

Introduction

Mission

Test planning & specification

Showing test results

The future of test management

Closure


A good test report

A good Test Report

  • Has useful information for the stakeholders

  • Is easy to understand for the stakeholders

  • Does not need any go/no go advice

  • Is brief and visual

  • Includes a forecast

  • Makes clear what you have (not) covered


Testistanbul 2013 conference future of testing new techniques and methodologies

Be VERY careful with coverage metricsTesting is investigation in an infinite space: numbers like 80% could easily be misunderstood


Testistanbul 2013 conference future of testing new techniques and methodologies

Be VERY careful with coverage metricsOnly testers understand these numbers… (do we?)

‘We have reached 91.7% multiple condition coverage’

Do our stakeholders know what this means for them?

Be honest, do you know? I’m not sure if I do…


Old school test reporting

Old school test reporting

  • We covered 95% of the area

  • We found 60 mines

  • We disarmed 58

  • The 2 mines we did not disarm are here

  • These are MSM MKII mines containing

    4Kg of Composition B explosives

    IS THIS USEFUL INFORMATION ?

    WOULD YOU APPRECIATE MY GO/NO GO ADVICE ?


Alternative use maps to show coverage test results

Alternative: Use maps to show coverage & test results


Alternative use maps to show coverage test results1

Alternative: Use maps to show coverage & test results


What is wrong with the good old traffic lights

What is wrong with the good old traffic lights?

  • Do not encourage to take very much responsibility

  • Don’t think, just obey the law !!

  • Is not how I want to communicate my test results

  • Remember: it’s not the test manager who decides!

  • Weather symbols send a better message

  • This is likely to happen

  • You have been informed (warned?) and now it’s up to you to make a decision

  • B.t.w.: a weather forecast is never 100% reliable


Example

Example


Longer term forecasting testing in the early project phases and i mean testing not just reviewing

Longer term forecastingTesting in the Early Project Phases (and I mean testing, not just reviewing)

Getting your message heard

  • Don’t complain, just report the facts

  • Friendly ask some questions (‘Are you OK with this?’)

  • And the best: let stakeholders experience the problem  no further comments needed


Agenda3

Agenda

Introduction

Mission

Test planning & specification

Showing test results

The future of test management

Closure


Trend 1 changing software development paradigms

Trend 1: Changing Software Development Paradigms

  • Agile

  • Continuous deployment

  • Smaller iterations

  • Test Driven Development

  • Quality is not ‘owned’ by the testers (anymore)


Trend 2 systems integration

Trend 2: Systems Integration

  • Chains, ever increasing in scope and complexity

  • Many software suppliers involved

  • COTS software, combined with bespoke software

  • Multi-stakeholder

  • Cross-company integration

  • Distributed data

  • Across the globe, 24/7 business processes

    ORGANIZING AN END-TO-END TEST CERTAINLY IS NOT A TRIVIAL TASK


The future

The future


A shift in qualities

A shift in qualities

To:

From:

  • Ensure quality

  • Go/No Go decision

  • Find defects

  • Number of test cases

  • Number of defects

  • Enforce, control

  • Proof

  • Follow rules & standards

  • Strictly follow detailed plans

  • Certification

  • Requirements freeze

  • Acceptance criteria

  • Quality assistance

  • Deep insight to stakeholders

  • Prevent failures

  • Coverage map

  • Business consequences

  • Serve, help, facilitate

  • Forecast

  • Investigate

  • Prepared for change

  • Diversity

  • Requirement management

  • Participation, co operation


What can we learn from the doctor

What can we learn from The Doctor?

  • Passionate about his mission

  • He enjoys his journey

  • No dogma’s

  • Not afraid of the unexpected, embraces change

  • Surrounded by a few smart and pro-active companions

  • Uses a few simple yet powerful tools

  • Easy going, BUT can be very persistent WHEN IT MATTERS


Almost done but tell me when is testing done

Almost Done!But tell me, when is testing done?


Thank you

Thank you !

[email protected]

+31 6 512 88 478


  • Login