testistanbul 2013 conference future of testing new techniques and methodologies
Download
Skip this Video
Download Presentation
TestIstanbul 2013 Conference “Future Of Testing: New Techniques and Methodologies”

Loading in 2 Seconds...

play fullscreen
1 / 43

TestIstanbul 2013 Conference “Future Of Testing: New Techniques and Methodologies” - PowerPoint PPT Presentation


  • 127 Views
  • Uploaded on

TestIstanbul 2013 Conference “Future Of Testing: New Techniques and Methodologies”. The Future Of Test Management “Expect The Unexpected” Gerlof Hoekstra. Introduction Who am I, the world I come from. Gerlof Hoekstra Started as software engineer Early 90’s: Testing !

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' TestIstanbul 2013 Conference “Future Of Testing: New Techniques and Methodologies”' - lam


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
testistanbul 2013 conference future of testing new techniques and methodologies

TestIstanbul 2013 Conference“Future Of Testing: New Techniques and Methodologies”

The Future Of Test Management

“Expect The Unexpected”

Gerlof Hoekstra

introduction who am i the world i come from
Introduction Who am I, the world I come from
  • Gerlof Hoekstra
  • Started as software engineer
  • Early 90’s: Testing !
  • Current job: Test consultant/test manager/solution consultant at Atos NL
  • From software testing to large scale business process integration tests
    • Complex application chains
    • Many stakeholders
    • Many software suppliers
intended audience
Intended audience
  • Do you sometimes
    • Feel that your message is not heard?
    • Feel that nobody cares about your test plans and test cases?
  • Does your customer
    • Complain that testing takes too long, asks you:
      • ‘When you’re done?’, ‘Can’t you go faster?’, ‘You are delaying the project’
    • Ignore your good advice and still go live with a flawed system?
  • Do you sometimes feel frustrated because
    • You only get half the time that you require for a proper test
    • Many of the defects you found are still not solved?
  • Do you worry
    • If your job still exists in a few years, when all projects are ‘Agile’?
presentation subject
Presentation subject
  • Being successful as a test manager often is very much dependent of the way you behave, set expectations, use words & symbols etc.
  • Very often (a combination of) subtle things
  • Some test managers I met and saw at work
    • are worried about the future
    • have been frustrated, burned out, replaced (yes, it can be a challenging job)

WHICH QUALITIES CAN HELP TESTMANAGERS TO DELIVER MORE VALUE TO THEIR CUSTOMERS

AND IN THE MEANTIME HAVE A MORE FUN JOB AND BE PREPARED FOR THE FUTURE

agenda
Agenda

Introduction

Mission

Test planning & specification

Showing test results

The future of test management

Closure

the test manager s mission
The Test Manager’s Mission

How the first 20 minutes of your (to be) assignment can be crucial

job interview 1
Job Interview 1

‘…. write a test plan, make sure the acceptance criteria are SMART, make test cases, have all requirements covered, get the test cases reviewed and formally accepted, execute the test cases, have blocking defects get solved, perform re-tests, regression testing, deliver test report and provide a go/no go advice’.

job interview 2
Job Interview 2

‘I expect you, the future test manager, to assure the quality and decide whether the system can go live, how will you do that?’

‘What the …?!??’

‘I don’t…….’

my mission where i stand for and not
My mission: Where I stand for (and not)
  • I am here to serve, to assist the stakeholders
  • My job is to provide USEFUL INFORMATION
    • that the stakeholders need
    • to make their own informed decisions
  • I am the weather man !
  • I am NOT the quality police, nor the judge
  • I do NOT ensure product quality either
then what do i ensure
Then what DO I ‘ensure’ ?
  • My observations are
    • Objective, facts, no opinions
    • Made by skilled & trained investigators
  • You will receive forecasts that you will learn to appreciate and are
    • Pretty reliable
    • Early in time (!)
  • The information you will get from me is very clear, in terms that the stakeholders understand, no explaining or advice needed, enables them to make their own decisions!
  • The information gathering process is efficient

I PROMISE YOU, IN CASE YOUR PROJECT IS HEADING INTO THE WRONG DIRECTION, I WILL BE THE FIRST TO TELL YOU, LOUD AND CLEAR !

two basic values
Two ‘basic values’
  • ‘Expect The Unexpected’
    • Projects are subject of constant change and movement
    • One cannot possibly plan all activities ahead
    • Test strategy should be able to cope with change
  • Management style
    • Most testers are highly skilled and trained professionals
    • Test manager should treat them appropriate
    • Leave room for personal initiative and creativity
agenda1
Agenda

Introduction

Mission

Test planning & specification

Showing test results

The future of test management

Closure

discussion with an old school test manager
Discussion with an ‘old school test manager’
  • How to we know your test set is complete?
  • Your test cases are not very detailed.
  • You did not define in the test plan the criteria when a defect is considered blocking.
  • You do not have certified testers in your team
  • You did not define acceptance criteria (coverage%, #test cases passed, #open defects)
  • You should complete the system test before you start the acceptance test
  • The most important deliverable (go/no go advice) is missing in your plan
  • ---
  • ---

Review test plan v0.5

slide14
Let me introduce you to ‘The Doctor’You don’t have to be a control freak to get missions accomplished
detailed preparation versus flexibility
Detailed preparation versus Flexibility
  • Have a simple but clear plan, not too detailed
  • Have attention to details by the time when really needed, be prepared !
  • Prepare / train your team so that they are able to respond when the Unexpected happens
writing a test plan the theory
Writing a Test Plan – The Theory

The creation of the test plan involves the following activities:

  • Establishing the assignment
  • Understanding the assignment
  • Determining the test basis
  • Analysing the product risks
  • Determining the test strategy
  • Estimating the effort
  • Determining the planning
  • Allocating test units and test techniques
  • Defining the test products
  • Defining the organisation
  • Defining the infrastructure
  • Organising the management
  • Determining the test project risks and countermeasures
  • Feedbackand consolidation of the plan    
writing a test plan the theory1
Writing a Test Plan – The Theory

The creation of the test plan involves the following activities:

  • Establishing the assignment
  • Understanding the assignment
  • Determining the test basis
  • Analysing the product risks
  • Determining the test strategy
  • Estimating the effort
  • Determining the planning
  • Allocating test units and test techniques
  • Defining the test products
  • Defining the organisation
  • Defining the infrastructure
  • Organising the management
  • Determining the test project risks and countermeasures
  • Feedbackand consolidation of the plan    

Now look at this !?!

some additional thoughts
Some additional thoughts
  • You can not write a test plan on your own, hiding in your office
  • Received no review comments? The reviewer did not read/understand the plan or is not interested. Don’t consider the plan done!
  • Strip to the bone, minimize #words, avoid meaningless phrases, use images & tables
  • A test plan is NOT a contract!
  • Expect the plan to change
coverage maps a great way to plan your test
Coverage MapsA great way to plan your test

Tip: you can use James Bach’s San Francisco DePOT acronym

test specification
Test specification
  • How many test cases & scripts do we really need?
  • How detailed should they be?
  • Scripted vs Exploratory testing?

IT IS ALL ABOUT BALANCE

My observation:

  • Many test educations (and as a result, many certified testers) heavily over-rate the value of specifying a lot of detailed test scripts
  • Often, you can get away with less test scripts than you might think
slide25
Example, a limited number of formal test cases, based on a decision table , completed with test charters
agenda2
Agenda

Introduction

Mission

Test planning & specification

Showing test results

The future of test management

Closure

a good test report
A good Test Report
  • Has useful information for the stakeholders
  • Is easy to understand for the stakeholders
  • Does not need any go/no go advice
  • Is brief and visual
  • Includes a forecast
  • Makes clear what you have (not) covered
slide28
Be VERY careful with coverage metricsTesting is investigation in an infinite space: numbers like 80% could easily be misunderstood
slide29

Be VERY careful with coverage metricsOnly testers understand these numbers… (do we?)

‘We have reached 91.7% multiple condition coverage’

Do our stakeholders know what this means for them?

Be honest, do you know? I’m not sure if I do…

old school test reporting
Old school test reporting
  • We covered 95% of the area
  • We found 60 mines
  • We disarmed 58
  • The 2 mines we did not disarm are here
  • These are MSM MKII mines containing

4Kg of Composition B explosives

IS THIS USEFUL INFORMATION ?

WOULD YOU APPRECIATE MY GO/NO GO ADVICE ?

what is wrong with the good old traffic lights
What is wrong with the good old traffic lights?
  • Do not encourage to take very much responsibility
  • Don’t think, just obey the law !!
  • Is not how I want to communicate my test results
  • Remember: it’s not the test manager who decides!
  • Weather symbols send a better message
  • This is likely to happen
  • You have been informed (warned?) and now it’s up to you to make a decision
  • B.t.w.: a weather forecast is never 100% reliable
longer term forecasting testing in the early project phases and i mean testing not just reviewing
Longer term forecastingTesting in the Early Project Phases (and I mean testing, not just reviewing)

Getting your message heard

  • Don’t complain, just report the facts
  • Friendly ask some questions (‘Are you OK with this?’)
  • And the best: let stakeholders experience the problem  no further comments needed
agenda3
Agenda

Introduction

Mission

Test planning & specification

Showing test results

The future of test management

Closure

trend 1 changing software development paradigms
Trend 1: Changing Software Development Paradigms
  • Agile
  • Continuous deployment
  • Smaller iterations
  • Test Driven Development
  • Quality is not ‘owned’ by the testers (anymore)
trend 2 systems integration
Trend 2: Systems Integration
  • Chains, ever increasing in scope and complexity
  • Many software suppliers involved
  • COTS software, combined with bespoke software
  • Multi-stakeholder
  • Cross-company integration
  • Distributed data
  • Across the globe, 24/7 business processes

ORGANIZING AN END-TO-END TEST CERTAINLY IS NOT A TRIVIAL TASK

a shift in qualities
A shift in qualities

To:

From:

  • Ensure quality
  • Go/No Go decision
  • Find defects
  • Number of test cases
  • Number of defects
  • Enforce, control
  • Proof
  • Follow rules & standards
  • Strictly follow detailed plans
  • Certification
  • Requirements freeze
  • Acceptance criteria
  • Quality assistance
  • Deep insight to stakeholders
  • Prevent failures
  • Coverage map
  • Business consequences
  • Serve, help, facilitate
  • Forecast
  • Investigate
  • Prepared for change
  • Diversity
  • Requirement management
  • Participation, co operation
what can we learn from the doctor
What can we learn from The Doctor?
  • Passionate about his mission
  • He enjoys his journey
  • No dogma’s
  • Not afraid of the unexpected, embraces change
  • Surrounded by a few smart and pro-active companions
  • Uses a few simple yet powerful tools
  • Easy going, BUT can be very persistent WHEN IT MATTERS
ad