Test vs inspection part 1
This presentation is the property of its rightful owner.
Sponsored Links
1 / 47

Test vs. inspection Part 1 PowerPoint PPT Presentation


  • 91 Views
  • Uploaded on
  • Presentation posted in: General

Test vs. inspection Part 1. Torbjørn Skramstad. What we will cover. Part 1 Introduction Inspection processes Testing processes Part 2 Tests and inspections – some data Inspection as a social process – two experiments and some conclusions. Introduction. Adam’s data - 1.

Download Presentation

Test vs. inspection Part 1

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Test vs inspection part 1

Test vs. inspectionPart 1

Torbjørn Skramstad


What we will cover

What we will cover

  • Part 1

    • Introduction

    • Inspection processes

    • Testing processes

  • Part 2

    • Tests and inspections – some data

    • Inspection as a social process – two experiments and some conclusions


Introduction

Introduction


Adam s data 1

Adam’s data - 1


Adams data 2

Adams’ data – 2

The main information that you get from the table on the previous slide is that

  • Some defects are important because they will happen quite often.

  • Most defects are not important since they will happen seldom.

    How can we tell the difference?


Testing alone is not the solution

Testing alone is not the solution

As can be seen from the next slide, testing is not an acceptable solution alone. It will

  • Take too long time

  • Cost too much

    We can generate tests automatically, but would never the less have to use large resources to check the result – the oracle problem


A limit result

A limit result

The following relation holds under a rather wide set of conditions:

The initial number of defects – N0 – must be estimated e.g. based on experience from earlier projects as number of defects per KLOC.


An example from telecom

An example from telecom


Testing and inspection the v model

Testing and inspection – the V model


Testing and inspection 1

Testing and inspection – 1

The important message here is that testing cannot always be done.

In the first, important phases, we have nothing to execute and will thus always have to do some type of inspection.

This might be considered one of the weaknesses of traditional software engineering over Agile development.


Testing and inspection 2

Testing and inspection – 2

In order to understand the main differences between testing and inspection, we should consider Fits’ list.

Based on this, we will give a short discussion of the relative merits of testing and inspection.


Man vs machine 1

Man vs. machine – 1

Good when we need the ability to

  • Handle variation

  • Be innovative and inductive

  • Recognize and handle patterns

    Not so good when we need the ability to

  • Do the same things over and over again in a consistent manner

  • Handle large amount of data


Man vs machine 2

Man vs. machine – 2

In order to do the best job possible we need processes where we let each part

  • Do what they are best at:

    • Man is innovative

    • Machine handles large amounts of data

  • Support the other with their specialties.

    • Machine supports man by making large amounts of information available

    • Man support machine by providing it with innovative input


General considerations documents

General considerations - documents

Architecture, system, sub-system and component design plus pseudo code. Here we can only use inspections.

Man will use experience and knowledge to identify possible problems

Machine can support by identifying information – e.g. find all occurrences of a string.


General considerations code 1

General considerations – code (1)

For executable code, we can use inspection, testing or a combination of both.

The size and complexity – degree of dynamism – of the code will, to a large degree, decide our choice.

Other important factors are the degree of experience with

  • The programming language

  • The algorithms used


General considerations code 2

General considerations – code (2)

Simple code

  • Start with inspection – all code

  • Design and run tests

    Complex code

  • Start with inspection – focus on algorithm and logic

  • Decide test completeness criteria – we cannot test everything

  • Design and run tests


Inspection processes

Inspection processes


Inspections 1

Inspections – 1

The term “inspection” is often used in a rather imprecise manner. We will look at three types of inspection:

  • Walkthrough

  • Informal inspection – also called informal review

  • Formal inspection – also called formal review or just inspection

    The first two types are usually project internal while the last one is used as a final acceptance activity for a document.


Inspections 2

Inspections – 2

For all types of inspections:

  • The quality of the results depends on the experience and knowledge of the participants. “Garbage in – Garbage out”

  • It might be a good idea to involve customer representatives.


The walkthrough process

The walkthrough process

Walkthrough is a simple process – mostly used for early decisions for an activity. The document owner:

  • Makes a rough sketch of the solution – architecture, algorithm etc.

  • Presents – explain – the sketch to whoever shows up.

  • Registers feedback – improvements.


Walkthrough pros and cons

Walkthrough – pros and cons

Pros:

  • Easy and inexpensive. Needs no extra preparation.

  • Collect ideas at an early stage of development.

    Cons:

  • No commitment from the participants

  • May collect many loose or irrelevant ideas


The informal inspection process

Planning

Rules,checklists,procedures

The informal inspection process

Loggingmeeting

Product document

Changerequests

Individualchecking


Informal inspections pros and cons

Informal inspections – pros and cons

Pros:

  • Is simple and inexpensive to perform.

  • Can be used at all stages of development

  • Usually has a good cost / benefit ratio

  • Needs a minimum of planning

    Cons:

  • No participant commitment

  • No process improvement


The formal inspection process

The formal inspection process

The formal inspection process described below is – with small variations – the most commonly used. The version shown on the following slides stem from T. Gilb and D. Graham.

We recommend this process as the final acceptance process for all important documents


Formal inspection process overview

Rules,checklists,procedures

Planning

Formal inspection process overview

Changerequests

Product document

Process improvements

Loggingmeeting

Walk-through

Edit and follow-up

Individualchecking

Kick-off


Distribution of resources

Distribution of resources


Initiating the inspection process

Initiating the inspection process

  • The inspection process starts with a “request for inspection” from the author to the QA responsible.

  • The QA responsible appoints an inspection leader.

  • First step is always to check that the document is fit for inspection.


Planning

Planning

Important planning points are:

  • Who should participate in the inspections

    • Who is interested?

    • Who have time available for preparation and meetings?

    • Who has the necessary knowledge concerning application, language, tools, methods?


Kick off

Kick-off

Important activities here are:

  • Distribution of necessary documents:

    • Documents that shall be inspected

    • Requirements

    • Applicable standards and checklists

  • Assignment of roles and jobs

  • Setting targets for resources, deadlines etc.


Individual checking

Individual checking

This is the main activity of the inspection. Each participant read the document to look for

  • Potential errors - inconsistencies with requirements or common application experience

  • Lack of adherence to company standards or good workmanship


Logging meeting

Logging meeting

The logging meeting has three purposes:

  • Log issues already discovered by inspection participants

  • Discover new issues based on discussions and new information that arises during the logging meeting.

  • Identify possible improvement to the inspection or development process.


Improve the product 1

Improve the product - 1

The author receives the log from the inspection meeting. All items - issues - in the log are categorised as one of the following:

  • Errors in the author’s document.

  • Errors in someone else’s document.

  • Misunderstandings in the inspection team.


Improve the product 2

Improve the product - 2

  • Errors in own document:Make appropriate corrections

  • Errors in someone else’s documents:Inform the owner of this document.

  • Misunderstandings in the inspection team:Improve document to avoid further misunderstandings.


Checking the changes

Checking the changes

This is the responsibility of the inspection leader. He must assure that all issues raised in the log are disposed of in a satisfactory manner:

  • The documents that have been inspected

  • Related documents - including standards and checklists

  • Suggested process improvements


Formal inspection pros and cons

Formal inspection – pros and cons

Pros:

  • Can be used to formally accept documents

  • Includes process improvement

    Cons:

  • Is time consuming and expensive

  • Needs extensive planning in order to succeed


Testing processes

Testing processes


Testing

Testing

We will look at three types of testing:

Unit testing – does the code behave as intended. Usually done by the developer

Function verification testing – also called systems test. Does the component or system provide the required functionality?

System verification testing – also called acceptance test. Does the hardware and software work together to give the user the intended functionality?


The unit testing process

The unit testing process

Unit testing is done by the developer one or more times during development. It is a rather informal process which mostly run as follows:

Implement (part of) a component.

Define one or more tests to activate the code

Check the results against expectations and current understanding of the component


Unit testing pros and cons

Unit testing – pros and cons

Pros:

Simple way to check that the code works.

Can be used together with coding in an iterative manner.

Cons:

Will only test the developer’s understanding of the spec.

May need stubs or drivers in order to test


The system test process

The system test process

A systems test has the following steps:

Based on the requirements, identify

Test for each requirement, including error handling

Initial state, expected result and final state

Identify dependencies between tests

Identify acceptance criteria for test suite

Run tests and check results against

Acceptance criteria for each test

Acceptance criteria for the test suite


Systems test pros and cons

Systems test – pros and cons

Pros:

Tests system’s behavior against customer requirements.

Cons:

It is a black box test. If we find an error, the systems test must be followed by extensive debugging


The acceptance test process

The acceptance test process

The acceptance test usually has three activities – all involving the customer or his representatives:

Rerun the systems test at the customer’s site.

Use the system to solve a set of real-world tasks.

Try to break the system – by stressing it or by feeding it large amounts of illegal input


Acceptance test pros and cons

Acceptance test – pros and cons

Pros:

Creates confidence that the system will be useful for the customer

Shows the system’s ability to operate in the customer’s environment

Cons:

Might force the system to handle input that it was not designed for, thus creating an unfavorable impression.


Testing vs inspection 1

Testing vs. inspection – 1

  • Testing – strong points:

    • Can be made fairly realistic

    • Can take into account the way the customer will use the system

    • Can, at least partly, be automated

    • Can be extended as we learn more of the system under test

  • Testing – weak points:

    • Is only a spot check

    • Can only be used for code

    • Requires a lot of personnel resources

    • May need several stubs and drivers


Testing vs inspection 2

Testing vs. inspection – 2

  • Inspection – strong points:

    • Can be used on all types of documents, not just code

    • Is not a spot check – sees the complete picture

    • Uses all information available in the team

  • Inspection – weak points:

    • Is difficult to use on complex, dynamic situations

    • May find problems that will not bother the customer – see Adams’ data


Testing vs inspection 3

Testing vs. inspection – 3

Experience shown that testing and inspections

  • Finds different types of defects

  • Are efficient at different stages in development

    Thus, we need both testing and inspection.


  • Login