Testing techniques
This presentation is the property of its rightful owner.
Sponsored Links
1 / 35

Testing Techniques PowerPoint PPT Presentation


  • 60 Views
  • Uploaded on
  • Presentation posted in: General

Testing Techniques. Equivalence class analysis Cause effect graphs All pair testing. Equivalence classes. Equivalence classes. Equivalence classes. Triangle Problem. Read 3 integer values in range [1..200].

Download Presentation

Testing Techniques

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Testing techniques

Testing Techniques

Equivalence class analysis

Cause effect graphs

All pair testing


Equivalence classes

Equivalence classes


Equivalence classes1

Equivalence classes


Equivalence classes2

Equivalence classes


Triangle problem

Triangle Problem

  • Read 3 integer values in range [1..200].

  • These 3 values represent the length of the sides of a triangle. The programme displays a message which establishes that the triangle is isosceles, equilateral or scalene.


Triangle problem1

Triangle Problem

  • we are interested in 4 questions:

    • Is it a triangle?

    • Is it an isosceles?

    • Is it a scalene?

    • Is it an equilateral?

  • We may define the input test data by defining the equivalence class through “viewing” the 4 output groups:

    • input sides <a, b, c> do not form a triangle

    • input sides <a, b ,c> form an isosceles triangle

    • input sides <a, b, c> form a scalene triangle

    • input sides <a, b, c> form an equilateral triangle


Valid classes

Valid Classes


Non valid classes

Non Valid Classes


Non valid classes1

Non Valid Classes

Could you find other non valid classes ?


Cause effect graph

Cause effect graph


Methodology

Methodology


Triangle example

Triangle example


Triangle problem2

Triangle Problem


Cause effect graph1

Cause Effect Graph


Decision table

Decision Table


Test cases

Test cases


All pairs testing

All pairs testing

  • Based on orthogonal Array

  • Drastically reduces number of test cases

  • A combination of worst cases and equivalence class testing

  • Well supported in the agile methods community

  • Supported by commercial and non-commercial products


Testing techniques

What Does All Pairs Do?

  • Input is a set of equivalence classes for each variable.

  • Sometimes the equivalence classes are those used in Boundary Value testing (min, min+, nominal, max-, max)

  • Output is a set of (partial) test cases that approximate an orthogonal array, plus some pairing information.

  • Why partial? No expected outputs are provided. natural enough, but a tester still needs to do this step.


Testing techniques

All Pairs Assumptions

  • Variables have clear equivalence classes.

  • Variables are independent.

  • Failures are the result of the interaction of a pair of variable values.


Example

Example

  • Suppose that we have three parameters.

  • For each parameter, there are two possible values.

    • Values are :

      • A, B for parameter 1.

      • J, K for parameter 2.

      • Y, Z for parameter 3.

  • Degree of interaction coverage is 2.

    • We want to cover all potential 2-way interactions among parameter values.


Set of all possible test configurations

A

J

Y

A

J

Z

A

K

Y

A

K

Z

B

J

Y

B

J

Z

B

K

Y

B

K

Z

Set of all possible test configurations

Three parameters, two

values for each.

There are 23 = 8 possible test configurations.


Set of all possible degree 2 interaction elements

A

J

A

Y

J

Y

A

K

A

Z

J

Z

B

J

B

Y

K

Y

B

K

B

Z

K

Z

Set of all possible degree 2interaction elements

There are ( )  2 2 = 12 possible interaction elements.

3

  • Coverage measure:

    • Percentage of interaction elements covered.

2


Test configurations as sets of interactions

A

J

A

Y

J

Y

Test configurations assets of interactions

A

J

Y

One test configuration...

… covers 3 possible

interaction elements.


Interaction test coverage goal

A

J

Y

A

J

Z

A

J

A

Y

J

Y

A

K

Y

A

K

A

Z

J

Z

A

K

Z

B

J

B

Y

K

Y

B

J

Y

B

K

B

Z

K

Z

B

J

Z

B

K

Y

B

K

Z

Interaction test coverage goal

Goal: cover all interaction

elements…

…using a subset of

all test configurations.


Selection of test configurations for coverage of interaction elements

A

J

A

Y

A

A

J

J

Y

Y

A

K

A

Z

A

J

A

Y

J

Y

A

J

Z

A

K

Y

B

J

B

Y

J

Y

A

K

Z

B

J

Y

B

K

B

Z

J

Z

B

J

Z

B

K

Y

K

Y

B

K

Z

K

Z

Selection of test configurations forcoverage of interaction elements

Interaction elements

Test configurations

Degree 2 coverage: 3 / 12 = 25%

Degree 3 coverage: 1 / 8 = 12.5%


Selection of test configurations for coverage of interaction elements1

A

J

Y

A

K

A

Z

A

K

Z

K

Z

Selection of test configurations forcoverage of interaction elements

Interaction elements

Test configurations

A

J

A

Y

J

Y

A

J

Z

A

K

A

Z

J

Z

A

K

Y

B

J

B

Y

K

Y

A

K

Z

B

K

B

Z

K

Z

B

J

Y

B

J

Z

B

K

Y

Degree 2 coverage: 6 / 12 = 50%

B

K

Z

Degree 3 coverage: 2 / 8 = 25%


Selection of test configurations for coverage of interaction elements2

A

A

J

J

Y

Y

A

J

Z

A

K

Y

J

Z

A

K

Z

B

J

A

K

Z

B

J

Y

B

Z

B

J

Z

B

J

Z

B

K

Y

B

K

Z

Selection of test configurations forcoverage of interaction elements

Interaction elements

Test configurations

A

J

A

Y

J

Y

A

K

A

Z

J

Z

B

J

B

Y

K

Y

B

K

B

Z

K

Z

Degree 2 coverage: 9 / 12 = 75%

Degree 3 coverage: 3 / 8 = 37.5%


Selection of test configurations for coverage of interaction elements3

A

A

J

J

Y

Y

A

J

Z

A

K

Y

A

K

Z

B

Y

K

Y

A

K

Z

B

J

Y

B

K

B

J

Z

B

J

Z

B

K

Y

B

K

Y

B

K

Z

Selection of test configurations forcoverage of interaction elements

Interaction elements

Test configurations

A

J

A

Y

J

Y

A

K

A

Z

J

Z

B

J

B

Y

K

Y

B

K

B

Z

K

Z

Degree 2 coverage: 12 / 12 = 100%

Degree 3 coverage: 4 / 8 = 50%


Choosing the degree of coverage

Choosing the degree of coverage

  • In one experiment, covering 2 way interactions resulted in the following average code coverage:

    • 93% block coverage.

    • 83% decision coverage.

    • 76% c-use coverage.

    • 73% p-use coverage.

    • Source: Cohen, et al, “The combinatorial design approach to automatic test generation”, IEEE Software, Sept. 1996.

  • Another experience report investigating interactions among 2-4 components:

    • Dunietz, et al, “Applying design of experiments to software testing”, Proc. Of ICSE ‘97.


Online mortgage application thanks bernie berger stareast

Online Mortgage Application(Thanks, Bernie Berger, STAREast)

Closing Cost

Bank Emp

Intro Rate

Residence

Refinance

Property

tier

Credit

LTV

NIV

NAV

Region


Testing techniques

Total number of combinations

Twelve variables, with varying numbers of values, have

7 x 6 x 6 x 5 x 3 x 3 x 2 x 2 x 2 x 2 x 2 x 2 = 725,760

combinations of values.

“All Pairs does it in 50.” (Bernie Berger, STAREast2003)


Testing techniques

1

NY

L

1 fam

A+

Pri

80%

Yes

Yes

Yes

Cust

Yes

Yes

66

2

NY

M

2 fam

A

Vac

90%

No

No

No

Bank

No

No

66

3

NJ

L

2 fam

A-

Inv

100%

Yes

No

Yes

Bank

Yes

No

54

4

NJ

M

1 fam

B

Pri

90%

No

Yes

No

Cust

No

Yes

46

47

NY

H+1

~Coop

B

~Inv

~80%

~No

~No

~Yes

~Cust

~Yes

~Yes

1

48

Other

H

2 fam

~B

~Vac

~100%

~No

~No

~Yes

~Cust

~No

~No

1

49

Other

H+2

3 fam

~A-

~Pri

~80%

~Yes

~No

~No

~Cust

~No

~No

1

50

Other

H+1

Condo

<B

~Vac

~90%

~No

~No

~Yes

~Cust

~No

~No

2

Selected All Pairs Test Cases (note fault resolution!)


Mortgage application observations

Mortgage Application Observations

  • The example uses values, not equivalence classes.

  • Values are independent,

  • System level testing (apparently).

  • The efficiency comes at the expense of fault isolation. There are 2C12 = 12!/(2! x (12 - 2)!) = 66 pairs in the first row of the table. If that test case fails, which of the 66 pairs caused the failure?

  • NB: excellent candidate for regression testing


What if the fault involves more than just a pair of variables

What if the fault involves more than just a pair of variables?

Number of combinations of n things taken p at a time is

n

!

pCn

=

p

!

(

n

p

)!

-

Worst case (for first test case):

12

å

pCn

2

^

12

1

4095

=

-

=

1


  • Login