Grey box testing
This presentation is the property of its rightful owner.
Sponsored Links
1 / 27

Grey Box testing PowerPoint PPT Presentation


  • 134 Views
  • Uploaded on
  • Presentation posted in: General

Grey Box testing. Tor Stålhane. What is Grey Box testing. Grey Box testing is testing done with limited knowledge of the internal of the system. Grey Box testers have access to detailed design documents with information beyond requirements documents.

Download Presentation

Grey Box testing

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Grey box testing

Grey Box testing

Tor Stålhane


What is grey box testing

What is Grey Box testing

Grey Box testing is testing done with limited knowledge of the internal of the system.

Grey Box testers have access to detailed design documents with information beyond requirements documents.

Grey Box tests are generated based on information such as state-based models or architecture diagrams of the target system.


State based testing

State based testing

The tests are derived from a state model of the system. We can derive the state model in several way, e.g. from

  • Expected system behavior

  • State part of a UML design or requirements specification.

  • Other state diagrams

    Most system will, however, have a large number of states


Binder s state control faults 1

Binder’s state control faults – 1

Binder has make a list of common state –related problems in software systems. This list may be used as an input to

  • State based testing

  • State machine or code inspection


Binder s state control faults 2

Binder’s state control faults – 2

  • Missing or incorrect

    • Transitions – new state is legal but incorrect

    • Events – valid message ignored

  • Extra, missing or corrupt state – unpredictable behavior

  • Sneak path – message accepted when it should not be accepted

  • Illegal message failure – unexpected message causes a failure

  • Trap door – system accepts undefined message.


State test criteria

State test criteria

We can choose one or more of the following test selection criteria:

  • All states – testing passes through all states

  • All events – testing forces all events to occur at least once

  • All actions – testing forces all actions to be produced at least once


State test strategies

State test strategies

All round-trip paths

  • All transition sequences beginning and ending in the same state

  • All simple paths from initial to final state

    This strategy will help you to find

  • All invalid or missing states

  • Some extra states

  • All event an action faults


Round trip path tree 1

A round-trip path tree

Is built form a state transition diagram

Includes all round-trip paths

Transition sequences beginning and ending in the same state

Simple paths for initial to final state. If a loop is present, we use only one iteration

Is used to

Check conformance to explicit behavioral models

Find sneak paths

Round-trip path tree – 1


Round trip path tree 2

Round-trip path tree – 2

A test strategy based on round-trip path trees will reveal:

  • All state control faults

  • All sneak paths – messages are accepted when they should not

  • Many corrupt states - unpredictable behavior


Challenge for round trip path testing

Challenge for round-trip path testing

In order to test a system based on state transitions via triggers, predicates (guards) and activities, we need to be able to observe and register these entities.

Thus, we may need to include “points of observations” in the code that gives us access to the necessary information.


Round trip tree small example

Round-trip tree – small example

a

a

A

a[p1] / w

A

B

B

A

C

a[p1] / w

b[p2] / u

b[p2] / u

C

W

A

W

W

B

A


Transitions

Transitions

Each transition in a state diagram has the form

trigger-signature [guard] / activity. All parts are optional

  • trigger-signature: usually a single event that triggers a potential change of state.

  • guard: a Boolean condition that must be true for the transition to take place.

  • activity: an action that is performed during the transition.


Test description 1

Test description – 1

Each test completes one branch of the round-trip tree – from a to W. The necessary transitions describes the test case.

The table on the next slide shows the test case for a -> A -> C -> A

a

A

B

A

C

A

W

W

B

A


Test description 2

Test description – 2

a

A

a[p1] / w

B

A

C

b[p2] / u

A

W

W

B

A


Sneak path test cases

Sneak path test cases

A sneak path – message accepted when it should not be accepted – can occur if

  • There is an unspecified transition

  • The transition occur even if the guard predicate is false


Sneak path test description

Sneak path test description

a

A

a[p1] / w

B

A

C

b[p2] / u

A

W

W

B

A


State diagram for a sensor 1

State diagram for a sensor - 1

B

w

a

E

D

C

A


State diagram for a sensor 2

a

w

A

B

C

E

D

State diagram for a sensor - 2


Sensor round trip path tree

a

w

w

w

D

C

E

E

E

B

E

A

D

Sensor round-trip path tree

[sensor alarm] / sound alarm

[no sensor alarm] / test

[false alarm] / test

[alarm OK / request reset]

[test fails] / replace

[test OK]

[ACK] / reset

[test fails] / replace

[test OK]

/ test

/ test


Acknowledgement

Acknowledgement

Most of the previous presentation is based on a slide set from the University of Ottawa, Canada


Mutation testing

Mutation testing

Tor Stålhane


Type 1 mutation testing 1

Type 1 mutation testing – 1

Type 1 mutation testing is done as follows:

  • Write a chunk of code

  • Write a set of tests

  • Test and correct until the test suite runs without errors

  • Change a random part of the code – e.g. a “+” to a “-”. This is called a code mutant. We will only consider mutants that compiles without error messages

  • Run the test suite again


Type 1 mutation testing 2

Type 1 mutation testing – 2

  • If the tests suite

    • runs without errors, then we need to extend the test suite until we discover the defect.

    • diagnoses the defect then got back to step 4 to create a new mutant.

      The test process stops when all of X new mutants are discovered by the current test suite.


Type 2 mutation testing

Type 2 mutation testing

Type 2 mutation testing – also called “fuzzing” – has many ideas in common with random testing. The main difference is that:

  • Random testing generate random tests from scratch.

  • Type 2 mutation testing starts with an input that works OK and then change part of it in a random way.


Software functions are not continuous

Software functions are not continuous

When we discuss mutation testing, it is important to remember that a function implemented in software is not continuous. E.g. x = 2.00 and x = 1.99 can give dramatically different results.

A small changes in input can thus have a large effect on the output.


Type 2 mutation testing example 1

Type 2 mutation testing example – 1

SUT – a system for computing F(x) – takes an input consisting of

  • F – a three character string identifying a probability distribution function.

  • A real number x. The allowed value range will depend on F, e.g. if F = “ exp”, then x must be a positive number, while if F “nor” then x may be any number.


Type 2 mutation testing example 2

Type 2 mutation testing example – 2

We can perform type 2 mutation testing as follows:

  • Run a test with input <“exp”, 3>

  • Check that the result is correct

  • Make a mutant by drawing a random integer value 1 (F) or 2 (x).

    • If we draw a 1, generate a random integer n from 0 to 10 – string size – and generate a random string of length n

    • If we draw a 2, generate a random real value x

  • Compute F(x)

  • Check the result – especially any error messages

  • If we are satisfied then stop, otherwise repeat from step 3


  • Login