Integration and system testing
1 / 87

Integration and system Testing - PowerPoint PPT Presentation

  • Updated On :

Integration and system Testing . Csci565 Fall 2012. Objectives. Integration Testing Simple ATM (SATM) discuss integration testing strategies Decomposition Based Integration Testing (DBIT) Top-down Bottom-up Call Graph Based Integration Testing (CGBIT)

Related searches for Integration and system Testing

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Integration and system Testing' - morwenna

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Integration and system testing

Integration and system Testing


Fall 2012



  • Integration Testing

  • Simple ATM (SATM)

  • discuss integration testing strategies

    • Decomposition Based Integration Testing (DBIT)

      • Top-down

      • Bottom-up

    • Call Graph Based Integration Testing (CGBIT)

    • Path-Based Integration Testing (PBIT)


A software testing strategy spiral model
A Software Testing Strategy: Spiral Model

System Testing

validation Testing

Integration Testing

Unit Testing




System eng.

Integration testing 1
Integration Testing:1

  • If all units/components work individually, do they work as a whole when we put them together?

    • Not necessarily

  • The problem is “putting them together” or interfacing them


Problems with interfacing
Problems with interfacing

  • Integration faults often traceable to incomplete or misunderstood interface specifications

    • mismatched assumptions about other components

    • Individually acceptable imprecision may be magnified to unacceptable levels

    • Global data structures can present problems

  • Inconsistent interpretation of parameters or values

  • Mixed units (meters/yards) in Martian Lander

  • Violations of value domains, capacity, or size limits


Integration testing
Integration testing

  • Tests complete systems or subsystems composed of integrated components

  • Integration testing should be black-box testing when tests derived from the specification

  • Main difficulty is localising errors

  • Incremental integration testing reduces this problem


Approaches to integration testing
Approaches to integration testing

  • Two major approaches

    • Incremental approaches

      • The decomposition-based techniques or tree

        • Stubs/Drivers

      • Call Graph-based techniques

        • No stubs or drivers

    • Non-incremental approaches


Incremental approaches top down
Incremental approaches: Top-down

  • Top-down testing

    • Start with high-level system

    • integrate from the top-down replacing individual components by stubs where appropriate

      • Depth-first

      • Breadth-first

      • No-best order

        • Critical sections

        • Early skeletal version using I/O modules



  • Stubs

    • Special module to simulate some functionality

    • Its production is nontrivial task because the code may simulate a very complicated tasks

      • E.g.

        • Writing a stub performing a database table search routine

    • Creating multiple version of same stub for various reasons


Top down complications
Top-down: Complications

  • The most common complication occurs when processing at low level hierarchy demands adequate testing of upper level

  • To overcome:

    • Either, delay many tests until stubs are replaced with actual modules (BAD)

    • Or, develop stubs that perform limited functions that simulate the actual module (GOOD)

    • Or, Integrate the software using bottom up approach


Incremental testing bottom up
Incremental testing: Bottom up

  • Bottom-up testing

    • Integrate individual components in levels until the complete system is created


Bottom up approach
Bottom-up approach

  • Starts with construction and testing with atomic modules

    • No need for stub

    • Low level components are combined into cluster (or builds) to perform a specific sub-function

    • A driver (a control program for testing) is written

      • Contain hardcoded test input, calls the module being tested, and display the results

    • Cluster is tested

    • Drivers are removed and clusters are combined moving upward in the program structure


Guideline for integration testing
Guideline for Integration testing

  • Integrate the components that implement the most frequently used functionality

  • Perform regression testing for old features

  • Perform progression testing for new features

  • Study the Architecture design of the Application

    • identify the Critical Modules.

  • Obtain the interface designs from the Architectural team and create test cases to verify all of the interfaces in detail.

    • E.g., Interface to database/external hardware/software application must be tested in detail.


Non incremental

  • Big-bang

    • Imposes no order (GOOD)

    • Test all the units (Modules) at once (GOOD)

    • Very easy (GOOD)

    • difficult to localize the source of errors (VERY BAD)


Top down vs bottom up
Top-down Vs. Bottom-up

  • Architectural validation

    • Top-down integration testing is better at discovering errors in the system architecture

  • System demonstration

    • Top-down integration testing allows a limited demonstration at an early stage in the development

  • Test implementation

    • Often easier with bottom-up integration testing


Primary design of the atm using tree based decomposition
Primary Design of the ATM using Tree-based decomposition



Terminal I/O







Card Entry

PIN Entry


Simple atm satm
Simple ATM (SATM)

  • An ATM simple

    • Provides 15 screens for interactions

    • includes 3 function buttons

      • B1, B2, B3

    • Modeled in structural analysis

      • Data Model (ERD)

      • Functional Model (DFD)

      • Behavioral model (STD)


Figure 12 7
Figure 12.7


Figure 12 8
Figure 12.8


Figure 12 9
Figure 12.9


Figure 12 10
Figure 12.10


Figure 12 12
Figure 12.12


Figure 12 13
Figure 12.13


Decomposition based strategies
Decomposition based strategies

  • Decomposition based

    • Top/down

    • Bottom up

    • Sandwich

    • Big bang


Figure 12 14
Figure 12.14


Figure 13 1
Figure 13.1


Decomposition based testing 1
Decomposition based testing:1

  • Discussion revolves around the tree-based decomposition and the order by which units are tested and combined

    • Top-to-bottom

    • Bottom-to-top

    • Sandwich

    • Big bang

  • The focus is on the structural compatibility among interfaces


Test sessions
Test Sessions

  • A test session refers to one set of tests for a specific configuration of actual code and stubs

  • The number of integration test sessions using a decomposition tree can be computed

    • Sessions=nodes – leaves + edges


Decomposition based testing 2
decomposition based testing: 2

  • For SATM system

    • 42 integration testing session (i.e., 42 separate sets of integration test cases)

      • top/down

        • (Nodes-1) stubs are needed

          • 32 stub in SATM

      • bottom/up

        • (Nodes-leaves) of drivers are needed

          • 10 drivers in SATM


Decomposition based strategies pros and con
Decomposition based strategies: Pros and con

  • Intuitively clear and understandable

    • In case of faults, most recently added units are suspected ones

  • Can be tracked against decomposition tree

  • Suggests breadth-first or depth-first traversals

  • Units are merged using the decomposition tree

    • Implies correct behavior should follow from individually correct units and interfaces

    • Stubs/Drives are major development Overhead


Call graph based integration testing
Call graph based integration testing

  • Call graph

    • A directed graph

    • Nodes corresponds to unit

    • Edges corresponds to the call

    • E.g.

      • AB (i.e., A is calling B)

  • Attempts to overcome the decomposition problem (structural)

  • Moves toward behavioral testing


Call graph based cg approaches
Call graph based(CG): approaches

  • Two main approaches based on Call Graph

    • Pair-wise integration

    • Neighborhood integration


Table 2 am
Table 2: AM


Pair wise integration
Pair-wise integration

  • The main idea is to eliminate the overhead (i.e., stub/drive)

  • Uses actual code by restricting a session testing to a pair of units in the Call Graph

    • One integration test for each edge in Call Graph

    • 40 edges means 40 integration tests for the SATM



Uses actual code

-one integration test session for each edge

-40 edges for SATM


Neighborhood integration
Neighborhood Integration

  • The neighborhood of a node refers to the nodes that are one edge away from the given nodes

    • SATM Neighborhoods

    • Number of neighborhoods can be computed:

      • 1) Neighborhoods = Interior node + source node

      • 2) Interior nodes =nodes – (source nodes + sink nodes)

        • ( x=1 if there exists leaf nodes connected directly to the root node otherwise X= 0)

  • Results a drastic reduction in the number of integration test session

    • In case of SATM (11 vs. 40)


Pros and cons
Pros and cons

  • Benefits (GOOD)

    • Mostly behavioral than structural

    • Eliminates sub/drive overhead

    • Works well with incremental development method such as Build and composition

  • Liabilities (BAD)

    • The fault isolation

      • E.g.,

        • Fault in one node appearing in several neighborhood


Questions about integration testing
Questions about Integration testing

  • How do perform Integration testing for non-tree based functional decomposition?

    • E.g.,

      • integration testing for OO

      • Integration testing for Client/server systems

      • Integration testing for Layered systems

      • ….


Path based integration testing
Path-based Integration Testing

  • The hybrid approach (i.e., structural and behavioral) is an ideal one integration testing

  • The focus is on the interactions among the units

    • Interfaces are structural

    • Interactions are behavioral

  • With unit testing, some path of source statements is traversed

    • What happens when there is a call to another unit?

      • Ignore the single-entry/single-exit

      • Use exist follows by an entry sequence

      • Suppress the call statement


New and extended concepts
New and extended concepts

  • Source node (begin)

    • A statement fragment at which program execution begins or resumes

    • E.g., BEGIN

  • Sink node (end)

    • A statement fragment at which program execution terminates

    • E.g., END


More on concepts
More on concepts

  • Module Execution Path (MEP)?

    • A sequence of statements that begins with a source node and ends with a sink node, with no intervening sink nodes

  • Message?

    • A programming language mechanism by which one unit transfers control to another unit

      • E.g.,

        • subroutine invocations

        • Procedure calls

        • Function references


The path based integration testing definition
The path based integration testing (definition)

  • Definition: MM-Path

    • An interleaved sequence of module execution paths(MEP) and messages

  • MM-Path can be used

    • to describe sequences of module execution paths including transfers of control among units using messages

    • To represent feasible execution paths that cross unit boundaries

    • To extent program graph

      • Where

        • nodes = MEP

        • edges = messages

  • Atomic system function (ASF)?

    • An action that is observable at the system level in terms of port input and output events (at GUI)


Figure 13 3
Figure 13.3



Mm path graph definition
MM-Path Graph (definition)

  • Definition: MM-Path Graph?

    • Given a set of units, their MM-Path graph is the directed graph in which

      • nodes = MEPs

      • edges = messages/returns


Figure 13 4
Figure 13.4


How long should an mm path be 1
How long should an MM-Path Be:1

  • MM-Path issues

    • How long is an MM-Path?

    • What is the endpoint?

  • The following observable behavior are used as endpoints

    • Event quiescence (event inactivity)

      • System level event

      • Happens when system is ideal/waiting

    • Message quiescence (msg inactivity)

      • Unit that sends no messages is reached (module C)

    • Data quiescence (data inactivity)

      • Happens when a sequences of processing generate a stored data that is not immediately used

      • E.g., account balance that is not used immediately


Mm path guidelines
MM-Path Guidelines

  • MM-Path guidelines:

    • Points of

    • (data, event, or msg) are natural endpoints for an MM-path

    • Atomic System Functions (system behavior) are considered as an upper limit for MM-Paths

      • MM-Paths should not cross ASF boundaries


Pros and cons1
Pros and cons

  • hybrid approach(GOOD)

  • The approach works equally well for software testing developed by waterfall model (GOOD)

  • Testing closely coupled with actual system behavior (GOOD)

  • Identification of MM-Paths which can be offset by elimination of sub/drivers(BAD)

  • Test case generation ?


Summary of integration testing 1
Summary of integration testing: 1

  • Integration testing

    • Testing of incompatibilities and interfaces between otherwise correctly working components

    • Performed with an incomplete system using scaffolding(drivers/stubs/mocks)

      • Stubs main job is to return values to the calling program

        • Simple stub assigns constant values to the outputs


Summary of integration testing 2
Summary of integration testing:2

  • Drivers main job is to control and set up testing ; they can include hard-code values or retrieve the values from an external source

  • Tools exist to generate both stubs/drivers automatically


Summary of integration testing 3
Summary of integration testing: 3

  • Integration testing focuses on interface

    • 40% of software errors are revealed during integration testing

    • Must be built on foundation of thorough unit testing

    • Integration faults often traceable to incomplete or misunderstood interface specifications

      • Prefer prevention to detection, and make detection easier by imposing design constraints

  • Strategies tied to project build order

    • Order construction, integration, and testing to reduce cost or risk

Type of integration faults
Type of Integration Faults

  • Inconsistent interpretation of parameters or values

    • Example: Mixed units (meters/yards) in Martian Lander

  • Violations of value domains, capacity, or size limits

    • Example: Buffer overflow

  • Side effects on parameters or resources

    • Example: Conflict on (unspecified) temporary file

  • Omitted or misunderstood functionality

    • Example: Inconsistent interpretation of web hits

  • Nonfunctional properties

    • Example: Unanticipated performance issues

  • Dynamic mismatches

    • Example: Incompatible polymorphic method calls

Integration testing using software architecture sa
Integration testing Using Software Architecture (SA)

  • Software Architectural Based Integration testing

    • How software architecture can be used to conduct tests to uncover errors related to the interface?

  • How can we test if a design and implementation comply using SA?

  • How can we specify a SA such that one or more properties can be tested or verified?

  • How integration testing can be planned and controlled based on the SA?


Software architecture testing
Software architecture & Testing

  • Using Traditional approach for formalizing and automating the integration test stage is difficult

    • the selection of the test cases for the subsystems stressstructureover behavior

    • the order in which the components are incrementally combined is entirely dependent on the structure (tree or graph)


Software architecture sa testing
Software Architecture (SA) & Testing

  • Most current testing approaches are either based

    • the implementation or structural

    • information of the system

    • a requirement specification

    • system design

    • the architecture level

  • Using the properties of formal architecture description languages (ADL), architecture based test criteria can be defined



  • what to test at the software architecture?

    • At the integration and system testing level, the predominant form for defining testing criteria is based on definition/use bindings

      • each module defines or provides a set of facilities that are available to the users or requires by other modules

  • Software architecture

    • focuses on the interaction between components

    • Interaction is language independent


Testing requirements for sa
Testing Requirements for SA

  • Testing component-to-connector connectivity

  • Testing connector-to-component connectivity

  • Testing component –to-internal interfaces

  • Testing connector internal interfaces

  • Testing direct-component-to-component connectivity

  • Testing indirect-component-to-component connectivity

  • Testing whole structure connectivity

  • Testing of NFR? How to test for usability, safety, etc?


Path based testing sa
Path-Based Testing & SA?

  • SA Method

    • Used for prediction of the system-level quality

  • The approach would belong to the black box techniques of state transition testing

    • The system specification is modeled by an automata

    • The generation of test cases is aimed at covering the arcs (the transitions) and the nodes ( the states) of it

    • Create sub-graphs from SPEC representing specific views of a system

    • Use these views as a base for the definition of coverage criteria and testing strategies.

    • Select test cases to cover these sub-graphs


The advantages of using the sa to derive the automaton
The advantages of using the SA to derive the automaton

  • for state transition testing are evident:

    • we have a formal description and the automaton can be automatically derived;

    • the SA is at a high level of abstraction, thus the number of states in the automaton can be kept manageable


References acknowledgements

  • 1) Software testing by Paul C. Jargensen

  • 2) Art of Software Testing by Mayer

  • 3) Writing Good Software Engineering Research Papers (Mini-tutorial), Mary Shaw, ICSE02

  • Software engineering: Practitioner ’s approach,7th edition, 2009 by Roger Pressman:

  • Software Testing and Analysis by Mauro Pezze and Michal Young

  • Software Testing by Paul Ammann and Jeff Offutt


Writing good software engineering research papers
Writing Good Software Engineering Research Papers

  • In a research paper, the author explains to an interested reader

    • what he or she accomplished

    • how the author accomplished it

    • why the reader should care.


What kinds of questions do software engineers investigate
What kinds of questions do software engineers investigate?

  • Generally speaking, software engineering researchers seek better ways to develop and evaluate software.

    • Development

      • includes all the synthetic activities that involve creating and modifying the software, including the code, design documents, documentation, etc.

    • Evaluation

      • includes all the analytic activities associated with predicting, determining, and estimating properties of the software systems, including both functionality and extra-functional properties such as performance or reliability.


A good research paper need to answer the following
A good research paper need to answer the following:

  • What, precisely, was your contribution?

  • What question did you answer?

  • Why should the reader care?

  • What is your new result?

  • What new knowledge have you contributed that can be used elsewhere?

  • What previous work (yours or someone else’s)do you build on?

  • What do you provide as a superior alternative ?

  • How is your result different from and better than this prior work?

  • What, precisely and in detail, is your new result?

  • Why should the reader believe your result?

  • What standard should be used to evaluate your claim?


What is your new result
What is your new result?

  • Explain precisely what you have contributed to the store of software engineering knowledge.

  • how this is useful beyond your own project?


About survey paper
About survey paper

  • Example of abstract for survey paper:

    • In this report (or paper), we survey a number of X-based representations/algorithms/ tool support used to generate, identify, execute, etc.

    • Many problems with references and citation

    • Analysis (Comparisons )

  • Examples of citation:


About presentations
About Presentations

  • Each presentation will be 30 minutes long, including 5 minutes for questions/discussion.

  • The presentation will be divided into the following parts

    • Problem definition/motivation

    • What are the challenges?

    • Background literature surveyed by the authors

    • Specific technique developed in this paper

    • Weaknesses of the technique.



  • Each group will need to develop a power-point presentation

  • The presentation will be evaluated using the following criteria

    • Originality

    • Clarity

    • Organization

    • Punctuation


System testing
System testing

  • Objective of unit and integration testing was to ensure that the code implemented the design properly

  • Functional Testing

    • Black-box testing

  • The goal is to Test the Quality + functionalities

  • Works with notion of threads (or scenarios)

  • A thread is a sequence of events (or ASFs)

    • Provides a unifying view of our three level of testing

      • e.g.,

        • A scenario of normal usage

        • A stimulus/response pair

        • A sequence of machine instructions

        • A sequences of ASFs

  • Identifying threads?

    • FSM (node/edge coverage metrics)

      • Top/down

      • Bottom/up


System testing techniques
System testing techniques

  • System testing attempts to validate system-wide behaviors and properties of the entire system

    • E.g.,

      • performance,

      • security,

      • recovery, etc.