using software architecture for code testing l.
Skip this Video
Loading SlideShow in 5 Seconds..
Using Software Architecture For Code Testing PowerPoint Presentation
Download Presentation
Using Software Architecture For Code Testing

Loading in 2 Seconds...

play fullscreen
1 / 55

Using Software Architecture For Code Testing - PowerPoint PPT Presentation

  • Uploaded on

Using Software Architecture For Code Testing Written By: Henry Muccini Antonio Bertolino Paola Inverardi Presented By: yong wang Introduction SA is emerging as a promising approach as it is used to design and analyze complex distributed system.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Using Software Architecture For Code Testing' - Leo

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
using software architecture for code testing

Using Software Architecture For Code Testing

Written By:

Henry Muccini Antonio Bertolino

Paola Inverardi

Presented By: yong wang

  • SA is emerging as a promising approach as it is used to design and analyze complex distributed system.
  • Using suitable abstraction it makes large application manageable.
  • SA support formal modeling of a system with topologic description and behavioral one in term of component and connector
  • Crucial part of development process is testing. It can be used to test that the dynamic behavior of program by observing the execution of the program on selected set of test case.
  • Specification-based testing checks the implementation Under Test(IUT) fulfils specifications, to test the important properties against which the ITU has to be tested.
  • This paper explains how complex systems dynamic behavior and specifications can be tested using SA.
goal approach
Goal & approach
  • For complex system, it is impossible to test all class.
  • The goal of approach is to provide a test manager with a systematic method to extract suitable test classes for higher level of testing and to refine them into concrete test at the code level.
  • It is based on specification of SA dynamics which is used to identify useful schemes of interaction between system components and to select test classes corresponding to relevant architectural behaviors.
  • SA dynamics are modeled by Labeled Transition System (LTS).
  • The approach consist of deriving suitable LTS abstractions, called Abstract LTS or ALTS. ALTS offer specific views of SA dynamics by concentrating on relevant features and abstracting away from uninteresting ones.
  • ALTS is a tool to understand what must be tested. It represents a model of the system
overview of approach to sa testing
Overview Of Approach to SA Testing
  • Four Steps

1. Software Architect looks at SA dynamics from different viewpoints, defines various obs-functions over SA model each one highlighting a specific perspective of interest for a test session.

2. Applying each obs-function to LTS, an Abstract LTS (ALTS) can be generated which is reduced LTS showing only interested behaviors with respect to selected views.

overview of approach to sa testing7
Overview Of Approach to SA Testing

3. The software architect chooses a set of important patterns and behavior (path over ALTS) to be tested.

4. Finally these high level tests are passed to the software tester, who actually has to run the test and observe whether the current implementation “conforms” to its architectural model. deals with the execution of these tests at the code level

overview of approach to sa testing8
Overview Of Approach to SA Testing
  • Steps 1 to 3 presents a rigorous method to extract architectural tests from an SA specification
  • Step 4 deals with the execution of these tests at the code level
definition labeled transition system lts
Definition:Labeled Transition System (LTS)
  • LTS consist of (S, L, S0, Sf, T)

S = set of states

L = set of distinguished labels (actions)

denoting the LTS alphabet.

S0=initial state.

Sf= Final State.

T = Transition Relation labeled with

elements of L.

case study the trmcs
Case Study: The TRMCS
  • The Teleservice and Remote Medical Case System (TRMCS) is a system that provides monitoring and assistance to disabled or elderly people.
  • A typical service is to send relevant information to a local phone center so that the family, along with those concerned with medical or technical assistance, can be timely notified of critical circumstances.
case study the trmcs arch
Case Study: The TRMCS(arch)

Box=components; arrow = connectors

Arrow label = data elements exchange

through channel

case study the trmcs12
Case Study: The TRMCS
  • Four components:

1. USER send either “alarm” or “check” message to the router process. Then he/she waits for acknowledgement from the router. In this case two users can concurrently send alarms and checks.

2. ROUTER waits for signals (check or alarm) from the user. It forwards the alarm message to Server and monitors the state of the User via check message.

3. SERVER dispatches the help requests.

4. TIMER sends a clock signal at each time unit.

the approach
  • 4 steps of our approach to SA-based testing.
  • STEP 1 Obs-Functions Definition
  • STEP 2 Abstract LTS Derivation
  • STEP 3 Test Selection
  • STEP 4 Code-Level Testing
step 1 obs functions defintion
Step 1: Obs-functions defintion


SA---> Labeled Transition System models SA dynamics. LTS provides a global, monolithic description. Vast information flattened into graphic. Extracting relevant for validation is difficult task.


provide software architect with a key to decipher the LTS.

Common practice is to derive a set of simplified models from global SA model which will provide different system views.

step 1 obs functions defintion15
Step 1: Obs-functions defintion
  • Ideal

Allow for the formal derivation from the LTS of reference models for testing, each representing a relevant pattern of behavior.


Test ing criterion: each of the selected patterns of behavior. It divide LTS action L into two groups:

Relevant Interaction R and Non Relevant Interaction NR. R U NR=L, R ∩ NR=null.

step 1 obs functions defintion16
Step 1: Obs-functions defintion
  • Using obs-function map the relevant interactions to domain D and Non relevant interactions to τ
  • Obs function can be considered as hiding operator as it hides non relevant information

Obs: LL’, obs(r ∈R)=d ∈D, obs(n ∈NR)=τ, L’=DU τ

LTS paths

If p = I1I2…..In

: obs(p)=obs(I1I2…..In)=obs(I1)obs(I2)…..obs(In)

We therefore have SA testing criterion an obs-function that maps the relevant LTS label.

step 2 abstract lts derivation
STEP 2 Abstract LTS Derivation
  • We need a small automaton from the LTS which still expresses all high level behavior. ALTS.
  • The ALTS is obtained by two transformations:
  • By relabeling each transition τ∈ LTS in the LTS transition according to obs-function.
  • By minimizing the resulting automaton with respect to a selected equivalence relation.

Trace-equivalence: reduce much number of τ transitions and nodes

Bisimulation equivalence: system evolves step by step

step 2 abstract lts derivation18
STEP 2 Abstract LTS Derivation

The abstraction is applied over this LTS with respect to selected obs-function. (LTS)

The trace equivalence minimization function is applied.


c) The resulting ALTS is shown in Fig

(trace-based ALTS)

This figure represent branch minimization.

(bisimulation-based ALTS)

step 3 test selection
STEP 3: Test Selection
  • Once the ALTS is derived we move to deriving a set of complete paths appropriately covering the ALTS.
  • Now ALTS have less paths the tester might think to apply extensive coverage criteria on it.
  • But ALTS corresponds to many possible LTS paths.
  • Therefore less thorough coverage criteria seems more practical
  • Use McCabe’s test technique as it is a good compromise between arc and path coverage in the case of ALTS coverage
step 3 test selection20
STEP 3: Test Selection
  • When considering what to take as the specification of an “architectural test” we have two options.
  • consider an ALTS complete path as the test specification. In this case, the test is specified at a more abstract level and tester instinctively focuses on testing restricted set of interactions(use)

2. identify those LTS paths of which the selected ALTS path is an abstraction. As LTS is more detailed than ALTS in this case tester will have more information about how to perform tests, but stricter requirements i.e. tester does not have much freedom in choosing the code level tests.

step 4 code level testing
STEP 4 Code-Level Testing
  • In this step tester use architectural paths from step 3 to test the implementation. Two problems are encountered while doing this

1. TRACEABILITY- concerns “relating the abstract values of the specification to concrete values of the implementation”.

2. TEST execution– entails forcing the Implementation Under Test (IUT) to execute the specific sequence of events that has been selected. This is a difficult as non-determinism is introduced.

step 4 code level testing22
STEP 4 Code-Level Testing
  • Step 4.1

Map architectural components and actions into their

low-level implementation. analyze the system implementation to understand how architectural actions have been implemented in the code by sequences of partially ordered method calls.

  • Step 4.2

Take into account how an ordered sequence of actions is implemented by a sequence of low-level functions. Map actions into sequences of method calls. Run sequentially and run concurrently.

step 4 code level testing23
STEP 4 Code-Level Testing
  • Step 4.3

Run code and evaluate execution traces with respect to the expected ones to analyze the source code conformance with respect to the architectural behavior.

Nondeterministic testing: repeat the launching of a program run under some specified input conditions several times until the sequence is observed.

applying the approach
Applying the Approach
  • TRMCS components behavior were specified by using Finite State Process(FSP)language
  • The behavior of each component is modeled by one or more FSP processes, and each process is described by an LTS which is automatically generated by Labeled Transition System Analyzer (LTSA) tools
applying the approach25
Applying the Approach
  • By running the LTSA tool on the TRMCS FSP specification
    • 256 states of LTS
  • The LTS labels of interest for analysis are:
    • u[i].sendAlarm_To_Router: an alarm message is sent by the

user to the Router

    • u[i].receiveAck_From_Router: the Router Acknowledgment

messages received by the user

    • r.[i].sendAlarm_To_Server: the Router forwards the Alarm

message to the Server

    • sa[0].sendAck_To_Router: the Server sends the Ack to the Router
    • r.sendNoFunc_To_Server: the Router sends the NoFunction

message to the Server

applying step 1
Applying step 1
  • Informal specification:

Test the flow of an Alarm message from the moment a User sends it to the moment the User receives an acknowledgment.

  • Corresponding obs-function: AlarmObs:
applying step 127
Applying step 1

All the interaction that involve this component

FR{a1,a2,no} messages the servers receivers from the Router

TR{ack1,ack2} message sent to the router

ServerRegression Obs-Function

applying step 3
Applying Step 3
  • After the reduction and minimization algorithms
    • Shaded circle represents the initial state
    • After the Alarm is issued, the system reacted with one of the possible actions:
      • Elaborate the Alarm and send back an Ack, (Alarm1Dispatch)
      • Receive another message from another user, (Alarm2Dispatch)
applying step 329
Applying Step 3
  • List of test paths derived according to McCabe’s coverage criterion follow corresponding test sequences

With reference to the ServerRegression criterion, Fig 4b is obtained

  • 4b shows the interactions involving the Server component
  • Receives Alarm1 (FRa1) or Alarm2 (FRa2) from the Router
  • Then, receive other Alarms or send back the Ack (Track)
  • It can also receive the NoFunc message from the Router (FRno)
  • Parts 1b, 3b, 7b, and 8b are verified that the NoFunc message (FRno in fig. 4b) can b received by the Server
applying step 4 1
Applying Step 4.1:
  • Test sequence generated from AlarmObs ALTS has 3 test components (User1, User2, and Router)
  • Two Architectural events are
    • AlarmDispatch and AckDispatch
    • Understand how the source code implements these architectural actions
      • AlarmDispatch – represents a system input
        • E.g. user pushes the Alarm button of the device
        • The device will start by analyzing the actions associating with the pushing the alarm button
        • Then, the Alarm functionality implements


    • The Alarm message is written into a socket
    • The Router periodically reads the socket for new messages
    • The Ack function is implemented
    • The Router writes into a socket and the user waits on a defined socket port

Objects and methods implements the Alarm

    • Fig. 5a: the User object creates the User Object
      • If_1 condition is verified
      • Afterward, (if_2), User presses the Alarm button
      • User then creates a ClientConnection
      • SendNMessages (SNM) objects
      • User calls the send() method of the SNM object
      • It then tries to create the socket
      • If the socket is created (if_3), SNM finally writes the Alarm message into the socket

From the Router side

  • Router graphical interface object (xRouter) creates a new instance of the MasterRouter object
  • In turn creates the ServerConnection object
  • Tries to create a ServerSocket
  • A process is then activated
  • Continuously checks the ServerSocket port
  • A new message is written in the socket (if_b) the socket is read and the Alarm is received
applying step 4 2
Applying step 4.2
  • Analyze how an architectural test(sequence of actions) is implemented by the code.
  • TS3a:Alarm1Dispatch.Ack1Dispatch.Alarm2Dispatch.Ack2Dispatch. Alarm1Dispatch happens before Ack1Dispatch.
  • Code level two option:
  • Code-level sequences implements AlarmiDispatch and AckiDispatch run sequentially.
  • Low-level scenarios can be run concurrently.

Method call may interleave.

applying step4 3
Applying step4.3
  • Execute the test sequence and verify the system implementation behaves as described by our SA test.
  • According to TRMCS implementation P, build an instrumented version P’ to store information about the execution of SM(synchronization method). Introduce the “synchronization Method” file. when an SM is called, a print operation is performed in this file which capture the following information:<SM ID, User ID, SM instance, time>
  • Running P’ several times with difference input order,

Get file report. Each alarm operation is always immediately followed by the relative acknowledgment.

some considerations
Some Considerations
  • Difficulty of Application

Step 1.Obs-functions: is an empirical task based on a SA’s experience

Tasks could be made easier through classification of observations

By assigning semantics to the messages exchanges

Step 2.Deriving ALTS: ALTS generation can be automated

Step 3. Test Classes Selection: coverage criteria like McCabe’s test technique could be automated easily

– extracting only paths that expose different behaviors


Step 4:Code-level Testing:

Testers used to generate test cases

Used to trace high-level info into code-level tests

4 Issues

need to understanding which classes and methods that implement an architectural functionality

More than one sequence can implement the desired behavior

  • This can be handled by requiring the intervention of the tester who manually evaluates each point of decision
  • And, performs a reasoned choice regarding which selection is most relevant for testing
  • Interactions are considered in SA description, not computations
  • Expected behaviors are described only
  • But, the implementation has to handle the exceptional behaviors
b approach generality
B. Approach Generality
  • Approaches are put into practice as well as independently
  • ADL is selected to describe the SA dynamics through LTS
  • Coverage criteria is selected to extract a limited number of ALTS paths
  • Implementation is used following the OO paradigm
  • SA dynamics is described via an LTS conforming the given definition
  • Different ADLs can be used to describe the SA
  • Different ALTS coverage criteria can be applied
  • Different implementation languages and programming paradigms can be used
related work
Related Work

Approach which is presented makes use of the following concepts.

Testing concurrent and real time systems

Conformance based testing

Tracing information

Architectural testing

related work cont
Related Work Cont
  • Most of the work has been done on Testing Concurrent and Real Time, both are specification and implementation based with focus on unit testing.
  • Objective was to derive test plans for integration testing using the LTS , abstraction and event sequences used in different way.
  • Approach to defining ALTS path for high-level test classes is same as Carver and Tai ( Syn-sequence) but constraint to specification based testing of concurrent programs.
related work cont44
Related Work Cont.
  • ALTS path for SA restricts the event of a concurrent program when test are selected, the main difference is in the distance of abstraction level of the reference model from the implementation.
  • Selection of a test suit based on specification of implementation under test, and which is assumed to be given in form of Finite State Machine (FSM)
  • Transition tour, W-method, DS-method, UIO method and UIO methods were proposed
related work cont45
Related Work Cont.
  • TAG (tan et al) test generation tool based on FSM and generated by given specifications but the draw backs of this tool were due to use of FSM with limited number of states and impossibility of ensuring complete coverage.
  • Software system using statecharts: natural language scenarios were designed through statechart explaining information on test case generation, thus enabling test cases generation through path traversal.
related work cont46
Related Work Cont.
  • Bogdanov presented a method to extract test cases from state charts to prove the implementation is behaviorally equivalent to design.
  • Das-Boot tool were also presented to generate test suit for OO (object oriented) systems through statechart. 
  • Studies have carried out by Fujiwara, Bochmann and others to generate automatically test suits to test the conformance of an implemented system.
  • Petrenko uses FSM to drive a conformance testing, where system specification (S) its implementation (I) are formalized using FSMs to define “Conformance relations” (equivalence, quasi-equivalence, reduction) and Wp-method is used to generate test sequences.
related work cont47
Related Work Cont.
  • Tretmans presented LTS-based conformance testing, specification (S) using LTSs, implementation (I) using LTSs or IOTS (Input/output Transition System) and tests (T) were formalized using LTSs: Given S and possible implementation I.

IMP (Implementation relations) to correlate S with I where I conforms to S iff I is “imp”with respect to S.

  • Fernandez et al. formalize the specification, implementations and test purpose by using IOLTS (input/output LTS). Automatic, on-the-fly generations of test cases were projected in form of TGV (test generation and verification) environment.
related work cont48
Related Work Cont.
  • They also used the SA-derived LTS as reference model but in different way. Implementation relations (conf, ioconf, ioco) were also defined and test whether the implementation is correct with respect to specification, based on the models described earlier.
  • Egyed proposed the “Tracing Information” where showed the way to detect traceability between software systems and their models, including some techniques.
  • Work has also been done in bridging the gap between software architectures and its requirements.
related work cont49
Related Work Cont.
  • The Architecture Testing has become more interesting and the authors have defined six criteria for architecture-based testing, adapting and specification based approaches.
  • They also analyzed the advantage of SA-level testing for reuse test and test further functional properties.
  • Harrold presented the approaches for using software architecture for effective regression testing.
  • She also proposed the architecture-based integration testing approach, which includes architecture traceability, simulation and slicing.
  • The research has been carried out on SA philosophy, methods and tools can be useful within software development process
  • Mainly targeted the using of SA descriptions to improve conformance testing of a large system.
  • This literature was heavily relied upon the Software Architecture and Specification based testing.
  • Includes tools for design and analysis but very little had been said about SA- based testing.
  • There are many problems exists due to the practicality of using SA models and tools despite the claim that SA must play a role throughout the software life cycle.
  • Relationship between high level architecture tests cases and the implementation under test may be very difficult and still there is no complete solution exists.
  • In specification based testing, methods and tools would be same but there are two main differences between proposed approach and existing ones.
  • The SA explanation confined to SA-relevant behaviors only and not focusing on system functions. This approach does not test the system completely as in traditional specification based testing. The proposed approach concentrates on integration testing stages.
  • The abstraction level of the referenced model and its relative distance is very high from the implementation under test in SA based testing approach.
  • While in existing approach it is assumed to be low. The high abstraction level is a basic assumption of this study. Where as the traditional approaches have a close distance between the reference model and implementation
  • Extract is being presented from the practical insight gained while performing SA based testing on the TRMC case study.
  • It is believed that in the most cases problems faced and the solutions found can easily be generalized to any SA environment, although the experience reported may be specific to the case study.
future work
future Work
  • It is considered that the approach presented here as a initial contribution that might not apply directly in practice, for instance it might not have global architecture model because of architecture may be described through complex models.
  • Secondly architecture model may be incomplete, which means that some component behaviors are unknown or components are not completely specified.
  • Studies are currently investigating the possibilities of generating abstract observation and test cases directly from partial architecture models.
future work cont
Future Work Cont.
  • Secondly how to further stricken the assumptions on SA based development process, it is showed in reference that how the testing process can be made completely systematic by refining general approach to more specific scenario. 
  • By adopting style architecture and related framework, they were able to handle the traceability among SA.
  • Adapting the category partition method solved the code problems.
  • And execution over the identified test cases problems through Argus-1 tool. This approach has been applied to a case study, which is generating interesting results