Loading in 2 Seconds...
Loading in 2 Seconds...
Automated Testing of Software Components Based on Algebraic Specifications -- Method, Tool And Experiments. Hong Zhu Dept. of Computing and Electronics, Oxford Brookes University, Oxford, OX33 1HX, UK Email: email@example.com. Outline. Background Motivation Related works
Dept. of Computing and Electronics,
Oxford Brookes University,
Oxford, OX33 1HX, UK
zero: -> nat;
succ: nat -> nat;
succ(x) = succ(y)
Algebraic specification (AS) emerged in the 1970s. In the past three decades, it has developed into a mature formal method.
By substituting constants into variables, we can generate test cases
S: stack, n: integer,
S.push(n).height() = S.height()+1
Checking the equivalence between the values of the left and right hand sides is to check the correctness of test results
A ground term corresponds to a sequence of procedure/ method/ operation calls
import Int, String;
push: Stack,Int ->Stack;
S: Stack; n: Int; x: String;
1: create(x).getId() = x;
2: findByPrimaryKey(x).getId() = x;
3: create(x).height() = 0;
4: S.push(n) = S;
if S.height() = 10;
5: S.pop() = S; if S.height() = 0;
6: S.push(n).pop() = S;
if S.height() < 10;
7: S.push(n).top() = n;
if S.height() < 10;
8: S.push(n).height() =S.height()+1;
if S.height() < 10;
9: S.pop().height() = S.height()-1;
if S.height() >0;
endCASOCC specification language
Definition 1. (Observable sort)
In an AS <S, E>, a sort s is an observable sort implies that there is an operation _ == _ : ssBool such that for all ground terms t and t’ of sort s,
E|-( (t == t’) = true) E |- ( t=t’ ).
An algebra A (i.e. a software entity) is a correct implementation of an observable sort s if for all ground terms t and t’ of sort s,
A |= (t=t’) A|= ( (t == t’) = true)
Pre-defined sorts of Java primitive classes and data types are observable.
Let U be a set of specification units in CASOCC and S be a set of sorts. For each sort sS, there is a unit UsU that specifies the software entity corresponding to sort s.
Let be the importation relation on S.
Definition 2. (Well founded specifications)
A sort sS is well founded if s is observable, or for all s’ in the import list of Us, s’ is an observable sort, or s’ is well founded.
A specification U is well founded if and only if the importation relation is a pre-order on the set S of sorts, and all sorts sS are well founded.
Definition 3. (Well structured specifications)
A specification U in CASOCC is well structured if it satisfies the following conditions.
(1) It is well founded;
(2) For every user defined unit Us U,
(a) there is at least one observer in Us;
(b) for every axiom E in Us, if the condition contains an equation t=t’, we must have, where s’ is the sort of terms t and t’.
A practice implication: for all sorts there are finite lengths of observable contexts.
Definition 1. (Observation context)
A context of a sort c is a term C with one occurrence of a special variable of sort c. The value of a term t of sort c in the context of C, written as C[t], is the term obtained by substituting t into the special variable .
An observation contextoc of sort c is a context of sort c and the sort of the term oc is. To be consistent on notations, we write _.oc: cs to denote an observation context oc.
An observation context is primitive if s is an observable sort. In such cases, we also say that the observation context is observable and call the context observable context for short.
written T1=T2, [if C]
Definition 2. (Checkable test cases)
A test case T1=T2, [if C]is directly checkable (or simply checkable), if and only if
(a) the sort of terms T1 and T2 is observable, and
(b) the sort of equations in C is observable, if any.
Spec s: CASOCC specification unit of the main sort;
Sigs s1, s2, …, sk: The signature of imported sorts;
TC: A subset of axioms in s (* the axioms to be tested *);
vc: Integer (*complexity upper bound of variables*);
oc: Integer (*complexity upper bound of observation contexts*) ;
rc: Integer (* the number of random values to be assigned to variables of primitive sorts*)
Step 1: Initialisation
Step 2: Generate normal form terms for non-primitive variables
Step 3: Generate random values for primitive variables
Step 4: Substitute normal forms into axioms
Step 5: Substitute random values into test cases
Step 6: Compose test case with observation context
Step 7: Output test set
Theorem 1. The test case generation algorithm will always terminate if the specification is well founded.
Theorem 2. The test cases generated are checkable, i.e. for all test cases <t1=t2; if c> generated by the algorithm, t1, t2 and c are of primitive or observable sorts.
Theorem 3. The test cases are valid. That is, if the specification is well-structured and the observable sorts satisfy the constraints in Definition 1, we have the following properties.
(a) The program correctly implements the specification with respect to the behavioural semantics of algebraic specifications implies that the evaluation of t1 and t2 using the program give equivalent results provided that c is evaluated to be true.
(b) If the evaluation of t1 and t2 gives non-equivalent values in an implementation when c is evaluated to true, then there are faults in the program.
This statement is supported by T-Test.
s,v. (s.setX(v).getX = v), Ifpre-condsetX(v)
s,v. (s.setX(v).getY = s.getY),
where X Y.
s,v. (s.[getX] = s),
s,x. (s.F(x).getX) = f(x, s.getX), if pre-condF(x),
s,x,y. (s. [A.P (x)].B = s.B.Q(y)), if pre-condA.P(x)
Table 1. Statistic Data of Student Capability
The time taken to write the algebraic specification excludes the time to understand the components.
 周长发, Java数值计算算法编程，电子工业出版社，2007.
 Bodoff, S. et al. 2004. The J2EE Tutorial, 2nd Edt., Pearson 2004.
Is writing algebraic specifications learnable?
The students’ learning experience is not hard. They attained the knowledge and skill of algebraic specification in just a few lessons.
How expensive to write an algebraic specification for a software component?
On average a student took about half hour to complete the writing of an algebraic specification for a typical software component.
Does writing algebraic specification need good mathematical skills?
We divide the students into the following four groups and calculated their scores in class tests.
Is writing algebraic formal specifications a job only for the most capable?
Writing algebraic specification must be the job of the most capable programmers. However, there is a potential bias.
The average scores and average times contain the results of the first and second class tests. Thus, they do not reflect the situation after the students completed their training.
y = -0.0002x2 - 0.0148x + 98.405 (3)
(b) Relationship between final class test times and programming capability
y = 0.0017x2 - 0.4558x + 56.043 (4)
After taking three lessons and class tests, the students are capable of writing algebraic specifications of almost equal quality, but the most capable ones took slightly less time.
Writing algebraic specifications can be a job for any well trained software developer rather than just for the few most capable ones.
The conclusions are only applicable to writing algebraic formal specifications.
A notable advantage of algebraic specification is that the syntax and semantics of axioms are simple and easy to understand. They use little mathematics notations.
The conclusions are only applicable to writing formal specification
These activities may well require much deeper understanding of the theories of formal methods and the semantics of formal specification languages. They may also rely on skills of using software tools that supports formal methods.
Gonnon, J., McMullin, P. and Hamlet, R., Data-Abstraction Implementation, Specification and Testing, ACM TOPLAS 3(3), 1981, 211-223.
Bernot, G., Gaudel, M. C., and Marre, B., Software testing based on formal specifications: a theory and a tool, Software Engineering Journal, Nov. 1991, 387- 405.
Doong,K. & Frankl, P., The ASTOOT approach to testing object-oriented programs, ACM TSEM3(2),1994, 101-130
Hughes, M. and Stotts, D., Daistish: systematic algebraic testing for OO programs in the presence of side-effects. ISSTA’96, 53-61.
Chen, H.Y., et al., In black and white: an integrated approach to class-level testing of object-oriented programs, ACM TSEM 7(3), 1998, 250-295.
Chen,H.Y., Tse,T.H. & Chen,T.Y., TACCLE: a methodology for object-oriented software testing at the class and cluster levels, ACM TSEM 10(1), 2001, 56-109.