PS3: Acceptance Test

1. PS3: Acceptance Test An example Based on Testing Extreme Programming By Lisa Crispin, Tip House

2. Goal • Check the functionality of the entire system as specified by the project's customer. • Acceptance tests should not include specific knowledge about the internals of the system and are said to be "black box" tests. • These tests touch the system only at predefined APIs or GUIs. • To be effective, system-level tests often require large amounts of domain-specific data. • The customer's involvement in creating this data and specifying system behavior is critical. Software Engineering Spring 2011

3. Example Application: Seller Reputation Software Engineering Spring 2011

4. Rate a transaction Select a seller to rate from a list of sellers Select a transaction to rate Rate the transaction Accurate description Communication Quick shipping Reasonable shipping charges Software Engineering Spring 2011

5. Major phases • High level Acceptance Tests • Write stories • Define some acceptance tests for each story • Find Hidden Assumptions in the stories • Customer’s view; User’s view; Programmer’s view • Add High level Acceptance Tests based on these assumptions • Obtain Data for testing • Bad, sad, happy scenarios • Write executable tests • Write code for each acceptance test defined • Bridge the code to the application Software Engineering Spring 2011

6. High Level Acceptance Test • Story I:Rate Transaction Story • The user can provide a rating for a transaction he took part in, expressing the degree to which he is satisfied with it in the following properties: Accurate description, Communication, Quick shipping, Reasonable shipping charges, and textual comment. • Define high-level acceptance tests for the Story. Software Engineering Spring 2011

7. Obtaining hidden Assumptions Story Software Engineering Spring 2011

8. Step1 – Customer view • I'm the customer. • We need a way to track performance of sellers in our system in order to compute the reputation of this seller. • For this purpose we need the users that interacted with this seller to rate the seller by 4 different aspects. • We don’t want users that did not have experience with the seller to rate it – so only user who interacted with the seller may provide ratings for him. • We don’t want the user to provide multiple good ratings or bad ratings – so we allow only one rating per transaction. • What business problem is it solving? How does it solve it? • Users would like to know a seller reputation before initiating a transaction with him. • By collecting users rating we use some Trust/Reputation algorithm to infer the seller’s reputation. Software Engineering Spring 2011

9. Step 2: User view • I'm a user, I want to provide a rating for a transaction I made • What are the worst and best things that can happen? • What would really irritate me? • Worst thing: • I can't locate the transaction I want to rate. It's in the system, but I can't find it. • Two rating properties appear to have a similar meaning —I can't tell the difference. • Best thing: • I can see all the transactions I have performed in the system sorted by date, seller rating –state (rated/unrated). • I can see the rating I gave for the rated ones . • I can also select an unrated transaction form my list and rate it. • How can I screw up? How should the system respond? • Can I accidentally rate the same transaction more than once? • Can I rate a seller that I didn’t interact with? • Can I change my opinion – change the rating I already gave a seller upon completion of a transaction? Software Engineering Spring 2011

10. Step 3: Programmer view • I'm the programmer. • What's the simplest implementation that could possibly work? • A list of all transactions each indicated as rated or not. • Click on an unrated transaction to select it and to rate it. • Click on a rated transaction impossible. • OR? • We need some way to identify a transaction • Do we? • by a number? Software Engineering Spring 2011

11. Step 4: Tester’s view • How likely is the implementation to solve the business problem? • Fairly likely.( under honest and cooperating users assumptions) • How likely is the implementation to solve the related problems? • Fairly likely. • How likely is the implementation to avoid irritating the user? • We need a way to indicate the user that • no more than one rating is allowed per transaction • There is no option to modify a rating already provided • The value they can provide is within a predefined range Software Engineering Spring 2011

12. Hidden assumptions • Identify problematic hidden assumptions in the story • The system provides a list of the user’s transactions • The user can view rated transactions. • The user cannot edit rated transaction. • All four properties are required fields. • The rating values range within a known scale. • Textual comments are optional • ? Software Engineering Spring 2011

13. Additional tests (based on the assumptions) • What other acceptance test can be added according to the assumptions? Software Engineering Spring 2011

14. Writing Executable Acceptance tests • The rateTransStory. • High-level acceptance tests for this story: • Rating an unrated transactions using correct values: succeeds – the transaction properties are updated, transaction is closed • Rating an unrated transactions using out-of –range values: fails– the transaction properties are not updated, transaction remains opened. • Data: • Users : Bob, Clair • Sellers: AutoShop, Buy4Cheap • Transactions: Bob, AutoShop - unrated transaction 1223, Clair, Buy4Cheap– rated transaction 1226 • This captures the essence of the test but leaves out the details. The details can go into an executable test, as follows: • assertTrue(rateTrans(1223,1,2,3,4,”bad experience”) ) • assertTrue(rateTrans(1223,1,2,3,4,””) ) • assertFalse(rateTrans(1223,7,2,3,4,”Called right on time”) ) • assertFalse(rateTrans(1226,3,2,3,4,”unreliable”) ) • assertFalse(rateTrans (1227,1,2,3,4,”blabla”) - Can blabla cause the test failure? Software Engineering Spring 2011

15. Sampled Test Data Table 16.3 • Defined with the customer • Happy, sad, bad… Software Engineering Spring 2011

16. Testing Method per Action • For each action of the high level acceptance test– write a testing method: public class rateTransStoryTest { public void testRateTrans() { // Test data and assumptions for action Rate go here } public void testListTrans() { // Test data and assumptions for action List go here } } Software Engineering Spring 2011

17. Testing Method per Action public class rateTransStoryTest extends SellerReputationTest{ public rateTransStoryTest(String name) {super(name); }\\ A pass-thru constructor public void testRateTrans() { assertTrue(rateTrans(“1223”,1,2,3,4,”blabla”) ) ; assertTrue(rateTrans(“1223”,1,2,3,4,””) ) ; assertFalse(rateTrans(“1223”,7,2,3,4,”blabla”) ) ; assertFalse(rateTrans(“1226”,3,2,3,4,”blabla”) ) ; assertFalse(rateTrans (“1227”,1,2,3,4,”blabla”); } public void testListTrans() { … } } rateTrans(<T_ID>, <score1>, <score2>, <score3>, <score4><comment>) should be bridged to the real application Software Engineering Spring 2011

18. Bridge by inheritance import junit.framework.*; //Define an application-test class that extends JUnit' s TestCase import SellerReputation.*; public class SellerReputationTest extends TestCase {// bridge class public SellerReputationTest(String name) {super(name); } public void assertFalse( boolean condition) { assertTrue(!condition); } // Make every story acceptance test a sub-class of the application test class. Public booleanrateTrans(String tId, int att1, int att2, int att3, int att4, string comment){ SellerReputationTrans trans= new SellerReputationTrans(); return trans.rate(tId, att1,att2,att3,att4, comment); } public booleanlistTrans( String uId, String sId) { ……. } } ABridge to the rate operation as implemented in the application Software Engineering Spring 2011

19. Bridge by inheritance • The SellerReputationTrans class in the SellerReputation system represents a transaction. • The SellerReputationTranshas a rate method that sets the transaction rating criteria: Accurate description, Communication, Quick shipping, Reasonable shipping charges and comment. • If the specified values are all valid, it then attempts to update the transaction, if the update is successful it returns true • otherwise - if the update fails or if any of the specified values is invalid, it returns false • The SellerReputationUser class in the SellerReputation system represents a user. • The SellerReputationUserhas a transListmethod that searches for a list of transactions the user had with a specified seller • If the user had interactions with the specified seller, it produces the list of transactions, if the list is nonempty it returns true • otherwise - if the list is empty or if seller id is invalid, it returns false Software Engineering Spring 2011

20. Bridge by inheritance • TestCase (junit ) • assertTrue • assertEqual • … Functionality provided by testing framework, e.g. junit • SellerReputationTest • assertFalse • rateTrans • listTrans • … Project Test Defines the bridge to the application • rateTransStoryTest • testRateTrans • testListTrans • … Actual tests Software Engineering Spring 2011

21. Bridge by Composition • <<Interface >> • SellerReputationBridge • rateT() • listT() • ……. • TestCase (junit ) • setUp() • assertTrue • assertEqual • … bridge real • SellerReputationTest • setUp() • assertFalse • rateTrans { ..bridge..} • listTrans{ ..bridge..} • … implements implements • Proxy Bridge • rateT() • listT() • …….. • Real Bridge • rateT() • listT() • …….. • rateTransStoryTest • testRateTrans{..rateTrans..} • testListTrans{..listTrans..} • … Software Engineering Spring 2011

22. Bridge by Composition How to avoid recompile the tests when replacing bridges ? • SellerReputationTest class receives the bridge as parameter (setter and/or in constructor). • SellerReputationTest class reads configuration file to select the right bridge (in setUp()). • SellerReputationTest class uses external class to receive the bridge (next slide). • Any other methods you can come up with. Software Engineering Spring 2011

23. Bridge by Composition • <<Interface >> • SellerReputationBridge • rateT() • listT() • ……. • TestCase (junit ) • setUp() • assertTrue • assertEqual • … protected void setUp() throws Exception { super.setUp(); … this.bridge=Driver.getBridge(); … } bridge real • SellerReputationTest • setUp() • assertFalse • rateTrans { ..bridge..} • listTrans{ ..bridge..} • … implements implements public static SellerReputationBridgegetBridge() { ProxyBridge bridge=new ProxyBridge (); // add when real bridge is ready bridge.real = new RealBridge (); return bridge; } • Proxy Bridge • rateT() • listT() • …….. • Real Bridge • rateT() • listT() • …….. <<use>> • rateTransStoryTest • testRateTrans{..rateTrans..} • testListTrans{..listTrans..} • … <<use>> <<use>> Driver getBridge() Software Engineering Spring 2011