Executable Requirements with Fitnesse

1. Executable Requirements with Fitnesse October 9, 2008 Ryan Hoegg Kenyatta Clark

2. A working definition of Executable Requirements Some of the challenges of requirements management The impact of requirements failures How Executable Requirements benefit the SDLC The difference between a requirement and a test The theory of devolving all software to tables that describe it Executable Requirements

3. Agile Commons Blog To our dismay, we seemed to have been off the mark in terms of what the client was expecting. The client seemed disappointed and our team was as well. After the review, we conducted a team retrospective and the main topic was "Why did we miss the mark so badly?". It was the right question to ask and sparked a very open discussion. We discovered that we had not adequately uncovered the business cases behind the user stories, that we had a communications gap with our client, and that we didn't do a thorough review of the client's existing application.

4. QA BA PMO Developer(s)‏ DBA Infrastructure

5. Requirements Communication Requirements are viewed differently between groups.

6. Cohesiveness Completeness Correctness Relevancy Testability Usability Specificity Requirements typically lack

7. Testing Requirements Development

8. Interpretation Boundaries “Entree comes with choice of ((salad or soup) and bread)” Requirements Development Testing “Entree comes with choice of (salad or (soup and bread))” “Entree comes with choice of salad or soup and bread”

9. TDD Scrum Project Community XP Tools, techniques, and practices are being used and refined on Agile teams to deal with the chaos.

10. Executable Requirement Development

15. Finding and fixing a software problem after delivery is often 100 times more expensive than finding and fixing it during the requirements and design phase. The Bottom Line Source: Software Defect Reduction Top 10 List Barry Boehm, University of Southern California Agile methods such as Scrum, TDD and Executable requirements move unit and acceptance testing forward in the process.

16. What is the difference between a Requirement and Test?

17. The distinction is generally only one of phrasing and intent. Requirement or Test? As a requirement... The software should eat pink jellybeans and burp purple flowers As a test... Feed software pink jellybeans and verify that it burps purple flowers Input Output

18. Software Requirements as Tables

19. Software requirements as table In the July 2001 issue of ACM Software Engineering Notes, J. P. Lewis published an article where he observes that: “...any software program can be thought of as a table that maps program inputs to their corresponding correct outputs.This characterization of programming assumes a finite number of finite inputs, which is fair for practical purposes”

20. Software requirements as tables Executable requirements are agile user stories expressed as tables that can be directly used for integration and acceptance testing.

21. Criteria for Completion Great Collaboration Tool Source of Feedback Real Data to measure progress Why Acceptance Tests?

22. How often would you press it? When would you press it? Who would press it? Testers Developers Managers Customers Spectators The Button

23. Open source tools for executable requirements Fitnesse and FIT

24. Why Fit ? framework that enables customers testers and programmers to learn what their software should do and to automatically compare that to what it actually does do. What will be covered in this session ? FIT/Fitnesse Introduction

25. Introduction Overview of Fit Fit Tools – Fitnesse Use Fitnesse to write / test FIT tests Fit tables Base Class Fit Fixtures Examples of Fixtures More on Fit Tools - Fitnesse integrations Conclusion References Agenda Agenda

26. How Fit Works ? Fit is the engine that actually runs your tests (originally written in JAVA). Fit works with HTML documents HTML documents include Fit tables Fixtures interpret tables Fit works by parsing HTML, finding tables and passing the information in the tables to fixtures that programmers write. The fixtures take the information from the tables and turn them into method calls on the actual application under test. Fixture Conventions What happens when a Fixture runs? Overview of Fit

27. Running Fit Fit is a command-line application The Standard File Runner takes a single HTML file, processes it, and outputs the result to a different file. Third-Party runners FolderRunner (FitLibrary)‏ Fit Pro (Luxoft)‏ Fit works with Microsoft Word/Excel documents (as HTML)‏ Current language implementations JAVA, .NET, Python, Perl, Smalltalk, Cpp, Ruby http://fit.c2.com/wiki.cgi?DownloadNow Overview of Fit

28. Use Fit to work with Customers !! Not just a testing tool Is really a Collaboration tool Take advantage of the fact that Fit reads/writes HTML Steer UI discussions to Example discussions Help Convert Examples to raw tables reformat tables so they're easy to implement with Fit Be careful not to lose the customer's perspective when you do this. Overview of Fit

29. FitNesse - wiki-based Integrated Development Environment for Fit. a wiki wrapper for FIT (engine). Rather than using Microsoft Word and the command-line to edit and run tests, all tests are created, edited, and run within a Wiki environment. A simple markup language lets you create headings, make text bold, underline, and italic, create bulleted lists, and do other kinds of simple formatting http://www.fitnesse.org/FitNesse.EditingFitNessePages Can we use Fitnesse to run tests written in other languages? an additional component called FitServer allows FitNesse to interface with FIT. Given an implementation of FIT and the corresponding FitServer, FitNesse can execute tests in any language. Fit Tools – Fitnesse

30. Fit Tools – Fitnesse

31. FitServer is executed by FitNesse and will communicate through a socket with FitNesse. The protocol used is described in FitServerProtocol. Current list of all languages with FIT/FitServer implementations. Java - Included in the fitnesse.jar. .NET - Included in the standard FitNesse distribution. C++ - C++ FIT and FitServer Delphi - Delphi FIT and FitServer Python - Python FIT and FitServer Ruby - Ruby FIT and FitServer Smalltalk - Visual Works Smalltalk FIT and FitServer Perl - Perl FitServer http://www.fitnesse.org/FitNesse.DownLoad Fit Tools – Fitnesse

32. Let’s see what we need to know about? FIT tables – different table styles for different purposes Begins with a row that contains the classname of the Fixturecode that will interpret the rest of the table FIT Fixtures – each style of test table has its own style of Fixture code Use Fitnesse to write / test FIT tests

33. Fit tables Benefits of having a tabular form ?? enhances communication, Agility and Balance Helps expressing tests for business rules that define How Calculations are made, e.g. discounts How decisions are made, e.g. whether to send a payment Business processes that are incorporated into a software system, e.g. steps to admit a patient to a hospital structured so that it can also act as automated tests that are run against an application system Fit Tables

34. Different Fit Table styles (Most common)‏ ColumnFixture tables - best for checking rules and calculations rows of data represent inputs and expected outputs 1st row specifies the fixture to be called header row maps to names of public properties of the fixture - inputs vs. outputs body rows for each body row fixture values set for the inputs methods called for outputs ActionFixture tables - best for step by step processing first column maps to methods of the action fixture itself. second column maps to names of additional fixtures (the screens) and methods on them third column maps to arguments or values that are used for checking RowFixture tables - best for checking sets of data for testing queries that should return an exact set of values Columns map to variables and methods Fixture will query to get computed rows that will be checked Expected rows (Parse objects) are checked with computed rows (Domain objects) using bucket sort algorithm Missing and surplus rows are marked Fit Tables

35. Creating Fixtures Fit comes with three CORE fixtures which you extend. These base fixtures do the heavy lifting. http//:Release/Source/fit/Fixture.java When do I use a Column Fixture ? What does it favor? smaller functions that can be setup with a few columns of data and then tested with a few more columns of expected results. implements fields and methods that the column fixture finds by reflection. http//:Release/Source/fit/ColumnFixture.java Base Class Fit Fixtures

36. When do I use a Row Fixture ? What does it favor? gathers domain objects which it organizes and inspects. only fixture that reflects directly on domain objects. fields and methods are like those of the column fixture except that fields are read, not written. Are always sub classes that vary by what rows and kind of rows. getTargetClass() describes the type and query() method actually fetches the rows. http:Release/Source/fit/RowFixture.java Base Class Fit Fixtures

37. When do I use an Action Fixture ? What does it favor? addresses the GUI facing parts of the domain model, without needing the GUI. Is the mapping similar? Only one that does not look at column headings for field and method names. first column names methods of the action fixture itself. second column names additional fixtures (the screens) and methods on them. What are the base class actions? start, enter, press, and check. spawns more fixtures from its start method for each "screen" the model expects. These spawn one or more domain objects as is the practice of the real screens they imitate. http//:Release/Source/fit/ActionFixture.java Base Class Fit Fixtures

38. User story in Rally – Demonstrate that legacy application ‘tt_prod’ (C and embedded SQL) generates and displays the same production report after converting it’s Informix specific SQL to ANSI compliant SQL. Customer’s initial examples – input data to the curses based Menu. Date range, summary/detail indicator, lens type range, etc… Acceptance Criteria for the story – Compare the 2 reports generated by the legacy application before and after the SQL conversion effort. The expected number of differences between the reports is 5. (Hint: The report has the date stamp in 5 places) Express this scenario as table of input data and expected output Simple Column Fixture Example - Essilor (User story to Fit test)‏

39. Initial table (with customer’s inputs)‏ Fit table (with programmer’s/tester’s additions) and Fit friendly formatting How did we put the table on the page? Used Fitnesse Wiki Markup for table (using Fitnesse framework)‏ |!- com.optifacts.fittest.RunTtProd -! | |execution status?| |5| Simple Column Fixture Example - Essilor (User story to Fit test)‏

40. Fit Fixture Simple Column Fixture Example - Essilor (User story to Fit test)‏

41. Inner details

42. User story – Demonstrate that Informix-ACE Reports executed by Informix specific tools (saceprep / sacego) generate output files that are consistent with ones generated by Valtech proposed open-source tool (Aubit). Customer’s initial examples – input data for the 13 ACE Reports Output file generated by ACE Report executables when using Informix tools Acceptance Criteria for the story – Compare the report generated when using Informix specific tools with the one produced by the open-source tool (Aubit). They should result in the same output files. Express this as table of input data and expected output Another Column Fixture Example - Essilor (User story to Fit test)‏

43. Fit table (with programmer’s/tester’s additions) and Fit friendly formatting Another Column Fixture Example - Essilor (User story to Fit test)‏

44. Markup for table (using Fitnesse framework)‏ |!-com.optifacts.fittest.RunAceReport-!| |''ACE report executable''|''arg1''|''arg2''|''arg3''|''arg4''|''report file''|''master file''|''num differences?''| | /progs/colorsp.aubit | | | | |./spool/colorsp.out |colorsp.sacego.out|2| | /progs/colorss.aubit | | | | | ./spool/colorss.out |colorss.sacego.out|21| | /progs/itemized.aubit | | | | | ./spool/itemized.out |itemized.sacego.out|1| | /progs/liteupdp.aubit | | | | | ./spool/liteupdp.out |liteupdp.sacego.out|3| | /progs/liteupds.aubit | | | | | ./spool/liteupds.out |liteupds.sacego.out|79| | /progs/liteupnp.aubit | | | | | ./spool/liteupnp.out |liteupnp.sacego.out|4| | /progs/liteupns.aubit | | | | | ./spool/liteupns.out |liteupns.sacego.out|79| | /progs/plistsp.aubit | | | | | ./spool/plistsp.out |plistsp.sacego.out|1| | /progs/plistss.aubit | | | | | ./spool/plistss.out |plistss.sacego.out|8| | /progs/acct_pl.aubit | 0 | '0 ' | '*' | | ./spool/acct_pl.out |acct_pl.sacego.out.0.0.star|4| | /progs/acct_pl.aubit | 0 | '-5000' | '9999999' | | ./spool/acct_pl.out |acct_pl.sacego.out.0.-5000.9999999|2| | /progs/acct_pl.aubit | 10 | ' 25' | '9999999' | | ./spool/acct_pl.out |acct_pl.sacego.out.10.25.9999999|2| | /progs/acct_pl.aubit | 0 | '~181' | '9999999' | | ./spool/acct_pl.out |acct_pl.sacego.out.10.tilda181.9999999|2| Another Column Fixture Example - Essilor (User story to Fit test)‏

45. Another Column Fixture Example – Essilor (User story to Fit test)‏ • Fit Fixture

46. Music Example: describe a music player as if we are operating it. We will see two kinds of tables in use, action and row. The action tables list things actions to be performed, step-by-step, one row at a time. Some actions will involve browsing for music, selecting songs, displaying details of selected songs, playing songs etc. We will check to be sure we've found the right music using a second kind of table, a row table. In the row table we will simply list each song, one per row, with columns showing different attributes of each song. Action and Row Fixtures Example - building a new Music app

47. The objects Generic fixtures that interpret action words. ActionFixture -- Calls up player screens and pushes their buttons. Realtime -- Adds time oriented commands to ActionFixture. Application specific fixtures that simulate screens and system calls. Browser (a plain Fixture) -- Stand in for the main application screen. Dialog (a plain Fixture) -- Stand in for a modal error dialog. Display ( a RowFixture) -- Search results from the library. Simulator (an Object) -- Simulates system behavior for realtime actions. Domain objects within the application. Music -- A song. The elements of the library. Played by the player. MusicLibrary -- Collects songs with mechanisms for finding them. MusicPlayer -- Plays Music with usual tape recorder style controls. Action and Row Fixtures Example - building a new Music app

48. Action - Browsing Music User story – Demonstrate that the music browser starts up looking at the whole library of songs. [We specify the library so that we know what we are talking about in this example.] Action and Row Fixtures Example - building a new Music app

49. User story – Demonstrate that we can pick songs and see details of our selection as we go. Action and Row Fixtures Example - building a new Music app

50. ActionFixture interprets the words in the first column of previous table. The actions operate on fields and buttons on the Browser screen we started in the first table. The Browser (or what ever other Fixtures we start) interprets the names in the second column. Each name maps to a method of the Browser. The third column provides data that are passed as arguments to Browser methods or compared with Browser method Action and Row Fixtures Example - building a new Music app