Using Tests as Agile Documentation

1. Using Tests as Agile Documentation Brian Button Principal Consultant

2. Introduction • Welcome to my experiment! • Not concrete yet, still taking shape • Trying to prove an idea • In Agile world, there is the claim that the tests are the documentation. • What would it take to make that true???

3. Who is the Audience? • Who is reading the documentation? • Programmers (task oriented) • Just want to get in and get out • Maintainers (holistic understanding) • Need to understand design • Use tests as backup during maintenance • Evaluators (kicking the tires) • Setup, overview, simple tasks

4. My Target Audience • This talk with focus on the Programmers using the library • Most basic need • If this need is not met, tests will not suffice • Also focusing on projects where code is the output

5. What Problem are We Solving?

6. What is the Problem? • We all need documentation • But its expensive to: • Create • Maintain • Written as separate activity by writers or programmers • And it needs to change as requirements change

7. What Can We Do? • Nothing • Limit changes to keep down costs • Embrace Change

8. Status Quo • Don’t change • Accept that documentation costs lots of money • We’re all comfortable with that • But costs keep increasing, budgets keep shrinking, projects keep getting shorter • Probably not the right answer

9. Limit Changes • Let documentation costs lead design decisions • Huh??? • Leads to missed opportunities • After all, we’re building software, not manuals • Also, probably not the way to go

10. The Agile Solution • Embrace changes • Because they’re gonna happen • Adapt documentation process to allow for easy changes • Leverage an existing, changing asset to keep documentation creation and maintenance costs low

11. Unit Tests!!!!

12. Unit Tests?? • Means different things to different people • My definition: • Specific, targeted tests that focus on one path through code under test • Not system-wide or complex transactions • Simple, method-level tests

13. Example – String Immutability public void ConvertingStringToUpperCaseCreatesNewStringInstance() { string lowerCaseString = "brian"; string upperCaseString = lowerCaseString.ToUpper(); Assert.AreEqual("brian", lowerCaseString); Assert.AreEqual("BRIAN", upperCaseString); }

14. Preconditions to Adoption • Assumes Agile Development practices • Must have Unit Tests for Everything • Tests must be run all the time • Tests must pass 100% all the time • Assumes leeway and understanding from audience • It’s gonna be different!

15. Agile Documentation • But if they’re open to it, we can really wow them • We can create • Simple statements of facts • Give simple usage examples • Document all behavior explicitly • Guarantee code and docs are in sync

16. Standard Programming Docs • Documentation for Stack in Visual Studio • Class level • Gives overview, class hierarchy, example • Example takes some effort to read and is incomplete • Peek() • Gives overview, exception, more specific example • Example gets worse as method gets more complex

17. Agile Docs for Stack public class StackBehaviorFixture { public void StackIsEmptyAtCreation() public void StackNotEmptyAfterItemPushed() public void StackChangesBackToEmptyAfterPushAndPop() public void ValuePushedOntoStackIsSameAsValuePoppedOffStack() public void MultipleItemsPushedOntoStackArePoppedOffInReverseOrder() public void PopFromEmptyStackThrowsInvalidOperationException() public void PeekDoesNotRemoveItemFromStack() public void PeekWillReturnItemPushedOnStack() public void PeekWillAlwaysReturnTheSameItem() public void PeekWillAlwaysReturnMostRecentPushedItem() public void PeekOnEmptyStackThrowsInvalidOperationException() public void NullItemsCanBePushedOntoStack() public void PopReturnsNullItemOffStack() public void PeekReturnsNullItemOffStack() }

18. Agile Docs for Stack • Each test is statement of fact • Human readable • Simple usage example • Would a Test List be better?

19. Creating Agile Documentation • Generate a Test List • List of candidate tests to flesh out interface and functionality. • Grows over time as you learn more • Task oriented • Flows from simple to complex • Implement tests

20. Anatomy of a Good Test • Strong names, simplicity, clarity • 3 A’s public void PeekDoesNotRemoveItemFromStack() { Stack stack = new Stack(); stack.Push("foo"); stack.Peek(); Assert.IsFalse(stack.IsEmpty); }

21. What’s Missing? • What versus Why • High-level UML diagrams • Guidance on where to look

22. (Some) Documentation == Necessary Evil • Unit tests tell what not why • For complex classes or subsystems, some level of written documentation is required • But not at same level of detail as before • Overview, design decisions, key UML diagrams • Unlikely to change as details change

23. Test Map • On complex project, there are lots of tests • Key question is how to find right tests easily • Is this harder than with regular documentation? • Lots of paper, web pages, etc, versus lots of test • Creation of a test map would go a long way to helping

24. What is a Test Map • Idea of my own creation • It simply maps from a method being tested to those tests that exercise that method • Creates a manual page that lists methods and has hotlinks to tests that exercise them • Use of good test names will help programmer find right test to look at

25. Test Maps and Automation • Test Map would be hard to keep in sync • Creation should be automated [Test] [TestFor("StackExample.Stack.Push")] [TestFor("StackExample.Stack.Pop")] public void ValuePushedOntoStackIsSameAsValuePoppedOffStack() { Stack stack = new Stack(); stack.Push("fred"); object poppedValue = stack.Pop(); Assert.AreEqual("fred", poppedValue); }

26. Downside of Markups • Has to be maintained manually • Failing test won’t point you to incorrect markups • Error prone • Drives up cost • Lessens value • Where is the balancing point?

27. Conclusion • Tests can replace much written documentation • Some written docs and UML still needed • Remaining documentation less change-prone • Requires right audience and context • Requires different mindset while writing tests