Lessons Learned in Testability

1. Lessons Learned in Testability Scott McMaster Google Kirkland, Washington USA scott.d.mcmaster (at) gmail.com

2. About Me • Software Design Engineer @ Google. • Building high-traffic web frontends and services in Java. • AdWords, Google Code • Ph.D. in Computer Science, U. of Maryland. • Formerly of Amazon.com (2 years), Lockheed Martin (2 years), Microsoft (7 years), and some small startups. • Frequent adjunct professor @ Seattle University, teaching software design, architecture, and OO programming. • Author of technical blog at http://www.scottmcmaster365.com.

3. Testing and Me • Doing automated testing since 1995. • Ph.D. work in test coverage and test suite maintenance. • Champion of numerous unit, system, and performance testing tools and techniques. • Co-founder of WebTestingExplorer open-source automated web testing framework (www.webtestingexplorer.org).

4. Agenda • What is Testability? • Testability Sins • Statics and singletons • Mixing Business and Presentation Logic • Breaking the Law of Demeter • Testability Solutions • Removing singletons. • Asking for Dependencies • Dependency Injection • Mocks and Fakes • Refactoring to UI Patterns

5. Testability: Formal Definition • Wikipedia: “the degree to which a software artifact (i.e. a software system, software module, requirements- or design document) supports testing in a given test context.” http://en.wikipedia.org/wiki/Software_testability

6. Some Aspects of Testability • Controllable: We can put the software in a state to begin testing. • Observable: We can see things going right (or wrong). • Isolatable: We can test classes/modules/systems apart from others. • Automatable: We can write or generate automated tests. • Requires each of the previous three to some degree.

7. Testability: More Practical Definition • Testability is a function of your testing goals. • Our primary goal is to write or generate automated tests. • Therefore, testability is the ease with which we can write: • Unit tests • System tests • End-to-end tests

8. Testers and Testability • At Google, test engineers: • Help ensure that developers build testable software. • Provide guidance to developers on best practices for unit and end-to-end testing. • May participate in refactoring production code for testability.

9. Example: Weather App

10. Weather App Architecture

11. Original Weather App Design

12. Testability Problems? • Can’t test without calling the cloud service. • Slow to run, unstable. • Can’t test any client-side components without loading a browser or browser emulator. • Slow to develop, slow to run, perhaps unstable.

13. Mission #1: Unit Tests for WeatherServiceImpl • Problem: Uses static singleton reference to GlobalWeatherService, can’t be tested in isolation. • Solution: • Eliminate the static singleton. • Pass a mock or stub to the WeatherServiceImpl constructor at test-time.

14. WeatherServiceImpl: Before private static GlobalWeatherServiceservice = new GlobalWeatherService(); public List<String> getCitiesForCountry(String countryName) { try { if (countryName == null || countryName.isEmpty()) { return new ArrayList<String>(); } return service.getCitiesForCountry(countryName); } catch (Exception e) { throw new RuntimeException(e); } } • What if we try to test this in its current form? • GlobalWeatherService gets loaded at classload-time. • This itself could be slow or unstable depending on the implementation. • When we call getCititesForCountry(“China”), a remote web service call gets made. • This remote web service call may: • Fail. • Be really slow. • Not return predictable results. •  Any of these things can make our test “flaky”.

15. Proposed Solution • First we need to get rid of the static singleton. • Then we need something that: • Behaves like GlobalWebService. • Is fast and predictable. • Can be inserted into WeatherServiceImpl at test-time.

16. A Word About Static Methods and Singletons • Never use them! • They are basically global variables (and we’ve all been taught to avoid those). • They are hard to replace with alternative implementations, mocks, and stubs/fakes. • They make automated unit testing extremely difficult.

17. Scott’s Rules About Static Methods and Singletons • Avoid static methods. • For classes that are logically “singleton”, make them non-singleton instances and manage them in a dependency injection container (more on this shortly).

18. Singleton Removal public class WeatherServiceImpl extends RemoteServiceServlet implements WeatherService{ private final GlobalWeatherService service; public WeatherServiceImpl(GlobalWeatherService service) { this.service= service; } ... • Also, make GlobalWeatherService into an interface. • Now we can pass in a special implementation for unit testing. • But we have a big problem…

19. We’ve Broken Our Service! • The servlet container does not understand how to create WeatherServiceImpl anymore. • Its constructor takes a funny parameter. • The solution?

20. Dependency Injection • Can be a little complicated, but here is what you need to know here: • Accept your dependencies, don’t ask for them. • Then your dependencies can be replaced (generally, with simpler implementations) at test time. • In production, your dependencies get inserted by a dependency injection container. • In Java, this is usually Spring or Google Guice.

21. Dependency Injection with Google Guice • Google Guice: A Dependency Injection framework. • When properly set up, it will create your objects and pass them to the appropriate constructors at runtime, freeing you up to do other things with the constructors at test-time. • Setting up Guice is outside the scope of this talk. • This will get you started: http://code.google.com/p/google-guice/wiki/Servlets

22. Fixing WeatherServiceImpl (1) • Configure our servlet to use Guice and tell it about our objects: • When someone asks for a “GlobalWeatherService”, Guice will give it an instance of GlobalWeatherServiceImpl. public class WeatherAppModule extends AbstractModule { @Override protected void configure() { bind(WeatherServiceImpl.class); bind(GlobalWeatherService.class).to(GlobalWeatherServiceImpl.class); } }

23. Fixing WeatherServiceImpl (2) • At runtime, Guice will create our servlet and the object(s) it needs: @Singleton public class WeatherServiceImpl extends RemoteServiceServlet implements WeatherService{ private final GlobalWeatherService service; @Inject public WeatherServiceImpl(GlobalWeatherService service) { this.service= service; } ... • The “@Inject” constructor parameters is how we ask Guice for instances.

24. After Testability Refactoring #1

25. Finally! We Can Test! • But how? • We want to test WeatherServiceImplin isolation. • For GlobalWeatherService, we only care about how it interacts with WeatherServiceImpl. • To create the proper interactions, a mock object is ideal

26. Mock Object Testing • Mock objects simulate real objects in ways specified by the tester. • The mock object framework verifies these interactions occur as expected. • A useful consequence of this: If appropriate, you can verify that an application is not making more remote calls than expected. • Another useful consequence: Mocks make it easy to test exception handling. • Common mocking frameworks (for Java): • Mockito • EasyMock • I will use this.

27. Using Mock Objects • Create a mock object. • Set up expectations: • How we expect the class-under-test to call it. • What we want it to return. • “Replay” the mock. • Invoke the class-under-test. • “Verify” the mock interactions were as-expected.

28. Testing with a Mock Object private GlobalWeatherServiceglobalWeatherService; private WeatherServiceImplweatherService; @Before public void setUp() { globalWeatherService= EasyMock.createMock(GlobalWeatherService.class); weatherService= new WeatherServiceImpl(globalWeatherService); } @Test public void testGetCitiesForCountry_nonEmpty() throws Exception { EasyMock.expect(globalWeatherService.getCitiesForCountry("china")) .andReturn(ImmutableList.of("beijing", "shanghai")); EasyMock.replay(globalWeatherService); List<String> cities = weatherService.getCitiesForCountry("china"); assertEquals(2, cities.size()); assertTrue(cities.contains("beijing")); assertTrue(cities.contains("shanghai")); EasyMock.verify(globalWeatherService); } • Observe: • How we take advantage of the new WeatherServiceImpl constructor. • How we use the mock GlobalWeatherService.

29. Mission #2: Unit Tests for GlobalWeatherService • Problem: Talks to external web service, does non-trivial XML processing that we want to test. • Solution: • Split the remote call from the XML processing. • Wrap external web service in an object with a known interface. • Pass an instance to the GlobalWeatherServiceImpl constructor. • Use dependency injection to create the real object at runtime, use a fake at test-time.

30. After Testability Refactoring #2

31. Fakes vs. Mocks • Mocks • Verifies behavior (expected calls). • Implementation usually generated by a mock object framework. • Often only usable in a single test case. • Often fragile as the implementation changes. • Fakes • Contains a simplified implementation of the real thing (perhaps using static data, an in-memory database, etc.). • Implementation usually generated by hand. • Often reusable across test cases and test suites if carefully designed. • Mocks and Fakes • Either can often be used in a given situation. • But some situations lend themselves to one more than the other.

32. Use a Fake, or Use a Mock? • Problem: Setting up a mock object for GlobalWeatherDataAccess that returns static XML is possible, but ugly (and perhaps not very reusable). • Idea: Create a fake implementation of GlobalWeatherDataAccess. • We can give the fake object the capability to return different XML in different test circumstances.

33. Implementing the Fake Object public class FakeGlobalWeatherDataAccess implements GlobalWeatherDataAccess { // Try http://www.htmlescape.net/javaescape_tool.html to generate these. private static final String CHINA_CITIES = "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<string xmlns=\"http://www.webserviceX.NET\"><NewDataSet>\n <Table>\n <Country>China</Country>\n <City>Beijing</City>\n </Table>\n <Table>\n <Country>China</Country>\n <City>Shanghai</City>\n </Table>\n</NewDataSet></string>"; private static final String BEIJING_WEATHER = "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<string xmlns=\"http://www.webserviceX.NET\">&lt;?xml version=\"1.0\" encoding=\"utf-16\"?&gt;\n&lt;CurrentWeather&gt;\n &lt;Location&gt;Beijing, China (ZBAA) 39-56N 116-17E 55M&lt;/Location&gt;\n &lt;Time&gt;Oct 27, 2012 - 04:00 PM EDT / 2012.10.27 2000 UTC&lt;/Time&gt;\n &lt;Wind&gt; from the N (010 degrees) at 9 MPH (8 KT):0&lt;/Wind&gt;\n &lt;Visibility&gt; greater than 7 mile(s):0&lt;/Visibility&gt;\n &lt;Temperature&gt; 39 F (4 C)&lt;/Temperature&gt;\n &lt;DewPoint&gt; 28 F (-2 C)&lt;/DewPoint&gt;\n &lt;RelativeHumidity&gt; 64%&lt;/RelativeHumidity&gt;\n &lt;Pressure&gt; 30.30 in. Hg (1026 hPa)&lt;/Pressure&gt;\n &lt;Status&gt;Success&lt;/Status&gt;\n&lt;/CurrentWeather&gt;</string>"; private static final String NO_CITIES = "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<string xmlns=\"http://www.webserviceX.NET\"><NewDataSet /></string>"; @Override public String getCitiesForCountryXml(String countryName) throws Exception { if ("china".equals(countryName.toLowerCase())) { return CHINA_CITIES; } return NO_CITIES; } @Override public String getWeatherForCityXml(String countryName, String cityName) throws Exception { return BEIJING_WEATHER; } }

34. Testing with a Fake Object private GlobalWeatherServiceImplglobalWeatherService; private FakeGlobalWeatherDataAccessdataAccess; @Before public void setUp() { dataAccess= new FakeGlobalWeatherDataAccess(); globalWeatherService= new GlobalWeatherServiceImpl(dataAccess); } @Test public void testGetCitiesForCountry_nonEmpty() throws Exception { List<String> cities = globalWeatherService.getCitiesForCountry("china"); assertEquals(2, cities.size()); assertTrue(cities.contains("beijing")); assertTrue(cities.contains("shanghai")); } @Test public void testGetCitiesForCountry_empty() throws Exception { List<String> cities = globalWeatherService.getCitiesForCountry("nowhere"); assertTrue(cities.isEmpty()); } The fake keeps the tests short, simple, and to-the-point!

35. Mission #3: Unit Tests for WeatherHome • Problem: UI and business logic / service calls all mixed together. • The view layer is difficult and slow to instantiate at unit test-time. • But we need to unit test the business logic. • Solution: • Refactor to patterns -- Model-View-Presenter (MVP). • Write unit tests for the Presenter using a mock or stub View.

36. Mixing Business and Presentation @UiHandler("login") void onLogin(ClickEvent e) { weatherService.getWeatherForUser(userName.getText(), new AsyncCallback<Weather>() { @Override public void onFailure(Throwable caught) { Window.alert("oops"); } @Override public voidonSuccess(Weatherweather) { if (weather != null) { fillWeather(weather); unknownUser.setVisible(false); } else { unknownUser.setVisible(true); } } }); } • How NOT to write a UI event handler for maximum testability: • Have tight coupling between the UI event, processing a remote service call, and updating the UI.

37. Model-View-Presenter (MVP) • UI pattern that separates business and presentation logic. • Makes the View easier to modify. • Makes the business logic easier to test by isolating it in the Presenter.

38. Model-View-Presenter Overview

39. Model-View-Presenter Responsibilities • Presenter uses the View interface to manipulate the UI. • View delegates UI event handling back to the Presenter via an event bus or an interface. • Presenter handles all service calls and reading/updating of the Model.

40. Passive View MVP • A particular style of MVP where the View is completely passive, only defining and layout and exposing its widgets for manipulation by the controller. • In practice, you sometimes don’t quite get here, but this is the goal. • Especially if you use this style, you can skip testing the View altogether.

41. After Testability Refactoring #3

42. Presenter Unit Test Using EasyMock @Test public void testOnLogin_unknownUser() { weatherService.expectGetWeatherForUser("unknown"); EasyMock.expect(weatherView.getUserName()).andReturn("unknown"); weatherView.setUnknownUserVisible(true); EasyMock.expectLastCall(); weatherView.setEventHandler(EasyMock.anyObject(WeatherViewEventHandler.class)); EasyMock.expectLastCall(); EasyMock.replay(weatherView); WeatherHomePresenterpresenter = new WeatherHomePresenter(weatherService, weatherView); presenter.onLogin(); EasyMock.verify(weatherView); weatherService.verify(); } This test uses a manually created mock to make handling the async callback easier.

43. Question • Why does the View make the Presenter do this: weatherView.setUnknownUserVisible(true); • Instead of this: weatherView.getUnknownUser().setVisible(true)

44. Answer weatherView.getUnknownUser().setVisible(true) • Is hard to test because it is hard to mock: • To mock this, we would have to mock not only the WeatherView, but also the UnknownUser Label inside of it. • The above code is “talking to strangers”.

45. Law of Demeter • Also known as the “Don’t talk to strangers” rule. • It says: • Only have knowledge of closely collaborating objects. • Only make calls on immediate friends. • Look out for long chained “get”-style calls (and don’t do them: a.getB().getC().getD() • Your system will be more testable (and maintainable, because you have to rework calling objects less often).

46. What’s the Point? • To write good unit tests, we need to be able to insert mocks and fakes into the code. • Some things that help us do that: • Eliminating static methods and singletons. • Asking for dependencies instead of creating them. • Using design patterns that promote loose coupling, especially between business and presentation logic. • Obeying the Law of Demeter. • Code that does not do these things will often have poor test coverage.

47. What Can I Do? • Developers: • Follow these practices! • Testers: • Educate your developers. • Jump into the code and drive testability improvements. • A good way to motivate this is to track test coverage metrics.

48. Questions? Scott McMaster Google Kirkland, Washington USA scott.d.mcmaster (at) gmail.com

49. Bonus Slides

50. Model-View-Controller (MVC)