iOS Unit Testing and Code Coverage

1. iOS Unit Testing and Code Coverage Amit Rao Mint iOS Developer rao.amit@gmail.com

2. Key Takeaway • Move Quality close to the developer desktop with unit testing, code coverage, static analysis, code reviews and warning free builds.

3. Assumptions and Pre-Requisites • Basic working knowledge of XCTest • Xcode 5.1.1 • Jenkins (1.564 or above) installed and basic working knowledge of Jenkins job creation and configuration • At least a couple of unit test already written • Unit tests can be invoked successfully from command line

4. High Level Steps • Turn on code coverage flags in Xcode Project • Add Logic to generate code coverage files in source code • Install Code Coverage Tool and Plugin for Jenkins Server • Configure Unit Test Job with Code Coverage

5. Unit Testing Resources • http://qualitycoding.org/ • OCMock (http://ocmock.org/reference/) • OCHamcrest/OCMockito • CoverStory

6. Unit Testing CI Methodolgy • Develop unit tests while developing app code • Use CoverStory to view coverage on desktop • Post check-in verify unit tests and code coverage report via CI system.

7. Step 1 Turn on Code Coverage Flags in xCode Project

8. Turn on Code Coverage Flags in xCode Project Set the following flags for the maintarget to ‘Yes’: • Generate Test Coverage Files • Instrument Program Flow

9. Step 2 Add Logic to Generate Code Coverage Files in Source Code

10. Add Logic to Generate Code Coverage Files in Source Code • As of Xcode 4.x, code coverage files are no longer auto generated by xCode. • __gcov_flush(); needs to be called in the application delegate after all tests are run

11. Add Logic to Generate Code Coverage Files in Source Code • To do so: • Create ‘test observer’ to notify application delegate when tests are done. *The test observer should be a member of the unit test target • Eg:

12. Add Logic to Generate Code Coverage Files in Source Code • Add observer when application delegate loads • Enhance applicationWillTerminate of application delegate to call __gcov_flush(); • Note might be a good idea to use #ifdef to only call this during unit testing

13. Step 3 Install Code Coverage Tool and Plugins to Jenkins Server

14. Install Code Coverage Tool and Plugins to Jenkins Server • Gcovr • generating summarized code coverage results from code generation files. • produces either compact human-readable summary reports, machine readable XML reports or a simple HTML summary. • Cobertura plugin • calculates the percentage of code accessed by tests. It can be used to identify which parts of the program are lacking test coverage • Renders the xml report generated by Gcovr to a navigable code coverage dashboard

15. Install gcovr • Download the latest gcovrinstallation on server hosting Jenkins from https://github.com/gcovr/gcovr/releases • Get the tar.gz file of the latest release and uncompress it in a temp directory • Copy the gcovr-X.X/scripts/gcovr to an accessible directory e.g. /usr/local/bin or /usr/bin or /Applications • Note: Gcovr is installed on /Applications/Scripts on the TT iPad build server • Run gcovr –-version to ensure installation > <your path>/gcovr--version gcovr 3.1 Note: if gcovr is in your path, you can just type gcovr --version

16. Install Cobertura plugin

17. Add Code Coverage Plugins to Jenkins- Install Cobertura plugin Start typing the word Cobertura in the Filter text box and the list below will expand and show the Cobertura (do not hit return after typing the word Cobertura

18. Add Code Coverage Plugins to Jenkins- Install Cobertura plugin

19. Step 4 Configure Unit Test Job with Code Coverage

20. Configure Unit Test Job with Code Coverage • Creating a separate job from main build is a good idea while configuring and fine tuning the process

21. Configure Unit Test Job with Code Coverage

22. Configure Unit Test Job with Code Coverage

23. Configure Unit Test Job with Code Coverage • Note: at this point, we will be writing a lot of instructions that can be executed on the command line • it would be a good idea to run these commands on the build server which hosts Jenkins to make sure they work before configuring them onto Jenkins

24. Configure Unit Test Job with Code Coverage – run unit test • *Change directory to location of xcode project cd /Users/devmac/Developer/Mint/trunk_svn1.7/Gala • Run xcodebuild command line xcodebuild –project Gala.xcodeproj-scheme ’Mint.com Debug-Internal' -sdk iphonesimulator7.1 clean test | ocunit2junit • *Note: You don’t have to change directory and instead use an absolute path but I’d like to break things up a bit so the commands are shorter and more readable • Ocunit2junit is a ruby script that generates the unit test summary report

25. Configure Unit Test Job with Code Coverage – run gcovr to generate Cobertura xml report • Change directory to location of code coverage files cd "`xcodebuild–project Gala.xcodeproj-showBuildSettings -sdk iphonesimulator7.1 test | grepOBJECT_FILE_DIR_normal |grep -oEi "/.*" | awk '{print $0"/i386"}'`” Note: The code generated files are placed in a xCodeDerivedData folder which is xCode assigns. The script above is one way to determine the DerivedData folder. The method I use is to run a script called “exportenv.sh” which generates a file called “env.sh” with an environment variable to the DerivedData folder. For more details please refer this link.

26. Configure Unit Test Job with Code Coverage – run gcovr to generate Cobertura xml report • Run gcovr and create the coverage.xml file /Applications/scripts/gcovr --object-directory . --exclude 'NS*’ --exclude '.*Frameworks.*' --exclude '.*include.*’ --xml > ${WORKSPACE}/coverage.xml • Note exclude what you don’t need! • Note {WORKSPACE} is a Jenkins env variable that indicates the Jenkin user’s workspace

27. Configure Unit Test Job with Code Coverage • Putting it together …

28. Configure Cobertura plugin to render coverage report • Add a post build action

29. Configure Cobertura plugin to generate coverage and unit tests report

30. Run unit test build job Now lets give our unit test job a run by selecting ‘Build Now’ If successful, an entry will show up under Build History, for our example, lets look at #45 Jun 5, 2014 3:04:37

31. Run unit test build job

32. Run unit test build job

33. Run unit test build job Drilling down to individual files Green section indicates code that are covered by unit test Pink section indicates code that are not covered

34. Add step to invoke unit test build from main build

35. Demo