DevOps Meetup | Test Automation | 3/19/2012

1. DevOps Meetup | Test Automation | 3/19/2012

2. Requirements and Purposes • Open source tools • Multiple browser and OS support • Regression / functional tests • Parallel execution • Tool for easily write tests (long term)

3. Used software • Selenium (webdriver) in Java • TestNG (+ReportNG) from Ant • Jenkins

4. Framework design • Window driver pattern • Locators / Translations / Fixtures • WebElement locator strategy • id > css ( > xpath) • DynamicLocator

5. Framework design • Organize tests by groups • Own page factory pattern design • Easy to read Test code • Wrap what you can ("wait for" utilities) • Report what you do

6. Framework design DLocator <WebElementname="GeneralVideoDiv"> <!-- This is a general locator for videos, can be used as a dynamic locator to address videos. --> <Locators> <css>#MyVideosList &gt; li</css> <class>+ li</class> </Locators> </WebElement> <WebElementname="VideoSpecificVideoDiv"> <!-- This is a specific locator for a video, can be used as a dynamic locator to address videos. --> <Locators> <css>#Video-VIDEO_ID</css> <value>VIDEO_ID</value> </Locators> </WebElement> <WebElementname="VideoDivEditButton"> <!-- Edit button on a video div, can be used as second part of a dynamic locator. --> <Locators> <css>button.button-bar.editButton</css> </Locators> </WebElement>

7. Framework design TestCode /** * Tests error messages and text format in edit video overlay price field. */ @Test(description = "Tests video edit overlay functionalities", groups = { "PPVTests", "PPVVod" }) publicvoid testVodEditVideoOverlay() { finalint videoToEdit = 3; HeaderPage headerPage = getHeaderPage(); DashboardPage dashboardPage = getDashboardPage(); PPVEventUser ppvUser = userFactory.getPpvEventUser(); DynamicLocator editButton = dashboardPage.getDLocator("GeneralVideoDiv").append("VideoDivEditButton"); headerPage.loginUser(ppvUser.getName(), ppvUser.getPassword()); headerPage.openDashboard(); dashboardPage.loadManageYourShow(); dashboardPage.loadShowToManage(ppvUser.getVodPPVEventShowName()); dashboardPage.click("ShowYourVideos"); dashboardPage.waitForPageLoad(); dashboardPage.click(editButton, videoToEdit); dashboardPage.waitForAjaxRequestDone();

8. Framework design TestGrps <!DOCTYPEsuiteSYSTEM"http://testng.org/testng-1.0.dtd"> <suitename="Social Pages Login Tests"verbose="1"parallel="tests"thread-count="4"> <testname="Social Pages Login Tests in Firefox"parallel="methods"thread-count="12"> <parametername="browserName"value="Firefox"/> <parametername="browserVersion"value="8"/> <groups> <run> <includename="SocialPagesLoginLogout"/> <excludename="NotRunOnFirefox"/> </run> </groups> <packages> <packagename="com.ustreamv3.tests"/> </packages> </test> <testname="Social Pages Login Tests in Chrome"parallel="methods"thread-count="12"> <parametername="browserName"value="Chrome"/> <groups> <run> <includename="SocialPagesLoginLogout"/> <excludename="NotRunOnChrome"/> </run> </groups>

9. Framework design Report

10. Flash Automation • JsApi is our friend • getProperty(...) • setProperty(...) • callMethod(...) • addListener(...) • Needs strong team cooperation

11. Flash Automation @Sequential(groupClasses = { MetricsUser.class }) @MultiBrowser @Test(description = "Tests the broadcast start and stop effect on channel page.") publicvoid testBroadcastStartStop() { MetricsUser metricsUser = userFactory.getMetricsUser(); ShowPage showPage = getShowPage(); JsAPI viewerApi = getJsAPI(); showPage.openShowPage(metricsUser.getShowName()); viewerApi.checkProperty(ViewerProperties.IS_PLAYING, false); incrementSteps(); finalint lastSession = startNewSession(); HeaderPage headerPage = getHeaderPage(lastSession); JsAPI bcApi = getJsAPI(lastSession); headerPage.loginUser(metricsUser.getName(), metricsUser.getPassword()); headerPage.openBroadcastConsole(); bcApi.checkProperty(BroadcasterProperties.IS_PUBLISHER_ACTIVE, true); bcApi.callMethod(BroadcasterMethods.START_BROADCAST); viewerApi.checkProperty(ViewerProperties.IS_PLAYING, true); incrementSteps(); bcApi.callMethod(BroadcasterMethods.STOP_BROADCAST); viewerApi.checkProperty(ViewerProperties.IS_PLAYING, false); }

12. Dynamic vs static fixtures • Dynamic fixture • backdoor setup • gui setup • internal api setup • Static fixture = manual shared fixture

13. Dynamic vs static fixturesBackdoor setup • Pros • full parallel • fast setup • accurate result • Cons • infrastructure • maintenance

14. Dynamic vs static fixturesGUI setup • Pros • black-box test • parallel • Cons • s l o w setup • fragile

15. Dynamic vs static fixturesInternal API setup • Pros • can be fast • parallel • custom • Cons • development • maintenance

16. Dynamic vs static fixturesManual Shared Fixture • Pros • flexible • fast test • Cons • sequential • creation • maintenance

17. Parallel execution • Tests with multiple browser instances→ sequential job execution • Native events = one browser / VM • IE = one instance / VM

18. Multiple browser support • Safari 5.x with selenium → Fail • IE webdriver = jvm killer app :(

19. Lessons learned • seeing is believing • be prepared for exceptional cases • multiple browser = Firefox • test a new selenium version before use • clean up trash that selenium forgot to • memory > cpu (cores > frequency)

20. Lessons learned • Parallel is better than fast tests • Tests must run on a daily basis • Keep locators organized • Automation must be planned carefully • Keep tests in good health

21. Questions / comments ?

22. Thank you Happy testing!