Why your se tests are so dang brittle and what to do about it
This presentation is the property of its rightful owner.
Sponsored Links
1 / 45

Why Your Se Tests are So Dang Brittle… …and what to do about it PowerPoint PPT Presentation


  • 39 Views
  • Uploaded on
  • Presentation posted in: General

Why Your Se Tests are So Dang Brittle… …and what to do about it. Patrick Wilson-Welsh Dawn Cannan. Introduction. Patrick Welsh Senior Agile Consultant Pillar Technology Group http://pillartechnology.com [email protected] http://patrickwilsonwelsh.com

Download Presentation

Why Your Se Tests are So Dang Brittle… …and what to do about it

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Why your se tests are so dang brittle and what to do about it

Why Your Se Tests are So Dang Brittle……and what to do about it

Patrick Wilson-Welsh

Dawn Cannan


Introduction

Introduction

  • Patrick Welsh

  • Senior Agile Consultant

  • Pillar Technology Group

  • http://pillartechnology.com

  • [email protected]

  • http://patrickwilsonwelsh.com

  • http://coderetreat.ning.com/

  • mobile: 248 565 6130

  • twitter: patrickwelsh


Why your se tests are so dang brittle and what to do about it

PrerequisitesYou’ll need to know some stuff. To follow, you’ll need to know Selenium, Java, xpath, css, HTML, and a bit of Object Oriented Design.And, I will be gentle. 


We re here to help

We’re here to help …


Why your se tests are so dang brittle and what to do about it

How hard is it to regression test an entire enterprise web app using any web-app-GUI-black-box testing tool?


Why your se tests are so dang brittle and what to do about it

too hard: avoid


Number 1 reason selenium rc through the web app gui tests are so brittle

Number #1 Reason Selenium RC, through the web app GUI tests are so brittle?


Because you are writing too many of them let me put that another way

Because you are writing too many of them.Let me put that another way.


Why your se tests are so dang brittle and what to do about it

You are committing too high a percentage of total test-automation and programming resources to that specific kind of automated testing. Have a plan for outgrowing that pattern.Actually, have a fairy tale:


Driving on the ice in michigan avoid it whenever you can know how to do it well

Driving on the ice in Michigan. Avoid it whenever you can. Know how to do it well.


3 kinds of automated tests

3 kinds of automated tests


Gui end 2 end integration story acceptance unit micro isolation

GUIend-2-end, integration, story, acceptanceunit/micro/isolation


Why your se tests are so dang brittle and what to do about it

TCO

% of automated testing work


Good least investment roi good most investment roi

good: least investment/ROIgood: most investment/ROI


Why your se tests are so dang brittle and what to do about it

least rework & waste; lowest TCO


But that s not your triangle yet

But that’s not your triangle, yet.


Why your se tests are so dang brittle and what to do about it

we often start here


Why your se tests are so dang brittle and what to do about it

good: easy to learn

bad: hard to learn

for good reason!


Why your se tests are so dang brittle and what to do about it

bad: high TCO, low ROI

good: low TCO,

high ROI

But again, the price


So you re stuck with more se testing than you should be doing that s ok so am i

So, you’re stuck with more Se testing than you should be doing. That’s ok. So am I.


How can we minimize the brittleness of these inherently brittle tests

How can we minimize the brittleness of these inherently brittle tests?


Following are things that help me and my teams

Following are things that help me and my teams.


Why your se tests are so dang brittle and what to do about it

Don’t tell me testing is “not your job”


Why your se tests are so dang brittle and what to do about it

Don’t say programming is “not your job”


Why your se tests are so dang brittle and what to do about it

Automated testing is everybody’s job, because the Definition of Done is everybody’s job.Se tests are written by testers and programmers together. Period. The whole team shares sufficient testing as part of Definition of Done.


Why your se tests are so dang brittle and what to do about it

Some issues in Java Selenium RC testing:Mixed abstraction layer logic (test + biz domain + testing framework + …)Obscure page-flow and page verification.Verifying state across multiple pages. DefaultSelenium (et al) access control.Dynamic HTML/Ajax


Why your se tests are so dang brittle and what to do about it

Example 1: Bad-Ole Se RC Multi-Page-FlowProcedural code for verifying that the same links show up on each of several pages. Even method extraction and custom assertions may not help as much as you need.


Why your se tests are so dang brittle and what to do about it

Se-Specific, General Guidelines :Only use Se as last resort; play to its strengthsDon’t use Se IDE except for prototypingHand-roll Object-Oriented Se RC testsSeparate framework code & test code


A principle to consider for se testing dry test framework wet test code

A principle to consider for Se testing:DRY test framework; “wet” test code.


Why your se tests are so dang brittle and what to do about it

src source folder: DRY reusabilitysrc folder contains only reusable domain-specific and domain-independent classes. Domain-specific pages, panes, and common components (nav menus, etc). Domain-independent framework (page elements/controls, Selenium and jQuery façade/decorator, locator strategy code)


Why your se tests are so dang brittle and what to do about it

test source folder: one-off, wet test codetest folder contains behaviorally-organized test scenarios. Nearly all of them happy paths. Test scenarios for page flow manipulation and state verification can include quite a bit of duplication. If you prefer, extract private helper methods like “loginAndNavigateToSuchAndSuchPage()”


Specific patterns in the selenium rc patterns eclipse project

Specific Patterns in the selenium-rc-patterns eclipse project.


Why your se tests are so dang brittle and what to do about it

  • Pattern: Reusable Element Objects Framework

  • Different kinds of page elements get matching classes; all extend BaseElement.

  • All are completely generic; unrelated to biz domain.


Why your se tests are so dang brittle and what to do about it

  • Pattern: Reusable Selenium Singleton Facade

  • All access to DefaultSelenium and SeleniumServer (jetty proxy) are outside test code.

  • All access to them is via singleton inside façade

  • Façade decorates DefaultSelenium with other cool stuff you need


Why your se tests are so dang brittle and what to do about it

  • Pattern: Self-Verifying, Lazy-Load Page Objects

  • Biz-domain-specific classes wrap the behavior of specific pages in your app, in page-specific ways.

  • When you navigate to a page by clicking on a link, the matching page object gets launched, and waits until it is fully loaded by that object’s definition (a css locator).


Why your se tests are so dang brittle and what to do about it

  • Pattern: Elements that are Page Object Factories

  • When you instantiate a DhtmlLink class, you parameterize it with the PageObject.class that you want to be launched when you click on that link.

  • This pushes page flow semantics deep into the actual link flow, emulating actual app, and keeps page flow DRY.


Why your se tests are so dang brittle and what to do about it

Remember the good old days of every browser-resident change resulting from an HTTP Response to an HTTP Request? (Sigh.)Well kiss em goodbye. We live in a dynamic, RIA world now. Ajax. Etc.Whole desktops, coming to browsers near you… except, differently!


Why your se tests are so dang brittle and what to do about it

Principles to consider for testing dynamic stuff:The rendered HTML is less and less your friend.XPATH is not standard, thus not your friend. The in-memory DOM is your friend. CSS element location strategy is your friend.jQuery is your friend.


Why your se tests are so dang brittle and what to do about it

  • Pattern: CSS Element locator strategies for DHTML/Ajax

  • Use id or name attributes if you got ‘em

  • Use css as a second-resort locator

  • Only use xpath as a last-resort element locator

  • Actually, forget it. Never use xpath.


Why your se tests are so dang brittle and what to do about it

  • Pattern: Using jQuery to Verify Dynamic Visibility

  • Consider usingSeto inject jQuery (or some such js library) into a page, to get direct access to the DOM, not the rendered HTML, when you need

  • There will increasingly be behaviors for which jQuery, being cross-browser in a fairly standard way, is a better choice for DOM state verification than older techniques

  • jQuery support: coming to a Selenium near you?


Why your se tests are so dang brittle and what to do about it

  • Pattern: Common Page Element Singletons

  • If all pages in an app have certain common elements, consider hanging a static singleton off a BasePage object.


Why your se tests are so dang brittle and what to do about it

  • Pattern: Multi-Page Rule Verification

  • A MultiPageTraverser is instantiated with a rule that implements a Verifiable interface.

  • Verifiable.verify() call on each page does the right thing.


Why your se tests are so dang brittle and what to do about it

Get this book...


Why your se tests are so dang brittle and what to do about it

…and this book.


Why your se tests are so dang brittle and what to do about it

Q/A


  • Login