Why Your Se Tests are So Dang Brittle… …and what to do about it. Patrick Wilson-Welsh Dawn Cannan. Introduction. Patrick Welsh Senior Agile Consultant Pillar Technology Group http://pillartechnology.com [email protected] http://patrickwilsonwelsh.com
PrerequisitesYou’ll need to know some stuff. To follow, you’ll need to know Selenium, Java, xpath, css, HTML, and a bit of Object Oriented Design.And, I will be gentle.
too hard: avoid
You are committing too high a percentage of total test-automation and programming resources to that specific kind of automated testing. Have a plan for outgrowing that pattern.Actually, have a fairy tale:
% of automated testing work
least rework & waste; lowest TCO
we often start here
good: easy to learn
bad: hard to learn
for good reason!
bad: high TCO, low ROI
good: low TCO,
But again, the price
Don’t tell me testing is “not your job”
Don’t say programming is “not your job”
Automated testing is everybody’s job, because the Definition of Done is everybody’s job.Se tests are written by testers and programmers together. Period. The whole team shares sufficient testing as part of Definition of Done.
Some issues in Java Selenium RC testing:Mixed abstraction layer logic (test + biz domain + testing framework + …)Obscure page-flow and page verification.Verifying state across multiple pages. DefaultSelenium (et al) access control.Dynamic HTML/Ajax
Example 1: Bad-Ole Se RC Multi-Page-FlowProcedural code for verifying that the same links show up on each of several pages. Even method extraction and custom assertions may not help as much as you need.
Se-Specific, General Guidelines :Only use Se as last resort; play to its strengthsDon’t use Se IDE except for prototypingHand-roll Object-Oriented Se RC testsSeparate framework code & test code
src source folder: DRY reusabilitysrc folder contains only reusable domain-specific and domain-independent classes. Domain-specific pages, panes, and common components (nav menus, etc). Domain-independent framework (page elements/controls, Selenium and jQuery façade/decorator, locator strategy code)
test source folder: one-off, wet test codetest folder contains behaviorally-organized test scenarios. Nearly all of them happy paths. Test scenarios for page flow manipulation and state verification can include quite a bit of duplication. If you prefer, extract private helper methods like “loginAndNavigateToSuchAndSuchPage()”
Remember the good old days of every browser-resident change resulting from an HTTP Response to an HTTP Request? (Sigh.)Well kiss em goodbye. We live in a dynamic, RIA world now. Ajax. Etc.Whole desktops, coming to browsers near you… except, differently!
Principles to consider for testing dynamic stuff:The rendered HTML is less and less your friend.XPATH is not standard, thus not your friend. The in-memory DOM is your friend. CSS element location strategy is your friend.jQuery is your friend.
Get this book...
…and this book.