From 3 weeks to 30 minutes a journey through the ups and downs of test automation
Download
1 / 52

- PowerPoint PPT Presentation


  • 81 Views
  • Uploaded on

From 3 weeks to 30 minutes – a journey through the ups and downs of test automation. Who am I?. Peter Thomas Chief Software Engineer Operations IT, UBS Investment Bank Developer (mostly) I do some architecture I have done Testing I talk a lot (Mentor/ Coach)

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about '' - luyu


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Who am i
Who am I? downs of test automation

  • Peter Thomas

    • Chief Software Engineer

    • Operations IT, UBS Investment Bank

  • Developer (mostly)

  • I do some architecture

  • I have done Testing

  • I talk a lot (Mentor/Coach)

  • From the dark side (consulting) but getting better


Where did we start
Where did we start? downs of test automation

  • Existing mainframe legacy application

  • 3 week manual PPT testing cycle

  • 12 week delivery cycle


What did we want to do
What did we want to do? downs of test automation

  • Belief there was a better way to deliver software

  • Incremental development to deliver business value quickly

  • Address the rapidly changing business landscape with flexibility in delivery

  • Build quality into the solutions

  • Deliver the software rapidly, but in a cost effective manner

  • Put the fun back into software delivery


From 3 weeks to 30 minutes a journey through the ups and downs of test automation

New York downs of test automation

London

Kiev

Hyderabad

Hong Kong

2M trades per day

100 billions settling per day in all major currencies

50+ exchanges across EMEA and APAC

15 scrum teams/120 people

9 applications

Production releases every 2 weeks


From 3 weeks to 30 minutes a journey through the ups and downs of test automation

New York downs of test automation

London

Kiev

Hyderabad

Hong Kong

200 commits per day

1000 artefacts updated per day

1 commit every 5 minutes peak


From 3 weeks to 30 minutes a journey through the ups and downs of test automation

New York downs of test automation

London

Kiev

Hyderabad

Hong Kong

24 Build Targets

60+ Test Targets

800 Automated Functional Tests

10, 000 Unit/Integration Tests

7, 000 Behavioural Tests


From 3 weeks to 30 minutes a journey through the ups and downs of test automation

But…….. downs of test automation


Our tests were
Our tests were….. downs of test automation

Complicated

Obscure

Random failures

Slow to run

Difficult to fix


The tdd rut
“The downs of test automation TDD rut”

Complicated

Obscure

Random failures

Slow to run

Difficult to fix


Test the right thing and test the thing right

Test the Right Thing downs of test automation and Test the Thing Right

When all you have is a hammer, everything looks like a nail


Why do you test
Why do you test? downs of test automation


Why do you test1
Why do you test? downs of test automation

  • Because TDD tells me so?

  • Because (insert favourite method here) says I should?

  • So I meet the 80% coverage metric?


Why do you test2
Why do you test? downs of test automation

  • To accept the solution

  • To understand and document the solution

  • To prove its not broken

  • To find the unknown unknowns

  • To help us design and build new features

  • To help us explore what is really needed

  • To show it won’t crash under load, to show it is secure (to test the ‘ilities)

    …?


Why do you test3
Why do you test? downs of test automation

Agile testing Quadrants – Lisa Crispin, Janet Gregory


Testing purposefully
Testing Purposefully downs of test automation


The right thing at the right level
The Right Thing At The Right Level downs of test automation

Unit

Component

System


The right thing at the right level1
The Right Thing At The Right Level downs of test automation

  • Tests a single class with no dependencies

  • If dependencies like Spring Context, Database used then called Unit Integration

  • Tests technical correctness and robustness

  • Very specific, a failing test indicates an issue in a specific class

  • Difficult to perform on poor quality code

  • Very fast to run, should run on the developer’s desktop in the IDE

Unit

Component

System


The right thing at the right level2
The Right Thing At The Right Level downs of test automation

  • Tests a group of components which are integrated to perform a business relevant function

  • Can test technical or business correctness, but should be expressed in Domain concepts

  • Specific, a failing test indicates problems in that component

  • Easier to perform on poor quality code, provided component boundaries are clear

  • Can be quick to run, doesn’t need the full application, should run on developers desktop

Unit

Component

System


The right thing at the right level3
The Right Thing At The Right Level downs of test automation

Unit

  • Tests a system at its boundaries as a ‘black box’

  • Primarily testing for business correctness

  • Not Specific, a failing test could be caused anywhere in the system flow

  • Easy to perform on legacy applications, requires little code modification

  • Slow to run, can be fragile, may not run on developers desktop

Component

System


What we wanted
What We Wanted downs of test automation


What we had
What We Had downs of test automation

Unit tests which weren’t really Unit Tests

End to End tests when unit tests would have been sufficient

Duplicate and redundant End to End tests


The tdd cycle
The TDD Cycle downs of test automation


From 3 weeks to 30 minutes a journey through the ups and downs of test automation
TDD? downs of test automation

@Test 
public void shouldBeEmtpyAfterCreation() { 
  ReportObjectaTrm = new ReportObject(); 

assertNull(aTrm.getEligibleTrm()); 
  assertNull(aTrm.getProcessedEvent()); 
  assertNull(aTrm.getPayloadXml()); 
}

@Test 
public void shouldCorrectlySetAttributesViaConstructor() { 
  ReportObjectaTrm = new ReportObject(eligibleObject, REPORTABLE_XML); 
 assertEquals(eligibleObject, reportableTrm.getEligibleTrm()); 
  assertEquals(REPORTABLE_XML, reportableTrm.getPayloadXml()); 
}

@Test 
public void shouldCorrectlySetFieldsViaSetters() { 
  ReportObjectaTrm = new ReportObject();  aTrm.setEligibleTrm(eligibleObject); 
  aTrm.setProcessedEvent(child); 
  aTrm.setPayloadXml(REPORTABLE_XML);

assertEquals(eligibleObject, aTrm.getEligibleTrm()); 
  assertEquals(child, aTrm.getProcessedEvent()); 
  assertEquals(REPORTABLE_XML, aTrm.getPayloadXml()); 
}


The hollow egg
The Hollow Egg downs of test automation


The hollow egg1
The Hollow Egg downs of test automation

98 Tests

2.5K LOC

30 Tests

200 LOC


R spec model
R downs of test automation Spec model


Outside in the tdd spiral
Outside In - The TDD Spiral downs of test automation


Make the intent clear

Make the Intent Clear downs of test automation

How to achieve acceptance without showing your IDE or log file to the users


Unit test naming
Unit Test Naming? downs of test automation

testProcessError()

whenWorkItemIsManuallyAssignedThenClientRuleShouldBeSetToManualOverride()

shouldAllowAnActioningWorkItemToBeUpdated()


Test data nightmare
Test Data Nightmare downs of test automation


What do you demo
What Do You Demo? downs of test automation


What do you demo1
What Do You Demo? downs of test automation


Executable specification
Executable Specification downs of test automation


Improve testing stability

Improve Testing Stability downs of test automation

Avoiding the Broken Windows syndrome


Separate progress regression tests
Separate Progress & Regression Tests downs of test automation


Speed up through parallelism
Speed-up Through Parallelism downs of test automation


Identify unstable tests
Identify Unstable Tests downs of test automation


Quarantine unstable tests
Quarantine Unstable Tests downs of test automation


Avoid external dependencies
Avoid External Dependencies downs of test automation


Introduce fakes
Introduce Fakes downs of test automation


Avoid time dependent tests
Avoid Time-Dependent Tests downs of test automation


Test isolation
Test Isolation downs of test automation


Asynchronous testing headache
Asynchronous downs of test automation Testing Headache


Don t
Don’t! downs of test automation

  • Does your test need to be asynchronous?

  • 80/20 rule?

  • Create synchronous test runner harness


Asynchronous testing using events
Asynchronous downs of test automation Testing using Events


From 3 weeks to 30 minutes a journey through the ups and downs of test automation

So…? downs of test automation


Treat your tests like you treat your code

Treat your Tests Like you Treat your Code downs of test automation

“it’s just a test class” is not an excuse

Clean Code applies to tests too


Think about why you are testing

Think about Why You are Testing downs of test automation

Specification tests for internal quality

Business tests for external quality


Think about who you are testing for

Think about Who You are Testing For downs of test automation

More people are interested in your tests than you may think


Zero tolerance to instability

Zero Tolerance to Instability downs of test automation

“It runs OK on my machine” is not a valid response


From 3 weeks to 30 minutes a journey through the ups and downs of test automation

Interested in a career at UBS? downs of test automation

peter.thomas@ubs.com

@peterrhysthomas

peterrhysthomas.wordpress.com