slide1 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Testing @ Axa PowerPoint Presentation
Download Presentation
Testing @ Axa

Loading in 2 Seconds...

play fullscreen
1 / 26

Testing @ Axa - PowerPoint PPT Presentation


  • 261 Views
  • Uploaded on

Testing @ Axa . Olivier De Lissnyder. UA Leerstoel - Testing - May 2011. Agenda. AXA in the world, and in Belgium IBT² @ AXA Belgium Testing @ AXA Belgium Testing Approaches Test Roles Challenges Facing a test team. A fresh look at AXA. The AXA Group in key figures

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Testing @ Axa' - jana


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Testing @ Axa

Olivier De Lissnyder

UA Leerstoel - Testing - May 2011

agenda
Agenda
  • AXA in the world, and in Belgium
  • IBT² @ AXA Belgium
  • Testing @ AXA Belgium
  • Testing Approaches
  • Test Roles
  • Challenges Facing a test team
a fresh look at axa
A fresh look at AXA

The AXA Group in key figures

80 million customers

present in 56 countries

91 billion euros in revenues

4 billion euros in underlying earnings

981 billion euros of assets under management

210,000 employees (exclusive distributors included)

2

axa in belgium
AXA In Belgium

AXA in Belgium

6,256 employees (exclusive distributors included)

more than 3.1 million customers

375 million euros in underlying earnings

4.7 billion euros in insurance turnover

23.4 billion euros of total consolidated balance sheet for AXA Bank Europe

an activity model integrating bank and insurance

central services supporting all local activities of AXA in Belgium: HR, finance, IT, communications,...

3

axa in top 5 in belgium
AXA in top 5 In Belgium
  • AXA is one of the 5 leading insurance companies on the market

Source: Assuralia 2009: 2008 market shares

4

agenda6
Agenda
  • AXA In The World, and in Belgium
  • IBT² @ AXA Belgium
  • Testing @ AXA Belgium
  • Testing Approaches
  • Test Roles
  • Challenges Facing a test team
key features of axa software engineering
Key features of AXA Software Engineering

Axa IBT² has chosen to work with

  • Delivery Centers
    • Industrialized approach to deliver components of high quality.
    • Members of Delivery Centers are technological specialists (Silverlight, Java, ETL) in charge of implementing the business requirements.
    • Specialization means more efficiency – Reduced time to market, reduced cost
    • Delivery Centers Services, Delivery Center Front End, Delivery Center Testing,…
  • Releases
    • Most applications and projects depend on each other, changing one application can have impacts on the entire AXA system environment.
    • Working in releases means to put all the changes together in production, after careful synchronized testing. This increases quality and availability of production.
  • Off shore partners
    • Provide additional capacity and reduces costs
agenda8
Agenda
  • AXA In The World, and in Belgium
  • IBT² @ AXA Belgium
  • Testing @ AXA Belgium
  • Testing Approaches
  • Test Roles
  • Challenges Facing a test team
what is testing
What is Testing?
  • Delivery Center Testing is an independent Test Team
    • 14 Test Coordinators
    • 35 Test Analysts
    • 40 Off shore testers
    • 4 Test managers
    • Defines, monitors and controls its own processes
    • Supported by HP Quality Center as Test Management Tool
    • Supported by HP Quick Test Pro as Test Automation Tool
    • Supported by HP Load Runner As Performance Testing Tool
  • N° of defects found last year – 8.781
  • N° of scenarios executed last year – 44.139
benefits of ibt organization for testing
Delivery center

Test professionals are continuously gaining on-the-job experience in testing

They are now involved in an education program aimed at improving their testing skills.

Off shoring

Off shore provides capacity required to execute tests and report defects

Test execution is made off shore at off shore costs.

Good test professionals are scare people, therefore a dedicated investment in knowledge acquisition is needed in order to keep their skills and motivation high. This investment would not be possible in an organization where testing is not made in an independent team

Good test professionals are rare and testing is a matter of absorbing peaks of activity in last weeks before Put in Production. Off shoring the test execution and detailed design, offers the possibility to handle multiple parallel testing peak periods while keeping the costs under control.

Benefits of IBT² organization for testing
benefits of ibt organization for testing11
Release mode

Imposes code freeze period 2 weeks before Put in Production in order to execute regression and performance tests. All developments must be finished 2 weeks before Put in Production

Imposes a scope freeze 12 weeks before Put in Production. Scope of testing activities is known and can be planned in advance.

Regression tests and performance tests could be forgotten in the rush to production, therefore a freeze period of 2 weeks is welcome. These are important tests to execute as a regression or slow performances in an application could have much more impact on the users than a defect on a new functionality

The test team has a few weeks of relative calm to organize itself in order to have a planned and controlled test session. Most often it will eventually turn to an uncontrolled rush to testing, but we want this to happen as late as possible in the project in order to minimize risks of putting defects into production.

Benefits of IBT² organization for testing
slide12

Added-value of testing

  • Why spending money on testing before Put in Production?
    • Any defect found AFTER Put in Production is found by many customers
      • Bad Imago !
      • Loss of new business !
      • Financial errors !
      • Loss of time by the broker
      • Customer not returning
    • Multiplied by the number of customers finding the defects at the same time.
    • Multiplied by the internet-speed !
  • This cost is most often hidden, because difficult to compute, but it exists
    • …And you will feel it as soon as 5 min. after the Put in Production…
benefit of ibt organization for testing
Benefit of IBT² organization for testing
  • Let’s assume the (very) conservative hypothesis that a defect in production costs to the company 400€ more than the test activities would have cost to find it.
    • 8000 defects found per year * 400€ = 3 200 000€ saved by testing activities.

You can hardly bypass testing

slide14

Added-value of a dedicated test team

  • Some organization are not working with an independent tests team
    • Tests are created and executed by the business analyst, system analyst or representative of ‘the business’.
    • There is no opportunity to capitalize on experience and trigger a continuous improvement loop.
  • A dedicated test team can dedicate time on continuous testing improvements
  • Deeper and Broader Test Coverage
  • Systematic Regression Testing
  • Improved Reviews (V-model)

Improving Quality of Production

KPI: N° defect prod/N° defect in test

AXA: 4%

  • Off-Shore component takes detailed test design and test execution in charge
  • Test automation is made possible
  • Re-use of test scenarios

Decreasing Cost of Testing

KPI: Euros spent on test activities/ N° of defects found by tests

agenda15
Agenda
  • AXA In The World, and in Belgium
  • IBT² @ AXA Belgium
  • Testing @ AXA Belgium
  • Testing Approaches
  • Test Roles
  • Challenges Facing a test team
2 different testing approaches
2 Different Testing Approaches
  • Large Projects are tested in a systematic, rigorous way dictated by the V-model.
      • This increases the probability to put in production on time a high quality software because such a rigorous approach allows to better control the schedule and scope uncertainties, the dependencies with other projects/applications and the test efforts spent.
      • But it has a cost !
  • Small demands and small projects are tested in a less rigorous, more agile way.
      • Limited risk level, limited uncertainties
      • No or limited dependencies
      • Minor test effort invested.
      • No reason to spend time in rigorous test coverage, reviews and follow up.
project testing
Project Testing
  • Acquire project knowledge and identify incoherencies thanks to Early Testability Reviews

Regression Tests

  • Verify that the critical features of the application are still working fine (Functional + Performance)

Requirement

Acceptance Tests

  • Validate that the application delivered meets thebusiness need

Functional Design

System Tests

  • Verify that the design documents are implemented as specified in the application

Technical Design

Integration Tests

Implementation And Unit Tests

  • Out of Test Process Scope

Test Design (Early and detailed))

testing in project life cycle

CONCEPT

INCEPTION

ELABORAT*

CONSTRUCT*

TRANSIT*

Warranty

RUN

IDEA

OFFER

PROJECT

AM

Go

Go

Go

Go

concept

concept

Go

Go

mobilize

mobilize

Go

Go

delivery

delivery

Go

Go

execute

execute

End

End

project

project

production

production

5

5

0

0

1

1

2

2

3

3

4

4

5

5

0

0

1

1

2

2

3

3

4

4

  • Support Solution Design
  • Define Test Strategy
  • Finalize collaboration agreement with project. Commit on:
    • Scope
    • Cost
    • Schedule
    • Quality

BUILD : Coordinate and deliver

  • Testability Reviews
  • Technical Design
  • Coding
  • Integration testing
  • Maintenance

TEST : Coordinate and deliver

  • System Testing
  • Regression and Performance Testing
  • Support to User Acceptance Testing

1101001100010110100011

Software factory

Testing In Project Life Cycle
small demands testing
Small Demands Testing
  • 1 Week Cycle Approach
    • Once per week, new demands are included in the test session.
    • The demands are tested as soon as possible, with a light formalism
  • Focus On Regression Tests
    • During the 2 weeks preceding the Put in Production, the applications are completely tested for regression, on a pre-production environment containing the entire release to be Put in Production.
    • No more modifications are allowed others than defect fixing. Code is frozen.
agenda20
Agenda
  • AXA In The World, and in Belgium
  • IBT² @ AXA Belgium
  • Testing @ AXA Belgium
  • Testing Approaches
  • Test Roles
  • Challenges Facing a test team
test roles
Test Roles
  • Test Coordinators
    • In charge of the daily management of the test sessions : day to day schedule, facilitation, fire-fighting, reporting, defect tracking, administration in Quality Center
    • This is an organization role, not a « content » role. The goal of the Test coordinator is to « make-it-happen  » that requires pro activity and drive.
  • Test Analysts
    • In charge of the test coverage The test analyst identifies the tests to be executed and the expected results that will guarantee that the major risks are covered by test scenarios.
    • This is a « content » tasks, the test analyst has a deep knowledge of the application/project and guarantees the quality of the test coverage.  
  • Off Shore Testers
    • In charge of the detailed design of the test scenarios identified by the Test Analyst, test data identification and test execution.
    • This an execution job, that requires the ability to absorbs peaks of activity and good handling of stressful situations
agenda22
Agenda
  • AXA In The World, and in Belgium
  • IBT² @ AXA Belgium
  • Testing @ AXA Belgium
  • Testing Approaches
  • Test Roles
  • Challenges Facing a test team
challenges facing a test delivery center
Challenges Facing a Test Delivery Center
  • Testing is between Development and Put in Production
    • Testing is on the Critical path of the project and is the step just before the Put in Production  (very) high pressure to shorten/skip the testing
    • Testing is not perceived as a « productive » task. Testing does not add lines of codes or provides analysis documents that describe a solution to a problem.

 (very) high pressure.

    • Axa Answer: release mode and mandatory freeze periods, off shore testing to add capacity, rigorous reporting based on functional risks taken.
  • Testing finds defects in the code BUT also in all other processes
    • Requirements Engineering, Requirement Management, Configuration Management, … are also generating defects. The earlier in the life cycle a defect is generated and not discovered, the most it costs to correct.
    • Testing could be perceived as a rude guest that underlines dysfunctioning. The tester needs good relational and communication skills in order to be perceived as a ally in the quest for good quality in production.
slide24

Challenges Facing a Test Delivery Center

  • Skills shortage, competence erosion
    • Test specialist are scare. The developments techniques are speeding up in order to shorten the time to market and applications are evolving fast in order to meet market needs
    • Axa answer: My.Future program (upskills and career moves development program) for test analysts, investments in knowledge acquisition and off shoring of test execution
  • If testing gets a cold, it could be because other processes have the flu…
    • Requirements unclear  generates false defects, rework in test design
    • No design  No testing ! (V-Model )
    • Uncontrolled changes  uncontrolled rework (budget!), false defects
    • Late deliveries, uncompleted code baselines,…
    • Testing is at the end of the project, it can be crippled by all the other preceding processes if they dysfunction.
    • Axa Answer: QA team accompanies all the processes in order to make sure they are healthy and continuously improving.