Highlights
This presentation is the property of its rightful owner.
Sponsored Links
1 / 27

Highlights PowerPoint PPT Presentation


  • 48 Views
  • Uploaded on
  • Presentation posted in: General

BSPIN Agile Practices Benchmarking Report October 2012. Highlights. Introduction. Agile Practice Benchmarking: BSPIN conducted a study during 2011-2012 to benchmark the Agile practices to provide insights on the effective ways of adopting Agile in small to large organizations.

Download Presentation

Highlights

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Highlights

BSPIN Agile Practices Benchmarking Report

October 2012

Highlights


Introduction

Introduction

  • Agile Practice Benchmarking:

    BSPIN conducted a study during 2011-2012 to benchmark the Agile practices to provide insights on the effective ways of adopting Agile in small to large organizations.

    • 12 Organizations presented their Agile experience across 3 Panel discussions

    • Agile Practice benchmarking survey was conducted in July 2012

      • Designing of Questionnaire, Consolidation, Analysis of feedback was done in coordination with Agile coach and Senior Consultant from Improvalue Consulting, Bangalore (www.improvalue.com).

  • Objectives :

    • Explore new areas like:

      • Domains where Agile has been adopted. Agile Practices in use.

      • Benefits and Barriers encountered by organizations on their Agile journey

      • Metrics Reported (Project and Product)

      • Coexistence/Compatibility with other standards (CMMi) and practices (Lean)

      • Best Practices

    • Focus is on IMPLEMENTATION aspects of AGILE

    • Outcome:

    • Learn from industry experience

    • Enable informed decisions


Agenda

Agenda

  • Demographics - Agile Spread

  • Usage of Agile Practices

  • Estimation Techniques

  • Automation, Tools

  • Agile Metrics

  • Agile – Coexistence with other Methodologies/Models

  • Benefits and Barriers

  • Challenges Faced, Best Practices

  • Conclusions/Recommendations


Demographics

Demographics

  • 94 responses from 75 Organizations

  • More participation from Small and Medium Organizations (compared to other surveys)

  • Product and Service Organizations covered equally

  • 98% of Large Organizations and 77% of Small an d Medium Organizations have external quality certifications

  • More than 50% of the respondents have been using Agile for more than 2 years

  • Panel discussions – 12 Organizations shared their Agile Experience ( 6 Large and 6 Small-Medium)


Support to agile

Support to Agile

How Agile adoption is supported?

Training Methods used:

  • This section was answered by 72% of the organizations

  • As the organization gains more experience on Agile, Agile becomes a part of QMS

  • Only 6% of the organization indicated “Self Management by teams” with no other support.

  • This was answered by 90% of respondents

  • About 40% of the organization had more than one method of training

  • About 20% of the organization indicated only “In-house” with no other support.


Agile spread

Agile Spread

  • SCRUM is the mostly adopted methodology

  • Increase in usage of KANBAN -

    • Many Organizations evolved from manufacturing background where LEAN is adopted

  • Agile experience of the organization has no major effect on the methodology chosen

  • Nearly 70% of the organizations are using 1 or 2 methods

  • Of the 7% using more than 4 methodologies, 3% are small organizations


Agile spread1

Agile Spread

  • Others Domains Reported:

  • OS, Middleware, Kernel

  • Platform development

  • Control Systems

  • Product Integration

  • Media and Publication, Education

  • Travel, Energy and Storage

  • Agile has been used in almost all domains including safety critical domains like Pharma, Avionics and other Embedded Systems.

  • Telecom domain which is mostly hardware dependent has reported highest percentage of Agile implementation.

  • More than 75% of the respondents have reported the implementation of ‘Distributed Agile’

  • Agile has been used in most of the project types – including Testing and Systems integration

ABREVIATIONS

(1) BFS – Banking and Financial Services (2) IA – Industrial Automation (3) A,A,A – Avionics, Aviation, Aerospace (4) SI – Systems Integration


Usage of agile practices

Usage of Agile Practices

82% have release cycles <= 4 weeks

  • Core Agile (Scrum) Practices have been well adopted.

  • Engineering Practices usage (Refactoring, Test Driven Development) needs to be strengthened.

  • 48% of the respondents use 2 to 3 weeks release cycles.


Agile practices vs agile experience

Agile Practices Vs Agile Experience

  • Core Agile practices are embraced right from the beginning

  • There is a significant increase in the usage of Open Work Areas (compared to other surveys)

  • Some of Scrum practices take more time to stabilize

  • Organizations can have increased rigor early in the Agile adoption for better benefits.

  • The Engineering practices strengthen over time

  • Refactoring and TDD need more focus for effective Agile adoption


Agile estimation techniques

Agile Estimation Techniques

  • About 53% of the respondents use "User Stories" estimation at proposal/bid stage

  • About 80% of the respondents are using one or more of the following estimation techniques:

    • User Stories estimation

    • WBS

    • Guesstimation

  • Organizations are taking more than 2(two) years to move away from Guesstimation to WBS which needs attention.

Usage of traditional estimation techniques at

proposal/bid stage and User Story Estimation/

WBS at iteration stage seems to be in vogue


Extent of automation

Extent of Automation

  • Only 76% of the respondents answered this section

  • Automation of unit testing is low, inline with low adoption of TDD

  • Automation of code reviews is low, as evident from low adoption rate of Refactoring

  • Automation done in later stages of testing and quality compromised- missing out the essence of Agile

  • The automation level increases with Agile experience of the organization

  • The size of the organization had no impact on the level of automation

Organizations have shared multiple Tools : For

Agile Project Management and Engineering.


Tools used

Tools Used

Requirements

Code Reviews

Testing

Defect Tracking

Project Management

  • Caliber RM

  • DOORS

  • Greenhopper

  • JIRA

  • RALLY

  • Rational Requisite Pro

  • Red Mine

  • ScrumWorks

  • Sharepoint

  • TFS

  • VersionOne

  • CheckStyle

  • Eclipse plugins

  • FxCop

  • IBM AppScan

  • Rational Software Analyzer

  • SONAR

  • StyleCop

  • TFS

  • Toad

  • xTest

  • xUnit

  • HP Quality Center

  • HP Quick Test Professional

  • JIRA

  • LoadRunner

  • MS VS

  • Selenium

  • Silk

  • VSTS

  • Winrunner

  • xTest

  • xUnit

  • Bugzilla

  • TeamForge

  • Clearquest

  • HP Quality Center

  • JIRA

  • Rational ClearQuest

  • TFS

  • CollabNet

  • Green Hopper

  • HP-Project and Portfolio Management

  • JIRA

  • Mingle

  • Rally

  • Scrum Works

  • TFS

  • Version One

  • XPlanner

Not a complete list. Widely used tools are reported here.


Traditional practices with agile

Traditional Practices with Agile

  • 56% of the respondents reported:

    • Upfront collection and finalization of requirements

    • Upfront architecture

  • This practice was predominantly observed in Product companies when compared to Service Providers


Management metrics

Management Metrics

Other Metrics Reported:

Schedule:Schedule Variance, Sprint Stretch Factor

Effort:Effort Variance,

Scope:Commitment Accuracy, (De)Scoping Index,

Requirements Stability, Scope burn down

Quality:Product Metrics, Review Efficiency,

Defect Turnaround Time,

Defect Resolution Productivity,

Defect Removal effectiveness

Testing:Test Yield, Test case pass percentage

  • 90% of the respondents have answered this question

  • The usage of metrics increases with Agile experience to measure effectiveness

  • Recommend BSPIN to conduct ‘Agile data

  • Benchmarking’ to obtain industry benchmark


Engineering metrics

Engineering Metrics

  • Monitoring Product Metrics is fairly good except for the Code related activities

  • Measuring Code complexity is key for refactoring – needs more focus

  • Measuring Code coverage is key for quality – needs more attention

  • There seems to be a steady improvement as the organization gains more experience on Agile

  • However, Code Complexity and Code Coverage metrics needs more focus


Agile cmmi coexitence

Agile - CMMi Coexitence

What does the Survey Report?

Reasons cited for “Yes”::

Agile does not throw away the processes. It is a mature way of looking how efficient the process

CMMi prescriptions can be applied to any methodology.

Reasons cited for “No” and “not Sure” :

Different focus (People centric Vs Process centric)

Agile is project management, engineering framework and CMMi is a process improvement framework

It is compatible till L3 except IPM, OPD

  • This question was answered by 72% of the respondents

  • 68% of the respondents think that Agile and CMMi can coexist. Of these 60% are from organizations that have ISO or CMMi certification.

  • 72% of the respondents reported that their Agile projects were also being audited (as per the corresponding certification process)


Cmmi on agile

CMMi on Agile

What does CMMi v1.3 say?

To help those who use Agile methods to interpret CMMI practices in their environments, notes have been added to selected process areas. These notes are added, usually in the introductory notes, to the following process areas in CMMI-DEV: CM, PI, PMC, PP, PPQA, RD, REQM, RSKM, TS, and VER.

. . .

Even if you are not using an Agile approach, you might still find value in these notes.

. . .

Some general background and motivation for the guidance given on Agile development approaches are found in the SEI technical note CMMI or Agile: Why Not Embrace Both! [Glazer 2008].

Reference: Section-5: Interpreting CMMI When Using Agile Approaches – CMMI-DEV, V1.3 by SEI

Possible reason for higher maturity organizations to agree that Agile and CMMi are compatible

Practitioner Groups of CMMi and Agile can integrate the practices

to adopt the best of both


Coexistence with other models

Coexistence with other models

Integrating LEAN with Agile

  • 33% of the respondents reported using LEAN

    techniques with Agile

  • Most organizations (26%) have adopted KANBAN

    Integrating Agile with other Life Cycle Models

  • 46% of the respondents reported using Agile best practices in other LCMs

  • Agile practices predominantly used:


Benefits of agile adoption

Benefits of Agile adoption

  • Key Benefits:

  • Better management of Requirement and Priority changes, Risks

  • Improved time-to-market and customer satisfaction


Benefits perceived role wise

Benefits Perceived – Role wise

  • Ability to handle risks and improved customer satisfaction were rated high by Agile team and Agile Coach

  • Managing changing priorities and better project visibility were rated high by senior management and process professionals

  • Benefits perceived by organizations increase with Agile experience, inline with practices adopted and metrics collected/analyzed


Barriers to agile adoption

Barriers to Agile adoption

These observations are in line with the world wide survey reports by other organizations

Agile Coaches/Consultants and Senior Management to focus on methodical transition to AGILE


Barriers perceived role wise

Barriers Perceived – Role wise

  • Barriers reported by Senior Management and Process Professional are much higher than Agile Coach/Consultant and Agile Team

  • The size and Agile experience of the organization had no significant impact on the barriers


Benefits and barriers impact

Benefits and Barriers - Impact

  • Time-to-market

  • IT and Business Collaboration

  • Handle risks earlier/ better

  • Manage changing priorities

  • Better project visibility

  • Customer/End-user satisfaction

Customer

Management

  • Agile cannot handle Maintenance

  • Lack of skilled resources

  • Agile produces Insufficient documentation

  • Lack of coordinated transition to agile

  • Lack of focus among PM/PMO Groups

  • Resistance to change

  • Manage changing requirements

  • Better Team-morale

  • Better collaboration

Team

  • Project technical complexity / size

  • Lack of onsite - offshore collaboration

  • Lack of tools/ Automation for short iterations


Challenges expressed

Challenges Expressed

Challenges Faced

Possible Solution

  • Fixed Price Model is a challenge, T&M have worked well.

  • Collection of Traditional Metrics in Agile Projects.

  • Moving away from Command & Control; Self Organizing Teams

  • Scope creep and sign-offs.

  • Compensation / Appraisal related issues due to no individual Heroes

  • Communication/ Collaboration challenges between the distributed sub-teams.

  • Infrastructure needs for Continuous integration. Teams Not exposed to usage of tools and automation.

  • Frequent release cycles in sprints thus not enough time for testing. Very less focus on Non functional Test. Some legacy application testing cannot be fully automated.

  • No predefined performance benchmarks.

  • User story articulation by different stakeholders.

  • Focus on Maintainability and code quality.

  • Not all teams will be empowered or trained for ‘Test Driven Development’. Need External Support.

  • Can think of Hybrid model based on the scope clarity.

  • Redefine/Customize the traditional metrics.

  • Handholding by Agile coach. Mandatory Agile skill training.

  • Stakeholders to come together and decide.

  • Policies need to be flexible to have Team benefits

  • Address the dependency issues in Daily stand-up meetings. Effective Scrum of Scrums

  • Senior management to focus on infrastructure needs.

  • Teams to look into non-functional aspects during planning meetings. Detail planning of testing effort for manual testing

  • Benchmark within the organization . Participate in external Forums.

  • Breakdown stories for ease of development. Help customers articulate User Stories and Product Backlog.

  • Usage of Tools

  • Enhance People Agile skills; especially on Engineering side.


Best practices reported

Best Practices Reported

  • Record the retrospection and use some Agile tools for tracking them.

  • Stake holder involvement in defining test scenarios.

  • More encouragement from management to adopt Test Driven and Behaviour Driven Development.

  • Invest on tools and infrastructure to run automated regression suite on every check in

  • Representative at onsite for each offshore team: Overlap of the key people from offshore with onsite team.

  • Slightly heavy onsite team for better coordination.

  • Good build plan. Report invocation from Build.

  • Teams based on functionality rather than skills

  • Buddy Testing from developer machine to save the deployment time.

  • All the team members who are part of particular sprint should be part of the scope and estimation.

  • Measurement of code complexity for quantification.

  • Use of Agile training from time to time, participation in Agile conferences and Forums.


Conclusions and recommendations

Conclusions and Recommendations

Agile as a methodology is well established - more than 50% of respondents have used it for past 2 years and included it in QMS. Scrum is the most widely used methodology followed by Kanban

Agile usage is spread across multiple domains and project types; Over 75% of the respondents have confirmed using Distributed Agile

While Management practices (predominantly from SCRUM) have been well adopted, the adoption of engineering practices (predominantly from XP) needs more rigor

Usage of metrics and automation level increased with Agile experience

Coexistence of Agile with other industry standards (like ISO and CMMi) has been agreed upon

NEXT STEPS

Data Benchmarking of Agile projects

Practitioner Groups of CMMi and Agile can integrate the practices to adopt the best of both

  • Bibliography

  • CMMi for Development, Version 1.3 by SEI

  • “2011 State of Agile Development Survey Results” by VersionOne

  • “Agile adoption in India Survey Report – 2011” by ThoughtWorks

  • “Agile India 2012 Survey Results” by Valtech


Discussions

Discussions

THANK YOU


  • Login