Guilt-Free Agile Development

1. Guilt-Free Agile Development Plan-Driven or Agile–Why Choose When You Can Have The Benefits of Both? John ManzoManaging Partner USC/CSEResearch Review March 18, 2004

2. Topics • Who Are We, And Why Might Our Perspective Be Of Value? • The Limitations of Plan-Driven and Agile Methods • A Brief Overview Of Agile+ - Best of Breed Practices from ADMs and Plan-Driven Approaches • Overcoming the Limitations of Agile Development – Preserving the Baby While Throwing Out The Bath Water • Wrap Up • Q & A

3. AgileTek Company Snapshot • A privately held professional services firm specializing in custom software development and software advisory and training services • Decades of experience • Comprised of distinguished software professionals • Led by a highly experienced management team • Conveniently located in Chicago (Just a 5-minute drive from O’Hare Airport) • Financially stable and profitable • No debt • No investors

4. AgileTek Company Details • We were early pioneers in Agile Development and among its first practitioners (our first eXtreme Programming project was conducted prior to Kent Beck even publishing his first book on XP) • We've also continuously refined our agile development methodology over the past six years... based on learning from real-world projects across a broad spectrum of application domains • AgileTek tends to work with clients that are developing new products with dynamic and changing requirements where time to market and the need for tight user feedback loops are critical factors

5. AgileTek’s Service Offerings • Custom Software Development and Co-Development • Advisory Services • Agile+ ,Agile, and Hybrid Development Consulting • Architecture Evaluation/Development • Project Risk-Based Assessments • Project Health Checks and Monitoring • Project Turnarounds/Rejuvenation • Software Size-Time-Effort Estimation • All-Aspect Software Development Environment Evaluation • Software Engineering Management • Managing Complex Systems Development • Training/Workshops • Managing Complex Systems Development • The Power of Architecture • Pre-Launch/Re-Launch Workshops • Custom Tailored Courses

6. Why Clients Engage AgileTek • Accelerate time to market • One of the highest productivity indices (PI) in our industry • Average of 35 LOCs per coding hour • Ability to accommodate change at minimum cost • Higher quality • 0.5 defects for 1KLOC (~5-sigma, 6-sigma on life or safety-critical projects) • Experience with emerging technologies • Pragmatic, battle-tested approaches Wireless XML XSL

7. Preamble “They [Barry and Rich] remind us: If one has strong discipline without agility, the result is bureaucracy and stagnation. Agility without discipline is the unencumbered enthusiasm of a startup company before it has to turn a profit.” Alistair CockburnIn his Foreword to: Balancing Agility and Discipline

8. AgileTek’s Secret Sauce • Top Software Engineering talent • Decades of real-world experience developing complex systems • An extremely effective Software Development Methodology influenced by: • Contemporary Agile Methods • Airlie Council’s Principal Best Practices • Traditional battle-tested methods

9. Complex DOD System Test Beds For The Traditional Methods Used In Agile+

10. Complex Computer System Test Beds For The Traditional Methods Used In Agile+

11. Complex Microprocessor Array Test Beds For The Traditional Methods Used In Agile+ 9.3 Million Transistors

12. Complex Telecommunication System Test Beds For The Traditional Methods Used In Agile+

13. 1. Norm Brown (Director) 6. Capers Jones 11. Tim Lister 12. Tom McCabe (not pictured) 2. Vic Basili 7. Tom DeMarco 13. Grady Booch (not pictured) 3. Frank McGrath 8. LarryPutnam 14. Roger Pressman (not pictured) 4. Ed Yourdon 9. John Manzo 5. Lou Mazzucchelli 10. Mike Evans The Airlie Council Developed The Nine Principal Best Software Practices For DOD

14. Contemporary Agile Methods

15. Impetus For A Best-Of-Breed Approach Traditional Plan-Driven Methods can be highly effective but, they have their limitations. They often: • Deny the need for software to evolve and assume that requirements are stable, and can be determined before initiation of coding activities • Define processes and plans with such detail and rigidity that it is difficult to deal with inevitable contingencies • Cause more emphasis to be placed on checking off a task than on providing value to a customer • Diffuse accountability and tend to invite bureaucracy • Result in documentation bloat • Put too many layers of formality between the developers and a palpable understanding of the problem and the state of the project

16. The Agile Software Manifesto • We are uncovering better ways of developing software by doing it, and helping others do it. Through this work we have come to value: • Individuals and interactions over processes and tools • Working software over comprehensive documentation • Customer collaboration over contract negotiation • Responding to change over following a plan • That is, while there is value in the items on the right, we value the items on the left more. February 2001 Jim Highsmith (+…)* *…15 others representing XP, and several other Agile Development Methods (ADMs)

17. Impetus For A Best-Of-Breed Approach XP (and other ADMs) can be highly effective… but, they have their limitations. They are: • Only predictable on a micro versus a macro level • Difficult to scale • Vulnerable to changes at the system level • Vague about the nature of tests • Inflexible to special needs • Impractical in some respects • Lack explicit management of risks

18. Overcoming the Limitations Adjusting and Extending XP Specific Approaches

19. AgileTek’s Secret Sauce = Agile+ Agile+ is… Contemporary Agile Development Methods + Best of the Last 20 Years of Proven Best Practices  Agile+ = XP +

20. Agile+ And XP  Agile+=XP+ • + Business Process Analyses (BPAs) • + Story “Actors” • + Delphi-STE Estimation • + Risk-Based Situation Audits (RBSAs) • + Componentized Architecture • + Wall Gantts and Instrument Panels • + Automated Contract and Regression Testing • + Automatic Document Generation • Strict Pair Programming • 40-Hour Work Week Restriction • + Flexibility to meet special needs

21. Meeting at the Wall-Gantt Everyone at any time can see the picture emerge and evolve. They can see how the whole depends on their work and how their work is connected to every other part of the effort

22. Overcoming the Limitations Adjusting and Extending XP Specific Approaches

23. Most ADMs Are Unpredictable At A Macro Level • Solution: • Delphi Estimation • STE usage for larger projects

24. Delphi And The Oracle

25. Size - Time - Effort Chart PI - Productivity Index MBI - Staff Buildup Index Region of Feasibility Region of Infeasibility

26. Beta-Distributed Risk Profile

27. Most ADMs Are Notoriously Difficult To Scale • Solution: • Componentized Architecture - Interface Definitions • Automated Build and Test Processes • (Virtual) Team Rooms

28. So Just What Is “Architecture?”(A Working Definition) Architecture - the structure of the components of a program/system, their interrelationships, and the principles and guidelines governing their design and evolution over time1. [1] David Garlan and Dewayne Perry, guest editorial to the April 1995 IEEE Transactions on Software Engineering devoted to software architecture

29. Some Commonly Ascribed Advantages/Benefits • Architectures permit independent evolution of the exploitation of changing technologies in the major elements of a system. • Architectures ease sharing of data and building of distributed applications. • Architectures ease communications and movement of data across levels. • Architectures facilitate assimilation of new employees. • Architectures preserve investment in training. In Essence, Architectures Enable and Facilitate Scaling and Distributed Development

30. Automated Build and Regression Test Processes Reduce Communication Load

31. Team Rooms Provide An Additional Dimension For Collaboration and Coordination

32. Most ADMs Leave The Software Vulnerable To Changes At the System Level • Solution: • Componentized Architecture

33. Good Architectures Ease The Effects Of Change… They keep visible the interrelationships among the elements of the system and help the team make informed change decisions in the context of these interrelationships They limit the degree of change and its sphere of influence

34. Most ADMs Are Vague About Testing • Solution: • Use Automated Contract and Regression Testing

35. Contract Testing Involves Testing For: Pre-Conditions Post-Conditions Class Invariants

36. Most ADMs Are Inflexible To Special Needs • Solution: • Treat the Special Need as a User Story and prioritize it accordingly

37. Documentation As An Example Of A Special Need Systems created under the auspices of a regulatory agency will require certain documentation specified by CFRs1 This can be treated as just another software deliverable and prioritized accordingly [1] CFR – Code of Federal Regulations

38. Some ADM Practices Are Impractical • Solution: • Use practices that make sense and work in real-world situations • Abandon or modify those that don’t

39. ADM Practices We’ve Found Impractical (Some Examples) • Pair Programming rigidly applied • Refusal to estimate total project size, time and effort • Building without at least a “sketch” • On-site customer • Weekly/bi-weekly iterations • 40-Hour work week

40. Most ADMs Do not Manage Risks Explicitly • Solution: • Use Risk-Based Situation Audits • Establish a risk management philosophy

41. ADMs And Risk Most ADMs manage risk through project transparency This is highly effective Adding up-front risk characterization and a philosophy of on-going risk mitigation yields superlative results If you’re not managing risk, you are not managing1 [1] Airlie Council

42. Do ADMs Add Risk To A Project? • The greatest risk in software development is not knowing what you don’t know until it’s too late… • Every Agile+project begins with an RBSA; then, • Though short iterations,Agile+provides a means to: • Challenge assumptions • Make informed choices and decisions • Evolve understanding • …early and often • Moreover, • Project progress is continuously visible to the developers • Wall Gantts • Daily Stand-Up Meetings • … and to our clients • Placed at the project’s center • Frequent reviews • Unlike some methodologies where Risk Management is “bolted on”, it’s an integral part of Agile+

43. The Agile Software Manifesto • We are uncovering better ways of developing software by doing it, and helping others do it. Through this work we have come to value: • Individuals and interactions over processes and tools • Working software over comprehensive documentation • Customer collaboration over contract negotiation • Responding to change over following a plan • That is, while there is value in the items on the right, we value the items on the left more. False Dichotomy February 2001 Jim Highsmith (+…)* *…15 others representing XP, and several other Agile Development Methods (ADMs)

44. The AgileTek Software ManifestoA Measured And Balanced Approach • We are uncovering better ways of developing software by doing it, and helping others do it. Through this work we have come to value: • Individuals and interactions over processes and tools • Working software over comprehensive documentation • Customer collaboration over contract negotiation • Responding to change over following a plan • That is, while there is value in the items on the right, we value the items on the left more. with streamlined with appropriate balanced by while being guided by That is, while we most value the items on the left, we also place importance in the items on the right

45. What Is Agile—Really? Agile  No Process Agile  Undisciplined Agile  Essential, Streamlined Processes Agile  No Nonsense, Intensely Practical Agile  Bias Toward Action Agile  Highly Disciplined Agile  Flexible and Adaptable Agile  Execution With Swiftness And Precision

46. Preserving The Essence Of Agility • Eliminate gratuitous processes, and rework any processes and behaviors that are not intensely practical • Wherever possible, minimize formality and ceremony • Maintain a bias toward action—come up with a 70% plan and begin to move forward while figuring out the remaining 30% as you go • Develop momentum quickly and maintain it • Stay flexible and adaptable • Know the “why” of processes and do what makes sense when the rules don’t

47. Can This Really Work And Is It Really Agile? I’m From Missouri—Show Me! The Numbers The Testimonials AGILETEK

48. Productivity and Quality • Using Agile+ we’ve achieved a PI1 of 22 • That translates into an average of ~ 35 ESLOC/Coding Hour • Our average defect rate is 0.5/KLOC ~ 5, higher for safety critical software • [1]Our most recent audit revealed an overall average Productivity Index (PI) of 22 (as defined by Larry Putnam in his Measures of Excellence: Reliable Software On Time, Within Budget). This index is a management scale corresponding to the overall process productivity achieved by an organization during the main software build. An index of 25 has been considered among the highest ever recorded.

49. A Legacy of Satisfied Clients With Whom We’ve Used Our Process

50. Some Recent Clients…