There is no magic
This presentation is the property of its rightful owner.
Sponsored Links
1 / 30

There is no magic Bob Martin [email protected] PowerPoint PPT Presentation


  • 80 Views
  • Uploaded on
  • Presentation posted in: General

There is no magic Bob Martin [email protected] “I assume you will show the alternative ways to use a cage, a whip, and a chair to manage software teams?” - Stu Feldman, VP Engineering Google. About Me. Built a lot of software ~100M lines of code System software Operating systems

Download Presentation

There is no magic Bob Martin [email protected]

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


There is no magic bob martin bobmartin08008 mac

There is no magic

Bob Martin

[email protected]

  • “I assume you will show the alternative ways to use a cage, a whip, and a chair to manage software teams?” - Stu Feldman, VP Engineering Google


About me

About Me

  • Built a lot of software ~100M lines of code

    • System software

      • Operating systems

      • Programming languages

      • Transaction control

    • Network management

    • Service control points

    • Embedded systems

      • Packet switches

      • Network equipment

      • Testing equipment

  • Love technology

    • EE and CS trained

    • Life long passion – now neural science, molecular & evolutionary biology

      • Help me convince Al to work in computational biology!

  • Not expert in contemporary software technology/tools

  • Conned into this talk by a great friend of 42+ years


Facts of life

Facts of Life

  • Failure is the norm

    • But it needn’t be

  • Error correction costs increase exponentially during the lifecycle

  • Programmers are a special breed*

    • They like machines not people

    • They have high need for approval/recognition

    • They must respect the leader

    • Beware, complicators and prima donnas lurk in the weeds

  • Software engineering is complexity management

    • Race with technology/tools

    • Complexity is still ahead

  • Good software engineering is old hat

    • But is not always used - why????

      “Two things are infinite: the universe and human stupidity; and I'm not sure about the the universe” - Einstein

* “Psychology of Computer Programmer” - Gerald Weinberg


Why software projects fail

Why Software Projects Fail *

Average overrun: 89.9% on cost, 121% on schedule, with 61% of content

* Barry Boehm - A View of 20th and 21st Century Software Engineering


Increase in software cost to fix vs phase 1976

Increase in Software Cost-to-fix vs. Phase (1976) *

1000

1000

500

500

200

200

100

100

Relative cost to fix defect

50

50

20

20

Smaller Software Projects

10

10

5

5

2

2

1

1

Requirements Design Code Development

Requirements Design Code Development

Acceptance Operation

Acceptance Operation

test

test

test

test

Phase in which defect was fixed

* Barry Boehm - A View of 20th and 21st Century Software Engineering


What year was this conference

What Year Was This Conference?

NATO Software Engineering Conference - 1968

Design

  • Guidelines

    • External function, e.g., user language

    • Internal function, e.g., parsers

  • Techniques

    • Deductive or inductive

    • Modularity and interfaces

    • Complexity control

  • Proscriptions

    • Completeness, modularity, efficiency

    • Self-monitoring and performance improvement

    • Incremental systems

    • Balance

      • Security, control, … vs. cost

      • Limited goals to attain excellent performance

  • Design problems

    • Data structures on system design

    • Fixed resources

    • Cooperating processes

  • Documentation

Production

  • Organization for producing software

    • Number and quality of people

    • Structure of large groups

    • Control and measurement

    • Internal communication

  • Production techniques

  • Monitoring

    Service

  • Distribution

    • Media

  • Acceptance criteria

  • Documentation

  • Adaptation

  • Maintenance

    • Error detection & reporting

    • Response and distribution

  • Documentation

  • Performance

  • Feedback into design


Impact of increased discipline

Impact of Increased Discipline

0.8

Industry-wide

0.7

Company X

0.6

0.5

0.4

0.3

0.2

0.1

0

1

2

3

4

  • Low software engineering maturity results in:

    • Reduced productivity (35%)*

    • Late detection of defects (22%)*

    • Longer time to market (19%)*

    • Higher post-release defect rates (39%)*

    • * median annual improvement

CMM** process maturity

  • Takes several years of focused improvement to become a high-performance software team

  • Focus on the programmer

    • Just management: well-controlled junk

  • Delicate touch to avoid process imprisonment

CMM Level

** Software Engineering Institute’s

Capability Maturity Model (CMM)


Learning

Learning

  • Disasters

    • Mine

    • Others

  • Really good companies

    • IBM - Discipline & inspections

    • Sun - Tempo

    • Japan - Quality circles & reuse

    • Microsoft - Reuse

    • Apple - User centered design & tool kits, “Design of Everyday Things” - Don Norman

  • Really smart professionals

    • Al Aho – Swamp drainer

    • Fred Brooks –“Mythical Man Month,”“No Silver Bullet”

    • Barry Boehm – Estimation and cost of errors

    • Michael Cusumano – Software factories

    • Edsger Dijkstra – Importance of time in systems

    • Watts Humphrey - Encapsulated team and personal discipline - “TSP, PSP”

    • Martyn Thomas - Formal methods

    • Ken Thompson - The programming craft


Two troubled projects

Two Troubled Projects

Switch

Test Head

PDP 11/20

Repair

Service

Bureaus

PDP 11/70

IBM 370

“Temporary”

Subsystem

“Robust

Pipe”

Univac 1100 with C

  • Loop Maintenance System

    • IBM MVS system

    • PDP 11/20 backup

    • 8 Months to go

      No schedule

      No performance plan

      No requirements

      Weak management

  • Loop Assignment System

    • Univac system

    • Failed trial

      • $250 million down the drain

      • Demoralized, weak team

    • Senior 18-month commitment

      No architecture or requirements

      No technology infrastructure

    • Working “temporary” subsystem

      • Unix based


Building the team

Building The Team

  • All - domain expertise, discipline & simplifiers

  • Team leads – programming, lateral & motivational skills

  • Architect – deep system software & computer science

    • Great ones are really rare

      “Everything should be made as simple as possible ... but not simpler” - Einstein

  • Specification - user-centered design & customer skills

  • Design/programming – proud craft persons

    • Read great books to learn to write, read great programs to learn to program

      “Lion’s Commentary on Unix” - John Lions

  • Testing – sadists & analysts

  • Performance – back of envelopers

  • Quality - superb communicators

  • Tools & environment – geeks emeritus

  • Product manger coordinated extended team

    • Sales, marketing, pricing, legal, service, …

      “Executives owe it to the organization and to their fellow workers not to tolerate nonperforming individuals in important jobs” - Drucker


Process is not a four letter word

Process is not a four-letter word

Iterative Definition/Development

b1

b2

b3

b4

b5

Integration, Test, & Trial

Phased Deployment

  • Early on

    • Architecture

    • New technology/risk

    • Performance

    • Factory operations

      Handoffs

      Automation

      Configuration control

  • Later with increasing process discipline

    • Features

    • Quality

R1

R2

R3

  • Iterative by many names

    • Build a little, test a little, refine requirements a little

    • Entrance/exit criteria

  • Templates & best cases

    • Lightweight

    • Web

  • Inspections & reviews

    • Requirements, design, code, test, etc

    • Lightweight to heavyweight

    • Especially on tough stuff and with new people

  • Lead customers

    • User-centered design

  • Screw-up early detection & action

    • Jeopardies and speaking up

    • Audits with smart friends

“I had the world’s greatest group, and I should have made the world’s two greatest groups. I didn’t realize there are benefits to having real implementers and real users” - Alan Kay


What s your favorite bug

What’s your favorite bug?


Architecture

Architecture

“Controlling complexity is the essence of computer programming” - Kernighan

  • Top-down decomposition of a complex problem to allow independent, bottom-up development and change

  • Goals

    • Changeability/customizability

    • Chunk independence

      • Reduce n2 effects

    • Inject huge-payoff CS technology

      • Tiny languages, protocols, finite state machines, search, algorithms & data structures, formal methods, etc

    • Performance

    • Reuse

  • Check it

    • Prototype

    • Audit with smart friends

  • Crumble with time

    • “An Architecture History of the Unix System” - Feldman Usenix ‘84


Calling the shot

Calling the shot

  • The true art

    • Best to wait until specification/design done

    • Domain expertise crucial

    • Sizing models help, but not magic (still +/- 30%)

      “Let the numbers do the talking” - Don Reifer

  • Aggressive but doable

  • Starvation not gluttony

    • Reduce communications, complexity & feature creep

      “Whilst it is true a large programming project might require a large programming team, a large programming team will always build a large project” - J. K. Buckle “Managing Software Projects”

  • Slips

    • If you must, do it once

    • Iterative development provides an out

      “A customer will always forgive you a slipped schedule or missed feature, they will never forgive you for bad quality or performance” - Buckle


Which project used 15x staff of the other

Which Project Used 15x Staff of the Other?

  • Loop Maintenance System

    • IBM MVS system

    • PDP 11/20 backup

    • 8 Months to go

      No schedule

      No performance plan

      No requirements

      Weak management

Switch

Test Head

PDP 11/20

Repair

Service

Bureaus

PDP 11/70

IBM 370

  • Loop Assignment System

    • Univac system

    • Failed trial

      • $250 million down the drain

      • Demoralized, weak team

    • Senior 18-month commitment

      No architecture or requirements

      No technology infrastructure

    • Working “temporary” subsystem

      • Unix based

Temporary

Subsystem

“Robust

Pipe”

Univac 1100 with C


How many lines of code

How many lines of code?

Determine the frequency count of the occurrence of words in text, most frequent first

  • “1” - Six Unix tools, piped together

  • 100 - C, Pascal

  • ~600 - ACM paper on commenting code

  • 1,000 - Cobol, PL/1

  • 5,000 - Unisys SW VP

  • 10,000 - IBM SW VP

  • ??? with very detailed requirements


Tempo

Tempo

“.. the disaster is due to termites, not tornadoes; and the schedule has slipped imperceptibly but inexorably. Indeed, major calamites are easier to handle; one responds with major force…” - Brooks

  • The plan

    • Delicate balance of top/down and bottom/up

    • PERT to plan, GANTT to manage

  • Critical path control

    • Don’t multitask

    • Buffer schedule

    • 50% estimates

  • Railroad train feature loading

    • Leave the station on time

    • Features left behind

  • The weekly/biweekly project meeting

    • Milestones

      • All milestones are important, particularly early ones and few big ones

      • Build team morale, tempo, customer confidence

        “No plan survives contact with the enemy” - Helmuth von Moltke the Elder


The project meeting

The Project Meeting

“The Screaming”


Whipping the rowers

Whipping the rowers

  • Project manager style

    • Most decisions are 50/50

    • Avoid acting like “A glacier with dignity”

      “An officer in charge on an Indian agency made a requisition in the autumn for a stove costing seven dollars, certifying at the same time that it was needed to keep the infirmary warm during the winter, because the old stove wore out.”

      Then, after glacial dignity

      “The stove is here. So is spring” *

* “An Autobiography” - Theodore Roosevelt


Testing

“Program testing can be a very effective way to show the presence of bugs, but it is hopelessly inadequate for showing their absence” - Dijkstra

Full lifecycle activity

One day build and test (tempo!)

Multi-level

Automated

Domain specific tools

Field driven test libraries

What to test

Requirements & design

Coverage

Usability

Performance

Measure with S curves

Testing

Buy death

march whip

}

Number

Announce new

features

Time

Test cases run

Bugs found

Bugs closed


Get your s curve you can t make this stuff up

Get your S-curve You can’t make this stuff up


Communications documentation

Communications & Documentation

“Schedule disaster, functional misfits, and systems bugs all arise because the left hand doesn’t know what the right hand is doing” - Brooks

  • Project meetings

    • Hierarchical

    • Weekly or biweekly

  • Jeopardies are good

  • Documents

    • “The act of writing forces hundreds of tiny decisions” - Brooks

    • Few key ones, all template driven, all reviewed/inspected

      • Requirements, architecture, design, test & project plan

      • Strong version and configuration control

  • Code

    • Structure - aim for one pagers

    • Comments

      • “This section of code is cursed” - Stroustrup C++

      • “You are not expected to understand this” - Unix


Metric driven improvement

Metric Driven Improvement

Number

Severity

4 3 2 1

4 3 2 1

4 3 2 1

….

>6 2 1

.

Age

.

Project

Teaches

.

.

Good

Organization

Learns

.

.

Project

Learns

Installability

Usability

Service

Performance

Functionality

Quality

Organization

Project

  • Quality

    • Measures

      • Fault density

      • Service responsiveness

    • Improvement

      • Root cause analysis

      • Team & individual

  • Productivity

  • Customer satisfaction

    • Huge fault density correlation


A project specific metric

A Project-Specific Metric

  • Project characteristics

    • Gluttonously staffed ~ 5,000 people

    • Long crumbled architecture - 20+ years

    • Kryptonite-weight processes

  • Humongously successful

    • 99.9999 uptime

      • Down 30 seconds a year for all causes

    • Drug dealer cash flow

  • Proposed programmer metric

    • Lines of code

    • Meetings attended

  • Improvement objective: exceed 1!


Contrarian views

Contrarian Views

  • Design methodologies

    • Great designers make great designs

    • Methodologies make lousy designs look good

  • Fancy measures

    • Confuse precision with accuracy


Making it stick

Making It Stick

  • People support what they invent

  • Thick dictates are good only for cellophane

    • Bellcore’s was ten one-pagers (what not how)

  • Participatory multi-day course

    • Crusty expert led

    • Project team takes course

      • Time it with a new release schedule

      • Output is their plan & their process

  • Team mentors

  • Senior support


My most notable disasters

My Most Notable Disasters

  • “Hi ho silver”

    • Loop maintenance system

  • Domain ignorance

    • Customer ordering system

      “Organizations can only do damage to themselves and to society if they tackle tasks that are beyond their specialized competence, their specialized values, their specialized functions” - Drucker

  • Second system effect

    • Switch assignment system

      “This second is the most dangerous system a man ever designs. …The general tendency is to over-design the second system, using all the ideas and frills that were cautiously sidetracked on the first one.” - Brooks


Maybe a little magic

Maybe a little magic

  • High quality, on time useful software can be built

    • Bellcore after six-years of continuous improvement (1994)

      • 95% on time/content

      • 3-4x competitive productivity

      • 10x competitive quality

  • It’s all about people

    • Picking and developing the best and getting rid of the worst

    • Helping them select and use good software engineering practices

    • Motivating and rewarding them for

      • Using and improving these practices

      • Behaving as teams

    • Giving them the time and resources to do their jobs

      • Aggressive but realistic goals

    • Making and holding them accountable

  • Crowd control can be fun


Things you might consider trying other than shooting alfie the shot caller

Things you might consider trying(Other than shooting Alfie “the shot-caller”)

  • Development process

    • Iterative development

      • Entrance/exit

    • Design template

    • User-centered design (pair with another team)

    • Inspections

      • Heavy and lightweight

    • Performance model

    • Defect metrics & root cause analysis

    • Tempo control

      • Project meeting

      • Milestones

      • Railroad

      • Jeopardy

  • Outside audit (pair with another team)

    • Changeability/customization

  • Automated testing

    • System & module

    • S curves

  • Lots of extra-bold Starbucks

    • Two shots


There is no magic bob martin bobmartin08008 mac

Final

That's All Folks!!!


  • Login