Software quality and management how the world s most powerful software makers do it
1 / 20

Software Quality and Management: How the World’s Most Powerful Software Makers do it - PowerPoint PPT Presentation

  • Uploaded on

Software Quality and Management: How the World’s Most Powerful Software Makers do it. Dien D. Phan Professor BCIS Department St. Cloud State University St. Cloud, MN 56301. Information Systems Management , Vol. 18, No.1, 2001. Software Quality Concepts.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Software Quality and Management: How the World’s Most Powerful Software Makers do it' - ulmer

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Software quality and management how the world s most powerful software makers do it

Software Quality and Management: How the World’s Most Powerful Software Makers do it

Dien D. Phan


BCIS Department

St. Cloud State University

St. Cloud, MN 56301.

Information Systems Management, Vol. 18, No.1, 2001.

Software quality concepts
Software Quality Concepts

Weinberg: Quality is relative. What is quality to one person may be lack of quality to another

IEEE: Quality is the probability of failure-free operation of software.

Fault is the cause of failure in software:

  • A failure is the departure of the external results of a program operation from requirements

  • A fault is the defect in the program that when executed under particular conditions, causes one or more failures.

Capability maturity integration model cmmi
Capability Maturity Integration Model (CMMI)

  • Level 1: Initial

  • Level 2: Repeatable. Config mgmt, Qual Assurance, Subcontract mgmt, Proj tracking, Proj planning, requirements mgt.

  • Level 3: Defined. Peer rev., Coordination, Prod. Engineering, Integr. Mgmt, Training, Org. process def., Org. process focus.

  • Level 4: Quantitatively Managed. Softw. Quality Mgmt., Quant. process mgmt.

  • Level 5: Optimized. Process change mgmt, Technology change mgmt, Defect prevention.

Impacts of cmm on software quality
Impacts of CMM on Software Quality

  • Putnam (1994): For organizations that moved from level 1 to 3:

    • Time reduced by a factor of 1.7

    • Peak staff reduced by a factor of 3.2

    • Efforts reduced by a factor of 5.7

  • Herbslef et al (1997):For organizations that moved from level 1 to 3:

    • 5 to 1 ROI at Hughes Aircraft

    • 7.7 to 1 ROI, 75% decrease on rework, 190% increase in productivity, and decrease in error rates per KLOC from 17.2 to 4.0 at Raytheon

Common quality measurements
Common Quality Measurements

  • Mean time between failure

  • Mean time to repair

  • Defect rate by time

  • Defect rate by size

  • Defect backlog size

  • Number of clean LOC passed first QA attempt

  • Cumulative defects per version

  • Timeliness to fix defect

  • Customer satisfaction level

  • Rate of defects per unit of software size discovered after first year of delivery

Quality control processes at ibm os 400 development
Quality Control Processes at IBM -OS/400 Development

  • During development:

    • Scope control: Requirements and resources control

    • Reviews: HiLvl Design, LoLvl Design, and Code reviews

    • Customer involvement

    • Tests and defect removal activities

    • Development and quality control tools: IDSS, DCR, PTR, and quality models

  • After delivery:

    Quality control for services, administration, and documentation.

Causes of bad fixes
Causes of bad fixes

  • Not thorough enough testing

  • Incorrect instructions to solve problems

  • Not all steps tested

  • Incomplete test bucket (data, programs, scenarios)

  • Inadequate reviews

  • Inability to test user hardware environment

  • Insufficient testing of special installations


  • Defect rate

  • Level of user satisfaction and complaints

    • Numbers of service calls decreased by 50%

    • Cost of service calls decreased by 20-40%

    • System installation time reduced to 2 weeks

    • Documentation errors reduced by 50%

More recent studies at ibm
More Recent Studies at IBM

  • Kaplan et al. (1995): Secret of Software Quality: 40 Innovations from IBM

    • IBM Santa Theresa lab: defects decreased by 46%, service costs decreased by 20%, revenues per employees increased by 58%, and customer satisfaction increased by 14%

    • Mixed results on the cleanroom process

The cleanroom process
The Cleanroom Process

Rather than developing a software and then working to remove defects, the cleanroom approach demands the discipline required to eliminate defects in specification and design then implement in a “clean” maner. The reasons that it is not widely used today are:

  • It’s too theoretical, mathematical, and radical

  • It abandons unit testing

  • It uses statistical (randomized testing) and thus cannot effectively test weak spots in software

  • It requires a high level of maturity in software development process

Quality management at microsoft
Quality Management at Microsoft

Agile process using small teams: team decides how the work get done

Process Model

Daily nightly build synchcronization and stabilization process
Daily (nightly) build/synchcronization and stabilization process

  • Check out source code to work ->private copy

  • Tool: Diff to find the difference between master source since the time code is checked out

  • Submit private copy (merging) to “quick test” before daily deadline (daily/nightly-build, daily synchronization and stabilization)

  • Problem: “Breaking the build”

  • Solution: Use Build-Master in daily-build

Microsoft s quality strategies
Microsoft’s Quality Strategies process

  • Grow rather than design and building software,

  • Change or replace 50% of code after each release,

  • Change in design must be justified by features that are at least twice as good,

  • Build multiple version simultaneously,

  • Develop software at single site,

  • Continuously test the products as software grows, and

  • Use Adaptive Software Development (ASD) process to build web-based software for clients

Ibm microsoft quality capabilities measured by sei s cmm
IBM, Microsoft Quality Capabilities measured by SEI’s CMM process

Legend: Y= Yes

- = Not defined

Lessons learned
Lessons learned process

  • Development Processes

    • IBM focuses on requirements, Microsoft focuses on vision

    • IBM processes are development oriented, less trade off resources, features, and schedule. Microsoft’s are customers oriented, more trade-offs allowed at each milestones.

    • Microsoft half-life (50% replacement) reduced software reuse

    • Microsoft teams enjoy greater freedom in the planning and development phases

Lessons learned1
Lessons learned process

  • CMMI:

    • IBM is at level 4 or 5

    • Microsoft is at level 3

  • Are there any Silver Bullets?

Key success factors
Key success factors process

  • Feedback, On-line monitoring,

  • Project management techniques, quality control standards, quality processes,

  • Team members are responsible of quality of components being built,

  • Test early, and test frequently: frequent synchronization,

  • Software reuse,

  • Quantitative Defect Removal Models,

  • Test cases should be ready before development.

  • Version Control, Configuration Management.

Conclusion process

  • IBM relies on both process and quantitative defect models: traditional software development oriented

  • Microsoft relies on its process model: agile development oriented.

  • Agile methodology is popular in Web projects