Overview of the software process
Download
1 / 66

Overview of the Software Process - PowerPoint PPT Presentation


  • 93 Views
  • Uploaded on

Overview of the Software Process. Bill Lorensen GE Research (retired) [email protected] Course Wiki. http://public.kitware.com/OpenSourceSoftwarePractice. Outline. Motivation The Process Case Study – Insight Toolkit Lessons Learned. A Personal Look Back.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Overview of the Software Process ' - kaz


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Overview of the software process

Overview of the Software Process

Bill Lorensen

GE Research (retired)

[email protected]

Open Source Software Practice


Course wiki

Course Wiki

http://public.kitware.com/OpenSourceSoftwarePractice

Open Source Software Practice


Outline
Outline

  • Motivation

  • The Process

  • Case Study – Insight Toolkit

  • Lessons Learned

Open Source Software Practice


A personal look back
A Personal Look Back

  • 1976- NASA’s Computer Software Management and Information System (COSMIC)

  • 1978 - MOVIE.BYU

  • 1984 - GE Research Workstation

  • 1984 - LYMB

  • 1994 - Visualization Toolkit

  • 2000 - Insight Toolkit

Open Source Software Practice


Flattener #4

Open Sourcing

Self-organizing collaborative communities

Open Source Software Practice


Open source anecdote
Open Source Anecdote *

IBM Manager: “So, walk me through the development process for e-commerce. What’s the underlying web server?”

Developer: “It’s built on top of Apache.”

Manager: “Apache?”

Developer: “A shareware program for web server technology, produced for free by a bunch of geeks working online in some open-source chat room.”

Manager: “How do you buy it?”

Developer: “You download it off a Web site for free.”

Manager: “Who supports it if something goes wrong?”

Developer: “I don’t know, it just works!”

*from The World is Flat

Open Source Software Practice


Successful open source projects

Successful Open Source Projects

Open Source Software Practice


Visualization toolkit vtk
Visualization Toolkit - vtk

Open source toolkit for scientific visualization, computer graphics, and image processing

www.vtk.org

Open Source Software Practice


Computer vision library vxl
Computer Vision Library - vxl

  • Based on Target jr and Image Understanding Environment

  • Numerics

  • Imaging

  • Geometry

  • Camera models

vxl.sourceforge.net

Open Source Software Practice


Slicer
Slicer

  • Developed by Brigham and Womens Surgical Planning Lab

  • User interface plus plug-ins for applications

    • Segmentation

    • Registration

    • Image Guidance

http://www.slicer.org

Open Source Software Practice


Scirun
SCIRun

  • Developed by U Utah SCI Institute

  • Computational Workbench

  • Visual Programming

  • Modeling, Simulation and Visualization

http://www.sci.utah.edu

Open Source Software Practice


National library of medicine segmentation and registration toolkit
National Library of MedicineSegmentation and Registration Toolkit

$14 million over 6 years

Leading edge algorithms

Open Source Software

www.itk.org

Open Source Software Practice


Open source menu for success
Open Source Menu for Success

  • A Community with a common vision

  • A pool of talented and motivated developers/scientists

  • A mix of academic and commercial

  • An organized, light weight approach to software development

  • A leadership structure

  • Communication

  • A business model

Open Source Software Practice


Project stages
Project Stages

  • Startup

  • Early

  • Middle

  • Mature

Open Source Software Practice


Project stages startup
Project Stages - Startup

  • Define goals and/or requirements

  • Select developer community

  • Define developer roles

  • Select host server

    • Roll your own

    • SourceForge, CollabNet, GForge

  • Select software development tools

    • Revision control system

    • Compilers

    • Code coverage

    • Dynamic analysis

    • Build system

    • Testing system

  • Select an open source license

Open Source Software Practice


Project stages early
Project Stages - Early

  • Refine tools

    • Change ASAP if necessary

  • Establish coding guidelines

  • Setup communication channels

    • Mailing lists

      • Developers

      • Users

  • Establish web presence

    • Project home pages

    • Wiki for developers

  • Establish nightly build/test

Open Source Software Practice


Project stages middle
Project Stages - Middle

  • Establish release mechanisms

  • Establish policy to add additional developers

  • Establish backward compatibility policy

  • Establish procedure to add new features

  • Marketing

    • Papers

    • Tutorials

    • Web buzz

Open Source Software Practice


Project stages mature
Project Stages - Mature

  • Project has an installed base

  • Everything should be hard to change

    • Code

    • Tools

    • Process

Open Source Software Practice


Inside insight itk

Inside Insight (itk)

Open Source Software Practice


What is itk
What is itk?

  • A common Application Programmers Interface (API).

    • A framework for software development

    • A toolkit for registration and segmentation

    • An Open Source resource for future research

  • A validation model for segmentation and registration.

    • A framework for validation development

    • Assistance for algorithm designers

    • A seed repository for validation case studies

Open Source Software Practice


Insight open source products
Insight - Open Source Products

Open Source Software Practice


Itk by the numbers
itk by the Numbers

  • 400K

    • # of lines of code

  • 100K

    • # of lines of test code

  • 45K

    • # of lines of examples

  • 160K

    • # of lines of Applications

  • > 300

    • # weekly t-cons

  • 85

    • # unique developers

  • March 2000

    • First code checkin

  • 1600

    • # of nightly builds

  • 1200

    • # tests run nightly

  • 50

    • # of platforms

  • 730

    • # of classes

  • 2000

    • # of files with code

Open Source Software Practice


Itk by the numbers1
itk by the Numbers

  • 186

    • # of subscribers to the developers mailing list

  • 900

    • # of subscribers to the users mailing list

  • 350

    • # of monthly posts to users-list

Open Source Software Practice


http://ohloh.net

Open Source Software Practice


http://ohloh.net

Open Source Software Practice


A common vision
A Common Vision

Create a dynamic, self-sustaining, public domain and extensible toolkit that will empower researchers throughout the world to develop new segmentation and registration algorithms and create new applications that leverage the NLM’s investment in the Visible Human Male and Female data sets

Open Source Software Practice


The original team
The Original Team

  • Six prime contractors

  • Industrial

    • Kitware (Schroeder)

    • Insightful (Ng)

      • UPenn (Gee)

    • GE Research (Lorensen)

      • Brigham and Womens (Kikinis)

  • Academic

    • UNC (Aylward)

      • Pitt (Stetton)

    • Utah (Whitaker)

    • Rutgers (Metaxas)

      • Columbia (Imielenski)

      • UPenn (Udupa)

Open Source Software Practice


Insight consortium
Insight Consortium

  • A competitive process selected the development team

  • Each group was evaluated individually, without regard to how they might integrate with each other

  • Each team had strengths in software development, validation or algorithm development, but no one group had the necessary skills to do the entire project

  • Distributed software developers with varying software engineering experience

Open Source Software Practice


Insight software architecture
Insight Software Architecture

  • Object-oriented design

  • Generic Programming

  • Design Patterns

  • Frameworks

  • Separation of Algorithms from Interfaces

Open Source Software Practice


Object oriented design
Object-Oriented Design

  • Dominated software systems throughout the 1990’s

  • Continues to be the accepted software design technique

  • Particularly useful for dealing with complexity

  • Provides programmatic abstractions to deal with generalization and encapsulation

  • C++ and Java have mechanisms to support OOD

Open Source Software Practice


Generic programming
Generic Programming

  • Organize libraries consisting of generic—or reusable—software components.

  • The essential ideas of generic programming are containers to hold data, iterators to access the data, and generic algorithms that use containers and iterators to create efficient, fundamental algorithms.

  • ITK uses generic programming to process n-dimensional “images”.

Open Source Software Practice


Design patterns
Design Patterns

  • Good object-oriented software systems have recurring designs (patterns) that occur frequently

  • ITK employs a number of powerful design patterns

    • object factories

    • command/observer

    • smart pointer memory management

Open Source Software Practice


Frameworks
Frameworks

  • Define how a group of participants can be put together to solve a particular task.

  • Particularly suitable for describing complex flows or algorithms that have a number of steps that can be varied

  • ITK Frameworks

    • A demand-driven data processing pipeline that connects algorithms to process n-dimensional image data

    • Registration framework

    • Level-set framework

Open Source Software Practice


Registration framework
Registration Framework

Open Source Software Practice


Level set framework
Level-Set Framework

Open Source Software Practice


Separation of algorithms from interfaces
Separation of Algorithms from Interfaces

  • Implement the algorithms with a clear separation from the applications and especially the user interfaces.

  • Uses the Command/Observer design pattern that permits applications to watch for significant events during the execution of an algorithm

  • ITK has no built-in visualization, but has been interfaced to several systems including 3D Slicer, Analyze, SciRun and Volview.

Open Source Software Practice



Google hits extreme programming 1 860 000

Google Hits

“Extreme Programming”: 1,860,000


Extreme programming
Extreme Programming

Open Source Software Practice


A light weight software engineering process
A Light Weight Software Engineering Process

  • Based on the new Extreme Programming process

    • High intensity design, test, implement cycle

    • Supported with web-enabled tools

    • Automated testing integrated with the software development

Open Source Software Practice


Extreme programming1
Extreme Programming

Compresses the standard analyze, design, implement, test cycle into a continuous process

Open Source Software Practice


A process supported by a suite of portable open source tools
A process supported by a suite of portable, open source tools

  • Apache, perl, php

    • Web services

  • cvs, subversion

    • Revision control

  • MediaWiki

    • Collaborative content capture

  • Doxygen

    • Automated documentation

  • CMake

    • Cross-platform program build

  • Dart

    • Continuous and distributed test reporting

Open Source Software Practice


Extreme programming2
Extreme Programming tools

The community owns the code

Although the identity of the original author is kept, other developers are free to correct defects and enhance each other's code

In the end, all of the software should appear as though one author wrote it

Open Source Software Practice


Extreme programming3
Extreme Programming tools

Release early, release often

Although developers are tempted to keep their code under wraps until it is perfect, the process encourages them to release their code as soon as it passes some minimum tests

The longer the code is visible to the community, the better integrated it will be

Open Source Software Practice


Extreme programming4
Extreme Programming tools

Continuous integration

There is no scheduled porting to computer platforms

All new software builds supported platforms every evening

Open Source Software Practice


Extreme programming5
Extreme Programming tools

All developers agree to keep the software defect free

Although everyone is encouraged to submit their code early, the code must compile and pass tests nightly

A continuous build process sends e-mails to developers who check in code that does not compile

More effectively, the community enforces the commitment though peer pressure

Open Source Software Practice


Insight development cycles
Insight - Development Cycles tools

  • Daily – dashboard

  • Weekly – telephone conferences

  • Periodic – architecture reviews

  • Quarterly – developer meetings

  • Yearly – work assignments

Open Source Software Practice


Extreme programming daily testing is the key
Extreme Programming toolsDaily Testing Is The Key

  • Testing anchors and drives the development process (Dart)

  • Opens up the development process to everyone

  • Developers monitor the testing dashboard constantly

  • Problems are identified and fixed immediately

  • Developers receive e-mail if they“Break the Build”

Open Source Software Practice


How dart enables collaboration

Results posted on web tools(the dashboard)

How DART Enables Collaboration

CVS maintainssource code revisions

DART compilessource code, runs tests

CVS

Developers check-in code

Developers review results

Open Source Software Practice


Dart tools

Dart

Open Source Software Practice



Multi platform builds
Multi-Platform Builds tools

Open Source Software Practice


Regression Testing tools

Open Source Software Practice


Continuous System Monitoring tools

Open Source Software Practice


Someone broke the build! tools

Open Source Software Practice


Insight toolkit lessons learned

Insight Toolkit toolsLessons Learned

Open Source Software Practice


Lessons learned the good
Lessons Learned: The Good tools

  • Good mix of commercial and academic

  • Communication is critical

  • The daily rhythm of Extreme Testing

  • The Whole >>> Sum of the parts

  • Process automation reduces the burden on developers

  • The process is open and repeatable

Open Source Software Practice


Lessons learned the bad
Lessons Learned: The Bad tools

Communication

  • Early in the project, communication was limited to the mailing list and the Quarterly meetings. Almost a year passed before we added the weekly telephone conferences. Developers felt isolated and the impersonal electronic communication methods failed to build the personal relationships required for any software development process.

    Solution

  • Weekly developer t-cons

  • Wiki

Open Source Software Practice


Lessons learned the bad1
Lessons Learned: The Bad tools

Lack of Design Reviews

  • The lightweight process has its drawbacks. The current process emphasizes coding and testing. This process is well defined and benefits from automation. But, the design process is not well supported by the current mechanisms.

    Solution

  • Insight Journal review process

Open Source Software Practice


Lessons learned the bad2
Lessons Learned: The Bad tools

Unstable Application Programming Interfaces (API’s)

  • The nightly test/build was so effective in empowering programmers to make changes, that API changes occurred too frequently without the necessary buy-in from the development community.

    Solution

  • Established backward compatibility policy

Open Source Software Practice


Insight observations the bad
Insight Observations: The Bad tools

Insufficient Automation

  • Automation is critical to maintain consistent style and robust software. Although the project provided many automatic techniques to catch developer errors, the project would benefit from more automation.

    Solution

  • KWStyle – tool to do style checking

Open Source Software Practice


Insight observations the bad1
Insight Observations: The Bad tools

Complexity and Feature Creep

  • Like most software development projects, this software suffers from complexity and feature creep. Early designs and implementations used many features of the C++ language to an “extreme”. The team realized and began removing the complexity.

    Solution

  • Ongoing reviews

Open Source Software Practice


Lessons learned the ugly
Lessons Learned: The Ugly tools

  • Still need a stick

    • Someone needs to monitor the daily process.

  • XP is not everyone’s cup of tea

    • Almost all bought into the process. A few tolerated it.

Open Source Software Practice


Open source resources
Open Source Resources tools

  • http://www.opensource.org

  • http://sourceforge.net

  • http://gforge.org

  • http://www.itk.org

  • http://www.vtk.org

  • http://vxl.sourceforge.net

  • http://www.slicer.org

  • http://www.sci.utah.edu

  • http://www.google.com/codesearch

  • http://ohloh.net

Open Source Software Practice


Quiz monday reading list
Quiz Monday – Reading List tools

  • Free Culture, Lawrence Lessig,

    • Available online at: http://www.free-culture.cc/freecontent/

    • Sections to read

      • Introduction

      • Chapter Four

  • Free Software Free Society, Richard Stallman

    • Available online at

      • http://www.gnu.org/doc/book13.html (HTML)

      • http://www.gnu.org/philosophy/fsfs/rms-essays.pdf (PDF)

    • Sections to read

      • Chapter One: "The GNU Project"

      • Chapter Two: "The GNU Manifesto"

      • Chapter Three: "Free Software Definition"

      • Chapter Four: "Why software should not have owners"

  • Open Source Software Licensing, Lawrence Rosen

    • Available online at: http://www.rosenlaw.com/oslbook.htm

    • Sections to read

      • Chapter Two: "Intellectual Property"

      • Chapter Four: "Taxonomy of Licenses"

  • Retrieved from "http://public.kitware.com/OpenSourceSoftwarePractice/index.php/Intellectual_Monopolies_Reading_Assignement_I"

Open Source Software Practice


Overview of the software process1

Overview of the Software Process tools

Bill Lorensen

GE Research (retired)

[email protected]

Open Source Software Practice


ad