software design verification and validation l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Software Design, Verification and Validation PowerPoint Presentation
Download Presentation
Software Design, Verification and Validation

Loading in 2 Seconds...

play fullscreen
1 / 29

Software Design, Verification and Validation - PowerPoint PPT Presentation


  • 146 Views
  • Uploaded on

Software Design, Verification and Validation. Elements of a Code Integrity Management System for Real-time Embedded Systems. Joao H. Silva, Ph.D. jsilva6@visteon.com joaosilva@comcast.net. Problem Definition. Typical Scenario: 1 Senior Manager  178 – 200 project releases

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Software Design, Verification and Validation' - holland


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
software design verification and validation

Software Design, Verification and Validation

Elements of a Code Integrity Management System for Real-time Embedded Systems

Joao H. Silva, Ph.D.

jsilva6@visteon.com

joaosilva@comcast.net

problem definition
Problem Definition

Typical Scenario:

  • 1 Senior Manager  178 – 200 project releases
  • 2 releases with major Problems
  • 176 releases without any major problem

Question:

  • How do we anticipate those two “Bad Releases” before they become a problem
consider these statistics
Consider These Statistics
  • Statistic #1: For every thousand lines of code developed by software engineers, there could be as many as 20 to 30 defects on average
  • Statistic #2: In 2003, Developer News reported 50 percent of all bug fixes are incorrectly performed the first time, often introducing new bugs in the fixing process
  • Statistic #3: As bugs progress through the development cycle, defects found become exponentially more expensive to fix – it is at least 30 times more costly to fix software in the customer versus during development – SEI 2001
  • For automotive releasing a bug into the field may promote a very costly recall of that vehicle(s)
software defects
Software Defects
  • Software defects are inevitable.
  • “…people who write software are human first and programmers only second – In short, they make mistakes, lots of them...”

The Economist 2003

where is the complexity
Where is the complexity?

Avionics

Automotive

2010 Premium  100 M LOC

1995 – 2000  52%/Year

2001 – 2010  35%/Year

Tony Scott, GM CIO

  • Boeing 747  0.4 M LOC
  • Boeing 777  4 M LOC

Technology Review 2002

use of models
Use of Models
  • CMM
  • CMMi
  • Complexity Models
  • Metrics Maturity Model
  • TMM –Testing Maturity Model
  • Others
a model to understand complexity
A model to understand complexity

PROBLEM

HW Solution #1

HW/SW Co-Design

SW Solution #1

COMPLEXITY

Error

Proneness

Size

EFFORT

Reliability

Usability

Change

Maintainability

Understandability

MIPS

Productivity Solution #1

Quality Solution #1

what influences the complexity of sw
What influences the complexity of SW?

Size

Error-Proneness

Environment

Competence

Methods

Unnecessary functionality

Etc…

  • Module length
  • Tools
  • Language
  • Features
  • Management
  • Reuse
  • Outsourcing
  • Unnecessary functionality
  • Etc…
it is a multi dimensional problem
It is a multi-dimensional problem

Cost Reduction

Performance improvement

Meet Requirements

Architecture roadmap

Reuse Objectives

Others

Improve Scalability

Identify Problems

who will succeed
Who will succeed?
  • The companies that will succeed will be those that can develop high-quality software faster, more reliably, and more cost-effectively than their competitors.
typical software activities
Typical Software Activities

Product Requirements

R

Architectural Analysis

Detailed Designs

Requirements Analysis

Coding

Release

R

R

R

Integration Testing

Unit Testing

Functional Testing

R

R

Hardware Design

Product Validation

level 2 repeatable process on individuals
Level 2: Repeatable Process on Individuals

Control

  • Budget

Construct the System

  • Size
  • Volatility
  • Size
  • Time

Input

Output

  • Size
  • Experience

Personnel

  • Software Size
  • Non-Commented Source Lines of code
  • Feature Function Points
  • Object or method count
  • Requirements Volatility
  • Requirements changes
  • Engineers experience
  • Domain/application
  • Development architecture
  • Tools and Methods
  • Overall years of experience
  • Personnel Effort
  • Actual person-month of effort
  • Reported person-months of effort

Employee turnover

level 3 process is defined and institutionalized
Level 3: Process is defined and institutionalized

Design method

Integration Method

Detailed Design method

Unit Testing method

  • Complexity
  • Quality
  • Complexity
  • Quality

Requirements

Unit Tested

Architectural Design

Coding

Unit testing

  • Complexity
  • Quality
  • Complexity
  • Quality

High-Level Design

Detailed Design

Integration

Testing

Detailed Design

Tested System

  • Requirements complexity
  • Number of distinct objects
  • Number of actions addressed
  • Test complexity
  • Path count
  • Number of interfaces to test
  • Design complexity
  • Number of design modules
  • Cyclomatic complexity
  • Quality Metrics
  • Defects discovered
  • Defect density – defects discovered per unit size
  • Requirements defects discovered
  • Design defects discovered
  • Code defects discovered
  • Fault density for each project
  • Code complexity
  • Number of code modules
  • Cyclomatic complexity
level 4 managed process
Level 4: Managed Process

Reporting requirements from senior management

Manage

Process

Product

People

[Metrics Database]

  • Design defects
  • Distribution of defects
  • Productivity of tasks
  • Planned vs. actuals
  • Resource allocation

Redesign Directive

High-Level Design

why is it important to achieve level 4
Why is it important to achieve level 4?
  • Process type
  • Assess reuse opportunities
  • Defect identification and classification
  • Analyze defect density
  • Assess project completion
level 5 optimized process
Level 5: Optimized Process
  • An optimizing process is the highest level of the process maturity levels
  • Measures from the activities defined before are used to tailor the process
why is it important to achieve level 5
Why is it important to achieve level 5?
  • Continuous assessment of the process
  • Refine the appropriate metrics
  • Use metrics to recommend new metrics, tools, and techniques
  • Estimate project size, costs, and schedule
  • Create project databases
  • Assess project costs and schedule
  • Evaluate project quality and productivity
building code integrity management systems
Building Code Integrity Management Systems
  • Requirements Integrity Management System
  • Design and Implementation Integrity management System
requirements integrity mgt system
Requirements Integrity Mgt System

Customer Requirements

Essential Model

Functionality

R

R

Requirements Analysis

Technical Constraints

Computing power

Memory Requirements

Network Bandwidth

Worst Case Execution Time

Technical

Model

Product Validation

R

Functional Testing

Quality Attributes

Performance

Maintenance

Safety

Reliability

Flexibility

R

Quality Attributes

Hardware Requirements

Time direction

Specification Baseline

Essential Model

Technical

Model

Quality Attributes

requirements integrity system
Requirements Integrity System

New

Requirements

Executable

Specifications

Component

Reuse

HW

Test

Procedures

Component Models

System Requirements

SW

V&V Test Plans

ME

V&V Test Scripts

COTS

ETC…

Change

Management

System

Filtering

and

Reporting

Configuration

Management

System

problem areas
Problem Areas
  • Maturity: Mostly operating at CMMi L1 – L2
  • Metrics: CMMi L1 – L2
  • Testing: TMM L1 – L4
  • Executable Specifications: Virtually non existent except for the HMI-Graphics,
  • Environment: Requirements and Validation teams need to work throughout the entire development life cycle
implementation integrity mgt system
Implementation Integrity Mgt System

Improvement Plan

Standards

Certification

[Quality Group]

New Code

Quote

Requirements

Gen Code

Requirements

Design

Legacy Code

Design

Coding

Release

Vault

COTS

Coding

Unit Testing

Release

Integration Testing

Validation

Other

Change

Management

System

Configuration

Management

System

Other

Validation

New Improvement Plan (revised)

Filtering

and

Reporting

Status Report

Quality Report

Productivity Report

conclusions
Conclusions
  • Maturity of Automotive Designs: Designs and architectures for automotive systems is a moving target. Change is continuous. Both the end-user needs more features and the developer devises more elaborated solutions.
  • SW Complexity: Complexity of the automotive software increases at an unprecedented pace.
  • “No Silver Bullet” or “No Silver Bullet Re-Fired”
  • Integrity management System: Elements of this system have been implemented and rolled out with various degrees of success…
  • Software reuse
  • Case Tools
  • Higher-Level Languages
  • Model-Driven Methodologies
  • Extreme programming and agile software development methods
  • Auto-Code and Auto-Testing
architecture
Architecture

An architecture is the set of significant decisions about the organization of a software system, the selection of the structural elements and their interfaces by which the system is composed, together with their behavior as specified in the collaborations among those elements, into progressively larger subsystems, and the architectural style that guides this organization – these elements and their interfaces, their collaborations, and their composition.

Krutchten – The Rational Unified Process

Booch, Rambaugh, and Jacobson –The UML Language User Guide, Addison-Wesley, 1999