Toward an ontology for measuring systems engineering return on investment l.jpg
This presentation is the property of its rightful owner.
Sponsored Links
1 / 29

Toward an Ontology for Measuring Systems Engineering Return on Investment PowerPoint PPT Presentation


  • 100 Views
  • Uploaded on
  • Presentation posted in: General

Toward an Ontology for Measuring Systems Engineering Return on Investment. Advancing the knowledge of systems engineering. Eric Honour +1 (850) 479-1985 [email protected] Dr. Ricardo Valerdi +1 (617) 253-8583 [email protected] Version. Topics. SE-ROI Project “Ontology” concept

Download Presentation

Toward an Ontology for Measuring Systems Engineering Return on Investment

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Toward an ontology for measuring systems engineering return on investment l.jpg

Toward an Ontology for Measuring Systems Engineering Return on Investment

Advancing the knowledge of systems engineering

Eric Honour

+1 (850) 479-1985

[email protected]

Dr. Ricardo Valerdi

+1 (617) 253-8583

[email protected]

Toward Ontology for SE-ROI

Version


Topics l.jpg

Topics

  • SE-ROI Project

  • “Ontology” concept

  • COSYSMO work toward ontology

  • Categorization from current standards

  • Future directions

Toward Ontology for SE-ROI


Systems engineering return on investment l.jpg

Systems Engineering Return on Investment

Summary of the SE-ROI Project

Toward Ontology for SE-ROI


Heuristic claim of se l.jpg

Traditional Design

Risk

SYSTEM

DESIGN

DETAIL

DESIGN

PRODUCTION

INTEGRATION

Time

TEST

Risk

Saved

Time/

Cost

“System Thinking” Design

Time

Heuristic Claim of SE

  • Better systems engineering leads to

    • Better system quality/value

    • Lower cost

    • Shorter schedule

Toward Ontology for SE-ROI


Value of se 2004 results l.jpg

Greater SE Effort led to better cost/schedule compliance and better predictability

Value of SE – 2004 Results

Problems/challenges:

  • Quantitative data on SE not available in program databases

    • All data points were subjective

  • Detailed structure not available

Value = 1.0 if program met cost/schedule goals

Each dot is one program, with sizes between $1M and $6.5B

Source: SECOE 01-03INCOSE 2003

Toward Ontology for SE-ROI


Se roi project l.jpg

SE-ROI Project

  • Interviews

  • Just-completed programs

  • Key PM/SE/Admin

  • Translate program data into project structure

  • Desired Results

  • Statistical correlation of SE methods with program success.

  • Leading SE indicators that can be used during a program.

  • Identification of good SE practices under different conditions.

  • Program characterization

  • Program success data

  • SE data (hours, quality, methods)

Statistical correlation

Toward Ontology for SE-ROI


Ontology concept l.jpg

“Ontology” Concept

What is this word and how does it relate to systems engineering?

Toward Ontology for SE-ROI


Current state of se definition l.jpg

Current State of SE Definition

  • Fragmented by domain opinions

  • Military – DOD/MOD

  • Space - NASA/ESA

  • Commercial products

  • Aircraft

  • Automobiles

  • Nuclear waste

  • Process engineering

  • Tool vendors

  • Etc. Etc. Etc.

  • Fragmented by discipline opinions

  • Technical leaders

  • System architects

  • System analysts

  • Requirements engineers

  • Operations analysts

  • Design engineers

  • Fragmented by standards

  • ANSI/EIA-632

  • IEEE-1220

  • ISO-15288

  • CMMI

  • MIL-STD-499C

Toward Ontology for SE-ROI


Ontology l.jpg

Ontology

“…a branch of metaphysics concerned with the nature and relations of being”

aesthetics

interfaces

functions

structure

inputs

components

methods

outputs

categories

understanding

  • POSIWID – The Purpose of a Systems Is What It Does

  • Jack Ring

  • The purpose of systems engineering is different in the eyes of different people, because they perceive different actions/results from SE

Toward Ontology for SE-ROI


Purpose of this paper l.jpg

Purpose of this Paper

  • Explore the variety of what people see in SE

  • Formulate some general categories

  • Interpret historical SE effort data

  • Provide a structure for the data-gathering in the SE-ROI project.

Toward Ontology for SE-ROI


Cosysmo work toward ontology l.jpg

COSYSMO work toward ontology

An exploration of ontology as performed in the COSYSMO project

Toward Ontology for SE-ROI


Cosysmo effort profile l.jpg

COSYSMO Effort Profile

  • How is Systems Engineering effort distributed over time?

Toward Ontology for SE-ROI


Slide13 l.jpg

Effort Distribution Across

ANSI/EIA 632 Fundamental Processes

Toward Ontology for SE-ROI


Slide14 l.jpg

Systems Engineering Effort Profile

Toward Ontology for SE-ROI


Categorization from current standards l.jpg

Categorization from Current Standards

The start of an ontology, by identifying the widely-accepted categories.

Toward Ontology for SE-ROI


Categories in the standards l.jpg

Categories in the Standards

  • Mission/Purpose Definition

  • Requirements Management

  • System Architecting

  • System Implementation

  • Technical Analysis

  • Technical Management/Leadership

  • Verification & Validation

CMMI

ANSI/EIA-632

MIL-STD-499C

IEEE-1220

ISO-15288

Colored boxes on following slides show the terminology used by each standard

Toward Ontology for SE-ROI


Mission purpose definition l.jpg

Mission/Purpose Definition

  • Define the mission or purpose of the new/changed system.

    • Typically described in the language of the system users rather than in technical language

  • CMMI

  • Develop customer requirements

  • ANSI/EIA-632

  • Not included

  • MIL-STD-499C

  • Not included

  • IEEE-1220

  • Define customer expectations

  • ISO-15288

  • Stakeholder needs definition

Toward Ontology for SE-ROI


Requirements management l.jpg

Requirements Management

  • Creation and management of requirements

    • Efforts to define, analyze, validate, and manage the requirements

  • CMMI

  • Requirements development

  • Requirements mgmt

  • ANSI/EIA-632

  • System design

  • Requirements definition

  • MIL-STD-499C

  • System requirements analysis and validation

  • IEEE-1220

  • Requirements analysis

  • ISO-15288

  • Requirements analysis

Toward Ontology for SE-ROI


System architecting l.jpg

System Architecting

  • Define the system in terms of its component elements and their relationships

    • Diagrams that depict the system, its environment, components, and relationships

  • CMMI

  • Technical solution

  • ANSI/EIA-632

  • System design

  • Solution definition

  • MIL-STD-499C

  • System product technical requirements analysis and validation

  • Design or physical solution representation

  • IEEE-1220

  • Synthesis

  • ISO-15288

  • Architectural design

  • System life cycle mgmt

Toward Ontology for SE-ROI


System implementation l.jpg

System Implementation

  • Development/completion of the system

    • Specific system-level efforts in the standards are system integration and transition to use

  • ANSI/EIA-632

  • Product realization

  • Implementation

  • Transition to use

  • CMMI

  • Product integration

  • MIL-STD-499C

  • Not included

  • IEEE-1220

  • Not included

  • ISO-15288

  • Implementation

  • Integration

  • Transition

Toward Ontology for SE-ROI


Technical analysis l.jpg

Technical Analysis

  • System-level technical analysis

    • Assessment of system performance

    • Trade-off analysis of alternatives

  • CMMI

  • Measurement and analysis

  • ANSI/EIA-632

  • Technical evaluation

  • System analysis

  • MIL-STD-499C

  • Functional analysis, allocations and validation

  • Assessments of system effectiveness, cost, schedule, and risk

  • Tradeoff analyses

  • IEEE-1220

  • Functional analysis

  • Requirements trade studies and assessments

  • Functional trade studies and assessments

  • Design trade studies and assessments

  • ISO-15288

  • Requirements analysis

Toward Ontology for SE-ROI


Technical management leadership l.jpg

Technical Management/Leadership

  • Guiding the engineering teams involved in system design programs

    • Size/complexity of teams demands leadership

  • CMMI

  • Project planning

  • Project monitoring & control

  • Supplier agreement mgmt

  • Process/product quality assur.

  • Configuration mgmt

  • Integrated project mgmt

  • Decision analysis/resolution

  • Quantitative project mgmt

  • Risk mgmt

  • ANSI/EIA-632

  • Technical Mgmt

  • Planning

  • Assessment

  • Control

  • MIL-STD-499C

  • Planning

  • Monitoring

  • Decision making, control, and baseline maintenance

  • Risk mgmt

  • Baseline change control

  • Interface mgmt

  • Data mgmt

  • Subcontract mgmt

  • Technical reviews/audits

  • IEEE-1220

  • Technical mgmt

  • Track analysis data

  • Track requirements and design changes

  • Track performance

  • Track product metrics

  • Update specifications

  • Update architectures

  • Update plans

  • Maintain database

  • ISO-15288

  • Planning, Assessment, Control

  • Decision mgmt

  • Config mgmt

  • Acquisition, Supply

  • Resource mgmt

  • Risk mgmt

Toward Ontology for SE-ROI


Verification validation l.jpg

Verification & Validation

  • Verification: comparison of the system with its requirements through objective evidence.

  • Validation: comparison of the system or requirements with the intended mission

  • ANSI/EIA-632

  • Technical Evaluation

  • Requirements validation

  • System verification

  • End products validation

  • CMMI

  • Verification

  • Validation

  • MIL-STD-499C

  • Design or physical solution verification and validation

  • IEEE-1220

  • Requirement verification

  • Functional verification

  • Design verification

  • ISO-15288

  • Verification

  • Validation

  • Quality mgmt

Toward Ontology for SE-ROI


Future directions l.jpg

Future Directions

Where is SE-ROI going?

Toward Ontology for SE-ROI


Project advisory group l.jpg

Project Advisory Group

  • Group of interested people/organizations

    • Communicate via web, telecon, meetings

    • Help define the data organization

    • Build public interest in the project

    • Provide access to real programs

    • View interim (protected) data as it develops

Current members come from:

AF Institute of Technology

Northrop Grumman

DOD Office of Secy of Defense

DRS

Johns Hopkins Univ

MIT

The Mitre Corp

NAVAIR

Raytheon

Rand Corporation

Rafael

Systems & Software Consortium

Univ of South Australia

USN Chief Engineer

For information, see http://www.hcode.com/seroi/

Toward Ontology for SE-ROI


Summary l.jpg

Summary

  • Systems engineering current state of knowledge is fragmented

  • Broadly-accepted ontology is needed

  • SE-ROI project needs categorization now

    • Structure the data to be correlated

    • Discover leading SE indicators

    • Identify SE best practices and methods

  • Categorization across standards helps develop the needed ontology

Toward Ontology for SE-ROI


Questions l.jpg

Questions?

Eric Honour

+1 (850) 479-1985

[email protected]

Dr. Ricardo Valerdi

+1 (617) 253-8583

[email protected]

For information, see http://www.hcode.com/seroi/

Toward Ontology for SE-ROI


Survey of se roi knowledge l.jpg

Survey of SE-ROI Knowledge

Toward Ontology for SE-ROI


Next steps l.jpg

Next Steps

  • Define interview data sheets

    • Use this categorization

  • Identify and interview trial projects

    • Obtain initial data

    • Evaluate the interview data sheets

  • Identify and interview projects

    • Ongoing effort for 2-3 years

  • Perform statistical correlation work

    • Ongoing effort for duration of project

    • Interim reports to participating organizations

  • Final report expected 2009

Toward Ontology for SE-ROI


  • Login