toward an ontology for measuring systems engineering return on investment l.
Skip this Video
Loading SlideShow in 5 Seconds..
Toward an Ontology for Measuring Systems Engineering Return on Investment PowerPoint Presentation
Download Presentation
Toward an Ontology for Measuring Systems Engineering Return on Investment

Loading in 2 Seconds...

play fullscreen
1 / 29

Toward an Ontology for Measuring Systems Engineering Return on Investment - PowerPoint PPT Presentation

  • Uploaded on

Toward an Ontology for Measuring Systems Engineering Return on Investment. Advancing the knowledge of systems engineering. Eric Honour +1 (850) 479-1985 Dr. Ricardo Valerdi +1 (617) 253-8583 Version. Topics. SE-ROI Project “Ontology” concept

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Toward an Ontology for Measuring Systems Engineering Return on Investment' - platt

Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
toward an ontology for measuring systems engineering return on investment

Toward an Ontology for Measuring Systems Engineering Return on Investment

Advancing the knowledge of systems engineering

Eric Honour

+1 (850) 479-1985

Dr. Ricardo Valerdi

+1 (617) 253-8583

Toward Ontology for SE-ROI


  • SE-ROI Project
  • “Ontology” concept
  • COSYSMO work toward ontology
  • Categorization from current standards
  • Future directions

Toward Ontology for SE-ROI

systems engineering return on investment

Systems Engineering Return on Investment

Summary of the SE-ROI Project

Toward Ontology for SE-ROI

heuristic claim of se

Traditional Design














“System Thinking” Design


Heuristic Claim of SE
  • Better systems engineering leads to
    • Better system quality/value
    • Lower cost
    • Shorter schedule

Toward Ontology for SE-ROI

value of se 2004 results

Greater SE Effort led to better cost/schedule compliance and better predictability

Value of SE – 2004 Results


  • Quantitative data on SE not available in program databases
    • All data points were subjective
  • Detailed structure not available

Value = 1.0 if program met cost/schedule goals

Each dot is one program, with sizes between $1M and $6.5B

Source: SECOE 01-03INCOSE 2003

Toward Ontology for SE-ROI

se roi project
SE-ROI Project
  • Interviews
  • Just-completed programs
  • Key PM/SE/Admin
  • Translate program data into project structure
  • Desired Results
  • Statistical correlation of SE methods with program success.
  • Leading SE indicators that can be used during a program.
  • Identification of good SE practices under different conditions.
  • Program characterization
  • Program success data
  • SE data (hours, quality, methods)

Statistical correlation

Toward Ontology for SE-ROI

ontology concept

“Ontology” Concept

What is this word and how does it relate to systems engineering?

Toward Ontology for SE-ROI

current state of se definition
Current State of SE Definition
  • Fragmented by domain opinions
  • Military – DOD/MOD
  • Space - NASA/ESA
  • Commercial products
  • Aircraft
  • Automobiles
  • Nuclear waste
  • Process engineering
  • Tool vendors
  • Etc. Etc. Etc.
  • Fragmented by discipline opinions
  • Technical leaders
  • System architects
  • System analysts
  • Requirements engineers
  • Operations analysts
  • Design engineers
  • Fragmented by standards
  • ANSI/EIA-632
  • IEEE-1220
  • ISO-15288
  • CMMI
  • MIL-STD-499C

Toward Ontology for SE-ROI


“…a branch of metaphysics concerned with the nature and relations of being”











  • POSIWID – The Purpose of a Systems Is What It Does
  • Jack Ring
  • The purpose of systems engineering is different in the eyes of different people, because they perceive different actions/results from SE

Toward Ontology for SE-ROI

purpose of this paper
Purpose of this Paper
  • Explore the variety of what people see in SE
  • Formulate some general categories
  • Interpret historical SE effort data
  • Provide a structure for the data-gathering in the SE-ROI project.

Toward Ontology for SE-ROI

cosysmo work toward ontology

COSYSMO work toward ontology

An exploration of ontology as performed in the COSYSMO project

Toward Ontology for SE-ROI

cosysmo effort profile
COSYSMO Effort Profile
  • How is Systems Engineering effort distributed over time?

Toward Ontology for SE-ROI


Effort Distribution Across

ANSI/EIA 632 Fundamental Processes

Toward Ontology for SE-ROI


Systems Engineering Effort Profile

Toward Ontology for SE-ROI

categorization from current standards

Categorization from Current Standards

The start of an ontology, by identifying the widely-accepted categories.

Toward Ontology for SE-ROI

categories in the standards
Categories in the Standards
  • Mission/Purpose Definition
  • Requirements Management
  • System Architecting
  • System Implementation
  • Technical Analysis
  • Technical Management/Leadership
  • Verification & Validation






Colored boxes on following slides show the terminology used by each standard

Toward Ontology for SE-ROI

mission purpose definition
Mission/Purpose Definition
  • Define the mission or purpose of the new/changed system.
    • Typically described in the language of the system users rather than in technical language
  • CMMI
  • Develop customer requirements
  • ANSI/EIA-632
  • Not included
  • MIL-STD-499C
  • Not included
  • IEEE-1220
  • Define customer expectations
  • ISO-15288
  • Stakeholder needs definition

Toward Ontology for SE-ROI

requirements management
Requirements Management
  • Creation and management of requirements
    • Efforts to define, analyze, validate, and manage the requirements
  • CMMI
  • Requirements development
  • Requirements mgmt
  • ANSI/EIA-632
  • System design
  • Requirements definition
  • MIL-STD-499C
  • System requirements analysis and validation
  • IEEE-1220
  • Requirements analysis
  • ISO-15288
  • Requirements analysis

Toward Ontology for SE-ROI

system architecting
System Architecting
  • Define the system in terms of its component elements and their relationships
    • Diagrams that depict the system, its environment, components, and relationships
  • CMMI
  • Technical solution
  • ANSI/EIA-632
  • System design
  • Solution definition
  • MIL-STD-499C
  • System product technical requirements analysis and validation
  • Design or physical solution representation
  • IEEE-1220
  • Synthesis
  • ISO-15288
  • Architectural design
  • System life cycle mgmt

Toward Ontology for SE-ROI

system implementation
System Implementation
  • Development/completion of the system
    • Specific system-level efforts in the standards are system integration and transition to use
  • ANSI/EIA-632
  • Product realization
  • Implementation
  • Transition to use
  • CMMI
  • Product integration
  • MIL-STD-499C
  • Not included
  • IEEE-1220
  • Not included
  • ISO-15288
  • Implementation
  • Integration
  • Transition

Toward Ontology for SE-ROI

technical analysis
Technical Analysis
  • System-level technical analysis
    • Assessment of system performance
    • Trade-off analysis of alternatives
  • CMMI
  • Measurement and analysis
  • ANSI/EIA-632
  • Technical evaluation
  • System analysis
  • MIL-STD-499C
  • Functional analysis, allocations and validation
  • Assessments of system effectiveness, cost, schedule, and risk
  • Tradeoff analyses
  • IEEE-1220
  • Functional analysis
  • Requirements trade studies and assessments
  • Functional trade studies and assessments
  • Design trade studies and assessments
  • ISO-15288
  • Requirements analysis

Toward Ontology for SE-ROI

technical management leadership
Technical Management/Leadership
  • Guiding the engineering teams involved in system design programs
    • Size/complexity of teams demands leadership
  • CMMI
  • Project planning
  • Project monitoring & control
  • Supplier agreement mgmt
  • Process/product quality assur.
  • Configuration mgmt
  • Integrated project mgmt
  • Decision analysis/resolution
  • Quantitative project mgmt
  • Risk mgmt
  • ANSI/EIA-632
  • Technical Mgmt
  • Planning
  • Assessment
  • Control
  • MIL-STD-499C
  • Planning
  • Monitoring
  • Decision making, control, and baseline maintenance
  • Risk mgmt
  • Baseline change control
  • Interface mgmt
  • Data mgmt
  • Subcontract mgmt
  • Technical reviews/audits
  • IEEE-1220
  • Technical mgmt
  • Track analysis data
  • Track requirements and design changes
  • Track performance
  • Track product metrics
  • Update specifications
  • Update architectures
  • Update plans
  • Maintain database
  • ISO-15288
  • Planning, Assessment, Control
  • Decision mgmt
  • Config mgmt
  • Acquisition, Supply
  • Resource mgmt
  • Risk mgmt

Toward Ontology for SE-ROI

verification validation
Verification & Validation
  • Verification: comparison of the system with its requirements through objective evidence.
  • Validation: comparison of the system or requirements with the intended mission
  • ANSI/EIA-632
  • Technical Evaluation
  • Requirements validation
  • System verification
  • End products validation
  • CMMI
  • Verification
  • Validation
  • MIL-STD-499C
  • Design or physical solution verification and validation
  • IEEE-1220
  • Requirement verification
  • Functional verification
  • Design verification
  • ISO-15288
  • Verification
  • Validation
  • Quality mgmt

Toward Ontology for SE-ROI

future directions

Future Directions

Where is SE-ROI going?

Toward Ontology for SE-ROI

project advisory group
Project Advisory Group
  • Group of interested people/organizations
    • Communicate via web, telecon, meetings
    • Help define the data organization
    • Build public interest in the project
    • Provide access to real programs
    • View interim (protected) data as it develops

Current members come from:

AF Institute of Technology

Northrop Grumman

DOD Office of Secy of Defense


Johns Hopkins Univ


The Mitre Corp



Rand Corporation


Systems & Software Consortium

Univ of South Australia

USN Chief Engineer

For information, see

Toward Ontology for SE-ROI

  • Systems engineering current state of knowledge is fragmented
  • Broadly-accepted ontology is needed
  • SE-ROI project needs categorization now
    • Structure the data to be correlated
    • Discover leading SE indicators
    • Identify SE best practices and methods
  • Categorization across standards helps develop the needed ontology

Toward Ontology for SE-ROI



Eric Honour

+1 (850) 479-1985

Dr. Ricardo Valerdi

+1 (617) 253-8583

For information, see

Toward Ontology for SE-ROI

survey of se roi knowledge
Survey of SE-ROI Knowledge

Toward Ontology for SE-ROI

next steps
Next Steps
  • Define interview data sheets
    • Use this categorization
  • Identify and interview trial projects
    • Obtain initial data
    • Evaluate the interview data sheets
  • Identify and interview projects
    • Ongoing effort for 2-3 years
  • Perform statistical correlation work
    • Ongoing effort for duration of project
    • Interim reports to participating organizations
  • Final report expected 2009

Toward Ontology for SE-ROI