slide1
Download
Skip this Video
Download Presentation
Improving Confidence in the Assessment of System Performance in Differing Scenarios.

Loading in 2 Seconds...

play fullscreen
1 / 44

Improving Confidence in the Assessment of System Performance in Differing Scenarios. - PowerPoint PPT Presentation


  • 48 Views
  • Uploaded on

Improving Confidence in the Assessment of System Performance in Differing Scenarios. T D Clayton. Cardinal Consultants. 1. Context 2. Scenario Dependency of Input Data 3. Choosing Scenarios to Assess 4. Modelling Widely Differing Scenarios 5. Example Study

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Improving Confidence in the Assessment of System Performance in Differing Scenarios.' - gerda


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Improving Confidence in the Assessment of

System Performance in Differing Scenarios.

T D Clayton

Cardinal Consultants

slide2

1. Context

2. Scenario Dependency of Input Data

3. Choosing Scenarios to Assess

4. Modelling Widely Differing Scenarios

5. Example Study

6. Summary and Conclusions

slide3

SYSTEM

EFFECTIVENESS

ASSESSMENT

slide4

SYSTEM

EFFECTIVENESS

ASSESSMENT

Warhead

Lethality

slide5

SYSTEM

EFFECTIVENESS

ASSESSMENT

Warhead

Lethality

Combat

modelling

slide6

Sensor

Performance

Guidance

System

Wargaming

SYSTEM

EFFECTIVENESS

ASSESSMENT

Warhead / Fuze

Performance

Combat

modelling

Operator

Performance

Tactical / Strategic

studies

Other

subsystems

slide7

Purpose of System Effectiveness Studies

  • Research / long term development objectives
  • Medium term procurement objectives
  • Design optimisation
  • Procurement decisions
  • Input to Operational / Tactical Studies
slide8

But, whatever the purpose, scenario assumptions are critical.

or, we should assume they are, unless proven otherwise.

slide9

Rule 1

Everything is scenario dependent.

slide10

Sensor

Performance

Guidance

System

Wargaming

SYSTEM

EFFECTIVENESS

ASSESSMENT

Warhead / Fuze

Performance

Combat

modelling

Operator

Performance

Tactical / Strategic

studies

Other

subsystems

slide11

Sensor

Performance

Guidance

System

Wargaming

SYSTEM

EFFECTIVENESS

ASSESSMENT

Warhead / Fuze

Performance

Combat

modelling

Operator

Performance

Tactical / Strategic

studies

Other

subsystems

slide13

Nature of ground around the target

  • Presence of adjacent trees, or protective earthworks
  • Azimuth distribution
  • Elevation distribution
  • Relative value of M-kill, F-kill, P-kill, K-kill
  • Likelihood of multiple hits
  • Using an MFK value as a probability ?
slide14

The Multi-Disciplinary Problem

Lethality

Expert

Combat

Modeller

Systems

Modeller

slide15

TheManagementSolution

Establish roles and responsibilities for managing the interfaces between expert groups.

slide16

Responsibilities of the Interface Manager

  • Understand methodologies and assumptions at all levels
  • Organise training / briefings to assist expert groups widen knowledge
  • Conduct studies to measure Scenario Dependencies of results
  • Maintain knowledge base of dependencies and “corrections”
  • Involvement in planning of studies, addressing assumptions
  • Involvement in reporting of studies, esp. assumptions
slide17

Study 1

Study 2

Study 3

MAIN DATABASE OF STUDY RESULTS

‘Offline’ analysis tools

Comparison & Analysis

Study planning and analysis

Data provided to

other studies

DATABASE OF

SCENARIO COMPENSATION FACTORS

slide18

Study 1

Study 2

Study 3

MAIN DATABASE OF STUDY RESULTS

Calculate SCF’s from new studies

DATABASE OF

SCENARIO COMPENSATION FACTORS

Assessment and comparison of SCF’s

Modified SCF’s

slide19

Rule 2

You will never assess the right scenarios.

slide20

Opposing - Technology

Ground - Numbers

Equipment - Own Intell.

Posture & - Posture (Defensive, attacking)

Deployment - Deployment and detectablity

Air - Aircraft types

Capability - Level of technology

- Numbers

- Own Intell.

Anti-Air - Numbers of units

Capability - Capability

- Own Intell.

Maritime - Maritime involvement

- Capability

BLUE ROLE - Peace keeping, combat (defensive)

combat (hunt and kill)

Scenario Parameters

Climate - Temperature

- Precipitation

Ground - Vegetation

- Topology

- Roads

Geography - Geographic isolation

& Politics - Neighbouring countries

- Local cilvilian population

Opposing - Nuc., Chem., Bio.

Max. Cap. - Short range Long range

Opposing - Numbers

Troops - Capability

slide23

Rule 3

A combat model cannot address

widely differing scenarios.

slide24

Example Study

Comparative assessment of two potential candidates

for a cannon system for light armoured vehicles.

slide26

ORIGINAL STUDY PLAN

Input data

Engagement Model (developed for this study)

Combat model (existing)

3 Scenarios

slide27

REVIEW OF PROVIDED DATA

1. When multiple hits are likely, SSKP may not be appropriate.

2. Lethality figures give no azimuth dependency.

3. No information on range dependency.

4. Data required for wider range of target types.

Lethality models re-run, in concert with Engagement model.

slide28

REVIEW OF EXISTING COMBAT MODEL

1. Tends to choose tanks as preferred target type.

2. All targets are land vehicles.

3. Terrain in all 3 scenarios tends to give long engagement ranges.

4. No variations in met-vis or day/night > long ranges

5. Same Blue positions for both System A and System B.

6. Units are static when firing.

slide30

THE ALTERNATIVE APPROACH

1. Use a range of methods, including Military Judgement, to derive intermediate data and distributions reflecting a wide range of scenarios.

  • relative frequencies of target types engaged
  • engagement range distributions
  • azimuth distributions
  • probability of kill per burst - function of range and target type
slide31

THE ALTERNATIVE APPROACH

1. Use a range of methods, including Military Judgement, to derive intermediate data and distributions reflecting a wide range of scenarios.

2. Develop a simple tool to calculate specific Measures of Effectiveness from the input data and distributions.

MoE 1: Military Worth of kills per burst

MoE 2: Military Worth of kills per ammunition load

slide32

THE ALTERNATIVE APPROACH

1. Use a range of methods, including Military Judgement, to derive intermediate data and distributions reflecting a wide range of scenarios.

2. Develop a simple tool to calculate specific Measures of Effectiveness from the input data and distributions.

  • quick to develop
  • quick to run
  • facilitates review and scrutiny of data
  • stores data and maintains audit trails
slide33

THE ALTERNATIVE APPROACH

1. Use a range of methods, including Military Judgement, to derive intermediate data and distributions reflecting a wide range of scenarios.

2. Develop a simple tool to calculate specific Measures of Effectiveness from the input data and distributions.

  • permit results to be adjusted by Military Judgement
    • to account for factors not addressed by calculations

- the value of the ability to fire on the move

- the value of the greater manoeuvrability afforded

by the lighter system

slide40

SUMMARY AND CONCLUSIONS

Appropriate methods of addressing scenario dependencies are

essential to ensure study conclusions are valid.

1. ALL DATA should be regarded as being scenario-dependent.

It is very useful to have an analyst in every team with

special responsibility for addressing this problem.

2. Using combat models to compare performance of systems

can be hazardous.

Consider using a range of methods to generate

intermediate results which are open to scrutiny

and to sensitivity studies.

slide42

Framework

Title

Model results

Feedback

Alternative approach

Contents

Rule 2

Data screen 1

Study levels

Results screen

Scen Pars

Study purpose

Conclusions

Histogram

Rule 1

Graph

Highlight top-level

Rule 3

Highlight all

Further Dev’t

TarDes pic

Example study

Data

Leth’y depends

Current issues

Original plan

MutliDisciplinary

Data review

Management Soln

Responsibilities

Model review

slide43

Further Development of the CST Tool

1. Development of proper library of routines

2. Improved statistical routines for increase in speed

3. Automated methods for parametric studies

4. Use of EDMS technologies to manage and access study reports

slide44

CURRENT ISSUES / PROBLEMS WITH CST-01

1. It is not clear how best to address the problem of

firing multiple bursts at a target, depending upon

whether it is perceived to be killed.

2. It is not clear whether (and how) costs (or numbers of units)

should be included, or handled separately.

ad