proposed metoc metrics system for socal nsw training events involving small boats l.
Download
Skip this Video
Download Presentation
Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats

Loading in 2 Seconds...

play fullscreen
1 / 47

Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats - PowerPoint PPT Presentation


  • 159 Views
  • Uploaded on

Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats. Presented by: Bruce W. Ford bruce@clearscienceinc.com fordbw@tsc-jax.navy.smil.mil http://www.clearscienceinc.com 904-379-9704 (office) 904-536-7180 (cell). This Brief is UNCLASSIFIED. For official use only.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats' - graceland


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
proposed metoc metrics system for socal nsw training events involving small boats
Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats

Presented by:

Bruce W. Fordbruce@clearscienceinc.comfordbw@tsc-jax.navy.smil.milhttp://www.clearscienceinc.com904-379-9704 (office)904-536-7180 (cell)

This Brief is UNCLASSIFIED

For official use only

http://www.ClearScienceInc.com

slide2

Contributors

  • LCDR Bill Swick – NSW Group 3
  • LTJG Megan Thomas – MSC
  • Luke Piepkorn - SPA

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

For official use only

slide3

The Questions

1. Can a viable METOC metrics system be set up for NSW operations?

2. Can a viable METOC metrics system be installed to collect and compute metrics for SOCAL Small Boat Team (SBT) training events?

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

For official use only

slide4

The Short Answer

Yes!

  • A viable METOC metrics collection and calculation system for NSW TEs in SOCAL involving boats is an attainable goal. To institute such a system, some changes may be needed in the way forecasts are assembled, verified, and changes must be instituted to collect customer debrief information. The extent of those changes will be decided upon by the NSW METOC community leadership and this document strives to lay out options for metrics data collection.

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

For official use only

slide5

The Study Goals

  • Define study and customers - SBTs
  • Dissect customer process
  • First-guess metrics
    • METOC Performance Metrics
    • Customer Performance Metrics
    • Operational Impact Metrics
    • Proxy Operational Impacts Metrics
  • Propose collection process
  • Describe data analysis
  • Describe operational modeling
  • Map to 3-D metrics space

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

For official use only

slide6

Process for Developing Metrics of

METOC Impacts on NSW Operations

Mission/TE Outcomes

Mission/TE Plans

EM Forecasts *

Scientific/Anecdotal Observations

Operational Performance Metrics

METOC Performance Metrics

Metrics of METOC Impacts on Operational Performance

We apply this process to both real world data and

output from a military mission model.

* or other products

From Murphree et al., METOC Metrics, Jan 07, murphree@nps.edu

slide7

Study Definition

  • Training Event (TE) types
    • *Small Boat Ops
    • *Combat Swimmer Ops
    • *Visit, Board, Search & Seizure (VBSS)
    • Land Warfare
    • Jump Ops
  • Geographical Region – SOCAL area

The mission types designated with the red asterisk (*) involve boats and thus are the subject of this study.

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

For official use only

slide8

The Study Goals

  • Define study and customers - SBTs
  • Dissect customer process
  • First-guess metrics
    • METOC Performance Metrics
    • Customer Performance Metrics
    • Operational Impact Metrics
    • Proxy Operational Impacts Metrics
  • Propose collection process
  • Describe data analysis
  • Describe operational modeling
  • Map to 3-D metrics space

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

For official use only

slide9

Mission Execution Brief

Mission Execution Brief

Mission Launch

Mission Launch

ULT Schedule Available

ULT Schedule Available

Logistics

Logistics

EM starts forecast

EM starts forecast

M-6

Months

M-6

Months

M-1

Month

M-1

Month

M-72

Hours

M-72

Hours

M-24

Hours

M-24

Hours

SOCAL TE/Mission Process

  • Planning
  • Execution
    • 3-48 hours in duration
    • Similar TEs may be held for up to 7-10 consecutive days
    • EM may be in contact with team
    • Team will usually go despite red conditions – Wait and see
  • Debrief
    • Often an informal “huddle” within 24 hours after TE – no set time or location
    • EM do not attend
    • Feedback to EM typically informal by exception

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

For official use only

slide10

The Study Goals

  • Define study and customers - SBTs
  • Dissect customer process
  • First-guess metrics
    • METOC Performance Metrics
    • Customer Performance Metrics
    • Operational Impact Metrics
    • Proxy Operational Impacts Metrics
  • Propose collection process
  • Describe data analysis
  • Describe operational modeling
  • Map to 3-D metrics space

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

For official use only

slide11

Proposed METOC Performance Metrics What we forecast

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

For official use only

slide12

Proposed METOC Performance Metrics What we can verify

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

For official use only

slide13

Proposed METOC Performance Metrics

Location of Mission-by-mission METOC Performance Metrics within 3-D Metrics Space. Purple circle denotes location of metrics.

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

For official use only

slide14

Proposed METOC Performance Metrics

Location of Multiple-mission METOC Performance Metrics within 3-D Metrics Space. Purple circle denotes location of metrics.

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

For official use only

slide15

Proposed Customer Performance Metrics

  • What SBTs use for measure of TE/Mission success is under investigation
  • Proposed single-TE/mission metrics:
    • Mission success – Were the pre-mission objectives met?
    • Small boat effectiveness – We small boats used to reach their planned destination? (Boat ops, VBSS)
    • Target vessel – Was the target boat reached and boarded? (VBSS)
    • Swimmer success – Were combat swimmer able to reach their destination? (Combat swimmer)
    • Others?

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

For official use only

slide16

Proposed Customer Performance Metrics

  • Proposed multi-TE/mission metrics:
    • Mission success percentage – percentage of missions that achieved their declared objective
    • Small boat effectiveness percentage – percentage of missions involving small boats that reached their destination? (Boat ops, VBSS)
    • Target vessel percentage– percentage of VBSS missions in which the target vessel was successfully boarded
    • Swimmer success percentage– percentage of missions involving swimmer in which the swimmer reached their destination

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

For official use only

slide17

Proposed Operational Performance Metrics

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

For official use only

slide18

Proposed Operational Performance Metrics

Location of Operational Impacts Metrics within 3-D Metrics Space. Purple circle denotes location of metrics.

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

For official use only

slide19

Proposed Proxy Operational Performance Metrics

  • Significant wave height – Strongly impacts nearly all types of boating operations
  • Breaker height – Strongly impacts all boating operations where near-shore disembarkation is involved.
  • Sea Surface Temperature – Strongly impacts missions involving personnel in the water.

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

For official use only

slide20

The Study Goals

  • Define study and customers - SBTs
  • Dissect customer process
  • First-guess metrics
    • METOC Performance Metrics
    • Customer Performance Metrics
    • Operational Impact Metrics
    • Proxy Operational Impacts Metrics
  • Propose collection process
  • Describe data analysis
  • Describe operational modeling
  • Map to 3-D metrics space

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

For official use only

slide21

Process for Developing Metrics of

METOC Impacts on NSW Operations

Mission/TE Outcomes

Mission/TE Plans

EM Forecasts *

Scientific/Anecdotal Observations

Operational Performance Metrics

METOC Performance Metrics

Metrics of METOC Impacts on Operational Performance

We apply this process to both real world data and

output from a military mission model.

* or other products

From Murphree et al., METOC Metrics, Jan 07, murphree@nps.edu

slide22

Data Collection for SOCAL TEs Involving BoatsWho inputs the data?

EM Forecasts

(pre-mission)

Anecdotal Obs

(debrief)

Scientific Obs

(post-mission)

Mission Plans(pre-mission)

Mission/TE Outcomes

(debrief)

Operational Performance Metrics

METOC Performance Metrics

Legend

Metrics of METOC Impacts on Operational Performance

EM Inputs

Non-EM

Inputs

Automated

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

slide23

Data Collection for SOCAL TEs Involving BoatsData flow

Forecasts

Plans

Scientific Obs

Anecdotal Obs

Outcomes

Unique mission records

on metrics server via SIPR web forms

Multi-level metrics computed

Level 1: Single Mission Metrics

Level 2: Multi Mission Metrics

Level 3: Multi Mission Metrics over time

Legend

EM Inputs

EM Inputs

Non-EM

Inputs

Automated

User Pulled

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

slide24

Forecasts

Plans

Scientific Obs

Anecdotal Obs

Outcomes

Unique mission records

on metrics server via SIPR web forms

Multi-level metrics computed

Level 1: Single Mission Metrics

Level 2: Multi Mission Metrics

Level 3: Multi Mission Metrics over time

Data Collection for SOCAL TEs Involving BoatsData flow

Legend

EM Inputs

EM Inputs

Non-EM

Inputs

System inputs

Automated

User Pulled

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

slide25

Forecasts

Plans

Scientific Obs

Anecdotal Obs

Outcomes

Unique mission records

on metrics server via SIPR web forms

Multi-level metrics computed

Level 1: Single Mission Metrics

Level 2: Multi Mission Metrics

Level 3: Multi Mission Metrics over time

Data Collection for SOCAL TEs Involving BoatsData flow

Legend

EM Inputs

EM Inputs

Non-EM

Inputs

Automated

User Pulled

Pre-mission web form

Post-mission web form

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

slide26

Data Collection for SOCAL TEs Involving BoatsProposed forecast creation tool

  • Currently little standardization of EM forecasts
  • Without automating the collection of forecast data, EM will have to re-enter forecast elements into metrics collection system
  • Forecast creation tool
    • Consists of a web interface where EM enters forecast elements into intuitive web form
    • Outputs PPT file or image to be used in PPT file to serve as the start to TE/Mission forecast
    • Standardizes the look of common boat-related METOC forecast elements
    • Allows for entry of numerical/narrative entry of METOC elements and automated threshold interpretation (e.g., EM inputs seas as 5-7 feet and the interface determines the R-Y-G threshold on the output)
    • Provides helpful links, thresholds and error checking functions to improve overall forecast quality
    • Collecting forecast elements for use in metrics calculations

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

For official use only

slide27

Forecasts

Plans

Scientific Obs

Anecdotal Obs

Outcomes

Unique mission records

on metrics server via SIPR web forms

Multi-level metrics computed

Level 1: Single Mission Metrics

Level 2: Multi Mission Metrics

Level 3: Multi Mission Metrics over time

Data Collection for SOCAL TEs Involving BoatsData flow

Legend

EM Inputs

EM Inputs

Non-EM

Inputs

Automated

User Pulled

Collected by

Forecast creation tool

decreasing imposition on the EM

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

slide28

Forecasts

Plans

Scientific Obs

Anecdotal Obs

Outcomes

Unique mission records

on metrics server via SIPR web forms

Multi-level metrics computed

Level 1: Single Mission Metrics

Level 2: Multi Mission Metrics

Level 3: Multi Mission Metrics over time

Data Collection for SOCAL TEs Involving BoatsData flow

Legend

EM Inputs

EM Inputs

Non-EM

Inputs

Automated

User Pulled

Development/transition serverat FNMOC a likely candidate tohost NSW metrics engine

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

slide29

The Study Goals

  • Define study and customers - SBTs
  • Dissect customer process
  • First-guess metrics
    • METOC Performance Metrics
    • Customer Performance Metrics
    • Operational Impact Metrics
    • Proxy Operational Impacts Metrics
  • Propose collection process
  • Describe data analysis
  • Describe operational modeling
  • Map to 3-D metrics space

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

For official use only

slide30

Forecasts

Plans

Scientific Obs

Anecdotal Obs

Outcomes

Unique mission records

on metrics server via SIPR web forms

Multi-level metrics computed

Level 1: Single Mission Metrics

Level 2: Multi Mission Metrics

Level 3: Multi Mission Metrics over time

Data Collection for SOCAL TEs Involving BoatsData flow

Legend

EM Inputs

EM Inputs

Non-EM

Inputs

Automated

User Pulled

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

slide31

Forecasts

Plans

Scientific Obs

Anecdotal Obs

Outcomes

Unique mission records

on metrics server via SIPR web forms

Multi-level metrics computed

Level 1: Single Mission Metrics

Level 2: Multi Mission Metrics

Level 3: Multi Mission Metrics over time

Data Collection for SOCAL TEs Involving BoatsData flow

Legend

EM Inputs

EM Inputs

Non-EM

Inputs

Automated

User Pulled

Products served by SBT, time span or geographical region or a combination for

a single METOC activity

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

slide32

Forecasts

Plans

Scientific Obs

Anecdotal Obs

Outcomes

Unique mission records

on metrics server via SIPR web forms

Multi-level metrics computed

Level 1: Single Mission Metrics

Level 2: Multi Mission Metrics

Level 3: Multi Mission Metrics over time

Data Collection for SOCAL TEs Involving BoatsData flow

Legend

EM Inputs

EM Inputs

Non-EM

Inputs

Automated

User Pulled

Products served by SBT, time span or geographical region or a combination. Emphasis on long time spans and multipleMETOC activities

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

slide33

Conclusions

  • A METOC metric collection and reporting system can be instituted for SOCAL NSW TEs involving boats.
  • Some process changes will be necessary to facilitate metrics data collection (e.g., attend TE debriefs)
  • Questions remain in some details, but nothing that should pose a serious roadblock to the ultimate goal of collecting quality metrics.

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

For official use only

slide34

Discussion Questions

  • Will a SIPR based collection system give access to those that need it (EMs, MSC, leadership)?
  • Is there interest in creating a forecast building tool, which would require some forecast standardization?
  • How big of a paradigm shift will it be for EMs to attend TE debriefs?
  • How will SBTs accept the EM attending debriefs?
  • How intrusive will it be for EMs to place debrief data into the system by returning to the interface following each debrief and entering information?
  • How intrusive will it be for the MSC or another organization to enter observation information if this process cannot be automated?
  • Would such an interface be of use to EM who are deployed?
  • Are there customer performance metrics not mentioned in this document?
  • When do we expand this to beyond the boats?
  • What options exist for entering data when the EM is not “plugged in”?

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

For official use only

questions
Questions

Bruce W. Fordbruce@clearscienceinc.comfordbw@tsc-jax.navy.smil.milhttp://www.clearscienceinc.com904-379-9704 (office)904-536-7180 (cell)

Ford B.W. et al, Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats, 22 Mar 07, bruce@clearscienceinc.com

proposed metoc metrics system for socal nsw training events involving small boats36
Proposed METOC Metrics System for SOCAL NSW Training Events Involving Small Boats

Presented by:

Bruce W. Fordbruce@clearscienceinc.comfordbw@tsc-jax.navy.smil.milhttp://www.clearscienceinc.com904-379-9704 (office)904-536-7180 (cell)

This Brief is UNCLASSIFIED

For official use only

http://www.ClearScienceInc.com

back up slides
Back-up slides

Ford B.W., NSW METOC Metrics - Next Steps, Feb 07, bruce@clearscienceinc.com

slide38

Online Real World Metrics System – System Schematic

Carrier Strike Group (CSG)

Data obtained

by CSG personnel via interaction with air wing

NPS

Data written to database

Data analyzed per NPS metrics procedures

Output reports delivered via web site.

Reports available to CSG on demand via automated metrics calculation process.

Reports delivered by OA division to air wing personnel

Data entered remotely via web site

CSG

From NPS thesis research of LT Jeremy Callahan, USN, 2006

Murphree et al., METOC Metrics, Jan 07, murphree@nps.edu

slide39

Online Real World Metrics System – Data Collection

Mission

Information

Debriefing

Information

Mission Data

Forecast

Negative Weather

Events

Observed

Negative Weather

Events

Weather

Events

Impacts of

Weather

on Mission

Mission

Impact

Changes Made

to Original

Mission by Pilot

Target Changes

And Cancellations

Mitigating

Actions

From NPS thesis research of LT Jeremy Callahan, USN, 2006

Murphree et al., METOC Metrics, Jan 07, murphree@nps.edu

slide40

Select 1-3 mission types that are most likely candidates for the establishment of a data collection system

    • Recommend the most predicable mission type for initial data collection effort
      • Allows process challenges to be solved with “low hanging fruit” before attempting more difficult mission types.
      • Established planning/execution/debrief cycle
      • Established, stable METOC products
    • Recommend stable mission area
      • Predictable tactics
      • METOC products not in flux
      • METOC verification possible
      • Debriefing is routine
      • METOC inputs are part of planning process
      • METOC personnel are welcomed participants at debriefs

For official use only

Ford B.W., NSW METOC Metrics - Next Steps, Feb 07, bruce@clearscienceinc.com

slide41

2. Diagram the mission flow process including each METOC and customer personnel step including planning, execution, and debrief.

  • Recommend timeline format where possible
  • Annotate METOC inputs (routine/non-routine/formal/informal) to the cycle.

For official use only

Ford B.W., NSW METOC Metrics - Next Steps, Feb 07, bruce@clearscienceinc.com

slide42

Select 1-3 mission types that are most likely candidates for the establishment of a data collection system

    • Recommend the most predicable mission type for initial data collection effort
      • Allows process challenges to be solved with “low hanging fruit” before attempting more difficult mission types.
      • Established planning/execution/debrief cycle
      • Established, stable METOC products
    • Recommend stable mission area
      • Predictable tactics
      • METOC products not in flux
      • METOC verification possible
      • Debriefing is routine
      • METOC inputs are part of planning process
      • METOC personnel are welcomed participants at debriefs

For official use only

Ford B.W., NSW METOC Metrics - Next Steps, Feb 07, bruce@clearscienceinc.com

slide43

3. Identify discrete and non-discrete data elements provided by METOC personnel. Determine what elements are routinely verified.

  • Use current METOC product examples
  • Discrete data element:
    • One forecast/one value
      • Ex: High Temp - 74
      • Ex: Sig wave ht (bottom) – 5 ft
      • Ex: Sig wave ht (top) – 8 ft
  • Non-discrete data element:
    • One forecast/many potential values
      • Ex: Satellite image
      • Ex: Coastal current image
  • List discrete elements and identify verification sources
    • Anecdotal verification from customer
    • Scientific verification
      • Trained observers
      • Collection equipment

For official use only

Ford B.W., NSW METOC Metrics - Next Steps, Feb 07, bruce@clearscienceinc.com

slide44

4. Determine how the customer determines mission success.

  • Determine currently used customer measures of success
    • Binary (e.g., success/failure)
    • Non-binary (e.g., T1-T4)
  • Determine our ability to collect customer measures of success. Is automation possible?
    • Pre-mission message
    • Post mission message
  • Determine the extent of customer self-analysis
    • What mission information is archived by the customer?

For official use only

Ford B.W., NSW METOC Metrics - Next Steps, Feb 07, bruce@clearscienceinc.com

slide45

5. Propose metrics to be collected (potentially multiple types)

Working definitions:

  • METOC performance metrics – Metrics of the quality of the METOC organization, including its operations, products, and services (e.g., accuracy of METOC forecasts of breaker height)
  • Customer performance metrics – Metrics of customer success (e.g., number of successful beach infiltrations)
  • Operational impacts metrics – Metrics of the impacts of METOC products on customer operations (correlation of breaker height forecast accuracy to beach infiltrations)
  • Proxy operational impacts metrics - Metrics of product performance that are acknowledged by METOC and customer personnel to be closely related to customer performance, so that the proxy operational impact metrics can be assumed to be good indicators of operational impacts (e.g., accuracy of breaker height forecasts which is assumed to be closely related to customer success)
  • Phenomena metrics – Metrics that relate product performance, customer performance, or operational impacts to specific environmental phenomena (e.g., breaker height forecast accuracy in the presence of high winds and seas; beach infiltrations in the presence of high winds and seas; correlation of breaker height forecast accuracy and beach infiltrations in the presence of high winds and seas)
  • Other examples of METOC metrics:
    • METOC Quality Metrics – Metrics of the overall quality of a product or service (e.g., forecast grade). These are a type of METOC performance metric.
    • METOC Capacity/Readiness Metrics – Metrics of organization’s operations (e.g., number of forecasters available, number of trained watchstanders). These are a type of METOC performance metric.
    • METOC Efficiency/ROI Metrics – Metrics of how well an organization uses its resources (e.g., number of forecasters required to publish forecast). These are a type of METOC performance metric.

For official use only

Ford B.W., NSW METOC Metrics - Next Steps, Feb 07, bruce@clearscienceinc.com

slide46

6. Propose system for metrics collection, calculation and display

  • Recommend SIPR web-based system
  • Minimize negative impact to METOC personnel
    • Strive for intuitive interface
      • Forecast collection
      • Verification input
      • Debrief input
  • Automate steps where possible
    • Message parsing
    • Forecast creator/collector
    • Verification collection
  • Strive to give METOC personnel a value-added tool
    • Streamline forecast process
    • Replace/enhance any current data collection efforts
    • Real time metrics calculation and display

For official use only

Ford B.W., NSW METOC Metrics - Next Steps, Feb 07, bruce@clearscienceinc.com

questions47
Questions

Ford B.W., NSW METOC Metrics - Next Steps, Feb 07, bruce@clearscienceinc.com