METOC Metrics for Naval Special Warfare Operations Presenter: Bruce W. Ford, Clear Science, Inc. Performers: Tom Murph

1. METOC Metrics for Naval Special Warfare Operations Presenter: Bruce W. Ford, Clear Science, Inc. Performers: Tom Murphree (NPS) Bruce Ford (CSI) Paul Vodola (SPA) Rick Allard (NRL-SSC) Presented 19 March 2008 at NRL-SSC. Updated on 20 March 2008.

2. METOC Metrics for Naval Special Warfare Operations Project Components • Data Collection, Analysis, and Reporting • Operational Modeling • Coordination and Collaboration

3. METOC Metrics for NSW FY07/08 Accomplishments Technical Objective/Funds • Develop and transition metrics system for METOC support of NSW operations including: • an online SIPRnet real world data collection, analysis, & reporting modules • an operational analysis tool set for developing model based metrics. • METOC Metrics Data Collection Demo Rel. 1-07 • METOC Metrics Data Collection Demo Rel. 1-08 • Analyzed mission planning process, developed preliminary small boat insertion/extraction mission model, and completed pilot analyses using model. • Coordinated/collaborated on: (1) assimilation of wave buoy data into SWAN adjoint model to provide improved initial conditions; and (2) bathymetric uncertainty using Delft3D ($K) FY07 FY08 FY09 6.2/6.4 0/200 0/255 0/255 MS Deliverables w/Delivery Dates FY08/09 Plans (Based on Projected Funds) • Demonstration of real-world metrics data collection system (Q2-08 and Q3-08) – TRL 3 • Real world metrics data Collection System (Q4-08) via METOC Metrics Web – TRL 4/5 • Demonstration of NSW operations/METOC support model (Q4-08) – Transition (Q4-09) – TRL 3 • Online modules demonstrated, tested, and delivered for operational use. • Finalize, develop, and test engagement model for small boat insertion/extraction operations • Coordinate/collaborate on development of ARCmap display for Delft3D results and measured field data; comparison of modeled and observed conditions for Ship Island experiment and RIMPAC08

4. Operational Outcomes Operational Plans METOC Forecasts * METOC Observations Operational Performance Metrics METOC Performance Metrics Metrics of METOC Impacts on Operational Performance Apply this process to both real world data and output from military mission models. Process for Developing METOC Metrics * or other products

5. Additional mission types Jump training SDV Ground Combat Small Boats MOUT Training Operational modeling Expand beyondtraining missions METOC Metrics for Naval Special Warfare OperationsProject Objectives Conceptual model: a fence with gates

6. METOC Metrics for Naval Special Warfare Operations Component 1: Data Collection, Analysis, and Reporting

7. METOC Metrics for Naval Special Warfare Operations Data Collection, Analysis, and Reporting • System Modules - Progress • Collect data on forecasted conditions – Ready to collect • Collect data on observed conditions (instrumented) – Ready to collect • Collect data on observed conditions (from warfighter) – 85% • Collect data on mission plans and outcomes (from warfighter) – 85% • Analyze data from modules 1-4, calculate and display metrics results – 15% • Administer metrics – 50%

8. METOC Metrics for Naval Special Warfare Operations Data Collection, Analysis, and Reporting • FY 08 Accomplishments • Demo release 1-07 (Oct 2007) • Prototype web interface for building threshold-based stoplight mission brief • Contains threshold viewer/administrative control • Adheres to top-level thresholds for color selection • Collect and stores mission and forecast information • Wish list collected from MSC personnel • Demo release 1-08 (Jan 2008) • Interface for recording multi-source observations • Enhancements to thresholds viewer

9. METOC Metrics for Naval Special Warfare Operations Data Collection, Analysis, and Reporting http://www.clearscienceinc.com/metrics/nsw_metrics/nsw_forecast_builder/Username: metrics Password: 915metrics*

10. METOC Metrics for Naval Special Warfare Operations Data Collection, Analysis, and Reporting

11. METOC Metrics for Naval Special Warfare Operations Data Collection, Analysis, and Reporting

12. METOC Metrics for Naval Special Warfare Operations Data Collection, Analysis, and Reporting

13. METOC Metrics for Naval Special Warfare Operations Data Collection, Analysis, and Reporting

14. METOC Metrics for Naval Special Warfare Operations Data Collection, Analysis, and Reporting

15. METOC Metrics for Naval Special Warfare Operations Data Collection, Analysis, and Reporting • FY 08/09 Objectives • Demo release 2-08 (Q3 FY08) • Complete modules 3 and 4 (warfighter obs/outcomes) • Deploy data collection to METOC Metrics Web - MMW (ASAP) • Awaiting final approval/implementation • ATOS2 environment (FNMOC) • Common landing pad for multiple METOC metrics projects • Deploy metrics display and admin modules (5 and 6) to MMW - Q1 FY09 • Complete online training for system users – Q1 FY 09

16. METOC Metrics for Naval Special Warfare Operations Component 2: Operational Modeling

17. Operational Modeling - Overview • Modeling simulates NSW ops, and the operational impacts of forecasted and actual METOC conditions • Inputs both actual METOC conditions and predictions with varying lead times and accuracies • Emulates actual mission planning processes weeks, days, and hours ahead of mission execution • Models customer decisions and outcomes based on METOC predictions and other factors • Value • Simulates impacts of METOC forecasts and phenomena on spectrum of NSW scenarios • Facilitates determination of phenomena which have significant impact on NSW operations • Establishes performance benchmarks for METOC support • Evaluates dependence of NSW effectiveness on varying types of METOC support

18. Model Design FY07 Lessons Learned Model as Laboratory METOC Data Collection Performance Thresholds Model Development • METOC Support Process • Products • Timelines • NSW Decision Process • Inputs • Timelines • Mission Planning Process • Etc… New Support Products & Concepts Support Evaluation Performance Benchmark Cyclical Metrics-Based Analysis of METOC Performance Updated benchmarks Operational Modeling Process

19. Ops Modeling – FY07 Accomplishments • Compiled database describing effects of METOC features on of SOF operations • Feature thresholds; operations & equipment impacted • Compiled from a variety of resources • Joint METOC Handbook • JTS/IWEDA • Critical METOC Thresholds for SOF Operations • Joint Doctrine & TTPS for METOC • Described NSW mission planning process • Detailed from the time Warning Order is received to the end of SOF extraction • Allows for high-resolution mission modeling • Helps answer the questions: • When do METOC products need to be provided to the mission planner? • How do METOC support & information affect the planning of various types of Missions? • Completed a pilot ops analysis using preliminary model built for small boat insertion/extraction operations • Metrics at multiple levels • Shows the impact that METOC support can have on NSW operations

20. Scenario Friendly Territory • NSW forces tasked with completing DA mission upon hostile island • SBU will utilize necessary assets to carry combat swimmers 30nm to chosen insertion point (IP), 1nm from landing beach • Swimmers will perform DA mission in no more than two hours, then head to extraction point (EP) • SBU will carry swimmers from EP to FOB • Operation must be completed in one eight-hour period of darkness FOB IP/ EP IP/ EP IP/ EP Landing Beach Mission Location Hostile Island Pilot Analysis - Scenario Possible SBU Transit Routes Possible Swimmer Routes Not Drawn To Scale

21. METOC features considered Nearshore currents Speed Direction Wave height Thresholds based on SOF-METOC database Feature forecasts provided 48 hours in advance of operation Mission planning based on predicted values Planning factors impacted Insertion asset Zodiac vs swimmers Insertion/extraction routes Multiple boat paths and landing beaches Actual features impact operations Mission performance based on actual values Operational MOEs Time to complete operation Probability of mission success Probability of increased risk to Special Boat Team Pilot Analysis - METOC Modeling

22. Deliberate Planning Timeline • METOC Impact on pre-execution timeline • Asset changes, COA rebrief, shifted execution times

23. Go/NoGo conditions correctly predicted 90% of the time 15% of missions predicted to be successful were not completed successfully 9% of missions failed due to actual NoGo conditions Pilot Analysis - Results • All NoGo conditions driven by Wave Heights • Current Speeds relatively benign

24. Operations Modeling – FY08 Goals • Finalize, develop, and test operational mission model for SBU operations (insertion/extraction) • Operational timelines • METOC support inputs • Mission planning process • Complete a full analysis cycle for SBU operations across a range of METOC features • Simulate the NSW mission planning process • Explore sensitivities to determine ranges of acceptable METOC support performance • Determine detailed benchmarks for METOC support to SBU operations

25. Operational Modeling: Transitioning Options Product / process End-users embrace products and process but play management role in methodology and toolsets Develop issues and initial conditions Evaluate progress, products, utility Similar to typical efforts for OPNAV N81/N84, well-defined schedule and purpose Focused tools All of functionality of Option 1, plus: Ability to use individual mission models to conduct data analysis as needed Well-defined interfaces with METOC and metrics inputs Turnkey system All of functionality of Options 1 & 2 plus: Ability to create and modify mission modules as needed User control over all key inputs and processes (environmental, tactical, mission planning, and control End-to-end analysis capability with automated guidance and error-prevention measures Lower Cost/Time More Functionality Transitioned Continuum of Ops Analysis Transition Options 1 2 3 Working with collaborators to identify most appropriate option.

26. Operational Modeling: Transitioning Steps Identify the user Characterize the functionality Specify outputs and objectives Identify data sources and interfaces Identify milestones and schedule Develop methodology / model design Implement and test Deliver and train Critical first steps FY08 FY09 Collaborating with user Need to identify who at CNMOC or elsewhere will oversee ops modeling transitioning and shape products

27. METOC Metrics for Naval Special Warfare Operations • Component 3: Coordination and Collaboration

28. Working with SWAN adjoint model examining the assimilation of wave buoy data to provide improved initial conditions. Examining NCEX exercise area (Scripps) Examined bathymetric uncertainty using Delft3D at Duck, NC Progress Since May 2007

29. Sandyduck97 • Chosen because part of Delft3D Validation Test Report. • Lots of data; various observations vs. time shown on right. • Three cases chosen (denoted with red dots in the figure on the right): • October 2, 1997; 01 EST • October 18, 1997; 13 EST • October 19, 1997; 07 EST

30. Sandyduck97 • Delft3D results compare well to measurements. • Shown are model-data comparisons for the October 18, 1997; 13:00 EST case.

31. Vignette 1: Bathymetry Changes Baseline (measured) bathymetry Vignette 1 (first hypothesized bathymetry) Difference between measured and hypothesized bathymetry

32. Vignette 1: Current Speed Changes Current speed given the Baseline (measured) bathymetry Current speed given the first hypothesized (Vignette 1) bathymetry Difference in the current speed given the differing bathymetries

33. Vignette 1: Wave Height Changes Significant wave height given the Baseline (measured) bathymetry Significant wave height given the hypothesized (Vignette 1) bathymetry Difference in the significant wave height given the differing bathymetries

34. Sandyduck97 Summary • The plan for Vignette 1 (alongshore uniform bathymetry) was carried out for three time instances of Sandyduck97. • Differences of 1 m between the baseline and vignette 1 bathymetries were evident. • The bathymetric differences caused differences of up to 0.5 m in the resulting wave predictions and 0.5 m/s in current magnitude predictions for the case of 1300 EST, 18 OCT 1997. • Future work can examine variations in bathymetry due to SEAL team surveys, inclusion of UAV-derived bathymetry and sand bar migration.

35. Ship Island Experiment12-14 March 2008 • Collaborate as source of SUROBs • FST to perform MBES and SBES survey of exercise area. • Use of UAV to collect bathymetry. • Delft3D to be run for Ship Island area. • High-res (5 km) COAMPS to force SWAN & Delft3D. • ADCP measurement of current flow near West Ship Island. • SUROBS to be provided for model evaluation. • Deployment of PSI wave buoy with ADCP.

36. Coordinate with related NSW/littoral projects Ship Island Experiment Potential source of SUROBs ARCOAS project: Geo-referencing interface for viewing key observations atop modeled environment. Potential tool available to MSC for displaying observations RIMPAC Source of SUROBs for use in verification Other Collaborationand Coordination Efforts

37. Objectives: Forecasters from reserve unit will be on the watch floor at FNMOC supporting reachback from USS Bonhomme Richard (LHD 6). In conjunction with NRL Probabilistic Prediction Research Office, FNMOC will provide products that illustrate the uncertainty in selected operational forecast parameters including surf. Plans call for ensemble runs of NOGAPS, WavewatchIII, SWAN and Delft3D or NSSM. Evaluation criteria still being developed. Timeframe: 26 June – 31 July 2008 POC’s: Dr. Jim Hansen (NRL-MRY), CDR Rebecca Stone (NPS) RIMPAC08

38. Possible Beaches • PMRF/Kauai (workups): southwest exposure • Bellows: east exposure • Kaneohe: north or southeast exposure • Kaena Point: north or southwest exposure • Kahukus: northwest through northeast exposures, but less likely Oahu OPS Kauai OPS

39. CAPT (s) Todd Monroe (DOO EXW) CDR Bruce Morris (NOSWC-SD) CDR Tim Gallaudet (NOSWC, NSWC) LCDR Damon Dixon (NOSWC) LCDR Bill Swick (NOSWC) LT Keith Plavnick (NOSWD Stennis) LT Doug Pearman (Fleet Survey Team) Bruce Gomes (NAVO NP2) Dean Wakeham (MSC) CDR Rebecca Stone (NPS) Todd Holland (NRL) Jim Hansen (NRL) Key Collaborators

40. METOC Metrics for Naval Special Warfare Operations Schedule Legend:S – Planned Start, D – Major Demonstration, T – Transition, R – Report, C – Complete Technology Readiness Levels: *TRL 5, **TRL 4, *** TRL 3

41. Technology Readiness Levels Definitions

42. METOC Metrics for Naval Special Warfare Operations Capabilities Summary • The products of this project will give NSW METOC • forecasters and managers the ability to: • Conduct semi-automated online collection of data from METOC units and customers • Quantify METOC product performance and impacts on customer operations • Assess uncertainty and confidence in METOC products • Predict impacts of METOC support on war fighting operations • Identify gaps in, and methods for improving quality and efficiency of, METOC support for NSW operations • Develop recommendations for allocation of METOC resources • Improve ability to incorporate METOC into Naval assessments

43. METOC Metrics for Naval Special Warfare Operations Capabilities Summary • The products of this project will give NSW operators • and managers the ability to: • Determine actual and potential impacts of METOC products on mission planning • Determine impacts of actual METOC conditions on mission execution • Estimate uncertainty and confidence in METOC products. • Identify gaps in, and methods for improving quality and efficiency of, mission planning and execution with respect to METOC factors • Develop recommendations for improving: (a) education and training; and (b) mission planning, execution

44. METOC Metrics for Naval Special Warfare Operations Transition Strategy and Status • Projected Transition Dates: • MS1: TRL 5 – Dec 08 • MS2: TRL 4 –Jul 09 • MS3: TRL 3 – Sep 09 • Transition Deliverables Status: • MS1: 73% - Left: Finalize collection modules, complete display/admin modules • MS2: 30% - Left: Finalize SBU ops model across METOC features, determine transition path • MS3: 50% - Left: MMW employment, documentation

45. METOC Metrics for Naval Special Warfare Operations Transition Strategy and Status • Transition POC • Overall: Dr. Tom Murphree • Component One: Bruce W. Ford (CSI) • Component Two: Dr. Paul Vodola (SPA) • Transition Issues • In final stages of procuring ATOS2 account (FNMOC) for MMW • Finalizing component two transition plan

46. METOC Metrics for Naval Special Warfare Operations Post-Project Efforts • Project is designed to operationally implement key elements of an NSW METOC metrics program. • But limited scope of project means that implementation of additional key elements will require follow-on 6.4 efforts. • Key elements that will still be needed at end of project include component one modules and component two models customized for all types of NSW operations and for real world operations. • Need to begin planning now for these follow-on efforts.

47. NSW METOC Metrics Project Sites NIPR: http://met.nps.edu/metrics/navy/nsw_metrics/ SIPR: http://web.ntsstl.nps.navy.smil.mil/metrics/nsw_metrics/

48. Project Contact Information • Systems Planning and Analysis, Inc. • 2001 N. Beauregard Street • Alexandria, VA 22311 • Fax: 703-399-7365 • Paul Vodola, Ph.D. • Email: pvodola@spa.com • Paul.Vodola_Contractor@spa.dtra.smil.mil • Phone: 703-399-7225 • Luke Piepkorn • Email: lpiepkorn@spa.com • Phone: 703-399-7239 • Naval Research Laboratory • Oceanography Division • NRL Code 7322 • Stennis Space Center, MS 39529-5004 • Fax: 228-688-4759 • Richard Allard • Email: allard@nrlssc.navy.mil • Phone: 228-688-4894 • Kacey Edwards • Email: kedwards@nrlssc.navy.mil • Phone: 228-688-5077 Naval Postgraduate School Department of Meteorology 254 Root Hall, 589 Dyer Road Monterey, CA 93943-5114 Fax: 831-656-3061 Tom Murphree, Ph.D. (project lead) Email: murphree@nps.edu murphrjt@nps.navy.smil.mil Phone: 831-656-2723 Clear Science, Inc. 7801 Lonestar Rd Suite #17 Jacksonville, FL 32211 Fax: 904-379-9704 Bruce W. Ford Email: bruce@clearscienceinc.com fordbw@tsc-jax.navy.smil.mil Phone: 904-379-9704