an independent technical review process for government developed models l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
An Independent Technical Review Process for Government Developed Models PowerPoint Presentation
Download Presentation
An Independent Technical Review Process for Government Developed Models

Loading in 2 Seconds...

play fullscreen
1 / 14

An Independent Technical Review Process for Government Developed Models - PowerPoint PPT Presentation


  • 158 Views
  • Uploaded on

An Independent Technical Review Process for Government Developed Models. HASDM As a Case Study Michael J. Gabor. Outline. Introduction JAWG HASDM Technical Review Process Lessons Learned Summary and Conclusions. Introduction. U.S. Space Command JAWG HASDM. JAWG.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

An Independent Technical Review Process for Government Developed Models


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
an independent technical review process for government developed models

An Independent Technical Review Process for Government Developed Models

HASDM As a Case Study

Michael J. Gabor

outline
Outline
  • Introduction
  • JAWG
  • HASDM
  • Technical Review Process
  • Lessons Learned
  • Summary and Conclusions
introduction
Introduction
  • U.S. Space Command
  • JAWG
  • HASDM
slide4
JAWG
  • Originally associated with SSWG
  • Led by USSPACE/J33 and AN
  • Then leadership transferred to AN
  • Coordinate R&D of SSA (especially space surveillance) between services
  • Attempted USSPACE Astro standards
hasdm
HASDM
  • DCA
    • Use catalog to determine common corrections that are expected to account for common atmospheric characteristics
    • Various approaches to DCA exist
      • HASDM
      • SSULI - Special Sensor Ultraviolet Limb Imager (NRL)
      • Nazarenko
      • EDR (Storz)
  • E10.7
    • New solar flux proxy intended to replace F10.7
    • Based on satellite EUV measurements
technical review process
Technical Review Process
  • Pre-meeting info dissemination
  • Pre-meeting Q&A
  • Briefings
  • Q&A
  • Feedback forms
  • Organizational memos
  • Feedback filtered for biases, inaccuracies, ignorance, attribution
feedback survey
Feedback Survey
  • Respondent information (not used for report)
  • HASDM Project Comments
    • Assumptions
    • Consistency of methods
    • Appropriateness of tests (complete, independent)
    • Meaningfulness of results
    • Traceability of conclusions
    • Additional thoughts
feedback survey8
Feedback Survey
  • Technical Review Comments
    • Success of review
    • Prior expectations
    • Difference between expectations and results
    • Value and effectiveness of process
    • Recommendations to improve
    • Additional thoughts
review outcome
Review Outcome
  • HASDM evaluation - too much to present - see paper for examples
lessons learned
Lessons Learned
  • Model development
    • Specific criteria need to be established to enable objective evaluations
    • Options should be investigated and documented first (approach, baseline, parameterization, validations)
    • Design tests that are independent
lessons learned11
Lessons Learned
  • Review process
    • Coordination of review needs to be improved
      • Participants were not sure who briefs, who pays, etc.
    • Insufficient time for review
    • Lack of formalism in process
      • No mechanisms for tracking resolution of criticism
      • After USSPACECOM demise, no accountability to process
recommendations hasdm
Recommendations - HASDM
  • E10.7 was not ready for implementation
  • The progress of the DCA portion of HASDM merited further evaluation
  • Establish evaluation criteria
  • Investigate other options
  • Conduct/Improve tests for performance evaluation and operational suitability
  • Perform cost/benefit analysis
  • Distribute as much data as possible to the research community to improve the science and allow for other ideas to mature
recommendations technical review
Recommendations - Technical Review
  • Process should be applied new models or significant modification
  • Review needs to become requirement for operational implementation
  • Organization needs to be assigned responsibility
    • Needs authority, independence, and impartiality
  • Need mechanisms for tracking resolution
summary and conclusions
Summary and Conclusions
  • Review process used only once for HASDM
  • Identified criticisms of execution in model development and highlighted areas of concern
  • Serves as example for future reviews
  • Technical review is necessary to gain community buy-in and identify loose ends
  • Data and technical info should be made available to industry and academia
  • Overcome institutional inertia obstructing team work, cooperation, and interoperability