Jason tuell office of science and technology
This presentation is the property of its rightful owner.
Sponsored Links
1 / 30

Jason Tuell Office of Science and Technology PowerPoint PPT Presentation


  • 63 Views
  • Uploaded on
  • Presentation posted in: General

AWIPS Tech Infusion The Evolution of AWIPS Operations Committee 5 September 2007. Jason Tuell Office of Science and Technology. Overview. The Why and What of AWIPS Evolution Objectives Re-architecture Approach Roadmap Exciting features and GIS What does AWIPS II mean to NWS Operations?

Download Presentation

Jason Tuell Office of Science and Technology

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Jason tuell office of science and technology

AWIPS Tech InfusionThe Evolution of AWIPSOperations Committee5 September 2007

Jason Tuell

Office of Science and Technology


Overview

Overview

  • The Why and What of AWIPS Evolution

  • Objectives

  • Re-architecture Approach

  • Roadmap

  • Exciting features and GIS

  • What does AWIPS II mean to NWS Operations?

  • Summary


Why change

Case for change briefed to NWS Corporate Board – Nov 2004

AWIPS Present State Summary

Hardware

AWIPS hardware was in good shape

Communications Infrastructure

AWIPS communications infrastructure was in OK shape

Data

AWIPS Data was in need of improvements

Software

AWIPS software was in critical need of improvements

Costly software development, maintenance and inability to meet NWS and customer needs

Corporate board direction to focus on addressing software shortcomings

Plan and requirements developed

Shaped portions of the AWIPS O&M re-compete activity

Why Change?


What is awips evolution

What is AWIPS Evolution?

  • AWIPS Evolution

    • A long-term project which delivers a modern, robust software infrastructure that provides the foundation for future system level enhancements

  • AWIPS II

    • Implements a modern Services Oriented Architecture (SOA) infrastructure

    • First output of AWIPS Evolution and provides the foundation for all subsequent improvements

  • AWIPS II Extended - Creation of a seamless weather enterprise spanning NWS operations

    • Migration of NAWIPS into the AWIPS II SOA

    • Delivery of thin client

      • Support to the Weather Service Offices and Center Weather Support Units

    • Integration of “orphan” systems (e.g., Weather Event Simulator)

  • AWIPS II Enterprise Enhancements

    • Data Delivery Enhancements

      • “Smart push-smart pull” data access

      • Katrina satellite WAN back up

    • Integrated visual collaboration

    • Visualization Enhancements

    • Information Generation Enhancements


Awips evolution objectives

AWIPS EvolutionObjectives

  • Establish Service Oriented Architecture for AWIPS and NAWIPS

    • Create a seamless weather enterprise that supports all levels of NWS operations from National Centers to WSOs

    • Build a common development environment that will be used by all developers

    • Establish infrastructure for GIS integration

  • Enable access to data independent of its location, i.e., provide access to data not resident locally at the WFO or RFC.

  • Provide infrastructure for real time graphical collaboration between

    • WFOs, RFCs and centers for enhanced internal collaboration

    • Other NOAA entities and

    • Trusted partners, e.g., Emergency Managers

  • Implement a Common AWIPS visualization environment (CAVE) used by all applications

  • Standardize generation of NWS products and services


Awips ii re architecture approach

AWIPS IIRe-Architecture Approach

  • Perform “black-box” conversion

    • Preserve existing functionality, look and feel on top of new infrastructure

  • Thorough field validation and acceptance before deployment

  • No loss of functionality

    • Deployed system current with deployed AWIPS capability (i.e., OB9)

  • Use open source projects - No proprietary code

    • JAVA and open source projects enable AWIPS II to be platform and OS independent

      • No plans to move from Linux

  • ADE enables collaborative development

    • OS, Platform independence allows non-Linux based research to be easily integrated into AWIPS II


Awips ii features

AWIPS II Features

  • AWIPS Development Environment (ADE)

    • Used by all AWIPS developers (National, Regional, & Local)

    • Developers concentrate on new capabilities, not re-implementing existing ones (i.e. screen I/O, communications protocols, data access routines, logging routines, or other previously developed capabilities)

    • Software can be developed on a variety of platforms

  • Robust infrastructure for improved software O&M

    • Use of plug-ins: visualization extensions; new data types and transforms

    • System level, remediation, core services reduce system complexity

    • Improved support for local requirements (e.g., local apps, scripts, plug-ins)

  • Common AWIPS Visualization Environment (CAVE)

    • Provides a common development and execution environment for AWIPS GUIs (e.g. D2D, NMAP, GFE, etc.)

    • Ability to pan/zoom large data sets (Raster & Vector) with flexibility over data rendering

    • GIS tools

    • Thin Client (Web Browser) enabled

  • Dynamic Load balancing

    • Processing dynamically allocated among available CPUs


Jason tuell office of science and technology

AWIPS IIRoadmap

AE OSIP Gates

2

3

4a

Meshed Topology

4b

Analysis

ADE Training Development

PIP

ADETraining

Migration Planning

RTSIRAD

ADE Development

AWIPS II 1.0

10/10/07

Migration

Strategy

2010

2006

2007

2008

2009

MPLS

OBx

8.3

9

7

8

8

10

Deployment

OB 9 Dev & Test

New Release

Paradigm

SW CTR (AWIPS II)

NWS New Capability Development in ADE

O & M Transition

O & M Transition Prep & Coordination

Baseline Application Migration

OTE / Deployment Support

Note: Task bar colors are

For speaker reference only

“User” Functional Tests

ADE Local App Training

Local App Migration

C & A

OTE

= Calendar Year

Deployment

Deployment Planning

= Fiscal Year

Field Ops Training

-- ITO, ESA


Jason tuell office of science and technology

AWIPS EvolutionRoadmap

2007

2008

2009

2010

2011

2012

2013

2014

Baseline Application Migration

WES Integration

IOC

FOC

Phase 1 Collaboration

Phase 2

Phase 3

= Calendar Year

= Fiscal Year

IOC

FOC

Information Generation

IOC

Visualization

AWIPS II

AWIPS II

OTE / Deployment

Governance Model

NAWIPS Migration

AWIPS II

Extended

Thin Client

CHPS

AWIPS II

Enhancements

Data Delivery


Awips evolution governance model

AWIPS EvolutionGovernance Model

  • What is it?

    • Governance model controls the development, test, integration, configuration management, deployment and support of the new system -- both hardware and software

  • Why?

    • AWIPS II offers new levels of flexibility and extensibility

    • New rules needed to take advantage of system capabilities and also define limits

    • Tension between unlimited modifications and ability to support the system

  • Sample issues for consideration

    • Monolithic configurations no longer required -- how do we manage site specific configurations?

    • Plug ins can be down loaded and installed on demand

    • Scripting that modifies AWIPS menus, functions

  • Flexible Governance Model needed for limited open source implementation


Awips ii what gets us excited so far

AWIPS IIWhat gets us excited so far…

  • Dynamic load balancing

    • Failover handled automatically

    • Enables consideration of tailored hardware configurations

  • Mathematically intensive calculations handed off to the graphics card

    • Significant performance improvements

  • Progressive disclosure of all data

    • Imagery via quad tree tiling, grids and observations

  • Integrated thin client

    • Allows baseline solution to be extended to CWSUs, WSOs, and IMETs

  • Integrated drawing and graphical collaboration

    • Tools built into the infrastructure, implemented in 2011

  • Built in GIS via geotools library

  • Scripting level access to practically all system level services and functions

  • LESS CODE

    • Potential order of magnitude reduction in amount of software with increase in functionality


Awips ii gis capabilities

AWIPS IIGIS Capabilities

  • GIS functionality integrated into AWIPS II architecture via GeoTools

  • GeoTools

    • Open Source Java code library

    • Provides OGC compliant methods for manipulation of geospatial data

    • Equivalent to ESRI’s arc engine

  • AWIPS II will support JavaScript access to GeoTools library

    • ADE 1.0 supports limited access

      • Read, write shape files

      • Create GeoTiffs

      • Spatial queries of ingested data

    • Task Order 10 will deliver full JavaScript access

  • GeoTools does not make AWIPS a stand alone GIS system

  • Reference:

    • GeoTools home: http://geotools.codehaus.org

    • GeoTools & Eclipse: http://docs.codehaus.org/display/GEOT/Eclipse+Developers+Guide


Awips ii operational impacts

AWIPS IIOperationalImpacts

  • Forecaster

    • Little to no impact anticipated

      • Look & Feel preserved

  • ESA/ITO

    • New architecture drives changes to:

      • Release Installations (projected to be easier & shorter in duration)

      • System Maintenance

      • System Troubleshooting

  • Application Focal Point

    • Definition of application changes under new architecture

    • Application configuration likely to change

      • Do not know by how much at this time. Better idea around end of calendar year (2007)

  • Local Application Developer

    • Local applications need to be migrated to new infrastructure

      • Migration path needs to be determined for each local app

    • New development accomplished within ADE

      • Will need to learn new concepts - object oriented programming, SOA prinicples

      • Will need to learn new languages -- JAVA script and potentially JAVA -- still defining requirements


Awips ii training

AWIPS IITraining

  • Strategic Training Plan developed

  • Training targeted for the following groups:

    • ESAs

    • ITOs

    • AWIPS and application focal points

    • Developers (both baseline and local)

    • NCF

    • SST

  • Training Organizations involved in planning, developing and implementing courses


Awips ii local applications

AWIPS IILocal Applications

  • Survey – completed August 31, 2007

    • 4191 local applications identified

    • 37% critical, 44 % important, 19% less important

  • Migration plan to address approach based on survey results

    • Training requirements and approach to be refined based on survey

    • Raytheon to provide sample migration and code samples for approach

    • “Integrate with” vice “interface to”

  • Level of effort required uncertain

    • Raytheon estimate that 80% of local apps will be able to be rewritten in Javascript, without extensive programming in the ADE


Awips ii testing

AWIPS IITesting

  • Layered testing strategy

    • Different testing phases overlap – functionality tested multiple times in different settings

  • ADE usage

    • Not formal testing, but expect to get feedback as developers use the ADE

  • Algorithm V&V –

    • Assist algorithm choice

    • Verify port of algorithms

  • V&V

    • Task order validation

    • Executed by dev orgs in controlled setting

    • Performance testing – Evaluate system against known and developing benchmarks

    • Acceptance testing – part of the task order acceptance process

  • Risk reduction

    • Side by side testing of new system by field personnel in a “lab” environment

    • Testing at limited number sites in manner that doesn’t impact operations

  • Operational Test & Evaluation (OT&E)

    • Formal testing of the entire system

    • Tests system interfaces, operations, support, training, etc.


Awips ii risks and challenges

AWIPS IIRisks and Challenges

  • Organizational challenges

    • Significant off contract workload on Government

    • Matrixed structure increases project complexity

  • Performance

    • Supporting the short fuse warning mission

    • Handling large global data sets

  • Schedule

    • Completing the migration and testing

  • Migration of local applications

    • Local applications outside the baseline

    • Not a Raytheon responsibility


Summary

Summary

  • New architecture defined and infrastructure delivered

    • ADE 1.0 delivered June 14, 2007

  • Application migration underway

    • Migration Plan delivered June 2007

    • AWIPS baseline migration to be completed FY09

    • Local applications migration planning in progress

  • Incremental approach will extend AWIPS II architecture throughout NWS

  • AWIPS II will deliver capabilities that enable NWS to be more responsive to emerging requirements


Back up

Back Up


Awips evolution outcomes

AWIPS EvolutionOutcomes

  • Short-term (1-3 years)

    • Shorten transition of research to operations

    • Improve software O&M and technology refresh

      • Fewer DRs and TTs

      • Focus on hardening and productionizing for life cycle support

    • Minimize adverse impacts on operations from software and hardware upgrades

  • Long-term (3-10 years)

    • Increase integration of AWIPS and National Center AWIPS

    • Improve performance and functionality of AWIPS

    • Improve collaboration at all levels of NWS operations

    • Increase access to all environmental data for decision making


Awips evolution data delivery

AWIPS EvolutionData Delivery

  • OSIP Project 05-040

  • Enables “smart push - smart pull” data delivery

    • Implements a discovery service within the SOA

    • Access to data not available locally

    • Freedom from the tyranny of the SBN

    • Enables consideration of new data delivery architecture

      • What data to you broadcast over SBN?

      • What data do you make available on servers?

  • Schedule

    • IWT starting Q4 2007 to define concept of operations and operational requirements

    • IOC - 2011 - software implementation for remote data access

    • FOC 2012 - enterprise configuration (servers, comms, etc.) that enables remote data access


Awips evolution collaboration

AWIPS EvolutionCollaboration

  • OSIP Project 05-041

  • Objective

    • Integrated graphical collaboration throughout the NWS Weather Enterprise and beyond

    • Phase 1 - Integrated collaboration between all levels of NWS operations

    • Phase 2 - Collaboration between NWS offices and other NOAA entities

    • Phase 3 -Collaboration between NWS offices and trusted external partners, e.g., Emergency Managers

  • Schedule

    • Phase 1 IOC - 2011

    • Phase 2 IOC - 2012

    • Phase 2 IOC - 2013


Awips evolution information generation visualization

AWIPS EvolutionInformation Generation & Visualization

  • OSIP Projects 05-042 (IG) and 05-021 (Vis)

  • Information Generation objective

    • Re-architect generation of all NWS products and services

    • Separation of content generation from formatting and dissemination

    • Enable faster response to emerging customer demands

  • Visualization objective

    • Common user interface - standardize User Interfaces across applications

    • 3-D visualization

    • Improve user interfaces based on latest principles and research


What is awips

NEXRAD

GOES/POES/NPOESS

Warnings

NCEPModels

Watches

Advisories

ASOS

Forecasts

Buoys,

River Gauges

What is AWIPS?

169 separate AWIPS systems at

137 geographical locations

AWIPS

Communications

AWIPS Workstations and

Servers

Service provided to

3066 US Counties

24 hrs/day,

365 days/yr.

~900 Workstations (total)

~1200 Servers (total)


Enterprise architecture

Systems Legend

Dissemination

Modeling & Product Generation

Observing

Enterprise Architecture

“AWIPS is the Integrating Element of the Modernized Weather Service”

Figure 6-8 of

NOAA EA v 1.6

* Note some systems perform multiple functions, e.g. LDAD disseminates information and receives observations


Architectural compliance

Architectural Compliance

  • Collaboration on Achieving Enterprise Solutions

    • AWIPS is the Integrating Element of the Modernized Weather Service

    • AWIPS Uses Enterprise Red Hat Linux support contract

    • AWIPS Contract supports Climate, Weather and Water, Transportation, and extensible to Coastal and Ocean Resources

  • Reuse of existing assets

    • HazCollect uses AWIPS IT infrastructure for message handling/creation and routing to dissemination systems

    • Shared use of NOAA-Net for Wide Area Network in progress

    • AWIPS provides NEXRAD visualization and data collection functions

    • AWIPS software used by other Government projects - RSA II, WES, FX-Net

  • Usage of Standards

    • Complies with statutory requirements

      • Information Quality (Section 515) & Accessibility (Section 508)

      • Government Paperwork Elimination Act (GPEA)

      • FISMA/NIST Standards and Guidance


Awips evolution an ea perspective

AWIPS EvolutionAn EA Perspective

  • EA perspective focuses on hydrological and meteorological mission operations

    • Administrative functions not part of AWIPS

    • Supercomputing functions outside of AWIPS

  • AWIPS II compliant with current DMIT data standards

  • Some services, e.g., discovery service, not implemented as part of AWIPS II

    • Will be implemented in the Data Delivery system enhancement

  • Exploring partnerships with other agencies within and outside of NOAA

    • NESDIS and NPOES NDE

    • NASA SPORT and NASA Goddard


Open source projects used in ade

Open Source ProjectsUsed in ADE


Soa architecture logical layered view layers separated by simple apis

<<Java>>

HDF5DataStore

<<Java>>

DataLayer

<<abstract>>

BaseDao

SOA Architecture – Logical Layered ViewLayers Separated By Simple APIs

Client/Presentation Services

JMX

CAVE

External Programs

Mission Services Layer

Mbean

ProductSrv

IngestSrv

NotifySrv

Hydro Models

AdapterSrv

AutoBldSrv

PersistSrv

IndexSrv

SubscribeSrv

UtilitySrv

LAPS

Security Services /Demilitarized Zone (DMZ)

Enterprise Service Bus - Communication

PurgeSrv

StagingSrv

FORTRAN/C/C++

Command Line

Programs

Data Access Layer

Hibernate

HDF5 API

Localization

Store

Platform Layer

PostgreSQL

HDF5

Data Persistence

Store

Metadata

Index

Spatial

Index


Awips ii ade high level system services soa services running in an esb container

AWIPS-II ADE High Level System ServicesSOA Services Running in an ESB Container

CORE SOA Services

JMX Remote

Service Management

Client

WAN Visible

ADE 1.0

Updated T06

Mbean

Mbean

Mbean

Mbean

Mbean

IngestSrv

StagingSrv

PersistSrv

PurgeSrv

IndexSrv

ProductSrv

AdapterSrv

CAVE

Visualization

Client

FileSystem

Meta Data Index

uEngine

Exec Adapter

Data Rendering

METAR Plug In

RDBMS via JDBC

Spatial Data Base

Manage Subscription

JNI Adapter

CAVE Bundles

Radar Plug In

HDF5 persistence

GFE IFPServer

Data Interrogation

Satellite Plug In

CAVE Procedures

GRIB Plug In

Wx Drawing

Wx Warning

Radar All Tilts

Hydro Visualization

Mbean

Mbean

Mbean

UtilitySrv

NotifySrv

AutoBldSrv

Collaboration

uEngine

Localization Data

Subscription Notify

XMPP

Color Tables

Maps and Topo

Enterprise Service Bus – HTTP, JMS, Virtual Memory, File Endpoints

Enterprise Service Bus – HTTP, JMS, Virtual Memory, File Endpoints

Services Independent of End Points


  • Login