The USGODAE Monterey Data Server 13 March 2007

1. The USGODAE Monterey Data Server13 March 2007 http://www.usgodae.org Phil Sharfstein GODAE Project Manager phil.sharfstein.ctr@metnet.navy.mil

2. The Global Ocean Data Assimilation Experiment (GODAE) • Make ocean monitoring and prediction a routine activity in a manner similar to weather forecasting. • GODAE will contribute to an information system for the global ocean that will serve interests from climate and climate change to ship routing and fisheries. • The USGODAE Server is developed and operated as a prototypical node for this global information system.

3. The International GODAE Project Time Line • 1998-1999Concept • USGODAE Server begins initial development and deployment • 2000-2002Development • Server brought on-line as a 24x7 operational system • Funding provided by ONR for the duration of GODAE • 2003-2006Operational Demonstration • USGODAE Server sees steady growth in usage • Increased involvement in International GODAE Projects • Increased breadth of data holdings • 2007-2008Consolidation and Transition • ONR funding ends 2008 • Server Decommissioned (If no additional funding is provided) • Goal: Seek alternative funding source to ensure that the system remains available to support both DoD operations and research, and the current broad range of R&D users.

4. USGODAE Server Schedule • 2006-2007:Full Support • ONR has committed to fund server operation through the end of the GODAE Project. • FNMOC will continue to provide full support for all server operations and data management tasks. • 2008: Server Decommissioned (If no additional funding is provided) • Transition Argo GDAC to other funded resource (if available). • Provide PKI data access for all DoD researchers currently using USGODAE. • Provide data distribution for FNMOC data sets only served by USGODAE that are used for DoD research (Computational Grid NOGAPS, Ocean QC, High-Resolution Computational Grid Ocean Model/Analysis Products).

5. USGODAE FTP Activity Through the Operational Demonstration Phase Monthly Downloads Monthly Sessions

6. GODAE Server • Hosted in the FNMOC operational data center • Provides 24x7 system support. • Development through a partnership of FNMOC and NOAA's Pacific Marine Environmental Lab (PMEL) • The server hosts: • In-situ oceanographic data • Atmospheric forcing fields suitable for driving ocean models • Unique GODAE data sets, including demonstration ocean model products • Primary Server users include: • GODAE participants • IOOS community • DoD activities • General earth and ocean science research community • The GODAE Server has played a significant role in the international GODAE program and the ongoing development of ocean models

7. USGODAE Hardware • ONR support for the final phase of GODAE included funding for a complete hardware upgrade. • High reliability/high availability architecture (ATOS2) • Scaled to meet the increasing server usage • Provides the resources to run the latest data serving applications

8. USGODAE Data Serving Goals • DoD Users: • Reliable, easy access to real-time and short-term archives of FNMOC public data sets • Operational access to unique GODAE data sets for NAVO and FNMOC • General Users: • Flexible access to data: users can quickly find and start using the data they need. • Provide well documented, standard Web interfaces by adhering to community data serving standards: • National Virtual Ocean Data System (NVODS) • Integrated Ocean Observing System Data Management and Communications (IOOS DMAC)

9. USGODAE Data Server Applications • All Server data are available through anonymous FTP and HTTP download • Custom Web Applications • Data Catalog – access and documentation links • Preview Generator – non-gridded data • THREDDS - Data discovery and catalog for NVODS • Local Data Manager (LDM) - Data push for large model output • OPeNDAP - A Protocol that facilitates access to data and data products via the Internet • Including GrADS/Data, Aggregation, and CGI servers • Live Access Server (LAS) - Human-intelligible, geo-referenced browsing for all NVODS data

10. USGODAE Data Discovery and Catalog NASA Global Change Master Directory Unidata THREDDS Catalog Custom USGODAE Data Discovery Application

11. USGODAE Data Serving Example:Argo Global Data Assembly Center (GDAC) • Transition Goal: Continue to provide the complete archive of quality-controlled Argo data on a Navy hosted server • Allow operational access for NAVO and FNMOC to one of the most important in-situ oceanographic data sets • Argo is a major project under GODAE and the Global Ocean Observing System (GOOS) • Will have a global array of ~3,000 drifting temp./salinity profiling floats sending data in real-time • USGODAE is one of two Argo GDACs • Hosts the complete set of quality controlled T/S profiles • Over 300,000 profiles on-line from 9 national/regional centers • Delayed Mode (Scientific QC’d data) on-line • Currently 90,000 delayed mode profiles • USGODAE Argo Data served through LAS and OPeNDAP • Provides complete integration of Argo data with NVODS and IOOS

12. Monthly Argo Coverage (Aug. 2005)Image from USGODAE Argo Data Browser Constrain By: Time, Region, DAC, Float ID, Delayed Mode Data Browser provides fast access to data previews and easy download for Argo data files

13. Argo GDAC Web Interface Output File Retrieval and Preview Options

14. Argo Data Served Through OPeNDAP (A.K.A DODS) • A Protocol that facilitates access to oceanographic data and data products via the Internet • Regardless of: • data type • where the data are stored • the format in which they are stored • the user's visualization package • the user's level of expertise.

15. OPeNDAP Server for Argo Data • All NetCDF files on the GDAC are available through OPeNDAP • OPeNDAP functionality • Export to binary or ASCII • Subset • Subsample • Browse • Standard Interfaces and APIs

16. OPeNDAP Aggregation Server for Argo Data (and THREDDS Catalog) • Create virtual collections of Argo data without hosting multiple copies of each profile. • FUTURE: THREDDS will provide an OpenGIS compliant interface to Argo data

17. Argo Data Served Through Live Access Server “human-intelligible, geo-referenced browsing for all NVODS data” • Basic graphics • Well annotated • All possible viewing planes and axes • Readable tables of values • Formatted subsets of data • NetCDF, tab-delimited, GIS formats, others… • Comparisons • Overlays • Side-by-side plots • Difference plots (with regridding) • and basic transformations Data Products Analysis Tools

18. LAS Interface for Argo Data • Select profile data by: • Time Period • Region • Depth Range • Additionally constrain by • Float ID • DAC • Real-Time/Delayed Mode • QC Values

19. Standard LAS Output Options for Argo Data Gaussian Pie Comma Separated Metadata HTML Text Waterfall

20. Custom LAS Argo Data ProductsColor coded legends list location, time, and float/profile ID Profile T/S Composite Track/Location

21. LAS Single/Multiple Profile Selection Select product and variables Select one or more profiles

22. Unique USGODAE Data Sets: FNMOC Model Forcing • Transition Goal: Provide public access to FNMOC data sets for non-operational Navy/DoD use • Reduces work load on FNMOC staff fulfilling custom archived data requests and new account requests from non DoD users • Academic and industry partners have instant access to data needed for their projects • Data available through FTP, HTTP, LDM, LAS, and OPeNDAP • Global NOGAPS • Data on 1° application grids sufficient to force the regional COAMPS model (~5500 files/watch) • Surface forcing fields, boundary conditions and fluxes on ½° Gaussian computational grids custom tailored to force ocean models • Regional COAMPS – Three Regions Surrounding CONUS • WW3 – Complete Wave Watch 3 144 Hr. forecast • NCODA – Global 3D Ocean Temperature, Salinity, and Surface Height Analysis • GHRSST – High Resolution 6 hr. SST/Sea Ice Analysis

23. NOGAPS Surface Pressure Data through LAS GRIB → GDS →LAS

24. Unique USGODAE Data Sets: Near Real-Time In-Situ • Transition Goal: Provide DoD and public access to data not generally distributed through the operational data feeds • Quality controlled internal model initialization data • Used by NRL, NPS, and NASA researchers • Observational data from the FNMOC Ocean QC • Contains all non-classified data used to initialize NOGAPS • Data files contain all background fields used in QC process • PROFILE: Fixed/drifting buoy, bathy, and PALACE float data, including real-time Argo profiles • SURFACE: Ship, fixed/drifting buoy, and CMAN in-situ SST • ALTIMETRY: Level 2B TOPEX, GFO, and ERS altimeter SSH • MCSST: Level 2B AVHRR SST retrievals • SSM/I: Level 2B DMSP sea ice concentration retrievals • Meteorological (MET): Three week archive of the latest 4x daily MET • NAVOCEANO Level 2B Satellite Data • SST and SSH: Including high resolution LAC SST

25. In-Situ Data Access Thumbnail Generator (From Files) LAS: Custom Database

26. The USGODAE Server User Communities • GODAE • DoD • Non-DoD Research/Commercial • IOOS

27. Server User Community: GODAE The Server acts as a focal point for US and international GODAE projects, data, and information. • Data for GODAE • Custom tailored model forcing fields • Near real-time In-situ atmospheric and oceanographic data • Operational high-resolution model and analysis fields for GHRSST • GODAE Project Hosting • Ocean QC Intercomparison, Equatorial Intercomparison, Model Forcing Intercomparison, LAS GODAE Modelers Output Site. • GODAE Information • Meeting agendas and presentations, project documentation

28. DoD Usage of the USGODAE Server • The DoD community is one of the largest server users. • 56.5% of HTTP Downloads (5,213,704 files/year) • 10.8% of FTP Downloads (3,690,638 files/year) • Top DoD Users: • Navy Operational Centers: Naval Pacific Meteorology and Oceanography Center, NAS Point Mugu, Sasebo Naval Base, US Pacific Command, Naval Oceanographic Office • Navy R&D: NRL Monterey, NRL Stennis, Naval Postgraduate School • Other DoD: Air Force Weather Agency, US Coast Guard, Army Research Labs • DoD Industry Partners: Applied Science Associates (Coast Guard), Lockheed Martin (Air Force), Barrodale Consulting Services (Navy), SAIC (Army)

29. Server User Community: General R&D • The USGODAE Server is the primary source of FNMOC model and observational data for a broad range of research projects and institutions. • Many funded research projects that will continue beyond the GODAE transition period rely on USGODAE as their principal data source. • The Server provides a framework and data archive for DoD R&D that will need to be replicated at each new (or existing) project.

30. Non-DoD R&D and Commercial Users • .gov • NOAA: FSL, NODC, NDBC, AOML, PMEL, GFDL • NASA: JPL, GSFC • .org • UNESCO, Alaska Ocean Observing System (AOOS), Southeastern Universities Research Association Coastal Ocean Observing and Prediction Project (SURA/SCOOP), Gulf of Maine Ocean Observing System (GOMOOS), Columbia Scientific Balloon Facility, Seaturtle.org • .edu • Several hundred institutions retrieving over 150,000,000 files each year through FTP, HTTP, and LDM. • Partnered with Unidata/UCAR to provide data for educational research. • .com (Excluding DoD Partners) • Orbimage, Accuweather, Weather3000, Surfline, Ocens, Weathernews

31. Server User Community: IOOSThe bridge between GODAE and IOOS • The USGODAE Server currently provides a wide range of real-time data, and oceanic/atmospheric models that are essential to the success of several IOOS projects. • The Server is one of the primary real-time data servers for global in-situ oceanographic data. • The USGODAE Server is designed and operated to adhere to the data serving standards specified by the IOOS/DMAC. • As a partner in IOOS, FNMOC can make many of its higher-resolution operational model fields covering the waters surrounding CONUS available to the IOOS community.

32. IOOS Participants Served by the USGODAE LDM Server(Each institution served over 400,000 files/day) • SCOOP • Bedford Institute of Oceanography, Louisiana State University, University of Florida, University of Miami, University of North Carolina, Virginia Institute of Marine Sciences, University of Alabama at Huntsville • SEACOOS • North Carolina State University,University of North Carolina Charlotte, University of North Carolina Chapel Hill, University of Miami, Florida State University, University of South Florida (FTP/HTTP only) • GoMOOS • Bedford Institute of Oceanography, University of Maine • GCOOS • Texas A&M University • AOOS • AOOS LDM/Data Server, University of Alaska Fairbanks

33. IOOS Products from USGODAE Server Data AOOS imagery from USGODAE satellite data collection Wave models for GoMOOS, SCOOP, and SEACOOS forced by COAMPS from USGODAE LDM

34. FNMOC Data and Models Currently Provided for IOOS • Global Wave Watch 3 • 1 degree resolution, forecast to 144 hours • COAMPS Application Grids • Data for the three regions surrounding CONUS • Currently used to force several IOOS wave models • Global NOGAPS Application and Computational Grid • Computational grids provide the most accurate fields for forcing models in the coastal zone. • Daily High Resolution Global Ocean Analysis • 2161x1051 34 Levels (0-5000m) • High Resolution Global SST/Sea Ice Analysis • 3241x2441 6 hourly

35. FNMOC Models Available for IOOS(all pending FNMOC approval for public release) • High Resolution Regional Wave Watch 3 • Covers East Pacific, West Atlantic, and Central America (Gulf of Mexico) • Forced by FNMOC Regional COAMPS Models available on Server • 27 km resolution, forecast to 48 hours • COAMPS Computational Grids • Data served on the original computational grids for three regions surrounding CONUS • Provides the most accurate fields for forcing ocean models in the coastal zone

36. USGODAE COAMPS and WW3 Coverage for IOOS Regional COAMPS and WW3 are available for the three regions surrounding CONUS: EPAC, CENTAM, and WATL

37. FNMOC Regional Atmospheric and Ocean Models for IOOS EPAC Regional WW3 COAMPS Comp. GridLambert Projection for EPAC

38. Summary • Thanks to ~8 years of funding from ONR, the USGODAE Monterey Data Server has been developed into an invaluable resource for both the R&D community and the Navy operational community. • As planned from the outset, ONR funding for the Server will end at the beginning of FY08. • To ensure continued USGODAE Server availability we must enhance its role as an R&D system, supported under the projects that have made the Server a key part of their data serving infrastructure. • If no additional funding is provided, the USGODAE Server will be decommissioned in 2008.