Satellite Data Provided by the NASA Ocean Biology Processing Group (OBPG) - PowerPoint PPT Presentation

Satellite Data Provided by the NASA Ocean Biology Processing Group(OBPG)

Deep-Water Chlorophyll Images

• common-bin 12-day composite, Winter 2002

• 0.01-1 mg/m^3

SeaWiFS R5

MODIS/Aqua R1

OBPG Satellite Data

- MODIS Aqua (2002-present)

- MODIS Terra (2000-present)

- SeaWiFS (1997-present)

- OCTS (1996-1997)

- CZCS (1978-1986)

HDF File Format

• Hierarchical Data Format (HDF) is a data file format
• OBPG data files stored as HDF (L1A, L1B, L2, L3)
• HDF can be thought of as a “container” to store a variety of data types and meta data in a single file

Example HDF file

HDF-EOS

• OBPG files stored in HDF-EOS format:
• specialized form of HDF
• created by NASA in 1993
• standard format for all EOS data
• SeaDAS is compatible with OBPG HDF-EOS

MODIS Data Levels

Level 0

Level 1A

Level 1B

Level 2

Level 3 binned

Level 3 mapped

raw digital counts

native binary format

Level 1A

raw digital counts

HDF formatted

• ATT & EPH
• spacecraft attitude
• spacecraft position

• GEO
• geolocation
• radiant path geometry

• Level 1B
• calibrated reflectances
• converted telemetry

• Ancillary data
• wind speed
• surface pressure
• total ozone
• Reynolds SST

• Level 2
• geolocated geophysical products for each pixel

raw digital counts

native binary format

Level 1A

raw digital counts

HDF formatted

• ATT & EPH
• spacecraft attitude
• spacecraft position

• GEO
• geolocation
• radiant path geometry

• Level 1A Subset
• reduced to standard ocean bands only

• Level 1B
• calibrated reflectances
• converted telemetry

• Ancillary data
• wind speed
• surface pressure
• total ozone
• Reynolds SST

• Level 2
• geolocated geophysical products for each pixel

Level 1A (HDF)

• raw radiance counts
• Onboard calibration data
• navigation data
• spacecraft telemetry
• instrument telemetry

• A2006027191500.L1A_LAC

Level 1B (HDF)

• calibrated, geolocated at-aperture radiances

i.e. Top of Atmosphere (TOA) radiances

• quality flags
• error estimates
• calibration data

A2006027191500.L1B_LAC

A2006027191500.L1B_HKM

A2006027191500.L1B_QKM

Level 2 (HDF)

• geophysical values for each pixel
• products include chlorophyll-a, SST, nLw’s, K490, flags
• geolocation data

A2006027191500.L2_LAC

A2006027191500.L2_LAC_SST

A2006027191500.L2_LAC_SST4

Chlorophyll-a

SST

Level 2 Flags

• Ocean Color (day only)
• Normalized water-leaving radiances - nLw()
• Chlorophyll - Ca
• Diffuse attenuation - Kd(490)
• Aerosol type and concentration
• Optical thickness - a
• Ångström exponent
• Atmospheric epsilon
• Processing flags
• Cloud, land, glint, atmfail, atmwarn, chlfail, chlwarn, etc.
• Ocean Temperature
• Long-wave SST (11-12 m) (day and night)
• Short-wave SST (3.9 - 4.0 m) (night only)
• SST quality level (0-4)

• Ocean Color
• Alternate Ca and Kd algorithms
• Chlorophyll fluorescence, FLH
• Particulate inorganic carbon, Calcite
• Inherent optical properties (various bio-optical models)
• absorption (total, phaeophytin, dissolved matter)
• backscatter (total, particulate)
• Photosynthetically active radiation, iPAR, PAR (coming)
• Euphotic depth (Zeu, Zsd)
• Subsurface PAR at 1st optical depth, Kd(PAR)
• Intermediate atmospheric correction products
• Ocean Temperature
• Brightness temperatures

L3 Binning

Level-2

After Level-2…

Level 3 Binned data products (HDF)

• data composites
• binned Level 2 data (spatially and temporally)
• arithmetic means with statistics for each bin
• daily, 8-day, monthly, yearly periods
• 0.5, 1, 2, 4, 9, 36 km2 resolutions
• stored as one file or a main file with subordinates
• useful for scientific research

A20071822007212.L3b_MO.main

Level 3 SMI (mapped) (HDF)

• Mapped L3 binned product
• Data is scaled to 16-bit values
• Equidistant cylindrical projection

A20071822007212.L3m_MO_CHLO_4

Ocean coverage over time for binned files

Ancillary data products

(HDF and other formats)

• NRT data and refined ancillary data
• used in various stages of processing:
• Geolocation (attitude and ephemeris)
• Calibration (coefficient look-up tables)
• atmospheric correction (MET/OZONE/SST)

Operational MODIS Ocean Band Suite

VIS/NIR

Ocean Color

Thermal

SST

Operational MODIS-Aqua Data Flow

NASA EDOS

System

Software process

April 1, 2014

Hardware system

Database table

NOAA Realtime

System

Ancillary input data

User

Community

MET

Full-resolution day- and nighttime via SEN (~60 GB per day)

MET, Ozone, and OISST data are dynamically selected for each L1A granule

Ozone

MODISA L2

table

Level-0

Ingest

L1B-L2

(MSl12)

Ocean Color

Web Server

OISST

Archive - Distrib

Server

Sensor

CAL

L1A-L1B

(MOD_PR02)

Ingest queue

table

Sensor

attribs

Browser

CGI /

httpd

MODISA L3-bin

table

L3BIN

(l2bin)

Atm

corr

Archive - Distrib

Server

ATT/EPH

Ingest

process

Archive - Distrib

Server

MySQL

DB

Geo-Location

(MOD_PR03)

MODISA atteph

table

MODISA L3-map

table

Product meta data are populated from production DB

L3MAP

(smigen)

Archive - Distrib

Server

L0

Ingest

process

Archive - Distrib

Server

Archive - Distrib

Server

L0-L1A

(MOD_PR01)

MODISA L0

table

MODISA L1

table

ODPS Data Processing System

Current Components

Distribution Servers (ftp)

6 storage nodes

7.2 TB

Distribution Servers (web)1 large server – 3 TB

58 storage nodes

69.6 TB

Ingest Servers

2 SeaSpace groundstations

5 storage nodes

6 TB

Cal/Val & QC Systems

Processing Cluster 34 processing nodes 1.5 TB

Extreme Networks

Black Diamond 6816

Gigabit Ethernet switch

Mission Operations Systems

Database Server

1 large server

876 GB

Development Servers

1 processing node

2 storage nodes

2.4 TB

Testing Cluster

13 test nodes

2 TB

Backup Servers

1 large server

876 GB+2 TB tape

5 storage nodes

6 TB

User Desktops

Network Support Systems

gigabit ethernet

• database controlled, distributed processing and storage system
• many Linux-based, processing nodes (34 dual 3GHz Xeon)
• 200X processing for MODIS/Aqua ocean color
• 4000X processing through-put for SeaWiFS ocean color
• 9-month OCTS mission can be reprocessed over lunch
• all data online for direct access (100TB RAID-5 array)
• capacity for global mission testing prior to reprocessing, allowing opportunity for community participation in reprocessing decisions
• direct and near real-time user access to data archives

database server

processing node

storage node

Ocean Biology Data Processing System - Processing Node

Supermicro 6024 rack mount server

2 Intel Xeon CPUs 3.06 Ghz

8 1 GB DDR2 RAM modules

2 Intel Gigabit Ethernet network interfaces

1 Seagate 80 GB EIDE disk (system)

5 Maxtor 73.4 GB Atlas 10K IV Ultra320

SCSI disks (processing)

1 CD-ROM (low profile)

1 Supermicro X5DP8-G2 motherboard with

built-in Ultra320 SCSI controller

1 Supermicro 6024H 2U rack mount server case

with dual power supplies

Operating system: Linux

Ocean Biology Data Processing System - Storage Node

NAS 1.2 TB rack mount server

1 Intel Pentium-4 CPU 3.06 Ghz

2 512 MB DDR2 RAM modules

1 Gigabit Ethernet network interface

1 Seagate 80 GB EIDE disk (system)

8 Maxtor 250 GB MaxLine Plus II EIDE disks

(RAID 5, 6 data + 2 hot spares)

1 CD-ROM (low profile)

1 Asus P4C800-E Motherboard

1 3ware Escalade RAID controller for 8 drives

1 Comix Computer RM-3U9SCA (IDE) ATX

rackmount chassis 3 U, 9 hot swappable

drive bays

Operating system: Linux

Ocean Biology Data Processing System - Large Server

SunFire V880 rack mounted

4-8 UltraSPARC-III+ processors

8-16 GB RAM

6-12 73 GB 10K RPM FibreChannel disks

2 Gigabit Ethernet nework interfaces

1-5 Fast Ethernet network interfaces

1 External FibreChannel adapter

Operating system: Solaris

Ocean Biology Data Processing System - Network Switch

Extreme Networks Black Diamond 6816

768 Gbps total switching capacity

Route/filter/forward 192 million pps

160 GigE ports in current configuration

All modules are hot swappable

4 MSM management modules

2 G8Ti 8 port Gigabit Ethernet modules

6 G24T3 24 port Gigabit Ethernet modules

4 Hot-swappable power supplies