progress on radar data assimilation at the ncep environmental modeling center l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Progress on Radar Data Assimilation at the NCEP Environmental Modeling Center PowerPoint Presentation
Download Presentation
Progress on Radar Data Assimilation at the NCEP Environmental Modeling Center

Loading in 2 Seconds...

play fullscreen
1 / 54

Progress on Radar Data Assimilation at the NCEP Environmental Modeling Center - PowerPoint PPT Presentation


  • 243 Views
  • Uploaded on

Progress on Radar Data Assimilation at the NCEP Environmental Modeling Center S. Lord, G. DiMego, D. Parrish, NSSL Staff With contributions by: J. Alpert, V. K. Kumar, R. Saffle, Q. Liu NCEP: “where America’s climate, weather, and ocean services begin” Overview Introductory remarks

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Progress on Radar Data Assimilation at the NCEP Environmental Modeling Center' - oshin


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
progress on radar data assimilation at the ncep environmental modeling center

Progress on Radar Data Assimilationat the NCEP Environmental Modeling Center

S. Lord, G. DiMego, D. Parrish,

NSSL Staff

With contributions by:

J. Alpert, V. K. Kumar, R. Saffle, Q. Liu

NCEP: “where America’s climate, weather, and ocean services begin”

overview
Overview
  • Introductory remarks
    • NEXRAD observations and Data Assimilation (DA)
  • History of NEXRAD data use in DA, including “precipitation assimilation” (Lin, Parrish)
  • CONUS impact study (Alpert)
  • Hurricane impact study (Liu)
  • Summary and outlook
nexrad wsr 88d radars
NEXRAD WSR-88D RADARS
  • 158 operational NEXRAD Doppler radar systems deployed throughout the United States
  • Provide warnings on dangerous weather and its location
    • Potentially useful for mesoscale data assimilation
  • Data resolution of Level 2 radar radial wind
    • 1/4 km radial resolution
    • 1 degree of azimuth
    • 16 vertical tilt angles
    • 200 km range
    • 8 minutes time resolution
  • Wind observation processing
    • VAD: cartesian (u,v) wind from radial wind processing
    • Level 3: dealiased radial wind at 4 lowest tilts
    • Level 2.5: on-site processing by NCEP “superob” algorithm
    • Level 2.0: raw radial wind
  • Data volume
    • 100 Billion (1011) potential reports/day for radar radial winds
    • Typically 2 Billion radial wind reports/day
    • 0.1 Tb/day computer storage
slide4

NEXRAD WSR-88D RADARS

  • A rich source of high resolution observations
    • Radial (Line of Sight) wind
    • Reflectivity  precipitation
slide6

Radar or Model Reflectivity?

WRF 24 hour 4.5 km forecast of 1 hour accumulated precipitation valid at

00Z April 21, 2004 and corresponding radar reflectivity

five order of magnitude increase in satellite data over next ten years

NPOESS Era Data Volume

Five Order of Magnitude Increase in Satellite Data Over Next Ten Years

Daily Satellite & Radar Observation Count

2005 210 M obs

2003-4 125 M obs

Level 2 radar data 2 B

2002 100 M obs

Count (Millions)

1990

2000

2010

2010-10%of obs

integration and testing of new observations
Integration and Testing of New Observations
  • Data Access (routine, real time) 3 months
  • Formatting and establishing operational data base 1 month
  • Extraction from data base 1 month
  • Analysis development (I) 6-18 months
  • Preliminary evaluation 2 months
  • Quality control 3 months
  • Analysis development (II) 6-18 months
  • Assimilation testing and forecast evaluation 1 month
  • Operational implementation 6 months
  • Maintain system* 1 person “till death do us part”

Total Effort: 29-53 person months per instrument

* Scientific improvements, monitoring and quality assurance

global data assimilation observations processing
Global Data AssimilationObservations Processing
  • Definitions
    • Received: The number of observations received operationally per day from providers (NESDIS, NASA, Japan, Europeans and others) and maintained by NCEP’s Central Operations. Counted observations are those which could potentially be assimilated operationally in NCEP’s data assimilation system. Observations from malfunctioning instruments are excluded.
    • Selected: Number of observations that is selected to be considered for use by the analysis (data numbers are reduced because the intelligent data selection identifies the best observations to use). Number excludes observations that cannot be used due to science deficiencies.
    • Assimilated: Number of observations that are actually used by the analysis (additional reduction occurs because of quality control procedures which remove data contaminated by clouds and those affected by surface emissivity problems, as well as other quality control decisions)
overview11
Overview
  • Introductory remarks
    • Observations and Data Assimilation (DA)
  • History of NEXRAD data use in DA, including “precipitation assimilation” (Parrish, Lin)
  • CONUS impact study (Alpert)
  • Hurricane impact study (Liu)
  • Summary and outlook
vad winds bill collins d parrish
VAD WindsBill Collins, D. Parrish
  • VAD winds reinstated 29 March 2000
    • First used by RUC (June 1997) and NAM-Eta (July 1997)
    • Withdrawn from operations (Jan. 1999) due to problems with observation quality
  • Error sources
    • Migrating birds (similar to errors in wind profilers)
      • Southerly wind component too strong (fall)
      • Northerly wind component too strong (spring)
      • Characteristic altitudes and temperatures
      • 5% of all winds
    • Winds of small magnitude
      • Source unknown
      • 8% of all winds
    • Outliers (large difference from model “guess”)
      • Source unknown
      • 7% of all winds
    • Random, normally distributed, errors
      • 2x magnitude expected from engineering error analysis
      • “Acceptably small”
    • Total 20% of observations have unacceptable errors
  • Quality control programs designed to filter erroneous observations
stage ii and stage iv multi sensor precipitation analyses ying lin
“Stage II and Stage IV”Multi-sensor Precipitation AnalysesYing Lin

http://www.emc.ncep.noaa.gov/mmb/

ylin/pcpanl/stage2/

Stage II

  • Generated at NCEP
  • Hourly radar and from hourly gauge
  • reports
  • First generated at ~35 minutes
  • after the top of the hour
  • 2nd and 3rd at T+6h and T+18h
  • No manual QC.

Stage IV

  • National mosaic; assembled
  • at NCEP
  • Input: hourly radar+gauge analyses by 12 CONUS River Forecast Centers (RFCs)
  • Manual QC by RFCs
  • Product available within an hour
  • of receiving any new data
assimilation of precipitation analyses 24 may 2001 ying lin
Assimilation of Precipitation Analyses24 May 2001 – Ying Lin
  • Motivation
    • Direct model precipitation contains large biases
      • Impacts all aspects of hydrological cycle
      • Soil moisture and surface latent heat flux particularly impacted
  • Real-time Stage II precipitation analyses are available
  • Assimilation technique
    • Precipitation nudging technique
      • Comparison of model and observed precipitation
      • Change model precipitation, latent heating and moisture in consistent way dependent on ratio Pmodel/Pobs
  • Expected improvements in NAM-Eta
    • Short-term (0-36 h) precipitation
    • Cycled soil moisture and surface fluxes
    • 2 meter temperature
  • No negative impact on other predicted fields
24 may 2001 cont

15 JUL 98

OPS EDAS:

15-DAY OBS PRECIP (1-15 JUL 98)

1-15 JUL

OPS EDAS:

15-DAY PRECIP

SOIL MOISTURE

(e)

(c)

(a)

15 JUL 98

TEST EDAS:

TEST EDAS:

1-HR STAGE IV PRECIP

1-15 JUL

15-DAY PRECIP

SOIL MOISTURE

(b)

(d)

(f)

24 May 2001 (cont)
  • Impacts as expected
    • Significantly improves the model's precipitation and soil moisture fields during data assimilation (e.g. North Americal Regional Reanalysis)
    • Often has a significant positive impact on the first 6 hours of the model's precipitation forecast
    • Occasional positive impact on precipitation forecasts 24h and beyond
    • Modestly positive impact on forecast skill scores
    • Not used in snow cases due to low observational bias
    • No negative impact is seen on the model forecast temperature, moisture and wind fields

Observed Precipitation

6-h Model Forecast

Without Assim.

With Assim.

8 july 2003 nam eta upgrade
8 July 2003 NAM-Eta Upgrade
  • Stage II and Stage IV hourly analyses merged precipitation assimilation
    • Analyses must arrive before data cutoff (H + 1:15)
    • Quality control added to merged product
  • Assimilation of Level 3 NEXRAD 88D radial wind data
    • Time and space averaged data (compression)
      • First 4 radar tilts (0.5, 1.5, 2.5, and 3.5 degrees) – the “NIDS” feed (1:4)
      • Hourly (~1:8)
      • Horizontal resolution of
        • 5 km radially (1:20)
        • 6 degrees azimuthally (1:6)
      • Overall compression: 1:3840
    • Quality control applied from VAD winds, including migrating bird contamination
  • “These radial wind runs show little positive or negative impact in the verification statistics, so it is certainly safe to include these winds treated this way in the 3DVAR”
  • First implementation: do no harm
overview17
Overview
  • Introductory remarks
    • Observations and Data Assimilation (DA)
  • History of NEXRAD data use in DA, including “precipitation assimilation” (Lin, Parrish)
  • CONUS impact study (Alpert)
  • Hurricane impact study (Liu)
  • Summary and outlook
conus impact study with level 2 5 winds
CONUS Impact Study with Level 2.5 Winds
  • Why compression?
    • Observations contain a high degree of redundancy
    • Communications cannot (until recently) handle the data volume for unprocessed observations
  • NCEP algorithm for winds processing (“Superobs”) installed on NEXRAD
    • Compression parameters can be modified without impacting code change management
    • Standard NCEP processing algorithm
slide19

Adaptable Parameters for the Level 2.5 Superob Product:

ParameterDefaultRange

Time Window 60 minutes [5-90 min]

Cell Range Size 5 km [1-10 km ]

Cell Azimuth Size 6 degrees [2-12 deg]

Maximum Range 100 km [60-230 km]

Minimum Number

of points required 50 [20-200]

Same as Level 3 products except for additional tilts and processing algorithm

slide20

Level 2.5

Level 3

Impact on Precipitation Forecasts

8-20 June 2004

(2 weeks)

24-h accumulated precipitation equitable threat score (upper) and bias (lower) from Eta 32-km 60-h forecasts from 8JUN2004 – 20JUN2004 for various thresholds in inches. The solid line (+) are the radial wind super-ob level 2.5 experiment and the dash is the Eta control (▲) with NIDS level 3.0 super-obs.

slide21

Level 3

Level 2.5

Impact of Level 2.5 Obs on Forecast Geop. Height

Improved RMS scores for height

Height RMS

Height Bias

Small improvements

in upper troposphere;

No degradation

slide22

Level 3

Level 2.5

Impact of Level 2.5 Obs on Forecast Winds

No degradation in Vector wind – slightly better near jet levels.

Wind RMS Vector Error

Small improvement

in upper troposphere

RMS vector wind errors against RAOBS over the CONUS from Eta 32-km 60-h forecasts, 8JUN2004 – 20JUN2004 (24 forecasts). The dash line is the radial wind super-ob Level 2.5 and the solid line is the Eta control with NIDS level 3.0 super-obs.

impact of level 2 5 obs on forecast precipitation
Impact of Level 2.5 Obs on Forecast Precipitation

24 h

Forecast

Control

Obs

Radar

Difference

Level

2.5

summary level 2 5 winds
Summary: Level 2.5 Winds
  • Winds received operationally from every radar site (April 2003)
  • Improved precip, height and wind scores (none from Level 3)
    • Data processing impacts forecast scores
  • Subjective evaluation shows positive impact
  • Quality control issues remain
    • Difficult to solve with processing at radar sites
    • Motivates transmission of full data set to NCEP and robust QC effort at central site
overview25
Overview
  • Introductory remarks
    • Observations and Data Assimilation (DA)
  • History of NEXRAD data use in DA, including “precipitation assimilation” (Lin, Parrish)
  • CONUS impact study (Alpert)
  • Hurricane impact study (Liu)
  • Summary and outlook
airborne doppler radar data analysis in hwrf model

Airborne Doppler RadarData Analysis in HWRF Model

Q. Liu, N. Surgi, S. Lord

W.-S. Wu, D. Parrish

S. Gopal andJ. Waldrop

(NOAA/NCEP/EMC)

John Gamache

(AOML/HRD)

background
Background
  • Initialization of hurricane vortex
    • GFDL model – “uncycled” system
    • “Spin-up” from axisymmetric model with forcing from observed parameters
      • Surface pressure
      • Maximum wind
      • Radii of max. wind, hurricane and T.S. winds
    • Increase of observations in hurricane environment
      • Dropsondes
      • Satellite winds
      • Scatterometer (QuikSCAT)
      • Sounding radiances (AMSU, AIRS, HIRS…)
      • Dopper radar (research)
    • $13 M program to add Doppler radar to GIV aircraft
  • Use of NEXRAD data in landfall situations
  • Hurricane is the only system uninitialized from observations at NCEP
cycled hurricane analysis summary
Cycled Hurricane AnalysisSummary
  • Capture short-term intensity changes
  • Account for storm motion
  • 6 hourly cycling
  • Use all available observations
  • When no observations, try to correct model intensity with axisymmetric correction
  • First time: use “bogus” vortex
3d var doppler radar data assimilation
3D-VAR Doppler Radar Data Assimilation
  • Data Quality Control

John Gamache (HRD)

  • Superobs

James Purser, David Parrish

Dx=10km, Dy=10km, Dz=250 m

Minimum number of data: 25

  • NCEP Gridpoint Statistical Interpolation (GSI) analysis
  • Hurricane Ivan 2004 September 7
    • Mature storm
future work
Future Work
  • Run more model forecast using the new analysis for weak storms
  • Study the impact of the airborne radar data on hurricane track and intensity forecasts, particularly for weak storms
  • Run HWRF complete cycling system during 2006 hurricane season
summary and outlook
Summary and Outlook
  • Use of NEXRAD wind data has proceeded in incremental steps over the past 9 years
    • Level 3  Level 2.5  Level 2
  • Use of reflectivity for
    • Precipitation analyses
    • Model initialization
  • Remaining issues
    • Quality control
    • Model initialization (increasing system complexity)
summary and outlook cont
Summary and Outlook (cont)
  • June 2005 - Implemented Level 2.5 (superobbed) data
  • June 2006 – Hierarchical radar data ingest for WRF-NAM
    • Level 2.0 (full resolution radial winds)
    • Level 2.5 (superobbed winds)
    • Level 3 (“NIDS” feed)
    • Precip. Assimilation impacts land surface only
  • Prototype data assimilation for hurricane initialization
    • Airborne Doppler radar
    • Coastal radar
    • 2004 cases as prototype
    • 2006 cases will be run as demonstration project
  • Integrating quality control codes into NCEP North American Model (NAM) run
    • Visiting scientist hired (on board at EMC 30 June, 2006)
  • Winds - expect steadily increasing impact
  • Reflectivity - long term project requiring advanced data assimilation techniques
doppler velocity data quality problems
Doppler Velocity Data Quality Problems

1. Noisy fields (due to small Nyquist velocity)

2. Irregular variations due to scan mode switches

3. Unsuccessful dealiasing

4. Contamination by migrating birds

5. Ground clutters due to anomalous propagation (AP)

6. Large velocities caused by moving vehicles & AP

7. Sea Clutter

EMC Working with NSSL and CIMMS to address all QC issues

level 2 radar data assimilation strategy
Level 2 Radar Data Assimilation Strategy
  • NAM assimilates Level 2 data – 20 June
  • QC codes are being ported from NSSL & CIMMS
    • Address all QC issues
  • Visiting Scientist on board at NCEP (30 June)
    • Former NSSL scientist
    • Prior experience with codes
    • Tuning and case studies
  • Assimilating reflectivity will be a long-term project, dependent on advanced data assimilation techniques
milestone and time table

Milestone and Time Table

FY06 Task 1.

Complete porting existing reflectivity QC C++ code executable together with the NCEP Fortran code into single compliable executable.

Shunxin Wang (QC C++ code developer) will work on this task as early as possible to meet NCEP's immediate needs.

FY06 Task 2.

Complete Phase 1 (by Sept, 2006)

Complete initial stages of Phase 2 (Dec, 2006)

Pengfei Zhang and Shunxin Wang will work together to design the NCEP/NSSL FORTRAN QC code. Li Wei working with Shunxin will combine various DA approaches towards an integrated Fortran DA for NCEP.

Code sets developed during the above two phases will be ported, tested, and refined on NCEP computers by Shun Liu (and others at NCEP).

FY07 and beyond …TBD

flowchart of real time migrating bird identification
Flowchart of Real-time Migrating Bird Identification

Raw data

Calculate QC parameters

Night?

no

yes

Bayes identification and calculate posterior probability

P(B|xi) >0.5

no

yes

Bird echo

Next QC step

current nssl radar data qc packages

Doppler Velocity Vr QC

Reflectivity Z QC

Current NSSL Radar Data QC packages

Input: Level II data

Input:

Fortran Data Structure

Pure clear air vol.echo removal

Ground Clutter Detection

Hardware test pattern vol.removal

Dealiasing

Speckle filter

Tilt-by-tilt Vr QC (bird, noisy Vr etc.)

Sun strobe filter

Pixel-by-pixel 3D Z QC (clear air, bird, insect, AP, sea clutter, interference etc.)

Output:

Fortran Data Structure

Fortran code

C++ code

phase i nssl ncep fortran qc package

Phase I:NSSL/NCEPFortran QC package

Reflectivity+ Doppler Velocity QC

Combined QC Filter from C++ code

Input:

Fortran Data Structure

Pure clear air vol.echo removal

Hardware test pattern removal

Ground Clutter Detection

Speckle filter

Dealiasing

Sun strobe filter

Rewrite in Fortran and integrate into Vr QC

Tilt-by-tiltVr QC

Output: QCed Z and Vr

Fortran Data Structure

Optimize the entire package

Fortran code

phase ii nssl ncep fortran qc package

Phase II: NSSL/NCEP Fortran QC package

Reflectivity+ Doppler Velocity QC

Input:

Fortran Data Structure

Combined QC Filter

New Dealiasing Algorithm (DA)

Ground Clutter Removal

Build test-case data base for comparing different DAs.

b. Develop optimum Fortran DA code set based on comparisons with research and operational DA approaches.

Tilt-by-tiltZ + Vr QC

Output: QCed Z and Vr

Fortran Data Structure

Fortran code

phase iii nssl ncep fortran qc package

Phase III: NSSL/NCEPFortran QC package

Reflectivity+ Doppler Velocity QC

Input:

Fortran Data Structure

Combined QC Filter

Ground Clutter Detection

Upgrade Vr QC to Z + Vr QC.

Improve tilt-by-tiltQC based on Bayes statistics.

Expand raw & “ground truth” data base optimize QC thresholds for radars at different regions (in terms of geographical and climatologic conditions).

New Dealiasing

Tilt-by-tilt Z + Vr QC

Output: QCed Z and Vr

Fortran Data Structure

Fortran code

strategies for developing unified fortran qc package

Strategies for Developing Unified Fortran QC package

  • Prioritize development phases based on anticipated QC ‘skill’ and difficulties for each phase.
  • Modularize individual components and routines (with on/off options) to facilitate CPU performance and optimization on NCEP computers.
  • Prioritize parameters in the QC package in order to simplify or enhance the package to fit the requirement and associated resources.
  • Develop and maintain QC archive important and/or challenging cases for comparing and testing. Includes collecting DA cases to assess different DA schemes, towards a optimum single DA code set.
  • Monitor and capture problematic cases, expand raw & “ground truth” data base, and optimize QC thresholds for each properly-classified category (such as VCP, diurnal, seasonal, regional, etc).
problems in operational dealiasing
Problems in Operational Dealiasing

Level-II raw data

Level-III NIDS

KBUF

KBUF

raw

dealiased

review three step d ealiasing for level ii v elocit ies
Review: Three-step Dealiasing for Level-II Velocities

Raw data

3-Step Noise Remove (BA88 )

Step 1

Select circles, Mod-VAD (u0,v0), Pre-dealiasing

VAD (u0,v0), Vertical check

Horizontal averaging & variance check

Step 2

Calculate Vr (refined reference)

Quality check (flag=0, 1 or 10)

Dealiasing with Vr (skip if flag =0 or 1)

Step 3

Dealiasing with continuity check

Output

Adopted

polarimetric koun vs wsr 88d ktlx
Polarimetric (KOUN) vs WSR-88D (KTLX)

KOUN

KTLX

Bird

Storm

rHV

Reflectivity

May 24 2003 0852UTC

slide54

Jung and

Zapotocny

JCSDA

Funded by

NPOESS IPO

Satellite data

~ 10-15% impact