slide1 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
NCL Introduction PowerPoint Presentation
Download Presentation
NCL Introduction

Loading in 2 Seconds...

play fullscreen
1 / 47

NCL Introduction - PowerPoint PPT Presentation


  • 1102 Views
  • Uploaded on

NCL Introduction Dennis Shea National Center for Atmospheric Research Workshop Overview Objective comfortable with NCL ; minimize learning curve workshop will not make you an expert access, process and visualize data Labs important: you must invest the time

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'NCL Introduction' - Gabriel


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

NCL

Introduction

Dennis Shea

National Center for

Atmospheric Research

workshop overview
Workshop Overview
  • Objective
    • comfortable with NCL; minimize learning curve
      • workshopwill notmake you an expert
    • access, process and visualize data

Labs important: you must invest the time

  • “osmosis” method of learning does not work
  • Schedule
    • day 1: Introduction, File I/O, [lunch] , Demo, Lab
    • day 2: Graphics [,Python], [lunch] , Web Tour, Lab
    • day 3: Functions and Data Analysis: Lab
    • day 4: Lab
ncl variable model is based on the netcdf variable model

Introduction

to

netCDF

NCL variable model is based on the netCDF variable model

netcdf files
netCDF files

self describing

  • (ideally)all info contained within the file
  • no external information needed to determine file contents

portable[machine independent]

  • Supported by many software tools / languages
    • NCL, IDL, Matlab, ferret, GrADS, F, C, C++,Java
    • command line file operators: NCO, CDO
      • perform various tasks very efficiently on files
        • http://nco.sourceforge.net
        • http://www.mpimet.mpg.de/fileadmin/software/cdo
    • ncview
      • quick visualization: COARDS
        • http://meteora.ucsd.edu/~pierce/ncview_home_page.html

Many modelling groups use netCDF [IPCC]

examining a netcdf file
Examining a netCDF file
  • ncdump file_name | less
    • dumps the entire contents of a file; prints every value
  • ncdump -h file_name | less
    • Dumps header information [most commonly used]
    • NCL equivalent: ncl_filedump file_name | less
  • ncdump -v U file_name | less
    • NCL equivalent: ncl_filedump –v U file_name| less
  • Note: ncdump is a Unidata utility
    • not a netCDF Operator (NCO / CDO)
    • not associated with NCL
  • ncview: visualize file contents [COARDS conven]
  • ncl_filedump file_name [ more general ]
    • netCDF3/4, GRIB-1,GRIB-2, HDF, HDF-EOS [HDF5]
parts of netcdf file
Parts of netCDF file

ncdump –h foo.nc (or ncl_filedump foo.nc)

DIMENSION SIZES

dimensions:

lat = 64

lon = 128

time = 12

VARIABLES:

Names , Types, Attributes, Coordinate Variables

variables:

floatlat(lat)

lat:long_name = "latitude"

lat:units = "degrees_north"

floatlon(lon)

lon:long_name = "longitude"

lon:units = "degrees_east"

inttime(time)

time:long_name = "time"

time:units = "Month of Year"

doubleT(time, lat, lon)

T:long_name = “Temperature”

T:units = “C"

T:missing_value = 1.e+20f

T:_FillValue = 1.e+20f

time=UNLIMITED (12 currently)

FILE ATTRIBUTES

global attributes:

title = “Temp: 1999”

source = “NCAR”

conventions = "None”

exercise:

ncl_filedump FOO.nc | less

netcdf ncl variable

4.35 4.39 0.27 -3.35 -6.90

4.36 4.66 3.77 -1.66 4.06

9.73 -5.84 0.89 8.46 10.39

17 3.68 5.08 0.14 -5.63

-0.63 -4.12 -2.51 1.76 -1.43

-4.29 0.07 5.85 0.87 8.65

x

netCDF/NCL variable
  • array[could be of length 1 (scalar)]
  • (may) have additional information

name: x

type: float [real]

shape: 2-dimensions

size: 7 (rows) x 5 (columns)

values:x(2,3) = 8.46 [row major]

long_name: “Temperature”

units: “C”

named dimensions: x(time,lat)

lat: (/ -60, -30 ,0, 30, 60 /)

time: (/2000,2001,2002,2003,2004,2005,2006 /)

Meta data

visual simple 2d netcdf variable
visual: simple 2D netCDF Variable

coordinate variables (rectilinear grid)

  • attributes @:
  • long_name
  • units

Latitude coordinate variable (1D, &)

Longitude coordinate variable (1D, &)

  • NCL is NOT LIMITEDto netCDF conforming variables
    • eg: 2D coordinate arrays (curvilinear coords)
netcdf ncl variable model

X

Scalar or Array

attributes

attributes

long_name

_FillValue

units

add_offset

scale_factor

etc.

long_name

_FillValue

units

add_offset

scale_factor

etc.

accessed via @

accessed via &

X

coord var

values

time

lev

lat

lon

etc.

coordinates

Scalar or Array

time

lev

lat

lon

etc.

netCDF [NCL] Variable model

f = addfile(“foo.nc”,”r”) ; grb/hdf

x = f->X

NCL reads the scalar/array, attributes, and coordinate variables as an object

detailed look netcdf variable ncl
Detailed Look netCDF Variable (NCL)

ncl<return> ; interactive mode

ncl0 >f =addfile("UV300.nc", "r") ; open file

ncl1 >u = f->U; import STRUCTURE

ncl2 >printVarSummary(u) ; overview of variable

Classic netCDF

Variable Model

Variable: u

Type: float

Total Size: 65536 bytes

16384 values

Number of Dimensions: 3

Dimensions and Sizes: [time | 2] x [lat | 64] x [lon | 128]

Coordinates:

time: [ 1 .. 7 ]

lat: [ -87.8638 .. 87.8638 ]

lon: [ 0 .. 357.185]

Number of Attributes: 5

_FillValue : 1e36

units : m/s

long_name : Zonal Wind Component

short_name : U

missing_value : 1e36

NCL

syntax/funcs

query

use

modify

add

any aspect of

data object

why know ncl netcdf variable model
Why Know NCL/netCDF Variable Model
  • variables imported (read) from nc, grib, hdf, hdf-eos
      • will have the same [consistent] structure
      • powerful programming aid
  • facilitates writing netCDF and HDF files
  • facilitates writing robust functions
      • functions can query, add, use meta data
  • some functions access/use the meta data
    • eg: gsn_csm graphic functions
      • aware of common netCDF conventions
      • will query and use for labels and mapping
integrated processing environment

pdf

Vis5D

NCL Overview

Integrated processing environment
  • freeware: supported, public domain
  • portable: linux/unix, windows (cygwin), MacOS
  • general purpose: unique capabilities
  • excellent graphics (2D, limited 3D)
ncl support
NCL Support
  • Documentation and Examples
    • http://www.ncl.ucar.edu/
      • numerous downloadable examples to get you going
    • downloadable reference manuals [pdf], FAQ

ncl-talk@ucar.edu

users must subscribe

http://www.ncl.ucar.edu/Support/ncl_talk.shtml

[haley@ucar.edu , shea@ucar.edu]

ncar command language
NCAR Command Language
  • Complete Programming Language
    • data types
    • variables
    • operators
    • expressions
    • conditional statements
    • loops
    • functions/procedures/graphics
  • Features
    • query / manipulate meta data
    • import data in a variety of formats
    • array syntax / operations
    • can use user fortran/C codes and commercial libraries
    • most functions/procedures ignore missing data
running ncl
Running NCL
  • Interactive Mode (Command line)
    • ncl[options][command-line-arguments]<return>

ncl> enter commands

ncl> quit <return>

    • can save interactive commands

ncl> record “file_name”

ncl> stop record

  • Batch Mode[ .ncl suffix is optional]
    • ncl[options][arguments]script.ncl
          • ncl< script.ncl[also acceptable]
    • ncl[options][arguments]script.ncl >&!out
    • ncl[options][arguments]script.ncl >&!out&
      • appending "&" means put in background
      • note: the >&!& are csh andtcshsyntax
ncl predefined options
NCL predefined options
  • ncl –hnxV [predfined options are preceded by dash]
  • may be used for interactive or batch mode
  • informational
    • ncl –h[display predefined options and usage and exit]
    • ncl –V[print the NCL version and exit]
  • action
    • ncl –x[echo statements as encountered (debug)]
    • ncl –n[don't enumerate dimensions of values in print()]
  • multiple predefined options
    • ncl –nx[ not ncl –n –x ]
sample batch script sample ncl
Sample Batch Script: sample.ncl

load “$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl“

load“$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl“

begin ; optional

f1 = addfile("TMP_58-97.nc", "r") ; open netCDF file

T = f1->Tmp ; T(time,lev,lat,lon) =>(480,17,73,144)

f2=addfile(“P_1958-1997.grb", "r") ; open GRIB file

P = f2->Pres ; P(time,lev,lat,lon)

f3=addfile(“Q_1958-1997.hdfeos", "r") ; open hdf –eos file

Q = f3->Specific_Humidity ; Q(time,lev,lat,lon)

pot = T* (1.+0.622*Q) *(1000/P)^0.286; potential temperature (array)

print("pot: min="+ min(pot) + " max="+ max(pot) )

wks =gsn_open_wks("ps", “sample") ; open a graphic workstation

gsn_define_colormap (wks,"gui_default") ; change from default

plot =gsn_csm_contour_map_polar (wks,T(0,5,:,:),False)

res = True;change default plot

res@cnFillOn = True; use colors

res@gsnSpreadColors = True; use entire color map

plot = gsn_csm_pres_hgt (wks,T(0,:,{50},:),res)

end ; only if begin is present

slide19

Language

Basics

outline ncl language basics
Outline: NCL Language Basics
  • special syntax characters
  • data types
  • Variables  netCDF/NCL variable model
  • attributes
  • _FillValue
  • named dimensions
  • coordinate variables
  • print and printVarSummary
  • shaping
  • subscripting
ncl syntax characters
NCL Syntax Characters
  • ;- comment [can appear anywhere]
  • @- reference/create attributes
  • ! - reference/create named dimension
  • & - reference/create coordinate variable
  • {…}- coordinate subscripting
  • $ - enclose strings when (im/ex)port variables via addfile
  • (/../)- array construct characters
  • : - array syntax
  • |- separator for named dimensions
  • \- continue character [statement to span multiple lines]
  • ::- syntax for external shared objects (eg, fortran/C)
  • ->- use to (im/ex)port variables via addfilefunction
data types
Numeric

double (64 bit)

float (32 bit)

long (32 or 64 bit)

integer (32 bit)

short (16 bit)

byte ( 8 bit, 0-255)

complex NOT supported

non-Numeric

string

character

graphic

file

logical

list

Data Types
simple variable assignment
Simple Variable Assignment

a_int = 1 , a_float = 2.0 [2e-5] a_double = 3.2d

a_string = "a” , a_logical = True [False]

  • array constructor characters (/…/)
    • a_integer = (/1,2,3/)
    • a_float = (/1.0, 2.0, 3.0/)
    • a_double = (/1., 2 , 3.2 /) *1d5
    • a_string = (/"abcd","e", "Hello, World”/)
    • a_logical = (/True, False,True/)
    • a_2darray = (/(/1,2,3/), (/4,5,6/), (/7,8,9/) /)
conversion between data types
Conversion between data types
  • NCL is a‘strongly typed’ language
  • coercion
    • implicit conversion of one type to another
  • automatic coercion when no info is lost
    • let i be integer and x be float or double
    • fortran: x=i and i=x
    • NCL: x=i and i=floattointeger(x)
  • many functions to perform conversions
attributes
Attributes
  • info about a variable or file [meta data]
    • attributes can be any data type except file or list
    • scalar, multi dimensional array (string, numeric)
  • assign/access with@character
    • T@units = “deg C”
    • T@wgts = (/ 0.25, 0.5, 0.25 /)
    • T@x2d = (/(/1,2,3/), (/4,5,6/), (/7,8,9/) /)
    • T@_FillValue = -999
    • title = x@long_name
  • attribute functions [isatt, getfilevaratts]
    • if (isatt(T,"units")) then .... end if
    • atts = getfilevaratts (fin, "T")
  • deletecan eliminate an attribute
    • delete(T@title)
fillvalue
_FillValue
  • Unidata and NCLreservedattribute
  • most NCL functions ignore_FillValue
  • Note: “missing_value” attribute: no special status to NCL
    • if “T” has “missing_value” attribute but no “_FillValue ”
      • NCL creates: T@_FillValue = T@missing_value
    • use built-in function “ismissing” to check for _FillValue
      • if (any(ismissing(T))) then … end if
      • NOTE: if (any(T.eq.T@_FillValue)) will not work
  • netCDF Operators[NCO]: _FillValue attribute
  • ncview: missing_value attribute
    • best to create netCDF files with both
  • NCL: best to not use zero as a _FillValue
    • OK except when contouring [random bug]
interactive variable 1
Interactive: Variable_1
  • http://www.ncl.ucar.edu/Training/Workshops/interactive.shtml

- demonstrate interactive mode

    • illustrate simple calculations and function use
    • first view of meta data [attributes]
arrays
Arrays
  • row major ….. like C/C++
    • left dimension varies slowest; right varies fastest
    • dimension numbering left to right [0,1,..]
  • indicies [subscripts] are zero based [0,N-1]

Consider T(:,:,:)

left dimension is 0 [ T!0 ]

middle dimension is 1 [ T!1 ]

right dimension is 2 [ T!2 ]

ncl netcdf dimensions
NCL (netCDF) Dimensions
  • may be “named”
    • provides alternative way to reference subscripts
    • recommendation: always name dimensions
    • use NCL syntax
  • assigned with ! character{let T(:,:,:)}
    • T!0 = "time" ; leftmost [slowest varying] dim
    • T!1 = "lat“
    • T!2 = "lon" ; rightmost [fastest varying] dim
  • dim names may be renamed, retrieved
    • T!1 = "LAT" … dName = T!2
  • delete can eliminate: delete (T!2)
  • Named dimensions used to reshape
ncl netcdf coordinate variables
NCL (netCDF) Coordinate Variables
  • data coordinates [ eg: time, level, lat, lon ]
    • strict netCDF definition
      • 1D numeric array
      • monotonically in[de]creasing numeric values
      • can only be assigned to a named dimension
      • 1D array must be the same size as dimension
      • should not have _FillValue attribute
    • coordinate values assigned via &character
      • T&lat = latitude
  • used in graphics, coordinate subscripting
  • coordinate functions
    • iscoord , isfilevarcoord
  • deletecan eliminate coordinate array
    • delete(T&time)
create and assign coordinate variables
Create and Assign Coordinate Variables
  • create 1D array
    • time = (/1980, 1981, 1982 /) ; integer
    • lon = ispan(0, 355, 5)*1.0 ; integer->float
  • assign dimension name [same as variable name]
    • time!0 = “time”
    • lon!0 = “lon”
  • assign values to named dimension
    • time&time = time
    • lon&lon = lon
  • let x be 2D: name dimensions
    • x!0 = “time” … x!1 = “lon”
  • assign coordinate variables to x
    • x&time = time … x&lon = lon
access change create delete meta data
Access/Change/Create/Delete Meta Data
  • @ attributes
    • u@long_name = "U"
    • lonName = u@long_name
  • !named dimensions
    • u!0 = "TIME"
    • tName = u!0
  • &coordinate variable
    • u&lat = (/ -90., -85, .... , 85., 90. /)
    • latitude = u&lat
  • $ substitute string
    • x = fin->$variable(n)$ …x = fin->$"T: p"$
netcdf ncl variable model33

X

Scalar or Array

attributes

attributes

long_name

_FillValue

units

add_offset

scale_factor

etc.

long_name

_FillValue

units

add_offset

scale_factor

etc.

accessed via @

accessed via &

X

coord var

values

time

lev

lat

lon

etc.

coordinates

Scalar or Array

time

lev

lat

lon

etc.

netCDF [NCL] Variable model

f = addfile(“foo.nc”, “r”) ; grb/hdf

x = f->X

NCL reads the scalar/array, attributes, and coordinate variables as an object

variable assignment
Variable Assignment
  • Variable-to-Variable assignment
    • consider y = xwhere x is previously defined
      • if y not defined:
        • y has same type/shape/values/meta data as x
      • if y predefined:
        • y must have same shape and type
        • or, x must be coerceible to the type of y
        • y attributes, dimension names and coordinate variables, if they exist, will be over written
  • Value-only assignment(no meta copy)
    • U multi-dimensional array with meta data
      • Uval = (/ U /) or Uval = (/f->U/)
      • the (/ ... /)operator pair strips meta data
interactive variable 2
Interactive: Variable_2
  • http://www.ncl.ucar.edu/Training/Workshops/interactive.shtml
    • named dimensions
    • create a variable data-object [structure] from scratch
    • standard and coordinate subscripting
    • reordering via named dimensions
variable reshaping reordering
Variable Reshaping/Reordering
  • functions may require data in specific order
      • map plot functions want array order T(...,lat,lon)
  • can and shouldbe done without loops
      • use NCL syntax or functions
      • very fast for variables in memory
  • how? … two approaches: let T(time,lat,lon)
    • named dimensions: t = T(lat|:, lon|:, time|:)
    • NCL functions:
      • ndtooned:t1D =ndtooned(T)
      • onedtond:t2D =onedtond(t1D, (/N,M/) )
variable subscripting 1 of 3
Variable Subscripting (1 of 3)

Standard Array Subscripting

  • ranges: start/end and [optional] stride
  • indices separated by :
  • omitting start/end index implies default begin/end

Consider T(time,lat,lon)

T  entire array [ don't use T(:,:,:) ]

T(0,:,::5)  1st time index, all lat, every 5th lon T(0,::-1,:50)  1st time index, reverse lat order, 1st 51 lon

T(:1,45,10:20)  1st 2 time indices, 46th index of lat, 10-20 indicies of lon

variable subscripting 2 of 3
Variable Subscripting (2 of 3)

Coordinate Variable Subscripting

  • only applies to coordinate variables
  • same rules apply for ranges, strides, defaults
  • use curly brackets {…}
  • standard and coordinate subs can be mixed

[if no reorder]

T(:,{-30:30},:)  all times/lon, lat -30° to +30° (inclusive)

T(0,{-20},{-180:35:3})  1st time, lat nearest - 20°, every 3rd lon between -180° and 35°

variable subscripting 3 of 3
Variable Subscripting (3 of 3)

Named Dimensions

  • only used for dimension reordering
  • indicated by |
  • dim names must be used for each subscript
  • named/coordinate subscripting can be mixed

Consider T(time,lat,lon)

t = T(lat|:, lon|:, time|:)  makes t(lat,lon,time)

t = T(time|:,{lon|90:120},{lat|-20:20})  all times,

90-120° lon, -20-20° lat

standard and coordinate subscripting
Standard and Coordinate Subscripting

Standard:

T(9:13,1:8)

Coordinate:

T({-10:20},{-170:-110})

Latitude coordinate variable (1D)

Combined:

T({-10:20}, 1:8)

Longitude coordinate variable (1D)

interactive variable 3
Interactive: Variable_3
  • http://www.ncl.ucar.edu/Training/Workshops/interactive.shtml
    • standard and coordinate subscripting
    • reorder via named dimensions
      • shape remains the same [same number of dimensions]
    • reshape via ndtooned and onedtond
      • number of dimensions changes
printing
“printing”
  • printVarSummary
    • gives gross overview of a variable
  • print
    • same info as printVarSummary
    • prints values
  • write_matrix
    • print to standard out or a file
    • format control of numerical output
    • can write to file also
printvarsummary
printVarSummary
  • Print out variable (data object) information
    • type
    • dimension information
    • coordinate information (if present)
    • attributes (if present)
  • printVarSummary(u)

Variable: u

Type: double

Total Size: 1179648 bytes

147456 values

Number of Dimensions: 4

Dimensions / Sizes: [time | 1] x [lev | 18] x [lat | 64] x [lon | 128]

Coordinates:

time: [4046..4046]

lev: [4.809 .. 992.5282]

lat: [-87.86379 .. 87.86379]

lon: [ 0. 0 .. 357.1875]

Number of Attributes: 2

long_name: zonal wind component

units: m/s

print 1 of 3
print (1 of 3)
  • Prints out all variable information including
    • meta data, values
    • T(lat,lon): print (T)

Variable: T

Type: float

Total Size: 32768 bytes

8192 values

Number of Dimensions: 2

Dimensions / Sizes: [lat | 64] x [lon | 128]

Coordinates:

lat: [-87.86379 .. 87.86379]

lon: [ 0. 0 .. 357.1875]

Number of Attributes: 2

long_name: Temperature

units: C

(0,0) -31.7

(0,1) -31.4

(0,2) -32.3

(0,3) -33.4

(0,4) -31.3 etc. [entire T array will be printed]

print 2 of 3
print (2 of 3)
  • can be used to print asubsetof array
    • meta data, values
    • T(lat,lon): print( T(:,103) ) orprint( T(:,{110}) )
  • Variable: T (subsection)
  • Type: float
  • Total Size: 256 bytes
  • 64 values
  • Number of Dimensions: 1
  • Dimensions / Sizes: [lat | 64]
  • Coordinates:
  • lat: [-87.86379 .. 87.86379]
  • Number of Attributes: 3
  • long_name: Temperature
  • units: C
  • lon: 109.6875[ added ]
  • (0) -40.7
  • (1) -33.0
  • -25.1
  • -20.0
  • (4) -15.3 etc.
print 3 of 3
print (3 of 3)
  • printwith embedded strings
    • no meta data
    • print ( "min(T)="+min(T)+" max(T)="+max(T) )

(0) min(T)=-53.8125 max(T)=25.9736

  • sprintf andsprinti provide formatting
    • often used in graphics
    • print ( "min(T) = "+ sprintf("%5.2f", min(T)) )

(0) min(T) = -53.81

  • sprintican left fill with zeros (ex: let n=3)
    • fnam = "h" + sprinti ("%0.5i", n) + ".nc"
    • print("file name = "+fnam)

(0) file name = h00003.nc

write matrix x fmt opt
write_matrix(x[*][*], fmt, opt)
  • pretty-print 2D array (table) to standard out
    • integer, float, double
    • user format control (fmt)
    • T(N,M), N=7, M=5: write_matrix (T, “5f7.2”, False)

4.35 4.39 0.27 -3.35 -6.90

4.36 4.66 3.77 -1.66 4.06

9.73 -5.84 0.89 8.46 10.39

4.91 4.59 -3.09 7.55 4.56 17 3.68 5.08 0.14 -5.63

-0.63 -4.12 -2.51 1.76 -1.43

-4.29 0.07 5.85 0.87 8.65

  • can also create an ASCII file

opt = True

opt@fout = “foo.ascii” ; file name

write_matrix (T, “5f7.2”, opt)