Statistics Function Implementation

1 / 77

# Statistics Function Implementation - PowerPoint PPT Presentation

Statistics Function Implementation. Traditional Programming Approach. minimum.pro. function minimum, x n = n_elements(x) answer = x(0) for i=1 L , n-1 do begin if( answer gt x(i) ) then \$ answer = x(i) endfor return, answer end. sum.pro. function sum, x

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## Statistics Function Implementation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Statistics Function Implementation

minimum.pro

function minimum, x

n = n_elements(x)

for i=1L, n-1 do begin

endfor

end

sum.pro

function sum, x

n = n_elements( x )

for i = 0, n-1 do begin

endfor

end

Overflow Problems
• What if the array is an integer array?
• Sums will have a maximum limit based on the number of bits specified by the integer type.
Corrected sum.pro

function sum, x

n = n_elements( x )

for i =0L, n-1 do begin

endfor

end

mean.pro

function mean, x

n = n_elements(x)

end

Integer Problems

IDL> b=9/5

IDL> print,b

1

Integer Problems

IDL> b=9/double(5)

IDL> print,b

1.8000000

Corrected mean.pro

function mean, x

n = n_elements(x)

end

sum_squares.pro

function sum_squares, x

squares = x^2.0

end

variance.pro

function variance, x

n = double(n_elements(x))

end

sd.pro

function sd, x

end

### Shell Programming

UNIX Tip of the Day
• Directory maneuvering commands pushd, popd, and, dirs

% cd /usr/tmp

% pwd

/usr/tmp

% pushd ~rvrpci

% pwd

/cis/staff/rvrpci

UNIX Tip of the Day

% dirs

/cis/staff/rvrpci /usr/tmp

% pushd

% pwd

/usr/tmp

% dirs

/usr/tmp /cis/staff/rvrpci

% pushd /usr/local/bin

UNIX Tip of the Day

% dirs

/usr/local/bin /usr/tmp

/cis/staff/rvrpci

% pwd

/usr/local/bin

% pushd

% dirs

/usr/tmp /usr/local/bin

/cis/staff/rvrpci

UNIX Tip of the Day

% dirs

/usr/tmp /usr/local/bin

/cis/staff/rvrpci

% pwd

/usr/tmp

% pushd +2

% pwd

/cis/staff/rvrpci

UNIX Tip of the Day

% dirs

/cis/staff/rvrpci /usr/tmp

/usr/local/bin

% popd

% dirs

/usr/tmp /usr/local/bin

IMPORTANT UNIX Concepts
• Environment and Shell Variables
• These allow you to customize your UNIX environment
• They are different in terms of their SCOPE
• SCOPE determines the visibility of a variable
Other IMPORTANT UNIX Concepts
• Environment Variable
• Examples are TERM and DISPLAY
• Set a particular variable to a value by using the setenv command
• You can print the value of a particular variable or all the environment variable using the printenv command
% Environment Variables %
• Examples
• To set environment variables

% setenv TERM vt100

% setenv DOG Goofy

• print out the terminal type

% printenv TERM

vt100

• print out all environment variables

% printenv

Shell Variables
• Shell variables are similar to Environment variables except they have a limited scope, i.e., they exist only in the shell which they are defined.
• Environment variables on the other hand, exist in all its children shells
• To illustrate this concept, let us look at the following example
Environment vs. Shell Variables

% set prompt = "Parent Shell > "

Parent Shell > setenv DOG Goofy

Parent Shell > set mouse=Mickey

Parent Shell > printenv DOG

Goofy

Parent Shell > echo \$mouse

Mickey

Parent Shell > xterm &

(YOU SHOULD NOW HAVE A NEW xterm WINDOW)

THIS IS KNOWN AS

“SPAWNING A NEW (OR CHILD) PROCESS”

Environment vs. Shell Variables

(IN THE NEW xterm WINDOW, DO THE FOLLOWING)

% set prompt = "Child Shell > "

Child Shell > printenv DOG

Goofy

Child Shell > echo \$mouse

mouse: Undefined variable.

Environment vs. Shell Variables

Child Shell > setenv DOG Pluto

Child Shell > set mouse=Minnie

Child Shell > printenv DOG

Pluto

Child Shell > echo \$mouse

Minnie

Child Shell > exit

(THE xterm WINDOW SHOULD NOW GO AWAY - THIS PROCESS HAS NOW BEEN KILLED)

Environment vs. Shell Variables

Parent Shell >

Parent Shell > printenv DOG

Goofy

Parent Shell > echo \$mouse

Mickey

Parent Shell >

Environment & Shell Variables
• Why is this important?
• UNIX uses Environment and Shell Variables control a number of processes
• Variables used for UNIX Scripts
• They are typically defined and initialized in your .login and .cshrc files
Useful Shell Variables

filec #Allows file completion

path #List of command

directories

cdpath #List of candidate

directories to cd into

history #Number of commands to

remember

What is shell programming?
• Shell programming
• automate a set of UNIX commands.
• Just like any programming language
• “wrappers”
• black box a customized collection of UNIX commands.
• Example of shell programs

.cshrc

set path=(\$HOME/bin /usr/local/bin \

/usr/ucb /usr/sbin /bin /usr/bin \

/usr/bin/X11 .)

stty dec new

tset -I -Q

set mail=/usr/spool/mail/\$USER

set editmode = emacs

biff n

date

.cshrcfile

if (\$?prompt) then

set notify

set history = 100

set savehist = 100

alias pd pushd

alias pop popd

alias vt100 "set term = vt100"

endif

When these files are executed?

.cshrc

• is automatically executed when you start a new shell

• only gets executed once when you first login

Can be re-executed by giving the source command

% source .cshrc

Other useful .login and .cshrc entries

set filec

set cdpath=(~ ~rvrpci/pub ~/mythesis)

Other common entries

set path=( \$path /usr/local/bin)

set path=(/usr/local/bin \$path)

User defined shell program
• Determine name of command
• Determine input, output, and option arguments
• Determine UNIX commands to execute
• Establish error trapping
• Make shell program executable
A simple shell program
• ddcommand to swap bytes

% dd if=input.dat of=output.dat bs=2 conv=swab

• Very difficult to remember
• Very little utility to non-UNIX geeks (normal people)
We would rather see...

% swap_bytes input.dat output.dat

Special Shell Variables Set

% swap_bytes input.dat output.dat

\$0

\$1

\$2

command

\$argv[1]

\$argv[2]

Another Special Shell Variables

% swap_bytes input.dat output.dat

\$#argv

Indicates how many arguments are present

In this case, 2

shell programswap_bytes

#!/bin/csh -f

dd if=\$1 of=\$2 bs=2 conv=swab

Making swap_bytesshell script executable

% ls -l swap_bytes

-rw------- ... swap_bytes

% chmod u+x swap_bytes

% ls -l swap_bytes

-rwx------ ... swap_bytes

To run swap_bytes
• swap_bytesbecomes just another unix command!

% swap_bytes input.dat output.dat

Limitation of swap_bytes
• No error trapping
• Should give usage when typing command

% swap_bytes

usage: swap_bytes input_file output_file

Improvement to swap_bytes

#!/bin/csh -f

if ( \$#argv != 2 ) then

echo "usage: \$0 input_file output_file"

exit 1

endif

dd if=\$1 of=\$2 bs=2 conv=swab

• By convention

exit 0

Indicates successful command completion

exit 1(or non-zero)

Indicates some error condition

Informational message from swap_bytes
• UNIX style informational message

% swap_bytes

usage: swap_bytes input_file output_file

Interactive swap_bytes
• If you want a “friendlier” shell program
• Have it query the user for the inputs
• Another special shell variable can be used

\$<

Interactive swap_bytes

#!/bin/csh -f

if ( \$#argv != 2 ) then

echo -n "Please enter the input file> "

set input=\$<

echo -n "Please enter the output file> "

set output=\$<

endif

dd if=\$input of=\$output bs=2 conv=swab

Interactive swap_bytesexample
• User simply types the command

% swap_bytes

Please enter the input file> input.dat

Please enter the output file> output.dat

### UNIX Quotes

A note about quotes in UNIX

% set a=ls

% echo a

% echo \$a

% set b=“\$a”

% echo \$b

% set b=‘\$a’

% echo \$b

% set b=`\$a`

% echo \$b

• Shell variables can also double up as arrays
• Using the previous example,

% echo \$b

% echo \$b[1]

% echo \$#b

% echo \$b[\$#b]

A more complex shell program
• In pbmplus utilities,

rawtopgm conversion exists

pgmtorawconversion does not

• A version of pgmtoraw in a programming language like C
• Time consuming
• Program will likely be used infrequently
• Solution: shell program
pgmtoraw shell script design
• Define input and output files
• Figure out dimensions of input image
• Determine number of bytes for input image
• Determine number of bytes for header
• Need to strip out the header bytes
• Write out headerless image data
Define input and output filespgmtoraw

#!/bin/csh -f

set command_name=\$0

set number_args=\$#argv

if( \$number_args != 1 ) then

echo “usage:\$command_name input_file_name”

exit 1

endif

.

.

.

Dimensions of input image ( pnmfile)

% more test_data.pgm

P2

3 3

255

1 2 3

4 5 6

7 8 9

% pnmfile test_data.pgm

test_data.pgm: PGM plain, 3 by 3 maxval 255

pgmtoraw(continued)

set input_file=\$1

set pnm_info = `pnmfile \$input_file`

set image_type = \$pnm_info[2]

set data_type = \$pnm_info[3]

set width = \$pnm_info[4]

set height = \$pnm_info[6]

set maxval = \$pnm_info[8]

set pixel_bytes = 1

@ image_bytes = \$width * \$height

pgmtoraw (continued)

set file_info=`wc -c \$input_file`

set bytes_in_file = \$file_info[1]

@ header = \$bytes_in_file - \$image_bytes

Resulting pgmtoraw utility
• Uses existing routines to obtain information

pnmfile

wc

dd

• Functional tool written in 20 command lines
Current Limitations of Resulting pgmtoraw utility
• No check between “ASCII” vs. “RAW” pgm

if( data_type == ‘plain,’) ...

• No provisions for multibyte per pixel case
• Use pnmfile results to check for above cases
• endian case needs to be addressed for multibyte case
• Above conditions can be addressed by suite of UNIX commands
Shell Scripts Wrappers and IDL
• Another utility formerly missing in pbmplus

jpegtopnm or pnmtojpeg

• IDL has jpeg read/write capability
• Create a “wrapper” that makes an idl program pbmplus-like
pnmtojpeg.pro

pro pnmtojpeg, input_file, output_file

write_jpeg, output_file, image

end

Usage of pnmtojpeg.pro in IDL

IDL> pnmtojpeg,‘image.pnm’,’image.jpg’

Usage of pnmtojpeg.pro in IDL

IDL> pnmtojpeg,‘image.pnm’,’image.jpg’

• For IDL to automatically find pnmtojpeg.pro
• It must be in the current working directory
• Directory containing pnmtojpeg.promust be defined in the ENVIRONMENT VARIABLE
• IDL_PATH
IDL_PATHenvironment variable

setenv IDL_DIR /cis/common/rsi/idl_5

setenv IDL_PATH \+\$IDL_DIR/lib:\+\$IDL_DIR/examples:

\+/dirs/common/rsi/idl_5:\+/dirs/common/lib/idl:\+~/lib/idl

pnmtojpeg.csh

#!/bin/csh -f

echo pnmtojpeg “,” “’”\$1”’” “,” “’”\$2”’” | idl

Usage of pnmtojpeg.csh

% pnmtojpeg.csh image.pnm image.jpg

Limitation of pnmtojpeg.csh
• Does not conform to pbmplus piping, i.e.,

% tifftopnm file.tif | pnmscale 2.0 > new_file.pnm

• No error trapping
Usage cases of pnmtojpeg(no command line arguments)

% tifftopnm file.tif | pnmscale 2.0 | pnmtojpeg > new_file.jpg

Usage cases of pnmtojpeg(1 command line argument)

% pnmtojpeg image.pnm > image.jpg

Yet another wrapperpnmtojpeg

#!/bin/csh -f

onintr stop

# \$0 is the command name

# \$#argv is the number of arguments

# \$\$ is the process id

Code for no argument case

if(\$#argv == 0) then

set input_file = /usr/tmp/\$0_input_\$\$

set output_file = /usr/tmp/\$0_output_\$\$

cat > \$input_file

pnmtojpeg.csh \$input_file \$output_file

cat \$output_file

.

.

.

Code for 1 argument case

else if(\$#argv ==1) then

set input_file = \$1

set output_file = /usr/tmp/\$0_output_\$\$

pnmtojpeg.csh \$input_file \$output_file

cat \$output_file

.

.

.

Code for 2 argument case

else

set input_file = \$1

set output_file = \$2

pnmtojpeg.csh \$input_file \$output_file

endif

#clean up when finished

stop:

rm -f /usr/tmp/\$0_input_\$\$

rm -f /usr/tmp/\$0_output_\$\$

pnmtojpegsummary
• Produced a “new” pbmplus utility
• Used UNIX shell scripting
• Argument handling
• Scratch space /usr/tmp
• Process id handling
• Clean up
• Integrated IDL program and commands
• 21 lines of UNIX commands
Summary
• The “dot” files
• Basics of Shell Scripting
• Special Shell Variables
• Seamless integration of UNIX to other utilities (IDL)