statistics function implementation
Download
Skip this Video
Download Presentation
Statistics Function Implementation

Loading in 2 Seconds...

play fullscreen
1 / 77

Statistics Function Implementation - PowerPoint PPT Presentation


  • 240 Views
  • Uploaded on

Statistics Function Implementation. Traditional Programming Approach. minimum.pro. function minimum, x n = n_elements(x) answer = x(0) for i=1 L , n-1 do begin if( answer gt x(i) ) then $ answer = x(i) endfor return, answer end. sum.pro. function sum, x

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Statistics Function Implementation' - EllenMixel


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
statistics function implementation

Statistics Function Implementation

Traditional Programming Approach

minimum pro
minimum.pro

function minimum, x

n = n_elements(x)

answer = x(0)

for i=1L, n-1 do begin

if( answergtx(i) ) then $

answer = x(i)

endfor

return, answer

end

sum pro
sum.pro

function sum, x

n = n_elements( x )

answer = 0

for i = 0, n-1 do begin

answer = answer + x(i)

endfor

return, answer

end

overflow problems
Overflow Problems
  • What if the array is an integer array?
  • Sums will have a maximum limit based on the number of bits specified by the integer type.
corrected sum pro
Corrected sum.pro

function sum, x

n = n_elements( x )

answer = 0.0D

for i =0L, n-1 do begin

answer = answer + x(i)

endfor

return, answer

end

mean pro
mean.pro

function mean, x

n = n_elements(x)

answer = sum(x) / n

return, answer

end

integer problems
Integer Problems

IDL> b=9/5

IDL> print,b

1

integer problems8
Integer Problems

IDL> b=9/double(5)

IDL> print,b

1.8000000

corrected mean pro
Corrected mean.pro

function mean, x

n = n_elements(x)

answer = sum(x) /double(n)

return, answer

end

sum squares pro
sum_squares.pro

function sum_squares, x

squares = x^2.0

answer= sum( squares )

return, answer

end

variance pro
variance.pro

function variance, x

n = double(n_elements(x))

answer=(sum_squares(x)-(sum(x)^2.0/n))/ (n-1)

return, answer

end

sd pro
sd.pro

function sd, x

answer = sqrt(variance(x))

return, answer

end

unix tip of the day
UNIX Tip of the Day
  • Directory maneuvering commands pushd, popd, and, dirs

% cd /usr/tmp

% pwd

/usr/tmp

% pushd ~rvrpci

% pwd

/cis/staff/rvrpci

unix tip of the day15
UNIX Tip of the Day

% dirs

/cis/staff/rvrpci /usr/tmp

% pushd

% pwd

/usr/tmp

% dirs

/usr/tmp /cis/staff/rvrpci

% pushd /usr/local/bin

unix tip of the day16
UNIX Tip of the Day

% dirs

/usr/local/bin /usr/tmp

/cis/staff/rvrpci

% pwd

/usr/local/bin

% pushd

% dirs

/usr/tmp /usr/local/bin

/cis/staff/rvrpci

unix tip of the day17
UNIX Tip of the Day

% dirs

/usr/tmp /usr/local/bin

/cis/staff/rvrpci

% pwd

/usr/tmp

% pushd +2

% pwd

/cis/staff/rvrpci

unix tip of the day18
UNIX Tip of the Day

% dirs

/cis/staff/rvrpci /usr/tmp

/usr/local/bin

% popd

% dirs

/usr/tmp /usr/local/bin

important unix concepts
IMPORTANT UNIX Concepts
  • Environment and Shell Variables
    • These allow you to customize your UNIX environment
    • They are different in terms of their SCOPE
      • SCOPE determines the visibility of a variable
other important unix concepts
Other IMPORTANT UNIX Concepts
  • Environment Variable
    • Examples are TERM and DISPLAY
    • Set a particular variable to a value by using the setenv command
    • You can print the value of a particular variable or all the environment variable using the printenv command
environment variables
% Environment Variables %
  • Examples
    • To set environment variables

% setenv TERM vt100

% setenv DOG Goofy

    • print out the terminal type

% printenv TERM

vt100

    • print out all environment variables

% printenv

shell variables
Shell Variables
  • Shell variables are similar to Environment variables except they have a limited scope, i.e., they exist only in the shell which they are defined.
  • Environment variables on the other hand, exist in all its children shells
  • To illustrate this concept, let us look at the following example
environment vs shell variables
Environment vs. Shell Variables

% set prompt = "Parent Shell > "

Parent Shell > setenv DOG Goofy

Parent Shell > set mouse=Mickey

Parent Shell > printenv DOG

Goofy

Parent Shell > echo $mouse

Mickey

Parent Shell > xterm &

(YOU SHOULD NOW HAVE A NEW xterm WINDOW)

THIS IS KNOWN AS

“SPAWNING A NEW (OR CHILD) PROCESS”

environment vs shell variables24
Environment vs. Shell Variables

(IN THE NEW xterm WINDOW, DO THE FOLLOWING)

% set prompt = "Child Shell > "

Child Shell > printenv DOG

Goofy

Child Shell > echo $mouse

mouse: Undefined variable.

environment vs shell variables25
Environment vs. Shell Variables

Child Shell > setenv DOG Pluto

Child Shell > set mouse=Minnie

Child Shell > printenv DOG

Pluto

Child Shell > echo $mouse

Minnie

Child Shell > exit

(THE xterm WINDOW SHOULD NOW GO AWAY - THIS PROCESS HAS NOW BEEN KILLED)

environment vs shell variables26
Environment vs. Shell Variables

Parent Shell >

Parent Shell > printenv DOG

Goofy

Parent Shell > echo $mouse

Mickey

Parent Shell >

environment shell variables
Environment & Shell Variables
  • Why is this important?
    • UNIX uses Environment and Shell Variables control a number of processes
    • Customizes your working environment
    • Variables used for UNIX Scripts
  • They are typically defined and initialized in your .login and .cshrc files
useful shell variables
Useful Shell Variables

filec #Allows file completion

path #List of command

directories

cdpath #List of candidate

directories to cd into

history #Number of commands to

remember

what is shell programming
What is shell programming?
  • Shell programming
    • automate a set of UNIX commands.
    • Just like any programming language
    • “wrappers”
      • black box a customized collection of UNIX commands.
    • Example of shell programs

.login

.cshrc

login file
.loginfile

set path=($HOME/bin /usr/local/bin \

/usr/ucb /usr/sbin /bin /usr/bin \

/usr/bin/X11 .)

stty dec new

tset -I -Q

set mail=/usr/spool/mail/$USER

set editmode = emacs

umask 077

biff n

date

cshrc file
.cshrcfile

if ($?prompt) then

set notify

set history = 100

set savehist = 100

alias pd pushd

alias pop popd

alias vt100 "set term = vt100"

endif

when these files are executed
When these files are executed?

.cshrc

  • is automatically executed when you start a new shell

.login

  • only gets executed once when you first login

Can be re-executed by giving the source command

% source .cshrc

other useful login and cshrc entries
Other useful .login and .cshrc entries

set filec

set cdpath=(~ ~rvrpci/pub ~/mythesis)

Other common entries

set path=( $path /usr/local/bin)

set path=(/usr/local/bin $path)

user defined shell program
User defined shell program
  • Determine name of command
  • Determine input, output, and option arguments
  • Determine UNIX commands to execute
  • Establish error trapping
  • Make shell program executable
a simple shell program
A simple shell program
  • ddcommand to swap bytes

% dd if=input.dat of=output.dat bs=2 conv=swab

  • Very difficult to remember
  • Very little utility to non-UNIX geeks (normal people)
we would rather see
We would rather see...

% swap_bytes input.dat output.dat

special shell variables set
Special Shell Variables Set

% swap_bytes input.dat output.dat

$0

$1

$2

command

$argv[1]

$argv[2]

another special shell variables
Another Special Shell Variables

% swap_bytes input.dat output.dat

$#argv

Indicates how many arguments are present

In this case, 2

shell program swap bytes
shell programswap_bytes

#!/bin/csh -f

dd if=$1 of=$2 bs=2 conv=swab

making swap bytes shell script executable
Making swap_bytesshell script executable

% ls -l swap_bytes

-rw------- ... swap_bytes

% chmod u+x swap_bytes

% ls -l swap_bytes

-rwx------ ... swap_bytes

to run swap bytes
To run swap_bytes
  • swap_bytesbecomes just another unix command!

% swap_bytes input.dat output.dat

limitation of swap bytes
Limitation of swap_bytes
  • No error trapping
  • Should give usage when typing command

% swap_bytes

usage: swap_bytes input_file output_file

improvement to swap bytes
Improvement to swap_bytes

#!/bin/csh -f

if ( $#argv != 2 ) then

echo "usage: $0 input_file output_file"

exit 1

endif

dd if=$1 of=$2 bs=2 conv=swab

commad exit status
Commad exit status
  • By convention

exit 0

Indicates successful command completion

exit 1(or non-zero)

Indicates some error condition

informational message from swap bytes
Informational message from swap_bytes
  • UNIX style informational message

% swap_bytes

usage: swap_bytes input_file output_file

interactive swap bytes
Interactive swap_bytes
  • If you want a “friendlier” shell program
    • Have it query the user for the inputs
  • Another special shell variable can be used

$<

interactive swap bytes47
Interactive swap_bytes

#!/bin/csh -f

if ( $#argv != 2 ) then

echo -n "Please enter the input file> "

set input=$<

echo -n "Please enter the output file> "

set output=$<

endif

dd if=$input of=$output bs=2 conv=swab

interactive swap bytes example
Interactive swap_bytesexample
  • User simply types the command

% swap_bytes

Please enter the input file> input.dat

Please enter the output file> output.dat

a note about quotes in unix
A note about quotes in UNIX

% set a=ls

% echo a

% echo $a

% set b=“$a”

% echo $b

% set b=‘$a’

% echo $b

% set b=`$a`

% echo $b

a note about shell variables
A note about shell variables
  • Shell variables can also double up as arrays
  • Using the previous example,

% echo $b

% echo $b[1]

% echo $#b

% echo $b[$#b]

a more complex shell program
A more complex shell program
  • In pbmplus utilities,

rawtopgm conversion exists

pgmtorawconversion does not

  • A version of pgmtoraw in a programming language like C
    • Time consuming
    • Program will likely be used infrequently
  • Solution: shell program
pgmtoraw shell script design
pgmtoraw shell script design
  • Define input and output files
  • Figure out dimensions of input image
  • Determine number of bytes for input image
  • Determine number of bytes for header
  • Need to strip out the header bytes
  • Write out headerless image data
define input and output files pgmtoraw
Define input and output filespgmtoraw

#!/bin/csh -f

set command_name=$0

set number_args=$#argv

if( $number_args != 1 ) then

echo “usage:$command_name input_file_name”

exit 1

endif

.

.

.

dimensions of input image pnmfile
Dimensions of input image ( pnmfile)

% more test_data.pgm

P2

3 3

255

1 2 3

4 5 6

7 8 9

% pnmfile test_data.pgm

test_data.pgm: PGM plain, 3 by 3 maxval 255

pgmtoraw continued
pgmtoraw(continued)

set input_file=$1

set pnm_info = `pnmfile $input_file`

set image_type = $pnm_info[2]

set data_type = $pnm_info[3]

set width = $pnm_info[4]

set height = $pnm_info[6]

set maxval = $pnm_info[8]

set pixel_bytes = 1

@ image_bytes = $width * $height

pgmtoraw continued57
pgmtoraw (continued)

set file_info=`wc -c $input_file`

set bytes_in_file = $file_info[1]

@ header = $bytes_in_file - $image_bytes

dd if=$input_file bs=$pixel_bytes skip=$header

resulting pgmtoraw utility
Resulting pgmtoraw utility
  • Uses existing routines to obtain information

pnmfile

wc

dd

  • Functional tool written in 20 command lines
current limitations of resulting pgmtoraw utility
Current Limitations of Resulting pgmtoraw utility
  • No check between “ASCII” vs. “RAW” pgm

if( data_type == ‘plain,’) ...

  • No provisions for multibyte per pixel case
    • Use pnmfile results to check for above cases
    • endian case needs to be addressed for multibyte case
  • Above conditions can be addressed by suite of UNIX commands
shell scripts wrappers and idl
Shell Scripts Wrappers and IDL
  • Another utility formerly missing in pbmplus

jpegtopnm or pnmtojpeg

  • IDL has jpeg read/write capability
    • Create a “wrapper” that makes an idl program pbmplus-like
pnmtojpeg pro
pnmtojpeg.pro

pro pnmtojpeg, input_file, output_file

read_ppm, input_file, image

write_jpeg, output_file, image

end

usage of pnmtojpeg pro in idl
Usage of pnmtojpeg.pro in IDL

IDL> pnmtojpeg,‘image.pnm’,’image.jpg’

usage of pnmtojpeg pro in idl63
Usage of pnmtojpeg.pro in IDL

IDL> pnmtojpeg,‘image.pnm’,’image.jpg’

  • For IDL to automatically find pnmtojpeg.pro
    • It must be in the current working directory
    • Directory containing pnmtojpeg.promust be defined in the ENVIRONMENT VARIABLE
      • IDL_PATH
idl path environment variable
IDL_PATHenvironment variable

setenv IDL_DIR /cis/common/rsi/idl_5

setenv IDL_PATH \+$IDL_DIR/lib:\+$IDL_DIR/examples:

\+/dirs/common/rsi/idl_5:\+/dirs/common/lib/idl:\+~/lib/idl

pnmtojpeg csh
pnmtojpeg.csh

#!/bin/csh -f

echo pnmtojpeg “,” “’”$1”’” “,” “’”$2”’” | idl

usage of pnmtojpeg csh
Usage of pnmtojpeg.csh

% pnmtojpeg.csh image.pnm image.jpg

limitation of pnmtojpeg csh
Limitation of pnmtojpeg.csh
  • Does not conform to pbmplus piping, i.e.,

% tifftopnm file.tif | pnmscale 2.0 > new_file.pnm

  • No error trapping
usage cases of pnmtojpeg no command line arguments
Usage cases of pnmtojpeg(no command line arguments)

% tifftopnm file.tif | pnmscale 2.0 | pnmtojpeg > new_file.jpg

usage cases of pnmtojpeg 1 command line argument
Usage cases of pnmtojpeg(1 command line argument)

% pnmtojpeg image.pnm > image.jpg

yet another wrapper pnmtojpeg
Yet another wrapperpnmtojpeg

#!/bin/csh -f

# If user interrupts process, jump to stop

onintr stop

# $0 is the command name

# $#argv is the number of arguments

# $$ is the process id

code for no argument case
Code for no argument case

if($#argv == 0) then

set input_file = /usr/tmp/$0_input_$$

set output_file = /usr/tmp/$0_output_$$

cat > $input_file

pnmtojpeg.csh $input_file $output_file

cat $output_file

.

.

.

code for 1 argument case
Code for 1 argument case

else if($#argv ==1) then

set input_file = $1

set output_file = /usr/tmp/$0_output_$$

pnmtojpeg.csh $input_file $output_file

cat $output_file

.

.

.

code for 2 argument case
Code for 2 argument case

else

set input_file = $1

set output_file = $2

pnmtojpeg.csh $input_file $output_file

endif

#clean up when finished

stop:

rm -f /usr/tmp/$0_input_$$

rm -f /usr/tmp/$0_output_$$

pnmtojpeg summary
pnmtojpegsummary
  • Produced a “new” pbmplus utility
  • Used UNIX shell scripting
    • Argument handling
    • Scratch space /usr/tmp
    • Process id handling
    • Clean up
  • Integrated IDL program and commands
  • 21 lines of UNIX commands
summary
Summary
  • The “dot” files
  • Basics of Shell Scripting
  • Special Shell Variables
  • Seamless integration of UNIX to other utilities (IDL)
ad