1 / 106

Lecture 9 Midterm Solutions

Solutions for the midterm exam questions discussed in Lecture 9, covering KornShell 93, CGI, Perl, regular expressions, and more.

akeith
Download Presentation

Lecture 9 Midterm Solutions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Lecture 9 Midterm Solutions ksh93 CGI (continued) Perl

  2. Question 1 • b,c,d • b,c,f • c,d,f • a,c

  3. Question 2 • same: both match everything • same side effect: both create /tmp/filesdifferent output: first one outputs number of lines, words and chars from ls; second one outputs 0 0 0 because wc gets empty stdin • different: first one outputs unique first fields; second one outputs first fields of unique lines • same: 6 is u=rw, 3 is g=wx, 1 is o=x

  4. Question 2 (cont.) e) different: first command outputs the contents of foobar in uppercase; second command output the contents of foobar with a and z substituted by A and Z respectively, because sed translation command does not take ranges. e) different: first command also prints contents of directories in the current directory; second command only prints contents of current directory

  5. Question 3 • find / -size +5000c –user elee –exec sed –n ’10,20p’ ‘{}’ \; • find / -size +5000c –user elee | xargs –L 1 sed –n ’10,20p’

  6. Question 4 • cut –f1 –d\& | tr a-z A-Z • egrep –c ‘&1&[0-9](\.|$)’ • cut –f1 –d\& | egrep ‘(^|[[:space:]])(Beef|Chicken)$’ • sort –k3rn –t \& | cut –f1 –d\&

  7. Question 5 • $FOO • ls a b • ls a b • * • a b • $BAR • \$BAR • ls a b • a b

  8. Question 6 • 1 • 1-7 • 3,6 • 1,5 • 1,3,5

  9. Question 7 s/^[^[:alnum:]]*// s/[^[:alnum:]]*$// y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/ /^$/c\ This line has been removed.

  10. Question 8 { for (i=1; i<=NF; i++) { if ($i ~ /^[0-9]+$/) { x[i]++; } } nfield=NF } END { for (i=1; i<=nfield; i++) { print x[i]/NR*100 “%” } }

  11. Question 9 #!/bin/sh linenum=1 while IFS= read line ; do for i in "$@" ; do if [ $i -eq $linenum ] ; then echo "$line" break fi done linenum=`expr $linenum + 1` done

  12. Question 9 (alternate) #!/bin/sh trap 'rm –f /tmp/input$$' EXIT cat > /tmp/input$$ linetot=`wc –l < /tmp/input$$` linenum=1 while [ $linenum –le $linetot ] ; do for i in "$@" ; do if [ $i -eq $linenum ] ; then head -$linenum /tmp/input$$ | tail -1 break fi done linenum=`expr $linenum + 1` done

  13. KornShell 93

  14. Variable Attributes • By default attributes hold strings of unlimited length • Attributes can be set with typeset: • readonly (-r) – cannot be changed • export (-x) – value will be exported to env • upper (-u) – letters will be converted to upper case • lower (-l) – letters will be converted to lower case • ljust (-L width) – left justify to given width • rjust (-R width) – right justify to given width • zfill (-Z width) – justify, fill with leading zeros • integer (-I [base]) – value stored as integer • float (-E [prec]) – value stored as C double • nameref (-n) – a name reference

  15. New Parameter Expansion • ${param/pattern/str} – Replace first pattern with str • ${param//pattern/str} – Replace all patterns with str • ${param:offset:len} – Substring with offset

  16. Patterns Extended Regular Expressions Patterns • Additional pattern types so that shell patterns are equally expressive as regular expressions • Used for: • file expansion • [[ ]] • case statements • parameter expansion

  17. ANSI C Quoting • $'…' Uses C escape sequences $'\t' $'Hello\nthere' • printf added that supports C like printing: printf "You have %d apples" $x • Extensions • %b – ANSI escape sequences • %q – Quote argument for reinput • \E – Escape character (033) • %P – convert ERE to shell pattern • %H – convert using HTML conventions • %T – date conversions using date formats

  18. Associative Arrays • Arrays can be indexed by string • Declared with typeset –A • Set: name["foo"]="bar" • Reference ${name["foo"]} • Subscripts: ${!name[@]}

  19. C Expressions • We have already seen built-in expressions with the [[ ]] operator: • [[ $var = *foo* ]] && print "contains foo" • New operator (( )) for C-like numeric expressions: • (( x > 10 )) && print "x=$x, greater than 10" • (( x ++ )) • Note variables don't have to be used with $ inside parens • Value of (( )) expression can be usedwith $(( )) • y=$(( x + 1 )) • print $(( x * y - sin(y) ))

  20. Compound Variables • Variables can contain subfields (like structures or classes) • Syntax: variable name containing . • Example: cust=(name=Jeff zip=10003) cust.state=NY print ${cust.name} print ${!cust.*}

  21. New for loop syntax • Regular syntax: for var in listdo …done • Additional syntax like C: for (( initialization; condition; increment )) do … done • Example: for (( i=0; i < $VAR; i++))

  22. Example: Word Count #!/home/unixtool/bin/kshinteger l=0 w=0 c=0 while read –r LINE do (( l++ )) set -- $LINE (( w += $# )) (( c += ${#LINE}+1 )) done < $1 print "$l lines, $w words, $c characters"

  23. Example: Word Count integer l=0 w=0 c=0 while read –r LINE do (( l++ )) set -- $LINE (( w += $# )) (( c += ${#LINE}+1 )) done < $1 print "$l lines, $w words, $c characters" • integer tag indicates variables will be used as integers • while loop is a command, so redirection works

  24. Example: Word Count integer l=0 w=0 c=0 while read –r LINE do (( l++ )) set -- $LINE (( w += $# )) (( c += ${#LINE}+1 )) done < $1 print "$l lines, $w words, $c characters" • set -- $LINEturns LINE into positional parameters ($1, …), splitting up the value with IFS • $# is the number of positional parameters

  25. Example: Word Count integer l=0 w=0 c=0 while read –r LINE do (( l++ )) set -- $LINE (( w += $# )) (( c += ${#LINE}+1 )) done < $1 print "$l lines, $w words, $c characters" • ${#LINE}returns the length of the value of LINE • We add 1 because the newline character is not part of LINE

  26. Example: Spell a Phone Number Given a number, finds possible words that the number spells on a telephone. Example: $ phonespell 8643 void

  27. Algorithm • Create function combo that prints all combinations of words. Check those against the dictionary. • function combo is recursive: • Pass in part of number, part of word spelled combo 8643 "" combo 643 t combo 643 u combo 643 v combo 43 vm combo 43 vn combo 43 vo … combo "" void

  28. Example: Spell a Phone Number function combo { typeset num=$1 word=$2 if [[ $num = '' ]] then print $word else typeset -L1 digit=$num for letter in ${get_letter[digit]} do combo "${num#?}" "$word$letter" done fi } • functions defined in ksh take arguments as positional parameters, like commands • typeset makes a variable local

  29. Example: Spell a Phone Number function combo { typeset num=$1 word=$2 if [[ $num = '' ]] then print $word else typeset -L1 digit=$num for letter in ${get_letter[digit]} do combo "${num#?}" "$word$letter" done fi } • End of recursion: If number is empty, just print the given word. Should end up happening for every combination

  30. Example: Spell a Phone Number function combo { typeset num=$1 word=$2 if [[ $num = '' ]] then print $word else typeset -L1 digit=$num for letter in ${get_letter[digit]} do combo "${num#?}" "$word$letter" done fi } • Extract leftmost digit from num

  31. Example: Spell a Phone Number function combo { typeset num=$1 word=$2 if [[ $num = '' ]] then print $word else typeset -L1 digit=$num for letter in ${get_letter[digit]} do combo "${num#?}" "$word$letter" done fi } • for loop goes through all letters that correspond to the number (stored in get_letter array, shown next slide) • Recursively calls itself for each letter, taking off one character from the left (using the # operator with pattern ?)

  32. Spell a Phone Number (cont’) set -A get_letter o i "a b c" "d e f" "g h i" "j k l" \ "m n o" "p r s" "t u v" "w x y" # method 1 combo $1 | comm -12 /usr/dict/words - # method 2 trap 'rm -f /tmp/full$$' EXIT combo $1 > /tmp/full$$ spell < /tmp/full$$ | comm -13 - /tmp/full$$ • set –A arraynamevalue value ... • sets elements of an array all at once

  33. Spell a Phone Number (cont’) set -A get_letter o i "a b c" "d e f" "g h i" "j k l" \ "m n o" "p r s" "t u v" "w x y" # method 1 combo $1 | comm -12 /usr/dict/words - # method 2 trap 'rm -f /tmp/full$$' EXIT combo $1 > /tmp/full$$ spell < /tmp/full$$ | comm -13 - /tmp/full$$ • Call function combo with first argument, pipe to comm • suppress fields 1 and 2 (show only matching lines) • comboemits sorted lines, and dictionary is sorted so comm works well

  34. Spell a Phone Number (cont’) set -A get_letter o i "a b c" "d e f" "g h i" "j k l" \ "m n o" "p r s" "t u v" "w x y" # method 1 combo $1 | comm -12 /usr/dict/words - # method 2 trap 'rm -f /tmp/full$$' EXIT combo $1 > /tmp/full$$ spell < /tmp/full$$ | comm -13 - /tmp/full$$ • Another method: use spell command • Create temporary file storing combos • Run through spell, generating list of misspelled words • Pipe to comm, suppressing fields 1 and 3 (show correct words)

  35. Example: Mortgage Calculator float rate=$1 principle=$2 payment integer months years=$3 [[ $1 ]] || read -r 'rate?rate in per cent: ' [[ $2 ]] || read -r 'principle?principle: ' [[ $3 ]] || read -r 'years?years to amoritization: ' print "\n\n\tprinciple\t$principle" print "\trate\t\t$rate" print "\tamortization\t$years" (( months = years*12 )) (( rate /= 1200. )) (( payment = (principle*rate)/(1.-pow(1.+rate,-months)) )) • Declare variables • Read in unspecified inputs

  36. Example: Mortgage Calculator float rate=$1 principle=$2 payment integer months years=$3 [[ $1 ]] || read -r 'rate?rate in per cent: ' [[ $2 ]] || read -r 'principle?principle: ' [[ $3 ]] || read -r 'years?years to amortization: ' print "\n\n\tprinciple\t$principle" print "\trate\t\t$rate" print "\tamortization\t$years" (( months = years*12 )) (( rate /= 1200. )) (( payment = (principle*rate)/(1.-pow(1.+rate,-months)) )) • Initialize values • Uses built-in arithmetic (pow, floating point /)

  37. Example: Mortgage Calculator printf "\tmonthly payment\t%8.2f\n\n" "$payment" print '\tYears Balance' print '\t====== =======' for (( months=0; principle > 0; months++)) do (( principle *= (1.+rate) )) (( principle -= payment )) if (( ((months+1)%12) == 0 )) then printf "\t%d\t%8.2f\n" months/12 "$principle" fi done • Print table header • Uses printf to format floating point number

  38. Example: Mortgage Calculator printf "\tmonthly payment\t%8.2f\n\n" "$payment" print '\tYears Balance' print '\t====== =======' for (( months=0; principle > 0; months++)) do (( principle *= (1.+rate) )) (( principle -= payment )) if (( ((months+1)%12) == 0 )) then printf "\t%d\t%8.2f\n" months/12 "$principle" fi done • C-style for loop with numerical calculations

  39. Documentation • Web version of Learning the KornShell documents ksh93. Good for learning ksh. • Glass documents ksh88 and bash • UNIX in a Nutshell has a chapter that is a great ksh93 reference. Documents: • Bourne shell compatible features • ksh88 compatible features • ksh93 features

  40. Last Week…. • HTML forms are used to collect user input • Data sent via HTTP request • Server launches CGI script to process data • Browser sends form variable as name-value pairs • name1=value1&name2=value2&name3=value3 • Two methods (HTTP_METHOD) to send form input: • GET method: name-value pairs in URL (QUERY_STRING) • POST method: name-value pairs in body or HTTP request

  41. CGI Script: Example

  42. Part 1: HTML Form <html> <center> <H1>Anonymous Comment Submission</H1> </center> Please enter your comment below which will be sent anonymously to <tt>kornj@cs.nyu.edu</tt>. If you want to be extra cautious, access this page through <a href="http://www.anonymizer.com">Anonymizer</a>. <p> <form action=cgi-bin/comment.cgi method=post> <textarea name=comment rows=20 cols=80> </textarea> <input type=submit value="Submit Comment"> </form> </html>

  43. Part 2: CGI Script (ksh) #!/home/unixtool/bin/ksh . cgi-lib.ksh # Read special functions to help parse ReadParse PrintHeader print -r -- "${Cgi.comment}" | /bin/mailx -s "COMMENT" kornj print "<H2>You submitted the comment</H2>" print "<pre>" print -r -- "${Cgi.comment}" print "</pre>"

  44. Debugging • Debugging can be tricky, since error messages don't always print well as HTML • One method: run interactively $ QUERY_STRING='birthday=10/15/03'$ ./birthday.cgi Content-type: text/html <html>Your birthday is <tt>10/15/02</tt>.</html>

  45. How to get your script run • This can vary by web server type http://www.cims.nyu.edu/systems/resources/webhosting/index.html • Typically, you give your script a name that ends with .cgi • Give the script execute permission • Specify the location of that script in the URL

  46. CGI Security Risks • Sometimes CGI scripts run as owner of the scripts • Never trust user input - sanity-check everything • If a shell command contains user input, run without shell escapes • Always encode sensitive information, e.g. passwords • Also use HTTPS • Clean up - don’t leave sensitive data around

  47. CGI Benefits • Simple • Language independent • UNIX tools are good for this because • Work well with text • Integrate programs well • Easy to prototype • No compilation (CGI scripts)

  48. Example: Dump Some Info #!/home/unixtool/bin/ksh . ./cgi-lib.ksh PrintHeader ReadParse print "<h1> Date </h1>" print "<pre>" date print "</pre>" print "<h1> Form Variables </h1>" print "<pre>" set -s -- ${!Cgi.*} for var do nameref r=$var print "${var#Cgi.} = $r" unset r done print "</pre>" print "<h1> Environment </h1>" print "<pre>" env | sort print "</pre>"

  49. Example: Find words in Dictionary <form action=dict.cgi> Regular expression: <input type=entry name=re value=".*"> <input type=submit> </form>

  50. Example: Find words in Dictionary #!/home/unixtool/bin/ksh PATH=$PATH:. . cgi-lib.ksh ReadParse PrintHeader print "<H1> Words matching <tt>${Cgi.re}</tt> in the dictionary </H1>\n"; print "<OL>" grep "${Cgi.re}" /usr/dict/words | while read word do print "<LI> $word" done print "</OL>"

More Related