Shell Scripting

1. Shell Scripting WeeSan Lee <weesan@cs.ucr.edu> http://www.cs.ucr.edu/~weesan/cs183/

2. Roadmap • Introduction • Prompt & Alias • How to Create a Shell Script? • Sharp-Bang & Comments • Always Use Absolute Pathname! • Command Separator , ||, && • Backquote/Backtick • Variables • Positional Parameters • Misc. • References

3. Introduction • Shell • A command interpreter • For example • sh, bash, csh, tcsh, kosh, zsh, … • Shell Script • Whole bunch of Unix commands saved into a text file with flow control • Your shell scripting skills depends on • # Unix commands you know • how well you put them together to do the right thing

4. Prompt & Alias • PS1="\u@\h:\w \$ “ • \u the username of the current user • \h the hostname • \w the current working directory • \$ if the effective UID is 0, a #, otherwise a $ • alias v='ls -l --color=auto‘ • ~/.bashrc • After editing • $ source ~/.bashrc

5. How to Create a Shell Script? • Edit hello.sh • #!/bin/sh • echo "Hello World!" • Make hello.sh an executable • $ chmod +x hello.sh • Run hello.sh • $ ./hello.sh • $ sh hello.sh • Debug hello.sh • $ sh -x hello.sh

6. Sharp-Bang (#!) & Comments • Which shell to run the script? • #! Sharp-Bang, a two-byte magic number • $ cat hello.sh • #!/bin/sh • # My first shell script • echo "Hello World!”

7. Always Use Absolute Pathname! • $ cat cal.sh • #!/bin/sh • cal 2008 • $ cat cal • #!/bin/sh • /usr/bin/bash • $ cat cal2.sh • #!/bin/sh • CAL=/usr/bin/cal • $CAL 2008

8. Command Separator, ||, && • $ cd /var/log ; tail message • $ alias todo='echo ; cat -n ~/TODO ; echo‘ • $ cd /var/log || { • echo "Cannot change to /var/log" • exit 1 • } • $ cat or.sh • #!/bin/sh • [ -e $1 ] || exit 1 • /usr/bin/less $1 • $ lpr file.tmp && rm file.tmp

9. Backquote/Backtick • $ echo "The date is `date`" • $ wget `cat z.txt` • $ for i in `cat kilo.txt`; do • echo “$i has `ssh $i who | wc -l` users” • done

10. Exit Status • Each Unix command returns a exit status ($?) • Unlike C, 0 means true, false otherwise • $ /bin/true • $ echo $? • 0 • $ cat abc • $ echo $? • 1 • $ touch abc • $ cat abc • $ echo $? • 0

11. Exit Status • Return a exit status using “exit” built-in command • $ cat exit2.sh • #!/bin/sh • exit 2 • $ exit2.sh • $ echo $? • 2 • $ exit-1.sh • #!/bin/sh • exit -1 • $ echo $? • 255

12. Exit Status • The script returns the status of the last executable • $ cat exit3.sh • #!/bin/sh • /usr/bin/who | /bin/false • $ exit3.sh • $ echo $? • 1

13. $ a=3 Define a variable a $ echo $a $ echo ${a} $ echo "a = $a" $a is a reference to its value $ a =3 run a with =3 as parameter $ a= 3 # run 3 with empty environment a $ a="a b c" $ echo $a $ echo "$a" Double quoting a variable preserves whitespaces $ echo '$a' Single quoting a variable disables var. referencing $ a= Set a to a null value $ unset a $ a=`date` $ echo $a $ a=`ls -l` Can assign anything to a variable $ echo $a $ echo "$a" Double quotes preserves formatting $ echo "'a' is $a" $ echo "\"a\" is $a" $ echo "\$a is $a" Variables

14. Variables • $ cat user.sh • #/bin/sh • NUM="`who | cut -d' ' -f1 | sort | uniq | wc -l`" • echo "`hostname` has $NUM users."

15. $ cat param.sh #!/bin/sh echo "Program = $0" echo "# of parameters = $#" echo "\$1 = $1" echo "\$2 = $2" echo "\$3 = $3" echo "\$10 = ${10}" echo "\$@ = $@" echo "\$* = $*" $ param.sh `seq 1 10` Program = ./param.sh # of parameters = 10 $1 = 1 $2 = 2 $3 = 3 $10 = 10 $@ = 1 2 3 4 5 6 7 8 9 10 $* = 1 2 3 4 5 6 7 8 9 10 $ echo $$ Current process ID Positional Parameters

16. Positional Parameters & Soft-link Trick • $ ln -s param.sh newcmd.sh • $ newcmd.sh `seq 1 10` • Program = ./newcmd.sh • # of parameters = 10 • $1 = 1 • $2 = 2 • $3 = 3 • $10 = 10 • $@ = 1 2 3 4 5 6 7 8 9 10 • $* = 1 2 3 4 5 6 7 8 9 10

17. Branching • Syntax • if<condition>; then • <stmts> • elif<condition>; then • <stmts> • else • <stmts> • fi

18. How to check if a given path is a file or a directory? • $ test.sh /etc/foo • /etc/foo does not exist. • $ test.sh /etc/passwd • /etc/passwd is a file. • $ test.sh /etc • /etc is a directory.

19. How to check if a given path is a file or a directory? If [ $# -ne 1 ]; then echo "Usage: $0 file" exit 1 fi • #!/bin/sh • if test -e $1; then • if [ -f $1 ]; then • echo "$1 is a file." • elif [ -d $1 ]; then • echo "$1 is a directory." • else • echo "$1 has an unknown type." • fi • else • echo "$1 does not exist." • fi [ $# -eq 1 ] || { echo "Usage: $0 file" exit 1 }

20. File Test Operators • -e file exists • -f regular file • -s file is not zero size • -d directory • -b block device • -c char. device • -L symbolic link • -r file has read permission • -w write • -x execute

21. Comparison Operators • #!/bin/sh • a=1 • b=2 • if [ "$a" -ne "$b" ]; then # Numerical comparision: 1 != 2 • echo "$a and $b are not equal" • else • echo "$a and $b are equal" • fi • if [ "$a" != "$b" ]; then # String comparision: "1" != "2” • echo "$a and $b are not equal" • else • echo "$a and $b are equal" • fi

22. Comparison Operators • Integer Comparison Operators • -eq is equal to • -ne is not equal to • -gt is greater than • -ge is greater than or equal to • -lt is less than • -le is less than or equal to • String Comparison Operators • = is equal to • == same as = • != is not equal to • -n string is not null • -z string is null

23. -n example • #!/bin/sh • # Wrong, should do [ $1 ] • if [ -n $1 ]; then • echo "\$1 is non-null" • else • echo "\$1 is null" • fi • # Correct • if [ -n "$1" ]; then • echo "\$1 is non-null" • else • echo "\$1 is null" • fi

24. Case Statement • Syntax • case "$var" in • value1) • <stmts> • ;; • value2) • <stmts> • ;; • *) • <stmts> • ;; • esac

25. Case Statement - an example • #!/bin/sh • echo -n "Hit a key, then hit Enter: " • read key • case "$key" in • a|b|c|d|e|f|g|h|i|j|k|l|m|n|o|p|q|r|s|t|u|v|w|x|y|z) • echo "$key is a lower case." • ;; • A|B|C|D|E|F|G|H|I|J|K|L|M|N|O|P|Q|R|S|T|U|V|W|X|Y|Z) • echo "$key is an upper case." • ;; • 0|1|2|3|4|5|6|7|8|9) • echo "$key is a number." • ;; • *) • echo "$key is a punctuation." • ;; • esac

26. Case Statement - a better example • #!/bin/sh • read -n1 -p "Hit a key: " key • echo • case "$key" in • [a-z]) • echo "$key is a lower case." • ;; • [A-Z]) • echo "$key is an upper case." • ;; • [0-9]) • echo "$key is a number." • ;; • *) • echo "$key is a punctuation." • ;; • esac

27. for loop • Syntax • for <var> in <list>; do • <stmts> • done • For example • for i in 1 2 3 4 5 6 7 8 9 10; do • echo $i • done for i in `seq 1 10`; do

28. for loop /usr/*/man gets expanded first! • $ for i in /usr/*/man; do echo $i; done • /usr/bin/man • /usr/csshare/man • /usr/kerberos/man • /usr/local/man • /usr/share/man

29. while & until loop • while loop syntax • while<condition>; do • <stmts> • done • until loop syntax • until<condition>; do • <stmts> • done

30. How to keep track if a user has been logged out? • Null command always returns true • $ : • $ echo $? • 0 • Create a zero-sized file • $ : > out.txt • $ cat /dev/null > out.txt • $ touch out.txt • #!/bin/sh • user=manager • while : ; do • who | grep $user &> /dev/null • if [ $? -ne 0 ]; then • break • else • sleep 30 • continue • fi • done • echo "$user has logged out at `date`" Do we care about the output?

31. How to keep track if a user has been logged out? • #!/bin/sh • user=manager • while who | grep $user &> /dev/null; do • sleep 30 • done • echo "$user has logged out at `date`"

32. How to keep track if a user has been logged in? • #!/bin/sh • user=weesan • until who | grep $user &> /dev/null; do • sleep 30 • done • echo "$user has logged in at `date`"

33. read • #!/bin/sh • while read line; do • echo "$line" • done < /etc/passwd • #!/bin/sh • OIFS=$IFS; IFS=: • while read name passwd uid gid fullname ignore; do • echo "$name ($fullname)" • done < /etc/passwd • IFS=$OIFS

34. How to find the UID of “weesan”? • Parse /etc/passwd line-by-line • while IFS=: read name passwd uid gid fullname ignore; do • if [ "$name == "weesan" ]; then • echo $uid • fi • done < /etc/passwd • Or, use grep & cut • $ grep weesan /etc/passwd | cut -d: -f3 • Or, simply • $ id -u weesan

35. How to remove “weesan” from /etc/passwd? • Parse /etc/passwd line-by-line • while IFS=: read name ignore; do • if [ "$name" != "weesan" ]; then • echo "$uid:$ignore” • fi • done < /etc/passwd • Or use grep • $ grep -v weesan /etc/passwd

36. How to find lines starting or ending with '1'? • $ cat data.txt • 1a • 2 * • 3 • 411 • $ grep 1 data.txt • 1a • 411 • $ grep ^1 data.txt • 1a • $ grep '1$' data.txt • 411

37. Command Line Options • $ argv.sh -a -b -c -d 1 -e -f foo.txt • Option a • Option b • Option c • Option d with 1 • Option e • Unknown option: -f • The rest is: foo.txt

38. $ cat argv.sh #!/bin/sh for arg; do case "$1" in -a) echo "Option a";; -b) echo "Option b";; -c) echo "Option c";; -d) shift echo "Option d with $1" ;; -e) echo "Option e";; -*) echo "Unknown option: $1";; *) FILE="$FILE $1";; esac shift done echo "The rest is: $FILE" $ cat getopts.sh #!/bin/sh while getopts "abcd:e" arg; do case "$arg" in a) echo "Option a";; b) echo "Option b";; c) echo "Option c";; d) echo "Option d with $OPTARG";; e) echo "Option e";; esac done shift $((OPTIND - 1)) echo "The rest is: $1" Command Line Options

39. Command Line Options • $ getopts.sh -abc -d1 -ef foo.txt • Option a • Option b • Option c • Option d with 1 • Option e • getopts.sh: illegal option -- f • The rest is: foo.txt

40. Here Document • #!/bin/sh • cat <<EOF • This is a here document. • Usually it has multiple lines • and long. • EOF

41. Basename, dirname • $ dirname /boot/System.map • /boot • $ basename /boot/System.map • System.map • $ basename /boot/System.map .map • System • Good for renaming • $ a="a.txt" • $ tmp="`basename $a .txt`.tmp” • $ grep -v weesan $a > $tmp ; mv $tmp $a

42. eval • #!/bin/sh • cmd='/usr/bin/grep $string $file' • read -p "File: " file • read -p "String: " string • eval $cmd

43. #!/bin/sh echo "\$user is not set" echo ${user-`whoami`} echo "\$user is set" user="foo" echo ${user-`whoami`} echo "\$user is set and non-empty" user="foo" echo ${user-`whoami`} echo ${user:-`whoami`} echo "\$user is set but empty" user= echo ${user-`whoami`} echo ${user:-`whoami`} $user is not set weesan $user is set foo $user is set and non-empty foo foo $user is set but empty weesan Parameter Substitution

44. Function Call • #!/bin/sh • start() { • echo "Start" • } • stop() { • echo "Stop" • } • restart() { • echo "Restart" • } • case "$1" in • [Ss]tart) start;; • [Ss]top) stop;; • [Rr]estart) restart;; • *) echo "Usage $0 start|stop|restart";; • esac

45. #!/bin/sh foo() { echo "# of para = $#" echo "\$0 = $0" echo "\$1 = $1" echo "\$2 = $2" echo "\$3 = $3" } foo 1 2 3 # of para = 3 $0 = ./func2.sh $1 = 1 $2 = 2 $3 = 3 Function Call - parameters

46. find • $ find -nouser -xdev • $ find /home -print0 | xargs -0 ls -l • $ find /bin /usr/bin /sbin -perm -4000 -or -perm -2000 • $ find /bin /usr/bin /sbin -perm -4000 -or -perm -2000 | xargs ls -l • $ find /bin /usr/bin /sbin -perm -4000 -or -perm -2000 -exec ls -l '{}' \;

47. References • Advance Bash-Script Guide • http://www.tldp.org/LDP/abs/abs-guide.pdf • Unix Power Tools, 3rd Edition • Shelley Powers, Jerry Peek, Tim O'Reilly, Mike Loukides