Shell Basics

1. CS465 - Unix Shell Basics

2. Shell Basics • Shells provide: • Command interpretation • Multiple commands on a single line • Expansion of wildcard filenames • Redirection of input and output • Pipelines • Background execution • Script-programming facility

3. Running Multiple Commands ; Causes commands to run in sequence. $ cmd1 ; cmd2 ( ) Causes sequenced commands to be grouped $ (cmd1 ; cmd2) && Runs the next command only if the current command succeeds. $ cmd1 && cmd2 || Runs the next command only if the current command fails. $ cmd1 || cmd2

4. Sequence Example Commands can be sequenced (run in sequence) using the semicolon (;): $ ls ; who ; date ; pwd file1 file2 file3 newfile small000 pts/1 Jun 4 14:41 Tue Jun 4 14:42:51 MDT 2003 /export/home/small000 $ If any one command fails, the others will still execute. NOTE: To make the command line easier to read, separate commands from the semicolon (;) with blanks.

5. Grouped Commands Example • Commands can be grouped together using parentheses • To create a “single command” out of a group of commands (especially useful before a pipe): $ (cat letter1; head -2 names) | sort

6. && Example To run the second command only if the first command succeeds (i.e. short-circuit “and” operator): BAD: $ cd /somedir; rm -r * GOOD:$ cd /somedir && rm -r *

7. || Example To run the second command only if the first command fails (i.e. short-circuit “or” operator): Example: $ ls file1 || cp fileX file1

8. Shell Environment • Your environment defines such characteristics as your user identity, where you are working on the system, and what commands you are running. • Your working environment is defined by both environment variables and shell variables. • Your default login shell uses environment variables and passes them to all processes and subshells that you create. • Shell variables are valid only for your current shell and are not passed to subshells.

9. Environment Variables HOME Name of your login directory. Set by login program. HOST Name of host machine running your shell program. LOGNAME Your login name. Set by login program. MAIL Pathname of the file used by the mail system to detect the arrival of new mail. Set by login program.

10. Environment Variables PATH Directory list and order that your system uses to search for, find, and execute commands. Set by login scripts. SHELL Your default shell. Set using the shell specified in your entry in the /etc/passwd file. TERM Type of terminal you are using. Set by your login script. TZ Your current time zone. Set by the system login script.

11. Environment Variables • To see the value of a defined variable, place a $ in front of the variable name: $ echo $HOME /home/small000 $ echo $TZ MDT $ • Note the difference if you leave off the "$": $ echo HOME HOME

12. Metacharacters • Each shell program recognizes a set of special characters called metacharacters: • ; & ( ) | \ ^ ~ % { } $ # ´ “ ‘ @ * ? [ ] ! • Metacharacters are used to create patterns to match filenames and command names. • For example, we often wish to apply a command to a selection of files or directories • The shell wildcard (often known as globbing) feature allows this.

13. Wildcard Metacharacters ?matches any character * matches any number of characters [abc] matches any one of the characters in the brackets [a-z] matches any one character in the range of characters in the brackets [!a-z] matches any character except the range of characters in the brackets

14. Wildcard Examples $ ls abc?def • matches abcXdef, abc.def, abc3def but not abcdef $ ls abc* • matches abc., abcd, abcdef, abc.another.part $ ls *.* • matches any name with a “.” in it $ ls * • matches any name (except names beginning with “.”) $ ls [a-z]* • matches any name beginning with a lower-case letter

15. Wildcard Exercise • Assume we have the following files under the current directory: 120 235 305 a.out c22 cap Doc.1 Doc.2 Doc.3 one.c two.c three.c • What would match for: • ls *.c ls ?2? • ls D* ls [a-z]*[!0-9] • ls ??? ls [23]* • ls * ls .*

16. Avoiding Wildcard Expansion • Sometimes you may NOT want wildcard characters expanded. • The Unix glob utility translates wildcard characters on the command line. • To turn off ALL wildcard expansion: $ set noglob • To turn ALL wildcard expansion back on: $ set glob

17. Avoiding Wildcard Expansion • Sometimes you need to selectively enter a wildcard character on a command line, and not have it expanded. • Method 1: Double quote the argument containing the character: $ ls “abc*def” abc*def ls the file whose name is abc*def (if the system allows such a name)

18. Avoiding Wildcard Expansion • Method 2: “Escape” the character $ ls abc\*def • again lists the file whose name is abc*def • Can use these methods in combination $ echo “Show asterisks *” and \* Show asterisks * and * $ • * inside the double quotes is not expanded • * after the backslash is not expanded

19. Double Quotes vs Single Quotes • Within double quotes, the reserved characters: • dollar sign ($) • grave accent (`) • backslash (\) keep their special meanings • All other characters are taken displayed literally.

20. Double vs Single Quote Example Displaying a variable name without it being interpreted by the shell: $ echo “Value of $USER is “ $USER Value of small000 is small000 $ $ echo 'Value of $USER is ' $USER Value of $USER is small000 $

21. Grave Accent/Backquotes • Within backquotes (graves), command substitution is forced: • Example: $ echo I am user whoami I am user whoami $ echo I am user `whoami` I am user small000 $

22. Summary of Quoting Metacharacters • Single quotes • Contents are used verbatim • No wildcard expansion • No variable interpolation • No command execution • Double quotes • Only turns off wildcard expansion • Backquotes • Forces execution of command in quotes • Output from command replaces quoted string

23. stdout (1) stdin (0) command stderr (2) Redirection of I/O Every program run under Unix is provided with three standard I/O streams: stdin, stdout and stderr By default: - the standard input (stdin) is the keyboard - the standard output (stdout) and standard error (stderr) are the display screen Your shell allows you to divert any or all of these streams.

24. Output Redirection • Use the “>” character to redirect the standard output of a command to a file: $ echo hello $USER > greeting $ cat greeting hello small000 $ echo hi there > greeting $ cat greeting hi there $ $ ls > myfiles $ cat myfiles file1 file2 file3 $

25. Output Redirection • Use the “>>” character after a command to redirect output to APPEND to a file: $ cat greeting hi there $ echo “How are you?” >> greeting $ cat greeting hi there How are you? $

26. Output Redirection • Can separate stdout (1) and stderr (2): $ echo roses are red 1> rhyme 2> errs $ echo violets are blue 1>> rhyme 2>> errs $ cat errs $ cat rhyme roses are red violets are blue $ $ ls x* ls: x*: No such file or directory $ ls x* 1>xfiles 2>errs $ cat xfiles $ cat errs ls: x*: No such file or directory

27. Output Redirection Redirecting cat’s output can create a file (no input file name given, so gets input from stdin): $ cat test type line 1 type line 2 $ $ cat > test type line 1 type line 2 <ctrl-d>  to end your input $

28. Output Redirection • Appending file1 to file2 $ cat file1 >> file2 • Typing to append to the end of a file $ cat >> test type line 3 type line 4 <ctrl-d> $ cat test type line 1 type line 2 type line 3 type line 4

29. Overwriting Files? • By default, the use of ‘>’ will overwrite any existing file, without warning. Use “noclobber” to prevent this: $ set –o noclobber (+o to revoke) • Gives error if you try to overwrite an existing file. • Can still use >| to force overwriting of an existing file. $ ls file1 file2 $ set -o noclobber $ cat file1 > file2 ksh: cannot create file2: File exists $ cat file >| file2 $

30. Input Redirection • Use the “<” character to redirect input from a file to a command: $ sort < names $ mail user < letter.txt • Can use input and output redirection together: $ sort < names > names.sort

31. Filters • Most UNIX utilities (commands) are filters • A filter is a program which: • reads input (text) from standard input only • writes output and errors to standard output only • Filters can be combined to work together using pipes • A pipe takes the stdout of one command and uses it as the stdin of another command

32. Pipes • The standard output of one program is “piped” to the standard input of another: $ who | sort • Several pipes can be connected: $ ls | sort | more • No limit on number of processes that can be chained by pipes

33. Pipes • Pipes and I/O redirection can be used together: $ ls | sort > files.sorted $ sort < names | more

34. Why Pipes? • Could chain commands together $ ls > temp; sort < temp; rm temp • Works, but requires temporary intermediate file, which must be manually removed • Solution – Use a pipe: $ ls | sort • No intermediate file, processes run in parallel

35. Multiple Outputs • What if you want the output of a command to be saved in a file AND displayed on the screen? • Use the tee command • Example: $ ls –la | tee filelistDisplays directory listing to the screen, AND saves it to the file filelist.

36. Summary of Shell Command Interpretation The shell reads a line from a script file or from a user. Then: • Meta-characters are "handled.“ • Executable is found (via PATH). • Arguments are passed to the program. • File redirection is setup. • The program is executed.