UNIX Shells

1. UNIX Shells CIS 218

2. Shells • A shell is a command line interpreter that is the interface between the user and the OS. It is also a programming language. • Shells may be used interactively or non-interactively. In interactive mode, they accept input typed from the keyboard. When executing non-interactively, shells execute commands read from a file. • The shell waits for synchronous commands to complete before accepting more input; asynchronous commands continue to execute in parallel with the shell while it reads and executes additional commands. • The shell: – analyzes each command – performs substitutions – determines what actions are to be performed – performs the actions • Shells also provide a small set of built-in commands (builtins) implementing functionality impossible or inconvenient to obtain via separate utilities. For example, cd, break, continue, and exec) cannot be implemented outside of the shell because they directly manipulate the shell itself. Builtin commands do not generate separate processes.

3. Which Shell? • sh – Bourne shell – Most common, other shells are a superset • csh or tcsh – “C” shell – C-like syntax – Used by UNIX programmers • bash –Bourne again shell’ default on Linux – Based on sh, newer versiond has some csh features. • korn – written by David Korn – Based on sh – combines features of bash and csh. • PERL – Program Extract and Reporting Language – sysadmin scripting language pre-dates Internet but used for web programmng.

4. Common shell facilities • Input-output redirection (STDIN, STDOUT): prog < infile > outfile Also << (here document) and >> (append) • Pipeline “|” commands send the output (STDOUT) from one command to STDIN on a subsequent command. • Semicolon ; allows multiple commands on one line • Job control - A job is a program whose execution has been initiated by the user. - Foreground job: a program which has control of the terminal - Background job: runs concurrently with the parent shell and does not take control of the keyboard. - Start a job in the background by appending & - Commands: ^Z, jobs, fg, bg, kill

5. File Name Expansion • * stands in for 0 or more (any) characters • ? stands in for exactly one (any) character • [..] stands for one character contained in the set of .. sh[1-6] stands in for one of 1, 2, 3, 4, 5, 6 • [^..] stands for one character NOT contained in the set of .. [^oa] stands in for any char except o or a • ~/ stands in for your home directory • Examples - ls *.c - rm file[1-6].? - cd ~/bin - ls ~krueger - ls *.[^oa]

6. Shell Commands • You can run any program in a shell by calling it as you would on the command line. • When you run a program like grep or ls in a shell program, a new process is created. • There are also some built-in UNIX commands where no new process is created (in addition to shell script commands). echo set read exit test shift wait

7. Variables • Variables are placeholders, named locations in memory, where values are stored. • Whitespace or IFS delineate values (see Quoting) – [set] name=value – assignment (no spaces around =) – [set] name= nulls variable, variable exists, no value – $name – replaced by value of name, – ${name} – replaced by value of name, {} delineates varaiable name from adjacent test – unset name - to remove variable • Single value: bindir="/usr/bin“ • List of values (separated by spaces): searchdirs="~/tests $HOME/test2 ."

8. String Quoting • Scripting languages are all about replacing text or strings. Usually with some form or “quoting”. • Escape “\” quotes the following single character. • Double quotes “” inhibit wildcard replacement only. • Single quotes ‘’ inhibit wildcard replacement, variable substitution and command substitution. • Back quotes ``cause command substitution: command output assigned to a variable. • Quoting is used to remove the special meaning of certain characters or words to the shell. • Quoting can be used to disable special treatment for special characters, to prevent reserved words from being recognized as such, and to prevent parameter expansion. • There are four quoting mechanisms: the escape character, single quotes, double quotes nad back quotes.

9. String Quoting • Escape Character: A non-quoted backslash ‘\’ is the Bash escape character. It preserves the literal value of the next character that follows, with the exception of newline. If a \newline pair appears, and the backslash itself is not quoted, the \newline is treated as a line continuation (that is, it is removed from the input stream and effectively ignored). • Single Quotes: Enclosing characters in single quotes (‘’’) preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash. • Double Quotes: Enclosing characters in double quotes (‘"’) preserves the literal value of all characters within the quotes, with the exception of ‘$’, ‘‘’, ‘\’, and, when history expansion is enabled, ‘!’. The characters ‘$’ and ‘‘’ retain their special meaning within double quotes. The backslash retains its special meaning only when followed by one of the following characters: ‘$’, ‘‘’, ‘"’, ‘\’, or newline. Within double quotes, backslashes that are followed by one of these characters are removed. Backslashes preceding characters without a special meaning are left unmodified. A double quote may be quoted within double quotes by preceding it with a backslash. If enabled, history expansion will be performed unless an ‘!’ appearing in double quotes is escaped using a backslash. The backslash preceding the ‘!’ is not removed. The special parameters ‘*’ and ‘@’ have special meaning when in double quotes, • Command substitution: `` encloses a command and allows re-assignment of command output to a variable. Example: DATE=`date`; echo $DATE

10. ANSI-C Quoting • A double-quoted string preceded by a dollar sign (‘$’) will cause the string to be translated according to the current locale. If the current locale is C or POSIX, the dollar sign is ignored. If the string is translated and replaced, the replacement is double-quoted. • Words of the form $'string' are treated specially. The word expands to string, with backslash-escaped characters replaced as specified by the ANSI C standard. Backslash escape sequences, if present, are decoded as follows:

11. ANSI-C Characters (man echo) • \a - alert (bell) • \b - backspace • \e - an escape character (not ANSI C) • \f - form feed • \n - newline • \r - carriage return • \t - horizontal tab • \v -vertical tab • \\ - backslash • \' -single quote • \nnn - the eight-bit character whose value is the octal value nnn (one to three digits) • \xHH -the eight-bit character whose value is the hexadecimal value HH (one or two hex digits) • \cx - a control-x character

12. SHELL Operation • Reads its input from a file from a string supplied as an argument to the ‘-c’ invocation option or from the user’s terminal. • Breaks the input into words and operators, obeying the quoting rules separated by whitespace or command separators (; | <LF>). Alias expansion is performed by this step • Parses the tokens into simple and compound commands. The first word or token in each string is the command. • Performs the various shell expansions breaking the expanded tokens into lists of filenames and commands and arguments. • Performs any necessary redirections and removes the redirection operators and their operands from the argument list. • Executes the command. • Optionally waits for the command to complete and collects its exit status, • When you give the shell a command, it forks (spawns) a child process to execute the command. While the child process is executing the command, the parent process sleeps. While a process is sleeping, it does not use any computer time but remains inactive, waiting to wake up. When the child process finishes executing the command, it tells its parent of its success or failure via its exit status and then dies. The parent process (which is running the shell) wakes up and prompts for another command. • Shell built-in commands do not spawn child processes. Neither do UNIX built-in commands (previous slide).

13. Commands from the SHELL • A simple command is the kind of command encountered most often. It's just a sequence of words separated by blanks, terminated by one of the shell's control operators – usually RETURN or EOF <LF> character. • A pipeline is a sequence of simple commands separated by one of the control operators ‘|’ or ‘|&’. The format for a pipeline is command1 [ [| or |&] command2 ...] The output of each command in the pipeline is connected via a pipe to the input of the next command. That is, each command reads the previous command's output. This connection is performed before any redirections specified by the command. • With ‘|&’ is used, the standard error of command1 is connected to command2's standard input through the pipe; it is shorthand for 2>&1 |. This implicit redirection of the standard error is performed after any redirections specified by the command.

14. Commands from the SHELL • A command list is a sequence of one or more pipelines separated by one of the operators ‘;’, ‘&’, ‘&&’, or ‘||’, and terminated one of ‘;’, ‘&’, or a newline <LF> character. • Of the list operators, ‘&&’ and ‘||’ have equal precedence, followed by ‘;’ and ‘&’, which have equal precedence. Parentheses () can be used to group commands as a single process or change precedence order within a string of commands. • A sequence of one or more newlines may appear in a list to delimit commands, equivalent to a semicolon. • If a command is terminated by the control operator ‘&’, the shell executes the command asynchronously in a subshell. This is known as executing the command in the background. The shell does not wait for the command to finish, and the return status is 0 (true). • The semicolon (;) is a command separator executes a series of commands sequentially by entering them on a single command line and separating each from the next with a semicolon (;). You initiate execution of the sequence of commands by pressing RETURN: $ x ; y ; z<LF>

15. Commands from the SHELL • Commands separated by a ‘;’ are executed sequentially; the shell waits for each command to terminate in turn. The return status is the exit status of the last command executed. • AND and OR lists are sequences of one or more pipelines separated by the control operators ‘&&’ and ‘||’, respectively. AND and OR lists are executed with left associativity. • Parentheses () can be used to group commands as a single process or change precedence order within a string of commands. • An AND list has the form: command1 && command2 command2 is executed if, and only if, command1 returns an exit status of zero. • An OR list has the form| command1 || command2command2 is executed if, and only if, command1 returns a non-zero exit status. • The return status of AND and OR lists is the exit status of the last command executed in the list.

16. Commands from the SHELL • When you enter a long command line and the cursor reaches the right side of the screen, you can use a backslash (\) character to continue the command on the next line. The backslash quotes, or escapes, the NEWLINE character that follows it so that the shell does not treat the NEWLINE as a command terminator. The > is a secondary prompt (PS2) indicating that the shell is waiting for you to continue the unfinished command. • You can use parentheses to group commands. The shell creates a copy of itself, called a subshell, for each group. It treats each group of commands as a job and creates a new process to execute each command . • Each subshell (job) has its own environment, meaning that it has its own set of variables with values that can differ from those of other subshells. The following command line executes commands a and b sequentially in the background while executing c in the background. The shell prompt returns immediately. $ (a ; b) & c &

17. Shell (meta)characters • Char Use • <LF> initiates execution of a command • ; Separates commands • ( ) Groups commands for execution by a subshell or identifies a function • & Executes a command in the background • | Sends output through a pipe • > Redirects standard output • >> Appends standard output • < Redirects standard input • >& Redirect the standard output and standard error to replace current contents. • >>& Redirect the standard output an standard error to append to current contents. • << Here document • * Any string of zero or more characters in an ambiguous file reference • ? Any single character in an ambiguous file reference • \ Quotes the following character • ' Single quotes a string, preventing all substitution • " Double quotes a string, allowing variable and command substitution • '...' Performs command substitution • [ ] Character class in an ambiguous file reference • $ References a variable • . Executes a command (only at the beginning of a line) • # Begins a comment • { } Used to surround the contents of a function • : null builtin Returns true • && Boolean AND Executes command on right only if command on left succeeds (returns a zero exit status) • || Boolean OR Executes command on right only if command on left fails (returns a nonzero exit status) • ! Boolean NOT Reverses exit status of a command • $() Performs command substitution (preferred form) • [] Evaluates an arithmetic expression