shell programming n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Shell Programming PowerPoint Presentation
Download Presentation
Shell Programming

Loading in 2 Seconds...

play fullscreen
1 / 90

Shell Programming - PowerPoint PPT Presentation


  • 160 Views
  • Uploaded on

Shell Programming. Guntis Barzdins Girts Folkmanis. Lecture outline. Shell features Helper utilities, introduction Connecting utilities with shell scripting Helper utilities in detail Piping, advanced examples Shell scripts as files, Internal shell commands. Shell features.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Shell Programming' - johana


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
shell programming

Shell Programming

Guntis Barzdins

Girts Folkmanis

lecture outline
Lecture outline
  • Shell features
  • Helper utilities, introduction
  • Connecting utilities with shell scripting
  • Helper utilities in detail
  • Piping, advanced examples
  • Shell scripts as files, Internal shell commands
shell features
Shell features
  • We will talk about bash, there might be differences for other shells.
    • bash - GNU Bourne-Again Shell
    • Authors: Brian Fox and Chet Ramey
    • Free Software Foundation
  • Popular in different distributions
  • Tip: To find your current shell, type following command

$ echo $SHELL

/bin/bash

shell features1
Shell Features
  • The shell itself is defined in SUS (Single UNIX Specification) as regardscalling conventions and switches. The language interpreted by the shell is alsopart of the standard
  • The shell standard derives from the POSIX.2standard, which is not freely available (thecurrent standard, SUS, stands as IEEE Std1003.1 2001and is identical to POSIX.2)
shell features2
Shell features
  • Two types of usage:
    • Command line - interactive
    • Shell script, usually non-interactive
  • Shell script defined as:
    • "Shell Script is series of commands written in plain text file. Shell script is just like batch file is MS-DOS but have more power than the MS-DOS batch file."
shell features3
Shell features
  • Two types of commands:
    • Internal commands – built in the shell interpreter
    • External commands – calling other executable files
  • Almost everything applies to both command line usage and shell scripts
external commands
External commands
  • Execution of external programs – most common task

External program: /bin/ls

girtsf@linux tmp $ ls -l /lib

total 4035

-rwxr-xr-x 1 root root 7488 Oct 6 12:33 cpp

drwxr-xr-x 13 root root 1024 Oct 25 15:57 dev-state

drwxr-xr-x 2 root root 1024 Jun 28 09:53 evms

drwxr-xr-x 2 root root 2048 Aug 23 15:25 iptables

-rwxr-xr-x 1 root root 92716 Oct 14 13:10 ld-2.3.4.so

-rwxr-xr-x 1 root root 22800 Oct 14 13:17 ld-linux.so.1

...

external commands1
External commands
  • Environment variable $PATH determines where to search for external programs.
  • girtsf@linux tmp $ echo $PATH

/bin:/usr/bin:/usr/local/bin:/opt/bin

  • “:” as separator
  • Current directory “.” is usually not in PATH for security reasons.
external commands2
External commands
  • girtsf@linux tmp $ echo $PATH

/bin:/usr/bin:/usr/local/bin:/opt/bin

  • With /bin in path, typing “ls” suffices to run /bin/ls.
  • Example of unsetting path:
  • girtsf@linux tmp $ unset PATH

girtsf@linux tmp $ ls

bash: ls: No such file or directory

girtsf@linux tmp $

internal commands
Internal commands
  • A large list of built in commands, that are handled internally without running an external command
  • Most commonly used internal command is cd, used to change the current working directory:
  • girtsf@linux girtsf $ cd /tmp/

girtsf@linux tmp $

aliasing
Aliasing
  • Aliasing is the process of assigning a command to a shorter “alias”
  • This allows you to type the shorter command instead of the longer one.
  • Aliasing is useful for changes that you want all of the time.
    • alias rm “rm –i”
  • Aliasing is similar to shell function definitions
    • dos2unix() { cat $1 | perl -pe 's/\r\n$/\n/g'; }
    • unix2dos() { cat $1 | perl -pe 's/\n$/\r\n/g'; }
internal commands1
Internal commands

girtsf@linux tmp $ help

GNU bash, version 2.05b.0(1)-release (i686-pc-linux-gnu)

These shell commands are defined internally. Type `help' to see this list.

Type `help name' to find out more about the function `name'.

Use `info bash' to find out more about the shell in general.

Use `man -k' or `info' to find out more about commands not in this list.

A star (*) next to a name means that the command is disabled.

%[DIGITS | WORD] [&] (( expression ))

. filename :

[ arg... ] [[ expression ]]

alias [-p] [name[=value] ... ] bg [job_spec]

bind [-lpvsPVS] [-m keymap] [-f fi break [n]

builtin [shell-builtin [arg ...]] case WORD in [PATTERN [| PATTERN].

cd [-L|-P] [dir] command [-pVv] command [arg ...]

...

sus shell grammar
SUS: shell grammar

complete_command : list separator

| list

;

list : list separator_op and_or

| and_or

;

and_or : pipeline

| and_or AND_IF linebreak pipeline

| and_or OR_IF linebreak pipeline

;

pipeline : pipe_sequence

| Bang pipe_sequence

;

pipe_sequence : command

| pipe_sequence '|' linebreak command

;

command : simple_command

| compound_command

| compound_command redirect_list

| function_definition

;

compound_command : brace_group

| subshell

| for_clause

| case_clause

| if_clause

| while_clause

| until_clause

;

subshell : '(' compound_list ')'

;

compound_list : term

| newline_list term

| term separator

| newline_list term separator

;

term : term separator and_or

| and_or

;

for_clause : For name linebreak do_group

| For name linebreak in sequential_sep do_group

| For name linebreak in wordlist sequential_sep do_group

;

name : NAME /* Apply rule 5 */

;

in : In /* Apply rule 6 */

;

wordlist : wordlist WORD

| WORD

;

case_clause : Case WORD linebreak in linebreak case_list Esac

| Case WORD linebreak in linebreak case_list_ns Esac

| Case WORD linebreak in linebreak Esac

;

case_list_ns : case_list case_item_ns

| case_item_ns

;

case_list : case_list case_item

| case_item

;

case_item_ns : pattern ')' linebreak

| pattern ')' compound_list linebreak

| '(' pattern ')' linebreak

| '(' pattern ')' compound_list linebreak

;

case_item : pattern ')' linebreak DSEMI linebreak

| pattern ')' compound_list DSEMI linebreak

| '(' pattern ')' linebreak DSEMI linebreak

| '(' pattern ')' compound_list DSEMI linebreak

;

pattern : WORD /* Apply rule 4 */

| pattern '|' WORD /* Do not apply rule 4 */

;

if_clause : If compound_list Then compound_list else_part Fi

| If compound_list Then compound_list Fi

;

else_part : Elif compound_list Then else_part

| Else compound_list

;

while_clause : While compound_list do_group

;

until_clause : Until compound_list do_group

;

function_definition : fname '(' ')' linebreak function_body

;

function_body : compound_command /* Apply rule 9 */

| compound_command redirect_list /* Apply rule 9 */

;

fname : NAME /* Apply rule 8 */

;

brace_group : Lbrace compound_list Rbrace

;

do_group : Do compound_list Done /* Apply rule 6 */

;

simple_command : cmd_prefix cmd_word cmd_suffix

| cmd_prefix cmd_word

| cmd_prefix

| cmd_name cmd_suffix

| cmd_name

;

cmd_name : WORD /* Apply rule 7a */

;

cmd_word : WORD /* Apply rule 7b */

;

cmd_prefix : io_redirect

| cmd_prefix io_redirect

| ASSIGNMENT_WORD

| cmd_prefix ASSIGNMENT_WORD

;

cmd_suffix : io_redirect

| cmd_suffix io_redirect

| WORD

| cmd_suffix WORD

;

redirect_list : io_redirect

| redirect_list io_redirect

;

io_redirect : io_file

| IO_NUMBER io_file

| io_here

| IO_NUMBER io_here

;

io_file : '<' filename

| LESSAND filename

| '>' filename

| GREATAND filename

| DGREAT filename

| LESSGREAT filename

| CLOBBER filename

;

filename : WORD /* Apply rule 2 */

;

io_here : DLESS here_end

| DLESSDASH here_end

;

here_end : WORD /* Apply rule 3 */

;

newline_list : NEWLINE

| newline_list NEWLINE

;

linebreak : newline_list

| /* empty */

;

separator_op : '&'

| ';'

;

separator : separator_op linebreak

| newline_list

;

sequential_sep : ';' linebreak

| newline_list

;

%token WORD

%token ASSIGNMENT_WORD

%token NAME

%token NEWLINE

%token IO_NUMBER

%token AND_IF OR_IF DSEMI

/* '&&' '||' ';;' */

%token DLESS DGREAT LESSAND GREATAND LESSGREAT DLESSDASH

/* '<<' '>>' '<&' '>&' '<>' '<<-' */

%token CLOBBER

/* '>|' */

/* The following are the reserved words. */

%token If Then Else Elif Fi Do Done

/* 'if' 'then' 'else' 'elif' 'fi' 'do' 'done' */

%token Case Esac While Until For

/* 'case' 'esac' 'while' 'until' 'for' */

/* These are reserved words, not operator tokens, and are

recognized when reserved words are recognized. */

%token Lbrace Rbrace Bang

/* '{' '}' '!' */

%token In

/* 'in' */

helper utilities
Helper utilities
  • Helper utilities – various small external programs that are helpful when working with shell scripts or command line
  • Called from shell (scripts or command line)
  • Somehow transforms input into output, based on the parameters
helper utilities1
Helper utilities
  • cat - concatenate files and print on the standard output
    • Syntax: cat [file1] [file2] … [fileN]

girtsf@linux etc $ cat gentoo-release shells

Gentoo Base System version 1.4.16

# /etc/shells: valid login shells

# $Header: /home/cvsroot/gentoo-src/rc-scripts/etc/shells,v 1.5 2003/07/15 20:36:32 azarah Exp $

/bin/sh

/bin/bash

/bin/tcsh

/bin/csh

/bin/esh

/bin/ksh

/bin/zsh

/bin/sash

helper utilities2
Helper utilities
  • echo – displays a line of text
  • Besides a program /bin/echo, also usually built in the shell (takes precedence)
    • Syntax: echo [STRING] ...

girtsf@linux girtsf $ echo quick brown fox

quick brown fox

Can be used to display environment variables

girtsf@linux girtsf $ echo $HOME

/home/girtsf

helper utilities3
Helper utilities
  • wc - print the number of newlines, words, and bytes in files
    • wc [options] [file1] [file2] … [fileN]
  • By default, newlines, words and byte counts are displayed
  • Options
    • -c : print only byte count
    • -w : print only word count
    • -l : print only line count
helper utilities4
Helper utilities
  • Example use of wc:

girtsf@linux etc $ wc /etc/passwd

50 76 2257 /etc/passwd

girtsf@linux etc $ wc -l /etc/passwd

50 /etc/passwd

lines words bytes

lines only

helper utilities5
Helper utilities
  • grep - print lines matching a pattern
    • grep PATTERN [file1] [file2] … [fileN]
  • The lines that contain PATTERN are printed to standard output.
  • If no files are specified, input is taken from standard input (more later).
  • Advanced versions of grep allow using regular expressions in PATTERN.
helper utilities6
Helper utilities
  • File “testfile” contains the following lines

girtsf@linux girtsf $ cat testfile

the quick brown

fox jumped over

the lazy dog

  • We search for “the”:

girtsf@linux girtsf $ grep the testfile

the quick brown

the lazy dog

  • Only lines containing the substring “the” are printed.
helper utilities7
Helper utilities
  • Some useful parameters for grep:
    • -i : ignore case (“the” finds “the”, “The”, “THE”,…)
    • -l : output only filenames that match, not the contents
    • -B <n> : output also n lines before the matching line
    • -A <n>: output also n lines after the matching line
  • See the man page (“man grep”) for all parameters
helper utilities8
Helper utilities
  • tee - read from standard input and write to standard output and files
  • Syntax: tee [File1] [File2] .. [FileN]
  • Example of tee taking user’s input from terminal and writing to 3 files:

girtsf@linux tmp $ tee a b c

some string^D

some string

girtsf@linux tmp $ cat a

some string

girtsf@linux tmp $ cat b

some string

girtsf@linux tmp $ cat c

some string

Inred – my input, ending with Control-D, which is the EOF (End of File) character.

This input is read as standard input by tee.

helper utilities9
Helper utilities
  • Any program can be used as a helper program
  • More examples later
connecting utilities with shell scripting
Connecting utilities with shell scripting
  • Standard I/O
  • I/O redirection to/from file
  • I/O redirection using a pipe
  • Backticks
standard i o
Standard I/O
  • Every process, when run, has 3 already open data streams (file descriptors):
    • Standard input
    • Standard output
    • Standard error
standard i o1
Standard I/O
  • When run interactively (from command line), these streams are attached to the terminal they are running from
    • Standard input is attached to user’s keyboard input
    • Standard output is attached to user’s terminal output
    • Standard error, similarly to output, is attached to user’s terminal output
  • Usually referred to as stdin, stdout, stderr.
standard output error
Standard output & error
  • “ls” command does not use input, but uses stdout, stderr.
    • The second line is the stdout from “ls” command:

girtsf@linux etc $ ls -l /etc/passwd

-rw-r--r-- 1 root root 2257 Oct 22 13:35 /etc/passwd

    • The second line is from stderr from “ls” command:

girtsf@linux etc $ ls -l /etc/asdfasdf

ls: /etc/asdfasdf: No such file or directory

    • Both stdout and stderr simultaneously:

girtsf@linux tmp $ ls -l /etc/passwd /etc/asdfasdf

ls: /etc/asdfasdf: No such file or directory

-rw-r--r-- 1 root root 2257 Oct 22 13:35 /etc/passwd

i o redirection to from file
I/O Redirection to/from file
  • By default, the 3 streams are attached to terminal
  • This can be overridden when executing the command and is called “redirection”
  • “>” specifies that stdout is redirected to file
  • “<“ specifies that stdin is taken from file
  • “2>” specifies that stderr is redirected to file
i o redirection to from file1
I/O Redirection to/from file
  • Syntax:
  • <cmd> [ > <file1>] [ < <file2> ] [ 2> <file3> ]
  • For those redirections that are specified, the respective stream will be attached to the specified file
  • None, one, two or all three types can be specified
  • If output file exists: > - replace file; >> - append to file
i o redirection to file
I/O Redirection to file
  • Example of stdout redirection to file

girtsf@linux tmp $ ls -l /lib/ > direktorijas_saraksts

girtsf@linux tmp $ cat direktorijas_saraksts

total 4035

-rwxr-xr-x 1 root root 7488 Oct 6 12:33 cpp

drwxr-xr-x 13 root root 1024 Oct 25 15:57 dev-state

drwxr-xr-x 2 root root 1024 Jun 28 09:53 evms

drwxr-xr-x 2 root root 2048 Aug 23 15:25 iptables

...

i o redirection to file1
I/O Redirection to file
  • Example of stdout redirection to file

girtsf@linux tmp $ ls -l /asdf > direktorijas_saraksts

ls: /asdf: No such file or directory

girtsf@linux tmp $ cat direktorijas_saraksts

  • The file is empty, as no output was sent to stdout, as error message was send to stderr, which still was attached to user’s terminal
i o redirection to file2
I/O Redirection to file
  • Example of stderr redirection to file

girtsf@linux tmp $ ls -l /asdfasdf 2> errlog

girtsf@linux tmp $ cat errlog

ls: /asdfasdf: No such file or directory

  • Now stderr was redirected to file and file contained the error message.
i o redirection to file3
I/O Redirection to file
  • Example of stdout, stderr redirection to file

girtsf@linux tmp $ ls -l /asdfasdf /lib 2>errlog >sar

girtsf@linux tmp $ cat errlog

ls: /asdfasdf: No such file or directory

girtsf@linux tmp $ cat sar

/lib:

total 4035

-rwxr-xr-x 1 root root 7488 Oct 6 12:33 cpp

drwxr-xr-x 13 root root 1024 Oct 25 15:57 dev-state

drwxr-xr-x 2 root root 1024 Jun 28 09:53 evms

drwxr-xr-x 2 root root 2048 Aug 23 15:25 iptables

...

i o redirection from file
I/O Redirection from file
  • Example of stdin redirection
    • First, we create file “a” with the following content

the quick brown fox

jumped over a quick brown fox

    • Use wc (word count) by not supplying the file name, but redirecting standard input

girtsf@linux tmp $ wc < a

2 10 50

i o redirection with pipes
I/O Redirection with pipes
  • Task: given a file, output the total number of words in those lines, that contain substring “the”.
  • Example input:

girtsf@linux girtsf $ cat testfile

the quick brown

fox jumped over

the lazy dog

  • Lines 1 and 3 match, total number of words = 6.
i o redirection with pipes1
I/O Redirection with pipes
  • Solution with redirection to files:
    • First find the lines, save them into temp file
    • Then use wc (word count) utility to count the number of words

girtsf@linux girtsf $ grep the testfile > tmpfile

girtsf@linux girtsf $ wc –w < tmpfile

6

i o redirection with pipes2
I/O Redirection with pipes
  • Temporary file was used to redirect the standard output of grep to file
  • The standard input to wc was taken from temporary file
  • Easier way – connect the standard output of one program to standard input of another one directly
i o redirection with pipes3
I/O Redirection with pipes
  • Syntax: program1 | program2 (| - pipe symbol)
  • Called “piping output from one program to another”
  • This example:

girtsf@linux girtsf $ grep the testfile | wc -w

6

  • No temporary files. Elegant!
backticks
Backticks
  • Backticks – reverse apostrophes “`” (usually the same key as tilde ~ character)
  • Using backticks sub-commands are executed, their result written in the place they are defined
  • Example:

girtsf@linux tmp $ cd `echo /home`

girtsf@linux home $

Substituted with “/home”

helper utilities10
Helper utilities
  • We will examine the following utilities:
    • cut
    • sort
    • uniq
    • awk
    • sed
slide41
cut
  • cut - remove sections from each line of files
  • Syntax: cut [OPTION]... [FILE]...
  • Options:
    • -d DELIM : use DELIM instead of TAB character
    • -f LIST : output only these fields (delimited by DELIM)
    • -c LIST : output only these characters
  • See man page for more options
slide42
cut
  • Example – output second word on each line:
    • Delimiter: space “ “
    • Fields: 2

girtsf@linux tmp $ cat a

the quick brown fox

jumped over a quick brown fox

girtsf@linux tmp $ cut -f 2 -d ' ' a

quick

over

slide43
cut
  • Example – output characters 1-3, 5, 7-end
    • Use –c to choose the needed characters

girtsf@linux tmp $ cat a

thequick brown fox

jumped over a quick brown fox

girtsf@linux tmp $ cut -c 1-3,5,7- a

theqick brown fox

jume over a quick brown fox

slide44
sort
  • sort - sort lines of text files
    • sort [OPTION]... [FILE]...
  • Writes sorted concatenation of all FILE(s) to standard output.
  • Interesting options:
    • -r : reverse
    • -n : compare according to string numerical value
  • See man page for more options
slide45
sort
  • sort - sort text file reversed

girtsf@linux tmp $ cat a

fish

dog

animal

bird

girtsf@linux tmp $ sort -r a

fish

dog

bird

animal

slide46
sort
  • Sort numeric file (as text)

girtsf@linux tmp $ cat a

5412 this line should go last

998 this line should go second

50 this line should go first

999 this line should go third

girtsf@linux tmp $ sort a

50 this line should go first

5412 this line should go last

998 this line should go second

999 this line should go third

slide47
sort
  • Sort numeric file as numbers

girtsf@linux tmp $ cat a

5412 this line should go last

998 this line should go second

50 this line should go first

999 this line should go third

girtsf@linux tmp $ sort -n a

50 this line should go first

998 this line should go second

999 this line should go third

5412 this line should go last

slide48
uniq
  • uniq - remove duplicate lines from a sorted file
    • uniq [OPTION]... [INPUT [OUTPUT]]
  • Discards all but one of successive identical lines from INPUT (or standard input), writing to OUTPUT (or standard output).
  • Can be used together with sort, to get file without duplicate lines.
slide49
Just sorted:

$ cat a | sort

bird

bird

dog

dog

fish

fish

fly

sort | uniq:

$ cat a | sort | uniq

bird

dog

fish

fly

uniq
slide50
awk
  • gawk (GNU awk) - pattern scanning and processing language.
  • Gawk is the GNU Project's implementation of the AWK programming language. It conforms to the definition of the language in the POSIX 1003.2 Command Language And Utilities Standard. This version in turn is based on the description in The AWK Programming Language, by Aho, Kernighan, and Weinberger, with the additional features found in the System V Release 4 version of UNIX awk.
slide51
awk
  • Complete language interpreter
    • Variables
    • User defined functions
  • Useful for small one-liners
  • Just some examples will be given
  • See man page or search for tutorials on the net
slide52
awk

Task: sum all the numbers in the file:

$ cat a

1

2

3

4

5

$ cat a | awk '{ sum += $1 } END { print sum }'

15

first field

executed on every line

executed at the end

slide53
awk
  • Sum 2nd field (separated by colons) of those lines, that contain letter “a”

girtsf@linux tmp $ cat a

zzz:1:a

zzz:2:b

zzz:3:b

zzz:4:b

zzz:5:a

girtsf@linux tmp $ cat a | awk -F ':' '{ if(/a/) sum += $2; } END { print sum }'

6

field delimiter

slide54
sed
  • sed – stream editor
  • A stream editor is used to perform basic text transformations on an input stream (a file or input from a pipeline). While in some ways similar to an editor which permits scripted edits (such as ed), sed works by making only one pass over the input(s), and is consequently more efficient.
  • Simpler than awk, smaller feature set.
  • Just some examples will be given.
  • See man page or search for tutorials on the net.
slide55
sed
  • Replace some substring with another

$ cat a

bird barks

mouse runs

$ sed 's/barks/flies/' < a

bird flies

mouse runs

Regular expression

slide56
sed
  • Replace some characters with others
    • Replacing ‘b’ with ‘Q’, ‘i’ with ‘X’

girtsf@linux tmp $ cat a

bird barks

mouse runs

girtsf@linux tmp $ cat a | sed 'y/bi/QX/'

QXrd Qarks

mouse runs

advanced example 1
Advanced example 1
  • Calculate total bytes transferred in Apache log file.
    • First, take only successful lines (containing error code 200)
    • Sum up byte field
  • Line format:

159.148.123.123 - - [28/Oct/2004:18:11:36 +0300] "GET /somefolder/file.php HTTP/1.1" 200 127602 "-" "Opera/7.54 (X11; Linux i686; U) [en]"

advanced example 1 cont
Advanced example 1 cont.

$ cat access_log | grep ' 200 ' | awk '{ bytes += $10 } END { print bytes }'

1105653994

  • Cat file
  • Grep for “ 200 “
  • Sum up 10th column, output it at the end
advanced example 2
Advanced example 2
  • Calculate number of hits per remote host in Apache log file, most active hosts first.
  • Line format:

159.148.123.123 - - [28/Oct/2004:18:11:36 +0300] "GET /somefolder/file.php HTTP/1.1" 200 127602 "-" "Opera/7.54 (X11; Linux i686; U) [en]"

advanced example 21
Advanced example 2

$ cat access_log | cut -d ' ' -f 1 | sort | uniq -c | sort –n -r

  • First, cut out the host part (1st field), sort it
  • get the number of repeated lines before the line (uniq –c : prefix lines by the number of occurrences),
  • sort it numerically, reversed so that largest number comes first
  • Output:

348698 159.148.111.222

123485 159.148.48.54

12313 80.123.123.4

...

helper utilities11
Helper utilities
  • Some more utilities that can be useful
    • head, tail - output the first/last part of files
    • basename - strip directory and suffix from filenames
    • bc - an arbitrary precision calculator
    • sleep – sleep specified number of seconds
    • tr – translate or delete characters
    • true, false – always return success/error code
  • Read the man pages
tips for working in a shell
Tips for working in a shell
  • Up arrow – repeats the previous command
  • history – internal bash command that shows command history
  • Use script to make a typescript of terminal session
  • CTRL-R and a few chars recalls last command that contains these chars:
    • (reverse-i-search)`re': grep text file*
the abcs of unix
A is for awk, which runs like a snail

B is for biff, which reads all your mail

C is for cc, as hackers recall

D is for dd, the command that does all

E is for emacs, which rebinds your keys

F is for fsck, which rebuilds your trees

G is for grep, a clever detective

H is for halt, which may seem defective

I is for indent, which rarely amuses

J is for join, which nobody uses

K is for kill, which makes you the boss

L is for lex, which is missing from DOS

M is for more, from which less was begot

N is for nice, which really is not

O is for od, which prints out things nice

P is for passwd, which reads in strings twice

Q is for quota, a Berkeley-type fable

R is for ranlib, for sorting a table

S is for spell, which attempts to belittle

T is for true, which does very little

U is for uniq, which is used after sort

V is for vi, which is hard to abort

W is for whoami, which tells you your name

X is, well, X, of dubious fame

Y is for yes, which makes an impression, and

Z is for zcat, which handles compression

The ABCs of Unix
another unix command refference
Another UNIX command refference
  • Use man pages for further details
shell scripts as files
Shell scripts as files
  • Everything that can be called from command line, can also be called from shell script
  • Shell Script is series of commands written in plain text file.
  • Shell script is just like batch file is MS-DOS but have more power than the MS-DOS batch file.
running a shell script
Running a shell script
  • As a parameter to shell interpreter
      • bash some_script.sh
  • Specifying interpreter on first line
      • First line:

#!/bin/bash

      • Make it executable (chmod +x some_script.sh)
      • ./some_script.sh
basics
Basics
  • Interpreted line by line
  • The same effect when entering the lines one by one in interactive shell
sample shell script
Sample shell script

Interpreter to be used

#!/bin/bash

# comment line

echo "what a fine day: "

date

  • Output, when called by “./test.sh”:

what a fine day:

Thu Oct 28 23:37:39 EEST 2004

Regular commands to execute

variables
Variables
  • Sample “hello world” with variables

#!/bin/bash

STR="Hello World!"

echo $STR

  • When assigning, no $ is used
  • When getting the contents – use $
  • No data types – string, number, character, all the same
  • No declaring, just assign
another sample shell script
Another sample shell script

#!/bin/bash

DATE=`date +%Y%m%d`

WHAT='/home/girtsf'

DEST="/backups/$DATE.tgz"

tar cvzf $DEST $WHAT

  • Results in calling:

tar cvzf /backups/20041028.tgz /home/girtsf

conditionals
Conditionals
  • Example:

#!/bin/bash

T1="foo"

T2="bar"

if [ "$T1" = "$T2" ]; then

echo expression evaluated as true

else

echo expression evaluated as false

fi

command line arguments
Command line arguments
  • Automatically defined variables
    • $0 – contains shell script name
    • $1 – contains first argument
    • $2 – 2nd
    • $* - contains all arguments as a string
command line arguments conditionals example
Command line arguments & conditionals example

#!/bin/sh

#

#Script to print file

#

if cat $1

then

echo -e "\n\nFile $1, found and successfully echoed"

fi

returns error code

usually 0 – no error

if executes branch if command returns 0

comparisons
Comparisons
  • Either “test <expression>” or “ [ <expression> ] “
    • if test $1 -gt 0
    • if [ $1 -gt 0 ]
  • Numeric comparisons:
    • -eq, -ne equal/not equal
    • -lt, -le, -gt, -ge : less than, less than or equal, greater than, greater than or equal
comparisons1
Comparisons
  • String comparisons
    • string1 = string2 string1 is equal to string2
    • string1 != string2 string1 is NOT equal to string2
    • string1 string1 is NOT NULL or not defined
    • -n string1 string1 is NOT NULL and does exist
    • -z string1 string1 is NULL and does exist
file tests
File tests
  • -s file Non empty file
  • -f file File exist or normal file and not a directory
  • -d dir Directory exist and not a file
  • -w file Is writeable file
  • -r file Is read-only file
  • -x file File is executable
logical operators
Logical Operators
  • ! expression Logical NOT
  • expression1 -a expression2 Logical AND
  • expression1 -o expression2 Logical OR
another example
Another example

#!/bin/sh

# Script to test if..elif...else

#

if [ $1 -gt 0 ]; then

echo "$1 is positive"

elif [ $1 -lt 0 ]

then

echo "$1 is negative"

elif [ $1 -eq 0 ]

then

echo "$1 is zero"

else

echo "Opps! $1 is not number, give number"

fi

arithmetic expansion
Arithmetic expansion
  • Arithmetic expansion allows the evaluation of an arithmetic expression and the substitution of the result. The format for arithmetic expansion is:
    • $((expression))

A=5

B=4

C=$(($A*$B))

echo $C

loops for
Loops: for

for { variable name } in { list }

do

execute one for each item in the list until the list is not finished (And repeat all statement

between do and done)

done

for i in 1 2 3 4 5

do

echo "Welcome $i times"

done

loops for1
Loops: for

for (( expr1; expr2; expr3 ))

do

repeat while expr2 is true

done

for (( i = 0 ; i <= 5; i++ ))

do

echo "Welcome $i times"

done

using for
Using for
  • You can use for together with file name expansion to perform the same action for several files

#!/bin/sh

for x in *txt;

do

cat $x

done;

loops while
Loops: while

while [ condition ]

do

command1

command2

command3

....

done

many more features
Many more features
  • See man page
  • Linux Shell Scripting Tutorial: A Beginner's handbook:
    • http://www.freeos.com/guides/lsst/
word guessing game
Word guessing game
  • Source:
    • http://www.ltn.lv/~guntis/unix/mini.txt

+------

|/ |

| o

| O

| /

|

MATER_AL

Juusu mineejums: i

main part
Main part

izveleties_vardu

stripas

gaj=0

while true;

do

zimet_karatavas $gaj

echo $v

echo -n "Juusu mineejums: "

read iev

gajiens $iev

if [[ $v == $vards ]]; then

uzvara; exit

fi

if [[ $gaj -eq 10 ]]; then

zaude; exit

fi

done;

getting a random word
Getting a random word

izveleties_vardu()

{

local sk r

if [[ -e /usr/share/dict/words ]];

then

echo Njemu vienu nejaushu vaardu no /usr/share/dict/words!

sk=`cat /usr/share/dict/words|wc -l`

r=$(($RANDOM % $sk + 1))

vards=`tail +$r /usr/share/dict/words|head -1|tr a-z A-Z`

else

echo /usr/share/dict/words nav atrasts, njemu nokluseeto vaardu!

vards="KARATAVAS"

fi

sleep 1

}

convert letters to underscores
Convert letters to underscores

stripas()

{

local i

i=${#vards}

v=

while [[ $i>0 ]];

do

i=$(($i-1))

v=${v}_

done

}

comparison part
Comparison part

while [[ $i < $l ]];

do

t=${vards:$i:1}

if [[ $t == $b ]];

then

v2=${v2}$b

ir=$(($ir+1))

else

v2=${v2}${v:$i:1}

fi

i=$(($i+1))

done