unit 3 shell programming n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Unit -3 Shell Programming PowerPoint Presentation
Download Presentation
Unit -3 Shell Programming

Loading in 2 Seconds...

play fullscreen
1 / 91

Unit -3 Shell Programming - PowerPoint PPT Presentation


  • 128 Views
  • Uploaded on

Unit -3 Shell Programming. What is the Shell. Shell is a user level program. It is an command language interpreter that executes commands read from the standard input device (keyboard) or from a file.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Unit -3 Shell Programming' - ogden


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
what is the shell
What is the Shell
  • Shell is a user level program.
  • It is an command language interpreter that executes commands read from the standard input device (keyboard) or from a file.
  • It is not part of system kernel, but uses the system kernel to execute programs, create files etc.
  • In MS-DOS, Shell name is COMMAND.COM which is also used for same purpose, but it's not as powerful as Linux Shells are!
slide3

Common Shells

  • Sh(1971) – Developed by Ken Thompson

- Introduced the concept of pipes, filters and redirection

Lacked the ability to script.

  • Bourne (1977) – Developed by Stephen Bourne

- Introduced the concepts of scripting, control flows, string literals and

command substitution

- Lacked the ability to define functions

  • Korn (1978) – Developed by David Korn (also called ksh)

- Includes features from other shells

- Includes advanced features from Ruby and Python languages like

floating point arithmetic and arrays

- File name completion

- History feature

slide4

Common Shells

  • C (1978) – Developed by Bill Joy (also called csh)

- Provides C language feel

- Advanced feature is Command history

  • Tenex C (1983) – Developed by Ken Greer (also called tcsh)

- Command line editing feature

- Command completion

  • Bourne Again Shell – Developed by Brijan Fox (also called bash)

- It combines the features of Korn and C shells

- Introduced some environment variables

- Default shell in Linux.

- specific shell variables.

- some built-in commands like set and man.

etc shells
/etc/shells

The file /etc/shells lists the shells available under Linux.

  • It contains a list of all the installed shells, available to all users
  • The first entry acts as the default shell

Note : that each shell does the same job, but each understand a different command syntax and provides different built-in functions.

login logout scripts

Shell

Scripts

Bash

.bash_profile: Login initialization file

.bashrc: BASH shell configuration file

.bash_logout: Logout script

Tcsh

.login: Login Initialization File

.tcshrc: TCSH shell configuration file

.logout: Logout file

Login/Logout Scripts

.cshrc .login (executed only at login)

.logout (executed only at logout)

Csh

shell responsibilities
Shell: Responsibilities
  • Command Interpretation
  • Providing a scripting language
  • A process that provides runtime environment
simple architecture of a shell
Simple architecture of a Shell

User

Lexical Analysis

Expansion

Execution

Kernel

interpretation cycle for shell commands
Interpretation cycle for shell commands

The Shell

  • Issues the prompt and waits for a command to be entered
  • Analyses the input and parses it
  • Scans the command line for meta-characters and performs expansions to give a simple command
  • Passes command line to the kernel for execution
  • Waits for command to execute
  • On completion gives the prompt again; and the cycle continues
shell variables
Shell: Variables
  • Control the environment provided by the shell
  • Set via any of the following activities
    • After user logs in through login scripts
    • As a result of scripts executed after login
    • Interactively by user
  • Also known as Environment / System variables
  • .bash_profile initialization file contains definitions and assignments of parameter variables.
shell variables1
Shell: Variables
  • Each shell variable is used for a specific purpose
  • A shell variable is typically entered in uppercase
  • To get its value, you precede the shell variable name with a $, as in $PWD
  • The set command shows the complete list
learning about shell variables
Learning about shell variables

Examples of shell variables:

  • PWD – the most recent working directory set with the cd command
  • OLDPWD – the previous working directory set with the cd command
  • BASH – the full path of the bash shell
  • HOME– each user’s home directory, typically /home/username, where username is a user’s login name
  • HOSTNAME – the current hostname of a Linux system
expansion
Expansion
  • Expansion is the process of using meta characters and special symbols to change the given text
  • Simple examples:
    • Variable Expansion
      • $HOME expands into current user’s home directory,
      • $BASH expands to give the full path of the bash shell
  • The types of expansion discussed:
    • Parameter and variable
    • Command substitution
parameter and variable expansion
Parameter and Variable Expansion
  • Parameter expansion substitutes values for parameter or variable names
  • Parameters and variables are names that contain data values
  • Example:
    • echo $x will display the contents of a variable named x
    • Notice=“Meeting will be held tomorrow”
    • Echo $Notice

Use of braces {…} allows one to mix variables and numbers. Example:

echo ${EmployeeName} will display an employee’s name on the screen

expansion command substitution
Expansion – Command Substitution

Command substitution allows you to substitute the output of a command in place of the command itself

Two forms:

  • Use $(command)

Example: echo "Today is " $(date)

  • Surround the command with a single back quote as in `command`

Example : Mycmd = `ls *.c`

echo $Mycmd

shell script
Shell Script
  • Normally shells are interactive i.e. shell accept command from you (via keyboard) and execute them.
  • But if required a set of related commands can be run by storing them to text file and telling the shell to execute this text file.
  • This is known as shell script.
  • Definition:
    • "Shell Script is series of command written in plain text file. Shell script is just like batch file is MS-DOS but have more power than the MS-DOS batch file."
why to write shell script
Why to write shell script?
  • Shell script can take input from user, file and output them on screen.
  • Useful to create our own commands.
  • Multiple commands can be bound by means of control constructs to define a logical flow of control.
  • Shell scripts execute commands sequentially.
  • You can alter the sequential order with decisions, loops and functions
  • Shell scripts
    • combine programming logic with operating system commands
    • help in automating job duties
creating shell scripts
Creating shell scripts
  • Use any editor like vi or gedit to write shell script.
  • After writing shell script set execute permission for your script as follows
  • Syntax:
    • chmod permission script-name
    • Examples:$ chmod +x script-name$ chmod 755 script-name
  • Note:This will set read write execute(7) permission for owner, for group and other permission is read and execute only(5).
executing shell scripts
Executing shell scripts

syntax:

$ shscript-name or$ ./script-name

Examples:

$ shscr or$ ./scr

user defined variables
User defined variables

Syntax: variable name=value

Rules for naming variables:

  • Variable name must begin with Alphanumeric character or underscore character (_)
  • Don't put spaces on either side of the equal sign when assigning value to variable
  • Variables are case-sensitive, just like filename in Linux
  • NULL variable (variable which has no value at the time of definition) can be defined as:
    • $ vech=
  • Do not use ?,* etc, to name your variable names

Variable value can be accessed using $vname

shell arithmetic
Shell Arithmetic

Syntax:

  • expr op1 math-operator op2Examples: $ expr 1 + 3$ expr 2 - 1$ expr 10 / 2$ expr 20 % 3$ expr 10 \* 3$ echo `expr 6 + 3`
exit status
Exit Status

In Linux when a particular command/shell script is executed, it returns two type of values, signifying success or failure

This value, returned via return statement, is known as Exit Status.

These values can be used to check whether command or shell script executed successfully or not.

  • If return value is zero (0), command is successful.
  • If return value is nonzero, command is not successful or some sort of error executing command/shell script.

$? can be used to find the exit status of last executed command.

exit status example
Exit status : example

For e.g. (This example assumes that unknownfile does not exist )

$ rm unknownfile

It will show error as followsrm: cannot remove `unkownfile': No such file or directoryand after that if you give command

$ echo $? it will print nonzero value to indicate error

The value of $? can be used in scripts for decision making

shell grammar i
Shell Grammar - I

A blank is a space or tab used to separate items

A word (token) is a sequence of characters considered to be a single unit to the shell

A name (identifier) is a word consisting of letters, numbers and underscore

A meta character is a character that, when unquoted, separates words

A Control Operator is a token that performs control function.

shell grammar ii
Shell Grammar - II

Metacharacter: Examples

  • | - the pipe symbol allows you to pass output to another command
  • & - the ampersand allows you to run a process in the background
  • ; - the semicolon allows you to sequence commands
  • <- less than redirects input
  • > - greater than redirects output
slide26

Shell Grammar - III

Control operators:

  • || - executes a command depending upon failure of another
  • && - executes a command depending upon success of another

A reserved word has special meaning to the shell and cannot be used for another purpose. Example: if, then, else, fi

slide27

Command Separators

The && Operator

  • Causes a command to execute only if the preceding command completes successfully (exit status of 0)
  • Example: rm file4.txt && ls
    • The ls command will only execute if rm file4.txt completes successfully

The || Operator

  • Causes a command to execute only if the preceding command completes unsuccessfully (exit status of 1)
  • Example: rm file4.txt || pwd
    • The ls command will only execute if rm file4.txt completes unsuccessfully
operators
Operators

Logical

  • -a: and
  • -o: or
  • !: not

Comparison

  • -eq, -ne, -lt, -gt, -le, -ge: Numerical Comparison
  • =, != : String comparison
  • -z: check string against null
if statement
if statement

The shell requires you to follow strict syntax when implementing the if statement

The syntax of the if statement follows:

if [condition]

then

Statements

else

Statements

fi

if statements can be nested if required

elif clause in an if statement
elif clause in an if statement

The optional elif clause allows you to further test an if statement

if [condition]

then

Statements

elif [condition]

then

Statements

else

Statements

fi

the case statement
The case statement

Another decision structure

Use when a decision is based upon multiple inputs

Syntax:

case word in

Pattern1)

statements;;

Pattern2)

statements;;

esac

Can use * to match all patterns, ? to match a single character or […] for a range.

the while statement
The while statement

A set of statements is executed till a condition evaluates to true

while [ condition ]

do

statements

done

the while statement1
The while statement

Use the break command to end a loop prematurely.

Example:

count=1

while [condition]

do

statements

if [ $count –gt 3 ]

then

break

fi

statements

done

the until statement
The until statement

A set of statements is executed till a condition does not evaluate to true

until [ condition ]

do

statements

done

the for statement
The for statement

The for statement syntax in the shell takes two forms:

  • Word list form:

for variable in list

do

statements

done

  • Arithmetic expression form:

for ((var=init; test; incr))

do

Statements

done

slide38

Understanding functions

  • Functions are self-contained blocks of code
  • Functions can accept values and return a result
  • Function code can be reused
  • Functions reduce redundancy
  • Functions provide for modular code
slide39

Components of a function

  • General format of a function in the shell:

function function-name ()

{

list

}

  • Function name is required
  • Either the keyword “function” or the parentheses are required but can use both
  • The left and right braces are required
  • The commands in list are any valid shell commands
slide40

Function call

Calling a function:

  • Done by using its name

Example:

function DisplayHello ()

{

echo "hello"

}

DisplayHello

slide41

Function basics

  • Functions are usually placed at the beginning of the script or prior to the function call
  • A function called prior to its definition will generate a “command not found” error
  • Functions are executed in context of the current shell (unlike shell scripts)
  • Functions can be exported to subshells using export -f
slide42

Function parameters

  • On execution the parameters become the positional parameters
  • $0 remains unchanged
  • Variables are shared between function and its caller
  • return returns control back to the caller & the positional parameters are restored to their original value
  • Variables local to the function can be declared using local keyword
slide43

Function : Example

function testFunc ()

{

for i in $*

do

echo $i

done

}

for i in $*

do

echo $i

done

testFunc 1 2 3

slide44

The return statement

  • Use the return command to cause a function return an exit status
  • General format:
    • returnn
  • Where n is the return status you specify
  • This value can be checked using $?
  • Typically 0 for success and non-zero for indicating cause of failure
slide45

Arrays

  • Array is a variable containing a series of elements
  • Refer to elements using an integer called a subscript
  • Subscripts begin with 0
  • Use an array when you have values that are similar in nature
  • All the data in the array can be accessed by referencing the whole array variable
  • A single element can be accesses by using its subscript value
slide46

Declaring an array

  • Use the declare -a command to declare an array
  • Can use typeset -a command too
  • Example to declare an array named courses:
    • declare -a courses
    • declare -ia codes
slide47

Assigning values to array elements

  • After declaring an array, you need to assign values to it
  • Example to set the first to elements of the courses array:
    • courses[0]="MCA"
    • courses[1]="B Tech."
  • Another method is to assign multiple elements as in:
    • courses=("MCA" "B Tech. ")
    • codes=(10 20 30)
  • Space acts as the element separator
slide48

Using array elements

  • Syntax to refer to a single element of an array:
    • ${array-name[subscript]}
  • Example to display the sixth element (subscript 5) in the courses array:
    • echo ${courses[5]}
  • Using array elements as rvalue:
    • course=${courses[5]}
  • The expression $courses[5]
    • is treated as $courses followed by[5]
slide49

Operations on an array

  • Reference all of the array elements:
    • echo ${array-name[*]} or echo ${array-name[@]}
    • Example: echo ${courses[*]} or echo ${courses[@]}
  • Determine the number of array elements:
    • echo ${#array-name[*]} or echo ${#array-name[@]}
    • Example: echo ${#food[*]} or echo ${#food[@]}
  • Determine an element’s length:
    • echo ${#array-name[subscript]}
    • Example: echo ${#food[2]} will give the length of 3rd element in the array.
    • Similarly echo ${#array} will give the length of the first lement in the array i.e. ${#array[0]}
slide50

Getting values from the user

  • It is possible to use arithmetic operations and variable name substitution to determine a subscript number
  • Reading i-th element from user:
    • read -p "Enter $i the element" arr[$i]
  • Displaying i-th element to user:
    • echo ${arr[$i]} " "
slide51

Operations on an array

  • Search and Replace array elements
  • Eg : echo ${unix[@] / mandriva / RedHat}

will replace ‘mandriva’ with ‘RadHat’. But this will not permanently replace the array relements.

slide52

Operations on an array

  • Add an element to an existing bash array :

array=(“${array[@]}” “new item”)

  • Copying an array

newarray = (“${oldarray[@]}”)

slide53

Operations on an array

  • Concatenate two arrays :

array3 = (“${array1[@]}” “${array2[@]}”)

  • Deleting an entire array :

unset arrayname

echo ${#array[@]} # would give 0.

slide54

which command

which <command>

Can be used to locate executables that are in one of the directories contained in the PATH variable

It prints the full path of the executable that would get executed in the command was given on the command prompt.

which -a commandprints all the occurrences of the command; not just the first one

slide55

grep

  • Used for matching patterns
  • Pattern: built using regular expressions
  • Options:
    • -i: Ignore case
    • -v: Don't display matching lines
    • -n: Display line numbers along with lines
    • -c: Display count of occurrences
    • -l: Display list of filenames that contain the pattern that is searched in a set of files
  • Variations
    • egrep : Equivalent to grep -E: use Extended Regular Expressions
    • fgrep: Equivalent to grep -f: read patterns from file
slide56

Basic regular expressions

  • A*: Look for 0 or more occurrences of A
  • . : Look for any one character (?)
  • .* : Look for any number of characters
  • ^A : Look for A in beginning of string
  • A$ : Look for A at the end of string
  • ^A$ : Look for A
  • [abc]: Look for a single character a, b or c
  • [a-d]: Look for a range
  • [^a-d]: Look for a character not in this range range
slide57

Extended regular expressions

  • A+ : Matches one or more occurrences of A
  • A? : Matches zero or one occurrence of A
  • exp1 | exp2 : Matches exp1 or exp2
  • ( e1 | e2 ) e3 : Matches e1e3 or e2e3
  • Use egrep or grep -E to make use of Extended Regular Expressions
slide58

Stream Editor - sed

  • A multipurpose tool
  • Combines work of several filters
  • Performs non-interactive operations on a data stream (file or input from a pipeline)
  • Uses instructions to work on text
  • An instruction combines:
    • An address for selecting lines of input
    • Action to be taken on them
  • General form:
  • sed options 'address action' file(s)
  • The 'address action' pair is enclosed in single quotes
slide59

Addressing and actions

Addresses in sed can be specified in two ways:

  • By specifying line numbers: single or range
  • By specifying a pattern enclosed in / /

Actions (commands) in sed are defined by sed's internal family of command.

  • Can be used for
    • Substitution (s)
    • Global substitution (g)
    • Duplication (p)
    • Deletion (d)
    • Stop searching (q)
    • Append (a)
    • Replace lines (c)
slide60

Text Editing : Substituting text

Command: s

Usage:

sed 's/exp1/exp2/' filename

  • First occurrence of exp1 will be changed to exp2 in all the lines of text

sed 's/exp1/exp2/g' filename

  • All occurrences of exp1 will be changed to exp2 in all the lines of text

sed '---s/exp1/exp2/' filename

  • Change will be made in lines addressed by ---

sed 's/^/exp2/' filename

  • Will prefix exp2 in beginning of all lines

sed 's/$/exp2/' filename

  • Will prefix exp2 at the end of all lines
slide61

Text Editing : Substituting text

  • Some examples :
  • $ sed ‘s/mr./Mr./g’ < file1 > file2

will replace all occurrences of mr. with Mr. and place it in file2

  • $ sed ‘1,3 s/mr./Mr./’ file

will make changes only in lines from 1 to 3

  • $ sed ‘5,$ s/yahoo/google/’ file

will replace yahoo with google in lines from 5 till end

  • Echo “hello how are you” | sed ‘s/h/H/g’

will give ouptput as : Hello How are you

slide62

Line Addressing : Example

‘q’ option is used to stop searching after a particular line.

sed '3q' <filename': Prints and quits after 3rd line

sed -n '3p' <filename : Prints the third line

Addressing rules:

  • Num : Single line
  • Num1, Num2: Range of lines
  • $: Last line
  • ! for negation: Used before command

Use -e for combining multiple commands

sed -n -e '1,2p' -e '5,6p' <filename>

Will print lines from 1 to 2 and 5 to 6

slide63

Appending and duplicating lines

  • For appending lines

sed ‘/pattern/ a\line to append’ filename

  • e.g.
  • sed ‘3 a\From here the 4th line begins’ file

The above command will append a line after 3rd line.

  • For duplicating lines

sed ‘/^$/p’ file

will duplicate all empty lines.

sed ‘1,5 p’ file

will duplicate lines from 1 to 5

slide64

Text replacement

‘c’ option is used to replace lines in a file

sed ‘/pattern/ c\new line’ file

Examples:

sed ‘1 c\This is a new line’ filename

will replace the first line with new line

slide65

Text Editing : Deleting lines

Command: d

Can be used with line / context addressing

sed '/pattern/d' filename

sed '1,3d' filename

Will remove lines from 1 to 3

Sed ‘1, /^ $/d’ file

Will remove header of a file until the first blank line.

slide66

The awk

  • An advanced filter
  • Available in Linux as gawk (GNU awk)
  • Named after its authors:
    • Aho, Weiberger and Kernigan
  • Important Features
    • Operates at field level, i.e. Can easily access, transform and format individual fields
    • Can work on extended regular expressions (ERE)
    • Has C-like programming contructs
    • Supports user defined variables
    • Allows usage of arrays and functions
slide67

Simple filtering using awk

  • Like sed, awk can be used to filter out lines (records) on basis of
    • Line number; or
    • Pattern
  • Syntax
  • awk options 'selection_criteria {action} files(s)
  • Selecting on basis of pattern (fixed or regular expressions)
  • awk '/pattern/ {print}' filename
  • Selecting on basis of line number
  • awk 'NR==n1 {print}' filename
  • awk 'NR==n1 {print}' filename // prints range of lines
slide68

Processing individual fields

  • awk identifies fields using special variables $1..$n
  • $0 signifies the entire record
  • These variables are different from the varibles as used by shell to idntify command line paramerters
  • awk 'NR==2 { print $3, $4 }' file
  • Specify field separator using -F
  • awk -F “:” 'NR==2 { print $3, $4 }' file
slide69

Generating formatted output

awk uses C-like format specifiers; as used in the printf function to generate formatted output.

Syntax:

awk '{ printf “%5d %10s”, $2, $5 }' filename

Output

  • The 2nd field as a 5 column wide number
  • The 5th field as a 10 column wide string

The output generated by individual print commands can be redirected to different files / commands enclosed in double quotes

awk '{ printf “%5d %10s”, $2, $5 > “f1” }' filename

awk '{ printf “%5d %10s”, $2, $5 | “wc” }' filename

slide70

Arithmetic Operators

Computation can be generated on numeric fields using the basic arithmetic operators: + , - , * , / , %

These operators can be used for both

  • Providing selection criteria
  • Generating Output

The operations for incrementing / decrementing (pre as well as post) are also available.

slide71

User defined variables

User defined variables in awk

  • Are created automatically without declaration
  • Are initialized to 0 to NULL depending upon context.

Can be used in statements for performing arithmetic, comparison or output operations

As awk commands become more and more complex; the -f option can be used to fetch an awk program from a file:

awk -f <awk command file name> <input file name>

slide72

Built-in Variables

The built-in variable available with awk are listed as follows:

NR: Cumulative number of lines read

FS: Input field separator

OFS: Output field separator

NF: Number of fields in current line

FILENAME: Current input file

ARGC: Number of arguments in command line

ARGV: List of arguments

slide73

BEGIN and END sections

Two special sections available with awk can be used for initialization and cleanup respectively.

One can use BEGIN for

  • giving initial values to variables
  • Printing headers; etc

Important because the commands given in {} are executed once for each line of input

Syntax:

awk 'BEGIN{--} {--} END {--}' filename

slide74

Using functions

Awk provides various functions for performing string and arithmetic operations

The parameters are passed using C-like syntax

Special case: parenthesis can be omitted when no parameters are being passed

Common functions:

  • int (x)
  • sqrt (x)
  • Length / length (x)
  • substr (str, start, len)
  • index (s1, s2): look for s2 in s1
  • split (str, arr, ch): split str and store into arr, use ch as delimiter
  • system(“cmd”)
slide75

Few Additional Commands

  • tr flags set1 set2
  • mail
  • diff, cmp, comm
  • cut
  • uniq
  • split
  • sort
  • head, tail
  • join
slide76

Using tr

  • The tr command allows you to translate or delete characters
  • Reads from Standard Input and write to Standard Output
    • “-t” : translates characters of its input that is in first set to the corresponding character in second set
    • “-d” : deletes those characters of its input that appear in the first set.
    • “-s” : replaces multiple consecutive occurrences of characters that are in set one with a single occurrence
  • Symbols that can be used are:
    • Escape sequences, ranges, character sequences, character classes
    • Character Classes: alnum, alpha, digit, lower, upper...
    • Example: tr [:lower:] [:upper:]to translate lower case letters to uppercase
slide77

Using mail

  • Can be used to send / receive mails.
  • Sending mail : mail <username>
  • Reading Mail : mail
    • Prints one line header of each message
    • Interactive Commands
        • # to read a particular mail
        • +, - to move back & forth
      • d to delete; u undelete
      • r to reply
      • q to quit
slide78

Using diff

Gives a formatted output after comparing

  • Two files
    • Gives commands that can be applied to convert one file to other
  • A file and a directory
    • Compares the given file with a file in the given directory having the same name.
  • Two directories
    • Compares corresponding files in both directories
    • “-r” for recursive comparison

Diff <file1> <file2>

slide79

Using cmp

  • Compares files
  • Silent if files are same
  • If they are different: reports the byte and line number at which first difference occurs.
  • Options
    • “-l” Print byte number & differing byte values for each difference
    • “-s” Silent mode, gives no output just returns status
    • “-i n:m” skip first n bytes of file 1 and m bytes of file 2
    • -”n” Compare at most n bytes
    • Can specify offsets for both files if required.
    • Cmp <options> <file1> <file2>
slide80

Using comm

Compare sorted files

Three sets of outputs

  • Lines unique to first file
  • Lines unique to second file
  • Lines in both files
  • “-n” to suppress nth column of output.

comm <file1> <file2>

slide81

Using cut

Extract sections of a file based upon a delimiter

Default delimiter: TAB

Options

  • “-f” specify field list
  • “-c” specify column list
  • “-d” specify delimiter
  • “-s” suppress lines that don't contain the delimiter

Example

cut -f1,4 -d”:” -s datafile

slide82

Using uniq

Extract unique / duplicate lines from a sorted file

Options

  • “-i” : ignore case
  • “-f” : ignore n fields while comparing
  • “-s” : ignore n characters while comparing
  • “-c” : print repeat count
  • “-d” : print only duplicate entries
  • “-u” : print only unique entries

Example

uniq -f2 -cd datafile

slide83

Using split

Split a file and generate sub-files

Options

  • “-a” : specify suffix length (default 2)
  • “-b” : specify bytes per output file (b:512, k:kilo, m:mega)
  • “-l” : specify lines per output file
  • Can give a prefix for files to be generated as the last option

Example

split -b k <filename> <prefix>

slide84

Using sort

Generate sorted output after reading a set of files

Options

  • “-b” : ignore leading blanks
  • “-f” : fold (ignore) case
  • “-n” : numeric sort
  • “-r” : reverse sort
  • “-c” : check whether input is sorted
  • “-k” : specify sort keys
  • “-t” : specify separator
  • “-o” : specify output file

Example

sort -k4,4 -k3 datafile -o outputfile

slide85

Using head

Extract top N bytes / characters from a set of files.

Options:

  • “-c N” : First N bytes
  • “-n N” : First N lines
  • “-q” : Quiet mode – Don't print file name headers
  • “-v” : Verbose mode – Print file name headers

Example:

head -n 5 datafile

slide86

Using tail

Extract top N bytes / characters from a set of files.

Options:

  • “-c N” : First N bytes
  • “-n N” : First N lines
    • If N is written as +3 => extract all after 3rd line / byte
    • If N is written as -3 => extract last 3 lines / bytes
  • “-q” : Quiet mode – Don't print file name headers
  • “-v” : Verbose mode – Print file name headers

Example:

tail -n 5 datafile

slide87

Using join

  • Perform join operation on two files treating each line as a record and each column as a field.
  • A white space separates fields by default.
  • If join fields are not specified then the operation is performed using first field of each file.
  • An output record (line) is generated for each pair of input records (lines) with identical join fields.
slide88

Using join

  • Options:
  • “-1 N -2 M” : perform join using Nth field of first file and Mth field of second file.
  • “-a SIDE” : print unpairable lines from SIDE
  • “-v SIDE : same as “a”; suppresses joined lines
  • “-e TEXT” : replace empty fields with TEXT
  • “-i” : ignore case
  • “-t CHAR” : specify field separator
  • “-o” : specify output format
  • Example:

join -1 2 -2 1 f1 f2 -o 1.2,1.3,2.3

short question
Short Question

Q.1: Write a shell script which renames all .txt files as .text files.

Q.2: Explain different type of shells in Linux. Write a shell script to generate Fibonacci series?

Q.3:Write a shell script which takes a name as parameter and returns the PID (s) of the processes with that name.

Q.4: What are shell programming functions. Write a function that adds two numbers.

Q.5: Explain the usage of the test command for checking the type of a file.

long questions
Long Questions
  • Write a shell script which takes a name as parameter and returns the PID(s) of processes with that name.
  • Write a shell program to evaluates an arithmetic expression like: ./calculator 10 + 20 ,where 10 is op1 and + is operator and 20 is op2. Using command line argument and operator may be (+,-,*,/ )
  • Write a shell script for copying files. where the source file exists and has read permission . Check for it else display error message. Also check the target file should not exist in the directory
references
References:
  • Sumitabha Das, “Unix Concepts and Applications”, TMH, 4th Ed., 2009.
  • Arnold Robbins, “Linux Programming by Examples The Fundamentals”, Pearson Education, 2nd Ed., 2008.