Cps 393 introduction to unix and c
This presentation is the property of its rightful owner.
Sponsored Links
1 / 24

CPS 393 Introduction to Unix and C PowerPoint PPT Presentation


  • 41 Views
  • Uploaded on
  • Presentation posted in: General

CPS 393 Introduction to Unix and C. START OF WEEK 3 (UNIX). 11/15/2014. Course material created by D. Woit. 1. Comment on quotes. Enclosing characters in single quotes preserves the literal value of each character in quotes.

Download Presentation

CPS 393 Introduction to Unix and C

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Cps 393 introduction to unix and c

CPS 393Introduction to Unix and C

START OF WEEK 3 (UNIX)

11/15/2014

Course material created by D. Woit

Course material created by D. Woit

1


Comment on quotes

Comment on quotes

  • Enclosing characters in single quotes preserves the literal value of each character in quotes.

  • Enclosing characters in double quotes preserves the literal value of all characters within the quotes with the exception of ‘$’, ‘`’, ‘\’ and when history is enabled ‘!’.

  • Example:

    • varx=1

    • echo $varx

      • 1

    • echo “$varx”

      • 1

    • echo ‘$varx’

    • $varx


Cut paste commands

Cut, Paste Commands

  • cut extract specified input columns

  • paste combine input columns

  • both take input from stdin or files

    • Examples:

    • cut -c8 cutexample # outputs 8th column of each line of myfile

    • cut -c5-7,25- # outputs columns 5,6,7,25... of each line of stdin

    • cut -f2 table # outputs second field of each line of file table (default field separator for –f switch is tab)

    • cut -d' ' -f3 # outputs field 3, fields delimited by one space (here field separator is changed from tab to space)

    • paste file1 file2 # linei to stdout contains linei of f1 followed by a tab # followed by linei of f2

  • cat –A shows tabs, newlines etc.

  • expand can convert tabs to spaces

11/15/2014

Course material created by D. Woit

Course material created by D. Woit

3


Cps 393 introduction to unix and c

HMWK

  • Create a file called stdnt-file with last-names in column 1, first-names in column 2, id numbers in column 3, userids in column 4.What combination of commands (and possibly temp files) can you use to create a file, stdnt-ids, that contains only the id numbers and userids from stdnt-file?

  • How can you combine commands (and possibly temp files) to create another file, stdnt1, that contains only the first and last students in stdnt-file?

  • How can you combine commands to find out exactly how long userid dwoit (or whoever you want) has been idle (has not been typing anything)? You will find the "w" command useful. It tells you who is logged in, and what they're doing. For an explanation do: man w. Note that "dwoit" (or whoever) will likely have multiple entries (one for each "terminal" connection), so your output may have multiple lines.

11/15/2014

Course material created by D. Woit

Course material created by D. Woit

4


Transformers and translators

Transformers and Translators

  • sed (stream editor)

  • Example change string unix into UNIX

    • sed -e "s/unix/UNIX/" sedexample

    • -e says the string following is an edit command

    • the "s" in the string says substitute last string for first

      • But only its first occurrence in line

    • sedexample is name of file to edit

    • output to stdout (but could redirect with > or >>)

    • input is taken from stdin or files

    • Regular expressions (like in grep) can be used.

11/15/2014

Course material created by D. Woit

Course material created by D. Woit

5


Sed stream editor cont

sed (stream editor) cont.

  • sed -e "s/dog/cat/g“ sedexample

  • changes all occurrences of "dog" to "cat" on each line (g=global)

  • sed -e "s/xx*/1234/" fx

    • changes first occurrence of x or xx or xxx etc on a line to 1234 in file fx

    • Character before * , i.e. x* can occur 0 or more times.

  • sed -e "1,5s/HI/hi there/"

    • changes "HI" to "hi there" on lines 1-5 of stdin

  • sed -e "/UNIX/ s/shell/Shell/" f1.txt

    • changes first "shell" to "Shell" only in lines containing the string "UNIX"

  • sed -e "/unix/ d"< myfile#deletes all lines containing string unix

  • sed -e "1,5d"< myfile delete lines 1-5

11/15/2014

Course material created by D. Woit

Course material created by D. Woit

6


Sed stream editor cont1

sed (stream editor) cont.

  • Sed commands can be submitted from a file e.g:

  • File sedfile has two lines:

    • s/shell/Shell/

    • s/c language/ C Language/

    • And can be used as sed –f sedfile infile

  • if we want to include '/' in the change, we use another character to delimit, such as ? Instead of /.

    • sed -e "s?his/her?their?g“

    • sed -e "sXhis/herXtheirXg“

  • Note: sed can be invoked in sequence e.g. change A to B and B to y

  • /home/dwoit> sed -e 's/A/B/' -e 's/B/y/‘

    • A

    • Y

11/15/2014

Course material created by D. Woit

Course material created by D. Woit

7


Tr translator

tr (translator )

  • translates input, character by character, and writes to stdout given translation table specified by from-string and to-string

  • uses a 1:1 correspondence between from/to strings

  • tr ":" ";" <inf

    • changes all colons into semi-colons (uses file inf as input)

  • tr "x" "AX" <inf

    • changes all blanks to "A" and all "x" to "X“

  • tr "A-Z" "a-z" <infile >outfile

    • changes all uppercase to lowercase letters (so it would make stdout a lowercase version of stdin)

  • tr "a-zA-Z" "A-Za-z" <infile

    • changes all upper to lower, and lower to upper

  • can use backslash escapes, \n (newline), \t (tab) etc

  • tr " " "\n“

    • changes all blanks to newlines

11/15/2014

Course material created by D. Woit

Course material created by D. Woit

8


Tr translator cont

tr (translator ) cont.

  • can use octal codes, if begin with a \ (see man ASCII or ascii command)

  • tr " x" "\012y" <inf

    • changes all blanks into newlines (\012 is octal newline) and all "x" to "y"

  • can use Character Classes (see man grep)

    • tr '[:lower:]' '[:upper:]'

  • tr -d "\n" <inf

    • removes all newline chars from inf (-d=delete) and displays on stdout

  • tr -d "a-z" <inf

    • removes all lower case letters from inf

  • tr -s " " " " <inf

    • (-s = squeeze) optionwilltake all repeated occurrences of char1 and replace by just char2. In example above char1=“ “ and char2=“ “

    • i.e., all strings of 1 or more blanks are replaced by just one blank

11/15/2014

Course material created by D. Woit

Course material created by D. Woit

9


Tr translator cont1

tr (translator ) cont.

  • More examples with –s squeeze option:

  • tr -s "Ae" " Y" <inf

    • All strings of one or more "A"s are replaced by one space

    • All strings of one or more "e"s are replaced by one "Y"

  • tr -s '[:punct:][:blank:]' '\n'

    • squeezes all punctuation and horizontal whitespace into one newline (so it puts each word of input on its own line)

  • More information for tr command can be found in man pages

11/15/2014

Course material created by D. Woit

Course material created by D. Woit

10


Pipes

Pipes

  • A pipe | connects stdout of one command to stdin of another

  • e.g., ls | more # lists all files in current dir in alpha order, one screen at a time

  • ls -t | head # lists the 10 most recently modified files in this dir. ls -t ------------------> head ---------> screen

  • redirection is also possible : ls -t | head >outfile

  • see if string "IEEEtran" is anywhere in the first 10 lines of huge file bigfile.tex:

    • head < bigfile.tex | grep "IEEEtran"

    • big.file ------> head -------------> grep ------> screen

    • stdin stdout stdin stdout

11/15/2014

Course material created by D. Woit

Course material created by D. Woit

11


Pipes cont

Pipes cont.

  • last | head

    • see last few times someone has logged in (last's output is LONG)

  • pipes contribute to shell's usefulness and flexibility

  • no need to create new commands to perform a task– we can *combine* existing commands

  • existing commands to perform more complex tasks

  • no need for "temp" files

11/15/2014

Course material created by D. Woit

Course material created by D. Woit

12


Tee command

Tee command

  • saves the data passing through the pipe

  • writes stdin to stdout *and* to a given file

  • Example we want to change

  • head < bigfile.tex | grep " IEEEtran"

  • To

  • head < bigfile.tex |tee first10| grep "IEEEtran"

    • Does exactly same, but also saves first 10 lines of bigfile.tex in file first10

11/15/2014

Course material created by D. Woit

Course material created by D. Woit

13


Example shell programs 1

Example shell programs - #1

#!/bin/bash

#source profs

#prints names of profs logged in

who | grep dwoit | cut -c1-8 | uniq

who | grep eharley | cut -c1-8 | uniq

who | grep jmisic | cut -c1-8 | uniq

exit 0

#or who | egrep "dwoit|eharley|jmisic" | cut -c1-8 | sort –u

Command who displays all users logged in

Command uniq prevents repeated output of a line

11/15/2014

Course material created by D. Woit

Course material created by D. Woit

14


Example shell programs 2

Example shell programs - #2

#!/bin/bash

#source prof1

#prints names of a prof logged in

#name supplied as command line argument

who | grep $1 | cut -c1-8

See what happens if you use prof1 without an argument.

How do you fix it? (hint: use quotes)

who | grep “$1” | cut -c1-8

Corrected pgm prof1a will interpret lack of argument as a null string and print all users

11/15/2014

Course material created by D. Woit

Course material created by D. Woit

15


Cps 393 introduction to unix and c

HMWK

  • Use various UNIX commands and pipes to do the following (do not

  • use any temp files):

  • 1. List, in long form, all those files in your home directory that have

    read, write and execute perms for you, and whose name contains the string "txt".

  • 2. Do the above question again, but list in short form (just the file names.)

  • 3. List any lines in the first 10 lines of a file, myf, that contain the word "dog". (you should create different versions of myf to test your commands.)

  • 4. Find all lines of file, myf, that contain all the words "cat", "dog" and "mouse", in any order, and start with the letter "A".

  • 5. List all the files in your home dir that contain the string "so" somewhere within them. Next, just list the *number* of files in your home dir that contain the string "so" (you should use, among other commands, the command "wc -l", word-count (lines option), which you should look up in the man pages.)

11/15/2014

Course material created by D. Woit

Course material created by D. Woit

16


Hmwk cont

HMWK cont.

  • 6. Use only ls, grep, cut, pipes to answer this question. What sequence

    of pipes and commands could you issue from your home dir to list, in alphabetical order, the absolute path names of all the directories in your file system (if you NEED to, use the sort command.)

  • 7. List the names of any subdirectories of your home directory that have

    rwx permissions for user and group, and no permissions for others--note: list just the directory names in short form (the names only.)

  • 8. List just names of all files/directories that have no extension;

  • 9. List just names of all files/directories that contain no vowels at all.

11/15/2014

Course material created by D. Woit

Course material created by D. Woit

17


Hmwk cont1

HMWK cont.

  • 10. Write a shell program called nw that takes either zero or one command line arguments.

    • nw with no arguments should print out the 10 newest (most recently modified) files/dirs in the current directory.

    • nw -n (nw with one argument, -n) prints out the n newest files/dirs in the current directory.

    • e.g., nw -3 prints out the 3 most recently modified files/dirs

    • If the argument is incorrect, then your program is allowed to do unpredictable things!

  • 11. List names in reverse alphabetical order, of directories that have "r"

    permissions for "other". (the "reverse" option of the sort command may be useful.)

  • 12. List in alphabetical order all files in the current directory that have been edited exactly 3 days ago. Do not list the same file more than once. (the uniq command will eliminate duplicate lines).

11/15/2014

Course material created by D. Woit

Course material created by D. Woit

18


Foreground background processing

Foreground/Background Processing,

  • execute commands in background with &

  • /home/dwoit> find / -name firefox -print >find.out 2>/dev/null &

  • /home/dwoit> # free to do other work

  • /home/dwoit> grep "^the" bigfile.tex >grep.out 2>/dev/null &

  • How to list the active processes?

  • /home/dwoit> ps -l # lists processes, ids, status

    • Status can be R running, S suspended (waiting for an event to complete), D uninterruptable sleep, T stopped by job control signal

  • ... PID ... CMD

  • ... 2498 ... find

  • ... 1483 ... grep

  • ... 3398 ... ps

11/15/2014

Course material created by D. Woit

Course material created by D. Woit

19


Foreground background processing1

Foreground/Background Processing

  • Note: ps x gives shorter list or alternatively

    • ps –u user

  • A shell associates a job with each pipeline

  • /home/dwoit> jobs

    • [1] Running find ... &

    • [2] Running grep ... &

  • /home/dwoit> fg %1 # now in foreground--no prompt until done

    • ^z # suspends foreground process (ctrl-z)

  • /home/dwoit> bg %1 # starts find again in background

  • /home/dwoit> kill %1 or

  • /home/dwoit> kill 2498 # find cmd is killed (^c if fg process)

  • might need guaranteed kill:

    • kill -9 2498

11/15/2014

Course material created by D. Woit

Course material created by D. Woit

20


Command history editing for bash

Command History Editing for bash

  • !! re-execute last command issued

  • !-n re-execute current command minus n

  • !cmd re-execute last command that started with string "cmd"

  • ^str1^str2^ re-execute last cmd but substitute str2 for str1

  • history (or fc -l) lists last commands (each is numbered)

11/15/2014

Course material created by D. Woit

Course material created by D. Woit

21


Examples

Examples

  • /home/dwoit> ls

  • lab1.jav lab2.jav lab1.c393 lab2.c393

  • /home/dwoit> !!

  • lab1.jav lab2.jav lab1.c393 lab2.c393

  • /home/dwoit> find . -name "*.jav"

  • lab1.jav lab2.jav

  • /home/dwoit> ^jav^c393^

  • lab1.c393 lab2.c393

  • /home/dwoit> history

    • 8 ls

    • 9 ls

    • 10 find . -name "*.jav" -print

    • 11 find . -name "*.c393" -print

    • 12 history

11/15/2014

Course material created by D. Woit

22

Course material created by D. Woit


Examples cont

Examples cont.

  • /home/dwoit> !10

  • lab1.jav lab2.jav

  • History in editing:

  • set -o vi makes command line editing use vi

  • set -o emacs makes command line editing use emacs

  • if set -o vi, then

    • esc-k brings back last command for editing (using vi) repeated "k"s move back in history, "j" move forward

  • if set -o emacs, then

    • up-arrow brings back last command for editing (using emacs) repeated "up-arrows"s move back in history, "down-arrows" move forward

11/15/2014

Course material created by D. Woit

Course material created by D. Woit

23


Cps 393 introduction to unix and c

HMWK

  • When the "find" command tries to look in a directory you don't have permission to read, it prints an error message on stderr (try this out to see what happens). If you redirect stderr to stdout, then "find" will print both on stdout and you can pipe all the output. Use find, pipes, grep and wc to print the *number* of directories that "find" cannot read in your filesystem.

  • --

  • Do the same thing, but have the numberwritten into a file called

    num.no.read (using redirection).

  • --

  • Repeat the previous question (output to file) but for the entire

    filesystem (i.e., start your "find" search at root)--but run the command in the background, or you will have to wait a long time for it to finish. If you want to logout before it's done, kill it first.

11/15/2014

Course material created by D. Woit

Course material created by D. Woit

24


  • Login