Introduction to Unix – CS 21

1. Introduction to Unix – CS 21 Lecture 5

2. Lecture Overview • Lab Review • Useful commands that will illustrate today’s lecture • Streams of input and output • File redirection • Piping between different programs • Putting it all together for some powerful tools

3. Important Points From Lab • Get used to both modes of chmod • Understand how directory permissions affect access • Write permission only doesn’t let you do anything • Start thinking about using wildcards to speed up tasks

4. Good Commands To Know: wc • Stands for word count • Will report the number of lines, words, and characters in a file • Works with all sorts of files, but only makes sense with text files • Useful for all sorts of applications • Either file direction or piping is required…

5. Example Of wc

6. sort • Will sort the lines of a file into alphabetical order • Useful on data files or output from programs • Usage: sort [OPTIONS] [FILE]

7. Example Of sort

8. uniq • Will go through a file and remove duplicate copies of a line • Will only remove duplicate copies of lines that are adjacent • Useful on files that might have repeated unnecessary data • Usage: uniq [OPTIONS] [FILE]

9. Example Of uniq

10. Wouldn’t It Be Nice If We Could sort And uniq A File? • No command exists that does both • By the end of the lecture, we should see if this is possible • Do you think this will be possible? • Hint: Remember Unix was designed so that tasks could be easily automated and so that tasks could work together

11. What Exactly Is Input And Output? • Input • Everything a Unix program reads • Usually from the keyboard • What you type • Output • Everything a Unix program writes • Usually to the screen • What you see

12. Streams Of Input And Output • stdin • Input from the keyboard • stdout • Output to the screen • stderr • Error messages that show up on the screen

13. stdin • Everything you type into the terminal • The programs mostly expect all of their input from the user (you) typing at the keyboard

14. stdout • The results of a running program • Goes to the terminal you ran the program from

15. stderr • Appears the same location as stdout • Used for error messages • Allows for the error messages to be separated from normal output

16. Redirecting These Streams • Deciphering “echo ‘hi’ > helloFile” • > • Sends all output that normally would go to stdout into a file • Completely overwrites whatever file was there or creates a new file if none existed

17. Example Of >

18. A Tricky Catch To Remember • You must be careful because ‘>’ will overwrite files

19. Redirecting stdin • < • Makes all of the input for a file come from a file as opposed to the keyboard • Allows you to input a lot of data without a lot of typing • Exercise: Go to lab and run cat without any parameters

20. Example Of <

21. Redirection Of Output Without File Overwrite • What if you want to add information to a file instead of overwriting it? • >> • Appends all output of a program to a file • Will create a file if it doesn’t exist, but won’t overwrite any information currently in a file

22. Example Of Append Redirection

23. One More Stream • Don’t forget about stderr • 2> • Not some weird smiley • This says redirect stderr to a file • Important to note why this notation exists • 0 = stdin • 1 = stdout • 2 = stderr

24. Example Of Redirecting stderr

25. Example Of The Difference Between stdout And stderr

26. Is This Ever Useful? • Short answer: Yes! • Longer answer: • Whenever you want to save the output of a program, redirect it • Whenever you want to avoid typing input, redirect it • Whenever you want to isolate any error messages, redirect it

27. Possible Uses For File Redirection • Saving data generated by one program for use by another • If you continually type the same input into different programs, place that data into a file and redirect it into the different programs

28. How To Remember Which Arrow Does What • cat < inputFile • Think of this as the inputFile dumping all of its contents into cat • In other words, inputFile is going into cat • cat > outputFile • In this case, cat is dumping its results into outputFile • In other words, cat is going into outputFile

29. What About Other Command Flags? • The input or output file must come directly after the redirection operator • Example: • Right: ls –l * > listingOutput • Wrong: ls > -l * listingOutput • Question: How do you think this line is interpreted? • As a general rule of thumb, redirection should come at the end of a line • Redirection should come after all arguments to the program to avoid problems like the above

30. Combining These Flags • Entirely possible to have both < and > on the same line • Each redirection operator assumes the next argument is what is being redirected • Question: Can you have > and >> on the same line?

31. Answer

32. What Does This Command Do? • ls /bin/doesntExist 2>> errorFile • Appends stderr to the end of errorFile

33. What About These Commands?

34. Piping: Tying Programs Together • We can combine some of the previous commands together with a pipe • Pipes feed the output of one program into another • | • Yes, that’s a vertical bar

35. What Does A Pipe Look Like?

36. sort And uniq • Answering our previous question, we see that it should be possible to both sort and uniq a file at once using pipes • How exactly would we do it?

37. sort And uniq With Redirection

38. Same Example With Piping

39. How To Read Pipes • You can have more than one pipe per line if you’d like • Always read left to right, and output flows left to right

40. Putting It All Together • What if we want to have both file redirection as well as piping? • Well, things start to look ugly, but this is entirely doable

41. Complex Example

42. Really Complex Example

43. Is All This Stuff Worth It? • Well, let’s compare this with how you do things in Windows • With piping, several tasks can be combined and run in one single step • What if I wanted to sort and uniq a Word file? • Open the file and highlight all of the text • Select Table->Sort (not intuitive) • Specify how to sort and click o.k. • Word doesn’t have a uniq feature…

44. Slowly, Your Power Grows • With this information, you are slowly becoming a Unix power user • Maybe you’ll save the world from dinosaurs yet…

45. Next Time • More thorough explanation of wildcards • Regular expression overview • grep – what exactly is it? • Quiz #1