Linux Intermediate

1. Linux Intermediate ITS Research Computing Center C. D. Poon, Ph.D. Email: cdpoon@unc.edu

2. Class Material • Point web browser to http://its.unc.edu/Research • Click on “Training” on the left column • Click on “ITS Research Computing Training Presentations” • Click on “Linux Intermediate – Commands, Tips and Tricks”

3. Linux Command Category Stdout/Stdin/Stderr, Pipe and Redirection, Wildcards Linux Command Review Break Tips and Tricks Conclusion Question and Exercise Outline

4. Linux Command Category

5. Linux Command Category • Communication ssh scp • File/Directory Management cat cd chmod cp ln ls mkdir more less mv pwd dirs rm head tail wc • Comparisons diff

6. Linux Command Category Cont’d • Searching grep find locate • Archiving compress uncompress gzip gunzip zcat tar • Text Processing cut paste sort sed awk uniq

7. Linux Command Category Cont’d • System Status chgrp chown date df du env who w uptime • Miscellaneous bc cal clear man

8. Stdout/Stdin/StderrPipe and RedirectionWildcards

9. stdout stdin stderr • Output from commands • usually written to the screen • referred to as standard output (stdout) • Input for commands • usually come from the keyboard (if no arguments are given • referred to as standard input (stdin) • Error messages from processes • usually written to the screen • referred to as standard error (stderr)

10. Pipe and Redirection • Pipe (|): stdout of one command to stdin of another command • Output Redirection (>): stdout of a command to a file • Output Appending (>>): stdout of a command appending to a file • Input Redirection (<): stdin of a command from a file

11. Stderr Redirection For tcsh &> filename For bash 2>&1 filename

12. Wildcards • Multiple filenames can be specified using special pattern-matching characters. The rules are: • ‘?’ matches any single character in that position in the filename • ‘*’ matches zero or more characters in the filename. • ‘[…]’ Characters enclosed in square brackets match any name that has one of those characters in that position • Note that the UNIX shell performs these expansions before the command is executed.

13. Linux Command Review

14. ssh Log on to remote machine from Linux ssh cdpoon@killdevil.unc.edu ssh killdevil.unc.edu –l cdpoon ssh killdevil ssh –X kure.unc.edu ssh –Y kure.unc.edu

15. ssh using SecureCRTin Windows Using ssh, login to killdevil.unc.edu To start ssh using SecureCRT in Windows, do the following. • Start -> Programs -> Remote Services -> SecureCRT • Click the Quick Connect icon at the top. • Hostname: killdevil.unc.edu • Login with your ONYEN and password

16. scp Copy files and directories to and from remote computers scp file1 killdevil.unc.edu:/nas02/home/cdpoon/. scp zircon.its.unc.edu:/home/cdpoon/file2 . scp –r dir1 killdevil.unc.edu:/netscr/cdpoon/. scp –r killdevil.unc.edu:/netscr/cdpoon/dir2 dir3 scp kure:/netscr/cdpoon/f killdevil:/nas02/home/cdpoon/.

17. cat Read one or more files and print the on stdout cat file1 cat file1 file2 file3 > file_all cat file4 >> file_all Append file4 to file_all cat > file5 Create file at stdin, end with EOF (cntl-d normally, use “stty –a” to find out) cat > file6 << STOP Create file at stdin, end with STOP

18. cd Change directory, build-in shell command cd /nas02/home/c/d/cdpoon cd ../../Change directory to 2 levels up cd ..Change directory to 1 level up cd ~Change directory to Home cdChange directory to Home cd –Change to previous directory

19. chmod Change the access mode of one or more files chmod u+x file1 chmod go-w file2 chmod u=rwx, g=rx, o=x file3 chmod 751 file3Same as above, 7=rwx, 5=rx, 1=x chmod =r file4 chmod 444 file4Same as above, 4=r, 2=w, 1=x

20. cp Copy a file/dir to another file/dir cp file1 file2Copy to the same directory and change filename cp file1 ../dir/file2Copy to different directory and change filename cp file1 ../dir/.Keep the same filename cp –r dir1 dir2Copy directory recursively cp –r dir1 new_dir/dir2 Copy directory recursively to another directory cp –p file3 file4Preserve the modification time and permission modes

21. ln Create links for file/dir and allow them to be accessed by different names ln file1 file2Hard link for file ln dir1 dir2Hard link not allowed for directory ln –s dir1 dir2Soft link for directory, dir2 -> dir1 ln –s file3 file4Soft link, file4 -> file3 ln –s dir/file5 file6Soft link, file6 -> dir/file5

22. ls List all files and directories in the current directory ls ls –a List files/directories starting with “.” too ls –lLong listing ls –lhList file sizes in human readable format and long list format ls –FFlag filenames by appending / to directories, * to executables files, and @ to symbolic links

23. mkdir Create one of more directories mkdir dir1 mkdir –p dir1/dir2/dir3 Create intervening parent directories if they don’t exist Same as mkdir dir1; cd dir1; mkdir dir2; cd dir2; mkdir dir3; cd ../../

24. more Display files on a terminal, one screen at a time more file1Hit space bar for another page, q to quit more –d file2Display the prompt “Press space to continue, ‘q’ to quit more –c file3Page through the file by clearing each window instead of scrolling

25. less Works like “more” but allows backward and forward movement less file1Hit space bar for another page, q to quit Hit b to scroll backward one window Hit /pattern to highlight “pattern” in the text Hit Return to scroll one line at a time

26. mv Move files and directories within the same machine and/or rename them mv file1 dir1/file1Move file1 to dir1, Same as mv file1 dir1/ mv file3 file4Rename file3 to file4 mv dir1 dir2Rename directory dir1 to dir2 mv dir3 dir4/dir5/dir6 Rename directory dir3 to dir6 and move to dir4/dir5 directory

27. pwd dirs Print the full pathname of the current directory pwd dirsC shell and bash shell built-in command, works like “pwd” dirs –lPrint working directory in long listing

28. rm Delete one or more files and directories Delete empty directory with “rmdir” rm file1 rm file* Remove all files with filename starting as “file” rm –i file*Prompt for y (remove the file) or n (do not remove the file) rm –r dir1Delete directory “dir1” and its content

29. head tail Print first/last few lines of one or more files head file1Print the first 10 lines of file “file1” head –n100 file2Print the first 100 lines of file “file2” tail file*Print the last 10 lines of files with filename starting as “file” tail –f file3Print the last 10 lines of file “file3” and follow file as it grows

30. wc Print a character, word, and line count for files wc –c file1Print character count for file “file1” wc –l file2 Print line count for file “file2” wc –w file3Print word count for file “file3”

31. diff Report lines that differ between file1 and file2, with file1 text flagged by < and file2 by > diff file1 file2 Show difference between file1 and file2

32. grep Search for lines that match a regular expression grep abc file1 Print line(s) in file “file1” with “abc” grep –i abc file2 Print line(s) in file “file2” with “abc” ignoring uppercase and lowercase distinctions

33. find Find particular groups of files find . –name tempFind file named “temp” in current directory find /etc –name ‘rc*’Find file(s) in /etc directory with name starting with “rc” find /usr/share/man –type d –name ‘man*’ Find directories in /usr/share/man with name starting with “man”

34. locate Find files with matching pattern in database prepared by updatedb, Database needed to be updated daily locate whichFind files named with pattern “which” in the OS locate –c whichCount number of files named with pattern “which” in the OS locate –i whichFind files named with pattern “which” in the OS ignoring case distinctions

35. compress uncompress Reduce or expand the size of one or more files using adaptive Lempel-Ziv coding Use uncompress to expand data compress file1Reduce the size of file1 and create new file named file1.Z compress –f file2Force to reduce the size of file2 and create new file named file2.Z uncompress file3.Z Expand file3.Z and restore file3

36. gzip gunzip Reduce or expand the size of one or more files using Lempel-Ziv coding (LZ77) Use gunzip to expand data gzip file1 Reduce the size of file1 and create new file named file1.gz gzip –f file2 Force to reduce the size of file2 and create new file named file2.gz gunzip file3.gz Expand file3.gz and restore file3

37. zcat Expand the size of one or more files created by compress or gunzip List file contents to stdout without deleting the .Z or .gz file zcat file1.Z Expand file1.Z and list the content of file1 in stdout zcat file2.gz Expand file2.gz and list the content of file2 in stdout

38. tar Archive files and directories Create a single file with extension .tar tar –cvf file123.tar file1 file2 file3Create archive file named file123.tar in verbose mode with contents, file1, file2, and file3 tar –xvf file123.tarExpand file123.tar in verbose mode and generate the original files and directories back

39. cut Remove sections from each line of files cut –d: -f1,5 /etc/passwdUse field delimiter “:” to locate fields 1 and 5 from file /etc/passwd to extract usernames and real names cut –c4 file1Take character 4 out from each line of file1 and display in stdout

40. Merge lines of files $ cat file1 1 2 $ cat file2 a b c paste $ paste file1 file2 1 a 2 b c $ paste –s file1 file2 1 2 a b c

41. sort Sort lines of text files sort –fd file1 Alphabetize lines (-d) in file1 and ignore lower and upper cases (-f) sort –t: -k3 -n /etc/passwdTake column 3 of file /etc/passwd separated by “:” and sort in arithmetic order

42. sed Edit one or more files without user interaction using stream editor sed s/xx/yy/g file1 Substitude all occurrences of “xx” in file1 with “yy” and display on stdout sed /abc/d file1 Delete all lines containing “abc” in file1 sed /BEGIN/,/END/s/abc/123/g file1 Substitute “XYZ” on lines between BEGIN and END with “xyz” in file1

43. awk Process files by pattern-matching awk –F: ‘{print $1}’ /etc/passwd Extract the 1stfield separated by “:” in /etc/passwd and print to stdout awk ‘/abcde/’ file1 Print all lines containing “abcde” in file1 awk ‘/xyz/{++i}; END{print i}’ file2 Find pattern “xyz” in file2 and count the number awk ‘length <= 1’ file3 Display lines in file3 with only 1 or no character

44. uniq Report or omit repeated lines uniq file1 Filter adjacent matching lines from file named file1 , writing to stdout uniq –c file1 Prefix lines by the number of occurrences from file named file1

45. chgrp Change the group ownership of one or more files or directories chgrp employee file1 Change group ownership to “employee” for file “file1” chgrp –R student dir1 Change group ownership to “student” for directory “dir1” including subdirectories recursively

46. chown Change the ownership of one or more files or directories chown employee file1 Change ownership to “employee” for file “file1” chown –R student dir1 Change ownership to “student” for directory “dir1” including subdirectories recursively

47. date Print the current date and time in certain format Set the current date and time date Print the current date and time date +%D Print the current date and time in mm/dd/yy format date 1201160108 Set the current date and time to Dec 01 4:01pm 2008 date –d fri Show the date of the coming Friday

48. df Report the number of used and free disk block on all mounted file systems df Print used and free disk block on all mounted file system df -h Print used and free disk block in human readable format df -k Print used and free disk block in kb

49. du Print disk usage of directories and its subdirectories du dir1 Print disk usage in kilobyte of directory “dir1” du –-block-size=1M dir2 Print disk usage in megabyte of directory “dir2” du –hs dir3Print summarized disk usage in human-readable format of directory “dir3”

50. env Display the current environment variables or set new values env Display all of the current environment variables