Unix Linux Administration II

Unix Linux Administration II Class 2: Working in your shell, review common commands and UNIX tools.

Agenda • Review topics discussed last week • Homework review: • Reading. • Prod host login using uwst14 account. • Account creation and disabling uwst14 • sudo logs and iptables • Unit 1: Files, Directories and your shell. • In class lab 2-a • Unit 2: common UNIX commands • In class lab 2-b • Unit 3: common UNIX tools and script basics • in class lab 2-c

Review – CentOS Redhat CentOS freely available, one install base Redhat – subscription based license, multiple installs available. Both based on the same core source, systems and applications certified for Redhat should also be able to run with CentOS.

Review: FHS FHS, File System Hierarchy Standard. Maintained by the Linux Foundation. Supported by organizations including IBM, Dell, RedHat, and HP. /sbin, /etc, /boot, /usr/local, /var…. */srv contains site-specific data which is served by this system.

Review: user and group mgmt. Local user accounts are maintained in /etc/passwd. This file and be edited directly but I would discourage this and instead recommend using standard tools such as useradd, usermod and userdel. The same applies to groups which are maintained in /etc/group and should be edited with groupadd, groupmod and groupdel

Review: SU and sudo Switch User (su). Account password required to execute commands. No access restrictions. Little tracking or auditing available by default. Super user do (sudo). Account password not required. Lots of permissions and access controls available.

Review: iptables The default config can be found under /etc/sysconfig/iptables Manual updates can be made to /etc/sysconfig/iptables which will be picked up when the iptables service is restarted. Dynamic updates can be applied using /sbin/iptables sudo /sbin/iptables -I INPUT 4 -p tcp --dport 53 -j ACCEPT

Review: certificates for ssh From your windows clients use PuTTygen to create a public and private key pair. The public key is placed into the authorized_keys file and the private key is then used to authenticate your account by providing a signature that can be decrypted by the public key. From your linux or mac host you can use ssh-keygen to accomplish the same tasks.

Class 2, Unit 1 What we are going to cover: • Linux files and directories, input, output, redirection, and pipes. What you should leave with from this session: • How Linux files systems are organized. How to navigate at the command line and work with data streams.

Linux file types There are 3 basic types of files • Ordinary (regular) files • Base file type, can contain data, text, compiled code, etc. • Directory files • Contains other files and directories • Special files • Special meaning, often some form of I/O

Basic file utilities • cat – print the contents of a file to stdout • ls – list files. • file – used to determine file type • mv – used to rename a file. • cp – copy a file • rm – remove a file • ln – link a file (similar to a shortcut) • wc – count the contents of a file.

It all starts at / Working with directories is always relative to root or “/” Each file in a directory must have a unique name However two more files may exist with the same name if they are NOT in the same directory.

Where is HOME • Users always have a home directory defined. • Typically this is where you start every time you login. • This is your current “working directory” at login. All files are defined either from / or your current working directory. • Where is . • Where is ..

Navigate and work with directories. • pwd – print current “working directory”. • mkdir – make a directory • rmdir – remove a directory • cd – change directory • run "env" how does your system track your last directory

Standard input, output and redirection Standard in (stdin) and standard out (stdout) are pre-connected channels. By default most *nix systems take input from your terminal and return the output to your terminal. Either input or output can be redirected.

Standard input and output When you enter data into your terminal to signal the end of the content you enter ctrl+d. Something like: Sort <enter> h d f ctrl-+d This will result in asciibetically sorted output.

Output (stdout) Redirection Output destined for standard out (stdout) can be redirected using “>” Meaning you can take the output from any tool and redirect it to another tool or file. who | wc –l > sessions.txt By default this will overwrite any data, but to append data use >> who | wc –l >> sessions.txt

Input (stdin) redirection Command input (stdin) can be redirected from a file using < For example with a data file you can grep the contents using < For example grep –i root < /etc/passwd Or using the word count utility wc –l < /etc/nsswitch.conf

Standard error (stderr) • stderr – When the command you enter results in an error the output is NOT sent to stdout it is usually sent to stderr For example search for a file that does not exist • ls –l zhy zhy* not found

Redirecting stderr Usually you want to see the error message even when redirecting output. However if this is not the case simply redirect the output type 2 to file • ls -l zhy 2> error.txt Note: no space is permitted between the 2 and > In this example we redirect error messages to /dev/null, the bit bucket. • find / -name log 2> /dev/null

Common usage: 2>&1 You may often come across this syntax in scripts. STDOUT = 1 STDERR = 2 The > is a redirect and & is an escape. So 2 is redirected to 1 where 1 is a file handle and not a file. The escape is used to facilitate this redirect to a file handle and not a real file called “1”. So to redirect both stdout and stderr to a file use: find /etc/ -name .profile > file.info.txt 2>&1 * note 0 = STDIN

pipes Output and input can be redirected but commands can also be linked or piped together using | who | wc –l With this syntax we are connecting the standard output from who with wc. Pipes can be used between any programs where the first generates stdout and the second reads from stdin

Multitasking Yes, already, we need to consider maximizing your time. You can type more than one command on the same line so long as you separate them with a semi-colon mkdir test; touch test/newfile.txt ; echo "new content" > test/newfile.txt You can also send commands to the “background” using the ampersand. sudo grep -i accept /var/log/messages* > /tmp/logins.txt &

Review File types: standard files, directories and special. It all starts at root Standard output, standard input, standard error Output and input redirection Filters and pipes Multitasking.

Lab 2a • Lab notes for this session can be found here: http://www.ulcert.uw.edu/class/ -> Home -> Labs ->

Class 2, Unit 2. What we are going to cover: • What is the shell What you should leave with from this session: • How the shell interacts with the kernel, how the shell manages your requests.

What is a shell? UNIX or Linux can be divided into two broad categories; the kernel and, the utilities. • The kernel is at the core of the system. It is loaded into memory at boot and remains there until the system is halted. • Tools exist on disk and are brought into memory as required. The shell is just another utility that is loaded into memory as needed.

Shell responsibilities The shell is responsible for: • Program execution • Variable and filename substitution • I/O redirection • Pipeline hookup • Environment control • Providing an interpreted programming language

Shell behavior Each line of input is analyzed, commands executed. If the shell finds a program to initiate it will request the kernel load it. This program will be loaded into memory and the shell will sleep until it completes and the shell is loaded into memory.

Terminals Terminals define the baud rate and present the login prompt for the session. For each physical terminal port a Get Teletype (getty) will be active Login is the process which interacts with the session, confirms the credentials and loads the shell.

Launching a terminal On SysV Linux based distributions such as RedHat/CentOS the Init process is responsible for launching the terminal. Direct logins to the console are typically managed by getty terminals Remote logins such as ssh and telnet are managed by pts terminals or pseudo terminals.

Review What two broad categories can be divide a Linux system into? What are some of the shell responsibilities? What are some terminal types?

Lab 2b • Lab notes for this session can be found here: http://www.ulcert.uw.edu/class/ -> Home -> Labs ->

Class 2, unit 3 What we are going to cover: • Common Linux utilities. What you should leave with from this session: • Ideas of how you might use and leverage some or all of these common tools.

regex cut paste sed tr grep sort uniq awk xargs find vi(m) Common tools in review

Common scripting tool review • We need to briefly discuss Regular Expressions, or regex • There are many editor tools available but for this course will assume you are using VI. While other editors may have benefits VI or VIM (VI enhanced) in often available.

Regular Expressions • * means zero or more, very greedy… • ? Means zero or one of the preceding character • […] means any characters in the brackets • [0123] or [A-Z] [4-9] [a-zA-Z] • . Means single character except line break. • ^ beginning of line • $ end of line • \ means escape the special meaning. • What does .$ match? • How can you match the a period at the end of the line?

Saving matched values • Matches are stored in memory on the system and can be recalled until the program ends or they are over-written • ^$.$ will match the first character of a line • ^$.$\1 will assign that value to $1 • matching if the first two character are the same. • regex matching three character blocks • ^$…$$…$ • Start ^, escape \ (, pattern match …, escape \ ) repeat.

cut • Used to extract fields of data. • cut –c<chars> file or data • Cut can be used with files and can interpret delimiters for the data. • -d = delimiter (default is tab) use ‘ ‘ for space. • -f = fields cut -d: -f4 /etc/passwd cat /etc/services

paste • Paste is just the opposite of cut, it will put lines together • Syntax paste files • paste file file2 • paste /etc/services /etc/passwd | more • Files are pasted side by side • If you want to insert delimiters between the files use –d ‘<value>’ paste –d ‘#’ /etc/passwd /etc/shadow | more paste –s merge lines from the same file.

sed: Stream editor sed cannot be used interactively. sed syntax is: sed <cmd> file sed applies the command to each line of input for the file specified. If no file is defined standard input is assumed. sed does not change the source file, all changes are directed to stdout. To change nologin to upper case in /etc/passwd sed ‘s/nologin/NOLOGIN/’ /etc/passwd If they were more than one instance of nologin per line you would need to apply the change globally /g

More on sed When making change with sed it is common to redirect the output to a temp file, confirm the changes and then overwrite the original file. sed ‘s/nologin/NOLOGIN/’ /etc/passwd > /tmp/nwp {verification logic} mv /tmp/nwp /etc/passwd Use –n to output specific lines from a file sed –n ‘#p’ /etc/passwd print lines 1 thru 5 sed –n ‘1,5p’ /etc/passwd print just the lines with "lp" sed -n '/lp/p' /etc/passwd

Sed cont. sed can also display what characters may be used for delimiters in a file such as the tab or space sed –n ‘l’ /path/to/file = list out tabs notations delete lines from a file with –d numbers or patterns. • sed ‘1d’ /etc/passwd • sed '3,6d' /etc/passwd Delete any line with “root” • sed ‘/root/d’ /etc/passwd

tr – translate or transliterate • tr is used to translate characters from stdin. • tr always expects input from standard in • The basic format is "tr from-char to-char" • tr takes two sets of characters and replaces the second set with those from the first set. • tr matches in order between character groups.

tr cont. Use the -s option with tr to “squeeze” out multiple occurrences of characters. tr -s ' ' ' ' < /etc/services Use the -d option to delete a character from a stream tr -d '\11' </etc/services *\11 represents the octal value for tab. switch case using either '[a-z]' '[A-Z]' or '[:lower:]' '[:upper:]'

grep – search for patterns grep allows you to search file/s for patterns. grep <pattern> file -i = means case insensitive or ignore case -v = not matching –v <username> /etc/passed -n = line numbering You can use regexp with grep also grep 'dbd*' /etc/services

sort • Sort takes each line of input and sorts in ascending order by default, basically ACSIIbetically. • ps –ef | sort • -u = remove duplicates from output • -r = reverse the sort order • -o = output to file • +2n = skip the first two lines • -t = sort by defined delimiter, colon, #, etc.

uniq • Helps to find duplicates in a file. • -d write out only the duplicates from a file • -c provides numbers of duplicate lines

awk Awk named after the creators; Alfred Aho, Peter Weinberger and Brian Kernighan Awk is a pattern scanning and processing language. Awk is similar to C and can be complex; however, a little Awk can go a long way.

awk cont. awk can be used to print a column who | awk '{print $2}‘ Or have awk print the entire line who | awk ‘{print $0}’ Or tell you the number of fields who | awk ‘{print NF $0}’ Awk can do math, create formatted output, select by comparison and more.

Unix Linux Administration II