BASH Shell Interactive Usage – Intermediate-Level Topics

1. BASH Shell Interactive Usage – Intermediate-Level Topics Presented by: Brian A. Seklecki <brian.seklecki@pnc.com> Brian A. Seklecki <bseklecki@fedex.com> Brian A. Seklecki <seklecki@gmail.com>

2. Presentation Overview Goal: To improve day-to-day UNIX/POSIX-system proficiency by utilizing advanced of features of the BASH shell (*) NOTE: This presentation will be limited to topics related to interactive shell usage (“at the command prompt”). Shell scripting can be covered by another presentation. (*) Available on EIS-ITO GNU/Linux Systems and some AIX systems.

3. Presented in 4 Acts • Act I: Interactive Shell Features (History, Command editing, VI / EMACS Modes, Tab Completion, etc.) • Act II: Job Control and Process Management • Intermission • Act III: Customizing Bash Behavior • Act IV: Advanced Topics (and GNU/Linux-specific commands to save you time) NOTE: This presentation will be limited to topics related to interactive shell usage (“at the command prompt”). Shell scripting can be covered by another presentation.

4. Interactive Presentation • Please have your laptop out and be logged into your assigned learning environment! • If you are logged into a Midrange environment system, please ensure that you are running bash by typing ‘bash’

5. Shell Prerequisite Knowledge • UNIX/POSIX Process Concepts • Variable Declaration • Standard UNIX I/O Concepts • stdin, stdout, stderr, etc. • Pipes & Redirection • File and Directory Concepts • Basic Shell Glob Expression Handling • Basic External Commands: grep, Awk, Sed • Basic VI and EMACS style editing

6. Introducing Bash • Bash is the standard shell for RedHat-based GNU/Linux systems • It implments a superset of the Bourne POSIX shell (sh) • Borrows features from sh, csh, tcsh, and ksh, zsh • Created by the Free Software Foundation licensed uner GNU Public License (GNU = GNU is Not UNIX)

7. What’s new in Bash • Many features from KSH • Powerful command line editing including tab completion • Simplified VIM and EMACS editing mode • Enhanced/Simplified History • Additional special variables and builtins (*) for simplified operation • Exceptional Documentation! (*) Does everyone know what a ‘builtin’ is?

8. Act I: Interactive Shell Features • Tab Completion • History • Command Editing • A few cheap hacks to save you time!

9. Tab Completion • Tab completion is a quick way to avoid unnecessary ls(1) searching (for both files and commands) by allowing one to easily tab through a ever-narrowing list of commands based on: 1) $PATH variable contents or 2) A specific path • In BASH, tab completion is completed by two [TAB] keys in immediate succession • You can tab complete both your commands and file arguments to commands

10. Tab Completion (How it works) • Tab completion is an assisted search • Tab completion works by offering an array search result candidates based on the current input on the command line • The less-ambiguous of an argument you provide before tab-completion, the more-narrow your search results will be • If you already have a valid command (e.g., ls, cd, rm, etc.) entered, and optionally any arguments/switches, tab completion will assume that you want to tab complete a UNIX path as an argument. • Unless it is prefixed with ‘/’, ‘..’, or any other absolute/relative path prefix identifier, it will assume that you want to complete the path argument relative to “./” (aka, $PWD)

11. Tab Completion (Example 1) Example 1: Completing from search $PATH variable $ ls /usr/{,s}bin/*ipmi* /usr/bin/ipmicmd /usr/bin/ipmilan /usr/bin/ipmish /usr/bin/ipmi_ui /usr/bin/openipmicmd /usr/bin/openipmish bs827200@fs0090nmd0:/home/bs827200$ [TAB-TAB] Display all 2128 possibilities? (y or n) n bs827200@fs0090nmd0:/home/bs827200$ open[TAB-TAB] open openipmicmd openipmish openjade openssl openvt bs827200@fs0090nmd0:/home/bs827200$ openipmi[TAB-TAB] openipmicmd openipmish bs827200@fs0090nmd0:/home/bs827200$ openipmish > help domain - Commands dealing with domains [...]

12. Tab Completion (Example 2) Example 2: Completing from absolute/relative path $ ls /opt/fedex/swak/[TAB-TAB] 1.0/ 1.4/ 1.4.1/ 2.0-BETA2/ .ssh/ $ ls /opt/fedex/swak/1.[TAB-TAB] 1.0/ 1.4/ 1.4.1/ $ ls /opt/fedex/swak/1.4/ [...]

13. You Try It: Tab Completion Example 3: Completing from absolute/relative path $ ls /etc/s[TAB] [...snip...] subversion/ sasl2/ selinux/ setuptool.d/ shells snmp/ scrollkeeper.conf sensors.conf skel/ sound/ sysconfig/ NOTE: Append ‘sy’ manually to your command $ ls /etc/sy[TAB] NOTE: The command will change to $ ls /etc/sys $ ls /etc/sys[TAB] sysconfig/ sysctl.conf syslog.conf $ ls /etc/syscon[TAB] NOTE: the command will change to (note the trailing slash indicating sub-dir entry): $ ls /etc/sysconfig/ $ ls /etc/sysconfig/sys[TAB] syslog system-config-netboot system-config-securitylevel system-config-users $ ls /etc/sysconfig/system[TAB] NOTE: The command will change to  $ ls /etc/sysconfig/system-config-[TAB] system-config-netboot system-config-securitylevel system-config-users $ ls /etc/sysconfig/system-config-n[TAB] NOTE: The command will change to  $ ls /etc/sysconfig/system-config-netboot -rw-r--r-- 1 root root 36 Apr 7 2008 /etc/sysconfig/system-config-netboot

14. History: Command Synopsis Bash keeps track of your commands in a database file in your home directory: ~/.bash_history Several ways to use history: • ‘history’ Command / Database File • The ‘fc’ mechanism • Previous command line editing (vi, emacs mode)

15. History: Command Synopsis (cont.) Some important environmental variables related to history: $ HISTFILE=/home/user/.bash_history $ HISTFILESIZE=1000 - The number of commands in the history file $ HISTSIZE=1000 – The number of commands in history memory The ‘history’ command shows all of the contents of the history database in reverse order by default. You can grep/head/tail this command with a pipe. Commands: • history [-rh] where: r – reverse listing h –hide numbers % ![historylinenumber] – allows one to re-execute a given command % ![prefix] – allows one to locate a command beginning with prefix % !?[string] – allows one to locate a previous command containing the string

16. History: Command (Contd.) $ history 10 586 ps gauxww -u bs827200|head 587 ps gauxww -u bs827200|head -1 588 ps gauxww -u bs827200|head -5 589 ps gauxww -u 827200|head -5 590 top 591 top 592 top 593 cd 594 history --help 595 history 10 $ !590 [top is re-run]

17. History: Command (You Try It!) • Log into any Linux or AIX System where Bash is your default shell • If bash is not your default shell, try the command ‘chsh’ • Start bash • Enter a 10-15 commands (ls /prod, ls /prod/sms, etc.) • Type history • Type !10 (example)

18. History: Edit a history entry with FC (Special) $ export EDITIOR=vim $ export FCEDITOR=$EDITOR $ history 10 589 ps gauxww -u 827200|head -5 595 history 10 $ fc $ ps gauxww –u bs827200|head -10 # modified

19. History: Command Line Editing • Use history to reduce keystrokes needed • Re-run the previous command with slight variations

20. Command Line Editing Command line editing lets you use the up/down arrow keys to: • Interactively navigate the history file to retrieve recent commands. • Once you select a command, you can edit it using the key macros for your edit mode • To select VI vs EMACS edit mode, simply run either: • % set –o emacs • % set –o vi

21. Command Line Editing (EMACS) bash provides a number of line editing commands; many are the same as emacs editing commands • M-b Move back one word • M-f Move forward one word • C-a Move to beginning of line • C-e Move to end of line • C-k Kill text from cursor to end of line • C-r Re-draw/refresh the screen • “M(meta)” in Bash with Putty default config is the “ESC” key

22. Command Line Editing (Vi/vim) In Vi mode, bash provides a number of line editing commands; many are the same as vi(1)editing commands • w Move forward one word • B Move backward one word • 0 (zero) Move to beginning of line • $ Move to end of line • dw Delete word • cw Change word • d$ Delete to end of line (*) See VIM documentation for more

23. VIM v.s. EMACS (The final Word) http://learning.stat.purdue.edu/wiki/courses/fall2010/598g/lab/week3

24. Command Line Editing (YouTube link not visible from PNC Proxy)http://www.youtube.com/watch?v=gbzMJrxGx98

25. History: Drill Down History Recall • To easily find a command outside of immediate history (that one would use keyboard history navigation to find) • Press Control+R (^R) at the shell and enter a substring text search query and Bash will search your memory and file history and bring it into the buffer • Optional: Once found, use your shell’s history navigation mode keys to edit before entering

26. History: Drill Down History Recall (You try it!)

27. Act II: Job Control • Job Control • A bit of advanced process management • A few cheap hacks to save you time

28. Interactive Job Control • All POSIX shells have basic interactive shell control • Jobs are localized / relative in the context of the current shell’s instance (different shell instances cant control another shell’s jobs) • Job controls allow a single TTY/Emulator session and single bash shell process run many commands • Analogous to multi-threaded window manager in a GUI In BASH, many improvements make the

29. Job Control and Process Mgmnt Advantages: • Reduce the number of shells and terminals • Simplified Debugging Tasks Job Control and Process Mgmnt Job States: • Foreground (running, controls STDIN) • Background (running, STDIN returned to shell) • Suspended (Job process is paused, at prompt)

30. Process Scheduling and ControlJob/Process Control Command Overview ps: Reports the process[es] status. jobs: Lists the jobs that you are running in the background and in the foreground. If the prompt is returned with no information no jobs are present. Not supported by all shells.  top: Display top CPU processes (external) sleep: Waits or (sleeps) an x amount of seconds (external) fg: Continues a stopped process by running it in the foreground. Not supported by all shells. kill: Kills a process. wait: Make process wait until completion of other process.

31. Jobs – command synopsis jobs command shows a list of current jobs for current user in current shell in the following format. Sample output: Job no. Status command [1]- Stopped man ps [2]+ Stopped top [3] Running sleep 120 & jobs can show PIDs as well with ‘-l’ on Linux to avoid the need for ps(1) + grep(1) $ jobs -l [1] 29411 Stopped ssh qpswkd01 [2]- 29412 Stopped ssh qpswkd02

32. Background Jobs • Background processing • Use the & meta-character tailing argument to run programs in the background • May run more than one process in background • PID number is displayed for background process • Each background process is given a job number • Can execute other commands with existing shell while background program is running • STDOUT/STDERR can be redirected, otherwise it may pollute other jobs

33. Managing Jobs • A SIGINT (interrupt, pause) signal may be sent from the terminal to a foreground job using key sequence: CONTROL + Z • You may background the jobs (so that it will continue running) with the ‘bg’ shell builtin command • ‘bg’ takes an optional argument of Job ID • Use the ‘jobs’ command to view the child processes and jobs of the current shell • Once a job is backgrounded, it will change state to RUN; loosing control of STDIN but not STDOUT/STDERR • You may bring a single job back to foreground with ‘fg’

34. Background Jobs (example) You may start a program directly into the background: • bs827209@drh00000:~/$ tail –F ITL_LOG & [1] 79116 • bs827209@drh00000 :~/$ • bs827209@drh00000 :~/$ jobs [1] Running tail -F ITO_LOG & • bs827209@drh00000 :~/$ ./App –logVerbose [date] App starting at …

35. Managing Jobs (example) linux$ sleep 90 [CONTROL+Z] [1]+ Stopped sleep 90 linux$ sleep 1000 & [2] 29200 linux$ jobs [1]+ Stopped sleep 90 [2]- Running sleep 1000 & linux$ sleep 15 & [3] 29203 linux$ fg 1 sleep 90 [3]- Done sleep 15 linux$ # NOTE: Job 1 completed, Job 3 Completed as well; Job 2 rages on Luinux$ jobs [2]+ Running sleep 1000 &

36. Managing Jobs (You Try It) linux$ top & [1]+ Stopped top # Note: This will fail because ‘top’ cannot be run in the background linux$ dd if=/dev/urandom > ~/random.data & linux$ jobs [1]+ Stopped top [2]- Running dd if=/dev/urandom > ~/random.data & linux$ fg 2 [no output] [Control+z] linux$ jobs [1]- Stopped top [2]+ Stopped dd if=/dev/urandom > ~/random.data linux$ kill -9 %2 [2]+ Stopped dd if=/dev/urandom > ~/random.data [2]+ Killed dd if=/dev/urandom > ~/random.data # Cleanup here linux$ du -h random.data 157M random.data linux$ rm !$ rm random.data linux$

37. Job Control: Signal Review • Historic Note: SIGHUP (“hangup”) was historically used to restart mgetty(1) dialup modem TTY handler. E.g., drop the current dial in call session, respawn, and re-listen on the serial line for “RING” • When a SSH connection is reset to the client, the child Shell process of the forked SSHD process will receive SIGHUP, and so will all child processes that haven‘t forked from the current TTY

38. Managing Jobs (kill a process review) • Sometimes a user process becomes unresponsive or ghosted from the terminal (Lost session, etc.) • Kill the process with kill(1) command Kill with an explicit Signal: • 1 - hang up • 15 - terminate (same as control-c ): default signal sent by kill without arguments • 9 - kill: will terminate processes that do not terminate with 15 • Example: % kill -9 35679 forces killing process 35679

39. Managing Jobs (kill a process review)

40. Tangent: Nohup to avoid SIGTERM • nohup command • At logoff background processes would be terminated by cascading SIGHUP of parent process (SSH daemon instance, shell, etc.) • Use nohup command to keep background processes running, even after logout • If running more than one command must use in a shell script (ex: sleep 30; echo HI) • Just prefix the command with ‘nohup’ % nohup ./logRunningJob.sh [&]

41. Tangent: Controlling STDOUT with SIGSTOP / SIGCONT • In Bash, SIGSTOP is mapped to Control+Q [^Q] and SIGCONT is mapped to Control+S [^S] • You may halt the current forground process to examine (for example) verbose debug logging messages to STDOUT/STDERR • Example: linux$ find /etc /etc /etc/sane.d [control+q] # Output stopped [control+s] # Output resumed /etc/sane.d/saned.conf

42. Tangent: Using GNU screen to virtualize the terminal (contd.) • Screen can simplify long-running interactive jobs • Screen is useful for bypassing unstable / unreliable parts of the network (Wifi, VPN) • [Quick preview here]

43. More on Job Control • shell-job.ppt attached – Great visual demonstration

44. Act III: Customizing Bash • Login Scripts • Aliases • Prompts • Writeable Variabables • Shell Options • Editing Mode • Editors

45. Login Scripts • Use the login scripts to permanently customize your shell’s behavior / configuration • Use login scripts to declare reusable macros, aliases, and functions to save time • You don’t want to have to manually enter aliases, set environment variables, etc., each time you log in • All of these things can be done in a script that is run each time the shell is started

46. Shell Modes Different scripts are sourced/read/executed based on the mode that the shell was started in • Interactive Shell: Started when a remote shell session is started via SSHD, Telnet, {m,}getty(1) + login(1) on the console or serial line • Non-interactive Shell: Started when a remote session is started (rcp, rexec, ssh, cron(8) job, sub-shell (bash ./myscript.sh, etc.)) • HINT: If your default shell is not bash, and you want to spawn bash from sh/ksh in interactive mode, run: lifesucks% bash --login

47. .bash_profile and .bashrc Every time you log in to bash, the commands in ~/.bash_profile are run • you can put any common startup commands you want into this file • useful for setting up aliases and other settings for remote login Every time you launch a non-login bash terminal, the commands in ~/.bashrc are run • useful for setting up persistent commands for local shell usage, or when launching multiple shells • often, .bash_profile is configured to also run .bashrc, but not always Note: a dot (.) in front of a filename indicates a normally hidden file, use ls –a to see

48. Login Scripts (contd.) • For bash in interactive mode, order is . . . • /etc/profile • ~/.bash_profile • ~/.bash_login (if no .bash_profile) • ~/.profile (if neither are present) • ~/.bashrc • After logout . . . • ~/.bash_logout • For non-interactive shell: • ~/.bashrc

49. Example .bash_profile (partial) % cat ~/.bash_profile # include .bashrc if it exists if [ -f ~/.bashrc ]; then . ~/.bashrc fi # Set variables for a warm fuzzy environment export CVSROOT=~/.cvsroot export EDITOR=vim export PAGER=/usr/local/bin/less # AIX / Solaris export http_proxy=http://internet.ground.fedex.com:80

50. Example .bashrc (partial) % .bashrc • # abbreviations for some common commands • alias f=finger • alias j=jobs • alias l='ls -lF' • alias la='ls -alF' • alias ls='ls -F' • ssh=“/usr/bin/ssh –v” • dev=“ssh dpPRJa08” # Compound