Introduction to Linux

1. Introduction to Linux U.Melbourne February 2007 Galloway, “X-ray data analysis”

2. What is Linux? • A Unix-like operating system family; a Linux system which includes elements of the GNU Project is sometimes referred to as GNU/Linux. • Mainly GNU development from 1984 to 1991; Linux kernel development from 1991 by Linus Torvalds. • Linux runs on Intel PC (“i686”) hardware; it is the de facto standard for astronomical computing. • There are many different Linux distributions, in addition to other Unix-like OSs including Solaris, Mac OS-X (Darwin), AIX, etc etc. Galloway, “X-ray data analysis”

3. Galloway, “X-ray data analysis”

4. Getting started • The kernel is the most basic (hardware-dependent) software which allows the computer to interpret and act on your commands • Terminal windows are command-line environments where you can interact with the computer, run code etc. • The shell is the 2nd most basic software element which determines the command line environment & syntax of elementary commands. Common examples are zsh, csh, tcsh etc. • Your shell should be tcsh (“T-shell”) • where to get help: man & info • Ubuntu help & WWW Galloway, “X-ray data analysis”

5. Galloway, “X-ray data analysis”

6. (right-click) Galloway, “X-ray data analysis”

7. Galloway, “X-ray data analysis”

8. Typing things & saving time • The [TAB] key will auto-complete the command or file name you are typing (if it can) • The up-arrow key allows you to recover previous commands, can then scroll through with the up & down-arrow • You can quickly go to the end or beginning of a line you’re typing with ^E and ^A (hold down control & press E or A, respectively) • The left button on the mouse can be used to highlight text, which can then be pasted using the middle button • Right mouse button usually brings up contextual menus Galloway, “X-ray data analysis”

9. Typing things & saving time 2 • The history command lists (all?) the previous commands you’ve entered into that terminal window • You can copy and paste previous commands using the mouse, or use !, I.e. !405 [RETURN] will re-execute the 405th command you entered; !405 [TAB] will bring up the command and allow you to modify it Galloway, “X-ray data analysis”

10. Command syntax & wildcards • typical command syntax is > command [-f1 [arg1] [-f2 [arg2] […]]] • Wildcards allow you to give multiple files as argument: • * matches any number of any character • ? Matches one of any character For example > ls *.txt list all the files with extension .txt > rm burst?.dat deletes burst1.dat, burst2.dat but not burst11.dat Galloway, “X-ray data analysis”

11. Redirection and pipes • output goes to the terminal, unless you redirect; > command > output.log [&] saves output to the specified file; optional & runs the job in the background • You can “daisy-chain” commands, by piping the output: > ls -l | grep “ 0 “ will list all zero-length files in the current directory Galloway, “X-ray data analysis”

12. The shell & customizing it • The .cshrc file contains commands to be run before the start of every terminal session. You can set environment variables, run programs, and many other things. 2 examples: • alias allows you to make shortcuts to frequently used (but long) commands; > alias s 'extremely long command' Then at the prompt just type “s”. Type > alias s to show what the command is aliased to • prompt allows you to customize your prompt; > set prompt = "%B%m [%t] [%~] >%b” then your prompt lists the machine name, current time (%t), and current path (%~) Galloway, “X-ray data analysis”

13. The shell & customising it 2 • List environment variables with > setenv SHELL=/bin/tcsh USER=duncang USERNAME=duncang : • List individually with > echo $PATH /data/xra1/software/ciao/ots/saord:/data… • Set (also) using setenv > setenv PATH /usr/bin/newpath:${PATH} (curly brackets not always necessary) • PATH is how the shell finds commands, and is perhaps the most important environment variable! Galloway, “X-ray data analysis”

14. Directories, paths etc. • mkdir to create a new directory • ls to show its contents • mv to rename a file, or change it’s directory • rm to remove a file • You can change to another directory using > cd /new/directory • Your home directory is just ~; for other users, it’s ~username (e.g. ~duncang) • To return to your home directory: > cd ~ Or just > cd Galloway, “X-ray data analysis”

15. Directories, paths etc. 2 • Temporarily visit another directory using pushd: > pushd /new/directory (cd to /new/directory and saves the old to a stack). Then popd to return to the last stack position • Alternatively, you can make permanent softlinks: > ln -s /extremely/long/path shortpath makes a link called “shortpath” as a subdirectory in the current directory, pointing instead to the specified location • If you get lost, pwd = print working directory Galloway, “X-ray data analysis”

16. Permissions and security > ls -l (“long” format) returns -rw-r--r-- 1 duncang astro 45690 2007-01-24 13:16 1900flux.ps drwxrwxr-x 2 duncang astro 4096 2007-01-23 17:02 archive -rw-rw-r-- 1 duncang astro 202059 2007-02-19 14:54 astro-ph_notable.txt lrwxrwxrwx 1 duncang astro 16 2006-03-21 14:04 burst -> /data/xra1/burst : • Leading “d” indicates a directory; “l” is a link. • Each of the next 3 sets of 3 letters indicate access permissions for (u)ser, (g)roup, and (o)ther: (r)ead, (w)rite and e(x)ecute • Control permissions via chmod, e.g. add write access for the group: > chmod g+w 1900flux.ps Also a shorthand numerical format… see the man page Galloway, “X-ray data analysis”

17. Compiling and running programs • Don’t know where a command (executable file) lives? Try > which command If it’s on the path, this will show it. • To run a program, just type it’s name, then return. To suspend a running (interactive) program, press ^Z. To continue running in the background, type bg. To bring it back to the foreground, type fg. Galloway, “X-ray data analysis”

18. Compiling and running programs 3 • To run a job initially in the background, use &: > command & (this doesn’t work well with interactive commands!) • To list all the running commands > ps PID TTY TIME CMD 11517 pts/6 00:00:00 tcsh 13109 pts/6 00:00:00 ps Use the PID to “kill” (stop) a process, if necessary: > kill [-9] 11517 Galloway, “X-ray data analysis”

19. Compiling and running programs 4 • For those running models on shared machines, please be “nice” (sorry) i.e. > nice model assigns a priority to the process, ensuring it only takes processor time when nothing of higher priority is running. See also renice. Also > nohup nice model will ensure the program is not terminated when you log out (thanks Matthias). You can also try > time model to see how long something takes • top shows a ranked list of running processes and their share of system resources; q to exit • programming languages including C and Fortran (gcc, g77) are included with Linux Galloway, “X-ray data analysis”

20. Working with text files • You will need to decide on an editor: emacs or vi(m)? To be fair, I would say they are both pretty strange for a new user, but they are both extremely powerful. To help you choose, there’s a vim tutorial: > vimtutor For the emacs equivalent, > emacs then ^H and T • You could also use the text editor built into the Ubuntu GUI, but you may then have problems working remotely • (You will likely format your thesis in LaTeX, which requires a text file as input). Galloway, “X-ray data analysis”

21. Working with text files 2 • cat lists a text file in the terminal; more does the same but allows pagination, and less even adds scrolling up and down • grep searches within text files: > grep string textfile.txt • find does the equivalent on a bunch of files. The syntax is rather complicated; I use > alias find ’find ./ -type f -print | grep' Galloway, “X-ray data analysis”

22. Working with text files 3 • diff will compare two files and list the differences: > diff file1.txt file2.txt • head prints the first few lines, tail the last • cut will select out columns, e.g. from a (text) data file; paste will put them back together • For even more advanced text processing, try sed or awk, or even perl Galloway, “X-ray data analysis”

23. Communicating with the outside world • There are a number of email packages built in to Ubuntu, including Evolution, Mozilla Mail, Thunderbird (part of the Firefox browser?). The other students may be able to suggest other options (but for god’s sake don’t use PINE or elm) • You can also read your Physics email via the web • ssh is used to open a terminal window to another machine: > ssh xray.ph.unimelb.edu.au • scp can be used to copy a file from one machine to another: > scp file.txt duncang@xray.ph.unimelb.edu.au:”~" • sftp & wget can be used to get files of remote servers, particularly useful for downloading data, packages etc. files • Compressing large files speeds transfers! Use > gzip file.txt to compress (creates file.txt.gz), & gzip -d to unpack Galloway, “X-ray data analysis”

24. Communicating 2 • Printing for postscript files with lp: > lp -P astrops file.ps Astrops is the printer in 359; substitute bakercolour for the colour printer upstairs in the baker lab • Jobs go into a queue, which you can examine using > lpq -P astrops • To set astrops as your default printer, set the PRINTER environment variable to astrops • To print duplex, add the option -o sides=two-sided-long-edge • See also Ivy’s message re. your personal webpage. This is a useful thing to have! Galloway, “X-ray data analysis”

25. Advanced topics • tcsh (and other shells) have the ability to run a file full of commands as a shell script. This is extremely useful for oft-repeated commands. Just > tcsh script.sh • Shells also have programming constructs like if…then, for loops etc. foreach is particularly useful: > foreach file (*.txt) foreach? mv $file ${file}.backup foreach? end Galloway, “X-ray data analysis”

26. Other useful tools • Perl is a scripting language with C-like syntax, many available libraries; particularly good for processing text files • IDL is terrific for all sorts of plotting and also allows you to write scripts, create “widgets”, and do many kinds of analysis. A huge repository of available software (particularly the IDLAstro tools) • Pgplot is a much simpler plotting package which can be linked in with Fortran or C code • Gnuplot is another plotting package which is good for quick plots of column-delimited data in text files Galloway, “X-ray data analysis”

27. Resources • Later I will try to post some example scripts, and a command summary list on the astro web page http://astro.ph.unimelb.edu.au • The Linux documentation project http://tldp.org including “Introduction to Linux: A Hands-on Guide” at http://tldp.org/LDP/intro-linux/html/index.html • Ubuntu documentation http://www.ubuntu.com/support/documentation • Ubuntu wiki http://www.ubuntu.com/wiki • Wikiguide (some tcsh incompatibilities) http://en.wikibooks.org/wiki/Guide_to_Unix • Linux command line tips http://www.pixelbeat.org/cmdline.html • Linux cartoons http://folk.uio.no/hpv/linuxtoons • In the beginning was the command line http://www.cryptonomicon.com/beginning.html Galloway, “X-ray data analysis”