Linux & Joker – An Introduction

1. ICT Cyber Infrastructure Architecture Team Linux & Joker – An Introduction

2. Part I Presenter: Dusan Jolovic Presentation by: Jelena Karapetrovic Tracey Fernandez

3. Why Linux? Why Joker? • Linux is ‘light’ OS • It is secure and reliable • Most supercomputers run on Linux • Joker is a supercomputer with 238 cores • Joker can help you with heavy computational tasks • Saves time

4. Do you have an account? • Why do you need an account? • Where to register for an account? • https://hpc.nmsu.edu/resources/

5. Accessing Joker from within the NMSU domain using a terminal emulator. • Joker may be accessed by any one of the many SSH terminal programs (such as “PuTTY”). • Download and install PuTTY (Google “putty ssh” for download URL). • Run putty. Enter joker.nmsu.edu in Hostname window. • Click open.

7. Enter your Joker account username and password when prompted. You will then be in the terminal window ready to go.

8. Accessing Joker from outside NMSU’s domain with a terminal emulator. • Joker is not exposed to “outside” connections. • You must first connect using NMSU's VPN service. • The VPN client may be found at “VPN.NMSU.EDU”. • Install and run the client.

9. Enter your NMSU Username and Password. Once connected, you may then connect to Joker with a terminal emulator (PuTTY).

10. Upload and download files to and from Joker using sftp. • Download and install a sftp client (let’s use WinSCP). • Google winscp, go to the site download, installation package. • After install, run the app. and enter the login information. Note: If “outside” the NMSU domain, you will need to connect with the VPN client before the sftp app.

11. WinSCP sftp Window • Local drive on left. • Joker on right. • Windows Explorer functionality (drag and drop, right click options, …) local drive on left, joker on right

12. Navigation • File System Organization: hierarchical directory structure • The files on Linux are organized in a tree-like pattern of directories. • Linux does not employ the concept of drive letters (drive letters used by legacy operating system split the file system into a series of different trees). Linux always has a single tree.

13. Root directory- the first directory; contains files and subdirectories. • The working directory- the directory you are in. When you first log on to a Linux system, the working directory is set to your home directory.

14. Commands • pwd- used to find the name of the working directory • ls- used to list the files in the working directory (ls -a will list hidden files as well) • cd- used to change the working directory (type cd followed by the pathname of the desired working directory) • Passwd – to change password • -help (eg: type ‘module –help)

15. Rules about file names • File names that begin with a period character are hidden. This only means that ls will not list them unless you say ls -a. • Case sensitive (“File1” and “file1” are not the same thing) • Do not use spaces • Limit punctuation to: • Period • Dash • Underscore.

16. Use the following commands: pwd, ls, cd • pwd (find the name of the working directory) • ls (list the files in the working directory) • cd/usr/bin (change the working directory) • pwd (find the name of the working directory- note it is different now) • ls (list the files in the working directory) • cd /home (change the working directory to home) • pwd (find the name of the working directory- note the change)

17. List hidden files: • cd /usr/bin (change the working directory) • ls (list the files in the working directory) • ls -a (list the files, including hidden files in the working directory. Note the difference!)

18. Remember • Commands • The following commands are amongst the most frequently used • mkdir- create directories • mv- move or rename files and directories • cp- copy files and directories • rm- remove files and directories

19. Useful Resources • https://cia.nmsu.edu/other-resources/ • http://linuxcommand.org/index.php • http://www.howtogeek.com/199687/how-to-quickly-create-a-text-file-using-the-command-line-in-linux/ • https://www.chpc.utah.edu/presentations/IntroLinux3parts.php • http://www.thegeekstuff.com/2010/11/50-linux-commands/?utm_source=feedburner • https://www-uxsup.csx.cam.ac.uk/pub/doc/suse/suse9.0/userguide-9.0/ch24s04.html

20. About Us • https://cia.nmsu.edu/meet-the-team/

21. Contact Info • JelenaKarapetroviciskra@nmsu.edu • Tracey Fernandez tfernnnn@nmsu.edu • Dusan Jolovic djolovic@nmsu.edu

22. Part II Presenter: Poo-Chou Su Prepared by: Poo-Chou Su Mohammed Tanash

23. UNIX TEXT EDITORS Editor Features • Enter text • Search and replace • Copy, cut and paste • Undo and redo • Importing and exporting text • Save and cancel Common Text Editors: • Vim • Pico • Nano • Emacs • Mousepad • Xedit

24. VIMTEXT EDITORS Input Mode • Creat/Open a file • Vim [File-name].[Ext] • Close a file • q(Quit) or wq(Write than Quit) • Online source • http://www.openvim.com/ • https://linuxconfig.org/vim-tutorial • http://vim.wikia.com/wiki/Tutorial • Google “vi cheatsheet” Esc Ins Command Mode Esc v Visual Mode

25. Module System • Allows you to add various path definitions to your shell environment. • Default compilers, applications and libraries can be set by individual or combinations of Modules commands. • Modules are not applications, rather they simply add the location of applications to your environment.

26. Module System • List the available Modules using the command: • $ module avail • You can load a module using the command: • $ module load <module name>

27. Module System You can load multiple modules in one line using the command • $ module load <module name1><module name2>... You can view the added modules using the command • $ module list

28. Module System • To unload a module you can use the command • $ module unload <module name> • To unload more than one out of many loaded modules use the command • $ module unload <module name1> <module name2>…

29. Module System • To unload all the loaded modules, use either commands • $ module purge • $ module clear

30. Advanced command “Awk” • Awk is a shell programming language designed for text processing and typically used as a data extraction and reporting tool. • Awk command searches files for text containing a pattern. When a line or text matches, awk performs a specific action on that line/text. • Syntax: • awk 'Program' input-file1 input-file2 ... awk -f PROGRAM-FILE input-file1 input-file2 ... • Online Sources: • https://www.tutorialspoint.com/unix_commands/awk.htm • https://www.tutorialspoint.com/awk/index.htm

31. Advanced command “Sed” • sed, short for "stream editor", allows you to filter and transform text. • Sed is used to perform basic text transformations on an input stream (a file, or input from a pipeline). • Whenever you want to make changes to the file automatically, sed comes in handy to do this. • Online source • https://www.gnu.org/software/sed/manual/sed.html#Introduction • http://www.computerhope.com/unix/used.htm • http://www.grymoire.com/Unix/Sed.html#uh-0

32. Use Joker • User guide • https://hpc.nmsu.edu/hpc-user-guide/ • IMPORTANT • Test on head nodebefore submit scriptto compute nodes • Use SMALL inputSIZE

33. The Slurm Scheduler • Job scheduler implemented on Joker • Mostly used commands • sinforeports the state of partitions and nodes managed by Slurm • squeuereports the state of jobs or job steps

34. The Slurm Scheduler • Mostly used commands (cont’d) • srunis used to submit a job for execution or initiate job steps in real time

35. The Slurm Scheduler • Mostly used commands (cont’d) • sbatchis used to submit a job script for later execution

36. The Slurm Scheduler • Mostly used commands (cont’d) • scancel is used to cancel a pending or running job or job step • Tutorial • https://hpc.nmsu.edu/hpc-user-guide/using-slurm-on-joker/

37. How to use Gaussian on Joker • Not ready on Module yet • Run set up script before you use Gaussian • Complete and test gaussian script on head node • Submit the script to compute node using sbatch Gaussian_script.sh • Example script • http://psurc-docs.readthedocs.io/en/latest/research_software/g09.html

38. X Windows • Graphic User Interface for Linux terminal on Windows OS • Xming • http://www.straightrunning.com/XmingNotes/