1 / 27

Lecture 1 Basic Skills

Lecture 1 Basic Skills. Presenter Name Presenter Institution Presenter email address Grid Summer Workshop June 21-25, 2004. Welcome!. You are at the Grid Summer School

avani
Download Presentation

Lecture 1 Basic Skills

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecture 1Basic Skills Presenter Name Presenter Institution Presenter email address Grid Summer Workshop June 21-25, 2004 Lecture 1: Basic Skills

  2. Welcome! • You are at the Grid Summer School • If you are looking for the underwater basket weaving classes, they are two block to the west. • Sponsored by: • Center for Gravitational Wave Astronomy • GriPhyN • iVDGL • NMI GRIDS Center Lecture 1: Basic Skills

  3. Goals for this week • Learn about Grids • What is a grid? • How do you… • Submit jobs? • Transfer data? • Cope with failure? • Where are Grids going? • Use grids • Hands-on exercises • Local and remote grids • Format of the week • Morning • Lecture • Exercises • Lunch! Mmm… • Afternoon • Lecture • Exercises Lecture 1: Basic Skills

  4. Day 1 • Basic Skills • What is a Grid? • Basics of Using a Grid Lecture 1: Basic Skills

  5. Day 2 • Running a real application on a grid • What requirements do real applications have? • How should you build a real application? • Taking care of an application • Condor-G • DAGMan • Staging your application Lecture 1: Basic Skills

  6. Day 3 Lecture 1: Basic Skills

  7. Day 4 Lecture 1: Basic Skills

  8. Day 5 Lecture 1: Basic Skills

  9. Without further adieu... Lecture 1: Basic Skills

  10. Step 1: Basic Networking Skills • We apologize in advance if you already know these concepts. • We want to make sure that there is a certain basic level of understanding for all students. Lecture 1: Basic Skills

  11. IP Addresses • All computers on the Internet use TCP/IP. • All computers have at least one IP address. • 32-bit number • Written as four numbers, like: 128.105.3.61 • An IP Address identifies a network interface, not a computer. • A computer can have multiple IP addresses. Lecture 1: Basic Skills

  12. DNS • DNS maps IP addresses to names, and vice-versa • www.amazon.com  207.171.163.30 • Discover this with “host” or “nslookup” or “dig” • Try all three—how do they differ? Lecture 1: Basic Skills

  13. Whois • Who owns or runs a domain? • Amazon.com: Amazon.com, Inc. (HOS276-ORG) hostmaster@AMAZON.COM PO BOX 81226 SEATTLE, WA 98108-1300 US +1 206 266 4064 fax: +1 206 266 7010 • Discover this with “whois”. Lecture 1: Basic Skills

  14. Note that a computer can have multiple names Ping! Are you awake? • Is a computer on the network? • Use ping to find out % ping www.cs.wisc.edu PING marzipan.cs.wisc.edu (128.105.7.11) 56(84) bytes of data. 64 bytes from marzipan.cs.wisc.edu (128.105.7.11): icmp_seq=1 ttl=63 time=0.327 ms 64 bytes from marzipan.cs.wisc.edu (128.105.7.11): icmp_seq=2 ttl=63 time=0.241 ms --- marzipan.cs.wisc.edu ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 2611ms rtt min/avg/max/mdev = 0.241/0.276/0.327/0.036 ms Lecture 1: Basic Skills

  15. Internet routes • Between you and a computer on the network, there is an often complex route. % traceroute www.cs.uwm.edu traceroute to miller.cs.uwm.edu (129.89.143.24), 30 hops max, 40 byte packets 1 svi-121.cisco1.cs.wisc.edu.105.128.in-addr.arpa (128.105.121.248) 0.423 ms 0.242 ms 0.227 ms 2 rh-cssc-b280c-2-core-vlan-492.net.wisc.edu (144.92.128.186) 0.404 ms 4.985 ms 0.489 ms … snip… 6 r-uwmilwaukee-isp-atm1-0-1.wiscnet.net (140.189.8.2) 2.730 ms 2.603 ms 2.689 ms 7 space-needle-mke.csd.uwm.edu (216.56.1.194) 2.836 ms 2.718 ms 2.748 ms 8 miller.cs.uwm.edu (129.89.38.24) 2.754 ms * 2.796 ms Lecture 1: Basic Skills

  16. Port numbers • A port number indicates which program to talk to on a computer. • Some port numbers are standard: • HTTP (web): port 80 • SMTP (mail): port 25 • Ping: port 7 • Some port numbers are assigned dynamically when you run a server. Lecture 1: Basic Skills

  17. Netstat • Netstat can answer the question: is a program running on a port on the local computer. netstat --protocol=inet –l tcp 0 0 *:finger *:* LISTEN • -l meant “listening for connections”. Look for active connections: netstat --protocol=inet | grep ssh % netstat --protocol=inet | grep ssh tcp 0 0 chopin.cs.wisc.edu:ssh ppp-67-38-160-108:20715 ESTABLISHED tcp 0 0 chopin.cs.wisc.edu:ssh 68.185.181.47:1176 ESTABLISHED … Lecture 1: Basic Skills

  18. Telnet • Telnet isn’t just for remote access to a computer • Telnet can tell you if some remote services are running correctly. • Is ssh running? • Find ssh port number in /etc/services. It’s 22. • telnet <host> 22. Example: telnet beak.cs.wisc.edu 22 Trying 128.105.146.14... Connected to beak.cs.wisc.edu (128.105.146.14). Escape character is '^]'. SSH-1.99-OpenSSH_3.6.1p2 ^] (That is control-right bracket) telnet> quit Lecture 1: Basic Skills

  19. Telnet & HTTP (Just for fun) % telnet www.cs.wisc.edu 80 Trying 128.105.7.31... Connected to www.cs.wisc.edu (128.105.7.31). Escape character is '^]'. GET http://www.cs.wisc.edu/~roy/index.html <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN"> <html> <head> <title> Alain Roy</title> … snip … • I typed the lines in red, and got back a web page. Lecture 1: Basic Skills

  20. Why are we telling you this? • We have a shameful secret: • Maintaining a grid is hard work. Things break. You will get involved in debugging them. • You can use all of these commands to figure out where a problem lies. Lecture 1: Basic Skills

  21. Example • Bad complaint: Waaah!!! My grid server-thingy is down • Good complaint: The Globus gatekeeper isn’t running, even though the computer is up. • Is the computer up? Use ping • Is the computer accessible on the network? Use traceroute • Can you telnet to the gatekeeper? Lecture 1: Basic Skills

  22. Grid Without a Grid (GWAG) • Now we’re going to learn how to do grid work without using grid technology. • What is a grid? We’ll tell you this afternoon. For now: • Run jobs on other computers • Transfer data • Discover information • Do it all securely • Could we have thought of a worse acronym? GWAG? Lecture 1: Basic Skills

  23. GWAG: Security • Explain OpenSSL • Explain OpenSSH • Explain HTTPS • Note password authentication • Describe ssh keygen stuff, to eliminate passwords. Note how this has to be set up on a per-computer basis. • This needs to be fleshed out. Lecture 1: Basic Skills

  24. GWAG: Job submission • ssh can run jobs remotely: • ssh roy@beak.cs.wisc.edu ls • This expects a few things: • Your program already exists on the remote computer • A shell is invoked to set up your environment, so you can find ls. Lecture 1: Basic Skills

  25. GWAG: File Transfer • How do you transfer data to be executed? • scp executable roy@beak.cs.wisc.edu:. • Note the final period at the end of that line • How do you transfer the output back? • scp roy@beak.cs.wisc.edu:output . • Submitting your job takes a minimum of three commands. If you don’t set up your keys, you’ll type your password three times. Lecture 1: Basic Skills

  26. GWAG: Issues • Issues you will find: • How do you run three jobs sequentially that require data transfer? You just run each command in a row. • How do you run three jobs in parallel that require data transfer? • Make each transfer/job/transfer a script. Run the scripts in parallel from the command line • How do you deal with failures? • What if scp or ssh returns an error? • What if scp or ssh never returns? (It hangs) • How do you keep track of everything? • In an ad-hoc way? Lecture 1: Basic Skills

  27. GWAG: Dealing With the Issues • We will talk about grid solutions to these problems. • Keep an eye out for: • GSI for security • GRAM or Condor-G for transfer of input/output files • Condor-G for job reliability • DAGMan for running sets of jobs Lecture 1: Basic Skills

More Related