The Gateway Computational Web Portal

1. The Gateway Computational Web Portal Marlon Pierce, Choonhan Youn, Geoffrey Fox ERDC, August 16 2001

2. Tutorial Overview • Demo • Grid and Gateway Overviews • HTML Forms • Java QuickStart Guide • JavaServer Pages Overview • Gateway JSP Tools • WebFlow Module Development • Installation and Security Issues

3. Computational Grids Survey A brief introduction to computational grid projects and goals.

4. What Is a Computational Grid? • Grids link distributed scientific resources. • Resources can be geographically, politically distributed • Goal: provide means for sharing resources between organizations. • Example “high-end” resources: • Supercomputers and clusters • Mass storage • Advanced visualization (CAVES) and collaboration (Access Grid). • Particle colliders, telescopes, earthquake detectors • www.globus.org/research/papers/anatomy.pdf

5. What Does a Grid Need? • Multi-institutional security • PKI or Kerberos • Information services • Manage, store, deliver information about resources. • Use information to make decisions • Scheduling and Queuing • Advance reservation • Meta-queuing • Remote execution, file transfer, monitoring

6. Example of a Grid Problem:CERN’s Large Hadron Collider • Goes on-line in 2005 • Will generate petabytes of raw, distributed data, terabytes of event summary data. • Computing resources for data analysis will be distributed between CERN and regional centers spread all over the world • 1500-2000 people will collaborate on experiments.

7. Grid Projects • Grid Infrastructure • Condor: www.cs.wisc.edu • Globus: www.globus.org • Legion: www.cs.virginia.edu/~legion • Grid Applications • Netsolve: www.cs.utk.edu/netsolve • Ninf: www.etl.go.jp • Global Grid Forum: www.gridforum.org

8. Examples of Deployed Grids • NASA’s Information Power Grid • Links NASA’s Ames, Glenn, and Langley Centers. • LaunchPad currently available • www.ipg.nasa.gov • DOE’s ASCI Distributed Resource Management • Links classified computing resources at Lawrence Livermore, Los Alamos, and Sandia National Labs. • Full deployment scheduled by Nov 2001.

9. Latest Grid News • NSF will spend $53 million on the Distributed Terascale Facility (DTF) • 13.6 teraflops, 600 terabytes, 40 Gigabit/sec • DTF sites: NCSA, SDSC, Argonne, CalTech • Industry partners: IBM, Intel, Qwest • See www.ncsa.uiuc.edu/News/Access/Releases for more information (August 9).

10. Example: Globus • Run applications remotely: • globus-job-run: interactive. • globus-job-submit: batch for PBS, LSF. • globusrun: most general version (RSL). • Split jobs between hosts. • Send and retrieve data securely (PKI). • Monitor jobs remotely. • Monitor hosts remotely.

11. What’s the Problem? • Globus client must be installed on desktop • Difficult installation • No ubiquitous access (PDAs, your grandmother’s PC) • Typical solution is to support Globus at particular sites and have users remotely log in. • Problems arise because many users are not Unix-savvy. • Lots of new commands to learn.

12. Computational Portals • Computational portals are designed to simplify access to grid technologies. • Also provide coarse-grained grid approach that ties grid and non-grid resources. • Not everyone uses Technology X. • Not everything at a TechX supporting site will use TechX. • Different TechX sites may remain separate.

13. Gateway Architecture • Gateway is implemented in a three-tiered architecture. • Browser Front End • JSP dynamically generates HTML pages. • Component Middle Tier • JavaBeans on the web server. • Distributed WebFlow servers. • HPC back end • Link to grid and non-grid services with rsh, ssh. • More sophisticated interfaces can be built.

14. Web Browser And Client Applications Web Browser And Client Applications HTTP(S) HTTP(S) JVM JavaBean Local Service JavaBean Service Proxy Web Server And Servlet Engine JavaBean Service Proxy JavaBean Local Service SECIOP SECIOP WebFlow Child Server WebFlow Child Server WebFlow Master Server WebFlow Child Server WebFlow Child Server RSH,SSH RSH,SSH HPC+LSF Data Storage HPC+PBS Condor Flock Globus Grid

15. Gateway Design Goals • Build a working portal for users. • Produce a tool chest for portal developers. • Targeted Services: • File Transfer • Problem organization and session archiving • Batch script generation • Job submission • Job monitoring • Shared visualization • Security

16. Levels of Use • Users and Admins can do everything through web. • Portal developers may want to edit pages, use our components. • Advanced developers can write modules. Portal Users and Administrators Sophistication Portal Developers Module Developers

17. Gateway Descriptors How to add your codes and your hosts to the portal.

18. Gateway Descriptors • Form the base of portal for any particular field. • Collect static info about applications, hosts in an XML data record. • Application Descriptors describe how to run codes. • Host descriptors describe HPC systems. • Users are described by another mechanism.

19. Sample Application Descriptor <XSIL Name="ANSYS" Type="csm.parseXMLDesc"> <Param Name="NumberOfInParams">0</Param> <Param Name="NumberOfInFiles">1</Param> <Param Name="NumberOfOutParams">0</Param> <Param Name="NumberOfOutFiles">1</Param> <Param Name="IOStyle">StandardIO</Param>

20. Sample Host Descriptor <XSIL Name="Modi4" Type="csm.parseXMLHost"> <Param Name="HostName">modi4</Param> <Param Name="QueueType">LSF</Param> <Param Name="ExecPath">/usr/bin/ansys57</Param> <Param Name="WorkDir">/scratch</Param> <Param Name="QsubPath">/usr/bin/bsub</Param>

21. Adding Your Application • We store application and host data in a single file. • Applications “contain” hosts. • You can create and edit this by hand, or • You can use administrator interface to edit the data record. • Admin interface also lets your verify data. • Did I give the right executable path.

22. Java Quick Start Guide A quick and dirty overview of the Java programming language.

23. Basic Elements • The Java language resembles C/C++: • Primitive types: int,float, double, char, boolean • Strings are actually classes (more later on this) • Standard control structures like for and while loops, if statements, case/switch statements, try/catch blocks. • Some important differences from C/C++: • No pointers • Method arguments are always passed by value. • No preprocessors or macros.

24. If/Else Statement Format if(condition1) { //conditionally executed code } else if(condition2) { //conditionally executed code } else { //conditionally executed code }

25. For Loops • Syntax: for(int i=0;i<MAX;i++) { //executed code } • MAX is a variable defined elsewhere.

26. Java Classes • Java is object-oriented • Classes encapsulates data and methods (functions) within a single entity. • Objects are instances of classes. • Analogy: the declaration “int i” creates an instance of an integer. • The Java SDK comes with an extensive library of pre-defined classes for you to use. • See the online API: • http://java.sun.com/j2se/1.3/docs/api/

27. Example Class: Hashtable • Hashtable allows you to store name/value pairs. • To create a new hashtable object: Hashtable myhash=new Hashtable(); • You can now use Hashtable methods myhash.put(“MyName”,”Marlon”); String name= (String)myhash.get(“MyName”);

28. References • The Java web site has API documentation and tutorials: • http://java.sun.com/j2se/1.3/docs/ • Excellent reference text: • “Core Java” Volumes I and II by Cay Horstmann and Gary Cornell (Prentice Hall) • O’Reilly publishes the API: • “Java in a Nutshell” by David Flanagan

29. Interactive HTML Using HTML forms to tie widgets to server actions.

30. The <form> Tag • The <form> …</form> tag pair surround all HTML input types. • Format: <form name=“myform” method=“Post” action=“/GOW/servlet/someAction”> … <!-- Input tags go here --> </form> • The “action” attribute specifies what happens when an input button is pressed. • Can be CGI, a servlet, or a JSP page

31. The <input> Tag • Input tags define text fields, submit buttons, radio buttons, menus, … • Several can be combined within a single <form> • Format: <form method=“GET” action=“servlet/myServlet”> <input type=“text” name=“text” value=“Sample”> <input type=“submit”> </form>

32. Putting It All Together <html> <body> <form action=“someaction” method=“Post”> Please type your name: <input type=“text” name=“myname” value=“Marlon”> <input type=“submit”> </form> </body> </html>

33. What Happens When I Click the Button? • The CGI Script/Servlet/JSP/… specified in the action receives all name/value fields of <input>. • These are sent in the HTTP request to the server. • Usually you should use “Post” instead of “Get” • No size limit on requests with Post • Requests are not shown in the browser location field. • The server-side code usually returns output to the browser.

34. JavaServer Pages (JSP) Putting Java into HTML to build dynamic web pages.

35. What Are JavaServer Pages? • JSP let you embed Java code into your HTML web pages. • Use .jsp extension • When the page is loaded by the browser, JSP is translated into a servlet, executed, and you see the output. • To run this, you need a special server • Apache’s Tomcat, IBM’s WebSphere, …

36. Embedding Scriptlets <%@ page import=“java.util.Date” %> <%@ page import=“java.util.Hashtable”%> <html><body> Hello, Marlon <% Hashtable Myhash=new Hashtable(); Date now=new Date(); Myhash.put(“date”,now); %> The time is <%= now.toString() %> <!-- More HTML and scriptlets to follow. --> </body></html>

37. What Does It Mean? • The “import” statements at the top point to the location of the Java class files. • Everything between <% and %> is interpreted as Java. • This sections are called scriptlets. • Java can be in-lined with html using the <%= %> tags. • These are referred to as expressions.

38. Ex: For Loop of Radio Buttons <% for (int i=0;i<5;i++) { %> Radio<%= i %> <input type=“radio”> <% } %>

39. Using JavaBeans in JSP • As presented so far, JSP still requires extensive knowledge of Java API. • O’Reilly’s Java API “Nutshell” is 600 pages. • JavaBeans are custom components that encapsulate specific sets of functions. • Develop a small set of classes for area-specific tasks. • It is good design to separate display from control code so that each is reusable. • You don’t want sprawling JSP pages.

40. Separation of Responsibility Portal Users Define Functionality, L&F Web Developers Work on L&F Java Programmers Develop Beans

41. JavaBeans in JSP • Create an instance “gem” of the gemBean class <jsp:useBean id=“gem” class=“gem.gemBean” scope=“session”/> • You can now use “gem” like any other object gem.loadData(); gem.runSim(); • By setting the scope, pages can share beans. • You can also use this tag to initialize once • Other HTML-like tags exist for accessing data.

42. Overview of Gateway JSPs • Welcome Page • Sets up most beans • Buttons are included in TrackNavigator.jsp

44. CodeSelect.jsp • Codes are read in from the Application Descriptor file. • The page is generated automatically from the descriptor. • Problem name is mapped to user context directory, where session data will be stored.

46. JobSubmit.jsp • Based on selected code, forms are generated automatically. • Application Descriptor file specifies the number of input files, parameters, etc.

48. Submitted.jsp • Shows the generated queue script, based on user requests. • The user has one last chance to edit. • The “Submit” button can be tied to an action to run the script.

50. ReturnPage.jsp • Job has been submitted. • The track navigator is again included at the bottom of the page.