Developing grid portals using portlets
This presentation is the property of its rightful owner.
Sponsored Links
1 / 80

Developing Grid Portals Using Portlets PowerPoint PPT Presentation


  • 118 Views
  • Uploaded on
  • Presentation posted in: General

Developing Grid Portals Using Portlets. Marlon Pierce Community Grids Lab Indiana University. Overview of Material. General remarks on portals and portlets. General remarks to provide context for the remainder of the session. OGCE Portal capabilities. Summary of things we do in OGCE.

Download Presentation

Developing Grid Portals Using Portlets

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Developing grid portals using portlets

Developing Grid Portals Using Portlets

Marlon Pierce

Community Grids Lab

Indiana University


Overview of material

Overview of Material

  • General remarks on portals and portlets.

    • General remarks to provide context for the remainder of the session.

  • OGCE Portal capabilities.

    • Summary of things we do in OGCE.

  • JSR 168 Review

  • Developing OGCE portlets

    • How to develop OGCE compatible portlets.

    • With launching points for Dennis and Gregor.

  • Useful third party testing tool: HttpUnit.

  • A brief JSR 168 tutorial.

  • At the risk of a low tutorial rating, I will cover higher level issues. Slides on nuts and bolts details are provided for homework.


Towards a common grid client hosting environment

Towards A Common Grid Client Hosting Environment

Grid portal background and emerging common frameworks


What is a grid computing portal

What Is a Grid Computing Portal?

  • Browser based user interface for accessing grid and other services

    • “Live” dynamic pages available to authenticated, authorized users.

    • Use(d) Java/Perl/Python COGs

    • Manage credentials, launch jobs, manage files, etc.

    • Hide Grid complexities like RSL

    • Can run from anywhere

    • Unlike user desktop clients, connections go through portal server, so overcome firewall/NAT issues

  • Combine “Science Grid” with traditional web portal capabilities

    • Get web pages for news feeds

    • Post and share documents

    • Search engine interfaces, calendars, etc.

    • Enabled by portlets, as we will see.

  • Customizable interfaces and user roles/views


What a grid portal is is not

What a Grid Portal Is/Is Not

  • It is

    • A tool for aggregating and managing web content

    • A user customizable view of these Web content pieces.

      • You see what you want/can see.

      • But you must log in.

    • Implemented on top of standard services

      • Like login, authorization, customization.

      • May include collaboration, etc, that depend on login.

    • A way to accomplish Grid tasks through browsers:

      • Launch, monitor jobs

      • Move files

      • Run science applications based on these services.

    • Compatible with emerging standards and best practices (such as portlets, JSR 168 and WSRP).

  • It is not (just)

    • A web page

    • A collection of links

    • An applet


Which is the portal

Which Is the Portal?


Which is the computing portal

Which Is the Computing Portal?

  • In fairness, the screenshots are not large enough to see, but you have to log in to the one on the right.

    • Point is that they are superficially similar to browser users, but have many differences under the hood.

  • The screen shot on the left is of the NASA JPL QuakeSim project page.

    • http://quakesim.jpl.nasa.gov/

  • The screen shot on the right is the NASA JPL QuakeSim portal.

    • http://www.complexity.ucs.indiana.edu:8282

    • Go here to run QuakeSim earthquake simulation codes, access earthquake databases, etc.


Let 10 000 flowers bloom

Let 10,000 Flowers Bloom

  • Many portal projects have been launched since late ’90s.

    • HotPage from SDSC, NCSA efforts, DOD, DOE Portals, NASA IPG

    • 2002 Special Issue of Concurrency and Computation: Practice and Experience.

  • Continue to be important component of many large projects

    • NEESGrid, DOE SciDAC projects, NASA, NSF, many international efforts

  • Global Grid Forum’s Grid Computing Environments (GCE) Research Group

    • Community forum


Three tiered architecture

Three-tiered architecture is accepted standard for accessing Grid and other services

Three-Tiered Architecture

Grid and

Web

Protocols

JDBC,

Local, or

Remote

Connection

Portal

Client

Stub

Database

Service

Database

Portal User Interface

Portal

Client

Stub

Grid Resource

Broker Service

HPC

or

Compute Cluster

Portal

Client

Stub

Information

and

Data Services

Grid Information

Services, SRB


Problem with portals

Problem with Portals

  • GCE revealed two things

    • Everyone was doing the same thing

      • Not quite, but significant

      • Everyone builds secure logins, remote file manipulation, command execution, access to info servers.

      • Everyone would at least like support for multiple user roles (administrators, users) and customization

    • No one could share components with other groups

      • No well defined way of sharing UI components or making services interoperate.

      • No well defined interfaces to portal services.

  • A research opportunity!

    • Two levels of integration: user interfaces and services

  • Our challenges

    • Stop reinventing things and provide ways for groups to reuse components.

    • Provide a portal marketplace for competing (advanced) services.

    • Provide APIs for service integration


A solution based on components

A Solution based on components

  • A software component is object defined by

    • A precise public interface

    • A semantics that includes a set of “standard” behaviors.

  • A Software component architecture is:

    • A a set of rules for component behavior &

    • A framework in which components can be easily installed and interoperate.

  • The component architecture of choice for the Portal community is the one based on portlets

    • (Java) components that generate content, make local and remote connections to services.

    • Portlet containers manage portlet lifecycles

  • We have now many, many components.

    • So don’t start from scratch.


Things to hate about portals

Things to Hate About Portals

  • If you involved in portal efforts, be aware of the following:

  • Browsers have limited interactivity.

    • Desktop GUIs provide much better interactivity but have other problems.

    • Applets are a solution, but they don’t interact with other parts of the browser very well.

    • Solution: Service Oriented portals let you use services through both portals and grid desktops.

  • Developing really useful user interfaces to your set of services is a time consuming, non-scaling process.

    • Get users involved early in design.

  • Browsers notoriously have incompatible features.

    • Things don’t work the same on IE, Mozilla, Konqueror, etc.

    • Same browsers on Macs, Windows don’t work the same.

    • No substitute for lots of testing.

  • Grid portals need grids.

    • Setting up grids is a fragile process.

    • Plenty of dumb mistakes like setting up CA signing policies that even experts make.

    • You will be blamed when things don’t go smoothly.


Ogce open grid computing environments

OGCE: Open Grid Computing Environments

Marlon Pierce

Indiana University


Nsf nmi project for reusable portal components who we are

NSF NMI Project for Reusable Portal Components: Who We Are

  • University of Chicago

    • Gregor von Laszewski

  • Indiana University

    • Marlon Pierce, Dennis Gannon, Geoffrey Fox, and Beth Plale

  • University of Michigan

    • Charles Severance, Joseph Hardin

  • NCSA/UIUC

    • Jay Alameda, Joe Futrelle

  • Texas Advanced Computing Center/San Diego State University

    • Mary Thomas


What is ogce s release 1

What Is OGCE’s Release 1

  • The OGCE Portal release is based on CHEF/Jetspeed 1.4

  • Available for download and installation from http://www.collab-ogce.org.

  • It comes with many pre-configured capabilities if you want a grid portal “out of the box”.

    • Except for the mysql jar.

    • You must still set up Grid services (MyProxy servers, Globus, etc).

    • Globus version compatibility through the Java CoG.

  • Apache Ant-based installation procedure:

    • Edit one properties file, run ant, and away you go.


User portlets

User Portlets


More user portlets and services

More User Portlets and Services


Grid portlet examples

Grid Portlet Examples

  • We’ll next overview several portal capabilities.

  • Jetspeed/CHEF acts as a clearing house for portal capabilities

    • User interface components can be added in well defined ways.

    • First level of integration

  • All Grid access goes through the Java COG.


Example capability portals for users

Example Capability: Portals for Users

User “Beth”

  • The user contacts the portal server and asks it to do “grid” things on behalf of the user.

  • To make this possible the server needs a “Proxy Certificate”

    • The user has previously stored a proxy cert in a secure MyProxy Server stored with a temporary password.

    • User give the portal server the password and the portal server contacts the proxy server and loads the proxy.

    • The portal server will hold the proxy for the user for a “short amount of time” in the user’s session state.

1. Load my

Proxy

Certificate!

Portal Server

MyProxy

Portlet

2. Give me

Beth’s proxy

certificate

COG

3.

I am

Beth’s

Proxy

MyProxy

Server


Example capability grid context service

Example Capability: Grid Context Service

  • User’s want to be able to use the portal to keep track of lots of things

    • Application and experiment records

      • File metadata, execution parameters, workflow scripts

    • “Favorite” services

      • Useful directory services, indexes, links to important resources

    • Notes and annotations

      • “Scientific Notebooks”


Portlet interfaces to grid context

Portal

Server

Portlet Interfaces to Grid Context

  • A Remote Service Directory Interface

    • Holds references and metadata about application services.

  • User selects interface to application service from the directory browser.

  • Examples: (near completion)

    • Select a link to a Dagman document and invoke the Condor service on the script.

    • Same for GridAnt/Ogre or BPEL workflow script.

    • Factory services for any grid apps that have interactive user interfaces.

Remote

Service

Directory

Service

Remote

Grid

Application

Service


Example capability topic based messaging systems

Example Capability: Topic Based Messaging Systems

  • XML metadata system based on messages.

  • Newsgroups

    • Topic based message posting and administration

  • Citation/reference browsers

    • Topic based, export/import bibtex

  • Portlets sit atop JMS-based message system

    • NaradaBrokering, used in JMS mode.


User privileges for group channels

User Privileges for Group Channels

  • Users request access to specific topics/channels.

    • Granted by administrator for that topic

  • Can request

    • Read/write by browser

    • Read/write by email (newsgroups)

    • Receive/don’t receive attachments.

  • Topic admin can edit these requests.

  • Super admins can manage administrators to topics


Gpir data

Load - aggregated CPU

Downtime data for a machine

Jobs: aggregated queue

MOTD

Nodes: job usage for each machine node

NWS: based on VO and Click model

Grid Monitoring

Based on TACC GMS System

Custom providers

Plans to include MDS3.0 and INCA data uderway

Expanding to include:

queuing system

application profiles

performance data

Application profiles

Doc links

Model allows generic inclusion of any XML data from any recognized source

Need schema

Need query

GPIR Data


Grid portal information repository gpir 1 1

Grid Portal Information Repository (GPIR 1.1)


Gpir components

GPIR Components

  • Web Services Ingestor

    • Web Services Ingestor and clients

    • XML Schemas - can be changed

  • Data Repository

    • Local Cache

    • Archival --> PostgreSQL

  • Web Service Query

    • retrieve data – XML Queries

    • Retrieving current snapshot and archived data

  • Clients

    • GridPort services

    • Portal/Web Interface (Portlets, servlets, JSP)

    • Command line

    • Any that speak web services


What s next for the ogce

What’s Next for the OGCE?

  • JSR 168 Compatible Grid portlet release at Supercomputing.

    • Basic capabilities: MyProxy, GridFTP, GRAM, GPIR, based on CoG 4.

    • Working in uPortal, GridSphere, Jetspeed2, ….

  • Join (all of) us at SC2004 for a portals BOF.

  • Solutions JSR 168 limitations that work in multiple JSR 168 containers.

  • Grid-compatible WSRP portlets.

    • Compatible with Sakai/CHEF 1.2 capabilities.

  • Backward compatibility bridges to previous portlets.

  • Fancy new capabilities available in separate downloads.


A jsr 168 overview

A JSR 168 Overview

A review of the latest portlet developments.


What is jsr 168

What is JSR 168?

  • Defines a standard for vendor container-independent portlet components.

  • Many implementations:

    • Gridsphere, uPortal, WebSphere, Jetspeed2, ….

  • From the portlet development point of view, it is really very simple:

    • You write a java class that extends GenericPortlet.

    • You override/implement several methods inherited from GenericPortlet.

    • You use some supporting classes/interfaces

      • Many are analogous to their servlet equivalents

      • Some (portletsession) actually seem to be trivial wrappers around servlet equivalents in Pluto.


Some terminology

Some Terminology


Some generic portlet methods

Some Generic Portlet Methods


Supporting classes interfaces

Supporting Classes/Interfaces


The infamous big picture

The Infamous Big Picture

  • As a portlet developer, the previous set of classes are all you normally touch.

  • The portlet container (such as Pluto or Gridsphere) is responsible for running your portlets.

    • Init, invoke methods, destroy.

  • Portlets have a very limited way of interacting with the container.

    • It is a black box.

    • The API is basically one-way.


Deploying portlet applications

Deploying Portlet Applications

  • The portlet container (i.e. uPortal) runs as a distinct web application.

    • That is, it has its own directory in tomcat.

    • Moreover, it runs as a separate context, with its own classloader, session management, etc.

  • Portlet applications are deployed as distinct war files/web applications.

    • You go through the container webapp to get to the portlet webapp.

    • Portlets in the same application share jars, classes, and runtime stuff like request and session variables.

    • Portlets in different portlet apps do not share anything.


A critique of jsr 168

A Critique of JSR 168

  • There is no way to share data/objects between portlet applications.

    • So all grid portlets would have to be in the same portlet app.

  • There is no way to extend the portlet API to add such services.

  • There is a lack of general purpose portlets.

    • Right now, you make specific extensions to GenericPortlet for each portlet you develop.

  • JSR 168’s MVC approach is incompatible with Turbine, Struts, ….

    • The issue is managing <form action=“”> and href URLs.

    • No fundamental problem, but this is a gap that must be filled.

    • JSF Portlets are the exception.

  • No defined way to integrate with portal-specific services (i.e. logins).

  • No inter-portlet communication.

  • Despite these problems, JSR168 (and WSRP) are the best available standards.

    • They are compatible.

    • WSRP overcomes many of the JSR 168 limitations.

    • But more work needs to be done to make WSRP ready for Grid portals (Web Service security).


Writing basic ogce portlet templates

Writing Basic OGCE Portlet Templates

Developing Portlets for OGCE Release 1


Writing a velocity portlet

Writing a Velocity Portlet

  • Imagine the following simplistic scenario:

    • I have a web form that helps a user submit a job to a supercomputer.

    • I only need one input form to generate the input file, pick a host computer, etc.

  • The output file is written back to the user’s home directory, so he or she can gridftp it later.

  • Before we can do this, we first need a bit of background on programming portlets.


Velocity portlet example

Velocity Portlet Example

  • Before handling the Grid execution, we will first construct a dummy form.

  • We will use Velocity templates.

    • Jetspeed has better support and documentation for Velocity.

    • Most of the portlets in the release use this.

  • We will examine a JSP example later.


Create the template

Open a new file in nmi/WEB-INF/templates/vm/portlets/html.

Give it a name, myTestApp.vm

Write the Velocity template

Write an xreg file (see next slide).

Restart Tomcat.

Customize your display to show the new portal.

So far, so good.

<h3> Enter the command you want to execute</h3>

<form method="post">

<table>

<tr>

<td> Command Name:</td>

<td> <input type="text" name="runCommnad" value="">

</td>

</tr>

</table>

</form>

Create the Template


Sample jsp portlet template

Sample JSP Portlet Template

<?xml version="1.0" encoding="UTF-8"?>

<registry>

<portlet-entry name="myScienceApp" hidden="false" type="ref"

parent="CustomizerVelocity" application="false">

<meta-info>

<title>Sample Proxy Form</title>

<description>Sample Proxy Form</description>

</meta-info>

<classname>org.apache.jetspeed.portal.portlets.VelocityPortlet</classname>

<parameter name="template" value="myScienceApp" hidden="true"

cachedOnName="true" cachedOnValue="true"/>

<parameter name="action" value="portlets.myScienceAppAction"

hidden="true" cachedOnName="true" cachedOnValue="true"/>

<media-type ref="html"/>

<url cachedOnURL="true"/>

</portlet-entry>

</registry>


Specifying the template

The parts in red (template and action) point to things that you must write.

The line below is used to name the VM template in the XREG.

Points to myScienceApp.vm

<parameter

name="template" value="myScienceApp"

hidden="true" cachedOnName="true”

cachedOnValue="true"/>

Specifying The Template


Actions in templates

Actions in Templates

  • Note our velocity template is just HTML (at this point) with a form action.

    • The action implementation is specified in the XREG file.

    • MyScienceAppAction.java is the code that does the work when you click the button.

  • Jetspeed action managers are responsible for calling your actions.

    • You just need to write the java code, put it in the right place, and connect it to a Velocity template in the XREG file.

    • Jetspeed will take care of the invocation.


Writing an action

Writing An Action

  • A portlet action is just a java class.

  • It should extend VelocityPortletAction

  • You should implement the buildNormalContext() method.

    • This method is called by default when you invoke an HTML form action.

    • This can do anything you want (i.e. make calls to Grid services through the Java COG).

    • You can also implement other action methods.


Getting started

Let’s give our simple portlet an action.

To do this, we first modify the

Don’t forget to shutdown tomcat first.

We then write the action and compile it into nmi/WEB-INF/classes.

See next slide

Set classpath correctly!

Restart the server.

<parameter name="action" value="portlets.myScienceApAction"

hidden="true" cachedOnName="true" cachedOnValue="true"/>

Getting Started


A minimal action myscienceappaction java

A Minimal Action: myScienceAppAction.java

package org.apache.jetspeed.modules.actions.portlets;

//Import Turbine packages.

import org.apache.turbine.util.RunData;

//Import Velocity packages

import org.apache.velocity.context.Context;

//Import Jetspeed packages.

import org.apache.jetspeed.modules.actions.portlets.VelocityPortletAction;

import org.apache.jetspeed.portal.Portlet;

public class myScienceAppAction extends VelocityPortletAction {

public void buildNormalContext(VelocityPortlet portlet,

Context acontext, RunData rundata) {

//Real action code goes here.

System.out.println("buildNormalContext called");

}

}


Some miscellaneous notes

Some Miscellaneous Notes

  • This action is automatically called whenever the JSP template’s form action is invoked.

  • In the portal release, the chef-1.0.7/modules/nmi-lib directory contains all the jars needed to compile this code.

    • You can use our ant build scripts as templates for writing your own.

  • The portlet actions’s system.out.println() is written to catalina.out.

  • If you prefer, you can use Jetspeed’s logger, which writes to nmi/WEB-INF/log.


Rundata requests and sessions

RunData, Requests, and Sessions

  • The method buildNormalContext includes an object called RunData in its arg list.

  • RunData is your access point to all HTTP request, response, and session data.

    • HttpSession session=rundata.getSession();

    • HttpServletRequest req=rundata.getRequest();

  • From here, you can do standard java.net.* development.

    • I am not planning to cover this, but we can discuss.


Connecting multiple templates

Connecting Multiple Templates

  • In reality, a single web form is not enough to set up a complicated input file, select a host and execute a job.

    • These may be spread over multiple linked forms.

  • Form actions in templates must be handled a bit differently.

  • <form action=“SomeOtherPage”>

    • This can’t point to a template, since it is not directly accessible (in WEB-INF).

    • Jetspeed actions handle this.


Redirecting to another page

Redirecting to Another Page

  • Setting up and running applications from a portal typically requires many successive HTML forms.

  • So we need to see how to do this with Jetspeed.

  • Let’s call this myScienceApp2.vm and place it in the same place as myScienceApp.vm.

myScienceApp

myScienceApp2

Response

Request

myScienceAppAction


First modify myscienceapp vm

Use the eventSubmit_{action} construction.

This action will be tied to a specific method in myScienceAppAction.java.

<h3> Enter the command you want to execute</h3>

<form method="post">

<table>

<tr>

<td>Command Name:</td>

<td><input type="text" name="runCommnad" value=""></td>

</tr>

</table>

<input type="submit" name="eventSubmit_doGet_command" value="Click to Select Host">

</form>

First, Modify myScienceApp.vm


Next modify your action code

Add the following method to myScienceAppAction.java.

Your method name should follow the pattern given in the form.

This particular action runs setTemplate(), which loads the indicated Velocity template.

When you click a form, Jetspeed will look for all methods matching eventSubmit’s pattern.

If it finds them, it executes them.

If not, buildNormalContext() is the default.

public void doGet_command( RunData runData, Context aContext ) {

setTemplate(runData, "myVelocityApp2");

}

Next, Modify Your Action Code


Write the destination template

The following is an example of myScienceApp2.vm.

Note the eventSubmit.

Both of our templates use the same action.

We can add a method doGet_host to myScienceAppAction.java to handle host events.

You can build

Once=anomaly

Twice=infinitely reproducible pattern

<form method="post">

BigIron: <input type=radio name=host value="BigIron">

<br>

IronBird: <input type=radio name=host value="IronBird">

<input type="submit" name="eventSubmit_doGet_host" value="Click to Run">

</form>

Write the Destination Template


Grid portlets

Grid Portlets

  • There is nothing fundamentally special about Grid portlets

    • Just implement your portlet action with Java COG calls.

  • We handle all access to the Grid through the Java COG kit.

    • Hides differences between Globus toolkit versions.

    • Currently a higher level programming API for the COG is under development as part of NMI.

    • GVL is one of our PIs, so we all have vested interest in this.

  • Basic procedure:

    • Proxy credentials acquired through MyProxy client portlet.

    • Modify your action java class (myScienceAppAction.java)

    • Use convenience methods to get local GSSCredential from memory.

    • Use GSSCredential in COG calls to Grid resources.

    • Develop Velocity user interfaces to collect necessary info from users.


Example code fetching credentials

The OGCE provides a convenience class ProxyManager for storing/fetching proxy creds.

Assumes you have logged in to the proxy portlet.

HttpSession session = runData.getSession();

GSSCrendential cred = ProxyManager.getDefaultProxy(session);

Example Code: Fetching Credentials


Using the cog portlet interfaces

Using the CoG Portlet Interfaces

  • From here, you can develop portlet actions using the Java CoG kit.

  • Gregor will discuss new interfaces in the next session.


Portal testing with httpunit

Portal Testing with HTTPUnit

Marcus Christie and Marlon Pierce


Types of unit testing

Types of Unit Testing

  • JUnit is a great way for testing code functionality.

  • But how do you test portals and portlets?

    • Portal interfaces need to be tested by simulated users.

      • Button clicks, link clicks, form interactions, etc.

  • For the NMI OGCE project, we have adopted HttpUnit for this.

    • http://httpunit.sourceforge.net/


Unique challenges for testing grid portals

Unique Challenges for Testing Grid Portals

  • Fluid GUI

  • Much setup required for Grid Portal

    • Portal account

    • Grid Software, accounts

    • Reliance on external services

  • Broken HTML

  • Infinite set of user operations

    • Use Cases + Simple Error Handling


Httpunit features

HTTPUnit Features

  • Extension of Junit

    • Integrates easily with Ant <junit> task

  • Fairly high level API

  • Browser emulation (cookies, JavaScript, etc.)

  • Includes a sloppy/quirky HTML parser (NekoHTML)


Setting up your test

The code on the right shows the boilerplate to put at the start of your test.

We will fill in the rest of the test in the following slides.

package xportlets.tests;

import com.meterware.httpunit.*;

import junit.framework.*;

public class XPortletTestCase extends TestCase {

....

public void setUp() throws Exception {

super.setUp();

}

//Rest of the test

}

Setting Up Your Test


Testing a login form

The code shows how to actually write a simple form test.

Create a WebConversation object.

Use WebResponse to work programmatically with the HTML.

Use WebForm to invoke HTML <forms>

Assertions are used to test the content of the returned page.

public WebConversation xPortletAuth() throws Exception {

WebConversation wc = new

WebConversation();

WebResponse resp =

wc.getResponse(url);

WebForm form = resp.getForms()[0];

form.setParameter("username", username);

form.setParameter("password", password);

form.submit();

WebResponse resp2 =

wc.getCurrentPage();

String page = resp2.getText();

assertTrue("Failed to log in", page.indexOf("Welcome to

your workspace") != -1);

return wc;

}

Testing a Login Form


Simulating button clicks

Create SubmitButton objects to simulate button clicks.

“myButton” is the name of the submit button in the HTML <form>

Calling click() causes the form action to take place.

Get the new page from the WebConversation object.

public void testSimplePing() throws Exception {

WebConversation wc =

xPortletAuthProxy();

WebResponse resp = wc.getCurrentPage();

WebForm form = resp.getForms()[3];

form.setParameter("service", "rainier.extreme.indiana.edu");

SubmitButton addServiceBtn = form.getSubmitButton(“myButton");

assertNotNull(“myButton",

addServiceBtn);

addServiceBtn.click();

WebResponse resp2 =

wc.getCurrentPage();

WebForm form2 = resp2.getForms()[3];

}

Simulating Button Clicks


Running httpunit with ant

Running HttpUnit with Ant

  • After you have developed a suite of tests, you can automate your portal tests using Ant’s optional JUnit task.

  • Even better, you can combine this with the <junitreport> task that will generate nice HTML report forms.

  • First, you will need to put these jars in ant’s lib directory.

    • httpunit.jar

    • junit.jar


An example unit test target

Let’s say all of your tests fit the pattern Test*.class.

Then run a <batchtest> to run all of them at once.

Will run all tests

Won’t exit on failed test.

Use errorProperty and failureProperty to catch these states.

Specify an XML formatter for pretty output (next).

<junit printsummary="true”

errorProperty="test.failed" failureProperty="test.failed">

<formatter type="xml"/>

<batchtest

todir="${report.dir}">

<fileset dir="classes"

includes="**/Test*.class"/>

</batchtest>

</junit>

An Example Unit Test Target


Generating html dashboards

The <junitreport> target can be used to convert the report outputs from <junit>.

Use <report> to generate HTML.

Finally the <fail> message checks if any tests failed.

See previous target.

<junitreport todir="${report.dir}">

<fileset dir="${report.dir}">

<include name="TEST*.xml"/>

</fileset>

<report format="frames" todir="${html.report.dir}"/>

</junitreport>

<fail message=“Tests failed." if="test.failed"/>

Generating HTML dashboards


Example report dashboard

Example Report Dashboard

List of

Test Classes

Test Results for One

Of the Test Cases


Some drawbacks to httpunit

Some Drawbacks to HTTPUnit

  • Tests are tied to page structures.

    • If you change your page (say, reordering links or forms), you will break your tests.

  • Tests also depend on

  • Test assertions typically depend on the content of the page.

    • If content changes, tests may break.

    • Checking content is not completely sufficient for testing Grid portals.


Additional capabilities

Additional Capabilities


Capability community scheduling framework portlets

Capability: Community Scheduling Framework Portlets

CSF Use Case

  • Researcher submits job through User Portal

  • User Portal uses GridPort to

    • authenticate user

    • optionally make advanced reservation to visualization system

    • submit job to CSF

  • CSF selects compute cluster with best fit and forwards job

  • Gridport sends results to visualization system

User Workstation

User Portal

GridPort

Visualization

System

Bandera

CSF

Blanco

Buda


Capability job sequencer portlets

Capability: Job Sequencer Portlets

GPIR

" xsi:schemaLocation="http://grids.tacc.utexas.edu/schemas/sequencer/jobSequence

C:\DOCUME~1\Maytal\Desktop\Maytal\Work\GP-IR\GP-IRX~1\motd.xsd">

<

<Status>New</Status>

<Step>

<Status>Unscheduled</Status>

<Type>CSFJob</Type>

<Parameter name="jobFactoryServiceHandle">http://129.116.218.36:15080/ogsa/services/metascheduler/JobFactoryService</Parameter>

<Parameter name="queue">normal</Parameter>

<Parameter name="executable">pam</Parameter>

<Parameter name="arguments">-g 1 mpichp4_wrapper /home/monitor/mpi_jobs/mpimd_5</Parameter>

<Parameter name="directory">/home/monitor/mpi_jobs</Parameter>

<Parameter name="count">4</Parameter>

<Parameter name="stdIn">/dev/null</Parameter>

<Parameter name="stdOut">/home/monitor/mpi_jobs/tomislavSequencerJobOut</Parameter>

<Parameter name="stdErr">/home/monitor/mpi_jobs/tomislavSequencerJobErr</Parameter>

</Step>

<Step>

<Status></Status>

<Type>GridFTP</Type>

<Parameter name="fromHost">[Previous]</Parameter>

<Parameter name="toHost">blanco.tacc.utexas.edu:2811</Parameter>

<Parameter name="fromFileFullName">/home/monitor/mpi_jobs/tomislavSequencerJobOut</Parameter>

<Parameter name="toFileFullName">/home/monitor/mpi_jobs/tomislavSequencerJobOutCopied</">/home/monitor/mpi_jobs/tomislavSequencerJobErr</Parameter>

<Parameter name="toFileFullName">/home/monitor/mpi_jobs/tomislavSequencerJobErrCopied</Parameter>

</Step>

</JobSequence>

Sequencer

User uses Portal to generate XML description of sequence.

Currently, sequence steps can consist of File Transfers and Job Submissions to the CSF meta scheduler

The XML is then decomposed and persisted to GPIR where the status information of each step in the sequence and of the sequence as a whole can be stored

GridPort returns a Sequence ID to the Portal immediately and then begins executing the Sequence to completion or to error. Status information can be obtained at any time with the Sequence ID


Developing grid portals using portlets

O.G.R.E.—A Job Management Engine

  • O.G.R.E. = Open Grid Computing Environments Runtime Engine

  • What Ant lacked, but we needed:

    • Broader conditional execution,

      • Ant: based on write-once String properties.

    • A general “loop” structure for Task execution.

    • Data-communication between Tasks (and with their containers).

    • Specialized tasks

      • File reading and writing

      • Local and remote file management (gridftp)

      • Web service related tasks

      • Event- and process-monitoring-tasks


Xdirectory a grid context service

XDirectory: A Grid Context Service

  • XDirectory is itself a Grid Service that is access by the portal.

    • An index over a relational database

    • Each node is either a “directory node” or a leaf.

    • Leaf nodes are xml elements which contain metadata as well as html annotations.


Example capability file management

Example Capability: File Management

User “Beth”

  • Grid FTP portlet– Allow User to manage remote file spaces

    • Uses stored proxy for authentication

    • Upload and download files

    • Third party file transfer

      • Request that GridFTP server A send a file to GridFTP server B

      • Does not involve traffic through portal server

Portal Server

GridFTP

portlet

Java

COG

GridFTP

Server A

GridFTP

Server B


Jsr 168 tutorial

JSR 168 Tutorial

Details on Writing portlets


In action get started

In Action: Get started.

public class JunkPortlet extends GenericPortlet {

public void init(){

//Do any initialization.

}

//Rest of the methods on following slides go here.

}


Override doview

Override doView()

protected void doView( RenderRequest req, RenderResponse res)

throws PortletException, IOException {

//Include the desired JSP or HTML page.

//We could also use out to write directly to the response.

WindowState state=req.getWindowState();

if(!state.equals(WindowState.MINIMIZED)) {

res.setContentType("text/html");

PortletRequestDispatcher rd=

getPortletContext().getRequestDispatcher(“MyJSP.jsp”);

rd.include(req,res);

}

}


The jsp page

The JSP Page

<portlet:defineObjects/>

<%

PortletSession portletSession=renderRequest.getPortletSession();

portletSession.setAttribute("localattname","localattval");

PortletURL url=renderResponse.createActionURL();

String theActionString=url.toString();

%>

HTML Content is here.

A form is below.

<form method=post action="<%=theActionString%>">

<input type=…>

</form>


Some notes

Some Notes

  • Include the <%portlet:definedObjects/%> tag, which will instantiate renderRequest, renderResponse, and portletConfig objects.

    • You can then just use them, as with request, response, and other JSP implicit objects.

  • The renderRequest gives you access to the PortletSession, if you want to store session variables.

    • One of the trouble points.

  • The renderResponse gives you access to the PortletURL object.

  • Use the PortletURL to generate a URL for the <form action>

    • So that it points to portlet container and gets handled by the processAction() method, rather than going of into space.

    • Handle href URLs similarly.

    • This is one of the sticking points.


Lastly override processaction

When you invoke the form on the previous JSP, the portlet container will pass the action handling to the processAction method.

The ActionRequest can be used to get any of the <input> parameters in a way similar to the usual HttpServletRequest.

When the processAction method returns, the container then invokes the appropriate do method (usually doView).

If you need to pass <form> parameters on to doView, add them to the ActionResponse.

This will give them to the RenderRequest.

The example shows how to add ALL parameters.

public void processAction (ActionRequest request, ActionResponse actionResponse) throws PortletException, java.io.IOException {

//Process request parameters

//Add any other request params

// to the renderRequest

actionResponse.setRenderParameters(request.getParameterMap());

}

Lastly, Override processAction()


  • Login