January 23 2006 ilkay altintas
1 / 17

January, 23, 2006 Ilkay Altintas - PowerPoint PPT Presentation

  • Updated On :

New Developments in Kepler . January, 23, 2006 Ilkay Altintas. Kepler System Architecture. Authentication. GUI. …Kepler GUI Extensions…. Vergil. Documentation. Smart Re-run / Failure Recovery. Provenance Framework. Kepler Object Manager. SMS. Type System Ext. Actor&Data

Related searches for January, 23, 2006 Ilkay Altintas

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'January, 23, 2006 Ilkay Altintas' - taite

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
January 23 2006 ilkay altintas l.jpg

New Developments in Kepler

January, 23, 2006

Ilkay Altintas

Kepler system architecture l.jpg
Kepler System Architecture



…Kepler GUI Extensions…




Re-run /


















Joint authentication framework l.jpg
Joint Authentication Framework

  • Requirements:

    • Coordinating between the different security architectures

      • GEON uses GAMA which requires a single certificate authority.

      • SEEK uses LDAP with has a centralized certificate authority with distributed subordinate Cas

    • To connect LDAP with GAMA

    • Coordinating between 2 different GAMA servers

    • Single sign-on/authentication at the initialize step of the run for multiple actors that are using authentication

      • This has issues related to single GAMA repository vs multiple, and requires users to have accounts on all servers.

      • Kepler needs to be able to handle expired certificates for long-running workflows and/or for users who use it for a long time.

      • A trust relation between the different GAMA servers must be established in order to allow for single authentication.

Functional prototype completed l.jpg
Functional Prototype Completed

  • APIs and tests cases in place

  • More work required on certificate renewal and multiple server access

Vergil is the gui for kepler l.jpg
Vergil is the GUI for Kepler

Actor Search

Data Search

  • Actor ontology and semantic search for actors

  • Search -> Drag and drop -> Link via ports

  • Metadata-based search for datasets

Actor search l.jpg
Actor Search

  • Challenges:

    • Building/searching a repository …

    • Making changes to MoML (see KAR)

    • GUI changes

    • Ontology management

  • Kepler Actor Ontology

    • Used in searching actors and creating conceptual views (= folders)

  • Currently 160 Kepler actors added!

Data search and usage of results l.jpg
Data Search and Usage of Results

  • Kepler DataGrid

    • Discovery of data resources through local and remote services

      • SRB,

      • Grid and Web Services,

      • Db connections

    • Registry of datasets on the fly using workflows

Vergil updates l.jpg
Vergil Updates

  • To make it more useful to the user

    • Updated actor icons

    • Menu redesign

  • Improve readability

  • Develop cohesive visual language

  • Follow standard HF principles

  • Improve organization


DB Query

Computation or Operation



File Operation

Web Service

Kepler archives l.jpg
Kepler Archives

  • Purpose: Encapsulate WF data and actors in an archive file

    • … inlined or by reference

    • … version control

      • More robust workflow exchange

      • Easy management of semantic annotations

      • Plug-in architecture (Drop in and use)

      • Easy documentation updates

  • A jar-like archive file (.kar) including a manifest

  • All entities have unique ids (LSID)

  • Custom object manager and class loader

  • UI and API to create, define, search and load .kar files

Kar file example l.jpg
KAR File Example

<entity name="Multiply or Divide" class="ptolemy.kernel.ComponentEntity">

<property name="entityId" value="urn:lsid:localhost:actor:80:1" class="org.kepler.moml.NamedObjId"/>

<property name="documentation" class="org.kepler.moml.DocumentationAttribute"></property>

<property name="class" value="ptolemy.actor.lib.MultiplyDivide" class="ptolemy.kernel.util.StringAttribute">

<property name="id" value="urn:lsid:localhost:class:955:1" class="ptolemy.kernel.util.StringAttribute"/></property>

<property name="multiply" class="org.kepler.moml.PortAttribute">

<property name="direction" value="input" class="ptolemy.kernel.util.StringAttribute"/>

<property name="dataType" value="unknown" class="ptolemy.kernel.util.StringAttribute"/>

<property name="isMultiport" value="true" class="ptolemy.kernel.util.StringAttribute"/></property>

<property name="divide" class="org.kepler.moml.PortAttribute">

<property name="direction" value="input" class="ptolemy.kernel.util.StringAttribute"/>

<property name="dataType" value="unknown" class="ptolemy.kernel.util.StringAttribute"/>

<property name="isMultiport" value="true" class="ptolemy.kernel.util.StringAttribute"/>


<property name="output" class="org.kepler.moml.PortAttribute">

<property name="direction" value="output" class="ptolemy.kernel.util.StringAttribute"/>

<property name="dataType" value="unknown" class="ptolemy.kernel.util.StringAttribute"/>

<property name="isMultiport" value="false" class="ptolemy.kernel.util.StringAttribute"/></property>

<property name="semanticType00" value="http://seek.ecoinformatics.org/ontology#ArithmeticMathOperationActor" class="org.kepler.sms.SemanticType"/>


Kepler object manager l.jpg
Kepler Object Manager

  • Designed to access local and distributed objects

  • Objects: data, metadata, annotations, actor classes, supporting libraries, native libraries, etc. archived in kar files

  • Advantages:

    • Reduce the size of Kepler distribution

      • Only ship the core set of generic actors and domains

    • Easy exchange of full or partial workflows for collaborations

    • Publish full workflows with their bound data

      • Becomes a provenance system for derived data objects

        => Separate workflow repository and distributions easily

Initial work on provenance framework l.jpg
Initial Work on Provenance Framework

  • Provenance

    • Track origin and derivation information about scientific workflows, their runs and derived information (datasets, metadata…)

  • Need for Provenance

    • Association of process and results

    • reproduce results

    • “explain & debug” results (via lineage tracing, parameter settings, …)

    • optimize: “Smart Re-Runs”

  • Types of Provenance Information:

    • Data provenance

      • Intermediate and end results including files and db references

    • Process (=workflow instance) provenance

      • Keep the wf definition with data and parameters used in the run

    • Error and execution logs

    • Workflow design provenance (quite different)

      • WF design is a (little supported) process (art, magic, …)

      • for free via cvs: edit history

      • need more “structure” (e.g. templates) for individual & collaborative workflow design

Kepler provenance recording utility l.jpg
Kepler Provenance Recording Utility

  • Parametric and customizable

    • Different report formats

    • Variable levels of detail

      • Verbose-all, verbose-some, medium, on error

    • Multiple cache destinations

  • Saves information on

    • User name, Date, Run, etc…

Provenance possible next steps l.jpg
Provenance: Possible Next Steps

  • Provenance Meeting: Last week at SDSC

    • Deciding on terms and definitions

    • .kar file generation, registration and search for provenance information

    • Possible data/metadata formats

    • Automatic report generation from accumulated data

    • A GUI to keep track of the changes

    • Adding provenance repositories

    • A relational schema for the provenance info in addition to the existing XML

What other system functions does provenance relate to l.jpg
What other system functions does provenance relate to?

  • Failure recovery

  • Smart re-runs

  • Semantic extensions

  • Kepler Data Grid

  • Reporting and Documentation

  • Authentication

  • Data registration

Re-run only the updated/failed parts

Guided documentation generation an updates

Hot topics in kepler l.jpg
Hot Topics in Kepler