neesgrid data and metadata technology n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
NEESGrid Data and MetaData Technology PowerPoint Presentation
Download Presentation
NEESGrid Data and MetaData Technology

Loading in 2 Seconds...

play fullscreen
1 / 26

NEESGrid Data and MetaData Technology - PowerPoint PPT Presentation


  • 102 Views
  • Uploaded on

NEESGrid Data and MetaData Technology. Kincho Law, Jun Peng, Jim Eng, Terry Weymouth, Paul Hubbard, Charles Severance. Outline. Model activities Metadata tools and repository Data as video What is done What we have to do. Data. Metadata. The Slide.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'NEESGrid Data and MetaData Technology' - yonah


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
neesgrid data and metadata technology

NEESGrid Data and MetaDataTechnology

Kincho Law, Jun Peng, Jim Eng, Terry Weymouth, Paul Hubbard, Charles Severance

outline
Outline
  • Model activities
  • Metadata tools and repository
  • Data as video
  • What is done
  • What we have to do
the slide

Data

Metadata

The Slide

There is a layer is where we develop tools which take advantage and begin to depend on of the “meaning” of the data – where we begin to depend on the meaning of a second.

Where we make a viewer capable of viewing a certain type of object.

This is where we build things which make use of knowledge.

This layer will never be complete but it is a large focus of the coming months.

Data Ingestors

Search

Data Mappers

Concepts

Data Viewers

09/2003

how to prioritize model exploration and development
How to prioritize model exploration and development
  • Focus on the following areas:
    • Areas where we have or are building tools
    • Areas where we already have incoming data in some format
    • Build the model through experiment based deployment - solve real problems in an open way and see if (with some adaptation) the solutions apply more broadly (i.e. Minnesota )

11/2003

go forward tools
Go Forward - Tools
  • Evaluate the ORST interface and use it to implement experiment-based interface to meta data repository
  • Investigate tools to represent structural data (like SAC data)
  • Extend and improve viewers – publish API so that sites can extend the viewers
  • Improve notebook
    • Single signon using CHEF/Grid credentials
    • Integration with Metadata
    • Smother integration with CHEF
  • Explore automated synchronized video and data capture and after-experiment replay of synchronized video and data (ORST UMinn)
  • Explore the capture of high quality still images as data (UMinn)
  • Investigate adopting a data-editing tool (XMLSpy)

09/2003

rdf integration
RDF Integration
  • Some of the data and meta data task force members are using Protégé-2000 to develop their models and expressing them in RDF.
  • RDF and NEESML are very similar but not identical so it may be challenging to ingest any arbitrary RDF
  • We expect that we will be able to map a subset of RDF to NEESML for ingestion or adapt an RDF parser (Jena or Raptor) to ingest that subset directly into the repository

11/2003

data model activities
Data Model Activities
  • Groups have been formed to develop coordinated models
    • Shake Table
    • Centrifuge
    • Tsunami
    • GeoTech
  • Gokhan Pekhan is organizing this effort
  • Kincho / Jun / Jim pushed forward with a RDF-Based Shake Table Model from Protege-2000 and developed software
models data model
Models + Data Model

Repo

Data

Load

RDF

<owl:ObjectProperty rdf:ID="hasPublications">

<rdfs:domain>

<owl:Class>

<owl:unionOf rdf:parseType="Collection">

<owl:Class rdf:about="#Project"/>

<owl:Class rdf:about="#Task"/>

</owl:unionOf>

</owl:Class>

</rdfs:domain>

<rdfs:range rdf:resource="#Publications"/>

</owl:ObjectProperty>

Configure

Models

RDF/

OWL

Configure

models data model1
Models + Data Model

Repo

Data

Load

RDF

Configure

Models

Protégé - 2K

RDF/

OWL

Configure

electronic notebook
Electronic Notebook
  • Collaborative effort with the DOE SciDAC
    • Electronic notebook - metadata entry
    • Data mapping
    • Data provenance
    • Data display
    • Slide data/metadata jakarta.apache.org/slide/
  • Ultimate integration will be via JSR-170
  • www.scidac.org/SAM/
  • collaboratory.emsl.pnl.gov/docs/collab/sam/samtechoverview.html
doe eln neesgrid integration to date
DOE ELN / NEESgrid Integration (to date)

NEESgrid

Repository

MyProxy

Chef

Grid

Security

SAM /

Slide

Repository

Technology Celebration

doe eln neesgrid integration ultimate
DOE ELN / NEESgrid Integration (ultimate)

MyProxy

NEESgrid

Repository

Chef

Grid

Security

skunkworks project

DAQ

“Skunkworks” project

0 3 4

0 6 8

<experiment>

<blah>

<public-view>

</experiment>

0 3 4

0 6 8

<experiment>

<blah>

<public-view>

</experiment>

09/2003

data turbine
Data Turbine
  • Commercial, free data streaming toolkit
data turbine cont
Data Turbine (cont)
  • Existing data viewers will be adapted to access and display data from data turbine
  • Data acquisition software will be adapted to place information in Data Turbine Channels
  • Metadata elements will be developed to represent data turbine live, stored, and derived channels
  • New efforts (video as data) will be developed from the ground up using Data Turbine
  • outlet.creare.com/rbnb/

11/2003

video as data
Video as Data
  • Follow on to initial demonstration at ORST
  • Experiment based development: Minnesota
  • Design phase complete
  • Joint effort, NEESGrid SI, ORST, Minnesota, UC Davis, Texas, Buffalo, and others
data turbine today
Data Turbine - Today

rbnbjcap

BT848

Axis

DT Client

Data

Capture

DAQ

DT Client

NTCP

Plugin

CTL

NTCP

Control

Control

Plugin

DT Main System

AXIS / DT

Gateway

NEES

NSDS

Driver

data turbine1
Data Turbine

Control

Make Smoothie

NTCP

Control

Control

Plugin

DT Main System

Thumbs

Technology Celebration

dt capturing
DT Capturing

Still

Capture

PTZ/

USB

DT Client

rbnbjcap

BT848

DT Client

Audio

Encoder

Audio

DT Client

Data

Capture

DAQ

DT Client

Each still capture produces two channels - Small 1-5fps stream + large single images when picture is taken

Camera

Control

Control

Plugin

DT Main System

NEES

NSDS

Driver

Still Capture - Minnesota / Paul Hubbard

Video capture - From Creare

Audio capture - From Creare (TBD)

Data Capture - From sites (upwards compatible)

NEES NSDS Driver - Paul Hubbard

Camera Control Plugin - Mich / Minn

user views still camera
User Views / Still Camera

Still Image / Camera Control

^

<

>

^

~

<

>

Thumbnail + Audio + Data

Data Viewer

<

>

+

Control

Plugin

Thumbnail - uBuffalo / Umichigan

Thumbnail viewer - Creare / Mich

Camera Control Applet - Minn / Mich

Quicktime Slicing tool - Mich (low)

Stored Data Viewer - Mich

JPEG Viewer - Creare

Quickime Viewer - Apple

DT Main System

Thumbnail

Process

Quicktime Storage System

Quicktime

Slicing

Tool

minnesota mock up
MinnesotaMock - up

If you area developer and interested in following / helping / participating in this activity, join the mailing list

neesgrid-dv@neesgrid.org

tool list to do
Tool List - To Do
  • Next release of repository
  • Integrate ELN into repository
  • DAQ Control Panel in CHEF
    • Set/Retrieve Metadata
    • Start / Stop
    • Ingest data from staging space
  • Data Turbine Control Panel in CHEF
    • Start / Stop / Configure Sources
      • Video | Audio | Data | Thumbnail
    • Control permanent storage of video
to do cont
To Do (cont)
  • NTCP Debugging and Monitoring in CHEF
    • Needed Data Turbine
  • Data as Video Client Tools in CHEF
    • New Monitor Tool
    • Still Image
    • Camera Control
  • Data Turbine Audio Capture
  • We may need to support XML Schema
  • QuickTime Capabilities
    • Archive, retrieve, slice, dice, convert, present
    • Probably will not be completed as part of SI effort
data model work
Data Model Work
  • Data Curation Summit
    • Understand issues form go-forward plan
    • Meeting 3/18/2004
  • Data and Metadata Task Force
    • Finish the tsunami and centrifuge models
  • DSAC Committee
    • Meeting 3/19/2004
summary
Summary
  • In September 2003, we met and “re-visioned” data
  • A bunch of requirements gathering and development has been done
    • The “high risk” elements are working now
  • There is more to do - We will run out of time
    • Evolutionary development approach - there will always be usable working code - we will stop when we run out of time
the people
The people…
  • Gokhan Peckan - Data Models
  • Kincho Law - Data Models / Software design
  • Jun Peng - Data Models / Software design
  • Jim Eng - Parse / Ingest / RDF / Project Browser
  • Jim Myers - Electronic Notebook
  • Terry Weymouth - Data Turbine
  • Paul Hubbard - DAQ and NTCP
  • Joe Futrelle - Data / Metadata Repository