Semantic grid services for video analysis
Download
1 / 19

slides - PowerPoint PPT Presentation


  • 239 Views
  • Updated On :

Semantic Grid Services for Video Analysis Gayathri Nadarajan, Yun-Heh Chen-Burger, James Malone Centre for Intelligent Systems and their Applications School of Informatics University of Edinburgh Scotland Introduction - EcoGrid

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'slides' - andrew


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Semantic grid services for video analysis l.jpg
Semantic Grid Servicesfor Video Analysis

Gayathri Nadarajan,

Yun-Heh Chen-Burger, James Malone

Centre for Intelligent Systems and their Applications

School of Informatics

University of Edinburgh

Scotland


Introduction ecogrid l.jpg
Introduction - EcoGrid

  • Utilises state-of-the-art Grid technologies to establish an infrastructure for ecological research in several locations in Taiwan.

  • Scientists can conduct data acquisition, data analysis and data sharing on this platform.

  • Divided into four components; network, data streaming, data management and workflow enactment.

  • Collaboration between National Centre for High-Performance Computing (NCHC), Taiwan and Artificial Intelligence Applications Institute (AIAI), University of Edinburgh on workflow enactment for EcoGrid.


Sample video clips l.jpg

Sample video clips

Ecological Motivation


Ecological motivation l.jpg

Problem:

- Vast amount of valuable and continuous real-time data of varying quality is collected over 3 years, unanalysed.

- Manual processing is time-consuming.

1 minute’s clip will take 15 minutes to analyse, thus 1 year’s recording will cost human experts 15 years’ effort x 3 under water cameras = 45 years.

Impractical situation and more appropriate automation methods must be deployed.

Semi-automated and automated way to speed up this process.

Ecological Motivation


Requirements l.jpg

Process Automation:

- Process model for video annotation:

Performance-Based Selection.

Iterative Processing.

Adaptive, Flexible and Generic Architecture.

Semantic-Based Compatibility.

Requirements

Key frame

detection

Segmentation

Recognition

Annotation


Related work l.jpg

Existing Grid workflow systems:

- Pegasus

- Triana

- Taverna

- Kepler

Limitations of current solutions for flexible workflow composition for complex video processing tasks.

Related Work



Proposed framework design layer l.jpg

Process Manager is responsible for composing sequence of processes to be executed based on available tools.

- Planning component

- CBR component

Ontologies give meaning to process and keep goals separate from capabilities.

- Goal

- Domain

- Tool/Capability

Proposed Framework – Design Layer


Slide9 l.jpg

Main interface between design and processing layers. processes to be executed based on available tools.

Workflow enactor acts as interpreter of events that occur within system.

Proposed Framework – Workflow Layer


Proposed framework processing layer l.jpg

Consists of a set of image and video processing tools. processes to be executed based on available tools.

Functions of tools represented in capability ontology in Design Layer.

Once a workflow has been established, tools may work directly on videos.

Depending on quality of video and task at hand, each tool will work on varying level of accuracy.

Feedback from domain expert would be useful.

Use of ML techniques to assist with performance measure predictions for image and video processing tools.

Proposed Framework – Processing Layer


Discussion l.jpg

Work in progress processes to be executed based on available tools.

- Understand video processing tools available.

- Build goal, domain and capability ontologies.

- Keep capability non-technical for ecologists but may have technical implication in underlying mechanism, e.g. “pitch dark” means “brightness” level of 0-25.

Concepts such as “brightness”, “focus”, types of fish species, “fish count”, “fish size” are taken into account.

Ontologies are related.

Discussion


Walkthrough l.jpg
Walkthrough processes to be executed based on available tools.

  • User (ecologist) interacts with system by providing high level goals and constraints as input.

  • Goal will match that in the Goal ontology.

  • Constraints will match those in the Domain ontology and will determine the video clip to be fetched.

  • The Planner will look for a sequence of processes for execution using task decomposition.

  • Sequence of processes constitute the workflow composition.

  • Instances of processes and their subtasks are contained in the process library.


Walkthrough13 l.jpg
Walkthrough processes to be executed based on available tools.

  • Each subtask will be matched with a video/image processing operation contained in the Capability ontology. The tool(s) to execute each operation is also contained in the Capability ontology.

  • In cases where more than one tool is available to perform the subtask, the tool with the best performing capability is selected.

  • A set of tools to perform VP task is now obtained.

  • VP tools may now work on the video clip directly.

  • By wrapping each tool as a Grid service, this could be performed in a distributed fashion on a Grid infrastructure.


Example l.jpg
Example processes to be executed based on available tools.

“Detect at least one tiger fish in high brightness”

Goal: Detection

Constraints: Occurrence = at_least_one;

Brightness = high

Process library

detect_presence(X) :-

pre_process(X), feature_extraction(X), segmentation(X), classification(X).

pre_process(tiger_fish) :-

keyframe_detection, enhancement.


Tools l.jpg

Open Computer Vision Library (OpenCV) processes to be executed based on available tools.

LTI library (C++)

HTK (HMM classifier)

Lib SVM

ITK (medical imaging)

STL (C++)

Readily available but not Grid-compatible.

Tools


Contributions l.jpg
Contributions processes to be executed based on available tools.

  • Grid workflow technology with semantic capabilities + AI Planning and CBR.

  • Vision - specialised tools used in workflow could help provide reasonable solutions for automatic video processing, e.g. annotation.


End of slides thank you l.jpg
End of Slides processes to be executed based on available tools.Thank you!


References l.jpg

EcoGrid, National Centre for High Performance, Taiwan, 2006. http://ecogrid.nchc.org.tw/

Workflow enactment in the EcoGrid, AIAI, University of Edinburgh, 2006. http://www.aiai.ed.ac.uk/~jessicac/project/NCHC

Y-H Chen-Burger, F-P Lin. “A Semantic Based Workflow Choreography for Integrated Sensing and Processing”. In Procs. CNNA 2005.

G. Nadarajan, Y-H Chen-Burger, J Malone. “Semantic-Based Workflow Composition for Video Processing in the Grid”. In Procs. WI-2006 (to appear).

D. D. O’Roure, N. R. Jennings, N. R. Shaldbolt. “The Semantic Grid: Past, Present and Future”. In Procs. the IEEE Vol. 93, Issue 3, 2005.

J. Yu, R. Buyya. “A Taxonomy of Workflow Systems for Grid Computing”. Journal of Grid Computing, 2006.

OpenCV. http://sourceforge.net/projects/opencvlibrary

References


The semantic grid technical motivation l.jpg

The Grid is aimed at enabling resource sharing and coordinated problem-solving between computers and people in a distributed and heterogeneous manner.

The Semantic Grid is an extension of current Grid whereby information and services are given well-defined and explicitly represented meaning, i.e. the application of Semantic Web technologies to the Grid.

Requires means for facilitating the composition of multiple resources and mechanisms for creating and enacting these in a distributed manner, i.e. composing and executing complex workflows.

Improvement of Grid workflow systems with the incorporation of semantic capabilities.

The Semantic Grid – Technical Motivation


ad