A distributed data architecture
This presentation is the property of its rightful owner.
Sponsored Links
1 / 10

A Distributed Data Architecture PowerPoint PPT Presentation


  • 54 Views
  • Uploaded on
  • Presentation posted in: General

A Distributed Data Architecture. Mark Jessop University of York. Swans. Grid Enabled Swans. London. Tokyo. Cape Town. Mexico City. How Big is that Lake?. Heathrow capped at 36 landings per hour. If half have 4 engines and half have 2, average aircraft carries 3 engines.

Download Presentation

A Distributed Data Architecture

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


A distributed data architecture

A Distributed Data Architecture

Mark Jessop

University of York


Swans

Swans


Grid enabled swans

Grid Enabled Swans

London

Tokyo

Cape Town

Mexico City


How big is that lake

How Big is that Lake?

  • Heathrow capped at 36 landings per hour.

  • If half have 4 engines and half have 2, average aircraft carries 3 engines.

  • Each engine generates around 1GB of data per flight.

  • 36 x 3 x 1 = 108GB raw engine data per hour.

  • Factor in the working day and the rest of the world…

  • …Terabytes and up!


Managing the flow of water

London

Tokyo

Cape Town

Mexico City

Managing the Flow of Water


Plumbing toolkit

Plumbing Toolkit

  • Data Repository

  • Catalogue

  • Pattern Match Engine


Pattern match engine

Pattern Match Engine

  • Pattern Match Control

  • Data Extractor/Encoder

  • AURA Encoder

  • AURA-G

  • Back Check


Data repository

DATA

DATA

DATA

DATA

DATA

DATA

MCAT

MCAT

MCAT

MCAT

MCAT

MCAT

MCAT

MCAT

DATA

DATA

DATA

Data Repository

  • SDSC Storage Request Broker.

  • Manages distributed storage resources.

  • Meta Data Catalogue.

  • Many configurations.

  • Heterogeneous.

  • Efficient data delivery.

  • C++ and Java APIs.


A distributed architecture

MCAT

A Distributed Architecture

  • One node per airport.

  • Single global MCAT.

  • Stream engine data.

  • Global Parallel Search.

  • Present Results.

  • Scalable.

  • Robust.


Summary

Summary

  • Large quantities of data arriving globally.

  • Distributed architecture for data management and search.

  • Scalable and Robust.


  • Login