an architectural approach to managing data in transit n.
Skip this Video
Download Presentation
An Architectural Approach to Managing Data in Transit

Loading in 2 Seconds...

play fullscreen
1 / 15

An Architectural Approach to Managing Data in Transit - PowerPoint PPT Presentation

  • Uploaded on

An Architectural Approach to Managing Data in Transit. Micah Beck Director & Associate Professor Logistical Computing and Internetworking Lab Computer Science Department University of Tennessee DOE Data Management Workshop 3/17/2004. “Data in Transit”.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'An Architectural Approach to Managing Data in Transit' - dale

Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
an architectural approach to managing data in transit

An Architectural Approach to Managing Data in Transit

Micah Beck

Director & Associate Professor

Logistical Computing and Internetworking Lab

Computer Science Department

University of Tennessee

DOE Data Management Workshop 3/17/2004

data in transit
“Data in Transit”
  • After being generated by an instrument or supercomputer
  • Not stored in a permanent archive
  • Serving the diverse purposes of a community of users and applications
  • Being transferred, processed and stored to meet changing and unanticipated needs
    • Visualization
    • Data Mining
    • Collaboration
    • Distributed Computing
interoperability via a common interface
Interoperability via a Common Interface
  • Span heterogeneous physical resources, operating systems, local management schemes
  • Serve changing and unexpected application requirements; enable application autonomy
  • We measure success in terms of infrastructuredeployment scalability
    • In networks and distributed systems, this means number, distribution, global reach, spanning administrative domains…
    • The Internet is the gold standard of infrastructure deployment scalability
layering as an architectural approach
Layering as An Architectural Approach
  • Abstractions at each layer can hide differences at lower layers
  • Exposed approaches avoid creating overly complex mechanisms at lower layers
  • The E2E Principle: Attributes of lower layers implemented on shared infrastructure enable deployment scalability
    • Generality: Serve diverse application needs, model diverse lower layer resources
    • Weak semantics: Don’t give too much away at one time!
the ip network stack
The IP Network Stack



common interface (IP)




ip s failure of scalability
IP’s Failure of Scalability
  • Today, IP is failing as a common interface
  • The design of IP is out of date
    • Application communities are more diverse
    • Link layer technologies violate IP assumptions
  • Application communities are defining their own common interfaces for general resource sharing, deploying their own infrastructure (e.g. the Grid)
  • Some networking communities have abandoned interoperability at the network layer between widely divergent link layer technologies (e.g. optical switching & IP)
the transit layer a new location for interoperability
The Transit Layer: A New Location for Interoperability
  • Expand the link layer to a local layer to model transfer, storage and processing resources
  • Insert a new transit layer between the local and network layers to implement a common interface to diverse technologies at the local layer
  • Adopt a highly general common interface at the transit layer, providing a uniform view of all of the resources of the network node
  • Build diverse network services on top of this common interface to model diverse application requirements
  • “Locating Interoperability in the Network Stack”, Micah Beck & Terry Moore, UT-CS-04-520, Univ. of TN CS Dept Tech Rpt
the transit network stack
The Transit Network Stack




common interface







transit networking a unified view
Transit Networking: A Unified View

“… memory locations … are just wires turned sideways in time”

Dan Hillis, 1982,Why Computer Science is No Good

logistical networking an overlay implementation of the transit layer
Logistical Networking: An Overlay Implementation of the Transit Layer
  • Logistical Networking is an overlay implementation of transit layer functionality built on top of the IP network
  • The Internet Backplane Protocol is the common transit layer interface for Logistical Networking
  • Network nodes are IBP “depots” that run as user level processes, communicate using TCP/IP as well as other link and network layer protocols
  • Depots also serve storage and processing resources to Logistical Networking clients
ln tools and deployment
LN Tools and Deployment
  • The Logistical Runtime System (LoRS) is a set of tools based on IBP that enable users to take advantage of the resources of IBP depots
  • Logistical Distribution Network (LoDN) is a data directory, monitoring and management system
  • The Logistical Backbone is a Resources Discovery service and global experimental IBP testbed
    • Over 35 TB of storage available
    • Over 300 depots in 21 countries
    • Leverages the resources of PlanetLab
  • Additional depots deployed at ORNL & NERSC
example ln applications
Example LN Applications
  • Astrophysics: Terascale Supernova Initiative (A. Mezzacappa, ORNL; J. Blondin, NCSU)
    • Management of massive datasets
  • Fusion Energy Research (S. Klasky, PPPL)
    • Streaming of simulation data during generation
  • Viewset-Based Visualization
    • Prestaging & caching of distant data
  • Content Distribution
    • Heroic data distribution problems (Linux ISOs)
  • Multimedia Networking
    • Creation, mgt & delivery of high value content
ln futures and directions
LN Futures and Directions
  • Storage
    • Implementation of file system services
    • Moving data through firewalls at line speed
    • QoS in highly controlled environments
  • Networking
    • Interoperability at ultrascale
    • Advanced services (e.g. multicast)
  • Computation
    • Offloading visualization to IBP depots
    • Developing sets of operations to support application communities

Thank you!