an architectural approach to managing data in transit
Skip this Video
Download Presentation
An Architectural Approach to Managing Data in Transit

Loading in 2 Seconds...

play fullscreen
1 / 15

An Architectural Approach to Managing Data in Transit - PowerPoint PPT Presentation

  • Uploaded on

An Architectural Approach to Managing Data in Transit. Micah Beck Director & Associate Professor Logistical Computing and Internetworking Lab Computer Science Department University of Tennessee DOE Data Management Workshop 3/17/2004. “Data in Transit”.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' An Architectural Approach to Managing Data in Transit' - dale

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
an architectural approach to managing data in transit

An Architectural Approach to Managing Data in Transit

Micah Beck

Director & Associate Professor

Logistical Computing and Internetworking Lab

Computer Science Department

University of Tennessee

DOE Data Management Workshop 3/17/2004

data in transit
“Data in Transit”
  • After being generated by an instrument or supercomputer
  • Not stored in a permanent archive
  • Serving the diverse purposes of a community of users and applications
  • Being transferred, processed and stored to meet changing and unanticipated needs
    • Visualization
    • Data Mining
    • Collaboration
    • Distributed Computing
interoperability via a common interface
Interoperability via a Common Interface
  • Span heterogeneous physical resources, operating systems, local management schemes
  • Serve changing and unexpected application requirements; enable application autonomy
  • We measure success in terms of infrastructuredeployment scalability
    • In networks and distributed systems, this means number, distribution, global reach, spanning administrative domains…
    • The Internet is the gold standard of infrastructure deployment scalability
layering as an architectural approach
Layering as An Architectural Approach
  • Abstractions at each layer can hide differences at lower layers
  • Exposed approaches avoid creating overly complex mechanisms at lower layers
  • The E2E Principle: Attributes of lower layers implemented on shared infrastructure enable deployment scalability
    • Generality: Serve diverse application needs, model diverse lower layer resources
    • Weak semantics: Don’t give too much away at one time!
the ip network stack
The IP Network Stack



common interface (IP)




ip s failure of scalability
IP’s Failure of Scalability
  • Today, IP is failing as a common interface
  • The design of IP is out of date
    • Application communities are more diverse
    • Link layer technologies violate IP assumptions
  • Application communities are defining their own common interfaces for general resource sharing, deploying their own infrastructure (e.g. the Grid)
  • Some networking communities have abandoned interoperability at the network layer between widely divergent link layer technologies (e.g. optical switching & IP)
the transit layer a new location for interoperability
The Transit Layer: A New Location for Interoperability
  • Expand the link layer to a local layer to model transfer, storage and processing resources
  • Insert a new transit layer between the local and network layers to implement a common interface to diverse technologies at the local layer
  • Adopt a highly general common interface at the transit layer, providing a uniform view of all of the resources of the network node
  • Build diverse network services on top of this common interface to model diverse application requirements
  • “Locating Interoperability in the Network Stack”, Micah Beck & Terry Moore, UT-CS-04-520, Univ. of TN CS Dept Tech Rpt
the transit network stack
The Transit Network Stack




common interface







transit networking a unified view
Transit Networking: A Unified View

“… memory locations … are just wires turned sideways in time”

Dan Hillis, 1982,Why Computer Science is No Good

logistical networking an overlay implementation of the transit layer
Logistical Networking: An Overlay Implementation of the Transit Layer
  • Logistical Networking is an overlay implementation of transit layer functionality built on top of the IP network
  • The Internet Backplane Protocol is the common transit layer interface for Logistical Networking
  • Network nodes are IBP “depots” that run as user level processes, communicate using TCP/IP as well as other link and network layer protocols
  • Depots also serve storage and processing resources to Logistical Networking clients
ln tools and deployment
LN Tools and Deployment
  • The Logistical Runtime System (LoRS) is a set of tools based on IBP that enable users to take advantage of the resources of IBP depots
  • Logistical Distribution Network (LoDN) is a data directory, monitoring and management system
  • The Logistical Backbone is a Resources Discovery service and global experimental IBP testbed
    • Over 35 TB of storage available
    • Over 300 depots in 21 countries
    • Leverages the resources of PlanetLab
  • Additional depots deployed at ORNL & NERSC
example ln applications
Example LN Applications
  • Astrophysics: Terascale Supernova Initiative (A. Mezzacappa, ORNL; J. Blondin, NCSU)
    • Management of massive datasets
  • Fusion Energy Research (S. Klasky, PPPL)
    • Streaming of simulation data during generation
  • Viewset-Based Visualization
    • Prestaging & caching of distant data
  • Content Distribution
    • Heroic data distribution problems (Linux ISOs)
  • Multimedia Networking
    • Creation, mgt & delivery of high value content
ln futures and directions
LN Futures and Directions
  • Storage
    • Implementation of file system services
    • Moving data through firewalls at line speed
    • QoS in highly controlled environments
  • Networking
    • Interoperability at ultrascale
    • Advanced services (e.g. multicast)
  • Computation
    • Offloading visualization to IBP depots
    • Developing sets of operations to support application communities

Thank you!