optiputer system software n.
Skip this Video
Download Presentation
OptIPuter System Software

Loading in 2 Seconds...

play fullscreen
1 / 21

OptIPuter System Software - PowerPoint PPT Presentation

  • Uploaded on

OptIPuter System Software. Eric Weigle et al. speaking for Andrew A. Chien Computer Science and Engineering, UCSD January 2006 OptIPuter All-Hands Meeting. OptIPuter System Software Architecture. Applications. Visualization. DVC API. Distributed Virtual Computer Middleware.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'OptIPuter System Software' - hamlet

Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
optiputer system software

OptIPuter System Software

Eric Weigle et al. speaking for

Andrew A. ChienComputer Science and Engineering, UCSD

January 2006

OptIPuter All-Hands Meeting

optiputer system software architecture
OptIPuter System Software Architecture




Distributed Virtual Computer Middleware

DVC Runtime Library

DVC Configuration

DVC Services



DVC Job Scheduling

DVC Core Services





Security Management

High Speed


Storage Services






Optical Signaling,








Transport Protocols




Distributed Applications/ Web Services





Data Services




performance across layers
Performance Across Layers

3 &5Layer Demos



Distributed Virtual


Novel Transport


Optical Network


Year 4 & 5: Integration, Performance Tuning, Tech Transfer

Valerie Taylor et al (TAMU)

cross team integration and demonstrations
Cross Team Integration and Demonstrations
  • 2-layer Demo, TeraBIT Juggling, [SC2004, Nov2004]
    • DVC middleware, high-speed transport (GTP)
    • Move data between OptIPuter network endpoints
      • 10 endpoints across UCSD, UvA, UIC, Pittsburgh
      • Achieved 17.8Gbps, a TeraBIT in less than one minute
  • 3-layer Demo [AHM2005, Jan2005]
    • Visualization (JuxtaView/LambdaRAM), DVC middleware, high-speed transport
    • Remote data visualization (visualization: NCMIR; storage: UIC and UvA)
      • Use DVC to establish visualization environments
        • Automated Grid resource selection and binding
      • Achieved 2.6 Gbps on ~7 Streams
  • 5-layer Demos [iGrid2005, Sep2005]
    • Applications, visualization, DVC middleware, high-speed transports (GTP), optical network configuration (PIN/PDC)
    • Demo #1: Collaborative Data Visualization with Earth-Sciences
    • Demo #2: Real-time Brain Data Acquisition, Assembly and Analysis
optiputer high performance transport protocols
OptIPuter High-Performance Transport Protocols
  • Bridge the Gap between High Speed Link Technologies and Growing Demands of Advanced Applications
    • TCP has well-documented performance problems on long-haul networks
  • Pursue complementary avenues of investigation
    • Efficient congestion/flow management, fairness among flows
    • High-speed group communication (multipoint-to-point, multipoint-to-multipoint)

Network Connection

Private Lambda

Shared, Routed















composite endpoint protocol cep accomplishments plans
Composite Endpoint Protocol (CEP)- Accomplishments & Plans
  • OptIPuter “Gold Roll” 1 (CEP v. 1.1)
    • Initial release, basic functionality
    • 32 Gbps in the LAN
    • TCP, some automatic tuning
  • Software Summit Release (CEP v. 1.2)
    • New file transfer, sockets API
    • New internal networking stack
      • Preliminary GTP, XIO carrier support
    • Support for 64-bit systems, more OSes
  • OptIPuter “Gold Roll” 2 (CEP v. 2.0)
    • Target: Fall 2006
    • Code stabilization
    • Documentation
    • Improved scalability
    • Improved GTP, XIO integration
    • Improved performance
    • Suitable for public release

Eric Weigle (UCSD)

group transport protocol gtp accomplishments plans
Group Transport Protocol (GTP)- Accomplishments & Plans







Active Sessions






Lambda Networks

  • Extend GTP to Sender Capacity Management
    • End node based allocation schemes at both sources and sinks to achieve good global performance and fairness
    • Proof of stability and convergence properties of GTP
  • Comprehensive comparison studies between GTP and other transport protocols
  • Implementation and Demonstrations with OptIPuter System Software
    • iGrid2005
    • OptIPuter “Gold Roll” 1 (Basic functionalities)
  • 3 Publications
  • Analytical studies
    • Finalize convergence proofs in asynchronous cases
    • More comparison studies
    • Study the interaction between GTP and other TCP traffic
  • Implementation: OptIPuter “Gold Roll” 2
    • GTP v. 2.0
    • Target: Summer 2006
    • Goals
      • Support both source and sink allocation schemes
      • Improved CPU efficiency and scalability
      • Improved CEP, XIO integration
      • Suitable for public release

Ryan Wu (UCSD)

udp based data transfer protocol udt accomplishments plans
UDP-based Data Transfer Protocol (UDT)– Accomplishments & Plans
  • Based upon experience to date, a new version of UDT is being developed called Composible-UDT
  • Composible-UDT supports multiple high-speed congestion control algorithms, including
    • UDT decreasing-increases AIMD congestion control
    • Reliable UDP blast
    • TCP, HighSpeed TCP, Scalable TCP, BiC, FAST, Vegas, Westwood
    • GTP
  • Recent Experimental Studies
    • iGrid 2005 - High performance mining of data streams over UDT
      • 8 Gb/s computing histograms on web traffic data streams
      • 14 Gb/s transferring data memory-memory around the world
    • iGrid 2005 - Remote exploration of Sloan Digital Sky Survey data
      • 1.2 Gb/s transfer disk-disk from San Diego to South Korea
  • Plan for UDT 3.0 release
    • Composable-UDT (first full release)
    • Congestion controlled unreliable messaging
    • Firewall punching
  • Further work on protocol toolkit underlying Composible-UDT
  • Experiments to understand how to provide secure transport using UDT

Robert L. Grossman & Yunhong Gu (UI-Chicago)

udp offload engines for lambdagrids
UDP Offload Engines for LambdaGrids
  • Offload UDP Based Protocols to the Network Cards.
  • Initial Results
    • 7.4 Gbps Maximum throughput.
    • 35% improvement over Host based UDP
    • Reduced CPU utilization.
    • 17% improvement in Latency
  • Future Plans
    • Build LambdaStream on the NIC.
    • Evaluate Partial Offload Engines with full offload.

A Case for UDP-offload engines in LambdaGrids – Venkatram Vishwanath, et al. (PFLDNet 2006) - In collaboration with OSU, LANL, VT)

Venkatram Vishwanath et al. (EVL)

Venkatram Vishwanath et al. (EVL)

  • An application-level transport protocol for streaming and data transfer for dedicated high-bandwidth networks.
  • Current Status
    • A single-stream version with a configurable design.
    • API which supports buffered and unbuffered communications.
    • Working towards a Multi-stream LambdaStream for Multipoint to Multipoint communication
  • Recent Results
    • 18 Gbps between Chicago and San Diego over TeraWave.
    • 18 Gbps between Chicago and San Diego over CaveWave.
  • Future Plans
    • Integrate with SAGE and other applications.
    • Performance Evaluation with MAGNET to identify end-system bottlenecks.

Venkatram Vishwanath et al (EVL)

  • A monitoring apparatus for generic kernel event tracing.
    • Identify end system performance bottlenecks in a generic linux kernel and improve next generation protocols, middleware, and software applications.
  • Current status:
    • Uses the “probes” mechanism in the kernel.
    • Designed as a kernel module.
    • Monitor the Network Stack.
  • Future Plans:
    • More instrumentation points.
    • Performance Analysis of LambdaStream.
    • Analysis and Synthesis for Adaptive Visualization applications

Joint work with Wu Feng, Mark GardnerLANL and Virginia Tech.

Venkatram Vishwanath et al. (EVL)

storage accomplishments plans
Storage- Accomplishments & Plans
  • RobuSTore Design
    • RobuSTore Architecture Consisting of Coding Algorithm, Metadata Service, Admission Controller, and Security Schemes
  • RobuSTore Evaluation Across a Wide Range of System Configurations
    • Evaluate RobuSTore against Conventional Parallel Storage Schemes (i.e. RAID):
    • Explore Five Dimensions
      • # Disks, Data Size, Block Size, Network Latency, Degree of Redundancy
    •  5x Improvement on Robustness; 15x on Access Bandwidth
    •  Moderate Overhead: 2~4x Storage Capacity, 1.5x Network, Disk I/O
  • Simulation Study with More Complex and Realistic Workloads
  • RobuSTore Implementation (based on Lustre) and Deployment
    • Experiments on the OptIPuter Testbed
    • Evaluation Using Benchmarks and Neuroscience and Geophysical Application Workloads

Huaxia Xia, Justin Burke (UCSD)

vision real time tightly coupled wide area distributed computing
Vision – Real-Time Tightly Coupled Wide-Area Distributed Computing

Real-Time Object network


  • High-precision Timings of Critical Actions
  • Tight Bounds on Response Times
  • Ease of Programming
    • High-Level Prog
    • Top-Down Design
  • Ease of Timing Analysis

Dynamically formed DistributedVirtual Computer

K. Kim (UCI)

real time progress
Real-Time Progress
  • RCIM (RT comm infrastructure mgt)
    • Study of TT Ethernet under way with the help of Hermann Kopetz
    • Hope to acquire the 1st unit some time in 2006.
  • IRDRM (Intra-RT-DVC resource mgt)
    • TMO (Time-triggered Message-triggered Object) Support Middleware (TMOSM)
    • Redesigned TMOSM improves modularity, concurrency, portability, and timing precision. It runs on Linux, WinXP, & WinCE.
    • Extending the TMOSM to exploit unique capabilities of Jenks’ cluster SPDS2.
  • Programming model
    • API for RT middleware enables high-level RT programming (TMO) without a new compiler.
    • The notion of Distance-Aware (DA) TMO, an attractive building-block for RT wide-area DC applications, was created created and a study for its realization is under way.
  • Enhancement the Network Infrastructure of OptIPuter
    • GPS receivers acquired from German vendor & installed in UCI/UCSD (Calit2 bldgs)
    • One-way message delay between UCI and UCSD measured
      • Jitters were less than 60 microsecs.
  • Application development experiments
    • Preparation of demos; at stage where LAN-based feasibility demos are working.
      • e.g., Low-jitter video, Fair and efficient Distributed On-Line Game Systems
  • Publications in IDPT2003, AINA2004, WORDS2005, ISORC2005, …

Source: Kim, Jenks, et al. at UCI

year 4 plan
Year 4 Plan
  • RCIM (RT comm infrastructure mgt)
    • Development of middleware support for TT Ethernet
  • IRDRM (Intra-RT-DVC resource mgt)
    • Extending the TMOSM to further exploit unique capabilities of Jenks’ cluster SPDS2
    • Full development of Support for Distance-Aware TMOs
    • Interfacing TMOSM to the Basic Infrastructure Services of OptIPuter
  • Demos
    • Remote access and control of electron microscopes at UCSD-NCMIR
    • Remote control of an electric car

OptIPuter Paths

Remote Controlnode

Local Relaynode

Source: Kim, Jenks, et al. at UCI

areas of security effort
Areas of Security Effort
  • Trusted remote computation (UCI)
    • Protection from cheating
    • Reduce effort on clients, coordinator effort
    • Solution: inject false positives (“chaff”) as a check
  • Broadcast keying (UCI)
    • Protect messages to large groups
    • Minimize number and distribution of keys
    • Solution: chromatic leap-frog keys, predeployed keysets
  • Secure network protocols (USC/ISI)
    • Transport disconnection, spoofing
    • Alleviate DOS attack impact
    • Solution: layer different algorithms, SPI-spinning

Joe Touch & Mike Goodrich, UCI/ISI

ipsec baseline performance
IPsec Baseline Performance


Packet size

Joe Touch & Mike Goodrich, UCI/ISI

effect of dos traffic
Effect of DOS Traffic
  • IETF TCPM WG doc
    • Need for IP-layer solution
    • Infrastructure-free security
  • IETF Triage session
    • Reducing load of spoofed DOS traffic
  • Goal: Internet Standard


Packet size

Joe Touch & Mike Goodrich, UCI/ISI

summary of accomplishments
Summary of Accomplishments
  • Integration and Demonstration of Capability
    • All five layers (application, visualization, DVC, transport protocols, Optical network control)
    • Across campus, national and international-scale test beds
  • Distributed Virtual Computer
    • Integrate with network configuration (PIN/PDC)
    • Simulation study of service models of configurable network
    • Simulation study of efficient resource selection algorithms
  • Advanced Transport Protocols
    • GTP: Analytic and simulation study, extend to sender capacity management
    • CEP: Implement and evaluate N-to-M communication; integrate with XIO
    • SABUL/UDT: Development of composable congestion control algorithms
    • LambdaStream: Configurable design, API for buffered/unbuffered communication
  • Real-Time Programming/Networking/DVC
    • Time-triggered Message-triggered Object support middleware & API
    • Enhancement of optiputer network infrastructure with GPS
    • Evaluation of delay/jitter between UCI/UCSD
  • File/Storage Systems
    • New RobuSTore architecture with Erasure coding, statistical guarantees
    • Coding algorithm, metadata service, admission controller, security schemes
    • Evaluation across a wide range of system configurations & conventional schemes
  • Performance Analysis/Modelling
    • Performance Analysis of VolaTile & other OptIPuter software
  • Network Security (Touch & Goodrich)
optiputer rocks roll releases
OptIPuter ROCKS Roll Releases
  • OptIPuter System SW Roll (Optigold) v.1 [July2005]
    • Stable, integrated for OptIPuter iGrid2005 Demos
    • OptIPuter software
      • DVC Middleware v.1.0
        • Core resource, security, namespace and job management services
        • Network binding service (interface with PDC)
        • DVC configuration and communication APIs
      • Advanced Transport Protocols
        • Group Transport Protocol (GTP) v.0.95
        • Composite Endpoint Protocol (CEP) v1.1
      • Optical Network Configuration
        • Photonic Domain Controller (PDC) v.2.0
    • External Software
      • Globus Toolkit 4.0
  • OptIPuter Software Summit 2006 (Jan-Feb, 2006)
    • Build a Completed, Tested, End-to-end OptIPuter Software Stack
    • Create Site for Downloadable OptIPuter Software
summary of plans
Summary of Plans
  • Lots of progress in 2005- Met and exceeded goals!
  • Planned Features & Improvements
    • Distributed Virtual Computer (DVC)
      • Second-gen architecture, prototypes, improved cross-team integration and resource selection
    • Real-Time (TMO)
      • RT communication infrastructure, Intra-RT DVC resource mgmt, applications (NCMIR)
    • Performance Analysis (Prophesy)
      • Develop and utilize archive for performance data
    • High Speed Protocols (CEP, LambdaStream, XCP, GTP, UDT)
      • CEP: version 2.0: code stabilization, integration, performance, documentation
      • GTP: version 2.0: convergence, comparison studies, integration with other protocols
      • UDT version 3.0: composible, congestion controlled, firewall punching, security
      • LamdaStream: UDP Offload, integration with SAGE & apps, bottleneck identification
    • Storage (Robustore)
      • Simulation/evaluation with more complex/realistic workloads
      • implementation and deployment (lustre, optiputer testbed)
    • Security
      • Explore IPsec variants to reduce attacker advantage
  • Planned 2006 releases
    • ~September 2006 – System Software Gold Roll version 2
    • ~November 2006 – cross-team 5-level demonstrations (SC2006)