Infopipes
Download
1 / 29

Infopipes and the Infopipe Stub Generator Galen Swint, Calton Pu, Younggyun Koh, Wenchang Yan - PowerPoint PPT Presentation


  • 95 Views
  • Uploaded on

Infopipes and the Infopipe Stub Generator Galen Swint, Calton Pu, Younggyun Koh, Wenchang Yan. Overview. The Infosphere project Goals of the ISG Implementation Current results Infopipes and OEP Future Work. The Infosphere Project. Distributed computing with RPC is hard

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Infopipes and the Infopipe Stub Generator Galen Swint, Calton Pu, Younggyun Koh, Wenchang Yan' - dillon-hooper


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Infopipes

and the

Infopipe Stub Generator

Galen Swint, Calton Pu, Younggyun Koh, Wenchang Yan


Overview
Overview

  • The Infosphere project

  • Goals of the ISG

  • Implementation

  • Current results

  • Infopipes and OEP

  • Future Work


The infosphere project
The Infosphere Project

  • Distributed computing with RPC is hard

  • RPC semantics do not fit streaming applications

  • Need services to be more composable

  • QoS concerns

    • “expanded” QoS

    • eliminate redundant QoS coding

  • Enhance portability

    • Same specification can be made for several different comm. machines


Motivation
Motivation

  • RPC

    • Procedure call abstraction hides network

      • Application must address QoS

      • Does not understand application packets

      • Request/Response is an unnatural fit for streaming media (movies/sound)

    • Composable?

    • Basis for CORBA, RMI, Sun RPC, SOAP

  • We need an information-centric abstraction that is network aware to complement RPC


Goals for isl isg
Goals for ISL/ISG

  • Form the core of an Infopipe toolkit

  • ISL

    • Simple description for flows

    • Support datatypes

    • Support “basic” composition

    • QoS requirements

  • ISG

    • Generate datatypes, communication stubs

    • Support multiple communication layers

    • Support multiple languages


Implementation
Implementation

  • ISG becomes part of a process

  • XSLT + C++

  • Brand new XIP (Young)

  • New comm. added more quickly (but needs more typing!)

  • XSLT allows easy modularity in templates (xsl:include)

  • Must re-implement the aspects

  • Buzzword compliant



Benchmarks

Mean Time

Std. Dev.

Java RMI

1.5 sec

0.1 sec

ECho/Infopipe

0.35 sec

0.04 sec

TCP socket

0.152 sec

0.001 sec

μ-benchmarks

  • Round trip time (ping-pong)

    • One number sent and received on separate simplex channels

    • 100 experiments of 1000 ping-pongs


Experiment 1 uav
Experiment 1 - UAV

  • UAV demo

    • DoD project

    • ECho

    • Remote camera sends data via wireless link

    • Demonstrates code uploading

    • ISG generates replacement comm. code from SIP/XIP


Experiment 1 uav results

Mean Time

Std. Dev.

Original

19.5 ms

0.6 ms

Infopipe

19.6 ms

0.7 ms

  • Frame transfer

    • 101 runs, discard first

    • No statistical difference

Mean Time

Std. Dev.

Original

598 ms

33 ms

Infopipe

606 ms

45 ms

Experiment 1 - UAV - Results

  • Initialization

    • 101 runs, discard first

    • No statisical difference

Same performance with 36% fewer LOC!


Conclusions
Conclusions

  • No performance hit

  • Substantial LOC savings

  • Time savings for development

  • Potential portability

  • LOC savings probably varies with App


Oep integration recap
OEP Integration Recap

  • Participation in BBN OEP

    • Source-based filters for adaptive constrained resource management in the Multi-UAV demo

    • New filters as Infopipes (compression, encryption)

    • Better DRE application programs and better QoS support (more platforms and dimensions)

  • Infopipe software tools

    • Infopipe Stub Generator, program viewer, other tools

    • Demo apps (BBN OEP, Boeing OEP, etc)


New infopipe functionality
New Infopipe Functionality

  • Three kinds of filters (and more)

    • Image filters (lower/higher resolution, B&W)

    • Compression (gzip, lzo, JPEG)

    • Encryption (Rijndael)

  • Two underlying platforms

    • BSD sockets/TCP and TAO AVStreams/UDP

    • And more: event channels, publish/subscribe

  • Automated QoS management

    • Adaptation by choosing appropriate combination of filters (also controlled manually at runtime)


Abstract infopipe view

DVDview

MPEG/PPM

PBIO/XML

PBIO/XML

Middle

Middle

PBIO/XML

PBIO/XML

Stub

Stub

Stub

Stub

parser

parser

method

method

generator

generator

WebCam

Abstract Infopipe View

Defragmenter

Assembler

BBN Video

Distributor

ATR (PPM)

UDP

Wireless

Link-TCP

QuO

Contract

Filter

Control

GUI

Source

Filter


Single filter experiments

Infopipe Stub

Generator

XML

XML

XMLparser

Middlemethod

XMLgenerator

Stub

Stub

Single Filter Experiments

Application

GUI for Infopipe

Specification

Infopipe Spec.

Lang. (ISL)

(evolving)

ISL2XIP

XIP

(fixed,

extensible)


Multiple filter experiment 1

DVDview

MPEG/PPM

Image Filter

Compress

Encrypt

Source

Decompress

Decrypt

Multiple-Filter Experiment (1)

  • Refining the information flow by combining Infopipes through connectors

ATR


Multiple filter experiment 2

DVDview

MPEG/PPM

Source

Multiple-Filter Experiment (2)

  • Self-configured Infopipes (when data stream is unencrypted, system bypasses the decryption stage)

Compress

Image Filter

Decompress

ATR


Multiple filter experiment 3

DVDview

MPEG/PPM

Compress

Intermediate Processing

Encrypt

UAV

Decompress

Decrypt

Multiple-Filter Experiment (3)

  • Adaptive and flexible distribution of Infopipe stages at runtime

Source

Image Filter

Distributor

ATR


Connector specialization

XMLparser

Middlemethod

XMLgenerator

XMLparser

Middlemethod

XMLgenerator

Stub

Stub

Stub

Stub

Middlemethod

XMLgenerator

Stub

Stub

XMLparser

Middlemethod

Stub

Stub

Connector Specialization

Different machines

SocketConnector

IPCConnector

Function Call Conn.

Same machine

Same process/CPU

Function Call Conn.


Future work
Future Work

  • Stabilize comm. layers

  • Some basic type checking

  • Use aspects for QoS (w/Lenin S.)

  • Wrap in specialization (w/Young K.)

  • Use aspects for performance adaptation


Planned experiments
Planned Experiments

  • Infopipe (filter) development and integration

    • Development cost/time, variety of filter code

    • Code quantity (ISL/XIP module size), quality (runtime overhead), portability (variety of underlying platforms)

    • Integration into OEP, making QoS work with new filters (measure QoS dimensions below)

  • QoS dimensions and trade-offs

    • Performance (e.g., latency, bandwidth, image resolution)

    • Security (e.g., encryption level)

    • Other platforms in OEP that support QoS


Credits
Credits

  • Initial stub generator – Morimori

  • ECho – Greg Eisenhauer, Karsten Schwan

  • Java socket template – Younggyun Koh

  • C socket template – Volkan Altuntas

  • Initial QoS – Wei Han

  • Demo harnesses, demo integration – Wenchang Yan

  • Initial UAV demo – Fabian Bustamante, [email protected] Widener


Xip example
XIP Example

  • Creating a data type

    • From the UAV demo, this holds a frame

<datatype name="Raw_data">

<arg type="integer" name="tag" />

<arg type="char" name="ppm1" />

<arg type="char" name="ppm2" />

<arg type="integer" name="size" />

<arg type="integer" name="width" />

<arg type="integer" name="height" />

<arg type="integer" name="maxval" />

<arrayArg type="byte" name="buff" size="155000" />

</datatype>


Xip example1
XIP Example

  • Creating a filter

    • From the UAV example, this crops an image to half size

<filter name="cropImage1Cto2C"

uri="cropImage.ecl"

inType="Raw_data1C"

outType="Raw_data2C"

takesParams="1"

paramType="Params_rectangle" />


Xip example2
XIP Example

  • Declaring a simple, singular pipe

    • From UAV example, the sender and receivers

    • Note each is only a half-pipe

<pipe name="uavSender" absMachine="ECho2">

<outport name="send1" type="Raw_data" />

</pipe>

<pipe name="uavReceiverNormal" absMachine="ECho2">

<inport name="receive1" type="Raw_data" />

</pipe>


Xip example3
XIP Example

  • Connecting pipes together

<composedPipe name="multiUAV">

<declarations>

<pipe name="sender" class="uavSender" />

<pipe name="receiver" class="uavReceiverNormal" />

</declarations>

<ports />

<connections>

<connection>

<from pipe="sender" port="send1" />

<to pipe="receiver" port="receive1" />

</connection>

</connections>

</composedPipe>


Isg template
ISG Template

  • Snippet for current version of ISG

    • Submit function for sending data

    • for-each command generates for multiple outputs

    • “nodePipeOutConnect” captures some semantic data about the template

<xsl:for-each select="/xip/pipes/pipe[@name=$thisPipeClass]/outport">

<xsl:variable name="portName" select="@name"/>

<nodePipeOutConnect mark="true">

int <xsl:value-of select="$thisPipeName"/>_<connType/>_<xsl:value-of select="$portName"/><connName/>Submit( );

</nodePipeOutConnect>

</xsl:for-each>


Et voila
Et voila!

  • The code produced

//int sender__send1Submit( CarrierStruct * );

int sender__send1Submit( );


Benchmarks1

Mean Time

Std. Dev.

ECho/Infopipe

2.6 sec

0.015 sec

TCP socket

0.122 sec

0.003 sec

  • 1000 Integers

    • Loop 10,000 times sending 1,000 integers (4000 bytes)

Mean Time

Std. Dev.

ECho/Infopipe

3.463 sec

0.004 sec

TCP socket

3.393 sec

0.003 sec

μ-benchmarks

  • Single Integer

    • Loop 100,000 times sending 1 integer (4 bytes)


ad