soap events l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
SOAP Events PowerPoint Presentation
Download Presentation
SOAP Events

Loading in 2 Seconds...

play fullscreen
1 / 69

SOAP Events - PowerPoint PPT Presentation


  • 178 Views
  • Uploaded on

SOAP Events. An Extensible and Interoperable Event System Using SOAP (xEvents Reference Implementation). Aleksander Slominski IU Extreme! Lab . Outline. Scope and Requirements Whirlwind survey of existing standards Event System Design SOAP Events Core (Base Event Type, Event Listener)

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'SOAP Events' - usoa


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
soap events

SOAP Events

An Extensible and Interoperable

Event System Using SOAP

(xEvents Reference Implementation)

Aleksander Slominski

IU Extreme! Lab

outline
Outline
  • Scope and Requirements
  • Whirlwind survey of existing standards
  • Event System Design
  • SOAP Events
    • Core (Base Event Type, Event Listener)
    • Advanced (Passing Remote References, Leasing Event Channels)
  • Implementation: xEvents
    • on top of SoapRMI 1.1 Java / C++
  • Further work
scope
Scope
  • Application and Component Level Events
  • Not real time or multimedia
  • Focus on events that glue disparate components and allows for easy application integration (as painless as possible…)
  • Optimal solution: simple (good for majority of cases) but extensible (for the rest) and easy to deploy (leveraging existing standards)
what is event
What is Event?
  • time stamped message
  • delivered from source
  • to subscribed listeners
  • that contains typed data
  • may contain sequence number to prevent duplication and time-to-live
requirements
Requirements
  • Simple
    • Make typical cases simple
  • Extensible
    • Make other cases possible
  • Easily Deployable
    • Uses standards
    • Language and Platform Independent
    • Internet Friendly (firewalls…)
requirements details
Requirements - details
  • Naming
    • How to find event publishers (LDAP, …)
  • Types - Extensible Data Format
    • Today it means XML and XML schemas…
  • Internet Enabled
    • Works with HTTP (firewalls, NAT, …)
    • Can survive and recover from failures …
  • Simple OO Extensible Interfaces (IDL, …)
  • Security – Leveraging Existing Standards
    • TLS/SSL, X.509 certificates, digital signatures, …
tour of existing standards
Tour of Existing Standards
  • Corba Events
  • Jini Events
  • Over 100 other event systems (or more)
    • http://www.cs.caltech.edu/~adam/isen/event-systems.html
  • Ad hoc solutions – application specific mechanism using proprietary format
    • write to file, socket, and other magic …
corba events
Corba Events
  • Naming Service to locate listeners
  • Push/Pull models and good performance
  • Available solutions to enable Internet computing through firewalls (but not designed for) and can have load balancing and failure recovering mechanisms
  • Use of IDL to describe interfaces
  • Available extensions for security
  • High Performance and Multiple Languages but Complex and Steep Learning Curve …
jini events
Jini Events
  • Naming with Jini Lookup Service (some limitations but can use optionally JNDI)
  • Event is Java Object (good extensibility)
  • Listener interface - flexible publisher model
  • Leasing - very good for self healing
  • Uses RMI (available solutions for firewalls)
  • Leverages Java for security
  • Easy to use but limited to Java (generally)
ad hoc solutions
Ad Hoc Solutions
  • It works! (for given application)
    • but: it does not works with others (on Grid) …
  • Can be very well optimized and probably is well contained (no need to change it!)
  • Existing applications are typically hard to modify therefore simple interoperable event mechanism is crucial (in FORTRAN/C just function call to library etc.)
  • Typically limited to the application and hard to connect to other event systems
conclusions
Conclusions
  • Any proposed event mechanism must work with existing event standards
  • No one solution is good for everything …
  • Therefore it is important to have simple and extensible event system that can easily be added to existing applications
  • It must allow for easy naming, object types, security and leverage Internet standards
event system design
Event System Design
  • Choices made for this event system
design objectives
Design objectives
  • Events are signals (small size) and actual data transfers should use more high performance messaging protocols
  • Platform and language independent
  • Pervasive and simple format - XML
    • Easy to receive, use (parse) and store
  • Works and adapts to Internet using most widespread global protocol – HTTP
  • Naming (LDAP) and Security (SSL/X.509)
soap 1 1
SOAP 1.1
  • Standard supported by both IBM and Microsoft (foundation of .NET strategy)
  • Elegant and simple
  • Builds on top of XML and XML Schemas
  • XML it is verbose but good to create common data format and easy to transform
  • XML Schemas to describe what we actually send and expect to receive (object types)
naming
Naming
  • Leverage existing and working standard
  • Minimum: support LDAP
  • Allow to work with emerging standards such as UDDI to describe event services
  • SOAP can be easily extended to allow to store remote references in naming services
types
Types
  • Extensible but simple system to describe event types
  • Not centralized and platform independent
  • Easy to store, process and query
  • XML and XML schemas fits the bill
internet enabled
Internet Enabled
  • SOAP RPC = HTTP + XML
    • Simple but extensible RPC model
    • Defines HTTP bindings for SOAP
  • Robust and easy to implement
  • Good to send signals and also to provide tle lowest common denominator protocol
    • ex. to negotiate compression, encryption, QoS or type of faster binary protocol used
  • Firewall friendly (leverages HTTP infrastructure)
wan distributed computing
WAN & distributed computing
  • Everything can fail just any time ..
  • Therefore it is important to incorporate leasing concept
  • However if necessary allow for unleased operations (when leasing can not work with existing applications or components)
simple oo interfaces
Simple OO Interfaces
  • Event interfaces should not hide but expose objects
  • Easy to understand and to extend
  • WSDL or IDL to describe what is event listener and how to subscribe with publisher
  • Promote interoperability
    • Must be language and platform independent
security
Security
  • Rarely can be added later
  • SOAP already can use HTTP level security (TLS/SSL)
  • Proposed extension to SOAP to support digital signatures
  • Event system should allow to operate in unsafe mode (for testing, debugging, …)
soap events core
SOAP Events Core
  • Translating requirements into event system architecture …
event design
Event Design
  • Builds on top of SOAP 1.1 encoding
  • eventType and source can be used for filtering and message dispatching
  • seqNum prevents against delivering duplicated events
  • message contains descriptive information
  • timestamp – time in milliseconds since UNIX epoch
soap event
SOAP Event
  • Event is a base class and it contains following fields:
    • eventNamespace – URI of namespace
    • eventType – dot hierarchical event type name
    • source – string describing source (depends on eventType)
    • timestamp – in milliseconds since UNIX epoch
    • seqNo – sequence number
    • message – human readable description of the event
    • handback – described later with subscription
  • Event type is defined by XML schema: (next slide)
slide24
<schema targetNamespace = “http://www.extreme.indiana.edu/soap/events/” >

<element name="Event">

<complexType><sequence>

<element name="eventNamespace" type="string"/>

<element name="eventType" type="string"/>

<element name="source" type="string"/>

<element name="timestamp" type="long"/>

<element name="seqNo" type="long"/>

<element name="message" type="string"/>

<element name="handback" type="string"/>

</sequence></complexType>

</element>

example resource event
Example: Resource Event

<MachineUtilizationEvent>

<eventNamespace>http://www.extreme.indiana.edu/soap/events/resdat#MachineUtilizationEvent</eventNamespace>

<eventType>resdata.machine.utilization</ev…>

<source>rainier.extreme.indiana.edu</source>

<timestamp>982862312897<timestamp>

<cpuUtilization>0.88< cpuUtilization >

<memoryUsed>123988</memoryUsed>

</MachineUtilizationEvent >

slide26
<schema targetNamespace=“http://www.extreme.indiana.edu/soap/events/resdat/” xmlns:m=“http://www.extreme.indiana.edu/soap/events/” >

<import namespace="http://www.extreme.indiana.edu/soap/events/"

schemaLocation="event.xsd"/>

<element name="MachineUtilizationEvent" base="m:Event" derivedBy="extension">

<complexType><sequence>

<element name="cpuUtilization" type="double"/>

<element name="memoryUsed" type="long"/>

</sequence></complexType>

</element>

</schema>

event listener
Event Listener
  • Very simple interface
  • Only one method that accepts Event
  • Can throw SOAP 1.1 Fault in case of exceptions
  • In Java pseudo code (WSDL etc. in future):

interface EventListener {

void handleEvent(Event ev) throws Exception;

}

soap events and others
SOAP Events and others
  • Interoperability
    • Any SOAP compliant implementation may publish and listen for events and use Event Channel
  • Mobility
    • To generate events there is no need to have full fledged SOAP – it is enough to write pre-formatted strings into socket.
do we need more
Do We Need More?
  • Actually core SOAP Events has all that is necessary to publish and listen for events!
  • However it would be nice to have more:
    • Using and storing simple remote references (in file or in Naming Service)
    • Lease and leasing event publishers
    • Event channels to decouple event publisher and listeners
soap events channels
SOAP Events Channels
  • Decoupling producers from consumers (Event Channels)
  • Supporting leased subscription (and simple XML Remote Reference) to deal with inherent distributed system failures
  • Using Naming Service for higher reliability: mobile event listeners and channels supports dynamic failure recovery (lease renew + lookup)
simple remote reference
Simple Remote Reference
  • Location of remote endpoint (web service)
  • What is the type of endpoint (uniquely identified by URI to its IDL)
  • That is it!
  • However sometimes it is necessary to have:
    • URI namespace of of the endpoint (required by Apache SOAP)
    • SOAPAction
example of remote reference
Example of Remote Reference

<Port>

<endpoint>

<location>http://192.168.1.7:4566/urn:soaprmi-v11:leasing-filtered-event-channel</location>

<binding><name>urn:soaprmi-v11:leasing-filtered-event-channel</name> </binding>

</endpoint>

<portType>

<uri>urn:soaprmi-v11:temp-java-port-type</uri> <name>soaprmi.events.LeasingFilteredEventChannel</name>

</portType></Port>

port remote reference
Port = Remote Reference
  • Port has Endpoint(s)
  • Endpoint has Location (URL)
  • Endpoint may have optionally Binding
  • Binding may contain endpoint dependent parameters (in example it is name that describes SOAP URI)
  • Port type is uniquely identified by pair (uri, name) and it may point simply to WSDL(ex. http://host/myService.wsdl#Listener)
  • This is just XML and can be stored in Directory Service
leased event subscription
Leased Event Subscription
  • Subscribe passing remote ref and asking for lease (EventListener is Port)
  • When subscribing an event lease is returned (simple object that contain duration of lease granted)

interface LeasingFilteredEventPublisher {

EventLease subscribeLease(

EventListener listener,

Event filter, long leaseDuration,

String handback) throws Exception;

}

yet another xml schema
Yet another XML schema …

<schema targetNamespace = “http://www.extreme.indiana.edu/soap/events/” >

<element name="EventLease">

<complexType>

<sequence>

<element name=“duration“ type="long"/>

</sequence>

</complexType>

</element>

example subscribe request
Example Subscribe Request
  • It is using SOAP-RPC
    • HTTP header
    • SOAP envelope and body and XML element with function name (subscribeLease) and namespace identifying endpoint (from binding)
    • Parameters
      • First parameter is remote reference (it is Port)
      • The rest of parameters
    • End of SOAP envelope
http headers
HTTP Headers

POST /leasing-filtered-event-channel HTTP/1.0

Host: localhost

Content-Type: text/xml

Content-Length: 1475

SOAPAction: "http://localhost:4566/leasing-filtered-event-channel#subscribeLease"

Connection: Close

soap envelope and call
SOAP Envelope and Call

<SOAP-ENV:Envelope

xmlns:SOAP-ENV='http://schemas.xmlsoap.org/soap/envelope/'

xmlns:SOAP-ENC='http://schemas.xmlsoap.org/soap/encoding/‘

xmlns:xsi='http://www.w3.org/1999/XMLSchema/instance/'

xmlns:xsd='http://www.w3.org/1999/XMLSchema/'

SOAP-ENV:encodingStyle='http://schemas.xmlsoap.org/soap/encoding/'>

<SOAP-ENV:Body>

<subscribeLease>

first param with remote ref
First param with remote-ref

<p1 id='id1' xsi:type='ns1:soaprmi.port.Port' xmlns:ns1='urn:soaprmi-v11:temp-java-xml-type'>

<endpoint id='id2' xsi:type='ns2:soaprmi.port.Endpoint' xmlns:ns2='urn:soaprmi-v11:temp-java-xml-type'>

<binding id='id3' xsi:type='ns3:soaprmi.port.Binding' xmlns:ns3='urn:soaprmi-v11:temp-java-xml-type'>

<name xsi:type='xsd:string'>urn:soaprmi-v11:simple-listener</name>

</binding>

<location xsi:type='xsd:string'>http://192.168.1.7:4561/urn:soaprmi-v11:simple-listener</location>

</endpoint>

<name xsi:type='xsd:string'>urn:soaprmi-v11:simple-listener</name>

<portType id='id4' xsi:type='ns2:soaprmi.port.PortType' xmlns:ns2='urn:soaprmi-v11:temp-java-xml-type'>

<name xsi:type='xsd:string'>soaprmi.events.EventListener</name>

<uri xsi:type='xsd:string'>urn:soaprmi-v11:temp-java-port-type</uri>

</portType>

<userName xsi:type='xsd:string'></userName>

</p1>

rest of parameters and closing of soap envelope
Rest of parameters and closing of SOAP envelope

<p2 xsi:type='xsd:long'>240000</p2>

<p3 xsi:type='ns1:soaprmi.events.Event' xmlns:ns1='urn:soaprmi-v11:temp-java-xml

-type' xsi:null='1'/>

<p4 xsi:type='ns1:string' xmlns:ns1='http://www.w3.org/1999/XMLSchema/' xsi:null

='1'/>

</subscribeLease>

</SOAP-ENV:Body>

</SOAP-ENV:Envelope>

event channels
Event Channels
  • Event Channel = combination of Even Listener with Event Publisher:

public interface EventChannel extends EventListener, EventPublisher {

}

public interface LeasingFilteredEventChannel extends EventChannel, LeasingFilteredEventPublisher {

}

implementation xevents
Implementation: xEvents
  • xEvents is implemented in SoapRMI C++ and Java (also can be used from Jython)
  • This is proof-of-concept implementation
  • To check that SOAP Events design will work with any SOAP 1.1 compliant implementation.
soaprmi design motivation
SoapRMI Design Motivation
  • RMI model
  • Web services
  • Distributed Events
typical rmi scenario
Typical RMI Scenario

Registry

[Java/C++/…]

Client

[Java/C++/…]

Server

[Java/C++/…]

2.

1.

3.

<Port>

<location>

http://host:8888/server

</location>

<binding>…

</Port>

SOAP on the wire

Internet space

stages in rmi scenario
Stages in RMI scenario

1. Server publishes remote reference:

Server serverImpl = new ServerImpl();

Naming.bind(“helloService”, serverImpl);

2. Client lookups server remote reference:

Server serverRef = (Server) Naming.lookup(“helloService”);

3. Client execute call on server:

serverRef.sayHello(“World”);

web services
Web Services

<Port>

<location> http://host:8888/server

</location>

<binding>…

</Port>

Server

[Java/C++/…]

HelloServlet.java

Client

[Java/C++/…]

HelloClient.java

SOAP on wire

Load service description

(SOAP/Files/WebDAV)

Publish service description

(SOAP/Files/WebDAV)

<Port>

<location> http://host:8888/server

</location>

<binding>…

</Port>

Service Directory

distributed events
Distributed Events

<Port>

<location> http://host:8888/server

</location>

<binding>…

</Port>

Publisher

[Java/C++/…]

Listener

[Java/C++/…]

SOAP on wire

Publish service description

(SOAP/Files/WebDAV)

Load service description

(SOAP/Files/WebDAV)

<Port>

<location> http://host:8888/server

</location>

<binding>…

</Port>

Subscription

soap events48
SOAP Events

SOAP handleEvent

Events

Listener

Events

Publisher

SOAP handleEventResponse

  • Implemented as:
  • Java application
  • Servlet
  • Python script
  • Others: Perl, VB, …
  • Implemented as:
  • Java application
  • C/C++ application
  • Python script
  • Others: Perl, VB, …

SOAP 1.1

soap and soaprmi
SOAP and SoapRMI
  • RMI system on top of SOAP (simple API)
  • Supports Web Services
    • SoapRMI remote object can be put into servlet
    • To contact specify URL and interfaces
  • Naming services
    • Standard RMI registry (Naming.bind, …)
    • JNDI for Java (and access OpenLDAP)
    • SoapRMI Registry-to-JNDI bridge for C++
implementation details
Implementation Details
  • Compliant with SOAP 1.1
  • Sample code available to show how to write event publishers and listeners in SoapRMI Java and C++
  • Extensibility
    • Events are objects and can be extended
    • XML Schema can be used to describe event format and we have support for mapping schema types into objects
xevents
xEvents
  • Publisher
    • Open socket, write string, close socket (or keep open for keep-alive)
  • Listener
    • Accept socket, parse simple XML, do stuff
  • Event channel with filtering (and persistence)
  • Easily pluggable: Event Listener or Event Channel can work standalone or in Servlet
eventlistener
EventListener
  • Very simple interface (throws RemoteException that maps to SOAP Fault):

interface EventListener {

void handleEvent(Event ev) throws soaprmi.RemoteException;

}

eventpublisher
EventPublisher
  • Each event publisher may have its own API
  • EventPublisher is just simple marker interface

interface EventPublisher {

}

leasingfilteredeventpublisher
LeasingFilteredEventPublisher
  • We provide one simple push subscribe with lease:

public interface LeasingFilteredEventPublisher extends EventPublisher {

EventLease subscribeLease(EventListener myReference,

long leaseDuration,

Event filter,

String handback

) throws RemoteException;

}

eventchannel
EventChannel
  • Event channel decouples event publishers and listeners and it is both publisher and listener:

public interface EventChannel extends EventListener, EventPublisher {

}

public interface LeasingFilteredEventChannel extends LeasingEventChannel, LeasingFilteredEventPublisher {

}

implementation
Implementation
  • Most of xEvents are interfaces and few basic classes such as Event
  • However in package soaprmi.events.util is simple implementation of
    • Listener (standalone and as servlet)
    • Publisher (produces machine utilization events)
    • Leasing Filtered Event Channel
performance
Performance
  • On the order of 100 round-trip events/sec
    • though raw XML parsing speed of 5000/sec
  • Can be fine tuned for particular event types or implemented in C++
  • To receive events parsing can be done by hand if received event types are limited or using simple and small XML Pull Parser both for C++ and Java
  • Persistent event channel uses JDBC and MySQL for backend
xml parsing
XML Parsing
  • Available strategies
    • DOM: put parsed tree in memory and allow to manipulate it
    • SAX: pick what you need from input stream
    • XPP (XML Pull Parser): parse incoming stream
    • Mixed: progressive DOM – combining SAX with creating DOM on demand
  • Layers: XPP -> SAX -> DOM
  • For events: low CPU and memory overhead
xml pull parser
XML Pull Parser
  • Designed to unmarshal complex data structures from XML (such as SOAP)
  • Can be easily embedded in any application
  • Minimized memory allocation during parsing and parser can be reused
  • Tokenizing done in fast automata
  • It is small (20KB for Java JAR file)
  • Used both in SoapRMI Java and C++
example resource monitoring
Example: Resource Monitoring
  • Publisher: Sensor collects data packs it into string and writes into socket connected to EC
    • Small specialized C program without XML parser!
  • Event Channel (EC) receives event parses it and delivers to subscribers
    • SoapRMI Java service: standalone or embedded in servlet
  • Listener: subscribes to EC to receive resource utilization events
    • Java application or servlet that uses SoapRMI
setting up publisher listener

SOAP Event Publisher

Event

Source

Format

SOAP Envelope

From Template

SOAP Events

Listener

Event

Instance

SOAP RPC CALL

handleEvent

HTTP over TCP

Setting up Publisher - Listener
  • Simple things should be easy to do, such as:
  • connecting producers to consumers:
    • run listener 2001
    • run publisher http://localhost:2001
  • Listener and Publisher can be in SoapRMI (C++ or Java) or event hand written.
slide62

Producers

Consumers

Listeners

Publishers/Sensors

Channels

Java

Simple Publisher

TC: Java

Test Channel

SL: Java

Simple Listener

test.*

C++

Resdat Sensor

FC: Java

Filtering Channel

CSL: C++

Simple Listener

test.*

Java

Resdat Sensor1

resdat.*

SL1: Java

Simple Listener1

RC: Java

Resdat Channel

SL2: Java

Simple Listener2

Java

Resdat Sensor2

example setup listeners
Setup event channels

run channel –rebind FC

run unfiltered –rebind TC –subscribe FC

run unfiltered –rebind RC [NOTE: no –subscribe FC]

Setup event listeners

run listener –rebind SL –subscribe TC

run listener –rebind SL1 –subscribe RC

run listener –rebind SL2 [NOTE: no –subscribe RC]

Setup dynamic connections:

run connect –publisher FC –listener RC

run connect –publisher RC –listener SL2

Example Setup: Listeners
example setup publishers
Example Setup:Publishers
  • Publishers – start pumping events
    • run publisher TC
    • run sensor FC
    • run sensor RC
  • C++ publisher
    • ./sensor FC
  • C++ listener
    • ./listener –bind FC
fault tolerance
Fault Tolerance
  • taking down any part of interconnected event network will not bring down system
  • event listeners (including event channels) are location independent – they rebind their reference on startup and during subscription renewal listeners get updated remote references
  • migration of listeners to another host is supported – publishers will lookup in directory new reference and use it
  • Example: kill FC or SL1 and move to another host
how it works
How it works?

public static void main(String[] args) {

parseArgs(args);

EventListener listener =

Util.getListenerRef(optListener, optCtxUrl);

Util.startSubscription(optPublisher,

listener, optCtxUrl, 4 * 60 * 1000, …);

}

how it works 2
How it works (2)?

public static LeasingEventChannel getLECRef(String loc,

String ctx)

throws soaprmi.RemoteException

{

Remote ref = soaprmi.util.Util.lookupRemote(

loc, null, ctx, LeasingEventChannel.class);

if(! (ref instanceof LeasingEventChannel)) {

throw new RemoteException("remote object is not"

+" LeasingEventChannel at location "+loc

+" (context "+ctx+")");

}

return (LeasingEventChannel) ref;

}

how it works 3
How it works (3)?

public static Remote lookupRemote(String loc, Remote ref,

String ctx, Class iface ) throws RemoteException

{

if(loc.startsWith("http://") || loc.startsWith("https://")) {

return soaprmi.soaprpc.SoapServices.getDefault()

.createStartpoint(loc, new Class[]{iface});

}

// do RMI registry lookup

String rmi = makeRMIRef(loc, ctx);

if(rmi != null) {

return Naming.lookup(rmi);

}

...

DirContext dirCtx = getInitialDirContext(ctx);

return (Remote) dirCtx.lookup(loc);

}

further work
Further Work
  • Exploring C++ and Java performance of xEvents
  • Interfacing with other SOAP implementation (currently works with Apache SOAP and preliminary with .NET)
  • Using TLS/SSL transport and leveraging SOAP Digital Signature Extension (SOAP DSig Spec)
  • Integrating xEvents into CCAT and Active Notebook (already working with previous incarnation)
  • Adding Persistent Event Channel