FIRMS: Federation & Implementation of Reliable Messaging Specifications for Web Services

1. FIRMS: Federation & Implementation of Reliable Messaging Specifications for Web Services Geoffrey Fox (managerial contact) Shrideep Pallickara (technical contact) October 21 2004 Southampton

2. Staffing and Contract Update • Contract still waiting negotiation of terms and conditions • e.g. is the State of Indiana part of British Empire and subject to its laws • Have interviewed two good software engineers • Shrideep gave them each a one day exam

3. Broker Broker Broker Broker Service NB as Appl. Logic Handler NB as Service Handler Appl. Logic FIRMS and FINS • Are built on NaradaBrokering Software with a different leaner deployment • Can use original deployment if need additional features Original Subscriber Publisher Subscriber Subscriber FIRMS

4. Current NaradaBrokering Features

5. Forthcoming Features • Production implementations of WS-Eventing, WS-Notification, WS-RM and WS-Reliability. • Time Differential Services: Preserve time spacing between events, that are time-stamped using high-resolution timers. • Active replay support: Pause and Replay live streams. • Replicated storage support for fault tolerance and resiliency to storage failures.

6. WS-Reliable Messaging • Specification from IBM, and Microsoft • Leverages the WS-Addressing and WS-Policy specifications. • Provides support for both positive and negative acknowledgements. • Provides operations for explicit creation and termination of sequences. • Delivery assurance mode supported: at-least-once, at-most-once, exactly-once, and ordered delivery

7. WS-Reliability • Specification from Fujitsu, Oracle, and Sun • Provides support for positive acknowledgements. • Provides support for a variety of message-exchange patterns. • Request/Response, One-way, Polling • Delivery assurance mode supported • Unreliable, at-least-once, ordered-and-exactly-once • Under consideration by the OASIS to be a standard

8. M(n) M(n+1) Service B Service A Mechanisms for Reliable Messaging I • There are essentially sequence numbers on each message • Unreliable transmission detected by non-arrival of a message with a particular sequence number • This is “just some TCP reliability” built at application level (Service level Internet) • One can either use ACK’s – Receiver (service B) positively acknowledges messages when received • Service A fully responsible for reliability • Or NAK’s – Service B is partially responsible and tracks message numbers – sends a NAK if sequence number missing

9. Mechanisms for Reliable Messaging II • Each message has a retransmission time; messages are retransmitted if ACK’s not received in time • Uses some increasing time delay if retransmit fails • Note need to be informed (eventually) that OK to throw away messages at sender; pure NAK insufficient • Note this is reliability from final end-point to beginning end-point: TCP reliability is for each link and has different grain size and less flexible reliability mechanisms • There are several efficiency issues • Divide messages into groups and sequence within groups • Do not ACK each message but rather sequences of messages • NAK based system attractive if high latency (some mobile devices) on messaging from receiver back to sender

10. Filter 2 NaradaBroker Filter 1 Custom Message Reliability 2 second PDA reply latency! Different endpoints may well need different reliability schemes. Another reason to use application layer. Need to define easy touse “standard reliabilityprofiles Wireless Optimized WS-RM WS-RM WS-Reliability

11. Handlers and Filters in-memory Processing Built-in Handlers ProcessWS-RM

12. WS-……..Handler WS-RMHandler Deployment Issues for “System Services” • “System Services” are ones that act before the real application logic of a service • They gobble up part of the SOAP header identified by the namespace they care about and possibly part or all of the SOAP body • e.g. the XML elements in header from the WS-RM namespace • They return a modified SOAP header and body to next handler in chain Header Body e.g. ……. Could be WS-Eventing WS-Transfer ….

13. Legacy Service WS-RMService WS-RM removed from SOAP Header WS-RM enabledSOAP Proxy Distributed WS-RM Processing • A handler is like an in memory “service” so one can build handlers that can alternatively be deployed “outside” application service and look like a WS-RM service. • Comments on handlers as services paradigm? • Will capture this as a deployment memo • Handlers could be just part of application logic – more natural for WS-Eventing than WS-RM

14. WSRM Implementation: Overview

15. Stable Storage • This is where messages are stored until receiver indicates that message has been successfully processed • In memory, Flat Files, MySQL supported • In memory (default?) or Flat File is sufficient for WS-RM messages that do not require sophisticated search • Can store messages at one or more distributed NaradaBrokering sites • Could keep messages for a long time! • All “new” communication will be done using SOAP and WSDL defined ports

16. Complicated NaradaBrokering • NaradaBrokering has more capabilities than OMII needs • We will make them optional to reduce code bloat • NaradaBrokering could implement Handler chains for protocols and WSDL bindings not supported by AXIS • UDP plus WS-RM (or equivalent) has been shown to outperform traditional TCP over high bandwidth high latency links • Also supports parallel TCP and GridFTP

22. WSRM/WSR Similarities • Messages are part of a sequence/group of messages. • Addresses issues pertaining to ordering and duplicate detection. • Quality of service constraints are applied to groups of messages. • Recommends message-level security: specifically WS-Security for secure delivery of messages. • Provides framework for reporting faults/errors in processing between endpoints involved in reliable delivery • Provide support for acknowledging multiple range of messages received within a group/sequence.

23. WSRM/WSR Differences • WSR has no support for negative acknowledgements. Retransmissions are always initiated by the source of messages. WSRM supports negative acknowledgements. • WSRM has an explicit operation for the creation of sequence and associated sequence identifier. WSR has no such operation, a new group begins when a receiver has encountered a new groupID. • disadvantage: difficult to resolve collisions in group id namespace • WSRM uses WS-Policy for specifying and exchanging featured info. WSR does not advocate any specification though it alludes to an abstract concept of agreements. • Order is always tied to duplication elimination in WSR. WSRM allows order and duplication detection to exist independent of each other • WSR incorporates a HTTP binding for its specification. WSRM currently does not have one; though one can simply use SOAP’s HTTP binding. • WSRM has explicit termination of sequences. WSR groups cannot be terminated. They are first closed and then removed.

24. Federation, Why? • WSRM being supported by powerful group: IBM/Microsoft. • WSR being actively considered for recommendation as a standard by OASIS • It is quite possible that these specification may continue to co-exist for a while. • Federation would allow end-points belonging to different specifications to communicate with each other.

25. Federation, How? • Mapping of physical (XML) elements and semantics associated with these specifications. • Mapping of sequence numbering. WSRM starts numbering at 1, WSR starts at 0. • NAKs in WSRM messages will simply be ignored, since WSR does not support it. • Mapping of policy elements and rules associated with where/when and the combination in which multiple policy/agreement elements may occur. • Enforcing constraints on delivery. WSR provides a subset of the delivery modes available in WSRM • Mapping of faults/error reporting • Creation/Termination of sequence in WSRM have no equivalents in WSR. • So terminate-sequence in WSRM will trigger multiple requests/retransmission to ensure WSR has received everything. Group expiry time then needs to be updated at the WSR side.

26. FIRMS Schedule

27. FINS: Federation & Implementation of Notification Specifications for Web Services Geoffrey Fox (managerial contact) Shrideep Pallickara (technical contact)

28. WS-Eventing • Delivery Modes: Push • Pull and Trap (UDP Notifications) through WS-Management • Subscription Manager can be a separate Web service or part of handler bundled with Source Web Service when it gives as in OGSI special ports for each Source service

29. WS-Notification • WS-BaseNotification, WS-BrokeredNotification, WS-Topics, WS-ResourceProperties and WS-ResourceLifetime

30. WS-Eventing/Notification Issues • IBM (Sun ..) joined Microsoft (BEA. Tibco) in new August 2004 specification • WS-Eventing will presumably replace WS-BaseNotification as core of WS-BrokeredNotification • We made WS-Eventing as our highest priority • Eventually support WS-Eventing, WS-Notification and JMS (Java Message Service) in a federated model • WS-Metadata needed to query WS-Eventing sources • Not in current OMII plans but clear how to add • Note FINS will use FIRMS in all messages if desired

31. FINS Schedule

35. WS-Eventing and WS-Notification