Geoffrey Fox, Shrideep Pallickara and Xi Rao

1. A Scaleable Event Infrastructure for Peer-to-Peer Gridshttp://www.naradabrokering.org/ACM Java Grande, Seattle 3-5 Nov 2002. Geoffrey Fox, Shrideep Pallickara and Xi Rao gcf, spallick, xirao@indiana.eduCommunity Grid Computing Laboratory,Pervasive Technology LabsIndiana University. http://www.naradabrokering.orggcf,spallick,xirao@indiana.edu

2. Talk Outline • NaradaBrokering overview • Results (Native, JMS, RTP) • Rationale for supporting P2P interactions • NaradaBrokering & JXTA • Results • Conclusions and Future work. http://www.naradabrokering.org gcf,spallick,xirao@indiana.edu

3. NaradaBrokering • Based on a network of cooperating broker nodes • Cluster based architecture allows system to scale to arbitrary size • Originally to provide uniform software multicast to support real-time collaboration linked to publish-subscribe for asynchronous systems. • Now has four major core functions • Message transport (based on performance) in multi-link fashion • General publish-subscribe including JMS & JXTA • Support for RTP-based audio/video conferencing. • Federation of multiple instances (just starting) of Grid services http://www.naradabrokering.org gcf,spallick,xirao@indiana.edu

4. Role of Event/Message Brokers • We will use events and messages interchangeably • An event is a time stamped message • Our systems are built from clients, servers and “event brokers” • These are logical functions – a given computer can have one or more of these functions • In P2P networks, computers typically multifunction; in Grids one tends to have separate function computers • Event Brokers “just” provide message/event services; servers provide traditional distributed object services as Web services • There are functionalities that only depend on event itself and perhaps the data format; they do not depend on details of application and can be shared among several applications • NaradaBrokering is designed to provide these functionalities • MPI provided such functionalities for all parallel computing http://www.naradabrokering.org gcf,spallick,xirao@indiana.edu

5. Engineering Issues Addressedby Event / Messaging Service • Application level Quality of Service – e.g. give audio highest priority • Tunnel through firewalls & proxies • Filter messages to slow (collaborative/real-time) clients • Hardware multicast is implemented • Scaling of software multicast • Efficient calculation of destinations and routes. • Elegant implementation of Collaboration. • Integrate synchronous and asynchronous collaboration • Supports local broker accesses • Transparently replace single server JMS systems with a distributed solution. http://www.naradabrokering.org gcf,spallick,xirao@indiana.edu

6. Features of Event Service • MPI nowadays aims at a microsecond latency • The Event Web Service aims at a millisecond (computer) latency • Typical distributed system travel times are many milliseconds (to seconds for Geosynchronous satellites) • Different performance/functionality trade-off • Messages are not sent directly from P to S but rather from P to Broker B and from Broker B to subscriber S. • Synchronous systems: B acts as a real-time router/filterer • Messages can be archived and software multicast • Asynchronous systems: B acts as a database & workflow engine • Subscription is in each case, roughly equivalent to a database query • Company X sets up a firewall • The event service sets up brokers either side of firewall to optimize transport through the firewall. http://www.naradabrokering.org gcf,spallick,xirao@indiana.edu

7. Data base Narada Broker Network (P2P) Community For message/events service Broker Broker (P2P) Community Resource Broker Broker Broker (P2P) Community Software multicast Broker (P2P) Community http://www.naradabrokering.org gcf,spallick,xirao@indiana.edu

8. NaradaBrokering Communication • Applications interface to NaradaBrokering through UserChannels. • UserChannels have publish/subscribe semantics • Links implement a single conventional “data” protocol. • Addition of new transport protocols within the Framework is easy to achieve. • Administrative channel negotiates the best available communication protocol • Link implementations can incorporate their own handshaking protocols to facilitate communications and data exchange. • Different links can have different underlying transport implementations • Implementations in the current release include support for TCP,UDP, Multicast, SSL and RTP. HTTP, HTTPS support will be available in next release. • Supports communication through proxies such as iPlanet, Netscape and Apache. • Supports communication through firewalls such as Microsoft ISA, Checkpoint. • NaradaBrokering brokers and links can be instantiated dynamically • Support communication between two application end points across firewall & proxy boundaries. http://www.naradabrokering.org gcf,spallick,xirao@indiana.edu

9. Application (user level) NaradaBrokering TraditionalTransport Protocols PhysicalLevel QoS, Priority Specification Performance Information Chosen Implementation Network Protocol Architecture e.g. Audio Specified as Low Bandwith Low Latency High Reliability e.g. QoS Request Satisfied as UDP on Long haul with Optimized TCP/IPthrough firewall Process priority within application So Audio would be set to high Priority so no interference with Video and shared application in for example collaboration session http://www.naradabrokering.org gcf,spallick,xirao@indiana.edu

10. NaradaBrokering Results http://www.naradabrokering.org gcf,spallick,xirao@indiana.edu

11. Sender/receiver/broker - (Pentium-3, 1 GHz, 256 MB RAM). 100 Mbps LAN. JDK-1.3, Red Hat Linux 7.3 http://www.naradabrokering.org gcf,spallick,xirao@indiana.edu

12. JMS Performance: Transit Delay & Std Deviation (NaradaBrokering & SonicMQ) Lower Payloads & High Publish Rates • NaradaBrokering is JMS compliant • Applications ported include • Anabas – JMS based distance education system. • OKC – Online Knowledge Center project at IU http://www.naradabrokering.org gcf,spallick,xirao@indiana.edu

13. H.263 video file Average bit-rate of 600Kbps. Frame-rate of 30 frames/sec • Jitter J = J + (|D(i-1, i)| - J)/16 http://www.naradabrokering.org gcf,spallick,xirao@indiana.edu

14. Why P2P? • Core features • Resource Sharing & Discovery • CPU cycles: SETI@home, Folding@HOME • File Sharing: Napster, Gnutella • Deployments user driven • No dedicated management • Management of resources • Expose resources & specify security strategy • Replicate resources based on demand • Dynamic peer groups, fluid group memberships • Sophisticated search mechanisms • Peers respond to queries based on their interpretations • Responses do not conform to traditional templates. http://www.naradabrokering.org gcf,spallick,xirao@indiana.edu

15. What are the downsides? • Interactions are attenuated • Localized • Fragmented world of multiple P2P subsystems • Routing not very sophisticated • Inefficient network utilization (Tragedy of Commons) • Simple forwarding • Peer Traces (to eliminate echoing) • Attenuations (to suppress propagation) • TTL’s associated with interactions. http://www.naradabrokering.org gcf,spallick,xirao@indiana.edu

16. JXTA • Set of protocols to support P2P interactions. • Planned bridges to (and from) other P2P systems. • Support for JXTA will allow us to leverage these P2P systems. http://www.naradabrokering.org gcf,spallick,xirao@indiana.edu

17. What we plan to accomplish • Better routing and network utilization • Bridge isolated P2P subsystems • Efficiently locate/replicate shared resources • Since interactions are routed through the broker network, brokers can build efficient resource maps. • Use JXTA search to locate resources stored in distributed XML database. • NaradaBrokering provides efficient routings for queries/responses. • Provide the notion of reliable interactions (between peers attached to Narada-JXTA proxies) • Grid/WebServices generated dynamically when complex tasks are initiated could be managed by peer groups. http://www.naradabrokering.org gcf,spallick,xirao@indiana.edu

18. NaradaBrokering & JXTA Interaction Model • Based on proxy model • Acts as both Rendezvous peer (JXTA routers) and NaradaBrokering client. • No changes to JXTA core or straitjacketing of interactions • Change made to Rendezvous layer • Peers are not aware that they interact with a Narada-JXTA proxy or Rendezvous peer. • Narada-JXTA provides JXTA guaranteed long distance delivery http://www.naradabrokering.org gcf,spallick,xirao@indiana.edu

19. Narada-JXTA Proxy • Glean relevant information from JXTA interactions. • Peer group advertisements (XML Doc describing resource) • Requests/Responses to be part of peer group. • Messages sent to a peer group. • Subscribe to relevant topics to ensure delivery • Construct corresponding Narada-JXTA event from interactions. • These events lend themselves to efficient routing. Peer group advertisement http://www.naradabrokering.org gcf,spallick,xirao@indiana.edu

20. Narada-JXTA events Requests/Responses to be part of a certain peer group http://www.naradabrokering.org gcf,spallick,xirao@indiana.edu

21. Narada-JXTA events - II Messages sent to a specific peer-group/peer http://www.naradabrokering.org gcf,spallick,xirao@indiana.edu

22. Apps & Setups • Applications • Integrated NaradaBrokering-JXTA environment tested under JXTA shell and myJxta (InstantP2P) • Plan to integrate myJxta into Anabas with NaradaBrokering managing P2P and middle-tier (JMS) style interactions. • Experimental Setup • Sender/receiver - (Pentium-3, 1 GHz, 256 MB RAM). • Every node (broker/router) hosted on a different machine (Pentium-3, 1 GHz, 256 MB RAM). • Machines reside on a 100 Mbps LAN • Run-time environment for all the processes is JDK-1.3 build Blackdown-1.3.1, Red Hat Linux 7.3 http://www.naradabrokering.org gcf,spallick,xirao@indiana.edu

23. http://www.naradabrokering.org gcf,spallick,xirao@indiana.edu

24. Performance measurements are used by Links in Reconfiguring Connectivity between nodes Deciding underlying transport protocol Determining possible filtering Performance “heuristic” to define links and their protocols Factors measured include Transit delays, bandwidth, Jitter, Receiving rates. Performance measurements are Spaced out at increasing intervals for healthy channels. Factors selectively measured for unhealthy channels. No repeated measurements of bandwidth for example. Injected into Narada network as XML events Narada Performance Web Service Administrative Interface http://www.naradabrokering.org gcf,spallick,xirao@indiana.edu

25. NBHub Federation of Services • If you have a service – Notification (as with Grid or JXTA advertisements), Search, Scheduling, Audio-Video conferencing …. • With a standard which client and server components • Then can build a “server” that services all clients • Alternatively can hierarchically consider collection of existing Server-client domains • IBM or Globus OGSA islands • Sun Grid Engine Schedulers • Polycom/Access Grid A/V sessions • Enterprise Search Engines • Federation links islands together • JXTA Search has XML specified federation approach – will try to include and generalize as a NaradaBrokering federation framework • DoD High Level Architecture HLA does this for simulation http://www.naradabrokering.org gcf,spallick,xirao@indiana.edu