1 / 31

The NaradaBrokering System

The NaradaBrokering System. Shrideep Pallickara and Geoffrey Fox Community Grid Computing Labs Indiana University. What to expect?. As promised this seminar is An attempt to spruce things up a bit As it only seems fit Lest I be accused of being remiss Seems to me that its only rational

meda
Download Presentation

The NaradaBrokering System

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The NaradaBrokering System Shrideep Pallickara and Geoffrey Fox Community Grid Computing Labs Indiana University

  2. What to expect? As promised this seminar is An attempt to spruce things up a bit As it only seems fit Lest I be accused of being remiss Seems to me that its only rational That the Q&A session is bi-directional So desist from launching another browser As you may be called upon to answer Dispossessed, I am not, of clairvoyance For aware, I am, that repay you will in kind For putting you in a bind And generally being an annoyance But then again that’s how seminars should be So without any further delays …. Shall we.

  3. Talk outline • The Story so far • What’s currently cooking • What needs more thought • And a preview of things to come • Closing remarks

  4. NaradaBrokering • Based on a network of cooperating broker nodes • Cluster based architecture allows system to scale to arbitrary size • Based on the publish/subscribe model • Also supports another model, peer-to-peer (P2P) via JXTA • Incorporates algorithms for • Topic matching and calculation of destinations • Efficient routing to computed destinations • Supports local broker accesses • Clients do not need to reconnect to the remote broker that it was last connected to.

  5. Controlled Uncontrolled settings

  6. Event in NaradaBrokering

  7. NaradaBrokering Results • Experimental Set-up • Topology of 22 broker. • Each broker node on 1 Sun SPARC Ultra-5 machine. • 100 client Node process, Sun SPARC Ultra-60 • JDK 1.2 run time environment • Parameters being Varied • Message Size • Publish Rates • Matching Rates

  8. NaradaBrokering Results

  9. JMS Compliance • Features: • JMS clients written while conforming to spec • JMS clients are vendor agnostic • JMS providers do not interoperate with each other • Rationale • Providing support for JMS clients within the system. • Mature specification with several existing applications • Opens NaradaBrokering to these applications • Bring NaradaBrokering functionality to JMS systems • Distributed solution, load balancing, scaling and failure resiliency.

  10. Providing JMS support • JMS Interactions • Supported locally or mapped to corresponding NaradaBrokering calls • JMS Interconnection Bridge • Operations supported locally or mapped to corresponding NaradaBrokering interactions • One bridge instance per connection • Maintains list of registered sessions • Responsible for routing events to appropriate sessions • Support for • Creation of different message types e.g. ObjectMessage, BytesMessage etc. • Operations that can be invoked on these message types.

  11. Providing JMS Support • Topic • NaradaBrokering topics are created as <tag,value> pairs. • JMS Topics are generally “/” separated. • JMS selector mechanism • We augment NaradaBrokering’s topic matching with the selector mechanism implemented by openJMS. • JMS subscriptions • Mapped to corresponding NaradaBrokering subscription requests. • The Narada JMS Event. • Encapsulates the entire JMS message as a payload for the event. • Matching is done based on the topic name contained in the message.

  12. Towards a distributed JMS solution • Features in NaradaBrokering best exploited in distributed settings. • Clients of distributed solution to inherit NaradaBrokering features • Routing efficiencies, load balancing, scaling etc. • Eliminate the Single point of failure model • Highly Available System • Existing systems should easily be replaced with distributed system

  13. Broker Locators: Distributed JMS Solution • Primary Function • Discovery of brokers that clients can connect to • Obviates need for client to keep track of broker states within the broker network. • Keeps track of • Broker additions and removals • Changes to network fabric • Published Limit on concurrent connections at a broker node • Set during broker initializations • Active connections at a broker node. • Individual brokers notify changes to broker locator.

  14. Broker Locators: Features & Constraints • Features • Dynamic Real time Load Balancing • Connection requests forked to best available broker. • Incorporation of new brokers into solution • A newly added broker is among the best brokers to handle a connection request. • Slower clients could all be hosted on specific brokers • Eliminates broker choking resulting from servicing very slow clients. • Constraints • Availability (multiple per domain) • Should not constitute a bottleneck or single point of failure. • Minimal logic • Should not affect processing pertaining to any node in the system.

  15. Broker Locator: Decision Metrics • IP-address of requesting client • Published limit on concurrent connections • Number of active connections still possible • Availability of broker • A simple ping test • If broker is not available, remove broker from list of available brokers. • Computing capabilities at a broker • CPU speed, RAM etc.

  16. Broker Locator: Sequence of Operations • Locate valid broker • Propagate broker information to client • Hostname/IP-address information • Port number on which it listens for connections • Transport protocol over which it communicates • Client then uses info to establish communication channel with broker • Done transparently. • Clients with multiple connections • A client could sometimes have connections to multiple brokers.

  17. JMS Performance Data • SonicMQ (version 3.0) and NaradaBrokering broker • Dual CPU (Pentium 3, 1 GHz, 256 MB RAM) machine. • 100 subscribers • Over 10 different JMSTopicConnections • All hosted on a Dual CPU (Pentium 3, 866 MHz, 256 MB RAM) machine. • Publisher and Measuring Subscriber • Hosted on another dual CPU (Pentium 3, 866 MHz, 256 MB RAM) machine. • Operating System and Run time Environment • Linux (version 2.2.16) • Java 2 JRE (Java 1.3.1, Blackdown-FCS, mixed-mode)

  18. Performance Data:Factors Measured • Transit Delay • No need for clock synchronization and accounting for clock drifts. • Standard Deviation in the transit delay for the sample of messages received. • System Throughput • Measured in terms of rate at which messages are received. • Factors measured under varying • publish rates • message payload sizes.

  19. Performance: Transit Delay & Standard Deviation Lower Payloads & Low Publish Rates

  20. Performance: Transit Delay & Standard Deviation Higher Payloads & Low Publish Rates

  21. Performance: Transit Delay & Standard Deviation Lower Payloads & High Publish Rates

  22. Performance: System Throughput Lower Payloads & Higher Publish Rates

  23. Why P2P • Deployments user driven • No dedicated management • Management of resources • Expose resources & specify security strategy • Replicate resources based on demand • Dynamic peer groups, fluid memberships • Sophisticated search mechanisms • Peers respond based on their interpretations • Responses do not conform to traditional templates.

  24. What are the downsides? • Interactions are attenuated • Localized • Fragmented world of multiple P2P subsystems • Routing not very sophisticated • Inefficient network utilization (Tragedy of Commons) • Simple forwarding • Peer Traces (to eliminate echoing) • Attenuations (to suppress propagation)

  25. JXTA • Set of protocols to support P2P interactions. • Planned bridges to (and from) other P2P systems.

  26. NaradaBrokering & JXTA Interaction Model • Based on proxy model • Acts as both Rendezvous peer (JXTA routers) and NaradaBrokering client. • No changes to JXTA core or straitjacketing of interactions • Change made to Rendezvous layer • Clients are not aware that they interact with a Narada-JXTA proxy or Rendezvous peer.

  27. What we plan to accomplish • Better routing and network utilization • Bridge isolated P2P subsystems • Efficiently locate/replicate shared resources • Since interactions are routed through the broker network, brokers can build efficient resource maps. • Incorporate GXOS support in NaradaBrokering • Then use JXTA search to locate resources stored in the distributed XML database. • Provide the notion of reliable interactions (for peers attached to Narada-JXTA proxies) • Grid/WebServices generated dynamically when complex tasks are initiated could be managed by peer groups.

  28. Narada-JXTA Proxy • Glean relevant information from JXTA interactions. • Peer group advertisements • Requests/Responses to be part of peer group. • Messages sent to a peer group. • Subscribe to relevant topics to ensure delivery • Construct corresponding Narada-JXTA event from interactions.

  29. Applications • Integrated NaradaBrokering-JXTA environment tested under JXTA shell and myJxta (InstantP2P) • Plan to integrate myJxta into Anabas with NaradaBrokering managing P2P and middle-tier (JMS) style interactions. • JXTA-Jabber interface via Narada/Anabas. • (Jabber, PDA, Anabas) linkage to be combined with (Jxta, NaradaBrokering, Anabas linkage).

  30. A preview of things to come • Adaptive protocol support • Switch transport protocols dynamically • TCP, UDP, HTTP/SHTTP, Multicast, RTP & SOAP. • Managing lightweight GXOS-based XML databases using NaradaBrokering/JXTA Search. • JXTA/Jabber/PDA linkage • Security infrastructure • Kerberos/PKI • Web Services ….

  31. Closing remarks • Intersection of P2P, WebServices, middleware and distributed systems • Lot of interesting research. • Lot of issues to be tackled/addressed • What you need to do • Start using it • Report bugs • suggest useful additions

More Related