1 / 16

Network-Aware Operator Placement for Stream-Processing Systems CS253 project presentation

Network-Aware Operator Placement for Stream-Processing Systems CS253 project presentation. Min Chen, Danhua Guo {michen, dguo}@cs.ucr.edu 12/4/2006. Roadmap. Motivation Background Knowledge Project Architecture Evaluation Conclusion. Motivation. Stream-based overlay network (SBON)

lexiss
Download Presentation

Network-Aware Operator Placement for Stream-Processing Systems CS253 project presentation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Network-Aware Operator Placement for Stream-Processing SystemsCS253 project presentation Min Chen, Danhua Guo {michen, dguo}@cs.ucr.edu 12/4/2006

  2. Roadmap • Motivation • Background Knowledge • Project Architecture • Evaluation • Conclusion {michen,dguo}@cs.ucr.edu

  3. Motivation • Stream-based overlay network (SBON) • Operator placement (RELAXATION) • Designed for large-scale DSPS • Network efficiency • Low delay • Cost space {michen,dguo}@cs.ucr.edu

  4. Motivation • Cost Space • Idea: Problem -> Virtual space -> physical node • Latency (& Load) dimensions {michen,dguo}@cs.ucr.edu

  5. Background Knowledge • Network Usage: A Blended Metric • Bandwidth-delay product • Individual applications perspective • Small delay • Network perspective • Scaling bandwidth by latency {michen,dguo}@cs.ucr.edu

  6. Architecture • Overview {michen,dguo}@cs.ucr.edu

  7. Operator Placement Algorithm • RELAXATION • Adapt to changing conditions • Decentralized • Considers impacts of shared placement decisions {michen,dguo}@cs.ucr.edu

  8. Operator Placement Algorithm • Algorithm • Virtual Operator Placement {michen,dguo}@cs.ucr.edu

  9. Operator Placement Algorithm • Algorithm • Physical Operator Mapping • Find k nodes whose coordinates are near the target cost space coordinate. • Sort the list by distance to the target coordinate. • Walk the sorted list, returning the first node already running the operator. • Failing that, return the nearest node that meets the application’s resource criteria {michen,dguo}@cs.ucr.edu

  10. Operator Placement Result • Δ=0.1 and Ft = 1 {michen,dguo}@cs.ucr.edu

  11. Evaluation • Based on PlanetLab trace files • Efficiency compared with other algorithm • Optimal: exhaustive search • Producer: place the operator at producer • Consumer: place the operator at consumer • Random: place the operator at a random node {michen,dguo}@cs.ucr.edu

  12. Evaluation • Set up • Bandwidth: 2KB/s • # of Nodes: 226 nodes from PlanetLab • # of Queries: 1000 • Query set up: 4 producers, 1 comsumer, 1 unpinned operator per query {michen,dguo}@cs.ucr.edu

  13. Evaluation • Result {michen,dguo}@cs.ucr.edu

  14. Evaluation • Delay penalty after placing 1000 queries {michen,dguo}@cs.ucr.edu

  15. Conclusion • Our experiment shows that SBON is efficient in terms of Network Usage (Bandwidth, Latency) {michen,dguo}@cs.ucr.edu

  16. Reference • Peter Pietzuch et. al. “Network-Aware Operator Placement for Stream-Processing Systems” • Peter Pietzuch, Jonathan Ledlie, and Margo Seltzer, “Supporting Network Coordinates on PlanetLab”, In Proceedings of WORLDS 2005, San Francisco, CA, December 2005 {michen,dguo}@cs.ucr.edu

More Related