1 / 121

网络专题选讲

网络专题选讲. 华中科技大学 电子与信息工程系 程文青 chengwq@mail.hust.edu.cn 2013 年 1 月. 社会网络应用专题选讲. 华中科技大学 电子与信息工程系 互联网技术与工程研究中心 黑晓军 Email: heixj@hust.edu.cn Web: http://itec.hust.edu.cn/~heixj 2013.1. Outline. Introduction Case study

penny
Download Presentation

网络专题选讲

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. 网络专题选讲 华中科技大学 电子与信息工程系 程文青 chengwq@mail.hust.edu.cn 2013年1月

  2. 社会网络应用专题选讲 华中科技大学 电子与信息工程系 互联网技术与工程研究中心 黑晓军 Email: heixj@hust.edu.cn Web: http://itec.hust.edu.cn/~heixj 2013.1

  3. Outline Introduction Case study Traffic transportNetTube: Exploring Social Networks for Peer-to-Peer Short Video Sharing, 2009 IncentiveP2P Trading in Social Networks: The Value of Staying Connected, 2010 RecommendationCircle-based Recommendation in Online Social Networks, 2012 《网络专题选讲 》

  4. Internet Topology

  5. Introduction 5 我们生活在一个关系的社会

  6. 社会网络应用 6

  7. Friend network in Facebook 7

  8. Co-authorship network 8

  9. Co-authorship in network science 9

  10. Ingredient networks 10

  11. 911事件——犯罪网络 11

  12. Social Networking(General public)

  13. Social Networking(General public)

  14. Social Networking(Academia)

  15. 专著

  16. 专著

  17. 研究现状 17 • 主要研究机构 • 国外:MIT、Stanford、Maryland、USC、HP、Michigan • 国内:IBM中国研究院、微软亚洲研究院、中科院、中国传媒大学、清华大学、南京大学 • 近年来社会网络成为国内外研究热点 • 美国国家科学基金会(NSF)将社会计算研究领域提供专项资金(2010) • 美国计算机协会ACM,Workshop on Social Network Mining and Analysis(2007~2012) • WWW会议成立“Social Network and Web2.0 Track”论坛(2009) • SIGCOMM,ACM SIGCOMM Workshop on Online Social Networks(2009~2012) • EuroSys, Workshop on Social Network System(2009~2012) • 互联网测量会议(IMC),海量数据仓库国际会议(VLDB),信息与知识管理(CIKM)大量关于社交网络文章 • 全国网络科学论坛(2004~2012),全国复杂网络会议(2005~2011)

  18. NetTube:Exploring Social Networks for Peer-to-Peer Short Video Sharing Xu Cheng and Jiangchuan Liu School of Computing ScienceSimon Fraser University British Columbia, Canada October 2009 IEEE INFOCOM, 2009

  19. Background (1) • Social Networked Media Sharing – new killer Internet application • Since 2005 • Rich user-generated content (UGC) sharing • Social networks • Among users • Among videos • Changing the popular culture

  20. Background (2) • YouTube – a representative • Popular • Market share of around 43% • More than six billion videos viewed in January 2009 • Consumed as much bandwidth as the entire Internet in 2000 • 3rd visits among all Internet sites (after Google and Yahoo) • Fast growing • 20% growth rate per month • 15 hours of new videos are uploaded every minute

  21. Motivation (1) Can we make it better ? • The YouTube Crisis – all other sites’ challenge • Severely hindered by client/server architecture • Bandwidth costs • Consumed as much bandwidth as the entire Internet in 2000 • $1 million a day for server bandwidth! • Sold to Google for $1.65 billion in Nov. 2006 • Performance and scalability • “Slow” among the surveyed sites by Alexa.com

  22. Motivation (2) • Peer-to-peer (P2P) – alternative to Client/Server • New generation of communication paradigm • Each peer contributes its bandwidth to serve others • Scale well with larger user base • More users, more resources contributed • Success already seen in • BitTorrent, eMule, eDonkey (file sharing) • Video broadcasting • …

  23. P2P架构 没有永远在线的服务器 任意主机可以同另一个主机进行通信 节点可以间歇性的连入系统,IP地址可能会变化 peer-peer 23

  24. 对等网络流媒体系统 两大设计空间 如何形成重叠网络? 如何传输内容? 现有体系结构 树状拓扑 + 推式内容传输 ESM, Yoid, CoopNet, SplitStream, Bullet, Chunkspread … 网状拓扑 + 拉式内容传输 -24- 《网络专题选讲》

  25. 网状-拉式对等网络流媒体系统 这类系统非常类似于BitTorrent -25- 《网络专题选讲》

  26. 节点软件结构 双缓存 积极下载 vs 保守下载 处理丢失的数据块 缓存控制 -26- 《网络专题选讲》

  27. 缓存映像(buffer map) 缓存映像反映了节点缓存所拥有的数据块信息 此映像可以被用来评估用户的播放质量 -27- 《网络专题选讲》

  28. 对等网络中的问题 28 内容组织和搜索 内容传输 信誉、激励及安全相关问题

  29. CoolStreaming • The first practical large-scale P2P NetTV • Origin of data-driven mesh design • With many follow-ups: PPLive, PPStream, UUSee … X. Zhang, J. Liu, B. Li, and T.-S. P. Yum, CoolStreaming/DONet: A Data-driven Overlay Network for Live Media Streaming, IEEE INFOCOM'05, March 2005. >800 citations J. Liu, S. G. Rao, B. Li, and H. Zhang, Opportunities and Challenges of Peer-to-Peer Internet Video Broadcast, Proceedings of the IEEE, Vol. 96, No. 1, pp. 11-24, January 2008.

  30. Data-Driven Mesh • Core operations • Every node periodically exchanges data availability information with a set of partners • Then retrieves unavailable data from one or more partners, or supplies available data to partners • Easy to implement • no need to construct and maintain a complex global structure • Efficient • data forwarding is dynamically determined according to data availability • Robust and resilient • adaptive and quick switching among multi-suppliers

  31. Challenges and Opportunities (1) • Challenges – Drastically different statistics • 1.5 year measurement of 5 million videos • http://netsg.cs.sfu.ca/youtubedata/ • Short video clips – stability • 99.6% are less than 700 seconds • “I don’t want to wait for 30 seconds for a two-minute video!” • Searching for sources • High churn rate: join/leave system • Huge number of videos – scalability • Highly skewed • Inefficient for unpopular videos • Very few users watch the same one

  32. Challenges and Opportunities (2) • Opportunities – Social networks • No longer independent – videos have related videos • Small-world – strong clustering • Important role

  33. NetTube Design (1) • Bi-layer overlay network • Lower-layer – per-video • Download and uploading • Peers stay in previous overlays as sources • Larger and more stable • Upper-layer – social network • Connected by the same peers in different lower-layer overlays • Conceptual relation for searching • Social network brings similar peers closer • Clustering • Efficient searching

  34. NetTube Design (2) • Bloom filter based indexing • An efficient approach to keep track of peers’ cached videos • Bloom filter • An m-bit array using k hash functions • Space-efficient • Scalable indexing table • Fast searching • Table size is scalable with the number of videos • Search locally and search in the upper-layer overlay • Social network clustering the similar video

  35. NetTube Design (3) Transmission scheduling: From which partner to fetch which data segment ? • Constraints • Data availability • Playback deadline • Heterogeneous partner bandwidth • Rarest-first (BitTorrent’s) doesn’t work !

  36. NetTube Design (3) • Variation of Parallel machine scheduling • NP-hard • Conventional Heuristics • Message exchanged • Window-based buffer map (BM): Data availability • Segment request (piggyback by BM) • Less suppliers first • Multi-supplier: Highest bandwidth within deadline first

  37. NetTube Design (3) • Short video ? • CODAS: Collaborative Delay-Aware Scheduling

  38. NetTube Design (4) • Social network assisted pre-fetching • Most peers finish downloading before playback ends - free time available (about 80 seconds on average) • Using free time to reduce startup delay • Prefix pre-fetching • Avoid wasting bandwidth and space • Enable multiple pre-fetching • Multiple pre-fetching • Accuracy increases greatly • Accuracy increases as watch more videos • Pre-fetching among neighbors • Easy to implement • Social network helps improve efficiency

  39. Performance Evaluation (1) • Simulation • Configuration • Based on about 7,000 crawled videos • Scale to more than 10,000 heterogeneous clients • Compare with PA-VoD (MSN Video) • Bandwidth reduction • Save significantly more • More scalable

  40. Performance Evaluation (2) • Simulation • Impact of social network • Find more sources: more than 95% within 2 hops • Greatly increase pre-fetching accuracy

  41. Performance Evaluation (3) • PlanetLab experiment • Configuration • Maximum 235 PlanetLab nodes • Experiment results • Server bandwidth reduction: more than 40% • Startup delay: average 2.2 s • Playback continuity

  42. Summary • Contribution • First social network assisted P2P system for short video sharing • IWQoS’08, INFOCOM’09, IEEE Transactions on Multimedia • Techniques • Bi-layer overlay network • Bloom filter based indexing • Social network assisted pre-fetching • Collaborative delay-aware scheduling • Evaluation results • Greatly reduce server bandwidth • Much lower maintenance cost: $1 million → $60 K • Inherently scalable – P2P • Greatly reduce playback delay • Satisfying startup delay • Continuous playback

  43. P2P Trading in Social Networks: The Value of Staying Connected Zhengye Liu, Hao Hu, Yong Liu, Keith Ross, Yao Wang, and Markus Mobius Polytechnic Institute of NYU Dept. of Economics, Harvard Unviversity IEEE INFOCOM, 2010 43

  44. Outline 44 Background: P2P Incentive Networked Asynchronous Bilateral Trading (NABT) NABT Efficiency Theory NABT Simulations Conclusions

  45. P2P Apps: BitTorrent 45

  46. P2P Apps: Skype 46

  47. Peer-Assisted Video Streaming 1 6 2 5 3 4 47 • Large scale deployments on Internet • thousands of live/on-demand channels • millions of world-wide users daily • Leading P2P Video Companies • CoolStreaming • PPStream • PPLive • Sopcast • UUSee

  48. Major P2P Issues 48 • Traffic localization • P4P • Security • Attacks on • Attacks from • Lack of uniform API • Incentives for peers to contribute resources

  49. Partially Successful P2P Incentive + P2P design  Resources Incentives 49 • BitTorrent is popular • 50+ client implementations • Dozen public trackers • 5-10 million users • Why BitTorrent? • First generation P2P applications: Gnutella • 70% of users are free-riders • Second generation P2P applications: BitTorrent

  50. The BitTorrent Incentive To get files faster… contribute more bandwidth 50 Implementation of incentive: • The rich play/trade with the rich

More Related