vod bittorrent framework n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
VoD BitTorrent Framework PowerPoint Presentation
Download Presentation
VoD BitTorrent Framework

Loading in 2 Seconds...

play fullscreen
1 / 32

VoD BitTorrent Framework - PowerPoint PPT Presentation


  • 122 Views
  • Uploaded on

VoD BitTorrent Framework. Background. BT is a very popular peer to peer protocol with many implementations: http://en.wikipedia.org/wiki/Comparison_of_BitTorrent_clients The first one was created in Python by Bram Cohen. Real world motivation. BitTorrent is common and Free

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'VoD BitTorrent Framework' - hamlet


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
background
Background
  • BT is a very popular peer to peer protocol with many implementations:
    • http://en.wikipedia.org/wiki/Comparison_of_BitTorrent_clients
  • The first one was created in Python by Bram Cohen.
real world motivation
Real world motivation
  • BitTorrent is common and Free
  • Good chance there is a swarm for your movie
  • VoD servers are expensive and complicated
  • Its better to use P2P for VoD
some definitions
Some definitions
  • Delay (D) = How much time I am allowed to hold before starting to play M (Movie).
  • Prefetch Time (P) = How much time before I watch byte B from M, it should arrive.
  • Playback Rate (R) = How many bytes/second I watch
  • B(t) – the index of the bit that must arrive by t.
  • B(t) = R *(P+t-D)
rules and examples
Rules and Examples
  • B(t) = R * (P+t-D)
  • If R=5 KB/sec and D=6 seconds and P=2 seconds, than:
    • At t=6 seconds, I must have 10KB continuous from the start of the movie.
    • At t=18 seconds I must have 70KB continuous from the start of the movie.
what does one vod client do
What does one VOD client do
  • Your client tries to get from the Bittorent clients pieces of data needed to play the video.
  • Each piece needs to arrive before its deadline.
  • If the piece does not arrive by the deadline you can download it “instantly” from the “server”.
  • Each VOD client counts the data from server (DFS).
what is my goal
What is my goal
  • Each byte taken from the server costs money, therefore you need to minimize DFS.
  • Goal - Minimize the accumulated DFS of all the VOD clients that run concurrently.
how do i create my client
How do I create my client
  • Download GROUP_VOD_CODE.tar.gz from “BitTornado - Start with this Code” link in the course web page.
    • It is also in /users/courses/hanochworkshop/group_vod
  • GROUP_VOD_CODE.tar.gz is a regular BT with StreamWatcher.py to monitor DFS. Any changed section is marked  ###### GROUP VOD ######
  • Modify it to minimize acumulated DFS.
  • Do not change StreamWatched.py.
standard bt parameters
Standard BT parameters
  • --saveas F
  • --max_upload_rate
  • F.torrent
vod client additional params
VOD client additional params
  • SET = defined inside run_all.sh, not a parameter.
  • delay – Wait before prefetch, in seconds. SET TO 20
  • prefetchT – How much time in advance the byte must arrive. In percent of movie length. SET TO 5%
  • rate – VOD rate in KB/sec. SET TO 90 KB.
  • order – How many run when client spawned. SET BY run_all.sh.
  • Parameter of run_multiple.sh.
  • gap – How much to wait between spawns – unit is pewfetchT.
  • alg – Name of the algorithm. Default is ORIG.
  • verbose – Print and kill the whole swarm when complete.
  • peers_num – how many regular peers.
  • seeds_num – how many seeds.
  • Vod_num – how many VoD peers.
how do i test my idea
How do I test my idea
  • cd /users/courses/hanochworkshop/voddiversegroup/vod_tests
  • run_multiple.sh
    • your_client - example: ~/GROUP_VOD_CODE/btdownloadheadless.py
    • Gap – example: 0
    • output_location example: ~/bt_out
    • iteration example: 4
    • empty/full example: empty
    • peers_num example: 16
    • seeds_num example: 2
    • vod_num example: 14
execution example

Tracker

BT

BT

BT

BT

BT

BT

BT

BT

BT

BT

VOD

BT

Execution example

run_multiple.sh

run-all.sh

VOD

BT

VOD

BT

VOD

BT

how does it work
How does it work?
  • Run tracker
  • Run the “world”
  • Run your clients
how does it work1
How does it work?
  • Both tracker and the “world” run on the same computer.
  • Running a test:
    • Reserve time on pc-hanoch-w3/w4 in:
      • http://bt-p2p-vod.wikispaces.com/
      • Or from course www - "Scheduling computers for tests.“
output to screen
Output to screen

==================================================================================

total time: 1 min 07 sec

share rating: 0.904 (4816.0 KB up / 5328.0 KB down)

saving: picture-100.bmp (27.5 MB)

percent done: 18.9

time left: 4 min 52 sec

download to: /home/pc-hanoch-w4/home/hillelav/name/picture.pc-hanoch-w4-4.bmp

download rate: 93.4 kB/s

upload rate: 80.6 kB/s

seed status: 2 seen now, plus 0.340 distributed copies

peer status: 28 seen now, 10.7% done at 1220.3 kB/s

--------------------------------StreamWatcher-------------------------------------

Csv stats: ~/out/1/statistics-order-4-gap-0.csv

DFS is: 8372224 bytes

DFS/Total is: 29 %

FullPieces: 63 / 220

DirtyPieces: 1 / 220

FaildHashChecks: 0

Prefetching 5 %

Playing point: 56 / 220 ( 25 %)

Only last VOD client prints

output to csv
Output to .csv
  • In ~/out you will find vod_num directories: /1, /2, …/iteration.
  • In each directory, you have vod_num files.
  • Statistics in - order-<1..vod_num>-gap-<GAP>.csv
  • Order is how many VOD clients ran before.
  • <GAP> is the parameter Gap.
csv file
.csv file

alg,dfs,p2p

ORIG,5,0

ORIG,10,2

ORIG,15,6

ORIG,19,11

ORIG,23,16

ORIG,28,20

ORIG,31,25

ORIG,34,31

ORIG,37,36

ORIG,39,41

.

.

.

total dfs and total p2p gap 2
total DFS and total P2P GAP = 2

File percentage

File percentage

run all sh defaults
run_all.sh defaults
  • Start
    • Use in run_multiple.sh
  • Stop
    • Kill all python threads
  • Kill
    • Kill all python threads and scripts.
to run client separately
To run client separately:
  • PATH_TO_YOUR_VOD_CLIENT/btdownloadheadless.py

--saveas F

--delay D

--prefetchT P

--rate R

--out_dir O Can add the parameters

--order X in the IDE

--gap G

--group_size S

--alg ‘ALG'

--verbose

F.Torrent

python
Python
  • Very common script language
  • IDE:
    • PyDev
    • WINGWARE Python IDE – not free, but very good.
  • References:
    • python.org :http://docs.python.org/reference/
    • http://wiki.python.org/moin/BeginnersGuide/NonProgrammers
    • http://wiki.python.org/moin/BeginnersGuide/Programmers
  • The book "Think like a Computer Scientist in Python" has a very good reputation:
getting started
Getting started
  • Run the environment
    • ssh pc-hanoch-w3 or pc-hanoch-w4
    • cd /users/courses/hanochworkshop/group_vod/vod_tests
    • run_multiple.sh MyClient GAP output-directory
    • Note tracker prints a message for each GET.
  • You can connect to the tracker, from the CS network, with a browser at:
    • http://pc-hanoch-wk:6970/
  • Do ls -l /home/pc-hanoch-wk/home/hillelav/$USER to see downloaded files
directory structure
Directory Structure
  • Root –
    • high level application
  • Root/BitTornado –
    • services
  • Root/BitTornado/BT1 –
    • Actual work
client operation
Client Operation
  • btdownloadheadless.py – First to run.
    • Performs initializations.
      • get_response – parse the .torrent
      • Infohash – the ID of the .torrent
      • startEngine – Start connection with peers
      • startRerequester - Start connection with tracker
    • Run rawserver.listen_forever from RawServer.py.
    • listen_forever is:
      • Polling port
      • executing func() from a task queue.
receiving messages
Receiving messages
  • From listen_forever loop it goes to:
    • BT1/Connecter.py/got_message
      • CHOKE
      • UNCHOKE
      • INTERESTED
      • NOT_INTERESTED
      • BITFIELD
      • REQUEST
      • CANCEL
      • PIECE
      • HAVE
influencing decisions
Influencing decisions
  • Can be done in one of the callbacks which spring from Connecter.py / got_message:
  • Downloader.py
    • HAVE->got_have->send_interested
      • Tune for VoD?
bt tracker
BT tracker
  • Send GET request in Rerequester.py / announce
  • Receive GET data in thread in Rerequester.py / _rerequest
policy
Policy
  • Choker.py/_round_robin – decide who to choke / optimistically un choke / send_have to.
  • Downloader.py / _request_more.
  • PiecePicker.py / next - rarest piece, etc. Tune for VoD?
  • got_unchoke->_request_more->next…
more about the code
More about the code
  • Start your project from “BitTornado - Start with this Code” link.
  • Click “BitTornado - Legacy Code Guide” for basic explanations about the code.