CS 414 – Multimedia Systems Design Lecture 27 – Media Server (Part 3)

1. CS 414 – Multimedia Systems DesignLecture 27 – Media Server (Part 3) Klara Nahrstedt Spring 2011 CS 414 - Spring 2011

2. Administrative MP3 – posted today CS 414 - Spring 2011

3. Some Interesting Facts • DBMS2.com Source (May 2009) • Facebook had 400 terabytes of disks managed by Hadoop/Hive with an approx. 6:1 compression ratio • Facebook’s Hadoop/Hive system ingests 15 terabytes of new data per day • Facebook had 610 Hadoop nodes (in May 2009) running in a single cluster and was heading for 1000 • Yahoo had 2000 nodes (in May 2009) and was heading for 4000 CS 414 - Spring 2011

4. Some Interesting Facts • Source: www.slideshare.net (March 2011) • Current data sets: • NYSE: 8PB; Google > 12PB; • Data Volumes: • NYSE: 1.5 TB daily; • Facebook: 350 M users; 3.5B shared items/week • Facebook adds > 100K users, 55M ‘status’ updates, 80M photos daily CS 414 - Spring 2011

5. Outline • Disk Scheduling • SCAN-EDF • Group Sweeping • Mixed Scheduling • Admission Control • File System Metadata/Indexing • Block Size Issues CS 414 - Spring 2011

6. Disk Scheduling Policies • Goal of Scheduling in Traditional Disk Management • Reduce cost of seek time • Achieve high throughput • Provide fair disk access • Goal of Scheduling in Multimedia Disk Management • Meet deadline of all time-critical tasks • Keep necessary buffer requirements low • Serve many streams concurrently • Find balance between time constraints and efficiency CS 414 - Spring 2011

7. EDF (Earliest Deadline First) Disk Scheduling • Each disk block request is tagged with deadline • Policy: • Schedule disk block request with earliest deadline • Excessive seek time – high overhead • Pure EDF must be adapted or combined with file system strategies CS 414 - Spring 2011

8. EDF Example Note: Consider that block number Implicitly encapsulates the disk track number CS 414 - Spring 2011

9. SCAN-EDF Scheduling Algorithm • Combination of SCAN and EDF algorithms • Each disk block request tagged with augmented deadline • Add to each deadline perturbation • Policy: • SCAN-EDF chooses the earliest deadline • If requests with same deadline, then choose request according to scan direction CS 414 - Spring 2011

10. Implementation of SCAN-EDF • Notation: • Dibe deadline of disk block request ‘i’ • Nibe track (block) position on disk • Nmax be maximum number of disk tracks • Deadline Modification: • Di + f(Ni) • f(Ni) converts track number of ‘i’ into a small perturbation of deadline • Perturbation small enough so that • Di + f(Ni) ≤ Dj + f(Nj) for Di ≤ Dj • Possible f(Ni) = Ni/Nmax CS 414 - Spring 2011

11. SCAN EDF Example (Nmax = 100) CS 414 - Spring 2011

12. Enhanced SCAN-EDF (1) Head Moves Upwards • Use more accurate perturbation of deadline • Consider • Actual track position of disk head ‘N’ • Nmax– max number of disk tracks • Ni– next track to be considered CS 414 - Spring 2011

13. Enhanced SCAN-EDF (2) • Algorithm: • If head moves upwards (towards Nmax), then • (a) • (b) CS 414 - Spring 2011

14. Enhanced SCAN-EDF (3) If head moves downwards (towards 1), then (a) (b) CS 414 - Spring 2011

15. Group Sweeping Algorithms • Policy: • Each Request consists of (Deadline, Block Number ) • Disk Block Requests served in cycles • In one cycle, requests divided into groups according to similar deadlines • Within group use SCAN • As we retrieve blocks, we may need smoothing buffers to ensure continuity CS 414 - Spring 2011

16. Group Sweeping Example CS 414 - Spring 2011

17. Mixed Scheduling (uses SSTF – Shortest Seek Time First) Example of SSTF CS 414 - Spring 2011

18. Mixed Scheduling SSTF (Shortest Seek Time First) + Balanced Strategy CS 414 - Spring 2011

19. Admission Control Client 1 retrieves K1 blocks in one round Client 2 retrieves K2 blocks Server Client 3 retrieves K3 blocks Client 4 retrieves K4 blocks CS 414 - Spring 2011

20. Admission Control • Disk block requests are timed • Media server must determine • admit a stream • serve (schedule) a stream without having negative effect on other streams already serviced. • Deterministic Guarantees • Admission control considers worst case scenario when admitting new stream • Constrained Disk Placement Example: M - size of blocks, G – size of gabs, rdt– data transfer of disk CS 414 - Spring 2011

21. Admission Control Cost to switch and move Ki blocks Minimal Intra- Ki blocks delay α – overhead switching from one round (‘j-1’) to another round (j), and then transmitting the first block of the ‘j’ round β – transmission time of (Ki-1) blocks in ‘j’ round, i=1,..4 Ki – number of blocks retrieved by client ‘i’ ηi – Block granularity retrieved for client ‘i’ (e.g., in Bytes) Ri – playback rates of client ‘i’ (e.g., in Bytes per second) CS 414 - Spring 2011

22. Admission Control • Statistical Guarantees • Deadlines are guaranteed with certain probability • Admission control considers statistical behavior of the disk system while admitting new stream (average performance) • Best effort Service • No guarantees CS 414 - Spring 2011

23. Conclusion The data placement, scheduling, are very important for any media server design and implementation. Still need to consider multimedia file system and caching – next lecture CS 414 - Spring 2011