1 / 36

     History of Internet Audio Experiments at McGill (and elsewhere) Jeremy R. Cooperstock

     History of Internet Audio Experiments at McGill (and elsewhere) Jeremy R. Cooperstock Centre for Intelligent Machines Centre for Interdisciplinary Research in Music Media and Technology McGill University. Software available from: http://ultravideo.mcgill.edu Research supported by:.

kstackhouse
Download Presentation

     History of Internet Audio Experiments at McGill (and elsewhere) Jeremy R. Cooperstock

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1.      History of Internet Audio Experiments at McGill (and elsewhere) Jeremy R. Cooperstock Centre for Intelligent Machines Centre for Interdisciplinary Research in Music Media and Technology McGill University

  2. Software available from:http://ultravideo.mcgill.edu Research supported by:

  3. Some Early (Pre-)History Eve Schooler: Distributed Music: A Foray into Net Performance (Sept. 1993) • synchronized three real-time streams from different hosts; delays in the order of 200 ms • difficult for performers to be listeners Paul Hoffert: CyberSoiree (Feb. 1996) • ATM-based technology for audio and video streaming of a four-way jazz performance • delay >0.5s delay but musicians earned to compensate through extensive practice Dimitri Konstantas: Distributed Musical Rehearsal Studio (May 1996) • ATM based distributed rehearsal with conductor at different location from musicians • 80 ms one-way delay for audio-video synch; echo resulted in "extreme confusion" Seiji Ozawa: Opening Ceremony Nagano Winter Olympics (1998) • conduct choruses on 5 continents: singers in Sydney, New York, Beijing, Berlin, False Bay • time lag adjustor used to eliminate satellite delay

  4. World's first Internet AC-3 streamMontreal-New York for the AES 107th Convention, Sept. 26, 1999

  5. AC-3 (640 Kbps) (5.1 channels,16 bits @ 48kHz) coded in AES/EBU SGI Indy 1.5 Mbps Dolby encoder SGI Indy Network 1.5 Mbps PC Server Cisco IP/TV PC Client System Diagram McGill UniversityMontreal, Canada NYU Cantor HallNew York, USA

  6. In action Sender Receiver Receive,check, playback Transmit Read Send Audio Queue Receive Audio Queue

  7. Recording Studio that Spanned a ContinentMontreal-Los Angeles for the AES 109th Convention, Sept. 23, 2000 12 channels, 96kHz, 24 bits/channel USC Norris Theatre, LA McGill Redpath Hall, Montreal

  8. Microphone Configuration

  9. 3 Mbps

  10. 27.6 Mbps

  11. Some Modern History Internet2 Initiative: World's First Remote Barbershop Quartet (Nov. 2000) • multi-location quartet; each of the 4 singers in different cities, conductor in 5th • network delay variances prevented singers from hearing each other or conductor Internet2 Initiative: Music Video Recording via Internet2 (Nov. 2000) • multi-location music video recording session using real-time streaming video • musicians simultaneously connected via timing tracks to a mixing board Chris Chafe: QoS Enabled Audio Teleportation (Nov. 2000) • CD quality sound (750 kbps) of 2 separated musicians in Dallas streamed to Stanford • musicians played "together" in same space (Stanford) but delay was severe John Wawrzynek: Network Musical Performance (May 2001) • gestural coding (e.g. MIDI) used to manage data for distributed musical performance • musicians at Berkeley and CalTech, playing on MIDI keyboards; local feedback only

  12. "Frères Jacques" experimentMontreal, RISQ 2000 Conference "low-latency" MPEG-2 videoconferencing

  13. "Playing Together" experimentNew York-Ottawa, Remote Masters class, Dec. 8, 2000 used Litton MPEG-2 codec @ 10 Mbps Pinchas Zuckerman, Columbia U, NYC Wu Ji, Canarie ARDNOC, Ottawa

  14. Quality vs. Latency • near-equivalent quality of encoded video at greatly reduced bandwidth • but... compression takes time

  15. musicalactivity smallensemble large ensemble lip synch compress acquire transmit MPEG* Tolerated Synchronicity 0 10 20 80ms 30 40 60 70 50 speed of light: Mtl-LA net latency Mtl-LA networks * This figure is a theoretical “best-case” scenario. Actual performance may vary based on local hardware. The manufacturer makes no warranties, either express or implied, and will not be held liable for damage resulting from use of the product. Certain restrictions may apply. Offer void where prohibited by law. Batteries not included.

  16. or put another way... Latency:The Interaction-Killer

  17. Interactive Success! Chris Chafe: SoundWIRE (August 2001) • streaming pro audio from remote locations for musical collaboration • UDP/IP streaming: good results between Armonk, NY and Stanford • round trip delay 125ms, musicians able to "catch-up" during pauses

  18. The future according to Sympatico?

  19. yeah, right! (tu n'es pas sérieux!) For those with DSL at home...

  20. So what does it take? • much bigger network pipes ("abusive bandwidth") • need Fast Ether (100Mbps) for audio • GigE (1000Mbps) for video • more efficient network topologies – hops add delay

  21. So what does it take? • much bigger network pipes ("abusive bandwidth") • need Fast Ether (100Mbps) for audio • GigE (1000Mbps) for video • more efficient network topologies – hops add delay • more powerful hardware • CPU speed (fast process switching, especially for mixed media) • pro-audio interfaces with small buffers

  22. So what does it take? • much bigger network pipes ("abusive bandwidth") • need Fast Ether (100Mbps) for audio • GigE (1000Mbps) for video • more efficient network topologies – hops add delay • more powerful hardware • CPU speed (fast process switching, especially for mixed media) • pro-audio interfaces with small buffers • more masochistic researchers

  23. Sometimes things work...

  24. Sometimes things work... note: look of extreme anguish and sometimes they don't

  25. Distributed Violin DuetMcGill-Université de Montréal for RISQnet Conference, Nov. 21, 2001 • connected two Montreal universities via IP over 1 Gbps fiber link • uncompressed audio and video • < 30ms latency

  26. "The medium is the message" Marshall McLuhan Cooperstock's Corollary: "The medium must be faster than the message."

  27. Size does matterMontreal-Ottawa (NRC), Distance Teaching Trial, Feb. 20, 2002 • connected McGill to National Research Council (Ottawa) with SDI video and multichannel 96kHz/24bit audio • display on 50" plasma screen (near life-size)

  28. Distributed Jazz JamMontreal-Stanford (CCRMA), Research Demo, Oct. 24, 2002 • uncompressed audio and video @ < 50ms latency

  29. Remote Video Interpretation (Sign Language)Montreal-Fredericton (UNB), research trials, 2003 • DV (patient, physician) exchanged with remote sign-language interpreter physician's setup patient's setup

  30. Vibrosensory TransmissionVRQ demo, March 8, 2004

  31. Coming next: DSD TransmissionAES demo, October 31, 2004

  32. Milestones Ahead • multicast • traffic shaping • region-of-interest client selection • multichannel echo-supression/cancellation

  33. Self-Signal Isolation: Echo-suppression • the big problem in videoconferencing • same issue for video and vibrosensory data

  34. achieving high-fidelity distributed interaction, at levels of presence and latency that support the most demanding applications and doing so in spite of sensor and bandwidth limitations

  35. http://ultravideo.mcgill.edu

More Related