1 / 17

Augmenting Film and Video Footage with Sensor Data

Augmenting Film and Video Footage with Sensor Data. N. Su, H. Park, E. Bostrom, J. Burke, M. Srivastava, D. Estrin PerCom ’04: March 14-17, 2004. Augmented Recording System. Wireless sensor network application for filmmaking and media production Seamless integration Mobility

saddam
Download Presentation

Augmenting Film and Video Footage with Sensor Data

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Augmenting Film and Video Footage with Sensor Data N. Su, H. Park, E. Bostrom, J. Burke, M. Srivastava, D. Estrin PerCom ’04: March 14-17, 2004

  2. Augmented Recording System • Wireless sensor network application for filmmaking and media production • Seamless integration • Mobility • Increased expressiveness • Synchronize sensor data with video footage • Sensor data allows post processing of video Herman Li

  3. Why not use image processing? • Image processing techniques can infer position, motion, light condition… • For example: Van Helsing Infrared LED Marker http://www.fxguide.com Herman Li

  4. Why not use image processing? http://www.fxguide.com Herman Li

  5. Why not use image processing? • Do not offer fine grain data • Cannot infer conditions outside of view, or quantities such as wind speed or temperature Herman Li

  6. Architecture • Sensor node neighbourhood • Serial port server • Timecode generator • Sylph server middleware • Jini client • SQL database Herman Li

  7. Sylph server lookup • Sylph translates query, forward to serial port server • Serial port server dispatches messages to base stations • Sensors begin collecting data • Base station forwards data to serial port server • Serial port server interpolates data • Sylph server announces new data • Jini client stores data in database Herman Li

  8. Sensors node neighbourhood • Uses CrossBow Mica 1 / Mica 2 motes • Thermistor, light sensor, microphone, accelerometer. • Radio range of few hundred feet, ~10kbps http://computer.howstuffworks.com/mote.htm • Runs on PALOS (Power Aware Light-weight Operating System) Herman Li

  9. Sensors node neighbourhood • Clustering • Each base station responsible for a few motes • Base station • Potentiometer calibration • Admit closest X sensors • Neighbours • Internal frame counter from 2 to infinity • Sends data every 13 frames • Filters redundant values Herman Li

  10. Serial port server • Controls base stations via serial connections • Takes SMPTE timecode and synchronizes with sensor data • Combines all sensor data for a frame and sends as one packet • Interpolates missing data Herman Li

  11. Serial port server • Time synchronization • At most 7s per second drift • One frame error per 4766.7 frames • Time sync every 2.5 min • Latency test • shows delay of at most 3 frames Herman Li

  12. Sylph server middleware • UCLA project, used in Smart Kindergarten • Allows queries on sensors • Defines JINI attributes such as light, period, command • JINI query: “READ LIGHT EVERY 30 SECONDS” • Translated: “SET PERIOD 30 SECONDS”, “SET COMMAND=STARTSENDING” Herman Li

  13. JINI client • Client retrieves sensor data per frame and stores in MS Access DB • Provides playback features and data collection controls Herman Li

  14. Evaluations • Deployed 2 base stations, 4 sensors each • Clustering algorithm took 2.59 min • Exp1: Graduate light intensity changes Herman Li

  15. Evaluations • Exp2: Delay measurement • Some delay of ~10 frames • Maximum of 20 frames Herman Li

  16. Future work • Dynamic control • Real time on-the-fly adjustment to studio equipments • Semantic indexing of video streams • Express interest and query high level events • Continuity Management • Allows checks for continuity in different footage Herman Li

  17. References • Augmenting Film and Video Footage with Sensor Data, Norman Makoto Su, Heemin Park, Eric Bostrom, Jeff Burke, Mani B. Srivastava, Deborah Estrin • ARS: http://www.ee.ucla.edu/~hmpark/ars • PALOS: http://deerhound.ats.ucla.edu:7777/pls/portal/docs/PAGE/CENS_REPOSITORIES/TECHNOLOGIES/PALOS-TUTORIAL.PDF • http://mmsl.cs.ucla.edu/sylph • http://www.fxguide.com • http://computer.howstuffworks.com/mote.htm Herman Li

More Related