1 / 19

MP 1: Audio/ Video Recorder and Player

MP 1: Audio/ Video Recorder and Player. CS414: Multimedia System Instructor: Klara Nahrstedt February 1 st , 2012. Learning Goals. Capturing a video and audio using a webcam Storing the audio and video data in PC using compression

edana
Download Presentation

MP 1: Audio/ Video Recorder and Player

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. MP 1: Audio/ Video Recorder and Player CS414: Multimedia System Instructor: KlaraNahrstedt February 1st, 2012

  2. Learning Goals • Capturing a video and audio using a webcam • Storing the audio and video data in PC using compression • Playing a video from the stored file with fast forward, rewind, pause and play functionalities • Playing an audio from the stored file • Comparing the media frames across different compression

  3. Covered Aspects of Multimedia Audio/Video Presentation Playback Image/Video Capture Audio/Video Perception/ Playback Image/Video Information Representation Transmission Transmission Compression Processing Audio Capture Media Server Storage Audio Information Representation A/V Playback

  4. System Architecture Image/Video Capture Audio/Video Perception/ Playback Image/Video Information Representation Compression Processing Audio Capture Audio Information Representation

  5. System Modules and Data Flow: Recording Frame rate, resolution • You need to create separate threads • Muxer is optional. You may need it to run the video file using standard player type Thread Thread Capture Filter Encoder Store Webcam Player Thread … … … … Encode Muxer Store … … Encode

  6. System Modules and Data Flow: Playback • Display the video on your GUI and place the audio data to the sound card • Demuxer is optional. You may need it run standard video files from Internet Media File type Capture Decode Player Rate control … … Decode … … … … Capture Demuxer Decode

  7. System Modules and Data Flow: Monitoring in Recording • Store the monitoring output to a file • [Optional] You may draw chart showing [time vs. compression ratio (r)], [time vs. compression time (t)] and [time vs. frame size (s)] [bonus point] • X-axis: Time in 5 second interval • Y-axis: Average values (of r, s or t) in last 5 seconds Thread Monitoring Component Monitoring Point Thread Thread Capture Filter Encode Store Webcam Player Thread

  8. System Modules and Data Flow: Monitoring in Playback • Store the monitoring output to a file • [Optional] You may draw plot showing [time vs. decompression time (d)] [bonus point] [Hint: Try JFreeChart library] • X-axis: Time in 5 second interval • Y-axis: Average values (of d) in last 5 seconds Monitoring Point Thread Media File Monitoring Component Thread Thread Capture Decode Player

  9. gstreamer Architecture • Goal: create a multimedia application using gstreamer • gstreamer uses 3rd party plugins for processing (over 150) Multimedia Application gstreamer Core Framework Encoder Decoder Muxer Demuxer Sink source 3rd Party plugins

  10. How does it work (1) ? • You need to create element for each system modules • Elements are equipped with Pads: source and sink pads Source Sink gstreamer-java elements • The number of source and sink pad varies depending on the element type • You can add/ create additional sink or source pad

  11. Creating gstreamer elements Sink Src Src Src Sink Sink • Java code example to create an element • Elements element_name = ElementFactory.make (“plugin name”, “your defined name”) • Java code example to set property of the element • element_name.set (“property”, “value”); Sink source Src Capture filter Filter encoder Encode muxer Sink Muxer sink Store

  12. How does it work (2) ? Src Sink • You need to link the elements • Linking define whom is receiving and sending data to whom • This is similar to linking the system modules Capture Filter Src Media Source Media Filter

  13. gstreamer-java pipeline Sink Src Src Src Src Sink Sink Sink • Java example code for creating, and linking a pipeline • Pipeline pipe= new Pipeline(“test”); • pipe.addMany(source, filter,encoder, muxer,sink); • Element.linkMany(source, filter,encoder, muxer,sink); • pipe.play(); Webcam Capture Filter Encode Store Media Encoder Media Muxer Media Filter Media Source Media Sink

  14. How to create multiple pipelining Src Sink Sink Src Src Src Sink Sink • Several pipelines from the same source or towards same sink • Example: recording and playback at the same time Sink Thread Thread Capture Filter Encoder Store Webcam Player Thread Media Encoder Media Muxer Media Sink Media Filter Media Source Media Sink Src

  15. How to create multiple pipelining Src Sink Sink Src Src Src Src Src Src Sink Sink Sink Sink Sink • You can create additional pads • You can use tee elements • Use queue elements after tee Sink muxer Media Sink encoder queue1 tee filter source Src queue2 Sink

  16. How to redirect frames to application Src Src Src Src Sink Sink Sink • How to deliver a frame to your application • Use appsink element Emit Signals Media Encoder AppSink Media Filter Media Source • Java example code for getting frames from AppSink • AppSinkappsink = (AppSink) ElementFactory.make(“appsink”, null); • appsink.set(“emit-signal”, true); • appsink.setSync(false);

  17. Some gstreamer plugins • Video webcam source: v4l2src • Audio webcam source:alsasrc • Video or Audio file source: filesrc • Video Encoder:ffenc_mpeg4 (mpeg4), jpegenc (mjpeg), … • Video Decoder:ffdec_mpeg4 (mpeg4), jpegdec(mjpeg), … • Audio Encoder: vorbisenc, alawenc, mulawenc • Audio Decoder: vorbisdec, alawdec, mulawdec • Muxer [optional]:avimux (avi), matroskamux(mkv), … • Demuxer [optional]: avidemux (avi), • Audio sink: alsasink

  18. Evaluations Required Points: 100, Optional Points: 20 Points are will be considered based on live demo and interview performance

  19. Group Directory • A shortcut link in desktop and home directory • cs414_Gn.work [note: n= your group number] • You can directly access • /team/cs414/Gn[note: n= your group number] • You are allowed to use any languages (C/C++/java) • You are allowed to you’re your own laptop any OS

More Related