Media extensions
Download
1 / 38

Media extensions - PowerPoint PPT Presentation


  • 96 Views
  • Uploaded on

Media extensions. Marcin Stankiewicz and Matthew Howard Development - Multimedia Platform Team Richard Fricks Windows Scenario Adoption Team Session 4-104.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Media extensions' - luana


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Media extensions

Media extensions

Marcin Stankiewicz and Matthew Howard

Development - Multimedia Platform TeamRichard FricksWindows Scenario Adoption TeamSession 4-104



Agenda
Agenda chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

  • Architecture overview

  • Media Foundation pipeline

  • Media extensions

  • Visual Studio 2012

  • Media source demo

  • Streaming media to JavaScript


Media application architecture
Media Application Architecture chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

Windows Store app

  • Media namespace

    • Capture

    • Transcode

    • PlayTo

    • PlaybackManager

    • MediaExtensionManager

JavaScript – HTML5

Video tag, audio tag

C#

MediaElement

Windows Runtime (WinRT)

Media engine

Playback and preview

Capture engine

Capture

Transcode

Sharing engine

PlayTo

Media Foundation


Media app architecture
Media app architecture chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Media Foundation pipeline

Media source

Decoder

Effect

Encoder

Sink

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect

Media engine

Playback and preview

Capture engine

Capture

Transcode

Sharing engine

Play To

Media Foundation


Media app architecture1
Media app architecture chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Media Foundation pipeline

Media source

Decoder

Effect

Encoder

Sink

  • Can be either native C++ DLL or a native C++/CX DLL

  • The DLL must ship in your app package

  • You can call WinRT APIs but you can use ONLY those desktop APIs that are marked for Windows Store apps:

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect

http://msdn.microsoft.com/en-us/library/windows/apps/hh452756.aspx


Registration process
Registration process chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

.RegisterAudioDecoder()

.RegisterVideoDecoder()

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Media Foundation pipeline

.RegisterAudioEncoder()

.RegisterVideoEncoder()

Windows.Media.MediaExtensionManager

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect

.RegisterSchemeHandler()

.RegisterByteStreamHandler()

?


Registration process effects
Registration process: Effects chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Media Foundation pipeline

Effects are registered using the class your code uses to process the media:

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect


Registration process sinks
Registration process: Sinks chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Media Foundation pipeline

Sinks are supported only by MediaCapture:

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect


Registration process1

Media source chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

Registration Process:

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Media Foundation pipeline

Audio

.RegisterAudioDecoder()

.RegisterVideoDecoder()

.RegisterAudioEncoder()

.RegisterVideoEncoder()

.RegisterSchemeHandler()

.RegisterByteStreamHandler()

.RegisterAudioDecoder()

.RegisterVideoDecoder()

.RegisterAudioEncoder()

.RegisterVideoEncoder()

.RegisterSchemeHandler()

.RegisterByteStreamHandler()

Audio encoder

Audio sink

Audio decoder

Audio effect

Windows.Media.MediaExtensionManager

?


Which to use

Which to use? chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

  • ByteStreamHandler

  • New container format (WebM, MKV)

  • SchemeHandler

  • Generating your own streams

  • Media stored in a location not accessible to inbox ByteStream

  • Custom streaming protocol


Imfbytestreamhandlerinterface
IMFByteStreamHandlerInterface chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.


Imfbytestreamhandlerinterface1
IMFByteStreamHandlerInterface chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

  • HRESULT BeginCreateObject( [in] IMFByteStream *pByteStream,

  • [in] LPCWSTR pwszURL,

  • [in] DWORD dwFlags,

  • [in] IPropertyStore *pProps,

  • [out] IUnknown **ppIUnknownCancelCookie,

  • [in] IMFAsyncCallback *pCallback,

  • [in] IUnknown *punkState );

MediaElement.Source= newUri(“video.mv4");


Imfbytestreamhandlerinterface2
IMFByteStreamHandlerInterface chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

  • HRESULT EndCreateObject( [in] IMFAsyncResult *pResult,

  • [out] MF_OBJECT_TYPE *pObjectType,

  • [out] IUnknown **ppObject );

typedef enum MF_OBJECT_TYPE { MF_OBJECT_MEDIASOURCE,

MF_OBJECT_BYTESTREAM,

MF_OBJECT_INVALID } MF_OBJECT_TYPE;


Schemehandlers and byte streamhandlers
SchemeHandlers and Byte-StreamHandlers chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

Byte-StreamHandler

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Media Foundation pipeline

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect


Imfschemehandlerinterface
IMFSchemeHandlerInterface chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.


Imfschemehandlerinterface1
IMFSchemeHandlerInterface chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

  • HRESULT BeginCreateObject( [in] LPCWSTR pwszURL,

    • [in] DWORD dwFlags,

    • [in] IPropertyStore *pProps,

    • [out] IUnknown **ppIUnknownCancelCookie,

    • [in] IMFAsyncCallback *pCallback,

    • [in] IUnknown *punkState );

MediaElement.Source= newUri("myscheme://square");


Imfschemehandlerinterface2
IMFSchemeHandlerInterface chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

  • HRESULT EndCreateObject( [in] IMFAsyncResult *pResult,

  • [out] MF_OBJECT_TYPE *pObjectType,

  • [out] IUnknown **ppObject );

typedef enum MF_OBJECT_TYPE { MF_OBJECT_MEDIASOURCE,

MF_OBJECT_BYTESTREAM,

MF_OBJECT_INVALID } MF_OBJECT_TYPE;


Schemehandlers and byte streamhandlers1
SchemeHandlers and Byte-StreamHandlers chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

Byte-StreamHandler

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Media Foundation pipeline

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect


Schemehandlers and byte streamhandlers2
SchemeHandlers and Byte-StreamHandlers chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

SchemeHandler

Byte-StreamHandler

ByteStream

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Media Foundation pipeline

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect


What is a bytestream

What is a ByteStream? chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

  • Represents a byte stream from some data source:

  • Local file

  • Network file

  • Some other source

  • This interface supports typical stream operations such as reading, writing, and seeking.


Imfbytestream interface
IMFByteStream Interface chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.


Schemehandlers and byte streamhandlers3
SchemeHandlers and Byte-StreamHandlers chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

SchemeHandler

Byte-StreamHandler

ByteStream

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Media Foundation pipeline

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect


Schemehandlers and byte streamhandlers4
SchemeHandlers and Byte-StreamHandlers chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

SchemeHandler

SourceResolver

Byte-StreamHandler

ByteStream

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Media Foundation pipeline

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect


Media foundation pipeline
Media Foundation pipeline chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

SchemeHandler

SourceResolver

ByteStream

Byte-StreamHandler

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect


Media foundation pipeline1
Media Foundation pipeline chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

SchemeHandler

SourceResolver

ByteStream

Byte-StreamHandler

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect


Media foundation pipeline2
Media Foundation pipeline chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

SchemeHandler

SourceResolver

ByteStream

Byte-StreamHandler

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect


Demo building a media extension

How to build an extension using C++/CX chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

The proper use of WRL to implement a SchemeHandler

Instantiating a media source from a SchemeHandler

Using the PropertySet class to implement two-way communication between the app and the extension

PropertySet change notification to support dynamic updates

Direct3D rendering

Demo: Building a media extension

  • Marcin Stankiewicz

  • Development - Multimedia Platform Team


Recap of demo
Recap of Demo chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

  • How to build an extension using C++/CX

  • The proper use of WRL to implement a SchemeHandler

  • Instantiating a media source from a SchemeHandler

  • Using the PropertySet class to implement two-way communication between the app and the extension

  • PropertySet change notification to support dynamic updates

  • Direct3D rendering


Media foundation pipeline3
Media Foundation pipeline chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

SchemeHandler

SourceResolver

ByteStream

Byte-StreamHandler

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect


Demo streaming media to an app

How to implement a sink media extension for audio chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

Capturing uncompressed audio

Processing audio buffers in a JavaScript app

Writing audio to a wave file

Asynchronous processing techniques

Demo: Streaming media to an app

  • Matthew Howard

  • Development - Multimedia Platform Team


Recap of demo1
Recap of Demo chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

  • How to implement a sink media extension for audio

  • Capturing uncompressed audio

  • Processing audio buffers in a JavaScript app

  • Writing audio to a wave file

  • Asynchronous processing techniques


Need more information

Need more information? chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

Come to our question-and-answer session!

3-106 Chalk Talk: Media apps

Thursday 4:15pm, B33 Hood


Media related sessions
Media-related sessions chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

3-117 10/31/2012 17:15 300 - Advanced B33 Hood Key technologies for building advanced media apps

3-122 11/1/2012 14:30 300 - Advanced B33 Hood Media app case studies

3-106 11/1/2012 16:15 300 - Advanced B33 Hood Chalk Talk: Media apps


Resources

Windows 8 media overview chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

http://blogs.msdn.com/b/b8/archive/2012/06/08/building-a-rich-and-extensible-media-platform.aspx

Media extensions SDK sample

http://code.msdn.microsoft.com/windowsapps/Media-extensions-sample-7b466096

Windows 8 multimedia desktop APIs available to Windows Store apps

http://msdn.microsoft.com/en-us/library/windows/apps/hh452756.aspx

SchemeHandlers and ByteStreamHandlers

http://msdn.microsoft.com/en-us/library/windows/desktop/aa371872(v=vs.85).aspx

Writing a Custom Media Source

http://msdn.microsoft.com/en-us/library/windows/desktop/ms700134(v=vs.85).aspx

Resources


Resources1

Resources chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.

  • Develop: http://msdn.microsoft.com/en-US/windows/apps/br229512

  • Design: http://design.windows.com/

  • Samples: http://code.msdn.microsoft.com/windowsapps/Windows-8-Modern-Style-App-Samples

  • Videos: http://channel9.msdn.com/Windows

Please submit session evals by using the Build Windows 8 app

or at http://aka.ms/BuildSessions


ad