media extensions
Download
Skip this Video
Download Presentation
Media extensions

Loading in 2 Seconds...

play fullscreen
1 / 38

Media extensions - PowerPoint PPT Presentation


  • 97 Views
  • Uploaded on

Media extensions. Marcin Stankiewicz and Matthew Howard Development - Multimedia Platform Team Richard Fricks Windows Scenario Adoption Team Session 4-104.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Media extensions' - luana


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
media extensions

Media extensions

Marcin Stankiewicz and Matthew Howard

Development - Multimedia Platform TeamRichard FricksWindows Scenario Adoption TeamSession 4-104

slide2
Engaging with rich media—whether watching a movie, video chatting, or playing music—is one of the most prevalent and enjoyable things we do on our PCs today.
agenda
Agenda
  • Architecture overview
  • Media Foundation pipeline
  • Media extensions
  • Visual Studio 2012
  • Media source demo
  • Streaming media to JavaScript
media application architecture
Media Application Architecture

Windows Store app

  • Media namespace
    • Capture
    • Transcode
    • PlayTo
    • PlaybackManager
    • MediaExtensionManager

JavaScript – HTML5

Video tag, audio tag

C#

MediaElement

Windows Runtime (WinRT)

Media engine

Playback and preview

Capture engine

Capture

Transcode

Sharing engine

PlayTo

Media Foundation

media app architecture
Media app architecture

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Media Foundation pipeline

Media source

Decoder

Effect

Encoder

Sink

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect

Media engine

Playback and preview

Capture engine

Capture

Transcode

Sharing engine

Play To

Media Foundation

media app architecture1
Media app architecture

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Media Foundation pipeline

Media source

Decoder

Effect

Encoder

Sink

  • Can be either native C++ DLL or a native C++/CX DLL
  • The DLL must ship in your app package
  • You can call WinRT APIs but you can use ONLY those desktop APIs that are marked for Windows Store apps:

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect

http://msdn.microsoft.com/en-us/library/windows/apps/hh452756.aspx

registration process
Registration process

.RegisterAudioDecoder()

.RegisterVideoDecoder()

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Media Foundation pipeline

.RegisterAudioEncoder()

.RegisterVideoEncoder()

Windows.Media.MediaExtensionManager

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect

.RegisterSchemeHandler()

.RegisterByteStreamHandler()

?

registration process effects
Registration process: Effects

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Media Foundation pipeline

Effects are registered using the class your code uses to process the media:

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect

registration process sinks
Registration process: Sinks

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Media Foundation pipeline

Sinks are supported only by MediaCapture:

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect

registration process1
Media sourceRegistration Process:

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Media Foundation pipeline

Audio

.RegisterAudioDecoder()

.RegisterVideoDecoder()

.RegisterAudioEncoder()

.RegisterVideoEncoder()

.RegisterSchemeHandler()

.RegisterByteStreamHandler()

.RegisterAudioDecoder()

.RegisterVideoDecoder()

.RegisterAudioEncoder()

.RegisterVideoEncoder()

.RegisterSchemeHandler()

.RegisterByteStreamHandler()

Audio encoder

Audio sink

Audio decoder

Audio effect

Windows.Media.MediaExtensionManager

?

which to use

Which to use?

  • ByteStreamHandler
  • New container format (WebM, MKV)
  • SchemeHandler
  • Generating your own streams
  • Media stored in a location not accessible to inbox ByteStream
  • Custom streaming protocol
imfbytestreamhandlerinterface1
IMFByteStreamHandlerInterface
  • HRESULT BeginCreateObject( [in] IMFByteStream *pByteStream,
  • [in] LPCWSTR pwszURL,
  • [in] DWORD dwFlags,
  • [in] IPropertyStore *pProps,
  • [out] IUnknown **ppIUnknownCancelCookie,
  • [in] IMFAsyncCallback *pCallback,
  • [in] IUnknown *punkState );

MediaElement.Source= newUri(“video.mv4");

imfbytestreamhandlerinterface2
IMFByteStreamHandlerInterface
  • HRESULT EndCreateObject( [in] IMFAsyncResult *pResult,
  • [out] MF_OBJECT_TYPE *pObjectType,
  • [out] IUnknown **ppObject );

typedef enum MF_OBJECT_TYPE { MF_OBJECT_MEDIASOURCE,

MF_OBJECT_BYTESTREAM,

MF_OBJECT_INVALID } MF_OBJECT_TYPE;

schemehandlers and byte streamhandlers
SchemeHandlers and Byte-StreamHandlers

Byte-StreamHandler

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Media Foundation pipeline

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect

imfschemehandlerinterface1
IMFSchemeHandlerInterface
  • HRESULT BeginCreateObject( [in] LPCWSTR pwszURL,
    • [in] DWORD dwFlags,
    • [in] IPropertyStore *pProps,
    • [out] IUnknown **ppIUnknownCancelCookie,
    • [in] IMFAsyncCallback *pCallback,
    • [in] IUnknown *punkState );

MediaElement.Source= newUri("myscheme://square");

imfschemehandlerinterface2
IMFSchemeHandlerInterface
  • HRESULT EndCreateObject( [in] IMFAsyncResult *pResult,
  • [out] MF_OBJECT_TYPE *pObjectType,
  • [out] IUnknown **ppObject );

typedef enum MF_OBJECT_TYPE { MF_OBJECT_MEDIASOURCE,

MF_OBJECT_BYTESTREAM,

MF_OBJECT_INVALID } MF_OBJECT_TYPE;

schemehandlers and byte streamhandlers1
SchemeHandlers and Byte-StreamHandlers

Byte-StreamHandler

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Media Foundation pipeline

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect

schemehandlers and byte streamhandlers2
SchemeHandlers and Byte-StreamHandlers

SchemeHandler

Byte-StreamHandler

ByteStream

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Media Foundation pipeline

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect

what is a bytestream

What is a ByteStream?

  • Represents a byte stream from some data source:
  • Local file
  • Network file
  • Some other source
  • This interface supports typical stream operations such as reading, writing, and seeking.
schemehandlers and byte streamhandlers3
SchemeHandlers and Byte-StreamHandlers

SchemeHandler

Byte-StreamHandler

ByteStream

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Media Foundation pipeline

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect

schemehandlers and byte streamhandlers4
SchemeHandlers and Byte-StreamHandlers

SchemeHandler

SourceResolver

Byte-StreamHandler

ByteStream

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Media Foundation pipeline

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect

media foundation pipeline
Media Foundation pipeline

SchemeHandler

SourceResolver

ByteStream

Byte-StreamHandler

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect

media foundation pipeline1
Media Foundation pipeline

SchemeHandler

SourceResolver

ByteStream

Byte-StreamHandler

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect

media foundation pipeline2
Media Foundation pipeline

SchemeHandler

SourceResolver

ByteStream

Byte-StreamHandler

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect

demo building a media extension
How to build an extension using C++/CX

The proper use of WRL to implement a SchemeHandler

Instantiating a media source from a SchemeHandler

Using the PropertySet class to implement two-way communication between the app and the extension

PropertySet change notification to support dynamic updates

Direct3D rendering

Demo: Building a media extension
  • Marcin Stankiewicz
  • Development - Multimedia Platform Team
recap of demo
Recap of Demo
  • How to build an extension using C++/CX
  • The proper use of WRL to implement a SchemeHandler
  • Instantiating a media source from a SchemeHandler
  • Using the PropertySet class to implement two-way communication between the app and the extension
  • PropertySet change notification to support dynamic updates
  • Direct3D rendering
media foundation pipeline3
Media Foundation pipeline

SchemeHandler

SourceResolver

ByteStream

Byte-StreamHandler

Video

Media source

Video decoder

Video effect

Video encoder

Video sink

Audio

Audio encoder

Audio sink

Audio decoder

Audio effect

demo streaming media to an app
How to implement a sink media extension for audio

Capturing uncompressed audio

Processing audio buffers in a JavaScript app

Writing audio to a wave file

Asynchronous processing techniques

Demo: Streaming media to an app
  • Matthew Howard
  • Development - Multimedia Platform Team
recap of demo1
Recap of Demo
  • How to implement a sink media extension for audio
  • Capturing uncompressed audio
  • Processing audio buffers in a JavaScript app
  • Writing audio to a wave file
  • Asynchronous processing techniques
need more information

Need more information?

Come to our question-and-answer session!

3-106 Chalk Talk: Media apps

Thursday 4:15pm, B33 Hood

media related sessions
Media-related sessions

3-117 10/31/2012 17:15 300 - Advanced B33 Hood Key technologies for building advanced media apps

3-122 11/1/2012 14:30 300 - Advanced B33 Hood Media app case studies

3-106 11/1/2012 16:15 300 - Advanced B33 Hood Chalk Talk: Media apps

resources
Windows 8 media overview

http://blogs.msdn.com/b/b8/archive/2012/06/08/building-a-rich-and-extensible-media-platform.aspx

Media extensions SDK sample

http://code.msdn.microsoft.com/windowsapps/Media-extensions-sample-7b466096

Windows 8 multimedia desktop APIs available to Windows Store apps

http://msdn.microsoft.com/en-us/library/windows/apps/hh452756.aspx

SchemeHandlers and ByteStreamHandlers

http://msdn.microsoft.com/en-us/library/windows/desktop/aa371872(v=vs.85).aspx

Writing a Custom Media Source

http://msdn.microsoft.com/en-us/library/windows/desktop/ms700134(v=vs.85).aspx

Resources
resources1

Resources

  • Develop: http://msdn.microsoft.com/en-US/windows/apps/br229512
  • Design: http://design.windows.com/
  • Samples: http://code.msdn.microsoft.com/windowsapps/Windows-8-Modern-Style-App-Samples
  • Videos: http://channel9.msdn.com/Windows

Please submit session evals by using the Build Windows 8 app

or at http://aka.ms/BuildSessions

ad