Presented by Steve Holmes Sr. Video Applications Engineer

1. File-Based Video QC Presented by Steve Holmes Sr. Video Applications Engineer

2. Agenda Overview of File-Based Video • Container and Codec Formats • Metadata Quality Control for File-Based Content • QC Methodology • Types of Errors Content Readiness Testing in Various Workflows • Video on Demand • Local Ad Insertion • Cloud Based QC • Adaptive Bit Rate Delivery Automating File-Based QC • Semi-Automated Workflows • Integration with Asset Management Systems

3. Signal source Monitor the Signal Processing Monitor the Camera’s output Up-Link Signal Processing Camera

4. The Old Way of Evaluating Video Quality

5. Content QC for Multi Platform Multi Display DistributionIs Visual Inspection Enough? • It’s been said “Content is King”…well, if that is the case then managing your content and assets is “Queen”. • The traditional approach to Quality Control is visual inspection • QC staff can see two main categories of technical impairments: • Signal levels, such as video luma and chroma, or audio loudness • Problems such as black sequences, frozen frames, blockiness, loss of audio, audio/video sync • Subjective and variable results • Effective when reviewing relatively small volumes of video content • Cycle time is content length plus administrative follow up SMPTE Regional Seminar Series

6. Content QC for Multi Platform Multi Display DistributionIs Visual Inspection Enough? • Human QC Falls Apart • Syntax Errors • Interoperability • Format Mismatches • Headers Not Matching Content • Not Meeting ALL Deliverable Requirements • Far Too Much Content to Handle Workload Cost Effectively • Girl Friend on the phone SMPTE Regional Seminar Series

7. Content QC for Multi Platform Multi Display DistributionCerify- Automated Quality Control • Instrumented or Optimized Decoder • Reporting syntax errors as it processes the encoded essence and metadata • Measuring video and audio quality • Scalable System • Determine system through put Integrate with MAM automation system Periodic monitoringfor new files File copy or Automatic file transfer WatchFolder QC System Output Quarantine

8. Quality Control forFile-Based Workflows Content Readiness for File-Based Video Workflows

9. What does a Automated File QC system do • An Automated File QC System test that your files meet your deliverables for that point in the system where you are checking. • Is the Syntax or Structure of the file correct • Is the Container what you expect it to be • Is the Video essence what you expect it to be • Is the Audio essence what you expect it to be • Does the metadata contain the information that you need Content Readiness for File-Based Video Workflows

10. Overview of File-Based VideoSyntax or Structure of the file • Syntax Checks the syntax and integrity of the file layer; that is, are there problems in the file structure, but not in the elements of the streams. • Testing for compliance to the detected standard. This will generate alerts for corrupted streams, and streams generated by misconfigured or noncompliant encoders. • Examples: • MPEG Syntax • Incorrect encoding, • File copy or transfer errors Content Readiness for File-Based Video Workflows

11. Overview of File-Based VideoContainer Formats • Containers, or “wrappers”, are file formats for identifying and organizing audio/video essence and its associated metadata • Sometimes independent of the audio/video encoding formats used within • May be optimized for specific applications, such as acquisition, playback (streaming) or non-linear editing • Examples: • MPEG Program Stream • MPEG-2 Transport Stream • MP4 • 3GP • QuickTime File Format (QT) • Material Exchange Format (MXF) • General Exchange Format (GXF) • Advanced Systems Format (ASF) Content Readiness for File-Based Video Workflows

12. Overview of File-Based VideoVideo Formats • Video essence • Gamut violations: RGB components or luma • Frozen frames, black frames • Unexpected letterbox/pillarbox presence • Blockiness • Examples: • MPEG-2 Transport Stream • H.263 • DV • ProRes • MP4 • VC-1 • H.264 • JPEG2000 • QuickTime File Format (QT) • Raw Video Content Readiness for File-Based Video Workflows

13. Overview of File-Based VideoAudio Formats • Audio essence and its associated metadata • Loudness and true peak limits • Clipping • Silence and mute • Examples: • MPEG • AC3 • Dolby-E • DV • AAC • PCM • WMA Content Readiness for File-Based Video Workflows

14. Overview of File-Based VideoVariety of Formats within the Workflow • Different container formats and codec types are optimized for different tasks in the workflow • MPEG Transport Stream is suitable for playout • QuickTime or MXF with Intra encoded video are suitable for editing and transcoding • “Mezzanine” files are working copies of the original source material, more convenient to use in the workflow • Compressed, but without noticeable loss of picture quality • Less storage space (and file transfer time) than the original • High-enough resolution avoids generation loss when transcoding • I-frame only, for easier non-linear editing • Often 10-bit 4:2:2 sampling (same as SDI) • Transcoding is the format conversion of media assets • From an ingest format to the common mezzanine format • From the mezzanine format to multiple distribution formats Content Readiness for File-Based Video Workflows

15. Quality Control for File-Based ContentAutomatic Quality Control • Saves time and resources • Automated systems operate 24/7 • Skilled QC operators are few and expensive • Allows QC staff to work only on content that is identified as errored • More thorough than visual inspection • Consistent, repeatable results • Catch errors “inside” the file, such as syntax errors, encoding parameters, and structural metadata Content Readiness for File-Based Video Workflows

16. Quality Control for File-Based ContentError Types and Error Detection • Errors can occur as a result of: • Incorrect source material, resulting in baseband errors being encoded into the file • Incorrect encoding, from misconfigured or faulty encoders • File copy or transfer errors • Errors can be detected by an instrumented decoder, capable of: • Reporting syntax errors as it processes the encoded essence and metadata, and • Measuring video and audio quality in the decoded image raster and audio channels Content Readiness for File-Based Video Workflows

17. Quality Control for File-Based ContentBaseband Quality Checks • Non-real-time measurements of file-based content are similar toreal-time waveform monitoring of SDI signals • Baseband video errors: • Gamut violations: RGB components or luma • Frozen frames, black frames • Unexpected letterbox/pillarbox presence • Baseband audio errors: • Loudness and true peak limits • Clipping • Silence and mute Content Readiness for File-Based Video Workflows

18. Quality Control for File-Based ContentEncoded Content Checks • Syntax errors and encoding errors may adversely affect picture quality • Over-compression may results in block artifacts, and lower subjective picture quality • MPEG errors such as incorrect slice order result in large block artifacts • Incorrect field order for interlaced video (e.g. encoded top field first, playout expected bottom field first) will result in undesirable motion artifacts • Syntax error checking can also revealfile-related issues • E.g. incomplete file transfer will result in missing end-of-sequence start code Content Readiness for File-Based Video Workflows

19. Quality Control for File-Based ContentStructural Checks • Checking the container structure and metadata can reveal errorssuch as: • Incorrect number of streams (e.g. missing audio) • Incorrect PIDs for MPEG-2 Transport Streams(e.g non-compliant with CableLabs specification for VOD content) • Measurements made on the content can reveal errors such as: • Mismatch between play duration of video and audio tracks • Mismatch between actual bit rate and signaled bit rate • Checking video and audio codec headers will identify “unexpected” essence formats and encoding: • Profile and level • GOP structure • Frame and sample rates • Picture size and aspect ratio • Interlaced or progressive • Color depth and color sampling Content Readiness for File-Based Video Workflows

20. Cloud Computing forVideo Workflows File-Based Video Workflows In the Cloud

21. Cloud Computing for Video WorkflowsWhat is Cloud Computing? • Can be simply defined as the:Delivery of Services from a Network-based Infrastructure • Entomology: “Cloud” is a widely-used metaphorfor the Internet • De-facto standard in network topology diagrams • Benefits of cloud computing: • Conversion of cost from CapEx to OpEx • Service Provider pays for equipment and maintenance • “Pay as you go” pricing • On-demand usage of shared resources results inbetter utilization of storage and computing assets • Centralization of infrastructure results in lower costs • Service providers can offer higher reliability anddisaster recovery than individual enterprises File-Based Video Workflows In the Cloud

22. Cloud Computing for Video WorkflowsEssential Characteristics of Cloud Computing • On-demand self-service • Users can provision server time and/or network storage without manual interaction from the service provider • Broad network access • Capabilities are available over a variety of access interfaces and client platforms • Resource pooling • Resources are shared by multiple users, with resources assigned dynamically • Rapid elasticity • Resources can be provisioned, re-sized, and released to scale with user demand • Measured service • Resource usage is metered File-Based Video Workflows In the Cloud

23. Cloud Computing for Video WorkflowsCloud Architecture File-Based Video Workflows In the Cloud

24. Cloud Computing for Video WorkflowsService Models • Infrastructure as a Service (IaaS) • Virtual computing resources and storage • Examples: Amazon EC2 and S3, Windows Azure • Billed like a utility, based on computing time and storage usage • API typically implemented with SOAP and REST • Platform as a Service (PaaS) • Provides a computing platform and solution stack • Includes the OS plus web server, database, and programming language • LAMP: Linux + Apache + MySQL + Python/Perl/PHP • WISA: Windows + IIS + SQL Server + ASP.NET • Software as a Service (SaaS) • “On demand software” provided by Application Service Providers (ASPs) • Centralized multi-tenant software architecture, accessed by thin clients via a web browser • Examples: Google Apps, iTunes, Office 365 File-Based Video Workflows In the Cloud

25. Cloud Computing for Video WorkflowsSeparation of Responsibilities File-Based Video Workflows In the Cloud

26. Cloud Computing for Video WorkflowsCloud-Based Video Workflow • Asset storage in the cloud • Mezzanine files uploaded after local ingest and editing • Content processing in the cloud • Transcoding and Quality Control • Input and output file locations are in cloud storage • Streaming delivery from the cloud • Adaptive Bitrate video for various client devices and network bandwidth Streaming Video Web Apps • Cloud Services • Storage (incl. Archive) • Content Distribution • Content Delivery • Transcoding • Quality Control Local Resources File-Based Video Workflows In the Cloud

27. Content Readiness Testingin Various Workflows Content Readiness for File-Based Video Workflows

28. Content ReadinessLocal Ad Insertion Architecture IngestServer GbESwitch Edge QAM Video withLocal Ads Ad Playout SCTE 30 Messages Splicer Ad Server Video with National Adsor Black Segments Digital Broadcast Feed with Embedded SCTE 35 Cues Analog Broadcast Encoder DTMFCues SCTE 104 Cue Detection Content Readiness for File-Based Video Workflows

29. Content ReadinessLocal Ad Insertion Workflow • Commercial content can be delivered from production houses in a myriad of codec and container formats • Tapes are recorded to files at ingest • Files may be received by physical media or file transfer from a content delivery network • Problematic content may be edited locally, but usually rejected back to the provider • Files may be ingested as a mezzanine format, and transcoded to playout formats • Ad splicer can only make switches on GOP boundaries Content Readiness for File-Based Video Workflows

30. Content ReadinessQuality Issues in Ad Insertion Workflows • Picture and audio quality errors • Gamut violations, audio loudness, field-order issues, etc. • Ingest errors • Excessive blockiness from over compression, dropouts, clips and other audio distortion, etc. • Does not meet submission guidelines • The operator may constrain delivery formats to a specific set to reduce complexity • Codec type, container type, audio channel assignments, picture size, frame rate, minimum/maximum bit rate, etc. • Format mismatch • Avoid windowboxed video by having separate ingest paths for SD and HD deliveries • Verify that video fills the active image (no letterbox and pillarbox regions) • Verify Active Format Descriptor code if present Content Readiness for File-Based Video Workflows

31. Adaptive Bitrate Streaming (ABR) Video Quality Assurance (QC) Presented by Steve Holmes Sr. Video Applications Engineer

32. Content ReadinessAdaptive Bitrate Streaming Architecture Mezzanine File Transcoder Video Server Web Server StreamSegmenter HTTP Media asset Manifest file Media asset High bitrate version Medium bitrate version Medium bitrate version Low bitrate version High bitrate version Low bitrate version Video segments Video segments Video segments Content Readiness for File-Based Video Workflows

33. Content ReadinessAdaptive Bitrate Streaming Architecture Mezzanine File Transcoder Video Server StreamSegmenter QC System Media asset QC Each of the Manifest files QC the Mezzanine File High bitrate version Now you know that each of the Manifest files are Good Quality Medium bitrate version Low bitrate version Content Readiness for File-Based Video Workflows

34. Content ReadinessAdaptive Bitrate Streaming Architecture QOS Items to check and Alert on We know the Content is good Now check that we can Deliver the content without error Web Server HTTP Media asset Manifest file High bitrate version Low bitrate version Medium bitrate version Video segments Video segments Video segments Content Readiness for File-Based Video Workflows

35. Content ReadinessAdaptive Bitrate Streaming Workflow • QC the Mezzanine File to make sure that we have a good starting point • QC each of the Manifest Files • Program material is encoded into multiple bitrates—multiple versions of the same content • Streams are usually “segmented” or “fragmented” such that only a short sequence of the content is streamed from the server side • Standard HTTP network transport is used for client access • Firewalls will pass HTTP traffic • Test the QOS to ensure that all Manifest segment can be delivered without error • Client-side player determines available bit rate and requests the appropriate content to meet current bandwidth availability • As bandwidth availability changes, the next segment is requested at the optimum bitrate • As streams are segmented they can be “paused” at an appropriate GOP boundary Content Readiness for File-Based Video Workflows

36. ABR Overview Customer Client Content Proxy Edge Cache Origin Cache Walk me through the initial process Hello! Here is my sub id, device type, bandwidth available, session ID. I would like Superman 3, please. Authentication Do I have the asset? Sub id, device type, bandwidth available, and session ID would like Superman 3, please Yes No Send manifest file (XML) Transfer file to Edge Cache Request asset file Process manifest file Profile 2 of Superman 3, to session ID, please Profile 2 of Superman 3, to session ID, please Transfer file segment Here is your file, enjoy Here is your file, enjoy Measure download speed and begin playback

37. ABR Streaming Video Edge Server Movie Profile 1 Profile 2 Internet Profile 3 Profile 4 Tablet Smart Phone Set Top Box Profile 5 Profile 6

38. Content ReadinessQuality Issues in Streaming Workflows • Files must be “fragmentable” • H.264 video needs periodic IDR (Instantaneous Decoder Refresh) frames • No frame after an IDR frame can reference any frame before it • Each fragment must begin with an IDR frame • Unacceptable picture quality for lower bitrates • Blockiness and other compression artifacts reduce the perceptual video quality of the program content • Picture Quality Analysis tools can be used to “tune” the encoder to the best possible quality for a given target bitrate P B I IDR Content Readiness for File-Based Video Workflows

39. Content ReadinessQuality Issues in Streaming Workflows • Files must be “fragmentable” • H.264 video needs periodic IDR (Instantaneous Decoder Refresh) frames • No frame after an IDR frame can reference any frame before it • Each fragment must begin with an IDR frame • Unacceptable picture quality for lower bitrates • Blockiness and other compression artifacts reduce the perceptual video quality of the program content • Picture Quality Analysis tools can be used to “tune” the encoder to the best possible quality for a given target bitrate Content Readiness for File-Based Video Workflows

40. Media Set Status: HTTP Status Codes Mouse over code blocks to get event counts or click on code block to drill down tfor Media Set detail graphing of just that status code.

41. Media Set Status and HTTP Codes • Alert and Report on Media Set Representations and Display ALL HTTP Status Codes. Mouse over code blocks to get event counts or click on code block to drill down tfor Media Set detail graphing of just that status code. Quickly and accurately determine HTTP request problems and missing profile content.

42. Media Set Status: HTTP Status Codes • HTTP response status code columns: • Show the reponses to HTTP requests for a particular URL • 100s: Informational. This class indicates a provisional response, consisting of the Status-Line and optional headers. • 200s: Success. The action requested by the client was received, understood, accepted and process successfully. • 300s: Redirection. Further action needs to be taken by the user agent to fulfill the request. • 400s: Client Error. The client seems to have erred. Except when responding to a HEAD request, the server should include an entity containing an explanation of the error situation, and whether it is a temporary or permanent condition. • 500s: Server Error. The server failed to fulfill an apparently valid request.

43. Media Set Detail Report

44. Media Set Detail Report • Allows you to drill down to specific HTTP Status code errors • Show any combination of your specific representations • HTTP Status column allows you to mouse over status code bar to get the quantity of status codes over a particular reporting interval • Click on the bar to update the HTTP Status code graph which will indicate the specific status code error

45. Media Set Detail Report: Statistics & Graphs • Fragment Load Time: This is the time measured between when the HTTP header for a fragment is detected and when enough bytes have been received to equal the fragment size. • Fragment Load Latency: This is the time between when a request is made for a fragment (at the socket level) and when the HTTP header for the fragment is detected.

46. Media Set Detail Report: Statistics & Graphs • Fragment Load Bitrate: This is the fragment size / fragment load time

47. Program Statistics • Allows for reviewing/exporting the statistics collected for the reporting period against the ABR content being monitored. Measurements include: • ABR Fragment Size/Duration • Average fragment size being received. • ABR Fragment Load Time • Time measured between when the HTTP header for a fragment is detected and when enough bytes have been received to equal the fragment size. • ABR Fragment Load Bitrate • This is the (fragment size / fragment load time) • ABR Fragment Load Latency • Is the time between when a request is made for a fragment (at the socket level) and when the HTTP header for the fragment is detected. • ABR HTTP Status • Percent of each HTTP Status code present. • The representation index shows the number of representations being reported on for a particular port/URL.

48. Program Dashboard • A User Definable Dashboard View to quickly trend errors based on Program Statistics. Graph shows the number of representations errored in each category, content unavailable to users.

49. System Status User sets bandwidth limit • Used to check on the current status of the URL’s joined and the bitrate being utilized on the network. • Displays the Origin Server Bitrate used and Bitrate Threshold set during configuration. • Status will display UP, DOWN or PENDING for each configured input.

50. ABR Testing • If you test the Mezzanine file to make sure you are starting with a quality product. Then test each of the transcoded output files, you will know that your content is good. • After the file is segmented (chunked) the files are scrambled in most cases (DRM is applied) but since you checked all of the content before hand you know that the content is good. Now we only need to test that we can deliver (QOS) the content to the end user with out error. • Testing for good QOS in the delivery of the content will ensure that we have good QOE with the end user. Content Readiness for File-Based Video Workflows