Towards a hybrid model
This presentation is the property of its rightful owner.
Sponsored Links
1 / 13

Towards a Hybrid Model PowerPoint PPT Presentation


  • 60 Views
  • Uploaded on
  • Presentation posted in: General

Towards a Hybrid Model. Provide a structure with building blocks Provide a programming and evaluation environment Invite researchers to evaluate and improve their algorithms Check performance improvements. Saliency and Visual Attention. Temporal Visual Attention. PVS. Image

Download Presentation

Towards a Hybrid Model

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Towards a hybrid model

Towards a Hybrid Model

  • Provide a structure with building blocks

  • Provide a programming and evaluation environment

  • Invite researchers to evaluate and improve their algorithms

  • Check performance improvements

Marcus Barkowsky, Savvas Argyropoulos


Towards a hybrid model

Saliency and

Visual Attention

Temporal

Visual Attention

PVS

Image

Characterization

Sequence

Characterization

Summarization

and

Mapping

Optimization

NR Image

Distortion Metrics

HMIX

Packet Loss

Analysis

Tracking of Codec-

Prediction for

Packet Loss Region

Bitstream Quality

Indicators

Temporal

behavior of

bitstream changes

Annex-B

Reference decoder

Quality of

decoder

implementation

Marcus Barkowsky, Savvas Argyropoulos


Properties of modules

Properties of Modules

  • Input

    • PVS, HMIX, output of other modules, …

  • Extent of output value

    • Pixel, Slice, Frame, Sequence, …

    • Example: distortion map (pixel), framerate (sequence)

  • Expected range of complexity and quality output

    • high/medium/low image quality

    • high/medium/low accuracy

    • high/medium/low complexity

    • Information may be used:

      • under temporal constraints (e.g. realtime)

      • to switch off modules that do not match the detected quality range, e.g. JND algorithms in the presence of packet loss

Marcus Barkowsky, Savvas Argyropoulos


Towards a hybrid model

eventually PVS

from

Reference decoder

Saliency and Visual Attention

PVS

Object detection

Background

segmentation

Texture, Luminance,

Color,

Masking Effects

HMIX

Motion Vectors

Motion based

algorithms

Perspective

estimation

Summary of

Spatial

Degradations

Attraction of

gaze by severe

degradations

Marcus Barkowsky, Savvas Argyropoulos


Towards a hybrid model

Temporal Visual Attention

PVS

HMIX

Motion Vectors

Temporal

visibility of

degradations

Influence of

Scene Cuts on

Attention

Behavioral

changes across

sequence

Marcus Barkowsky, Savvas Argyropoulos


Towards a hybrid model

eventually PVS

from

Reference decoder

Image Characterization

PVS

HMIX

Motion Vectors

Amount of Motion

Spatial Frequency,

e.g. flat regions

Content type

classification

Detection of

Faces, Persons, ...

Related to:

Saliency and

Visual Attention

Marcus Barkowsky, Savvas Argyropoulos


Towards a hybrid model

Sequence Characterization

PVS

HMIX

Motion Vectors

Mean motion,

global motion

uniformity

Content type

classification, e.g.

“Cartoon”

Behavioral

changes across

sequence

Marcus Barkowsky, Savvas Argyropoulos


Towards a hybrid model

NR Image Distortion

PVS

HMIX

QP, DCT Coeffs

Blockiness,

Blurriness

on block level

Related to

Packet Loss

Analysis

Blurriness,

Sharpness

on image level

Related to

Packet Loss

Analysis

Frame rate

Pausing and

skipping,

Rebuferring

Marcus Barkowsky, Savvas Argyropoulos


Towards a hybrid model

Packet Loss Analysis

PVS

Discontinuities,

Artifacts

eventually PVS

from

Reference decoder

Efficiency of

Error Concealment

in decoder

HMIX

Position and

length in

bitstream

Hypothetical

reference decoder,

Leaky Bucket

Editors note: These blocks are not independant

Marcus Barkowsky, Savvas Argyropoulos


Towards a hybrid model

Tracking of Codec Prediction for

Packet Loss Prediction

PVS

Discontinuity

Analysis

HMIX

Tracking via

MV, Block Type

Influence of QP

Editors note: These blocks are not independant

Marcus Barkowsky, Savvas Argyropoulos


Towards a hybrid model

HMIX

Bitstream Quality Indicators

Framerate

Picture Size

MB type and QP

MB type and Bitrate

Motion Vectors

and subblock

pattern

DCT coeff.

distribution

Marcus Barkowsky, Savvas Argyropoulos


Towards a hybrid model

HMIX

Temporal behavior of bitstream changes

Rapid changes

in QP

Marcus Barkowsky, Savvas Argyropoulos


Towards a hybrid model

Saliency and

Visual Attention

Temporal

Visual Attention

Functions that

produce a

distortion map

Spatial error

visibility

Summation

Temporal Error

Visibility

Match with Continuous Quality

Evaluation of subj. experiments

Summation

OMOS

Optimization

Toolbox of

mapping function,

linear, sigmoid etc.

Marcus Barkowsky, Savvas Argyropoulos


  • Login