spatio temporal quincunx sub sampling
Download
Skip this Video
Download Presentation
Spatio-Temporal Quincunx Sub-Sampling

Loading in 2 Seconds...

play fullscreen
1 / 41

Spatio-Temporal Quincunx Sub-Sampling - PowerPoint PPT Presentation


  • 71 Views
  • Uploaded on

Spatio-Temporal Quincunx Sub-Sampling. . . and how we get there David Lyon. Overview. Sampling in Television and Film The problems of aliasing Filtering requirements Conversion between differing formats Problems that can occur

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Spatio-Temporal Quincunx Sub-Sampling' - shlomo


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
spatio temporal quincunx sub sampling

Spatio-Temporal Quincunx Sub-Sampling

. . and how we get there

David Lyon

overview
Overview
  • Sampling in Television and Film
  • The problems of aliasing
  • Filtering requirements
  • Conversion between differing formats
  • Problems that can occur
  • How we can mitigate some of the problems and maintain or improve quality
sampling theory
Sampling Theory
  • Harry Nyquist – 1889 to 1976
    • “The number of independent pulses that can be put through a telegraph channel per unit time is limited to twice the bandwidth of the channel”
sampling theory1
Sampling Theory
  • Harry Nyquist – 1889 to 1976
    • “The number of independent pulses that can be put through a telegraph channel per unit time is limited to twice the bandwidth of the channel”
  • Later Nyquist-Shannon
    • “Exact reconstruction of a continuous-time baseband signal from its samples is possible if the signal is bandlimited and the sampling frequency is greater than twice the signal bandwidth”
sampling theory3

Amplitude

Fs

Frequency

Sampling Theory
  • Audio:
    • 20kHz bandwidth, Fs = 44.1kHz, 48kHz
sampling theory4

Amplitude

Fs

Frequency

Sampling Theory
  • Audio:
    • 20kHz bandwidth, Fs = 44.1kHz, 48kHz
  • Video:
    • 5.75MHz bandwidth, Fs = 13.5MHz
    • 30MHz bandwidth, Fs = 74.25MHz
aliasing

Amplitude

Fs

Frequency

Aliasing

Nyquist

Frequency

aliasing1

Amplitude

Fs

Frequency

Aliasing

Nyquist

Frequency

  • Frequencies above Fs/2 are “reflected” into the lower portion of the spectrum and become entangled with the low-frequency signals
aliasing2

Amplitude

Fs

Frequency

Aliasing

Nyquist

Frequency

  • Frequencies above Fs/2 are “reflected” into the lower portion of the spectrum and become entangled with the low-frequency signals
  • These signals CANNOT be removed afterwards
aliasing3

Amplitude

Fs

Frequency

Aliasing

Nyquist

Frequency

  • Frequencies above Fs/2 are “reflected” into the lower portion of the spectrum and become entangled with the low-frequency signals
  • These signals CANNOT be removed afterwards
  • Filtering BEFORE sampling is needed
image sampling
Image Sampling

Temporal – frames

Vertical - lines

Horizontal - pixels

image sampling1
Image Sampling
  • Horizontal resolution
    • Sampling rate of 720, 1280, 1920 or 2048 samples/picture width
      • Resulting resolution of 360, 640, 960 or 1024 cycles/pw
image sampling2
Image Sampling
  • Horizontal resolution
    • Sampling rate of 720, 1280, 1920 or 2048 samples/picture width
      • Resulting resolution of 360, 640, 960 or 1024 cycles/pw
  • Vertical resolution
    • Sampling rate of 480, 576, 720, 1080 samples/picture height
      • Resulting resolution of 240, 288, 360 or 540 cycles/ph
image sampling3
Image Sampling
  • Horizontal resolution
    • Sampling rate of 720, 1280, 1920 or 2048 samples/picture width
      • Resulting resolution of 360, 640, 960 or 1024 cycles/pw
  • Vertical resolution
    • Sampling rate of 480, 576, 720, 1080 samples/picture height
      • Resulting resolution of 240, 288, 360 or 540 cycles/ph
  • Temporal resolution
    • Sampling rate of 24, 25, 30, 50, 60 . . . samples/second
      • Resulting resolution of 12, 15, 25, 30 cycles/sec
re sampling
Re-sampling
  • Image size changes are common
re sampling1

1080

Amplitude

Vertical Frequency

Potential Alias

480

Amplitude

Vertical Frequency

Re-sampling
  • Image size changes are common
    • Simple example of interpolating a 1080 picture to 480:
      • Input resolution is 540 cycles/ph
      • Output resolution is 240 cycles/ph (division by 2.25)

Filter

re sampling2
Re-sampling
  • Interpolation is only one part of the problem
    • Filtering is needed to control the signal spectrum and avoid the introduction of aliases
    • Simple interpolators are generally poor filters
re sampling3
Re-sampling
  • Interpolation is only one part of the problem
    • Filtering is needed to control the signal spectrum and avoid the introduction of aliases
    • Simple interpolators are generally poor filters
  • Alias terms are “folded” about the Nyquist point
    • Inverted in frequency, inverted “movement”
    • Highly noticeable to the human eye, which references its own internal 3D model
re sampling4
Re-sampling
  • Interpolation is only one part of the problem
    • Filtering is needed to control the signal spectrum and avoid the introduction of aliases
    • Simple interpolators are generally poor filters
  • Alias terms are “folded” about the Nyquist point
    • Inverted in frequency, inverted “movement”
    • Highly noticeable to the human eye, which references its own internal 3D model
  • Alias terms left in the image will be shifted again in any subsequent operations
    • Potentially cumulative problems
3d sampling

Restricted by practical limitations

Linked by aspect ratio and pixel shape

3D Sampling

Temporal – frames

Vertical - lines

Horizontal - pixels

spatio temporal sampling

Spatial Frequency

No of Lines

Potential alias

Potential alias

Frame Rate

Temporal Frequency

Spatio-Temporal Sampling

Temporal – frames

Spectrum

Spatial - lines

spatio temporal sampling1

Spatial Frequency

No of Lines

Potential alias

Potential alias

Frame Rate

Temporal Frequency

Spatio-Temporal Sampling
  • Filtering:
    • Spatial – optical LPF and lens MTF

Temporal – frames

Spectrum

Spatial - lines

spatio temporal sampling2

Spatial Frequency

No of Lines

Potential alias

Potential alias

Frame Rate

Temporal Frequency

Spatio-Temporal Sampling
  • Filtering:
    • Spatial – optical LPF and lens MTF
    • Temporal – integration time of sensor system

Temporal – frames

Spectrum

Spatial - lines

spatio temporal sub sampling

Potential alias

Potential alias

Spatio-Temporal Sub-Sampling

Spatial Frequency

  • Where is the filter?

No of Lines

Temporal – frames

Spectrum

Frame Rate

Spatial - lines

Temporal Frequency

up conversion

Horizontal

?

Up-conversion

Spatial Frequency

No of Lines

Temporal

Frame Rate

Vertical

Spectrum

Temporal Frequency

up conversion1

Horizontal

?

Up-conversion

Spatial Frequency

  • Adaptive filtering

No of Lines

Temporal

Frame Rate

Vertical

Spectrum

Temporal Frequency

up conversion2

Horizontal

?

Up-conversion

Spatial Frequency

  • Adaptive filtering
  • Motion compensation

No of Lines

Temporal

Frame Rate

Vertical

Spectrum

Temporal Frequency

format interchange

Film

1080p

720p

480i

1080i

1080p (24)

Format Interchange

Spatial Frequency

500c/ph

250c/ph

0c/ph

0c/s

15c/s

30c/s

Temporal Frequency

format interchange1

Film

1080p

720p

480i

1080i

1080p (24)

Format Interchange
  • Conversion between formats requires care

Spatial Frequency

500c/ph

250c/ph

0c/ph

0c/s

15c/s

30c/s

Temporal Frequency

format interchange2

Film

1080p

720p

480i

1080i

1080p (24)

Format Interchange
  • Conversion between formats requires care
  • Mixing formats such as film and video is to be avoided

Spatial Frequency

500c/ph

250c/ph

0c/ph

0c/s

15c/s

30c/s

Temporal Frequency

format interchange3

Film

1080p

720p

480i

1080i

1080p (24)

Format Interchange
  • Conversion between formats requires care
  • Mixing formats such as film and video is to be avoided
  • 1080p down-conversion might raise new challenges

Spatial Frequency

500c/ph

250c/ph

0c/ph

0c/s

15c/s

30c/s

Temporal Frequency

over sampling

96

Amplitude

Frequency

48

Amplitude

Frequency

Over-sampling
  • Commonly applied to audio – eg 96kHz down to 48kHz
    • Allows the use of a high performance digital filter:

Filter

over sampling1
Over-sampling
  • Commonly applied to audio – eg 96kHz down to 48kHz
    • Allows the use of a high performance digital filter:
over sampling2
Over-sampling
  • Commonly applied to audio – eg 96kHz down to 48kHz
    • Allows the use of a high performance digital filter:
  • 1080p allows similar gains for outputs of 720p and 1080i
    • Good temporal filtering must introduce delay
over sampling3
Over-sampling
  • Commonly applied to audio – eg 96kHz down to 48kHz
    • Allows the use of a high performance digital filter:
  • 1080p allows similar gains for outputs of 720p and 1080i
    • Good temporal filtering must introduce delay
  • Film sampling at >1080 lines/ph also allows controlled down-sampling
conclusion
Conclusion
  • Spatio-temporal quincunx sub-sampling (aka interlace) is likely to be with us for some time
conclusion1
Conclusion
  • Spatio-temporal quincunx sub-sampling (aka interlace) is likely to be with us for some time
  • Modern cameras and processing can stress the format unless care is taken
conclusion2
Conclusion
  • Spatio-temporal quincunx sub-sampling (aka interlace) is likely to be with us for some time
  • Modern cameras and processing can stress the format unless care is taken
    • Imprinted alias is difficult (or impossible) to remove
    • Camera integration is an important filter for interlace
conclusion3
Conclusion
  • Spatio-temporal quincunx sub-sampling (aka interlace) is likely to be with us for some time
  • Modern cameras and processing can stress the format unless care is taken
    • Imprinted alias is difficult (or impossible) to remove
    • Camera integration is an important filter for interlace
  • Poor anti-alias filtering leads to additional compression concatenation artefacts
conclusion4
Conclusion
  • Spatio-temporal quincunx sub-sampling (aka interlace) is likely to be with us for some time
  • Modern cameras and processing can stress the format unless care is taken
    • Imprinted alias is difficult (or impossible) to remove
    • Camera integration is an important filter for interlace
  • Poor anti-alias filtering leads to additional compression concatenation artefacts
  • 1080p down-conversion could make the stress worse
ad