Loading in 5 sec....

Spatio-Temporal Quincunx Sub-SamplingPowerPoint Presentation

Spatio-Temporal Quincunx Sub-Sampling

- 71 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about ' Spatio-Temporal Quincunx Sub-Sampling' - shlomo

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Overview

- Sampling in Television and Film
- The problems of aliasing
- Filtering requirements
- Conversion between differing formats
- Problems that can occur
- How we can mitigate some of the problems and maintain or improve quality

Sampling Theory

- Harry Nyquist – 1889 to 1976
- “The number of independent pulses that can be put through a telegraph channel per unit time is limited to twice the bandwidth of the channel”

Sampling Theory

- Harry Nyquist – 1889 to 1976
- “The number of independent pulses that can be put through a telegraph channel per unit time is limited to twice the bandwidth of the channel”

- Later Nyquist-Shannon
- “Exact reconstruction of a continuous-time baseband signal from its samples is possible if the signal is bandlimited and the sampling frequency is greater than twice the signal bandwidth”

Fs

Frequency

Sampling Theory- Audio:
- 20kHz bandwidth, Fs = 44.1kHz, 48kHz

- Video:
- 5.75MHz bandwidth, Fs = 13.5MHz
- 30MHz bandwidth, Fs = 74.25MHz

Fs

Frequency

AliasingNyquist

Frequency

- Frequencies above Fs/2 are “reflected” into the lower portion of the spectrum and become entangled with the low-frequency signals

Fs

Frequency

AliasingNyquist

Frequency

- Frequencies above Fs/2 are “reflected” into the lower portion of the spectrum and become entangled with the low-frequency signals
- These signals CANNOT be removed afterwards

Fs

Frequency

AliasingNyquist

Frequency

- Frequencies above Fs/2 are “reflected” into the lower portion of the spectrum and become entangled with the low-frequency signals
- These signals CANNOT be removed afterwards
- Filtering BEFORE sampling is needed

Image Sampling

- Horizontal resolution
- Sampling rate of 720, 1280, 1920 or 2048 samples/picture width
- Resulting resolution of 360, 640, 960 or 1024 cycles/pw

- Sampling rate of 720, 1280, 1920 or 2048 samples/picture width

Image Sampling

- Horizontal resolution
- Sampling rate of 720, 1280, 1920 or 2048 samples/picture width
- Resulting resolution of 360, 640, 960 or 1024 cycles/pw

- Sampling rate of 720, 1280, 1920 or 2048 samples/picture width
- Vertical resolution
- Sampling rate of 480, 576, 720, 1080 samples/picture height
- Resulting resolution of 240, 288, 360 or 540 cycles/ph

- Sampling rate of 480, 576, 720, 1080 samples/picture height

Image Sampling

- Horizontal resolution
- Sampling rate of 720, 1280, 1920 or 2048 samples/picture width
- Resulting resolution of 360, 640, 960 or 1024 cycles/pw

- Sampling rate of 720, 1280, 1920 or 2048 samples/picture width
- Vertical resolution
- Sampling rate of 480, 576, 720, 1080 samples/picture height
- Resulting resolution of 240, 288, 360 or 540 cycles/ph

- Sampling rate of 480, 576, 720, 1080 samples/picture height
- Temporal resolution
- Sampling rate of 24, 25, 30, 50, 60 . . . samples/second
- Resulting resolution of 12, 15, 25, 30 cycles/sec

- Sampling rate of 24, 25, 30, 50, 60 . . . samples/second

Re-sampling

- Image size changes are common

Amplitude

Vertical Frequency

Potential Alias

480

Amplitude

Vertical Frequency

Re-sampling- Image size changes are common
- Simple example of interpolating a 1080 picture to 480:
- Input resolution is 540 cycles/ph
- Output resolution is 240 cycles/ph (division by 2.25)

- Simple example of interpolating a 1080 picture to 480:

Filter

Re-sampling

- Interpolation is only one part of the problem
- Filtering is needed to control the signal spectrum and avoid the introduction of aliases
- Simple interpolators are generally poor filters

Re-sampling

- Interpolation is only one part of the problem
- Filtering is needed to control the signal spectrum and avoid the introduction of aliases
- Simple interpolators are generally poor filters

- Alias terms are “folded” about the Nyquist point
- Inverted in frequency, inverted “movement”
- Highly noticeable to the human eye, which references its own internal 3D model

Re-sampling

- Interpolation is only one part of the problem
- Filtering is needed to control the signal spectrum and avoid the introduction of aliases
- Simple interpolators are generally poor filters

- Alias terms are “folded” about the Nyquist point
- Inverted in frequency, inverted “movement”
- Highly noticeable to the human eye, which references its own internal 3D model

- Alias terms left in the image will be shifted again in any subsequent operations
- Potentially cumulative problems

Restricted by practical limitations

Linked by aspect ratio and pixel shape

3D SamplingTemporal – frames

Vertical - lines

Horizontal - pixels

No of Lines

Potential alias

Potential alias

Frame Rate

Temporal Frequency

Spatio-Temporal SamplingTemporal – frames

Spectrum

Spatial - lines

No of Lines

Potential alias

Potential alias

Frame Rate

Temporal Frequency

Spatio-Temporal Sampling- Filtering:
- Spatial – optical LPF and lens MTF

Temporal – frames

Spectrum

Spatial - lines

No of Lines

Potential alias

Potential alias

Frame Rate

Temporal Frequency

Spatio-Temporal Sampling- Filtering:
- Spatial – optical LPF and lens MTF
- Temporal – integration time of sensor system

Temporal – frames

Spectrum

Spatial - lines

Potential alias

Spatio-Temporal Sub-SamplingSpatial Frequency

- Where is the filter?

No of Lines

Temporal – frames

Spectrum

Frame Rate

Spatial - lines

Temporal Frequency

?

Up-conversionSpatial Frequency

No of Lines

Temporal

Frame Rate

Vertical

Spectrum

Temporal Frequency

?

Up-conversionSpatial Frequency

- Adaptive filtering

No of Lines

Temporal

Frame Rate

Vertical

Spectrum

Temporal Frequency

?

Up-conversionSpatial Frequency

- Adaptive filtering
- Motion compensation

No of Lines

Temporal

Frame Rate

Vertical

Spectrum

Temporal Frequency

1080p

720p

480i

1080i

1080p (24)

Format InterchangeSpatial Frequency

500c/ph

250c/ph

0c/ph

0c/s

15c/s

30c/s

Temporal Frequency

1080p

720p

480i

1080i

1080p (24)

Format Interchange- Conversion between formats requires care

Spatial Frequency

500c/ph

250c/ph

0c/ph

0c/s

15c/s

30c/s

Temporal Frequency

1080p

720p

480i

1080i

1080p (24)

Format Interchange- Conversion between formats requires care
- Mixing formats such as film and video is to be avoided

Spatial Frequency

500c/ph

250c/ph

0c/ph

0c/s

15c/s

30c/s

Temporal Frequency

1080p

720p

480i

1080i

1080p (24)

Format Interchange- Conversion between formats requires care
- Mixing formats such as film and video is to be avoided
- 1080p down-conversion might raise new challenges

Spatial Frequency

500c/ph

250c/ph

0c/ph

0c/s

15c/s

30c/s

Temporal Frequency

Amplitude

Frequency

48

Amplitude

Frequency

Over-sampling- Commonly applied to audio – eg 96kHz down to 48kHz
- Allows the use of a high performance digital filter:

Filter

Over-sampling

- Commonly applied to audio – eg 96kHz down to 48kHz
- Allows the use of a high performance digital filter:

Over-sampling

- Commonly applied to audio – eg 96kHz down to 48kHz
- Allows the use of a high performance digital filter:

- 1080p allows similar gains for outputs of 720p and 1080i
- Good temporal filtering must introduce delay

Over-sampling

- Commonly applied to audio – eg 96kHz down to 48kHz
- Allows the use of a high performance digital filter:

- 1080p allows similar gains for outputs of 720p and 1080i
- Good temporal filtering must introduce delay

- Film sampling at >1080 lines/ph also allows controlled down-sampling

Conclusion

- Spatio-temporal quincunx sub-sampling (aka interlace) is likely to be with us for some time

Conclusion

- Spatio-temporal quincunx sub-sampling (aka interlace) is likely to be with us for some time
- Modern cameras and processing can stress the format unless care is taken

Conclusion

- Spatio-temporal quincunx sub-sampling (aka interlace) is likely to be with us for some time
- Modern cameras and processing can stress the format unless care is taken
- Imprinted alias is difficult (or impossible) to remove
- Camera integration is an important filter for interlace

Conclusion

- Spatio-temporal quincunx sub-sampling (aka interlace) is likely to be with us for some time
- Modern cameras and processing can stress the format unless care is taken
- Imprinted alias is difficult (or impossible) to remove
- Camera integration is an important filter for interlace

- Poor anti-alias filtering leads to additional compression concatenation artefacts

Conclusion

- Spatio-temporal quincunx sub-sampling (aka interlace) is likely to be with us for some time
- Modern cameras and processing can stress the format unless care is taken
- Imprinted alias is difficult (or impossible) to remove
- Camera integration is an important filter for interlace

- Poor anti-alias filtering leads to additional compression concatenation artefacts
- 1080p down-conversion could make the stress worse

Download Presentation

Connecting to Server..