1 / 51

On the perception of Bandlimited Phase Distortion in Natural scenes

On the perception of Bandlimited Phase Distortion in Natural scenes. Kedar Vilankar , Logesh Vasu and Damon Chandler Computational Perception and Image Quality Laboratory School of Electrical and Computer Engineering Oklahoma State University. Importance of Phase.

arich
Download Presentation

On the perception of Bandlimited Phase Distortion in Natural scenes

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. On the perception of Bandlimited Phase Distortion in Natural scenes KedarVilankar, LogeshVasu and Damon Chandler Computational Perception and Image Quality Laboratory School of Electrical and Computer Engineering Oklahoma State University

  2. Importance of Phase • Oppenheim and Lim (1981) demonstrated the importanceof phase in signals. • Phase spectrum contributesmore to the image’s visual appearance. Magnitude Spectrum Phase Spectrum

  3. Cells Compute Local Information • Primates V1 is dominated by complex cells. • V1complex cells are insensitive to phase • V1complex cells encode the magnitude information only. • V1complex cells are localized to receptive fields.

  4. Local Magnitude is all we need • Phaseinformation is implicitlyencodedin local magnitude. • Morrone and Burr demonstrated computation of location of lines and edges (Phase congruency) from local magnitude (Complex cell response). • Also, other researchers have shown only local magnitude is required for scene recognition and categorization.

  5. Local Magnitude is all we need • Morgan et. al demonstrated local phase is of lesser importance than local magnitude. • HVS uses local magnitude information to determine global (image-wide) phase information. Phase Spectrum Magnitude Spectrum

  6. Does HVS use Local Phase ? • Signal processing perspective, local phase is important. • If HVS uses only local magnitude, then we should not see any impact of distortion in local phase. Local Phase Distorted Image Original Image

  7. Does HVS use Local Phase ? • Complex cells compute local magnitude by combining the responses of two simple cells. • May also exist a visual mechanism to compute local phase using simple cells.

  8. Does HVS use Local Phase ? • Complex cells compute local magnitude by combining the responses of two simple cells. • May also exist a visual mechanism to compute local phase using simple cells. • If HVS has mechanism to compute local phase, • then do we inferglobal phase from • only local magnitude or • both local magnitude and local phase

  9. Summary • Global phase ismost important for image appearance. (Oppenheim & Lim) • Local magnitude can implicitlyencode global phase. (Morroneet al. and Shams et al.) • Local phase is of lesser importance for image appearance • However, local phase distortion has substantial impact on image quality.

  10. Outline • Experiment (Local Phase Contribution) • Results (Interesting and surprising) • Discussion (Open questions. We need help) • Algorithm (Local magnitude and phase Distortion Rater) • Conclusion (Our belief)

  11. Experiment • Measure the relative contribution of local magnitude and local phase towards image appearance. • Stimuli were created by forming hybrid in complex wavelet subbands. • Each subbands local magnitude and local phase was taken from 2- 4 original images.

  12. Experiment Local Magnitude Local Phase Low Frequency High Frequency Frequency (cyc/deg) Original Images

  13. Experiment Local Magnitude Local Phase Low Frequency High Frequency Frequency (cyc/deg) Original Images

  14. Experiment • Based on permutations to make hybrid images 14 combination were created. • For each combination 12 stimuli were created. • Five subjects were asked to rate how much each original image contribute to the appearance of the stimulus. • Viewing distance : 60cm. • Five choices : 5%, 10%, 25%, 50% and 75%

  15. Result : Combination 1 Local Magnitude Local Phase Low Frequency High Frequency Frequency (cyc/deg) Original Images

  16. Result : Combination 1 Local Magnitude Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) Original Images

  17. Result : Combination 1 59% Local Magnitude 6% Local Phase 27% 8% Stimulus Low Frequency High Frequency Frequency (cyc/deg) Original Images

  18. Result : Combination 1 59% Local Magnitude 6% Local Phase 27% 8% Stimulus Low Frequency High Frequency Frequency (cyc/deg) When across frequencies, local magnitude and phase have non cooperative information, then HVS relies mostly on high frequency local magnitude information. Original Images

  19. Result : Combination 3 Local Magnitude Local Phase Low Frequency High Frequency Frequency (cyc/deg) Original Images

  20. Result : Combination 3 Local Magnitude Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) Original Images

  21. Result : Combination 3 39% Local Magnitude 6% 55% Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) Original Images

  22. Result : Combination 3 39% Local Magnitude 6% 55% Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) When local phase across entire frequency channels cooperate, then local phase information dominates local magnitude information. Original Images

  23. Result : Combination 12 Local Magnitude Local Phase Low Frequency High Frequency Frequency (cyc/deg) Original Images

  24. Result : Combination 12 Local Magnitude Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) Original Images

  25. Result : Combination 12 48% Local Magnitude 52% Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) Original Images

  26. Result : Combination 12 48% Local Magnitude 52% Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) Local phase and local magnitude have equal importance for the appearance of an image Original Images

  27. Result : Combination 13 Local Magnitude Local Phase Low Frequency High Frequency Frequency (cyc/deg) Original Images

  28. Result : Combination 13 Local Magnitude Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) Original Images

  29. Result : Combination 13 76% Local Magnitude 24% Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) Original Images

  30. Result : Combination 13 76% Local Magnitude 24% Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) High frequency information is more important than low frequency for image appearance. Original Images

  31. Discussion • Do we infer global phase from local phase?

  32. Result : Combination 12 48% Local Magnitude 52% Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) Original Images

  33. Discussion • Do we infer global phase from local phase? • Is there visual summation of local phase across frequency channels?

  34. Result : Combination 3 39% Local Magnitude 6% 55% Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) Original Images

  35. Discussion • Do we infer global phase from local phase? • Is there visual summation of local phase across frequency channels? • Why HVS needs local phase information?

  36. Discussion • Do we infer global phase from local phase? • Is there visual summation of local phase across frequency channels? • Why HVS needs local phase information? • How local phase is computed in HVS?

  37. Discussion • Do we infer global phase from local phase? • Is there visual summation of local phase across frequency channels? • Why HVS needs local phase information? • How local phase is computed in HVS? • What is the neural basis for this computation?

  38. Discussion • What is more important low frequency or high frequency?

  39. Result : Combination 13 76% Local Magnitude 24% Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) Original Images

  40. Discussion • What is more important low frequency or high frequency? Previous research: • Low and high convey independent information about image structure • For categorization task • Between class – Low frequency • Within class – High frequency • Information content in low and high frequency. Is this task dependent?

  41. Algorithm LMPDLocal Magnitude and Phase Distortion Rater • Algorithm was developed to rate the quality of local phase distorted images using experimental results. • Algorithm computes local phase as well as local magnitude distortions in an image.

  42. Algorithm LMPDLocal Magnitude and Phase Distortion Rater • Local Magnitude distortion • Decompose original and distorted images in five scale and ten orientation log-Gabor subbands. • For each scale • Compute local energy maps of the original and distorted images. • Compute local MSE map between local energy maps of the original and distorted images. Use block size of 16 × 16 for local MSE. • Collapse local MSE map via the L2 − norm into a single scalar value. • Compute correlation between local energy maps of original and distorted images. • Compute local magnitude distortion Si(where, i is the index for current scale) by multiplying scalar values obtained in step (c) and (d).

  43. Algorithm LMPDLocal Magnitude and Phase Distortion Rater • Using Equation combine the local magnitude distortion obtained for each scale in step (e) to compute local magnitude distortion rating in the distorted image. = () + () + () + () + () 1, 2, 3, 4, 5 are power coefficients with values of 4, 4, 2, 1.5 and 0.143 respectively

  44. Algorithm LMPDLocal Magnitude and Phase Distortion Rater • Local phase distortion • Decompose original and distorted images in four levels and four orientation complex wavelet subbands. • For each level of the complex wavelet subband • Extract local phase information of the original and distorted image. • Compute local phase distortion Ei(where, i is the index for current level) by computing MSE between local phase of the original and distorted image obtained in step (a).

  45. Algorithm LMPDLocal Magnitude and Phase Distortion Rater • Using Equation combine the local Phase distortion obtained for each scale in step (b) to compute local phase distortion rating in the distorted image. = () + () + () + () 1, 2, 3, 4are power coefficients with values of 2.1, 2.4, 2.3, and 2.2 respectively

  46. Algorithm LMPDLocal Magnitude and Phase Distortion Rater • Final quality rating of distorted image α = 0.6

  47. Algorithm LMPDLocal Magnitude and Phase Distortion Rater • Database of 48 phase distorted images. • Five subjects rated the distorted images. • Performance of LMPD compared with other image quality assessment algorithms.

  48. Algorithm LMPDLocal Magnitude and Phase Distortion Rater • Database of 48 phase distorted images. • Five subjects rated the distorted images. • Performance of LMPD compared with other image quality assessment algorithms.

  49. Conclusion • Local magnitude is most important information for image appearance. • Local phase also has a substantial, sometimes dominating contribution. • Local phase distortion of the images also has a substantial effect on image quality.

  50. Conclusion • Local magnitude is most important information for image appearance. • Local phase also has a substantial, sometimes dominating contribution. • Local phase distortion of the images also has a substantial effect on image quality. We believe that an explicit mechanism does exist in visual cortex for the computation of local phaseinformation.

More Related