1 / 79

Visual Perception in Realistic Image Synthesis

Visual Perception in Realistic Image Synthesis. Ann McNamara. Outline . Introduction Modeling important characteristics of the human visual system (HVS) Perception based rendering Image quality metrics Tone reproduction operators Summary. Realism. Architecture Stage lighting

greta
Download Presentation

Visual Perception in Realistic Image Synthesis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Visual Perception in Realistic Image Synthesis Ann McNamara

  2. Outline • Introduction • Modeling important characteristics of the human visual system (HVS) • Perception based rendering • Image quality metrics • Tone reproduction operators • Summary

  3. Realism • Architecture • Stage lighting • Entertainment • Safety systems • Archaeology

  4. Human Visual System • Physical structure well established • Perceptual behaviour is a complex process

  5. Modeling Important Characteristics of the Human Visual System

  6. Visual Acuity • How well we can see fine detail • Adaptation level • Rods and cones Cones

  7. Spatial Frequency • Number of grating that fall on one degree of the retina • Dependent on distance

  8. Spatial Frequency • Spatial mechanisms (channels) which are used to represent the visual information at various scales and orientations as it is believed that primary visual cortex does.

  9. Contrast Sensitivity Function • Contrast sensitivity function which specifies the detection threshold for a stimulus as a function of its spatial frequencies.

  10. Contrast Sensitivity Campbell-Robson contrast sensitivity chart

  11. Masking • Visual masking affecting the detection threshold of a stimulus as a function of the interfering background stimulus which is closely coupled in space and time.

  12. Masking

  13. Colour Appearance

  14. Perceptually Based Rendering

  15. Low Sampling Densities [Mitchell 1987] • Non-uniform sampling is less conspicuous • Optimise using how the eye perceives noise as a function of contrastand colour Raytracing -> Point Samples-> Aliasing

  16. Uniform Non-Uniform Adaptive Sampling Schemes [Mitchell 1987]

  17. Low Sampling Densities [Mitchell 1987] • Contrast • Colour R 0.4G 0.3B 0.6

  18. Low Sampling Densities [Mitchell 1987]

  19. Frequency Based Raytracing [Bolin &Meyer 1992] • Synthesise directly into frequency domain • Simple vision model to control • Where to cast rays • How to spawn rays

  20. Frequency Based Raytracing [Bolin &Meyer 1992] • Vision model • Contrast sensitivity • Spatial frequency • Masking

  21. Frequency Based Raytracing [Bolin &Meyer 1992] • Specific luminance difference at low intensity more important than same luminance difference at high intensity • Colour spatial frequency variations given fewer samples • Decrease rays spawned in high frequency regions

  22. Colour Abberation Limited sampling of receptor Spatial acuity of opponent channels Limited Color Acuity [Meyer & Liu1998]

  23. [Meyer & Liu1998]

  24. Application • How much computation is enough? • How much reduction is too much? • An objective metric of image quality which takes into account basic characteristics of the HVS could help to answer these questionswithout human assistance.

  25. Questions of Appearance PreservationThe Concern Is Not Whether Images Are the Same Rather the Concern Is Whether Images Appear the Same

  26. Perceptually Based Adaptive Sampling Algorithm [Bolin &Meyer 1998] • Image quality model embedded into image synthesis • Use statistical information about spatial frequency to determine where to estimate values where samples were yet to be taken

  27. VDM JND’s Perceptually Based Adaptive Sampling Algorithm [Bolin &Meyer 1998]

  28. Convergence Evaluation [Myszkowski 1997] Deterministic radiosity  = 200s  = 400s  = 800s  = 1600s Monte Carlo radiosity

  29. Termination Criterion [Myszkowski 1997] 0.5 vs.   vs. reference

  30. Physical Based Perceptual Metric [Ramasubramanian et al1999] • Threshold model defines a physical error metric • Handles luminance-dependent and spatially dependent processing independently • Allowing pre-computation of spatially-dependent component

  31. Physical Based Perceptual Metric [Ramasubramanian et al1999]

  32. Image Quality Metrics

  33. Image Quality • Compare and validate lighting simulations • Use comparisons to guide rendering more efficiently • Compute less without altering perception • Pixel by pixel comparison might be > 0, human might not see any difference

  34. RMSE 9.5 RMSE 5.2 Pixel by Pixel Comparison Prikryl, 1999

  35. Amplitude Nonlinear. Contrast Sensitivity Function Cortex Transform Masking Function Unidirectional or Mutual Masking + Psychometric Function Probability Summation Visualisation of Differences Amplitude Nonlinear. Contrast Sensitivity Function Cortex Transform Masking Function Visible Differences PredictorVDP [Daly ‘93, Myszkowski ‘98] Image 1 Image 2

  36. Pixel differences: Standard - Comparison Comparison Pixel differences The VDP response: probability of perceiving the differences Standard VDP response VDP: Results

  37. Daly’s VDP: Features Daly, 1993 • Predicts local differences between images • Takes into account important visual characteristics: • Amplitude compression • Advanced CSF model • Masking • Uses the cortex transform, which is a pyramid-style, invertible & computationally efficient image representation

  38. Visible Discrimination Model Lubin, 1997 • Map of Just Noticeable Differences • Point sample function to model optics • Resample the image according to foveal eccentricity • Band pass response • Contrast pyramid • steerable filters

  39. Visible Discrimination Model Lubin, 1997 • Both images subjected to Identical processing • Distance measure • Difference in responses for Each channel and summing Them to obtain a JND Map of the two images

  40. An Experimental Evaluation of Computer Graphics Imagery [Meyer et al, 1986] • Comparing image to real-world scene • An approach to image synthesis consisting of • A physical module • A perceptual module

  41. Simulated Measured Difference An Experimental Evaluation of Computer Graphics Imagery [Meyer et al, 1986]

  42. An Experimental Evaluation of Computer Graphics Imagery [Meyer et al, 1986]

  43. An Experimental Evaluation of Computer Graphics Imagery [Meyer et al, 1986]

  44. Image Quality Metrics [Rushmeier et al, 1995] • Components of perceptually based metrics adapted from image compression • Gervais et al 1984 • Mannos et al 1974 • Daly 1993

  45. Real Room Simulated Model of Room Image Quality Metrics [Rushmeier et al, 1995] • Daly tested very well

  46. Visual Psychophysics • Determine the relationship between the physical world and human’s subjective experience of that world • Measure the response (“psycho”) to a known stimulus (“physics”)

  47. Why Lightness ? [Gilchrist 1977]

  48. A Psychophysical Investigation [McNamara et al 1998, 2000] • Painted 5-sided cube • Objects painted with different grey paints • Complex illumination, with secondary reflections

  49. Graphic Reconstructions [McNamara et al 1998, 2000]

  50. Real Scene Rendered Experiment [McNamara et al 1998, 2000]

More Related