1 / 56

Imaging Colorimetry using a Digital Camera with Dental Applications

Imaging Colorimetry using a Digital Camera with Dental Applications. Jan P. Allebach School of Electrical and Computer Engineering Purdue University 5 November 2002. Applications of tooth colorimetry: assessment of efficacy of whitening agents. After. Before.

dirk
Download Presentation

Imaging Colorimetry using a Digital Camera with Dental Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Imaging Colorimetry using a Digital Camera with Dental Applications Jan P. Allebach School of Electrical and Computer Engineering Purdue University 5 November 2002

  2. Applications of tooth colorimetry:assessment of efficacy of whitening agents After Before Image obtained from: http://www.westcentraldental.com/cosmetic.htm

  3. Applications of tooth colorimetry:matching color of restored tooth to neighbors

  4. Colorimetry research at Purdue • Phase I • Development of theoretical framework for model-based colorimetry system and filter design • Simulation results only • M. Wolski, C. A. Bouman, and J. P. Allebach, “Optimization of Sensor Response Functions for Colorimetry of Reflective and Emissive Objects,” IEEE Trans. on Image Processing, Vol. 5, pp. 507-517, March 1996. • Phase II • Application to colorimetry with a digital camera • Characterization of digital camera • Regression-based approach • Experimental results with color patches and extracted teeth • W. Wu, J. P. Allebach, M. Analoui, “Imaging Colorimetry Using a Digital Camera,” Journal of Imaging Science and Technology, Special issue on color imaging, Vol. 44, pp. 267-279, July/August 2000.

  5. Colorimetry research at Purdue (cont.) • Phase III • Refinement of experimental system • Development of reflectance estimation based approach • Experimental results with extracted and live teeth • D.Y. Ng, J. P. Allebach, Z. Pizlo, and M. Analoui, “Non-Contact Colorimeter for Human Tooth Color Assessment using a Digital Camera,” Proceedings of the IS&T/SID 10th Color Imaging Conference, Scottsdale, AZ, 12 November – 15 November 2002.

  6. Collaborators/Sponsors • Mark Wolski,General Motors Research Laboratory • Color Savvy Systems,Inc. • Indiana University School of Dentistry • Dr. Mostafa Analoui, (now withPfizer,Inc.) • Dr. George Stookey • Indiana 21st Century Fund

  7. Synopsis • What is color? • Phase II • System architecture • Filter design • Preliminary results • Phase III • System refinement • “Clinical” system • Experimental results • Future research (Phase IV?)

  8. What does color mean?

  9. Each device has its own language

  10. Spectral Representation of Color

  11. Trichromatic Representationof Color for Human Vision • 3-D subspace

  12. Finite-Dimensional Approximation

  13. Finite-Dimensional Approximation (cont.)

  14. Sensor Subspace • Example 1: N = 2 wavelengths, one monochromatic sensor stimuli s1and s2 are metamers to sensor q

  15. Role of Illuminant illuminant modifies sensor response

  16. Effect of Illuminant Metamerism • Example 2: N = 2 wavlengths, two monochromatic sensors stimuli s1and s2 are metamers to sensor q1 but not to q2

  17. Challenges of tooth color measurement • Smooth surface gives rise to specular reflection • Irregular pattern of scattering of light gives rise to non-uniform color on the surface • Irregular size and shape complicate measurements • Fluorescence makes color less predictable A device with high spatial resolution is needed

  18. Measurement device Calibration mapping What is a colorimetry system?

  19. Prior work: colorimetry system design • Calibration mapping • 3 X 3 matrix (Wandell & Farrell, 1993, Finlayson & Drew, 1996) • 3 X 6 matrix (Farrell et al, 1994) • Measurement device • Design a device (Wolski et al, 1996) • Alter device sensitivity (Chen & Trussell, 1995) • Increase the number of device channels (Tominaga 1999, Haneishi et al, 1995)

  20. Prior Work: colorimetry with a digital camera • Two main approaches have been adopted: • Physics-based approach: • Model the electro-optics of the camera, [Daligault, 1991], [Quan et al, 2000], no experiments have been reported. • Statistically-based approach: • Focus on what the camera ‘sees’ instead of how the camera ‘sees’ • Linear regression is widely used • Results have been reported for color patches such as MacBeth Color Checkers ( [Lenz, 1996], [Balas, 1997]), Munsell Color Chips ( [Finlayson, 1997]) and IT8 target ( [Pointer, 2001])

  21. Phase II research • Digital camera + filters & calibration matrices • Attack the problem simultaneously on both the design of filters in measurement device and the mapping in color calibration • Obtain tristimulus values of a sample under pre-selected illumination conditions rather than recover its reflectance

  22. measurement device illuminant L Camera Filter 1 Camera Filter Nf reflectance r under illuminant L’ Multi-exposure colorimetry system

  23. Multi-exposure colorimetry system • A calibration problem: if filters are given • Model-based or regression-based method • A system design problem: if filter design is desired • Model-based method only • Interaction between measurement device and calibration mapping

  24. System design: notation fi : 31-pt. Filter transmittance of ith filter D: camera sensitivities A: CIE XYZ color matching function

  25. Design goal and approach • Find the “best” F-M’ pair • Small perceptual errors • Robust system • Smooth filter transmittance • How? • RMS errors of linearized DE • Perturbation of F • Cost on filter roughness

  26. System design: summary Minimize: Subject to: • Optimal design through constrained optimization • Restricted search among candidate solution set

  27. Calibration mappings • Model-based method • Regression-based method

  28. Evaluation of system performance • DE in L* a* b* space • T1-T2: error in CIE XYZ space • Mean DE

  29. Perceptual error - 1

  30. Perceptual error - 2

  31. Robustness to filter variation

  32. Smoothness constraint and overall cost function

  33. Design a colorimetry system • Camera sensitivity • Illumination conditions • Training samples • Number of exposures • Robustness and smoothness (Kt,Ks) • Filter design/restricted search

  34. Monochromator Output Camera Sensitivity 5 x 10 2.5 0.12 red 0.1 green 2 blue 0.08 1.5 counts sensitivity 0.06 1 0.04 0.5 0.02 0 0 300 400 500 600 700 800 400 450 500 550 600 650 700 wavelength (nm) wavelength (nm) Camera sensitivity • Request data directly from manufacturer • Estimate it through calibration

  35. Illuminant Spectra Chromaticity Diagram 0.44 25 Flash 0.42 E 20 A A 0.4 F F 15 D65 power y 0.38 10 0.36 5 0.34 Flash E D65 0.32 0 0.3 0.35 0.4 0.45 0.5 400 450 500 550 600 650 700 x wavelength (nm) Illumination conditions • Daylight from viewing booth & camera flash • E, A, F, D65

  36. 2 10 Flash E A 1 10 F E D65 D mean 0 10 -1 10 1 2 3 4 5 6 No. of bases Training samples • Application dependent • 120 DuPont paint chips • 170 natural objects • 1269 Munsell color chips

  37. No noise With noise Robustness issue: Ks = 1 Robustness issue: Ks = 1 2 10 7 6 5 E E D 4 D 1 10 mean mean 3 2 1 0 10 0 -8 -6 -4 -2 0 -8 -6 -4 -2 0 10 10 10 10 10 10 10 10 10 10 Kt Kt Smoothness issue: Kt = 1e-4 Smoothness issue: Kt = 1e-4 2 10 7 6 5 E E D 4 D 1 10 mean mean 3 2 1 0 0 10 -2 -1 0 1 2 -2 -1 0 1 2 10 10 10 10 10 10 10 10 10 10 Ks Ks Robustness and smoothness Flash-Flash

  38. 1 1 0.8 0.8 0.6 0.6 transmittance transmittance 0.4 0.4 0.2 0.2 0 0 400 450 500 550 600 650 700 400 450 500 550 600 650 700 wavelength (nm) wavelength (nm) Design results Optimal filters Optimal Wratten filters Optimal filter pairs for 2-exposure Flash-{Flash,E,A,F,D65} colorimetry systems when (Kt,Ks) = (10-4,1)

  39. 0.1 0.08 0.08 0.06 0.06 sensitivity sensitivity 0.04 0.04 0.02 0.02 0 0 400 450 500 550 600 650 700 400 450 500 550 600 650 700 wavelength (nm) wavelength (nm) Design results (Channelsensitivities) Optimal filters Optimal Wratten filters

  40. Simulation results

  41. 1 0.8 0.6 0.4 0.2 0 0 0.5 1 Experimental results Chromaticity Diagram Experimental results on 278 Munsell color chips using Wratten filter pair (WR11+WR38A,WR80B+WR85N6) y x

  42. Phase II conclusion • Comparison • Filters used in regression-based method is the design from model-based technique

  43. Phase III research • Continuation of Wu’s work. • Modify the measurement setup to achieve higher accuracy. Improve prediction of the system • Make the prediction more robust to different illumination conditions. • Adapt the system for clinical use. • Adopt the statistical approach.

  44. Regression Models • Illuminant-Dependent Colorimeter (IDC): • Slight modification of Wu’s Regression Model, use an affine instead of homogeneous calibration matrix • Only output CIE XYZ values • In this model, we calibrate, measure and match the output under the same illuminant • Prediction is only meaningful under the same lighting condition under which the calibration matrix was derived • Illuminant-Independent Colorimeter (IIC): • Output spectral reflectance • In this model, we calibrate and measure under the same illuminant but can match the output under any illuminant

  45. IIC and IDC for 2-exposure system

  46. Experiment • Apparatus • Kodak DCS460 Digital Camera • Kodak wratten filters (WR11+WR85N6 and WR38A+WR80B) • Photo Research PR705 spectroradiometer • Experiment 1 • 85 extracted teeth were measured under 3 different light sources: • MacBeth Daylight simulator • 2 AC-powered Vivitar 285 flash units • PL900 DC regulated Halogen light with bifurcated fiber optic light guide • Experiment 2 • 21 human subjects were recruited. The top two incisors of each subjects were measured with Halogen Light

  47. Setup For Experiment 1 • For Daylight simulator, the teeth were placed in the viewing booth. The incident angle of the light is approximately 45 degree

  48. Picture for Experiment 1 with flash units

  49. Picture for Experiment 2 with a subject

  50. Performance Evaluation • Experiment 1 • In each random selection, 70 of 85 extracted teeth were randomly picked as training set, the remaining 15 comprised the test set. • Experiment 2 • In each random selection, 34 of 42 live human teeth were randomly picked as training set, the remaining 8 comprised the test set. • In both experiments, 10,000 selections were performed. • The average of each selection was recorded.

More Related