1 / 25

High-resolution Hyperspectral Imaging for Cultural Heritage

High-resolution Hyperspectral Imaging for Cultural Heritage. Rei Kawakami 1 John Wright 2 Yu-Wing Tai 3 Yasuyuki Matsushita 2 Moshe Ben-Ezra 2 Katsushi Ikeuchi 3 1 University of Tokyo, 2 Microsoft Research Asia (MSRA), 3 Korea Advanced Institute of Science and Technology (KAIST)

kolya
Download Presentation

High-resolution Hyperspectral Imaging for Cultural Heritage

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. High-resolution Hyperspectral Imaging for Cultural Heritage Rei Kawakami1 John Wright2 Yu-Wing Tai3 Yasuyuki Matsushita2 Moshe Ben-Ezra2 Katsushi Ikeuchi3 1University of Tokyo, 2Microsoft Research Asia (MSRA), 3Korea Advanced Institute of Science and Technology (KAIST) 2011 Dunhuang Forum

  2. Giga-pixel Camera Large-format lens CCD Giga-pixel Camera M. Ben-Ezra et al.

  3. Spectrum Electromagnetic Spectrum [nm] 0.00001 1 meter 3100 meter 0.001 0.1 10 200 5000 Cosmic Rays Gamma Rays Ultra Violet TV And Radio Waves Electric Waves X-rays Infrared Orange Yellow Green Violet Blue Red 100 80 Solar radiation reaching earth’s surface (Relative Energy) Ultraviolet Visible light Infrared 60 40 20 0 [nm] 200 300 400 500 600 700 1000

  4. RGB vs. Spectrum

  5. Applications Light simulation Layered surface decomposition Morimoto et al. CVPR2010: Estimating Optical Properties of Layered Surfaces Using the Spider Model

  6. Why difficult?

  7. Approach Combine High-res RGB Low-res hyperspectral High-resolution Hyperspectral image

  8. Two-step approach • Factorize low-res hyperspectral image intobasis functions of spectra and coefficients • For each pixel in high-res RGB image, estimate coefficients of the basis functions

  9. Problem formulation Goal: H (Image height) (Spectral wavelength) S W (Image width) Given:

  10. Representation: Basis function Reflectance vectors H (Image height) + + + = x 0 x 1.0 x 0 x 0 S W (Image width) 0 1.0 0 … 0 … =

  11. 1: Matrix factorization • At each pixel of , only a few () materials are present 0 0.4 0 … 0.6 … H (Image height) = Sparse S Reflectance matrix W (Image width) For all pixel (i,j) Sparse matrix

  12. 2: Reconstruction • At each pixel of , materials should be even much fewer H Sparse S Reconstruction W

  13. Simulation experiments Balloons Beads Sponges Oil painting Flowers CD Peppers Face Spectral image database F. Yasuma, T. Mitsunaga, D. Isoand S. K. Nayar. Generalized assorted pixel camera: Postcapture control of resolution, Dynamic range, and spectrum. IEEE Trans. IP, 19(9):2241-2253, 2010

  14. Input images: Balloons and Beads examples Ground truths Reconstruction using component substitution method Reconstruction by the proposed method 460 nm 550 nm RGB/620 nm 460 nm 550 nm RGB/620 nm

  15. Input images: Sponges examples Ground truths Reconstruction by the proposed method Error images of the proposed method 430 nm 490 nm 550 nm 610 nm 670 nm

  16. RGB image

  17. Ground Truth (430 nm)

  18. Estimated 430 nm

  19. RMSE Balloons Beads Sponges Oil painting Flowers CD Peppers Face

  20. HS camera Lens CMOS Aperture Focus Filter Translational stage

  21. Real data experiment Input RGB Input (550nm) Estimated (550nm) Input (620nm) Estimated (620nm)

  22. Summary • Method to reconstruct high-resolution hyperspectral image from • Low-res hyperspectral camera • High-res RGB camera • Spatial sparsity of hyperspectral input • Search for a factorization of the input into • basis functions • set of maximally sparse coefficients

  23. Acknowledgement • This work was in part supported by Microsoft CORE 6 project.

More Related