280 likes | 441 Views
High-resolution Hyperspectral Imaging for Cultural Heritage. Rei Kawakami 1 John Wright 2 Yu-Wing Tai 3 Yasuyuki Matsushita 2 Moshe Ben-Ezra 2 Katsushi Ikeuchi 3 1 University of Tokyo, 2 Microsoft Research Asia (MSRA), 3 Korea Advanced Institute of Science and Technology (KAIST)
E N D
High-resolution Hyperspectral Imaging for Cultural Heritage Rei Kawakami1 John Wright2 Yu-Wing Tai3 Yasuyuki Matsushita2 Moshe Ben-Ezra2 Katsushi Ikeuchi3 1University of Tokyo, 2Microsoft Research Asia (MSRA), 3Korea Advanced Institute of Science and Technology (KAIST) 2011 Dunhuang Forum
Giga-pixel Camera Large-format lens CCD Giga-pixel Camera M. Ben-Ezra et al.
Spectrum Electromagnetic Spectrum [nm] 0.00001 1 meter 3100 meter 0.001 0.1 10 200 5000 Cosmic Rays Gamma Rays Ultra Violet TV And Radio Waves Electric Waves X-rays Infrared Orange Yellow Green Violet Blue Red 100 80 Solar radiation reaching earth’s surface (Relative Energy) Ultraviolet Visible light Infrared 60 40 20 0 [nm] 200 300 400 500 600 700 1000
Applications Light simulation Layered surface decomposition Morimoto et al. CVPR2010: Estimating Optical Properties of Layered Surfaces Using the Spider Model
Approach Combine High-res RGB Low-res hyperspectral High-resolution Hyperspectral image
Two-step approach • Factorize low-res hyperspectral image intobasis functions of spectra and coefficients • For each pixel in high-res RGB image, estimate coefficients of the basis functions
Problem formulation Goal: H (Image height) (Spectral wavelength) S W (Image width) Given:
Representation: Basis function Reflectance vectors H (Image height) + + + = x 0 x 1.0 x 0 x 0 S W (Image width) 0 1.0 0 … 0 … =
1: Matrix factorization • At each pixel of , only a few () materials are present 0 0.4 0 … 0.6 … H (Image height) = Sparse S Reflectance matrix W (Image width) For all pixel (i,j) Sparse matrix
2: Reconstruction • At each pixel of , materials should be even much fewer H Sparse S Reconstruction W
Simulation experiments Balloons Beads Sponges Oil painting Flowers CD Peppers Face Spectral image database F. Yasuma, T. Mitsunaga, D. Isoand S. K. Nayar. Generalized assorted pixel camera: Postcapture control of resolution, Dynamic range, and spectrum. IEEE Trans. IP, 19(9):2241-2253, 2010
Input images: Balloons and Beads examples Ground truths Reconstruction using component substitution method Reconstruction by the proposed method 460 nm 550 nm RGB/620 nm 460 nm 550 nm RGB/620 nm
Input images: Sponges examples Ground truths Reconstruction by the proposed method Error images of the proposed method 430 nm 490 nm 550 nm 610 nm 670 nm
RGB image
Ground Truth (430 nm)
Estimated 430 nm
RMSE Balloons Beads Sponges Oil painting Flowers CD Peppers Face
HS camera Lens CMOS Aperture Focus Filter Translational stage
Real data experiment Input RGB Input (550nm) Estimated (550nm) Input (620nm) Estimated (620nm)
Summary • Method to reconstruct high-resolution hyperspectral image from • Low-res hyperspectral camera • High-res RGB camera • Spatial sparsity of hyperspectral input • Search for a factorization of the input into • basis functions • set of maximally sparse coefficients
Acknowledgement • This work was in part supported by Microsoft CORE 6 project.