Image Mosaic Techniques for the Restoration of Virtual Heritage

1. Image Mosaic Techniquesfor the Restoration of Virtual Heritage 2003. 8. 28 Yong-Moo Kwon, Ig-Jae Kim, Tae-Sung Lee, Se-Un Ryu, Jae-Kyung Seol KIST KOREA

2. Contents Revisiting Image Mosaic Technique Our Researches for Image Mosaic IR Reflectography Image Mosaic X-Ray Image Mosaic Summary

3. Revisiting Image Mosaic Technique Image Mosaicing Panorama Image Image Based Rendering (IBR) Basic Algorithm Registration using Features Image Warping based on Homography Matrix Blending Images

4. Target Dimension in view of Image Mosaicing 2D Target Planar Paintings Image Homography Technique Feature-Based Image Mosaicing 3D Target 3D Real World Image Limitation using Homography Due to Depth Difference b/w Features in Target

5. Our Research for Image Mosaic 2D Target IR Reflectography Mural Underdrawings Mosaic Special 3D Target X-Ray Imaging Old Sword X-Ray Image Mosaic Research Topics How to extract and use Features Imaging Media (IR, X-Ray) Feature’s characteristics are different from the previous ones

6. IR Image Mosaic

7. IR Reflectography System • IR Source • IR Filter • IR Camera Murals

8. IR Reflectography Principle Visible Light IR Color Painting, Dust Reflection Reflection UnderDrawing Absorbed Back Frame

9. IR Reflectography Camera • IR Camera : Super eye C2847 (~1.9㎛) • Hamamatsu • IR Source : ~1.9㎛ • IR Filter : CVI Laser Corp. • NIR bandwidth filter • 800nm ~ 2000nm Pass • Bandpass Filter • every 100 nm bandpass filter (800nm, 900, …, 2000nm) • IR Characteristics to Mural according to WL

10. IR Camera • HAMAMATSU Super eye C2847 • WL Range : 0.4㎛ ~ 1.9㎛ • IR Source • HAMAMATSU C1385-02

11. Filter • HAMAMATSU IR-D80A : 0.8㎛ ~ 1.9㎛ • CVI Laser corporation • Near IR Interference BP filter 800nm, 900nm, … 2000nm • Sony PC-115 • Digital Image Capture • Night Shot

12. IR Image Mosaic for Mural Underdrawing • Basic Method • Automatic Feature Extraction • Registration Using Features • Image Warping • Image Blending • Main Considerations • IR Wavelength Characteristics • Penetration Ratio into Paintings • Color (Red, Green, Blue etc) • Color Painting Depth

13. Our Approach • ▶ Automatic Feature Extraction & Registration • - Cross Points in IR Underdrawing Image • - Grid Pattern for Blank Space • ▶ Adaptive Overlapping Area For Image Blending • - Trade-Off between Registration and Blending • * Large Overlapping Area: Good for Registration • * Small Overlapping Area: Good for Blending • ▶ Use feature of IR Spectrum • - Use Different IR Wavelength according to paining color

14. Automatic Feature Extraction  Feature of Korea Murals - Many Blank Space - Not so much good features 1> Visible Light Pattern 2> Twice Captures - w/o IR Filter - w/i IR Filter

15. IR Image Mosaic - Homography Estimation using Grid Image & IR Image - Apply Homography to IR Image

17. X-RAY Image Mosaic

18. Why we use X-ray Technique ? • Old Sword • Old Sword is inside Sword Cover • Weak for Touch & Manipulation • Can’t Open Sword Cover • Use X-Ray Technique for the restoration of Old Sword inside Sword Cover

19. Sword for experiment

20. Schema of a x-ray imaging using a linear X-Ray Camera • X-Ray Image • X-Ray Tube • X-Rays • X-Ray Detector • PC • Object

21. Why X-Ray Image Mosaic ? For High Resolution Imaging Multiple X-Ray Imaging Setting Object X-Ray Image Capture Move Object Upward or Downward Step-By-Step Stitching X-Ray Images into High Resolution Image

22. X-Ray Imaging Principle Basic Principle X-Ray Particle Penetrates through Target One Point Depth -> Grey Value Pixel Dependency Target Depth Target Material

23. X-Ray Image Characteristics: 2D or 3D ? Target Dimension in view of Image Mosaic Well Controlled Penetration Angle Image Pixel Depends on Penetration Angle Usually Same Penetration Angle for Each Capture Orthogonal axis Movement according to X-Ray Beam Just Planar 2D Image Using CCD Camera Object -> X-Ray Camera -> CCD Camera 2D Target: Homography Technique

24. X-RAY Image Equipment X-TEK X-Ray System X-Ray Source & Object (Sword)

25. X-RAY Image Capture • For High-Resolution Restoration • Multiple X-Ray Imaging • Image Stitching Technique • Feature-based Registration • Problem ? • Difficult to use features in X-ray Image • Using Feature Pattern

26. Feature Extraction • Feature Extraction From Known Pattern • Circle Type & Rectangular Type • Circle Type -> Pattern Matching • Rectangular -> Feature Points

27. Feature Extraction • Method • Circle Type Pattern -> Apply Image Labeling • Rectangular Type Pattern -> Corner Detection - For every pixel of image, computes first derivatives Dx and Dy. - The eigenvalues are found by solving det(C- λI )= 0 If λ1, λ2> t, where t is some threshold, then a corner is found at that location

28. Feature Point Matching Semi-auto Demo • Semi-Auto(Present) • Automatic Feature Extraction of Rectangle Type pattern • Manual Matching • Automatic Matching (On-going) • Classify the features using pattern ID from Circle Type Pattern • Homography Matrix • Apply LS-Method(Least Square Method) using Matched feature Points

29. Implemented S/W X-ray Image File Handling Feature Extraction & Select Points Homography Matrix Estimation & Stitching Generated High-Resolution X-ray Image

30. More Experimentation

31. Summary Application of Image Mosaicing Techniques Infrared Image X-Ray Image Our Approach Feature Pattern Automatic Feature Extraction & Registration Homography Technique Imaging Media (IR, X-Ray) & Feature’s Characteristics

32. Thank You ! ymk@cherry.kist.re.kr