Glasses Removal from Facial Image Using Recursive Error Compensation

1. Glasses Removal from Facial ImageUsing Recursive Error Compensation 作者：Jeong-Seon Park, You Hwa Oh, Sang Chul Ahn, Seong-Whan Lee 讲解人：卿来云

2. 作者的相关信息 • Jeong-Seon Park • Computer Science and Engineering, Korea University • 文章列表 • Jeong-Seon Park, Seong-Whan Lee: Resolution Enhancement of Facial Image Using An Error Back-Projection of Example-based Learning. FGR 2004: 831-836 • Jeong-Seon Park, Seong-Whan Lee: Enhancing Low-Resolution Facial Images Using Error Back-Projection for Human Identification at a Distance. ICPR (1) 2004: 346-349 • Seong-Whan Lee, Jeong-Seon Park, Bon-Woo Hwang: How Can We Reconstruct Facial Image from Partially Occluded or Low-Resolution One?. SINOBIOMETRICS 2004: 386-399 • Jeong-Seon Park, You Hwa Oh, Sang Chul Ahn, Seong-Whan Lee: Glasses Removal from Facial Image Using Recursive PCA Reconstruction. AVBPA 2003: 369-376 • 研究方向/学术贡献点：图像重构（误差处理） • Seong-Whan Lee • 研究方向：计算机视觉（相机标定、跟踪）和模式识别（手写体识别、人脸识别等） • 主要文章发表在PAMI、PR、IJPRAI和AVBPA、ICPR上

3. 文章出处 • IEEE Trans. On PAMI • Jeong-Seon Park, You Hwa Oh, Sang Chul Ahn, and Seong-Whan Lee. Glasses Removal from Facial Image Using Recursive Error Compensation. IEEE Trans. On PAMI, 27(5): 805~811, 2005 • Submitted Aug. 2003 • Revised Oct. 2004 • Accepted Nov. 2004 • Recommended for acceptance by K. Yamamoto. • AVBPA • J.-S. Park, Y.-H. Oh, S.-C. Ahn, and S.-W. Lee. “Glasses Removal from Facial Image Using Recursive PCA Reconstruction, Proc. Int’l Conf. Audio- and Video-based Biometric Person Authentication, pp. 369-376, June 2003.

4. 摘要 • 本文提出了一种从一幅人脸图像中去除眼镜的新方法。首先我们检测被眼镜遮挡的区域并通过对PCA重构的图像进行误差补偿产生一幅看起来很自然的人脸图像。结果图像没有眼镜的痕迹或者由眼镜引起的反射和影调。实验结果表明本方法 对眼镜遮挡问题提出了一种有效的解决方案，而且我们相信本方法能用来提高人脸识别系统的性能。

5. 文章结构 • 1 INTRODUCTION • 2 GLASSES REMOVAL METHODS • 2.1 Extraction of Glasses Region • 2.2 PCA Reconstruction Method • 2.3 Example-Based Reconstruction Method • 3 RECURSIVE ERROR COMPENSATION PROCEDURE • 3.1 Proposed Recursive PCA Reconstruction • 3.2 Advantages of the Proposed Recursive PCA Reconstruction • EXPERIMENTAL RESULTS AND ANALYSIS • 4.1 Experimental Data • 4.2 Experimental Results and Discussion • 5 CONCLUDING REMARKS AND FURTHER RESEARCH

6. Introduction • 问题：眼睛遮挡 • 对成功的人脸识别系统很重要 • 可能的解决方案 • 预处理：重构非遮挡人脸图像 • PCA reconstruction (Saito, Kenmochi, and Kotani, ICIP99) • 仍留有眼镜痕迹 • LLE based on morphable model (Hwang and Lee, PAMI03) • 点对点对齐 • 只用可见部分识别 • Local match (Martinez, PAMI02)

7. PCA Based Glass Removal Methods (1) • PCA Reconstruction Method • Framework • Experimental Results • Limits • Depends on training set：由于训练集没有眼镜，重构时误差会扩散到整个人脸图像

8. PCA Based Glass Removal Methods (2) • Example-Based Reconstruction Method • Framework • Experimental Results

9. Recursive PCA Reconstruction (1) • Framework (a) PCA reconstruction (b) Difference (c) Difference stretching (d) Difference enhancing (e) Error compensating

10. Recursive PCA Reconstruction (2) • Recursive error compensating • Iteration stops if 其中 被遮挡 不确定 无遮挡

11. Recursive PCA Reconstruction (3) • More Results Input faces Regions of glasses PCA reconstruction Final compensated faces

12. Experimental Results and Analysis (1) • Experimental Data • KFDB (Korean Face Database) • Frontal • 300 people with/without glasses • Numerous kinds of glasses • 3 illumination conditions • Training set: 200 people • Test set: 100 people

13. Experimental Results and Analysis (2) • Experimental results Input faces PCA reconstruction Example-based method Recursive PCA Original faces without glasses

14. Experimental Results and Analysis (3) • Face recognition

15. 对文章的评价 • 文章的贡献 • 提出了一种误差补偿跌代减小重构误差的方法 • 对其他涉及重构的问题有一定借鉴意义 • 优缺点 • 优点:简单 • 缺点:实验有限，尚不能有力证明方法的有效性 • 可借鉴的东西： • 研究一个比较新的问题容易被关注 • 解决新问题时，首先考虑利用老方法 • 误差补偿方法用于其他遮挡重构问题 • 光照补偿中的噪声去除、姿态遮挡、其他饰物遮挡…

16. 参考文献 • 自引太多 • 缺：Chenyu Wu, Ce Liu, Heung-Yueng Shum, Ying-Qing Xu, Zhengyou Zhang. "Automatic Eyeglasses Removal from Face Images," IEEE Trans. on PAMI, 26(3): 322-336,  March  2004.