Domain Adaptive Faster R-CNN for Object Detection in the Wild

1. Domain Adaptive Faster R-CNN for Object Detection in the Wild CVPR 2018 李一帆 2019年1月16日

2. Introduction • Object detection in the real world faces challenges from the large variance in viewpoints, object appearance, backgrounds, illumination, image quality, etc., which may cause a considerable domain shift between the training and test data. • We tackle the domain shift on two levels:1) the image-level shift, such as image style, illumination, etc., and 2) the instance-level shift, such as object appearance, size, etc.

3. Introduction Contribution : • 1) We provide a theoretical analysis of the domain shift problem for cross-domain object detection from a probabilistic perspective. • 2) We design two domain adaptation components to alleviate the domain discrepancy at the image and instance levels, resp. • 3) We further propose a consistency regularization to encourage the RPN to be domain-invariant. • 4)We integrate the proposed components into the Faster R-CNN model, and the resulting system can be trained in an end-to-end manner.

4. Preliminaries • the baseline model: Faster R-CNN • a two-stage detector mainly consisting of three major components: shared bottom convolutional layers, a region proposal network (RPN) and a region-of-interest (ROI) based classifier.

5. Preliminaries • The H-divergence is designed to measure the divergence between two sets of samples with different distributions. • Let us denote by x a feature vector. A source domain sample can be denoted as xS and a target domain sample as xT . We also denote by h : x→｛0,1｝a domain classifier, which aims to predict the source samples xS to be 0, and target domain sample xT to be 1.

6. Domain Adaptation • Image-Level Adaptation: P(C,B,I) = P(C,B|I)P(I) • I :feature map • B: the bounding-box of an object • C:the category of the object • 根据 Covariate Shift Assumption，条件概率P(C,B|I) 在目标域和源域应该是一样的，所以只需要对齐 P(I) 即PS(I) = PT (I)。

7. Domain Adaptation • Instance-Level Adaptation: P(C,B,I) = P(C|B,I)P(B,I) • 同样根据 Covariate Shift Assumption，条件概率P(C|B,I) 在目标域和源域应该是一样的，所以只需要对齐 P(B, I) 即PS(B, I) = PT (B, I) 。

8. Domain Adaptation • Joint Adaptation: P(B,I) = P(B|I)P(I) P(B|I)对应的是bounding boxes预测器，即 Faster RCNN中的RPN(region proposal network) • the bounding box annotation is only available for source domain training data, therefore P(B|I) is learned using the source domain data only, which is easily biased toward the source domain.

9. Domain Adaptation • Joint Adaptation(consistency regularization): • 想要对齐两个数据域，需要训练一个域分类器h(x), x 可以是image-level的表示 I 或者 instance-level级别的(B, I)。h(x) 就代表一个样本x属于目标域的概率。用 D 来代表域标签，则Image-level分类器可以表示为P(D|I)， Instance-level 可以表示为P(D|B,I) • 使用Bayes公式得到：P(D|B,I)P(B|I) = P(B|D,I)P(D|I) • 通过使得两个域分类器一致，即P(D|B,I) = P(D|I)， 可以使得P(B|D,I)接近P(B|I)

10. Loss Function • image-level adaptation loss： • instance-level adaptation loss：

11. Loss Function • consistency regularization : • final training loss:

12. Model • An overview of our Domain Adaptive Faster R-CNN model:

13. Experiments • From synthetic data to real data • Datasets: SIM 10k(source domain) Cityscapes(target domain) • Results:

14. Experiments • From Cityscapes data to Foggy Cityscapes • Datasets: Cityscapes(source domain) Foggy Cityscapes(target domain) • Results:

15. Experiments • Between two real datasets • Datasets: KITTI Cityscapes • Results:

16. Thanks!