Begum Demir Francesca Bovolo Lorenzo Bruzzone

1. Detection of Land-Cover Transitions in Multitemporal Remote Sensing Images with Active Learning Based Compound Classification Begum Demir Francesca Bovolo Lorenzo Bruzzone E-mail: demir@disi.unitn.it Web page: http://rslab.disi.unitn.it

2. Outline Introduction 1 Aim of the work 2 Proposed Joint Entropy based Active-Learning Method for Compound Classification 3 4 Experimental results Conclusion and future developments 5 B. Demir, F. Bovolo, L. Bruzzone

3. Introduction • Detection of land-cover transitions between a pair of remote sensing images acquired on the same area at different times (i.e., multitemporal images) is very useful in many applications. • Usually, this is achieved by supervised classification techniques, as unsupervised change detection methods have a reduced reliability in detecting explicitly different land-cover transitions. • Such an approach requires ground reference data to detect changes and identify transitions . • Due to the properties of the last generation of VHR passive sensors, supervised change-detection methods in real applications is becoming more and more important. • Problem: The collection of a large multitemporalreference data is time consuming and • expensive. B. Demir, F. Bovolo, L. Bruzzone

4. Aim of the Work • Goals • Compute a map of land-cover transitions betweena pair of remote sensing images acquired on the same area at different times. • Take advantage of temporal dependence between images. • Define a training set as small as possible. • Assumptions • The same set of land-cover classes characterizes the images. • Initial training set with small number of labeled samples is available. • Solution: Develop a novel Active Learning (AL) technique for compound classification of multitemporal remote sensing images that takes advantage of the temporal dependence among images. B. Demir, F. Bovolo, L. Bruzzone

5. Active Learning for Single Image Classification General AL Scheme U I Expanded T T Update T G G Q S G: Supervised classifier; Q: Query function; S: Supervisor; T: Training set; U: Unlabeled data I: Image [1] S. Rajan, J. Ghosh, and M. M. Crawford, “An active learning approach to hyperspectral data classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 46, no. 4, pp. 1231-1242, Apr. 2008. [2]B. Demir, C. Persello, and L. Bruzzone, “Batch mode active learning methods for the interactive classification of remote sensing images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 49, no.3, pp. 1014-1031, March 2011. B. Demir, F. Bovolo, L. Bruzzone

6. Proposed Method: Block Scheme Training Set (pairs of temporally correlatedlabeledsamples) Active Learning X1 t1 image Expanded Training Set Compound Classifier X2 Different kinds of changes t2 image Map of land-cover transitions B. Demir, F. Bovolo, L. Bruzzone 6

7. Proposed Method: Block Scheme Training Set (pairs of temporally correlatedlabeledsamples) Active Learning X1 t1 image Expanded Training Set Compound Classifier X2 Different kinds of changes t2 image Map of land-cover transitions B. Demir, F. Bovolo, L. Bruzzone 7

8. Proposed Method: Compound Classifier Estimation of Classifier Parameters X1 Training Set (pairs of temporally correlated labeled samples) t1 image Compound Classifier Different kinds of changes Map of land-cover transitions X2 Joint posterior probability t2 image Bayesiandecision rule forcompound classification: Number of classes L. Bruzzone, D. Fernandez Prieto, and S.B. Serpico, “A neural-statistical approach to multitemporal and multisource remote-sensing image classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 37, no.3, pp. 1350-1359, May 1999. B. Demir, F. Bovolo, L. Bruzzone

9. Proposed Method: Compound Classifier Assumption: class-conditional independence in the time domain Joint class-conditional density Joint prior probability Joint prior probabilities of land-cover transitions can be estimated on the basis of the expectation-maximization (EM) algorithm: Image size L. Bruzzone, D. Fernandez Prieto, and S.B. Serpico, “A neural-statistical approach to multitemporal and multisource remote-sensing image classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 37, no.3, pp. 1350-1359, May 1999. B. Demir, F. Bovolo, L. Bruzzone

10. Proposed Method: Block Scheme Training Set (pairs of temporally correlatedlabeledsamples) Active Learning X1 t1 image Expanded Training Set Compound Classifier X2 Different kinds of changes t2 image Map of land-cover transitions B. Demir, F. Bovolo, L. Bruzzone 10

11. Proposed Method: Active Learning Class conditional densities X1 Estimation of Statistical Distributions Joint Entropy Joint Entropy t1 image Training Set X2 Joint prior probability Joint prior probability Uncertain Samples Selection t2 image Update Training Set No Yes Expanded Training Set Convergence? Proposed AL Procedure B. Demir, F. Bovolo, L. Bruzzone

12. Proposed Method: Active Learning • We propose to use the joint entropy to measure the uncertainty: • If joint entropy is small, the corresponding pair of pixels will be classified with high confidence, i.e., the decision on compound classification of these samples is reliable. • If joint entropy is high, the decision is not reliable, and therefore the corresponding pair of samples is considered as uncertain and critical for the classifier. Joint entropy Joint posterior probability B. Demir, F. Bovolo, L. Bruzzone 12

13. Proposed Method: Active Learning • We adopted two possible simplifying assumptions that result in two different algorithms of the proposed AL technique: • Algorithm (JEAL) is defined under the assumption of class-conditional independence: Classconditional densities Joint prior probability B. Demir, F. Bovolo, L. Bruzzone

14. Proposed Method: Active Learning • Algorithm (JEALInd) is defined under the assumption of temporal independence: + the independence of a-priori class probabilities on the two images the class-conditional independence Classconditional densities Joint prior probability Marginal entropies B. Demir, F. Bovolo, L. Bruzzone

15. Proposed Method: Active Learning Algorithm (JEAL) is defined under the assumption of class-conditional independence: Algorithm (JEALInd) is defined under the assumption of temporal independence: Mutual information B. Demir, F. Bovolo, L. Bruzzone

16. Experimental Setup • Two different multitemporal and multispectral data sets are used (one made up of very high resolution images and one made up of medium resolution images). • Class conditional densities are estimated from the available initial training set assuming Gaussian distribution. • Results achieved with the proposed method are compared with Standard Marginal-Entropy based AL technique applied to the post classification comparison ruleignoring temporal dependence (Fully Independent). B. Demir, F. Bovolo, L. Bruzzone

17. Data Set Description Multitemporal data set: Two images acquired by the TM sensor of Landsat-5 satellite in September 1995 and July 1996 (Lake Mulargia, Sardinia Island, Italy). Land–coverclasses: Pasture, Forest, Urban Area, Water, Vineyard September 1995 July 1996 B. Demir, F. Bovolo, L. Bruzzone

18. Experimental Results Proposed 1=Proposed JEAL method defined under the assumption of class-conditional independence. Proposed 2=Proposed JEAL method defined under the assumption of temporal independence. B. Demir, F. Bovolo, L. Bruzzone

19. Experimental Results Proposed 1=Proposed JEAL method defined under the assumption of class-conditional independence. Fully Independent=Standard Marginal-Entropy based AL technique applied to the post classification comparison rule ignoring temporal dependence. B. Demir, F. Bovolo, L. Bruzzone

20. Conclusion • A novel AL method has been defined on the basis of joint entropy defined in the context of compound classification for the detection of land-cover transitions. • Two different joint entropy based AL algorithms are implemented under two possible simplifying assumptions: i) the class-conditional independence; and ii) the temporal independence between multitemporal images. • Experiments show that the proposed joint entropy based AL technique, which takes advantage of temporal correlation, gives higher accuracies in detection of transitions. • Proposed AL method • decreases significantly the cost and effort required for multitemporal reference data collection; • achieves high accuracy with a minimum number of multitemporal reference samples; • improves the performance of the standard marginal entropy based active learning method by exploiting temporal dependence between images. B. Demir, F. Bovolo, L. Bruzzone

21. Future Development • Extend the proposed active learning algorithms • by including a diversity criterion defined in the context of compound classification. • considering label acquisition costs, which depend on locations and accessibility of the visited points for labeling the uncertain samples. B. Demir, F. Bovolo, L. Bruzzone