1 / 50

Optimization of Artificial Neural Networks in Remote Sensing Data Analysis

Optimization of Artificial Neural Networks in Remote Sensing Data Analysis. Tiegeng Ren Dept. of Natural Resource Science in URI (401) 874-9035 tren8835@postoffice.uri.edu 10/07/2002. Outline. Introduction of Satellite Remote Sensing Remote Sensing Image Classification Methodology

quinn-tran
Download Presentation

Optimization of Artificial Neural Networks in Remote Sensing Data Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Optimization of Artificial Neural Networks in Remote Sensing Data Analysis Tiegeng Ren Dept. of Natural Resource Science in URI (401) 874-9035 tren8835@postoffice.uri.edu 10/07/2002

  2. Outline • Introduction of Satellite Remote Sensing • Remote Sensing Image Classification Methodology • Data and Experiments • Discussion and Conclusion

  3. Earth Surface and Land Cover Map How to achieve an accurate land cover map in natural resources research?

  4. Remote Sensing Remote Sensing is defined as: The science of acquiring, processing and interpreting images that record the interaction between electromagnetic energy and matter.

  5. Different Type of Land Use and Land cover and Spectral Characteristics Blue Band Green Band Red Band Near-infrared Band

  6. From Digital Image to Land-use Map • Multi-spectral Digital Image • Pseudo Color Image • Land-use and Land-cover Map Classification Visualization

  7. Classification System • 6. Brush land • 7. Urban land • 8. Water • 9. Non-forest wetland • 10. Forest wetland In this study, we used the USGS classification with 10 land cover categories: 1. Turf/Grass 2. Barren land 3. Conifer forest 4. Deciduous forest 5. Mixed forest

  8. Remote Sensing Image Classification Statistical Methods Supervised Unsupervised Artificial Neural Network Approaches Rule-based ...

  9. Supervised Remote Sensing Image Classification Soil Vegetation Band 2 (0 ~ 255) Band 1 (0 ~ 255) Water

  10. Classification Process Landsat TM Band1 Band2 Band3 Band4 Band5 Band6 Band7 Observation space Solution space Mapping Relationship 0~255 0~255 0~255 0~255 0~255 0~255 0~255 Category 1 Category .. Category … Category … Category … Category … Category N Water wetland Forest Agri. Urban Residential Methods: Statistical classifier ANN-based classifier 40 45 61 193 80 112 25 (Pattern) Category: Forest

  11. Band 2 (0 ~ 255) Vegetation Soil Water Band 1 (0 ~ 255) Statistical Methods - Need Gaussian (Normal) distribution on the input data which is required by Bayesian classifier. - Restrictions about the format of input data.

  12. Conifer forest Deciduous forest Mixed forest Non-forest Wetland Forest Wetland Brush Land Statistical Methods: How to find a boundary for the following patterns?

  13. Artificial Neural Network Approach No need for normal distribution on input data Flexibility on input data format Improved classification accuracy Robust and reliability

  14. Artificial Neural Network Is Defined by ... … … … … … … … … … • Processing elements • Organized topological structure • Learning rules

  15. Processing Element (PE) Artificial counterparts of neurons in a brain PE Wj1 Wj2 Wj3 Output Input  f(x) Wj4 Wj5

  16. … … … … … … … … … PE’s Inputs unitj wj1 o1 wj2 f o2 oj o3 wj3 Function of Processing Elements • Receive outputs from each PEs locate in previous layer. • Compute the output with a Sigmoid activation function F(Sumof(Oi*Wji)) • Transfer the output to all the PEs in next layer PE’s Output

  17. Artificial Neural Network Architecture - Backpropagation ANN (BPANN) Input layer Hidden layer Output layer … … … … … … … … … Input vector i(x1, x2, … xn) Output vector i(o1, o2, … om) Feed Information Analysis Result

  18. Band 2 (0 ~ 255) Vegetation Soil Water Band 1 (0 ~ 255) Pattern Recognition Vegetation: (10, 89) ------> (1,0,0) (11,70) ------> (1,0,0) … … … (1,0,0) Water: (10, 21) ------> (0,1,0) (15, 32) ------> (0,1,0) … … … (0,1,0) Soil: (50, 40) -------> (0,0,1) (52, 40) -------> (0,0,1) … … … (0,0,1)

  19. ANN Design Input layer Hidden layer Output layer Band 2 (0 ~ 255) Vegetation … … … … … … … … (0 ~ 1) Soil (0 ~ 255) (0 ~ 1) (0 ~ 255) (0 ~ 1) How many PEs we need - Basic rules in designing an ANN. Input layer PEs - by dimension of input vector Output layer PEs - by total number of patterns (classes) Water Feed Forward Band 1 (0 ~ 255) Pattern Back-Propagate

  20. ANN Training - From Pattern to Land Cover Category Vegetation: (10, 89) ------> (1,0,0) (11,70) ------> (1,0,0) … … … (1,0,0) Water: (10, 21) ------> (0,1,0) (15, 32) ------> (0,1,0) … … … (0,1,0) Soil: (50, 40) -------> (0,0,1) (52, 40) -------> (0,0,1) … … … (0,0,1) A vegetation pixel (10, 89) ------> (1,0,0) … … … … … … … … 1 (Vegetation) 10 0 (Water) 89 0 (Soil) Feed Forward Back-Propagate Pattern Land Cover Category

  21. Vegetation Soil water After Training A Well-trained ANN … … … … … … … … 1 (Vegetation) A new pixel (x,y), x in band 1, y in band 2 x 0 (Water) y 0 (Soil)

  22. Problems with Traditional Back-Propagation ANN Approaches Time-consuming Always over 10000 iteration, over 5 hours for a small case. Black box - uncontrollable training Easily trapped in local minimum. Training result unpredictable

  23. Optimization Techniques to ANN Approach Data Representation - Transform to Binary and Gray code format. Improved Convergence Algorithms - Apply Conjugate Gradient and Resilient Propagation. Weight initialization - Linear Regression.

  24. Data Representation Data representation - Use Binary format and gray code format instead of integer value. Avoid to compute in the saturation range of the activation function Increased the computation space (from 6 to 48) Make the value more smoothly - Gray Code

  25. Saturation Range of Activation Function - Sigmoid Function Saturation Original Inputs: 0 ~ 255 New Format: 0 or 1

  26. Coding Method Forest Pixel Integer format Binary format 40 45 61 193 80 112 25 0 0 1 0 1 0 0 0 0 0 1 0 1 1 0 1 0 0 1 1 1 1 0 1 1 1 0 0 0 0 0 1 0 1 0 1 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 1 1 0 0 1 6 Integers 48 Integers

  27. Gray Code vs. Binary Format • Gray Code is another coding system, similar to binary • Can better represent continuous value Integer Binary format Gray code format 127 0 1 1 1 1 1 1 1 0 1 0 0 1 0 0 0 128 1 0 0 0 0 0 0 0 0 1 0 1 1 0 0 0

  28. Improved Convergence algorithms To Make the convergence more robust and reliable, apply: • Conjugate Gradient (CG) • Resilient Propagation (RPROP)

  29. Error Space and Weight Adaptive Steps Total Error Wkl Wij

  30. Weight Adaptive with Conjugate Gradient Error Global minimum Wij

  31. Resilient Propagation (RPROP)

  32. Weight initialization Cut convergence time, do weight initialization using linear regression to pre-process the internal weight, make ANN adapted to the target pattern before training.

  33. Data and Experiments Study Area Data Distribution Fine tuning the ANN structure Classification Result

  34. Study Area - Rhode Island 1999 Landsat-7 Enhanced Thematic Mapper Plus (ETM+) Image Band 4,3,2 In RGB Band 5,4,3 In RGB

  35. Training and Testing Pattern

  36. Turf / Grass Barren land Conifer forest Deciduous forest Mixed forest Brush Land Urban area Water Non-Forest Wetland Forest Wetland Distribution of Training Data Patterns Plotted by band 3 x 4 Patterns Plotted by band 4 x 5

  37. ANN Design and Tuning Number of PEs in input layer = Number of spectral Bands in the remote sensing image Number of PEs in output layer = Number of land cover categories Number of PEs in hidden layer(s) to be determined for best performance

  38. ANN Design and Tuning- Before And After Optimization

  39. Classification Result

  40. Classification Result- A Close Look Rhode Island 1999 Land-use and Land-cover map Rhode Island 1999 ETM+

  41. Accuracy Assessment

  42. Discussion and Conclusion • Accuracy • Gray code Vs. Integer format • Robust and reliability • Discuss on number of hidden layer PEs

  43. Accuracy Comparison Non-optimized ANN Accuracy - 79.58% Optimized ANN Accuracy - 92.27%

  44. Gray-code vs. Integer Value Comparison Between Gray Code and Integer Format

  45. Resilient Propagation and Weight Initialization Improvement in robust and reliability

  46. Hidden Layer PE Number’s Influence On Classifier’s Performance. • 48-150-10 ( 48 inputs, 150 hidden neurons, 10 output classes) • 48-250-10 ( 48 inputs, 250 hidden neurons, 10 output classes) • 48-350-10 ( 48 inputs, 350 hidden neurons, 10 output classes)

  47. Hidden Layer PE Number’s Influence On Classifier’s Performance. 350 hidden layer PEs provide the best performance

  48. Conclusion • Coding method, especially the Gray code is important to Remote Sensing image classification. • Optimized ANN provide a robust and reliable classification solution • Optimized ANN lead to higher classification accuracy • Improved natural resources mapping

  49. AcknowledgementThe research was funded by NASA (Grant No. NAG5-8829) to Dr. Y.Q. Wang Dr. Y.Q. Wang (major professor) Dr. Pete August (NRS, committee member) Dr. Ken Yang (EE&CE) Dr. Tom Boving (GEO) Lab for Terrestrial Remote Sensing Environmental Data Center

  50. Thank You!

More Related