Chapter 8

322 Views

Download Presentation
## Chapter 8

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Chapter 8**Geocomputation Part B: Artificial Neural Networks (ANNs) & Genetic Algorithms (GAs) www.spatialanalysisonline.com**Geocomputation: ANNs**In this presentation on geocomputation: ANNs discussed include • Multi-level perceptrons (MLPs) • Radial basis function neural networks (RBFNNs) • Self organising feature maps (SOFMs) ANNs are particularly concerned with • Function approximation and interpolation • Image analysis and classification • Spatial interaction modelling www.spatialanalysisonline.com**Geocomputation: Evolutionary computing**In this presentation on geocomputation: EC elements discussed include • Genetic algorithms (GAs) • Genetic programming (GP) EC is particularly concerned with • Complex problem solving using GAs • Model design using GP methods www.spatialanalysisonline.com**Geocomputation**• Artificial Neural Networks (ANNs) • A computational model based on emulating biological neural networks • A form of non-linear modelling tool • Often a 3-layer network structure is used: input, hidden, output • The output layer of such structures are typically modified weighted sums of intermediate layers, which are modified weighted sums of the input layer www.spatialanalysisonline.com**Artificial Neural Networks**Hence at each output node (hidden or final) a two-step process takes place: www.spatialanalysisonline.com**Artificial Neural Networks**• Simple 3-layer feedforward ANN • Fully inter-connected; each connection is given a weight, w • Known as a Multi-level perceptron (MLP) • In this case: 3 input nodes, 5 hidden nodes, 2 output nodes and 2 bias nodes (bias, B, is similar to the constant term in regression models) • At hidden node 1 we have: where the wij are weights to be determined, b1=1, and the xi are the observed input values www.spatialanalysisonline.com**Artificial Neural Networks**is simply a linear weighted sum of the inputs. To generate a non-linear output it must be modified by some (well behaved) non-linear function, g(), e.g. the logistic function: i.e. Sample activation functions www.spatialanalysisonline.com**Artificial Neural Networks**We can now compute the output layer values as the weighted sum Suppose we have known input values x1=1, x2=-3, x3=5, and known outputs of 0 and 1. Can we select the weights to ensure the inputs generate the known outputs? Suggestion: <build your own worked example & program here> www.spatialanalysisonline.com**Artificial Neural Networks**• Learning • Supervised learning • Split training/test data sets (control data) • Known inputs and output (target) values for training data • (Network output-Target output) = Error signal, e • Systematically adjust weights to minimise sum of e2 • Adjustment typically based on backpropagation and gradient descent • Used in many classification/pattern recognition applications and in function approximation • Unsupervised learning • No training data • Must create clusters by analysing dataset for patterns/clusters www.spatialanalysisonline.com**Artificial Neural Networks**• Some basic issues: • local vs global minimisation • Initialisation and selection • Data normalisation and coding • Momentum • Model design and over-fitting • Overtraining • Interpolation vs Extrapolation/Forecasting www.spatialanalysisonline.com**Artificial Neural Networks**• MLP: Example 1 function approximation source data fitted solution curve RMSE vs epochs www.spatialanalysisonline.com**Artificial Neural Networks**• MLP Example 2: LCM www.spatialanalysisonline.com**Artificial Neural Networks**• MLP Example 2: LCM www.spatialanalysisonline.com**Artificial Neural Networks**• MLP Example 2: LCM www.spatialanalysisonline.com**Artificial Neural Networks**• MLP Example 2: LCM weights matrix www.spatialanalysisonline.com**Artificial Neural Networks**• MLP Example 3: Spatial interaction model • Generalised model: Tij=f(Oi,Dj,dij) • Sample data format (log transformed): www.spatialanalysisonline.com**Artificial Neural Networks**• MLP Example 3: Spatial interaction model www.spatialanalysisonline.com**Artificial Neural Networks**• Radial Basis Function Networks Basic functional form: Gaussian RBF: www.spatialanalysisonline.com**Artificial Neural Networks**• Self organising function maps • SOM as an output space • Neighbourhood relations • Grid size, form and topology www.spatialanalysisonline.com**Artificial Neural Networks**• Self organising function maps • Dimensional reductions • Mapped output – similar vectors (units) are close to each other • Typically an unsupervised procedure • Spatial mapping of SOM can follow using simple assignment to best matching unit (BMU) www.spatialanalysisonline.com**Artificial Neural Networks**• Self organising function maps • Choose a grid size, form and topology • Train the network • Identify the best matching units • Modify the BMU and its neighbours (spatially biased learning rule) • Map the trained network www.spatialanalysisonline.com**Artificial Neural Networks**• Self organising function maps – some issues • Initialisation • Pre-processing • Normalisation • Missing data • Masking and weighting • Learning and tuning • Distance metrics • Neighbourhood functions (kernels) • Learning rate functions www.spatialanalysisonline.com**Artificial Neural Networks**• Self organising function maps – Idrisi www.spatialanalysisonline.com**Artificial Neural Networks**• Self organising function maps – Idrisi www.spatialanalysisonline.com**Genetic Algorithms**• Solutions are represented as individuals • Individuals are modelled as chromosomes • Chromosomes are comprised of genes • Genes have values known as alleles • Chromosomes have a measurable fitness • New chromosomes (children) are created by reproduction and mutation processes • The fittest individuals survive • The creation process is then iterated www.spatialanalysisonline.com**Genetic Algorithms**• GAs: Example 1 - TSP allele=12 (ID of town in TSP problem set) chromosome genes • Each chromosome contains complete list of towns • create a set of m randomly permuted strings and compute lengths, d • evaluate the fitness of each string (e.g. 1/d) • select random pairs of tours (biased by fitness) • combine pairs by crossover operation • evaluate fitness of offspring • apply replacement rule (fittest retained) and iterate till stable www.spatialanalysisonline.com**Genetic Algorithms**• GA components • Encoding or representation – binary, list, tree etc • Fitness function selection – use of rank transforms • Population initialisation • Selection: roulette, tournament, uniform random • Reproduction • Crossover e.g. A = [a b c d e f g h] B = [1 2 3 4 5 6 7 8] and the crossover point is 3, the following children are generated: child 1 = [a b c 4 5 6 7 8] child 2= [1 2 3 d e f g h] • Mutation • Local search • Termination www.spatialanalysisonline.com**Genetic Algorithms**• GAs: application areas • TSP (as above) • Clustering • Map labelling • Optimum location with capacity constraints • Concept can be extended to alleles that are expressions or program elements rather than numerical values Genetic programming www.spatialanalysisonline.com