Loading in 5 sec....

Neural Networks and Classical Linear RegressionPowerPoint Presentation

Neural Networks and Classical Linear Regression

- 86 Views
- Uploaded on
- Presentation posted in: General

Neural Networks and Classical Linear Regression

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Neural Networks and Classical Linear Regression

Szu Hua Huang Jianjun Luo

Texas Tech University

6/10/2014

Contents

- Neural Networks and Classical Linear Regression
- Introduction Neural Networks
- Neural Networks VS Classical Linear Regression

- Case study

STRUCTURE OF A NEURAL NETWORK

Multilayer feedforward networks

A simple MLP (multilayer perceptron)

Hidden Layer

Output Layer

Input Layer

Weights(Regression Coefficients)

Back-propagation Model

Updating (Learning)

3

W13

1

W36

Node

W14

W15

6

W46

Output

Input

4

W23

W56

W24

2

Data from True Function

W25

5

Errors

Transfer Function, g(s)

Neural Network VS Classical Regression Model

Neural Network

Classical Regression

Linear

Normality

Constant variability of error terms

Independence Assumption

- Non-linear

Neural Network VS Classical Regression Model

Neural Network

Classical Regression

Regression coefficient shows the effect

- Weight estimates(regression coefficient) do not tell you the effect
- No guarantee the best linear combination of parameter estimates

Neural Network VS Classical Regression Model

Multi-collinearity

Outlier or Influential

No effect to Neural Network ?

It does hurt the Classical Regression

- No effect to Neural Network ?
- It does hurt the Classical Regression

Overfitting

Neural Network

Classical Regression

Not very sensitive to the given data.

- Sensitive to the given data, too much flexibility to the underlying distribution of data.
- Big Sample size can help to solve the problem of overfitting.

Assessment

Neural Network

Classical Regression

R square

- Optimization plot based on the updated weight estimates at each iteration of the iterative grip search routine.
- Using valid data

- 1. When the new weights are only incrementally different from those of the preceding iteration
- 2. When the misclassification rate reaches a required threshold
- 3. When the limit on the number of runs is reached

ASE

training

validation

0

5

10

15

20

Iteration

Supplement-Other Optimization Algorithms:

- Newton
- Quasi-Newton method
- Levenberg-Marquardt
- Gauss-Newton Method
- etc
Stanford Open Course – Machine Learning

- Dataset
- The School Children Data Set from Lewis & Taylor “Introduction to Experimental Ecology” (1967)
- Includes 126 male records
- Variables:
- Age (months)
- height (inches)
- weight (pounds)

- Predicting the weight of male school children based on their age and height.
- Comparing neural networks with OLS

Classical Linear Regression Model

procreg data=men;

model weight=height age;

output out=regout p=pred r=resid;

run;

INPUT

OUTPUT

HIDDEN

COMBINATION

w1+w2S_Height+w3S_Age = H11

TRANSFORMATION

tanh(H11)) =A

Standardization

H11

Height

S_Height

Weight

COMBINATION

w7+w8A+w9B=Weight

S_Age

Age

Standardization

TRANSFORMATION

tanh(H12) =B

COMBINATION

w4+w5S_Height+w6S_Age = H12

H12

18

Neural Network in SAS SAS Enterprise Miner

- The SAS neural network procedure
- PROC NEURAL

- A visual programming with a GUI interface

Neural Network Modeling using SAS Enterprise Miner

- To save time, I recorded the following video to show how to build the Neural Network Model with SAS Enterprise Miner.
- In case you are interested, I uploaded this video to YouTube:
http://www.youtube.com/watch?v=Bb3K7xAcJbk&feature=youtu.be

Comparing Neural Network and Classical Linear Regression predicted values

Output: predicted valuesObserved and Predicted Values of Male's Weight against Age

References predicted values

Eric Roberts. Neural networks. Available online at: http://www-cs-faculty.stanford.edu/~eroberts/courses/soco/projects/neural-networks/

Jim Georges, 2009. Applied analytics using SAS Enterprise Miner 6.1 Course Notes. SAS Institute Inc.

Lewis, T. and Taylor, L.R. 1967. Introduction to Experimental Ecology, Academic Press, Inc.

Randall Matignon, 2005. Neural Network Modeling using SAS Enterprise Miner. AuthorHouse

SAS Institute, 1999. SAS/STAT User’s Guide Version 8. Available online at: http://ciser.cornell.edu/sasdoc/saspdf/common/mainpdf.htm

Sue Walsh, 2002. Applying Data Mining Techniques Using Enterprise Miner Course Notes. SAS Institute Inc.

Wikipedia. Neural network. Available online at: http://en.wikipedia.org/wiki/Neural_network

Thank You! predicted values