- 72 Views
- Uploaded on
- Presentation posted in: General

Neural Networks and Classical Linear Regression

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Neural Networks and Classical Linear Regression

Szu Hua Huang Jianjun Luo

Texas Tech University

6/10/2014

- Neural Networks and Classical Linear Regression
- Introduction Neural Networks
- Neural Networks VS Classical Linear Regression

- Case study

Multilayer feedforward networks

Hidden Layer

Output Layer

Input Layer

Weights(Regression Coefficients)

Back-propagation Model

Updating (Learning)

3

W13

1

W36

Node

W14

W15

6

W46

Output

Input

4

W23

W56

W24

2

Data from True Function

W25

5

Errors

Transfer Function, g(s)

Neural Network

Classical Regression

Linear

Normality

Constant variability of error terms

Independence Assumption

- Non-linear

Neural Network

Classical Regression

Regression coefficient shows the effect

- Weight estimates(regression coefficient) do not tell you the effect
- No guarantee the best linear combination of parameter estimates

Multi-collinearity

Outlier or Influential

No effect to Neural Network ?

It does hurt the Classical Regression

- No effect to Neural Network ?
- It does hurt the Classical Regression

Neural Network

Classical Regression

Not very sensitive to the given data.

- Sensitive to the given data, too much flexibility to the underlying distribution of data.
- Big Sample size can help to solve the problem of overfitting.

Neural Network

Classical Regression

R square

- Optimization plot based on the updated weight estimates at each iteration of the iterative grip search routine.
- Using valid data

Optimization plot

- 1. When the new weights are only incrementally different from those of the preceding iteration
- 2. When the misclassification rate reaches a required threshold
- 3. When the limit on the number of runs is reached

ASE

training

validation

0

5

10

15

20

Iteration

- Newton
- Quasi-Newton method
- Levenberg-Marquardt
- Gauss-Newton Method
- etc
Stanford Open Course – Machine Learning

B: Case Study

- Dataset
- The School Children Data Set from Lewis & Taylor “Introduction to Experimental Ecology” (1967)
- Includes 126 male records
- Variables:
- Age (months)
- height (inches)
- weight (pounds)

- Predicting the weight of male school children based on their age and height.
- Comparing neural networks with OLS

Exploration of the dataset

procreg data=men;

model weight=height age;

output out=regout p=pred r=resid;

run;

Neural Network Model

INPUT

OUTPUT

HIDDEN

COMBINATION

w1+w2S_Height+w3S_Age = H11

TRANSFORMATION

tanh(H11)) =A

Standardization

H11

Height

S_Height

Weight

COMBINATION

w7+w8A+w9B=Weight

S_Age

Age

Standardization

TRANSFORMATION

tanh(H12) =B

COMBINATION

w4+w5S_Height+w6S_Age = H12

H12

18

- The SAS neural network procedure
- PROC NEURAL

- A visual programming with a GUI interface

- To save time, I recorded the following video to show how to build the Neural Network Model with SAS Enterprise Miner.
- In case you are interested, I uploaded this video to YouTube:
http://www.youtube.com/watch?v=Bb3K7xAcJbk&feature=youtu.be

Output: Observed and Predicted Values of Male's Weight against Age

References

Eric Roberts. Neural networks. Available online at: http://www-cs-faculty.stanford.edu/~eroberts/courses/soco/projects/neural-networks/

Jim Georges, 2009. Applied analytics using SAS Enterprise Miner 6.1 Course Notes. SAS Institute Inc.

Lewis, T. and Taylor, L.R. 1967. Introduction to Experimental Ecology, Academic Press, Inc.

Randall Matignon, 2005. Neural Network Modeling using SAS Enterprise Miner. AuthorHouse

SAS Institute, 1999. SAS/STAT User’s Guide Version 8. Available online at: http://ciser.cornell.edu/sasdoc/saspdf/common/mainpdf.htm

Sue Walsh, 2002. Applying Data Mining Techniques Using Enterprise Miner Course Notes. SAS Institute Inc.

Wikipedia. Neural network. Available online at: http://en.wikipedia.org/wiki/Neural_network

Thank You!