neural networks and classical linear regression
Download
Skip this Video
Download Presentation
Neural Networks and Classical Linear Regression

Loading in 2 Seconds...

play fullscreen
1 / 26

Neural Networks and Classical Linear Regression - PowerPoint PPT Presentation


  • 101 Views
  • Uploaded on

Neural Networks and Classical Linear Regression. Szu Hua Huang Jianjun Luo Texas Tech University 6/10/2014. Contents. Neural Networks and Classical Linear Regression Introduction Neural Networks Neural Networks VS Classical Linear Regression Case study. Neural Network - Neuron.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Neural Networks and Classical Linear Regression' - krista


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
neural networks and classical linear regression

Neural Networks and Classical Linear Regression

Szu Hua Huang Jianjun Luo

Texas Tech University

6/10/2014

contents
Contents
  • Neural Networks and Classical Linear Regression
    • Introduction Neural Networks
    • Neural Networks VS Classical Linear Regression
  • Case study
structure of a neural network
STRUCTURE OF A NEURAL NETWORK

Multilayer feedforward networks

a simple mlp multilayer perceptron
A simple MLP (multilayer perceptron)

Hidden Layer

Output Layer

Input Layer

Weights(Regression Coefficients)

Back-propagation Model

Updating (Learning)

3

W13

1

W36

Node

W14

W15

6

W46

Output

Input

4

W23

W56

W24

2

Data from True Function

W25

5

Errors

Transfer Function, g(s)

neural network vs classical regression model
Neural Network VS Classical Regression Model

Neural Network

Classical Regression

Linear

Normality

Constant variability of error terms

Independence Assumption

  • Non-linear
neural network vs classical regression model1
Neural Network VS Classical Regression Model

Neural Network

Classical Regression

Regression coefficient shows the effect

  • Weight estimates(regression coefficient) do not tell you the effect
  • No guarantee the best linear combination of parameter estimates
neural network vs classical regression model2
Neural Network VS Classical Regression Model

Multi-collinearity

Outlier or Influential

No effect to Neural Network ?

It does hurt the Classical Regression

  • No effect to Neural Network ?
  • It does hurt the Classical Regression
overfitting
Overfitting

Neural Network

Classical Regression

Not very sensitive to the given data.

  • Sensitive to the given data, too much flexibility to the underlying distribution of data.
  • Big Sample size can help to solve the problem of overfitting.
assessment
Assessment

Neural Network

Classical Regression

R square

  • Optimization plot based on the updated weight estimates at each iteration of the iterative grip search routine.
  • Using valid data
slide12
Optimization plot
  • 1. When the new weights are only incrementally different from those of the preceding iteration
  • 2. When the misclassification rate reaches a required threshold
  • 3. When the limit on the number of runs is reached

ASE

training

validation

0

5

10

15

20

Iteration

supplement other optimization algorithms
Supplement-Other Optimization Algorithms:
  • Newton
  • Quasi-Newton method
  • Levenberg-Marquardt
  • Gauss-Newton Method
  • etc

Stanford Open Course – Machine Learning

slide14
B: Case Study
  • Dataset
    • The School Children Data Set from Lewis & Taylor “Introduction to Experimental Ecology” (1967)
    • Includes 126 male records
    • Variables:
        • Age (months)
        • height (inches)
        • weight (pounds)
  • Purpose
    • Predicting the weight of male school children based on their age and height.
    • Comparing neural networks with OLS
classical linear regression model
Classical Linear Regression Model

procreg data=men;

model weight=height age;

output out=regout p=pred r=resid;

run;

slide18
Neural Network Model

INPUT

OUTPUT

HIDDEN

COMBINATION

w1+w2S_Height+w3S_Age = H11

TRANSFORMATION

tanh(H11)) =A

Standardization

H11

Height

S_Height

Weight

COMBINATION

w7+w8A+w9B=Weight

S_Age

Age

Standardization

TRANSFORMATION

tanh(H12) =B

COMBINATION

w4+w5S_Height+w6S_Age = H12

H12

18

neural network in sas
Neural Network in SAS
  • The SAS neural network procedure
        • PROC NEURAL
  • SAS Enterprise Miner
        • A visual programming with a GUI interface
neural network modeling using sas enterprise miner
Neural Network Modeling using SAS Enterprise Miner
  • To save time, I recorded the following video to show how to build the Neural Network Model with SAS Enterprise Miner.
  • In case you are interested, I uploaded this video to YouTube:

http://www.youtube.com/watch?v=Bb3K7xAcJbk&feature=youtu.be

slide25
References

Eric Roberts. Neural networks. Available online at: http://www-cs-faculty.stanford.edu/~eroberts/courses/soco/projects/neural-networks/

Jim Georges, 2009. Applied analytics using SAS Enterprise Miner 6.1 Course Notes. SAS Institute Inc.

Lewis, T. and Taylor, L.R. 1967. Introduction to Experimental Ecology, Academic Press, Inc.

Randall Matignon, 2005. Neural Network Modeling using SAS Enterprise Miner. AuthorHouse

SAS Institute, 1999. SAS/STAT User’s Guide Version 8. Available online at: http://ciser.cornell.edu/sasdoc/saspdf/common/mainpdf.htm

Sue Walsh, 2002. Applying Data Mining Techniques Using Enterprise Miner Course Notes. SAS Institute Inc.

Wikipedia. Neural network. Available online at: http://en.wikipedia.org/wiki/Neural_network

ad