Neural networks and classical linear regression
This presentation is the property of its rightful owner.
Sponsored Links
1 / 26

Neural Networks and Classical Linear Regression PowerPoint PPT Presentation


  • 60 Views
  • Uploaded on
  • Presentation posted in: General

Neural Networks and Classical Linear Regression. Szu Hua Huang Jianjun Luo Texas Tech University 6/10/2014. Contents. Neural Networks and Classical Linear Regression Introduction Neural Networks Neural Networks VS Classical Linear Regression Case study. Neural Network - Neuron.

Download Presentation

Neural Networks and Classical Linear Regression

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Neural networks and classical linear regression

Neural Networks and Classical Linear Regression

Szu Hua Huang Jianjun Luo

Texas Tech University

6/10/2014


Contents

Contents

  • Neural Networks and Classical Linear Regression

    • Introduction Neural Networks

    • Neural Networks VS Classical Linear Regression

  • Case study


Neural network neuron

Neural Network - Neuron


Structure of a neural network

STRUCTURE OF A NEURAL NETWORK

Multilayer feedforward networks


A simple mlp multilayer perceptron

A simple MLP (multilayer perceptron)

Hidden Layer

Output Layer

Input Layer

Weights(Regression Coefficients)

Back-propagation Model

Updating (Learning)

3

W13

1

W36

Node

W14

W15

6

W46

Output

Input

4

W23

W56

W24

2

Data from True Function

W25

5

Errors

Transfer Function, g(s)


Transfer function

Transfer Function


Neural network vs classical regression model

Neural Network VS Classical Regression Model

Neural Network

Classical Regression

Linear

Normality

Constant variability of error terms

Independence Assumption

  • Non-linear


Neural network vs classical regression model1

Neural Network VS Classical Regression Model

Neural Network

Classical Regression

Regression coefficient shows the effect

  • Weight estimates(regression coefficient) do not tell you the effect

  • No guarantee the best linear combination of parameter estimates


Neural network vs classical regression model2

Neural Network VS Classical Regression Model

Multi-collinearity

Outlier or Influential

No effect to Neural Network ?

It does hurt the Classical Regression

  • No effect to Neural Network ?

  • It does hurt the Classical Regression


Overfitting

Overfitting

Neural Network

Classical Regression

Not very sensitive to the given data.

  • Sensitive to the given data, too much flexibility to the underlying distribution of data.

  • Big Sample size can help to solve the problem of overfitting.


Assessment

Assessment

Neural Network

Classical Regression

R square

  • Optimization plot based on the updated weight estimates at each iteration of the iterative grip search routine.

  • Using valid data


Neural networks and classical linear regression

Optimization plot

  • 1. When the new weights are only incrementally different from those of the preceding iteration

  • 2. When the misclassification rate reaches a required threshold

  • 3. When the limit on the number of runs is reached

ASE

training

validation

0

5

10

15

20

Iteration


Supplement other optimization algorithms

Supplement-Other Optimization Algorithms:

  • Newton

  • Quasi-Newton method

  • Levenberg-Marquardt

  • Gauss-Newton Method

  • etc

    Stanford Open Course – Machine Learning


Neural networks and classical linear regression

B: Case Study

  • Dataset

    • The School Children Data Set from Lewis & Taylor “Introduction to Experimental Ecology” (1967)

    • Includes 126 male records

    • Variables:

      • Age (months)

      • height (inches)

      • weight (pounds)

  • Purpose

    • Predicting the weight of male school children based on their age and height.

    • Comparing neural networks with OLS


  • Neural networks and classical linear regression

    Exploration of the dataset


    Classical linear regression model

    Classical Linear Regression Model

    procreg data=men;

    model weight=height age;

    output out=regout p=pred r=resid;

    run;


    Output of ols

    Output of OLS


    Neural networks and classical linear regression

    Neural Network Model

    INPUT

    OUTPUT

    HIDDEN

    COMBINATION

    w1+w2S_Height+w3S_Age = H11

    TRANSFORMATION

    tanh(H11)) =A

    Standardization

    H11

    Height

    S_Height

    Weight

    COMBINATION

    w7+w8A+w9B=Weight

    S_Age

    Age

    Standardization

    TRANSFORMATION

    tanh(H12) =B

    COMBINATION

    w4+w5S_Height+w6S_Age = H12

    H12

    18


    Neural network in sas

    Neural Network in SAS

    • The SAS neural network procedure

      • PROC NEURAL

  • SAS Enterprise Miner

    • A visual programming with a GUI interface


  • Neural network modeling using sas enterprise miner

    Neural Network Modeling using SAS Enterprise Miner

    • To save time, I recorded the following video to show how to build the Neural Network Model with SAS Enterprise Miner.

    • In case you are interested, I uploaded this video to YouTube:

      http://www.youtube.com/watch?v=Bb3K7xAcJbk&feature=youtu.be


    Neural network weight estimates

    Neural Network Weight Estimates


    Neural network or classical linear regression

    Neural network or classical linear regression?


    Comparing neural network and classical linear regression predicted values

    Comparing Neural Network and Classical Linear Regression predicted values


    Neural networks and classical linear regression

    Output: Observed and Predicted Values of Male's Weight against Age


    Neural networks and classical linear regression

    References

    Eric Roberts. Neural networks. Available online at: http://www-cs-faculty.stanford.edu/~eroberts/courses/soco/projects/neural-networks/

    Jim Georges, 2009. Applied analytics using SAS Enterprise Miner 6.1 Course Notes. SAS Institute Inc.

    Lewis, T. and Taylor, L.R. 1967. Introduction to Experimental Ecology, Academic Press, Inc.

    Randall Matignon, 2005. Neural Network Modeling using SAS Enterprise Miner. AuthorHouse

    SAS Institute, 1999. SAS/STAT User’s Guide Version 8. Available online at: http://ciser.cornell.edu/sasdoc/saspdf/common/mainpdf.htm

    Sue Walsh, 2002. Applying Data Mining Techniques Using Enterprise Miner Course Notes. SAS Institute Inc.

    Wikipedia. Neural network. Available online at: http://en.wikipedia.org/wiki/Neural_network


    Neural networks and classical linear regression

    Thank You!


  • Login