Neural networks and classical linear regression
Download
1 / 26

Neural Networks and Classical Linear Regression - PowerPoint PPT Presentation


  • 86 Views
  • Uploaded on
  • Presentation posted in: General

Neural Networks and Classical Linear Regression. Szu Hua Huang Jianjun Luo Texas Tech University 6/10/2014. Contents. Neural Networks and Classical Linear Regression Introduction Neural Networks Neural Networks VS Classical Linear Regression Case study. Neural Network - Neuron.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha

Download Presentation

Neural Networks and Classical Linear Regression

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Neural networks and classical linear regression
Neural Networks and Classical Linear Regression

Szu Hua Huang Jianjun Luo

Texas Tech University

6/10/2014


Contents
Contents

  • Neural Networks and Classical Linear Regression

    • Introduction Neural Networks

    • Neural Networks VS Classical Linear Regression

  • Case study



Structure of a neural network
STRUCTURE OF A NEURAL NETWORK

Multilayer feedforward networks


A simple mlp multilayer perceptron
A simple MLP (multilayer perceptron)

Hidden Layer

Output Layer

Input Layer

Weights(Regression Coefficients)

Back-propagation Model

Updating (Learning)

3

W13

1

W36

Node

W14

W15

6

W46

Output

Input

4

W23

W56

W24

2

Data from True Function

W25

5

Errors

Transfer Function, g(s)



Neural network vs classical regression model
Neural Network VS Classical Regression Model

Neural Network

Classical Regression

Linear

Normality

Constant variability of error terms

Independence Assumption

  • Non-linear


Neural network vs classical regression model1
Neural Network VS Classical Regression Model

Neural Network

Classical Regression

Regression coefficient shows the effect

  • Weight estimates(regression coefficient) do not tell you the effect

  • No guarantee the best linear combination of parameter estimates


Neural network vs classical regression model2
Neural Network VS Classical Regression Model

Multi-collinearity

Outlier or Influential

No effect to Neural Network ?

It does hurt the Classical Regression

  • No effect to Neural Network ?

  • It does hurt the Classical Regression


Overfitting
Overfitting

Neural Network

Classical Regression

Not very sensitive to the given data.

  • Sensitive to the given data, too much flexibility to the underlying distribution of data.

  • Big Sample size can help to solve the problem of overfitting.


Assessment
Assessment

Neural Network

Classical Regression

R square

  • Optimization plot based on the updated weight estimates at each iteration of the iterative grip search routine.

  • Using valid data


Neural networks and classical linear regression

Optimization plot

  • 1. When the new weights are only incrementally different from those of the preceding iteration

  • 2. When the misclassification rate reaches a required threshold

  • 3. When the limit on the number of runs is reached

ASE

training

validation

0

5

10

15

20

Iteration


Supplement other optimization algorithms
Supplement-Other Optimization Algorithms:

  • Newton

  • Quasi-Newton method

  • Levenberg-Marquardt

  • Gauss-Newton Method

  • etc

    Stanford Open Course – Machine Learning


Neural networks and classical linear regression

B: Case Study

  • Dataset

    • The School Children Data Set from Lewis & Taylor “Introduction to Experimental Ecology” (1967)

    • Includes 126 male records

    • Variables:

      • Age (months)

      • height (inches)

      • weight (pounds)

  • Purpose

    • Predicting the weight of male school children based on their age and height.

    • Comparing neural networks with OLS



  • Classical linear regression model
    Classical Linear Regression Model

    procreg data=men;

    model weight=height age;

    output out=regout p=pred r=resid;

    run;



    Neural networks and classical linear regression

    Neural Network Model

    INPUT

    OUTPUT

    HIDDEN

    COMBINATION

    w1+w2S_Height+w3S_Age = H11

    TRANSFORMATION

    tanh(H11)) =A

    Standardization

    H11

    Height

    S_Height

    Weight

    COMBINATION

    w7+w8A+w9B=Weight

    S_Age

    Age

    Standardization

    TRANSFORMATION

    tanh(H12) =B

    COMBINATION

    w4+w5S_Height+w6S_Age = H12

    H12

    18


    Neural network in sas
    Neural Network in SAS

    • The SAS neural network procedure

      • PROC NEURAL

  • SAS Enterprise Miner

    • A visual programming with a GUI interface


  • Neural network modeling using sas enterprise miner
    Neural Network Modeling using SAS Enterprise Miner

    • To save time, I recorded the following video to show how to build the Neural Network Model with SAS Enterprise Miner.

    • In case you are interested, I uploaded this video to YouTube:

      http://www.youtube.com/watch?v=Bb3K7xAcJbk&feature=youtu.be





    Neural networks and classical linear regression

    Output: predicted valuesObserved and Predicted Values of Male's Weight against Age


    Neural networks and classical linear regression

    References predicted values

    Eric Roberts. Neural networks. Available online at: http://www-cs-faculty.stanford.edu/~eroberts/courses/soco/projects/neural-networks/

    Jim Georges, 2009. Applied analytics using SAS Enterprise Miner 6.1 Course Notes. SAS Institute Inc.

    Lewis, T. and Taylor, L.R. 1967. Introduction to Experimental Ecology, Academic Press, Inc.

    Randall Matignon, 2005. Neural Network Modeling using SAS Enterprise Miner. AuthorHouse

    SAS Institute, 1999. SAS/STAT User’s Guide Version 8. Available online at: http://ciser.cornell.edu/sasdoc/saspdf/common/mainpdf.htm

    Sue Walsh, 2002. Applying Data Mining Techniques Using Enterprise Miner Course Notes. SAS Institute Inc.

    Wikipedia. Neural network. Available online at: http://en.wikipedia.org/wiki/Neural_network


    Neural networks and classical linear regression

    Thank You! predicted values


    ad
  • Login