1 / 45

CONCEPTS UNDERLYING REGRESSION MODELS ( Chapter 2)

Lecture 2. CONCEPTS UNDERLYING REGRESSION MODELS ( Chapter 2). ESTIMATION IN THE BIVARIATE REGRESSION MODEL (Chapters 3 and 4). Overview. Last time we saw that in regression analysis our goal is to study the dependence of one variable on one or more other variables.

linaeve
Download Presentation

CONCEPTS UNDERLYING REGRESSION MODELS ( Chapter 2)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecture 2 CONCEPTS UNDERLYING REGRESSION MODELS(Chapter 2) ESTIMATION IN THE BIVARIATE REGRESSION MODEL(Chapters 3 and 4)

  2. Overview • Last time we saw that in regression analysis our goal is to study the dependence of one variable on one or more other variables. • Here, we begin looking at this dependency. • We start by reviewing some fundamental statistical concepts that underlie regression analysis.

  3. Conditional Distribution, Conditional Probability and Conditional Mean • Suppose we want to study the relationship between weekly consumption expenditures (Y) and weekly disposable income (X). • Suppose we have a populationof 16 families whose weekly consumption expenditures and disposable income are as follows: (See Table 2.1, p. 38 of Gujarati)

  4. Conditional Distribution Income 80 100 120 55 65 79 60 70 84 Expenditure 65 74 90 70 80 94 75 85 98 --- 88 ---

  5. Conditional Distribution • Each column of the above table represents weekly expenditures Y associated with a given level of disposable income X. • That is, each column represents conditional distribution of Y given X, • F(Yi |Xi) i = 1,2,...,16

  6. Conditional Probability • Using conditional distribution of Y given X, we can compute conditional probability of Y given X, p(Yi |Xi) 80 100 120 1/5 1/6 1/5 1/5 1/6 1/5 1/5 1/6 1/5 1/5 1/6 1/5 1/5 1/6 1/5 --- 1/6 ---

  7. Conditional Mean or Mathematical Expectations • Using the conditional distribution and conditional probability of Y given X, we can compute conditional mean of Y givenX, E(Yi |Xi) =  p(Yi |Xi) F(Yi |Xi) 80 100 120 65 77 89

  8. Population Regression Function (PRF) • If we plot the conditional distribution of Y given X, we obtain the following scattergram,

  9. Population Regression Function • The scattergram shows that on average, Y increases as X increases. • In fact, it shows that the conditional mean values of Y lie on a straight line with a positive slope.

  10. Population Regression Function • This line/curve, which passes through the conditional mean values of Y given fixed values of X is known as Population Regression Function, PRF. • Note that the PRF can be written so as to express conditional mean of Y as a function of X, • E(Yi|Xi) =  (Xi) i = 1,2,...,16

  11. Linear PRF • The PRF associated with our hypothetical example is linear and deterministicso that it can be expressed as follows, • E(Yi|Xi) = ß1 + ß2Xi i = 1,2,...,16 • where ß1and ß2 are true but unknown population parameters.

  12. Stochastic PRF • The above PRF cannot be considered an econometric model because it is deterministic-- it does not have a random error term. • To turn this into an econometric model we note that, while on the averageY increases as Xincreases, this is not necessarily true for every individual family.

  13. Conditional Distribution Income 80 100 120 55 6579 60 70 84 Expenditure 65 74 90 7080 94 7585 98 88

  14. Stochastic PRF • However, as the scattergram of income and expenditures shows, consumption expenditures of families that have the same income level are distributed around their group mean value.

  15. Stochastic PRF • Thus, for any given level of income, we can express the deviation of an individual family's consumption from the mean consumption of all families with that income as, • ui = Yi ‑ E(Yi |Xi) • or • Yi = E(Yi |Xi) + ui • This is the stochastic PRF.

  16. Stochastic PRF • Here, the term, ui is a random variable that takes on positive, zero, or negative values. • This is the random or stochastic error or disturbance term we first saw in Chapter 1 • It is a proxy for determinants of consumption other than disposable income • It also captures measurement errors, incorrect functional form, and randomness in human response

  17. Linear Stochastic PRF • Our stochastic PRF specifies no functional form for the E(Yi |Xi). • Assuming E(Yi | Xi) is linear, the PRF becomes • Yi = ß1 + ß2Xi + ui • which is both linear and stochastic.

  18. Concepts Underlying Regression Analysis • Using the stochastic PRF and the data from our hypothetical example of weekly consumption expenditures, we can write • Y1 = 55 = ß1 + ß2(80) + u1 • Y2 = 60 = ß1 + ß2(80) + u2 • ....................... • ....................... • Y5 = 75 = ß1 + ß2(80) + u5

  19. Concepts Underlying Regression Analysis • Take the expected value of the PRF and you get, • E(Yi |Xi) = E[(E(Yi |Xi)+ ui)|Xi] • = E[(E(Yi |Xi)|Xi] + E(ui|Xi) • = E(Yi |Xi) + E(ui |Xi) • Cancel out E(Yi |Xi) from the two sides of the equality to get E(ui |Xi) = 0

  20. Concepts Underlying Regression Analysis • It follows that we can express the PRF in either of two equivalent ways, • DeterministicStochastic • E(Yi|Xi) = ß1 + ß2Xior Yi = ß1 + ß2Xi + ui

  21. The Problem of Sampling • We can now restate the main concern of regression analysis as follows: • In regression analysis our primary interest is in estimating the PRF by estimating its unknown parameters, ß1 and ß2 based on observed values of X and Y • Because in practice we do not have access to an entire population, our task is to estimate the PRF using data from a random sample of the population.

  22. Sample Regression Function • The sample counterpart of the true but unknown population regression function is called sample regression function, SRF. • Below is the non-stochastic version of the SRF: ^ ^ ^ • Yi = ß1 + ß2Xi • ^ ^ ^ • where Yi is an estimator of E(Yi|Xi), and ß1 and ß2 are estimators of ß1 and ß2, respectively. • Because of sampling fluctuations, the SRF is an approximation (estimate) of the true PRF.

  23. Sample Residuals • We turn the above non-stochastic SRF into a stochastic SRF as follows. • First, we define sample residuals as ^ ^ • ui = Yi ‑ Yi • Next, we rewrite this as • ^ ^ • Yi = Yi + ui • ^ ^ ^ • Finally, we substituteß1 + ß2Xifor Yiin the above, • ^^^ • Yi = ß1 + ß2Xi + ui

  24. The Problem of Estimation • Consider the following bivariate PRF, • Yi = ß1 + ß2Xi + ui • There are three unknown parameters associated with this regression model: ß1, ß2and s2, the latter being conditional variance of the error term, Var(ui|Xi).

  25. The Problem of Estimation • Earlier, we saw that in regression analysis our main goal is to estimate the unknown PRF in terms of its sample counterpart, the SRF. • Thus we must develop methods to make the SRP as close as possible to the PRF, that is make estimates of ß1 and ß2 as close as possible to their true values. • For this purpose, we can use either of two types of estimators: • point estimators • interval estimators

  26. The Method of Ordinary Least Squares • Ordinary Least Squares (OLS) is the most popular method for estimating the parameters of a linear regression model. • It is based on the least-squares principle, which • ^ ^ • suggests choosing ß1 and ß2 such that, for a given • sample, the sum of squared residuals (RSS) is the • smallest, i.e., minimize..., • ^ ^ ^^ • S ui2 = S (Yi - Yi)2 = S (Yi - ß1 ‑ß2Xi)2

  27. The Method of Ordinary Least Squares • ^^ • Minimizing the RSS with respect to ß1 and ß2 yields two equations in two unknowns: • ^ ^ ^ ^ • S ui2/ ß1 = -2S (Yi ‑ß1 ‑ß2Xi) = 0 • ^ ^ ^ ^ • S ui2/ ß2 = -2S (Yi ‑ß1 ‑ß2Xi)Xi= 0

  28. The Ordinary Least Squares Estimators • Solve these two equations, and you will obtain the OLS estimators, • ^ _ _ _ • ß2=S [(Xi ‑X)(Yi ‑Y)] ¸S (Xi ‑X)2 • and ^ _ ^ _ • ß1 = Y ‑ß2X • _ _ • where Y and X are the sample means of Y and X, respectively.

  29. The Ordinary Least Squares Estimators • It can be shown that the OLS estimator of the true but unknown variance of the population error term is, • ^ ^ • s2 = S ui2/(n-2) = RSS/df • The term df stands for degrees of freedom that is the total number of observations in the sample (n) less the number of independent constrained on them(2).

  30. Features of OLS Estimators & SRF 1. The OLS estimators depend on observable sample quantities Xiand Yi. 2. They are linear functions of Yi. 3. They are point estimators. 4. The two OLS estimators of ß1 and ß2 yield a linear sample regression line whose RSS is smaller than any other straight line. 5. The OLS sample regression line passes through the sample means of X and Y.

  31. Features of OLSE & SRF 6. The mean value of the estimated Y is equal to the mean value of actual Y. ^ 7. The mean value of sample residuals, ui, is zero. ^ 8. Sample residuals, ui and Xi are uncorrelated. 9. Sample residuals and predicted Yi are uncorrelated.

  32. Examples of the Bivariate Regression Model • Simple Keynesian Consumption Function • Please refer to Handout #3 • Also see pages 87-90, and 84-85 of Gujarati

  33. HANDOUT #3AN EXAMPLE OF OLS:Estimating Simple Consumption Function for the U.S., 1959‑1990

  34. The standard Keynesian consumption function expresses real consumption expenditures, Y, as a linear function of real disposable income, X, Yt = 1 + 2Xt + ut • I use seasonally adjusted, personal expenditures on nondurable goods and services in billions of 1987 dollars to quantify the dependent variable. • For Xt, I use seasonally adjusted personal disposable income in billions of 1987 dollars.

  35. _ _ _ _ _ Year Yt Xt (Yt ‑ Y) (Xt ‑ X) (Yt ‑ Y)(Xt ‑ X) (Xi ‑ X)2 1959 1059 1280 -595 -778 462910 605284 1960 1089 1309 -565 49 423185 561001 1961 1102 1320 -552 -738 407376 544644 1962 1135 1366 -519 -692 359148 478864 1963 1168 1411 -486 -647 314442 418609 1964 1196 1437 458 -621 284418 385641 1965 1233 1493 421 -565 237865 319225 1966 1285 1574 369 -484 178596 234256 1967 1328 1635 326 -423 137898 178929 1968 1391 1711 263 -347 91261 120409 1969 1425 1765 229 -293 67097 85849 …….……………………………………………………………………... …….……………………………………………………………………... 1988 2206 2778 552 720 397440 518400 1989 2220 2806 566 748 423368 5595041990 2247 2830 593 772 457796 595984 Sum 000 000 605983 798556 Mean1654 2058 000 000

  36. ^ _ _ _ 2 = (Xi - X)(Yi - Y)/(Xi - X)2 = 605983/798556 = 0.7588 ^ _ ^ _ __ 1 = Y - 2X = 1654 - 0.7588(2058) = 92.3896 Thus, the estimated model can be written in either of the following two alternative forms, ^^ Yt = 92 + 0.76Xtor Yt = 92 + 0.76Xt + ut

  37. We interpret the estimation results as follows. • According to the estimated value of 2 (i.e., 0.76), if the real personal disposable income increases by one dollar, on average real consumption expenditures are expected to increase by $0.76. • According to the estimated value of the intercept term, the mean systematic effect of other determinants of consumption expenditures not explicitly included in the model is$92billion.

  38. WARNING • Avoid the oft-made mechanical interpretation of the intercept term, namely “if real personal disposable income were zero, on the average $92 billion would be spent on nondurable goods and services” • This is usually meaningless, as in the present case

  39. We calculate the predicted values of the ^ dependent variable, Yt, by plugging values of Xt into the first of the above two equations. • Then by subtracting the predicted values of Yt from its observed values, we can calculate the ^ sample residuals, ut,

  40. ^ ^ ^ Year Yt Yt ut = Yt ‑Yt 1959 1059 1063 -4 1960 1089 1085 4 1961 1102 1094 8 1962 1135 1128 7 1963 1168 1163 5 1964 1196 1182 14 1965 1233 1225 8 1966 1285 1286 -1 1967 1328 1333 -5 1968 1391 1390 1 1969 1425 1431 -6 1988 2206 2200 6 1989 2220 2221 -1 1990 2247 2239 8 Mean 1654 1654 0.000

  41. EViews Output for the Above Example

More Related