Regression and correlation analysis (RaKA)

1. Regression and correlation analysis (RaKA)

2. Investigating the relationships between the statistical characteristics: • Investigating the relationship between qualitative characteristics, e.g. AB , calledmeasurement of association • Investigating the relationship between quantitative characteristics –Regression and correlation analysis

4. Regression and correlation analysis: • examining causal dependency, exploring the relationship between cause and effect • When one or more effects (attributes, independent variables) cause resulting effect – dependent variable Y = f (X1 X2…...Xk ,Bo , B1 ,….Bp )+e Random, unspecified effects Unknown parameters of afunctionalrelationship Dependent variable - effect Independent variables - cause

5. Example of false correlation One of the famous spurious correlations: If the skirt lenght gets shorter, quotation of stocks gets higher Apart from that it is not always true, it would be false, or spurious correlation.

6. Examples ofstatistical - free - dependence • Examination how consumption of pork depend on income, price of pig meat, beef, poultry and tradition resp. another unspecified, or random effects. • Examination of dependence of GNP on Labour and Capital... • Ivestigation if the nutrition of the population depend on the degree of economic development of the country

7. Opposite of the statistical dependence is the functional dependence Y = f(X1 X2…...Xk ,Bo , B1 ,…., Bp) Where the dependent variable is clearly determined by functional relationship, Examples from physics, chemistry – this kind of relationship is not the subject of statistical investigation.

8. Regression and correlation analysis (RaKA) • Two basic task of RaKA: • Regression a) find a functional relationship by which the dependent variable changes with the change of independent variables - find a suitable regression line (function). b) It is also necessary to estimate the parameters of the regression function. • Correlation -to measure strength of the examined dependence (relationship).

9. y x Illustration of the correlation field in two cases (scatter plot) y x

10. According to the number of independent variables are distinguished: • Simple dependence, when we consider only one independent variable X, we investigate the relationship between Y and X. • Multiple dependence , we are considering at least two independent variables veličiny X1, X2, … Xk , fork  2

11. Simple regression and correlation analysis • Consider statistical sign X and Y which are in the population in linear relationship Y = Bo + B1 X +e point estimate of the regression function is a straight line yj = b0 + b 1 xj + ej , with coefficients calculated from the sample data Which method to use ???

12. The least square method (LSM)  We get set ofp+1equation withp+1unknown parameters => Ordinary least square method (OLS)

13. yj = b0 + b 1 xj + ejwe can rewrite yj = yj , + ejandej = y j - yj ,Principle of the LSM (ej )2 = (y j - y j’)2 (ej ) = y j - y j’

14. Can be proved that coefficientsbo, b1 , …, bpdeterminedby OLS are “best estimates” of parametersB 0 , B1 , …, Bpif the random error meet the assumptions: • E (ej ) = 0, • D (ej ) = E (ej2 ) = 2 , • E(ej1 , ej2 ) = 0 , for each j1  j2 Verbal formulation : Random errors are required to have zero mean, constant variance and should be independent.

15. Coefficients of the simple regression function can be derived:

16. After transformation we get two normal equations with two unknown parameters: The system of equation can be solved by elimination method , or by using determinants. We get the coefficients b oa b 1

17. The procedure for calculating the coefficients LRF xj yj xjyj xj 2 x1 y1 x1y1 x12 x2 y2 xn yn

18. Interpretation of simple linear regression coefficients • bo …intercept- expected value of dependent variable if the independent variable is equal to zero • b 1 …. Regression coefficient express the change in dependent variable, if the independent variable will change by one unit. • ifb1 >0 …positive correlation (dependence) • ifb1< 0 ….negative correlation (dependence)

19. Properties of least square method: Regression function passes throught the coordinatesa

20. When OLS can be applied? • If the regression function is linear • Linear in parameters (LiP) • Or we can transform regression function to be linear in parameters • Consider in which of the following regression functions can be used OLS

21. Some types of simple regression function:

22. Examples from micro- and macroeconomy • Phillips curve ???? • Cobb -Douglas production curve • Engel curves • Curve of economic growth • Any other? …...

23. Examining the consumption of selected commodities (depends on the level of GNP)

24. y x Comparison of two cases of correlationWhich correlation is closer? y x

25. Confidence interval for linear regression In addition to point estimates of parameters of linear regression functions are often calculated also interval estimates of parameters, which are called confidence intervals. Calculations of confidence intervals can be done with standard deviations of parameters and residual variance. Residual variance, if all the conditions of classical linear model are satisfied, is undistorted estimate of the stochastic parameter and is calculated according to equation

26. Interval estimate of any parameters for the regression line Assumes that if the assumptions formulated in classical linear model has variable tdistribution with n – p degrees of freedom. For the chosen confidence level 1 – is confidence interval for parameter given by relationship

27. And for parameter Analogically is constructed confidence interval for regression line Where is quantile of t distribution S with (for regression line n-2) degrees of freedom.

28. Role of the correlation • Examine tightness - strength - of dependence • We use various correlation indices • Should be bounded in interval • and within that interval increased to a higher power of dependence

29. Correlation analysis providesmethods and techniques which are used for verifying of explanatory abilityof quantified regression modelsas a whole and its parts.Verification of explanatory ability of quantified regression modelsleads to calculation of numerical characteristics, which in concentrated form describe the quality of the calculated models.

30. Index of correlation andindex of determinationIn population Iyxestimate from sample data is iyxest Iyx = iyx . Principle lies in the decomposition of variability of dependent variable Y Variability unexplained by regression function - Residual variability Variability of dependent variableexplained by regression function Total variability of dependent variable

31. . Its obvious that there is a relationship:  T = E + U T = Total sum of squares (of deviation) E = is explained sum of squares U = is unexplained (residual) sum of squares.

32. Index of correlationiyx Index of determinationiyx2

33. Index of determination can take values from 0 to 1, when the value of the index is close to 1, the greatproportion of the total variability is explained by the model and vice versa, if the index of determination is close to zero, the low proportion of the total variability is explained bythemodel.Index of determination is commonly used as a criterion in deciding about the shape of the regression function. However, if the regression functions has different number of parameters, it is necessary to adjust the index of determination to the corrected form:

34. Variability Sum of squares Degrees of freedom Variance F test Explained Unexplained V = N = F = Total C =

35. Test criterion in the table can be used for simultaneous testing the significance of the regression model, the index of determination and also correlation index. We compare calculated value of F test and quantile of F distribution with p-1 and n-p degrees of freedom. if F regression model is insignificant, as well as the index of correlation and index of determination. ifF> regression model is statistically significant as well as the index of correlation and index of determination.

36. For a detailed evaluation of the parameters quality of regression model is used t tests. We formulate the null hypothesis H0 :pre i = 0, 1H1 : where we assume zero therefore insignificant effect or impact of the variable at which the parameter is. The test criterion is defined by relationship:

37. Where is value of the parameter of regression function and is standard error of the parameter. We will compare calculated value of test criterion with quantile of t distribution at significance level and degrees of freedom .: - if      we do not reject null hypothesis about insignificance of the parameter. -  if       we reject null hypothesis, and confirm statistical significance of the parameter.

38. Nonlinear regression and correlation analysis In addition to linear regression functions, in practice are very often used nonlinear functions, which can be used also with two or more parameters. Some non-linear regression functions can be suitably transformed to be linear in parameters, and we can then use the method of least squares.. Most often, we can transform nonlinear function with two parameters to shape:

39. We estimate regression function in form where Function is then calculated as a linear function. Not all non-linear functions can be converted in this way, only those which are linear in parameters, ie there is some form of transformation called the linearising transformation, most often it is the substitution and logarithmic transformation for example

40. Hyperbolic function

41. Logarithmic function

42. Exponential function

43. power (Cobb-Douglas production function)

44. Similarly, it is possible to modify some more parametrical nonlinear functions such as.second degree parabola

45. Second degree hyperbole

46. It should be noted that the transformed regression functions do not always have the same parameters as the original non-linear regression function, so it is necessary for the estimated parameters of the transformed functions to do backwards calculations of the original parameters. Thus obtained estimates of the original parameters, do not have optimal statistical properties, but are often sufficient to solve specific tasks.Some regression function can not be adjusted or transformed to functions linear in parameters. Estimates of the parameters of such functions are obtained using different approximate or iterative methods. Most of them are based on so-called gradual improvement of initial estimates, which may be eg. expert estimates, or the estimates obtained by the selected points and so on.

47. Multiple regression and correlation analysis Suppose that the dependent variable Y and explanatory (independent) variables Xi ,i = 1, 2, ..., k  Are in linear relationship, we have already mentioned in previous sections, can be written: Which we estimate:

48. Coefficients, which are estimates of parameters , should meet the condition of the Least squares method since we assume a particular shape of the regression functions, we can install it into previous relationship and look for a minimum of this function ie.: we determine the minimum of the function similarly like in the case of a simple regression equation using partial derivatives of functions

49. Which leads to system of equations:

50. The solution of this system of equations will be the coefficients of linear regression equations Like for the simple linear relationship, we can calculate estimate of the parameters from the matrix equation The quality of a regression model can be evaluated similarly to the simple linear relationship, which we described in the previous section.