Applications The General Linear Model
Transformations to Linearity • Many non-linear curves can be put into a linear form by appropriate transformations of the either • the dependent variable Y or • some (or all) of the independent variables X1, X2, ... , Xp . • This leads to the wide utility of the Linear model. • We have seen that through the use of dummy variables, categorical independent variables can be incorporated into a Linear Model. • We will now see that through the technique of variable transformation that many examples of non-linear behaviour can also be converted to linear behaviour.
Intrinsically Linear (Linearizable) Curves 1Hyperbolas y = x/(ax-b) Linear form: 1/y = a -b (1/x) or Y = b0 + b1 X Transformations: Y = 1/y, X=1/x, b0 = a, b1 = -b
2.Exponential y = aebx = aBx Linear form: ln y = lna + b x = lna + lnB x or Y = b0 + b1 X Transformations: Y = ln y, X = x, b0 = lna, b1 = b = lnB
3. Power Functions y = a xb Linear from: ln y = lna + blnx or Y =b0 + b1 X
Logarithmic Functions y = a + b lnx Linear from: y = a + b lnx or Y =b0+ b1X Transformations: Y = y, X = ln x,b0 = a,b1= b
Other special functions y = aeb/x Linear from: ln y = lna + b 1/x or Y =b0 +b1X Transformations: Y = ln y, X = 1/x,b0= lna, b1= b
The ln-transformation is a member of the Box-Cox family of transformations with l= 0 • If you decrease the value of lthe effect of the transformation will be greater. • If you increase the value of lthe effect of the transformation will be less.
The effect of the ln transformation • It spreads out values that are close to zero • Compacts values that are large
y up y up x down x up x down x up y down y down The Bulging Rule
Non-Linear Models Nonlinearizable models
Non-Linear Growth models The Mechanistic Growth Model • many models cannot be transformed into a linear model Equation: or (ignoring e)“rate of increase in Y”=
The Logistic Growth Model Equation: or (ignoring e) “rate of increase in Y”=
The Gompertz Growth Model: Equation: or (ignoring e)“rate of increase in Y”=
Polynomial Models y = b0 + b1x + b2x2 + b3x3 Linear form Y = b0 + b1 X1 + b2 X2 + b3 X3 Variables Y = y, X1 = x , X2 = x2, X3 = x3
Suppose that we have two variables • Y – the dependent variable (response variable) • X – the independent variable (explanatory variable, factor)
Assume that we have collected data on two variables X and Y. Let (x1, y1) (x2, y2) (x3, y3) … (xn, yn) denote thepairs of measurements on the on two variables X and Y for n cases in a sample (or population)
The assumption will be made that y1,y2, y3 …, yn are • independent random variables. • Normally distributed. • Have the common variance, s. • The mean of yiis:
Each yi is assumed to be randomly generated from a normal distribution with mean and standard deviation s.
The matrix formulation The Model
Example In the following example two quantities are being measured X = amount of an additive to a chemical process Y = the yield of the process
Comment: A cubic polynomial in x can be fitted to y by defining the variables X1 = x, X2 = x2, and X3 = x3 Then fitting the linear model The Model – Cubic polynomial (degree 3)
Response Surface Models Extending polynomial regression models to k independent variables
Dependent variable Y and two independent variables x1 and x2. (These ideas are easily extended to more the two independent variables) Response Surface models (2 independent vars.) The Model (A cubic response surface model) Compare with a linear model:
The response surface model can be put into the form of a linear model : Y = b0 + b1X1 +b2X2 + b3X3 +b4X4 + b5X5 + b6X6 + b7X7 + b8X8 + b9X9+ e by defining
More Generally, consider the random variable Y with 1. E[Y] = g(U1 ,U2 , ... , Uk) = b1f1(U1 ,U2 , ... , Uk) + b2f2(U1 ,U2 , ... , Uk) + ... + bpfp(U1 ,U2 , ... , Uk) = and 2. var(Y) = s2 • where b1, b2 , ... ,bp are unknown parameters • and f1 ,f2 , ... , fp are known functions of the nonrandom variables U1 ,U2 , ... , Uk. • Assume further that Y is normally distributed.
Now suppose that n independent observations of Y, (y1, y2, ..., yn) are made corresponding to n sets of values of (U1 ,U2 , ... , Uk) : (u11 ,u12 , ... , u1k), (u21 ,u22 , ... , u2k), ... (un1 ,un2 , ... , unk). Let xij = fj(ui1 ,ui2 , ... , uik) j =1, 2, ..., p; i =1, 2, ..., n. Then or
Polynomial Regression Model: One variable U. Quadratic Response Surface Model: Two variables U1, U2.
Trigonometric Polynomial Models y = b0 + g1cos(2pf1x) + d1sin(2pf1x) + … + gkcos(2pfkx) + dksin(2pfkx) Linear formY = b0 + g1C1 + d1S1 + … + gk Ck+ dk Sk Variables Y = y, C1 = cos(2pf1x) , S2 = sin(2pf1x) , … Ck = cos(2pfkx) , Sk = sin(2pfkx)
The Normal equations: given data General set of models
Polynomial Models Two important Special Cases Trig-polynomial Models
Definition Consider the values x0, x1, … , xnand the polynomials are orthogonal relative to x0, x1, … , xnif: If in addition , they are called orthonormal
This is equivalent to a polynomial model. Rather than the basis for this model being The basis is ,polynomials of degree 0, 1, 2, 3, etc Consider the model
Derivation of Orthogonal Polynomials With equally spaced data points