Purpose of Factor AnalysisFactor analysis is one of the techniques to reduce dimension of the observed variables. Suppose that we have p-dimensional continuous variable vector x = (x1,x2,,,xp). We want describe correlation between them with m dimensional vector – y=(y1,y2,,,ym). Suppose that the joint probability distribution of these two variables is normal and equal to f(x,y). Conditional probability distribution of x given y is also normal and given by g(x|y). Conditional mean of x is linear in y and the covariance matrix does not depend on y. Then we can write:
Where e is normal random vector with 0 mean and constant dispersion. It is additionally assumed that elements of e are independent of each other and y. Moreover it is assumed that elements of y are independent each other and they are standard normal variables. We can write:
Where is the diagonal mxm matrix. In this model e is specific variables and weights are factor loadings.
Without loss of generality we will assume that mean of x s 0, i.e. =0.
Vector x is what we can observe and vector y is what we think is the vector of independent variables. We want to deduce from the observations independent variables