Thursday, November 17, 2011

Dimensionality Reduction

Another interesting Machine Learning algorithm (unsupervised learning this time) is Dimensionality Reduction.

Here is a short video explaining the theory. In the example presented in this video the mean (μ vector) is the simple arithmetic average per column of the matrix X. Σ (or covariance) matrix is simply (1 ⁄ M)⋅(X-μ)T⋅(X-μ) (where X-μ is per column operation, i.e. mean is extracted from each element of the relevant column, T – is transpose operator and M is the number of rows in the matrix X). Eigen values and vectors are of the Σ matrix, i.e. satisfying Σ⋅v=λ⋅v, where λ is a scalar value.

And here is another video explaining the algorithm's applicability.