Principal Component Analysis

The first Principal Component of a dataset with average zero in N-dimensional space is the unit vector such that the projection of the dataset onto that vector has the highest possible variance. The second Principal Component is a unit vector orthogonal to the first principal component such that the projection of the dataset onto that vector has the highest possible variance. Proceeding like so, always picking unit vectors orthogonal to the previously defined vectors, a set of N orthogonal vectors is obtained. When the dataset is represented in this basis, the first few coefficients can cover most of the variance in the data. Thus, re-writing the data on such basis and ignoring the coefficients after the first few is a common technique for Dimensionality Reduction. The calculation of the N Principal Components is referred to as Principal Component Analysis, or PCA.
Related concepts:
Dimensionality Reduction