... | ... | @@ -23,4 +23,22 @@ The machine learning models we have covered so far can also be interpreted from |
|
|
|
|
|
## Dimensionality reduction methods
|
|
|
|
|
|
... |
|
|
\ No newline at end of file |
|
|
PCA is one of the most popular dimensionality reduction methods. It is a linear, orthogonal projection method where the high dimensional data is reflected onto a lower dimensional space in a way the variance in the projected data is maximized. We can again make an analogy with the shadow game. This time our objective is to find the right direction for the light so that the features of the object with high dimensions (3D) is kept as much as possible in the lower dimensional space (2D). In other words, we will perform the data projection in a way that it minimizes the information loss.
|
|
|
|
|
|
How does the data compression process work? We again have the data matrix X, where each row represents a different instance, while the columns (dimensions) are the features. The process is distance based (Euclidian).
|
|
|
|
|
|
The first step of the process is to figure out the row wise mean values for all the features, so that we can look at the covariance. For N number of examples:
|
|
|
|
|
|
```math
|
|
|
\overline{x}= 1/N \sum_{n=1}^{N} x_{n}
|
|
|
|
|
|
\overline{X}= \begin{bmatrix} 1 \\ . \\ 1 \end{bmatrix} \overline{x}
|
|
|
```
|
|
|
|
|
|
Then, we used the mean values of the feautres to re-center it around zero:
|
|
|
|
|
|
```math
|
|
|
X = X - \overline{X}
|
|
|
```
|
|
|
|
|
|
In the next step, |