...  ...  @@ 141,6 +141,25 @@ We have already discussed a component analysis method, PCA. At this point, you m 





In PCA, the way we extract the unit vectors of the new coordinate system relies on the variance. If we apply it to a problem composed on two independent phenomena, it will lead to a merged transformation, which is definitely wrong (see middle). In such cases, we first aim to filter out the these "independent" behaviors within the data (see right). So, how are we going to do that?






Let's remember the problem. We have mixed signals recorded as X, and they are mixed via some unknown "mixing coefficients" A. What we want is to figure out the unmixed signals, S. At this point, we may imagine a set of unmixing coefficients:






```math



s_1 = \alpha_{11}x_1 + \alpha_{12}x_2



```






```math



s_2 = \alpha_{21}x_1 + \alpha_{22}x_2



```






```math



S = WX



```






All we need to do is finding the values for $`\alpha_{ij}`$ in W. Note that we need to find W when A is unknown (we cannot simply use the inverse A). What we know though is, W defines the vectors in the mixture space and each vector (e.g. $`[\alpha_{11},\alpha_{12}`$) basically extracts one source signal (here it is $`\s{1}`). If you look at the above sketch of ICA, we see that these vectors must be orthogonal to the samples associated with all sources except the one it describes.So, we need to find W such that each vector in W is orthogonal to all sources but one. Okay, now we are getting closer to define an optimization problem.






We also said that we are after independent signals. By saying so, we assume that the sources reflect this property, than the merged signals. With this constraint, we can say, "I will find such a W that "independency" is maximized in the extracted signals.













Note: In ICA, we assume that we do not have Gaussian distributions in the variables.




...  ...  