... | ... | @@ -163,54 +163,53 @@ At this point, after our discussions on regression, classification and clusterin |
|
|
|
|
|
Statistical independence is expressed in terms of PDFs. Two variables i & j are independent if the [joint PDF](https://en.wikipedia.org/wiki/Joint_probability_distribution) for i & j is:
|
|
|
|
|
|
´´´math
|
|
|
```math
|
|
|
p_{xy}(i,j)=p_i(i)p_j(j)
|
|
|
´´´
|
|
|
```
|
|
|
|
|
|
In simpler terms, it means if we know the PDFs of i & j, their joint PDF can be constructed. This implies that:
|
|
|
|
|
|
´´´math
|
|
|
```math
|
|
|
E(i^p,j^q) = E(i^p)E(j^q)
|
|
|
´´´
|
|
|
```
|
|
|
|
|
|
Let's start with something we already see in the lecture, p=q=1:
|
|
|
|
|
|
´´´math
|
|
|
```math
|
|
|
E(i,j) = E(i)E(j)
|
|
|
´´´
|
|
|
```
|
|
|
|
|
|
where ´$E(i,j)$´ is the covariance between i and j (both i and j are zero mean):
|
|
|
where `$E(i,j)$` is the covariance between i and j (both i and j are zero mean):
|
|
|
|
|
|
´´´math
|
|
|
```math
|
|
|
E(i,j) = \sum_{n=1}^{N} i_nj_n
|
|
|
´´´
|
|
|
```
|
|
|
|
|
|
"Correlation" is nothing but the normalized covariance:
|
|
|
|
|
|
´´´math
|
|
|
```math
|
|
|
\rho(i,j) = \sum_{n=1}^{N} i_nj_n / \sigma_i\sigma_j
|
|
|
|
|
|
´´´
|
|
|
```
|
|
|
|
|
|
where ´$\sigma$´ is the standard deviation:
|
|
|
|
|
|
´´´math
|
|
|
```math
|
|
|
\sigma_i = E(i,i)^0.5, \sigma_j = E(j,j)^0.5
|
|
|
´´´
|
|
|
```
|
|
|
|
|
|
We already talked in the lecture that this normalization helps us to scale between -1 and 1. To give you vivid examples, let's see the limits. ´$\rho(i,j)=1$´ means that i and j increases in proportion (j in proportion to i). 0 means j does not increase or increase (in average behavior of course) as x increases. If ´$\rho(i,j)=-1$´, y decreases in proportion to x, as it increases.
|
|
|
We already talked in the lecture that this normalization helps us to scale between -1 and 1. To give you vivid examples, let's see the limits. `$\rho(i,j)=1$` means that i and j increases in proportion (j in proportion to i). 0 means j does not increase or increase (in average behavior of course) as x increases. If `$\rho(i,j)=-1$`, y decreases in proportion to x, as it increases.
|
|
|
|
|
|
To check whether correlation exists or not, we look at the a limited case, the first moment of the joint PDF (p=q=1) and say that they are uncorrelated if:
|
|
|
|
|
|
´´´math
|
|
|
```math
|
|
|
E(i^p,j^q) = E(i^p)E(j^q)
|
|
|
´´´
|
|
|
```
|
|
|
|
|
|
In order to claim that it is independent, the general form of the equation must be satisfied (p>0, q>0; where p & q are integers). So our criteria is:
|
|
|
|
|
|
´´´math
|
|
|
```math
|
|
|
\rho(i^p,j^q) = 0
|
|
|
´´´
|
|
|
```
|
|
|
|
|
|
|
|
|
|
... | ... | |