1.13 Correlation and covariance of a random signal

When we take the expected value , or average, of a random process , we measure several important characteristics about how the process behavesin general. This proves to be a very important observation. However, suppose we have several random processes measuringdifferent aspects of a system. The relationship between these different processes will also be an important observation. Thecovariance and correlation are two important tools in finding these relationships. Below we will go into more details as towhat these words mean and how these tools are helpful. Note that much of the following discussions refer to just randomvariables, but keep in mind that these variables can represent random signals or random processes.

Covariance

To begin with, when dealing with more than one random process, it should be obvious that it would be nice to be able to havea number that could quickly give us an idea of how similar the processes are. To do this, we use the covariance , which is analogous to the variance of a single variable.

Covariance

A measure of how much the deviations of two or more variables or processes match.

Mathematically, covariance is often written as ${\sigma }_{xy}()$ and is defined as

Useful properties

• If $X$ and $Y$ are independent and uncorrelated or one of them has zero mean value, then $${\sigma }_{xy}()=0$$
• If $X$ and $Y$ are orthogonal, then $${\sigma }_{xy}()=-((X)(Y))$$
• The covariance is symmetric $$\mathrm{cov}(X, Y)=\mathrm{cov}(Y, X)$$
• Basic covariance identity $$\mathrm{cov}(X+Y, Z)=\mathrm{cov}(X, Z)+\mathrm{cov}(Y, Z)$$
• Covariance of equal variables $$\mathrm{cov}(X, X)=\mathrm{Var}(X)$$

Correlation

For anyone who has any kind of statistical background, you should be able to see that the idea of dependence/independenceamong variables and signals plays an important role when dealing with random processes. Because of this, the correlation of two variables provides us with a measure of how the two variables affect one another.

Correlation

A measure of how much one random variable depends upon the other.

Correlation coefficient

It is often useful to express the correlation of random variables with a range of numbers, like a percentage. For agiven set of variables, we use the correlation coefficient to give us the linear relationship between our variables. The correlation coefficient of twovariables is defined in terms of their covariance and standard deviations , denoted by ${\sigma }_x()$ , as seen below

Example

Now let us take just a second to look at a simple example that involves calculating the covariance and correlation of twosets of random numbers. We are given the following data sets: $$x=\{3, 1, 6, 3, 4\}()$$ $$y=\{1, 5, 3, 4, 3\}()$$ To begin with, for the covariance we will need to find the expected value , or mean, of $x$ and $y$ . $$\langle x\rangle =\frac{1}{5}(3+1+6+3+4)=3.4$$ $$\langle y\rangle =\frac{1}{5}(1+5+3+4+3)=3.2$$ $$\langle xy\rangle =\frac{1}{5}(3+5+18+12+12)=10$$ Next we will solve for the standard deviations of our two setsusing the formula below (for a review click here ). $$\sigma =\sqrt{((X-(X))^2)}$$ $${\sigma }_x()=\sqrt{\frac{1}{5}(0.16+5.76+6.76+0.16+0.36)}=1.625$$ $${\sigma }_y()=\sqrt{\frac{1}{6}(4.84+3.24+0.04+0.64+0.04)}=1.327$$ Now we can finally calculate the covariance using one of the two formulas found above. Since we calculated the threemeans, we will use that formula since it will be much simpler. $${\sigma }_{xy}()=10-3.4\times 3.2=-0.88$$ And for our last calculation, we will solve for thecorrelation coefficient, $\rho$ . $$\rho =\frac{-0.88}{1.625\times 1.327}=-0.408$$

Matlab code for example

The above example can be easily calculated using Matlab. Below I have included the code to find all of the valuesabove.