Why you can add means and expected values

In applied statistics you often have to combine data from different samples or distributions. One of the most frequently used operation here is to add means and expected values. For instance, you could sample people’s leg length, body and head height. The result when adding these means? It is the average body height, I hope!

Relationship between expected value and mean

My reasoning, why adding means works, will be based on expected values. The expected value of a random variable is defined as

$\displaystyle \mathrm{E}[X]=\sum \limits_{x} x \times p_{x}(x)$

where $p_{x}(x)$ is the probability of receiving a specified value $x$ when drawing from a specified population. In case you already received a sample from an unknown population, you may think of the probability for each value, which $x$ can take, to be 1/n. You then receive the well-known formula

$\displaystyle \overline{x}=\frac{1}{n} \sum\limits_{x} x=\sum\limits_{x} \frac{x}{n} \approx \mathrm{E}[X]$

Mean or expected value; the subsequent explanation is valid either way.

Adding two random variables

Now let’s assume we wanted to add two random variables – say $X$ and $Y$ – and calculate the expected value. Instead of $p_{x}(x)$ we now need to give the probability when the random variable $X$ takes a value $x$ AND the random variable $Y$ takes a value $y$ . We are going to call this probability

$p_{x,y}(x,y)=P\big(X=x, Y=y\big)$

With two variables we now have to process all possible combinations of $x$ and $y$ . Here is the result:

$\displaystyle \mathrm{E}\big[X+Y\big]=\sum\limits_{x} \sum\limits_{y} (x+y) \times p_{x,y}(x,y)=\sum\limits_{x} \sum\limits_{y} \big(x \times p_{x,y}(x,y)+y \times p_{x,y}(x,y)\big)$

We can split the right-hand side into two separat sums. As it doesn’t matter whether you sum the values for $x$ or the ones for $y$ first, we can swap the order of the second term.

$\displaystyle \mathrm{E}\big[X+Y\big]=\sum\limits_{x} \sum\limits_{y} x\times p_{x,y}(x,y)+ \sum\limits_{x} \sum\limits_{y} y\times p_{x,y}(x,y)$

$\displaystyle \mathrm{E}\big[X+Y\big]=\sum\limits_{x} \sum\limits_{y} x\times p_{x,y}(x,y)+ \sum\limits_{y} \sum\limits_{x} y\times p_{x,y}(x,y)$

This is where it gets tricky. On the one hand, we can drag $x$ out of the sum of all $y$ , because $x$ is constant in terms of $y$ . The same accounts for $y$ .

$\displaystyle \mathrm{E}\big[X+Y\big]=\sum\limits_{x} x \times \sum\limits_{y} p(x,y)+ \sum\limits_{x} y\times \sum\limits_{y} p(x,y)$

Further we can simplify the probabilities. See, adding $p_{x,y}(x,y)$ for all $x$ will result in $p_{y}(y)$ and vice versa.

$\displaystyle \mathrm{E}\big[X+Y\big]=\sum\limits_{x} x \times p_{x}(x)+ \sum\limits_{y} y\times p_{y}(y)$

But what do we have here? Each of the right-hand side’s terms is equal to the definition of an expected value!

$\mathrm{E}\big[X+Y\big]=\mathrm{E}\big[X\big] + \mathrm{E}\big[Y\big]$

Some comments

The last equation has shown that we can, indeed, add expected values. Have you noticed that we didn’t make any assumptions regarding the independence of the random variables? Hence, the rule holds even if your random variables are correlated! And remember the introductory example? We used a continous measure there. Just replace all sum signs by integral signs and you’ll see that the rule works with continous random variables too.

Always keep in mind that you can apply the addition to means as well. So, the mean $\overline{x+y}$ can be calculated by adding $\overline{x}$ and $\overline{y}$ . But this is what most people intuitively do already.