11.1 Mathematical expectation; general random variables

In this unit, we extend the definition and properties of mathematical expectation to the general case. In the process, we note the relationship ofmathematical expectation to the Lebesque integral, which is developed in abstract measure theory. Although we do not develop this theory, which lies beyond the scope of this study, identificationof this relationship provides access to a rich and powerful set of properties which have far reaching consequences in both application and theory.

Extension to the general case

In the unit on Distribution Approximations , we show that a bounded random variable X can be represented as the limit of a nondecreasing sequence of simple random variables. Also, a real random variablecan be expressed as the difference $X=X^{+}-X^{-}$ of two nonnegative random variables. The extension of mathematical expectation to the general case is based on these factsand certain basic properties of simple random variables, some of which are established in the unit on expectation for simple random variables. We list these properties and sketch how the extension is accomplished.

Definition. A condition on a random variable or on a relationship between random variables is said to hold almost surely , abbreviated “a.s.” iff the condition or relationship holds for all ω except possibly a set with probability zero.

Basic properties of simple random variables

• (E0) If $X=Y\mathrm{a}.\mathrm{s}.$ then $E\left[X\right]=E\left[Y\right]$ .
• (E1) $E\left[a{I}_E\right]=aP\left(E\right)$ .
• (E2) Linearity . ${\displaystyle X=\sum _{i=1}^n{a}_i{X}_i\text{implies}E\left[X\right]=\sum _{i=1}^n{a}_{i}E\left[X_i\right]}$
• (E3) Positivity; monotonicity
  1. If $X\ge 0\mathrm{a}.\mathrm{s}.$ , then $E\left[X\right]\ge 0$ , with equality iff $X=0\mathrm{a}.\mathrm{s}.$ .
  2. If $X\ge Y\mathrm{a}.\mathrm{s}.$ , then $E\left[X\right]\ge E\left[Y\right]$ , with equality iff $X=Y\mathrm{a}.\mathrm{s}.$
• (E4) Fundamental lemma If $X\ge 0$ is bounded and $\{X_n:1\le n\}$ is an a.s. nonnegative, nondecreasing sequence with ${\displaystyle \underset{n}{lim}{X}_n\left(\omega \right)\ge X\left(\omega \right)}$ for almost every ω , then ${\displaystyle \underset{n}{lim}E\left[X_n\right]\ge E\left[X\right]}$ .
• (E4a) If for all n , $0\le X_n\le X_{n+1}\mathrm{a}.\mathrm{s}.$ and $X_n\to X\mathrm{a}.\mathrm{s}.$ , then $E\left[X_n\right]\to E\left[X\right]$ (i.e., the expectation of the limit is the limit of the expectations).

Ideas of the proofs of the fundamental properties

• Modifying the random variable X on a set of probability zero simply modifies one or more of the A _i without changing $P\left(A_i\right)$ . Such a modification does not change $E\left[X\right]$ .
• Properties (E1) and (E2) are established in the unit on expectation of simple random variables..
• Positivity (E3a) is a simple property of sums of real numbers. Modification of sets of probability zero cannot affect the expectation.
• Monotonicity (E3b) is a consequence of positivity and linearity.
  $$X\ge Y\text{iff}X-Y\ge 0\mathrm{a}.\mathrm{s}.\text{and}E\left[X\right]\ge E\left[Y\right]\text{iff}E\left[X\right]-E\left[Y\right]=E[X-Y]\ge 0$$
• The fundamental lemma (E4) plays an essential role in extending the concept of expectation. It involves elementary, but somewhat sophisticated, use of linearity and monotonicity, limitedto nonnegative random variables and positive coefficients. We forgo a proof.
• Monotonicity and the fundamental lemma provide a very simple proof of the monotone convergence theoem, often designated MC. Its role is essential in the extension.