Mathematical expectation: simple random variables

Introduction

The probability that real random variable X takes a value in a set M of real numbers is interpreted as the likelihood that the observed value $X\left(\omega \right)$ on any trial will lie in M . Historically, this idea of likelihood is rooted in the intuitive notion that if the experiment is repeated enough times the probability is approximately the fraction of times the value of X will fall in M . Associated with this interpretation is the notion of the average of the values taken on. We incorporate the concept of mathematical expectation into the mathematical model as an appropriate form of such averages.We begin by studying the mathematical expectation of simple random variables, then extend the definition and properties to the general case. In the process, we note the relationship ofmathematical expectation to the Lebesque integral, which is developed in abstract measure theory. Although we do not develop this theory, which lies beyond the scope of this study, identificationof this relationship provides access to a rich and powerful set of properties which have far reaching consequences in both application and theory.

Expectation for simple random variables

The notion of mathematical expectation is closely related to the idea of a weighted mean, used extensively in the handling of numerical data. Consider the arithmetic average $\overline{x}$ of the following ten numbers: 1, 2, 2, 2, 4, 5, 5, 8, 8, 8, which is given by

Examination of the ten numbers to be added shows that five distinct values are included. One of the ten, or the fraction 1/10 of them, has the value 1, three of the ten, or the fraction 3/10 of them,have the value 2, 1/10 has the value 4, 2/10 have the value 5, and 3/10 have the value 8. Thus, we could write

The pattern in this last expression can be stated in words: Multiply each possible value by the fraction of the numbers having that value and then sum these products. The fractions are often referred to as the relative frequencies . A sum of this sort is known as a weighted average .

In general, suppose there are n numbers $\{x_1,x_2,\cdots x_n\}$ to be averaged, with $m\le n$ distinct values $\{t_1,t_2,\cdots ,t_m\}$ . Suppose f ₁ have value t ₁ , f ₂ have value $t_2,\cdots ,f_m$ have value t _m . The f _i must add to n . If we set $p_i=f_i/n$ , then the fraction p _i is called the relative frequency of those numbers in the set which have the value $t_i,1\le i\le m$ . The average $\overline{x}$ of the n numbers may be written

In probability theory, we have a similar averaging process in which the relative frequencies of the various possible values of are replaced by the probabilities that those values are observed onany trial.

Definition . For a simple random variable X with values $\{t_1,t_2,\cdots ,t_n\}$ and corresponding probabilities $p_i=P(X=t_i)$ , the mathematical expectation , designated $E\left[X\right]$ , is the probability weighted average of the values taken on by X . In symbols