Conditional expectation, regression (Page 2/9)

Applied probability Page 2 / 9

Definition . The conditional expectation of X , given event C with positive probability, is the quantity

E [X | C] = \frac{E [I_{C} X]}{P (C)} = \frac{E [I_{C} X]}{E [I_{C}]}

Remark . The product form $E [X | C] P (C) = E [I_{C} X]$ is often useful.

A numerical example

Suppose $X \sim$ exponential $(λ)$ and $C = {1 / λ \leq X \leq 2 / λ}$ . Now $I_{C} = I_{M} (X)$ where $M = [1 / λ, 2 / λ]$ .

P (C) = P (X \geq 1 / λ) - P (X > 2 / λ) = e^{- 1} - e^{- 2} and

E [I_{C} X] = \int I_{M} (t) t λ e^{- λ t} d t = \int_{1 / λ}^{2 / λ} t λ e^{- λ t} d t = \frac{1}{λ} (2 e^{- 1} - 3 e^{- 2})

Thus

E [X | C] = \frac{2 e^{- 1} - 3 e^{- 2}}{λ (e^{- 1} - e^{- 2})} \approx \frac{1.418}{λ}

Got questions? Get instant answers now!

Conditioning by a random vector—discrete case

Suppose $X = \sum_{i = 1}^{n} t_{i} I_{A_{i}}$ and $Y = \sum_{j = 1}^{m} u_{j} I_{B_{j}}$ in canonical form. We suppose $P (A_{i}) = P (X = t_{i}) > 0$ and $P (B_{j}) = P (Y = u_{j}) > 0$ , for each permissible $i, j$ . Now

P (Y = u_{j} | X = t_{i}) = \frac{P (X = t_{i}, Y = u_{j})}{P (X = t_{i})}

We take the expectation relative to the conditional probability $P (\cdot | X = t_{i})$ to get

E [g (Y) | X = t_{i}] = \sum_{j = 1}^{m} g (u_{j}) P (Y = u_{j} | X = t_{i}) = e (t_{i})

Since we have a value for each t _i in the range of X , the function $e (\cdot)$ is defined on the range of X . Now consider any reasonable set M on the real line and determine the expectation

E [I_{M} (X) g (Y)] = \sum_{i = 1}^{n} \sum_{j = 1}^{m} I_{M} (t_{i}) g (u_{j}) P (X = t_{i}, Y = u_{j})

= \sum_{i = 1}^{n} I_{M} (t_{i}) [\sum_{j = 1}^{m}, g, (u_{j}), P, (Y = u_{j} | X = t_{i})] P (X = t_{i})

= \sum_{i = 1}^{n} I_{M} (t_{i}) e (t_{i}) P (X = t_{i}) = E [I_{M} (X) e (X)]

We have the pattern

(A) E [I_{M} (X) g (Y)] = E [I_{M} (X) e (X)] where e (t_{i}) = E [g (Y) | X = t_{i}]

for all t _i in the range of X .

We return to examine this property later. But first, consider an example to display the nature of the concept.

Basic calculations and interpretation

Suppose the pair ${X, Y}$ has the joint distribution

P (X = t_{i}, Y = u_{j})

$X =$	0	1	4	9
$Y = 2$	0.05	0.04	0.21	0.15
0	0.05	0.01	0.09	0.10
-1	0.10	0.05	0.10	0.05
$P X$	0.20	0.10	0.40	0.30

Calculate $E [Y | X = t_{i}]$ for each possible value t _i taken on by X

$E [Y | X = 0] = - 1 \frac{0.10}{0.20} + 0 \frac{0.05}{0.20} + 2 \frac{0.05}{0.20}$
$= (- 1 \cdot 0.10 + 0 \cdot 0.05 + 2 \cdot 0.05) / 0.20 = 0$
$E [Y | X = 1] = (- 1 \cdot 0.05 + 0 \cdot 0.01 + 2 \cdot 0.04) / 0.10 = 0.30$
$E [Y | X = 4] = (- 1 \cdot 0.10 + 0 \cdot 0.09 + 2 \cdot 0.21) / 0.40 = 0.80$
$E [Y | X = 9] = (- 1 \cdot 0.05 + 0 \cdot 0.10 + 2 \cdot 0.15) / 0.10 = 0.83$

The pattern of operation in each case can be described as follows:

For the i th column, multiply each value u _j by $P (X = t_{i}, Y = u_{j})$ , sum, then divide by $P (X = t_{i})$ .

The following interpretation helps visualize the conditional expectation and points to an important result in the general case.

For each t _i we use the mass distributed “above” it. This mass is distributed along a vertical line at values u _j taken on by Y . The result of the computation is to determine the center of mass for the conditional distribution above $t = t_{i}$ . As in the case of ordinary expectations, this should be the best estimate, in the mean-square sense, of Y when $X = t_{i}$ . We examine that possibility in the treatment of the regression problem in [link] .

Although the calculations are not difficult for a problem of this size, the basic pattern can be implemented simply with MATLAB, making the handling of much larger problems quite easy. Thisis particularly useful in dealing with the simple approximation to an absolutely continuous pair.

X = [0 1 4 9];             % Data for the joint distributionY = [-1 0 2];P = 0.01*[ 5  4 21 15; 5  1  9 10; 10  5 10  5];jcalc                      % Setup for calculations
Enter JOINT PROBABILITIES (as on the plane)  PEnter row matrix of VALUES of X  X
Enter row matrix of VALUES of Y  YUse array operations on matrices X, Y, PX, PY, t, u, and P
EYX = sum(u.*P)./sum(P);   % sum(P) = PX  (operation sum yields column sums)disp([X;EYX]')             % u.*P = u_j P(X = t_i, Y = u_j) for all i, j0         0
    1.0000    0.30004.0000    0.8000
    9.0000    0.8333

<< Chapter < Page Page > Chapter >>

Read also:

Get Jobilize Job Search Mobile App in your pocket Now!

100% Free Mobile Applications
Receive real-time job alerts and never miss the right job again

Source: OpenStax, Applied probability. OpenStax CNX. Aug 31, 2009 Download for free at http://cnx.org/content/col10708/1.6

Google Play and the Google Play logo are trademarks of Google Inc.

Notification Switch

Would you like to follow the 'Applied probability' conversation and receive update notifications?

Ask

	27 AP 27 Reproductive System Essay By OpenStax Start Flashcards
	Subject-verb Agreement By Dindin Secreto Start Quiz
	4 BOD Hemolymphatic -Dr. Han By Brooke Delaney Start Exam
	17 AP 17 Endocrine System MCQ By OpenStax Start Quiz
	Art Reviewer MCQ By Edgar Delgado Start Quiz
	Real Estate Finance By Mldelatte Start Quiz
	Are you part of the R5 Family? By Anonymous User Start Quiz
	21 Kidney/Liver Biopath By Brooke Delaney Start Exam
	Social Dances 1 By Marion Cabalfin Start Test
	14 AP Key Terms 14 Brain Cranial Nerves By OpenStax Start Key Terms