Conditional independence, given a random vector

In the unit on Conditional Independence , the concept of conditional independence of events is examined andused to model a variety of common situations. In this unit, we investigate a more general concept of conditional independence, based on the theory of conditionalexpectation. This concept lies at the foundations of Bayesian statistics, of many topics in decision theory, and of the theory of Markov systems. We examine in this unit, verybriefly, the first of these. In the unit on Markov Sequences , we provide an introduction to the third.

The concept

The definition of conditional independence of events is based on a product rule which may be expressed in terms of conditional expectation, given an event. The pair $\{A,B\}$ is conditionally independent, given C , iff

If we let $A=X^{-1}\left(M\right)$ and $B=Y^{-1}\left(N\right)$ , then $I_A=I_M\left(X\right)$ and $I_B=I_N\left(Y\right)$ . It would be reasonable to consider the pair $\{X,Y\}$ conditionally independent, given event C , iff the product rule

holds for all reasonable M and N (technically, all Borel M and N ). This suggests a possible extension to conditional expectation, given a random vector.We examine the following concept.

Definition . The pair $\{X,Y\}$ is conditionally independent, given Z , designated $\{X,Y\}\mathrm{ci}|Z$ , iff

Remark . Since it is not necessary that $X,Y$ , or Z be real valued, we understand that the sets M and N are on the codomains for X and Y , respectively. For example, if X is a three dimensional random vector, then M is a subset of R ³ .

As in the case of other concepts, it is useful to identify some key properties, which we refer to by the numbers used in the table in Appendix G. We note twokinds of equivalences. For example, the following are equivalent.

(CI1) $E\left[I_M\left(X\right)I_N\left(Y\right)\right|Z]=E\left[I_M\left(X\right)\right|Z]E\left[I_N\left(Y\right)\right|Z]\mathrm{a}.\mathrm{s}.\text{for}\text{all}\text{Borel}\text{sets}M,N$

(CI5) $E\left[g\right(X,Z\left)h\right(Y,Z\left)\right|Z]=E[g(X,Z)\left|Z\right]E\left[h\right(Y,Z\left)\right|Z]\mathrm{a}.\mathrm{s}.\text{for}\text{all}\text{Borel}\text{functions}g,h$

Because the indicator functions are special Borel functions, (CI1) is a special case of (CI5) . To show that (CI1) implies (CI5) , we need to use linearity, monotonicity, and monotone convergence in a manner similar to that used in extending properties (CE1) to (CE6) for conditional expectation.A second kind of equivalence involves various patterns. The properties (CI1) , (CI2) , (CI3) , and (CI4) are equivalent, with (CI1) being the defining condition for $\{X,Y\}\mathrm{ci}|Z$ .