Conditional probability

The calculations, as in [link] , are simple but can be tedious. We have an m-procedure called bayes to perform the calculations easily. The probabilities $P\left(A_i\right)$ are put into a matrix PA and the conditional probabilities $P\left(E\right|A_i)$ are put into matrix PEA. The desired probabilities $P\left(A_i\right|E)$ and $P\left(A_i\right|E^c)$ are calculated and displayed

The following variation of Bayes' rule is applicable in many practical situations.

(CP3*) Ratio form of Bayes' rule ${\displaystyle \frac{P\left(A\right|C)}{P\left(B\right|C)}=\frac{P\left(AC\right)}{P\left(BC\right)}=\frac{P\left(C\right|A)}{P\left(C\right|B)}\cdot \frac{P\left(A\right)}{P\left(B\right)}}$

The left hand member is called the posterior odds , which is the odds after knowledge of the occurrence of the conditioning event. The second fraction in the right hand member is the prior odds , which is the odds before knowledge of the occurrence of the conditioning event C . The first fraction in the right hand member is known as the likelihood ratio . It is the ratio of the probabilities (or likelihoods) of C for the two different probability measures $P(\cdot |A)$ and $P(\cdot |B)$ .

The following property serves to establish in the chapters on "Independence of Events" and "Conditional Independence" a number of important properties for the concept of independence and of conditional independence of events.

(CP4) Some equivalent conditions If $0<P\left(A\right)<1$ and $0<P\left(B\right)<1$ , then

where $*\text{is}<,\le ,=,\ge ,\text{or}>$ and $\diamond \text{is}>,\ge ,=,\le ,\text{or}<$ , respectively.

Because of the role of this property in the theory of independence and conditional independence, we examine the derivation of these results.

VERIFICATION of (CP4)

1. $P\left(AB\right)*P\left(A\right)P\left(B\right)$ iff $P\left(A\right|B)*P(A)$ (divide by $P\left(B\right)$ — may exchange A and A ^c )
2. $P\left(AB\right)*P\left(A\right)P\left(B\right)$ iff $P\left(B\right|A)*P(B)$ (divide by $P\left(A\right)$ — may exchange B and B ^c )
3. $P\left(AB\right)*P\left(A\right)P\left(B\right)$ iff $[P\left(A\right)-P\left(A{B}^c\right)]*P\left(A\right)[1-P\left(B^c\right)$ iff $-P\left(A{B}^c\right)*-P\left(A\right)P\left(B^c\right)$ iff $P\left(A{B}^c\right)\diamond P\left(A\right)P\left(B^c\right)$
4. We may use c to get $P\left(AB\right)*P\left(A\right)P\left(B\right)$ iff $P\left(A{B}^c\right)\diamond P\left(A\right)P\left(B^c\right)$ iff $P\left(A^c{B}^c\right)*P\left(A^c\right)P\left(B^c\right)$

— $\square$

A number of important and useful propositons may be derived from these.

1. $P\left(A\right|B)+P\left(A^c\right|B)=1$ , but, in general, $P\left(A\right|B)+P\left(A\right|B^c)\ne 1$ .
2. $P\left(A\right|B)>P(A)$ iff $P\left(A\right|B^c)<P\left(A\right)$ .
3. $P\left(A^c\right|B)>P\left(A^c\right)$ iff $P\left(A\right|B)<P(A)$ .
4. $P\left(A\right|B)>P(A)$ iff $P\left(A^c\right|B^c)>P\left(A^c\right)$ .

VERIFICATION — Exercises (see problem set)

— $\square$

Repeated conditioning

Suppose conditioning by the event C has occurred. Additional information is then received that event D has occurred. We have a new conditioning event $CD$ . There are two possibilities:

1. Reassign the conditional probabilities. $P_C\left(A\right)$ becomes
   $$P_C\left(A\right|D)=\frac{P_C\left(AD\right)}{P_C\left(D\right)}=\frac{P\left(ACD\right)}{P\left(CD\right)}$$
2. Reassign the total probabilities: $P\left(A\right)$ becomes
   $$P_{CD}\left(A\right)=P\left(A\right|CD)=\frac{P\left(ACD\right)}{P\left(CD\right)}$$

Basic result : $P_C\left(A\right|D)=P\left(A\right|CD)=P_D\left(A\right|C)$ . Thus repeated conditioning by two events may be done in any order, or may be done in one step. This result extends easilyto repeated conditioning by any finite number of events. This result is important in extending the concept of "Independence of Events" to "Conditional Independence" . These conditions are important for many problems of probable inference.