Using the facts on repeated conditioning and the equivalent conditions
for independence, we may produce a similar table of equivalent conditions
for conditional independence. In the hybrid notation we use for repeated
conditioning, we write
P
C
(
A
|
B
)
=
P
C
(
A
)
or
P
C
(
A
B
)
=
P
C
(
A
)
P
C
(
B
)
P
C
(
A
|
B
)
=
P
C
(
A
)
or
P
C
(
A
B
)
=
P
C
(
A
)
P
C
(
B
)
(9)This translates into
P
(
A
|
B
C
)
=
P
(
A
|
C
)
or
P
(
A
B
|
C
)
=
P
(
A
|
C
)
P
(
B
|
C
)
P
(
A
|
B
C
)
=
P
(
A
|
C
)
or
P
(
A
B
|
C
)
=
P
(
A
|
C
)
P
(
B
|
C
)
(10)If it is known that C has occurred, then additional knowledge of the
occurrence of B does not change the likelihood of A.
If we write the sixteen equivalent conditions for independence in terms of the
conditional probability measure PC(·)PC(·), then translate as
above, we have the following equivalent conditions.
Table 1: Sixteen equivalent conditions
|
P
(
A
|
B
C
)
=
P
(
A
|
C
)
P
(
A
|
B
C
)
=
P
(
A
|
C
)
|
P
(
B
|
A
C
)
=
P
(
B
|
C
)
P
(
B
|
A
C
)
=
P
(
B
|
C
)
|
P
(
A
B
|
C
)
=
P
(
A
|
C
)
P
(
B
|
C
)
P
(
A
B
|
C
)
=
P
(
A
|
C
)
P
(
B
|
C
)
|
|
P
(
A
|
B
c
C
)
=
P
(
A
|
C
)
P
(
A
|
B
c
C
)
=
P
(
A
|
C
)
|
P
(
B
c
|
A
C
)
=
P
(
B
c
|
C
)
P
(
B
c
|
A
C
)
=
P
(
B
c
|
C
)
|
P
(
A
B
c
|
C
)
=
P
(
A
|
C
)
P
(
B
c
|
C
)
P
(
A
B
c
|
C
)
=
P
(
A
|
C
)
P
(
B
c
|
C
)
|
|
P
(
A
c
|
B
C
)
=
P
(
A
c
|
C
)
P
(
A
c
|
B
C
)
=
P
(
A
c
|
C
)
|
P
(
B
|
A
c
C
)
=
P
(
B
|
C
)
P
(
B
|
A
c
C
)
=
P
(
B
|
C
)
|
P
(
A
c
B
|
C
)
=
P
(
A
c
|
C
)
P
(
B
|
C
)
P
(
A
c
B
|
C
)
=
P
(
A
c
|
C
)
P
(
B
|
C
)
|
|
P
(
A
c
|
B
c
C
)
=
P
(
A
c
|
C
)
P
(
A
c
|
B
c
C
)
=
P
(
A
c
|
C
)
|
P
(
B
c
|
A
c
C
)
=
P
(
B
c
|
C
)
P
(
B
c
|
A
c
C
)
=
P
(
B
c
|
C
)
|
P
(
A
c
B
c
|
C
)
=
P
(
A
c
|
C
)
P
(
B
c
|
C
)
P
(
A
c
B
c
|
C
)
=
P
(
A
c
|
C
)
P
(
B
c
|
C
)
|
Table 2
|
P
(
A
|
B
C
)
=
P
(
A
|
B
c
C
)
P
(
A
|
B
C
)
=
P
(
A
|
B
c
C
)
|
P
(
A
c
|
B
C
)
=
P
(
A
c
|
B
c
C
)
P
(
A
c
|
B
C
)
=
P
(
A
c
|
B
c
C
)
|
P
(
B
|
A
C
)
=
P
(
B
|
A
c
C
)
P
(
B
|
A
C
)
=
P
(
B
|
A
c
C
)
|
P
(
B
c
|
A
C
)
=
P
(
B
c
|
A
c
C
)
P
(
B
c
|
A
C
)
=
P
(
B
c
|
A
c
C
)
|
The patterns of conditioning in the examples above belong
to this set. In a given problem, one or the other of these conditions may seem a reasonable
assumption. As soon as one of these patterns is recognized, then
all are equally valid assumptions. Because of its simplicity and
symmetry, we take as the defining condition
the product ruleP(AB|C)=P(A|C)P(B|C)P(AB|C)=P(A|C)P(B|C).
Definition. A pair of events {A,B}{A,B} is said to be
conditionally independent, givenC, designated {A,B} ci |C{A,B} ci |C
iff the following product rule holds: P(AB|C)=P(A|C)P(B|C)P(AB|C)=P(A|C)P(B|C).
The equivalence of the four entries in the
right hand column of the upper part of the table, establish
The replacement rule
If any of the pairs {A,B},{A,Bc},{Ac,B}{A,B},{A,Bc},{Ac,B}, or {Ac,Bc}{Ac,Bc}
is conditionally independent, given C, then so are the others.
— □□
This may be expressed by saying that if a pair is conditionally independent, we may replace
either or both by their complements and still have a conditionally independent pair.
To illustrate further the usefulness of this concept, we note some other common
examples in which similar conditions hold: there is
operational independence, but some chance factor which affects both.
- Two contractors work quite independently on jobs in the same city. The
operational independence suggests probabilistic independence. However, both
jobs are outside and subject to delays due to bad weather. Suppose A is the
event the first contracter completes his job on time and B is the event the
second completes on time. If C is the event of “good” weather, then arguments
similar to those in Examples 1 and 2 make it
seem reasonable to suppose {A,B} ci |C{A,B} ci |C and {A,B} ci |Cc{A,B} ci |Cc. Remark. In formal probability theory, an event must be sharply defined:
on any trial it occurs or it does not. The event of “good weather” is not
so clearly defined. Did a trace of rain or thunder in the area constitute
bad weather? Did rain delay on one day in a month long project constitute
bad weather? Even with this ambiguity, the pattern of probabilistic analysis
may be useful.
- A patient goes to a doctor. A preliminary examination leads the doctor
to think there is a thirty percent chance the patient has a certain disease.
The doctor orders two independent tests for conditions that indicate the
disease. Are results of these tests really independent? There is certainly
operational independence—the tests may be done by different laboratories,
neither aware of the testing by the others. Yet, if the tests are
meaningful, they must both be affected by the actual condition of the patient.
Suppose D is the event the patient has the disease, A is the event the
first test is positive (indicates the conditions associated with the disease)
and B is the event the second test is positive. Then it would seem
reasonable to suppose {A,B} ci |D{A,B} ci |D and {A,B} ci |Dc{A,B} ci |Dc.
In the examples considered so far, it has been reasonable to assume
conditional independence, given an event C, and conditional independence,
given the complementary event. But there are cases in which the effect of
the conditioning event is asymmetric. We consider several examples.
- Two students are working on a term paper. They work quite separately. They
both need to borrow a certain book from the library. Let C be the event
the library has two copies available. If A is the event the first completes
on time and B the event the second is successful, then it seems reasonable
to assume {A,B} ci |C{A,B} ci |C. However, if only one book is available, then
the two conditions would not be conditionally independent. In general
P(B|ACc)<P(B|Cc)P(B|ACc)<P(B|Cc), since if the first student completes on time, then
he or she must have been successful in getting the book, to the detriment
of the second.
- If the two contractors of the example above both need material which may be
in scarce supply, then successful completion would be conditionally independent,
give an adequate supply, whereas they would not be conditionally independent,
given a short supply.
- Two students in the same course take an exam. If they prepared separately,
the event of both getting good grades should be conditionally independent. If
they study together, then the likelihoods of good grades would not be independent.
With neither cheating or collaborating on the test itself, if one does well, the
other should also.
Since conditional independence is ordinary independence with respect to a conditional
probability measure, it should be clear how to extend the concept to larger classes of sets.
Definition. A class {Ai:i∈J}{Ai:i∈J}, where J is an arbitrary index set,
is conditionally independent, given event C, denoted {Ai:i∈J} ci |C{Ai:i∈J} ci |C,
iff the product rule holds for every finite subclass of two or more.
As in the case of simple independence, the replacement rule extends.
The replacement rule
If the class {Ai:i∈J} ci |C{Ai:i∈J} ci |C, then any or all of the events Ai may
be replaced by their complements and still have a conditionally independent class.
Catalogue of Useful Matlab Files
