# Connexions

You are here: Home » Content » Independence of Events

### Lenses

What is a lens?

#### Definition of a lens

##### Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

##### What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

##### Who can create a lens?

Any individual member, a community, or a respected organization.

##### What are tags?

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

#### Affiliated with (What does "Affiliated with" mean?)

This content is either by members of the organizations listed or about topics related to the organizations listed. Click each link to see a list of all content affiliated with the organization.
• Rice Digital Scholarship

This module is included in aLens by: Digital Scholarship at Rice UniversityAs a part of collection: "Applied Probability"

Click the "Rice Digital Scholarship" link to see all content affiliated with them.

#### Also in these lenses

• UniqU content

This module is included inLens: UniqU's lens
By: UniqU, LLCAs a part of collection: "Applied Probability"

Click the "UniqU content" link to see all content selected in this lens.

### Recently Viewed

This feature requires Javascript to be enabled.

# Independence of Events

Module by: Paul E Pfeiffer. E-mail the author

Summary: Historically, the notion of independence has played a prominent role in probability. If events form an independent class, much less information is required to determine probabilities of Boolean combinations and calculations are correspondingly easier. We give a precise formulation of the concept of independence in the probability sense. As in the case of all concepts which attempt to incorporate intuitive notions, the consequences must be evaluated for evidence that these ideas have been captured successfully. There are many situations in which we have an “operational independence,” which indicates that knowledge that one of the events has occured does not affect the likelihood that the other will occur. For a pair of events, this is suggested by the condition P(A|B) = P(A), which implies sixteen equivalent conditions. One of these, the product rule P(AB) = P(A)P(B), is taken as definitive. If any one holds so do all the others, including the product rule. For a larger class of events we require that the product rule hold for every finite subclass of two or more events in the class.

Historically, the notion of independence has played a prominent role in probability. If events form an independent class, much less information is required to determine probabilities of Boolean combinations and calculations are correspondingly easier. In this unit, we give a precise formulation of the concept of independence in the probability sense. As in the case of all concepts which attempt to incorporate intuitive notions, the consequences must be evaluated for evidence that these ideas have been captured successfully.

## Independence as lack of conditioning

There are many situations in which we have an “operational independence.”

• Supose a deck of playing cards is shuffled and a card is selected at random then replaced with reshuffling. A second card picked on a repeated try should not be affected by the first choice.
• If customers come into a well stocked shop at different times, each unaware of the choice made by the others, the the item purchased by one should not be affected by the choice made by the other.
• If two students are taking exams in different courses, the grade one makes should not affect the grade made by the other.

The list of examples could be extended indefinitely. In each case, we should expect to model the events as independent in some way. How should we incorporate the concept in our developing model of probability?

We take our clue from the examples above. Pairs of events are considered. The “operational independence” described indicates that knowledge that one of the events has occured does not affect the likelihood that the other will occur. For a pair of events {A,B}{A,B}, this is the condition

P ( A | B ) = P ( A ) P ( A | B ) = P ( A )
(1)

Occurrence of the event A is not “conditioned by” occurrence of the event B. Our basic interpretation is that P(A)P(A) indicates of the likelihood of the occurrence of event A. The development of conditional probability in the module Conditional Probability, leads to the interpretation of P(A|B)P(A|B) as the likelihood that A will occur on a trial, given knowledge that B has occurred. If such knowledge of the occurrence of B does not affect the likelihood of the occurrence of A, we should be inclined to think of the events A and B as being independent in a probability sense.

## Independent pairs

We take our clue from the condition P(A|B)=P(A)P(A|B)=P(A). Property (CP4) for conditional probability (in the case of equality) yields sixteen equivalent conditions as follows.

 P ( A | B ) = P ( A ) P ( A | B ) = P ( A ) P ( B | A ) = P ( B ) P ( B | A ) = P ( B ) P ( A B ) = P ( A ) P ( B ) P ( A B ) = P ( A ) P ( B ) P ( A | B c ) = P ( A ) P ( A | B c ) = P ( A ) P ( B c | A ) = P ( B c ) P ( B c | A ) = P ( B c ) P ( A B c ) = P ( A ) P ( B c ) P ( A B c ) = P ( A ) P ( B c ) P ( A c | B ) = P ( A c ) P ( A c | B ) = P ( A c ) P ( B | A c ) = P ( B ) P ( B | A c ) = P ( B ) P ( A c B ) = P ( A c ) P ( B ) P ( A c B ) = P ( A c ) P ( B ) P ( A c | B c ) = P ( A c ) P ( A c | B c ) = P ( A c ) P ( B c | A c ) = P ( B c ) P ( B c | A c ) = P ( B c ) P ( A c B c ) = P ( A c ) P ( B c ) P ( A c B c ) = P ( A c ) P ( B c )
 P ( A | B ) = P ( A | B c ) P ( A | B ) = P ( A | B c ) P ( A c | B ) = P ( A c | B c ) P ( A c | B ) = P ( A c | B c ) P ( B | A ) = P ( B | A c ) P ( B | A ) = P ( B | A c ) P ( B c | A ) = P ( B c | A c ) P ( B c | A ) = P ( B c | A c )

These conditions are equivalent in the sense that if any one holds, then all hold. We may chose any one of these as the defining condition and consider the others as equivalents for the defining condition. Because of its simplicity and symmetry with respect to the two events, we adopt the product rule in the upper right hand corner of the table.

Definition. The pair {A,B}{A,B} of events is said to be (stochastically) independent iff the following product rule holds:

P ( A B ) = P ( A ) P ( B ) P ( A B ) = P ( A ) P ( B )
(2)

Remark. Although the product rule is adopted as the basis for definition, in many applications the assumptions leading to independence may be formulated more naturally in terms of one or another of the equivalent expressions. We are free to do this, for the effect of assuming any one condition is to assume them all.

The equivalences in the right-hand column of the upper portion of the table may be expressed as a replacement rule, which we augment and extend below:

If the pair {A,B}{A,B} independent, so is any pair obtained by taking the complement of either or both of the events.

We note two relevant facts

• Suppose event N has probability zero (is a null event). Then for any event A, we have 0P(AN)P(N)=0=P(A)P(N)0P(AN)P(N)=0=P(A)P(N), so that the product rule holds. Thus {N,A}{N,A} is an independent pair for any event A.
• If event S has probability one (is an almost sure event), then its complement Sc is a null event. By the replacement rule and the fact just established, {Sc,A}{Sc,A} is independent, so {S,A}{S,A} is independent.

The replacement rule may thus be extended to:

### Replacement Rule

If the pair {A,B}{A,B} independent, so is any pair obtained by replacing either or both of the events by their complements or by a null event or by an almost sure event.

### CAUTION

1. Unless at least one of the events has probability one or zero, a pair cannot be both independent and mutually exclusive. Intuitively, if the pair is mutually exclusive, then the occurrence of one requires that the other does not occur. Formally: Suppose 0<P(A)<10<P(A)<1 and 0<P(B)<10<P(B)<1. {A,B}{A,B} mutually exclusive implies P(AB)=P()=0P(A)P(B)P(AB)=P()=0P(A)P(B). {A,B}{A,B} independent implies P(AB)=P(A)P(B)>0=P()P(AB)=P(A)P(B)>0=P()
2. Independence is not a property of events. Two non mutually exclusive events may be independent under one probability measure, but may not be independent for another. This can be seen by considering various probability distributions on a Venn diagram or minterm map.

## Independent classes

Extension of the concept of independence to an arbitrary class of events utilizes the product rule.

Definition. A class of events is said to be (stochastically) independent iff the product rule holds for every finite subclass of two or more events in the class.

A class {A,B,C}{A,B,C} is independent iff all four of the following product rules hold

P ( A B ) = P ( A ) P ( B ) P ( A C ) = P ( A ) P ( C ) P ( B C ) = P ( B ) P ( C ) P ( A B C ) = P ( A ) P ( B ) P ( C ) P ( A B ) = P ( A ) P ( B ) P ( A C ) = P ( A ) P ( C ) P ( B C ) = P ( B ) P ( C ) P ( A B C ) = P ( A ) P ( B ) P ( C )
(3)

If any one or more of these product expressions fail, the class is not independent. A similar situation holds for a class of four events: the product rule must hold for every pair, for every triple, and for the whole class. Note that we say “not independent” or “nonindependent” rather than dependent. The reason for this becomes clearer in dealing with independent random variables.

We consider some classical exmples of nonindependent classes

### Example 1: Some nonindependent classes

1. Suppose {A1,A2,A3,A4}{A1,A2,A3,A4} is a partition, with each P(Ai)=1/4P(Ai)=1/4. Let
A=A1A2B=A1A3C=A1A4A=A1A2B=A1A3C=A1A4
(4)
Then the class {A,B,C}{A,B,C} has P(A)=P(B)=P(C)=1/2P(A)=P(B)=P(C)=1/2 and is pairwise independent, but not independent, since
P(AB)=P(A1)=1/4=P(A)P(B)andsimilarlyfortheotherpairs,butP(AB)=P(A1)=1/4=P(A)P(B)andsimilarlyfortheotherpairs,but
(5)
P(ABC)=P(A1)=1/4P(A)P(B)P(C)P(ABC)=P(A1)=1/4P(A)P(B)P(C)
(6)
2. Consider the class {A,B,C,D}{A,B,C,D} with AD=BD=AD=BD=, C=ABDC=ABD, P(A)=P(B)=1/4P(A)=P(B)=1/4, P(AB)=1/64P(AB)=1/64, and P(D)=15/64P(D)=15/64. Use of a minterm maps shows these assignments are consistent. Elementary calculations show the product rule applies to the class {A,B,C}{A,B,C} but no two of these three events forms an independent pair.

As noted above, the replacement rule holds for any pair of events. It is easy to show, although somewhat cumbersome to write out, that if the rule holds for any finite number k of events in an independent class, it holds for any k+1k+1 of them. By the principle of mathematical induction, the rule must hold for any finite subclass. We may extend the replacement rule as follows.

### General Replacement Rule

If a class is independent, we may replace any of the sets by its complement, by a null event, or by an almost sure event, and the resulting class is also independent. Such replacements may be made for any number of the sets in the class. One immediate and important consequence is the following.

### Minterm Probabilities

If {Ai:1in}{Ai:1in} is an independent class and the the class {P(Ai):1in}{P(Ai):1in} of individual probabilities is known, then the probability of every minterm may be calculated.

### Example 2: Minterm probabilities for an independent class

Suppose the class {A,B,C}{A,B,C} is independent with respective probabilities P(A)=0.3P(A)=0.3, P(B)=0.6P(B)=0.6, and P(C)=0.5P(C)=0.5. Then

{Ac,Bc,Cc}{Ac,Bc,Cc} is independent and P(M0)=P(Ac)P(Bc)P(Cc)=0.14P(M0)=P(Ac)P(Bc)P(Cc)=0.14

{Ac,Bc,C}{Ac,Bc,C} is independent and P(M1)=P(Ac)P(Bc)P(C)=0.14P(M1)=P(Ac)P(Bc)P(C)=0.14

Similarly, the probabilities of the other six minterms, in order, are 0.21, 0.21, 0.06, 0.06, 0.09, and 0.09. With these minterm probabilities, the probability of any Boolean combination of A,BA,B, and C may be calculated

In general, eight appropriate probabilities must be specified to determine the minterm probabilities for a class of three events. In the independent case, three appropriate probabilities are sufficient.

### Example 3: Three probabilities yield the minterm probabilities

Suppose {A,B,C}{A,B,C} is independent with P(ABC)=0.51P(ABC)=0.51, P(ACc)=0.15P(ACc)=0.15, and P(A)=0.30P(A)=0.30. Then P(Cc)=0.15/0.3=0.5=P(C)P(Cc)=0.15/0.3=0.5=P(C) and

P ( A ) + P ( A c ) P ( B ) P ( C ) = 0 . 51 so that P ( B ) = 0 . 51 - 0 . 30 0 . 7 × 0 . 5 = 0 . 6 P ( A ) + P ( A c ) P ( B ) P ( C ) = 0 . 51 so that P ( B ) = 0 . 51 - 0 . 30 0 . 7 × 0 . 5 = 0 . 6
(7)

With each of the basic probabilities determined, we may calculate the minterm probabilities, hence the probability of any Boolean combination of the events.

### Example 4: MATLAB and the product rule

Frequently we have a large enough independent class {E1,E2,,En}{E1,E2,,En} that it is desirable to use MATLAB (or some other computational aid) to calculate the probabilities of various “and” combinations (intersections) of the events or their complements. Suppose the independent class {E1,E2,,E10}{E1,E2,,E10} has respective probabilities

0 . 13 0 . 37 0 . 12 0 . 56 0 . 33 0 . 71 0 . 22 0 . 43 0 . 57 0 . 31 0 . 13 0 . 37 0 . 12 0 . 56 0 . 33 0 . 71 0 . 22 0 . 43 0 . 57 0 . 31
(8)

It is desired to calculate (a) P(E1E2E3cE4E5cE6cE7)P(E1E2E3cE4E5cE6cE7), and (b) P(E1cE2E3cE4E5cE6cE7E8E9cE10)P(E1cE2E3cE4E5cE6cE7E8E9cE10).

We may use the MATLAB function prod and the scheme for indexing a matrix.

>> p = 0.01*[13 37 12 56 33 71 22 43 57 31];
>> q = 1-p;
>> % First case
>> e = [1 2 4 7];                  % Uncomplemented positions
>> f = [3 5 6];                    % Complemented positions
>> P = prod(p(e))*prod(q(f))       % p(e) probs of uncomplemented factors
P = 0.0010                         % q(f) probs of complemented factors
>> % Case of uncomplemented in even positions; complemented in odd positions
>> g = find(rem(1:10,2) == 0);     % The even positions
>> h = find(rem(1:10,2) ~= 0);     % The odd positions
>> P = prod(p(g))*prod(q(h))
P = 0.0034


In the unit on MATLAB and Independent Classes, we extend the use of MATLAB in the calculations for such classes.

## Content actions

PDF | EPUB (?)

### What is an EPUB file?

EPUB is an electronic book format that can be read on a variety of mobile devices.

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

#### Definition of a lens

##### Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

##### What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

##### Who can create a lens?

Any individual member, a community, or a respected organization.

##### What are tags?

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks