Foundations of Probability Theory: Basic Definitions
1.2
2003/05/12
2003/08/01 15:24:24.703 GMT-5
Don
Johnson
dhj@rice.edu
Don
Johnson
dhj@rice.edu
Eileen
Krause
erkrause@rice.edu
Kyle
Clarkson
kclarks@rice.edu
Elizabeth
Gregory
lizzardg@rice.edu
Kevin
Duh
kevinduh@rice.edu
Mariyah
Poonawala
mariyah@rice.edu
Matthew
Jeanes
mjeanes@rice.edu
Jeffrey
Silverman
jsilv@rice.edu
sample space
event
mutually exclusive
probability measure
axioms of probability
a priori probabilities
conditional probability
statistically independent
Bayes' Rule
This module covers the basic ideas of Probability Theory. It reviews the laws of boolean algebra, describes how to compute a priori and conditional probabilities, and uses these properties to obtain Bayes' Rule.
Basic Definitions
The basis of probability theory is a set of events - sample
space - and a systematic set of numbers - probabilities -
assigned to each event. The key aspect of the theory is the
system of assigning probabilities. Formally, a sample
space is the set Ω
of all possible outcomes
ω i
of an experiment. An event is a collection of
sample points ω
i determined by some
set-algebraic rules governed by the laws of Boolean algebra.
Letting A and
B denote events, these laws are
Union:
A
B
ω
ω
A
ω
B
Intersection:
A
B
ω
ω
A
ω
B
Complement:
A
ω
ω
A
A
B
A
B
The null set is the complement
of Ω. Events are said to
be mutually exclusive if there is no element
common to both events:
A
B
.
Associated with each event
A
i
is a probability measure
A
i
, sometimes denoted by
π
i
, that obeys the axioms of probability.
-
A
i
0
-
Ω
1
- If
A
B
, then
A
B
A
B
.
The consistent set of probabilities
·
assigned to events are known as the a
priori probabilities. From the axioms,
probability assignments for Boolean expressions can be
computed. For example, simple Boolean manipulations (
A
B
A
A
B
) lead to
A
B
A
B
A
B
Suppose
B
0
. Suppose we know that the event
B has occurred; what is the
probability that event A has
also occurred? This calculation is known as the
conditional probability of
A given
B and is denoted by
B
A
. To evaluate conditional probabilities, consider
B to be the sample space rather
than Ω. To obtain a
probability assignment under these circumstances consistent
with the axioms of probability, we must have
B
A
A
B
B
The event is said to be statistically independent
of B if
B
A
A
: the occurrence of the event
B does not change the
probability that A occurred.
When independent, the probability of their intersection
A
B
is given by the product of the a
priori probabilities
A
B
. This property is necessary and sufficient for the
independence of the two events. As
B
A
A
B
B
and
A
B
A
B
A
, we obtain Bayes' Rule.
A
B
B
A
B
A