Connexions

You are here: Home » Content » Foundations of Probability Theory: Basic Definitions
Content Actions

Foundations of Probability Theory: Basic Definitions

Module by: Don Johnson

Summary: This module covers the basic ideas of Probability Theory. It reviews the laws of boolean algebra, describes how to compute a priori and conditional probabilities, and uses these properties to obtain Bayes' Rule.

Basic Definitions

The basis of probability theory is a set of events - sample space - and a systematic set of numbers - probabilities - assigned to each event. The key aspect of the theory is the system of assigning probabilities. Formally, a sample space is the set ΩΩ of all possible outcomes ω i ω i of an experiment. An event is a collection of sample points ω i ω i determined by some set-algebraic rules governed by the laws of Boolean algebra. Letting AA and BB denote events, these laws are Union: AB={ω|ωAωB} Union: A B ω ω A ω B Intersection: AB={ω|ωAωB} Intersection: A B ω ω A ω B Complement: A={ω|ωA} Complement: A ω ω A AB=AB A B A B The null set is the complement of ΩΩ. Events are said to be mutually exclusive if there is no element common to both events: AB= A B .
Associated with each event A i A i is a probability measure Pr A i A i , sometimes denoted by π i π i , that obeys the axioms of probability.
  • Pr A i 0 A i 0
  • PrΩ=1 Ω 1
  • If AB= A B , then PrAB=PrA+PrB A B A B .
The consistent set of probabilities Pr· · assigned to events are known as the a priori probabilities. From the axioms, probability assignments for Boolean expressions can be computed. For example, simple Boolean manipulations ( AB=AAB A B A A B ) lead to
PrAB=PrA+PrB-PrAB A B A B A B (1)
Suppose PrB0 B 0 . Suppose we know that the event BB has occurred; what is the probability that event AA has also occurred? This calculation is known as the conditional probability of AA given BB and is denoted by PrA|B B A . To evaluate conditional probabilities, consider BB to be the sample space rather than ΩΩ. To obtain a probability assignment under these circumstances consistent with the axioms of probability, we must have
PrA|B=PrABPrB B A A B B (2)
The event is said to be statistically independent of BB if PrA|B=PrA B A A : the occurrence of the event BB does not change the probability that AA occurred. When independent, the probability of their intersection PrAB A B is given by the product of the a priori probabilities PrAPrB A B . This property is necessary and sufficient for the independence of the two events. As PrA|B=PrABPrB B A A B B and PrB|A=PrABPrA A B A B A , we obtain Bayes' Rule.
PrB|A=PrA|BPrBPrA A B B A B A (3)

Comments, questions, feedback, criticisms?

Discussion forum

Send feedback