Skip to content Skip to navigation

Connexions

You are here: Home » Content » Foundations of Probability Theory: Basic Definitions

Navigation

Recently Viewed

This feature requires Javascript to be enabled.

Foundations of Probability Theory: Basic Definitions

Module by: Don Johnson. E-mail the author

User rating (How does the rating system work?)
Ratings

Ratings allow you to judge the quality of modules. If other users have ranked the module then its average rating is displayed below. Ratings are calculated on a scale from one star (Poor) to five stars (Excellent).

How to rate a module

Hover over the star that corresponds to the rating you wish to assign. Click on the star to add your rating. Your rating should be based on the quality of the content. You must have an account and be logged in to rate content.

:
(0 ratings)

Summary: This module covers the basic ideas of Probability Theory. It reviews the laws of boolean algebra, describes how to compute a priori and conditional probabilities, and uses these properties to obtain Bayes' Rule.

Note: Your browser may not currently support MathML. See our browser support page for additional details. You can always view the correct math in the PDF version.

Basic Definitions

The basis of probability theory is a set of events - sample space - and a systematic set of numbers - probabilities - assigned to each event. The key aspect of the theory is the system of assigning probabilities. Formally, a sample space is the set ΩΩ of all possible outcomes ω i ω i of an experiment. An event is a collection of sample points ω i ω i determined by some set-algebraic rules governed by the laws of Boolean algebra. Letting AA and BB denote events, these laws are Union: A⋃B={ω|ω∈A∨ω∈B} Union: A B ω ω A ω B Intersection: A⋂B={ω|ω∈A∧ω∈B} Intersection: A B ω ω A ω B Complement: A′={ω|ω∉A} Complement: A ω ω A A⋃B′=A′⋂B′ A B A B The null set ∅ is the complement of ΩΩ. Events are said to be mutually exclusive if there is no element common to both events: A⋂B=∅ A B .

Associated with each event A i A i is a probability measure Pr A i A i , sometimes denoted by π i π i , that obeys the axioms of probability.

  • Pr A i ≥0 A i 0
  • PrΩ=1 Ω 1
  • If A⋂B=∅ A B , then PrA⋃B=PrA+PrB A B A B .
The consistent set of probabilities Pr· · assigned to events are known as the a priori probabilities. From the axioms, probability assignments for Boolean expressions can be computed. For example, simple Boolean manipulations ( A⋃B=A⋃A′B A B A A B ) lead to
PrA⋃B=PrA+PrB−PrA⋂B A B A B A B (1)

Suppose PrB≠0 B 0 . Suppose we know that the event BB has occurred; what is the probability that event AA has also occurred? This calculation is known as the conditional probability of AA given BB and is denoted by PrA|B B A . To evaluate conditional probabilities, consider BB to be the sample space rather than ΩΩ. To obtain a probability assignment under these circumstances consistent with the axioms of probability, we must have

PrA|B=PrA⋂BPrB B A A B B (2)
The event is said to be statistically independent of BB if PrA|B=PrA B A A : the occurrence of the event BB does not change the probability that AA occurred. When independent, the probability of their intersection PrA⋂B A B is given by the product of the a priori probabilities PrAPrB A B . This property is necessary and sufficient for the independence of the two events. As PrA|B=PrA⋂BPrB B A A B B and PrB|A=PrA⋂BPrA A B A B A , we obtain Bayes' Rule.
PrB|A=PrA|BPrBPrA A B B A B A (3)

Content actions

Give Feedback:

E-mail the module author | Rate module ( How does the rating system work?)

Rating system

Ratings

Ratings allow you to judge the quality of modules. If other users have ranked the module then its average rating is displayed below. Ratings are calculated on a scale from one star (Poor) to five stars (Excellent).

How to rate a module

Hover over the star that corresponds to the rating you wish to assign. Click on the star to add your rating. Your rating should be based on the quality of the content. You must have an account and be logged in to rate content.

(0 ratings)

Download:

Module PDF

Add module to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections directly in Connexions. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need a Connexions account to use 'My Favorites'.

| A lens (?)

Definition of a lens

Lenses

A lens is a custom view of Connexions content. You can think of it as a fancy kind of list that will let you see Connexions through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to Connexions materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual Connexions member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks