We can divide mathematics into pure and applied maths. Pure maths is the theory of maths and is very abstract. The work you have covered on algebra is mostly pure maths. Applied maths is all about taking the theory (or pure maths) and applying it to the real world. To be able to do applied maths, you first have to learn pure maths.

But what does this have to do with probability? Well, just as mathematics can be divided into pure maths and applied maths, so statistics can be divided into probability theory and applied statistics. And just as you cannot do applied mathematics without knowing any theory, you cannot do statistics without beginning with some understanding of probability theory. Furthermore, just as it is not possible to describe what arithmetic is without describing what mathematics as a whole is, it is not possible to describe what probability theory is without some understanding of what statistics as a whole is about, and statistics, in its broadest sense, is about 'processes'.

A process is how an object changes over time. For example, consider a coin. Now, the coin by itself is not a process; it is simply an object. However, if I was to flip the coin (i.e. putting it through a process), after a certain amount of time (however long it would take to land), it is brought to a final state. We usually refer to this final state as 'heads' or 'tails' based on which side of the coin landed face up, and it is the 'heads' or 'tails' that the statistician (person who studies statistics) is interested in. Without the process there is nothing to examine. Of course, leaving the coin stationary is also a process, but we already know that its final state is going to be the same as its original state, so it is not a particularly interesting process. Usually when we speak of a process, we mean one where the outcome is not yet known, otherwise there is no real point in analyzing it. With this understanding, it is very easy to understand what, precisely, probability theory is.

When we speak of probability theory as a whole, we mean the way in which we quantify the possible outcomes of processes. Then, just as 'applied' mathematics takes the methods of 'pure' mathematics and applies them to real-world situations, applied statistics takes the means and methods of probability theory (i.e. the means and methods used to quantify possible outcomes of events) and applies them to real-world events in some way or another. For example, we might use probability theory to quantify the possible outcomes of the coin-flip above as having a 50% chance of coming up heads and a 50% chance of coming up tails, and then use statistics to apply it a real-world situation by saying that of six coins sitting on a table, the most likely scenario is that three coins will come up heads and three coins will come up tails. This, of course, may not happen, but if we were only able to bet on ONE outcome, we would probably bet on that because it is the most probable. But here, we are already getting ahead of ourselves. So let's back up a little.

To quantify results, we may use a variety of methods, terms, and notations, but a few common ones are:

You may notice that all three of the above examples represent the same probability, and in fact ANY method of probability is fundamentally based on the following procedure:

The term 'measure' may be confusing, but one may think of it as a ruler. If we take a ruler that is 1 metre long, then half of that ruler is 50 centimetres, a quarter of that ruler is 25 centimetres, etc. However, the thing to remember is that without the ruler, it makes no sense to talk about proportions of a ruler! Indeed, the three examples above (50%, 5/10, and 1/2) represented the same probability, the only difference was how the total measure (ruler) was defined. If we go back to thinking about it in terms of a ruler '50%' means '50/100', so it means we are using 50 parts of the original 100 parts (centimetres) to quantify the outcome in question. '5/10' means 5 parts out of the original 10 parts (10 centimetre pieces) depict the outcome in question. And in the last example, '1/2' means we are dividing the ruler into two pieces and saying that one of those two pieces represents the outcome in question. But these are all simply different ways to talk about the same 50 centimetres of the original 100 centimetres! In terms of probability theory, we are only interested in proportions of a whole.

Although there are many ways to define a 'measure', the most common and easiest one to generalize is to use '1' as the total measure. So if we consider the coin-flip, we would say that (assuming the coin was fair) the likelihood of heads is 1/2 (i.e. half of one) and the likelihood of tails is 1/2. On the other hand, if we consider the event of not flipping the coin, then (assuming the coin was originally heads-side-up) the likelihood of heads is now 1, while the likelihood of tails is 0. But we could have also used '14' as the original measure and said that the likelihood of heads or tails on the coin-flip was each '7 out of 14', while on the non-coin-flip the likelihood of heads was '14 out of 14', and the likelihood of tails was '0 out of 14'. Similarly, if we consider the throwing of a (fair) six-sided die, it may be easiest to set the total measure to '6' and say that the likelihood of throwing a '4' is '1 out of the 6', but usually we simply say that it is 1/6, i.e. '1/6 of 1'.

There are three important concepts associated with a random experiment: 'outcome', 'sample space', and 'event'. Two examples of experiments will be used to familiarize you with these terms:

The term random experiment or statistical experiment is used to describe any repeatable process, the results of which are analyzed in some way. For example, flipping a coin and noting whether or not it lands heads-up is a random experiment because the process is repeatable. On the other hand, your reading this sentence for the first time and noting whether you understand it is not a random experiment because it is not repeatable (though making a number of random people read it and noting which ones understand it would turn it into a random experiment).

Venn diagrams

A Venn diagram can be used to show the relationship between the possible outcomes of a random experiment and the sample space. The Venn diagram in Figure 1 shows the difference between the universal set, a sample space and events and outcomes as subsets of the sample space.

Figure 1: Diagram to show difference between the universal set and the sample space. The sample space is made up of all possible outcomes of a statistical experiment and an event is a subset of the sample space.

We can draw Venn diagrams for experiments with two and three events. These are shown in Figure 2 and Figure 3. Venn diagrams for experiments with more than three events are more complex and are not covered at this level.

Figure 2: Venn diagram for an experiment with two events

Figure 3: Venn diagram for an experiment with three events.

Note: Interesting fact:

The Greek, Russian and Latin alphabets can be illustrated using Venn diagrams. All these alphabets have some common letters. The Venn diagram is given below:

Figure 4

The union of AA and BB is the set of all elements in AA or in BB (or in both). AorBAorB is also written A∪BA∪B. The intersection of AA and BB is the set of all elements in both AA and BB. AandBAandB is also written as A∩BA∩B.

Venn diagrams can also be used to indicate the union and intersection between events in a sample space (Figure 5 and Figure 6).

Figure 5: Venn diagram to show (left) union of two events, AA and BB, in the sample space SS.

Figure 6: Venn diagram to show intersection of two events AA and BB, in the sample space SS. The black region indicates the intersection.

We use n(S)n(S) to refer to the number of elements in a set SS, n(X)n(X) for the number of elements in XX, etc.

Exercise 1: Random Experiments

In a box there are pieces of paper with the numbers from 1 to 9 written on them. A piece of paper is drawn from the box and the number on it is noted. Let SS denote the sample space, let PP denote the event 'drawing a prime number', and let EE denote the event 'drawing an even number'. Using appropriate notation, in how many ways is it possible to draw: i) any number? ii) a prime number? iii) an even number? iv) a number that is either prime or even? v) a number that is both prime and even?

Solution

1. Step 1. Consider the events: :
   • Drawing a prime number: P={2;3;5;7 }P={2;3;5;7 }
   • Drawing an even number: E={2;4;6;8 }E={2;4;6;8 }
2. Step 2. Draw a diagram :

   Figure 7

3. Step 3. Find the union :

   The union of PP and EE is the set of all elements in PP or in EE (or in both). P∪E=2,3,4,5,6,7,8 P∪E=2,3,4,5,6,7,8 .

4. Step 4. Find the intersection :

   The intersection of PP and EE is the set of all elements in both PP and EE. P∩E=2P∩E=2.

5. Step 5. Find the number in each set :
   ∴ n ( S ) = 9 n ( P ) = 4 n ( E ) = 4 n ( P ∪ E ) = 7 n ( P ∩ E ) = 2 ∴ n ( S ) = 9 n ( P ) = 4 n ( E ) = 4 n ( P ∪ E ) = 7 n ( P ∩ E ) = 2

   (1)

[ Show Solution ] [ Hide Solution ]

Exercise 2

100 people were surveyed to find out which fast food chain (Nandos, Debonairs or Steers) they preferred. The following results were obtained:

• 50 liked Nandos
• 66 liked Debonairs
• 40 liked Steers
• 27 liked Nandos and Debonairs but not Steers
• 13 liked Debonairs and Steers but not Nandos
• 4 liked all three
• 94 liked at least one

1. How many people did not like any of the fast food chains?
2. How many people liked Nandos and Steers, but not Debonairs?

Solution

1. Step 1. Draw a Venn diagram: The number of people who liked Nandos and Debonairs is 27, so this is the intersection of these two events. The number of people who liked Debonairs and Steers is 13, so the intersection of Debonairs and Steers is 13. We are told that 4 people like all three options, and so this means that there are 4 people in the intersection of all three options. So we can work out that the number of people who like just Debonairs is 66–4–27-13=2266–4–27-13=22 (This is simply the total number who like Debonairs minus the number of people who like Debonairs and Steers, or Debonairs and Nandos or all three). We draw the following diagram to represent the data:

   Figure 8

2. Step 2. Work out how many people like none: We are told that there were 100 people and that 94 liked at least one. So the number of people that liked none is: 100–94=6100–94=6. This is the answer to a).
3. Step 3. Work out how many like Nandos and Steers but not Debonairs: We can redraw the part of the Venn diagram that is of interest:

   Figure 9

   Total people who like Nandos: 50
   Of these 27 like both Nandos and Debonairs, and 4 people like all three options. So we find that the total number of people who just like Nandos is: 50–27–4=1950–27–4=19
   Total people who like Steers: 40
   Of these 13 like both Steers and Debonairs, and 4 like all three options. So we find that the total number of people who like just Steers is: 40–13–4=2340–13–4=23
   Now use the identity n(N or S)=n(N)+n(S)–n(N and S)n(N or S)=n(N)+n(S)–n(N and S) to find the number of people who like Nandos and Steers, but not Debonairs.
   n(N or S) = n(N)+n(S)–n(N and S) 28 = 23+19–n(N and S) n(N and S) = 14 n(N or S) = n(N)+n(S)–n(N and S) 28 = 23+19–n(N and S) n(N and S) = 14

   (2)
   The Venn diagram with that represents all this information is given:

   Figure 10

[ Show Solution ] [ Hide Solution ]

Probability Identities

The following results apply to probabilities, for the sample space SS and two events AA and BB, within SS.

P ( S ) = 1 P ( S ) = 1

(4)

P ( A ∩ B ) = P ( A ) × P ( B ) P ( A ∩ B ) = P ( A ) × P ( B )

(5)

P ( A ∪ B ) = P ( A ) + P ( B ) - P ( A ∩ B ) P ( A ∪ B ) = P ( A ) + P ( B ) - P ( A ∩ B )

(6)

We can demonstrate this last result using a Venn diagram. The union of A and B is the set of all elements in A or in B or in both.

Figure 11

The probability of event A occurring is given by P(A)P(A) and the probability of event B occurring is given by P(B)P(B). However, if we look closely at the circle representing either of these events, we notice that the probability includes a small part of the other event. So event A includes a bit of event B and vice versa. This is shown in the following figure:

Figure 12

And then we observe that this small bit is simply the intersection of the two events.

So to find the probability of P ( A ∪ B )P(A∪B) we notice the following:

• We can add P ( A )P(A) and P ( B )P(B)
• Doing this counts the intersection twice, once in P ( A )P(A) and once in P ( B )P(B).

So if we simply subtract the probability of the intersection, then we will find the total probability of the union: P ( A ∪ B ) = P ( A ) + P ( B ) - P ( A ∩ B ) P ( A ∪ B ) = P ( A ) + P ( B ) - P ( A ∩ B )

Exercise 6: Probabilty identities

What is the probability of selecting a black or red card from a pack of 52 cards

Solution

1. Step 1. Write the answer :

   P(S)=n(E)n(S)=5252=1P(S)=n(E)n(S)=5252=1 because all cards are black or red!

[ Show Solution ] [ Hide Solution ]

Exercise 7: Probabilty identities

What is the probability of drawing a club or an ace with one single pick from a pack of 52 cards

Solution

1. Step 1. Identify the identity which describes the situation :
   P ( club ∪ ace ) = P ( club ) + P ( ace ) - P ( club ∩ ace ) P ( club ∪ ace ) = P ( club ) + P ( ace ) - P ( club ∩ ace )

   (7)
2. Step 2. Calculate the answer :
   = 1 4 + 1 13 - 1 4 × 1 13 = 1 4 + 1 13 - 1 52 = 16 52 = 4 13 = 1 4 + 1 13 - 1 4 × 1 13 = 1 4 + 1 13 - 1 52 = 16 52 = 4 13

   (8)

   Notice how we have used P(C∪A)=P(C )+P(A)-P(C∩A)P(C∪A)=P(C )+P(A)-P(C∩A).

[ Show Solution ] [ Hide Solution ]

The following video provides a brief summary of some of the work covered so far.

Figure 13

Khan academy video on probability

Probability Identities

Answer the following questions

1. Rory is target shooting. His probability of hitting the target is 0,70,7. He fires five shots. What is the probability that all five shots miss the center?
   Click here for the solution.
2. An archer is shooting arrows at a bullseye. The probability that an arrow hits the bullseye is 0,40,4. If she fires three arrows, what is the probability that all the arrows hit the bullseye?
   Click here for the solution.
3. A dice with the numbers 1,3,5,7,9,11 on it is rolled. Also a fair coin is tossed. What is the probability that:
   1. A tail is tossed and a 9 rolled?
   2. A head is tossed and a 3 rolled?
   Click here for the solution.
4. Four children take a test. The probability of each one passing is as follows. Sarah: 0,80,8, Kosma: 0,50,5, Heather: 0,60,6, Wendy: 0,90,9. What is the probability that:
   1. all four pass?
   2. all four fail?
   Click here for the solution.
5. With a single pick from a pack of 52 cards what is the probability that the card will be an ace or a black card?
   Click here for the solution.

Mutually Exclusive Events

Two events are called mutually exclusive if they cannot be true at the same time.

Examples of mutually exclusive events are:

1. A die landing on an even number or landing on an odd number.
2. A student passing or failing an exam
3. A tossed coin landing on heads or landing on tails

This means that if we examine the elements of the sets that make up AA and BB there will be no elements in common. Therefore, A∩B=∅A∩B=∅ (where ∅∅ refers to the empty set). Since, P(A∩B)=0P(A∩B)=0, equation Equation 6 becomes:

P ( A ∪ B ) = P ( A ) + P ( B ) P ( A ∪ B ) = P ( A ) + P ( B )

(9)

for mutually exclusive events.

We can represent mutually exclusive events on a Venn diagram. In this case, the two circles do not touch each other, but are instead completely separate parts of the sample space.

Figure 14: Venn diagram for mutually exclusive events

Mutually Exclusive Events

1. A box contains coloured blocks. The number of each colour is given in the following table.

   Table 1

   Colour	Purple	Orange	White	Pink
   Number of blocks	24	32	41	19

   A block is selected randomly. What is the probability that the block will be:
   1. purple
   2. purple or white
   3. pink and orange
   4. not orange?
   Click here for the solution.
2. A small private school has a class with children of various ages. The table gies the number of pupils of each age in the class.

   Table 2

   3 years female	3 years male	4 years female	4 years male	5 years female	5 years male
   6	2	5	7	4	6

   If a pupil is selceted at random what is the probability that the pupil will be:
   1. a female
   2. a 4 year old male
   3. aged 3 or 4
   4. aged 3 and 4
   5. not 5
   6. either 3 or female?
   Click here for the solution.
3. Fiona has 85 labeled discs, which are numbered from 1 to 85. If a disc is selected at random what is the probability that the disc number:
   1. ends with 5
   2. can be multiplied by 3
   3. can be multiplied by 6
   4. is number 65
   5. is not a multiple of 5
   6. is a multiple of 4 or 3
   7. is a multiple of 2 and 6
   8. is number 1?
   Click here for the solution.

There are two approaches to determining the probability associated with any particular event of a random experiment:

Relative frequency is defined as the number of times an event happens in a statistical experiment divided by the number of trials conducted.

It takes a very large number of trials before the relative frequency of obtaining a head on a toss of a coin approaches the probability of obtaining a head on a toss of a coin (in fact, if the probability of an event occurring is something other than 1, i.e. it is always true, or 0, i.e. it is never true, then probability is only absolutely accurate when an infinite number of trials is conducted). For example, the data in Table 3 represent the outcomes of repeating 100 trials of a statistical experiment 100 times, i.e. tossing a coin 100 times.

The following two worked examples show that the relative frequency of an event is not necessarily equal to the probability of the same event. Relative frequency should therefore be seen as an approximation to probability.

Perform an experiment to show that as the number of trials increases, the relative frequency approaches the probability of a coin toss. Perform 10, 20, 50, 100, 200 trials of tossing a coin.

The probability of an event is generally represented as a real number between 0 and 1, inclusive. An impossible event has a probability of exactly 0, and a certain event has a probability of 1, but the converses are not always true: probability 0 events are not always impossible, nor probability 1 events certain. There is a rather subtle distinction between "certain" and "probability 1".

For example, we can say that the sun will always rise in the east. This is a certain event, the sun will not suddenly rise in the north. But if we looked at the event of Penny Heyns winning a swimming race against your maths teacher, then this event is almost certain, since there is a very small chance that your teacher could win the race.

Most probabilities that occur in practice are numbers between 0 and 1, indicating the event's position on the continuum between impossibility and certainty. The closer an event's probability is to 1, the more likely it is to occur.

For example, if two mutually exclusive events are assumed equally probable, such as a flipped or spun coin landing heads-up or tails-up, we can express the probability of each event as "1 in 2", or, equivalently, "50%" or "1/2".

Probabilities are equivalently expressed as odds, which is the ratio of the probability of one event to the probability of all other events. The odds of heads-up, for the tossed/spun coin, are (1/2)/(1 - 1/2), which is equal to 1/1. This is expressed as "1 to 1 odds" and often written "1:1".

Odds a:b for some event are equivalent to probability a/(a+b). For example, 1:1 odds are equivalent to probability 1/2, and 3:2 odds are equivalent to probability 3/5.

We can summarize some of the key concepts in this chapter in the following table:

Table 5

Term	Meaning	Representation	Venn diagram
Union	Everything in A and B	A∪BA∪B	Figure 15
Intersection	Everything in A or B	A∩BA∩B	Figure 16
Complement	Everything that is not in A	AcAc	Figure 17
Only one	All that is only in A	A-BA-B	Figure 18