Introduction to Probability

Topic 2.3 estimated probabilities by repeating a random process many times. Sometimes you don't need to: if a fair four-sided die is rolled twice, you can list everything that can happen, see that each possibility is equally likely, and get an exact probability by counting. The counting only works if the list of possibilities is built correctly, so that comes first.

The sample space

The sample space of a random process, written $S$, is the list of every possible outcome of one trial. Two requirements make a list a sample space:

• Complete: every possible result is on the list, so every trial produces something on it.
• Nonoverlapping: no two outcomes on the list can happen on the same trial, so every trial produces exactly one outcome on it.

Because some outcome in $S$ happens on every trial, the event "the outcome is in $S$" always happens. Its probability is $P(S) = 1$.

For one roll of a six-sided die, $S = \{1, 2, 3, 4, 5, 6\}$. The list $\{\text{even}, \text{odd}\}$ is also a valid sample space for that roll: every roll is exactly one of the two. But $\{\text{even}, \text{greater than } 3\}$ is not. A roll of 4 is in both (overlap), and a roll of 1 is in neither (incomplete).

There is usually more than one valid sample space, and the choice matters. For two flips of a coin, both of these are complete and nonoverlapping:

The four outcomes in $S_1$ are equally likely, since each flip is a fair 50-50 and the flips don't affect each other. The three outcomes in $S_2$ are not: "1 head" happens two ways (HT and TH), so it is twice as likely as "0 heads." Which leads to the counting rule and its one condition.

Equally likely outcomes: probability by counting

When every outcome in the sample space has the same chance of happening, the probability of an event $E$ is

Here is why. Suppose $S$ has $k$ equally likely outcomes. Their probabilities are all the same number, and they must add to $P(S) = 1$, so each one has probability $\frac{1}{k}$. An event made of $m$ of those outcomes then has probability $m \cdot \frac{1}{k} = \frac{m}{k}$.

Take two rolls of a fair four-sided die (faces 1 to 4). The sample space has $4 \times 4 = 16$ equally likely ordered pairs. The table shows the sum for each pair:

Let $E$ be "the sum is 5." Four cells contain 5, so $P(E) = \frac{4}{16} = 0.25$. Let $T$ be "the sum is 2"; one cell, so $P(T) = \frac{1}{16} = 0.0625$. The seven possible sums 2 through 8 are not equally likely, which is exactly why you count the 16 pairs and not the 7 sums.

The condition is the whole rule. A lottery ticket either wins or loses, but that does not make $P(\text{win}) = \frac{1}{2}$. Without equally likely outcomes, probabilities come from long-run relative frequencies (topic 2.3) or from a model someone has built, not from counting.

Every probability is between 0 and 1

For any event $E$, $0 \le P(E) \le 1$. Under equal likelihood this is clear from the formula: the count of outcomes in $E$ is at least 0 and at most the count in $S$. Under the long-run view it is just as clear: a relative frequency can't be negative and can't exceed 1. A probability of 0 means the event never happens (a sum of 9 on two four-sided dice), and 1 means it always happens (a sum of at least 2).

When outcomes aren't equally likely, someone assigns each outcome a probability, and the result is a probability model. A model is valid only if each probability is between 0 and 1 and the probabilities of all outcomes in $S$ add to exactly 1. A model with a negative entry, or one whose entries sum to 0.95, is not a probability model at all. Finding a missing entry in a model is the same fact used in reverse: whatever is left over from 1 belongs to the missing outcome.

The complement rule

The complement of an event $E$ is the event "$E$ does not happen": all the outcomes in $S$ that are not in $E$. It is written $E'$, $\bar{E}$, or $E^C$, all meaning the same thing.

$E$ and its complement split the sample space into two parts that don't overlap and together make up all of $S$. Their probabilities therefore add to $P(S) = 1$:

For the two four-sided dice, the probability that the sum is not 5 is $1 - 0.25 = 0.75$, and counting agrees: 12 of the 16 cells hold something other than 5.

The complement rule earns its keep when the event itself is long to count but its complement is short. "At least one" is the standard case. Let $A$ be "at least one of the two rolls is a 4." Its complement is "neither roll is a 4," which means both rolls are 1, 2 or 3: $3 \times 3 = 9$ pairs. So

Counting directly confirms it: the last row and last column of the table have $4 + 4 - 1 = 7$ cells (the corner cell, 4 and 4, is in both and is counted once). With more rolls, the direct count grows quickly while the complement stays a single case, "none."

Worked examples

Where students lose points