Random Sampling

A high school with 1,800 students wants to know how many hours a week its students spend on homework, and it can afford to survey 90 of them. Which 90? If a person picks them, the choice drifts toward students who are easy to find, friendly, or sitting nearby, and those students may not do typical amounts of homework. Letting chance do the choosing removes that drift, and there are several ways to put chance to work. Each is a random sampling method, and each has studies it fits and studies it doesn't.

With or without replacement

Every method below is built from draws, and a draw can work two ways.

• Sampling without replacement: once an individual is picked, they are out of the pool. Nobody can be selected twice. This is how real surveys work, since interviewing the same person twice adds nothing.
• Sampling with replacement: after each pick, the individual goes back into the pool, so the same individual can be selected again.

The difference matters most when the population is small. Draw 3 names, with replacement, from a hat of 10. The chance that all three draws are different people is $\frac{10}{10} \cdot \frac{9}{10} \cdot \frac{8}{10} = 0.72$, so there is a 0.28 chance that someone is picked twice. Do the same with 10,000 names and the chance of a repeat is about 0.0003. In a large population the two kinds of sampling behave almost identically, which is why the inference topics in Units 3 and 4 let you treat a sample drawn without replacement as if the draws were independent, provided the 10% condition holds: the sample size $n$ is at most 10% of the population size $N$ ($n \le 0.10N$). The hat of 10 fails it (3 draws is 30% of the names), and that is exactly where the repeat chance of 0.28 showed up; 3 draws from 10,000 is 0.03%.

Simple random sample

A simple random sample (SRS) of size $n$ is chosen so that every possible group of $n$ individuals from the population has the same chance of being the sample. It is the basic random method; the others below use an SRS as one of their steps.

The definition is about groups, not individuals, and the difference has teeth. Suppose a teacher with 20 students flips a coin: heads, she surveys the first 10 names on her roster; tails, the last 10. Each student has a $\frac{1}{2}$ chance of being chosen, yet only 2 of the 184,756 possible groups of 10 can ever occur. Two students from the top of the roster can be in the sample together, but a student from the top and one from the bottom never can. That is not an SRS.

Two standard ways to get one:

• Random number generator. Label the individuals $1$ to $N$. Have a calculator or computer produce random integers from $1$ to $N$, skip any label that repeats, and continue until you have $n$ different labels. The individuals with those labels form the sample.
• Slips of paper. Write each individual's name on an identical slip, put the slips in a container, mix thoroughly, and draw $n$ slips without replacement.

For the school, label the 1,800 students $1$ to $1{,}800$ from the enrollment list and generate random integers from 1 to 1,800, ignoring repeats, until there are 90 different labels. Every one of the roughly $10^{154}$ possible groups of 90 is equally likely.

Stratified random sample

A stratified random sample first splits the whole population into groups called strata (singular: stratum). Every individual belongs to exactly one stratum, and the strata are formed so that individuals inside each one are alike on some characteristic, such as grade level. Then you take a separate SRS from every stratum and put those together as the sample.

At the school, the strata could be the four grades: 500 ninth graders, 460 tenth graders, 440 eleventh graders and 400 twelfth graders. Ninety students is 5% of 1,800, so taking 5% of each grade gives SRSs of 25, 23, 22 and 20 students. Sampling the same fraction from each stratum is called proportional allocation; it is common but not required, and a study that needs a reliable estimate for a small group may sample that group more heavily.

Stratifying pays off when the response differs from stratum to stratum. Seniors in this school might average many more homework hours than ninth graders. An SRS of 90 could, by bad luck, contain only 12 seniors instead of their fair share of 20, and the estimate of the school's average would come out too low. A stratified sample guarantees each grade its share. Because the chance variation in how many of each grade land in the sample is removed, estimates from a stratified sample vary less from sample to sample than estimates from an SRS of the same size. Stratifying also gives you a separate SRS for each grade, so you can estimate each grade's average too.

Cluster random sample

A cluster random sample also splits the population into groups, called clusters, but uses them in the opposite way. You take an SRS of clusters and then collect data from every individual in the chosen clusters.

At the school, the 1,800 students sit in 60 homerooms of 30. Choose 3 homerooms at random and survey all 90 students in them. The appeal is practical: you need a list of homerooms, not a list of students, and you visit 3 rooms instead of tracking down 90 individuals across the building. For a national survey, picking a few city blocks and knocking on every door is far cheaper than traveling to 1,000 scattered addresses.

Clusters work well when each one looks like a small copy of the population: mixed inside, and similar to one another. Homerooms that mix all four grades fit that description. If the school assigned homerooms by grade, a cluster sample could easily land on three senior homerooms and badly misjudge homework time for the school.

Systematic random sample

A systematic random sample picks a random starting point and then takes every $k$th individual from an ordered list or stream. To take 90 from a list of 1,800, use $k = \frac{1800}{90} = 20$, pick a random integer from 1 to 20 as the start, say 7, and take positions 7, 27, 47, and so on up to $7 + 20 \cdot 89 = 1787$.

This method needs no complete list in advance, so it suits a stream of individuals: every 20th customer through a door, every 50th part off an assembly line. Each individual has a $\frac{1}{20}$ chance of selection, but there are only 20 possible samples (one per starting point), so it is not an SRS. The random start is what makes it random; without one, the person running the study picks the sample.

The danger is a list with a repeating pattern that lines up with $k$. A machine that makes bottle caps in a mold with 10 cavities produces caps from the cavities in rotation. Sampling every 10th cap measures caps from one cavity over and over, and a defect in any other cavity never shows up.

Choosing a method

No method is best everywhere. The question being investigated, and what you know about the population, decide.

Worked examples

Where students lose points

On the exam

Multiple-choice questions describe a study and ask which method it used or which method fits. Free-response Question 1 is a multi-part question built mainly on Practice 1 (formulate questions) and Practice 2 (collect data), so a part of it may ask you to describe how to select a sample or explain why one method beats another. An explanation earns credit only when it names the variable that makes the method work or fail ("commute time depends on where a worker lives") in the context of the study. No calculator work is involved beyond simple proportions.