3.2 Randomized Controlled Trials

One approach for addressing the fundamental problem of causal inference is to simulate two potential states of the world through random assignment: Randomized Controlled Trials / Experiments

Experiments approximate factual vs. counterfactual comparison

• We randomly assign one group to receive a “treatment” and another not to receive a treatment (the control)
• When treatment assignment is randomized, the only thing that distinguishes the treatment group from the control group, besides the treatment itself, is chance.

This allows us to compare the average outcomes between groups in order to estimate our causal effects (more on this below).

3.2.1 Experiments: Why Randomize?

Randomization is essential for being able to “identify” and isolate the causal effect of the treatment on the outcome. Without randomization, there may be several reasons why two groups differ beyond the treatment of interest.

For example, if we randomly assigned half of Rutgers seniors to watch the movie Groundhog Day and half to watch Parasite we would expect the groups to have about equal proportions of female students, average age, racial composition, majors, etc.

• (If we didn’t randomly assign, and just let people “select” into watching a particular movie, the groups could look very different.)

But because we randomized assignment, on average, we’d expect the two groups to be identical except for the treatment– in this case, which movie they watched.

• Great news! This means any differences in the outcomes between the two groups can be attributed to the treatment. So if we wanted to see if Parasite leads people to have nightmares about people living in their basements, we could compare the average number of reported nightmares between the seniors that watched Parasite vs. Groundhog Day

3.2.2 Experiments: How to Analyze

Difference in Means: We compare each group’s average outcome by subtracting one from the other to estimate the average treatment effect (ATE) aka the average causal effect of the treatment.

• \(\widehat{ATE} = \bar{Y}(treatment) - \bar{Y}(control)\)

This is an estimate of, on average, how much our outcome would change if units went from being untreated to treated.

• E.g., on average how much a person donates to a campaign if contacted by phone compared to if not contacted by phone.

3.2.3 Ingredients of an Experiment

From Bit by Bit

For every experiment, you should be able to

• State the causal question or relationship of interest
• Describe how the experiment will be implemented (e.g., recruitment of subjects)
• Identify and describe the randomization into treatment group(s) and control group and what happens in each group
• Identify the outcome of interest, how it is measured
• Evaluate the relevant comparison

We will turn to an example in the next section.