4.4 Application: Changing Minds on Gay Marriage

We now turn to a study that asks the question

Research Question Can we effectively persuade people to change their minds?
- Contact Hypothesis: outgroup hostility diminishes through extended positive contact

The authors conduct two randomized control trials in Los Angeles

Target population: voters in Los Angeles
Recruitment: select people from a registered voter list
Randomized treatment conditions:
- Canvassers have a conversation about same-sex marriage vs.
- Recycling scripts (placebo)
- Control group: no canvassing
Outcome measures:
- Feeling towards gay couples (survey responses over multiple waves)
Comparison
- Compare average change in feelings between treatment conditions

Let’s load the data. Data available through QSS. See QSS Chapter 2 for additional discussion.

study: Which study is the data from (1 = Study1, 2 = Study2)
treatment: Five possible treatment assignment options
therm1: Survey thermometer rating of feeling towards gay couples in waves 1 (0–100) (asked before people were canvassed)
therm2: Survey thermometer rating of feeling towards gay couples in waves 2 (0–100) (asked after people were canvassed)

marriage <- read.csv("gayreshaped.csv", stringsAsFactors = T)

## How many rows and columns
dim(marriage)

## [1] 11948     6

## How many observations in each treatment group, in each study
table(marriage$treatment, marriage$study)

##                                                 
##                                                     1    2
##   No Contact                                     5238 1203
##   Recycling Script by Gay Canvasser              1046    0
##   Recycling Script by Straight Canvasser         1039    0
##   Same-Sex Marriage Script by Gay Canvasser      1151 1238
##   Same-Sex Marriage Script by Straight Canvasser 1033    0

For a video explainer of the code for the barplot, scatter plot and histogram created with this application, see below. The video only discusses the code. Use the notes and lecture discussion for additional context. (Via youtube, you can speed up the playback to 1.5 or 2x speed.)

Let’s focus on study 1 only.

marriage1 <- subset(marriage, study ==  1)

We have to do some work to prepare our outcome and treatment conditions.

In experiments, we compare the mean from the treatment group(s) \(\bar{Y}(1)\) to the control \(\bar{Y}(0)\) on some outcome

Here are outcome is Change in Support for gay couples: Wave 2 - Wave 1 feeling thermometer scores

marriage1$outcome <- marriage1$therm2 - marriage1$therm1

4.4.1 Creating new variable

Let’s create a new variable treatmentnew that collapses the two Recycling and Same-Sex marriage conditions.

marriage1$treatmentnew <- NA
marriage1$treatmentnew[marriage1$treatment == "No Contact"] <- "No Contact"
marriage1$treatmentnew[marriage1$treatment == "Recycling Script by Gay Canvasser" |
                         marriage1$treatment == 
                         "Recycling Script by Straight Canvasser"] <- "Recycling"
marriage1$treatmentnew[marriage1$treatment == "Same-Sex Marriage Script by Gay Canvasser" |
                         marriage1$treatment ==
                         "Same-Sex Marriage Script by Straight Canvasser"] <- "Marriage"
marriage1$treatmentnew <- as.factor(marriage1$treatmentnew)

table(marriage1$treatmentnew)

## 
##   Marriage No Contact  Recycling 
##       2184       5238       2085

4.4.2 Using ifelse to create new variable

An alternative way we could create a variable is to use ifelse

Let’s try another way using the ifelse command.

Can be read: If this relational statement is TRUE, I assign you A (in this case “No Contact”), otherwise (ifelse())
if this alternative relational statement is TRUE, I assign you B (in this case “Recycling”), otherwise (ifelse())
if this alternative relational statement is TRUE, I assign you C (in this case “Marriage”), otherwise
If all of those were FALSE I assign you D (in this case an NA)

marriage1$treatmentnew2 <- ifelse(marriage1$treatment == "No Contact", "No Contact",
                                  ifelse(marriage1$treatment == 
                                           "Recycling Script by Gay Canvasser" |
                                           marriage1$treatment ==  
                                           "Recycling Script by Straight Canvasser",  
                                         "Recycling",
                                    ifelse(marriage1$treatment ==
                                        "Same-Sex Marriage Script by Gay Canvasser" |  
                                            marriage1$treatment ==
                                        "Same-Sex Marriage Script by Straight Canvasser",
                                        "Marriage", 
                                        NA)))
marriage1$treatmentnew2 <- as.factor(marriage1$treatmentnew2)

4.4.3 Calculating the Average Treatment Effect

We now have our outcome and our treatment conditions. In an experiment, we want to look at the difference in means between conditions. Let’s calculate the means.

outs <- tapply(marriage1$outcome, marriage1$treatmentnew, mean, na.rm=T)

Note: Sometimes data include missing cells. In R, these have an NA. To ignore these when calculating a mean, we add na.rm = T to the mean() or tapply() functions.

4.4.4 Visualize means in a barplot

Let’s also add a line at 0 using abline()

barplot(outs,
        col="black",
        ylim =  c(-2, 2), # y-axis dimensions
        border = NA, # removes bar borders
        main = "Change in FT W2-W1 by Type of Treatment", # plot title
        cex.main = .8, # size of plot title
        ylab = "Mean Change in FT W2-W1", # yaxis label
        cex.lab = .8,# size of yaxis label
        las = 1) # controls angle of axis labels
abline(h=0, lty=2, col = "red", lwd=2) # adds horizontal line at 0 with dashes

How should we interpret these results?

In the Marriage condition, it looks like on average, views toward gay couples became warmer (the bar is positive) after the conversations with canvassers about same-sex marriage.
In contrast, the views of people in the Recycling or No Contact conditions did not change much and if anything, became slightly colder.
Comparing between these bars, then, it seems like there is an “average treatment effect” given that the change in the Marriage condition was different from the Recycling an No Contact control groups.