10.6 Additional Applications

We will carry out the same type of hypothesis testing process for a range of different scenarios. Though the process will remain very similar, the underlying statistical test/calculation/R function will change.

  • If dependent variable is numeric / continuous, can use regression and lm() (type of independent variable does not matter)
  • If dependent variable is numeric and want to compare between two specific groups, can use t.test()
  • If dependent variable is binary, such as assessing the proportion of call back rates, and want to compare between two specific groups, can use prop.test()

Other tests exist for different situations

10.6.1 Example Using Regression

1. Start with a research question

Do past election results help explain future election results?

2. Develop a theory of how the world works

A third-party candidate’s performance in one election will help us predict the success of a future third-party candidate.

3. Construct “null” and “alternative” hypotheses and 4. Carry out a test of the hypothesis, such as a regression.

In a regression, our key hypothesis test is about whether there is a significant, non-zero relationship between an independent variable and the outcome.

  • The null hypothesis is that \(\beta = 0\). The alternative is that \(\beta \neq 0\).

Going back to our Florida example: The null hypothesis would be the 1996 Perot vote does not help us explain the Buchanan 2000 vote (that \(\beta = 0\)). Our alternative is \(\beta \neq 0\), that there is some relationship between 1996 Perot votes and the 2000 Buchanan vote in Florida counties.

florida <- read.csv("https://raw.githubusercontent.com/ktmccabe/teachingdata/main/florida.csv")

We estimate \(\hat \beta\): for every 1 additional vote Perot received in 1996, we expect Buchanan to receive .036 additional votes in 2000.

fit <- lm(Buchanan00 ~ Perot96, data = florida)
coef(fit)
## (Intercept)     Perot96 
##  1.34575212  0.03591504

Is that relationship significant? In other words, is it helpful to know the Perot 1996 vote to help explain the Buchanan 2000 vote? Or should we treat the 0.036 number as essentially 0, just noise?

Recall that the \(\hat \beta\) represents the estimated slope of the relationship.

We ask: Is that relationship (the slope) significant (i.e., statistically different from 0 slope)?

5. Calculate the uncertainty around this estimate.

Our regression lm function will also generate estimates of uncertainty related to hypothesis testing

round(summary(fit)$coefficients, digits=4)
##             Estimate Std. Error t value Pr(>|t|)
## (Intercept)   1.3458    49.7593  0.0270   0.9785
## Perot96       0.0359     0.0043  8.2752   0.0000

6. Decide whether you can reject or fail to reject the hypothesis of no difference

We see the p-value for Perot96 is essentially 0– well less than 0.05. Therefore, it is highly unlikely we would observe a slope as big or bigger (in magnitude) as 0.0359 if Perot96 and Buchanan00 were actually unrelated.

  • We consider the effect “statistically significant.”
  • We reject a null hypothesis that the Perot 1996 vote is unrelated to the Buchanan 2000 vote.

In social science papers, regressions are often presented in tables:

## 
## =======================
##              Model 1   
## -----------------------
## (Intercept)    1.35    
##              (49.76)   
## Perot96        0.04 ***
##               (0.00)   
## -----------------------
## R^2            0.51    
## Adj. R^2       0.51    
## Num. obs.     67       
## =======================
## *** p < 0.001; ** p < 0.01; * p < 0.05

10.6.2 Example Using prop.test()

Sometimes our outcomes are 0 vs. 1 variables, which become proportions when we take the mean()

  • For example, applicants who get a call back vs. do not
  • For example, voters who turn out to vote vs. do not

When you have this type of outcome variable, you may want to use a test designed specifically for testing the differences in proportions of two groups.

Experimental Example: Going Back to Resume and Race study

1. Start with a research question

Does race influence hiring decisions?

2. Develop a theory of how the world works

Theory: Black applicants face discrimination in hiring.

3. Construct “null” and “alternative” hypotheses

We can conduct a two-sided hypothesis test

  • \(H_o\): No difference in call back rates for Black and white applicants
  • \(H_a\): Some difference in call back rates for Black and white applicants

The two-sided means we aren’t specifying a direction of our alternative hypothesis. Instead, we are conducting test just trying to reject the idea of no difference between racial groups. Sometimes researchers may specify the alternative hypothesis in a directional way, such as Black applicants will have a lower call back rate than white applicants. However, it is more common to use a two-sided test, even if researchers have a theoretical hypothesis in a particular direction.

resume <- read.csv("https://raw.githubusercontent.com/ktmccabe/teachingdata/main/resume.csv")

head(resume)
##   firstname    sex  race call
## 1   Allison female white    0
## 2   Kristen female white    0
## 3   Lakisha female black    0
## 4   Latonya female black    0
## 5    Carrie female white    0
## 6       Jay   male white    0

4. Carry out a test of the hypothesis, such as a test of proportions.

Does being black (vs. white) decrease call backs?

table(resume$race, resume$call)
##        
##            0    1
##   black 2278  157
##   white 2200  235
test <- prop.test(x=c(157, 235), n=c(157+2278, 235+2200))
  • The prop.test() function includes 4 inputs
  • The number of 1’s from the first group, the number of 1’s in the second group c(n1, n2)
  • The total observations from the first group, the total observations in the second group c(total1, total2)

5. Calculate the uncertainty around this estimate.

test
## 
##  2-sample test for equality of proportions with continuity correction
## 
## data:  c(157, 235) out of c(157 + 2278, 235 + 2200)
## X-squared = 16.449, df = 1, p-value = 4.998e-05
## alternative hypothesis: two.sided
## 95 percent confidence interval:
##  -0.04769866 -0.01636705
## sample estimates:
##     prop 1     prop 2 
## 0.06447639 0.09650924

You can extract the p.value for the difference in proportions and confidence interval

6. Decide whether you can reject or fail to reject the hypothesis of no difference

In the prop.test function, you see a X-squared statistic. You can treat this like the z-score. When you are doing a test of two groups, the X-squared is essentially the z-score squared. The X-squared refers to a Chi-squared test. In our case, the equivalent z-score would be about 4– well more than our threshold of about 2.

round(test$p.value, digits=3)
## [1] 0

What do you conclude about the hypothesis? Is it significant?