Non-Compliance and Instruments

Intro to Non-Compliance

Randomization Natural and Otherwise
- Randomized experiments are a wonderful thing
  - Help us solve the fundamental problem of causal inference
- Examples:
  - Randomized Control Trials
    - Drug Trials
    - Program evaluation
      - Health insurance, Welfare programs
  - Natural Experiments
    - Lottery determines admission to charter schools
- However, they don't often work out as nicely in practice as we would hope
  - Medical trial
    - Not all the subjects in the treatment condition take the pill
    - Some of subjects in the control condition find a way to take the pill
  - School Lottery
    - Some lottery winners don't attend the charter school
    - Some lottery losers find a way in.
Noncompliance: The Problem
- We don't want to simply compare people who did and didn't take the pill
  - Taking the pill wasn't random
  - Choosing to attend charter school conditional on offer wasn't random
- The people who chose not to take the pill despite being in the treatment condition might have really different $Y_0$ $Y^{ 0 }$ 's than those who did take the pill
  - Why might this be the case?
  - What if there are side effects, and we know treatment is not working?
- Similarly, we don't want to simply drop people who didn't comply with the experiment
- Either of these corrections would fail to make an apples-to-apples comparison
- What can we estimate?
  - Experiment guarantees that we can estimate the effect of the treatment assignment
  - Noncompliance makes it hard for us to estimate the effect of the actual treatment
  - It turns our that with some added assumptions, we can estimate the average effect of the treatment for the people who did comply with the treatment assignment.
    - This is less we might want in an ideal world.
  - Set-up
    - $Y$ : outcome of interest
    - $T$ $T$ : Treatment of interest
      - $T=1$ : treatment
      - $T=0$ : non-treatment
    - $Z$ $Z$ : Assignment Variable
      - $Z=1$ : Assigned to treatment
      - $Z=0$ : Assigned to control
  - Encouragement:
    - Think of the assignment as encouraging take up of the treatment
    - $T_{1_i}\equiv$ the value of $T$ that we would observe for individual $i$ if $Z=1$ (assigned to treatment)
    - $T_{0_i}\equiv$ the value of $T$ that we would observe for individual $i$ if $Z=0$ (assigned to control)
    - If encouragement design worked in the sense that it increased take up of the treatment, then $E[T_{1_i}-T_{0_i}]>0$ $E [T^{ 1^{ i } } - T^{ 0^{ i } }] > 0$ .
      - Those assigned to the treatment were more likely to take the treatment than those assigned to the control
Different Types of Subjects
- Always Takers: $T_1=T_0=1$ $T^{ 1 } = T^{ 0 } = 1$
  - No matter the assignment, these people find a way to access the treatment
- Never Takers: $T_1=T_0=0$ $T^{ 1 } = T^{ 0 } = 0$
  - No matter the assignment, these people do not take the treatment
- Compliers: $T_1=1$ $T^{ 1 } = 1$ and $T_0=0$ $T^{ 0 } = 0$
  - These are the people we want
  - They obey the rules
- Defiers: $T_1=0$ $T^{ 1 } = 0$ and $T_0=1$ $T^{ 0 } = 1$
  - They rebel the rules
  - Do the opposite of what we hope/expect

Measuring Non-Compliance and First Stage

Measuring Compliers
- Assuming no defiers, $E[T_{1_i}-T_{0_i}]$ $E [T^{ 1^{ i } } - T^{ 0^{ i } }]$ tells us the proportion that are compliers.
  - For always takers: $T_{1_i}-T_{0_i}=1-1=0$ $T^{ 1^{ i } } - T^{ 0^{ i } } = 1 - 1 = 0$
    - $E[T_{1_i}-T_{0_i}\mid\text{Always Takers}]=0$
  - For never takers $T_{1_i}-T_{0_i}=0-0=0$ $T^{ 1^{ i } } - T^{ 0^{ i } } = 0 - 0 = 0$
    - $E[T_{1_i}-T_{0_i}\mid\text{Never Takers}]=0$
  - For compliers, $T_{1_i}-T_{0_i}=1-0=1$ $T^{ 1^{ i } } - T^{ 0^{ i } } = 1 - 0 = 1$
    - $E[T_{1_i}-T_{0_i}\mid\text{Compliers}]=1$
- For defiers , $T_{1_i}-T_{0_i}=0-1=-1$ $T^{ 1^{ i } } - T^{ 0^{ i } } = 0 - 1 = - 1$
  - $E[T_{1_i}-T_{0_i}\mid\text{Defiers}]=-1$
Estimating Proportion of Compliers
- We don't know which individuals in our data set are compliers
- But we can estimate how many there are by simply comparing
  - the average value of $T$ in our treatment group
  - the average value of $T$ in our control group
- Sometimes, we will call this the "first stage" effect, and we can estimate it by calculating $\text{Avg}[T\mid Z=1]-\text{Avg}[T\mid Z=0]$ $Avg [T ∣ Z = 1] - Avg [T ∣ Z = 0]$
  - First stage estimates the effect of assignment on taking the treatment
- This simple estimator is unbiased, because $Z$ $Z$ is randomly assigned.
  - There should be an equal portion of always takers, never takers, and compliers in control and treatment groups in expectation
  - Only difference between $\text{Avg}[T\mid Z=1]$ $Avg [T ∣ Z = 1]$ and $\text{Avg}[T\mid Z=0]$ $Avg [T ∣ Z = 0]$ will come from presence of compliers
    - $\text{Avg}[T\mid Z=1]-\text{Avg}[T\mid Z=0]=0$ $Avg [T ∣ Z = 1] - Avg [T ∣ Z = 0] = 0$
      - Same take up of treatment in both groups so only never takers and always takers
    - $\text{Avg}[T\mid Z=1]-\text{Avg}[T\mid Z=0]=1$ $Avg [T ∣ Z = 1] - Avg [T ∣ Z = 0] = 1$
      - All those in treatment group take the treatment take treamtnet ( $\text{Avg}[T\mid Z=1]=1$ )
      - No on in control group takes the treatment ( $\text{Avg}[T\mid Z=0]=0$ )
      - Everyone is a complier
What Effect can we estimate?
- We'd really like to estimate the average effect of $T$ $T$ on $Y$ $Y$ .
  - The effect of treatment on the outcome
- We can't quite estimate that, but we can estimate the average effect of $Z$ $Z$ on $Y$ $Y$ .
  - The effect of being offered the treatment on the outcome
- Just compare the average value of $Y$ $Y$ in the treatment and control condition
  - Regardless of who actually took the pill -> an unbiased estimate of being assignmed to the treatment condition
- We sometimes call this the intent-to-treat (ITT) effect or the "reduced form" effect.
  - We can measure the effect of our intent to treat on the population, but not the actual treatment
Exclusion Restriction
- Assumption: $Z$ $Z$ only influences $Y$ $Y$ through $T$ $T$
  - This assumption is called the exclusion restriction
- If the exclusion restriction holds, then the ITT is just the average effect of $T$ on $Y$ for the compliers times the proportion of the compliers in population
  - Sometimes called the complier average treatment effect (CATE) time $Pr(\text{complier})$
IV Assumptions: 4 main assumptions for IV design to yield unbiased estimates of the CATE
- Exogeneity: Instrument must be randomly assigned or "as if" randomly assigned
  - Allows us to obtain unbiased estimates of the first stage of reduced form effects
- Exclusion: All of the reduced form effect occurs through the treatment of interest
  - There is no other way that $Z$ influences $Y$ except through its effect on $T$ .
- Compliers: There must be some compliers
  - Otherwise, our estimates are meaningless
- No Defiers: If there are defiers, then our estimate will give us a weighted average effect for compliers and defiers with negative weight on the defiers.
  - So, unless the proportion of defiers is negligible OR the effect for defiers and compliers is the same, it's pretty hard to make sense of our estimate.

Wald Estimator and 2SLS

Review
- $\text{First Stage Effect}=Pr(\text{complier})$ $First Stage Effect = P r (complier)$
  - Estimated by $\text{Avg}[T\mid Z=1]-\text{Avg}[T\mid Z=0]$
  - Difference in take up between treatment and control assignments
- $\text{ITT}=\text{Reduced Form}=\text{CATE}\times Pr(\text{Complier})$ $ITT = Reduced Form = CATE \times P r (Complier)$
  - Estimated by $\text{Avg}[Y\mid Z=1]=\text{Avg}[Y\mid Z=0]$
  - Difference in outcome between treatment and control
Wald Estimator
- If we divide the reduced form by the first stage, we will get the CATE.
  - This procedure is called the Wald Estimator
    - Gives an unbiased estimate of the average effect of $T$ on $Y$ for compliers.
    - Note: can't possibly learn about effect of the treatment for always takers and never takers.

$\text{Wald Estimate}=\dfrac{\text{Avg}[Y\mid Z=1]-\text{Avg}[Y\mid Z=0]}{\text{Avg}[T\mid Z=1]-\text{Avg}[T\mid Z=0]}.$

Regression Approach
- We could have generated the exact same estimate using regression.
- Regression $T$ $T$ on $Z$ $Z$ and generate predicted values of $T$ $T$ for each observation.
  - Regression: $T_i=\alpha+\beta Z_i+\epsilon_i$
  - $\hat{T_i}=\hat{\alpha}+\hat{\beta}Z_i$
  - In this simple case, the predicted values are simply a rescaled version of $Z$ $Z$ .
    - Instead of $0$ 's and $1$ 's, we now have $\text{Avg}[T\mid Z=0]$ and $\text{Avg}[T\mid Z=1]$ and the two possible values.
  - Then, regress $Y$ $Y$ on these fitted values:
    - Regression: $Y_i=\tau+\pi\hat{T_i}+\mu_i$
  - This is called the Two-Stage Least Squares (2SLS)
Instrumental Variable Approach
- 2SLS and Wald are both special cases of a kind of analysis we call "instrumental variables" of IV.
- We say $Z$ (assignment) is an instrument for $T$ (taking the treatment).
- Practitioners use 2SLS more often than Wald because it gives us additional flexibility.
  - Can include control variables
  - Could have instruments and treatments that are continuous rather than binary.
  - Could even have multiple instruments and multiple treatments.

Module 10 Non-Compliance and Instruments

Non-Compliance and Instruments

Intro to Non-Compliance

Measuring Non-Compliance and First Stage

Wald Estimator and 2SLS

results matching ""

No results matching ""