A Difference-in-Differences Approach using Mixed-Integer Programming Matching

class: center, middle, inverse, title-slide

# A Difference-in-Differences Approach using Mixed-Integer Programming Matching
## Magdalena Bennett McCombs School of Business, UT Austin
### AEFP 46th Annual Conference Mar 19, 2021

---

# Diff-in-Diff as an identification strategy

<img src="mbennett_did_files/figure-html/dd-1.svg" style="display: block; margin: auto;" />
---
# Diff-in-Diff as an identification strategy

---
# Diff-in-Diff as an identification strategy

---

# What about parallel trends?

.pull-left[
.center[
![:scale 80%](https://raw.githubusercontent.com/maibennett/website_github/master/exampleSite/content/images/data_comic.jpg)]]

.pull-right[
- Can matching help solve this?

- It's complicated (?) .small[(Zeldow & Hatfield, 2019;Lindner & McConnell, 2018; Daw & Hatfield, 2018 (x2); Ryan, 2018; Ryan et al., 2018)]
  
- Most work has focused on **.darkorange[matching outcomes]**]

---

# This paper

.pull-left[
- Identify contexts when matching can recover causal estimates under **.darkorange[violations in the parallel trend assumption]**.
 
 
- Use **.darkorange[mixed-integer programming matching (MIP)]** to balance covariates directly.
 
 
- Matching for **.darkorange[panel]** and **.darkorange[repeated cross-sectional]** data.

]
--
.pull-right[
 
.box-5[Simulations: Different DGP scenarios]
 
.box-5[Application: School segregation & vouchers]
]

---
background-position: 50% 50%
class: left, bottom, inverse
.big[
Let's get started
]
---

# DD Setup

- Let `$Y_{it}(z)$` be the potential outcome for unit `$i$` in period `$t$` under treatment `$z$`.

- Intervention implemented in `$T_0$` `$\rightarrow$` No units are treated in `$t\leq T_0$`

- Difference-in-Differences (DD) focuses on ATT for `$t>T_0$`:
`$$ATT = E[Y_{it}(1) - Y_{it}(0)|Z=1]$$`
- **.darkorange[Assumptions for DD]**:
  - Parallel-trend assumption (PTA)
  
  - Common shocks
  
  `$$E[Y_{i1}(0) - Y_{i0}(0) | Z=1] = E[Y_{i1}(0) - Y_{i0}(0) | Z=0]$$`
---
# DD Setup (cont.)

- Under these assumptions:
$$
`\begin{align}
\hat{\tau}^{DD} = &\color{#900DA4}{\overbrace{\color{black}{E[Y_{i1}|Z=1] - E[Y_{i1}|Z=0]}}^{\color{#900DA4}{\Delta_{post}}}} - \\
&\color{#F89441}{\underbrace{\color{black}{(E[Y_{i0}|Z=1] - E[Y_{i0}|Z=0])}}_{\color{#F89441}{\Delta_{pre}}}}
\end{align}`
$$
  - Where `$t=0$` and `$t=1$` are the pre- and post-intervention periods, respectively.
  
  - `$Y_{it} = Y_{it}(1)\cdot Z + Y_{it}(0)\cdot(1-Z)$` is the observed outcome.

---

# Violations to the PTA

.pull-left[
- Under PTA, `$g_1(t) = g_0(t) + h(t)$`, where:

- `$g_z(t) = E[Y_{it}(0) | Z=z, T=t]$`
  - `$h(t) = \alpha$`
  
- Bias in a DD setting depends on the structure of `$h(t)$`.

]

.pull-right[
![](https://media.giphy.com/media/L8yQ0RQBItqso/giphy.gif)
]
---
# Two distinct problems when combining matching + DD

.pull-left[
- **.darkorange[Regression to the mean]**:

- Both groups come from different populations
  - Particularly salient when matching on previous outcomes
  
- **.darkorange[Bias when matching on time-varying covariates]**:
  
  - Depends on the structure of time variation
]

.pull-right[
<img src="mbennett_did_files/figure-html/reg_to_the_mean-1.svg" style="display: block; margin: auto;" />
]

---
background-position: 50% 50%
class: left, bottom, inverse
.big[
Simulations
]

---

# Different scenarios

.small[
.box-1[S1: Time-invariant covariate effect]

.box-2[S2: Time-varying covariate effect]

.box-3[S3: Treatment-independent covariate]

.box-4[S4: Parallel evolution]

.box-6[S5: Evolution differs by group]

.box-7[S6: Evolution diverges in post]
]
.source[Following Zeldow & Hatfield (2019)]

---

# Different scenarios

.small[
.box-1n[S1: Time-invariant covariate effect]

.box-2n[S2: Time-varying covariate effect]

.box-3n[S3: Treatment-independent covariate]

.box-4[S4: Parallel evolution]

.box-6[S5: Evolution differs by group]

.box-7[S6: Evolution diverges in post]
]
.source[Following Zeldow & Hatfield (2019)]

---
#Parameters:

.center[
Parameter | Value
-------------------------------------|----------------------------------------------
Number of obs (N) | 1,000 
Pr(Z=1) | 0.5 
Time periods (T) | 10 
Last pre-intervention period (T0) | 5 
Matching PS | Nearest neighbor
MIP Matching tolerance | .05 SD
Number of simulations | 1,000
]

- Estimate compared to sample ATT (_different for matching_)
- When matching with post-treat covariates `$\rightarrow$` compared with direct effect `$\tau$`

---
# Summary of results

.pull-left[
- For **.darkorange[time-invariant covariates]**, matching actually **.darkorange[helps]**.

- For **.darkorange[time-varying covariates]**, matching on pre-intervention covariates can improve estimation **.darkorange[unless they diverge post]**.

- Can try to **.darkorange[bind effects]**.
  
  
- **.darkorange[Autocorrelation]** for time-varying covariates also plays an important role:

- If they come from different distributions, **.darkorange[high-autocorrelation is better]**.

]

.pull-right[
<img src="mbennett_did_files/figure-html/res2b-1.svg" style="display: block; margin: auto;" />
]

---
background-position: 50% 50%
class: left, bottom, inverse
.big[
Application
]
---
#Preferential Voucher Scheme in Chile

- Universal **.darkorange[flat voucher]** scheme `$\stackrel{\mathbf{2008}}{\mathbf{\longrightarrow}}$` Universal + **.darkorange[preferential voucher]** scheme

- Preferential voucher scheme:

- Targeted to bottom 40% of vulnerable students

- Additional 50% of voucher per student

- Additional money for concentration of SEP students.

--
 
.pull-left[
.center[
.box-5b[Students: - Verify SEP status - Attend a SEP school]]
]

.pull-right[
.center[
.box-5b[Schools: - Opt-into the policy - No selection, no fees - Resources ~ performance]]
]

---
#Impact of the SEP policy

- **.darkorange[Positive impact on test scores]** for lower-income students (Aguirre, 2019; Nielson, 2016)

- Design could have **.darkorange[increased]** socioeconomic segregation
  - Incentives for concentration of SEP students
  
- Key decision variables: Performance, current SEP students, competition, add-on fees.

- **.darkorange[Diff-in-diff (w.r.t. 2007) for SEP and non-SEP schools]**:
 - Only for **.darkorange[private-subsidized schools]**
 
 - Matching between 2005-2007 --> Effect estimated for 2008-2011
 
 - Outcome: Average students' household income
 
---
#Before Matching
 
.pull-left[
<img src="https://raw.githubusercontent.com/maibennett/presentations/main/content/presentations/DD/aefp_20210319/images/dd_all.svg" alt="diagram" width="800"/>
]
.pull-right[
- No (pre) parallel trend

- Covariates evolve differently in the pre-intervention period
]

---
# [Pre] parallel trends

.pull-left[
<img src="https://raw.githubusercontent.com/maibennett/presentations/main/content/presentations/DD/aefp_20210319/images/pta_all.svg" alt="diagram" width="800"/>
]
.pull-right[
<img src="https://raw.githubusercontent.com/maibennett/presentations/main/content/presentations/DD/aefp_20210319/images/pta_match.svg" alt="diagram" width="800"/>
]

---
#After Matching
 
.pull-left[
<img src="https://raw.githubusercontent.com/maibennett/presentations/main/content/presentations/DD/aefp_20210319/images/dd_match.svg" alt="diagram" width="800"/>
]
.pull-right[
- **.darkorange[MIP Matching]**: 
 - Mean balance (0.05 SD): Rural, enrollment, number of schools in county, charges add-on fees

- Fine balance: Test scores, monthly average voucher.

- **.darkorange[6% increase in the income gap]** between SEP and non-SEP schools
]
---
background-position: 50% 50%
class: left, bottom, inverse
.big[
Let's wrap it up
]
---
#Conclusions

.pull-left[
- **.darkorange[Matching can be an important tool to address violations in PTA]**.

- Relevant to think whether groups come from the **.darkorange[same]** or **.darkorange[different]** populations.

- **.darkorange[Serial correlation]** also plays an important role: Don't match on random noise.

.center[
.box-7a[Match well and match smart!]]
]
.pull-right[
![](https://media.giphy.com/media/drwxYI2fxqQGqRZ9Pe/giphy.gif)
]

---
background-position: 50% 50%
class: center, middle, inverse
.notsobig[
A Difference-in-Differences Approach using Mixed-Integer Programming Matching
]
##Magdalena Bennett
 
 
 www.magdalenabennett.com

---
background-position: 50% 50%
class: center, middle
.notsobig[
Additional Slides
]

---
# Results: Time-invariant covariates

---
# Results: Time-varying covariates

<img src="mbennett_did_files/figure-html/res2-1.svg" style="display: block; margin: auto;" />
---
# Other simulations

- Test **.darkorange[regression to the mean]** under no effect:

- Vary autocorrelation of `$X_i(t)$` (low vs. high)
  - `$X_0(t)$` and `$X_1(t)$` come from the same or different distributions.