A Difference-in-Differences Approach
using Mixed-Integer Programming MatchingMagdalena Bennett
McCombs School of Business, UT AustinAEFP 46th Annual Conference
Mar 19, 2021
  

Diff-in-Diff as an identification strategy

What about parallel trends?

Can matching help solve this?
- It's complicated (?) (Zeldow & Hatfield, 2019;Lindner & McConnell, 2018; Daw & Hatfield, 2018 (x2); Ryan, 2018; Ryan et al., 2018)
Most work has focused on matching outcomes

This paperIdentify contexts when matching can recover causal estimates under violations in the parallel trend assumption.

Use mixed-integer programming matching (MIP) to balance covariates directly.

Matching for panel and repeated cross-sectional data.

This paper

Identify contexts when matching can recover causal estimates under violations in the parallel trend assumption.
Use mixed-integer programming matching (MIP) to balance covariates directly.
Matching for panel and repeated cross-sectional data.

Simulations:
Different DGP scenarios
Application:
School segregation & vouchers

Let's get started

DD Setup

Let $Y_{i t} (z)$ be the potential outcome for unit $i$ in period $t$ under treatment $z$ .
Intervention implemented in $T_{0}$ $\to$ No units are treated in $t \leq T_{0}$
Difference-in-Differences (DD) focuses on ATT for $t > T_{0}$ : $A T T = E [Y_{i t} (1) - Y_{i t} (0) | Z = 1]$
Assumptions for DD:
- Parallel-trend assumption (PTA)
- Common shocks
$E [Y_{i 1} (0) - Y_{i 0} (0) | Z = 1] = E [Y_{i 1} (0) - Y_{i 0} (0) | Z = 0]$

DD Setup (cont.)

Under these assumptions: $\begin{aligned} {\hat{τ}}^{D D} = & \overset{Δ_{p o s t}}{\overset{⏞}{E [Y_{i 1} | Z = 1] - E [Y_{i 1} | Z = 0]}} - \\ \underset{Δ_{p r e}}{\underset{⏟}{(E [Y_{i 0} | Z = 1] - E [Y_{i 0} | Z = 0])}} \end{aligned}$
- Where $t = 0$ and $t = 1$ are the pre- and post-intervention periods, respectively.
- $Y_{i t} = Y_{i t} (1) \cdot Z + Y_{i t} (0) \cdot (1 - Z)$ is the observed outcome.

Violations to the PTA

Under PTA, $g_{1} (t) = g_{0} (t) + h (t)$ , where:
- $g_{z} (t) = E [Y_{i t} (0) | Z = z, T = t]$
- $h (t) = α$
Bias in a DD setting depends on the structure of $h (t)$ .

Two distinct problems when combining matching + DD

Regression to the mean:
- Both groups come from different populations
- Particularly salient when matching on previous outcomes
Bias when matching on time-varying covariates:
- Depends on the structure of time variation

Simulations

Different scenarios

S1: Time-invariant covariate effect

S2: Time-varying covariate effect

S3: Treatment-independent covariate

S4: Parallel evolution

S5: Evolution differs by group

S6: Evolution diverges in post

Following Zeldow & Hatfield (2019)

Different scenarios

S1: Time-invariant covariate effect

S2: Time-varying covariate effect

S3: Treatment-independent covariate

S4: Parallel evolution

S5: Evolution differs by group

S6: Evolution diverges in post

Following Zeldow & Hatfield (2019)

Parameters:

Parameter
Value


Number of obs (N)
1,000 

Pr(Z=1)
0.5 

Time periods (T)
10 

Last pre-intervention period (T0)
5 

Matching PS
Nearest neighbor

MIP Matching tolerance
.05 SD

Number of simulations
1,000


Estimate compared to sample ATT (different for matching)
When matching with post-treat covariates →→ compared with direct effect ττ

  

Parameter	Value
Number of obs (N)	1,000
Pr(Z=1)	0.5
Time periods (T)	10
Last pre-intervention period (T₀)	5
Matching PS	Nearest neighbor
MIP Matching tolerance	.05 SD
Number of simulations	1,000

Summary of results

For time-invariant covariates, matching actually helps.

For time-varying covariates, matching on pre-intervention covariates can improve estimation unless they diverge post.
- Can try to bind effects.

Autocorrelation for time-varying covariates also plays an important role:
- If they come from different distributions, high-autocorrelation is better.

Application

Preferential Voucher Scheme in Chile

Universal flat voucher scheme $\overset{2008}{⟶}$ Universal + preferential voucher scheme
Preferential voucher scheme:
- Targeted to bottom 40% of vulnerable students
- Additional 50% of voucher per student
- Additional money for concentration of SEP students.

Preferential Voucher Scheme in Chile

Universal flat voucher scheme $\overset{2008}{⟶}$ Universal + preferential voucher scheme
Preferential voucher scheme:
- Targeted to bottom 40% of vulnerable students
- Additional 50% of voucher per student
- Additional money for concentration of SEP students.
  
  Students:
  - Verify SEP status
  - Attend a SEP school

Schools:
- Opt-into the policy
- No selection, no fees
- Resources ~ performance

Impact of the SEP policy

Positive impact on test scores for lower-income students (Aguirre, 2019; Nielson, 2016)
Design could have increased socioeconomic segregation
- Incentives for concentration of SEP students
Key decision variables: Performance, current SEP students, competition, add-on fees.
Diff-in-diff (w.r.t. 2007) for SEP and non-SEP schools:
- Only for private-subsidized schools
- Matching between 2005-2007 --> Effect estimated for 2008-2011
- Outcome: Average students' household income

Before Matching

diagram

No (pre) parallel trend
Covariates evolve differently in the pre-intervention period

[Pre] parallel trends

diagram

After Matching

diagram

MIP Matching:
- Mean balance (0.05 SD): Rural, enrollment, number of schools in county, charges add-on fees
- Fine balance: Test scores, monthly average voucher.
6% increase in the income gap between SEP and non-SEP schools

Let's wrap it up

Conclusions

Matching can be an important tool to address violations in PTA.
Relevant to think whether groups come from the same or different populations.
Serial correlation also plays an important role: Don't match on random noise.

Match well and match smart!

A Difference-in-Differences Approach
using Mixed-Integer Programming Matching

Magdalena Bennett

www.magdalenabennett.com

Additional Slides

Results: Time-invariant covariates

Results: Time-varying covariates

Other simulations

Test regression to the mean under no effect:
- Vary autocorrelation of $X_{i} (t)$ (low vs. high)
- $X_{0} (t)$ and $X_{1} (t)$ come from the same or different distributions.

↑, ←, Pg Up, k	Go to previous slide
↓, →, Pg Dn, Space, j	Go to next slide
Home	Go to first slide
End	Go to last slide
Number + Return	Go to specific slide
b / m / f	Toggle blackout / mirrored / fullscreen mode
c	Clone slideshow
p	Toggle presenter mode
t	Restart the presentation timer
?, h	Toggle this help
s	Toggle scribble toolbox

A Difference-in-Differences Approachusing Mixed-Integer Programming Matching

Magdalena BennettMcCombs School of Business, UT Austin

AEFP 46th Annual ConferenceMar 19, 2021

Diff-in-Diff as an identification strategy

Diff-in-Diff as an identification strategy

Diff-in-Diff as an identification strategy

What about parallel trends?

This paper

This paper

DD Setup

DD Setup (cont.)

Violations to the PTA

Two distinct problems when combining matching + DD

Different scenarios

Different scenarios

Parameters:

Summary of results

Preferential Voucher Scheme in Chile

Preferential Voucher Scheme in Chile

Impact of the SEP policy

Before Matching

[Pre] parallel trends

After Matching

Conclusions

Magdalena Bennett

Results: Time-invariant covariates

Results: Time-varying covariates

Other simulations

Diff-in-Diff as an identification strategy

Help

A Difference-in-Differences Approach
using Mixed-Integer Programming Matching

Magdalena Bennett
McCombs School of Business, UT Austin

AEFP 46th Annual Conference
Mar 19, 2021