(adapted from twitter: https://twitter.com/besttrousers/status/1147925037648928769)

This Rubin Report interview with Andrew Yang starts with a great example of why its important to distinguish between Y hat and Beta hat^1.

Yang:

Studies have shown that 70 to 75% of kids academic performance is determined by out-of-school factors…and so right now we're going to teachers 'Hey take a hundred percent responsibility for a process that you can control 25 percent of.'

I'm not sure which studies he's referring to, but it's clear from context that he's talking about studies that are trying to estimate Y-hat. i.e., you can run a regression on GPA and find that out-of-school factors (parental engagement, household income, neighborhood) predicts 70% of the variance in academic outcomes.

But you can not (as Yang does) infer from that finding that there's only 25% left for in-school factors.

Why? Because many of the in-school factors are determined by the out-of-school factors!

For example, schools are paid for by property taxes. People from wealthy communities typically have better funded schools (and better outcomes).

Imagine a world where education outcomes is entirely determined by funding, which in turn is entirely determined by household wealth. A regression would show that out-of-school factors completely explain 100% of academic outcomes.

But that doesn't mean we can effect school outcomes through education policy. We'd see increases in academic outcomes if we increased funding in areas with low household wealth.

If we want to examine whether we can effect academic outcomes by education policy, we need to run studies that estimate beta hat.

This is an active research are! There is substantial evidence that changes in school quality change substantially effect academic outcomes.

• There's evidence from charter schools https://www.nber.org/papers/w19275.pdf

• There's evidence from variation in teacher quality: http://www.equality-of-opportunity.org/assets/documents/teachers_wp.pdf

• And there's evidence from busing: https://www.nber.org/digest/may11/w16664.html

Yang's claim that the best way to improve our education systems is by giving household more money doesn't follow from the arguments he makes. It's based on a misunderstanding of statistics.

1 - For context:

• Y hat: Trying to predict an outcome variable (say, academic performance) - usually, by including lots of different input variables.

• Beta hat: Trying to estimate the causal effect of a given input variable on the outcome variable.