Fiona McQuarrie asks:
Just curious whether you have any opinion on the Hoxby class size research (in Connecticut) that Gladwell discusses.
Here’s an interesting summary of class size research from Brookings. It is worth reading if you’re interested in the topic.
There’s a lot of stuff going on with class size research (it is, in fact, the topic going through the Stock and Watson undergraduate econometrics textbook because it has been attacked through most standard econometrics methods).
A couple of important things to note about external validity for these studies:
1. Natural experiments (and, indeed, standard experiments) are only as externally valid as the experiment itself. That means that a study that finds an effect on kindergarteners is not going to necessarily say much about high school students. We know a lot about class size and K-3, we don’t know so much about middle grades or higher. This particular experiment is on 4th and 6th grade. It argues that it gets cumulative effects of class size by cohort size, but when a cohort is expected to be a certain size, districts may plan differently by moving bad teachers to small cohorts and good teachers to larger cohorts etc. They may do the same with aides when deciding where to make a class-size split, or they may make specific decisions about where to put the problem kids or whether to do tracking or clustering. That kind of planning would completely wash out the effect in a way that you would not see if all classes were restricted to a certain size because of a policy change. That kind of planning is more likely to be going on in the type of natural experiment that Hoxby examines in this study.
2. Class size decisions are not made in isolation. A policy asking for extra money from the federal government to reduce class size is going to provide different results than a policy that is forced to take that extra money out of another budget. Generally, research suggests that, believe it or not, most schools are doing the best that they can with the budgets that they have. When you give them an unfunded mandate, outcomes are hurt in ways that they wouldn’t be if you gave them a funded mandate. Hiring more new teachers and buying portables while taking money away from other programs may end up having a negative effect even if smaller class-sizes are beneficial. The type of natural experiment Hoxby is looking at is one of these situations– the budget isn’t changing based on class-sizes, they get the same $/kid whether they’re in a large cohort or a small cohort. The only thing that changes is the expense from economies of scale (whether they need one teacher/classroom or two). That’s a different situation than one in which expenses for everything else stays the same but the district gets extra money to hire more teachers and buy portables.
So, do Hoxby’s results mean that class size is unimportant? No. They just show that it seems to be unimportant in the type of situation that she’s studying, one in which variations in elementary school class-size are caused by variations in cohort size. That’s why there’s a large literature on this topic– the answer is different in different situations. We need a lot of experiments and natural experiments to get the full picture.
Side note: Caroline Hoxby is one of my personal heroes. If I ever decide to give up this academia thing, I’m totally going to beg her for an RA job. She is an amazing economist. Also, rumor has it (aka multiple of her coauthors has mentioned) that she is one of those people who sleeps 4 hours/night every night because of low sleep need.