All dressed up but nowhere to go
There are a lot of things that drive me crazy about the current practice of econometrics. People who think over-identification tests validate their indentifying assumptions. People who think that if you fail to reject the null at the 0.05 level, it's fine to proceed in your analysis as if the null was true (i.e. people who don't believe in type II error).
But one of the biggest is the practice of thinking we do no harm by using estimators we know to be inappropriate for the data at hand and thinking we somehow fully fix that issue by using robust standard errors.
I annually beat my head against the wall trying to get my students to appreciate these issues (only to often have my work undone by their reading papers/books that make these mistakes), but now on this last point, I have some help!
Continue reading below the fold
There is a new working paper by King & Roberts that spells out and gives examples of the kinds of problems that arise when we think robust standard errors somehow fix specification problems. Here's a fun quote:
According to the contrary philosophy of science implied by the most common use of robust standard errors, if it looks like a duck and smells like a duck, it could well be delicious peanut butter sandwich.
This paper is gold, and will be required reading in my 'metrics class.
The other highly relevant paper here is Leamer's short, classic 2010 piece in the Journal of Economic Perspectives (ungated version here).
Here's the most relevant snippet:
An earlier generation of econometricians corrected the heteroskedasticity problems with weighted least squares using weights suggested by an explicit hetero- skedasticity model. These earlier econometricians understood that reweighting the observations can have dramatic effects on the actual estimates, but they treated the effect on the standard errors as a secondary matter. A “robust standard” error completely turns this around, leaving the estimates the same but changing the size of the confifidence interval. Why should one worry about the length of the confifidence interval, but not the location? This mistaken advice relies on asymp- totic properties of estimators.5 I call it “White-washing.” Best to remember that no matter how far we travel, we remain always in the Land of the Finite Sample, infifinitely far from Asymptopia. Rather than mathematical musings about life in Asymptopia, we should be doing the hard work of modeling the heteroskedasticity and the time dependence to determine if sensible reweighting of the observations materially changes the locations of the estimates of interest as well as the widths of the confifidence intervals.