The study on the determinants of college graduation rates is making a big splash. However, its analysis is, with all due respect, crap. Correlation is not causation, there are generally multiple explanations for a correlation and it is not correct to simply pick one and assert its truthfulness.
Consider this part of the story:
Students shouldn't settle for less in a college: Thousands of bright, qualified students apply only to lower-ranked schools where their grades and tests scores are above those of the average student. But the new study finds that those who attend such "safety" schools are far more likely to drop out than those who get into "reach" schools. "It is counterintuitive," Bowen says. "You might think that if Sally goes to a school where she is top dog, she will have a much easier time graduating. But that's not true. She has a better chance of graduating if she goes to school with other people as talented she is."
Well, his (Bowen is one of the authors of the study) interpretation of the correlation certainly is counter-intuitive. It is also almost certainly incorrect! How about this instead: Students who pick an easy school when higher quality options are available to them are not very interested in higher education and are signaling by their very choice that they are unlikely to complete a degree. The last sentence in the quote above should read: "She has a much better chance of graduating if she WANTS to go to a school with other people as talented as she is".
In other words, a lack of desire to get a college degree is driving both the choice of an easy school and the failure to graduate.
I am not saying that my interpretation is 100% correct, but it least it posits a causal mechanism that makes the correlation un-puzzling. The authors, to me are being almost willfully dense. They admit their view is "counter-intuitive" but can't bring themselves to think about anything else.
The next paragraph in the story makes the same mistake again:
Admissions tests don't predict graduation: SAT and ACT test scores are no help in predicting who will graduate from many, if not most, colleges. The widely used tests do help identify those likely to succeed at elite schools, the study found. But for many less selective colleges, students with higher scores were actually more likely to drop out.
Again, it's counter-intuitive unless you consider that highly qualified people picking an easy alternative are showing their actual lack of interest in the endeavor and thus are intrinsically less likely to complete said endeavor.
Putting students who don't want to go to a competitive college into a competitive college is NOT going to raise graduation rates in any significant way. Putting them into one or two year (instead of 4 year) certificate/professional training / apprenticeship programs or just getting off their backs and letting them go to work is a better way to make them happy and improve the economic health of the polity.
2 comments:
your explanation assumes that there is a lot of unobserved heterogeneity that it has strong effects. personally, i'm inclined to believe the Bowen et al study because personal experience shows me plausible micro mechanisms.
when i was a grad student and TA at a highly selective private school we had a couple kids who blew off classes, did shoddy work, etc, and the institution did not let them fail. i don't mean this in a social promotion kind of way, but their house masters were on the look-out for problems early on and intervened to solve the students' problems.
contrast this with the slightly less selective state school where i am now faculty. not only do we lack the kind of aggressive counseling to catch students who screw up, but we don't even provide enough resources for students who are being reasonably responsible to graduate in four years. most notably, there simply aren't enough seats to go around and most classes not only have full wait lists but a long list of less formal petitioners.
given treatment effects like this, i don't need to believe in selection by unobserved heterogeneity. i find it extremely plausible that a student who got in to both my grad school's UG college and my current institution would be more likely to graduate if he/she chose the former.
Gabriel, your argument is plausible, but determining whether that explanation holds should be remarkably simple: include a measure of expenditure per enrolled student in the regression. After all, your argument essentially revolves around resources devoted to student retention, which should primarily show up in the expenditure measure.
I would assume (hope?) that Bowen would already have included this obvious regressor in the analysis, in which case the resulting partial correlations can't be explained that way. If, on the other hand, Bowen's analysis omits this variable, then all the results are suspect for an entirely different reason. Either way, Bowen's not off the hook.
Post a Comment