Wednesday, December 05, 2012

Grand Game: Academic "Honor" Edition


UPDATE:  Okay, so don't even read my post.  It is lame and ad hoc.  GR answers all the questions, way better than I did.  Just read what he said.  Seriously.


So, this piece, based on this book, says that there is a giant conspiracy of "honor" in academic political science.

I'll admit that perhaps academic rank of PhD institution is more important than it "should" be (and I'm not sure what that means, but I'll admit it).  But if admissions processes are based on grades, recommendations, and GRE scores, and if the top ten PS departments* choose the 150 best applicants each year, wouldn't it be amazing if those departments did NOT dominate the job market?  Let's assume that the admissions criteria are only 50% predictive of later success.  Still, year after year, 75 of the best young political scientists in the country are going to the top ten departments. 

Sure, that means that there may be no value added in "top" PhD programs.  But if the admissions process selects based on traits that are actually correlated with ability, this is just sorting.   To put it another way, the admissions processes at top ten poli sci departments would have to be pretty dumb for anything other than dominance to occur.

Anyway, see if you agree with me, and pick out some of the logical howlers in this piece.

(*Based on this ranking, which the researchers used, Duke is 9th, so perhaps I'm just biased? Or drunk on all that "honor"?)

And, since @lordsutch linked it in the first place, apparently (?) approvingly...what say you, Lord Sutch?  You called it "in-breeding."  Why isn't it just probabilistic sorting?


Chris Lawrence said...

Tomato, tomahto.

In all seriousness, my POV is a little more complex than can be summarized in 140 characters. We know the most meritorious candidates don't necessarily get into the top 5 (or top 10) in the first place; certainly the mean aptitude of the Harvard PhD intake is higher than the mean aptitude of the North Texas* PhD intake, but it doesn't necessarily follow that the whole of the Harvard PhD intake is better than the most qualified UNT PhD student. Yet for a non-visible-minority UNT grad the chances of an interview for, much less placing in, a job in a doctoral-granting department are asymptotically close to zero, something unlikely the case for the low-to-mid-end Harvard student with equal aptitude.

* Not picking on UNT here; we could substitute any school in Texas not named "UT-Austin" or "Texas A&M," any SEC school other than Georgia and maybe South Carolina and pre-receivership Vandy, etc.

Natalie Jackson said...

Your points are valid. Of course these are just descriptive statistics at this point--there are things that need to be (and will be) controlled for as the project develops. My perspective is the same as Chris's. Just because you didn't have the prior advantages (top undergraduate education, contact with people who can get you on the path to the top programs, etc) that are needed to get into an Ivy or top PhD program doesn't mean your aptitude is automatically lower than the universe of students in those top programs. There is an element of sorting, but the question that needs to be answered is whether the advantage for the top programs holds after accounting for the sorting.

Anonymous said...

there was an earlier argument to this effect in a sociology journal article and I wrote a simulation of a more parsimonious explanation very similar to what you're proposing. (btw, this is an admission against interest for me since i've published an article on "status")

Dirty Davey said...

There is potential for a self-perpetuating phenomenon here, too. Just as schools choose students, students choose schools. And while ranking is one component of that decision, historically so are things like location, areas of focus, the presence of particular faculty members or particular research projects, and so one.

If it becomes significantly harder to get a job at even slightly lower-ranked schools, then the incentives change. So a prospective student might feel the difference between #9 and #13 (and the ability to be above the "top ten" line) is enough to make choosing Duke over Chapel Hill an obvious choice--even if 0.2 points on the scale really doesn't amount to a hill of beans.

I actually think that 50% is an overestimate for the predictive value of the admissions criteria, especially at the upper end--unless undergrad research work is a lot more advanced and undergrad recommendation letters a lot more substantive than they were twenty years ago.

(By "especially at the upper end", I mean: of course there are people who are badly prepared for graduate work and the admissions criteria work to eliminate a lot of no-hopers. But among the applicants with very good or better grades and GREs, I doubt that the ranking on these criteria is at all useful as a predictor of graduate and research success.)

Brad said...

While I agree the article is fairly absurd (didn't read the book), I am forced to question your conclusions, Munger, especially given your comments a few days ago about writing letters for students who aren't publishing (which is a lot at the top schools).

Let's except the students from top-10 departments who have great pubs. Let's assume for the sake of argument that they're effectively as rare as students from departments ranked 20-30 with great pubs (an assumption I don't think is too far out, fwiw). And these students generally clean up. Let's also not assume we're talking about the students at departments ranked 20-30 who never published (and agree that they're doomed when it comes to good jobs).

So what we're talking about is the students with middling publication records from the best schools, sstudents who are given access to significantly more resources than most students at lesser ranked schools. Students with less teaching requirements, more conference money, summer funding, etc. We're also talking about students with good publication records at "middling" schools. Those are the comparison groups.

Who should a department prefer? A student from Indiana or Michigan State with a decent solo-authored article or two, or a student from Columbia or Duke with one article co-authored with their advisor?

Assuming the latter is better because they got better grades at 20 (or 15) and scored better on a standardized test at 22 (or 17) is fairly absurd. But that's the sorting mechanism you're talking about. And 4 times out of 5 the top-10 ABD gets the job over the better-published (despite less resources) kid from IU or MSU.

Unknown said...

I appreciate all of the feedback that you are providing, but before dismissing this out-of-hand as sorting, may I make two suggestions:
1) Read the book to get the theory and don't dismiss the attachment to "honor" - otherwise, you aren't engaging the substance are are either attacking a straw man or engaging in red herring.
2) Realize that this is the beginning of a much larger research piece that is being constructed. There is much more forthcoming.

Mungowitz said...

Fair enough. I was dismissing the summarized conclusion from Lord Sutch, and as LS himself (correctly) notes, it was a TWEET, fercrissakes.

My own response would be this: it's a BLOG POST. Chill.

Chris Lawrence said...

Actually FWIW I appreciated the post; my tweet was probably a little too finals-inspired-snarky.

Anonymous said...

I get the sorting argument, but it assumes full mobility on the part of the applicants. Granted, I would say that most of the best prospective doctoral students are quite mobile, but we do see a decent sized group of students who are pretty tied to a certain area (often for family or cultural reasons).

Of course, the real problem here is whether we can demonstrate that a reasonable number of prospective students could have been admitted to a top ten institution but did not apply or accept.

An interesting conversation either way, though.