There's an old joke, what do you call someone who graduated last in his class at Med School?.....Doctor!
Now we can add, what do you call someone who graduated in the middle of his PhD. class in Econ at Harvard 6 years ago?.......I don't know, but it's not Associate Professor.
Interesting short paper from Vanderbilt on class ranking and publishing productivity at top grad schools.
Here's the abstract:
We study the research productivity of top Ph.D. programs in economics. We find that class rank is as important as departmental rank as predictors of future research productivity. For example, the best graduate from UIUC or Toronto in a given year will have roughly the same number of American Economic Review (AER) equivalent publications at year six after graduation as the number three graduate from Berkeley, U. Penn or Yale. We also find that research productivity of graduates drops off very quickly with class rank at all departments. For example, even at Harvard, the median graduate has only 0.04 AER paper at year six, an untenurable record at almost any department. These results provide guidance on how much weight to give to place of graduation relative to class standing when hiring new assistant professors. They also suggest that even the top departments are not doing a very good job of training students to be successful research economists for any not in the top of their class.
That's right, the median Harvard Grad has 0.04 AER equivalent papers at tenure time! Ouch.
Do departments concentrate all their resources on the top few students?
Is the talent pool in economics very shallow and schools are taking students who shouldn't be there just to fill up their TA slots?
Now we can add, what do you call someone who graduated in the middle of his PhD. class in Econ at Harvard 6 years ago?.......I don't know, but it's not Associate Professor.
Interesting short paper from Vanderbilt on class ranking and publishing productivity at top grad schools.
Here's the abstract:
We study the research productivity of top Ph.D. programs in economics. We find that class rank is as important as departmental rank as predictors of future research productivity. For example, the best graduate from UIUC or Toronto in a given year will have roughly the same number of American Economic Review (AER) equivalent publications at year six after graduation as the number three graduate from Berkeley, U. Penn or Yale. We also find that research productivity of graduates drops off very quickly with class rank at all departments. For example, even at Harvard, the median graduate has only 0.04 AER paper at year six, an untenurable record at almost any department. These results provide guidance on how much weight to give to place of graduation relative to class standing when hiring new assistant professors. They also suggest that even the top departments are not doing a very good job of training students to be successful research economists for any not in the top of their class.
That's right, the median Harvard Grad has 0.04 AER equivalent papers at tenure time! Ouch.
Do departments concentrate all their resources on the top few students?
Is the talent pool in economics very shallow and schools are taking students who shouldn't be there just to fill up their TA slots?
5 comments:
I suspect networking makes a big difference. The top MIT PhD grad will go to Harvard or Princeton and work with co-authors at the top of the field, gaining instant credibility at the AER, Ecta, QJE, JPE...
Low-ranked MIT grads will work in a top-50 or top-100 university, without an impressive network of co-authors. Good ideas and execution are not enough to publish in AER, you need credibility too.
I'm thinking it's the latter. I remember reading about a big fight between the Admin and faculty at some university a couple of years ago over something like this. The admin wanted to slash enrollment in a number of graduate programs because many of them were not finding jobs in their fields of study, whether academic or otherwise. Tenured professors revolted, I believe, when this led to a shortage of TA's.
The data section implies that they are looking at publication counts for the entire cohort at each school, even the students who took government and private sector jobs. Since a lot of econ grads leave academia, even from top programs, the results are not as surprising as I first thought.
Of course, now we have a data censoring problem. How well would the students outside academia have published if they had stayed? Better than they do now, but by how much depends on whether those outside options are a dumping ground for the research-inept or not.
Lots of Good information in your post, I favorited your blog post so I can visit again in the future, Thanks.
Andrew: various censoring issues are at play in this study, but they are easier to address than usual.
The stated purpose of the working paper (to give predictive advice to hiring committees) suggests that the past publication data that is of interest are those who were hired into tenure-track positions (or were in decent postdoc positions). In both such cases, publications are an expected output.
The relevant sub-population can be approximately identified by identifying the members of the base population who have anything published anywhere in any ISI or SCOPUS indexed publication, and examining the institutional affiliation (SCOPUS is better than ISI for this).
There is a right-censoring problem as well; publication lag for journals may mean that an article is accepted before the six year window, but appears after. Acceptance date is probably the relevant datum here.
It is possible that more relevant dates may be the date of conditional-acceptance (or possibly R&R, since a revised version was published). This data may not be available, though submission dates might be.
Pre tenure-track postdoc positions, and pre-dissertation tenure-track hiring may also make the year the dissertation was received a poor choice (especially if the defense takes place early or late in the year).
Date of starting tenure-track position may a better choice.
Where the study really falls apart is in it's use of an ad-hoc scoring formula, which gives article length relative to AER equal weight to quality relative to AER (so Science == bad).
There is a weak approach to dealing with multiple authors that ignores factors such as order of contributors, let alone explicit statements of which parts of the paper which author was responsible for.
The weakest part of the methodology is the use of an explicitly ad-hoc formula for journal quality (e.g. 1 AER = 2 Journal of Econometrics = 5 Journal of Public Economics).
There are enough empirical metrics available that the quality metric chosen is motivated solely as a means of giving weight to the ad-hoc judgements of hiring committees of what the ad-hoc judgements of tenure committees would be.
The results should be recalculated using other measures of quality, such as Impact or Eigen Factor, and the results compared to determine how sensitive the results are to journal ranking methodology
The validity of the metric should also be tested by regressing against whether the individual was actually tenured or not. A proxy for this might be the existence of conference papers showing the same affiliation that continue several years past the predicted tenure deadline.
Post a Comment