Wednesday, May 09, 2012
Fooled by randomness?
Wikipedia tells me Philip Tetlock is a distinguished chaired professor at Penn, who is best known for being highly skeptical of the value of "expert" forecasts.
Yet today here is a guest post up at Marginal Revolution from him this morning that I find extremely confusing. Tetlock is describing a multi-year forecasting experiment and doing things that I find quite surprising/shocking.
First consider the claim is that the forecasters did well because an ex-post weighted average of the "most insightful and engaged forecasters" crushed a control group.
Yikes!
In fact, they did so well that the Tetlock's group is breaking their planned experimental study after one year and is creating a group of "super-forecasters" based on their first year track record.
Double Yikes!
People, in a small sample stochastic environment, there will always be individuals that do better than others. This could purely be by chance. If you ex-post weight them higher, you can get a very impressive short term performance.
Generally speaking though, godding them up and following their forecasts into the future is a fools game, because their success was just short term randomness, or in plainer terms, luck.
Or as Mrs. Angus succinctly put it: "mean reversion, y'all".
If you do decided to enter the contest, you may find this information helpful.
Subscribe to:
Post Comments (Atom)
5 comments:
He can apply to be Steven Landsburg's "Royal Head Flipper"
http://books.google.com/books?id=_aZmzdlrBZMC&lpg=PA309&ots=QAVbX0KqHy&dq=the%20royal%20head%20flipper&pg=PA309#v=onepage&q&f=false
"... the forecasters did well because an ex-post weighted average of the 'most insightful and engaged forecasters' crushed a control group."
Sounds like survival of the fittest.
I sincerely hope that Mrs. Angus does not apply the same comment to your offspring when they misbehave.
http://www.patriotsjerseysprosale.com/aaron-hernandez-jersey/aaron-hernandez-jersey-new-england-patriots-black-shadow-81-jersey.html
http://www.patriotsjerseysprosale.com/andre-carter-jersey-/andre-carter-jersey-new-england-patriots-blue-93-jersey.html
http://www.patriotsjerseysprosale.com/andre-tippett-jersey/andre-tippett-throwback-jersey-new-england-patriots-mitchell-and-red-56-ness-nfl-authenticjersey.html
http://www.patriotsjerseysprosale.com/benjarvus-green-ellis-jersey/reebok-benjarvus-green-ellis-jersey-new-england-patriots-black-shadow-42-nfl-authentic-jersey.html
Unser Online-Shop bietet günstige Monster Beats By Dre, Beats By Dre Outlet,Beats Dr Dre Studio Und Dr. Dre Beats Solo-Sale. Hohe Qualität und lLowest Preis,Genießen Sie sparen bis zu 70%.
http://www.beatsbydrebilligkaufen.net/
http://www.clarisonicdeutschland.com/
Post a Comment