So one of the courses I'm teaching this semester is introductory probability, with a calculus prerequisite. Not surprisingly this attracts a lot of students in various subjects that are math-heavy but that are not actually mathematics or statistics. They also tend to be juniors and seniors.
In office hours today, one of my students got to criticizing the fact that lots of consulting, finance, etc. jobs like to ask questions like "how many ping pong balls fit in a 747" in interviews, on the basis that this has nothing to do with what they'd actually have to do on the job. I pointed out that there's some difficulty in knowing whether your hiring practices are working. You'd like to compare the success of the people that you hire into your company with the success that the people that you don't hire would have had in your company. The former might be difficult to measure, but it's probably not impossible assuming your company does something quantifiable. But the latter is essentially impossible.
So what's the right way to design this sort of study? Is there any serious research on which interview techniques actually succeed in finding the best people?
Subscribe to:
Post Comments (Atom)
5 comments:
This is a classic problem in insurance. How many of the people you didn't insure would have made claims. There simply aren't too many good ways to tell without resorting to some questionable extrapolations. In this case one of these imperfect extrapolation methods would be
1. Hire as best you can but make everyone that applies do some arbitrary tests. (e.g. calculate the ping pong balls in a plane).
2. See if the people that performed better on these arbitrary tests actually perform better at work later.
Obviously the trouble here is that alot of the people you hire might perform well on the arbitrary test.
seems like a good segue into the secretary problem :)
Oddly enough, Suresh, I saw Jonathan (who commented above) give a talk once on the secretary problem.
Perhaps what you really care about is whether the people you hire based on some systematic evaluation perform better than other people who you hired using previous, presumably more subjective, evaluations. Probably you have plenty of the latter at your company already, so it might be worthwhile to compare the two groups, once you have a reasonable number of the former.
Malcolm Gladwell wrote an interesting article on this:
http://www.newyorker.com/reporting/2008/12/15/081215fa_fact_gladwell
If I recall correctly, he talks about some studies which indicate the many standard methods, like interviews, don't work very well.
I think he may also have talked about this in an expanded form in one of his books.
Post a Comment