This study was from 2014 and titled, “What Aspects of Letters of Recommendation Predict Performance in Medical School? Findings From One Institution.”
Do you have a guess?
Being rated as “the best” among peers and having an employer or supervisor as the LOR author were associated with induction into AOA, whereas having nonpositive comments was associated with bottom of the class students.
Best = good
Nonpositive comments = nice way of saying you don’t really know someone or think they were lame = bad
Now, this was in reference to medical student performance based on college LORs, but some parallels exist in any situation where the current evaluation and the future task are not substantially similar, and—most importantly—bias knows no bounds.
Almost all LORs are positive, even if most people will, by definition, end up somewhere closer to average.
And because most LORs are based on classroom performance, they’re useless. We have more objective measures of classroom performance. They’re called grades (and—blech—the MCAT), and even those aren’t very good at predicting outcomes. At least your boss, whether in the lab or at a paying job, actually knows if you can show up to work and get stuff done.
I’d almost completely forgotten about the LOR process in applying to medical school. I remember I had one from my lab PI/thesis advisor. And then…I honestly have no clue.