The NBME Shelf exams are enjoyable standardized tests that every first year looks forward to with almost unbearable glee. Each tests a single subject (“Anatomy”) and is (for the preclinical years) made up from the old or junior varsity questions from the USMLE Step 1, a test that makes the MCAT look like the GRE and the SAT look like building with Lincoln logs.
Some schools force their students to take a variety of Shelf exams (spending/wasting $30 a pop) to help measure how well their students have mastered the material (AKA how they are doing compared to their national counterparts). What is a bit amusing and misleading about the whole ordeal is that the national norms are probably a big crock.
Different schools use the “shelves” differently. Some use them as a just-for-fun intellectual exercise, others as extra-credit, and still others as a true final exam. Don’t get me wrong, it’s not a bad thing to get some USMLE Step 1 experience, but it’s highly dependent on the environment: if you take five shelf exams in a single week, you are clearly not going to be prepared or even particularly focused. If it’s your final exam, you are going to do your best to rock it.
So if the national average is computed from all of these groups together, then it’s going to have a huge unseen left tail: if people are taking the exam who don’t care how they perform, they’re going to be dragging the average down from where it would otherwise be. So while the test is technically normalized, it’s not the same normal as a regular standardized test: Unlike the MCAT, not every student has something riding on the exam. I personally knew people who filled out all C’s on an exam that was for extra-credit only.
While your school receives the group’s average and your grade relative to your test group (classmates), the theoretically more interesting numbers a student receives are the grade based on the national average and corresponding percentile. I’m curious as to how far off the scores really are. If all those people who weren’t making a good faith effort actually tried (as they do on the USMLE Steps 1, 2, 3), then I’d wager it’d be a different ball game. It’s essentially an unstandardized standardized test.
Further reading: How NBME Shelf Scores Work