Ben White

NBME Shelf Exam scores, with a grain of salt

04.21.09 // Medicine

The NBME Shelf exams are enjoyable standardized tests that every first year looks forward to with almost unbearable glee. Each tests a single subject (“Anatomy”) and is (for the preclinical years)  made up from the old or junior varsity questions from the USMLE Step 1, a test that makes the MCAT look like the GRE and the SAT look like building with Lincoln logs.

Some schools force their students to take a variety of Shelf exams (spending/wasting $30 a pop) to help measure how well their students have mastered the material (AKA how they are doing compared to their national counterparts). What is a bit amusing and misleading about the whole ordeal is that the national norms are probably a big crock.

Different schools use the “shelves” differently. Some use them as a just-for-fun intellectual exercise, others as extra-credit, and still others as a true final exam. Don’t get me wrong, it’s not a bad thing to get some USMLE Step 1 experience, but it’s highly dependent on the environment: if you take five shelf exams in a single week, you are clearly not going to be prepared or even particularly focused. If it’s your final exam, you are going to do your best to rock it.

So if the national average is computed from all of these groups together, then it’s going to have a huge unseen left tail: if people are taking the exam who don’t care how they perform, they’re going to be dragging the average down from where it would otherwise be. So while the test is technically normalized, it’s not the same normal as a regular standardized test: Unlike the MCAT, not every student has something riding on the exam. I personally knew people who filled out all C’s on an exam that was for extra-credit only.

While your school receives the group’s average and your grade relative to your test group (classmates), the theoretically more interesting numbers a student receives are the grade based on the national average and corresponding percentile. I’m curious as to how far off the scores really are. If all those people who weren’t making a good faith effort actually tried (as they do on the USMLE Steps 1, 2, 3), then I’d wager it’d be a different ball game. It’s essentially an unstandardized standardized test.

Further reading: How NBME Shelf Scores Work

Small Gestures

04.03.09 // Writing

My very short story “Small Gestures” appeared in Six Sentences today, a really interesting online publication that only publishes stories of exactly six sentences. 6S is full of really great quick reads—and it’s only because of wonderful sites publishing flash fiction like this that I’m reading any stories at all these days. So, thanks.

Lip service surveys

03.17.09 // Medicine, Miscellany

The world is full of surveys: surveys for free meals at TGI Friday’s, surveys for news polls, and at school, surveys for curricular reform:

“In order to improve this course for next year, we would appreciate it very much if you would take a few minutes and fill out this evaluation form.”

And the idea behind a survey is a good (nay, excellent) one: to gather feedback and ostensibly make changes and corrections based on it. The issue is in survey construction and follow-through. The usual survey has a variety of broadly worded statements with answer choices 1-5, 1 being “strongly agree” and 5 being “strongly disagree.” There will usually be a text-box for general comments at the end. You take this survey and your answers disappear into the depths of the internet never to be heard from again.

But from the beginning, the idea that you can sum up whether something works effectively or not based on a numerical average is a kludge. Furthermore, even if an average of 4 does approximate satisfaction, that doesn’t mean there aren’t better ways to do things. It’s an understandable shorthand, but anyone hoping that it’s sufficient to understand reception  is fooling themselves. If people’s responses show that weekly quizzes are on the whole useful, that doesn’t mean they wouldn’t prefer or think it better if they were biweekly, on Mondays, on Fridays, longer, shorter, or anything else. If people say dividing the year into 4 chunks is no good, it doesn’t mean 7 would be better. A number is all well and good, but at the end of the day, how someone feels isn’t the crux: it’s why they feel the way they do.

In order for a survey to be effective, it has to take time. Each question needs to have its own comment box. Then, someone needs to go through those comment boxes and compile all of the suggestions and problems. Take the suggestions and complaints, then formulate new courses of action. Then, before implementing them, offer them anew in a survey:  What do you think about these choices? Do they sound good? How good? Better then before? If not, why not? If that takes too much time to do, have students volunteer to do the grunt work. They’ll put in on their CVs, the administrators can continue doing whatever it is that administrators do, and everyone is happy. This is also how you make changes quickly. It doesn’t need to take years.

People tend to make incremental changes to the status quo. It’s hard to make drastic changes, especially if those changes reverse your hard work or go against your own inclinations; it’s even harder to come up with these changes yourself when necessary. This difficulty then breeds the stagnation that allows bad systems to continue even when their obsolescence is practically taken for granted. And yet, this is how you get curricular form with a stethoscope on the heartbeat of a student body.

Sometimes things don’t work—but if a goal is truly to teach a subject effectively, then no one can tell you better what does and does not work than students. This is how you don’t spin your wheels around a problem, making arbitrary changes. You need to ask for feedback, but more importantly, you need to be willing to listen to it.

Anatomy of an NBME Shelf Exam

03.04.09 // Medicine

The NBME offers comprehensive subject exams, ostensibly to torture students and devour whatever scraps of self-worth they have left. The typical question format:

A X-year-old [type of person] reports to the doctor/ER with a X-hour/day/week/month/year history of not-feeling-so-hot. Upon examination, distracting details. Irrelevant information. Single key relationship. More words to make the test take longer. Talkie-talkie. What is the likely cause of this you-should-have-learned-in-your-class-and-probably-did-but-maybe-you-didn’t-who-knows-there-are-so-many-questions-on-this-test-it’s-all-a-blur person’s problem?

Rinse and repeat for three hours. Then do it for it every class you take. The joys of biochemistry could never be fully appreciated until they were compiled in such a form.

In-ear, shmin-ear

02.22.09 // Medicine

Every few months, I see some news report on the revelation that listening to loud music can cause hearing loss.  Yesterday, Time online posted “iPod Safety: Preventing Hearing Loss in Teens,” the latest in this series of mediocre ear-science.  I’ll admit that in this case the facts are accurate, it reminds me of a lot of stories that are less so.

The point, which is true, is that listening to loud noises of any kind tends to be bad for the ears. The louder the noise, the less time it takes to cause damage. Prolonged exposure to loud noise leads to both tinnitus (ear-ringing, which sucks) and sensorineural hearing loss (which sucks and is irreversible). This is actually a serious problem, and it’s caused the kind of hearing loss in teens that used to be reserved for old-time factory workers. My beef is that there is one finding several years ago that has given rise to a huge misconception:

In-ear head phones generate more sound pressure at a given volume setting than over-the-ear counterparts [source].

This is logical, given that in-ear headphones are actually in your ear, literally closer to your tympanic membrane, which transmits the physical pressure waves to your middle ear. Because iPods are generally used with in-ear headphones, some news outlets and people came away with the idea that in-ear headphones are automagically more dangerous—which is hogwash.

This is silly because pressure and volume are essentially the same thing. When the in-ear headphones produce more “volume” at a given setting, the user actually hears the music louder. If I were to switch from over-the-ear to in-ear phones, chances are I’d adjust the volume accordingly. The fact that earbuds can pump out more decibels in and of itself is meaningless. Admittedly, there has been some work that has shown that some earbuds don’t cancel external sound all that well and therefore might lead to higher volumes when used, but this varies wildly between brands. All that means is that the government should subsidize some new Bose headphones for people who work in loud places, because good sound-canceling headphones are the only ones that eliminate this problem effectively. Being closer to the ear is not an inherent problem unless the volume isn’t adjusted accordingly. This is not an unnoticeable danger increase.

What studies have shown is that individuals have a preferred ambient listening volume. Some very angry teenagers who like thrash metal tend to like to blow a hole out of their eardrum, but the rest of us tend to fall somewhere on a decent curve. What matters is what relative volume we prefer, not what method we use to get there. When people taken off the street were tested for average listening volume, the data reflect this reality: the biggest problem is background noise. We tend to like our music somewhere around 60dB. If the ambient noise is 20dB, many people will turn up the volume to 80dB. If you correct for background noise, preferred volume is nearly constant. So when people listen to their iPod somewhere loud (on an airplane or the subway), they’re probably doing a lot more damage than if they’re at home. It really is that simple.

The idea that in-ear headphones are actually worse for you is based on this distortion. They’re not; your preferences and habitat may be.

Sympathy-inspiring?

02.09.09 // Miscellany

This is not: “You Try to Live on 500k in This Town,” a NY Times article that explains why Obama’s proposal will make life unlivable for the executives whose daily life is inexorably tied to wasteful extravagance. When you don’t follow your own advice and save some of your 1+ million dollar salary, it’s harder for me to take your mortgage seriously. Just the same, when your necessary annual expenses include paying your Tom Daschle-style chaffeur, it keeps America firmly on the revenge train. If extravagance is required by corporate culture, then the culture-change brought about by salary-caps or (better yet) progressive tax increases can be a good thing.

Doctorate Factory

01.19.09 // Miscellany

There’s no surprise people are realizing and writing about the low-value of advanced degrees. Schools have used grad students as educational fodder, producing far more than the demand for academics can keep up with. People hoping for their PhD to be something beyond their pursuit of an intellectual interest unfortunately will continue to be disappointed when tenure remains forever out of reach.

Twiction

01.19.09 // Writing

Is it the fate of the internet to endlessly combine two words to make memes? Will these neologisms always make normal, everyday people throw up in their mouths, just a little? I recently discovered twiction, a combination of “twitter” (for the microblogging service) and “fiction” (as in fiction). Twiction, AKA twitter-fiction or (even worse) tweetfic, is fiction in a maximum of 140 characters, which usually translates to somewhere between 1 and 3 sentences. It’s been around for a couple of years and seems to fall somewhere between kinda popular and vaguely interesting.  It comes in two forms:

First, the more common form, standalone microfiction: stories told in a sentence, usually feeling like something in between Ernest Hemingway’s famous “story in six words” experiment (For Sale: Baby shoes, never worn) and more traditional flash fiction. Given the inherent limitations of the Twitter service, writing Twiction is a sort of writer’s challenge—can you write a compelling story in a sentence? Can you fit a character, a conflict, and a resolution in a line? Or, in another view of what makes a story: can your character change from the beginning of such a story to the end, all in less than 25 words? The answer? Sometimes.

Any twitterer trying to write twiction comes across the problem that it is actually very difficult to produce a lot of super-short stories with distinct plots, characters, and resolutions. 140 characters isn’t a lot. What I’ve noticed in reading around is that one common crutch is to resort to melodramatic endings (“and he was never seen again” sort of stuff), like this piece from the now defunct twitterfiction:

The poison cut deep rivulets in her flesh, her blood caught fire and her heart slowed. Soon she would be dead.

Death is an easy resolution, but it doesn’t make for a very compelling or thoughtful piece. Who is she? Why is she being poisoned? It’s impossible to fit in everything, but all we have here is an ending—not a story.

Another common “mistake” is to paint a scene or a scenario, but not a real story. After all, it’s much easier to write a character sketch or a lyrical description of a forest in 140 characters than it is to write a complete story. An example from 3S Stories:

Karma can be a bitch. Ike wasn’t sure what he did to deserve being reincarnated as a function call in Vista, but it must have been horrible.

It’s funny, and I actually rather like this one, but there’s no motion. There’s nothing inherently displeasing about it (and new writers struggle with the same issue even with no artificial restrictions), but it does mean a sizable chunk of twiction is more like the creative writing of interesting sentences than true standalone stories. In some ways, twitter might be more suited for twitter poetry, where an image alone—well painted—can stand on its own.

A second type of tweetfic is the serial-story (two examples), a more conventional length piece written in 140-character installments (like the serialized novels in magazines that were common in the olden days). The biggest issue here is writing a story that moves along at a decent clip in small segments. Writing a bunch of entries back to back defeats the purpose entirely. Pacing becomes a problem because there is a tension between condensing action too much (boring) and not making any progress per entry (also boring). On top of that, it’s impossible to go back and alter tweets. The story must go on, no matter if you think you’ve made some serious mistakes in previous entries. A strong detailed outline probably couldn’t hurt. Still, word-choice and character-limits will never be as frustrating as in the first type—if something can’t fit, then move it to the next installment.

So what we have in twitter-fiction, I think, is a challenging medium that is almost at odds with the nature of Twitter (in the sense that Twitter users generally post frequently and have conversations with other Twitter users). At best, it’s a literary diversion in the blink of an eye with at least some degree of artistic merit. More often, just words. Though, after seeing the content of your average “tweet,” just words might be just fine.