The ABR’s virtual Core Exam worked

Last year as the pandemic spiraled out of control, the ABR resisted–as they have for years–calls for disseminated exams away from their centers in Chicago and Tuscon. The lack of a foreseeable endpoint and pressure from advocates was finally enough for the ABR to make the switch. And to their credit, when the ABR came around, they went all the way: all exams are to be virtual from this point forward.

And guess what? It worked.

Apparently, it worked really well.

And I, for one, am not surprised.

People I’ve spoken to were overall very pleased with the remote experience. Were there rare technical difficulties? Sure. But reports are that the ABR was generally responsive and helpful in aiding candidates when issues cropped up, and multiple residents I spoke to gave ABR customer service high marks.

So while perhaps they shouldn’t have needed the worst pandemic in a century to make these changes, credit where it’s due: the ABR successfully pulled off the transition to at-home testing.

The ABR’s testing centers, though physically inconvenient, were always pretty nice compared with most commercial centers. But the ability to take the exam from a location of your choosing with no travel required and your choice of preferred snacks, clothing, thermostat settings, and bathroom is pretty nice. Having the exam over three days also probably helped with test-fatigue.

Future Fix Requests

There were a few complaints the ABR should address in future administrations:

  • Answer choice strikethrough. This was a common request, and it’s a common feature including one available on the USMLE exams that residents are all used to.
  • Cine clip optimization. This has been a longstanding complaint, but in this case, at least sometimes clips are presented in a separate window from the question and answer choices. They should be embedded the same way as normal images with easily controllable playback speed and the ability to manually scroll.
  • Remove the 30 question auto-lock. The need to lock previously seen questions makes perfect sense at the end of a 60-question block and whenever a candidate takes an optional break. But I’m not sure I buy any justification for auto-locking mid-section. This is a true functional change from prior exam administrations that has a negative impact on those who would like to review all related questions before moving on. It’s also difficult to know how much time to allot to question review when you break up 120 questions into 4 blocks instead of two, making time management more difficult.
  • Announce the section order. This was a big complaint I heard and one I agree completely with. For years the ABR has avoided publicizing the section order (e.g. Breast, then Cardiac, then GI) despite keeping it consistent across testing administrations. While people obviously aren’t supposed to discuss the exam, in the real world this has meant that candidates taking the exam later always know what sections are coming on which days, allowing them to cram most effectively. Unless every candidate has a randomized order, keeping this information semi-hidden in this setting just isn’t appropriate and should be a no-brainer to fix. Knowing you’re going to have ridiculous radioisotope safety microdetails on a specific day means you can prepare for that much more effectively and seriously jeopardizes the exam integrity. Again, this is not a new issue.

The Core Exam is still the Core Exam

Ultimately, the biggest complaint–no surprise–wasn’t the software but the test itself. It’s not as though the content magically became more on-point just because you got to wear pajamas.

If I were to limit myself to one content suggestion, it’s this:

I feel very strongly that the ABR’s reduction of physics and radiation safety to nonsense microdetails does our specialty a disservice. Residents constantly complain that the test material seems random and is not found in most review materials. This means either the Core Exam treats this material poorly or that the residents are studying the wrong information.

The problem is that this material is important. The ABR needs to make it clear what information candidates should know and release it as a packet of specific information like non-interpretive skills (NIS). In its current form, the combination of physics/radiation biology/radiation safety/nuclear medicine/RISE is a limitless almost black-box from which residents have no idea what to focus on or what material is high-yield. The end result is that most radiologists are taught low-yield or confusing information from physicists and end up with a poor understanding of these concepts. Candidates simply don’t really know what they should know and so don’t really know anything.

MR safety and contrast safety are included in the NIS study guide already (in addition to mission-critical information like the ACGME core competencies and how to create a “Culture of Safety”). The vast majority of the information I just described is also “non-interpretive” and needs to be included.

9 Comments

Tony Filly 02.28.21 Reply

Couldn’t agree more about relevant info. I took the test many years ago but remember a question about where bile salts were absorbed. I remembered that it was the terminal ileum but they had as choices 1. Terminal ileum by active transport 2. Terminal ileum by passive transport. I remember thinking, you have 200 or so questions to determine if someone should be a radiologist and this is what you’re asking? Ridiculous!

Nick 03.09.21 Reply

Also couldn’t agree more! I’m going to redact my favorite line: “The end result [of the Core exam] is that most [“trainees”] are [forced to choke down] low-yield or confusing information from [board review books] and end up with a poor understanding of [Radiology]. Candidates simply don’t really know what they should know and so don’t really know anything.” – Pretty much summarizes the whole exam, haha.

Thomas A Kavic, M.D.; FACR 02.28.21 Reply

Concerning virtual ABR exams: I am an hearing impaired physician (Radiologist) that uses lip reading and closed caption. This means that I would not be able to understand a slide or subject with background audio narrative. I would be able to understand the subject if provided with captioning or English subtitles. This is also true for Residency and Fellowship programs.
Americans with Disability Act (1973) is a federal directive encouraging
use of whatever means (technology, computors etc) to help individuals with disabilities.
Thank you for this forum.

Ben 02.28.21 Reply

There is no audio with the current ABR Core and Certifying Exams (or the online MOC called OLA). It’s just an image-based multiple choice exam and so is considerably more friendly than the old oral boards from that perspective.

JT 03.07.21 Reply

Thank you for writing this– agree completely. As someone that took this first virtual exam, I can say that all your points are spot on. The interface worked quite well, minus those missing features.

However, the true issue is the terrible, irrelevant, and esoteric questions that were on the exam. As Tony (prior comment) also noted, the questions continue to be ridiculous, and the answer choices even more so.

I know it may be wishful thinking, but I really think the ABR needs to be more transparent about the raw percent/score of test takers. If we all see that, for example, the median score is 30%, we can all agree that the test needs to be re-tooled or rewritten.

If the test was a minimum competency test that needs no preparation beyond showing up to residency, the ABR completely failed. To let such an irresponsible and out of touch governing body fail almost 20% of graduating residents in prior years is ridiculous and unacceptable.

Ben 03.07.21 Reply

I deeply agree with the need for transparency, both regarding the exams themselves and the ABR’s finances. This is an organization that WILL NOT release even basic passage statistics for the Certifying Exam and is also obfuscating the same information for OLA.

JT 03.08.21

Wow, thanks for those articles and your writing on it. I like your point that “This isn’t about you”. Which leads me to think…for the CORE…do we pray that we are in the 85-90% that pass and then shrug it off?

What about those that fail or condition? Seems like no real recourse for that group of people, from studying better materials, or having any better idea of what questions to better prepare for. Scary thoughts.

Leave a Reply to Ben Cancel reply