I appreciate that not everyone is on Twitter—and frankly that’s probably for the best since it’s largely a toxic dumpster fire—but I did want to share this tweet/thread about a real situation unfolding at the American College of Radiology, the largest and most important radiology organization in the US. Among other things, the ACR sets standards for imaging center accreditation, creates the appropriateness criteria and incidental findings white papers we all love, influences reimbursement, and performs congressional lobbying on behalf of radiologists.

BREAKING: Did newly elected ACR board member violate election COI policy, failing to disclose that he works for @Rad_Partners? Here's a snip from the election manual indicating that he works for a medical school. Just another ACR academic insider, right? A thread…you decide. pic.twitter.com/6OANwAV8Yo

— Exit ACR (paRADee) (@ExitACR) May 27, 2023

Insofar as anything involving organized radiology is newsworthy, this is news.

Who knows whether transparency here would have affected the election outcome. But we do know that this disclosure issue was debated fiercely a couple years ago, and this situation is exactly what people had in mind. I don’t know this radiologist, but it’s not just a paper relationship: he apparently went straight from the ACR annual meeting to be on stage at the RP Leadership Summit.

That said, this isn’t really a “private equity is just the worst” issue, because he apparently made the disclosures he had to make in order to follow the rules. It’s more of a problem/oversight with the ACR’s internal process compliance. However, it does reinforce how important these PE companies feel it is to infiltrate professional organizations (and especially to be high in the ACR leadership). It helps control the narrative and steer policy.

Every big democratic institution at least voices that it cares a lot about transparency and conflicts of interest. On that front, this is a big miss.

I posted two tweets the other day that deserve some further discussion:

RadPartners is now behind on paying its “unique” “profit sharing” proceeds to its “partners.” pic.twitter.com/KGCiTL87MK

— Ben White, MD (@benwhitemd) May 26, 2023

I’ve since by told by another source at RP that this is actually the third quarter in a row that profit-sharing has been delayed.

These “unique” payments are the ubiquitous practice of a group putting money in your 401k. “Profit sharing” is just the actual term used by the IRS. Practically, these contributions are just a portion of your compensation that is tax-deferred. For reference, my group contributes to my 401k on a monthly basis.

In other assuredly unrelated news, RP’s SVP of finance is resigning. pic.twitter.com/MbW3lOtDap

— Ben White, MD (@benwhitemd) May 26, 2023

Now, I am obviously not privy to RP’s internal workings, but I suspect these delays are twofold.

One, RP is suffering from cashflow/liquidity issues. That’s what they essentially say in the email snippet I’ve shared above.

Two, businesses have an incentive to delay payments/hold onto cash thanks to the time value of money: having money now instead of later is itself worth money–because you can invest it. By holding onto their radiologists’ money for longer, they can keep these funds earning interest, which helps their bottom line. This is a big reason why insurance companies delay care through denials and prior auths even for the things they know they will eventually cover. It’s also why Starbucks is basically a bank that sells coffee: they have over $1 billion in giftcards. Starbucks gets to invest all of that prepaid money before they incur the cost of actually giving you that delicious brown sugar oat milk shaken espresso.

The easiest way to make money is to have your money work for you.

RP needs (or believes they need) to do this now. Also note, these delays also started around the time RP laid off some of its nonclinical workforce.

This feels like part of a story.

When a “Partner” isn’t a Partner

The other word we need to address is partner.

It should almost go without saying that I can’t vouch for how every contract looks, but here’s the language for one of RadPartner’s “partnership” employment agreements:

Partnership Designation:
During the Term, the relationship between Physician and Practice shall be that of employee and employer and shall not modify or affect the physician/patient privilege or relationship. Unless otherwise directed in writing by the Chief Executive Officer of Practice, the Physician may refer to himself/herself as a “Partner”, allow others to refer to him/her as a “Partner” and refer to such other employees of Practice who have executed this Form of Employment Agreement with Practice as his/her “Partner”, provided, however, that the designation of “Partner” shall be in name only and the Physician shall not be an owner/partner of Practice under the law. Further, Physician shall not have any power or authority to bind Practice in any way, to pledge its credit or to render it financially liable for any purpose unless formally appointed an officer of Practice with such authority pursuant to Practice’s governing procedures and law or authorized in writing by the Chief Executive Officer of Practice.

You are a “partner” in name only.

This is the inescapable reality of choosing a “partnership” track job with an RP group. You are putting in the work in order to take on the responsibility of running the practice without actually owning the practice. It’s just verbal sleight of hand.

Evaluating “Partnership” Opportunities

Sometimes people reach out to me with employment offers and other quandaries for my opinion. (NB: Please note that I am a Person on the Internet and not an expert on most things including contract review).

A reader recently reached out asking for my thoughts on their partnership-track teleradiology-only employment offer with an RP-owned group. The offer included a decent workup salary with high productivity demands that I doubted most people fresh in practice would be comfortable hitting. As in, the W2 sounded very competitive on paper but was actually still pretty extractive taking into account the desired production. That’s not really news. All practices function this way at least to some extent. Partners make money on their employees.

The job also promised “full partnership” in two years with “equal profit sharing.” And this is the crux:

It’s true that whether you work at an independent practice or a private equity-owned group, the “profits” can always be zero. But the profits at an independent group are the profits (revenues minus costs). The profits at an RP group are something else. As United Healthcare argued in its recent lawsuit:

In exchange for these services, Radiology Partners siphons off large amounts of revenue from the medical groups. Indeed, on information and belief, the affiliated medical groups no longer retain any profits resulting from the radiology services that they provide, and all profits are instead kept by Radiology Partners.

An equal share of zero is still zero.

The stock offered to new RP employees is also almost certainly worthless. Don’t view the chance to catch a falling knife as a growth opportunity.

*  *  *

I promise I don’t begrudge anybody their career choices.

And you absolutely don’t need to consider what Random Guy with a Website says.

But if I were considering a job offer at an RP group, I would consider only the workup/employee salary and not make a decision based on the possibility of future increased income as a “partner.” I keep annoyingly using air quotes here for the same reason RP does: There are no partners. There is no partnership.

In each group, there are people who make less money and people who make more money, but they are all employees, and none of them are really actually entitled to much of anything. I won’t pretend to tell you what fraction of groups are happy with their sales and what fraction of groups are making good money and what, if anything, reliably differentiates the successful groups from the struggling ones. That kind of granularity is something that only RP knows, if anyone knows at all. But this much is undeniable: the partners are just employees who are usually paid more. 

*  *  *

If trainees flock instead to independent groups, then radiology private practice will stabilize and the independent model will survive. If they instead take one of the infinite positions offered by RP and their ilk, then they are casting votes for the corporate practice of medicine. I don’t have a crystal ball, but I remain concerned that the downstream consequences of that often understandable individual choice made en masse will be the tacit endorsement of the funding model and the acceleration of falling reimbursement and radiologist replacement.

If you want to work for RP, another PE company like Envision or Lucid, or ride the current wave of teleradiology positions that pay relatively well, then you can do that. You don’t owe the field of radiology more than you owe yourself or your family. But it would probably be wise to assume that it is a temporary play and that some component of your job, either the money itself or the quantity of work asked of you, will change in the coming years. Radiology is in the middle of a nationwide shortage that will morph into a big unpredictable shift. Lots of radiologists change jobs, so you certainly won’t be alone.

Some of these are undeniably good employee positions right now. But don’t think for a second that a private equity partnership means you own the business. Because you don’t.

Here is the video for the American Board of Radiology’s town hall discussion about the new oral boards, which are coming to a computer near you in 2028:

Some highlights:

1. The ABR would like you to know that discussions about revamping the Certifying Exam started internally and “did not arise from an assumption that there was something wrong with the Certifying Exam.” (There is.) They did acknowledge that “nuance is lacking in the current exam.”
2. Any interesting formats such as simulation-based assessments weren’t possible due to “practical constraints.”
3. With regard to data about the effectiveness of either the old oral boards format or the current exams, Executive Director Dr. Wagner said: “We have no data that it DOES work.” He went on to say that proving the ABR exams have an impact “would be a difficult experiment to run.”
4. The initial timing will be during the second half of fellowships (first offered in 2028), but while the format is set, the timing would “not be hard to change” in the future if needed.
5. They will send out a “mock session” in the next few weeks apparently. I hope they also intend on releasing sample cases with sample scoring rubrics as well.
6. When asked about exam preparation/support from fellowships, Dr. Wagner said: “The ABR doesn’t really have a position on that, as to how a candidate should prepare.”
7. In the following discussion, the implication was that likely most preparation would take place during the fourth year of residency. It was not specified as to why it should be deep into fellowship (the phrase “the least bad choice” was used.) When asked why not just offer the Core and Certifying exams simultaneously or back to back, the ABR’s answer was that they were not interested in changing the need to pass the Core exam first in order to take the Certifying Exam, and the Core Exam takes time to grade. (But, yes, we could, again, in principle, just have written and oral exams like we used to.)
8. There will be no “hardcore” physics or non-interpretive skills.
9. The plan is for 7 25-min sessions with 10 min breaks between each. There will be an extra session (“recovery block”) at the end in order to deal with internet failures during the exam day.
10. The ABR currently spends more than $200 per item to develop its multiple-choice question collection. This exam won’t cost more, because no one will travel, the number of items is far smaller, and the judges are volunteering. In reality, this exam will be much cheaper. But also: no, they won’t be dropping fees.

Want more? Here is my initial discussion of the coming change.

All of this recent talk of the nationwide radiology shortage and I’ve been remiss in not also mentioning: like every other practice in the country, we’re hiring!

Our 100%-independent physician-owned radiology practice of which I am a partner/shareholder is hiring in almost every subspecialty including breast, body, neuro, neuro IR, MSK, ER, and general. Everything except VIR at the moment. While all of our partners are in the Dallas/Fort Worth metroplex, we have some openings for remote employees and independent contractors.

We’re privademic: we have part of the practice that works with residents at a university hospital and we have part of the practice that does not. I enjoy a nice mix.

I’ve written before about why I believe some job healthcare models are problematic, and why not all attending jobs are created equal. I’ve also written before about how to approach getting your first job out of training. My perspective and biases about radiology practice are on full display.

Our group was/is my first job out of training. It was the job I wanted–so much so that the day I got the interview invitation email (after already having job offers waiting for a response), I did an actual Street Fighter dragon punch of victory and told the others they were going to have to wait past their response deadlines. I was drawn by two things:

1. A well-established successful privademic model combining teaching-focused academics with the no-BS of private practice (with positions leaning more in different directions based on interest including pure no-teaching PP), which gave me the chance to teach and work with trainees in a more flexible environment than a traditional big academic bureaucracy. I’m currently the associate program director for our residency, and our residents are awesome.
2. A sustainable job model combining high-quality (as opposed to only high-volume) radiology practice with reasonable daily expectations and the goal of a standard 4-day workweek for those who want it (tomorrow is my day off!). I wanted the time and mental space to also be a partner at home and have the flexibility to do the other things that are important to me (like this). My colleagues are good at what we do, and I still learn from them every day.

So if you’re in the market, come work with me and check out our great team in Dallas. If you’re interested, send me your CV at ben.white@americanrad.com and I’ll make sure it gets where it needs to go.

Years ago, when I was a resident and the ABR Core Exam was still novel, the ABR offered a lengthy “ABR CORE Examination Study Guide” PDF, which–in addition to over a hundred pages of endless bullet points listing every conceivable topic in radiology–included 57 official practice questions at the end (with an answer key).

They took that down years ago. But, thanks to the miracle that is The Internet Archive, you can still enjoy a copy of that PDF here. Well worth doing in the final weeks leading up to the June exam. Enjoy!

(Other nonofficial question resources are discussed here.)

((Lest there be any confusion, I have no working relationship with the ABR. In fact, I’m probably a persona non grata. These are just questions that they once posted on a public-facing website.))

It should go without saying, but I’ll say it anyway: these are my opinions, formed from the combination of my biases, my experience as a radiologist since beginning residency 10 years ago, and my many conversations with radiologists across the country. You don’t have to agree with me.

Not Enough and No Help Coming

For today’s needs and today’s technology, we have simply produced far too few radiologists. There is a sizable and worsening radiologist shortage, and there is no end in sight on the basis of increased radiologist supply. There are currently 1788 separate job postings on the ACR job board. Imaging volumes are increasing between 3 to 5% per year (increasingly including low-yield complex exams), but no one is seriously attempting to address utilization at any level. Nationwide, the supply of radiologists is basically flat. The anticipated wave of retirements from vested PE buyouts is just beginning.

It’s true that reimbursement has been steadily falling and that radiologists have been forced to read more in order to maintain their income, but it seems that even there, any excess workforce tolerance for higher workloads has been saturated. If anything, the tighter job market after the 2008 crash and the desire to maintain income against that downward reimbursement pressure masked the problem. Burnout is now so rampant and commonly discussed that it’s mostly just meme bait on social media.

Turnaround times are worsening. For example, a memo to the medical staff of Ascension Saint Mary in Chicago was making the rounds back in March:

Our current Radiologist group, RadPartners, has been experiencing challenges with physician coverage for some time. As a result of this radiologist shortage, outpatient exams are taking anywhere from one to six weeks to be read.

Six weeks?! I don’t think most practices have much more to give before flaming out, and many groups are shedding contracts in an attempt to right-size their workloads. (This was in fact a contract in its final months that RP had already terminated. [Also note, given comments I’ve received: this quotation was included for flavor, not because RP or this one group is the basis for this article’s argument.])

There’s a generational shift contributing as well. People’s understandable desire to have a better lifestyle also means that in some cases we require more young radiologists to cover the jobs of those leaving the workforce. It wasn’t that long ago that most radiologists covered their own nights on a rotation (or paid for tele coverage) and no one had dedicated night teams. Then it was normal to see a 7-on/7-off schedule. Now 7/14 and even 7/21 schedules are increasingly common. When you need three people to do the job of one person, that isn’t going to help with the workforce shortage.

Frankly, I think there’s no chance of radiologists meeting demand without a paradigm shift of some kind, either the long-awaited mass efficiency gains from meaningfully helpful AI products (maybe good?) or the significant expansion of the role of midlevel providers in image interpretation (highly suboptimal and currently not permitted). It’s hard to imagine a world where volumes actually go down in this country, but that would also work.

The training pipeline is essentially fixed in size and long in duration. Even opening up more training spots would take years to help. If the shortage gets worse, then turnaround times will continue to lengthen (and patients suffer) and hospitals will struggle to get coverage (and patients suffer). That will be the time when the government/Medicare/national organizations start advocating against the currently protected role that radiologists hold for imaging interpretation.

Years from now, there may be a world where there are too many radiologists, but that world is one where radiologists are performing a substantially different role than they are today, and I’m not sure there’s any way to meaningfully prepare for that possible future while also solving the problem of getting today’s work done.

“It’s Cyclical”

A lot of radiologists, particularly people who have been practicing for a while, think things are just going to go back to normal. Radiology job markets are cyclical, the argument goes, there are hot markets and cold markets. Whenever the pendulum swings too far in one direction, it will swing back. This probably seems historically true. Various factors have contributed to historical “oversupply” including the 2008 recession, which cooled the job market as older radiologists put off their retirements, or the invention/implementation of PACS, which helped radiologists read more studies efficiently. But to assume that each swing of the pendulum is a predictable back-and-forth cycle correction to small shifts and underlying market forces I think misses a greater point about the difference in supply/demand for a professional service like radiology and some typical commercial widget.

I personally have yet to be convinced that there are any secular market trends pointing in that direction. The invention of PACS was a singular event that changed the field of radiology. It will take something equally momentous in terms of efficiency or work distribution to deal with the current and worsening workforce shortage. “AI” will likely be that paradigm shift, but it remains to be seen if those tools will evolve fast enough to help us with a shortage that’s already here. They simply aren’t ready yet to move the needle in anything close to the level we need. There will only be “too many” radiologists after a change whose timing may be impossible to predict and/or impossible to prevent.

The world we should want to see is one where radiologists use machine learning tools to produce higher quality more efficiently. The problem in the short term is that it’s very hard to argue only radiologists can interpret imaging when there aren’t enough of us to interpret all the imaging.

It’s also extremely challenging to combat rampant overutilization in our medicolegal and fee-for-service reimbursement climate. Neither the clinicians nor the patients want fewer scans, and radiologists aren’t actually paid to be gatekeepers.

It’s not hard to see that many non-radiologists (especially administrators) would feel that any read would be better than no read. Especially when some practices–overstressed, and yes, perhaps focused on profit–may be willing to produce substandard work, the delta between the trained radiologist and the less trained non-radiologist physician or NP/PA is going to seem smaller and smaller. Perhaps first it would be preliminary reads on DEXA, plain films, or ultrasounds. Maybe some places would start relying on machine-generated or midlevel prelims overnight in overtaxed EDs that can’t get coverage. It’s a slippery slope, they say, and they’re right. There are a lot of ways the specifics could play out, and we’ve seen it happen in other fields.

Currently, the shortage is helping combat falling reimbursement. Some rads see it and are pleased: we’re on the right side of supply and demand for income and job security.

But it’s also hard to protect turf that you can’t handle.

Hard to Make Predictions, Especially About the Future

So, yes, there will likely come a day in the future when there are too many radiologists. But in the meantime, there are far too few.

Some hospitals have been hard-pressed to find coverage. Imaging services, typically an important revenue source, are getting more expensive. If radiology moves from a profit center to a cost center, that impacts patient care and jeopardizes the important role that radiologists play in the healthcare system.

Early AI tools are no panacea, and automation bias alone–not to mention the costs and liability concerns–tells us that we don’t know yet how to best utilize their growing capabilities. So far, very few are making us faster.

If we can’t address the current workforce shortage over the next few years by adapting to new technologies and/or decreasing overutilization while working together to ensure that the quality of radiology services remains high, then we’re going to have a very hard time preventing the expansion of non-radiologists interpreting imaging.

From an opinion piece in AJR recently titled “The Case for Presence as a Source of Professional, Educational, and Social Fulfillment“:

Although the long-term impact on social wellbeing of working virtually compared to working in-person is not yet fully understood, physical presence is likely to be conducive to establishing connectedness.

This is a valid knock on teleradiology. Working alone by yourself just isn’t the same as having real colleagues, hanging out with peers, and enjoying spontaneous interactions throughout your work day. I absolutely agree. Sure, having some remote work is great. It’s flexible and efficient. But I took the specific job in private practice I did precisely because I wanted to have peers, teach residents, and otherwise have a varied work-life experience (yes, including working from home sometimes).

All things being equal, we really benefit from spending time with real, live humans. The data show that having a “best friend at work” is a powerful force.

But.

The problem with this kind of article is that it’s a reflection of the academic bubble. When we draw a comparison between a typical academic center or hospital-based practice and teleradiology, we miss the fact that many, many radiology jobs are not team-based daily work. Yes, there are still jobs where you’ll drive to the hospital and work with your peers. There are even some with communal reading rooms and the chance to socialize. But the reality of modern private practice is that a lot of radiologists drive around town to sit by themselves in small reading rooms in the back of outpatient imaging centers located in strip malls.

The false dichotomy (tele = inescapable loneliness, non-tele = Shangri-La) misses the fact that the so-called downside of a remote/at-home practice applies equally well to the reality of private practice in many locales. If you’re commuting just to cover contrast from a dark closet somewhere, you’re not really benefiting from the perks of presence.

The key to meaningful comparison always rests on a foundation of fairness (apples to apples, not apples to oranges). Considering the enrichment you might get from physical presence in an academic medical center radiology position is one important consideration, but it’s ultimately a poor reference when comparing the jobs that many residents in many markets will decide between.

The more salient distinction between a teleradiology position and a local private practice job is the difference between sitting in your pajamas at home versus providing contrast coverage and the occasional procedure at an outpatient imaging center. Realistically, for better or worse, plenty of recent graduates don’t like doing procedures and don’t socialize with the techs. With that reality, it’s not hard to see why even trainees are interested in jumping straight into teleradiology. They’ve never experienced the relative isolation of being the only radiologist at a facility, let alone the isolation of being entirely remote.

Radiologists can argue themselves red in the face about how important it is to be visible and available to clinicians in order to demonstrate our value and the importance of face-to-face communication. That’s all well and good. But it is also outside the locus of control for an individual radiologist pursuing an individual job. The majority of imaging volume is outpatient imaging, and the majority of communication we do is over the phone. The volume is there, and the positions exist. How can we blame radiologists for taking the jobs that are available? The market consolidation from the growth of massive academic medical centers and nationwide private equity conglomerates coupled with a worsening radiologist shortage has fundamentally changed the workforce.

The reality when assessing an individual position is that there are good and bad types of every job. There are assuredly some teleradiology positions that have good support with built-in ways to reach out to colleagues for second opinions and reasonable productivity demands. And there are jobs that are local and in-person but spread out enough with bad IT infrastructure that you may feel even more alone.

You have to know what it is you want, and we have to evaluate each job on its own merits. You have to ask questions.

Yes, we’d probably all be happier feeling like we were part of something. I like my remote work days, but I have no interest in an exclusively teleradiology position. I agree with the thrust of the paper: presence matters. Unfortunately, most conventional jobs simply don’t offer that much presence and many that do are so busy that you can’t enjoy it.

So, ultimately, the distinction isn’t really just teleradiology versus in-person. It’s community vs isolation.

From “Automation Bias in Mammography: The Impact of Artificial Intelligence BI-RADS Suggestions on Reader Performance“:

Inexperienced radiologists were significantly more likely to follow the suggestions of the purported AI when it incorrectly suggested a higher BI-RADS category than the actual ground truth compared with both moderately (mean degree of bias, 4.0 ± 1.8 vs 2.4 ± 1.5; P = .044; r = 0.46) and very (mean degree of bias, 4.0 ± 1.8 vs 1.2 ± 0.8; P = .009; r = 0.65) experienced readers.

Small but pretty clever study.

“Automation bias” is an insidious combination of anchoring and the authority fallacy, and it demonstrated a huge (though experience-mediated) effect here. We are still very much in the early days here (most radiologists are still very skeptical about the current powers of “AI” tools).

As machine learning tools grow in power and complexity, they will undoubtedly become a larger part of the radiology workflow. But counter to enabling inexperienced practitioners to function without oversight (e.g. a non-trained non-radiologist working independently with AI bypassing radiologists), we will instead need more robust skills: Raising the floor to miss fewer fractures and PEs is the easy part; it takes knowledge and experience to countermand the computer you increasingly rely on.

This isn’t going to be easy.

Earlier this year at the Texas Radiological Society annual meeting, I attended an ABR update given by current ABR president, Bob Barr, where he announced the rapid progress of the ABR’s plan to revitalize the Certifying Exam to address widespread discontent. I wrote about it here. The plan was to announce the change no later than June but potentially as early as April. The takeaway summary at the end of that short post?

But the ABR did reiterate that their hope for the Certifying Exam is a better demonstration of the skills needed for general practice.

But no, I don’t think they’ll be bringing back the oral boards.

The news broke on April 13.

I was wrong.

The Status Quo

The current ABR Initial Certification process–comprised of a multiple-choice Core Exam during residency followed by another multiple-choice Certifying Exam at least 12 months after residency–is flawed in many ways.

The ABR suggests that the initial certification pathway is meant to convey minimal general competence for independent practice, but it is unclear how a second multiple-choice exam after fellowship–especially with choose-your-own-adventure content that is often more subspecialized–adds any value to this calculation after the Core Exam.

Short answer: it doesn’t.

The final paragraph of that post summarizes the situation:

Despite how challenging the Certifying Exam may feel, it’s an exam designed to “psychometrically” validate whether you suffered a traumatic brain injury in the two years since you passed the Core exam.

The bottom line is that everyone, the ABR included, knows the Certifying Exam is truly useless.

A few years ago, I wrote a summary of a Twitter chat with ABR Executive Director Brent Wagner. I called it “The ABR Defines the Intent of the Core Exam,” and I think it’s worth reading for our discussion here. One of his tweets:

#RGchat T1: the ABR certification exam is intended to test knowledge as it relates to competence, and critical thinking as it relates to image interpretation. Other elements of competence (procedures, professionalism, etc) are better tested (assessed) by the residency faculty.

— Brent Wagner (@brentwagner99) April 14, 2020

Per the ABR, its mission is “to certify that our diplomates demonstrate the requisite knowledge, skill, and understanding of their disciplines to the benefit of patients.”

The current certification process tests for knowledge. It does not test for skill.

This is its core flaw.

Radiology in the Post-Orals Era

Many old-guard radiologists bemoan a perception of decreased real-world readiness and confidence of recent graduates in the era of the Core Exam. It is felt–with some merit–that the loss of the high-stakes hot-seat oral boards format (and the style and volume of preparation it required) has reduced radiology trainees’ ability to speak cogently about radiology. A lot of radiologists have suggested that a multiple-choice exam simply can’t test for the soft skills that the oral boards could (which is undeniable).

From that same Twitter conversation, Dr. Wagner summarized the exam’s goal:

#RGchat T1: Much of the exam seeks to assess a candidate's ability to choose the most likely diagnosis based on a set of images. Similar to what I expect of myself and my colleagues in daily work.

— Brent Wagner (@brentwagner99) April 14, 2020

This comically obvious disconnect–between real-life radiology and selecting the best answer from a list of predefined possibilities–is a great source of consternation for many radiology stakeholders.

The reality is that, while oral examinations are also a contrived format, the radiology educators who have worked during both eras aren’t wrong. I began my diagnostic radiology residency in 2013 as the Core Exam was about to be administered for the first time. I remember the way seniors could take cases when I was an R1, and I know how subsequent classes took cases. They weren’t the same (we were clearly worse). If we believe casemanship is an important skill, then it’s a skill that has atrophied. You play like you practice.

So: the perception is that the certification pathway is flawed and that a second MCQ exam in particular isn’t adding value or testing meaningfully different skills. Its timing after residency is awkward and confusing and makes it harder to test general competence after a period of typically intense fellowship subspecialization. And lastly, there’s the usual intergenerational conflict with its grumbling about the good old days and bemoaning the foibles of millennials.

While the ABR doesn’t want to admit that the post-orals era under the leadership of Dr. Valerie Jackson was a failure, they are willing to admit it wasn’t exactly a success insofar as their willingness to change one half of the new process.

If you’d asked me to provide a differential for the ABR’s plan to revamp the Certifying Exam, a return to the oral boards would not have been high on the list.

I thought, perhaps foolishly, that the ABR was going to put in the hard work to offer real-world simulation as a key part of the initial certification process (something I’ve long advocated for).

If we want to see if radiology trainees can practice radiology, perhaps we should just have them practice radiology and see how they do. If it’s not a PACS, then it’s probably not a good test.

The Details of Back to Future (so far)

Instead, the future is the past: In 2028, DR residents training from 2023-2027 will trade the current Certifying Exam for the new “DR Oral Exam.” The Core Exam will, despite its flaws, remain unchanged. The new oral boards will take place after the completion of radiology training just like the current Certifying Exam. Instead of an awkward slog parading through examiners’ hotel rooms in Louisville, Kentucky (i.e. the old oral boards), it will take place in the safe space of the ABR’s remote videoconferencing platform. Unlike the current MCQ offerings, you probably shouldn’t wear pajamas.

But this is a “new” old exam for a new world.

One pitfall of the old oral exam was bias and subjectivity:

To mitigate subjectivity and potential bias inherent in an oral exam, examiners will use a standard set of cases, and detailed rubrics will be used to score each candidate. This is an improvement over the prior oral exam model and is facilitated by current technology, including software developed specifically for this purpose by the ABR and currently used for oral exams in the three other disciplines (interventional radiology, radiation oncology, and medical physics). As in the past, examiner panels will meet after each session to discuss candidate results to ensure fairness and consistency. A conditioned exam result will be possible. The panels will be balanced for geography, gender, and new vs experienced examiners.

Instead of a panel, candidates will meet with one examiner virtually at a time (technically much easier in a Zoom-style setting).

We anticipate there will be seven individual exam sessions. The clinical categories are breast, chest (includes cardiac and thoracic), abdominal (includes gastrointestinal, genitourinary, and ultrasound), musculoskeletal, nuclear radiology, pediatrics, and neuroradiology.

The content will include critical findings as well as common and important diagnoses routinely encountered in general practice.

It will take one day and be offered “winter/spring” and “fall” each year.

A candidate’s first oral exam opportunity will be the calendar year following the completion of residency training. Candidates are eligible to take the exam through the end of their board eligibility, which extends six full calendar years after the completion of residency training. We anticipate having two administrations of the new DR Oral Exam each year.

Back to mitigating scoring conflicts:

For each session and category, the examiner will use a standard set of cases for all candidates. Detailed rubrics will be used to score each candidate. This is an improvement over the prior oral exam model and is facilitated by current technology, including software developed specifically for this purpose by the ABR.

As in the past, examiner panels will meet after each session to discuss candidate results to ensure fairness and consistency. The panels will be balanced for geography, gender, and new vs experienced examiners.

How does a pow-wow ensure fairness and consistency instead of groupthink? Good question.

Note, the costs to traineess will be the same (trainees still pay much more than diplomates). We’re just swapping one exam you take at home that everyone passes for another exam taken at home that we currently know very little about.

One thing that’s also not clear is how the ABR will handle the manpower needs. While developing the test will be straightforward, getting enough volunteer evaluators together will be no small feat. If they intend to pay attending radiologists for their time, then we are planning for a deeply inefficient high perpetual cost for the foreseeable future. Radiology is already among the most expensive medical specialty certification processes. I for one would like to see those costs go down, especially for trainees.

Timing

There’s also the issue of timing: radiology is certainly not alone in pushing final board certification until after residency. Orthopedic surgery for example includes an oral exam Part II where examiners grill you on a log of your cases and complications two years after finishing training.

But, in our field, there is absolutely no good reason to delay this assessment. What magical general diagnostic radiology content and experience do most radiologists have a year after residency that they don’t have at the end of residency? None.

Pushing a general competence assessment until after finishing subspecialty training only makes sense if the ABR intends to argue that subspecialization is making young radiologists worse and that somehow forcing people to spend their fellowship maintaining general skills and prepping for orals will prevent that atrophy (no easy feat without the daily hot seat conferences that were once so common in residency). That, so far, does not appear to be the party line.

This exam needs to be at the end of residency like it used to be. If anything, it might help combat the post-Core senioritis that many fourth-years struggle with, particularly when rotating through services outside of their chosen specialty. I appreciate that many program directors don’t want this during residency because in the past seniors used to disappear from service (and especially the call pool) before Orals just like they do now before the Core Exam. It’s easier to run a residency with only one class preparing for one big test at a time. But convenience shouldn’t be our primary metric.

The choice of returning to orals aside, the timing of this exam remains a mistake. We should be graduating board-certified radiologists.

What’s Old is New Again

In the end, we will again get to pretend that a high-stress, closed-book, hot seat examination is the best way to assess for the requisite absolutely nonnegotiable critical skill of a real-world diagnostic radiologist: the ability to sit in a dimly lit room and crank out mostly correct reports.

From the ABR’s announcement:

The new DR Oral Exam will include select critical findings as well as common and important diagnoses routinely encountered in general DR practice, focusing on examples that optimally assess observation skills, communication, judgment, and reasoning (application of knowledge learned during residency).

It is not meant to represent a comprehensive review of clinical content. The oral exam aims to assess higher-level skills that are needed to be an effective diagnostic radiologist and are valued by referring physicians and patients.

Now, for what it’s worth, I don’t want to suggest that this change may not be in several ways an improvement. Yes, this will be a considerably more stressful examination, especially for the first few years. But the Certifying Exam is duplicative, and if nothing else, this is not. I suspect the real reason oral boards could potentially produce “better” radiologists will be because–unlike the current exam–they will again require real longitudinal preparation for an additional extended period of time. Said improvements, if they occur and if they can be measured, will likely be a reflection of the increased prep time (and perhaps the increased focus on verbalized explicit reasoning that programs will emphasize in hot-seat conferences in the future).

However, the soft skills that the oral boards will presumably purport to test are unlikely to be particularly well or fairly evaluated during the examination itself, especially if we want to ensure it’s unbiased, fair, and less susceptible to the individual preferences of the examiners. To assume that the test itself will actually differentiate between competence and incompetence is analogous to believing that the now defunct USMLE Step 2 CS differentiated between good and bad clinicians. It’s mostly anachronistic theatre.

However, I’ll attempt to reserve judgment for now. I’m almost willing to believe it could be an improvement, even if it will inevitably be a source of stress for trainees and a burden on programs.

Losing the Plot

However, we can acknowledge that a change may have some benefits but still ultimately be the wrong choice.

The DR Oral Exam potentially having some value over the current paradigm does not make it the optimal choice for certification reform.

I have little doubt that giving someone a case and making them perform reflects their understanding and skills differently than their ability to answer a multiple-choice question. I don’t even think that’s arguable. Oral Exams test different abilities. Verbal articulation in a high-stakes situation is a unique skill.

But is it the right skill?

Is that *waves hands encompassingly* the sine qua non of a modern radiologist?

Forgive the tautology, but: the true core competency of a radiologist, whether we want to admit it or not, is the ability to practice radiology.

All things being equal, sure, we do want radiologists to interface well with ordering providers and have good bedside manner for direct patient care. We should want radiologists to dazzle crowds with their facile wit during multidisciplinary conferences. Don’t we love it when the clinicians add on a bunch of last-minute cases and the radiologist still hits a series of home runs on the spot without breaking a sweat? Excellence is a beautiful sight!

But it’s not hard to realize that we are doing something with vague potential benefits that nonetheless is still off-target from the true purpose of board certification.

We may produce better radiologists because of how programs and trainees will respond to this anxiogenic and more opaque exam format–but not because said format itself is particularly well-suited to doing much of anything. Every time we contrive a new evaluation that tests some subset of radiology, we are trying desperately to design a test that can serve as a proxy for real-world competence.

We are constantly assessing knowledge, reasoning, etc “as they relate” to radiology ability. We are not, however, directly measuring a person’s skills: their ability to do this actual job day in and day out. We are fooling ourselves when we pretend that the radiology report is not our core work product. When we shake our heads at the quality of radiology from the “outside hospital,” we’re talking about misses and useless reports.

We can test for that directly. We can have trainees do real work under real conditions and see how they compare to a reference standard of practicing radiologists. We can see if they “demonstrate the requisite knowledge, skill, and understanding of their disciplines to the benefit of patients.” Imagine a test when a trainee just did several 8-hour shifts comprised of curated cases?

We already have the internet at our fingertips, but we are also on the precipice of a world with increasingly powerful AI tools. Any radiology certification process that does not include high-fidelity simulation with clear metrics that reflect real-world performance is ultimately a waste of time.