The Machinery of Skepticism

Carl Sagan, famous astronomer and author of Contact and Cosmos (among others), writing about “The Burden of Skepticism” way back in 1987:

It seems to me what is called for is an exquisite balance between two conflicting needs: the most skeptical scrutiny of all hypotheses that are served up to us and at the same time a great openness to new ideas. Obviously those two modes of thought are in some tension. But if you are able to exercise only one of these modes, whichever one it is, you’re in deep trouble. If you are only skeptical, then no new ideas make it through to you. You never learn anything new. You become a crotchety old person convinced that nonsense is ruling the world. (There is, of course, much data to support you.) But every now and then, maybe once in a hundred cases, a new idea turns out to be on the mark, valid and wonderful. If you are too much in the habit of being skeptical about everything, you are going to miss or resent it, and either way you will be standing in the way of understanding and progress. On the other hand, if you are open to the point of gullibility and have not an ounce of skeptical sense in you, then you cannot distinguish the useful ideas from the worthless ones. If all ideas have equal validity then you are lost, because then, it seems to me, no ideas have any validity at all.

Some ideas are better than others. The machinery for distinguishing them is an essential tool in dealing with the world and especially in dealing with the future.

That summarizes so much.

Flywheels and Doom Loops

Jim Collins, author of Built to Last and Good to Great, talking on The Knowledge Project:

What we found is that the most durable results happen as a series of good decisions that accumulate one upon another over a very long period of time, that create a massive compounding effect. And just like investing, where it’s buy quality assets you would presume to hold forever, then largely do and let them compound, this is the idea that you get a really good thing and you build strategic compounding over a very long period of time, and then you end up with this spectacular result.

He calls that the “flywheel effect.” And, as you might expect, there’s an opposite phenomenon:

Let me just describe the inverse of the flywheel, which is the doom loop. Something happens that produces disappointing results. And it could be that it was a random event or something just happened that was out of your control or something that you just made a mistake or you bungled something, whatever. You get disappointing results. But unlike really understanding why that happened so that you can correct, what happens is a company reacts without understanding. “Oh my gosh, we had disappointing results,” and often what happens is they panic. They look for a new direction or a new program or a new leader or a new acquisition or a new technology or something, and because that never really produces a great result, it produces a burst of false hope, but it’s like drinking a sugar drink as opposed to getting back to your core training. It doesn’t give you any accumulated momentum, which then creates another negative inflection, more disappointing results, which then more reaction without that understanding. Then another new direction, new fad, new program, new whatever, and then another failure to build momentum, more disappointing results, and then you’re in the doom loop.

If you’re honest, how many quality initiatives have you seen as a true flywheel of progress and not just a spoke in the doom loop?

Review: CaseStacks Radiology Call Prep

Before we get to it, the usual disclosure: this is not a paid review, but it is the usual kind where I get to offer a reader discount combined with an affiliate link, a win-win that makes it worth my time to write these reviews for products that I believe in. Coupon code benwhite gets you 15% off.

One of the most difficult things about radiology residency is the transition to call, especially for those programs that still have independent call. You go from generating draft reports that your attending may never read before just telling you what to say to suddenly being responsible for actual words that directly impact patient care. Compounding this stress is the fact that you may not have seen everything that you need to see during your rotations to prepare you for this experience. Reading books and articles and doing questions from casebooks or question banks are all certainly helpful, but they don’t simulate the process of actually opening a case on PACS, working through it, and mentally making a decision.

Enter CaseStacks, a new subscription site created by two Neuroradiology fellows at Wake Forest.

CaseStacks aims to be the way that radiology residents prepare for call.

CaseStack has multiple “courses” of different case types each with a combination of high-end bread and butter and some more complicated pathology, all presented with PACS simulation. Currently available courses are Neuro CT, Neuro MRI, Body CT, Chest CT, MSK Radiographs, Peds Radiographs, Chest Radiographs, and KUB. Several of these are subdivided. For example, Neuro CT includes Nontraumatic Brain, Traumatic Brain, Head & Neck, and Spine. Most also include a combination of “classic” and “practice” modules, the latter adding a combination of more subtle findings and negatives to keep you on your toes.

CaseStacks uses a web-based PACS, so all cross-sectional cases are scrollable, allowing you to really experience the case as you would in real life (can also window/level and zoom/pan). Each case is accompanied by findings, a diagnosis, teaching points, and a “preliminary report” that puts some real words on the page for how you might dictate the case in real life (very neat). Cases also include incidental findings that are invariably present in real life but never included in qbanks or casebooks, which typically only include a few static images and don’t reflect the breadth and variety of a real shift.

There are also 5 assessment modules, which include a combination of unknown cases including normal exams for a self-assessment (or for programs to test you) prior to taking call. This feature isn’t as fleshed out yet. The 5 offerings vary widely in length and do not combine all courses (or have a peds variant), so there is no single assessment that, say, covers neuro/body/chest CT + variety of radiographs. That would be clutch, but they’re not there yet.

The site technically works on mobile but doesn’t play that nicely. You’re better off with at least a laptop size screen.

Cost

They offer plans in 3-, 6- and 12-month increments for cross-sectional, plain film, or everything (“pro”). The pro version is $33.33/mo for a full year (~$400, pricey), and the price goes up to $45.33/mo for the shortest duration (3 months = $135.99). Definitely expensive but possibly a more practical use of your book fund than collecting books you probably won’t read.

Free Stuff

The free sample is a breath of fresh air. You can just navigate to the site, click on courses, and see a few complete cases from each class. No login required to see if it tickles your fancy.

Also free with no login required? Anatomy modules, incidental findings tables, normal head CT findings/variant/mimics (things all residents mistake for pathology at some point), peds radiographs normals by age (extremely helpful, especially for musculoskeletal radiographs). All 100% worth checking out and a great resource for call. Definitely bookmark it. I would have really loved the peds normals my first time taking solo peds call.

Take-Home

If CaseStacks existed when I was an R1 or R2, I absolutely would have paid for the service for a few months before starting call. It would’ve been invaluable for my confidence going into a challenging experience.

 

America’s Advisor

From David Roth’s The New Republic piece, “American Psycho,” an absolutely scathing brief profile of Jared Kushner’s role in unquestionable pandemic mismanagement:

So here we have Kushner, a powerful special adviser with no meaningful expertise in public health or epidemiology, using a breathtakingly specious chart produced by an economist who’d flubbed the biggest prediction he’d ever made—all as a justification for the federal government to do less to confront a rampaging pandemic. While the Trump years have offered many such crystalline and bottomless moments of executive abandonment, this one felt uniquely Jared. The collaboration is what makes it—a legacy figure in the field of elite ineptitude, delivering the old egregiousness in a style optimized for the vacuous new avatar of elite incompetence. The gilded tools of one generation of catastrophic conservative governance pass into the soft and clammy hands of the next. If it weren’t for all those people dying, it would be beautiful.

Translating the ABR’s Response to Exam Postponement Discontent

Two days after they announced the second cancellation of scheduled exams this year, the ABR felt compelled to rapidly address the massive discontent in the radiology committee. Perhaps hearing that a joint letter from every trainee and many rad organizations was in the pipeline gave them the extra incentive to try and preempt (unsuccessfully, see below) formal censure from its stakeholders.

On the whole, ABR President Vincent P. Mathews’ letter is…defensive. And it addresses the core concerns by calling them out but largely without meaningful explanation that would make the ABR’s repeated unforced errors in the handling of the pandemic look like anything other than inept and out of touch. It sounds like he’s inherited this mess, but the ABR needs more than a bigger listening ear: They need a paradigm shift.

I’ve been asked if I could translate from ABR-speak into English, so without further ado.

Colleagues,

Based on feedback received after the ABR’s announcement that we are canceling most of our exams for 2020 due to the COVID-19 pandemic, I want to provide more information regarding the considerations that led to this decision.

Restrictions on large gatherings in many locations, including Chicago, where we have our largest testing center, are very unlikely to be lifted before the fall. In addition, our staff had thoroughly investigated the logistics required to deliver an in-person exam, and our conclusion was that we could not guarantee the safety of our candidates, diplomates, and staff to the level required to proceed.

We know everyone told us that planning for an in-person exam in November was nuts. We know there was never any chance the virus would magically disappear or that chilly Chicago in November would ever be a better place for humans than June let alone a safer place for virus-free travel. I know people told us it never had a chance of happening and that if somehow we did have the ability to force people to come that it would be a disaster. Instead of listening, we wasted valuable time that we could have spent preparing a remote solution trying to figure out a way to use the testing centers we should have never built.

Many continue to ask about giving our exams at local testing centers. We do offer some of our Radiation Oncology and Medical Physics Exams at Pearson VUE centers, and we still have some scheduled in December. Neither Pearson VUE nor Prometric has the technical capability to deliver our Diagnostic Radiology Exams and have not been interested in developing those capabilities when we have inquired repeatedly over several years. Currently, these centers are closed, and it is uncertain when and where they will open.

A long time ago when we developed these exams we didn’t take into account the available capabilities of commercial centers (though we haven’t and have no intention of ever disclosing what those missing capabilities are or why we didn’t just go back and pay some nerds to re-make the software to make this feasible). After all, a bunch of our own exams as well as those offered by the American Board of Nuclear Medicine and the ACR In-service are offered at places like this. In the end, building our own centers so that we could reduce the ambient light by a few lux seemed like an easier and more profitable solution.

Others have asked why we can’t deliver a virtual exam. We are actively exploring alternatives and are collaborating with other ABMS boards to evaluate options. Currently, we have no reliable, secure option for a high-stakes exam such as the initial certifying exams that we deliver. We are aware that the American Board of Surgery and the American Board of Ophthalmology have announced they will give virtual oral exams in 2020. Their exams are different than ours, and they have not validated the ability to use radiologic images in those exams. No board has developed a secure initial certification written exam virtual platform that would be needed to deliver our Diagnostic Radiology and Interventional Radiology Core and Certifying Exams. When the breast content on the core exam was not successfully delivered to all candidates in 2017, we devised a solution for individuals to take that content remotely rather than come back to the testing centers. During this administration, some candidates had technical problems that we were able to fix “on the fly,” which was possible because of the small number of candidates taking the short exam at any given time. That exam contained very limited content and, in fairness to the candidates, we removed some items that were potentially too subtle without the monitor display control we have in our testing centers. Fortunately, it worked for a session of less than an hour for a minority of our candidates; however, we are not confident that it would work for a 10-hour exam for more than 1,000 candidates. Because of the magnitude of the significance of our initial certification exams, we must ensure near complete reliability. As we have experienced on two occasions in the past few years, even a minor failure in the delivery of these exams is extremely disruptive to everyone involved.

I know that no one actually expects us to have a ready-made, rapidly-deployable remote solution, but I feel compelled to conflate the fact that that we currently lack this capacity with our general unwillingness to pursue this goal with full vigor and the financial might of our near limitless cash reserves.

Our exams aren’t like other exams. Each exam is truly unique, like a snowflake.

We know that our continued harping on “image quality” is tone-deaf and completely meaningless given the functionality of literally every computing device used over the past twenty years, the existence of image-rich exams in multiple settings including our very own OLA, and the fact that cross-sectional modalities use images that could have been displayed by a Super Nintendo.

We also know that trying to crap on the “reliability” of our one-time remote experiment may also fall flat given that said effort came because our home center was even less reliable in the first place.

These mental gymnastics may be nonsensical, but they’re our party line, and they’re the best we got.

To reiterate, we just don’t have a reliable platform to give our computerized or oral exams right now. We are exploring our options for each of our specific types of exams in our various specialties and, hopefully, we can find solutions in the coming months. Programs have been asking for a decision soon because they are creating their resident schedules for the next academic year; therefore, we decided to move ahead with our announcement. We know that it will be highly disruptive to have two years of residents taking the Diagnostic Radiology and Interventional Radiology/Diagnostic Radiology (IR/DR) Core Exam in June 2021. Therefore, we hope to offer an administration in Chicago and Tucson in February 2021 in the event we do not have a virtual option. We are finalizing the dates and will make an announcement as soon as possible. We will also engage program directors and others as we progress with any virtual solutions.

We are looking, but we don’t want to promise to do the right thing. We’d rather arbitrarily postpone the date to another fictional timepoint instead of promising to carry out a remote solution on the dates we already provided and for which all programs and individuals have since prepared.

That said, we know—deep down—that we must provide a remote testing solution. And we know, since commercial centers won’t be able to take on this business, that it must either involve programs/institutions or people at home. But we are simply not ready to admit this to ourselves or others except in the most oblique of ways.

This is another unfortunate situation created by an unprecedented pandemic, and I hope you understand that this decision was not made lightly. We are fully aware of the disruption this causes for our candidates and their programs and desperately would have liked to have an acceptable solution sooner. We take responsibility for not engaging more stakeholders in the conversation leading to this decision, and we commit to doing better in this regard. Over the next two years of my service as ABR President, with the full support of the entire organization and Board, I will make this a priority. However, I do not think we would have arrived at a different outcome at this point in time.

We know that announcing a plan isn’t the same thing as having a solution in place, but we hope you understand that we believe all the current drama is entirely secondary to an unavoidable pandemic and not at all a reflection of our systemic deficiencies. We are sorry that we didn’t listen to your concerns formally before ignoring them. We promise to have an increased number of superfluous token conversations in the future.

And so

This may have delayed the formal response from a ton of radiologists by a day or two, but it changed nothing:

It’s a good letter, and you should read it. The joint effort covers the salient points in measured tones and doesn’t once devolve into the ruthless mockery that less restained commentators might have been unable to resist.