The ABR Online Testing Experience

11.17.21 // Radiology

When I finally got around to taking the Neuroradiology CAQ Exam this fall, I finally had the pleasure of partaking in the ABR’s remote testing platform. I put a lot of work into my posts about the Core and Certifying Exams back in the day, but they do hail from the pre-pandemic era.

Here are some thoughts on taking an ABR exam from the comfort of not Chicago or Tucson:

Big Brother

~~ABR relies on a third-party company called Proctorio for monitoring.~~ Update: As of 2022, the ABR is now handling remote monitoring in-house instead of relying on a third-party vendor.

Everything—including the real-time video monitoring, capturing a photo of your picture ID during the start-up process, and the “room scan”—rely on the webcam. This feature was clearly intended to be used on a laptop, which you could easily pick up and spin around the room. The ABR now requires a separate dedicated side-view webcam, which makes the acrobatics a bit more complicated.

Instead of buying a separate webcam, I used a super old DSLR with a $15 HDMI to USB video capture card (the knockoff of the better Cam Link). This works fine (and it’s what I use for Zoom calls to have that beautiful bokeh so that people like me more), but it’s worth noting that if you don’t have good autofocus (and mine is so old that it does not), that it may take a few tries to get the screengrab of your photo ID good enough to satisfy Proctorio’s artificially intelligent needs. Depending on your desk set up, a relatively short cable connecting the camera to the computer or a tripod to have the camera high enough to actually see anything if you don’t have a dresser or something suitable nearby can make the room scan awkward and a bit tenuous. A longer USB cord helps. After around 5 minutes of finagling, eventually, I was allowed to take the test.

You could also use your phone as a webcam by using one of many inexpensive apps like Camo or EpocCam, which is probably the best option, especially if you already have some sort of suitable stand or one of the infinite variety of cheapo phone tripods.

Anyway, what I’m trying to say is I don’t think most people need to buy a dedicated external webcam just for the purpose of taking one of these ABR tests. Especially since most webcams are terrible.

I will say that Proctorio didn’t make me install any apps on my phone, and the Chrome Extension is easy to uninstall. I don’t think anyone is going to have their identity stolen or be harassed by call center employees. While I am admittedly not very experienced with the myriad options for education spyware, it seems like the ABR picked one of the better players.

Computer Set-up

I really wanted to use my home workstation. It has direct fiber internet and a system I use every day for radiology-type stuff. However, Proctorio and the ABR have a 1 monitor rule, and I—in my radiologist glory—have 4 monitors. To satisfy Proctorio, any extra monitors in the space have to be missing from your system entirely, so simply turning them off isn’t good enough. They need to be totally unplugged from your computer so that they don’t show up. I use a dual PC system set-up to read simultaneously for the hospital and our outpatient PACS by swapping monitors and peripherals between two different PCs using a KVM switch. It’s awesome. But the nest of monitor cords is no joke, and the settings are just how I like them, and there was no way I was going spend the time unplugging stuff and jeopardize any of that in order to take a 3-hour exam with a bunch of low-quality JPGs.

Instead, I took it on my 2017 iMac over WiFi, which is across the house from the router, and it was fine. The biggest problem was getting Proctorio happy with the backlighting from the adjacent window overexposing the camera and making me look too dim.

Yes, I did roll over my attending chair, the Herman Miller Embody, from across the house for the experience, and it was worth it.

The Practice Test

The practice test is a 98 question process, but really it’s just designed to get you to make sure you can pass Proctorio’s welcome barriers and familiarize yourself with the software. The content is a random hodgepodge of questions, but there is no performance feedback or answer key, limiting usefulness as an actual practice exam experience.

The content parallels the Core Exam, which is to say I found it more difficult now as a super high-performing experienced attending radiologist than I would have as a resident. Go figure.

The Online Platform

I would show you some screenshots, but, you know, I can’t.

The software is totally fine. It works. You have a bunch of tools to manipulate the images, but since most images are just static screen grabs of middling quality, you usually don’t need to do anything. In fact, the image sizes are small enough and the questions themselves short enough that you really don’t need or even want a large monitor or anything fancy to take the exam. Your average laptop screen has plenty of real estate.

The ABR locks your past performance every 30 questions to prevent you from altering your responses to prior questions after you’ve left the surveillance zone for breaks (aka cheating). Locking so frequently is a touch annoying, and I’m sure the policy frustrates the people who want to blaze through the whole test and then agonize for hours on all the questions they flagged at the end. Psychologically, this is probably healthier, though it does make flagging a question basically useless.

I wish they just locked questions at breaks and not after so short a period, but given intermittent connectivity issues, the shorter batch size probably helps substantially reduce their tech support demands. It’s a worthwhile price to pay for the convenience.

Another irritating feature is that the ABR really wants you to move forward through the exam and not backward to revisit old questions, so much so that if you try to visit the previous question, the software prompts you with a pop-up saying (paraphrasing) “forward is so much better, are you sure you want to go back?”…which is annoying. I think the idea here is that they especially don’t want you to see a question, not answer, go backward, lose access in some way, and then have to unlock the question for you and/or question your integrity.

On exam day, I got a brief “connection lost” warning twice, lasting for a few seconds before being reestablished automatically, otherwise no interruptions. I know people personally and online who had less good luck, sometimes being completely kicked out and needing to call to have questions unlocked. To their credit, I’ve only heard good things about ABR customer service when it comes to helping with exam connectivity issues.

Overall

Overall these inconveniences are a small price for being able to wear pajamas and take a break literally whenever you want, which is wonderful. The ability to use a bathroom mere feet away or get drinks and snacks on a whim creates a completely different ambiance than a typical testing experience.

Not having to pay for flights and hotels, waste days on both ends for travel, and leave your family is also almost priceless.

In some ways, it’s a little harder to get in the normal high-stakes exam headspace without the suffering, but it’s a breath of fresh air that I enjoyed nonetheless. I bought a twelve-pack of Diet Mountain Dew just for the occasion, and I made myself a nice fresh ice coffee as well (see above, hat tip double-walled insulated glasses) to sip while answering ludicrous questions about differentiating spinal cord tumors.

If that’s not the dream, I don’t know what is.

Review: Sulcus Neuroradiology CAQ Prep

11.12.21 // Radiology, Reviews

The neuroradiology subspecialty exam (aka the CAQ or certificate of added qualification) is a tedious, basically redundant, and expensive waste of time taken by a relatively small number of people every year.

As a result, there are very few dedicated recourses.

In fact, there is only one, and it’s Sulcus.

Based on an informal survey I did of CAQ takers, the Sulcus bank is relatively popular despite what are generally poor reviews online. In my informal poll, almost half of respondents admitted to using it. I chalk this up to the CAQ exam being exceedingly expensive, people not wanting to fail it, and this being the only dedicated product.

Related note: Sulcus also now offers MSK and Mammography programs, neither of which I looked at.

Reviewer Disclosure

Sulcus did not pay for this review nor have they seen it prior to publication.

They did provide me with 7 days of free access. I have a life, so that wasn’t sufficient for me to view the entire product, but it was enough for me to form an opinion. I’m not sure I would have finished it regardless, and I did not purchase the product afterward for further studying.

I also have no discount code for you, dear reader.

Platform

It’s a little clunky, but it works.

You must “create a quiz” as a separate step before you can actually take said quiz.

There is no tutor mode.

The site is technically mobile-friendly but the left sidebar (which is just the question numbers) takes priority, meaning that you need to scroll down past the question navigation to do every single question and read every explanation. Very tedious.

Content

There are a total of 562 questions grouped into 175 cases.

As in, there are several questions per case/diagnosis/image(s). For example, doing my first 5 cases used up 16 questions.

The cases themselves are solid. But when each case has three or sometimes four questions, the first is usually high yield, the second is borderline, and the third is basically just for giggles.

The question format is nothing like CAQ. Most CAQ questions are diagnosis-based.

Many Sulcus questions are multiple-choice “pick the correct statement” nonsense, where you evaluate a bunch of statements and then pick the one that is true. It also contains questions that are negatively framed (“which of these are NOT…”) questions that are garbage psychometrically (in their defense, these still crop on ABR’s exams rarely as well).

You’ll also get a lot of extended multiple T/F questions, where you need to evaluate a whole bunch of statements about the diagnosis and get each of them right in order to be “correct” overall. So, the end result—as other reviewers have alluded to—is mostly irritating/demoralizing.

Extended matching questions are pretty rare but fine.

Overall, the bank trends too much toward hyper-detailed in-the-weeds stuff, making it very inefficient to review in quantity for a busy practicing radiologist. The CAQ is an image-based test mostly of diagnosis, so testing a whole bunch of second-order factoids is not worth your time.

In Sulcus’ defense, I have a suspicion that the CAQ exam has changed significantly over the years. It very well may be that its current format more closely resembled the CAQ of old before changes brought the question style more in line with the modern multiple choice offerings of the Core and Certifying Exams. I support this theory with two points.

No one would choose to write questions like this for no reason.
The current (but clearly out of date) CAQ exam study guide from 2017 suggests that most questions would be part of linked question sets: “After the candidate makes her/his choice, one to three follow-up questions will typically be asked.” This is simply not the case. Most questions are solitary single best answer and do not go on to hammer you on miscellaneous second-order details.

Survey Results

I did a brief survey online and got 38 responses. Of the 18 people who said they used Sulcus, there were only two written comments: “didn’t think was relevant” and “was the highest yield by far.”

I think the reality is somewhere in between.

Pricing

$399.

Free Sample

There is one sample case in a slideshow at the bottom of the main page.

Verdict

Eh, pass.

A decent collection of wide-ranging cases; not a bad learning tool but extremely expensive related to other options and not an efficient CAQ review for the practicing neuroradiologist.

Unless you really have a CME fund to burn and nothing else to spend it on, you don’t need something this tedious and demoralizing to get through for the purposes of passing the CAQ exam. Some regular cases books, like Neuroradiology: A Core Review, will get the job done.

If you really just want a chance to see some pretty solid cases, take some great and some excruciating multiple-choice questions, and really hammer some low-yield esoterica to impress trainees and specialists, then this is an excellent choice.

A Glimpse at the Coming Metaverse

11.05.21 // Miscellany

Ben Thompson of Stratechery talking about why he’s becoming more bullish on virtual reality than augmented reality during an interview with Mark Zuckerberg about the Metaverse:

I do have to say, the last couple of years, particularly the COVID era, has changed my perspective a little bit as there does seem to be more and more of sort of a bifurcation between your online reality and your offline reality. It’s something I wrote about in the context of work, where people call it working from home, but I actually think that’s a misnomer: it’s actually working online, and you can work online from anywhere but when you go online, you’re in a different place cognitively speaking than you are when you’re at home or playing with your kids, or you’re seeing your friends or whatever it might be.

I get to work from home sometimes, but I think there is a key nuance here. One of the hardest things about working online from home is that the people around you don’t want or feel like you are somewhere else, cognitively speaking. Frankly, getting to that other cognitive place in the first place can be quite difficult when surrounded by the context of all the non-work things around you.

Does a future where you can slip further into the internet make it easier or harder to be productive online?

Well, it depends on how you define productivity. In the interview, Zuckerberg claims that fostering human connection is his life’s work. He then goes on to freely admit that ruining meaningful unadulterated human connection is a “killer use case”:

Although I do think that for augmented reality, for example, one of the killer use cases is basically going to be you’re going to have glasses and you’re going to have something like EMG on your wrist and you’re going to be able to have a message thread going on when you’re in the middle of a meeting or doing something else and no one else is even going to notice. Think about what we’ve had over the last couple of years during the pandemic where everyone’s been on Zoom, and one of the things that I’ve found very productive is you can have side channel conversations or chat threads going while you’re having the main meeting. I actually think that would be a pretty useful thing to be able to have in real life too where basically you’re having a physical conversation or you’re coming together, but you can also receive incoming messages without having to take out your phone or look at your watch and even respond quickly in a way that’s discreet and private. So I think that there are going to be those use cases. I think that there are going to be easier ways to get in and out of experiences where you’re experiencing that deep sense of presence.

The problem with social media on cell phones is that your kids, friends, and colleagues know you are being rude and self-absorbed when you ignore them. The problem—of course!—is not the behavior and our inability to be fully present in our interactions. The problem is being so transparent in informing others that they are insufficiently interesting to hold our full attention.

This is the promise of the coming metaverse.

Functional Embezzlement

10.25.21 // Finance

From Charlie Munger’s Herb Kay Memorial Lecture, “Academic Economics: Strengths and Weaknesses, after Considering Interdisciplinary Needs” (University of California at Santa Barbara, 2003):

…I asked the question “Is there a functional equivalent of embezzlement?” I came up with a lot of wonderful affirmative answers. Some were in investment management. After all, I’m near investment management. I considered the billions of dollars totally wasted in the course of investing common stock portfolios for American owners. As long as the market keeps going up, the guy who’s wasting all this money doesn’t feel it, because he’s looking at these steadily rising values. And to the guy who is getting the money for investment advice, the money looks like well-earned income, when he’s really selling detriment for money, surely the functional equivalent of undisclosed embezzlement. You can see why I don’t get invited to many lectures.

Fee-drag is insidious and nearly invisible to the human mind at a glance. As COVID-19 demonstrated, we are not wired to intuitively understand compound growth. When you see your accounts growing, you are happy. Even if you see your fees, they may seem reasonable on a snapshot basis.

What you don’t see, of course, is the effect of those fees year after year. Every loss is another piece that can’t undergo the magic of compounding in your favor. As the saying goes, “it’s time in the market, not timing the market.”

If you ever wonder how nice people can practice in an Assets Under Management model, the same problem works in both directions. Your money is still going up, so they feel they are providing a valuable service, especially in holding you to a plan and preventing you from otherwise hamstringing yourself (like, say, risking your nest egg on chasing meme stocks on Reddit or buying start-up cryptocurrencies).

Psychologically, we’re very good at cognitive dissonance: of not seeing what is inconvenient for us. Those professionals would rather see the “value” they create in terms of investment growth and the end-result financial security and not the excessive value removed from larger investors (and the even larger wealth those clients might otherwise enjoy).

The Big (Temporary) PSLF Expansion

10.08.21 // Finance

You may have heard the news by now: PSLF has been (temporarily) expanded (again).

Back in 2018, TEPSLF created a new pot of money to help borrowers who had used the wrong payment plans in the past.

Now, in a final heave of their national emergency powers, the government will finally fulfill the spirit of the original law: more people getting forgiveness, fewer people missing out because of technicalities and bad servicing.

All “federal” loans are forgivable.

The inclusion of FFEL loans in the PSLF program is more noteworthy than you might think. You see, Direct Loans (the only current option and always part of PSLF) are provided and held by the federal government. The government forgiving its own loans is the whole point of the program. The now defunct FFEL program however was instead a public-private partnership: loans provided by private banks and secured by the federal government. In order to pay off FFEL loans, the government is going to encourage tens if not hundreds of thousands of borrowers to consolidate loans into the Direct system in order to forgive them, paying private companies real money in the process. This is why PSLF has specifically never included FFEL loans in the past (even though one could consolidate those FFEL loans and trade them in for a Direct Consolidation loan, making them eligible with minimal effort).

The fact is that for recent graduates the news is largely irrelevant. Very very few people graduating in recent years hold any FFEL loans or Perkins loans, and nearly everyone is using the correct payment plans. It’s just much easier for new graduates to set themselves up for the program than the older borrowers who were further along in the process (and who have been getting rejected or lost years of payments [often due to bad servicing]).

At baseline, people need to stop worrying about the PSLF rug being pulled out from underneath them, but hopefully, this second expansion will assuage lingering doubts. The program is still real, and it’s never going away retroactively.

Here is the Department of Education’s “Fact Sheet” about the overhaul.

And here is the very readable official description of what it all means and what to do next. This is the official party line, and it’s what you need to read.

The bottom line is that if you have any FFEL or Perkins loans, you need to consolidate those now and file a PSLF form (well at least by October 31, 2022). There are a lot of people working in public service and academics who are magically eligible for forgiveness this week that weren’t before (and there are going to be some very anxious people trying to track down employment verifications from back in 2008).

The ABR is Sorta Changing Its Fees

10.06.21 // Radiology

In recent years, the American Board of Radiology (ABR) has utilized a membership fee model, where—for example—those working towards an initial certification in diagnostic radiology would pay a $640 annual fee until passing the Certifying Exam. Since one takes the Certifying Exam 15 months after finishing residency, that has meant recent diplomates have paid a specialty tax of around 1% of their gross income for a total of five years before enjoying the privilege of paying a mere $340 per year for MOC forever.

The fee schedule looked like this:

To illustrate, here’s my payment history (the annual fee actually increased a bit during my training because money).

As of September 2021, ABR has moved to “an exam fee model.” How does that look? Well, a one-time $640 application fee followed by a $1280 Core Exam fee and a $1280 Certifying Exam fee.

It doesn’t require a doctorate to note that the total cost for initial certification is the same: $3200.

That fee continues to put radiology in the highest echelon of medical specialties in terms of board costs, as enumerated in this 2017 paper (which incidentally undercounted the radiology costs).

What has changed is that this fee structure is now standard across other exams and is resulting in a decrease in the (otherwise ludicrous) subspecialty exam fees.

You see, until now, the much shorter half-day CAQ exams actually cost the most! As above, you can see I paid $3,280 this spring for the privilege of spending a morning taking a poorly formulated exam to pseudo-prove that I can totally do the thing I already do every day. That’s more than the cost of the combined total of the much, much bigger Core and Certifying Exams.

But, as of this September 17, 2021 update, it’s merely the same $640 application fee + $1280 exam fee for a total of $1,920 (a savings of $1,360!).

Of course, before you get any warm fuzzies about their generosity, keep in mind that the CAQ exams comprise a relatively small proportion of ABR revenues since only ~200 people take them every year, and, meanwhile, MOC revenues continue to grow year after year. The ABR, per its internal narrative and official documents, has recently been operating at a loss.

Thankfully, they have some retained earnings on hand to mitigate the red.

Bedside Business (Podcast)

10.02.21 // Medicine

I did a Q&A about student loans and the transition to residency (as well as a dash of passion is overrated and medical education is toxic) with the fine students across the DFW Metroplex at TCOM this spring, and it’s now available as an episode of the Bedside Business Podcast (Apple | Spotify | Google | Stitcher).

The Zoom recording audio is a smidge choppy at times but not enough to hurt as long as you slow down to 1.5x to account for my speed!

Driving at Stable

09.23.21 // Medicine, Miscellany

A classic Jeff Bezos quotation:

I very frequently get the question: “What’s going to change in the next 10 years?” That’s a very interesting question.

I almost never get the question: “What’s not going to change in the next 10 years?” And I submit to you that that second question is actually the more important of the two.

You can build a business strategy around the things that are stable in time. In our retail business, we know that customers want low prices, and I know that’s going to be true 10 years from now. They want fast delivery; they want vast selection. It’s impossible to imagine a future 10 years from now where a customer comes up and says, “Jeff I love Amazon, I just wish the prices were a little higher.” Or, “I love Amazon, I just wish you’d deliver a little slower.” Impossible.

So we know the energy we put into these things today will still be paying off dividends for our customers 10 years from now. When you have something that you know is true, even over the long term, you can afford to put a lot of energy into it.”

I recently attended a “leadership” seminar about (radiology) healthcare ecosystems and change. As with all virtual events since early 2020, discussion of the Covid-19 pandemic played an outsized role, and the nature of complexity and change were much pontificated about.

But no one over the course of two days—no one—mentioned the stability of the core mission. The strategic analyses—such as explicit or implicit utilization of SWOT—were happy to focus on anticipation and interception of perceived changes and threats, but no one spared a breath for what they thought wouldn’t change. We talked about trends in corporatization and productivity metrics, group consolidation, encroachment by midlevels and other specialties, downward reimbursement pressure, the push for 24/7 subspecialty staff coverage, lifestyle and burnout, and AI and data science.

To be sure, these and all other big changes are important, but you also can’t lose sight of the underlying purpose of the business in all the pivoting.

What can we say about medicine that is not going to change in 10 years? What is our stability north star?

(Yes this is a rhetorical question cop-out.)

The Stress Heuristic

09.09.21 // Miscellany

Cal Newport, author of the beloved Deep Work (among others), writing in The New Yorker.

…most workers who are fortunate enough to exert some control over their efforts—such as knowledge workers and small-business entrepreneurs—tend to avoid working way too much, but also tend to avoid working a reasonable amount. They instead exist in a liminal zone: a place where they toil, say, for the sake of fixing a specific number, twenty percent more than they really have time for. This extra twenty percent provides just enough overload to generate persistent stress—there’s always something that’s late, always a message that can’t wait until the next morning, always a nagging sense of irresponsibility during any moment of downtime. Yet the work remains below a level of unsustainable pain that would force a change.

…

If you’re a professor, or a mid-level executive, or a freelance consultant, you don’t have a supervisor handing you a detailed work order for the day. Instead, you’re likely bombarded with requests and questions and opportunities and invites that you try your best to triage. How do you decide when to say no? In the modern office context, stress has become a default heuristic. If you turn down a Zoom-meeting invitation, there’s a social-capital cost, as you’re causing some mild harm to a colleague and potentially signaling yourself to be uncoöperative or a loafer. But, if you feel sufficiently stressed about your workload, this cost might become acceptable: you feel confident that you are “busy,” and this provides psychological cover to skip the Zoom. The problem with the stress heuristic is that it doesn’t start reducing your workload until you already have too much to do. Like Parkinson’s naval bureaucracy, which expanded at a regular rate regardless of the size of the Navy, this stress-based self-regulation scheme ensures that you remain moderately overloaded regardless of how much work is actually pressing.

“The Stress Heuristic” is a great term for people’s default strategy for avoiding more work: being literally too busy for more work.

But while saying ‘no’ is easiest when saying ‘yes’ is impossible, it forces you to live without margin. And margin—space in your life for yourself, serendipity, and the chance to chase down things of interest—is where the magic happens.

Even for academics, consider the words of psychologist Amos Tversky (whose work with Daniel Kahneman yielded the Nobel-prize-winning Prospect Theory and the crazy popular book Thinking, Fast and Slow): “The secret to doing good research is always to be a little underemployed. You waste years by not being able to waste hours.”

Writing Makes It “True”

08.30.21 // Miscellany, Writing

From “How to Leverage Language to Cultivate Your Creative Process” by Nicole He in Killscreen.

I had a concept in my mind—maybe I felt it emotionally, I had a feeling about what this thing is supposed to represent. Now what I’m saying makes it real. After that, I started responding to journalists the next morning. What blew my mind was this: the thing I wrote about my project became true about the project. Many of the things I would say, the lazier journalists would just copy and paste. Weirdly, seeing my own ideas in the media suddenly made them even more true—things about a project being about intimacy, about computers knowing more than we know about ourselves. All of this became true, because someone else was saying things about my project, but based on what I had said about my project.

There’s a lot to be said for putting in the effort of distilling a vague idea into a clear concept and writing it down (and, yes, maybe even sharing it). Not just for a specific project, but for you and your whole career.

There is power in storytelling, both internally and externally.

And, perhaps weirdest of all is that—on the whole—you get to define your own narrative.