The ABR Mammography Saga Continues

This week’s ABR Core exam snafu update:

Dear ABR Candidate,

The ABR board members and staff sincerely apologize for the problems with the diagnostic radiology Core Exam on Thursday, June 8, 2017 at our Chicago Exam Center. We did not start the exam on time, had intermittent interruptions, and we failed to deliver the breast imaging content to many candidates.  Candidates in Tucson were not affected, nor were candidates who took the exam in either center on June 12-13. We were extremely disappointed, and we know you were too. We have closely examined the situation and made changes to prevent another event like this.  In addition, we have developed a preliminary plan for administration of the breast imaging content to the candidates who did not receive it.

Here is information regarding our plans:

— We are on track to release the Core Exam results by the end of July, 2017.
— Candidates who did not receive breast imaging content will get their Core Exam pass/fail/condition result at the same time as those who did receive the breast content.
— Preliminary results for this Core Exam are very similar to results from previous administrations.
— There are no candidates for whom the presence of the breast imaging module was responsible for a pass or fail result. In other words, people who failed did poorly enough in multiple areas that even a stellar performance on the breast module would not have allowed them to pass.
— Candidates who did not receive breast imaging content will be required to pass a separate breast module, which will be distributed online in September 2017. We are finalizing our plans for this — it will not require travel or additional expense for candidates or their programs, and we anticipate that it will take only about an hour to complete.
— Performance on the breast content for those taking the separate module will not affect their Core Exam result; however, these candidates will be required to pass the separate breast content module in order to be eligible to take the Certifying Exam.

Again, we are truly sorry.  We greatly appreciate your patience while we have worked on the solution to this situation.

The email style has improved a bit since last time.

Summary impression:

  • The sections are a farce
  • The “separate breast module” is a meaningless box-checking endeavor
  • If they can disseminate the “breast content,” then they can distribute the whole thing

The sections are a farce

Of course the ABR would claim that the presence or absence of the mammo section had no bearing on anyone’s actual Core exam results. This conclusion was essentially guaranteed by the ABR’s claim/decision in past years that no one has ever conditioned an individual section outside of physics (which has a higher passing threshold). Essentially, the exam grading paradigm has been structured such that the gap between an overall passing performance and an individual section failure is so wide that no one (n > 4500) has ever managed to fail a single section without first doing so poorly on the exam on the whole that they fail the whole thing outright. This, of course, begs the question, why even pretend to grade each section separately if no one can really fail one?

The corollary to this is that the Core exam cannot actually ensure when you’re really competent in an individual section outside of its overall passing rate. It’s been essentially shown that if you can pass the exam in general, there is no meaningful way for you to fail mammography (or anything else) by itself. The ABR cannot by its own grading system guarantee meaningfully adequate performance in an individual area. Because the grading scheme’s details are kept secret, we can never know what percentage is required to condition or fail the exam. We do know that after four administrations of the exam, it is likely nearly impossible. In real life, we know people are not equally good in all sections. It is not hard to imagine that in some cases someone may just barely pass the exam but still truly be pretty terrible in one section. And yet, this has never borne out with a single non-physics conditioning performance.

This is not to say that I think people should be forced to travel across the country again just to take a one-hour section test—because that would be stupid. Preventing this from happening is presumably one of the reasons why the conditioning threshold for individual sections is so low.

Therefore, the breast module is also a meaningless box-checking endeavor

Based on history and the ABR’s admission that breast module performance had no effect on Core Exam passage, whether or not mammography is actually included in the exam or not is irrelevant from any practical standpoint. Any section(s) could be missed and would likely have absolutely no effect on overall exam passage. What the ABR is admitting with this gesture is not that the Core exam can even guarantee satisfactory competency in an individual section (i.e. that you can actually interpret a mammogram), but rather that it is too embarrassing to simply not test an entire region of the body, perhaps particularly so when the majority of examinees did eventually receive the content.

I do wonder a few things:

  1. Where did this decision come from? Was it from the ABR’s own problem-solving toolbox, or is it a reaction to some perceived MQSA deficiency or sub-specialty push back? Originally, the ABR implied the loss of the mammography content “should” have no bearing on MQSA.
  2. Would this be the solution if it was a different section that was missing? Would people be dealing with the same nonsense for the cardiac section (which as we know is meaningless to the majority of practicing radiologists)? Of course, at this point, the answer to this question would be “Absolutely!” So we’ll never really know.
  3. What happens if you “fail” the breast module? The Core Exam result isn’t malleable, so if this is even possible, does one just simply take it again and again until you pass? Is this the radiology equivalent of the online training modules about information security and fire safety that you pretend to read every year?

If you can distribute one section, you can distribute them all

If the ABR carries out its plan to somehow disseminate a single exam section without any cost to the examinees or programs, they are only two solutions that I can readily think of:

  1. Do it at local commercial venues like Pearson VUE or Prometric and pay the fees for all test-takers out of the ABR’s surely overflowing coffers.
  2. Offer a web-based version (like the ABR’s online practice exam) that can be taken at the resident’s institution (presumably with some form of proctoring).

Either way, making this section and releasing it in either form destroys any claims about the ABR’s inability to do this for the exam as a whole.

Again, I don’t want to diss the ABR’s testing center proper. It’s pretty nice, and the rapid/open bathroom break policy is a welcome change compared with the police state supervision of commercial testing centers. But, it’s still not worth forcing people to travel across the country for.

7 Comments

Christopher 07.18.17 Reply

On that last point, a curiosity. Coming from upper levels in my program, ABR dues have historically been paid ~ March. This year, as a new R1, our $640 in dues must be paid September 30th or a $400 late fee will be applied. Perhaps this is the ABR trying to grab the money they need to cover for their mammo problems.

In any case, it’s taken a lot of my fellow R1s by surprise, many just having moved across country and all. $640 isn’t easy to come by.

Ben 07.19.17 Reply

Interesting.
Transparency would be nice here too, it would seem. I do believe I paid around February in the past.

Joseph D. 07.30.17 Reply

Regarding your points here, and in the referring article ‘ABR Botches 2017 Core Exam’, you have a clear and logical thought process, and you posit many good points. For brevity’s sake, I will cut to the point, because you seem to be in a position to offer at least preliminary guidance, and you may find my particular situation interesting, especially given the blunders of this year’s ABR CORE administration (and, I might add, the very new Board of Administrators that have been appointed from February through April of 2017). Basically, I failed the exam by 10 points TOTAL (my score 340, rather than 350- for the record I took exam in Tuscon June 8/9)…. But I passed every section (>200), including physics (420). Many of the sections I did poorly on were those I considered my stronger areas going into the test, and felt relatively confident about after the exam (including Neuro, and BREAST)…. which leads me to the question:

How do I challenge this score, and/or at least demand transparency in the grading process?

I am very concerned about the lack of said transparency this year, and cannot help but wonder if I was some sort of sacrificial lamb (one of many) in a cover-up scheme that was neither ‘criterion based’ nor ‘norm-based’ but rather some screwed up amalgamation of the two.
Regardless, a) thank you for your continued coverage of this topic, b) thank you in advance from any information or guidance you can offer in this regard. I need to move quickly, I realize, to establish opposition, but also start studying for the re-test I will ultimately have to take regardless. It is disconcerting, to say the least, that this conspiracy theory seems as plausible as the ‘honest’ explanation that people just plain failed!
Best,
Joe

Ben 08.02.17 Reply

I know this isn’t the best response, but: I seriously doubt you’ll get any more details about the grading process from the ABR. While I doubt they are purposefully failing people out of their norms, the opaque two-step grading scheme is something they clearly have put some thought into and will stand by. The purpose of the initial step is to determine the overall passing score–until that is met, the individual section performance is unfortunately irrelevant. The fact that you can pass Stage 2 basically doesn’t matter, because Stage 2 solely concerns cringe-worthy section performance and not overall passage.

Personally, I’d just study for the retake and do your best. Since you were so close the first time, I have no doubt you can pass the second time (after all, a little bit of luck would have probably done the trick as well).

Leave a Reply to Joseph D. Cancel reply