The Devil’s Tic Tac

From “Nobody Has My Condition But Me” by Beverly Gage in The New Yorker:

Plus, the longer you stay on it the harder it becomes to stop. Prednisone is sometimes referred to as “the Devil’s Tic Tac”: cheap and available and effective, but at potentially scorching long-term costs.

Great term.

I always find it interesting that The New Yorker changes the titles of its stories for the internet so often. In print, this personal essay was titled “One of a Kind.”

Noise in Medicine

Some medical takeaways from Nobel-laureate Daniel Kahneman’s Noise: A Flaw in Human Judgement:

The large role of noise in error contradicts a commonly held belief that random errors do not matter, because they “cancel out.” This belief is wrong. If multiple shots are scattered around the target, it is unhelpful to say that, on average, they hit the bull’s-eye.

I like this line. Some radiologists, for example, over-call questionable findings while others are too cavalier and miss subtle features. They do not cancel out.

In Noise, Kahheman breaks noise down into three big categories: Level Noise, Pattern Noise, and Occasion Noise (each with its own causes and with its own mitigation strategies).

  1. Level noise: The deviation between a single judge from the average judger. For example, some teachers are tough graders.
  2. Pattern noise: The deviation of judges related to a unique or specific situation. For example, a teacher is generally an easy grader but really really likes Oxford commas and tends to grade harsher than average for students who fail to use them.
  3. Occasion noise: Variability related to random irrelevant/undesirable factors (weather, time of day, mood, recent performance of a local sport’s franchise). For example, a teacher grades harsher when finishing up their work from home.

Some doctors prescribe more antibiotics than others do. Level noise is the variability of the average judgments made by different individuals. The ambiguity of judgment scales is one of the sources of level noise. Words such as likely or numbers (e.g., “4 on a scale of 0 to 6”) mean different things to different people.

A massive problem, to be sure, and the reason why radiology trainees hate reading degenerative spine cases (no matter how you grade neural foraminal stenosis, it feels like you’re always “wrong”).

When there is noise, one physician may be clearly right and the other may be clearly wrong (and may suffer from some kind of bias). As might be expected, skill matters a lot. A study of pneumonia diagnoses by radiologists, for instance, found significant noise. Much of it came from differences in skill. More specifically, “variation in skill can explain 44% of the variation in diagnostic decisions,” suggesting that “policies that improve skill perform better than uniform decision guidelines.” Here as elsewhere, training and selection are evidently crucial to the reduction of error, and to the elimination of both noise and bias.

Algorithms are powerful, but for those that assume that checklists and knee-jerk medicine can provide equivalent outcomes, apparently not.

There is variability in radiologists’ judgments with respect to breast cancer from screening mammograms. A large study found that the range of false negatives among different radiologists varied from 0% (the radiologist was correct every time) to greater than 50% (the radiologist incorrectly identified the mammogram as normal more than half of the time). Similarly, false-positive rates ranged from less than 1% to 64% (meaning that nearly two-thirds of the time, the radiologist said the mammogram showed cancer when cancer was not present). False negatives and false positives, from different radiologists, ensure that there is noise.

The massive amount of noise in diagnostic medicine is one of several reasons why “AI” is so enticing. Essentially no one chooses their radiologists, and radiologists are often an out-of-sight/out-of-mind commodity. With our fee-for-service system combined with corporatized profit-seeking and a worsening radiologist shortage, it seems–at least anecdotally–that quality may be falling. These factors all combine to pave the way to make AI tools look even better in comparison.

Later, they go on:

Pattern noise also has a transient component, called occasion noise. We detect this kind of noise if a radiologist assigns different diagnoses to the same image on different days.

This definitely happens. Consistency is hard.

A separate study discusses another human foible, occasional noise related to the time of day:

But another study, not involving diagnosis, identifies a simple source of occasion noise in medicine—a finding worth bearing in mind for both patients and doctors. In short, doctors are significantly more likely to order cancer screenings early in the morning than late in the afternoon. In a large sample, the order rates of breast and colon screening tests were highest at 8 a.m., at 63.7%. They decreased throughout the morning to 48.7% at 11 a.m. They increased to 56.2% at noon—and then decreased to 47.8% at 5 p.m. It follows that patients with appointment times later in the day were less likely to receive guideline-recommended cancer screening.

How can we explain such findings? A possible answer is that physicians almost inevitably run behind in clinic after seeing patients with complex medical problems that require more than the usual twenty-minute slot. We already mentioned the role of stress and fatigue as triggers of occasion noise (see chapter 7), and these elements seem to be at work here. To keep up with their schedules, some doctors skip discussions about preventive health measures. Another illustration of the role of fatigue among clinicians is the lower rate of appropriate handwashing during the end of hospital shifts. (Handwashing turns out to be noisy, too.)

Taking a human factors engineering approach, we know that both patients and doctors will be better off in a system designed with human limitations in mind. For example, not just a deluge of interrupting EHR reminders to ignore, but a system that allows for the right kind of low-friction actionable prompts to be delivered at a useful time during a clinical encounter that is already scheduled in a way to allow for real-time documentation completion without running behind. Wouldn’t that be something?

Concerning metrics:

Focusing on only one of them might produce erroneous evaluations and have harmful incentive effects. The number of patients a doctor sees every day is an important driver of hospital productivity, for example, but you would not want physicians to focus single-mindedly on that indicator, much less to be evaluated and rewarded only on that basis.

See: Goodhart’s Law and patient satisfaction.

Discussion of job interviews and candidate selection has obvious parallels with the residency selection process:

If a candidate seems shy and reserved, for instance, the interviewer may want to ask tough questions about the candidate’s past experiences of working in teams but perhaps will neglect to ask the same questions of someone who seems cheerful and gregarious. The evidence collected about these two candidates will not be the same.

One study that tracked the behavior of interviewers who had formed a positive or negative initial impression from résumés and test scores found that initial impressions have a deep effect on the way the interview proceeds. Interviewers with positive first impressions, for instance, ask fewer questions and tend to “sell” the company to the candidate.

This is an incredibly on-point summary of how most institutions conduct interviews. Those candidates who are good on paper and not painfully awkward during the initial pleasantries basically get a pass. Even when given questions, those answers are often contextualized within the pre-formed opinions. This focus on “selling the program” would even be reasonable if the metrics and data that programs receive were actually helpful at predicting residency success.

Kahneman and his team offer a lot of advice on how to conduct better interviews in the book. Some of it I suspect is too inefficient and awkward for the residency process, but what a lot of programs do (subjectively grade an applicant on a few broad metrics during a committee meeting and then pretend the process is objective) is a bit of a farce.

Summary: highly recommended reading.

 

Incidental Pain and Suffering

From Matthew Davenport’s upcoming article in AJR, “Incidental Findings and Low-Value Care“:

It is increasingly recognized that incidental findings are incompletely understood, expensive, and surprisingly harmful. Rather than a benefit of imaging, they are usually a harm. They are not sought, the odds of them being important is low, and they create tremendous uncertainty and low-value care.

A good primer on an often unintuitive and yet incredibly important problem in medical diagnostics (both for radiologists and clinicians).

Death of the Noncompete?

Last week the FTC announced a proposed rule banning non-compete agreements. You can read the announcement here and the actual rule here. The rule would, if enacted, not just ban all non-competes going forward but nullify previous agreements as well. Non-competes are ubiquitous in medicine and a big factor locking doctors into their jobs, typically by preventing them from practicing in the same geographic region for a period of time after leaving their employer. And, for example, whenever a large organization like an academic medical center or a private equity company buys a practice or otherwise dominates a region, these non-competes form an effective moat against competition by preventing doctors from reorganizing after fleeing.

In some areas/fields, noncompetes are universal and have been functionally unavoidable. Many employers rely on lock-in to mitigate their bad culture and sleazy practices; shifting that power dynamic would I think change things very quickly.

It’s intuitive and straightforward how such a rule would affect employed physicians: you can just quit and hang up your shingle elsewhere. And yes, that means a clinician could join an academic practice for a few years, build up a patient panel, get more comfortable in their skin as an attending physician, and then leave and use that experience as a springboard to a new practice. This is, of course, part of the fear that led to non-competes in the first place. Employers put money into new hires between training, onboarding, early decreased efficiency, marketing, etc. Perhaps in a world without non-competes, employers will be less inclined to invest in their employees; that’s the typical business counterargument. The counter-counterargument also holds water: perhaps, if employers don’t invest in their employees, then their employees will leave. Value shouldn’t be a one-way street.

Too good to be true?

Several immediate reactions have been common. One, that somehow doctors will be exempted because woe is us. Two, that companies will use the magic of lawyers to get around the intent of the law. Three, that practice owners/shareholders (think partners in a large private practice) will be exempted because they are business owners and not employees. Four, that this will be litigated into oblivion.

The announcement had this to say:

Companies use noncompetes for workers across industries and job levels, from hairstylists and warehouse workers to doctors and business executives. In many cases, employers use their outsized bargaining power to coerce workers into signing these contracts. Noncompetes harm competition in U.S. labor markets by blocking workers from pursuing better opportunities and by preventing employers from hiring the best available talent.

So the FTC specifically includes doctors when they think of who this rule will affect.

The language of the rule itself also addresses a few of these concerns:

(1) Non-compete clause means a contractual term between an employer and a worker that prevents the worker from seeking or accepting employment with a person, or operating a business, after the conclusion of the worker’s employment with the employer.

(2) Functional test for whether a contractual term is a non-compete clause. The term non-compete clause includes a contractual term that is a de facto non-compete clause because it has the effect of prohibiting the worker from seeking or accepting employment with a person or operating a business after the conclusion of the worker’s employment with the employer.

So, in theory, clever machinations to functionally bind workers without the use of naughty catchphrases would still be against the law. How easy it would be to prove a functional non-compete in court, how expensive and stressful that process would be for an individual worker, and how aggressive companies will be in toeing the line remains to be seen. How desirable/how effective of a deterrent such schemes would be for employers depends on those answers.

There is an exception for business owners:

The requirements of this Part 910 shall not apply to a non-compete clause that is entered into by a person who is selling a business entity or otherwise disposing of all of the person’s ownership interest in the business entity, or by a person who is selling all or substantially all of a business entity’s operating assets, when the person restricted by the non-compete clause is a substantial owner of, or substantial member or substantial partner in, the business entity at the time the person enters into the non-compete clause.

The FTC defines “Substantial ownersubstantial member, and substantial partner” to “mean an owner, member, or partner holding at least a 25 percent ownership interest in a business entity.”

By that language, the ban would still apply to a physician owner in a practice of 5 or more people. Your average radiologist whose group sold to private equity, could, after the contract period, turn around and start working for other groups locally. They could, even, start a new group.

Well?

How is this likely to play out? I have no idea. In reviewing the media coverage, the overall consensus points towards the final rule being similar to the proposal, it not being stopped by congress (democrat-controlled senate), and then being litigated immediately. How long it takes to work its way through the courts and its eventual fate I don’t know. I’m sure plenty of lawyer and journalist ink will be spilled when the time comes to predict the outcome, but that is far outside my circle of competency.

For Radiology:

In radiology, the ability to do teleradiology work has taken some of the bite out of noncompetes, but this would still be a massive change for physicians in general. In particular, if the carve-out for owners/shareholders were to stay a similar size, the proposed rule provides a window into how a post-PE world might look for practices struggling after the sale.

No one has poured through every contract out there, but one of the common post-sale questions for the past few years has been: how can we get out of this? Common refrains: the things we were promised haven’t been provided, we can’t recruit, our rads are being poached to help elsewhere in the organizational umbrella, we can’t earn enough with the cut to make this sustainable. What recourse do the doctors who sold a practice have if things aren’t working out post-sale?

If this rule were to come to pass, there would be a light at the end of the tunnel. A failing group post-sale could run out the clock and conceivably form a new group to compete with the shell entity they’d leave behind (though presumably companies would still mitigate competition through non-solicitation agreements, for example). RadPartners and friends would still be buying the profits from your work and the goodwill of your relationships for several years, but the lack of a noncompete would make it impossible for them to guarantee their long-term stranglehold if/when their management fails. They’d have real skin in the game.

In practice, that could easily just hasten a lot more hospital-employed radiologists as institutions look to bring in rads and secure imaging services in this uncertain world. There are certainly groups out there that would rather work for the hospital they’ve been staffing for decades than the PE company they sold to. But even that trend could be temporary if a group of employed rads could then leave and form a group.

Boom.

The dynamism that such a rule enables is the real deal. The bargaining table permutations are infinite, and that’s exactly why the FTC wants to ban noncompetes.

WCICON23

Registering through the link here is also one of those effortless ways you can support this site while doing what you were going to do anyway.

I’m very much looking forward to speaking again this coming year at the Physician Wellness & Financial Literacy Conference (aka WCICON23), which will take place March 1-4 at the very nice JW Marriott Desert Ridge in Phoenix. If you have a CME fund to burn, I can’t think of anything else I’d rather spend it on.

 

 

Code CON23BW will net you $200 off an in-person registration through January 25.