An attempt to model the ROI of using narrow AI tools like PE and brain bleed detection, from “Efficiency and Financial Gains From Artificial Intelligence Algorithm Implementation” in JACR:
The pulmonary embolism triage algorithm also reduced interpretation time, but the magnitude of savings was insufficient to offset AI-related costs. The observed time savings of 0.83 min per case produced a negative contribution margin change from 6.0% to −10.1% and a negative return on invested capital of −76.6% (Table 2). In sensitivity analysis, the algorithm would require a total reduction in interpretation time of approximately 3.55 min per case, or a decrease in annual AI cost from $200,000 to approximately $47,000, to become financially favorable relative to baseline. These results highlight that even measurable efficiency gains may not be sufficient when algorithm costs are high, case volumes are limited or baseline interpretation times are already relatively efficient.
The actual modeling the authors used contains many assumptions that may or may not generalize, but the broader point they make is that we should be able to measure whether any tool is worth it. Both in and out of medicine, many AI projects are outright failures. (The brain bleed tool they tested was worth it, in their analysis.)
A generally insufficiently addressed question for all AI-makes-you-more-efficient discussions is what is the leakage of time saved? What fraction of that expensive deep breath gets applied to the next case? Is the increase in productivity really just taking that time and cranking the hamster wheel perfectly to add in the exact amount of work to fill that gap? If so, does that scale as more and more tools are added? Is that effect durable over time, or do we partially revert after the Hawthorne effect wears off? Do we enjoy our tool-assisted work more or less, and how will that change over time?
A broader financial question for these types of narrow, mission-critical tasks is what parts of the task/job are expensive and/or inefficient? Doing the work or verifying the work or both? AI might help you avoid some mistakes as a second reader, but that’s not the economic model anyone is interested in.