How AI is reshaping malpractice carriers

This episode of Out-Of-Pocket is brought to you by…

After migrating to Whereby’s video call API, practice management platform Jane, saw 90% of video call sessions rated 4-5 stars for quality. Their team spends less time on maintenance and troubleshooting, and their clinicians spend more time on patients.

If your platform’s video quality is costing you support time or clinician trust, it’s worth switching. See why Whereby is a trusted solution for telehealth platforms.

—

Wanna get in front of the smartest readers in healthcare? Go somewhere else. Jk, we got a lot of sponsorship options for you. Let's chat.

Malpractice redux

Last week we talked about what malpractice was and how practices that deviate from the standard of care create new scenarios for malpractice.

Anticipating the hype sucklers frothing at the mouth to ask about AI, I already had this part two written.

I think AI is going to change malpractice in a few interesting ways.

1. Ambient scribing has ground truth on what happened
2. AI clinical decision support is making judgment calls
3. Autonomous agents create liability at scale
4. AI becoming the standard of care raises questions on hospital resourcing

You coming to Ship It? Our Software Engineering Conference?

Speaking of AI changing things…you coming to Ship It? Our healthcare software engineering conference?

Applications are due in 10 days. If you’re a software engineer in healthcare and want to actually hear from other people building and managing engineering teams and know what a BAA is…you should come to this. We also put a lot of work into the nautical theme, please look.

Or…maybe you’re thinking about taking our Healthcare 101 course, where I’ll teach you how US healthcare actually works. You could try to ChatGPT it all of it, but you don’t even know what you don’t know 🙂.

I’ll give you the basics of how money flows, who the major players are, and other things that I nearly lost my (social) life learning over 10 years. Specifically meant for people building in healthcare that want to get right to the point, not the academic fluff.

Ambient scribing has ground truth on what happened

Scribes create a much clearer picture of what actually happened during a patient encounter. These transcripts can resolve he-said-she-said disputes and also show a jury of peers what the exchange looks like from the patient POV. My guess is that for most doctors this will actually help them in a trial.

The other question here is whether doctors would practice differently knowing it’s being recorded. Some doctors have told me they might not be as “honest” with patients if they knew they were being recorded. Too bad I recorded that convo and snitched on all of them. But there’s also an argument that choosing to be recorded itself is a self-selection mechanism for doctors less likely to DO indefensible things.

IMO I think malpractice carriers will start figuring out how to include scribes as part of their offering, even if it means it’s because “safer” doctors choose that option. In the same way Metromile puts a device in your car to track your driving and give you better rates, I can see a world where malpractice carriers partner with a scribe to offer lower rates to people that opt-in.

Is AI clinical decision support making judgments?

Clinicians make mistakes - it happens. Right now, all of these AI copilot companies use a “human-in-the-loop” approach where they don’t actually diagnose, they just…sorta make a suggestion. But what if the suggestion is wrong and the doc still goes with it? What if that suggestion is wrong for every doc that’s using the tool?

This introduces questions:

1. As doctors become reliant on AI assistance, who owns the liability if these issues happen?
2. Who’s responsible for monitoring changes in the software? How does this change diligence and procurement?
3. Who's responsible when the marketing of a tool implies clinical reliability but the legal structure shifts clinical responsibility to the doctor?

The 2023 case Sampson v. HeartWise Health Systems is an interesting case here. A patient went to a clinic and got the HeartWise-licensed cardiovascular screening program. It said everything was normal but six weeks later he died from an arrhythmia caused by hypertrophic cardiomyopathy. He went specifically because his dad had just died from a congenital heart defect, which is really sad.

A HeartWise rep said no independent doctor reviewed the software-generated results before they went back to the patient. The Alabama Supreme Court said the physicians should have independently scrutinized the software’s output rather than just passing along the report. Heartwise was left off the hook for negligence.

However, though HeartWise argued it wasn’t practicing medicine and didn’t exercise clinical judgment over patients the court did say Heartwise’s company’s marketing implied a level of diagnostic reliability that patients relied on. So it did allow some of the “fraud” claims to survive against Heartwise.

There’s an interesting tension popping up around how the tools are being marketed vs. who owns the liability. Tools want to market clinical value without accepting clinical responsibility, and physicians are increasingly deferring to algorithmic outputs they may not fully understand.

One reason this matters is because companies are using the human-in-the-loop approach to get scale across many providers and used on many patients. When a clinician makes a mistake, it affects one patient. If a flawed AI clinical decision tool is deployed across a health system then a single error propagates through thousands of decisions before anyone catches it. So the aggregate harm can dwarf anything a single physician could cause, which increases the potential damage.

Is malpractice the right liability framework for this? It’s worth noting that the Sampson case wasn’t even a malpractice suit, it was a wrongful death action. Part of the reason is that HeartWise didn’t qualify as a healthcare provider under Alabama’s medical liability act, so the existing malpractice framework literally didn’t have a slot for the software company.

That gap is only going to get wider as AI tools become more embedded in clinical workflows. If the tool vendor isn’t a provider and can’t be reached through malpractice, then maybe the answer is enterprise liability where the health system holds the bag for the full stack of tools and people it deploys. They might require AI vendors to carry their own clinical liability coverage.

But when an AI tool functions like a diagnostic device deployed at scale by a manufacturer, the FDA’s products liability framework starts lookin’ pretttttty reasonable. This looks more like product liability. In that case the manufacturer bears responsibility if a cleared product is used properly and has issues, but liability shifts to the physician if they use it differently. But then the products will have to go through the FDA process so…choose your battles.

Autonomous patient-facing agents creates liability at scale

What if we stop using humans in the loop entirely, and we start using autonomous patient-facing agents that can do clinical things. Doctronic’s Utah pilot, launched in January, authorized AI to autonomously renew prescriptions with no physician in the loop at $4 per refill. To make it work with malpractice, Doctronic obtained a custom policy from Beazley explicitly covering the AI’s decisions.

In the future, malpractice carriers will have to do their own evaluation and adversarial testing to see if a physician’s agents are up to snuff. You’re seeing this outside of healthcare, with AIUC providing insurance to ElevenLabs Voice Agents after putting them through the ringer. This becomes a new underwriting muscle malpractice carriers need.

But it also means that the agents will need some level of explainability to a jury. Is the expert witness someone that understands agents or clinical practice? Does the agent need to explain itself, or have a level of auditability under the hood? Have you tried explaining to your parents what an “agent” is without sounding like a buffoon?

Will AI become the standard of care?

When will standard of care itself include whether you use an AI tool or not?

Today there are certain procedures that would be considered malpractice if you didn’t use an ultrasound machine. AI assistance will probably be viewed similarly at some point for certain workflows (e.g. doing a second read for radiology).

Will this punish lower-resourced health systems that don’t keep their software up to date? If it’s non-emergent and standard of care is to do a procedure with the ultrasound, you’re supposed to transfer the patient if you don’t have the ultrasound. Maybe the same thing will happen here, or a forward deployed engineer is going to need to go to the sticks and figure out how to run an AI tool in MS-DOS.

As an anecdote, one person told me their hospital adopted an AI solution explicitly because they lost a huge lawsuit over a missed follow-up. A patient had a suspected issue and was told to come back to check. The patient cancelled twice and ended up requiring emergency surgery due to the issue. The jury ruled the hospital didn’t adequately emphasize urgency.

So now their hospital spends a lot of money on an AI that flags incidental findings and ensures patients get called for follow-up as a “cover your ass”. The relative cost is pegged against the lawsuit payout. Will hospitals that can’t afford that AI be in a more vulnerable position?

Malpractice and the future

Malpractice insurance is old-school as hell. Honestly it might even be the very first school.

The core of malpractice insurance is built around a few concepts that are changing wildly:

• There is 1 doctor with 1 patient and 1 encounter at a time. Now we’re moving to continuous monitoring and tools that can see many patients at once.
• There is a standard of care and other doctors are the best to judge that. But what if a non-doctor is delivering care + the standard of care starts fracturing.
• Underwriting that doctor’s risk uses proxy measures to figure it out (specialty, geography, previous suits). But we now have ground truth in the form of ambient scribing + underwriting requires testing the software as much as the people.

I think you’ll start seeing new models of malpractice emerge. Some companies are trying to bring technology into the space. Indigo raised $50 million in January but seems to focus more on using AI for the documentation gathering/quoting piece though that feels uninspired so I’m hoping they’re dreaming big. There will be more activity here, and I think malpractice carriers will start offering their own vetted technology as part of the bundle itself.

We’ll also probably see new forms of coverage that are not just claims-based and occurrence based. Maybe you’ll see more tiers based on what types of technology you have available, or bundling with other insurance like enterprise liability for the tools used. Or insurance that’s specific to non-standard of care protocols

The question is whether existing malpractice carriers will adapt to this new landscape, or a new carrier needs to come in and be built from the ground up.

Thinkboi out,

Nikhil aka. “It’s not a mistake, it’s a reimagining of the diagnosis”

This was written with help from Lauren Risenhoover. Thanks to Aqil Rashid and Dr. Eric Funk for reading drafts.

Twitter: @nikillinit

IG: @outofpockethealth

Other posts: outofpocket.health/posts

{{sub-form}}

If you're enjoying the newsletter, do me a solid and shoot this over to a friend or healthcare slack channel and tell them to sign up. The line between unemployment and founder of a startup is traction and whether your parents believe you have a job.