Whose Risk Is It Anyway? Building the AI Accountability Model

Healthcare has crossed a threshold. AI isn't sitting in a research lab anymore. It's writing clinical summaries, sitting inside documentation workflows, running decision support, fielding patient messages, and quietly optimizing the revenue cycle. More than 80% of physicians now use AI professionally, according to the American Medical Association, up from a fraction of that just two years ago. The tools got smart fast. The org charts didn't keep pace.

That's the gap Clearwater's Krissy Safi opened with when she sat down with Chris Cuellar, Chief Compliance Officer at Elevate ENT Partners, and Blaine Hebert, CISO at Onvida Health, for the first session of Clearwater's AI Healthcare Summit. Compliance sees privacy and billing exposure. Security sees shadow AI and vendor access. Clinical leadership sees patient safety and workflow disruption. Three departments, three risk registers, and when something actually breaks, an organization needs one accountability model, not three parallel stories about whose job it was to catch it.

The accountability vacuum is real, and it's not going away

Ask Chris Cuellar where AI risk lives at Elevate ENT, and he doesn't pretend there's a tidy org chart answer. "It lives everywhere you want it to live, but it's going to keep evolving," he said. The firm assigns an operational champion to every AI project and pulls in the relevant subject-matter experts (compliance, IT security, clinical leadership) so ownership is shared rather than orphaned. Blaine Hebert's answer was more blunt: it depends entirely on where an organization decides to put it, because most health systems, Onvida Health included, don't have a chief AI officer or chief risk officer to hand it to by default.

What both agreed on is that the ambiguity has a name. Cuellar called it "the accountability vacuum": the space that opens up when technology accelerates faster than governance can formalize around it. "The technology has grown at such a rapid pace that people are just trying to keep up," he said. "You get back to square one, and then it's pushed forward again."

I think the question to ask at the end of the day is: who owns enterprise risk? A lot of fingers can point: well, you're the co-chair of the AI subcommittee, but I don't own risk as a CISO for the organization. Leadership does. Blaine Hebert, VP & CISO, Onvida Health

That distinction matters more than it sounds. A CISO or compliance officer can build the scoring matrix, staff the subcommittee, and make a recommendation. But the decision to accept residual risk on behalf of the organization sits with the CEO, the COO, or the board. Hebert put it plainly: a RACI chart can list a name next to every AI initiative, but "when things go upside down, that's when you really find out who's going to take charge."

Where compliance ends and security begins

Krissy Safi pushed both leaders on where their mandates actually start and stop across the AI lifecycle: procurement, deployment, ongoing monitoring. For Cuellar, compliance touches all three. Procurement means steering physician-driven vendor interest (the scribe tool a doctor heard about from a friend at a social event) through governance instead of letting a dozen markets run parallel pilots of the same product. Ongoing monitoring means staying current on a landscape that "is so different than it was two years ago, and it's going to be so different two years from now," including hallucinations, algorithmic bias, and the black-box problem of not knowing exactly how a model reached its answer.

For Hebert, the overlap with compliance is constant and deliberate. His team and Cuellar's meet every other week to work through risk management, business impact analysis, privacy assessments, and penetration testing, now with an AI lens layered on top of standard IT security review. The division, when it exists, is mostly technical depth: Hebert's team gets into architecture and access controls; Cuellar's gets into contract terms, disclosure obligations, and patient advocacy.

Freedom of contract is freedom from contract. If we don't need an AI tool, let's walk away. There are plenty of other fires to put out across the organization. Chris Cuellar, VP & Chief Compliance Officer, Elevate ENT Partners

Third-party AI risk is vendor risk, sharpened

Neither leader treats AI vendor risk as an entirely separate discipline from standard third-party risk management; they treat it as the same discipline asked harder questions. Hebert's team starts at the contract's fine print: if AI language shows up, general counsel and privacy get pulled in immediately to understand how a vendor is harvesting data and whether a business associate agreement actually limits that exposure.

Both flagged a problem that's easy to miss until it costs real money: scope creep after signature. A vendor rolls AI into a platform a year into the contract, sometimes without notifying the client, sometimes rebranding a modest bot as a premium feature at a steep markup. "You're going from a Chevy to a Cadillac, but they're going to charge you a Lamborghini price," Cuellar said. Hebert's answer is procedural: if a vendor introduces AI functionality post-signature, that should trigger an addendum to the third-party risk assessment and a fresh look at whether the original contract required notice in the first place. Frameworks like the NIST AI Risk Management Framework give organizations a structured, vendor-agnostic way to ask those questions consistently.

Then there's the risk nobody puts in a vendor contract at all: shadow AI. Staff using unauthorized AI tools to save time, often with PHI or proprietary data pasted directly into a public model with no business associate agreement in place. It's a growing concern documented across the industry, and Hebert doesn't pretend it can be policed away.

People do what they do. If you don't give them a viable, sanctioned option, they're going to find the workaround. And if you don't have guardrails to keep PHI and proprietary information out of public AI tools, go take a look at that now. Blaine Hebert, VP & CISO, Onvida Health

A minimum viable governance model, for organizations that can't hire their way out

Not every organization has HCA-scale resources to throw at AI governance. Cuellar and Hebert both offered a version of the same floor: a charter, a written AI policy, and a subcommittee with real authority to recommend a yes or a no, reporting to someone senior enough to escalate when a decision is bigger than the room can handle.

Hebert's team runs a simple scoring matrix: a 7 out of 10 moves forward, a 3 gets a recommended no-go. It's formalized enough to be defensible, light enough not to bottleneck the organization. Resources like CHIME and the NIST framework above are both worth a look for organizations building this structure from scratch. For smaller and mid-sized systems without budget for a large AI team, Cuellar's advice was to get creative with staffing: short-term contractors or AI consultants for a 30- to 90-day evaluation window, rather than permanent headcount for a landscape that will look different in a year.

Start with ownership, not perfection

Asked what they'd have put in place sooner, both leaders gave the same answer before adding their own twist. Hebert: stand up the governance structure and the AI policy first, regardless of how robust it needs to eventually become, and don't let "the shiny thing" (his phrase for a flashy new AI product) skip the due diligence. Cuellar: the accountability question comes first, but so does the reason the organization is doing any of this in the first place.

At the end of the day, all these AI tools exist to drive efficiency: to automate lower-level work so people can do higher-order work. If a project isn't making things simpler or more patient-centric, and it's causing this much consternation about risk, maybe it's not the fight worth having right now. Chris Cuellar, VP & Chief Compliance Officer, Elevate ENT Partners

Neither leader treats governance as a brake pedal for its own sake. It's the thing that lets an organization say yes to AI with confidence instead of hoping nothing goes wrong. As Clearwater's Krissy Safi framed it, most healthcare organizations still can't point to a single owner of AI risk, and that's precisely the gap a formal accountability model is built to close.

Questions this session answers

See the full AI Healthcare Summit series & session replays →