A threat actor handed an AI agent a set of stolen credentials and one instruction: go. It ran the entire attack alone, roughly 600 coordinated steps, in real time.
A year ago, this same panel talked about AI as an emerging threat: useful for planning, still mostly speculative. Phishing volume was rising, voice cloning had just appeared as a documented risk, and nation-state actors were beginning to experiment. That conversation is over. As Clearwater's Dave Bailey opened the third session of the AI Healthcare Summit, "none of this is speculative anymore. It's confirmed that named actors are deploying AI in operations."
AI-driven business email compromise now accounts for most high-value fraud losses. Voice cloning has crossed what Bailey called the indistinguishable threshold: staff generally cannot tell a cloned voice from the real one by ear. And in testing of one clinical AI tool, prompt injection succeeded 94% of the time. "This isn't a trend line," Bailey said. "There's been a phase change from this year to last year. Everything that was treated as a future risk twelve months ago is now documented operational reality."
One confirmed attack, almost no human involved
Steve Akers, Clearwater's Corporate CISO, opened with the clearest evidence that AI-run attacks have moved from theory to practice. Researchers recently documented what was reported as a fully autonomous AI-driven intrusion. Akers was careful about the word autonomous: a human threat actor pointed the AI at a specific victim, supplied working credentials, an IP address, and a domain, and set up the command-and-control infrastructure. From there, the agent took over.
"The agent completed the entire intrusion chain, from initial access to database extortion," Akers said. "It executed something around 600 coordinated actions to achieve its goal." It harvested credentials, established persistence, mapped internal resources, encrypted roughly 1,300 records, deleted the original tables, and left a Bitcoin ransom demand. At one point, adapting in real time to a storage server's defenses, it went from a failed login to a working bypass in 31 seconds.
If the hair is not standing up on the back of your neck, then you must not be paying attention, because it is mine, and I live in this every day. Steve Akers, Corporate CISO, Clearwater
Frontier AI model day, and why Washington moved in three weeks
Bailey framed the moment with a term he called Frontier AI Model Day: the point where the vulnerability-discovery capability of the most advanced AI models reaches the adversary groups already targeting healthcare, and they start using it at scale. He cited three data points behind that claim: a frontier model used in an authorized government red-team exercise found vulnerabilities across classified systems in hours, not weeks; a distillation campaign ran 28.8 million exchanges through roughly 25,000 fraudulent accounts to extract a frontier model's vulnerability-hunting capability; and the average time from initial breach to lateral movement is now less than half of a World Cup soccer match, a fraction of what most incident response plans were built to assume.
That framing lines up with what the Five Eyes intelligence alliance (the US, UK, Canada, Australia, and New Zealand) said publicly this June: frontier AI's cyber-offense capability is months away, not years, a rare joint statement from five national intelligence agencies on the record together.
Created a formal review process for covered frontier models, expanded CISA's AI defense tools to critical infrastructure including rural hospitals, and directed DOJ to prioritize AI-enabled cybercrime enforcement.
Two advanced Anthropic models were suspended worldwide under a Commerce Department export control order, triggered by a reported jailbreak. The first real-world, government-mandated shutdown of a deployed frontier model.
Commerce restored access. From policy statement to enforcement to reversal: three weeks, start to finish.
The lesson for governance programs isn't which model or which order, Bailey argued. It's that frontier AI is now squarely a matter of national security policy, with direct implications for AI vendor dependency, contract risk, and continuity planning. "Could you imagine if a regulator suspended your AI vendor overnight? If that isn't already a scenario in your continuity plan, you should start thinking about it right now."
A $200-a-month attacker toolkit
What's driving the shift, according to Akers, is price. The tools that once required real technical skill are now rented, packaged, and supported like any other SaaS product.
A targeted healthcare campaign, Akers said, can be assembled for under $200 a month. "What's happening is that you're really getting the best players' capability for minor-league money. You're getting the major-league fastball, but paying a minor-league price." He expects deepfakes specifically to keep gaining believability, because voice and video break the trust barrier faster than text ever could: "I know this voice, it's Dave, so I immediately take down my guard." And because each AI-generated phishing email costs the attacker pennies or less to produce, the economics favor volume in a way that traditional phishing defenses were never built to absorb.
What the SOC is actually seeing
Justin Sun, Clearwater's Director of Security Operations, described a pattern showing up across multiple client environments: business email compromise attempts that were unusually well-crafted, missing the typos and awkward grammar that used to be an easy tell. Within the same campaign, redirect links and attachment names were changing dynamically, which broke static, signature-based detection. "It's almost like you have to respect the game," Sun said. "They're using AI to make it harder for us to detect them, so we had to use AI in our environment to gather those indicators quickly and shift toward behavioral detection instead."
Voice cloning hasn't hit Sun's SOC directly yet, but he isn't waiting for it. Generating a convincing clone now takes less than 10 seconds of a person's recorded voice, which is why the FBI's IC3 issued an advisory this year flagging AI voice impersonation of senior officials as an active, not theoretical, threat. Sun's guidance is simple: never verify a wire transfer or password reset request over the same channel it arrived on. If it comes by phone, confirm over Teams or email instead, and don't treat a familiar-sounding voice or a recognized phone number as proof of identity; both can be spoofed.
The offensive view: what pen testing looks like now
Philip Burnham, a penetration tester on Clearwater's technical testing team, described AI as a force multiplier on both sides of the table. For attackers, the skill floor has effectively dropped to zero: reconnaissance, org-chart mapping, and target profiling that used to take real expertise now happen in minutes. For his own team, AI still supports human-led testing rather than replacing it, speeding up social engineering campaign development, validating findings, and helping translate technical results into plain-language impact analysis for a C-suite audience.
Burnham's biggest concern isn't the AI model itself; it's what surrounds it. "If your LLM or AI is 100% prompt injection resilient, but the backend API is open and misconfigured, attackers can still get in and do what they like." He called shadow AI "the hugest inventory gap right now," pointing out that every Windows laptop ships with Copilot enabled by default, often with access to documents, PHI, and PII that security teams never explicitly reviewed or approved.
You can't govern, test, or defend what you haven't cataloged. Philip Burnham, Penetration Tester, Technical Testing Services, Clearwater
Governance is still the biggest gap
Akers cited industry breach data putting a number on what happens when AI runs ungoverned. Across breached organizations, a majority either lacked AI governance entirely or were still developing it, and organizations that experienced a shadow AI-related security incident saw meaningfully more personally identifiable information and intellectual property disclosed than the average breach. Within healthcare specifically, roughly 40% of staff were aware that colleagues were using unauthorized AI tools for clinical notes and patient communications, a pattern likely already leaking protected health information. Industry data from IBM's Cost of a Data Breach research tells a similar story: breaches involving shadow AI carry a real cost premium and take longer to detect than the average incident.
"AI risk has to be a named risk category," Akers said. "You've got to update your governance and compliance posture to match that." Bailey put it more bluntly: "You can't protect what you don't know you have. AI can't be a nebulous, cloud thing anymore. It has to be part of the known inventory."
What to do in the next 90 days
Asked what a CISO should prioritize immediately, each panelist gave a specific, actionable answer.
Steve Akers: treat AI risk as its own named category on the enterprise risk register, not a subset of general IT risk.
Justin Sun: static, signature-based detection can't keep up with AI-generated, constantly mutating attachments and redirects. Baseline what normal looks like instead.
Justin Sun: require a second communication channel to verify any wire transfer, password reset, or urgent request, regardless of how convincing the voice sounds.
Philip Burnham: map read and write access for every AI tool, then pen test it, including API endpoints and prompt injection scope.
Bailey's closing advice was aimed above the security team entirely: get AI risk in front of senior leadership and the board now, if it isn't there already. "We're soon going to be at a pace where the way we run security and compliance programs today just isn't going to be good enough. We're going to have to adapt quicker, and that starts with the board understanding what the risk actually is."
On the question of shadow AI specifically, the panel's advice converged on enablement over prohibition. Akers argued for offering staff a small number of sanctioned, well-governed tools rather than trying to block AI use outright, since most employees adopting unauthorized tools are trying to solve a real problem, not act maliciously. Sun added the operational piece: pairing an approved-tool list with automatic detection that blocks sensitive data, like Social Security numbers or PHI, from ever reaching an ungoverned AI model in the first place. Bailey tied both back to risk tolerance: authorize quickly for low-stakes use cases, and slow down deliberately whenever regulated or crown-jewel data is involved.
Questions this session answers
Is AI already being used to run full cyberattacks in healthcare?
Yes. Clearwater's security team has reviewed a documented case where a threat actor supplied an AI agent with stolen credentials and target details, then let it run. The agent completed the entire intrusion chain on its own, from initial access through credential harvesting, persistence, internal mapping, encryption, and a ransom note, executing roughly 600 coordinated actions with minimal further human input.
What is prompt injection and why is it a growing risk for clinical AI tools?
Prompt injection is an attack that manipulates an AI system's instructions to make it act outside its intended purpose. In testing of one clinical AI tool discussed during the session, prompt injection attempts succeeded 94% of the time. Combined with an exposed or misconfigured API, a prompt-injection-resilient model does not protect an organization if the backend around it is left open.
How is voice cloning changing business email compromise and wire fraud risk?
Fewer than 10 seconds of a person's recorded voice is enough to generate a convincing clone today, and staff generally cannot tell a cloned voice from the real one by ear. That breaks verbal verification controls built around human judgment, which is why wire transfer and password reset requests need confirmation through a second communication channel, not just a familiar-sounding voice or a recognized phone number.
What is shadow AI and why is it a governance gap for healthcare security teams?
Shadow AI is AI tool use that hasn't been reviewed, cataloged, or governed by an organization's security and compliance functions, often because tools like Copilot ship enabled by default or employees adopt public AI tools on their own. Security teams cannot test, govern, or defend a system they haven't inventoried, and industry data shows organizations with shadow AI incidents see materially higher rates of PII and IP exposure.
What should a healthcare CISO prioritize in the next 90 days for AI-related risk?
Name AI risk as its own category on the enterprise risk register rather than folding it into general IT risk. Inventory every AI-enabled tool and map its read and write access, shift detection from static signatures to behavioral analytics, rebuild incident response playbooks to account for voice-cloning scenarios, and put AI risk on the board's agenda if it isn't there already.
Do you know everywhere AI already touches your environment?
Clearwater helps healthcare organizations inventory AI risk, close the shadow AI gap, and build governance that keeps pace with how fast attackers are moving.
Explore AI Risk & Governance ServicesSession 4: From Commitment to Execution: Operationalizing AI Governance in Healthcare
Dave Bailey, VP of Consulting Solutions & Strategy at Clearwater, sits down with Cate Ciccolone, Associate Director of Commercial Health IT Advisory at Guidehouse, to examine what it actually takes to operationalize AI governance: standing up governance structures, building and managing AI inventories, prioritizing risk, and navigating the organizational realities that shape implementation.
Read Session 4 →See the full AI Healthcare Summit series & session replays →


