Your EHR's New AI Agents: How to Evaluate Them

By Jude Lee · · News

Office manager and front-desk staff reviewing practice management software on a monitor at a clinic reception desk

What the recent announcements signal — and what they don’t

August 2026 produced a cluster of releases that all point the same direction: systems of record are becoming systems of action. Fierce Healthcare reported Epic expanding into an agent platform with predictions drawn from its Cosmos dataset and deeper workflow automation. Orthodontic Products reported Planet DDS adding AI agents and modular suites on the dental side. HIT Consultant covered Assort Health shipping an agent aimed at end-to-end specialty referral conversion. If a link has moved, search each outlet’s archive by publication month — the underlying vendor press materials are the primary source, and worth reading before the trade summary.

What none of these announcements tell you is whether any of it will land in your practice, on your edition of the software, at a price a six-provider group can absorb. Enterprise agent platforms tend to reach large health systems first. Independent practices usually get the feature a release cycle or two later, sometimes in a thinner form, sometimes bundled into a tier upgrade. Treat the news as a signal about direction — not as a delivery date.

The three layers an agent can live in

Almost every AI automation decision in 2026 collapses into a choice among three architectures. They aren’t mutually exclusive, and most practices end up with two of them.

Layer 1 — Native to your PM/EHR. The agent ships inside Dentrix, athenahealth, eClinicalWorks, Epic, Open Dental, a behavioral-health EHR, whatever you run. It already sees your schedule, ledger, and chart. Your existing business associate agreement usually covers it, though confirm that in writing rather than assume.

Layer 2 — A point solution. A specialist vendor does one job deeply: voice agents for inbound calls, eligibility, claim scrubbing, referral intake, scribing. It integrates with your PM/EHR through an API or an interface engine. You sign a new BAA and take on a new integration.

Layer 3 — A governed layer you own. An AI assistant (Claude, or another model) connected to your systems through MCP — the Model Context Protocol, an open standard for exposing tools and data to an AI under explicit permissions. You define which functions the agent can call, what data comes back, and where a human has to approve. We walk through this build in the HIPAA-aware guide to connecting Claude to your EHR via MCP.

Native PM/EHR agent
Fastest to turn on. No integration project. Covered by an existing vendor relationship. But: it only does what the vendor built, only sees data inside that system, and you can’t reshape the workflow. If your billing lives elsewhere, the agent is half-blind.
Your own MCP layer
Spans systems — PM, clearinghouse, email, e-fax, spreadsheets. You control scope, logging, and approval gates. But: it’s a real build with real maintenance, it needs someone accountable for it, and it’s overkill if the native feature already covers most of the job.

The evaluation test to run before you enable anything

  1. Name the job, not the technology

    Write one sentence: an agent should take X input and produce Y outcome, ending in Z human decision. If you can’t write that sentence, you’re buying a demo, not a workflow. Our job-by-job buyer’s map is a useful starting inventory.

  2. Ask what the agent can actually *do*, not what it can answer

    There’s a wide gap between a chatbot that drafts a message and an agent that writes to your schedule, posts a note, or submits a claim. Ask for the exact list of write actions and which ones can be disabled.

  3. Find the human-in-the-loop seams

    Which steps auto-execute, which queue for approval, and can you change that per action? Anything touching clinical content, money, or a patient’s coverage should default to review. We break down where those handoffs belong in designing escalation for front-office agents.

  4. Read the data terms before the feature list

    Under HIPAA, a vendor handling PHI on your behalf is a business associate and needs a BAA — see the HHS Office for Civil Rights guidance on business associates for the governing text. Then go further: is PHI used to train models, where is it processed, how long is it retained, and can you get an audit log of every agent action?

  5. Demand a failure story

    Ask the vendor to describe what happens when the agent is wrong. If the answer is that it’s very accurate, that’s not an answer. You want: how errors surface, who gets notified, and how you roll back an action.

An agent that can only see one system will confidently give you a one-system answer to a two-system problem.

Where PHI rules genuinely narrow the field

This is the constraint that quietly decides most architecture questions. The HIPAA Privacy Rule’s minimum necessary standard means an agent should receive the smallest slice of PHI needed for the task — not a whole chart because that was the easiest API call to wire up. That principle is easier to honor when you define the tool surface, which is one of the honest advantages of a custom MCP server: you decide that the eligibility agent gets member ID, date of birth, and plan, and nothing else.

It also rules things out. A general-purpose consumer AI account with no BAA is not a place to paste patient information — we covered that in whether ChatGPT is HIPAA compliant for practices. Have your compliance officer or counsel confirm how these rules apply to your specific setup, against the primary HHS text, before any PHI moves.

Modeling the value without making numbers up

Don’t accept a vendor’s ROI slide. Build your own with four inputs you can actually measure in your practice.

tasks/week × minutes each
Baseline manual time — measure it for one week before you buy
hours saved × loaded hourly rate
Labor value recovered (use real payroll cost, not salary alone)
visits recovered × avg reimbursement
Captured revenue from fewer dropped referrals or no-shows
license + integration + oversight hours
Total cost of ownership — the term most models omit

The term practices most often forget is oversight. Every approval queue costs staff attention. If an agent drafts 40 items a week and someone reviews each in two minutes, that’s real time you must subtract before claiming a net gain. Plug in your own numbers; if the math only works when you assume zero review time, the math doesn’t work.

Training is the line item nobody budgets

Training is the step practices most often tell us they skipped: the tool arrives, and nobody owns it. Budget for a named internal owner, a written scope document for each agent, a monthly sample review of what the agent produced, and a short escalation script for staff. Treat the rollout like a hire — scope, shadowing, supervised work, then loosened supervision — which is the framing in onboarding an AI agent like a new front-office hire.

An opinionated default for 2026

My heuristic, offered as opinion rather than measured finding: enable the native agent features your PM/EHR already includes for high-volume, low-stakes work — reminders, recall outreach, drafting routine responses — because the integration cost is near zero and the BAA already exists. Reach for a point solution when one job is both painful and clearly bounded, like inbound call handling or eligibility. Build your own MCP layer only when the workflow crosses systems your vendor will never connect, and you have someone who can own it. If you’re weighing that decision seriously, the tradeoffs are laid out in custom vs off-the-shelf healthcare automation.

And sometimes the right answer is a rule, not an agent. A deterministic reminder cadence beats a language model at sending a text on day three. Save the agents for work that requires judgment across messy inputs — and keep a human on the decisions that carry clinical or financial weight.

Not sure where to start?

Get a free automation audit: we map your scheduling, intake, insurance, billing, and patient communication and show you what's worth automating — before you spend a dollar.

Get a free automation audit

Related guides