Key takeaways
- Every AI tool that talks to patients will make mistakes. The practice's job is to find them quickly, grade them and respond in proportion.
- Grade errors by harm, not embarrassment: a wrong clinical route is urgent, a wrong parking answer is not.
- Tell affected patients promptly, plainly and with the fix, and never blame the software.
- An error that sends patient information to the wrong person may be a HIPAA breach with notification deadlines of up to 60 days[1].
Errors are certain; slow detection is optional
AI output already reaches patients in many practices. In the AMA's 2026 survey, 28% of physicians said they use AI for documentation such as billing codes, charts or visit notes[2], 19% to draft responses to patient portal messages[2] and 18% for translation[2]. Add front office voice and text tools and the number of AI-generated words patients see each week becomes large.
Any system producing that much language will sometimes be wrong. It will state an outdated hour, route a call to the wrong team, mishear a date of birth or, in a poorly configured tool, say something it should not have said at all. Some errors come from the AI misunderstanding; many more come from the information it was given being out of date. Either way, the practice owns the result. What separates a safe deployment from a risky one is not whether errors happen but how quickly they are found and how well the practice responds.
Grade every error by potential harm
| Grade | Examples | Response target | Who leads |
|---|---|---|---|
| 1: Safety | Emergency language not escalated; clinical advice given; urgent request captured as routine | Same day, ideally within hours | Clinical lead |
| 2: Privacy | Information disclosed to the wrong person; message sent to wrong number | Same day; start incident assessment | Privacy officer |
| 3: Access | Wrong appointment booked or cancelled; request lost; patient told wrong availability | Within one business day | Front desk lead |
| 4: Information | Wrong hours, address, accepted plan or policy given | Fix answer same day; contact patient if they may act on it | Answer owner |
| 5: Experience | Awkward wording, repeated questions, long greeting | Next weekly review | Pilot or AI owner |
Grading on potential harm matters. A missed emergency phrase that happened to involve a caller who was fine is still a grade 1 error, because the same failure on another call could be serious. Grading by outcome teaches the team to relax about near misses, which is exactly backwards.
How to find errors before patients complain
- Daily sample: read or listen to a set number of conversations, weighted toward calls that ended in a transfer, a hang-up or a request for a person.
- Keyword sweep: search transcripts each day for symptom words, medication names and words like wrong, mistake or that's not right.
- Outcome checks: compare automated bookings and cancellations against the schedule each morning.
- Staff reports: a one-minute form for anyone who hears a patient mention a problem.
- Patient signals: complaints, portal messages that say I called and repeat calls about the same request within a day.
- Change watch: review more heavily for a week after any change to answers, routes or vendor software.
Correcting an error, step by step
Contain
Stop the error repeating. Fix or remove the wrong answer, pause the route, or switch the affected call type back to staff while you investigate.
Find everyone affected
Search logs for other calls with the same answer or route since the last known good configuration. One reported error often means several unreported ones.
Contact patients who may act on it
Anyone given wrong clinical routing, a wrong appointment or wrong information they might rely on is contacted, by phone where the issue is significant.
Fix the cause
Correct the answer, rule or route, and test it with fictional calls. If the vendor's software caused it, report it in writing with examples.
Verify
Review a targeted sample over the next few days to confirm the fix worked and created no new problem.
Document
Record the error, grade, patients contacted, cause, fix and sign-off in the error log.
Where errors usually come from
When you trace errors back, the causes cluster. Knowing the usual suspects speeds up investigation and points to prevention.
- Stale facts. Hours, providers, locations and accepted plans changed, but the approved answers did not. This is the most common cause, and the cheapest to prevent with an owner and a review date on every answer.
- Edits without a second check. Someone changed one answer and accidentally broke another, as in the example below.
- Gaps in the answer list. A caller asked something the tool had no answer for, and it guessed instead of saying it did not know. The fix is both a new answer and a firmer fallback rule.
- Mishearing. Names, dates of birth and phone numbers misheard on poor lines or with background noise. Read-back confirmation catches most of these.
- Routing logic. A request type mapped to the wrong queue, or a transfer target that changed numbers.
- Vendor changes. A software or model update altered behavior without notice. Ask for change notices in the contract and review more heavily after updates.
Track the cause of every logged error using these categories. After a few months, the pattern tells you where to invest: better answer ownership, tighter edit controls, firmer fallback rules or a conversation with the vendor.
Telling affected patients
Patients forgive mistakes that are owned and fixed quickly. They do not forgive discovering a mistake themselves. Three principles keep these conversations short and constructive: tell them promptly, say plainly what happened and what you have done, and take responsibility as the practice rather than pointing at the technology.
Wrong appointment: “I'm calling because our phone system moved your appointment to the wrong day when you called on Tuesday. That was our mistake. Your appointment is back on Thursday at 10, and I've made sure it's confirmed. I'm sorry for the confusion.”
Missed urgent route: “When you called yesterday you mentioned chest discomfort, and our system should have connected you to a nurse straight away. It didn't, and we've fixed that. How are you feeling today? I'd like to put you through to our nurse now.”
For the second kind of call, a clinician should decide who makes it and whether the patient needs to be seen. Document the conversation in the record.
When an error is a privacy incident
Some errors expose patient information: a text with appointment details sent to an old number, a captured message read back to the wrong caller, a transcript shared with the wrong practice. Treat any of these as a potential breach and involve your privacy officer the same day. Under HIPAA, notifications to affected individuals must be made without unreasonable delay and no later than 60 calendar days after discovery[1]. Breaches affecting fewer than 500 people must also be logged and reported to HHS within 60 days after the end of the calendar year[3]. Your vendor's business associate agreement should require it to tell you promptly when its systems cause such an incident.
Errors in other languages
Errors are harder to spot when the conversation is not in English, because reviewers may not understand it. Federal Section 1557 rules require that machine translation of critical or technical patient content be reviewed by a qualified human translator[4]. Apply the same thinking to monitoring: include non-English calls in your daily sample, and arrange for a qualified reviewer to check them regularly. Do not let a language gap become a blind spot. More on this in AI for multilingual patient calls.
Example: one wrong answer, three patients
Example: a fictional practice's front desk lead hears a patient at check-in say the phone told her the practice no longer takes her insurance plan. The lead checks: the answer for that plan was edited the previous week by mistake, when a different plan was removed. It is a grade 4 information error with access consequences.
Example, continued: the answer is corrected within the hour. A transcript search finds two more callers given the wrong answer, both prospective new patients. The front desk lead calls both, apologizes, confirms the plan is accepted and offers appointments; one books. The error log records the cause, a missing second check on answer edits, and the practice adds a rule that changes to accepted-plan answers need a second person to approve them.
Measuring error handling
- Errors found by the practice versus reported by patients. The first number should be much larger.
- Time from error to detection, and from detection to fix, by grade.
- Patients contacted after errors that affected them.
- Repeat errors with the same cause.
- Grade 1 and grade 2 errors per month, reviewed by the clinical lead and privacy officer.
Staff habits that make this work are covered in training staff to work alongside AI, and what call data can reveal beyond errors in call analytics.
Frequently asked questions
What should a practice do when its AI gives a patient wrong information?
Stop the error repeating, find everyone else affected, contact patients who may act on the wrong information, fix and test the cause, then document it. Grade the error by potential harm to set how fast each step happens.
Should we tell patients when AI made a mistake?
Yes, when the mistake could affect them. Call promptly, explain plainly what happened and what you fixed, and take responsibility as the practice.
Is an AI texting or calling error a HIPAA breach?
It can be, if patient information reached someone who should not have it. Involve your privacy officer the same day; HIPAA sets notification deadlines of up to 60 calendar days after discovery.
References
Legal Information Institute, Cornell Law School (Electronic Code of Federal Regulations text) (current). 45 CFR § 164.404 - Notification to individuals (opens in a new tab) ↑
American Medical Association (2026). More than 80% of physicians use AI professionally: AMA survey (opens in a new tab) ↑
Legal Information Institute, Cornell Law School (Electronic Code of Federal Regulations text) (current). 45 CFR § 164.408 - Notification to the Secretary (opens in a new tab) ↑
Legal Information Institute, Cornell Law School (Electronic Code of Federal Regulations text) (current). 45 CFR § 92.201 - Meaningful access for individuals with limited English proficiency (opens in a new tab) ↑