Key takeaways
- No business associate agreement, no pilot. A vendor handling patient information must give that assurance in writing[1].
- Ask for proof of accuracy on your own fictional test calls, not a demo script, and for a clear way to see and correct every answer.
- Check integration with your actual EHR. Small practices are far less likely to use one of the market-leading systems[2].
- Insist that every capability is labelled live or planned, and put the live list in the contract.
How to use this checklist
Most practices evaluate AI vendors the way they would a phone system: watch a demo, compare quotes, pick one. AI tools need more scrutiny because they speak for your practice and handle patient information in ways a phone system never did. This checklist groups the questions into six areas. Send it to each shortlisted vendor in writing, ask for written answers, and score each area from zero to two: zero for no or unclear, one for partly, two for a clear yes with evidence.
Weight the areas to suit your practice, but treat the privacy and safety items marked as gates: a zero on any gate removes the vendor, whatever its other scores. Governance is still rare in practices; only 20% of medical groups have AI governance or a formal AI-use policy[3]. A written evaluation is a good first piece of that governance.
1. Privacy and security
- Gate: will the vendor sign a business associate agreement before any patient information flows? HIPAA allows a covered entity to let a vendor handle protected health information only with satisfactory written assurance that it will be safeguarded[1].
- Where are call recordings, transcripts and messages stored, in which country, and for how long by default?
- Can you set retention periods and delete data on request, including at contract end?
- Is patient data used to train or improve models, for you or other customers? Can you opt out in writing?
- Who at the vendor can access your data, and is access logged?
- Which subcontractors process your data, and do they sign equivalent agreements?
- How quickly will the vendor notify you of a breach? HIPAA requires notice to affected individuals without unreasonable delay and no later than 60 calendar days after discovery[4], so you need to hear first, and fast.
- Does the vendor have an independent security assessment it will share?
The stakes are not small. Inflation-adjusted HIPAA civil money penalties range from $145 per violation at the lowest tier up to a calendar-year cap of $2,190,294 for identical violations[5]. Patients are wary too: 37% of US adults expected AI in health care to make the security of patients' records worse[6].
2. Accuracy and safety
- Gate: does the tool answer only from facts your practice approves, and say it does not know when an answer is missing?
- Gate: can emergency language trigger a fixed message that you and your clinical lead approve, with no follow-up questions?
- Will the vendor run your own fictional test calls, including tricky phrasing, before you sign?
- Can you see every call's transcript and summary, and correct an answer in minutes rather than by support ticket?
- How does the tool handle accents, background noise and the languages your patients speak?
- What happens on a system outage: do calls fall back to your phone line or voicemail?
- How are model or script updates tested and announced, and can you hold an update until you have tested it?
3. Integration
Many vendors integrate well with the largest EHR systems and poorly with everything else. That matters most for small practices. In 2024, only 32% of physicians in solo practice used an EHR from one of the five market-leading developers, compared with 90% of physicians in practices with more than 50 physicians[2].
Source: Office of the National Coordinator for Health IT (ASTP/ONC), HHS, 2024[2]
Show the data
| Item | Value |
|---|---|
| Solo practice | 32% |
| More than 50 physicians | 90% |
- Which EHR and practice management systems do you connect to today, in production, at practices like ours?
- Is the connection read-only or does it write back? Which fields?
- If there is no direct connection, what is the fallback: email, task queue, a shared inbox?
- Who pays for and maintains the integration, and what happens when the EHR updates?
- Can we see a reference practice using our exact EHR?
4. Operations and support
- Who builds the initial answer set and call routes, and how long does setup take?
- Is there a named contact, and what are the response times for urgent issues such as a broken transfer?
- What reports will we get, and can we export the raw data?
- How do we add a location, provider or new call type?
- What training do staff receive, and is it repeated for new hires?
5. Contract terms
| Term | What to ask for | Red flag |
|---|---|---|
| Term length | A pilot period or short initial term | Multi-year commitment before any pilot |
| Exit | Data export in a usable format and deletion confirmation | Data held hostage or fees to export |
| Capabilities | A written list of live capabilities | Roadmap features described as included |
| Service levels | Uptime commitment and outage fallback | No commitment, or credits only |
| Data use | No training on your data without written consent | Vague improve our services language |
| Changes | Notice before material changes to scripts or models | Unilateral changes at any time |
| Liability | Responsibility for vendor-caused failures addressed explicitly | Blanket disclaimers for all errors |
Keep copies of the vendor's written answers with the contract. HIPAA requires covered entities to retain required policy and compliance documentation for six years from creation or last effective date[7], and your vendor due diligence is part of the record you may need to show.
6. Honest labels
The most revealing question is simple: which of the features you showed us are live in production today, and which are planned? A trustworthy vendor answers precisely and puts the live list in writing. Be wary of answers like it's on the roadmap for next quarter for anything your business case depends on. Buy what exists, and treat planned features as a possible bonus.
Red flags in the sales process
How a vendor sells often predicts how it will behave once you are a customer. Watch for these signals during the evaluation, and note them next to the scores so the decision-maker sees them.
- The demo only ever uses the vendor's own script. A confident vendor lets you call in with your own fictional scenarios.
- Questions about data use, retention or training get answered verbally but not in writing.
- The business associate agreement arrives late, heavily modified, or only after you sign the main contract.
- Accuracy claims come as a single percentage with no explanation of how it was measured or on which calls.
- The vendor says its AI can safely answer clinical questions, or offers to configure symptom checking at the front desk.
- References are all large health systems when you are a small practice, or all on a different EHR.
- Pressure to sign before a quarter end, with a discount that expires.
- Nobody can tell you who your support contact will be after go-live.
One red flag is not always disqualifying; a small vendor may simply be new to health care. Several together usually mean the evaluation should stop. If a vendor is otherwise strong, raise the concern directly and ask how they will address it in the contract. The answer, and how quickly it arrives, tells you a great deal.
Scoring and deciding
Remove gate failures
Any vendor scoring zero on a gate item is out.
Score the rest
Add up scores per area and apply your weights. Note where answers were vague.
Test the top two
Run the same fictional call script with both. Compare transcripts, transfers and how quickly you could fix a wrong answer.
Check references
Speak to a practice of similar size on the same EHR. Ask what surprised them after go-live.
Pilot before committing
Sign for a pilot with clear success criteria, not a long term. See running a 30-day front office AI pilot.
Example: two vendors, one decision
Example: a fictional three-physician practice using a smaller EHR shortlists two vendors. Vendor A has the better demo and a strong integration list, but its connection to the practice's EHR is described as coming soon. Vendor B has no direct EHR connection but posts captured requests to the practice's task queue today, signs a business associate agreement without changes, and lets the manager correct answers herself.
Example, continued: on the practice's own fictional test calls, both handle office questions well, but Vendor A's emergency message can only be edited by its support team. That fails a gate. The practice pilots Vendor B, with a contract clause that lets it revisit the decision if Vendor A's integration goes live. The decision took two weeks, and every answer is on file. For what changes for staff afterwards, see will AI replace medical receptionists.
Frequently asked questions
What should I ask an AI vendor before signing?
Ask whether it will sign a business associate agreement, where data is stored and for how long, whether your data trains its models, how it handles emergencies, which EHRs it connects to in production, and which features are live versus planned.
Does an AI phone vendor need a business associate agreement?
If it will hear, record or store protected health information on your behalf, yes. HIPAA requires satisfactory written assurance from the vendor before it handles that information.
How should a small practice compare AI vendors?
Use a written checklist with gate items for privacy and safety, test the top two with the same fictional calls, check a reference on your EHR, and sign for a pilot rather than a long term.
References
Legal Information Institute, Cornell Law School (Electronic Code of Federal Regulations text) (current). 45 CFR § 164.502 - Uses and disclosures of protected health information: General rules (opens in a new tab) ↑
Office of the National Coordinator for Health IT (ASTP/ONC), HHS (2024). Office-based Physician Electronic Health Record Adoption, 2008-2024 (ONC Data Brief) (opens in a new tab) ↑
Medical Group Management Association (MGMA) (2026). AI governance in medical group practices: Rules for the humans in the loop (opens in a new tab) ↑
Legal Information Institute, Cornell Law School (Electronic Code of Federal Regulations text) (current). 45 CFR § 164.404 - Notification to individuals (opens in a new tab) ↑
Legal Information Institute, Cornell Law School (Electronic Code of Federal Regulations text) (current). 45 CFR § 102.3 - Penalty adjustment and table (opens in a new tab) ↑
Pew Research Center (2022). 60% of Americans Would Be Uncomfortable With Provider Relying on AI in Their Own Health Care (opens in a new tab) ↑
Legal Information Institute, Cornell Law School (Electronic Code of Federal Regulations text) (current). 45 CFR § 164.530 - Administrative requirements (opens in a new tab) ↑