Phone Operations

Mystery Caller Audit: Testing Your Own Phones

Run a mystery caller audit of your own practice phones: realistic test scenarios, a consistent score sheet, a time-of-day call schedule and fair follow-up.

AssistCall research teamPublished Reviewed by Faruque Amin, VP of Sales and Marketing7 min read

On this page
  1. What a mystery caller audit shows you
  2. Write realistic scenarios
  3. Score every call the same way
  4. Build a call schedule across the week
  5. Who should make the calls
  6. Test the other front doors too
  7. Run it fairly and safely
  8. Worked Example: results by time of day
  9. Sharing results with the team
  10. Turning findings into fixes
  11. Frequently asked questions
  12. References
Phone Operations: Mystery Caller Audit: Testing Your Own PhonesArticle coverPHONE OPERATIONSMystery Caller Audit:Testing Your Own PhonesAva Medical IntakeInsightsPhone Operations: Mystery Caller Audit: Testing Your Own PhonesArticle coverPHONE OPERATIONSMystery Caller Audit:Testing Your OwnPhonesAva Medical Intake

Key takeaways

  • A mystery caller audit shows what patients actually experience, which internal reviews of recorded calls cannot.
  • Write four to six realistic scenarios, including a new patient and an insurance question, and score each call the same way.
  • Spread test calls across days and times; most problems show up at peaks, lunch and near closing.
  • Tell staff audits happen, use results for fixes and coaching, and never book real appointments you will not keep.

What a mystery caller audit shows you

Recorded call reviews show how calls go once someone answers. A mystery caller audit shows the whole journey from the patient's side: how long the phone rings, how long the menu takes, whether the caller is put on hold, whether a new patient can get an appointment, and what happens at 12:15 or 4:55. It is the closest a practice can get to hearing its own front door.

Researchers use the same method to study access. A meta-analysis of 34 US audit, or secret shopper, studies found that callers presenting with Medicaid coverage were 1.6 times less likely to successfully schedule a primary care appointment and 3.3 times less likely for specialty care than callers with private insurance[1]. Whatever your payer mix, the lesson for practices is that how the phones handle different callers can shape who gets care, and that it is measurable.

Lower likelihood of getting an appointment for Medicaid vs privately insured callers, secret shopper studiesMeta-analysis of 34 US audit studies
Lower likelihood of getting an appointment for Medicaid vs privately insured callers, secret shopper studiesComparison. Specialty care: 3.3Γ—; Primary care: 1.6Γ—.3.3Γ—Specialty careSpecialty care: 3.3Γ—1.6Γ—Primary carePrimary care: 1.6Γ—Lower likelihood of getting an appointment for Medicaid vs privately insured callers, secret shopper studiesComparison. Specialty care: 3.3Γ—; Primary care: 1.6Γ—.3.3Γ—Specialty careSpecialty care: 3.3Γ—1.6Γ—Primary carePrimary care: 1.6Γ—

Source: Inquiry: The Journal of Health Care Organization, Provision, and Financing (via PubMed Central), 2019[1]

Show the data
ItemValue
Specialty care3.3Γ—
Primary care1.6Γ—

Write realistic scenarios

Example mystery caller scenarios for a primary care practice
ScenarioWhat it testsOpening lineA pass looks like
New patient, common planNew patient access and insurance checkβ€œI just moved here and need a new doctor. Do you take Blue Cross?”Plan confirmed, first available offered, what to bring explained
New patient, MedicaidEqual access by payerβ€œAre you taking new patients? I have Medicaid.”Clear answer, same booking process if accepted, referral if not
Established patient, sick visitSame-day access and urgency screenβ€œI have had a fever since yesterday and want to be seen.”Symptom questions or nurse transfer, same-day or next-day option
Refill requestMessage handlingβ€œI need a refill on my blood pressure pill.”Right route given, callback window stated
Billing questionRouting and courtesyβ€œI got a bill I do not understand.”Transferred or message taken with clear next step
Emergency symptomEmergency instructionβ€œMy father has chest pain right now, can he come in?”Told to call 911 immediately

Keep scenarios short and believable. Testers should have a consistent fictional profile, such as a name, age, plan and reason, but should not use a real patient's identity or details. For the emergency scenario, the tester ends the call as soon as the instruction is given, before anyone dispatches help.

Score every call the same way

  • Time to answer: seconds from first ring to a person or the menu.
  • Menu time: seconds from the menu starting to reaching a person.
  • Hold time: total seconds on hold, and whether permission was asked.
  • Outcome: appointment offered, message taken, transferred, or no resolution.
  • First available appointment offered, in calendar days, for new and sick visit scenarios.
  • Accuracy: correct information on insurance, location, hours and what to bring.
  • Urgency handling: correct action for the sick and emergency scenarios, scored pass or fail.
  • Courtesy: one simple rating, such as did the caller feel helped, yes or no.
  • Abandon: whether the tester gave up after a set limit, such as five minutes without a person.

Record numbers, not impressions, wherever possible. Two testers can disagree about whether a staff member sounded warm; they will not disagree about a 4-minute hold.

Put the score sheet in a shared spreadsheet with one row per call and one column per measure, plus the date, time window, scenario and tester. That layout makes it easy to sort by time of day or scenario later, and it lets you compare the rerun with the first audit line by line instead of relying on memory.

Build a call schedule across the week

  1. Pick the time windows

    Choose five windows: first hour after opening, mid-morning, lunch, mid-afternoon and the last half-hour before closing. Add one after-hours call to test the evening greeting and on-call route.

  2. Spread across weekdays

    Include at least one Monday morning and one Friday afternoon, since those are where most practices struggle.

  3. Assign scenarios to windows

    Rotate scenarios so each one is tested in different windows. A new patient call at 8:05 on Monday tests something different from the same call at 2:30 on Wednesday.

  4. Plan the volume

    Twenty to thirty test calls over two to three weeks gives a useful picture for a small or mid-sized practice without burdening staff.

  5. Repeat the same plan later

    Rerun the audit with the same scenarios and windows three to six months after making changes, so results are comparable.

Who should make the calls

The best testers sound like ordinary patients and are not known to the staff. Options include a manager from another site in the same group, a trusted friend or family member of the practice owner, a board member, or an outside firm that specializes in healthcare mystery shopping. Whoever calls needs a short briefing: the scenario, the fictional profile, what to say if asked for details they do not have, such as β€œI will have to check my card and call back,” and how to end the call politely without taking a slot.

  • Give each tester a printed score sheet and ask them to complete it immediately after each call, while details are fresh.
  • Ask testers to record the exact time the call started and the times of key moments, using a phone clock.
  • Use two or three different testers so results do not depend on one person's style.
  • Brief testers on the emergency scenario carefully so they end the call before any help is dispatched.

Test the other front doors too

Patients do not only call. Add a few tests of the other routes into the practice in the same audit window: submit a new-patient request through the website form and time the reply, send a portal message asking a simple non-clinical question, and call after hours to hear the evening greeting and check that the on-call route is explained clearly. A practice with good daytime phones but a website form that nobody answers for three days still loses patients. Score these with the same simple measures: time to response, accuracy and a clear next step.

Run it fairly and safely

  1. Tell staff that mystery calls will happen during the quarter, without giving dates. The goal is learning, not catching people out.
  2. Never let testers take real appointment slots and fail to show. If a booking is offered, decline politely at the end of the call or cancel within minutes.
  3. Use an outside tester, a colleague from another site or a trusted friend of the practice; staff will recognize a coworker's voice.
  4. Do not use real patient names, dates of birth or insurance numbers.
  5. Share team-level results first, and handle any individual concerns privately and as coaching.

If a test call reveals a safety problem, such as an emergency scenario not being directed to 911, act on it the same day with a team refresher. Do not wait for the full report.

Worked Example: results by time of day

Example mystery caller results for a fictional family practice
Time windowExample callsExample median holdExample abandonedExample new patient booked
8:00-9:0064 min 10 s21 of 2
10:00-11:0050 min 40 s02 of 2
12:00-1:005Closed greeting30 of 1
2:00-3:0051 min 05 s01 of 1
4:30-5:0042 min 30 s10 of 1

Example reading: the practice handles calls well mid-morning and mid-afternoon, but loses callers in the first hour, over lunch and near closing. The two new-patient callers who were not booked were both told to call back. Example actions: stagger morning start times, cover lunch with a rota, and let the last half-hour's calls be booked rather than messaged. The practice reruns the same plan in four months.

Sharing results with the team

How results are shared decides whether the next audit is welcomed or resented. Present the findings at a team meeting as a picture of the system, not a ranking of people. Start with what went well, such as accurate insurance answers or warm greetings, then show the two or three biggest gaps with the time-of-day data. Ask the team why they think those gaps happen; front desk staff usually know exactly why the 8 am hour is hard. Agree on one or two changes, assign an owner to each, and set the date for the rerun. Any individual coaching happens privately, using the recorded calls rather than the mystery call notes where possible.

Turning findings into fixes

Mystery caller results are most useful when tied to what practice leaders already care about. Phone access was named as the top patient access focus for 2026 by 22% of practice leaders in an MGMA poll[2], and 28% of medical groups said new-patient wait times were longer in 2026 than in 2025[3]. A new patient who cannot get through, or who is told to call back, adds to both problems.

New patients are a minority of visits but a large share of growth: they accounted for 16.8% of US physician office visits in 2019[4]. If your audit shows new-patient callers struggling at certain times, fix that first. Then use call quality monitoring to coach the details and track the effect on patient satisfaction with phone access.

Frequently asked questions

What is a mystery caller audit?

It is a planned set of test calls to your own practice, using realistic scenarios and a consistent score sheet, to measure what patients experience on the phone at different times.

Is it fair to staff?

It is fair when staff know audits happen, results are shared at team level first, and findings are used to fix processes and coach rather than to punish.

How many test calls do we need?

For a small or mid-sized practice, twenty to thirty calls over two to three weeks, spread across days and times, gives a useful picture.

Can testers book real appointments?

They should not keep them. Testers should decline at the end of the call or cancel within minutes so real patients are not blocked from slots.

References

  1. Inquiry: The Journal of Health Care Organization, Provision, and Financing (via PubMed Central) (2019). Medicaid Patients Have Greater Difficulty Scheduling Health Care Appointments Compared With Private Insurance Patients: A Meta-Analysis (Inquiry, 2019) (opens in a new tab) ↑

  2. Medical Group Management Association (MGMA) (2025). Patient access priorities for 2026: Tackling wait times, phones, no-shows and more (opens in a new tab) ↑

  3. Medical Group Management Association (MGMA) (2026). New-patient wait times largely hold flat in 2026 as some groups add providers in bid to meet demand (opens in a new tab) ↑

  4. CDC National Center for Health Statistics (2019). National Ambulatory Medical Care Survey: 2019 National Summary Tables (opens in a new tab) ↑

Book a demo

See Ava handle your calls.

Book a short demo. We’ll walk through your calls and workflow, and agree what Ava should take care of.

Book a demo