← Case studiesManaz Afzal · voice receptionist
Clinics, salons and trades · voice agent · built by Manaz

A receptionist that answers every call.

A phone receptionist for a clinic group. It listens to the caller, works out whether they want to book, check or cancel, reads the day and time, checks the diary and answers out loud. When it isn’t sure, it takes a message for the team.

Evaluated on 598 real recordings of 362 spoken requests from SLURP (Bastianelli et al., 2020, CC BY 4.0), passed through a telephone-line filter. Run 18 Nov 2026, 23:42 Melbourne time.

Callerplease add practice on feb four at king's park at two pm
HeardPlease add practice on February 4th, I can spot at 2pm.
AgentI have you with Priya on Thursday 4 February at 2 pm. Is that right?
A real test recording through the whole loop, on a three-core cloud computer with no GPU.
Ringing

The phone rings while everyone is busy.

In a clinic, salon or trade business the people who could answer the phone are with a customer. Calls go to voicemail, and many callers try the next business instead of leaving a message.

The job is narrow and repetitive: most calls are someone wanting to book, move or cancel a time, or to check one. A receptionist that handles those reliably, and passes everything else to a person, gives staff their hands back without risking a wrong booking.

Today

  • Calls during appointments go unanswered.
  • Staff return calls between customers, often hours later.
  • After-hours callers get voicemail.

With the receptionist

  • Every call is answered at once, day or night.
  • Bookings with a clear day and time are read back and written to the diary.
  • Everything else becomes a short message for the team.
Answered

The build

A complete voice loop that runs on an ordinary computer: no GPU and no paid speech service.

Each call passes through five parts: speech-to-text, a reader that decides what the caller wants, a reader for spoken days and times ("the twenty first at seven", "tomorrow ten am"), a clinic diary, and a spoken reply. The agent never books on a guess: it reads every booking back and asks the caller to confirm.

Recorded calls through the full loop

Booked, details read back
Caller saidplease add practice on feb four at king's park at two pm
Speech-to-text heardPlease add practice on February 4th, I can spot at 2pm.
Agent repliedI have you with Priya on Thursday 4 February at 2 pm. Is that right?
speech-to-text 0.55 s for 4.2 s of speech · understanding and diary 0 ms · voice 0.56 s
Booked, details read back
Caller saidadd conference call at four pm to my reminders for today
Speech-to-text heardAddFlora UNCLASSIFIEDServ:" at 4pm to my reminders for today.
Agent repliedWednesday 18 November at 4 pm is taken, I'm sorry. I'll ask the front desk to call you back with the nearest time.
speech-to-text 0.66 s for 4.0 s of speech · understanding and diary 0 ms · voice 0.69 s
Asks for what it missed
Caller saidevent reminder mona tuesday
Speech-to-text heardEvent reminder, Mona Chphia804.
Agent repliedHappy to book that. Which day and time suit you?
speech-to-text 5.18 s for 3.3 s of speech · understanding and diary 0 ms · voice 0.38 s
Not a booking: message for the team
Caller saidmake me a cup of coffee with salted carmel flavored cream
Speech-to-text heardMake me a cup of coffee with sodded camera for flavored cream.
Agent repliedI'll pass your message to the team and someone will call you back today.
speech-to-text 0.50 s for 4.7 s of speech · understanding and diary 0 ms · voice 0.44 s

Recordings from the SLURP test set, through the phone-line filter. Speech-to-text output is shown as produced, with unreadable characters removed. SLURP's requests were spoken to a home assistant, so some are reminders rather than appointments; the agent treats any "set" request as a booking.

Listening

How it works

The call, step by step

Call comes inphone line, 8 kHzSpeech to textfaster-whisper base.en, CPUWhat do they want?book · check · cancel · otherWhich day, what time?date and time readerDiarythree practitioners, live slotsSpoken replyPiper voice, read-backMessage for the teamunsure, or not a bookingcaller says no → staff call back

What it runs on

On the linePhone number → SIP trunk → audio stream (next step; the study runs on recorded calls)Earsfaster-whisper base.en, int8, three CPU threads, no GPUUnderstandingRequest type: TF-IDF words and letter patterns + logistic regression, trained on SLURP train text · day and time reader written for spoken datesDiarySQLite diary: three practitioners, Monday–Saturday hours, 62% of slots already takenVoicePiper text-to-speech, en_GB alba voice, on CPUPeopleEvery unsure call and every "no, that's wrong" goes to the front desk as a message
Booked

What it means for the front desk

Out of 598 test calls, this is what the agent’s first reply did.

66%of calls got the right first reply (61%–71%), against 54% for keyword rules
35%of calls were handled with no staff time: bookings read back, missing details asked for, and checks or cancellations started
7%of the calls it took on went the wrong way (3%–11%); the read-back is what stops these becoming wrong bookings
Where each call ended up

Per 100 calls, about 35 need no staff time, 31 are rightly passed on as messages, and 31 are passed on although the agent could have handled them.

First reply for each test call, by method. The agent row uses the confidence rule chosen on the tuning half.

At 3 minutes of staff time per handled call, that is about 1.8 hours saved per 100 calls.

The cost of being careful is the 31% of calls passed on that the agent could have handled. Lowering the confidence bar would hand more calls to the agent but let more go the wrong way.

Behind the line

Behind the line

Method, results and failure cases.

Data and test design

  • Speech: SLURP, a public set of people speaking requests to a voice assistant, recorded on a headset and from across the room. The study uses the headset recordings, which sound closest to a phone held to the ear, and passed every clip through a telephone-line filter (8 kHz, 300–3,400 Hz).
  • Requests: SLURP's calendar requests ("set", "query", "remove") became book, check and cancel; a random selection of its other requests ("play some jazz", "make me a coffee") stand in for calls that aren't bookings.
  • Split: the request reader learned only from SLURP's training text. The test recordings were split by sentence into a tuning half (553 recordings, used once to choose the confidence bar) and a reporting half (598 recordings of 362 sentences). No sentence is in both.
  • Intervals: 95% bootstrap intervals, resampling sentences so that repeat recordings of one sentence move together.
Request type: agent
Rows: what the caller wanted. Columns: what the agent chose before the confidence rule.
Request type: keyword rules
Same calls, sorted by keywords such as "book", "cancel" or "what".
Where accuracy is lost

Speech-to-text costs the most: request type drops from 91% on the human transcript to 78% on what it heard; the day reader drops from 100% to 84%.

Accuracy on the reporting half: on the speech-to-text output, and on the human transcript of the same calls.
Choosing the confidence bar

On the tuning half, 0.85 was the lowest bar keeping wrong turns at or under 5%. On the reporting half it gave 7.0%: the target was not held.

Share of calls with the right first reply, and share of calls taken on that went the wrong way, as the bar rises. Dashed: tuning half. Solid: reporting half.

All outcomes

First replyAgentKeyword rulesAgent on the human transcript
booking, every detail right463179
booking, asks for the missing detail830
booking, no detail given7152104
check or cancel, right flow8699176
passed to staff, correctly185140183
passed to staff, could have been handled18620128
booking, a detail misheard010
wrong flow167128

Reporting half: 598 recordings. "Could have been handled": the caller wanted to book, check or cancel, but the agent wasn’t confident enough.

Speed on an ordinary computer

Speech-to-text took a median 0.51 s for a median 2.8 s request (90th percentile 0.63 s), about 0.22 s per second of speech. Understanding and the diary take a few milliseconds, and the spoken reply 0.38–0.69 s on the four calls above, so the caller hears an answer roughly a second after they stop talking.

Findings

Far-field speech is much harder. On SLURP's recordings from across the room, word error rose from 35% to 67%, and request type accuracy fell to 59%. A speakerphone in a noisy shop would sit between the two.
One misheard word flips the request. "Delete the shopping trip I have scheduled March twenty third" came back as "He did the shopping trip…", and "Cancel business meeting on Wednesday" as "What is this meeting on Wednesday?". Both turned a cancellation into a check. Of the true booking requests, 12 had the day misheard and 8 the time; the read-back is there to catch those.
Some labels are debatable. SLURP labels "show meetings" and "show me the events in my work calendar for next week" as calendar "set" requests. The agent called them checks, which a receptionist would too; they still count against it here. Of the request-type errors, 26 happen on the human transcript as well, so they are not hearing errors.
A bug I fixed. The first version of the time reader missed "8pm" written as one word and booked it for 8 in the morning. It now reads "pm" attached to a number, and ordinals such as "the third" only after "the" or "on".
In detail

The problem in real terms

A physiotherapy, dental or allied-health clinic with two to six practitioners and one person on the front desk. That person is also checking patients in, taking payments and answering Medicare and health-fund questions, so the phone rings out in the busiest hours and goes to voicemail after 5 pm. Call audits in Australian multi-site clinics put unanswered calls at around one in ten, and practices that add an automated line report a large share of calls handled without the desk, most of them bookings.

Team
One receptionist, sometimes shared across two sites
Systems
Cliniko, Halaxy, Nookal or Timely diary; a VoIP line from Telstra, Aussie Broadband or Twilio
Calls
Mostly book, move, cancel or check a time; a minority are clinical or billing questions for a person
Constraints
Privacy Act and the APPs for health information; call-recording consent; no clinical advice

Baselines

Keyword rulessame calls54%Receptionist95% CI 61%–71%66%Receptionist on a perfect transcriptceiling if speech-to-text made no errors91%
The receptionist gives the right first reply on 66% of 598 test calls, against 54% for keyword rules (gain 7% to 17% points, 95% CI). With a perfect transcript it would reach 91%, so speech-to-text is the part to improve first.
Heard by speech-to-textHuman transcriptRequest type91%Day100%Time100%
Most accuracy is lost in hearing, not understanding: the day reader is right on every human transcript and on 84% of what was heard.
Where calls went wrongCalls
Misheard the request type34
Request type missed on the human transcript too26
Day misheard12
Time misheard8

Assumptions

  • Three minutes of staff time per handled call.
  • Headset recordings through a telephone filter stand in for mobile calls; far-field recordings were much worse (word error 67% against 35%).
  • SLURP calendar requests stand in for appointment calls.

What a clinic would do next

  • Week 1: connect the diary read-only and route after-hours calls to the receptionist, with staff call-back on every message.
  • Weeks 2 to 4: record consenting calls, re-measure word error and wrong turns on the clinic’s own callers.
  • Weeks 5 to 8: allow bookings with read-back; keep cancellations and anything clinical with staff.
  • Then: review missed-call rate, after-hours bookings and wrong bookings monthly.
After the call

Limits, and what comes next

  • SLURP was recorded for home assistants, not clinics. The accents, phrasing and background noise of real callers will differ, so a pilot should re-measure on the business's own calls (with consent).
  • The phone connection itself is not part of this study: the agent ran on recorded calls. A live line needs a SIP or telephony provider and call recording consent.
  • A larger speech model (small.en or a hosted one) should cut misheard words, at several times the computing time. I have not measured it here.
  • The agent only books, checks and cancels. It takes no payments and gives no advice; a live version should also send urgent callers straight to a person.

Run it yourself

# code MIT; SLURP audio and text CC BY 4.0
git clone https://github.com/manazafzal/case-studies && cd case-studies/ai-receptionist/code
python -m venv .venv && . .venv/bin/activate && pip install -r requirements.txt
python asr.py test base.en 800 asr_test_base.parquet   # about 25 minutes on three CPU cores
python agent.py receptionist.json && python media.py

Run 18 Nov 2026, 23:42 AEDT (Melbourne time) · seed 20261118

Sources

  1. Bastianelli, E., Vanzo, A., Swietojanski, P. & Rieser, V. (2020). SLURP: a spoken language understanding resource package. EMNLP. Data CC BY 4.0, via huggingface.co/datasets/marcel-gohsen/slurp.
  2. Radford, A. et al. (2023). Robust speech recognition via large-scale weak supervision (Whisper). ICML. Run with faster-whisper (SYSTRAN).
  3. Piper text-to-speech (Rhasspy / Open Home Foundation), en_GB alba medium voice.