The Best AI Phone Agents Know Exactly When to Hand Over to a Person

Judith Butterfield is 71 and had a stroke in January. It left her speech fragmented and her right side weak. When she tried to book an appointment at her GP surgery near Doncaster, she called five times and never got one. The practice’s AI receptionist, a system called EMMA, asked her not to use speakerphone — but she needed one hand free, because the other no longer worked. “There was a point in the conversation where I had to repeat my name and she just couldn’t understand me,” she said. The surgery has since switched EMMA off and hired more reception staff.

The detail that matters for anyone thinking about putting an agent on their phone line is not that the automation failed a caller. It is that the fix already existed and nobody had turned it on.

The way out was a setting

EMMA’s vendor, QuantumLoopAI, told reporters that every practice using the system can enable “priority routing”, letting a caller skip the automation and reach a human receptionist directly. The same vendor says EMMA is trained on a wide range of accents, supports 17 languages, and that more than 90% of patients report an improvement. Both can be true. A system that serves the large majority well but has no reliable exit for the minority it cannot hear is not a technology failure — it is a configuration failure.

Healthwatch Rotherham found the system “does not respond well to variations in speech such as regional accents or those with speech impediments”, and collected callers who simply gave up: “I could never get it to understand me, I ended up just hanging up.” English GP practices have been required since 2016 to identify and meet patients’ communication needs under the NHS Accessible Information Standard — a duty that does not lapse because a machine picks up.

Church offices meet this caller before most businesses do

Very few small organisations take calls from a population as old, as unwell or as anxious as a doctor’s surgery does. Churches are one of them: the people who ring a church office skew elderly, are often recently bereaved, and frequently speak English as a second language.

Half of those calls are easy: service times, parking, kids check-in, whether there is a lift. The other half are not, and an agent that tries to handle a bereavement call itself has done more damage than an unanswered phone. The line that separates an assistant from a wall is whether it recognises a pastoral or care request and routes it to a person instead of answering it. That is the standard worth holding any agent on a church phone line to.

What to check before your phone line goes automated

Test with your hardest callers, not your easiest. Ring your own number on speakerphone, with slow or slurred speech, with a heavy accent, from a bad line, and listen.

Then configure the handoff before you configure anything else. Decide what reaches a human — a spoken word, two unrecognised answers, five seconds of silence — and verify it works at 9pm as well as at 11am. Write down the categories that never get automated: bereavement, counselling, hospital visits, emergencies.

Then watch what you could not see before. Butterfield called five times; the surgery only found out when she went to the press. Every agent worth running logs the calls it could not complete — read that list weekly.

Doing nothing is not neutral either — an office phone that only answers Tuesday to Friday, 9 to 4, misses the visitor asking about Sunday at ten at night. The goal is something that answers every call and message and passes the human ones to a human, quickly and without making anyone prove they deserve it.