Dial 1 for Robot: I Let a Voice LLM Answer Support Calls for a Week and My Mom Was Caller #4

Every side project I've ever shipped has exactly one support channel: me, half-asleep, answering DMs at 11pm. So when I read yet another thread about AI phone agents, I figured the lab needed a new experiment. The question wasn't whether an LLM could do support. The question was: what would a week of real humans calling a robot about my app reveal about my app?

Spoiler: a lot. Mostly unflattering.

The Build: Cheaper Than You'd Think

The whole stack took one Saturday:

• Twilio number with voice forwarding ($1.15/month plus per-minute rates)
• OpenAI's Realtime API doing speech-to-speech, no separate transcription layer needed
• A system prompt stuffed with my app's README, FAQ, and the last 40 GitHub issues
• A webhook logger that dumped every transcript into a SQLite database, because obviously I was going to want receipts

Total setup time: about five hours, three of which were me fighting Twilio's webhook signature validation on a fresh Ngrok tunnel. If you've done that dance, you know. If you haven't, bring snacks.

The system prompt was the real work. I told the agent it was support for "TileMate," my habit-tracking side project, gave it a friendly Midwest personality, and — critically — instructed it to say "I don't know, I'll flag that for the developer" instead of hallucinating. That instruction failed maybe 30% of the time. More on that in a second.

The Callers: Friends, Mom, and One Guy from Reddit

I posted the number in my app's Discord, my personal newsletter (all 212 subscribers, hi), and one carefully worded r/SideProject post. Then I braced.

Caller #1 was my friend Marcus, who immediately tried to break it. He asked the bot to write a haiku about his procrastination. It did. He asked it to refund money he never spent. It apologized and said a refund was "being processed." There is no refund system. There is no billing system. I nearly fell out of my chair.

Caller #4 was my mom, who did not know she was calling a robot. She called because I told her to. She spent four minutes explaining her phone screen was too small, and the bot — bless its heart — walked her through accessibility settings that don't exist in TileMate. She hung up satisfied. I have complicated feelings about this.

Caller #11 was a stranger from Reddit who asked a genuinely hard question about CSV export edge cases. The bot nailed it, because I had literally pasted the relevant GitHub issue into its context. It sounded brilliant. It was quoting me to me.

By Sunday night: 23 calls, 71 minutes of audio, one database full of transcripts.

The Failure Modes, Ranked

1. The Confident Inventor. When asked about features that don't exist, the bot would sometimes describe them in loving detail. A "streak freeze" mechanic? Sounded great. I never built it. Two callers asked how to buy it.
2. The Endless Apologizer. Interrupt the bot mid-sentence and it enters a politeness death spiral. "I'm so sorry — I apologize — let me start over —"
3. The Accent Wall. My buddy's dad has a thick Boston accent. The transcription fell apart, and the bot started answering questions nobody asked. He rated the experience "a real gas," which I choose to read as sarcasm.
4. The Refund Ghost. See Marcus. Never let your support bot acknowledge financial remedies you cannot provide.

The Part Nobody Warns You About

Here's the thing I didn't expect: reading those transcripts was like getting an X-ray of my own documentation. The bot failed — truly failed, hallucination territory — exactly where my docs were thin. It couldn't answer anything about data export because I have one paragraph about data export. It improvised wildly on pricing because my FAQ says "free for now," which is not a pricing page.

An LLM support agent is a documentation smoke detector. Every confident wrong answer is a smoke alarm going off over a gap in your README. I found nine gaps in one week. I've closed six.

The other thing: people prefer calling it to emailing me. Callers got instant answers at 2am, no wait, no judgment about asking "dumb questions." My mom won't email me about screen sizes. She'll absolutely call a phone number. That's not a bug in human behavior; that's a feature I'd been ignoring.

Would I Keep It On?

Honestly, yeah — with guardrails. I've since added a hard rule: the bot cannot mention refunds, payments, or anything financial, full stop. I added a one-line disclosure at the start of each call ("You're talking to an AI assistant") after caller #7 asked if she could speak to a human and got a weird non-answer. And I review every transcript within 24 hours, because "I'll flag that for the developer" is only honest if a developer actually reads the flags.

The whole thing costs about $9 a month at current call volume. My closet server bill gets a new line item, but the lab notebook gets a whole new chapter.

If you've got a side project with users — even three users, even two of whom are your parents — I'd recommend this experiment. Not because AI support is the future. Because the transcripts will show you, in your users' own words and your bot's confident mistakes, exactly where your project is held together with tape.

Mine was held together with a lot of tape. Working on it.