How Do You Test an AI Chatbot? Try to Break It
How to test an AI chatbot: bait questions, replayed failures and real conversations — and why a green test suite isn't proof your visitors are safe.
From the AnswerRails development blog — plain-English notes on the technology behind trustworthy AI chat.
Most people test a chatbot by asking it questions they know it can answer. That’s a demo, not a test. A test is trying to break it.
Here’s how we try to break ours.

Installing isn’t enough
Before each release we ran an automatic check: install the plugin on a fresh WordPress site, switch it on, make sure nothing crashes. It passed every time.
Then a bug shipped that broke chat conversations on the paid tier. The check installed the plugin and stopped. It never held a conversation.
A person found the bug in minutes, by opening the chat and typing. This was before launch, on our own sites. The rule we took from it: an install test proves the install. Only a conversation proves the product.
Ask it things it can’t answer
AI models like to fill gaps. Ours once invented two products that never existed, with confident descriptions and dead links.
So we keep a stock of bait questions: products that don’t exist, policies that were never written. We ask them regularly on live sites. The pass mark is “I don’t know — would you like to speak to a human?” An assistant that answers everything is dangerous.
Last run, every bait question was declined and handed off. Boring, which is the point.
Replay your failures
When the assistant gets something wrong, that conversation becomes a permanent test. The invented-products question is still asked after every change. A bug you’ve seen once shouldn’t surprise you twice.
We also rehearse rare jobs, like erasing a visitor’s conversation, before anyone asks. The first real request is the wrong time to find a gap.
Green ticks aren’t proof
Our plugin ships with over nine hundred automated checks. They catch broken promises fast. But one afternoon, with all of them green, ninety minutes of ordinary conversation turned up three real defects.
Automated checks prove the parts. Conversations prove the whole.
So before a release we run a road test: realistic conversations through the live widget, with evidence kept for every finding. One run came back with fifteen findings. One was wrong in a useful way: the tester marked an answer as an error because it contradicted a page on the site. The answer was right; the page was out of date. The owner is the final authority on what’s true about the business, not the website and not the tester.
Three questions for any vendor
Ask it something it can’t know, in the demo. If it invents an answer, you’re done. Ask whether real failures become permanent tests. Ask whether a human talks to it before each release.
A chatbot that has only been demonstrated has never been tested. Try to break it first.
AnswerRails is a WordPress plugin bringing safe, grounded AI chat to small-business websites. More at answerrails.com.