Try the live assistant
blog / Can You Trust Your AI Chatbot’s…

Can You Trust Your AI Chatbot’s Email Notifications?

When your website's AI assistant says it has emailed your team, is that true? How honest email notifications work — and what proof of delivery looks like.

From the AnswerRails development blog — plain-English notes on the technology behind trustworthy AI chat.

A visitor asks to speak to a human. Your AI assistant replies: “I’m escalating this to our support team. A human will follow up shortly.” That sentence makes two claims: an email has left your website, and it will arrive somewhere your team reads. This post is about what it takes to make both true.

The afternoon our plugin promised follow-ups nobody would make

During a live test sweep on one of our own sites, we found the site’s email transport was broken — the server had no working way to send email at all. On top of that infrastructure fault sat a product fault.

Seven visitors who asked for a human were told their request had been passed to our support team. Nothing had been sent. And the chat log recorded email sent: true — not a measurement, a constant written into the code. A hardcoded value in a log is a truth claim with no source.

Why had nothing caught it? Our automated tests used a stand-in for the email system that always reported success. A pretend post office that cannot lose a parcel will never catch you promising deliveries you didn’t make. The test suite can now lose parcels on demand.

One detail made it worse. For privacy, the plugin keeps only a masked copy of a visitor’s contact address; the full address travels only in the email itself. So a failed send destroyed the address — while the visitor was thanked and promised a follow-up at it.

No claim without evidence of the send

The repair was a discipline: a claim about a send is gated on the send’s actual result. If the email fails now, the visitor is told so, plainly, with no invitation to leave contact details over a channel we’ve just watched fail. The log records what was observed. And the owner gets a dashboard warning whenever notification emails are failing — it can’t be dismissed, and clears only when a send succeeds.

Diagram: one support request, two honest outcomes. A successful send tells the visitor a human will follow up, claimed only because the send's result says so; a refused send tells the visitor plainly and warns the site owner until a send succeeds. Below, a hardcoded email-sent-true log entry is struck out as a claim with no source, replaced by the send's observed result; and the receiver's own verdict — SPF, DKIM and DMARC all passing — read at the destination mailbox, is proof of delivery rather than proof of postage.
Two claims, two kinds of evidence: the send’s observed result, and the receiver’s verdict at the destination.

Sent is still not arrived

Proof of postage is not proof of delivery. Arrival is decided by the receiving mail service, which stamps every message with the results of three standard checks — SPF, DKIM and DMARC: did this really come from a server allowed to send for your domain? Messages that fail don’t bounce; they rot in a junk folder.

So when we commissioned the email path for our live site, the acceptance test was reading those verdicts at the destination mailbox: pass, pass and pass. A receipt written by the receiver, not a claim made by the sender. A citation is a promise you can check; “message sent” is the same kind of promise.

That header check is a one-off commissioning test. Day to day, what protects you is the send-gated claims and the owner warning — a check, not a hope. (Escalating to your human team is part of our paid tier; the questions below apply to anything that sends email in your name.)

Three questions for any vendor

When the assistant says it has contacted your team, is that checked against the send’s actual result, or printed regardless? When a send fails, what do the logs record, and how would you find out? And has anyone read the authentication results on a real message at the destination mailbox — or does “email works” mean “nobody has complained yet”?

The worst state isn’t a system that fails. It’s one that fails while telling both sides it worked.


AnswerRails is a WordPress plugin bringing safe, grounded AI chat to small-business websites. More at answerrails.com.

← All posts