When it says “I don’t know”
Most AI assistants are built to answer. AnswerRails is built to answer only when your content holds the answer — and the decision about which is which is not left to the model.
Why refusing is the feature
A language model will produce a fluent answer to almost anything, including questions it has no basis for answering. On a company website that is not a quirk, it is a liability: an invented delivery time, a made-up returns policy, a link to a page that doesn’t exist. AnswerRails treats a confident wrong answer as the worst outcome available, worse than no answer at all. So the whole design leans the other way. The assistant is given your content, told what it may and may not claim, and then checked on the way out.
What the model is given
Before each answer, AnswerRails searches your knowledge base for the passages most relevant to the question — by default the five best matches — and puts those, and only those, in front of the model. It also supplies a standing list of the real, published pages on your site: their titles and addresses, nothing else. When that list is complete, the model is told plainly that those are the only pages that exist and that if something isn’t in them, it should say so. When your site is too large for the list to be complete, the model is told that instead, and is forbidden from claiming a page doesn’t exist.
The model is also told which links it may use: a page from that list, a source it was just given, or a page it linked earlier in the same conversation. A web address typed in by a visitor is never a reason to link to it.
If you have declared a JSON source as a lookup table — part numbers to product names, say — and a visitor’s question matches one of its keys, AnswerRails runs one extra search for the matched term before answering. That trigger is a deterministic match on your data, not a judgement call by the model.
What is checked on the way out
The model’s reply does not go straight to the visitor. It is returned in a structured form that includes the model’s own signal about whether it could ground the answer, and then a series of checks run in a fixed order:
- Invented internal links. Every link to your own site is checked against the pages that actually exist. A link to a page that doesn’t exist causes the whole turn to be deflected: the visitor gets an honest “I don’t have that” and the invented answer is never shown. One special case: if a dead link has exactly one confident match among your real pages, the link is repaired rather than the answer discarded. Two possible matches, or none, means deflection.
- Unsupported claims. A substantive answer the model produced without grounding in your content is replaced with a short, honest message saying the information isn’t available here. The discarded answer is kept in the log so you can see what would have gone out.
- Bad external links. A reply whose only problem is a broken outside link is served with the link removed.
- Small talk. A greeting or a thank-you is recognised as conversation rather than a question. It gets a normal reply, no Sources list, and is not scored — and if the model tries to smuggle a link into what it called small talk, the small-talk label is rejected and the reply is checked like any other.
None of these checks is a model call. Whether a layer runs, and what it does, is decided by code you can read, on evidence you can inspect in the log.
What the refusal looks like
A deliberate refusal is delivered as an ordinary assistant message in the conversation, not as a red error. Errors are kept for things that actually went wrong — the provider was unreachable, or the visitor hit a rate limit. A visitor should never be left unsure whether the assistant declined or broke.
The wording of the standard refusal points the visitor at your site’s usual contact routes. It does not promise to pass the question to a person, because on the free plugin there is no machinery that would. On Pro, where escalation by email is switched on and a recipient is configured, the assistant is allowed to offer a handoff — and only then. The rule is the same one that governs everything else here: a page or a prompt can ground a fact; only machinery can ground an action.
What you will see in the log
Each turn is recorded with the outcome it earned, so the pattern of refusals is something you can measure rather than guess at. The outcome vocabulary is short:
| Outcome | What it means |
|---|---|
| grounded | Answered from your content, with sources. |
| ungrounded_deflected | The model answered without grounding; the answer was withheld and the honest message served instead. |
| fabricated | The reply contained a link to a page that doesn’t exist; the whole turn was deflected. |
| answered_ungrounded | Served without grounding on a path that could not be checked structurally — visible to you, so you can decide what to do about it. |
| moderated | The visitor’s message was blocked before any answer was attempted. |
| conversational | Small talk; not scored. |
| handoff | Pro only: the turn triggered an escalation to a person. |
Link repairs and lookup-table searches are recorded alongside the outcome as provenance, not as outcomes of their own: a repaired answer keeps the outcome it earned. Historical records are never relabelled when the vocabulary changes.
What you can tune
The base instructions the model is given are editable. The size of the site-page list can be raised from its default of 150 entries (to a ceiling of 500) if your site is large and you want the complete-list guarantee back. Retrieval can be pinned to the local keyword index rather than a vector store. And the single most useful thing you can do on a new install is also the simplest: put your contact details in the knowledge base. On an empty knowledge base, “how do I contact you?” is correctly refused, because nothing grounds it.
