Reading the Overview dashboard

What each badge and figure on the AI System Health panel under AnswerRails → Overview means, what produces it, what clears it, and whether you need to do anything. It describes AnswerRails 1.0.0.

What the panel tells you

The AI System Health panel on the Overview says whether chat and knowledge search are working, judged on the calls it has recorded. It has a Chat card and a Knowledge Retrieval card, each with its own badge, and an Overall Status above them.

Overall Status is the worse of the Chat badge and the Knowledge Retrieval badge, in the order Healthy, Recovering, Degraded, Down. It reads Unknown only when both cards read Unknown. One exception: while the local keyword fallback has recently grounded an answer, a Down knowledge search counts as Degraded in Overall, so “Retrieval Down, Overall Degraded” is expected rather than a fault (see the Knowledge Retrieval card).

The badges and the Chat card’s figures are counted over the range you pick on the Overview, and each range counts whole UTC days: 24 Hours counts today and yesterday; 7 Days and 30 Days count those days; 90 Days and All Time both count the 31 days that are kept.

The five states

The Chat badge and the Knowledge Retrieval badge each show one of five states. A failure stays outstanding until the next successful call.

Down

What it means: the provider is refusing requests in a way that retrying cannot clear, or a failure has kept repeating. What produces it: a single failure for no request capacity, a reached usage limit or a rejected API key reads Down straight away. A busy or other failure reads Down once it has repeated 3 or more times in a row over at least 10 minutes. What clears it: a successful call. Down only ever comes from failures that have not yet been followed by a success, so it does not fade on its own with time.

Degraded

What it means: something is wrong but not yet Down. What produces it: the latest request failed (and has not reached Down); or under 95% of the counted calls succeeded with no outage behind them; or the counted calls took more than 10 seconds on average. What clears it: a successful call ends an outstanding failure; the other two causes clear when the success rate is back to 95% or more and the average is back to 10 seconds or less.

Recovering

What it means: nothing is failing now. Requests are succeeding again after an outage that read Down, but under 95% of the counted calls succeeded because that outage’s failures are still in the count. What clears it: the success rate reaching 95%; the outage’s last failure leaving the selected range; a passing Test APIs, which restarts the count; or a new failure, which takes the badge back to Degraded or Down.

Unknown

What it means and what produces it: either no calls were counted in the selected range and no failure is outstanding, or the endpoint is one the selected model cannot use. In the second case its figures are kept, and Test APIs and the automatic checks skip it. What clears it: the first recorded call, or changing to a model that can use the endpoint.

A quiet site does not always read Unknown. An outstanding failure keeps the badge at Down or Degraded even after its calls have left the range, so a site with little traffic can stay Down until a call succeeds.

Healthy

What it means: none of the above. Calls are succeeding, at least 95% of the counted calls succeeded, they averaged 10 seconds or less, and no failure is outstanding.

The Chat card

The Chat card shows the endpoint that is serving your visitors: its badge, Success Rate, Response Time (the average per call, in milliseconds) and Total Calls, all over the selected range. On OpenAI, which endpoint serves visitors follows the model you chose and the structured-outputs setting.

The count, and restarting it

The success rate and the badge are worked out from the calls counted in the selected range. A passing Test APIs restarts the count from that test, on every range. Nothing is deleted. The automatic re-check (below) does not restart the count.

The Recovering line

Only while the Chat badge reads Recovering, the card adds: “Requests have succeeded again since …; the success rate below still counts the outage. Run Test APIs to start the count afresh.” It tells you when requests began succeeding again. A passing Test APIs restarts the count, which ends Recovering.

The Last error line

When the Chat badge reads anything other than Healthy and the last error is under 24 hours old, the card shows Last error: how long ago it happened and a plain-language diagnosis. Hover over it to see the provider’s own error message. This line is history. It tells you what last went wrong, not that chat is failing now; the badge says whether it still is.

Per-API breakdown

Open Per-API breakdown to see each endpoint on its own. On OpenAI there are two sub-cards, Chat Completions API and Responses API, and each is judged only on its own calls. Mistral and Anthropic have one sub-card. On OpenAI, the endpoint that is not serving visitors says it is not serving visitors with your model, when it last recorded a call, and when its next automatic check is due. If the selected model cannot use that endpoint at all, the sub-card says so instead.

Automatic re-checks

When a chat endpoint that the selected model can use reads Down, AnswerRails re-checks that endpoint automatically: 10 minutes after the failure, then after 30 minutes, then after 2 hours, then every 6 hours, stopping at the first success. Saving an API key re-checks a Down endpoint straight away. The badge still needs a recorded success before it turns green. Re-checks are for chat only: knowledge search and moderation are never re-checked automatically.

What you do, and what you wait for

• Key rejected, usage limit reached, or no request capacity: fix it at your provider or under Chat Settings. Saving the key re-checks the endpoint at once.
• Busy: wait. If the busy failure reaches Down, the automatic re-check runs.
• Recovering: nothing is wrong. Wait, or run Test APIs to restart the count.
• Unknown, with no traffic and no failure outstanding: nothing is wrong.
• Down on a quiet site: the failure is still outstanding. The automatic re-check or a passing Test APIs clears it.

The Knowledge Retrieval card

The Knowledge Retrieval card covers knowledge search. Its engine badge shows whether search runs on the local keyword index or on an OpenAI vector store. Below that are Success Rate, Search Time and Searches. With the local index the card shows an index line: chunks, items and the last self-test. With a vector store it shows a store line: files indexed, any that failed, and when it was last checked. Until it has been checked, it says the store has not been checked yet and asks you to run Test APIs.

The local keyword fallback

In vector mode, when the vector search fails, the answer is grounded from the local keyword index instead, and the card counts those uses. While the fallback has recently grounded an answer, a Down knowledge search counts as Degraded in Overall Status.

Why OpenAI appears when you use another provider

In vector mode your knowledge base is synced to an OpenAI vector store and searched there, whichever provider writes the reply. The local copy is the automatic fallback.

Notices and Test APIs

Notices

Two notices can appear on every AnswerRails screen, and both link to Chat Settings:

• “AnswerRails chat is failing: …” appears while a rejected-key failure is outstanding and the error is under 24 hours old. It goes on the next successful call. It also goes after 24 hours with no success, even though the failure is still outstanding and the Chat card still reads Down.
• “AnswerRails chat is not answering visitors: …” appears while your provider reports no request capacity.

Busy and usage-limit failures have no notice. They show on the Chat card.

Test APIs

Test APIs probes three things. It probes each chat endpoint the selected model can use. It probes knowledge search: in vector mode a store check and a search, with the local index a self-test search on one of your site’s own titles. And it probes moderation, with a harmless fixed text. Each probe that passes restarts that service’s count, and the page reloads with the result. It does not probe an endpoint the selected model cannot use, and nothing repeats it automatically for knowledge search or moderation.

Experience Scores and Performance Metrics

API Success Rate, in Experience Scores and in Performance Metrics, is the Chat card’s success rate for the selected range. It is the same figure, not a separate measure.

Quality Score (Pro) is the average grounding score of sampled answers. Deflection Rate (Pro) is the share of sampled conversations that ran to more than two turns with no request to hand off to a person.

Response Time in Performance Metrics is the average reply time across your 50 most recent conversations, in seconds, whatever range you pick. The Chat card’s Response Time is different: it is per call, in milliseconds, over the selected range. The two are expected to differ. Knowledge Items is the number of published items in your knowledge base.

Diagnosis: the badge says X — check Y

Each row names what the Overview shows and what to check. The Last error line on the Chat card usually names the cause in plain language.

The Overview shows	What it means, and what to check
Chat Down, and the notice “AnswerRails chat is failing”	Your provider rejected the API key. Check the key at your provider, enter it under Chat Settings and save. Saving re-checks the endpoint at once. The failure ends on the first successful call; the badge then reads Healthy, or Recovering until the success rate is back to 95%.
Chat Degraded, and the Last error says the provider is busy	The provider is turning requests away for now. Wait. The next successful call ends the failure; the badge returns to Healthy once the success rate is back to 95%.
Chat Down after repeated busy failures	A busy failure has repeated 3 or more times in a row over at least 10 minutes. Wait: the automatic re-check runs 10 minutes after the failure, then after 30 minutes, then 2 hours, then every 6 hours, stopping at the first success.
Chat Down, and the Last error says the usage limit is reached	Your provider account has reached its usage limit. Fix it at your provider; there is no notice for this one.
Chat Down, and the notice “AnswerRails chat is not answering visitors”	Your provider reports no request capacity for your account. Fix it at your provider or under Chat Settings.
An OpenAI sub-card reads Unknown, and says the model cannot use the endpoint	The model you selected cannot use that endpoint, so Test APIs and the automatic checks skip it. Choosing a model that can use it clears the Unknown.
Knowledge Retrieval Down, the fallback count rising, Overall Degraded	The vector search is failing and answers are being grounded from the local keyword index instead. Run Test APIs to re-check the store: knowledge search is never re-checked automatically.
Chat Recovering	Nothing is wrong: requests are succeeding again, and the outage’s failures are still counted. Wait, or run Test APIs to restart the count.
Unknown on a site with little traffic	No calls were counted in the selected range and no failure is outstanding. Nothing is wrong. It clears on the first recorded call.
Down on a site with little traffic	A failure is still outstanding, even if its calls have left the range. A passing Test APIs clears it. For a chat endpoint, the automatic re-check also clears it when one succeeds.
The “chat is failing” notice has gone, but the Chat card still reads Down	The notice shows for 24 hours at most. The rejected-key failure is still outstanding. Fix the key under Chat Settings, as in the first row.