When the bot gets it wrong, who answers for it? The question everyone avoids before production.

An LLM facing customers will get things wrong. Not by intent, but because it produces plausible text from incomplete cues. An answer can sound confident while being false, incomplete, or out of scope. The real subject is therefore not the model's perfection. It is the organization's ability to contain its deviations.

The riskiest scenario is always the same. The bot is connected, a few simple scenarios pass acceptance testing, then the channel opens. The first disputed answer arrives before the roles are fixed. At that stage, the problem is no longer technical. It becomes contractual, operational, and reputational.

A customer bot is acceptable only if its scope, its traces, and its chain of responsibility are written down before go-live. Otherwise, every published sentence can be read as a promise, a binding statement, or a contestable refusal.

The scope must be written before the prompt

A bot is not allowed to say everything. It must answer within a bounded scope, validated by the business, compliance, and legal. Without that frame, the slightest variation in wording can become an enforceable statement.

The mechanism is simple. By default, an LLM cannot tell a commercial answer from a regulated one. It fills the gaps with what looks consistent, not with what is authorized. If you do not list the permitted cases, the mandatory refusals, and the escalations, the model invents continuity where a barrier is needed.

Ask the vendor or the integrator: "Which exact intents can the bot handle, with positive and negative examples?" Also require: "Which wordings are forbidden word for word, and where in the chain are they blocked?" Verify contractually that no answer on price, timing, cancellation, coverage, or eligibility can be issued without an explicit, traceable rule.

Frequent trap: the bot "understands" a sensitive question and answers in a careful tone, but with no authorized basis. The symptom is clear: a plausible, polite answer that is nonetheless impossible to justify line by line. In that case, the scope is not a framing document. It is an illusion of control.

Every answer must leave a usable trace

Fuzzy memory protects nobody. To defend an answer, you must reconstruct the exact exchange, the model version, the injected context, the rule applied, and the source consulted. Without that chain, the audit becomes a matter of opinion.

The right criterion is not "we have logs". The right criterion is: for every interaction, can you produce the exact transcript, the timestamp, the model version identifier, the documents used, and the justification for any refusal? If the answer is no, traceability is decorative.

Ask precisely: "Which logging format do you deliver, and how long do you retain each field?" Then: "Can you export a complete interaction without manual reconstruction or access to internal consoles?" Require that the traces be readable by a lawyer, an auditor, and an operator, not only by an engineer.

Classic trap: technical logs exist, but they do not say why the answer was given. You see latency metrics, confidence scores, sometimes request IDs, but not the authorization rule. The symptom is immediate: impossible to prove why the bot said yes, or why it refused. Without an evidential chain, the dispute wins by default.

Without an exact transcript and an authorization rule, there is no proof. There is only a lost conversation.

Responsibility must be named before go-live

When a bot says something false, someone answers for it. Not "the AI". Not "the platform". A legal entity, an operator, a vendor, or a business owner carries the burden. If that point stays vague, the contract protects only on paper.

The risk mechanism is known. The longer the chain, the more everyone assumes someone else arbitrates. In an incident, that dilution slows the fix, delays notification, and complicates proof of diligence. The bot speaks fast. The organization must be able to react without ambiguity.

Ask in black and white: "Who guarantees what, on which scope, and with what limit of liability?" Then: "Who investigates a report, who blocks the service, and who decides on a customer notification?" Check the RACI matrix on sensitive answers. Who validates the content? Who arbitrates exceptions? Who suspends the channel if the deviation repeats?

Precise trap: a contract that mentions "best efforts" with no remediation procedure and no reaction time. The symptom, in operations, is a team that opens tickets without being able to cut the service. Diluted responsibility always produces the same result: everyone has an opinion, nobody owns the shutdown.

The failure mode must be designed as a normal function

A reliable bot is not a bot that never gets it wrong. It is a bot that knows how to stop. It must recognize uncertainty, decline to answer, and hand over to a human when the rule is not clear-cut.

The real test is not about the easy cases. It is about the grey areas: an ambiguous pricing condition, a contractual clause, regulated data, a journey exception. If the bot improvises to stay useful, it turns uncertainty into assertion. Fallback must therefore be designed as native behavior, not as a patch added afterwards.

Ask: "Which confidence thresholds trigger a refusal or an escalation?" Then: "Which refusal messages are validated by compliance, and in which exact cases does the human handover activate?" Also require a verifiable emergency-stop criterion: who cuts what, how fast, with what proof of shutdown and what incident log.

Common trap: forcing the bot to answer in order to "stay useful". The symptom is an answer that fills the gap instead of flagging it. As soon as a system prefers inventing over stopping, it exceeds the acceptable risk level for a customer channel.

Go-live is acceptable only with proof of governance

Launch is not a technical victory. It is a risk decision. A smooth demonstration or a flattering satisfaction score is not enough to authorize an LLM in production.

Governance must be proven, not merely declared. You must be able to show the authorized scope, the complete traceability, and the contractual and operational responsibility. Without those three pieces, the project remains an experiment dressed up as a service.

Ask: "Can you provide a production release file with the scope, the logs, the escalation rules, and the incident owner?" Then impose an end-to-end test on a sensitive interaction. Obtain the bot's answer, the rule that authorized it, the timestamped trace, and the name of the person who handles the deviation. If a single piece is missing, the green light is premature.

Concrete trap: a go-live approved on simple scenarios, with no proof on sensitive cases. The symptom appears at the first dispute: nobody can explain every sentence that went out in production. The right criterion is not "the bot works". It is "we can defend every answer, line by line, in front of a customer, an auditor, or a judge".