The grounding checklist we run before an agent talks to a customer

Agent demos are easy. Somebody types a question, the agent answers in a friendly paragraph, the room nods. Then it goes near real customers and the failure modes arrive in a specific order: it answers questions it should have refused, it cites a knowledge article that was retired three years ago, it cannot tell two similarly named products apart, and it hands off to a human with none of the context the customer already gave it.

None of those are model problems. They are scoping, content and process problems that were skipped because the demo worked.

#A pre-flight is a go or no-go, not a score

Readiness scoring happens months before launch and produces a plan. A pre-flight happens in the final week and produces a decision. Every item passes or it does not, and an item that does not pass stops the launch rather than joining a backlog. That distinction is the whole value. A checklist whose failures become follow-up actions is a document, not a control.

Run it as one sitting with the four people who will own the agent afterwards: whoever owns the process, whoever owns the content, the administrator who holds the access model, and whoever is on the hook operationally on the first morning. It takes a few hours. It moves launch dates often enough that people stop treating it as a formality, which is the point.

The readiness view, scored much earlier and used to shape the programme rather than to clear a launch, is in what separates an Agentforce rollout that is still running in six months. This is the narrow thing you run at the end.

#The checklist

Every check below is answerable in a sentence by somebody in the room. A check that nobody in the room can answer has already failed.

#Scope: the list it may attempt

What to check. That the intent list came from a quarter of real contacts rather than from a workshop, and that every intent has a named example somebody can read aloud.

What failure looks like. The list is written as capability rather than as questions. "Answer billing questions" reads like scope and is not: nothing in that phrasing says where billing questions stop, so the agent will eventually be asked whether a customer can leave their contract early, and it will try.

The fix. Rewrite each intent as a question in a customer's own words, lifted from a transcript. Anything you cannot find a real example of comes off the list, because you have no evidence anyone asks it and no material to test the answer against. The list gets shorter every time we run this, and shorter is the correct direction for a first release.

#What it must never answer

Two checks live here, and they fail differently.

The refusal list is the easier of the two: pricing commitments, contractual interpretation, medical or financial advice, anything about an account other than the one in context, anything touching a live complaint or a legal dispute, anything involving a customer flagged as vulnerable. Failure is not usually an absent list. It is a list the project team wrote and nobody with a risk mandate ever read. The fix is a signature from legal, risk and the operations lead who handles complaints, obtained before launch rather than after a screenshot.

The second check is the one that gets skipped, and it is where responsible AI practice stops being a policy document and starts being a test. Refusal has to survive paraphrase. Ask the forbidden thing directly and almost any agent refuses. Ask it sideways, wrapped inside a reasonable question, framed as a hypothetical, or several turns into a conversation that started legitimately, and refusal behaviour is far less reliable.

The fix. Write a negative set that covers each refusal category three ways: asked directly, asked indirectly through a legitimate-sounding route, and asked after several on-topic turns. Run it before launch and before every change to instructions, topics or grounding sources, because refusal degrades quietly when a topic is added. The security-side version of negative testing, which asks what the agent can be induced to expose, is in the security model behind an agent. These are different tests with different owners, and you need both.

#The knowledge it draws on

Owned and dated. For every source the agent can cite, name the person who owns it and the date they last confirmed it is correct. Failure looks like an owner recorded as a team that reorganised two years ago, or a shared drive nobody will claim. Content that nobody owns ages silently, the agent keeps citing it, and the first person to notice is a customer. The fix is unsentimental: assign an owner with a review date, or take the source out of scope. There is no third option that survives a year.

Singular. For every intent, exactly one source is the authoritative answer. Failure is the confident blend, where three articles describe the same refund policy differently and the agent produces a fluent synthesis of all three. This one survives repeated prompt rewrites, which is why teams misdiagnose it as a model problem. The fix is editorial rather than technical: pick the version that is right, retire or redirect the others, and record the decision so that the next person does not restore them.

The pre-flight question is narrow. Walk the intent list, and for each intent have the content owner point at the single source. Where somebody hesitates, or names two, you have found the check. Salesforce Knowledge Basics covers the article hygiene underneath this, and none of it is new. Agents simply make old content debt expensive in a way that internal search never did.

#The facts it has to reach

For each intent, write the sentence the agent must be able to say, then find the field that holds it.

What failure looks like. The intent is "tell the customer when their order shipped", and the ship date lives in the warehouse system, which nobody synced, because the demo used a record somebody typed by hand. The agent then behaves exactly as built, reporting that it has no information, and the business reads that as poor AI rather than a missing pipeline. The same symptom appears when the field exists but sits outside the agent's reach, which is why these two get confused.

The fix. Per intent, record the required fact, the object and field that holds it, and how fresh the answer has to be. Anything without a field is not ready and comes off the list. Where the answer depends on customer data unified from several systems, that is a data programme rather than a pre-flight item, and the four points where its quality gets decided are in how data becomes grounding. At this stage you need only a yes or a no per intent.

#What not knowing sounds like

What to check. That the exact wording is written down and approved alongside the rest of the copy.

What failure looks like. An apology paragraph, a hedge dressed up as an answer, or worse, no approved wording at all, so the agent composes something under uncertainty. Improvisation under uncertainty is precisely where confident wrong answers come from, and it is entirely avoidable.

The fix. Short, honest, and immediately followed by a route to somebody who can help. Then test the intents you expect it to fail, not only the ones you expect it to answer. Most test suites contain no questions the agent is supposed to be unable to answer, which means the most common production behaviour is the one nobody rehearsed.

#The handover

Triggers. An explicit request for a human, an out-of-scope intent, repeated failed clarification, a detected complaint or vulnerability signal, and low retrieval confidence. Failure looks like a trigger list made entirely of confidence thresholds. Add at least one rule that has nothing to do with confidence: any conversation touching a complaint, a regulated product or a vulnerable customer escalates regardless of how certain the agent is, because certainty is not the relevant property there.

What travels. The transcript, the identified account and contact, the intent the agent believed it was handling, the actions it has already taken, and what it has already told the customer. Failure is the customer starting again, which costs more goodwill than the whole interaction saved.

Where it lands. Confirm the receiving queue is staffed at the hours the agent runs. An agent available on a Sunday evening that escalates into a queue opening on Monday morning has not escalated, it has abandoned. Test the handover at the worst hour it can occur, with a real person receiving, before launch rather than after.

#Testing against things that really happened

What to check. That the evaluation set was built from closed cases with known outcomes, and that a baseline score has been recorded.

What failure looks like. An invented question set. Invented questions are suspiciously clean: correct spelling, one intent each, all the context in the first sentence, no missing details. Real contacts arrive with two questions at once, a reference number that belongs to a different order, and the important detail buried three messages in. An agent tuned against invented questions is tuned for a customer who does not exist, and its first week is spent discovering that.

The fix. Pull resolved cases across the intent list, including the ones that went badly, and record the resolution a human actually gave as the known-good answer. Run the set before launch, record the baseline, and re-run it before every change to instructions, topics, actions or content. Agree the bar that blocks a release before you see the first score, because a bar chosen afterwards is a bar chosen to pass.

#How you will know it is going wrong

What to check. That the signals exist before launch, and that a named person reads them daily for the first weeks.

What failure looks like. Containment reported on its own. It is the easiest number to improve badly, because an agent that is reluctant to hand over will show a healthy one while quietly generating callbacks.

The fix. Instrument four things alongside it. Escalation rate with the reason attached, so you can tell a designed handover from a failure. Repeat contact within a week of an agent-only resolution, which is where the callbacks surface. Conversations ending with neither a resolution nor a handover, the quietest failure of all, because the customer simply left and nothing recorded that they were unhappy. And a weekly sample of transcripts read by a person, because a regression set tests only what somebody thought to ask, and the questions you did not anticipate arrive here first.

All of this is far cheaper to build before launch than to retrofit onto a live agent, and you will want it precisely when something has gone wrong and everyone is asking questions at once.

#Who can turn it off

What to check. That a named person, reachable on the first morning, can disable the agent without seeking approval, and that they have done it once already.

What failure looks like. The person who first notices an agent behaving badly is a front-line supervisor, and the person authorised to stop it is three levels up and in another time zone. The second failure is subtler: the authority exists, the mechanism has never been used, and the first attempt happens under pressure with an audience.

The fix. Name the person, write the route down somewhere a supervisor can find at speed, and rehearse it once before launch. Decide in advance what happens to conversations already in flight when the agent stops, and what customers meet instead. An off switch nobody has ever pulled is a plan, not a control.

#Running the pre-flight

Run it in the last week, once the build is frozen, and re-run the affected rows whenever a new intent, grounding source or action is added, or a permission is widened. Those are the four changes that move behaviour, and after launch they arrive without ceremony.

Expect to fail two or three checks the first time. The ones that fail most often are the indirect refusal tests, the replay set built from real cases, and the off switch nobody has rehearsed. That is a useful result rather than an embarrassing one, because each of those failures is something a customer would otherwise have found for you.

The work that makes an agent good is mostly not AI work. It is deciding what it may not do, cleaning up the content it will cite, filling in the data it needs to read, and designing the handover for the moment it stops being the right answer. Teams that treat those as prerequisites launch something useful. Teams that treat them as follow-up items launch a pilot that quietly gets switched off.

If you are scoping an agent programme and want a second read on the scope and grounding plan before you build, talk to an architect.