Customer support agent with LLM and tools

01 · CONTEXT

A customer service agent is a risk question

Exponent's interview guide, compiled from questions reported by candidates, says that designing an agent has become a standard question, especially at companies whose product is an agent (Sierra, Scale, and Glean).

Among the listed questions are building a customer service agent for a fictional outdoor-gear company and designing an agentic support system for Spotify, both attributed to Sierra.

The same guide sums up that these questions reward guardrails, cost control, and human fallback, and that agent questions are, at bottom, risk management questions.

02 · AGENT OR FIXED WORKFLOW

When the LLM directs the process

Anthropic distinguishes two arrangements: workflows, in which LLMs and tools are orchestrated through predefined code paths, and agents, in which the LLM dynamically directs its own process and tool use.

For Anthropic, customer support combines the familiar chatbot interface with tool integration. Tools can pull customer data, order history, and knowledge base articles; actions such as refunds or ticket updates can be handled by code; and success is measurable through user-defined resolutions.

A useful pattern in this domain is routing: directing different types of queries (general questions, refund requests, technical support) to distinct processes, prompts, and tools, and sending easy questions to smaller, cheaper models.

Agents serve open-ended problems where it is hard or impossible to predict the number of steps and fix the path. In exchange, autonomy brings higher costs and the risk of compounding errors, and Anthropic recommends extensive testing in sandboxed environments, with appropriate guardrails.

03 · BASIC ARCHITECTURE

Knowledge, tools, and the agent loop

A RAG design follows these steps: chunk and index the source documents into a vector store with metadata and, at query time, convert the question into a vector and retrieve a lean set of the best snippets, with hybrid search if needed.

According to Anthropic, agents are typically just LLMs using tools based on environmental feedback in a loop; that is why tool definitions deserve as much prompt engineering attention as the overall prompt.

1. The customer reaches the system through the entry gateway, the single point of access in this design.

2. The orchestrator keeps the loop in which the model uses tools based on feedback from the environment.

3. Help documents are chunked and indexed with metadata; at query time, the system retrieves a lean set of the best snippets.

4. The tool gateway exposes queries on customer data and order history, and actions such as refunds and ticket updates, handled by code.

5. The orders and tickets database is the node that the tools query and modify in this design.

At each step, the agent needs to obtain ground truth from the environment, such as tool call results, to assess progress.

04 · ACTIONS, GUARDRAILS, AND HANDOFF

Who can do what, and when to call a person

The guide's tip is to always design the human fallback and the guardrails: refusing out-of-scope topics, validating the output, and confidence thresholds.

In the guide's reference framework, the action step defines, for agents, the tools and approval gates, and the guardrails step refuses what is out of scope, validates the output, and routes to a human above thresholds.

In the example of an insurance claims agent, the model proposes a decision, but anything above a confidence or monetary-value threshold goes to a person, because a wrong automatic approval is expensive.

Anthropic also describes a guardrail in which one model instance processes queries and another screens them for inappropriate content or requests; according to the text, this tends to work better than having the same LLM call handle both.

1. A model instance screens queries for inappropriate content or requests, separate from the instance that handles the main response.

2. The policy node represents the approval gate, defined together with the agent's tools.

3. In the guide's claims example, anything above a confidence or monetary-value threshold goes to a person; here the approval queue plays that role.

4. The agent can pause for human feedback at checkpoints or when it hits blockers; the human agent is the one who decides in these cases.

5. Decisions are logged with their citations for auditing.

Agents can pause for human feedback at checkpoints or when they hit blockers, and it is common to include stopping conditions, such as a maximum number of iterations, to keep control.

05 · EVALUATION AND COST

Measuring the agent without fooling yourself

The guide says to separate retrieval evaluation from decision or answer evaluation and to log decisions with their citations, for auditing. If an LLM acts as a judge, it must be validated against human labels, since judges drift and carry biases.

The guide also asks that token cost and latency be made explicit: caching, lean retrieval, model routing, and per-request budgets.

The scoping step of the guide's framework covers inputs, outputs, latency budget, cost ceiling, and how much a wrong answer costs.

06 · WRAP-UP

Assumptions out loud

Confidence thresholds, value limits, and cost targets depend on the business. State your assumptions to the interviewer, justify each limit, and show how the design changes if the risk increases.