From Chatbots to AI Systems

What makes up an AI-enabled system beyond the model: instructions, tools, memory, retrieval, and orchestration.

About 15 minutes

By the end of this lesson you should be able to

Name the parts that can surround a model in an AI-enabled service, and say what each contributes
Trace where a described system gets its facts, who or what decides each step, and what it can change
Mark a system’s boundary: what it can see, what it can change, and what it records

From an answer to a system

Until now the course has mostly followed one pattern: you give an AI tool a task and some context, it gives you an output, and you check it. Many useful services do more than that. A library assistant that answers “when is the citation workshop?” correctly usually does not “know” the schedule. Another part of the service looked it up in an approved source and handed the result to the model. A tool that adds the workshop to your calendar is doing something the model cannot do on its own.

This level is about supervising that kind of service. The first step is to see it clearly: the model is one component, and often not the one that explains why an answer was right or wrong. When you evaluate an AI system, ask about the whole system, not just the model.

What can sit around the model

Not every system has every part below. A plain chat assistant may have little more than a model, instructions, the conversation, and a chat window. The terms are working descriptions; products name and combine them differently.

Model: The trained system that generates the output. By itself it produces text (or images, audio, code) from what it is given; it does not look things up, remember you, or act unless other parts let it.
System or developer instructions: Standing instructions set by whoever built or configured the service: its scope, tone, what to refuse, which sources to use. Users often cannot see them.
User input: The request, plus anything the user supplies with it, such as a pasted passage or an uploaded file.
Context: Everything the model is given for one step: instructions, input, earlier turns, retrieved passages, tool results. The model can only work with what reaches its context (Intermediate Lesson 2).
Retrieval: A component that searches a chosen source, such as a set of documents or a database, and passes the relevant passages into the context. An answer is rarely better than what retrieval found.
Memory and state: What carries over. State is where the work has got to, such as the conversation so far or the progress of a multi-step task; memory is information saved for later sessions, such as preferences. State lets a system continue work; it can also carry an early mistake forward.
Tools: Capabilities the system can invoke: search, read a file, query a database, create a draft, or send, change, or delete something (Lesson 2).
Orchestration: The logic that decides what happens next: a fixed sequence written in advance, a choice the model makes, a person’s choice, or a mix.
Interface: What the user sees and does: a chat box, a form, a button to approve an action, a notice that something was sent.
Permissions: What the system is allowed to reach or change, and under whose identity. A tool the system can call is not the same as an action it is permitted to take.
Logs and observability: Records of what happened in a run: the input, what was retrieved, which tools were called, the output, and any approvals. Records let people review, diagnose, and account for a result.

One pass through a system

A map of the layers a request can pass through. It is not a fixed order: orchestration may run retrieval before the model sees anything, call the model several times, or skip a layer entirely.

User and task. Someone asks for something, for a purpose.
Instructions. The service’s standing rules frame the request.
Model. Generates text: an answer, a plan, a proposed tool call.
Context and state. What the model is given, and what carries over from earlier steps.
Retrieval. Passages from an approved source are fetched into the context.
Tools. The system invokes a capability, such as a search or a calendar entry, if permitted.
Orchestration. Fixed logic, the model, or a person decides the next step.
Output or action. Words for someone to read, or a change in the world.
Review and records. A person checks what matters; the run is logged.

Make the boundary visible

Every system has an edge. Inside it are the sources it can read, the tools it can use, and the records it keeps; outside it is everything it cannot know or touch unless someone supplies it. Three questions locate the edge:

What can it see? Which sources, whose files, how current.
What can it change? Which tools write, send, or delete, and under whose account.
What does it record? What is logged, for how long, and who looks at it.

A fourth question is organizational: who is responsible for each part? The model may come from one company, the service from another, the approved source from a Stanford office, and the decision to rely on the output from you.

Worked example: diagnosing a wrong answer by component

A constructed example. The service and its settings are fictional.

A law library offers a Research Guide Assistant on its public website. A student asks, “Where should I start researching state data-breach notification laws?” and gets a confident answer pointing to a guide that was retired last year.

Research Guide Assistant: components (fictional)
Component	In this system	Question to ask about the wrong answer
Instructions	Answer only from the library’s guides; refer legal questions and anything not covered to a librarian	Did the instructions tell it to prefer current guides?
Retrieval	Searches an index of the library’s research guides	Was the retired guide still in the index?
Model	Drafts an answer from the retrieved guides, with links	Did it describe the retrieved guide accurately?
Orchestration	Fixed: always retrieve, then answer; no other steps	None here; the sequence ran as designed.
Tools and permissions	None that act; read-only access to public guides	None; nothing outside the chat changed.
Logs	Questions and the IDs of retrieved guides, kept for 30 days	Which guide did the log show was retrieved?

The log shows the retired guide was retrieved. The model summarized it faithfully; the fault was in the source index. Replacing the model would not have fixed anything. Removing retired guides from the index, and adding a date check to the instructions, would. Reading the system component by component turns “the AI was wrong” into a problem someone can fix.

Try it: read an event assistant as a system

A fictional system card. Nothing here describes a real Stanford service.

Crown Events Helper. A chat box on a library’s events page. Instructions: answer questions about library events only; cite the event listing; never confirm a registration. Retrieval: searches the library’s approved event listing, refreshed each night. Model: drafts each answer from what retrieval returns. Tool: “create calendar entry,” which can add an event to the user’s university calendar after the user signs in. Orchestration: the model decides whether to call the calendar tool when a user asks to “add” or “save” an event. Logs: questions, retrieved listings, and tool calls.

Which component supplies the event facts?
Which component decides what operation occurs?
Which component would need permission to create a calendar item, and whose permission?
What would need to be logged or reviewed?
A user reports that the helper gave the wrong room. Where would you look first?

Answers and discussion

Retrieval, from the approved listing. The model phrases the answer, but the facts come from the listing; if the listing is wrong or stale, a fluent answer will be too.
Orchestration, which here delegates the choice to the model. A designer could instead use fixed logic: the helper offers an “Add to calendar” button and acts only when the user presses it.
The calendar tool, acting in the user’s account. It needs the user’s authorization to reach the calendar, and a sensible design asks the user to approve each entry. Being able to call the tool is not the same as being permitted to create this entry.
At least: the question, which listing was retrieved and its refresh date, the answer, and any calendar action proposed, approved, declined, or completed. Someone should periodically compare a sample of answers with the listing, and look at every failed or unexpected tool call.
The log: which listing was retrieved, and whether it was current. Then whether the answer matched the listing. Only then the model or the instructions. Reasonable people might start elsewhere, for example by checking whether the room changed after the nightly refresh.

Decision criteria: questions to ask of any AI system

Facts: where do they come from, and how current are they?
Decisions: who or what chooses each step: fixed logic, the model, or a person?
Actions: what can it change, under whose identity, and with whose approval?
Records: what is logged, and who reviews it?
Responsibility: who owns each component, and who owns the result?

If nobody can answer these questions about a service, that is a finding in itself: it is not yet clear enough to rely on for consequential work.

The model is not the product. Instructions, sources, tools, orchestration, permissions, and records shape what an AI service does. Supervise the whole system.

Resources

Anthropic: Building effective agents (opens in a new tab) — a provider’s engineering article (December 2024) describing a model “enhanced with augmentations such as retrieval, tools, and memory” as the basic building block of agentic systems.
OpenAI: Tracing (opens in a new tab) — provider documentation showing what a run record can contain: model responses, tool calls, and delegated work.
Agents Without the Hype, modules 1–3 — optional deeper material on what makes something an agent, its job, and its context.