Module 1 · Lesson 2
LLM vs AI Agent: What Is the Difference?
An LLM produces model outputs. An AI agent is an application that uses a model inside an execution loop with tools, state and control logic.
The terms are often mixed together because agents are usually built on top of large language models. But the distinction matters when you design software, estimate cost or decide what needs to be tested.
An LLM is a model
A large language model accepts input context and generates output according to its learned parameters and the instructions you provide. A direct application can be as simple as:
request = user_message
response = model(request)
return responseThe model may be capable of reasoning about a task, but without an application runtime it cannot independently execute your database query, call your private API or persist application state.
An agent is a system
An agent wraps a model with capabilities and control flow. OpenAI's Agents SDK describes agents as models configured with instructions and tools, with optional handoffs, guardrails and structured outputs. The runner manages the loop around model turns and tool calls.
| Capability | LLM call | Agentic application |
|---|---|---|
| Generate text | Yes | Yes |
| Choose a tool | Can request one if tools are exposed | Runtime can execute and continue |
| Use external data | Only if included in context | Can retrieve it during the run |
| Multi-step execution | Not by itself | Yes, through a loop |
| Persist memory | No application persistence by itself | Can add sessions, checkpoints or stores |
| Human approval | Not an application mechanism | Can pause before actions |
Tool calling alone does not remove the distinction
Modern model APIs can return structured tool-call requests. That still leaves important work to your application: deciding which tools are exposed, validating arguments, executing calls, handling failures, returning results and deciding whether the loop should continue.
You can implement that loop yourself or use an agent framework. OpenAI explicitly distinguishes direct use of the Responses API, where you own the loop, from the Agents SDK, where the runtime can manage turns, tools, guardrails, handoffs and sessions.
Why the distinction matters in production
Testing
A model output is probabilistic. Your tool implementations, authorization rules and business constraints should be deterministic where possible. Tests should separate model behavior from application behavior.
Cost
A single model call may be cheap and predictable. An agent can make several model turns and several tool calls before finishing. Your cost model therefore depends on loop length, context size and tool behavior.
Latency
Each additional model turn or remote tool adds latency. An agent that requires five sequential decisions can feel dramatically slower than a single-response application.
Reliability
Agent systems have more failure surfaces: model decisions, malformed tool arguments, remote APIs, timeouts, partial state and repeated calls. Frameworks can help manage these, but they do not eliminate them.
A useful architecture boundary
Keep business-critical rules outside the model. For example, the model can decide that issuing a refund may be appropriate, but deterministic code should verify order ownership, refund eligibility and limits before a refund API is invoked.
Can you build an agent without an agent framework?
Yes. At its simplest, an agent runtime is a loop: call model, inspect result, execute requested tool, append tool output, repeat. Frameworks become useful when you need sessions, traces, handoffs, structured interrupts, guardrails, retries or multiple orchestration patterns.