AI Agent

An AI agent is a system that wraps a language model in a loop: instead of producing one response and stopping, it lets the model observe the result of its own actions, decide what to do next, and repeat — turning a single-shot question-answering tool into something that can complete a multi-step task on its own. The core architectural difference from a plain chatbot is that loop: a chatbot maps one prompt to one response, while an agent maps a goal to a sequence of model calls, tool calls, and observations that continues until the goal is met or a stopping condition is hit. At a high level, one iteration of that loop looks like this:

Tool use (also called function calling) is the mechanism that turns a language model from a text generator into something that can actually affect the world, and it works by a surprisingly simple trick. The model is given a list of available tools described in structured text (a search function, a calculator, a database query, a code execution sandbox), and instead of answering directly, it's trained to output a structured request to call one of them when appropriate; the actual tool runs outside the model entirely, in regular code, and its output gets fed back into the model's context as the next input. The model never truly "does" anything itself — it only ever generates text, and the agent framework around it is what interprets that text as an instruction to run real code.

Every step of an agent's loop is a full LLM inference call, which means agentic workflows multiply inference cost in a way a single chatbot response never does. A simple question might take one model call; a multi-step agent task (research a topic, call a search tool three times, synthesize the results, verify an answer, format a final response) might take ten or more sequential model calls, each one reading the model's full context window again, which grows with every added observation. This compounding cost — not model accuracy — is the main practical reason agent frameworks fight hard for efficiency: caching repeated context (prompt caching), using smaller/cheaper models for intermediate reasoning steps and a larger model only for the final answer, and capping the maximum number of loop iterations before giving up.

Agentic reliability, not raw model capability, is the actual bottleneck holding back more autonomous deployment, and it compounds multiplicatively across steps. If a single tool call succeeds correctly 95% of the time, a ten-step agentic task that depends on every step succeeding has roughly a 60% chance of completing the whole task correctly end to end (0.95 to the 10th power) — this multiplicative error compounding is why longer, more autonomous agent chains are disproportionately harder to deploy reliably than the per-step accuracy numbers alone would suggest, and why most production agent systems today keep a human in the loop at key checkpoints rather than running fully autonomously.

Read an AI agent not as a smarter model but as an ordinary control loop wrapped around an unchanged language model: every "decision" the agent makes is still just next-token prediction, every "action" is a separate inference call whose cost and latency compound with every added step, and the real engineering problem agent frameworks solve is managing that compounding cost and compounding error rate, not making the underlying model any more capable than it already was.