What Is Agentic AI? What Makes LLMs Agentic (2026)

What is agentic AI?

Agentic AI is an AI system that pursues a goal over multiple steps with limited human supervision, deciding for itself which actions to take along the way. In practice it is a large language model running in a loop: it plans, calls a tool, reads the result, and plans again until the task is done or it needs to ask a human.

The most careful definition comes from OpenAI's Practices for Governing Agentic AI Systems, which treats agenticness as a matter of degree: "the degree to which a system can adaptably achieve complex goals in complex environments with limited direct supervision." It breaks that into goal complexity, environmental complexity, adaptability, and independent execution. A chatbot scores low on all four; a coding agent that fixes a failing test suite on its own scores high.

Anthropic's Building Effective Agents draws the line engineers use day to day. It calls all of these agentic systems, then splits them in two: "Workflows are systems where LLMs and tools are orchestrated through predefined code paths," while agents "are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks." The more decisions the model makes instead of your code, the more agentic the system is.

Agentic AI vs AI agents vs generative AI

The three terms overlap, and most confusion comes from using them interchangeably:

For the anatomy of a single agent (the model, tools, memory, and loop) see our guide to what AI agents are. For the frameworks that implement the loop, see open-source AI agent frameworks compared, and for the production layer underneath them, agent runtimes explained. The rest of this post looks one level down: what has to be true of the model for any of this to work.

Standalone and Agentic LLMs

A standalone model answers once and stops. An agentic system runs a loop that thinks, decides, acts, observes, and revises. This enables multistep goals, tool use, memory, and collaboration. For methods see ReAct, Reflexion, and Tree of Thoughts.

A Simple Task, Two Ways

Example query to an LLM: Fix the failing tests in this open source repository, write a minimal patch, and open a pull request with a clear summary.

Memory and State

Agentic LLMs benefit from memory beyond the current context. A useful pattern is a two tier design. Keep a small set of core memories in context and a larger store outside the model. Retrieve by similarity, recency, and importance. See MemGPT and the production framework Letta.

• Episodic memory and reflection. Store observations and actions, then summarize and abstract. See Generative Agents.
• Dynamic organization. Link related memories and let new memories update older ones. See A MEM.
• Practical tips. Separate user profile, project knowledge, and tool history. Age memories that are stale. Log tool inputs and outputs for provenance.

How to Evaluate Agentic LLMs

Prefer execution based tasks over static questions. Measure success, safety, and efficiency.

Tool Calling

When ChatGPT browses the web to answer a question, that is tool calling in action. The model outputs a <tool_calling> marker, or a function call object, the runtime pauses, runs a web search, returns the results into context, and the model continues the answer.

This pattern became popular with Toolformer, which showed how to teach models when to call a tool and how to fill arguments, rather than calling everything all the time.

In agentic systems, tool calls are how LLMs act. The system invokes APIs, runs code, queries a database, or reads and writes files, then observes the outcome and takes the next step. That turns a one shot reply into a simple loop, plan then act then observe then revise.

How to train

Supervised finetuning and RL with human or AI feedback shape both the decision to call and the argument schema. Datasets include structured function calling pairs, for example xlam function calling 60k.

query: Where can I find live giveaways for beta access and games?

answer: [{"name": "live_giveaways_by_type", "arguments": {"type": "beta"}}, {"name": "live_giveaways_by_type", "arguments": {"type": "game"}}]

Tools: [{"name": "live_giveaways_by_type", "description": "Retrieve live giveaways from the GamerPower API based on the specified type.", "parameters": {"type": {"description": "The type of giveaways to retrieve (e.g., game, loot, beta).", "type": "str", "default": "game"}}}]

Planning

Another important agentic characteristic is to make goal driven autonomous decisions without the user giving a step by step guide. This is unlocked with chain of thought reasoning. Thinking out loud in a multistep way gives the model context about what to do next, which is exactly what a plan is.

How to train

Reinforcement learning can reward both the outcome and the reasoning process. See Chain of Thought as Policy.

Outcome reward and process reward

Coding Abilities

Most applications today automate software workflows, so coding skill matters. For how labs train models to excel at coding see StarCoder 3. This involves changes at every stage.

Tokenization

Special tokens support code tasks. For example, <|endoftext|> marks the end of a sequence. <|fim_prefix|>, <|fim_middle|>, and <|fim_suffix|> enable Fill in the Middle. <|fim_pad|> is for padding. <|repo_name|> identifies repositories, and <|file_sep|> separates files for repository level learning.

Pretraining

Mix general language data with code specific data. Use public repositories, pull requests, commits, notebooks, and Kaggle sets. Balance code, math, and text for a coding oriented base model.

File level pretraining

Learn from single files.

<|fim_prefix|>{code_pre}<|fim_suffix|>{code_suf}<|fim_middle|>{code_mid}<|endoftext|>

Repository level pretraining

Extend context across many files and learn project structure.

<|repo_name|>{repo_name}
<|file_sep|>{file_path1}
{file_content1}
<|file_sep|>{file_path2}
{file_content2}
<|file_sep|>{file_path3}
<|fim_prefix|>{code_pre}<|fim_suffix|>{code_suf}<|fim_middle|>{code_fim}<|endoftext|>

Instruction finetuning

Stage one uses many diverse but lower quality instructions. Stage two adds filtered high quality data with rejection sampling and supervised finetuning.

RL with execution feedback

For algorithmic tasks, run tests for correctness in Python, Java, and other languages. For complex snippets, use LLM as a judge.

Safety and Governance

• Use least privilege tools and scopes. Start read only and ask before escalation, see Governing Agentic AI.
• Sandbox code and browsers. Use network allow lists and secrets isolation, see Claude Code Security.
• Add verifiers before actions and auditors after actions. Keep tool logs with provenance. Structured outputs help, see Structured Outputs.

Frequently Asked Questions