What Makes LLMs Agentic?

Standalone and Agentic LLMs

A standalone model answers once and stops. An agentic system runs a loop that thinks, decides, acts, observes, and revises. This enables multistep goals, tool use, memory, and collaboration. For methods see ReAct, Reflexion, and Tree of Thoughts.

A Simple Task, Two Ways

Example query to an LLM: Fix the failing tests in this open source repository, write a minimal patch, and open a pull request with a clear summary.

Memory and State

Agentic LLMs benefit from memory beyond the current context. A useful pattern is a two tier design. Keep a small set of core memories in context and a larger store outside the model. Retrieve by similarity, recency, and importance. See MemGPT and the production framework Letta.

• Episodic memory and reflection. Store observations and actions, then summarize and abstract. See Generative Agents.
• Dynamic organization. Link related memories and let new memories update older ones. See A MEM.
• Practical tips. Separate user profile, project knowledge, and tool history. Age memories that are stale. Log tool inputs and outputs for provenance.

How to Evaluate Agentic LLMs

Prefer execution based tasks over static questions. Measure success, safety, and efficiency.

Tool Calling

When ChatGPT browses the web to answer a question, that is tool calling in action. The model outputs a <tool_calling> marker, or a function call object, the runtime pauses, runs a web search, returns the results into context, and the model continues the answer.

This pattern became popular with Toolformer, which showed how to teach models when to call a tool and how to fill arguments, rather than calling everything all the time.

In agentic systems, tool calls are how LLMs act. The system invokes APIs, runs code, queries a database, or reads and writes files, then observes the outcome and takes the next step. That turns a one shot reply into a simple loop, plan then act then observe then revise.

How to train

Supervised finetuning and RL with human or AI feedback shape both the decision to call and the argument schema. Datasets include structured function calling pairs, for example xlam function calling 60k.

query: Where can I find live giveaways for beta access and games?

answer: [{"name": "live_giveaways_by_type", "arguments": {"type": "beta"}}, {"name": "live_giveaways_by_type", "arguments": {"type": "game"}}]

Tools: [{"name": "live_giveaways_by_type", "description": "Retrieve live giveaways from the GamerPower API based on the specified type.", "parameters": {"type": {"description": "The type of giveaways to retrieve (e.g., game, loot, beta).", "type": "str", "default": "game"}}}]

Planning

Another important agentic characteristic is to make goal driven autonomous decisions without the user giving a step by step guide. This is unlocked with chain of thought reasoning. Thinking out loud in a multistep way gives the model context about what to do next, which is exactly what a plan is.

How to train

Reinforcement learning can reward both the outcome and the reasoning process. See Chain of Thought as Policy.

Outcome reward and process reward

Coding Abilities

Most applications today automate software workflows, so coding skill matters. For how labs train models to excel at coding see StarCoder 3. This involves changes at every stage.

Tokenization

Special tokens support code tasks. For example, <|endoftext|> marks the end of a sequence. <|fim_prefix|>, <|fim_middle|>, and <|fim_suffix|> enable Fill in the Middle. <|fim_pad|> is for padding. <|repo_name|> identifies repositories, and <|file_sep|> separates files for repository level learning.

Pretraining

Mix general language data with code specific data. Use public repositories, pull requests, commits, notebooks, and Kaggle sets. Balance code, math, and text for a coding oriented base model.

File level pretraining

Learn from single files.

<|fim_prefix|>{code_pre}<|fim_suffix|>{code_suf}<|fim_middle|>{code_mid}<|endoftext|>

Repository level pretraining

Extend context across many files and learn project structure.

<|repo_name|>{repo_name}
<|file_sep|>{file_path1}
{file_content1}
<|file_sep|>{file_path2}
{file_content2}
<|file_sep|>{file_path3}
<|fim_prefix|>{code_pre}<|fim_suffix|>{code_suf}<|fim_middle|>{code_fim}<|endoftext|>

Instruction finetuning

Stage one uses many diverse but lower quality instructions. Stage two adds filtered high quality data with rejection sampling and supervised finetuning.

RL with execution feedback

For algorithmic tasks, run tests for correctness in Python, Java, and other languages. For complex snippets, use LLM as a judge.

Safety and Governance

• Use least privilege tools and scopes. Start read only and ask before escalation, see Governing Agentic AI.
• Sandbox code and browsers. Use network allow lists and secrets isolation, see Claude Code Security.
• Add verifiers before actions and auditors after actions. Keep tool logs with provenance. Structured outputs help, see Structured Outputs.