Learn Agents from Pi
中文
On this page

03. Tool Design Fundamentals

Tools are the agent's only channel to the outside world. Design tools badly and the model learns the wrong behavior; make tool output unstable and the loop becomes hard to test; grant tools too much authority and the security boundary becomes decoration. In a usable coding agent, the tool layer has to serve the model, the user interface, the log, and the security policy all at once.

A tool is not a function wrapper

Exposing a local function to the model as-is usually fails. An ordinary function's parameters are written for programmers, and so are its error messages; an agent tool's parameters and errors are consumed by the model. Models are bad at inferring "what to change next" from a stack trace, but very good at correcting a call based on structured, explicit, concise feedback.

A tool definition should answer four questions:

  • When to use this tool.
  • How the parameters are expressed.
  • What the output will contain, and how it may be truncated.
  • How the model can correct itself when the call fails.

For example, do not describe grep as just "searches text." A better description: search the workspace when you do not know where a file lives; make queries as specific as possible; at most the first 100 results are returned, and overly long lines are truncated; if there are too many results, narrow the keywords or restrict the directory. Every number in the description must match the runtime's actual behavior — if the description says 50 and the implementation returns 100, the model will plan its next step on a wrong expectation.

Argument validation

Model output is unknown. Even if the provider claims to return arguments that conform to the schema, the runtime must validate them again — streamed tool args can be truncated, the model may add extra fields, and a restored old session may carry arguments shaped by an old schema.

The teaching project can start with hand-written validation:

type ReadInput = {
  path: string;
};

function parseReadInput(value: unknown): { ok: true; input: ReadInput } | { ok: false; message: string } {
  if (typeof value !== "object" || value === null) {
    return { ok: false, message: "Expected an object with a path field." };
  }
  const record = value as Record<string, unknown>;
  if (typeof record.path !== "string" || record.path.length === 0) {
    return { ok: false, message: "Expected path to be a non-empty string." };
  }
  return { ok: true, input: { path: record.path } };
}

A production system can use JSON Schema, Zod, or Valibot, but the principle is the same: a validation failure must produce a tool result, not crash the runtime. Error messages must be actionable — "path must be a relative path," not "validation failed."

Validation also comes with a counterintuitive lesson: do not be overly strict. Real models frequently stuff one or two fields beyond the schema into the arguments (for example, adding a description to each replacement passed to edit). If your validator hard-rejects any unknown field, you will manufacture a lot of avoidable failed rounds — and every model retry is a real token cost. The mature approach is to add an argument-normalization step before validation: strip harmless extra fields, apply lenient type coercion (a numeric string to a number), and only raise an error for problems that genuinely affect the semantics. Be lenient in what you accept, strict in what you produce.

Write output for the model

Tool output has two audiences: the model and humans. The model needs text that is concise, stable, and something it can keep reasoning from. Humans may need the full diff, command exit codes, elapsed time, the truncation policy, and expandable details. Do not cram every structure the human UI needs into the tool result text.

Have each tool return two layers of results:

type ToolResult<Details> = {
  message: ToolResultMessage;
  details: Details;
};

message goes into the LLM context; details goes into the event stream, the log, or the UI. The edit tool can tell the model "replacement succeeded, 2 lines changed" while handing the UI a structured diff. Truncation information is also part of details: the UI can show "output truncated to 50KB" and offer a way to expand the full output, while the model only needs to know that truncation happened and how to retrieve more. This saves tokens without sacrificing observability.

The read-only toolbox

Before giving the agent write access, implement a read-only toolbox first:

  • read: read a text file, with support for line offsets and line-count limits, truncating from the top when it is too long.
  • ls: list a directory, distinguishing files, directories, symlinks, and hidden entries.
  • grep: search text, cap the number of results, return matching lines and paths.
  • find: find files by name, with limits on traversal scope and result count.

A few implementation details from real systems are worth getting right in the very first version. read's truncation limit can be 2000 lines or 50KB (whichever comes first), and it should handle one edge case: if the very first line exceeds the byte limit, return an explicit explanation rather than an empty string. grep and ls should respect .gitignore, or the model will surface hundreds of noise matches from node_modules; delegating grep to a mature tool like ripgrep is faster and more correct than writing your own traversal. find needs a default result cap too (a common value is around 1000), telling the model to narrow its criteria when it is exceeded rather than truncating silently; it should also accept glob patterns and share the same ignore rules as grep — paths matched by .gitignore, the .git directory, and common build artifacts are excluded from the traversal by default. The real value of a search tool is high-signal-to-noise localization, not dumping every file in the repository at the model. read should also strip the BOM — if that invisible UTF-8 marker slips into the context, the model will forever fail to match when it later tries an exact replacement.

The goal of the read-only tools is not feature completeness; it is to instill in the model the habit of "observe before acting." The system prompt should also state the requirements explicitly: read the target file before editing it; search first when you do not know the path; never guess file contents.

What you should observe

A good read result should look like this:

Read src/config.ts lines 1-42.
Output was truncated after 200 lines. Request a narrower range if needed.

1 export type Config = {
2   model: string;
3   maxTurns: number;
...

It gives the model facts, boundaries, and a suggested next step all at once. Bad output is either "file too long" or tens of thousands of lines pasted in raw. The former leaves the model nothing to continue with; the latter wastes context.

Production tradeoffs

The tool layer needs at least these policies:

  • Paths must first resolve inside the workspace boundary, and .. must not escape it; symlinks must be resolved to their real path before the check.
  • Text reads have to handle encodings and binary files: on detecting a binary, refuse to read and state the file type, rather than stuffing garbled bytes into the context.
  • Long output must be truncated, and the model must be told explicitly that truncation happened and on what dimension it was truncated (line count or bytes).
  • Command-style tools need timeouts and process-tree cleanup — killing only the direct child leaves grandchild processes orphaned and still holding ports.
  • File-writing tools must participate in the per-file write queue to avoid parallel overwrites.
  • Tool results need stable ids so the UI and the log can correlate them.
  • Every tool receives an AbortSignal, and long operations must respond to cancellation at reasonable checkpoints.

There is also one interface worth laying down early: abstract file reads and writes and command execution into an operations object (readText, writeText, exec, and so on), and have tools reach the world through it rather than importing node:fs directly. The local implementation is just one option; when you later want to support an SSH remote workspace, a container sandbox, or an in-memory file system for tests, you swap in a different operations implementation and all the tool logic stays untouched. This is precisely the layer that lets mature systems run the same set of tools in both local and remote environments.

Finally, execution mode. Some tools inherently cannot run in parallel — two bash calls running at once would interfere with each other's output and cwd. The tool definition can declare its execution mode (parallel-safe or must-be-serial), and the executor in chapter 6 schedules them uniformly, rather than each tool adding its own lock.

These policies sound like details, but they decide whether the agent "occasionally demos well" or "can be used in a real repository."

Exercise

Implement the three read-only tools: read, ls, and grep.

Acceptance criteria:

  • A nonexistent path returns isError: true, and the error includes a correctable suggestion.
  • Overly long files are truncated, with the truncation policy stated and a way to retrieve more content.
  • Binary files are refused rather than having garbled bytes stuffed into the context.
  • Every path must resolve inside the workspace, and symlinks are resolved before the check.
  • Extra fields in the arguments do not cause an outright failure; execution continues after normalization.
  • Use the faux provider to make the model grep first and then read, verifying that the two tool calls chain together.