Learn Agents from Pi
中文
On this page

05. Streaming Output and the Event Model

An agent without streaming output feels sluggish. After the user submits a task, the model may think for several seconds, then request a tool, and the tool may run for several more seconds. If the interface only refreshes when the final result appears, the user cannot tell whether the system is working, stuck, or has already failed. Streaming events are not a visual optimization — they are the foundation of an agent's observability.

The real difficulty of streaming

Ordinary chat streaming only needs to keep appending text. Agent streaming has to handle far more events:

  • Assistant text deltas.
  • Thinking (reasoning) content deltas — models that support chain-of-thought emit a stretch of reasoning first, and the UI usually needs to render it distinctly.
  • Tool call start, argument deltas, and argument completion.
  • Tool execution start, progress, and end.
  • Turn end.
  • Abort, retry, compaction, and queue updates.

Tool call arguments especially. Many providers emit the JSON arguments in fragments. The UI can show "the model is preparing to call read" early on, but the runtime must wait until the arguments are complete and validated before executing. Executing a half-finished JSON payload as arguments is a common bug in streaming agents.

Half-finished JSON is not entirely useless, though. Mature systems use "repairing parsing" to let the UI see the readable parts of the arguments early: first try to fix common problems (bare control characters inside strings, invalid escapes), then run a truncation-tolerant partial JSON parser, and fall back to an empty object if both fail. The parse result is used only for rendering, never for execution. Execution only accepts the object that was fully validated after the argument stream ended. This boundary is worth writing into a code comment.

The event union

Start by defining a stable set of events. They should not be tied to any UI framework. Take the protocol-layer event shape of a mature system as a reference:

type StreamEvent =
  | { type: "start"; partial: AssistantMessage }
  | { type: "text_start"; contentIndex: number; partial: AssistantMessage }
  | { type: "text_delta"; contentIndex: number; delta: string; partial: AssistantMessage }
  | { type: "text_end"; contentIndex: number; content: string; partial: AssistantMessage }
  | { type: "toolcall_start"; contentIndex: number; partial: AssistantMessage }
  | { type: "toolcall_delta"; contentIndex: number; delta: string; partial: AssistantMessage }
  | { type: "toolcall_end"; contentIndex: number; toolCall: ToolCallBlock; partial: AssistantMessage }
  | { type: "done"; reason: "stop" | "length" | "toolUse"; message: AssistantMessage }
  | { type: "error"; reason: "error" | "aborted"; error: AssistantMessage };

Two design choices are worth noting. First, every event carries a partial — an assistant message that is built up progressively as streaming advances. Consumers do not need to accumulate deltas themselves; at any moment, an event gives them the current complete state to render, which also makes reconnect-after-drop and mid-stream subscription scenarios simple. Second, contentIndex marks which content block in the message the delta belongs to. A single assistant message can contain text, thinking, and multiple tool calls at once; without the index, the UI cannot attribute a delta to the correct block.

done and error are mutually exclusive termination events. error also carries an assistant message: the stop reason is error or aborted, errorMessage explains why, and any content fragments already received are preserved in content. This way a "stream interruption" and a "normal completion" leave through the same typed exit, and neither the log nor the UI needs a special case.

Dual views: an iterable of events and a final message

A streaming interface should ideally support two consumption styles at once:

  • The UI consumes events one by one.
  • The agent loop awaits the final assistant message.

A teaching project can express this idea with a small wrapper:

type EventStream<TEvent, TResult> = {
  events: AsyncIterable<TEvent>;
  result: Promise<TResult>;
};

The provider adapter emits deltas during streaming while accumulating the final assistant message. The agent loop can await result to decide the stop reason; the UI can iterate over events to render in real time. Both come from the same underlying stream — there is no need to send the request twice.

Two layers of events: the protocol stream and the lifecycle

The event union above describes what happens inside a single model request. The agent layer also needs a coarser-grained set of lifecycle events that string together multiple requests, tool executions, and queue state:

agent_start
turn_start
message_start / message_update / message_end
tool_execution_start / tool_execution_update / tool_execution_end
turn_end
agent_end

Protocol-stream events are wrapped into message_update and passed upward. This lets subscribers choose the granularity they care about: a status bar listens only to agent_start and agent_end, the chat area listens to message_update, and a tool panel listens to tool_execution_*. Chapter 06 gives the full ordering guarantees.

Abort is not a leaked exception

When the user presses stop, the underlying request receives an AbortSignal. The runtime should not simply let an AbortError bubble all the way to the top. A better approach is:

  1. Cancel the provider request and any tools currently executing.
  2. Emit an error event with reason aborted, carrying the partial message built so far.
  3. Form an assistant message with stop reason aborted, preserving the text already received.
  4. Write it to the session log.

That way, when the user resumes the session, they can see where the task was interrupted and how far the model got. Extensions and the UI can also clean up resources based on an explicit state.

What you should observe

A streaming turn that reads a file might produce these events:

start
text_delta: Let me check the config file first.
toolcall_start (contentIndex=1)
toolcall_delta: {"path":
toolcall_delta: "src/config.ts"}
toolcall_end: read {"path":"src/config.ts"}
done reason=toolUse
tool_execution_start: read
tool_execution_end: read isError=false

Note that done has reason toolUse: this model request has ended, but the turn has not — once tool execution completes, the next model request must follow. Streaming events tell the user what happened; the stop reason tells the runtime what to do next.

Production tradeoffs

Events are a public contract. Once the UI, SDK, and extensions depend on them, you cannot change fields casually. When designing events, keep them stable, fine-grained, and composable. Do not name an event after the action of some interface component, such as appendToChatBubble; name it after a domain fact, such as text_delta.

Events also need enough correlation ids. A single assistant turn can request multiple tools in parallel, and multiple tools may emit progress at the same time. Without a tool call id, the UI cannot attribute progress to the right tool, and the log cannot be replayed.

Frequency is a real problem too. Fast-scrolling bash output might produce hundreds of updates per second, and rendering each one turns the terminal into a slideshow. Mature systems throttle tool progress events (say, at most one update every 100 milliseconds, with end events not throttled) and cache render results on the consumer side. Whether the throttle lives on the producing side or the consuming side is debatable, but a rate-unlimited event stream wired directly to the UI is guaranteed to break under long output.

Exercise

Add an event stream to the loop from the previous chapter.

Acceptance criteria:

  • The concatenated text deltas match the final assistant message.
  • The partial message carried by every event is structurally valid at any moment.
  • No tool executes before its tool call arguments are complete; the partial arguments are used only for rendering.
  • Every tool execution has started and finished events.
  • After a user abort, the event stream terminates with error(reason=aborted), and the partial text is preserved.
  • Record an event sequence with the faux provider and write an assertion that guarantees a stable event order.