The agent loop for one session: a GenServer with pi's lifecycle
(pi-agent-core/src/agent-loop.ts; events in Rho.Events).
A run starts with prompt/2 (or steer/2 / follow_up/2 while idle) and
repeats turns: one model call (streamed), then the reply's tool calls in
parallel (up to :max_tool_concurrency, 4; one at a time, in the
model's order, when any of them is an :exclusive tool,
Rho.Tool.concurrency/1), until the model stops calling tools
and nothing is queued.
steer/2while running queues a user message injected at the next step boundary: after the current tool batch, before the next model call (or, if the model just stopped, as one more turn).follow_up/2while running runs after the agent would stop.stop/1kills the in-flight model call and skips tools not yet started; started tools finish, each sent{:rho_stop, loop}first, so one that waits on others (task's subagents) can wrap up early. The run ends with a notice entry.
The slow parts never run in this process: the model call runs in a
monitored worker, each tool in a task under :task_supervisor, so the
loop always answers steer / stop. Guards: :max_iterations model calls
per run (12), explicit :max_tokens (4096), up to :max_retries (2)
retries with exponential backoff on 5xx / transport errors, and
with :budget (a data dir) the per-day cost cap (Rho.Budget):
checked before each model call, each reply's cost recorded. A run at
:max_iterations doesn't just stop: a notice asks the model for a short
status (what's done, what's in progress or broken, the next steps) and
it gets exactly one more call, with no tools offered, before the run
ends (:max_iterations), so the user and the next run know where
things stand.
Rate limits (a 429 or a 529 overloaded, Rho.LLM.rate_limit/1)
are waited out more patiently, as omp does: up to :rate_limit_retries
(8) retries, each after the response's retry-after, or until the used
up window resets, else the exponential backoff capped at
:rate_limit_cap_ms (60 s), as long as the waits add up to no more
than :rate_limit_budget_ms (5 min). A wait that would go past it (a
subscription window used up for hours) isn't started. Then the run
continues on the fallback model (omp's fallback chain, one link):
:fallback_model (default the host's Rho.Settings.fallback_model/1),
if :fallback_check (default Rho.LLM.usable/1: signed in,
not used up itself) passes, with a model_change entry and a notice
saying why; the session's own model comes back (another model_change)
when the run ends. Without one, the run ends with the error and what to
do: when the limit lifts, or to prompt again to continue.
Compaction (pi's; Rho.Compaction): before each model call, a
context over the window (:context_window, default the model's) minus
the reserve (:reserve_tokens, default pi's) has its older part
summarized by the model into a compaction entry, keeping about
:keep_recent_tokens (pi's 20,000) of recent messages. The summary call
uses :compaction_model (default the session's) with an explicit
:compaction_max_tokens (4096), runs in a worker like a model call and
counts toward the cost cap; if it fails, the run ends with an error
notice and nothing else is persisted. A call the provider rejects as
over the context window is compacted and retried once.
A run about to end (the model stopped calling tools, nothing queued)
with items still open on the session's todo list
(Rho.Tools.Todo.open_items/2, when todo is offered) gets
one more turn, once per run: a notice naming them, as omp's reminder
does, so the agent finishes them or says why it can't.
A tool's ctx is %{session_id, call_id, data_dir, loop, withheld}: the
loop's pid and the names in :withhold_tools, besides
Rho.Tool's three.
A tool that answers with details ({:ok, {:details, output, details}},
Rho.ToolRunner) has them sent to the subscribers with
tool_execution_end as it gave them (a Dyn tool's as plain data,
Rho.Untrusted.plain/1); what goes in the session is the host's choice,
:keep_details (fn tool_name, details -> kept | nil end, called in
this process, so keep it quick): its non-nil answer is stored as the
toolResult entry's details (pi's), for a screen drawing the session
later. The default keeps nothing. A keep function that raises or keeps
something JSON can't encode is logged and nothing is stored. Pictures
the model makes (the reply's :images, Rho.LLM) go in the
assistant entry, as pi's image parts.
Everything is persisted to the session as it happens
(Rho.Session, omp/pi's JSONL).
Summary
Functions
A question from another agent (another node's, through the host's
clustering): queued like follow_up/3 (a run starts if idle), as a user
entry marked "ask" => id and attributed to node from, so the asker
can find the answer after it.
Returns a specification to start this module under a supervisor.
%{session, opts}: this loop's session and the options it was started
with (no :session / :entries), so a tool can start a loop configured
like its caller's (task's subagents: same model, LLM, gate, budget).
The req_llm messages the next model call would send (no system prompt).
Runs text once the agent would otherwise stop; starts a run if idle.
Adds a notice entry (kind :notice / :aside) to the session, as the
loop's own are: the user sees it, and the model with the next call. A
run in progress gets it when the run ends, so it never lands between a
tool call and its result.
Starts a run. {:error, :running} if one is in progress (use steer/3).
attachments (Rho.Session.attachment/0) go with the message;
text may then be empty.
Switches the model (a req_llm spec) for later calls; idle only.
%{session_id, path, title, model, status, entries, totals, streaming, queue}: what a screen needs to draw the session from scratch.
Options: :session and :entries (from Rho.Session.new/3 or
Rho.Session.open/2), :system_prompt, :data_dir (given to tools),
:llm ({module, opts}), :tools (:registry or a list of tool
modules), :withhold_tools (tool names never offered nor run: task
withholds itself from its subagents), :before_tool_call (fn call, ctx -> :allow | {:block, reason} end), :keep_details (fn tool_name, details -> term() | nil end: the part of a tool's details stored in its
result entry; nil, the default, stores none), :task_supervisor, :budget
(the data dir holding the cost ledger and cap; nil means no cap),
:inputs (what the model takes, for attachments and tool pictures;
default Rho.Models.inputs/1 of the session's model), and the
compaction, rate-limit and fallback options and guards above.
Injects text at the next step boundary; starts a run if idle.
Aborts the model call and unstarted tools; no-op when idle.
Sends pid (the caller by default) {:rho, session_id, event}
for every event, until it exits or unsubscribes.
Stops sending the caller this loop's events (subscribe/2).
Functions
@spec ask(GenServer.server(), String.t(), String.t(), node()) :: :ok
A question from another agent (another node's, through the host's
clustering): queued like follow_up/3 (a run starts if idle), as a user
entry marked "ask" => id and attributed to node from, so the asker
can find the answer after it.
Returns a specification to start this module under a supervisor.
See Supervisor.
@spec config(GenServer.server()) :: %{session: Rho.Session.t(), opts: keyword()}
%{session, opts}: this loop's session and the options it was started
with (no :session / :entries), so a tool can start a loop configured
like its caller's (task's subagents: same model, LLM, gate, budget).
@spec context(GenServer.server()) :: [ReqLLM.Message.t()]
The req_llm messages the next model call would send (no system prompt).
@spec follow_up(GenServer.server(), String.t(), [Rho.Session.attachment()]) :: :ok
Runs text once the agent would otherwise stop; starts a run if idle.
@spec note(GenServer.server(), :notice | :aside, String.t()) :: :ok
Adds a notice entry (kind :notice / :aside) to the session, as the
loop's own are: the user sees it, and the model with the next call. A
run in progress gets it when the run ends, so it never lands between a
tool call and its result.
@spec prompt(GenServer.server(), String.t(), [Rho.Session.attachment()]) :: :ok | {:error, :running}
Starts a run. {:error, :running} if one is in progress (use steer/3).
attachments (Rho.Session.attachment/0) go with the message;
text may then be empty.
@spec set_model(GenServer.server(), String.t()) :: :ok | {:error, :running}
Switches the model (a req_llm spec) for later calls; idle only.
@spec snapshot(GenServer.server()) :: map()
%{session_id, path, title, model, status, entries, totals, streaming, queue}: what a screen needs to draw the session from scratch.
@spec start_link(keyword()) :: GenServer.on_start()
Options: :session and :entries (from Rho.Session.new/3 or
Rho.Session.open/2), :system_prompt, :data_dir (given to tools),
:llm ({module, opts}), :tools (:registry or a list of tool
modules), :withhold_tools (tool names never offered nor run: task
withholds itself from its subagents), :before_tool_call (fn call, ctx -> :allow | {:block, reason} end), :keep_details (fn tool_name, details -> term() | nil end: the part of a tool's details stored in its
result entry; nil, the default, stores none), :task_supervisor, :budget
(the data dir holding the cost ledger and cap; nil means no cap),
:inputs (what the model takes, for attachments and tool pictures;
default Rho.Models.inputs/1 of the session's model), and the
compaction, rate-limit and fallback options and guards above.
@spec steer(GenServer.server(), String.t(), [Rho.Session.attachment()]) :: :ok
Injects text at the next step boundary; starts a run if idle.
@spec stop(GenServer.server()) :: :ok
Aborts the model call and unstarted tools; no-op when idle.
@spec subscribe(GenServer.server(), pid()) :: :ok
Sends pid (the caller by default) {:rho, session_id, event}
for every event, until it exits or unsubscribes.
@spec unsubscribe(GenServer.server()) :: :ok
Stops sending the caller this loop's events (subscribe/2).