Rho.Loop (rho v0.1.0)

Copy Markdown View Source

The agent loop for one session: a GenServer with pi's lifecycle (pi-agent-core/src/agent-loop.ts; events in Rho.Events).

A run starts with prompt/2 (or steer/2 / follow_up/2 while idle) and repeats turns: one model call (streamed), then the reply's tool calls in parallel (up to :max_tool_concurrency, 4; one at a time, in the model's order, when any of them is an :exclusive tool, Rho.Tool.concurrency/1), until the model stops calling tools and nothing is queued.

  • steer/2 while running queues a user message injected at the next step boundary: after the current tool batch, before the next model call (or, if the model just stopped, as one more turn).
  • follow_up/2 while running runs after the agent would stop.
  • stop/1 kills the in-flight model call and skips tools not yet started; started tools finish, each sent {:rho_stop, loop} first, so one that waits on others (task's subagents) can wrap up early. The run ends with a notice entry.

The slow parts never run in this process: the model call runs in a monitored worker, each tool in a task under :task_supervisor, so the loop always answers steer / stop. Guards: :max_iterations model calls per run (12), explicit :max_tokens (4096), up to :max_retries (2) retries with exponential backoff on 5xx / transport errors, and with :budget (a data dir) the per-day cost cap (Rho.Budget): checked before each model call, each reply's cost recorded. A run at :max_iterations doesn't just stop: a notice asks the model for a short status (what's done, what's in progress or broken, the next steps) and it gets exactly one more call, with no tools offered, before the run ends (:max_iterations), so the user and the next run know where things stand.

Rate limits (a 429 or a 529 overloaded, Rho.LLM.rate_limit/1) are waited out more patiently, as omp does: up to :rate_limit_retries (8) retries, each after the response's retry-after, or until the used up window resets, else the exponential backoff capped at :rate_limit_cap_ms (60 s), as long as the waits add up to no more than :rate_limit_budget_ms (5 min). A wait that would go past it (a subscription window used up for hours) isn't started. Then the run continues on the fallback model (omp's fallback chain, one link): :fallback_model (default the host's Rho.Settings.fallback_model/1), if :fallback_check (default Rho.LLM.usable/1: signed in, not used up itself) passes, with a model_change entry and a notice saying why; the session's own model comes back (another model_change) when the run ends. Without one, the run ends with the error and what to do: when the limit lifts, or to prompt again to continue.

Compaction (pi's; Rho.Compaction): before each model call, a context over the window (:context_window, default the model's) minus the reserve (:reserve_tokens, default pi's) has its older part summarized by the model into a compaction entry, keeping about :keep_recent_tokens (pi's 20,000) of recent messages. The summary call uses :compaction_model (default the session's) with an explicit :compaction_max_tokens (4096), runs in a worker like a model call and counts toward the cost cap; if it fails, the run ends with an error notice and nothing else is persisted. A call the provider rejects as over the context window is compacted and retried once.

A run about to end (the model stopped calling tools, nothing queued) with items still open on the session's todo list (Rho.Tools.Todo.open_items/2, when todo is offered) gets one more turn, once per run: a notice naming them, as omp's reminder does, so the agent finishes them or says why it can't.

A tool's ctx is %{session_id, call_id, data_dir, loop, withheld}: the loop's pid and the names in :withhold_tools, besides Rho.Tool's three.

A tool that answers with details ({:ok, {:details, output, details}}, Rho.ToolRunner) has them sent to the subscribers with tool_execution_end as it gave them (a Dyn tool's as plain data, Rho.Untrusted.plain/1); what goes in the session is the host's choice, :keep_details (fn tool_name, details -> kept | nil end, called in this process, so keep it quick): its non-nil answer is stored as the toolResult entry's details (pi's), for a screen drawing the session later. The default keeps nothing. A keep function that raises or keeps something JSON can't encode is logged and nothing is stored. Pictures the model makes (the reply's :images, Rho.LLM) go in the assistant entry, as pi's image parts.

Everything is persisted to the session as it happens (Rho.Session, omp/pi's JSONL).

Summary

Functions

A question from another agent (another node's, through the host's clustering): queued like follow_up/3 (a run starts if idle), as a user entry marked "ask" => id and attributed to node from, so the asker can find the answer after it.

Returns a specification to start this module under a supervisor.

%{session, opts}: this loop's session and the options it was started with (no :session / :entries), so a tool can start a loop configured like its caller's (task's subagents: same model, LLM, gate, budget).

The req_llm messages the next model call would send (no system prompt).

Runs text once the agent would otherwise stop; starts a run if idle.

Adds a notice entry (kind :notice / :aside) to the session, as the loop's own are: the user sees it, and the model with the next call. A run in progress gets it when the run ends, so it never lands between a tool call and its result.

Starts a run. {:error, :running} if one is in progress (use steer/3). attachments (Rho.Session.attachment/0) go with the message; text may then be empty.

Switches the model (a req_llm spec) for later calls; idle only.

%{session_id, path, title, model, status, entries, totals, streaming, queue}: what a screen needs to draw the session from scratch.

Options: :session and :entries (from Rho.Session.new/3 or Rho.Session.open/2), :system_prompt, :data_dir (given to tools), :llm ({module, opts}), :tools (:registry or a list of tool modules), :withhold_tools (tool names never offered nor run: task withholds itself from its subagents), :before_tool_call (fn call, ctx -> :allow | {:block, reason} end), :keep_details (fn tool_name, details -> term() | nil end: the part of a tool's details stored in its result entry; nil, the default, stores none), :task_supervisor, :budget (the data dir holding the cost ledger and cap; nil means no cap), :inputs (what the model takes, for attachments and tool pictures; default Rho.Models.inputs/1 of the session's model), and the compaction, rate-limit and fallback options and guards above.

Injects text at the next step boundary; starts a run if idle.

Aborts the model call and unstarted tools; no-op when idle.

Sends pid (the caller by default) {:rho, session_id, event} for every event, until it exits or unsubscribes.

Stops sending the caller this loop's events (subscribe/2).

Functions

ask(loop, text, id, from)

@spec ask(GenServer.server(), String.t(), String.t(), node()) :: :ok

A question from another agent (another node's, through the host's clustering): queued like follow_up/3 (a run starts if idle), as a user entry marked "ask" => id and attributed to node from, so the asker can find the answer after it.

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

config(loop)

@spec config(GenServer.server()) :: %{session: Rho.Session.t(), opts: keyword()}

%{session, opts}: this loop's session and the options it was started with (no :session / :entries), so a tool can start a loop configured like its caller's (task's subagents: same model, LLM, gate, budget).

context(loop)

@spec context(GenServer.server()) :: [ReqLLM.Message.t()]

The req_llm messages the next model call would send (no system prompt).

follow_up(loop, text, attachments \\ [])

@spec follow_up(GenServer.server(), String.t(), [Rho.Session.attachment()]) :: :ok

Runs text once the agent would otherwise stop; starts a run if idle.

note(loop, kind, text)

@spec note(GenServer.server(), :notice | :aside, String.t()) :: :ok

Adds a notice entry (kind :notice / :aside) to the session, as the loop's own are: the user sees it, and the model with the next call. A run in progress gets it when the run ends, so it never lands between a tool call and its result.

prompt(loop, text, attachments \\ [])

@spec prompt(GenServer.server(), String.t(), [Rho.Session.attachment()]) ::
  :ok | {:error, :running}

Starts a run. {:error, :running} if one is in progress (use steer/3). attachments (Rho.Session.attachment/0) go with the message; text may then be empty.

set_model(loop, model)

@spec set_model(GenServer.server(), String.t()) :: :ok | {:error, :running}

Switches the model (a req_llm spec) for later calls; idle only.

snapshot(loop)

@spec snapshot(GenServer.server()) :: map()

%{session_id, path, title, model, status, entries, totals, streaming, queue}: what a screen needs to draw the session from scratch.

start_link(opts)

@spec start_link(keyword()) :: GenServer.on_start()

Options: :session and :entries (from Rho.Session.new/3 or Rho.Session.open/2), :system_prompt, :data_dir (given to tools), :llm ({module, opts}), :tools (:registry or a list of tool modules), :withhold_tools (tool names never offered nor run: task withholds itself from its subagents), :before_tool_call (fn call, ctx -> :allow | {:block, reason} end), :keep_details (fn tool_name, details -> term() | nil end: the part of a tool's details stored in its result entry; nil, the default, stores none), :task_supervisor, :budget (the data dir holding the cost ledger and cap; nil means no cap), :inputs (what the model takes, for attachments and tool pictures; default Rho.Models.inputs/1 of the session's model), and the compaction, rate-limit and fallback options and guards above.

steer(loop, text, attachments \\ [])

@spec steer(GenServer.server(), String.t(), [Rho.Session.attachment()]) :: :ok

Injects text at the next step boundary; starts a run if idle.

stop(loop)

@spec stop(GenServer.server()) :: :ok

Aborts the model call and unstarted tools; no-op when idle.

subscribe(loop, pid \\ self())

@spec subscribe(GenServer.server(), pid()) :: :ok

Sends pid (the caller by default) {:rho, session_id, event} for every event, until it exits or unsubscribes.

unsubscribe(loop)

@spec unsubscribe(GenServer.server()) :: :ok

Stops sending the caller this loop's events (subscribe/2).