# `Rho.LLM`
[🔗](https://github.com/GenericJam/rho/blob/v0.1.0/lib/rho/llm.ex#L1)

The model client the loop talks to. `Rho.LLM.ReqLLM` is the real
one (Anthropic, OpenAI, OpenAI Codex and Google through req_llm, with
the credential from `Rho.Auth`); tests inject a scripted fake.

`c:stream/3` runs in a process the loop owns (and kills on Stop). It calls
`sink` for every streamed delta and returns the whole reply:

  * events: `{:text, delta}`, `{:thinking, delta}`
  * `{:ok, %{text:, thinking:, tool_calls: [%{"id", "name", "arguments"}],
     usage: map | nil, finish_reason: atom}}`, and optionally `images:
     [{mime_type, bytes}]`, pictures the model made (`Rho.Session.assistant/2`
     keeps them in the session; they aren't sent back to the model)
  * `{:error, error}` where `error` is normalized: `{:http, status, message}`,
    `{:rate_limited, limit}` (a 429, or a 529 overloaded, with what the
    response and `Rho.Usage` say about when it lifts: see
    `t:limit/0`), `{:transport, reason}`, `{:signed_out, provider}` (no
    credential for the model's provider), `{:auth, provider, message}` (its
    token couldn't be refreshed) or `{:other, message}`

`describe/1`'s remedies say where the user signs in or switches model
as the host words it (`Rho.Config.hint/1`).

Two kinds of 429 need telling apart (`Rho.Loop` does): a
short throttle ("would exceed your account's rate limit"), worth waiting
out, and a subscription window used up for hours (a window at 100 %, a
Codex "usage limit has been reached"), where waiting is pointless: the
limit's `resets_at` says which.

# `error`

```elixir
@type error() ::
  {:http, integer(), String.t()}
  | {:rate_limited, limit()}
  | {:transport, term()}
  | {:signed_out, Rho.Auth.provider()}
  | {:auth, Rho.Auth.provider(), String.t()}
  | {:other, String.t()}
```

# `limit`

```elixir
@type limit() :: %{
  status: integer(),
  message: String.t(),
  provider: Rho.Auth.provider() | nil,
  retry_after_ms: non_neg_integer() | nil,
  resets_at: integer() | nil,
  window: String.t() | nil
}
```

A rate limit: `retry_after_ms` from the response's `retry-after(-ms)`,
and `resets_at` (unix s) / `window` (its label, e.g. `"5 hours"`, or nil)
when a usage window is used up; `provider` is nil when unknown.

# `reply`

```elixir
@type reply() :: %{
  :text => String.t(),
  :thinking => String.t(),
  :tool_calls => [map()],
  :usage => map() | nil,
  :finish_reason => atom() | nil,
  optional(:images) => [{String.t(), binary()}]
}
```

# `request`

```elixir
@type request() :: %{
  :model => String.t(),
  :system_prompt => String.t(),
  :messages => [ReqLLM.Message.t()],
  :tools => [ReqLLM.Tool.t()],
  :max_tokens => pos_integer(),
  optional(:session_id) => String.t()
}
```

# `stream`

```elixir
@callback stream(request(), opts :: term(), sink :: (term() -> any())) ::
  {:ok, reply()} | {:error, error()}
```

# `clock`

```elixir
@spec clock(integer(), integer()) :: String.t()
```

Unix time `at` on the device's clock: `HH:MM` today, `Mon D HH:MM` on
another day. `now` (unix s) is for tests.

# `describe`

```elixir
@spec describe(error()) :: String.t()
```

What the user sees for a failed call, with the remedy where there is one.

# `rate_limit`

```elixir
@spec rate_limit(error()) :: limit() | nil
```

The rate limit an error is (a 429 or a 529 overloaded, with what is
known about it), or nil for any other error.

# `retryable?`

```elixir
@spec retryable?(error()) :: boolean()
```

Worth retrying: rate limits, provider 5xx, dropped connections.

# `usable`

```elixir
@spec usable(String.t(), Path.t(), integer()) :: :ok | {:error, String.t()}
```

Can `model` take over a run another model's limit stopped: its provider
signed in, and none of its usage windows used up for more than a minute
(`Rho.Usage`)? `:ok` or `{:error, why}`.

---

*Consult [api-reference.md](api-reference.md) for complete listing*