Rho.LLMCatalog (rho v0.1.0)

Copy Markdown View Source

Gives llm_db (the model catalog req_llm resolves models through) a snapshot that works on a phone.

llm_db reads its packaged priv/llm_db/snapshot.json (10 MB, 230 providers) at runtime, but deps' priv/ isn't shipped to a mob device and Application.app_dir/2 doesn't resolve there. compile_embed: true works but costs a 12.9 MB beam and a 2.4-20 s first lookup on a 1-scheduler phone BEAM. rho only talks to Anthropic, OpenAI (req_llm resolves openai_codex: models through openai's) and Google, so this module trims the packaged snapshot to those providers at compile time (its own snapshot_id, so llm_db's strict integrity check still holds), and install!/0 writes it to the data dir and points config :llm_db, :snapshot_path at it before the first lookup (Rho.Supervisor does this when it starts).

Summary

Functions

Writes the trimmed snapshot to the data dir and points llm_db at it. Call before the first model lookup; Rho.Supervisor does (its :llm_catalog option), so only a host that looks models up before starting it needs to.

The providers the trimmed snapshot keeps.

Functions

install!()

@spec install!() :: :ok

Writes the trimmed snapshot to the data dir and points llm_db at it. Call before the first model lookup; Rho.Supervisor does (its :llm_catalog option), so only a host that looks models up before starting it needs to.

providers()

@spec providers() :: [String.t()]

The providers the trimmed snapshot keeps.