Compiles a source set into generation n (part of the pipeline's step 2, see
Rho.Dyn) and loads, unloads and purges generations' modules.
Versioned names. The agent writes Dyn.Foo (the namespace,
Rho.Dyn.Config); rewrite/2 turns every Dyn.* alias in the AST
(aliases, module references, template sigils such as ~MOB) into
Dyn.G<n>.*, so generation n loads next to the running one and never
over it: loading a new version of a module the BEAM runs, then loading
again, would kill every process still in the old code. __MODULE__ is
the versioned name, as it should be.
Containment. The compile runs in its own process with a timeout and a
max_heap_size (module bodies run at compile time). If it fails, every
module of generation n it loaded is purged: nothing runs them. Its
errors are formatted in that process (Rho.Untrusted.format/4) and its
diagnostics come back as plain strings; if it dies, its exit reason is
only described (Rho.Untrusted.exit_text/2).
Screens report to the Keeper. Each screen module (as the host's
Rho.Dyn.Screens tells them: mob's use Mob.Screen with a mount/3)
gets a wrapping entry function (mount/3) that registers the screen
process with Rho.Dyn.Keeper.watch/3 before calling the original, so
the Keeper counts its crashes and knows the generation is in use.
Unloading an old generation (unload/1) happens only when no process
runs its code: no module of it is anywhere on a process's stack or its
initial call; then :code.delete/1, and :code.soft_purge/1 (which
refuses while :erlang.check_process_code/2 finds a process in the old
code); anything still in it is purged later (purge_old/1).
Summary
Types
A compiled generation: its binaries, how long the compile took, its
warnings, its compile dependencies (Rho.Dyn.Reuse) and the
files reused from the parent rather than compiled.
Functions
Checks sources and compiles them into generation n. Options:
:keeper (the Keeper screens report to), :src_dir (absolute dir the
sources live in, for file names and diagnostics), :compile_timeout_ms,
:compile_max_heap_mb, :reuse (%{n: parent, deps: parent's deps, beams: %{"Elixir.Dyn.G<parent>.X" => binary}, unchanged: MapSet of files whose source is the parent's}: compile incrementally,
Rho.Dyn.Reuse).
Compiles checked files (Rho.Dyn.Check.run/1's result) into generation n.
A digest of the Core's code as this VM runs it: every module below a
Core prefix (Rho.* and the host's, Rho.Dyn.Config's :core) except
the Dyn generations' (the namespace's), by the MD5 of its loaded code, or
of its .beam on the code path while it isn't loaded (what will load).
The modules are the :rho and host (:app) applications' plus every
loaded one below a Core prefix (a delivered update can add modules);
without an application's .app, every module below a Core prefix on the
code path. A delivered Core (mob_deliver loads it at launch, before the
Dyn layer boots) counts with its delivered code.
The generation a versioned module belongs to, or nil for any other
module. A loaded protocol implementation for a Dyn module, which Elixir
names <Protocol>.<module> (Inspect.Dyn.G3.Card, from a
defimpl or a library's use), belongs to its module's generation:
what says so is its own __impl__ attribute (protocol and target, and
the name they make), not the name alone. The build only lets one
through as that (Rho.Dyn.Check.beam/2).
Loads binaries; one already loaded with identical code is skipped.
Loads generation gen from its stored binaries (checked against the
manifest's hashes). Binaries built by another runtime (an app update
since, stale?/1) are rebuilt from the sources and selftested again
before they load (rebuild/3, then store_rebuild/4). Modules already
loaded with the same code are left alone.
Every loaded module of generation n, its protocol implementations
(Inspect.Dyn.G<n>.Card) included.
The name the agent wrote: Dyn.G3.Notes → "Dyn.Notes",
a loaded Inspect.Dyn.G3.Card (an implementation, see
generation_of/1) → "Inspect.Dyn.Card".
Fully unloads modules nothing runs (a candidate's): purge, delete, purge; what the loop kept of their tool declarations goes too.
Purges the old code of mods that no process executes
(:code.soft_purge/1, which asks :erlang.check_process_code/2);
returns the rest.
Rebuilds generation gen from its stored sources for this runtime and
selftests it: its modules are loaded, nothing is stored (that's
store_rebuild/4), and a failure leaves none of them loaded. Touches
only the code server, so it can run outside the Keeper (it does at
launch: Rho.Dyn.Keeper).
Rewrites Dyn.* (the namespace) to Dyn.G<n>.* throughout ast.
Identifies the runtime binaries were built by: OTP, Elixir, the host
app's version (Rho.Dyn.Config's :app, else rho's) and the Core's
code (core_digest/0), fixed for the VM's lifetime. A Core release, by
cable or over the air, changes it from the next launch on, so every
generation built before is rebuilt and selftested against the new Core
when it loads.
Were gen's stored binaries built by another runtime (runtime/0)?
Stores rebuild/3's binaries and selftest results for gen: the
manifest gets the new hashes and this runtime, and a proven generation
goes back on probation (its code is new). The caller unloads the modules
on an error.
Unloads an old generation's modules if no process runs them. :in_use
leaves everything loaded; {:ok, stuck} deleted them all and purged all
but stuck, whose old code a process is still executing (retry with
purge_old/1).
Types
@type build() :: %{ modules: [{module(), binary()}], compile_ms: non_neg_integer(), warnings: [String.t()], deps: Rho.Dyn.Reuse.deps(), reused: [String.t()] }
A compiled generation: its binaries, how long the compile took, its
warnings, its compile dependencies (Rho.Dyn.Reuse) and the
files reused from the parent rather than compiled.
@type failure() :: {:check, [Rho.Dyn.Check.violation()]} | {:compile, String.t()}
Functions
@spec build(%{required(String.t()) => String.t()}, pos_integer(), keyword()) :: {:ok, build()} | {:error, failure()}
Checks sources and compiles them into generation n. Options:
:keeper (the Keeper screens report to), :src_dir (absolute dir the
sources live in, for file names and diagnostics), :compile_timeout_ms,
:compile_max_heap_mb, :reuse (%{n: parent, deps: parent's deps, beams: %{"Elixir.Dyn.G<parent>.X" => binary}, unchanged: MapSet of files whose source is the parent's}: compile incrementally,
Rho.Dyn.Reuse).
@spec compile([{String.t(), Macro.t()}], pos_integer(), keyword()) :: {:ok, build()} | {:error, {:compile, String.t()}}
Compiles checked files (Rho.Dyn.Check.run/1's result) into generation n.
Compiling loads what it compiles at once, VM-wide, before any check of
the result can run. So the compile process's tracer refuses a module
outside the generation as its definition starts (Rho.Dyn.Trace.guard/3), and
reports every module this compile defines, with what was loaded under
that name before. Afterwards, whatever the outcome, each one other than
the generation's modules and its own protocol implementations
(Rho.Dyn.Check.generation_module?/4) is undone: a new one unloaded, a replaced
one reloaded from its file. A module the compile returned that the
tracer never saw (Module.create/3 with a keyword list, not an env,
traces nothing) and that isn't the generation's is undone too: unloaded,
and reloaded from the code path when a file for it is there. Modules
other processes define meanwhile (an eval, a subagent) are left alone.
Rho.Dyn.Check.beam/2 then rejects a build that had any other.
@spec core_digest() :: String.t()
A digest of the Core's code as this VM runs it: every module below a
Core prefix (Rho.* and the host's, Rho.Dyn.Config's :core) except
the Dyn generations' (the namespace's), by the MD5 of its loaded code, or
of its .beam on the code path while it isn't loaded (what will load).
The modules are the :rho and host (:app) applications' plus every
loaded one below a Core prefix (a delivered update can add modules);
without an application's .app, every module below a Core prefix on the
code path. A delivered Core (mob_deliver loads it at launch, before the
Dyn layer boots) counts with its delivered code.
@spec generation_of(module()) :: pos_integer() | nil
The generation a versioned module belongs to, or nil for any other
module. A loaded protocol implementation for a Dyn module, which Elixir
names <Protocol>.<module> (Inspect.Dyn.G3.Card, from a
defimpl or a library's use), belongs to its module's generation:
what says so is its own __impl__ attribute (protocol and target, and
the name they make), not the name alone. The build only lets one
through as that (Rho.Dyn.Check.beam/2).
Loads binaries; one already loaded with identical code is skipped.
@spec load_generation(Path.t(), Rho.Dyn.Generation.t(), keyword()) :: {:ok, [module()], Rho.Dyn.Generation.t()} | {:error, String.t()}
Loads generation gen from its stored binaries (checked against the
manifest's hashes). Binaries built by another runtime (an app update
since, stale?/1) are rebuilt from the sources and selftested again
before they load (rebuild/3, then store_rebuild/4). Modules already
loaded with the same code are left alone.
@spec loaded(pos_integer()) :: [module()]
Every loaded module of generation n, its protocol implementations
(Inspect.Dyn.G<n>.Card) included.
The name the agent wrote: Dyn.G3.Notes → "Dyn.Notes",
a loaded Inspect.Dyn.G3.Card (an implementation, see
generation_of/1) → "Inspect.Dyn.Card".
@spec purge([module()]) :: :ok
Fully unloads modules nothing runs (a candidate's): purge, delete, purge; what the loop kept of their tool declarations goes too.
Purges the old code of mods that no process executes
(:code.soft_purge/1, which asks :erlang.check_process_code/2);
returns the rest.
@spec rebuild(Path.t(), Rho.Dyn.Generation.t(), keyword()) :: {:ok, build(), [Rho.Dyn.Generation.selftest()]} | {:error, String.t()}
Rebuilds generation gen from its stored sources for this runtime and
selftests it: its modules are loaded, nothing is stored (that's
store_rebuild/4), and a failure leaves none of them loaded. Touches
only the code server, so it can run outside the Keeper (it does at
launch: Rho.Dyn.Keeper).
@spec rewrite(Macro.t(), pos_integer()) :: Macro.t()
Rewrites Dyn.* (the namespace) to Dyn.G<n>.* throughout ast.
@spec runtime() :: String.t()
Identifies the runtime binaries were built by: OTP, Elixir, the host
app's version (Rho.Dyn.Config's :app, else rho's) and the Core's
code (core_digest/0), fixed for the VM's lifetime. A Core release, by
cable or over the air, changes it from the next launch on, so every
generation built before is rebuilt and selftested against the new Core
when it loads.
@spec stale?(Rho.Dyn.Generation.t()) :: boolean()
Were gen's stored binaries built by another runtime (runtime/0)?
@spec store_rebuild(Path.t(), Rho.Dyn.Generation.t(), build(), [ Rho.Dyn.Generation.selftest() ]) :: {:ok, Rho.Dyn.Generation.t()} | {:error, String.t()}
Stores rebuild/3's binaries and selftest results for gen: the
manifest gets the new hashes and this runtime, and a proven generation
goes back on probation (its code is new). The caller unloads the modules
on an error.
Unloads an old generation's modules if no process runs them. :in_use
leaves everything loaded; {:ok, stuck} deleted them all and purged all
but stuck, whose old code a process is still executing (retry with
purge_old/1).
"Runs them" looks at each process's whole stack (its backtrace), not
only the function it is in: a process sleeping in Process.sleep/1,
called from a Dyn function, returns into that function.