core.models
core.models
Zone A — model serving (BUILD-SPEC §5, §7).
Two lifecycles kept separate: the model lifecycle (pull/update + the two-slot loader) lives here; the agent lifecycle (the factory + registry) arrives in Phase 5. Nothing is baked into Ollama — personas and params are injected at request time.
InferenceClient
Bases: Protocol
Backend-agnostic local inference. Implementations: OllamaClient (default) and
LlamaServerClient. Deliberately EXCLUDES ps/load/unload/list_models — those are
residency-manager operations that exist only because Ollama owns residency; under
dn-local-model-runtime §2.3 residency becomes child-process existence and they have no
counterpart. A protocol that included them would force one implementation to lie.
runtime_checkable buys an honest but shallow isinstance — method presence only, never
signatures. It is used as a test ratchet; the real conformance check is mypy's.
embed(model, inputs, *, keep_alive=None)
Batch-embed inputs. One vector per input, order preserved.
chat(model, messages, *, num_ctx=None, temperature=None, keep_alive=None, think=None)
Single-shot, non-streaming chat. Returns the assistant text.
healthy()
Up AND ready to serve — not merely reachable. See the module docstring.
ContextOverflowError
Bases: LlamaServerError
The prompt did not fit the server's loaded context window.
llama-server's context is fixed at spawn (-c), so this is a LOUD, fail-closed signal that
the window was sized wrong for the traffic — which is exactly why note §2.3 right-sizes the
embedder to 8192 and asks V-D to confirm no embed call can exceed it. The server's own
numbers are carried as attributes, not flattened into prose.
n_prompt_tokens = n_prompt_tokens
instance-attribute
n_ctx = n_ctx
instance-attribute
LlamaServerClient
dataclass
An InferenceClient over one local llama-server process.
One server serves ONE model (note §2.1 D: even Ollama is really N single-model servers behind
a manager), so model is passed through for wire compatibility but does not select anything.
port = DEFAULT_PORT
class-attribute
instance-attribute
host = LOOPBACK_HOST
class-attribute
instance-attribute
request_timeout_s = 120.0
class-attribute
instance-attribute
generation_timeout_s = 600.0
class-attribute
instance-attribute
base_url
property
healthy()
/health: 503 while the model loads → 200 when ready (measured, note §2.1 G).
Returns False rather than raising for BOTH not-ready states — still loading (503) and nothing listening (connection refused) — because a readiness probe that throws is a readiness probe every caller has to wrap. "Up but not ready" is the state a version string could not express, and it is the reason this method exists at all.
embed(model, inputs, *, keep_alive=None)
Batch-embed inputs via /v1/embeddings. One vector per input, order preserved.
keep_alive is accepted for protocol compatibility and DELIBERATELY IGNORED: it is an
Ollama residency knob (how long a third party's timer keeps a model warm). Here residency
is process existence — the model is loaded because we hold the process, and no timer can
evict it (note §2.3). Silently honoring it would be a lie; erroring on it would break the
seam. The whole batch goes in one request, as the Ollama client does; client-side batch
sizing for cancellation granularity is V-E, not this plan's.
chat(model, messages, *, num_ctx=None, temperature=None, keep_alive=None, think=None)
Single-shot, non-streaming chat via /v1/chat/completions.
⚑ Wire-contract only — see the module docstring: no upstream-loadable chat blob exists yet (§2.1 E), so this path has never been exercised against a real model.
num_ctx is accepted and IGNORED because llama-server fixes its window at spawn (-c);
the process manager sizes it per role (§2.3). That is not a silent truncation: a prompt
over the window comes back as a typed ContextOverflowError carrying both numbers, which
is louder than Ollama's per-request reload. keep_alive is ignored for the reason given
on embed. think maps to llama.cpp's chat_template_kwargs.enable_thinking (the Qwen3
hybrid-thinking toggle); that mapping is UNVERIFIED against a loaded model and re-enters
at V-B with the upstream GGUFs.
LlamaServerError
Bases: RuntimeError
Any failure talking to the local llama-server.
TwoSlotLoader
dataclass
config
instance-attribute
client
instance-attribute
registry
instance-attribute
last_load_seconds = 0.0
class-attribute
instance-attribute
last_reconcile
property
The most recent measurement. complete is False => any surface that renders residency
must say PARTIAL. The loader deliberately does not print it: core does not own
presentation (bp-107 §11).
resident_models()
resident_gb()
Registry-costed resident GB. Deliberately unchanged in meaning — callers and the
two-slot algebra both reason over registry models. The ceiling additionally charges
external_resident_gb(); see _check_ceiling.
external_resident_gb()
Ceiling-consuming residency outside the registry (today: the embedder, measured).
Charged by _check_ceiling, reported in ReconcileReport.known_gb.
uncostable_resident()
Resident names nothing can cost. Non-empty => the accounting is partial AND the fail-closed rule is active for non-pinned loads.
reconcile()
Replace belief with measurement: ask Ollama what is ACTUALLY resident.
Called at construction and before every _check_ceiling. Never raises — a probe failure
degrades to today's behaviour and is REPORTED as unreconciled, because Ollama being
unreachable means no load can succeed anyway (so refusing adds nothing but a brick risk).
ps() is the ONE reconciliation source (OllamaClient.ps); no second probe exists, by
design (bp-107 §9). It returns names only, which is the whole reason this returns a report
that can say "partial" instead of a number that pretends to be complete.
ensure(name, *, warm=True)
Make name resident, swapping/evicting as the two-slot rules require.
Refuses (raises MemoryCeilingError) before touching Ollama if it would breach
the ceiling.
⚑ Order matters and is the fix. reconcile() runs FIRST — before the idempotence
early-return, which is what killed the false-resident state (a model Ollama's keep-alive
timer had evicted was still claimed resident and the needed load was skipped), and before
_check_ceiling, which is what killed the false-absent state (the eviction loop below now
iterates a MEASURED set, so the evictions the ceiling arithmetic assumes actually happen
instead of being assumed).
⚑ warm=False does not probe, for the same reason it does not load: it is the
"no live server" affordance the accounting tests run under (module docstring), and ps()
is a server call. Every production caller is warm=True (scheduler/supervisor.py:42);
construction reconciles regardless, so even a warm=False loader starts from measurement.
ensure_tier(tier, *, warm=True)
ensure_pinned(*, warm=True)
Message
Bases: TypedDict
One chat turn, Ollama chat-API shaped. Deliberately duplicated from
core.constitution.Message (structurally identical, so mypy treats them as
interchangeable) to keep this client standalone; runtime-identical to the
plain dict it replaced. Both fields are ReadOnly (PEP 705) — kept in lock-step
with core.constitution.Message so the two stay assignable in both directions
(a ReadOnly/mutable mismatch would break that interchangeability).
role
instance-attribute
content
instance-attribute
OllamaClient
dataclass
config
instance-attribute
version()
list_models()
Names of models available on disk (pullable -> resident).
ps()
Names of models currently loaded (resident in memory).
healthy()
Up and serving — the readiness member of core.models.inference.InferenceClient.
ADDED by bp-115, the one method the seam needs beyond this client's existing surface;
no existing body was touched (Item 1's falsifier). Ollama has no readiness transition to
express — a model is either served or the request blocks — so a non-empty version string
IS its liveness signal. llama-server's form is /health 503→200 during load, which is
exactly why the protocol carries healthy() rather than version(): a caller that read
"responded" as "ready" would dispatch into a still-loading server.
load(model, *, num_ctx=None, keep_alive='30m')
Warm a model into memory without generating. An empty /api/generate with
keep_alive loads it; num_ctx sets the load-time window (changing it reloads).
unload(model)
Evict a model now (keep_alive=0).
embed(model, inputs, *, keep_alive=None)
Batch-embed inputs. Returns one vector per input, order preserved.
chat(model, messages, *, num_ctx=None, temperature=None, keep_alive=None, think=None)
Single-shot, non-streaming chat. Returns the assistant text.
OllamaError
Bases: RuntimeError
Any failure talking to the local Ollama server.
MemoryCeilingError
Bases: RuntimeError
Raised when a requested load would breach the two-slot / usable-RAM budget (Invariant 8). The scheduler refuses breaching work rather than crashing.
Registry
dataclass
config
instance-attribute
pinned
property
by_name(name)
by_tier(tier)
ModelServer
dataclass
config
instance-attribute
client
instance-attribute
loader
instance-attribute
version()
The Ollama server's version string — asked of the LOADER's client, which stays
concretely OllamaClient-typed on purpose (bp-115 §3 Q2: residency-manager operations
have no llama.cpp counterpart, and core/models/loader.py is bp-116's to replace).
version() is deliberately NOT on the inference protocol: llama-server's readiness is a
503→200 transition that a version string cannot express, so the seam carries healthy()
instead. build_model_server hands the same client object to both fields, so this
returns exactly what self.client.version() returned before.
ensure_pinned(*, warm=True)
chat(tier, messages, *, think=None, temperature=None)
build_inference_client(config, *, tier=None)
The PER-ROLE selector (dn-local-model-runtime §4): tier=None is the embedding role,
a tier name reads the [runtime] chat_backend per-tier override. Per role, not global —
that is what makes the embedder cutover (P4) a real, independently reversible step.
Defaults return OllamaClient for every role, so this changes nothing at landing. Flipping
is the owner's, in config/ouroboros.toml, at P4/P5 — never here (plan §9).
get_registry()
build_model_server(config=None)
inference
The backend-agnostic inference seam (dn-local-model-runtime §2.6 P1).
One protocol, two implementations: OllamaClient — today's default, unchanged — and
LlamaServerClient, which speaks llama.cpp's OpenAI-compatible surface. Everything above this
line (the Embedder, the ModelServer, every ingest lane) names a capability, never a vendor.
Why the seam exists at all. It is the whole reversibility story of the runtime migration (note §2.6): with inference reached through a protocol, rolling back any later phase is a config flip rather than a revert, and a third backend (MLX, parked) becomes a cheap experiment instead of a refactor. Nothing observable changes when this module lands — that is the acceptance bar, not a caveat.
Deliberately three methods. ps, load, unload and list_models are NOT here. Those are
residency-manager operations that exist only because Ollama owns residency; under note §2.3
residency becomes child-process existence and they have no llama.cpp counterpart. A protocol
carrying them would force one implementation to lie four times. The residency manager keeps
talking to OllamaClient concretely (core/models/loader.py) until its successor replaces the
question outright.
healthy() rather than version(). llama-server's readiness is a 503→200 transition while
the model loads (note §2.1 G measured it); a version string cannot express "up but not ready",
and a caller that reads "responded" as "ready" would dispatch into a loading server.
Stdlib-only binds the seam, not just one client. The rule is stated in full on
core/models/ollama_client.py:3-6 and is reproduced here as a property of every implementation
of this protocol: a sealed-core inference client must not import a network-capable third-party
package (CONVENTIONS). urllib is permitted because each request targets a 127.0.0.1 literal
(no DNS) that the egress guard core.sealing allows, and the static import firewall
(ops/import_lint.py, NETWORK_ALLOWLIST) audits the exception by filename. See that docstring
for the reasoning rather than a second copy of it.
Message
Bases: TypedDict
One chat turn, Ollama chat-API shaped (bp-006 T2 convention: TypedDict).
Runtime-identical to the plain dict this replaced — TypedDict is erased to
dict — but the shape now crosses module boundaries visibly to the checker.
Both fields are ReadOnly (PEP 705): a turn is a write-once record, so
msg["content"] = … after construction is a type error — the immutability an
assembled context relies on (Invariant 6: nothing nested may rewrite a prior
turn) made unrepresentable, not merely discouraged. Runtime is unchanged.
role
instance-attribute
content
instance-attribute
InferenceClient
Bases: Protocol
Backend-agnostic local inference. Implementations: OllamaClient (default) and
LlamaServerClient. Deliberately EXCLUDES ps/load/unload/list_models — those are
residency-manager operations that exist only because Ollama owns residency; under
dn-local-model-runtime §2.3 residency becomes child-process existence and they have no
counterpart. A protocol that included them would force one implementation to lie.
runtime_checkable buys an honest but shallow isinstance — method presence only, never
signatures. It is used as a test ratchet; the real conformance check is mypy's.
embed(model, inputs, *, keep_alive=None)
Batch-embed inputs. One vector per input, order preserved.
chat(model, messages, *, num_ctx=None, temperature=None, keep_alive=None, think=None)
Single-shot, non-streaming chat. Returns the assistant text.
healthy()
Up AND ready to serve — not merely reachable. See the module docstring.
build_inference_client(config, *, tier=None)
The PER-ROLE selector (dn-local-model-runtime §4): tier=None is the embedding role,
a tier name reads the [runtime] chat_backend per-tier override. Per role, not global —
that is what makes the embedder cutover (P4) a real, independently reversible step.
Defaults return OllamaClient for every role, so this changes nothing at landing. Flipping
is the owner's, in config/ouroboros.toml, at P4/P5 — never here (plan §9).
llama_server_client
Thin HTTP client for a LOCAL llama-server (dn-local-model-runtime §2.6 P1).
The second implementation of core.models.inference.InferenceClient. It speaks the
OpenAI-compatible surface upstream llama.cpp exposes — /v1/chat/completions, /v1/embeddings,
/health — measured working on b10090 (7347430f4), note §2.1 G, not read from documentation.
Stdlib-only by design, exactly as core/models/ollama_client.py:3-6 states the rule for the
Ollama channel: the sealed core must not import a network-capable third-party package
(CONVENTIONS). urllib is network-capable, but every request here targets a 127.0.0.1
LITERAL — never a hostname, so no DNS is involved — and the egress guard (core.sealing) permits
exactly that and blocks everything else. core/models/inference.py states this as a property of
the seam; this file is the second audited exception in ops/import_lint.py's allowlist.
⚑ This client never spawns a server. Spawning, readiness-gating, the budget check and the SIGTERM→grace→SIGKILL stop are the process manager's (note §2.4; bp-116). A client that also spawned would hold two responsibilities and would break the argv-as-capability story that makes the spawned server's egress a tier-2 claim. It connects to a port someone else is listening on.
⚑ Typed errors are kept typed. llama-server answers a too-long prompt with structured JSON
(exceed_context_size_error carrying n_prompt_tokens and n_ctx) rather than Ollama's opaque
string. That is one of the named concrete wins of the migration (note §2.1 G); flattening it into
a generic message at this boundary would throw away part of the reason for migrating. It surfaces
as ContextOverflowError, which carries both numbers as attributes.
⚑ What is NOT verified here. Ollama's chat blobs (qwen3.5:2b/9b, qwen3.6:27b, GGUF arch
qwen35) FAIL to load in upstream llama-server (key qwen35.rope.dimension_sections has wrong
array length; expected 4, got 3 — note §2.1 E). The chat path is therefore exercised against the
WIRE CONTRACT only, never against a loaded chat model; claiming otherwise would be a false
completion claim (plan §7 Item 3 falsifier). Only the embedder blob is portable, and its
cross-runtime cosine floor was measured at 0.999990 (§2.1 F). Real chat verification re-enters at
V-B, when upstream-convention GGUFs are placed.
DEFAULT_PORT = 8080
module-attribute
LOOPBACK_HOST = '127.0.0.1'
module-attribute
LlamaServerError
Bases: RuntimeError
Any failure talking to the local llama-server.
ContextOverflowError
Bases: LlamaServerError
The prompt did not fit the server's loaded context window.
llama-server's context is fixed at spawn (-c), so this is a LOUD, fail-closed signal that
the window was sized wrong for the traffic — which is exactly why note §2.3 right-sizes the
embedder to 8192 and asks V-D to confirm no embed call can exceed it. The server's own
numbers are carried as attributes, not flattened into prose.
n_prompt_tokens = n_prompt_tokens
instance-attribute
n_ctx = n_ctx
instance-attribute
LlamaServerClient
dataclass
An InferenceClient over one local llama-server process.
One server serves ONE model (note §2.1 D: even Ollama is really N single-model servers behind
a manager), so model is passed through for wire compatibility but does not select anything.
port = DEFAULT_PORT
class-attribute
instance-attribute
host = LOOPBACK_HOST
class-attribute
instance-attribute
request_timeout_s = 120.0
class-attribute
instance-attribute
generation_timeout_s = 600.0
class-attribute
instance-attribute
base_url
property
healthy()
/health: 503 while the model loads → 200 when ready (measured, note §2.1 G).
Returns False rather than raising for BOTH not-ready states — still loading (503) and nothing listening (connection refused) — because a readiness probe that throws is a readiness probe every caller has to wrap. "Up but not ready" is the state a version string could not express, and it is the reason this method exists at all.
embed(model, inputs, *, keep_alive=None)
Batch-embed inputs via /v1/embeddings. One vector per input, order preserved.
keep_alive is accepted for protocol compatibility and DELIBERATELY IGNORED: it is an
Ollama residency knob (how long a third party's timer keeps a model warm). Here residency
is process existence — the model is loaded because we hold the process, and no timer can
evict it (note §2.3). Silently honoring it would be a lie; erroring on it would break the
seam. The whole batch goes in one request, as the Ollama client does; client-side batch
sizing for cancellation granularity is V-E, not this plan's.
chat(model, messages, *, num_ctx=None, temperature=None, keep_alive=None, think=None)
Single-shot, non-streaming chat via /v1/chat/completions.
⚑ Wire-contract only — see the module docstring: no upstream-loadable chat blob exists yet (§2.1 E), so this path has never been exercised against a real model.
num_ctx is accepted and IGNORED because llama-server fixes its window at spawn (-c);
the process manager sizes it per role (§2.3). That is not a silent truncation: a prompt
over the window comes back as a typed ContextOverflowError carrying both numbers, which
is louder than Ollama's per-request reload. keep_alive is ignored for the reason given
on embed. think maps to llama.cpp's chat_template_kwargs.enable_thinking (the Qwen3
hybrid-thinking toggle); that mapping is UNVERIFIED against a loaded model and re-enters
at V-B with the upstream GGUFs.
loader
Two-slot model loader (BUILD-SPEC §5).
The model lifecycle's executor: it loads, swaps, and evicts weights while enforcing the hardware ceiling (Invariant 8). The router decides tier/window; this code does the load — model advises, code acts.
Two slots, never more:
* Slot 1 — the pinned tiny model (router + watchdog), kept warm indefinitely.
* Slot 2 — a single swappable worker. Loading a worker evicts the prior worker.
A stretch model that declares evicts_pinned also evicts the pinned model and runs as
the sole resident for its duration (the documented §5 tradeoff).
The ceiling is checked BEFORE any Ollama call, so breaching work is refused, not
half-applied. The warm flag lets the eviction/accounting logic be unit-tested
without a live server.
⚑ RESIDENCY IS MEASURED, NOT BELIEVED (bp-107, finding-0199). _resident used to be an
in-process dict that started EMPTY on every construction, and nothing reconciled it against what
Ollama — a separate, long-lived process that outlives the supervisor — is actually holding. That
made three states reachable, and all three were REPRODUCED LIVE (dn-local-model-runtime §2.1 B,
2026-07-25): false-absent (a fresh loader believed 1 model / 6.6 GB while ollama ps held 2),
guard-pass on a real breach (_check_ceiling passed 23.0 ≤ 24.0 while the true prospective was
25.7 GB, because the eviction loop iterated an empty dict and so nothing was ever really unloaded),
and false-resident (a 0.0 ms stale early-return skipped a load that was needed after Ollama's own
30-minute keep-alive timer evicted a worker). reconcile() now replaces belief with measurement at
construction and before every ceiling check, which is what makes the ceiling an enforcement rather
than an advisory (non-negotiable #8).
⚑ THE ACCOUNTING IS PARTIAL, AND SAYS SO. ps() returns names only, so a resident name with
no registry entry cannot be costed. ReconcileReport.complete is the honesty flag; any surface that
prints residency must render "partial" when it is False. Do not let a partial reconcile read as a
full one — a false claim of completeness is the very defect class this closes, one level up.
⚑ THIS IS THE INTERIM GUARD. dn-local-model-runtime §2.3 replaces this whole class with a
process manager for which residency is child-process existence, and the false-absent and
false-resident states lose their representation instead of being detected. That is bp-116. This
buys correctness for the interval between now and then, and deliberately does not pre-empt it:
max_resident_models keeps counting exactly what it counted before, and resident_gb stays the
declared weights-only constant (finding-0174 is made VISIBLE here, not fixed — see
_MEASURED_NON_REGISTRY_GB).
ReconcileReport
dataclass
What one ps() measurement could and could NOT account for.
⚑ complete is a narrow claim: every resident NAME was costable. It is emphatically NOT a
claim that the GB figures are right — they are still the declared weights-only resident_gb
constants, and finding-0174 (declared vs real, context-dominated) remains open until
dn-local-model-runtime §2.3 lands. A caller must read complete=True as "nothing resident was
invisible to the sum", never as "the sum is correct".
reconciled
instance-attribute
known_gb
instance-attribute
unknown
instance-attribute
complete
instance-attribute
measure(*, reconciled, known_gb, unknown)
classmethod
The ONLY constructor used in anger, so complete cannot drift from its definition.
Item 1's falsifier — "reports complete=True while unknown is non-empty" — is
unreachable by construction rather than merely untested.
TwoSlotLoader
dataclass
config
instance-attribute
client
instance-attribute
registry
instance-attribute
last_load_seconds = 0.0
class-attribute
instance-attribute
last_reconcile
property
The most recent measurement. complete is False => any surface that renders residency
must say PARTIAL. The loader deliberately does not print it: core does not own
presentation (bp-107 §11).
resident_models()
resident_gb()
Registry-costed resident GB. Deliberately unchanged in meaning — callers and the
two-slot algebra both reason over registry models. The ceiling additionally charges
external_resident_gb(); see _check_ceiling.
external_resident_gb()
Ceiling-consuming residency outside the registry (today: the embedder, measured).
Charged by _check_ceiling, reported in ReconcileReport.known_gb.
uncostable_resident()
Resident names nothing can cost. Non-empty => the accounting is partial AND the fail-closed rule is active for non-pinned loads.
reconcile()
Replace belief with measurement: ask Ollama what is ACTUALLY resident.
Called at construction and before every _check_ceiling. Never raises — a probe failure
degrades to today's behaviour and is REPORTED as unreconciled, because Ollama being
unreachable means no load can succeed anyway (so refusing adds nothing but a brick risk).
ps() is the ONE reconciliation source (OllamaClient.ps); no second probe exists, by
design (bp-107 §9). It returns names only, which is the whole reason this returns a report
that can say "partial" instead of a number that pretends to be complete.
ensure(name, *, warm=True)
Make name resident, swapping/evicting as the two-slot rules require.
Refuses (raises MemoryCeilingError) before touching Ollama if it would breach
the ceiling.
⚑ Order matters and is the fix. reconcile() runs FIRST — before the idempotence
early-return, which is what killed the false-resident state (a model Ollama's keep-alive
timer had evicted was still claimed resident and the needed load was skipped), and before
_check_ceiling, which is what killed the false-absent state (the eviction loop below now
iterates a MEASURED set, so the evictions the ceiling arithmetic assumes actually happen
instead of being assumed).
⚑ warm=False does not probe, for the same reason it does not load: it is the
"no live server" affordance the accounting tests run under (module docstring), and ps()
is a server call. Every production caller is warm=True (scheduler/supervisor.py:42);
construction reconciles regardless, so even a warm=False loader starts from measurement.
ensure_tier(tier, *, warm=True)
ensure_pinned(*, warm=True)
ollama_client
Thin HTTP client for the LOCAL Ollama server (BUILD-SPEC §7).
Stdlib-only by design: the sealed core must not import a network-capable third-party
package (CONVENTIONS). urllib is network-capable, but every request here targets the
loopback Ollama endpoint, and the egress guard (core.sealing) permits exactly that
and blocks everything else. Personas and per-call parameters are injected at REQUEST
time via this API — never baked into a Modelfile (CONVENTIONS / BUILD-SPEC §5).
Message
Bases: TypedDict
One chat turn, Ollama chat-API shaped. Deliberately duplicated from
core.constitution.Message (structurally identical, so mypy treats them as
interchangeable) to keep this client standalone; runtime-identical to the
plain dict it replaced. Both fields are ReadOnly (PEP 705) — kept in lock-step
with core.constitution.Message so the two stay assignable in both directions
(a ReadOnly/mutable mismatch would break that interchangeability).
role
instance-attribute
content
instance-attribute
OllamaError
Bases: RuntimeError
Any failure talking to the local Ollama server.
OllamaClient
dataclass
config
instance-attribute
version()
list_models()
Names of models available on disk (pullable -> resident).
ps()
Names of models currently loaded (resident in memory).
healthy()
Up and serving — the readiness member of core.models.inference.InferenceClient.
ADDED by bp-115, the one method the seam needs beyond this client's existing surface;
no existing body was touched (Item 1's falsifier). Ollama has no readiness transition to
express — a model is either served or the request blocks — so a non-empty version string
IS its liveness signal. llama-server's form is /health 503→200 during load, which is
exactly why the protocol carries healthy() rather than version(): a caller that read
"responded" as "ready" would dispatch into a still-loading server.
load(model, *, num_ctx=None, keep_alive='30m')
Warm a model into memory without generating. An empty /api/generate with
keep_alive loads it; num_ctx sets the load-time window (changing it reloads).
unload(model)
Evict a model now (keep_alive=0).
embed(model, inputs, *, keep_alive=None)
Batch-embed inputs. Returns one vector per input, order preserved.
chat(model, messages, *, num_ctx=None, temperature=None, keep_alive=None, think=None)
Single-shot, non-streaming chat. Returns the assistant text.
registry
Model registry + memory-ceiling accounting (BUILD-SPEC §5).
Agents are not models. This is the model lifecycle's reference data: the configured lineup keyed by tier/name, plus the rule for what may be resident at once. The router decides which tier to use; this code only describes and accounts for the weights.
MemoryCeilingError
Bases: RuntimeError
Raised when a requested load would breach the two-slot / usable-RAM budget (Invariant 8). The scheduler refuses breaching work rather than crashing.
Registry
dataclass
config
instance-attribute
pinned
property
by_name(name)
by_tier(tier)
get_registry()
server
ModelServer — the facade agents use to talk to local models.
Combines the registry, the two-slot loader, and the Ollama client so callers say "chat at the synthesis tier" and the right model is made resident first (model advises, code acts). Persona/params are passed through at request time.
ModelServer
dataclass
config
instance-attribute
client
instance-attribute
loader
instance-attribute
version()
The Ollama server's version string — asked of the LOADER's client, which stays
concretely OllamaClient-typed on purpose (bp-115 §3 Q2: residency-manager operations
have no llama.cpp counterpart, and core/models/loader.py is bp-116's to replace).
version() is deliberately NOT on the inference protocol: llama-server's readiness is a
503→200 transition that a version string cannot express, so the seam carries healthy()
instead. build_model_server hands the same client object to both fields, so this
returns exactly what self.client.version() returned before.