scheduler — top-level modules
scheduler.budget
Deterministic context budgeter (BUILD-SPEC §13).
A tokenizer + assembler — code, not a model. It composes each agent invocation to fit the active model's window with headroom for the reply, in the §13 priority order:
Constitution -> role -> retrieved RAG chunks -> history -> tool outputs -> task
When it won't fit, it trims in the §13 order: tighten retrieval depth first (the primary
lever — retrieval is usually over-fetched), then compact history (sliding window, oldest
first), then truncate tool outputs, and — if even the mandatory frame won't fit — flag
escalate so the caller routes to a larger-window tier rather than silently dropping the
Constitution. The Constitution, role, and task are never trimmed (Invariant 6); keeping the
Constitution lean is therefore context-budget discipline, not just style.
Token counts are a deterministic ESTIMATE (no model in the loop); we bias slightly high and
reserve reply headroom so the estimate is safe. A real tokenizer can be injected via
Budgeter(estimator=...) without changing callers.
DEFAULT_REPLY_RESERVE = 1024
module-attribute
Estimator = Callable[[str], int]
module-attribute
ContextParts
dataclass
role
instance-attribute
task
instance-attribute
retrieved = ()
class-attribute
instance-attribute
history = ()
class-attribute
instance-attribute
tool_outputs = ()
class-attribute
instance-attribute
constitution = None
class-attribute
instance-attribute
BudgetReport
dataclass
window
instance-attribute
reserve
instance-attribute
used_tokens
instance-attribute
retrieved_kept
instance-attribute
retrieved_dropped
instance-attribute
history_kept
instance-attribute
history_dropped
instance-attribute
tool_truncated
instance-attribute
fits
instance-attribute
escalate
instance-attribute
BudgetedContext
dataclass
messages
instance-attribute
report
instance-attribute
ConstitutionFrameError
Bases: RuntimeError
Refused: a caller tried to assemble a context whose outermost frame is not the canonical Constitution (Invariant 6/9) without a deliberate, visible override. The Constitution is a fixed point, not caller-substitutable content — closing the Threat-B assembly-logic seam.
Budgeter
dataclass
window
instance-attribute
reserve = DEFAULT_REPLY_RESERVE
class-attribute
instance-attribute
estimator = estimate_tokens
class-attribute
instance-attribute
assemble(parts, *, allow_constitution_override=False)
estimate_tokens(text)
Deterministic token estimate (~4 chars/token, rounded up). Stable across runs and biased slightly high so budgeting with reply headroom stays safe.
suggest_num_ctx(p95_tokens, *, headroom_frac=0.25, floor=2048, step=1024)
Right-size a role's load-time window from tracked usage (§13): p95 + headroom,
rounded up to a step multiple, never below floor. The deterministic basis for the
per-(model, role) window safe-lever (§14) — the OS agent may tune within bounds.
scheduler.cron
Cron / trough jobs — wiring the curator + dreaming agents into the supervisor (BUILD-SPEC §9, §13).
These are the §9 cognitive-tier jobs that "earn the big model": dreaming synthesis and
curation. Both kinds already route to the synthesis tier (scheduler.router), and the
supervisor's foreground gate keeps that tier out of foreground time (HEAVY_TIERS) — so
these run trough-only, never concurrent with the owner's use (§13), and the two-slot
loader's ceiling + swap discipline apply to them for free.
This module only builds the handlers and the enqueue helpers; the supervisor owns the loop.
A full dreaming pass is run-to-completion here (the dreamer caps max_clusters); rewriting
it as checkpointed per-cluster steps (the queue.checkpoint seam, roadmap §7) is a later
refinement if a pass ever grows long enough to want to yield mid-way.
DREAM_KIND = 'dream'
module-attribute
CURATE_KIND = 'curate'
module-attribute
SHADOW_KIND = 'shadow'
module-attribute
CHAT_EVENTS_KIND = 'chat_events'
module-attribute
INTEGRATE_KIND = 'integrate'
module-attribute
Handler = Callable[[Job], 'str | None']
module-attribute
dream_handler(dreamer)
curate_handler(curator)
cron_handlers(dreamer, curator)
The handler map for a supervisor that runs the trough jobs.
enqueue_dream(queue, router)
shadow_handler(runner)
enqueue_shadow(queue, router)
enqueue_curate(queue, router)
chat_events_handler(projector, *, max_per_pass)
enqueue_chat_events(queue, router)
Enqueue one background L1 projection pass. project() is incremental (a session is skipped
when its transcript digest is unchanged) and idempotent, so duplicate jobs are harmless.
integrate_handler(integrator, *, max_per_pass)
enqueue_integrate(queue, router)
Enqueue one background integration pass. integrate() is incremental (a session is skipped
when its L1 digest is unchanged) and idempotent, so duplicate jobs are harmless.
research_handler(airlock, embedder, store)
Run one research job: reconstruct the de-identified criteria from the payload, drive
emit → collect → rank, and return a plain reading list. If no criteria is present, or no
result is back from the fetcher yet, it degrades to a plain message — never raises.
enqueue_research(queue, router, criteria)
Enqueue a research job (synthesis tier, background — trough-gated). The payload carries the
ALREADY de-identified criteria (to_request()), never raw query text (Inv 11): the criteria
was scrubbed by research_criteria/deidentify at the enqueue boundary, and emit() will
re-assert cleanliness on the way out.
scheduler.interface
Wire the Ambassador / interface gateway into the scheduler (Track B / B1).
The interface substrate (edge gateway, core inbox, the Ambassador) was built but never
scheduled — scheduler/cron.py had no reference to it. This module is that missing wiring,
and it lives on the SCHEDULER side because it owns the queue: the Ambassador never imports the
scheduler (it stays pure + testable), so the delegation seam (task → gate → queue) and the
completed-result surfacing are injected from here as plain closures over the queue.
Two job kinds (scheduler/router.py):
* ambassador — the inbox-drain tick: drive CoreInbox.process_once() (pinned tier,
reactive) — for the scheduled/daemon path.
* ambassador_task — the Ambassador's DELEGATED heavy work: a deep grounded answer on the
synthesis tier, run trough-gated by the supervisor, result surfaced later.
ConversationRuntime is the in-process driver the CLI (scripts/talk.py) and the e2e test use:
owner text → gateway → inbox → Ambassador → reply, with run_pending_tasks() standing in for
the supervisor to complete delegated jobs between turns.
AMBASSADOR_KIND = 'ambassador'
module-attribute
AMBASSADOR_TASK_KIND = 'ambassador_task'
module-attribute
Handler = Callable[[Job], 'str | None']
module-attribute
ConversationRuntime
dataclass
inbox
instance-attribute
gateway
instance-attribute
adapter
instance-attribute
queue
instance-attribute
task_handler
instance-attribute
research_handler = None
class-attribute
instance-attribute
send(text, *, conversation='default')
Drive one full turn in-process: owner text → reply text (through the real gateway → filesystem handoff → core inbox → Ambassador → handoff → gateway path).
run_pending_tasks()
Complete any queued delegated tasks (the supervisor's job; the CLI calls this between
turns so a delegated result is ready to surface on the owner's next message). Handles both
the general ambassador_task and — when wired — the airlock research kind.
ambassador_inbox_handler(inbox)
Drain the core inbox each tick (the scheduled path). The per-message Ambassador reasoning
happens inside process_once via the inbox's handler.
ambassador_task_handler(librarian)
Run one DELEGATED task: a deep grounded answer over the mirror. Returns the answer text, which the supervisor stores as the job result for the Ambassador to surface later.
enqueue_ambassador_inbox(queue, router)
Enqueue one inbox-drain tick (pinned tier, reactive).
build_task_delegation(queue, router, *, gate=None, librarian=None)
Return (delegate, pending_results) closures over the queue. delegate records the task
in the gate (the routed-request ledger — visible, never auto-approved) and enqueues the
delegated job; pending_results reads completed jobs back for the conversation. The
Ambassador holds only these closures — never the queue itself.
A research-shaped TASK (external-literature cue, is_research_request) routes to the airlock
"research" kind instead of the general librarian.answer path — but ONLY the de-identified
criteria cross into the payload: research_criteria scrubs the raw query at THIS boundary
(Inv 11), the gate ledger records only the scrubbed topic, and no raw conversation text
reaches the outbound path. Without a librarian, the research route is off (mirror only).
build_conversation_runtime(config=None, *, server=None, embedder=None, store=None, drift=None, airlock=None)
Wire the full delegating Ambassador + inbox + gateway + queue for in-process use.
server/embedder/store/airlock are injectable (offline CLI + tests). The delegated-task
librarian runs on the synthesis tier (heavy work) over the SAME store the Ambassador reads,
so a delegated result lands where the next conversation can find it. The same librarian gives
the delegate its research_criteria (de-identify) seam; airlock defaults to the configured
core-side airlock, and the research trough handler completes research jobs between turns.
scheduler.presence
Foreground-presence detection (BUILD-SPEC §13 foreground check).
Heavy synthesis must not fire while the owner is actively using the machine. This reads the OS HID idle time (deterministic, no model) and reports whether the owner is present. The idle source is injectable so the scheduler is testable off-host and a non-macOS worker can supply its own probe.
Fail safe: if idle time is unknown, assume the owner IS present — never run heavy batch work on a blind guess.
DEFAULT_IDLE_THRESHOLD_S = 300.0
module-attribute
IdleProbe = Callable[[], 'float | None']
module-attribute
Presence
dataclass
idle_probe = macos_idle_seconds
class-attribute
instance-attribute
threshold_s = DEFAULT_IDLE_THRESHOLD_S
class-attribute
instance-attribute
assume_present_when_unknown = True
class-attribute
instance-attribute
idle_seconds()
foreground_active()
True if the owner is actively using the machine (so heavy tiers are gated).
macos_idle_seconds()
Seconds since the last HID (keyboard/mouse) event, via ioreg. None if unavailable
(e.g. not macOS, or the field is missing).
scheduler.queue
Durable job queue — the scheduler's heartbeat (BUILD-SPEC §8, §13; roadmap §7).
SQLite, WAL mode, single-writer by design: one supervisor owns this queue, so there is no write contention to reason about. The queue is the single safe serialization point — agents are config (re-composed per invocation from the stores), not OS processes, so "restoring" a job is cheap; the only heavyweight cost is a model load, which is what the scheduler is built to minimize.
Scheduling is cooperative and acts at job boundaries (roadmap §7): claim() selects the
next job by priority, skipping tiers the caller says are currently blocked (the foreground
gate), and — within the top-priority band — prefers a job that needs no model swap. A
reactive escalation is just a high-priority job; it is dispatched next, never mid-generation.
checkpoint/resume support long jobs (dreaming, curation) written as yielding steps.
Two hygiene properties keep the queue bounded and self-healing (bp-101, findings 0170/0173):
enqueue coalesces an idempotent kind onto the QUEUED row that is already waiting instead of
stacking a duplicate behind a busy worker, and sweep_orphans reclaims RUNNING rows stranded by
a run that died without finishing them. Both are keyed off _IDEMPOTENT_KINDS — see its comment for
what earns membership, and never widen it by assumption.
PRIORITY_REACTIVE = 0
module-attribute
PRIORITY_INTERACTIVE = 10
module-attribute
PRIORITY_DEFAULT = 50
module-attribute
PRIORITY_BACKGROUND = 100
module-attribute
AgingPolicy
dataclass
Anti-starvation aging (gap G6 — the liveness fix). A QUEUED job's EFFECTIVE priority
improves (its number falls) the longer it waits, so background work (dreaming, curation)
eventually outranks a perpetual stream of newer higher-priority jobs instead of starving
under sustained foreground load — ◇ queued jobs eventually run.
Bounds, deliberately conservative:
* a job that has waited < step_seconds ages zero steps, so NORMAL-load ordering is
unchanged (jobs are usually claimed within seconds of enqueue);
* aging never lifts a job above floor (default = INTERACTIVE), so an aged background
job can come to tie with interactive work and win on FIFO, but can NEVER preempt a
genuine REACTIVE escalation (a low-memory alarm must still go first — if those arrive
perpetually the system is in crisis and background SHOULD wait).
step_seconds = 900.0
class-attribute
instance-attribute
step = 10
class-attribute
instance-attribute
floor = PRIORITY_INTERACTIVE
class-attribute
instance-attribute
Job
dataclass
id
instance-attribute
kind
instance-attribute
tier
instance-attribute
num_ctx
instance-attribute
priority
instance-attribute
state
instance-attribute
payload
instance-attribute
result
instance-attribute
error
instance-attribute
attempts
instance-attribute
checkpoint
instance-attribute
created_at
instance-attribute
started_at
instance-attribute
finished_at
instance-attribute
claimed_by_run = None
class-attribute
instance-attribute
lease_expires_at = None
class-attribute
instance-attribute
load_key
property
The (tier, window) that must be resident to run this job. Changing either forces a model reload (§13), so the supervisor batches jobs sharing a load_key.
OrphanSweep
dataclass
What one sweep_orphans pass did — the job ids it requeued and the ones it failed. Empty
tuples mean a clean previous exit (the normal case), which is why the sweep is safe to run on
every start.
lease_expired is the third tuple and the one this pass did NOT act on: rows THIS run owns
whose deadline has lapsed. They are reported, never reclaimed — reclaiming a row whose stamp
says a live run holds it is the double-execution falsifier (finding-0173 / the sweep's own
guard), and enforcement is a later plan's (dn-supervision-and-liveness §2.6; bp-109 §9). So
total deliberately counts only what MOVED: it is the answer to "what did this sweep do?",
and a reported-but-untouched row did not move.
requeued = ()
class-attribute
instance-attribute
failed = ()
class-attribute
instance-attribute
lease_expired = ()
class-attribute
instance-attribute
total
property
render()
JobQueue
dataclass
path
instance-attribute
aging = field(default_factory=AgingPolicy)
class-attribute
instance-attribute
active_run_id = None
class-attribute
instance-attribute
job_budgets = field(default_factory=dict)
class-attribute
instance-attribute
enqueue(kind, tier, num_ctx, *, priority=PRIORITY_DEFAULT, payload=None)
Add a job and return it. Coalescing (extension, finding-0170): for a kind in
_IDEMPOTENT_KINDS (chat_sync, vault_sync, code_sync, code_backfill,
chat_events, integrate) this returns the job ALREADY WAITING instead of inserting a
duplicate. Every other kind behaves exactly as before — an unconditional INSERT.
Four properties make the collapse safe to reason about — the fourth is a correction,
not a feature: a checkpointed row WAS a collapse target and should never have been (V6,
read off the code in bp-105 journal CP1 and confirmed by dn-supervision-and-liveness
V6 before any batch-yield protocol lands):
- Only QUEUED rows collapse. A job that is RUNNING is past the point where a new
request can be folded into it (it may already have read the state the new request is
about), so enqueueing
code_backfillwhile one runs still inserts. Dropping that row would silently cancel the follow-up pass — the falsifier this rule exists to prevent. - The key is
(kind, payload), neverkindalone, so a payload-bearing job is never swallowed by an unrelated one.tier/num_ctxmust match too — a strictly narrower key that can only ever create a row, never drop one (routing is deterministic per kind today, so it never fires; it is here so a future re-route cannot silently mis-tier a job). - The FIRST row wins, and keeps its
created_at(plan Q3): anti-starvation aging (_effective_priority) measures wait fromcreated_at, so collapsing onto a fresh row would reset the clock and could starve a kind that sustained load keeps re-enqueueing. The one field a collapse may improve ispriority— if the incoming request is more urgent than the waiting row, the waiting row is promoted rather than the urgency lost. - A CHECKPOINTED row is not a collapse target (
AND checkpoint IS NULL).checkpointre-queues a partially-advanced job with its resume token still on the row, so it sits in QUEUED matching every other clause of this key. Without this clause a freshenqueue("code_backfill", …)returned that half-advanced row: the caller asked for a full re-derivation and was silently handed a resume from mid-pass, with nothing to re-derive the units the earlier pass had already claimed to cover. Like thetier/num_ctxnarrowing this can only ever CREATE a row, never drop one — the conservative direction this docstring commits to in its last paragraph.
Payload matching is on the stored JSON text, so two dicts with the same items in a different key order do NOT collapse. That errs toward an extra row, never a lost job.
claim(*, loaded_key=None, blocked_tiers=frozenset(), now=None)
Select + mark RUNNING the next eligible job (§13 policy): highest EFFECTIVE priority
first (priority + anti-starvation aging, gap G6); within the top band prefer the job
needing no model swap (matching loaded_key), then FIFO; skip tiers in blocked_tiers
(the foreground gate) — they stay QUEUED and are revisited once the block clears.
Returns None if nothing is runnable now.
Stamps claimed_by_run = self.active_run_id on the row it takes (extension,
finding-0173). A supervisor that calls sweep_orphans at start therefore leaves the
running rows TRUSTWORTHY: each one is either this run's live work or was already
reclaimed. Without that call the stamp is NULL and running means only what it used to.
Also stamps the lease (extension, dn-supervision-and-liveness §2.6). This is the queue's
ONLY constructor of a RUNNING row — nothing else in this file writes state = RUNNING, a
fact tests/unit/test_queue_leases.py re-derives from the AST on every run rather than
trusting this sentence — so stamping here is what lets a reader treat an expired deadline as
orphanhood by definition instead of waiting for a sweep someone must remember to call.
lease_expires_at = started_at + job_budgets[kind], computed off the same single clock read
as started_at so deadline - started_at == budget exactly; NULL when the kind has no
configured budget, which is every kind by default.
Note what checkpointing buys for free: checkpoint clears the lease and re-queues, and the
next claim stamps a fresh one, so for a yielding lane the deadline is per batch, not
per job-elapsed — the shape §2.10 requires ("deadlines must be per-batch, or a healthy
14-hour backfill dies at hour N on schedule"). A non-yielding lane gets a per-job-elapsed
deadline, which is precisely why no budget is configured for one yet (bp-110 lands the
yield protocol for code_backfill).
The selection policy above is untouched by any of this: the stamp is written after chosen
is decided, and the deadline is stamped, never ENFORCED here (§9 — a queue that terminates
its own jobs would be the ledger written by the actor it must record).
complete(job_id, result=None)
fail(job_id, error)
defer(job_id, reason)
Park a job that cannot run under current conditions (e.g. ceiling breach). Not
re-selected until revive_deferred() puts it back when conditions change.
Called on a row claim has already put in RUNNING (supervisor.py defers on a ceiling
breach), so it is one of the edges that ends a claim and therefore clears the lease.
revive_deferred()
Return deferred jobs to QUEUED (call when conditions change, e.g. RAM freed).
sweep_orphans(active_run_id)
Reclaim RUNNING rows left behind by a run that died without finishing them
(finding-0173 — job 300246 is the live example). Idempotent kinds go back to QUEUED;
everything else is FAILED with an explicit error, so stranded work is visible rather
than silently pending forever. Adopts active_run_id as this queue's owning run, so every
later claim() stamps claimed_by_run and a subsequent sweep can tell live from stranded.
Call it at supervisor start, BEFORE the first claim(). That ordering is what makes it
safe, and it is safe for two independent reasons:
- Run ids come from
ops.lifecycle.runs(INTEGER PRIMARY KEY AUTOINCREMENT), so a freshly-opened run's id is greater than any id already stamped on a row — no pre-existing row can be mistaken for this run's work. - The guard is positive, not inferential: a row is reclaimed only if its
claimed_by_runis notactive_run_id. A job this run actually claimed carries the stamp and is therefore never touched — the double-execution falsifier. A NULL stamp (every row written before this column existed, job 300246 included) is reclaimable precisely because no live run can have written it.
The lease is a second, independent reason a row is orphaned — and it does NOT widen what
this method reclaims (extension, dn-supervision-and-liveness §2.6). For the population
reason (2) already selects, the two reasons agree, so the deadline adds nothing the sweep
needs. For the complement — a row stamped by THIS run — the stamp stays an absolute VETO
that no deadline overrides, because reason (1)'s ordering argument is what protects rows
this run actually claimed, and a lapsed deadline is not evidence that the holder is dead:
a hung but alive worker has exactly that shape, and reclaiming under it would hand the
same job to a second worker. Worse than the orphan it fixes. So such rows are counted into
OrphanSweep.lease_expired and reported, never touched (§9: stamping is here, killing
is a later plan's; an expired-deadline row is reported orphaned, never silently reclaimed).
The derived-reader half of the mechanism — orphanhood visible with no sweep having run at
all — lives in lease_expired above and in ops.lifecycle.snapshot.read_queue_stats.
done/failed/queued/deferred rows are never examined. Runs the same UPDATE-over-a-
state-class shape as revive_deferred, split in two only because the two classes of kind
land in different states. Returns what it did; empty on a clean previous exit.
checkpoint(job_id, token)
Persist a resume token for a checkpointed-step job, then re-queue it so the next unit is dispatched at a job boundary (cooperative yielding, roadmap §7).
Clears the lease, which is what stops a checkpointed row from reading as an orphan: the
claim that minted the deadline ended when the handler yielded, and the row is now waiting,
not running. claim stamps a fresh deadline on the next batch, so a yielding lane's bound
is per-batch (see claim). A row left QUEUED with a stale deadline would be a lie the
derived reader has to be defended against instead of one the writer never tells — and
enqueue no longer collapses onto this row either (V6, the fourth property there).
get(job_id)
list(state=None)
depth()
Number of jobs waiting to run (queue depth — a vital, §8).
counts()
close()
deadline_lapsed(deadline, now)
Has an ISO deadline passed? The one implementation of that question, so the polarity
cannot drift between the two readers that ask it (this file's sweep and status's
ops.lifecycle.snapshot.read_queue_stats).
⚑ None ⇒ False. A row with no deadline is not expired, it is undeadlined — which is
exactly what every row written before this column existed carries (_MIGRATIONS). Inverting
this one line would mass-orphan 300k rows on first open. Unparseable ⇒ False for the same
conservative reason: a reader must never invent an orphan, and status must not crash on a
corrupt cell during the incident it exists to describe.
lease_expired(job, now)
True iff this row's claim has demonstrably lapsed. NULL deadline ⇒ False (today's behaviour, and every row written before this column existed). The orphan question stops being "did someone remember to sweep?" and becomes a property of the row.
The conjunction is the whole content of the tier-2 claim (dn-supervision-and-liveness §2.6):
RUNNING is still necessary — a QUEUED row is waiting, not orphaned — but it is no longer
sufficient, and the second conjunct is a fact about the clock rather than a byte some dead
actor wrote. Gating on state here is also structural insurance: a leftover deadline on a
non-RUNNING row (a writer that forgot to clear one) cannot manufacture an orphan, so the
reader is safe even if the invariant claim/checkpoint/defer maintain is ever broken.
scheduler.router
Router + watchdog — the pinned tiny model's role, done in rules first (BUILD-SPEC §9, §13).
RULES FIRST (roadmap §8): the scheduler is rule-capable by design and the tiny router model is an enhancement, not a single point of failure — so Phase 3 ships a deterministic rule-based router with a seam for the model router and a fall-back-to-rules path. The router decides role/tier/window; deterministic code acts (loads, assembles, dispatches) — model advises, code acts (Invariant 3).
The watchdog reads system vitals (the reactive tier, §9) and raises flags only when a threshold is crossed; those become high-priority jobs the supervisor dispatches next, at the following job boundary (roadmap §7) — never a mid-generation interrupt.
Plan
dataclass
kind
instance-attribute
tier
instance-attribute
num_ctx
instance-attribute
priority
instance-attribute
Flag
dataclass
metric
instance-attribute
value
instance-attribute
threshold
instance-attribute
note
instance-attribute
Router
dataclass
config
instance-attribute
tier_for(kind)
plan(kind, *, priority=None)
Resolve a job kind to (tier, window, priority) from the rules + the model lineup.
The window is the model's configured load-time num_ctx (§13); same-window jobs
batch together to avoid reloads.
Watchdog
dataclass
reader
instance-attribute
min_available_gb = 2.0
class-attribute
instance-attribute
check()
Read the latest vitals and return any crossed thresholds. Deterministic; escalates to a model only by enqueuing a job, which the supervisor dispatches by priority.
scheduler.supervisor
The supervisor — one loop owns the queue and the worker slot (BUILD-SPEC §13; roadmap §7).
Cooperative, job-boundary scheduling:
0. refuse to start anything at all while the battery is below the floor (the POWER axis,
dn-supervision-and-liveness Amendment A1) — the refusal sits ahead of the claim, so no
RUNNING row is minted for a machine that may not survive to close it;
1. claim the next eligible job (priority; swap-avoidance within a priority band; heavy
tiers gated while the owner is present — the foreground check — and shed while the machine
is on battery — the power check);
2. make its (tier, window) resident via the two-slot loader, which refuses any load that
would breach the RAM ceiling (Invariant 8) — such a job is deferred, not crashed;
3. dispatch it, in ONE OF TWO MODES (see below), counting worker swaps
(the pinned router doing interstitial work never evicts the worker, roadmap §7);
4. record vitals (queue depth, model-load time) and repeat.
[banner: correction] Step 3 used to read "run its handler to completion (or one checkpointed
step)", which was the only shape a dispatch could have. dn-supervision-and-liveness §2.5 adds a
second, and names why the first is not enough: a synchronous in-process call cannot be budgeted
from outside it, and cannot be observed from the loop it blocks. The two modes are:
inproc(the DEFAULT, and the documented behaviour for every unmigrated kind). Exactly as before:handler(job)runs to completion or to one checkpointed step, on this thread. Nothing about it changes, and no kind moves off it without an owner flippingworker_modeand a per-lane deskcheck (note §4).subprocess. The kind's registered COMPUTE half runs in apython -m scheduler.workerchild holding no store handle; it streams bounded batches back and the supervisor lands each one itself. The supervisor stays live throughout — it is reading a pipe, not computing — so the batch becomes both the fairness unit and the in-band progress signal.
[cross-ref: extension] "A reactive escalation is simply a high-priority job; it is dispatched at
the next boundary, never as a mid-generation interrupt." That remains true, and is about
SCHEDULING. dn-supervision-and-liveness §2.5 adds a separate power that must not be confused
with it: a WEDGED WORKER is now killable mid-compute (SIGTERM -> grace -> SIGKILL, a kernel fact
rather than cooperation). Distinguish the two — the split did not introduce preemption. No job is
ever interrupted to run another job; a worker is only ever killed because it overran its own
bound, and the escalation targets ONLY the worker, never the supervisor (killing the supervisor
mid-landing is how you create the partial write oq-0035 worried about).
A handler that raises must not take down the loop. Neither must a worker that dies.
HEAVY_TIERS = frozenset({'synthesis', 'stretch'})
module-attribute
Handler = Callable[[Job], 'str | None']
module-attribute
Supervisor
dataclass
queue
instance-attribute
loader
instance-attribute
handlers
instance-attribute
presence = field(default_factory=Presence)
class-attribute
instance-attribute
power = field(default_factory=Power)
class-attribute
instance-attribute
telemetry = None
class-attribute
instance-attribute
secrets = None
class-attribute
instance-attribute
warm = True
class-attribute
instance-attribute
swaps = 0
class-attribute
instance-attribute
compute = field(default_factory=dict)
class-attribute
instance-attribute
worker_mode = INPROC
class-attribute
instance-attribute
rows = None
class-attribute
instance-attribute
mint_token(role, ttl='10m')
Mint an ephemeral token scoped to role's policy (vault-runtime-auth.md §2). Returns
the whole MintedToken: the supervisor passes .token to the agent (Phase 5) and records
.accessor in the action's attestation (the Step-5 join) — it holds minting authority
only, never reading the secret it mints a token for; that happens later when the agent
itself calls get_secret(name, token=...).
blocked_tiers()
THE FOREGROUND GATE, and nothing else. Deliberately not extended with the single-model-in-flight rule (bp-110 §7 Item 4's invariant: "the foreground gate keeps its meaning and is not overloaded") — two different reasons to refuse a tier, conflated into one predicate, is how a reader later cannot tell which rule refused a job.
The siblings, so all THREE are findable from any one of them: model_blocked_tiers ("is a
model already out?") and power_blocked_tiers ("is there energy?", Amendment A1). Three
predicates, three questions, composed only by union at the ONE claim site in tick.
model_blocked_tiers()
THE SINGLE-MODEL-IN-FLIGHT RULE, verbatim from dn-supervision-and-liveness §2.7:
At most one in-flight MODEL-USING job. While one is out, the supervisor may dispatch
only jobs sharing its `load_key` or doing landing/housekeeping.
Today "≤ 2 resident models" (non-negotiable #8) holds IMPLICITLY because tick is serial.
A supervisor that stays live while a worker computes can claim a second model-using job,
and ensure_tier for job B would evict the model job A is mid-generation on — a failure
that would look like a model bug, not a scheduler bug. This adds no resident model; it
prevents one, so ceiling accounting is numerically unchanged.
The pinned tier is never blocked: it is always resident and the pinned router doing interstitial work never evicts the worker slot (the module docstring's step 3), which is exactly the "landing/housekeeping" the rule carves out.
⚑ Enforced at the ONE claim site, via claim's existing blocked_tiers — no new queue
API (Item 4's invariant). Tier accounting, stated honestly per §2.7: this is a dispatch
guard, tier 5 with a tier-4 test, not a capability. The ceiling REFUSAL itself
(_check_ceiling, raising before any load) is untouched by this plan.
⚑ Scope note, so this is not read as stronger than it is. Under the SYNCHRONOUS
dispatch this plan ships (_dispatch_to_worker streams to completion inside one tick),
the window this guards is currently EMPTY — no second claim can happen while a worker is
out. The guard ships anyway, with the mechanism that will create the hazard, so that
concurrency cannot later be introduced without it (§3 Q7: "shipping the split without it
is a regression"). finding-0229 records that the liveness half of §1's objective needs
non-blocking dispatch, which needs the serve loop — out of this plan's scope by §5.
power_blocked_tiers()
THE POWER AXIS, and nothing else (dn-supervision-and-liveness Amendment A1; issue
12): on battery, shed the heavy lanes.
⚑ The third sibling, deliberately NOT folded into blocked_tiers() — the amendment's
one load-bearing pin, for exactly the reason that method's docstring already gives about
the model rule. A power refusal and a presence refusal answer different questions ("is
there energy?" vs "is the owner here?"), and a reader who cannot tell which rule refused a
job cannot fix the one that is wrong. Composition happens only at the ONE claim site.
HEAVY_TIERS is READ here, never reshaped: the shed vocabulary is the existing one, so
there is a single answer to "which lanes are heavy?" rather than two that can drift (A1's
parked selector decision — load_key was rejected as the default because it introduces a
second, finer vocabulary whose interaction with this set nobody has designed).
Tier accounting, stated honestly per A1.4: a dispatch guard — tier 5 with a tier-4
test, deliberately identical to what §2.7 claims for the memory ceiling. Power is a
sampled reading of the physical world, so no value can be made to not inhabit "the battery
is low": tier 1 is unreachable here and claiming it would be the overclaim §0's ladder
names as the foot-gun. What the tier-4 test buys is tests/integration/test_supervisor.py
proving the union in tick actually contains this term, and that the probe's None path
fails closed — a predicate nobody calls is the finding-0187 shape (deleting bp-105's sweep
call left 85/85 green).
⚑ The honest limit, recorded rather than hidden (A1.4): this bounds what is STARTED,
never what is already running. Jul 24's code_backfill was in flight when the throttle
hit, so this would not have prevented that emergency outright. In-flight energy bounding
needs the job-timeout machinery (finding-0178) and is not designed here; nothing in this
path ever kills a running job.
tick()
Dispatch at most one job. Returns False when nothing is runnable right now.
⚑ THE ONE CLAIM SITE. All three refusal predicates — presence, single-model, power —
compose HERE, by union, and nowhere else (model_blocked_tiers's pin: "Enforced at the ONE
claim site, via claim's existing blocked_tiers — no new queue API"). Each stays
separately readable so a reader can still tell which rule refused a job; the union is the
only place they are indistinguishable, and it is one line long.
run(*, max_ticks=None)
Drain the queue cooperatively. Returns the number of jobs dispatched. Stops when nothing is runnable (e.g. only heavy jobs remain while the owner is present, or while the machine is on battery — and, below the power floor, when nothing at all may start).
Below the floor this returns 0 on the FIRST tick and returns control; it never loops or sleeps waiting for mains. That is the hold, and it is deliberately the caller's duty cycle rather than one invented here (Amendment A1's parked hold-for-AC decision).
scheduler.vault_sync
Wire the vault watcher + incremental re-ingest into the scheduler (vault-sync task).
The watcher (core) only signals; this scheduler-side module turns that signal into a durable
background vault_sync job and provides the handler that runs the idempotent re-ingest. So
all store mutation happens on the single supervisor writer (the queue's discipline), and the
core watcher stays free of any scheduler import (clean layering: scheduler depends on core,
never the reverse).
vault_sync is routed to the pinned tier (the router): it needs no chat model — it calls
the embedder directly — so making it "resident" is a no-op and the worker slot is never
evicted. It runs at BACKGROUND priority (yields to interactive/reactive work) but is NOT
in HEAVY_TIERS, so a note saved mid-session is re-ingested promptly rather than waiting for a
trough (the owner may want to query what they just wrote).
VAULT_SYNC_KIND = 'vault_sync'
module-attribute
Handler = Callable[[Job], 'str | None']
module-attribute
vault_sync_handler(sync)
enqueue_vault_sync(queue, router)
Enqueue one background re-ingest. Coalescing happens upstream (the watcher debounce);
duplicate jobs are harmless because rescan() is idempotent.
build_vault_watcher(queue, router, config=None)
A watcher whose on_change enqueues a background vault_sync job. Call .start() to run.
The supervisor must have the vault_sync handler registered (see vault_sync_handler) to
actually process the enqueued jobs.