Module persistent

Expand description

PersistentSubprocessController: manages agent CLI as a long-running process with bidirectional NDJSON streaming via stdin/stdout.

Architecture: A single CLI process is spawned on first message and kept alive for the entire session. User messages are written as NDJSON lines to stdin without closing it. This enables mid-turn input (redirecting the agent while it is still processing).

State machine: INIT ─(first message)─> RUNNING ─(idle between turns)─> RUNNING RUNNING ─(kill/stop)─> DONE RUNNING ─(process crash)─> DONE (auto-restart possible via session_id)

I/O model (3 async tasks per session):

stdin_writer: mpsc channel → process stdin (NDJSON lines)
stdout_reader: process stdout → .jsonl persistence + MPS blockfile events
process_waiter: wait for exit, update status

Module layout (split 2026-09-22; one impl PersistentSubprocessController block per file, all private methods pub(super)):

mod.rs — types, shared helpers, construction, Controller impl
status.rs — runtime status snapshot/publication and heartbeat
eager_resume.rs — spawn-on-construction when a session id is on file
queue.rs — send_message, spawn claim, queue drain/replay
resume_retry.rs — stale---resume recovery effects and retry batch
input.rs — user messages, question answers, tool decisions, stdin
spawn.rs — spawn_process: builds the command, spawns, wires the tasks
stderr_reader.rs, stdout_reader.rs, process_waiter.rs — the bodies of those per-session I/O tasks
lifecycle.rs — stop/restart, session_id, needs_spawn
tests/ — one file per former inline test module

Modules§

eager_resume 🔒: Eager resume: spawning the CLI on controller construction (before the first message) when a prior session id is on file, so the first turn is not charged the full cold-start.
input 🔒: User-facing input surface: user messages, AskUserQuestion answers/denials, tool-permission decisions, raw stdin, and inbound control frames.
lifecycle 🔒: Stop/restart and session-identity queries: stop_process, the kill request path, session_id, needs_spawn, and the deferred-restart flag.
process_waiter 🔒: The persistent CLI’s process waiter task, started by spawn_process (spawn.rs): the child.wait() arm (the process exited on its own) and the kill arm (a stop or restart asked for it).
queue 🔒: Message intake and the spawn-claim/queue-drain protocol: send_message and everything that decides whether a message is written to stdin now, queued behind an in-flight spawn, or replayed after one.
resume_retry 🔒: Stale---resume recovery: executing the effects persistent_resume decides, locating a recoverable session id, and the retry-batch rewrite.
segments 🔒: This controller’s side of the segment index (backend::continuity_segments): each spawn is one segment of its agent’s conversation. Best-effort throughout: a failed write is logged and never affects the process.
spawn 🔒: spawn_process: builds the CLI argv/env, spawns the child, and starts the per-session I/O tasks (stdin writer, stdout reader, stderr reader, waiter). The stdin writer runs inline; the other three bodies live in stderr_reader.rs, stdout_reader.rs and process_waiter.rs.
status 🔒: Runtime status: the BlockControllerRuntimeStatus snapshot, its publication to the frontend, the status heartbeat, and turn-activity marking.
stderr_reader 🔒: The persistent CLI’s stderr reader task, started by spawn_process (spawn.rs): logs every line and reacts to a stale --resume.
stdout_reader 🔒: The persistent CLI’s stdout reader task, started by spawn_process (spawn.rs): control frames, session-id capture, resume-state events, failure classification, the task feed and the output append.

Structs§

DeferredDropReport 🔒: The last line of defence for §4.5: whatever is still deferred when the controller goes away is reported, not silently discarded with it.
PersistentInner 🔒: Inner state protected by mutex.
PersistentSpawnConfig: Configuration for spawning the persistent process. PartialEq is derived for persistent_resume::RetryPayload’s own derive and its exhaustive unit tests — not used elsewhere in this file itself.
PersistentSubprocessController: PersistentSubprocessController keeps a long-running CLI process alive, sending user messages as NDJSON lines on stdin.
QueuedMessage 🔒: A single entry in pending_send_messages: the formatted stdin JSON payload plus whether it’s already been persisted to the blockfile. codex P2 on PR #2360 (sixth review pass, round 7): a stale-resume retry’s redelivery has already been correctly persisted on its original (failed) attempt — see retry_after_resume_failure’s own doc comment — but the drain used to persist EVERY message it delivers unconditionally, double-persisting a replayed retry batch into the blockfile transcript.

Enums§

EagerResumeOutcome 🔒: Result of try_eager_resume — see start()’s doc comment. DeclinedTo carries a short, non-exhaustive reason for the log line; it is not meant to be pattern-matched on by callers, just read.
KillRequest 🔒: What the process-waiter task’s kill arm is asked to do.
QueueDrainClaim 🔒: Which exclusivity claim a queue-drain task runs under — the spawn claim (spawning_in_progress, the post-spawn drain) or the retry flush claim (drain_claim, issue #2367). The task must release exactly the claim its initiator took, and nothing else.
RetryBatchAction 🔒: What decide_retry_batch_action determined a stale-resume retry batch’s fate should be (issue #2367; spec §4). Split from SendAction because the batch’s live-process outcome is not a direct delivery: the batch goes through the queue under the drain claim, never through a caller’s own try_send.
SendAction 🔒: What decide_send_action determined a message’s fate should be — see its own doc comment and PersistentInner::spawning_in_progress.
WatchdogStep 🔒: Whether the deferred-delivery watchdog keeps running after a tick.

Constants§

ANSWER_RESUME_FALLBACK_MS 🔒: How long to wait after delivering an AskUserQuestion answer before assuming the turn did not resume and re-delivering the answer as a follow-up message. See answer_question and SPEC_ASK_USER_QUESTION_2026_06_15.md §10.1.
ASK_USER_QUESTION_DENY_MESSAGE 🔒: Fixed, server-owned decline message for a cancelled AskUserQuestion (the Cancel button / Escape in AgentQuestionPanel.tsx) — not client-suppliable, since Cancel and Escape both mean exactly one thing and there is no user- authored content to carry. See deny_question and docs/specs/SPEC_AGENT_CONTROL_PROTOCOL_2026_06_15.md.
BLOCK_CONTROLLER_PERSISTENT
CONTINUATION_TAIL_BYTES 🔒: How much of a transcript’s tail find_continuation_session_id reads. The provider stamps its session id on every stream event, so the last few events suffice; a long-lived agent’s transcript runs to many MB.
DEFERRED_ORPHAN_GRACE_TICKS 🔒: Consecutive watchdog ticks with no process and no spawn in flight before the queue is declared stranded and reported (spec §4.5). Long enough to ride out the gap between a process exit and the respawn that follows it (a stale---resume retry, a fallback respawn); short enough that a dead agent’s backlog is surfaced rather than held forever.
DEFERRED_WATCHDOG_TICK 🔒: How often the deferred-delivery watchdog re-checks a non-empty queue. Messages are queued only while the process cannot take a write (starting up, restarting, stopping), and the watchdog is what writes them once it can, so this bounds how long after the process comes up they arrive.
MAX_DEFERRED_DELIVERIES 🔒: Bound on PersistentInner::deferred_deliveries. Past this, enqueueing returns an error instead of a false success, so a caller is never told a message was delivered and then has it silently dropped (SPEC_NO_MIDTURN_DELIVERY_2026_09_23.md §4.5). Senders already handle transient delivery failure — the reactive handler, muxbus and the bridges all have to cope with an unreachable target instance — so surfacing a full queue as one more retryable failure needs no new machinery on their side.
PERSISTENT_OUTPUT_SUBJECT: MPS file subject name for persistent subprocess output.

Functions§

build_answer_resume_message 🔒: Compose the directive follow-up message used by the AskUserQuestion dead-air fallback. answers maps each question’s text to the selected label(s) or free text (the same object delivered in the control_response). The message is deliberately directive so the model resumes the task instead of treating it as a no-op (the “user sent an empty message” failure mode).
build_deny_resume_message 🔒: Compose the directive follow-up message used by the AskUserQuestion dead-air fallback when the answer was a DECLINE rather than a real answer. Mirrors build_answer_resume_message’s directive tone (resume the task, don’t wait for further input) but tells the model the outcome was a decline so it doesn’t hallucinate a value the user never provided.
build_tool_decision_resume_message 🔒: Compose the directive follow-up message used by the tool-permission dead-air fallback (mirrors build_answer_resume_message / build_deny_resume_message’s directive tone — resume the task, don’t wait for further input — but for an ordinary tool-permission decision rather than an AskUserQuestion answer).
classify_exit_line 🔒: Classify a flushed error line, if it’s confident enough to surface to the pane’s failure-recovery UI. line is usually the same result-frame JSON the mid-stream error-result path handles, but persistent_resume::ResumeEffect::FlushErrorLine can also carry an earlier held-back turn’s line — falls back to raw-text keyword matching (the same classify() uses for a stderr tail) if it doesn’t parse as JSON. exit_code is Some at the process-exit arm and None at the mid-generation flush sites (a superseded spawn generation, or a SessionCaptured resolution), where no fresh exit code exists for the line being flushed. Returns None for an unrecognized, non-retryable FailureClass::UnknownNonZero to avoid a low-confidence banner from noisy flushed text — every other recognized class is surfaced, not just the retryable ones, since the recovery banner has value for e.g. Auth too (see SPEC_AGENT_FAILURE_RECOVERY_UI_2026_06_16.md’s per-class action matrix), just without auto-retry. See SPEC_PERSISTENT_CONTROLLER_FAILURE_CLASSIFICATION_2026_08_04.md.
fresh_start_needs_disclosure 🔒: Should a spawn that attached NO --resume disclose itself as a fresh start (subject to the caller also confirming prior history actually exists — see has_prior_transcript)?
held_elsewhere_error 🔒: The one spawn refusal that is a decision, not a failure: --resume <sid> was asked for while another pane still holds (or is still closing) that same conversation — session_held_elsewhere. Built here so that is_held_elsewhere_error is checked against the same text spawn_process actually returns, rather than a second copy that could drift; tests/reopen_guard.rs pins the wording users see.
interrupt_control_request_line 🔒: The Claude control-protocol interrupt (docs/specs/SPEC_AGENT_CONTROL_PROTOCOL_2026_06_15.md). Ends the turn in progress — measured at ~2s, with result: error_during_execution — while leaving the process and session alive (pane-close spec §5.1).
is_held_elsewhere_error 🔒: Did spawn_process refuse because of session_held_elsewhere? The answer decides whether a caller may fall back to a FRESH spawn: for any other failure a blank session is a reasonable recovery, but for this one it silently hands the user’s accepted prompt to a new conversation while the real one is open next door — exactly what the guard exists to prevent (codex P1 on PR #3551).
last_session_id_in_stream 🔒: The last non-empty field on a complete JSON line of tail. starts_mid_line: tail was cut from a longer file, so its first line is a fragment and is dropped.
muxbus_agent_id_from_env 🔒: Resolve the muxbus address (the agent’s display name) from a spawn env map. AGENTMUX_AGENT_ID (= agent.name, set at block creation) is canonical; WAVEMUX_AGENT_ID is the legacy fallback. Returns None — i.e. not muxbus-addressable — when neither is present (a non-agent persistent block).
next_registration_nonce 🔒: Draws the next process-wide registration nonce (≥ 1) for a persistent spawn’s muxbus/registry registrations — see the doc comment at the my_registration_nonce binding in spawn_process for why this is a srv-wide counter rather than the controller-local spawn generation (codex P1 on PR #2500: generations restart per controller instance and can collide across a resync_controller replacement).
park_tool_permission_request 🔒: Park an ordinary (non-AskUserQuestion) tool-permission can_use_tool control_request into PersistentInner::pending_permissions, mirroring how AskUserQuestion is parked into pending_questions two branches up in handle_control_frame — same in-memory-only, per-controller-instance lifetime, and the same “process respawned” failure mode on decide (PersistentSubprocessController::decide_tool_permission’s error text explains it the same way answer_question’s does).
publish_resume_retry_status 🔒: Publish a mps::EVENT_AGENT_RESUME_RETRY status ping — a free function (not a method) for the same reason session_outcome_line above is one: callable from the stdout-reader/process-waiter match arms, which only hold _read/_wait-suffixed clones, not &self. status is "retrying" (set when a stale --resume is detected and a retry/recovery attempt is about to fire) or "resolved" (set the moment the retry’s outcome — Fresh or Resumed — is actually known). See docs/status/STATUS_STALE_RESUME_LIVE_REPRO_AND_FIX_PLAN_2026_08_23.md §6.2. No-ops if broker is None (tests / non-wired controllers), same posture as every other best-effort MPS publish in this file.
session_outcome_line 🔒: Builds the NDJSON line for a persistent_resume::ResumeEffect:: EmitSessionOutcome — a free function (not a method) so it’s callable directly from the stdout-reader, process-waiter, and stop-path match arms below, which only hold _read/_wait-suffixed clones, not &self. See SPEC_AGENT_PANE_HISTORY_ALIGNMENT_2026_08_05.md §2.1.
session_outcome_line_with 🔒: session_outcome_line, plus whether the fresh session was given AgentMux’s record of the conversation (continued, SPEC_DURABLE_ CONVERSATION_MEMORY_2026_09_23.md §4.8). The outcome stays fresh: the provider session is new, and every consumer that scopes scrollback on fresh keeps doing so. Only the pane’s label changes.
should_route_to_decision_panel 🔒: Whether a can_use_tool control_request for tool_name (anything except AskUserQuestion, which is always parked separately — see handle_control_frame) should be routed to the frontend decision panel instead of auto-allowed.
surface_error_line 🔒: Classify a final error line with classify_exit_line and, if it is a failure, surface_failure it. Returns whether it was surfaced.
surface_failure 🔒: Record failure as the block’s agent:last_failure, then publish it as agent-failure on block:<id> (persisted for replay) when there is a broker.