Skip to main content

Module indexing

Module indexing 

Source
Expand description

output.idx rebuild: a lazily-built, self-validating line-offset cache.

Structs§

IndexedRead 🔒
Lines [offset, offset + limit) of output, read through output.idx.
OutputIndex 🔒
output and its output.idx as they were at ONE moment — one database snapshot (Codex on #3634): separate reads of the index’s size, header and label could each see a different version of an index another srv instance is rebuilding, and pass an old count off as fresh.

Enums§

Extend 🔒

Constants§

IDX 🔒
IDX_GEN_META 🔒
Key in output.idx’s file metadata naming the output generation the index was built for (Phase 5a-2b; filestore/counter.rs). Written in the same transaction as the index, so the label always describes the bytes.
OUTPUT_IDX_HEADER_LEN 🔒
Magic header size for output.idx: the first 8 bytes are the output byte-size the index was built for. The index is valid iff this equals output’s current size; otherwise it is stale and must be rebuilt.

Functions§

build_output_idx_from 🔒
Shared scanner behind both entry points.
extend_output_idx 🔒
Bring an existing output.idx up to output’s current size by scanning ONLY the appended bytes, instead of rescanning from byte zero.
labelled_for 🔒
Whether an index with metadata meta may describe output of generation output_gen, apart from its covered size.
output_index 🔒
See OutputIndex. None when there is no output.
output_now 🔒
output’s size and valid generation, from the database — never this process’s stat cache, which another srv instance’s append makes stale. A counter only behind (an older build appended since) is caught up first, so its generation holds across those appends (filestore/counter.rs).
read_indexed 🔒
read_via_index 🔒
Read lines [offset, offset + limit) of output through a fresh output.idx, with the freshness check, the index entries and the output bytes all from ONE database snapshot (Codex on #3634): validating in one read and fetching entries and bytes in later ones would let another srv instance replace output and rebuild the index in between — the answer would mix the new file’s lines with the old file’s total.
read_via_stale_index 🔒
read_via_index from an index that covers only a prefix of output: lines appended since it was built aren’t returned (total is the index’s count), nor is the last indexed line, whose end the index can’t give. For the read right after extending the index, when an append landed in between — before this, that read fell back to loading the whole transcript (1.3 GB, ~15 s, under the store’s lock) whenever an agent’s spawn was writing to the transcript its pane was opening (docs/reports/REPORT_AGENT_OPEN_STALL_RCA_2026_10_05.md). The lines it leaves out are the newest few, which the pane’s live stream delivers. None under the same conditions as read_via_index, and when nothing requested is within the indexed prefix.
try_extend 🔒