rebuild_output_idx

Function rebuild_output_idx 

Source
pub(crate) fn rebuild_output_idx(
    fs: &FileStore,
    block_id: &str,
    output_size: u64,
) -> Option<u64>
Expand description

Rebuild output.idx from output in a single streaming scan and atomically replace it. The index is the byte offset of every non-blank line, matching the reader’s line addressing (String::lines().filter(!trim().is_empty())).

Layout: [covered_size: u64-LE][offset_0: u64-LE][offset_1].... covered_size records the output size this index reflects so the read path can detect staleness in O(1) and rebuild only when output actually grew.

The scan streams output in 1 MiB windows so memory stays O(one line + offsets) rather than loading the whole (potentially multi-GB) file. Line splitting is done on raw bytes; since UTF-8 continuation bytes never collide with \n, lines are never split mid-codepoint, so per-line from_utf8_lossy matches the reader.

Returns the number of indexed (non-blank) lines on success, or None if output is unreadable or the index write fails (caller falls back to slow path).