- 06-BUILD-SYSTEM-SETUP: note which QEMU proof scripts use
qemu-login-expect.py vs host expect
- 01-REDOX-ARCHITECTURE, LOCAL-FORK-SUPREMACY-POLICY,
SYSTEM-STABILITY-AND-UPSTREAM-SYNC-PLAN, UPSTREAM-SYNC-PROCEDURE:
sync with current fork-model and sync-procedure state
- Delete archived/SOURCE-ARCHIVAL-POLICY.md and
fork-push-status/2026-07-12-Round-5-phase-4.1.md — superseded by the
current local-fork model docs and newer fork-push-status rounds
- config/redbear-mini.toml: trailing newline
Enable GBM (required by the EGL/GBM/GLES2 surface used by Qt6/KWin).
Drop crocus/iris from the gallium driver set for the 26.1.4 build —
the Intel hardware drivers require DRM uapi surfaces that are not
available in the Redox sysroot yet. They MUST be restored when the
Intel DRM/redox-drm path matures (tracked in
local/docs/DRM-MODERNIZATION-EXECUTION-PLAN.md Stage 5); this is a
build-surface constraint, not a feature removal.
Also widen the -Wno-error set for the 26.1.4 cross-compile
(missing-prototypes, return-type, empty-body, incompatible-pointer-types,
int-conversion, format) — upstream Mesa enables -Werror by default and
the Redox sysroot headers trip these classes; the underlying warnings
remain visible in the build log.
Writing startup diagnostics to stderr interleaves with the interactive
console on the live image; /scheme/debug reaches the serial log without
disturbing the shell's tty.
On the live image, a fork/exec during login-shell startup can stall the
shell before its first prompt. Replace command substitutions with pure
builtins in the startup path:
- bash.bashrc: read /etc/redox_chroot with the 'read' builtin instead of
$(cat ...)
- profile: derive the root prompt from EUID/USER/LOGNAME (set by
login(1)) instead of $(id -u)
- skel/.bashrc: disable lesspipe and dircolors eval (both spawn
subprocesses); keep the ls --color=auto alias
base -> fb421083:
- fbcond: retry handoff while display driver is not ready (bounded
250x10ms; fixes the 2026-07-20 '-vga std' boot freeze at 'Performing
handoff' where a transient not-ready driver left the framebuffer VT
permanently blank)
- xhcid/virtio-netd: info-level IRQ/MSI-X delivery and reactor startup
milestones for runtime-proof observability
- usb: demote diagnostic ATTACH/HUBFLOW traces to debug!
- usbhubd: gate port-indicator SetPortFeature on hub capability
userutils -> 3b02e0b9:
- getty: harden the console<->PTY bridge against transient
read/write/event errors (log-and-continue instead of panic; a panic
here killed the live shell's I/O)
The kernel fork (submodule/kernel bd1b251f) now provides a proper
'install' target honoring DESTDIR, so the recipe no longer hand-rolls
the staging copy. Fork also gains post-merge compile fixes (import
dedup + UnmapVec).
Sync the tracked source tree to the wayland-protocols 1.49 release
tarball already pinned in recipe.toml (blake3-verified). Upstream 1.49
drops the pkg-config .pc.in files, switches scanner discovery to a
native wayland-scanner dependency with fallback, and adds the new
stable/staging protocol XMLs (ext-image-*, xdg-toplevel-icon,
cursor-shape, content-type, alpha-modifier, ...).
Convert the remaining expect-based test scripts (phase1-6 runtime, usb,
wifi, bluetooth, dbus, greeter, live-iso, posix) to the stdlib python
harness, completing the migration started in 9ec00d4c81. Each script now
prefers booting the live ISO when present (falling back to harddrive.img
with snapshot=on), and retries up to 5 times when QEMU exits early or is
killed (rc 3 / 137) — the host-contention failure mode recorded in the
2026-07-20 runtime-proof status.
qemu-login-expect.py: treat an incomplete step sequence as a failure even
when a pass marker was seen partway through — partial sequences must not
count as a pass.
Post-review fixes:
- Phase 2 no longer breaks early when only --fail_marker is set: the
'if not args.pass_marker: break' guard skipped all output reads, so a
fail marker was never detected (exit 0 instead of 1). Now breaks only
when neither pass nor fail markers are configured.
- docstring referenced --pass/--fail; argparse registers
--pass_marker/--fail_marker. Align the docstring.
- retry echo said 'attempt N/3' but the loop limit is 5 attempts; say /5.
- warn and skip malformed steps (missing expect:/send: prefix) instead of
spinning in select until timeout.
IRQ plan + validation matrix: the login-based QEMU proofs (PS/2 + serio,
monotonic timer, IOMMU first-use, USB storage BOT) now run through
local/scripts/qemu-login-expect.py (stdlib python) instead of the host expect
tool. MSI-X (virtio-net) and xHCI interrupt-driven mode were re-confirmed on
the current redbear-mini ISO. Re-confirmation of the login-based legs with the
new harness is pending an uncontended host (parallel QEMU workload kills QEMU
processes and stalls guest boots). Also record the fbcond Performing-handoff
boot freeze observed with -vga std (separate graphics-path issue).
Drop the host expect dependency from the PS/2, timer, IOMMU, and USB
storage (BOT) runtime proofs. local/scripts/qemu-login-expect.py drives
the guest over serial-on-stdio with ordered expect/send steps, per-step
timeouts (matching expect's per-pattern semantics), and pass/fail markers.
It distinguishes premature QEMU death (exit 3) from genuine check failure
(exit 1) so the scripts retry external interruptions without retrying real
failures. Scripts use headless -display none -vga none (fbcond serial
mirror), a 3600s per-step timeout, and a retry loop (max 5 attempts).
All six validation scripts (xhci-irq, msix, ps2, timer, iommu,
usb-storage) booted build/<arch>/<config>/harddrive.img, which is only
produced by older make-all flows and goes stale (the Jul-17 image).
Running them against it measured the wrong tree — the same trap that
produced the false '+rb0.3.0' bootloader reading and the first hub-test
failure. Each now prefers build/<arch>/<config>.iso via -cdrom and only
falls back to harddrive.img when no ISO exists. extra.img and
usb-storage.img remain as data drives in both modes.
Root cause of the duplication: a stalled migration. driver-manager (new
Red Bear framework) was written and packaged, but never wired in — its
00_driver-manager.service ran cmd="pcid-spawner" (the legacy binary),
and redbear-mini.toml already notes it is 'not yet ready'. Meanwhile the
base recipe's 00_pcid-spawner.service (oneshot_async) is the live
spawner. Both services ran pcid-spawner async, so pcid-spawner launched
TWICE in mini/full, racing to spawn the same drivers.
Consolidation (user directive: consolidate on pcid-spawner for now,
revisit driver-manager later):
- drop the unused driver-manager = {} package (source recipe kept for
the future migration),
- remove the duplicate 00_driver-manager.service,
- point 13_driver-params.service requires_weak at the real
00_pcid-spawner.service,
- base recipe's oneshot_async pcid-spawner is now the sole PCI driver
spawner — no more double launch.
driver-manager is intentionally left out of configs until its driver
config migration is completed (documented in the file).
Per-entry assessment (file was never deleted — no D commits in history;
continuously present, last touched Jul 16 by d5561184f0 which fixed
00_base.service zsh->mkdir and unblocked boot):
- KEEP 00_base.service (mkdir /tmp): base provides no 00_base.service;
the Jul-16 fix is the sole source and is required.
- REMOVE 20_audiod.service: redundant with base's identical oneshot_async.
- REMOVE zsh = {}: redundant with minimal.toml.
- REMOVE 00_pcid-spawner.service (type=oneshot): HARMFUL — the blocking
oneshot overrode the base recipe's fixed oneshot_async pcid-spawner,
re-introducing the boot-wedge the base recipe explicitly fixed. Base's
oneshot_async now wins (config /etc/init.d no longer shadows it).
Resolves the pcid-spawner service conflict (task T-7fdcd5e5).
Correct the P3-A runtime-validation note and matrix row: the
intermittent apparent boot freeze is startup latency under load (guest
clock ~10-15x slower than wall), not a deadlock or an xhcid bug. A short
50-60s test timeout occasionally cuts the spawn+enumerate sequence,
producing a 'frozen' log while the guest is still running — proven by
18 boots (one apparent short-timeout loss), a 12s log-quiet 'freeze'
the guest resumed from, and every >=180s run enumerating fully. No fix
required beyond adequate test timeouts; no speculative kernel-deadlock
fix was implemented.
The usb-kbd behind the QEMU usb-hub now enumerates fully: debounce ->
port reset -> enable -> second debounce -> pre-attach -> attach ->
ATTACH 5.2 -> slot 3 -> addressed -> descriptors read -> HID class match
-> 'Loading subdriver "USB HID" for port 5.2' (class 3.1 proto 1,
keyboard boot protocol).
Root cause of the final stall: an unconditional SetPortFeature
(PORT_INDICATOR) for every connected+enabled port NAK-hung on QEMU's
indicator-less hub (no control-transfer timeout), blocking the flow
before the post-enable debounce. Now gated on
wHubCharacteristics.HUB_CHAR_PORTIND like Linux has_indicators.
test-usb-hub-qemu.sh: check #5 now requires the hub-child subdriver
line ('Loading subdriver "USB HID" for port <root>.<child>') so a
root-port HID (tablet) can no longer satisfy it — closes a false-
positive hole. Timeout raised to 360s for slow TT-path driver bring-up.
Docs (plan P3-A runtime validation + matrix): full hub + hub-child
chain recorded as QEMU-proven; remaining items (intermittent reactor
startup race, slow TT-path Evaluate Context) noted.
Hub enumeration is quick, but a hub-child's class driver then performs
its own descriptor/control transfers on the slow emulated TT path, and
the guest clock runs several times slower than wall time under load.
180s cut the boot before a hub-child HID driver could finish
registering after being spawned.
USB-IMPLEMENTATION-PLAN.md P3-A: record the 2026-07-19 QEMU runtime
validation (test-usb-hub-qemu.sh) — full usbhubd chain proven
(descriptor, EP1 change detection, initial scan, debounce, reset to
port-enable), the five latent bugs it surfaced and that were fixed
(scheme attach bootstrap, bitmap off-by-one, missing initial scan,
unbounded EP1 wait, debounce clear deadlock), QEMU's ClearPortFeature
stall handled gracefully by P2-C recovery, and the remaining gap:
hub-child device enumeration (usb-kbd to HID driver) blocked on
xhcid's hub-child attach path (debug-level; TT/route investigation)
plus an intermittent xhcid IRQ-reactor startup race.
HARDWARE-VALIDATION-MATRIX.md: hub row to QEMU-validated for the
enumeration chain, with the hub-child gap noted.
Serial output from dozens of concurrent daemons interleaves mid-line,
so full-sentence matches like 'usbhubd: 8 port(s) detected' fail even
when the content is present (observed mangled as 'usbhubd: 548: port(s)
detected'). Match short fragments ('port(s) detected', 'change
detection', 'registered producer') that survive interleaving.
Runtime log output is user-visible system surface; cross-references to
other operating systems belong in code comments and commit messages
(per project convention), not in runtime logs. The debounce parameters
and the comment-level references stay unchanged.
The 'USB HUB driver spawned' log line is emitted before setup_logging
and never reaches the serial console, so check #2 could never pass.
Use the hub-descriptor read line (emitted only after XhciClientHandle
open + 8-port descriptor fetch succeed) as the spawn proof, and fold
the old duplicate descriptor check into it.
Cat 1 discovery treated every local/recipes/*\/source/Cargo.toml as an
in-house crate, including git-fetched upstream sources like brush
(git = github.com/reubeno/brush). Rewriting upstream crate versions to
the branch version broke upstream-internal version requirements
(brush-shell was forced to 0.3.1 while brush required ^0.4.0 —
'failed to select a version for the requirement brush-shell = ^0.4.0').
Guard: skip any crate whose Cargo.toml is not tracked by the parent
repo. In-house crates are tracked trees (no independent .git); fetched
sources are gitignored and untracked. Restores correct classification:
75 crates checked (12 fetched-source crates now excluded), drift 0,
brush cooks successfully again.
- sync-versions.sh: 11 brush workspace crates 0.x -> 0.3.1 (Cat 1
in-house version policy).
- test-usb-hub-qemu.sh: boot the live ISO (canonical build-redbear.sh
artifact) instead of harddrive.img, which is only produced by older
make-all flows and goes stale — a stale Jul-17 image with a
pre-version-sync bootloader (+rb0.3.0) masked the current build and
crashed the first proof attempt. Falls back to harddrive.img only
when no ISO exists.
The script cd's into the fork dir for the clean-tree check (line ~202)
and never returns to PROJECT_ROOT. Every subsequent relative
'cd "$fork_dir"' subshell then fails with 'No such file or directory',
making single-fork and multi-fork runs always FAIL at the upstream-ref
resolution step. Make fork_dir absolute at assignment so all subshell
cd/git calls work regardless of current directory.
Found while attempting the base fork sync (verify-fork-functions gate).
Track base pointer to bd379e07 (usbhubd debounce/reset-wait/U0-wait/
wHubDelay + SET_ISOCH_DELAY, HubDescriptorV3 device_removable fix).
USB-IMPLEMENTATION-PLAN.md: P3-A marked CORE COMPLETE with the full
Linux equivalence inventory and documented follow-ups (USB 2.0 bitmaps,
SS.Inactive warm-reset loop, multi-TT, hub LPM). Hub gap-table row
updated (249 -> ~700 LoC with Linux state machine).
HARDWARE-VALIDATION-MATRIX.md: hub row updated — enumeration core
complete with 14 unit tests; runtime hub proof tracked under P8-B.
Track base pointer to d20557f7 (xHCI 1.1+ feature gating: LEC/U3C/CIC
gates, HW LPM substrate with protocol-caps HLC detection, PORTPMSC
register-target fix, bogus HCCPARAMS1-HLC accessor removed).
Mark P2-B COMPLETE in USB-IMPLEMENTATION-PLAN.md with the full gating
inventory and Linux 7.1 cross-references. Per-device L1 enablement
(BESL, MEL Evaluate Context) explicitly deferred to P3 hub work.
Expand XhciControllerQuirkFlags from 7 to all 51 flags defined in
Linux 7.1 xhci.h:1587-1649, each using 1u64 << N matching the kernel
bit positions. Add XhciQuirkEntry struct with vendor/device/revision
matching and a WILDCARD constant for catch-all rules.
Create xhci_table.rs — the canonical controller quirk table ported
from Linux 7.1 xhci-pci.c:xhci_pci_quirks() (lines 251-517). Covers
~85 entries across all vendors (Fresco Logic, NEC, AMD, ATI, Intel,
ETRON, Renesas, VIA, CDNS, Phytium, ASMedia, TI, Broadcom, Cavium,
Zhaoxin). 38 unit tests verify flag assignment for every vendor family.
Update lookup_xhci_controller_quirks_full() signature to accept
revision: u8 and hci_version: u16, enabling the two table entries
that depend on hci_version (AMD_0x96_HOST, >= 0x120 spec rule).
Bugs fixed vs xhcid's prior self-contained table:
- Intel 0x9c31 duplicate removed; WildcatPoint corrected to 0x9cb1
- Panther Point corrected to 0x1e31
- VIA VL805 corrected to LPM_SUPPORT + TRB_OVERFETCH
- Phytium corrected to RESET_ON_RESUME
- ASMedia 0x1142 corrected to NO_64BIT_SUPPORT + ASMEDIA_MODIFY_FLOWCONTROL
- Intel 0x9a13 moved off RESET_TO_DEFAULT list
xhcid now consumes this canonical path via a thin re-export shim
(base submodule pointer bumped to 9fc1947d).
Reference: local/reference/linux-7.1/drivers/usb/host/xhci-pci.c
local/reference/linux-7.1/drivers/usb/host/xhci.h
Boxing the Command(Command, ExitStatus) variant (204 bytes) and
Pkgar(pkgar::Error) variant (272 bytes) reduces the Error enum from
272+ bytes to ~80 bytes, eliminating all 70 result_large_err warnings.
Changes:
- Error::Command(Command, ExitStatus) → Command(Box<(Command, ExitStatus)>)
- Error::Pkgar(pkgar::Error) → Pkgar(Box<pkgar::Error>)
- Updated 2 construction sites in fs.rs (Command)
- Updated 1 construction site in package.rs (Pkgar)
- Updated 1 From conversion in lib.rs (Pkgar)
- Updated Display match in lib.rs (Command destructuring)
This is a heap allocation on error paths only — zero cost on the happy path.
Box<T> auto-implements Display/Debug when inner type does, so formatting
is unchanged.
Verification: cargo check ✅, cargo test --lib ✅ 38/38.
Clippy: 74 → 4 warnings (70 eliminated).
Tier 1 message fixes (Part 5.3):
- M18/M19 (build-redbear.sh:523-524): reworded patch-application messages
to reflect local fork model (forks have patches committed; recipe patches
for non-fork recipes applied atomically by repo fetch)
- M46 (build-redbear.sh:923): 'immutable archives' → 'local forks'
Tier 2 doc consolidation (Part 7):
- GRUB-INTEGRATION-PLAN.md: updated all 'make all CONFIG_NAME=' commands
to canonical './local/scripts/build-redbear.sh' equivalents
- SLEEP-IMPLEMENTATION-PLAN.md: archived (referenced 0.2.4 branch, pre-0.3.1)
Assessment doc Part 7 updated: all Tier 1 (items 1-6) and Tier 2 (items 7-13)
marked as verified complete with per-item status.
All Q1-Q10 quality improvement opportunities from Part 6.2 and all Tier 1-3
items from Part 7 are now resolved.
Q1 from assessment Part 6.2 — closes the largest robustness gap (Mechanism #2):
corrupt cached pkgar that passes mtime check was previously not detected
until runtime.
Publish side (repo_builder.rs):
- Accumulates BLAKE3 hash of each published file (pkgar, toml, dep_hashes,
auto_deps) into a BinaryStoreManifest
- Writes <recipe>.manifest.toml alongside published artifacts in repo/
Restore side (cook_build.rs):
- BinaryStoreManifest struct with read() returning Result<Option<Self>, String>
(same pattern as DepHashes: Ok(None) for missing = backward compat,
Err for corrupt TOML = loud WARN + skip)
- After restoring from binary store, verifies each file's BLAKE3 against
the manifest. On mismatch or missing file: WARN + all_restored=false
(forces rebuild). Missing manifest = backward compatible (older cookbook
published without one).
Helper (fs.rs):
- compute_file_blake3_hex(): 64KB chunked BLAKE3 hash, avoids loading
entire pkgar into memory
Tests: 3 new (roundtrip, missing-file Ok(None), corrupt-TOML Err).
Total: 38/38 pass. cargo check clean. No new clippy warnings.
Assessment doc Part 6.2/6.3/7 updated: Q1-Q6 all marked resolved.
Auto-correction table updated: transient network failure (Q2 retry) and
corrupt cached pkgar (Q1 BLAKE3 manifest) moved from 'does NOT auto-correct'
to 'auto-corrects'.
Tier 3 code improvements (from BUILD-SYSTEM-ASSESSMENT-2026-07-18 Part 6.2):
- Q2: Add run_command_with_retry() helper with exponential backoff (1s, 2s, 4s).
Wrap download_wget and git clone with 3 attempts. Git clone closure cleans
partial tmp dir between retries (git refuses non-empty clone targets).
- Q3: Preserve auto_deps.toml alongside dep_hashes.toml in binary store publish
(repo_builder.rs). Restore path prefers the preserved copy, falling back to
declared-depends-only reconstruction only when the preserved copy is absent
(e.g. published by an older cookbook). Preserves full ELF dynamic dep graph.
- Q4: DepHashes::read now returns Result<Option<Self>, String> discriminating
'missing' (legitimate first build, fall back to mtime) from 'corrupt' (loud
WARN + force rebuild). Closes a silent-swallow gap where a corrupt
dep_hashes.toml produced an incorrect mtime-fallback rebuild.
- Q6: Standardize error reporting in repo.rs nonstop path to {:#} format,
matching the pattern established in the earlier Tier 1 message commit.
Tier 1 message fixes (remaining items from assessment Part 5.3/5.4):
- M26: build-redbear.sh:569 '30-60 minutes' -> config-dependent range
- A6/A8: Makefile container messages add remediation hint
- B3: mk/config.mk sccache hint adds install guidance
- H2: mk/qemu.mk Unsupported ARCH lists supported values
- mk/redbear.mk comment 'fully-patched' -> 'versioned'
Verification: cargo check 0 errors, cargo test --lib 35/35 passed,
cargo clippy 0 new warnings vs baseline, make -n live + validate clean.
vasilito