From 3139b4880e4784ea9c50c427ff24d0ed82a27505 Mon Sep 17 00:00:00 2001 From: vasilito Date: Fri, 24 Jul 2026 05:49:51 +0900 Subject: [PATCH] =?UTF-8?q?docs:=20v3.2=20=E2=80=94=20QEMU=20gate=20passed?= =?UTF-8?q?,=20thread::scope=20platform=20quirk,=20staleness=20fix?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Plan v3.2: runtime gate PASSED (initfs ahcid bind, switchroot, rootfs e1000d concurrent bind with zero deferred, scheme live, resident hotplug loop); the four runtime-only bugs it found and their fixes; cookbook cargo-path-dep staleness fix verified. - Assessment § 12: gate evidence table + runtime-only bug register. - PACKAGE-BUILD-QUIRKS.md: new platform quirk — std::thread::scope hangs on the Redox target (park/unpark unvalidated); spawn+join is the required pattern; RwLock::write and park/scope remain unvalidated. - Pointers bumped to v3.2. --- docs/README.md | 2 +- local/AGENTS.md | 2 +- local/docs/DRIVER-MANAGER-MIGRATION-PLAN.md | 60 ++++++++++++++++++- local/docs/PACKAGE-BUILD-QUIRKS.md | 29 +++++++++ .../driver-manager/ASSESSMENT-2026-07-22.md | 30 ++++++++++ 5 files changed, 119 insertions(+), 4 deletions(-) diff --git a/docs/README.md b/docs/README.md index 27e5d6838b..ccc239d892 100644 --- a/docs/README.md +++ b/docs/README.md @@ -65,7 +65,7 @@ console-to-KDE plan. - `../local/docs/ACPI-IMPROVEMENT-PLAN.md` — ACPI ownership, robustness, validation - `../local/docs/IRQ-AND-LOWLEVEL-CONTROLLERS-ENHANCEMENT-PLAN.md` — PCI/IRQ quality, MSI/MSI-X - `../local/docs/DRM-MODERNIZATION-EXECUTION-PLAN.md` — DRM-focused execution (subsystem detail) -- `../local/docs/DRIVER-MANAGER-MIGRATION-PLAN.md` (v3.1, 2026-07-23) — migration from `pcid-spawner` to `driver-manager`. **CUTOVER COMPLETE (2026-07-23, operator-ratified):** driver-manager owns the boot-time PCI match/claim/spawn path in every `redbear-*` config; pcid-spawner is retired from configs and gated behind `/etc/driver-manager.d/disabled` as the operator fallback (never deleted). See `../local/docs/evidence/driver-manager/ASSESSMENT-2026-07-22.md` (major assessment, findings B1–G10 + § 11 services integration). Delivered: claim-via-channel collapse (pcid `ENOLCK` exclusivity — the pcid-spawner model); init `ConditionPathExists` gate (systemd-style, `!` negation); spawned-mode `pci_register_driver` in linux-kpi (honors `PCID_CLIENT_CHANNEL` — single ownership of match-claim-spawn for native and Linux-port daemons); linux-kpi real MSI/MSI-X via pcid_interface + `pci_request_regions` + `pcie_capability_*` + PM state; redbear-iwlwifi `--daemon` onboarding + `70-wifi.toml`; `--import-linux-ids` Linux id_table→TOML pipeline; scheme operator surface (`bind`/`unbind`/`new_id`/`remove_id`/`driver_override`/`rescan`); Tier-1 driver_override; success-triggered deferred retry; concurrent probes with real `Driver::probe()`; real SIGCHLD/SIGHUP handlers; heartbeat counters; AER `route_to_driver`; `exclusive_with` mutual exclusion; `pci=nomsi` env var; PciQuirkFlags spawn env hints + `REDBEAR_DRIVER_{IOMMU_GROUP,NUMA_NODE,MSIX_VECTORS}`; driver-params bridge live; D-Bus correctly absent (bridge-on-demand). Open: P0-4 QEMU runtime gate (in progress), P2-1 pcid AER/pciehp producers, P3 policy/hygiene. 93 tests pass; 0 audit-no-stubs violations; zero crate-local warnings on host and redox target. +- `../local/docs/DRIVER-MANAGER-MIGRATION-PLAN.md` (v3.2, 2026-07-24) — migration from `pcid-spawner` to `driver-manager`. **CUTOVER COMPLETE (2026-07-23, operator-ratified):** driver-manager owns the boot-time PCI match/claim/spawn path in every `redbear-*` config; pcid-spawner is retired from configs and gated behind `/etc/driver-manager.d/disabled` as the operator fallback (never deleted). See `../local/docs/evidence/driver-manager/ASSESSMENT-2026-07-22.md` (major assessment, findings B1–G10 + § 11 services integration). Delivered: claim-via-channel collapse (pcid `ENOLCK` exclusivity — the pcid-spawner model); init `ConditionPathExists` gate (systemd-style, `!` negation); spawned-mode `pci_register_driver` in linux-kpi (honors `PCID_CLIENT_CHANNEL` — single ownership of match-claim-spawn for native and Linux-port daemons); linux-kpi real MSI/MSI-X via pcid_interface + `pci_request_regions` + `pcie_capability_*` + PM state; redbear-iwlwifi `--daemon` onboarding + `70-wifi.toml`; `--import-linux-ids` Linux id_table→TOML pipeline; scheme operator surface (`bind`/`unbind`/`new_id`/`remove_id`/`driver_override`/`rescan`); Tier-1 driver_override; success-triggered deferred retry; concurrent probes with real `Driver::probe()`; real SIGCHLD/SIGHUP handlers; heartbeat counters; AER `route_to_driver`; `exclusive_with` mutual exclusion; `pci=nomsi` env var; PciQuirkFlags spawn env hints + `REDBEAR_DRIVER_{IOMMU_GROUP,NUMA_NODE,MSIX_VECTORS}`; driver-params bridge live; D-Bus correctly absent (bridge-on-demand). Gate: PASSED in QEMU q35 (initfs ahcid bind, rootfs e1000d concurrent bind, scheme live, resident hotplug). Platform finding: `thread::scope` hangs on Redox (spawn+join used instead). Cookbook now hashes Cargo path-dep sources (staleness hole closed). Open: P2-1 pcid AER/pciehp producers, P3 policy/hygiene. 93 tests pass; 0 audit-no-stubs violations; zero crate-local warnings on host and redox target. - `../local/docs/WAYLAND-IMPLEMENTATION-PLAN.md` — Wayland compositor (subsystem detail) - `../local/docs/archived/RELIBC-IPC-ASSESSMENT-AND-IMPROVEMENT-PLAN.md` — relibc IPC surface - `../local/docs/GREETER-LOGIN-IMPLEMENTATION-PLAN.md` — greeter/login design diff --git a/local/AGENTS.md b/local/AGENTS.md index 6602fa30e2..b3286c4654 100644 --- a/local/AGENTS.md +++ b/local/AGENTS.md @@ -1422,7 +1422,7 @@ When mainline updates affect our work: also be treated as first-class subsystem plans, not as side notes. - `local/docs/IRQ-AND-LOWLEVEL-CONTROLLERS-ENHANCEMENT-PLAN.md` is the current umbrella plan for IRQ delivery, MSI/MSI-X quality, IOMMU validation, and other low-level controller completeness work. -- `local/docs/DRIVER-MANAGER-MIGRATION-PLAN.md` (v3.1, 2026-07-23) is the canonical planning +- `local/docs/DRIVER-MANAGER-MIGRATION-PLAN.md` (v3.2, 2026-07-24) is the canonical planning authority for the migration from `pcid-spawner` (`local/sources/base/drivers/pcid-spawner/`) to `driver-manager` (`local/recipes/system/driver-manager/`). **CUTOVER COMPLETE (2026-07-23, operator-ratified):** driver-manager owns the boot-time PCI match/claim/spawn path in every diff --git a/local/docs/DRIVER-MANAGER-MIGRATION-PLAN.md b/local/docs/DRIVER-MANAGER-MIGRATION-PLAN.md index 4b9765d069..9323b7f846 100644 --- a/local/docs/DRIVER-MANAGER-MIGRATION-PLAN.md +++ b/local/docs/DRIVER-MANAGER-MIGRATION-PLAN.md @@ -1,11 +1,67 @@ # Red Bear OS — `pci-spawner` → `driver-manager` Migration Plan -**Document status:** v3.1 canonical planning authority (supersedes v3.0 with the completed P0/LDR/P2 implementation and the operator-ratified cutover: driver-manager now owns the boot path in all `redbear-*` configs; pcid-spawner is gated as the fallback) +**Document status:** v3.2 canonical planning authority (supersedes v3.1 with the QEMU runtime gate PASSED: cutover validated end-to-end in QEMU q35 — initfs storage bind, rootfs concurrent probe/claim/spawn, scheme live, resident hotplug loop — plus the `thread::scope` platform finding and the cookbook cargo-path-dep staleness fix) **Generated:** 2026-07-20 **Toolchain:** Rust nightly-2026-05-24 (edition 2024) **Architecture:** Microkernel OS in Rust (Redox fork) **Cross-reference baseline:** Linux kernel 7.1 at commit `ab9de95c9` (`local/reference/linux-7.1/`), CachyOS (`local/reference/cachyos/` — linux-cachyos `0001-cachyos-base-all.patch` 6.17.9 + desktop ISO 260628) -**Assessment:** `local/docs/evidence/driver-manager/ASSESSMENT-2026-07-22.md` (findings B1–G10, § 11 services integration) +**Assessment:** `local/docs/evidence/driver-manager/ASSESSMENT-2026-07-22.md` (findings B1–G10, § 11 services integration, § 12 QEMU gate) + +**v3.2 status update over v3.1 (QEMU runtime gate round, 2026-07-23/24):** + +The P0-4 runtime gate **passed** in QEMU (q35, e1000 + AHCI topology). +The cutover is now validated end-to-end: initfs driver-manager claims and +spawns `ahcid` via the channel model (`bound: 0000--00--1f.2 -> ahcid`), +switchroot proceeds, the resident rootfs manager binds `e1000d` through +the concurrent worker pool (`bound: 0000--00--02.0 -> e1000d`, zero +deferred), registers `scheme:driver-manager`, and enters the resident +hotplug loop. pcid-spawner stays dormant on both phases via +`ConditionPathExists`. + +Four runtime bugs found and fixed by the gate (none visible at compile +grade — exactly why the runtime column exists): + +1. **`thread::scope` hangs on Redox.** The scoped worker completed all + its work (probe, claim, spawn) but the scope join never returned — + every concurrent enumeration stalled the boot. The pool now uses + plain `thread::spawn` + `JoinHandle::join` (captures were already + owned/`Arc`); the counting semaphore is `Arc`-based. **Platform + lesson: `thread::scope`'s park/unpark path is unproven on the Redox + target — prefer spawn+join until relibc/std park is validated.** +2. **pcid config file had no EOF.** The `Handle::Config` read handler + returned 1–4 bytes at any offset forever, so any unbounded + `fs::read` of `/scheme/pci//config` looped infinitely (found + via `propose_msix_vectors`' config read hanging the initfs oneshot). + pcid now EOFs at the config-space boundary (256 fallback / 4096 + ECAM, advertised via fstat), and `read_msix_capability` does a + bounded 256-byte read with a real MSI-X capability-chain parse. +3. **initfs scheme registration collided.** The transient initfs + manager registered `scheme:driver-manager`; the registration + survived into the rootfs phase and the resident manager's + registration failed `EEXIST` → `exit(1)`. initfs mode no longer + registers the scheme (it is transient; the rootfs manager owns it). +4. **Matchless driver TOML broke config loading.** `[[driver]]` + without `[[driver.match]]` (legitimate for USB-class entries) failed + the required-field parse and aborted all config loading. Now + `serde(default)` + regression test. + +Also in this round: pcid PCI 3.0 fallback returns `0xFFFFFFFF` for +extended config reads instead of panicking (i440fx class), two +pre-existing pcid warnings fixed, `redox-driver-pci` reads only the +64-byte header, `30-graphics.toml` moved from the shared config to +`redbear-full.toml` (redox-drm is full-only), vesad removed from +drivers.d (init-managed service), and **the cookbook's content-hash +cache now tracks Cargo `path =` dependency source trees** +(`cook::cargo_path_deps`) — the staleness hole that let +`driver-manager` stay cached while `redox-driver-core` sources changed +is closed and verified (`DEBUG: cargo path-dep source hashes changed` +→ rebuild → cached). + +Open items (updated): **P2-1** pcid AER/pciehp producers, **P3** +policy/hygiene (quirk phases, AER recovery actions, per-vendor +firmware packaging, udev-shim driver-binding view, quirks TOML/parser +flag drift — `/etc/quirks.d/15-audio.toml` uses flags the parser does +not know, dep-crate warnings, `acpid pci_fd is not registered` error). **v3.1 status update over v3.0 (implementation + cutover round, 2026-07-23):** diff --git a/local/docs/PACKAGE-BUILD-QUIRKS.md b/local/docs/PACKAGE-BUILD-QUIRKS.md index 3db2f0768e..f68c40ec6e 100644 --- a/local/docs/PACKAGE-BUILD-QUIRKS.md +++ b/local/docs/PACKAGE-BUILD-QUIRKS.md @@ -16,6 +16,35 @@ in this environment. This document captures all known quirks and their fixes. --- +## Platform Quirk: `std::thread::scope` hangs on the Redox target + +### Problem + +`std::thread::scope`'s join path relies on thread park/unpark semantics that +are **unvalidated on the `x86_64-unknown-redox` target**. Observed in the +driver-manager QEMU gate (2026-07-23): a scoped worker thread completed all +of its work (probe, claim, spawn — visible in logs), but the `thread::scope` +join never returned, stalling the process forever. No panic, no error. + +### Fix Pattern + +Use plain `std::thread::spawn` + `JoinHandle::join` instead of +`thread::scope`. When the scoped closure's captures are all owned +(`Arc`-shared or moved values), scoped threads are unnecessary; the only +thing `thread::scope` adds is structured-join semantics, which +`JoinHandle::join` provides through the proven futex path. Reference: +`local/recipes/drivers/redox-driver-core/source/src/concurrent.rs` (worker +pool), found during the driver-manager cutover runtime gate. + +### Rule + +**Do not use `std::thread::scope` in target code paths until relibc/std +park/unpark is validated on Redox.** `thread::spawn`, `JoinHandle::join`, +`Mutex`, `mpsc`, `Condvar`, and `RwLock::read` are all proven in production; +`RwLock::write` and `thread::park`/`unpark`/`scope` remain unvalidated. + +--- + ## Cookbook Environment: `DYNAMIC_INIT` ### Problem diff --git a/local/docs/evidence/driver-manager/ASSESSMENT-2026-07-22.md b/local/docs/evidence/driver-manager/ASSESSMENT-2026-07-22.md index e05376f879..2dcde3931d 100644 --- a/local/docs/evidence/driver-manager/ASSESSMENT-2026-07-22.md +++ b/local/docs/evidence/driver-manager/ASSESSMENT-2026-07-22.md @@ -335,3 +335,33 @@ udev-shim, driver-params, and the init system: | **initfs storage path** | `40_driver-manager-initfs.service` (oneshot, `--initfs`) is default-on; pcid-spawner-initfs gated as fallback. The initfs driver-manager exits after enumeration (oneshot) — correct for the pre-switchroot phase. | ✅ Wired; first QEMU gate target | *Cutover commits: `e30f2499` (init gate), `c72d4247` (service gates), `b338760111` + parent bumps (config cutover).* + +--- + +## 12. QEMU runtime gate (added 2026-07-24) — PASSED + +The P0-4 gate ran in QEMU q35 (e1000 + AHCI). Evidence: + +| Phase | Result | +|---|---| +| pcid-spawner dormant | `[SKIP] Skipping PCI driver spawner (ConditionPathExists not met)` on both initfs and rootfs | +| initfs driver-manager | `bound: 0000--00--1f.2 -> ahcid`, `enumeration complete: 1 bound, 0 deferred`, exits via oneshot | +| switchroot | proceeds after the initfs oneshot completes | +| rootfs driver-manager | `bound: 0000--00--02.0 -> e1000d` via the concurrent worker pool, **zero deferred** | +| scheme | `driver-manager: registered scheme:driver-manager` | +| resident loop | `hotplug: starting event loop (250 ms poll)` | + +Runtime-only bugs the gate found (all invisible at compile grade): + +1. **`thread::scope` hangs on Redox** — worker completed, scope join never returned. Pool now uses spawn+join; semaphore Arc'd. **Until relibc/std park/unpark is validated on target, `thread::scope` is banned in target code paths.** +2. **pcid config file had no EOF** — unbounded reads looped forever; pcid now EOFs at 256 (fallback) / 4096 (ECAM) with matching fstat; `read_msix_capability` bounded + real MSI-X parse. +3. **initfs scheme registration survived into rootfs** — `EEXIST` on the resident manager's registration → `exit(1)`. initfs mode no longer registers. +4. **Matchless `[[driver]]` TOML broke all config loading** — `serde(default)` + regression test. + +Also fixed: pcid PCI 3.0 fallback returns `0xFFFFFFFF` for extended +reads (no more daemon panic on i440fx); `redox-driver-pci` reads only +the 64-byte header; `30-graphics.toml` moved to `redbear-full.toml`; +vesad dropped from drivers.d; **cookbook staleness hole closed** — +`cook::cargo_path_deps` hashes Cargo `path =` dep source trees into the +dep-hash record (proven: path-dep edit → `cargo path-dep source hashes +changed` → rebuild → cached).