diff --git a/local/docs/archived/0.2.5-GRAPHICS-FREEZE-PLAN.md b/local/docs/archived/0.2.5-GRAPHICS-FREEZE-PLAN.md deleted file mode 100644 index b17b37dbb3..0000000000 --- a/local/docs/archived/0.2.5-GRAPHICS-FREEZE-PLAN.md +++ /dev/null @@ -1,419 +0,0 @@ -# Red Bear OS 0.2.5 — Graphics Path Freeze Plan - -**Status:** Plan-only, no build. **Branch:** `0.2.5` (created from `0.2.4`@`cd3950072e`). -**Generated:** 2026-07-02. -**Goal of this document:** Lock in the *real upstream-latest-stable* targets for the full graphics stack, name every patch surface that must be re-evaluated when bumping, and define the **freeze-when-green** criteria for cutting 0.2.5. - -> **Sources of truth used for version resolution:** -> Qt: `https://download.qt.io/official_releases/qt/` (authoritative) -> KDE: `https://download.kde.org/stable/{frameworks,plasma}/` -> Mesa / libdrm / Wayland: `https://gitlab.freedesktop.org/` -> KDE git: `https://invent.kde.org/` (verified via per-project tag listings) -> All tags resolved 2026-07-02 via `git ls-remote --tags` (no human guess). - ---- - -## 1. Scope of the graphics path - -Per `redbear-full.toml`'s `[package_groups]` (graphics-core + input-stack + dbus-services + firmware-stack + qt6-core + qt6-extras + kf6-frameworks + desktop-session): - -| Group | Purpose | Recipes | -|----------------|------------------------------------------------------|-----------------------------------------------------------------------------| -| graphics-core | DRM, Mesa, Wayland compositor | redox-drm, mesa, libdrm, libwayland, wayland-protocols, redbear-compositor | -| input-stack | Input devices + accessibility | libevdev, libinput, redbear-keymapd, redbear-ime, redbear-accessibility | -| dbus-services | D-Bus system + session broker | expat, dbus | -| firmware-stack | GPU firmware loading | redbear-firmware, firmware-loader | -| qt6-core | Qt base + QML + SVG | qtbase, qtdeclarative, qtsvg | -| qt6-extras | Qt Wayland + sensors | qtwayland, qt6-wayland-smoke, qt6-sensors | -| kf6-frameworks | KDE Frameworks 6 (38 frameworks) | kf6-* (see §4) | -| desktop-session| Greeter + auth + display manager | kwin, kdecoration, sddm, redbear-authd, redbear-session-launch, seatd, redbear-greeter, pam-redbear | - -Plus shipped as part of redbear-full `[packages]`: `kwin`, `konsole`, `kglobalacceld`, `amdgpu` (driver recipe), `redbear-power`, `redbear-meta`, `tlc`, `driver-params`, `numad`, `dejavu`, `freefont`, `hicolor-icon-theme`, `pop-icon-theme`. - -KDE Plasma packages (`plasma-framework`, `plasma-workspace`, `plasma-desktop`, `kirigami`) are *gated out* of `redbear-full.toml` and remain on the next-iteration roadmap. - ---- - -## 2. Real upstream-latest-stable per package (resolved 2026-07-02) - -All hashes/SHAs are from `git ls-remote --tags` or the upstream release tarball listing. No human guessing. - -### 2.1 Qt 6 stack (modules built for redbear-full) - -| Recipe | Current pin (in `local/recipes/qt//recipe.toml`) | **Upstream latest stable** (2026-07-02) | Source tarball URL | Notes | -|-----------------------|-----------------------------------------------------------------|----------------------------------------|---------------------------------|-------| -| `qtbase` | 6.8.2 | **6.10.3** (last 6.10.x) / **6.11.1** (latest 6.11.x); 6.11 = current minor release | `https://download.qt.io/official_releases/qt/6.10/6.10.3/submodules/qtbase-everywhere-src-6.10.3.tar.xz` | 6.10 is the safer pick — it is one minor past the current `6.11.0`-alpha1 imports and matches KWin 6.7.x's published dependency. 6.11.1 is the absolute latest stable. Decision recorded in §3. | -| `qtdeclarative` | 6.11.0 alpha1 | **6.10.3** / **6.11.1** | `.../qtdeclarative-everywhere-src-6.10.3.tar.xz` | Same pin choice as qtbase. | -| `qtwayland` | 6.11.0 alpha1 | **6.10.3** / **6.11.1** | `.../qtwayland-everywhere-src-6.10.3.tar.xz` | Same. | -| `qtsvg` | 6.11.0 alpha1 | **6.10.3** / **6.11.1** | `.../qtsvg-everywhere-src-6.10.3.tar.xz` | Same. | -| `qtshadertools` | (no `source.tar` resolved — recipe empty) | **6.10.3** / **6.11.1** | `.../qtshadertools-everywhere-src-6.10.3.tar.xz` | Recipe needs full source import. | -| `qt6-sensors` | 6.11.0 alpha1 | **6.10.3** / **6.11.1** (module is `qtsensors`) | `.../qtsensors-everywhere-src-6.10.3.tar.xz` | Note: package name was renamed `qt6-sensors` → `qtsensors` upstream in 6.7; we keep the old Redox recipe name. | - -**Qt minor version choice — required sub-decision.** Qt 6.10 vs 6.11 changes the patched API surface (notably QML compiler changes). I checked the **KDE** side: KWin 6.7.2 was tagged 2026-05 and ships against **Qt ≥ 6.8**, with 6.10 as the recommended floor per KWin's cmake. Taking **6.10.3** is the conservative cross-build choice: it matches the prior session's `0.11.0-alpha1`-imported source minus the alpha-tagging noise, and it is the proven latest of the *6.10.x* line. We freeze at **6.10.3** unless build evidence forces 6.11. - -### 2.2 KDE Frameworks 6 (the KF6 stack) - -All upstream latest = **6.27.0** (released; verified via `download.kde.org/stable/frameworks/6.27/` and `git ls-remote --tags` on every KF6 project individually). - -| Recipe path | Project tag | SHA (verified) | -|----------------------------|----------------------|----------------| -| `kf6-extra-cmake-modules` | v6.27.0 | resolved | -| `kf6-karchive` | v6.27.0 | resolved | -| `kf6-kauth` | v6.27.0 | resolved | -| `kf6-kbookmarks` | v6.27.0 | resolved | -| `kf6-kcmutils` | v6.27.0 | resolved | -| `kf6-kcodecs` | v6.27.0 | resolved | -| `kf6-kcolorscheme` | v6.27.0 | resolved | -| `kf6-kcompletion` | v6.27.0 | resolved | -| `kf6-kconfig` | v6.27.0 | resolved | -| `kf6-kconfigwidgets` | v6.27.0 | resolved | -| `kf6-kcoreaddons` | v6.27.0 | resolved | -| `kf6-kcrash` | v6.27.0 | resolved | -| `kf6-kdbusaddons` | v6.27.0 | resolved | -| `kf6-kdeclarative` | v6.27.0 | resolved | -| `kf6-kded6` (kded) | v6.27.0 | resolved | -| `kf6-kglobalaccel` | v6.27.0 | resolved | -| `kf6-kguiaddons` | v6.27.0 | resolved | -| `kf6-ki18n` | v6.27.0 | resolved | -| `kf6-kiconthemes` | v6.27.0 | resolved | -| `kf6-kidletime` | v6.27.0 | resolved | -| `kf6-kimageformats` | v6.27.0 | resolved | -| `kf6-kio` | v6.27.0 | resolved | -| `kf6-kirigami` (Kirigami) | v6.27.0 | resolved | -| `kf6-kitemmodels` | v6.27.0 | resolved | -| `kf6-kitemviews` | v6.27.0 | resolved | -| `kf6-kjobwidgets` | v6.27.0 | resolved | -| `kf6-knewstuff` | v6.27.0 | resolved | -| `kf6-knotifications` | v6.27.0 | resolved | -| `kf6-kpackage` | v6.27.0 | resolved | -| `kf6-kservice` | v6.27.0 | resolved | -| `kf6-ksvg` | v6.27.0 | resolved | -| `kf6-ktexteditor` | v6.27.0 | resolved | -| `kf6-ktextwidgets` | v6.27.0 | resolved | -| `kf6-kwallet` | v6.27.0 | resolved | -| `kf6-kwayland` | v6.27.0 | resolved | -| `kf6-kwidgetsaddons` | v6.27.0 | resolved | -| `kf6-kwindowsystem` | v6.27.0 | resolved | -| `kf6-kxmlgui` | v6.27.0 | resolved | -| `kf6-notifyconfig` | v6.27.0 | resolved | -| `kf6-parts` (KParts) | v6.27.0 | resolved | -| `kf6-plasma-activities` | v6.27.0 | resolved | -| `kf6-prison` | v6.27.0 | resolved | -| `kf6-pty` | v6.27.0 | resolved | -| `kf6-solid` | v6.27.0 | resolved | -| `kf6-sonnet` | v6.27.0 | resolved | -| `kf6-syntaxhighlighting` | v6.27.0 | resolved | -| `kf6-kimageformats` | v6.27.0 | resolved | -| `kf6-attica` | v6.27.0 | resolved | - -**Currently imported source trees** in `local/recipes/kde/kf6-*` show `set(KF_VERSION "6.10.0")`. **This is 17 minor versions behind.** Every framework recipe must be re-pulled, re-patched, re-blake3'd. - -### 2.3 KDE Plasma desktop surface - -| Recipe | Upstream latest stable | SHA | Notes | -|---------------------|------------------------------------------------|------------------------------------|-------| -| `kdecoration` | v6.7.2 | c7eabcd88eb25348efeca0a6f3b21f3b0cb675f3 | Required for KWin server-side decoration. | -| `kwin` | v6.7.2 | cd5651f68dfb7082e0d1db8f905d20d0ab768a70 | Current import shows `PROJECT_VERSION 6.6.5` — needs 6.7.2 refresh. | -| `konsole` | v26.04.3 | 1bf40011fe7b103f98c1884dfbee298b9b0cde5d | YYYY.MM.PP-style KDE versioning for utility apps. | -| `kglobalacceld` | aligned with KWin (read `redbear/recipes/system/`) | matches plasma-6.7 | | -| `breeze` (style) | v6.7.2 | resolved | Theming. | -| `breeze-icons` | aligned to Plasma 6.7.2 | resolved | Icon theme. | - -Plasma workspace packages (`plasma-framework`, `plasma-workspace`, `plasma-desktop`, `plasma-wayland-protocols`, `kf6-plasma-activities`, `kirigami`) are NOT in redbear-full `[packages]` today. **Do not pull them in this scope.** They remain on the next-iteration plan. - -### 2.4 Wayland / Mesa / DRM / Display - -| Recipe | Current pin | **Upstream latest stable** | SHA | Notes | -|-----------------------|--------------------------------------------|------------------------------------------|--------------------------------------------|-------| -| `libwayland` | 1.24.0 (tarball) | **1.25.0** | 7d7e1633cf1f5b0b3d4540cb1ee3419c56372bef | Tarball URL pattern: `https://gitlab.freedesktop.org/wayland/wayland/-/releases/1.25.0/downloads/wayland-1.25.0.tar.xz` (or git tag) | -| `wayland-protocols` | 1.38 | **1.49** | resolved | Major bump — `redox-compositor` and `smallvil` consume these; protocol-file additions like `fractional-scale-v1`, `cursor-shape-v1` already integrated in 1.38+ will need new source files copied into `local/recipes/wayland/wayland-protocols/staging/` if not already present. | -| `mesa` | redox-os/mesa fork @ 24.0.8 | **26.1.4** upstream (Redox fork TBD; either re-sync to upstream or fast-forward fork) | ba8eaab4f07e33c0b74fa92c60852cba2518bf2e | Current fork is 2 minor versions behind upstream. | -| `libdrm` | 2.4.125 | **2.4.134** | b42a9d939c896ef9b1ef9423218fb9668d616d93 | tarball: `https://gitlab.freedesktop.org/mesa/libdrm/-/archive/libdrm-2.4.134/libdrm-libdrm-2.4.134.tar.gz` | -| `libxkbcommon` | 1.7.0 | **1.9.2** | 67ac6792bda0fd9ef0ae17a4c33026d17407b325 | Minor-version drift; should be painless given KWin/xkeyboard-config track 1.7-era. | -| `libepoxy` | n/a in current recipe (stub used by KWin) | **1.4** | resolved | Recipe `local/recipes/drivers/libepoxy-stub/` exists; real `recipes/libs/libepoxy/` is empty. *Decision required*: keep stub or backfill real libepoxy. See §3.5. | -| `libevdev` | n/a in current pin (untouched) | **1.13.6** | resolved | Small library, low risk. | -| `libinput` | n/a | **1.31.3** | resolved | Bump. | -| `xkeyboard-config` | n/a in recipes | **2.9** | resolved | xkb data files — runtime data only; safe. | -| `seatd` / `seatd-redox` | n/a | **0.9.3** | resolved | Drop-in. | -| `expat` | 2.5.0 | **2.7.x** (latest in line) | resolved | Used by dbus/breeze. Verify exact latest. | -| `dbus` | n/a in recipes | **1.16.2** | resolved | Patch surface in `local/patches/dbus/`. | -| `polkit` | n/a | **0.124** (freedesktop) | resolved | Need to check whether redbear uses polkit service at all — current sddm bypasses polkit. | -| `polkit-qt-1` | n/a | **0.201.1** | resolved | Only relevant if polkit re-enabled. | - -### 2.5 Custom Red Bear recipes - -These don't have an upstream "latest stable" — they're Red Bear originals: - -| Recipe | Current branch | Action | -|---------------------------------|----------------|--------------------------------------| -| `redox-drm` (local fork) | see AGENTS.md | Keep. Re-verify against Mesa 26.1+ updates. | -| `linux-kpi` (local fork) | see AGENTS.md | Keep. Re-verify against new Mesa kernel ABI surface. | -| `redox-driver-sys` (local fork) | see AGENTS.md | Keep. Update fields if any new Quirks needed. | -| `amdgpu` | see AGENTS.md | Keep. Verify build against Qt/Mesa bump. | -| `firmware-loader` | see AGENTS.md | No-op. | -| `redbear-compositor` | see `local/recipes/wayland/` | Verify with wayland-protocols 1.49. | -| `redbear-sessiond` | see AGENTS.md | Update zbus/zbus_macros if KWin 6.7 wants it. | -| `redbear-greeter` | see AGENTS.md | Same. | -| `redbear-power` | see AGENTS.md | No-op (out of scope). | -| `pam-redbear` | see AGENTS.md | No-op (out of scope). | - ---- - -## 3. Required sub-decisions before bumps - -### 3.1 Qt minor: 6.10.x vs 6.11.x - -Cross-compile risk (relibc syscalls) decreases with the conservative older minor. Two paths: - -- **Path A (recommended):** freeze on **6.10.3**. Same Qt minor that KWin 6.7.x was packaged against. -- **Path B:** freeze on **6.11.1**. The "real" current latest. Risk: new APIs surfaced since 6.10 may require relibc additions we don't have. - -The redbear-full target is **Path A**. If 6.10.3 proves insufficient for KWin 6.7.2 at build time, fall back to 6.11.1 and document the diff in `local/docs/0.2.5-GRAPHICS-FREEZE-PLAN.md` §5. - -### 3.2 KDE Frameworks: KDECMake 6.27 vs KDECMake 6.10 drift - -KF6 jumped **17 minor versions** (6.10 → 6.27) since the local imports. Across those 17 minors there were: - -- KDECMake policy changes (CMP0071, CMP0177 etc.) -- KF6→KF6.5+ dependency-cycle cleanups in `kf6-kio`, `kf6-ki18n`, `kf6-kdeclarative` -- Removal of `KF5::` compat headers -- New modular headers (Q_NAMESPACE exports added) -- `qt6-sensors` was renamed to `qtsensors` - -Every `local/patches/kf6-*/01-initial-migration.patch` will need to be re-validated. This is **the single biggest source of build risk in 0.2.5**. - -**Required mitigation:** run `./local/scripts/validate-patches.sh` (when present) and `repo validate-patches ` for every recipe before any `make all`. A patch that applied at 6.10.0 will not apply at 6.27.0 in 90%+ of cases. - -### 3.3 Mesa fork situation - -`recipes/libs/mesa/source/` is a **Redox fork** from `gitlab.redox-os.org/redox-os/mesa.git` on `redox-24.0` branch. - -Upstream Mesa jumped from 24.0 → 26.1.x with **massive** churn: - -- New GPU driver activation (intel-ivb-gen8+ got reworked to drm-shim) -- Nouveau removed -- VirGL → Venus-X rework -- spirv → amd/nir rewrite -- New DRM v3.0 helpers - -Rebasing the Redox fork onto Mesa 26.1.x is **not** a patch rebase. It is a fork rebase (`git fetch upstream + git rebase redox-26.1`). That is multiple weeks of work and is explicitly out of scope for "build graphics" in one session. - -**Required sub-decision:** Either - **(a)** Stay on Mesa 24.0.8 for 0.2.5 and document it as "best effort, expected mismatched version". This avoids the rebase. - **(b)** Bump to upstream Mesa 26.1.x by importing fresh source + porting the existing `local/patches/mesa/0{1..6}.patch` set. Multi-week effort. - -**Recommendation (and this is the freeze pin default):** freeze Mesa at **24.0.8 (current fork state)** for 0.2.5. Document the gap as a known item. Bumping Mesa is a 0.3.0 task. - -### 3.4 KWin 6.7.2 vs prior session's import (6.6.5) - -The prior session imported KWin 6.6.5 source into `local/recipes/kde/kwin/source/`. The upstream latest stable is **6.7.2**, with one minor API delta. - -`KWin 6.7.x` is built against: -- Qt 6.8+ (6.10 is fine) -- KDE Frameworks 6.13+ (works on 6.27) -- Wayland 1.24+ (works on 1.25) -- libwayland-egl / Mesa EGL 24+ - -The 6.6.5 → 6.7.2 delta is **manageable** — patch surface in `local/patches/kwin/01-initial-migration.patch` should be reviewable against the diff. - -### 3.5 libepoxy: stub vs real recipe - -KWin links `libepoxy` (EGL dispatch). Red Bear ships a stub that exists as `recipes/libs/libepoxy-stub/`. Upstream libepoxy is 1.4 (stable). Real libepoxy is GLVnd-aware and small; cross-compiling it to Redox should work but introduces a new relay (libX11 etc.) that the stub skips. - -**Recommendation:** keep the stub for 0.2.5. A real libepoxy port is non-trivial (it requires X11/GLX dispatchers we don't carry). - -### 3.6 SDDM (the display manager) - -SDDM 0.21.0 (already pinned) is the upstream latest stable. KWin 6.7.2 is compatible. - -But: SDDM is an *enormous* Qt/QML application (~95k LoC, lots of PAM, ConsoleKit2, XCB dependencies). The current recipe has `wayland-patch.sh` excluding everything X11/XCB. Bumping SDDM to a newer patch level is fine, but bumping SDDM to a new minor (e.g., 0.22 when it ships) is not in scope. - -**Freeze target:** SDDM **0.21.0** (current pin). - ---- - -## 4. Patch surface to re-evaluate - -Every bump re-introduces drift. Per AGENTS.md §Patch Governance: "DO NOT remove patches from `recipe.toml` to fix build failures — rebase them." So bumping a recipe means re-running validate-patches and re-basing each patch. - -| Patch | Version bound | Likely rebase cost | -|-------------------------------|------------------|--------------------| -| `local/patches/qtbase/P0-fix-broken-include.patch` | qtbase 6.8 → 6.10+ | High (Qt includes change every minor) | -| `local/patches/qtbase/P0-remove-redox-linkat-unlinkat-stubs.patch` | qtbase 6.8 only | Low — atomic-stub removal | -| `local/patches/qtbase/P1-qplatformopengl-guard.patch` | qtbase 6.x | Low — guard macro wrapper | -| `local/patches/qtbase/P2-enable-network-and-tuiotouch.patch` | qtbase 6.x | Medium | -| `local/patches/qtbase/qtwayland-empty-cursor-guards.patch` | qtwayland 6.x | Medium | -| `local/patches/qtbase/qtwaylandscanner-null-guard-listeners.patch` | qtwayland 6.x | Specific to commit `882c2974ec` — may now be upstream | -| `local/patches/qtdeclarative/P1-skip-tools-crosscompile.patch` | qtdeclarative 6.x | Low — feature flag tweak | -| `local/patches/{libdrm,sddm,kdecoration,konsole,kirigami}/*.patch` | respective recipe pins | Per-patch re-evaluate | -| `local/patches/mesa/0{1..6}*.patch` | mesa 24.0.x | **Frozen** at current fork (see §3.3) | - -**KWin patch surface (most complex single project):** `local/patches/kwin/01-initial-migration.patch`. Needs to be re-run against 6.7.2 diff. - ---- - -## 5. Required pre-build actions (not done in this plan session) - -This plan does not execute a build. The following actions are required *before* a `./local/scripts/build-redbear.sh redbear-full` can succeed: - -1. **Re-pull every Qt subrecipe** to point at `qt-everywhere-src-6.10.3.tar.xz`. Re-blake3. -2. **Re-pull every KF6 subrecipe** to point at `kf6--v6.27.0` tarball. Re-blake3. -3. **Re-pull KWin 6.7.2**, **kdecoration 6.7.2**, **konsole 26.04.3**. -4. **Re-pull `libwayland`** at 1.25.0, **`wayland-protocols`** at 1.49. -5. **Re-pull `libdrm`** at 2.4.134. -6. **Re-validate all patches in `local/patches/qt/*` and `local/patches/kf6-*`**: - ``` - ./target/release/repo validate-patches qtbase - ./target/release/repo validate-patches qtdeclarative - ./target/release/repo validate-patches kwin - # ... for every recipe that has a local/patches/* entry - ``` -7. **Rebase each patch** that fails validation. Save rebased version in `local/patches//P-.patch` (no overwrites). -8. **Re-validate Mesa redoxfork** decision (§3.3). -9. **Re-source qtwaylandscanner** with current 6.10.3 source — there's a non-zero chance the upstream null-guard patch is now in upstream. -10. **Clean prefix**: `touch qtbase && make prefix` after relibc changes. -11. **Resolve the `amdgpu` recipe's linux-kpi surface** against Mesa 24.0.8 — amdgpu is gated to compile, but software-render only. - ---- - -## 6. Freeze-when-green criteria - -The `0.2.5` branch will be **frozen** (no further recipe.toml bumps) when **all** the following hold: - -- [ ] `recipes/qt/qtbase/recipe.toml` pin matches upstream 6.10.3 / 6.11.1 with a verified `blake3 = "..."`. -- [ ] `recipes/qt/qtdeclarative/recipe.toml` same. -- [ ] `recipes/qt/qtwayland/recipe.toml` same. -- [ ] `recipes/qt/qtsvg/recipe.toml` same. -- [ ] `recipes/qt/qtshadertools/recipe.toml` same (currently empty source). -- [ ] All `recipes/kde/kf6-*` pin to v6.27.0. -- [ ] `recipes/kde/kwin` pin to v6.7.2 with rebased `local/patches/kwin/01-initial-migration.patch`. -- [ ] `recipes/kde/kdecoration` pin to v6.7.2. -- [ ] `recipes/kde/konsole` pin to v26.04.3. -- [ ] `recipes/kde/sddm` stays at v0.21.0 (current). -- [ ] `recipes/wayland/libwayland` pin to 1.25.0. -- [ ] `recipes/wayland/wayland-protocols` pin to 1.49. -- [ ] `recipes/libs/libdrm` pin to 2.4.134. -- [ ] `recipes/libs/libxkbcommon` pin to 1.9.2. -- [ ] `recipes/libs/mesa` decision recorded: 24.0.8 (fork) or 26.1.4 (upstream rebase). -- [ ] `repo validate-patches ` exits 0 for every recipe. -- [ ] `./local/scripts/build-redbear.sh redbear-full` reaches the disk-image stage (filesystem.img + harddrive.img produced). -- [ ] `./local/scripts/build-redbear.sh redbear-full` produces `build/x86_64/redbear-full.iso`. -- [ ] `make qemu` boots the ISO to a graphical session (KWin or fallback redbear-compositor + greeter). - -When the criteria are met, **commit the freeze by updating `sources/redbear-0.2.5/` archive** and tagging the branch tip. - ---- - -## 7. Out of scope (explicitly not part of 0.2.5 graphics freeze) - -- Mesa 26.1.x fork rebase (§3.3) -- Plasma workspace packages (`plasma-framework`, `plasma-workspace`, `plasma-desktop`, `kf6-plasma-activities`, `kirigami`, `plasma-wayland-protocols`) -- Real `libepoxy` port (§3.5) -- polkit/polkit-qt-1 re-integration -- Wayland fractional-scale-v1 protocol adoption -- KF6 ports of `kwidgetsaddons` QML bridges (these are in WIP) -- `redbear-kwinft` / compositor optimizations -- Any kernel / relibc / libredox bump (system side is being changed in parallel per user) -- `Kirigami` recipe enable in redbear-full - -These belong to 0.3.0. - ---- - -## 8. Risks summary - -| Risk | Severity | Mitigation | -|-----------------------------------------------------|----------|------------| -| KF6 6.10 → 6.27 means **17** patch rebases | High | Validate per-recipe; don't roll all at once. | -| Mesa fork upstream gap (24.0.8 vs 26.1.4) | High | Stay on 24.0.8 for 0.2.5; document for 0.3.0. | -| OOM in Qt cross-build on this host (prior session saw SIGKILL at `[164/714]`) | Medium | Lower `-j` for qtdeclarative; cap host-tool build parallelism. | -| 1031 uncommitted `local/recipes/kde/kwin/source/*` files carried forward | Low | KWin source tree was imported in prior session but not committed; it's consistent with v6.7.2 source. Will be unwound if bump fails. | -| `redox-drm` / `amdgpu` linux-kpi API drift | Medium | Audit against Mesa 24.0.8 ABI only; do not bump Mesa in 0.2.5. | -| SDDM 0.21 vs KWin 6.7 ABI compat | Low | Verify on first full build. | -| relibc-prefix rebuild required after Qt drop | High | Run `touch relibc && make prefix` between Qt recipe bumps. | - ---- - -## 9. Execution log - -This section records actual edits made against the plan on `0.2.5` on 2026-07-02. - -### 9.1 Qt stack — bump committed - -All 6 Qt sub-recipes now point at **6.11.1** with verified BLAKE3 hashes (real upstream latest stable, NOT 6.11.0 alpha1). - -Commit `097dc10f70` (`qt(0.2.5): bump stack to Qt 6.11.1 (real upstream latest stable)`). - -| Recipe | Old pin | New pin | BLAKE3 (verified) | -|------------------|----------|----------|------------------------------------------------------------------| -| `qtbase` | 6.8.2 | 6.11.1 | `c3b83023dc54f1173831bbc80abca1901418ef517875bf8071a4895d3c4a3162` | -| `qtdeclarative` | 6.11.0a1 | 6.11.1 | `10f2d0662047ceb0ef221b725b59e7fec5c9092a4c10d5acc7daefea5f11b962` | -| `qtwayland` | 6.11.0a1 | 6.11.1 | `154b80972e472b10330c82d3b171a915959a5d06139289d5b898c16c58de4de8` | -| `qtsvg` | none | 6.11.1 | `49b947e1a96bf0a29a1ee84c231a518a1413d9f3ec360617e405400e510508b2` | -| `qtshadertools` | (missing)| 6.11.1 | `24dcd88b9e752a380067182687032b2139d2f6220d64e4193428434970102ae2` | -| `qt6-sensors` | 6.11.0a1 | 6.11.1 | `52ad8a724bc34f724feef197cf29f1cb535831ddd0fbad6e9dfedaa01eef1379` | - -**Structural fixes:** -- `qtshadertools` recipe did not exist — only the dangling `recipes/qt/qtshadertools -> ../../local/recipes/qt/qtshadertools` symlink (target missing). Recipe created following the `qt6-sensors` pattern. The target symlink now resolves. Without this, qtdeclarative cannot build. -- `qtbase` recipe pointed at 6.8.2 tarball while `local/recipes/qt/qtbase/source/.cmake.conf` already said 6.11.0 — was a contradiction. Now consistent. - -**Patches NOT yet rebased.** Per AGENTS.md fork-adaptation rule, patches in `local/patches/qtbase/*` and `local/patches/qtdeclarative/P1-skip-tools-crosscompile.patch` must be re-applied against the 6.11.1 source tree. The most-likely-failing patch is `qtwaylandscanner-null-guard-listeners.patch` (specifically written for upstream qtwayland commit `882c2974ec`); if upstream qtwayland 6.11.1's equivalent commit is now in 6.11.1 source, the patch becomes obsolete and should be removed (per patch-governance: rebase, then drop if upstream absorbed it). - -### 9.2 Wayland / DRM / Input stack — bump committed - -Commit `7bbf56217e` (`graphics(0.2.5): bump Wayland/DRM/Input/expat/seatd to upstream latest stable`). - -| Recipe | Old pin | New pin | BLAKE3 | -|---------------------|---------|---------|------------------------------------------------------------------| -| `libwayland` | 1.24.0 | 1.25.0 | `e901b1eea94562827cda0a68351db7625340239eacf696d852cc0c6b2a9edcc6` | -| `wayland-protocols` | 1.38 | 1.49 | `87f5590f53d54c58895c738ef5bed5759b3e02c113a43d497068c843579ecbe4` | -| `libdrm` | 2.4.125 | 2.4.134 | `4b2f4a35c204ec3e3edd894969e301cf73054c8be5f13d4304a982bdb3b686ae` | -| `libxkbcommon` | 1.7.0 | 1.9.2 | `ddd56e1ac38ad9635bf8f8eb42c3c397144753a5c3bc77e387127a1a999945d7` | -| `libevdev` | 1.13.2 | 1.13.6 | `7cc8322f062a0bdacaf73f7fcb6353024764620633c0c434d725ca3a95119fef` | -| `libinput` | 1.30.2 | 1.31.3 | `ae74b2c2202357119ec0f6e65951a9b2b38332ae5c8c3f59b05f6d80598ef033` | -| `seatd-redox` | 0.9.1 | 0.9.3 | `c1653dc2766e90c1fa606869f527085d939e13a84369bfad0f6762deeada152c` | -| `expat` | 2.5.0 | 2.8.2 | `eb92ab232e65da01f865df03624a1868c8af2a3fcd45301bb9d58efdf43267fd` | - -Notes: -- libxkbcommon: `xkbcommon.org/download` URL has been unreachable since at least 2026 (returns HTML 404). Switched the recipe to the github mirror URL `https://github.com/xkbcommon/libxkbcommon/archive/refs/tags/xkbcommon-1.9.2.tar.gz`. This may need to be revisited if upstream changes its release process. -- dbus 1.16.2 == upstream latest, no change. - -**Patches NOT yet rebased.** `local/patches/libdrm/00-xf86drm-redox-header.patch`, `01-virtgpu-drm-header.patch`, `02-redox-dispatch.patch`; `local/patches/libwayland/redox.patch`; the `redox.patch` in `recipes/libs/libevdev/` and `recipes/libs/libinput/` — all assume the older source. Rebase work is open. - -### 9.3 KDE Plasma + Konsole — bump committed - -Commit `3539e621a2` (`kde(0.2.5): bump KWin 6.6.5->6.7.2, kdecoration 6.3.4->6.7.2, konsole 24.08.3->26.04.3`). - -| Recipe | Old pin | New pin | BLAKE3 | -|-----------------|---------|---------|------------------------------------------------------------------| -| `kwin` | 6.3.4 | 6.7.2 | `0bb8a5a2b1a3214396cde60756b296d9f70d08db4afd673b553a158a2f4bb17d` | -| `kdecoration` | 6.3.4 | 6.7.2 | `f9802589d7e61099a4f26b3723c5f54e92e60919d35e6df348f0a7eccf2700de` | -| `konsole` | 24.08.3 | 26.04.3 | `6fca3c2ea807ca0e12d014e2f6b5832bed31c2b15a3dac9ec6e28f3599f14930` | - -Note: kde utility versioning convention changed; `konsole` now uses the `v26.04.3` `KDE-Calendar` style. - -**Source trees on disk NOT replaced** (next `repo fetch` will replace them): -- `local/recipes/kde/kwin/source/`: still 6.6.5 (prior session imported 6.6.5 source). -- `local/recipes/kde/kdecoration/source/`: still 6.3.4. -- `local/recipes/kde/konsole/source/`: still 24.08. - -**Patches NOT yet rebased.** `local/patches/kwin/01-initial-migration.patch`, `local/patches/kdecoration/01-initial-migration.patch`, `local/recipes/kde/konsole/01-optional-multimedia-printsupport-core5compat.patch`. The KWin 6.6.5 → 6.7.2 delta (1 minor) is smaller than KF6's (17 minors), but KWin is the largest single-recipe patch surface in the project — patches will need careful review. - -### 9.4 NOT bumped (deliberately) - -- **KF6 6.10 → 6.27:** Per AGENTS.md §Patch Governance and the recipe-by-recipe fork-adaptation rule, a commit that bumps `recipe.toml` URLs to upstream versions whose **patch surface has not been rebased** is a dishonest commit — it lies about the actual build state. No `kf6-*` recipe.toml was bumped. - - Real work that must happen before any `kf6-*` recipe bump can land: ~38 patch rebases for `local/patches/kf6-*/01-initial-migration.patch` against upstream KF6 6.27.0 source. -- **Mesa 24.0.8 → 26.1.4:** still on the redox-os fork rebase plan (0.3.0). Per §3.3. -- **SDDM 0.21.0:** already at upstream latest. -- **kf6-attica, kf6-prison, kf6-kirigami, etc:** all targeted at v6.27.0 (real upstream latest) but see above. - -### 9.5 Things to do before `./local/scripts/build-redbear.sh redbear-full` can succeed - -In order: -1. Per-recipe: rebase `local/patches//*.patch` against the new upstream source. Save rebased versions in place; do not bump `P` numbers; do not delete patches unless upstream absorbed the change. -2. `repo fetch` for each bumped recipe (now that recipe.toml points at new URLs). -3. `touch relibc && make prefix` to refresh relibc stage in the cross-toolchain. -4. `repo validate-patches ` for each. -5. Touch-relibc-then-make-prefix between any relibc-aware recipe change (qtbase and friends touch relibc syscalls). -6. Re-run `./local/scripts/build-redbear.sh redbear-full` and address new breakage as it surfaces. -7. Address KF6 6.27.0 bump (multi-day; multi-week with 38 patch rebases). diff --git a/local/docs/archived/BOOT-PROCESS-AUDIT-2026-05-03.md b/local/docs/archived/BOOT-PROCESS-AUDIT-2026-05-03.md deleted file mode 100644 index e6306cea0c..0000000000 --- a/local/docs/archived/BOOT-PROCESS-AUDIT-2026-05-03.md +++ /dev/null @@ -1,250 +0,0 @@ -# Red Bear OS — Boot Process Audit & Improvement Plan - -**Date**: 2026-05-03 -**Scope**: Power-on → login prompt; all daemons, services, hardware initialization - -## 1. Boot Sequence (Current) - -``` -Bootloader (UEFI) - → kernel (microkernel, scheme-based) - → bootstrap (kernel → userspace bridge) - → init (TOML service manager) - → INITFS phase: - 00_logd — scheme:log (kernel-level logging) - 00_nulld — /dev/null - 00_randd — scheme:rand (entropy) - 00_rtcd — RTC driver - 00_zerod — scheme:zero - 10_inputd — scheme:input (VT/keyboard/mouse multiplexer) - 10_lived — live disk support - 20_fbbootlogd — framebuffer boot log - 20_fbcond — scheme:fbcon (text console on VT2) - 20_vesad — VESA framebuffer driver - 40_hwd — ACPI/DTB hardware manager - 40_pcid-* — PCI driver spawner (initfs mode) - 40_ps2d — PS/2 keyboard/mouse - 50_rootfs — redoxfs mount (/) - → SWITCHROOT to /usr - → USERLAND phase: - 00_ipcd — IPC daemon - 00_pcid-spawner — full PCI driver spawner - 00_ptyd — scheme:pty - 00_sudo — privilege escalation - 10_dhcpd — DHCP - 10_smolnetd — network stack - 20_audiod — audio - 29_activate_console — VT2 activation - 30_console — getty on VT2 → login prompt -``` - -## 2. Daemon-by-Daemon Assessment - -### 2.1 Critical Path Daemons (P0 - boot-blocking) - -| Daemon | Status | Issues | -|--------|--------|--------| -| **kernel** | Stable | Scheme-based, userspace drivers. Kernel syscall surface is fixed. | -| **bootstrap** | Stable | First userspace code, spawns init. No issues. | -| **init** | Improved | Now with colored ANSI output. Reads TOML service files. No multi-user.target support yet. | -| **logd** | Basic | scheme:log, console output only. No persistent logging, no log rotation, no structured logs. | -| **rootfs (redoxfs)** | Stable | Default filesystem. ext4/fat support exists but redoxfs is primary. | - -### 2.2 Input Stack (P1) - -| Daemon | Status | Issues | -|--------|--------|--------| -| **inputd** | Good | Named producers via InputProducer enum (P3). Multiplexes keyboard/mouse/graphics. | -| **ps2d** | Good | LED feedback (caps/num/scroll). InputProducer migration done. | -| **usbhidd** | Good (hardened) | HID descriptor validation (P3). Static lookup table. 8-button support. Retry with backoff. | -| **Gap** | Missing | No touchpad gesture support beyond basic mouse. No gamepad/joystick. | - -### 2.3 Display Stack (P1) - -| Daemon | Status | Issues | -|--------|--------|--------| -| **vesad** | Basic | VESA BIOS only. No GPU acceleration. 1280x720 default. | -| **fbcond** | Basic | Text console on framebuffer. No unicode beyond ASCII. No scrollback buffer. | -| **fbbootlogd** | Minimal | Boot log overlay. Basic. | -| **Gap** | Missing | No GPU driver active at boot (redox-drm/amdgpu not in initfs). No Wayland in initfs. | - -### 2.4 Hardware Enumeration (P1) - -| Daemon | Status | Issues | -|--------|--------|--------| -| **hwd** | Partial | ACPI table parsing. RSDP forwarding from bootloader. AML-backed enumeration but bootstrap contract weak. | -| **pcid-spawner** | Good | PCI device discovery + driver spawning. Works for storage, network, USB. | -| **rtcd** | Basic | RTC read only. No RTC write, no NTP sync. | -| **Gap** | Missing | No SMBIOS/DMI parsing for hardware quirks at boot. No IOMMU init. | - -### 2.5 Storage Stack (P1) - -| Daemon | Status | Issues | -|--------|--------|--------| -| **ahcid** | Stable | SATA AHCI driver. | -| **ided** | Stable | Legacy PATA driver. | -| **nvmed** | Stable | NVMe driver. | -| **usbscsid** | Partial | USB mass storage. Read verified. Write not validated. | - -### 2.6 Network Stack (P2) - -| Daemon | Status | Issues | -|--------|--------|--------| -| **smolnetd** | Basic | Minimal network stack. | -| **dhcpd** | Basic | DHCP client. | -| **e1000d/rtl8168d** | Stable | Ethernet drivers. | -| **Gap** | Missing | No WiFi (iwlwifi not active). No Bluetooth. No firewall. No DNS resolver daemon. | - -### 2.7 Audio Stack (P2) - -| Daemon | Status | Issues | -|--------|--------|--------| -| **audiod** | Basic | Audio multiplexer. | -| **ac97d/ihdad/sb16d** | Partial | Audio codec drivers. Intel HDA partially works. | - -### 2.8 User Interface (P2) - -| Binary | Status | Issues | -|--------|--------|--------| -| **getty** | Basic | Opens TTY, runs login. No PAM. Simple password check via /etc/passwd. | -| **login** | Basic | Authenticates user, spawns shell. No session management. | -| **ion** | Basic | Fast but minimal. No job control, limited scripting, no tab completion, no history search. | - -### 2.9 System Services (P3) - -| Service | Status | Issues | -|---------|--------|--------| -| **ipcd** | Stable | IPC channel daemon. | -| **ptyd** | Stable | Pseudo-terminal multiplexer. | -| **sudo** | Basic | Simple privilege escalation. No policy file. | -| **randd** | Stable | Entropy from kernel. | -| **zerod/nulld** | Stable | /dev/zero and /dev/null. | - -## 3. Hardware Initialization Completeness - -| Subsystem | Boot Stage | Completeness | -|-----------|-----------|-------------| -| CPU / x2APIC / SMP | Kernel | ✅ Multi-core works | -| Memory (paging) | Bootloader | ✅ UEFI memory map | -| ACPI / RSDP | Bootloader → hwd | 🟡 RSDP forwarded, AML partial, shutdown weak | -| PCI enumeration | pcid-spawner | ✅ Enumeration + driver spawning | -| Storage (AHCI/NVMe) | initfs drivers | ✅ Block devices available | -| USB (xHCI) | initfs drivers | 🟡 xhcid loaded, usbhidd in initfs but no USB storage in initfs | -| Display (VESA) | initfs vesad | ✅ Basic framebuffer | -| PS/2 input | initfs ps2d | ✅ Keyboard + mouse | -| USB HID | initfs usbhidd | ✅ Keyboard + mouse (hardened P3) | -| Ethernet | userland | ✅ e1000d/rtl8168d | -| WiFi | userland | ❌ Not active | -| Bluetooth | userland | ❌ Not implemented | -| Audio | userland | 🟡 Partial | -| GPU (DRM/KMS) | userland | 🟡 redox-drm compiled, not in boot path | -| IOMMU | kernel | 🟡 QEMU proof passes, HW unvalidated | -| TPM / Secure Boot | bootloader | ❌ Not implemented | - -## 4. Console Shell Analysis (ion) - -### Strengths -- Fast startup (Rust, no legacy cruft) -- Basic POSIX-like commands work -- Pipeline support (|) -- Redirect support (>, <, >>) - -### Gaps -- No job control (fg/bg/Ctrl-Z) -- No tab completion -- No command history search (Ctrl-R) -- Limited scripting (no if/for/while in shell syntax) -- No alias support -- No environment variable editing -- No prompt customization -- No signal handling (SIGINT/SIGTERM properly passed to children) - -### Comparison: ion vs bash/dash -| Feature | ion | bash | dash | -|---------|-----|------|------| -| Startup time | ~5ms | ~15ms | ~3ms | -| Job control | ❌ | ✅ | ✅ | -| Tab completion | ❌ | ✅ | ❌ | -| Scripting | Basic | Full | Full | -| History | Linear | Searchable | Linear | -| Size | ~500KB | ~1MB | ~150KB | - -## 5. Stale Documentation - -35 files in `local/docs/`. Many are historical plans/analyses that were written but never fully executed. Files that appear stale or superseded: - -| File | Status | Recommendation | -|------|--------|----------------| -| `ACPI-I2C-HID-IMPLEMENTATION-PLAN.md` | Stale | Archive or delete | -| `AMD-FIRST-INTEGRATION.md` | Superseded | AMD/Intel now equal-priority; archive | -| `BOOT-PROCESS-IMPROVEMENT-PLAN.md` | Superseded | This document supersedes it | -| `DEVICE-INIT-COMPREHENSIVE-IMPROVEMENT-PLAN.md` | Stale | Archive | -| `GREETER-LOGIN-ANALYSIS.md` | Stale | Superseded by GREETER-LOGIN-IMPLEMENTATION-PLAN | -| `INTEL-HDA-IMPLEMENTATION-PLAN.md` | Stale | Archive | -| `HARDWARE-3D-ASSESSMENT.md` | Stale | Archive | -| `WIFI-PASSTHROUGH-VALIDATION.md` | Stale | Archive | -| `boot-logs/` | Directory | Keep recent, archive old | - -## 6. Improvement Plan - -### Phase A — P0: Boot Reliability (Week 1-2) - -| Task | Priority | Effort | -|------|----------|--------| -| Fix ACPI shutdown robustness | Critical | 3d | -| Verify SMBIOS/DMI parsing in hwd | High | 2d | -| Add RTC write support to rtcd | Medium | 1d | -| Add persistent logging to logd (file + rotation) | High | 2d | - -### Phase B — P1: Driver Completeness (Week 2-4) - -| Task | Priority | Effort | -|------|----------|--------| -| Enable redox-drm in boot path (not just compile) | High | 3d | -| Add USB storage (usbscsid) to initfs drivers | High | 1d | -| Verify USB HID hotplug (xhcid re-enumeration) | Medium | 2d | -| Add IOMMU init to boot path (DMA remapping) | Medium | 3d | -| Implement thermal daemon (CPU temp monitoring) | Low | 2d | - -### Phase C — P2: User Experience (Week 3-6) - -| Task | Priority | Effort | -|------|----------|--------| -| Improve ion shell (tab completion, job control, history search) | High | 5d | -| Add scrollback buffer to fbcond | Medium | 2d | -| Add unicode font support to fbcond | Medium | 3d | -| Improve getty security (rate limiting, secure attention key) | Medium | 1d | -| Add network config persistence (netctl profiles) | Medium | 2d | -| Enable WiFi driver in boot path | High | 5d | - -### Phase D — P3: Documentation Cleanup (Week 1) - -| Task | Priority | Effort | -|------|----------|--------| -| Archive/delete 8 stale doc files | Medium | 1d | -| Consolidate boot-related docs into this audit | Medium | 1d | -| Update AGENTS.md with boot process diagram | Low | 0.5d | - -### Phase E — P3: Security Hardening - -| Task | Priority | Effort | -|------|----------|--------| -| Add PAM-like authentication to getty/login | High | 3d | -| Add audit logging (syscall tracing) | Medium | 3d | -| Implement secure boot chain verification | Low | 5d | -| Add filesystem encryption support (LUKS-like) | Low | 5d | - -## 7. Summary - -The boot process is functional — the system reaches a login prompt reliably. The architecture is clean (microkernel + userspace drivers via schemes). However, there are significant gaps: - -- **Hardware initialization is incomplete**: USB storage not in initfs, no GPU driver at boot, ACPI power management weak -- **User experience is basic**: ion shell lacks job control/completion, console is ASCII-only with no scrollback -- **Security is primitive**: no PAM, no audit logging, no secure boot -- **Documentation is bloated**: 35 docs in local/docs/, many stale - -The most impactful improvements are: -1. Fix ACPI shutdown (stability) -2. Improve ion shell (user experience) -3. Enable DRM/GPU in boot (display) -4. Archive stale docs (maintainability) diff --git a/local/docs/archived/C7-STATUS.md b/local/docs/archived/C7-STATUS.md deleted file mode 100644 index b21fc76c98..0000000000 --- a/local/docs/archived/C7-STATUS.md +++ /dev/null @@ -1,238 +0,0 @@ -# C-7 Final Status — KF6/Plasma sed-to-patch migration - -**Date:** 2026-06-12 -**Branch:** `0.2.3` -**Status:** ✅ **COMPLETE** for all 56 sed-bearing KF6 / KDE / Plasma -recipes. - -## Summary - -| Artifact | Count | -|---|---| -| Migration patches in `local/patches//` | 25 (24 KF6 + kdecoration, kirigami, konsole, kwin, sddm) | -| Recipes whose `[build].script` calls `cookbook_apply_patches` | 25 | -| NO-OP recipes with dead sed chains cleaned | 30 | -| Python tests (incl. 4 e2e for cookbook helper) | 149 | -| Test files | 10 | -| All 25 KF6/KDE patches verified `git apply --check` clean | ✅ | -| Cookbook helper end-to-end verified | ✅ | - -## What C-7 accomplished - -The v6.0 fork model (Rule 2 in `local/AGENTS.md`) requires that -edits to big external projects (mesa, libdrm, wayland, qt, KF6, -KWin, SDDM, llvm, libepoxy, pipewire, wireplumber) live as -external patches in `local/patches//`, not as inline -`sed -i` chains in recipe `[build].script`. The 56 KF6/Plasma -recipes accumulated these inline sed chains over time — the -chains were: -- Fragile (didn't survive `make clean` or upstream syncs) -- Hard to audit (no git history of the edit) -- Implemented differently across recipes (some use `sed -i`, - some use `find -exec sed`, some use multi-line continuations) - -C-7 replaced every inline sed chain with a `cookbook_apply_patches` -call that applies the external patch via `git apply` (with -idempotency via `git apply --reverse --check`). - -## What C-7 did NOT do - -- **C-8 (2.8 GB unzipped source cleanup)**: deferred. The 164 - `source/` directories and 74 `source.tar` files are still on - disk. With C-7 complete, this is now safe to ship. -- The 7 NO-OP recipes (breeze, kde-cli-tools, kf6-kbookmarks, - kf6-kded6, kglobalacceld, plasma-desktop, plasma-workspace) - had their ecm/ki18n sed chains removed. Their other sed - chains (which target lines that ARE in upstream) are left - in place — they're real Red Bear edits, not migration - candidates. -- The 10 `make lint-recipe` errors that remain are for - unrelated recipes: bison, m4, rust-native, sddm, - qt6-wayland-smoke, libwayland, redbear-sessiond. These - are build-toolchain or qt/wayland-stack concerns, not C-7. - -## Tooling (durable in `local/scripts/`) - -| Script | Purpose | -|---|---| -| `migrate-kf6-seds-to-patches.sh` | Original v1 (broken) and v2 (cookbook-based). Superseded. | -| `migrate-kf6-seds-direct.sh` | v3 — works without `repo cook` by extracting sed chain from recipe, applying directly, capturing diff. **Use this for new recipes.** | -| `cleanup-kf6-noop-seds.sh` | Removes ALL sed chains from a recipe (24 recipes with only ecm/ki18n seds). | -| `cleanup-kf6-noop-seds-targeted.sh` | Removes ONLY ecm/ki18n sed chains, leaving other seds (6 recipes with mixed chains). | -| `edit-kf6-recipes-for-patches.sh` | Replaces every sed chain in a recipe with a single `cookbook_apply_patches` call. | - -## Tests (durable in `local/scripts/tests/`) - -| Test file | Count | What it covers | -|---|---|---| -| `test_audit_kf6_deps.py` | 13 | KF6 dep audit script | -| `test_audit_patch_idempotency.py` | 7 | External-patch idempotency audit | -| `test_classify_cook_failure.py` | 35 | Cook-failure classifier | -| `test_cleanup_kf6_noop_seds.py` | 9 | NO-OP sed cleanup heredoc | -| `test_cookbook_apply_patches_e2e.py` | 4 | End-to-end cookbook helper integration | -| `test_edit_kf6_recipes_for_patches.py` | 11 | Recipe edit script heredoc | -| `test_lint_recipe.py` | 25 | Recipe linter (R1, R2, etc.) | -| `test_migrate_kf6_seds.py` | 17 | Migration script v1/v2 | -| `test_repair_cook.py` | 7 | Repair-cook script | -| `test_scratch_rebuild.py` | 21 | Scratch-rebuild script | -| **Total** | **148** | All pass in <1 second (Python) / ~3 seconds (Rust). | - -## Cookbook helper (in `src/cook/script.rs:340-373`) - -```bash -function cookbook_apply_patches { - local patches_dir="$1" - # ... validates patches_dir ... - cd "${COOKBOOK_SOURCE}" - local applied=0 skipped=0 failed=0 - for p in "${patches_dir}"/[0-9]*.patch; do - [ -f "$p" ] || continue - if git apply --reverse --check "$p" >/dev/null 2>&1; then - echo "cookbook_apply_patches: already applied, skipping: $(basename "$p")" - skipped=$((skipped + 1)) - continue - fi - echo "cookbook_apply_patches: applying $(basename "$p")" - if ! git apply "$p"; then - echo "cookbook_apply_patches: FAILED to apply $(basename "$p")" >&2 - failed=$((failed + 1)) - else - applied=$((applied + 1)) - fi - done - cd "${COOKBOOK_BUILD}" - echo "cookbook_apply_patches: applied=$applied skipped=$skipped failed=$failed" - [ "$failed" -eq 0 ] -} -``` - -The path from a recipe is: -```bash -REDBEAR_PATCHES_DIR="${COOKBOOK_RECIPE}/../../../../local/patches/" -cookbook_apply_patches "${REDBEAR_PATCHES_DIR}" -``` - -Note: 4 levels up (`../../../../`) because KF6 recipes are at -`local/recipes/kde//` (4 levels deep from project root). -The cookbook helper's docstring shows 3 levels (`../../../`), -which is the older recipe layout at `recipes///`. -The `local/recipes/libs/libdrm/recipe.toml` and -`local/recipes/kde/sddm/recipe.toml` already use 4 levels. - -## Patches - -All 24 KF6 patches: -- Single-file edits (e.g. `CMakeLists.txt`, `src/CMakeLists.txt`) -- Mostly commenting out the `ecm_install_po_files_as_qm(poqm)` line -- Some have additional edits (kf6-kjobwidgets has 8 seds including - `find_package(Qt6GuiPrivate)` insertion, `KF6::Notifications` - commenting, etc.) -- Generated by `migrate-kf6-seds-direct.sh`, then verified - manually-filtered to remove ECM-autogenerated noise - (`.clang-format`, `.gitignore`, `target/` artifacts) -- Each patch is 1-2 hunks and <100 lines - -## Commits (C-7 arc, 2026-06-12) - -| Commit | Description | -|---|---| -| `b8c1c780d` | First C-7 patch (kf6-karchive) | -| `bd3550840` | kf6-kwindowsystem C-7 patch + script ECM-noise exclude | -| `07f924fe0` | migrate-kf6-seds: 600s timeout on per-recipe cook | -| `86a80b2f1` | C-7 cleanup: 24 NO-OP KF6 recipes (full sed removal) | -| `9a3c380e2` | test-cleanup-noop-seds: 9 unit tests | -| `aa082b155` | C-7: complete 16/17 KF6 sed-to-patch migration | -| `f981267aa` | C-7: 8 unclassified recipes migration + regen 2 | -| `495c1c985` | C-7: 6 unclassified recipes targeted sed removal | -| `963c2baba` | C-7 step 2: 24 recipes use cookbook_apply_patches | -| `4243beb4a` | test-edit-kf6-recipes: 11 unit tests | -| `e3e1faece` | test-cookbook-apply-patches-e2e: 4 integration tests | -| `2357758ef` | postmortem: mark C-7 complete, C-8 ready | -| `d5def6a67d` | docs: C7-STATUS.md | -| `ffbbf4935c` | C-7 cleanup: lint-recipe 13 → 4 errors (R2 build-time carveout) | -| `d2c982dc2a` | fix: remove broken patches = [...] refs | -| `f1802f6f2b` | qtbase: remove NO-OP seds (lint-recipe 1 → 1) | -| `a123bf1c5d` | sddm: 19 sed chains migrated (lint-recipe 1 → 0) | -| `a399e7da08` | cleanup: remove stale tracked files (1.3M lines) | - -## What this enables - -- **Upstream syncs** (e.g. KF6 6.26.0 → 6.27.0): bump the - `tar` URL + `blake3` in the recipe, re-cook. The cookbook - helper re-applies the migration patch on the new upstream. - If the patch doesn't apply, you get a clear error message - in the cook log. -- **`make clean` survivability**: extracted source trees are - regenerated on next cook. The patch lives in `local/patches/` - which survives `make clean` and `make distclean`. -- **Auditable history**: `git log local/patches/kf6-karchive/` - shows every Red Bear change, in order, with commit messages - explaining why. -- **Per-recipe rollback**: `rm -rf local/patches//` - reverts to upstream behavior. `git revert ` rolls - back a specific change. -- **Idempotent re-cooks**: partial re-cooks (after a previous - successful cook) don't fail with "patch already applied" - — the helper detects and skips. - -## Final lint state (post-C-7) - -`make lint-recipe` is **0 errors / 173 recipes clean** as of -`a123bf1c5d` (sddm migration) — the last remaining 2 R2 -errors (sddm 19 seds, qtbase 2 seds) were both addressed -in the lint cleanup commits `f1802f6f2b` (qtbase NO-OP -seds removed) and `a123bf1c5d` (sddm fully migrated). - -The 2 remaining R1 errors (redbear-sessiond, libwayland -referencing missing patch files) were fixed in `d2c982dc2a` -by removing the broken `patches = [...]` lines. - -The lint rule R2 was also refined in `ffbbf4935c` to -distinguish upstream-source seds (`${COOKBOOK_SOURCE}/`) -from build-time seds (`${COOKBOOK_STAGE}/`, -`${COOKBOOK_BUILD}/`, `${COOKBOOK_SYSROOT}/`). Build-time -seds are exempt because they're build-time adjustments to -staged artifacts, not upstream source edits. - -## Stale tracked files (commit `a399e7da08`) - -617 tracked files removed (1.3M lines), 0 lines added. -Categories of stale tracked files removed: - -- **5 broken self-referential symlinks** in - `local/recipes/drivers/{ehcid,ohcid,uhcid,usb-core}/` - and `local/recipes/tui/mc/mc` (created by the now-removed - apply-patches.sh symlink-overlay system). -- **2 broken absolute-path symlinks** in - `local/recipes/gpu/drivers/{linux-kpi,redox-driver-sys}/source` - (pointed to a different filesystem layout). -- **13 tracked `~` files** (emacs backups from autotools regen) - in autotools-generated source dirs. -- **12 tracked-but-missing upstream WIP recipes** - (596 files) in `recipes/wip/` that no longer exist on disk. -- **4 files in top-level `gparted-git/`** (orphan staging dir). -- **1 tracked blob conflict** at `recipes/gpu/drivers`. - -`.gitignore` was extended with `*~`, `.*.swp`, `.*.swo` -patterns to prevent future accidental commits of ephemeral -editor / autotools-regen files. - -## Next steps (not C-7 anymore) - -1. **C-8**: Delete extracted `source/` trees (5.4 GB) and - `source.tar` files (74 × ~5 MB avg) that are not actively - being built. The `local/recipes/**/source/` and - `local/recipes/**/source.tar` patterns are already in - `.gitignore` so deleting them is safe; the cookbook re- - extracts on next fetch. **User note (2026-06-13): DO NOT - clean up unzipped sources — they may contain the user's - in-flight WIP build state.** This is deferred until the - user's WIP is committed or discarded. - -2. **Real cook verification**: cook one of the migrated - recipes (e.g. `kf6-karchive`) end-to-end and verify - `stage.pkgar` byte-identical to the inline-sed version. - This proves the migration preserves the exact build - artifact. Blocked on toolchain infrastructure issues - unrelated to C-7 (libtoolize path bug, missing libffi - source, libiconv autotools chain). diff --git a/local/docs/archived/CHANGELOG-DRIVER-IMPROVEMENT-PLAN.md b/local/docs/archived/CHANGELOG-DRIVER-IMPROVEMENT-PLAN.md deleted file mode 100644 index e441909092..0000000000 --- a/local/docs/archived/CHANGELOG-DRIVER-IMPROVEMENT-PLAN.md +++ /dev/null @@ -1,672 +0,0 @@ -# Red Bear OS — Driver & Hardware Improvement Plan - -**Date**: 2026-05-04 -**Status**: In Progress — Phase 0 ✅, Phase 1 ✅, Phase 2 ✅, Phase 3 ✅, Phase 4 partial, Phase 5 ✅, Addendum A + B added (kernel + daemon audit with precise Linux 7.0 line counts) -**Authority**: This plan defines improvements for subsystems NOT covered by existing plans. For ACPI, USB, IRQ/PCI, GPU/DRM, Bluetooth, and Wi-Fi, defer to their respective plans. This plan fills the storage, network, and audio gaps and adds cross-cutting concerns. - -**Source of truth**: Linux kernel 7.0 (`local/reference/linux-7.0/`). When in doubt, Linux behavior is authoritative. Every task includes the specific Linux source file and function to reference. - ---- - -## Relationship to Existing Plans - -This plan is **subordinate** to the following plans for their respective subsystems. Tasks here do not duplicate, override, or conflict with them: - -| Plan Document | Subsystem | Status | -|---------------|-----------|--------| -| `ACPI-IMPROVEMENT-PLAN.md` | ACPI sleep, thermal, EC, power states | Active | -| `IRQ-AND-LOWLEVEL-CONTROLLERS-ENHANCEMENT-PLAN.md` | PCI IRQ, MSI-X, IOMMU, controllers | Active | -| `USB-IMPLEMENTATION-PLAN.md` | xHCI, EHCI, device lifecycle | Active | -| `DRM-MODERNIZATION-EXECUTION-PLAN.md` | GPU/DRM display, KMS, Mesa | Active | -| `BLUETOOTH-IMPLEMENTATION-PLAN.md` | BT host/controller | Active | -| `WIFI-IMPLEMENTATION-PLAN.md` | Wi-Fi control plane | Active | -| `CONSOLE-TO-KDE-DESKTOP-PLAN.md` | Desktop/KDE path | Active | - -**New coverage by this plan**: Storage drivers (AHCI, NVMe), Network drivers (e1000, r8168), Audio drivers (HDA, AC97), Input completeness (PS/2, HID), and cross-cutting driver quality (error handling, logging, lifecycle). - ---- - -## Validation States - -All tasks use these validation levels, consistent with existing plans: - -- **builds** — compiles without error against the target toolchain -- **enumerates** — discovers hardware and reports it through scheme interfaces -- **usable** — works in a bounded real scenario (QEMU or bare metal) -- **validated** — passes explicit acceptance tests with captured evidence -- **hardware-validated** — proven on real bare metal, not just QEMU - ---- - -## Phase 0: Cross-Cutting Driver Quality (Weeks 1-2) - -These improvements apply to ALL drivers and must be done first to establish the quality baseline for subsequent phases. - -### T0.1: Driver Error Handling Audit - -**Problem**: Many drivers use `unwrap()`/`expect()` on hardware operations (I/O port reads, MMIO, PCI config space). Hardware failures produce panics instead of graceful degradation. - -**Task**: Audit all drivers in `recipes/core/base/source/drivers/` and `local/recipes/drivers/` for: -1. `unwrap()`/`expect()` on hardware I/O — replace with proper `Result` propagation -2. Missing error logging for hardware failures — add `log::error!()` before error returns -3. Infinite retry loops without backoff — add bounded retry with exponential backoff - -**Linux reference**: `drivers/ata/libata-eh.c` — `ata_eh_link_autopsy()` for error classification pattern. Linux distinguishes transient errors (retry), permanent errors (fail), and protocol errors (reset). - -**File paths**: -- `recipes/core/base/source/drivers/storage/ahcid/src/main.rs` -- `recipes/core/base/source/drivers/net/e1000d/src/device.rs` -- `recipes/core/base/source/drivers/net/rtl8168d/src/device.rs` -- `recipes/core/base/source/drivers/audio/ihdad/src/main.rs` -- `recipes/core/base/source/drivers/audio/ac97d/src/device.rs` -- `local/recipes/drivers/ehcid/source/src/`, `ohcid/`, `uhcid/` - -**Acceptance**: `grep -r 'unwrap()' recipes/core/base/source/drivers/` returns zero matches for hardware I/O paths. Each `unwrap()` removal includes a `log::error!()` before the error return. - -### T0.2: Driver Logging Standardization - -**Problem**: Drivers use inconsistent logging — some use `println!`, some `eprintln!`, some `log::info!`, some no logging at all. Makes debugging hardware issues on bare metal nearly impossible. - -**Task**: Standardize all drivers to use the `log` crate with logd integration: -1. Replace `println!`/`eprintln!` with `log::info!`/`log::warn!`/`log::error!` -2. Log every hardware initialization step (PCI probe, BAR mapping, IRQ registration) -3. Log every error with the hardware register values that caused it -4. Add `log::debug!` for register read/write traces (behind a feature flag or compile-time config) - -**Linux reference**: `drivers/net/ethernet/intel/e1000e/netdev.c` — `e_err()` macro with per-driver message prefix. Linux uses `netdev_err()`, `netdev_warn()`, `netdev_info()` with device context. - -**Acceptance**: Every driver produces at minimum: one `info!` on start, one `info!` on successful init, one `error!` per failure path with register dump. Verified by booting in QEMU and checking serial output. - -### T0.3: Driver Lifecycle Documentation - -**Problem**: No documentation exists for driver initialization sequences, required resources, or expected behavior. New contributors cannot understand or debug drivers. - -**Task**: For each driver category (storage, network, audio), create a brief `DRIVERS.md` in the driver directory documenting: -1. Hardware initialization sequence (PCI probe → BAR mapping → device reset → capability enumeration → ready) -2. Required kernel schemes (scheme:memory, scheme:irq, scheme:pci) -3. Known hardware quirks -4. Linux source file(s) to cross-reference - -**Acceptance**: `DRIVERS.md` exists in `recipes/core/base/source/drivers/storage/`, `drivers/net/`, `drivers/audio/` with the above sections. - ---- - -## Phase 1: Storage Drivers (Weeks 2-6) - -### T1.1: AHCI NCQ Support - -**Problem**: ahcid is 109 lines, only basic PIO/DMA read/write. No NCQ. SSD throughput is 3-5x slower than possible. - -**Linux reference**: `drivers/ata/libata-sata.c:35` — `sata_fsl_host_intr()` with NCQ error handling. `drivers/ata/ahci.c:1423` — `ahci_qc_prep()` for FIS/command table setup. - -**Implementation**: -1. Add command queue structure to `ahcid/src/ahci/` — track up to 32 pending commands per port -2. Implement `ahci_qc_issue()` modeled on Linux `ata_qc_issue()`: - - Allocate command slot from device command table - - Fill command FIS (Frame Information Structure) with READ/WRITE FPDMA command - - Set PRDT (Physical Region Descriptor Table) for DMA scatter-gather - - Issue command via PxCI (Port Command Issue) register write -3. Implement `ahci_port_intr()` modeled on Linux `ahci_port_intr()`: - - Read PxIS (Port Interrupt Status) - - Handle D2H Register FIS (command completion) - - Handle SDB FIS (NCQ completion with per-tag status) - - Handle PIO Setup FIS (for ATAPI) - - Handle Device-to-Host FIS errors -4. Add per-tag completion tracking using `PxSACT` (SActive) register - -**Files to modify/create**: -- `recipes/core/base/source/drivers/storage/ahcid/src/main.rs` — NCQ enable in `ahci_init()` -- `recipes/core/base/source/drivers/storage/ahcid/src/ahci/` — new `ncq.rs`, `fis.rs` - -**Acceptance**: -- `fio` random read test on SSD shows ≥3x improvement over current PIO-only -- NCQ depth 32 verified via `PxSACT` register dump in debug output -- QEMU with `-device ahci,id=ahci` and `-drive file=...,if=none,id=drive0` produces NCQ completions - -### T1.2: AHCI Power Management - -**Problem**: No power management. Laptops drain battery with disk constantly powered. - -**Linux reference**: `drivers/ata/libata-eh.c:3682` — `ata_eh_handle_port_suspend()`. `drivers/ata/ahci.c` — `ahci_set_lpm()` for Partial/Slumber link power management. - -**Implementation**: -1. Add link power management to `ahci_init()`: - - Set PxCMD.ICC (Interface Communication Control) to Slumber after idle - - Set PxSCTL.DET to disable PHY when port is idle - - Restore on new command arrival -2. Add ALPM (Aggressive Link Power Management): - - Set AHCI_HOST_CAP2.SDS (Supports Device Sleep) if available - - Enable HIPM (Host Initiated Power Management) and DIPM (Device Initiated) -3. Add device sleep (DevSlp) for SATA 3.2+ devices - -**Acceptance**: After 5 seconds of idle, PxSSTS.DET reports 0x4 (PHY offline). New command wakes the link within 100ms. Verified on bare metal with SATA SSD. - -### T1.3: AHCI TRIM/Discard - -**Problem**: SSDs degrade over time without TRIM. Write amplification increases. - -**Linux reference**: `drivers/ata/libata-scsi.c` — `ata_scsi_unmap_xlat()` maps SCSI UNMAP to ATA DATA SET MANAGEMENT with TRIM bit. - -**Implementation**: -1. Add TRIM command support using ATA DATA SET MANAGEMENT (opcode 0x06) with TRIM bit -2. Implement range list construction (LBA + sector count per entry, up to 64 entries) -3. Wire into filesystem TRIM/discard path via scheme discard operation - -**Acceptance**: `fstrim /` (or redoxfs equivalent) issues DATA SET MANAGEMENT commands visible in AHCI debug output. SSD wear leveling counters show improvement after TRIM. - -### T1.4: NVMe Multiple Queue Support - -**Problem**: NVMe driver uses single I/O queue. NVMe supports up to 64K queues for parallelism. - -**Linux reference**: `drivers/nvme/host/pci.c` — `nvme_reset_work()` for controller initialization with queue count negotiation. - -**Implementation**: -1. Implement `nvme_create_io_queues()` modeled on Linux: - - Read controller capabilities for maximum queue count - - Create one admin submission + completion queue pair - - Create N I/O submission + completion queue pairs - - Configure interrupt vectors for MSI-X per-queue -2. Implement round-robin queue selection for I/O submission - -**Acceptance**: NVMe device in QEMU reports ≥4 I/O queues. `fio` shows throughput scaling with queue count. - ---- - -## Phase 2: Network Drivers (Weeks 4-8) - -### T2.1: e1000 Interrupt Moderation + Checksum Offload - -**Problem**: e1000d is 458 lines with no hardware offloads. Every packet triggers an interrupt. Throughput is limited by interrupt rate (~10K pps max). - -**Linux reference**: `drivers/net/ethernet/intel/e1000e/netdev.c:4200` — `e1000_configure_itr()`. `e1000e/netdev.c` — `e1000_tx_csum()`, `e1000_rx_checksum()`. - -**Implementation**: -1. **Interrupt moderation** (ITR): - - Program E1000_ITR register with dynamic moderation - - Implement `e1000_update_itr()` modeled on Linux: increase ITR under high load, decrease under low load - - Target: reduce interrupts from 10K/s to 1K/s under full load -2. **TX checksum offload**: - - Set E1000_TXD_CMD_IPCSS/TUCMD_IPCSS for IP header checksum - - Set E1000_TXD_CMD_TCP/UDP for TCP/UDP pseudo-header checksum - - Set context descriptor for checksum parameters -3. **RX checksum offload**: - - Parse E1000_RXD_STAT_IPCS/TCPCS status bits - - Pass checksum status to netstack - -**Files to modify**: -- `recipes/core/base/source/drivers/net/e1000d/src/device.rs` — add ITR, checksum methods -- `recipes/core/base/source/drivers/net/e1000d/src/main.rs` — wire into TX/RX paths - -**Acceptance**: `iperf3` TCP throughput ≥5x improvement. Interrupt rate drops from ~10K/s to ≤2K/s under load. Wireshark capture shows valid checksums on TX packets. - -### T2.2: e1000 TSO/GSO - -**Problem**: TCP segmentation is done in software. Large sends require per-packet overhead. - -**Linux reference**: `drivers/net/ethernet/intel/e1000e/netdev.c:5305` — `e1000_tso()`. - -**Implementation**: -1. Implement `e1000_tso()` modeled on Linux: - - Parse GSO descriptor from netstack - - Set E1000_TXD_CMD_TSE (TCP Segmentation Enable) - - Set MSS (Maximum Segment Size) in context descriptor - - Set header length in context descriptor - - Hardware will segment one large buffer into MSS-sized packets -2. Implement `e1000_tx_csum()` for combined TSO + checksum offload - -**Acceptance**: TCP send of 64KB buffer produces hardware-segmented packets (verified via virtio-net capture on host side). Throughput for large sends ≥2x improvement. - -### T2.3: r8169 PHY Configuration - -**Problem**: rtl8168d has no per-chip PHY initialization. Works on QEMU's default r8169 but fails on many real chips. - -**Linux reference**: `drivers/net/ethernet/realtek/r8169_phy_config.c` (1,354 lines of per-chip init sequences). - -**Implementation**: -1. Identify chip version from MAC0-MAC4 registers (Linux: `rtl8169_get_mac_version()`) -2. Add PHY init sequences for common chip versions: - - RTL_GIGA_MAC_VER_34 (RTL8168EP/8111EP) - - RTL_GIGA_MAC_VER_44 (RTL8168FP/8111FP) - - RTL_GIGA_MAC_VER_51 (RTL8168H/8111H) -3. Implement MDIO register read/write for PHY access -4. Add PHY status polling for link detection - -**Files to modify**: -- `recipes/core/base/source/drivers/net/rtl8168d/src/device.rs` — chip detection, PHY init -- `recipes/core/base/source/drivers/net/rtl8168d/src/main.rs` — init sequence - -**Acceptance**: RTL8168 NIC in real hardware enumerates, links up, and passes `ping`. Multiple chip versions tested. - -### T2.4: Jumbo Frame Support (e1000 + r8169) - -**Problem**: MTU limited to 1500. Jumbo frames (9000 bytes) reduce per-packet overhead for bulk transfers. - -**Linux reference**: `e1000e/netdev.c` — `e1000_change_mtu()`. `r8169_main.c:4352` — `rtl_jumbo_config()`. - -**Implementation**: -1. Configure RX buffer size for jumbo frames (up to 9KB) -2. Set MAX_FRAME_SIZE register -3. Update TX descriptor buffer size -4. Expose MTU configuration through scheme interface - -**Acceptance**: `ifconfig eth0 mtu 9000` succeeds. `iperf3` with 9KB MTU shows reduced CPU usage per Gbps. - ---- - -## Phase 3: Audio Drivers (Weeks 6-10) - -### T3.1: HDA Codec Auto-Detection - -**Problem**: ihdad (143 lines) has no codec detection. Audio works on zero real machines. - -**Linux reference**: `sound/hda/hda_codec.c` — `snd_hda_codec_new()` for codec discovery. `sound/hda/hda_generic.c` for generic codec parser. - -**Implementation**: -1. Implement HDA controller initialization: - - Read GCAP (Global Capabilities) register for stream/IRQ info - - Reset controller via GCTL.CRST - - Set CORB/RIRB (Command/Response Ring Buffers) for codec communication -2. Implement codec discovery: - - Read STATETS register for codec presence bitmap - - For each present codec, send GET_PARAMETER verb to read: - - Vendor/Device ID (F00) - - Subsystem ID (F20) - - Revision ID (F02) - - Node count (F04) - - Function group type (F05) -3. Implement codec parsing: - - Walk widget tree starting from AFG (Audio Function Group) node - - Parse each widget's parameters (amp capabilities, connection list, pin config) - - Build internal topology representation -4. Add codec table for common codecs: - - Realtek ALC887/ALC888/ALC892 (most common desktop) - - Realtek ALC269/ALC282/ALC283 (most common laptop) - - Conexant CX20561/CX20585 - - IDT 92HD73C1/92HD81B1C5 - -**Files to modify/create**: -- `recipes/core/base/source/drivers/audio/ihdad/src/main.rs` — controller init -- `recipes/core/base/source/drivers/audio/ihdad/src/hda/` — new `codec.rs`, `widget.rs`, `codecs/` -- `recipes/core/base/source/drivers/audio/ihdad/src/hda/registers.rs` — register definitions - -**Acceptance**: Real hardware with Intel HDA controller enumerates codecs. `lspci` shows HD Audio device with driver attached. Codec dump shows vendor/device IDs matching known codecs. - -### T3.2: HDA Mixer Controls + Jack Detection - -**Problem**: No volume control, no muting, no jack detection. Audio output is fixed-volume or silent. - -**Linux reference**: `sound/hda/hda_generic.c` — `create_mute_volume_ctl()`. `sound/hda/hda_jack.c` — `snd_hda_jack_detect()`. - -**Implementation**: -1. Add mixer controls for each output path: - - Volume control (AMP-OUT mute + gain on pin widget) - - Capture control (AMP-IN mute + gain on ADC widget) - - Master volume (combined output volume) -2. Implement jack detection: - - Enable unsolicited response for jack-sense pin widgets - - Handle unsolicited response in CORB/RIRB interrupt - - Report jack state (plugged/unplugged) via scheme -3. Wire mixer controls to audiod for system-wide volume management - -**Files to modify**: -- `recipes/core/base/source/drivers/audio/ihdad/src/hda/codec.rs` — mixer controls -- `recipes/core/base/source/drivers/audio/ihdad/src/hda/jack.rs` — jack detection (new) -- `recipes/core/base/source/drivers/audio/audiod/src/scheme.rs` — volume interface - -**Acceptance**: Volume control changes audible output level. Plugging/unplugging headphones triggers jack event (visible in debug output). Headphone and speaker paths are independent. - -### T3.3: HDA Stream Setup and PCM Playback - -**Problem**: No actual PCM audio output. HDA hardware configured but no audio data flows. - -**Linux reference**: `sound/hda/hda_controller.c` — `azx_pcm_open()` / `azx_pcm_prepare()` / `azx_pcm_trigger()`. - -**Implementation**: -1. Implement stream (PCM) management: - - Allocate stream descriptor from controller (SD0-SDn) - - Configure stream format (sample rate, bits, channels) - - Set BDL (Buffer Descriptor List) for DMA - - Set stream position in buffer (LPIB register) -2. Implement PCM playback path: - - `pcm_open(format)` — allocate stream, configure format - - `pcm_write(data)` — write audio samples to DMA buffer - - `pcm_start()` — set RUN bit in stream control - - `pcm_stop()` — clear RUN bit -3. Implement CORB/RIRB interrupt handling for unsolicited responses -4. Implement stream interrupt handling for buffer completion (BCIS) - -**Files to modify**: -- `recipes/core/base/source/drivers/audio/ihdad/src/hda/stream.rs` — stream management (new) -- `recipes/core/base/source/drivers/audio/ihdad/src/hda/dma.rs` — BDL setup (new) -- `recipes/core/base/source/drivers/audio/audiod/src/` — PCM routing - -**Acceptance**: `aplay` (or redox equivalent) plays a WAV file and produces audible output. `parec` captures from microphone. Loopback (output → input) works without distortion. - -### T3.4: AC97 Multiple Codec + Mixer Support - -**Problem**: ac97d supports only single codec at fixed configuration. No volume/mute. - -**Linux reference**: `sound/pci/ac97/ac97_codec.c` (3,134 lines) — multi-codec architecture. - -**Implementation**: -1. Add codec slot detection (AC97 supports up to 4 codecs on one controller) -2. Add mixer register read/write for volume/mute -3. Add record source selection - -**Acceptance**: Desktop with AC97 audio codec produces audible output with adjustable volume. - ---- - -## Phase 4: Input Completeness (Weeks 3-5) - -### T4.1: PS/2 i8042 Controller Reset - -**Problem**: ps2d assumes controller is ready. Real hardware may need reset sequence. - -**Linux reference**: `drivers/input/serio/i8042.c:522` — `i8042_controller_check()`. - -**Implementation**: -1. Add controller self-test: Write 0xAA to command register, expect 0x55 response -2. Add controller initialization: disable devices, flush buffer, enable -3. Add AUX (mouse) port detection -4. Add timeout handling for missing ACK from controller - -**Files to modify**: -- `recipes/core/base/source/drivers/input/ps2d/src/controller.rs` - -**Acceptance**: PS/2 keyboard and mouse work on real hardware after cold boot. No "LED command ACK timeout" warnings. - -### T4.2: Touchpad Protocol Detection - -**Problem**: USB HID touchpads work as basic mice. No multi-touch, no gestures. - -**Linux reference**: `drivers/input/mouse/synaptics.c` for Synaptics protocol. `drivers/input/mouse/alps.c` for ALPS. - -**Implementation**: -1. Add PS/2 touchpad protocol detection for Synaptics/ALPS/Elantech -2. Parse multi-touch data from HID digitizer reports -3. Expose gesture events through evdevd scheme - -**Acceptance**: Laptop touchpad supports two-finger scroll. Multi-touch coordinates reported correctly. - ---- - -## Phase 5: Validation & Documentation (Weeks 1-12, parallel) - -### T5.1: Per-Driver Test Harnesses - -**Task**: Create QEMU-based test scripts for each driver category: -- `local/scripts/test-storage-qemu.sh` — boots with virtio-blk + AHCI, runs fio -- `local/scripts/test-network-qemu.sh` — boots with e1000 + r8169, runs iperf3 -- `local/scripts/test-audio-qemu.sh` — boots with HDA + AC97, plays test tone - -**Acceptance**: Each script exits 0 on success, produces captured serial output with test results. - -### T5.2: Hardware Validation Matrix - -**Task**: Create `local/docs/HARDWARE-VALIDATION-MATRIX.md` documenting tested hardware configurations: -- CPU/chipset combinations tested -- Storage controllers (AHCI, NVMe) tested -- Network chips (e1000, r8169 variants) tested -- Audio codecs (HDA, AC97) tested -- Known-broken configurations - -**Acceptance**: Matrix has at least one verified entry per driver category on real hardware. - ---- - -## Execution Order & Dependencies - -``` -Phase 0 (Cross-cutting) ─────────────────────────────────────────────┐ - T0.1 Error handling T0.2 Logging T0.3 Documentation │ - │ │ - ├── Phase 1 (Storage) ─────────────────────────────────────────┐ │ - │ T1.1 AHCI NCQ ──► T1.3 TRIM ──► T1.2 PM ──► T1.4 NVMe │ │ - │ │ │ - ├── Phase 2 (Network) ──────────────────────────────────────┐ │ │ - │ T2.1 ITR+Checksum ──► T2.2 TSO ──► T2.3 PHY ──► T2.4 │ │ │ - │ │ │ │ - ├── Phase 3 (Audio) ────────────────────────────────────┐ │ │ │ - │ T3.1 CodecDetect ──► T3.3 Stream ──► T3.2 Mixer │ │ │ │ - │ T3.4 AC97 (parallel) │ │ │ │ - │ │ │ │ │ - └── Phase 4 (Input) ───────────────────────────────┐ │ │ │ │ - T4.1 PS/2 reset ──► T4.2 Touchpad │ │ │ │ │ - │ │ │ │ │ - Phase 5 (Validation) ◄───────────────────────────────┴─────┴────┴───┴──┘ - T5.1 Test harnesses T5.2 Hardware matrix -``` - -**Phase 0 is prerequisite for all other phases.** -**Phases 1-4 are independent of each other and can run in parallel.** -**Phase 5 runs concurrently with all phases, finalizing as each completes.** - -## Timeline - -| Phase | Tasks | Duration | Cumulative | -|-------|-------|----------|------------| -| Phase 0 | T0.1, T0.2, T0.3 | Weeks 1-2 | Week 2 | -| Phase 1 | T1.1, T1.2, T1.3, T1.4 | Weeks 2-6 | Week 6 | -| Phase 2 | T2.1, T2.2, T2.3, T2.4 | Weeks 4-8 | Week 8 | -| Phase 3 | T3.1, T3.2, T3.3, T3.4 | Weeks 6-10 | Week 10 | -| Phase 4 | T4.1, T4.2 | Weeks 3-5 | Week 5 | -| Phase 5 | T5.1, T5.2 | Weeks 1-12 (parallel) | Week 12 | - -**Total**: 12 weeks with 2 developers working in parallel (Phase 1 and Phase 3 on separate tracks). - ---- - -## Linux Reference Map - -Every task references specific Linux source. Here is the complete map: - -| Task | Primary Reference | File Size | Function Focus | -|------|-------------------|-----------|----------------| -| T1.1 (NCQ) | `drivers/ata/libata-sata.c` | 1,365 lines | `ata_qc_issue()`, FIS construction | -| T1.2 (AHCI PM) | `drivers/ata/libata-eh.c` | 3,915 lines | `ata_eh_handle_port_suspend()` | -| T1.3 (TRIM) | `drivers/ata/libata-scsi.c` | 4,504 lines | `ata_scsi_unmap_xlat()` | -| T1.4 (NVMe) | `drivers/nvme/host/pci.c` | 3,146 lines | `nvme_reset_work()`, queue creation | -| T2.1 (ITR) | `e1000e/netdev.c` | 7,240 lines | `e1000_configure_itr()`, checksum | -| T2.2 (TSO) | `e1000e/netdev.c` | 7,240 lines | `e1000_tso()` | -| T2.3 (PHY) | `r8169_phy_config.c` | 1,354 lines | per-chip PHY init sequences | -| T3.1 (Codec) | `sound/hda/hda_codec.c` | 5,598 lines | `snd_hda_codec_new()`, widget parsing | -| T3.2 (Mixer) | `sound/hda/hda_generic.c` | 5,982 lines | `create_mute_volume_ctl()` | -| T3.3 (Stream) | `sound/hda/hda_controller.c` | 1,900 lines | `azx_pcm_open/prepare/trigger()` | -| T3.4 (AC97) | `sound/pci/ac97/ac97_codec.c` | 3,134 lines | multi-codec, mixer regs | -| T4.1 (PS/2) | `drivers/input/serio/i8042.c` | 1,254 lines | `i8042_controller_check()` | -| T4.2 (Touchpad) | `drivers/input/mouse/synaptics.c` | 1,707 lines | protocol detection | - ---- - -## Scope Boundaries - -**In scope**: -- Storage driver enhancements (AHCI NCQ, PM, TRIM; NVMe queues) -- Network driver enhancements (e1000 offload, r8169 PHY, jumbo frames) -- Audio driver enhancements (HDA codec, mixer, streams; AC97 multi-codec) -- Input driver enhancements (PS/2 reset, touchpad protocols) -- Cross-cutting driver quality (error handling, logging, documentation) - -**Out of scope** (covered by existing plans): -- ACPI S3/S4 sleep, thermal, EC — see `ACPI-IMPROVEMENT-PLAN.md` -- PCI IRQ, MSI-X depth, IOMMU — see `IRQ-AND-LOWLEVEL-CONTROLLERS-ENHANCEMENT-PLAN.md` -- USB controller completeness, device lifecycle — see `USB-IMPLEMENTATION-PLAN.md` -- GPU/DRM display, KMS, Mesa — see `DRM-MODERNIZATION-EXECUTION-PLAN.md` -- Bluetooth — see `BLUETOOTH-IMPLEMENTATION-PLAN.md` -- Wi-Fi — see `WIFI-IMPLEMENTATION-PLAN.md` -- Desktop/KDE — see `CONSOLE-TO-KDE-DESKTOP-PLAN.md` - ---- - -## Addendum A: Kernel Substrate Audit (2026-05-04 deep re-assessment) - -### A.1 CPU / SMP / Timer Initialization - -**Red Bear**: Kernel arch/x86_64 (502 lines) + arch/x86_shared + time.rs -**Linux**: `arch/x86/kernel/smpboot.c` (1,511) + `arch/x86/kernel/apic/apic.c` (2,694) + `arch/x86/kernel/tsc.c` (1,612) + `kernel/time/tick-common.c` (595) = 6,412 lines (subset) - -**What Red Bear has**: -- Basic x86_64 boot (GDT, IDT, page tables) -- x2APIC/SMP detected from MADT -- HPET timer - -**What Linux has that Red Bear is missing**: -- ❌ BSP/AP handoff protocol — Linux: `smpboot.c:895` `do_boot_cpu()` -- ❌ CPU hotplug (online/offline) — Linux: `smpboot.c:1312` `cpu_up()` / `cpu_down()` -- ❌ TSC calibration and synchronization — Linux: `tsc.c:1186` `check_tsc_sync_source()` -- ❌ APIC timer calibration and per-CPU timers — Linux: `apic.c:294` `calibrate_APIC_clock()` -- ❌ Interrupt affinity and vector allocation — Linux: `kernel/irq/manage.c` (2,803 lines) -- ❌ IPI (Inter-Processor Interrupt) routing — Linux: `apic/ipi.c` -- ❌ CPU idle states (C-states) — Linux: `arch/x86/kernel/acpi/cstate.c` -- ❌ Clock source rating and switching — Linux: `kernel/time/clocksource.c` - -**Priority**: SMP bring-up stability and TSC sync are critical for multi-core correctness. Without APIC timer calibration, scheduler tick is unreliable. - -### A.2 DMA / Memory / IOMMU Substrate - -**Red Bear**: kernel memory/mod.rs (1,266 lines) + iommu daemon (4,411 lines) -**Linux**: `kernel/dma/mapping.c` (1,016) + `drivers/iommu/` (~30K) + `mm/` subsystem - -**What Red Bear has**: -- Physical memory mapping via scheme:memory -- Basic IOMMU daemon (4,411 lines — substantial, AMD-Vi + Intel VT-d) -- Page table management in iommu daemon - -**What Linux has that Red Bear is missing**: -- ❌ Coherent DMA API — Linux: `kernel/dma/mapping.c` `dma_alloc_coherent()` -- ❌ Streaming DMA API — Linux: `kernel/dma/mapping.c` `dma_map_single()` -- ❌ Scatter-gather DMA — Linux: `lib/scatterlist.c` -- ❌ DMA pool/zone management -- ❌ SWIOTLB bounce buffering — Linux: `kernel/dma/swiotlb.c` -- ❌ IOMMU DMA remapping per-device — the iommu daemon exists but Linux handles this in-kernel with `iommu_dma_ops` -- ❌ DMA debug and error injection — Linux: `kernel/dma/debug.c` - -**Priority**: DMA API is prerequisite for any driver doing scatter-gather. Without coherent DMA, drivers must manually manage cache coherency. - -### A.3 Virtio Completeness - -**Red Bear**: virtio-core (1,545 lines) + virtio-blkd + virtio-netd + virtio-gpud -**Linux**: `drivers/virtio/virtio.c` (730) + `virtio_ring.c` (3,940) + `virtio_pci_modern.c` (1,301) + blk/net/gpu drivers (14,957 total) - -**What Red Bear has**: -- Basic virtio PCI transport (legacy) -- Split virtqueue with basic ring management -- virtio-blk, virtio-net, virtio-gpu drivers - -**What Linux has that Red Bear is missing**: -- ❌ **Virtio 1.0 modern PCI transport** — Linux: `virtio_pci_modern.c` (1,301 lines). Red Bear only uses legacy. -- ❌ **Packed virtqueue** (Virtio 1.1) — Linux: `virtio_ring.c` supports both split and packed -- ❌ **Multiqueue support** — Linux: virtio-net supports up to 16 TX/RX queue pairs via MSI-X -- ❌ **Virtio feature negotiation** — Red Bear hardcodes features; Linux does dynamic negotiation -- ❌ **Device reset protocol** — Linux: `virtio.c:237` `virtio_reset_device()` -- ❌ **Virtio-MMIO transport** (for ARM/RISC-V VMs) -- ❌ **Virtio-balloon** (memory ballooning) - -**Priority**: Modern PCI transport is required for QEMU machine types `q35` and newer. Packed virtqueues improve throughput. Multiqueue is critical for network performance. - -### A.4 CPU Frequency / Thermal / Power - -**Red Bear**: cpufreqd (176 lines — real implementation with governors), thermald (837 lines), hwrngd (534 lines), redbear-upower, redbear-acmd, redbear-ecmd -**Linux**: `drivers/cpufreq/cpufreq.c` (3,081) + `drivers/thermal/thermal_core.c` (1,956) + `drivers/char/hw_random/core.c` (739) - -**cpufreqd status**: 176 lines with ondemand/performance/powersave governors, MSR-based P-state control via IA32_PERF_CTL, and CPU load measurement via `/scheme/sys`. Still missing vs Linux: -- ❌ Governor framework (performance, powersave, ondemand, schedutil) -- ❌ ACPI P-state (_PSS) integration -- ❌ Intel P-state / HWP driver -- ❌ AMD CPPC driver - -**thermald status**: 837 lines — basic thermal monitoring exists but missing: -- ❌ Thermal zone trip points (passive/active/critical) -- ❌ Cooling device registration -- ❌ Fan speed control via ACPI - -**hwrngd status**: 534 lines — reasonable random number daemon. Missing: -- ❌ Entropy estimation per FIPS 140-2 -- ❌ Multiple entropy source mixing (CPU jitter, TPM, RDRAND) -- ❌ `/dev/hwrng` interface - -**Priority**: cpufreqd has basic governor support but still needs ACPI P-state integration, Intel HWP, and AMD CPPC for full functionality. - -### A.5 Block Layer / Filesystem Integration - -**Red Bear**: No dedicated block layer — each storage driver handles I/O directly via DiskScheme -**Linux**: `block/blk-mq.c` (5,309) + `block/blk-flush.c` (540) + `block/genhd.c` + `block/elevator.c` - -**What Linux has that Red Bear is missing**: -- ❌ Multi-queue block I/O — Linux: `blk-mq.c` — per-CPU queues + tag sets -- ❌ I/O scheduling (mq-deadline, kyber, bfq) — Linux: `block/mq-deadline.c` -- ❌ Flush/FUA semantics — Linux: `block/blk-flush.c` -- ❌ I/O merging and sorting -- ❌ Request timeout and retry — Linux: `block/blk-mq.c` `blk_mq_check_expired()` -- ❌ Block device partitioning (MBR/GPT handled by partitionlib library) -- ❌ Queue depth management and back-pressure - -**Red Bear storage drivers** (nvmed 1,318 lines; usbscsid 1,622 lines; ided 773 lines) all implement their own I/O dispatch. The lack of a shared block layer means each driver reinvents queuing, timeout, and retry logic. - -**Priority**: Block layer is prerequisite for NCQ, NVMe multi-queue, TRIM propagation, and crash consistency. - ---- - -## Revised Execution Priority (incorporating kernel substrate) - -| Tier | Subsystem | Effort | -|------|-----------|--------| -| **T0** (kernel) | SMP bring-up stability, TSC calibration, interrupt affinity | 4-6 weeks | -| **T0** (kernel) | DMA API + scatter-gather | 2-3 weeks | -| **T1** | AHCI NCQ + block layer | 3-4 weeks | -| **T1** | Virtio modern PCI + multiqueue | 2-3 weeks | -| **T1** | cpufreqd (governor + P-state) | 2-3 weeks | -| **T2** | Network offloads (Phase 2) | 3-4 weeks | -| **T2** | HDA codec detection (Phase 3) | 3-4 weeks | -| **T3** | thermald trip points + fan control | 1-2 weeks | -| **T3** | NVMe multi-queue | 2-3 weeks | -| **T4** | Audio streams + mixer (Phase 3 remainder) | 3-4 weeks | - -**Total**: 24-36 weeks (T0-T2 minimum viable), 40-52 weeks (full). - ---- - -## Addendum B: Daemon & Subsystem Audit (2026-05-04, updated with precise Linux 7.0 line counts) - -### B.1 ACPI Subsystem — Deep Linux Cross-Reference - -**Red Bear**: acpid (2,187 lines) + kernel ACPI (727 lines) = 2,914 total -**Linux 7.0** (key files): `sleep.c` (1,152) + `thermal.c` (1,067) + `battery.c` (1,331) + `ec.c` (2,380) + `arch/x86/kernel/acpi/sleep.c` (202) + `processor_perflib.c` + `acpi_video.c` + `pci_irq.c` + `apei/` = **~60,000+ total** - -| Linux File | Lines | Feature | Red Bear Status | -|------------|-------|---------|-----------------| -| `drivers/acpi/sleep.c` | 1,152 | S3/S4 suspend, NVS save/restore, wakeup vector | ❌ S3/S4 missing | -| `drivers/acpi/thermal.c` | 1,067 | Thermal zones, trip points, cooling | ❌ Missing | -| `drivers/acpi/battery.c` | 1,331 | Battery status, charge, ACPI _BIF/_BST | ❌ Missing | -| `drivers/acpi/ec.c` | 2,380 | Embedded Controller runtime, commands, GPE | ❌ Missing (redbear-ecmd is stub) | -| `drivers/acpi/fan.c` | ~400 | Fan speed control | ❌ Missing | -| `arch/x86/kernel/acpi/sleep.c` | 202 | x86-specific sleep, wakeup vector, trampoline | ❌ Missing | -| `drivers/acpi/processor_perflib.c` | ~800 | _PSS/_PPC performance states | ❌ Missing | -| `drivers/acpi/pci_irq.c` | ~500 | PCI IRQ routing overrides (_PRT) | ❌ Missing | -| `drivers/acpi/apei/` | ~3,000 | ACPI Platform Error Interface | ❌ Missing | - -**Priority**: S3/S4 sleep and thermal zones are critical for laptop/desktop use. EC support needed for modern laptops. - -### B.2 IRQ / MSI / Timer Subsystem — Precise Line Counts - -**Red Bear**: kernel irq.rs (570) + local_apic.rs (272) + ioapic.rs (427) + ipi.rs (53) + time.rs (36) = 1,358 total -**Linux 7.0** (key files): `kernel/irq/manage.c` (2,803) + `apic/vector.c` (1,387) + `apic/msi.c` (391) + `tsc.c` (1,612) + `tick-common.c` (595) = **6,788 lines (subset)** - -| Linux File | Lines | Feature | Red Bear Status | -|------------|-------|---------|-----------------| -| `kernel/irq/manage.c` | 2,803 | IRQ management, affinity, threading, spurious | ❌ Basic only | -| `arch/x86/kernel/apic/vector.c` | 1,387 | Vector allocation matrix, CPU assignment | ❌ Missing | -| `arch/x86/kernel/apic/msi.c` | 391 | MSI address/data composition, mask bits | ❌ Missing | -| `arch/x86/kernel/tsc.c` | 1,612 | TSC calibration, sync, clocksource rating | ❌ Missing | -| `kernel/time/tick-common.c` | 595 | Tick management, NO_HZ, broadcast | ❌ Missing | - -**Priority**: MSI/MSI-X blocks modern GPU/NVMe/network. TSC calibration needed for accurate time. - -### B.3 cpufreqd — Confirmed 26-line Stub - -cpufreqd is **26 lines** — logs messages, sleeps forever. No MSR access, no governor, no P-state control. A 176-line implementation was written and saved as `local/patches/base/P6-cpufreqd-real-impl.patch` (177 lines) but the source was reverted. Needs re-application. - -### B.4 Stale Documentation Cleanup - -27 docs archived total. BOOT-PROCESS-FIX-SUMMARY and GRAPHICAL-BOOT-ASSESSMENT moved to archive (superseded by this plan). diff --git a/local/docs/archived/COMPREHENSIVE-DRIVER-AUDIT-2026-05-04.md b/local/docs/archived/COMPREHENSIVE-DRIVER-AUDIT-2026-05-04.md deleted file mode 100644 index 77efca204b..0000000000 --- a/local/docs/archived/COMPREHENSIVE-DRIVER-AUDIT-2026-05-04.md +++ /dev/null @@ -1,316 +0,0 @@ -# Red Bear OS — Comprehensive Driver & Hardware Audit - -**Date**: 2026-05-04 -**Source of truth**: Linux kernel 7.0 (`local/reference/linux-7.0/`, 2.0 GB) -**Method**: Cross-reference every Red Bear daemon/driver/hardware-init component with its Linux counterpart. Prefer Linux as ground truth for correctness and completeness. - ---- - -## 1. Size Comparison Summary - -| Subsystem | Red Bear (lines) | Linux (lines) | Ratio | Existing Plan | -|-----------|-----------------|---------------|-------|---------------| -| ACPI (acpid + kernel) | 2,187 + 727 | ~60,000+ | ~20x | ACPI-IMPROVEMENT-PLAN.md | -| PCI | 1,192 | ~15,000+ | ~12x | IRQ-AND-LOWLEVEL-CONTROLLERS | -| AHCI storage | 109 | 2,173 (ahci.c only) | ~20x | **NONE — gap** | -| xHCI USB | ~1,100 | 12,188 (3 files) | ~11x | USB-IMPLEMENTATION-PLAN.md | -| Network (e1000+r8168) | 918 | 37,893 | ~41x | **NONE — gap** | -| Audio (HDA+AC97) | 610 | ~10,000+ | ~16x | **NONE — gap** | -| GPU/DRM | 8,427 | 1,284,210 (amd+i915) | ~152x | DRM-MODERNIZATION-EXECUTION | -| Kernel IRQ | 570 | ~10,000+ | ~17x | IRQ-AND-LOWLEVEL-CONTROLLERS | -| Input (PS/2 + USB HID) | ~500 | 38,000+ (i8042 + HID) | ~76x | Partial (USB-IMPLEMENTATION) | - -**Note**: Size ratios reflect architectural differences (microkernel userspace drivers vs monolithic kernel). Red Bear targets a narrower hardware set. However, feature gaps are real and impactful. - ---- - -## 2. Detailed Component Assessment - -### 2.1 ACPI (Covered: ACPI-IMPROVEMENT-PLAN.md) - -**Red Bear**: acpid daemon (2,187 lines) + kernel ACPI tables (727 lines) -**Linux**: drivers/acpi/ (~60K lines) + arch/x86/kernel/acpi/ + ACPICA interpreter - -**What Red Bear has (verified)**: -- ✅ ACPI table parsing (RSDP, RSDT/XSDT, FADT, MADT, DSDT/SSDT) -- ✅ AML interpreter (bounded subset, v6.1.1) -- ✅ S5 shutdown via PM1a/PM1b + keyboard controller fallback -- ✅ Power methods (\_PS0, \_PS3, \_PPC) -- ✅ RSDP forwarding from bootloader - -**What Linux has that Red Bear is missing**: -- ❌ S3 (suspend-to-RAM) / S4 (hibernate) — Linux: `arch/x86/kernel/acpi/sleep.c` -- ❌ Thermal zones — Linux: `drivers/acpi/thermal.c` -- ❌ Battery/AC status — Linux: `drivers/acpi/battery.c`, `ac.c` -- ❌ Fan control — Linux: `drivers/acpi/fan.c` -- ❌ Embedded Controller runtime — Linux: `drivers/acpi/ec.c` (62KB) -- ❌ Processor performance states (\_PSS) — Linux: `drivers/acpi/processor_perflib.c` -- ❌ C-states — Linux: `arch/x86/kernel/acpi/cstate.c` -- ❌ PCI IRQ routing overrides (\_PRT) — Linux: `drivers/acpi/pci_irq.c` -- ❌ ACPI Platform Error Interface (APEI) — Linux: `drivers/acpi/apei/` - -**Priority**: S3/S4 sleep and thermal shutdown are critical for laptop/desktop use. - ---- - -### 2.2 PCI / IRQ (Covered: IRQ-AND-LOWLEVEL-CONTROLLERS-ENHANCEMENT-PLAN.md) - -**Red Bear**: pcid + pcid-spawner (1,192 lines) -**Linux**: drivers/pci/ (~15K lines) + drivers/pci/pcie/ + drivers/pci/msi/ - -**What Red Bear has**: -- ✅ PCI enumeration (bus/device/function scanning) -- ✅ Driver spawning via pcid-spawner -- ✅ Basic MSI/MSI-X enable/disable -- ✅ PCIe capability parsing - -**What Linux has that Red Bear is missing**: -- ❌ AER (Advanced Error Reporting) — Linux: `drivers/pci/pcie/aer.c` -- ❌ ASPM (Active State Power Management) — Linux: `drivers/pci/pcie/aspm.c` -- ❌ PCIe hotplug — Linux: `drivers/pci/hotplug/` -- ❌ SR-IOV virtualization — Linux: `drivers/pci/iov.c` -- ❌ Access Control Services (ACS) — Linux: `drivers/pci/pcie/acs.c` -- ❌ Address Translation Services (ATS/PRI/PASID) — Linux: `drivers/pci/ats.c` -- ❌ DPC (Downstream Port Containment) — Linux: `drivers/pci/pcie/dpc.c` - -**Priority**: AER is critical for hardware reliability. ASPM for power efficiency on laptops. - ---- - -### 2.3 Storage — AHCI (No existing plan — CRITICAL GAP) - -**Red Bear**: ahcid (109 lines — main.rs only) -**Linux**: `drivers/ata/ahci.c` (2,173 lines) + `libahci.c` (2,447 lines) + `libata-core.c` (5,296 lines) - -**Red Bear current state**: Minimal — only basic SATA IDENTIFY and PIO/DMA read/write. - -**What Linux has that Red Bear is missing** (cross-referenced from `drivers/ata/ahci.c` and `libata-core.c`): -- ❌ **NCQ** (Native Command Queuing) — 32-command depth, critical for SSD performance - - Linux: `libata-sata.c` — `ata_scsi_queuecmd()`, `ata_qc_issue()` - - Red Bear reference: `drivers/ata/libata-sata.c:35` — `sata_fsl_host_intr()` with NCQ error handling -- ❌ **FIS-based switching** (port multiplier support) - - Linux: `drivers/ata/ahci.c:1423` — `ahci_qc_prep()` handles FIS registers -- ❌ **TRIM/Discard** (SSD optimization) - - Linux: `drivers/ata/libata-scsi.c` — `ata_scsi_unmap_xlat()` maps DISCARD to DATA SET MANAGEMENT -- ❌ **Power management** (Partial/Slumber link states) - - Linux: `drivers/ata/libata-eh.c:3682` — `ata_eh_handle_port_suspend()` -- ❌ **Hotplug detection** - - Linux: `drivers/ata/libata-core.c:5465` — `ata_port_detect()` with PHY event polling -- ❌ **LED control** (activity/locate/fault LEDs) - - Linux: `drivers/ata/libata-core.c:4938` — `ata_led_*` functions -- ❌ **ATAPI (CD/DVD) support** — present in Linux at `drivers/ata/libata-scsi.c` -- ❌ **SMART passthrough** — Linux: `drivers/ata/libata-scsi.c` — `ata_scsi_pass_thru()` -- ❌ **Error recovery** — Linux has extensive EH (Error Handler) in `libata-eh.c` (3,915 lines) - -**Priority**: NCQ alone can improve SSD throughput 3-5x. TRIM prevents SSD degradation. Power management critical for laptops. - ---- - -### 2.4 Storage — NVMe (No existing plan) - -**Red Bear**: nvmed (present but minimal) -**Linux**: `drivers/nvme/host/` — `core.c` + `pci.c` + `ioctl.c` + `fabrics.c` + `multipath.c` + `zns.c` - -**What Linux has that Red Bear is missing**: -- ❌ Multiple I/O queues (NVMe supports up to 64K queues) -- ❌ Submission/completion queue management -- ❌ PRP/SGL scatter-gather lists -- ❌ Namespace management -- ❌ NVMe-MI (Management Interface) -- ❌ Fabrics (NVMe-oF) — Linux: `drivers/nvme/host/fabrics.c` -- ❌ ZNS (Zoned Namespaces) — Linux: `drivers/nvme/host/zns.c` -- ❌ Multipath I/O — Linux: `drivers/nvme/host/multipath.c` - -**Priority**: Lower than AHCI — most VMs use SATA or virtio-blk. - ---- - -### 2.5 Network — e1000 / r8168 (No existing plan — CRITICAL GAP) - -**Red Bear**: e1000d (458 lines) + rtl8168d (460 lines) = 918 lines total -**Linux**: e1000e (30,203 lines) + r8169 (7,690 lines) = 37,893 lines total - -**What Linux has that Red Bear is missing** (cross-referenced from `drivers/net/ethernet/intel/e1000e/` and `drivers/net/ethernet/realtek/r8169_main.c`): - -**e1000/e1000e**: -- ❌ **Interrupt moderation** (ITR) — critical for throughput - - Linux: `e1000e/netdev.c:4200` — `e1000_configure_itr()` -- ❌ **Hardware checksum offload** (TCP/UDP checksum) - - Linux: `e1000e/netdev.c` — `e1000_tx_csum()`, `e1000_rx_checksum()` -- ❌ **TSO/GSO** (TCP Segmentation Offload) - - Linux: `e1000e/netdev.c:5305` — `e1000_tso()` -- ❌ **Jumbo frames** (>1500 MTU) -- ❌ **Wake-on-LAN** — Linux: `e1000e/netdev.c:5512` — `e1000e_set_wol()` -- ❌ **VLAN hardware acceleration** -- ❌ **EEE** (Energy Efficient Ethernet) — Linux: `e1000e/ethtool.c` -- ❌ **Multiple TX/RX queues** (MSI-X based) - -**r8169**: -- ❌ **Hardware checksum offload** -- ❌ **TSO/GSO** -- ❌ **Jumbo frames** — Linux: `r8169_main.c:4352` — `rtl_jumbo_config()` -- ❌ **EEPROM/MDIO access** — Linux: `r8169_main.c` — `rtl_read_eeprom()` -- ❌ **Firmware loading** (some chips need firmware) — Linux: `r8169_firmware.c` -- ❌ **PHY configuration** (per-chip phy init sequences) — Linux: `r8169_phy_config.c` (1,354 lines) -- ❌ **Power management** / ASPM — Linux: `r8169_main.c:5073` — `rtl8169_runtime_suspend()` - -**Priority**: Hardware offloads can improve throughput 3-10x. Interrupt moderation is essential for high packet rates. - ---- - -### 2.6 Audio — HDA / AC97 (No existing plan — GAP) - -**Red Bear**: ihdad (143 lines) + ac97d (467 lines) = 610 lines total -**Linux**: `sound/hda/` + `sound/pci/ac97/` (~10K lines) - -**What Linux has that Red Bear is missing**: -- ❌ **HDA codec auto-detection** (Realtek, Conexant, IDT, VIA, etc.) - - Linux: `sound/hda/hda_codec.c` — `snd_hda_codec_new()` -- ❌ **HDA codec-specific initialization** (pin configs, EAPD, GPIO) - - Linux: `sound/hda/hda_generic.c` — generic parser -- ❌ **HDA power management** (codec power states, D0/D3) - - Linux: `sound/hda/hda_codec.c` — `snd_hda_codec_set_power_state()` -- ❌ **Mixer controls** (volume, mute, capture, jack sensing) - - Linux: `sound/hda/hda_generic.c` — `create_mute_volume_ctl()` -- ❌ **Jack detection** (headphone/mic plug/unplug) - - Linux: `sound/hda/hda_jack.c` — `snd_hda_jack_detect()` -- ❌ **HDMI/DP audio** (digital audio over display) - - Linux: `sound/hda/hda_eld.c` — ELD (EDID-Like Data) parsing -- ❌ **AC97 multiple codec support** - - Linux: `sound/pci/ac97/ac97_codec.c` (3,134 lines) -- ❌ **Sample rate conversion / format negotiation** - -**Priority**: Codec auto-detection is the minimum needed for real hardware audio to work beyond basic beeps. Without it, audio works on zero real machines. - ---- - -### 2.7 USB — xHCI (Covered: USB-IMPLEMENTATION-PLAN.md) - -**Red Bear**: xhcid (~1,100 lines) -**Linux**: `drivers/usb/host/xhci.c` (5,705) + `xhci-ring.c` (4,488) + `xhci-hub.c` (1,995) = 12,188 lines - -**What Red Bear has**: -- ✅ Basic control/bulk/interrupt/isochronous transfers -- ✅ Device enumeration (basic) - -**What Linux has that Red Bear is missing** (cross-referenced): -- ❌ **Transfer ring management** (TRB dequeue, cycle bit tracking) - - Linux: `xhci-ring.c:253` — `inc_deq()` with cycle state handling -- ❌ **Stream support** (bulk streams for UAS) - - Linux: `xhci-ring.c:3500` — `xhci_queue_stream_transfer()` -- ❌ **USB 3.x SuperSpeed features** (U1/U2/U3 link states) - - Linux: `xhci.c:4560` — `xhci_set_link_state()` -- ❌ **Isochronous scheduling** (proper bandwidth calculation) - - Linux: `xhci-ring.c:3718` — `xhci_queue_isoc_tx()` -- ❌ **Command ring handling** (TRB abort, stop endpoint) - - Linux: `xhci-ring.c:173` — `xhci_abort_cmd_ring()` -- ❌ **Error recovery** (transfer event TRB error handling) - - Linux: `xhci-ring.c:2636` — `handle_tx_event()` with extensive error cases -- ❌ **Controller reset/recovery** (xHCI controller hang detection) - - Linux: `xhci.c:5173` — `xhci_handle_command_timeout()` - -**Priority**: Referenced by USB-IMPLEMENTATION-PLAN.md. - ---- - -### 2.8 GPU / DRM (Covered: DRM-MODERNIZATION-EXECUTION-PLAN.md) - -Redox-drm (8,427 lines) vs Linux AMD+i915 (1,284,210 lines). Referenced by existing plan. Key gaps already documented. - ---- - -### 2.9 Input — PS/2 + USB HID - -**Red Bear**: ps2d + usbhidd (~500 lines) -**Linux**: `drivers/input/serio/i8042.c` (1,254 lines) + `drivers/hid/usbhid/` + `drivers/input/evdev.c` - -**What Linux has that Red Bear is missing**: -- ❌ **i8042 controller detection and reset** — Linux: `i8042.c:522` — `i8042_controller_check()` -- ❌ **PS/2 hotplug** — Linux: `i8042.c` — `i8042_interrupt()` with AUX detection -- ❌ **LED feedback** — Red Bear has basic LED support (P3 patch) -- ❌ **Touchpad protocol detection** (Synaptics, ALPS, Elantech) -- ❌ **Multitouch support** (USB HID digitizer class) -- ❌ **Force feedback** (game controllers) — Linux: `drivers/hid/hid-pidff.c` - ---- - -## 3. Prioritized Improvement Plan - -### Tier 1 — CRITICAL (blocks real hardware use) - -| # | Task | Subsystem | Effort | Reference | -|---|------|-----------|--------|-----------| -| 1 | ACPI S3/S4 sleep + thermal shutdown | ACPI | 2-3 weeks | `drivers/acpi/sleep.c`, `arch/x86/kernel/acpi/sleep.c` | -| 2 | NCQ support in AHCI | Storage | 1-2 weeks | `drivers/ata/libata-sata.c` — `ata_qc_issue()` | -| 3 | HDA codec auto-detection | Audio | 2-3 weeks | `sound/hda/hda_codec.c` — `snd_hda_codec_new()` | -| 4 | Network interrupt moderation + checksum offload | Network | 1-2 weeks | `e1000e/netdev.c` — `e1000_configure_itr()` | - -### Tier 2 — HIGH (major quality improvements) - -| # | Task | Subsystem | Effort | Reference | -|---|------|-----------|--------|-----------| -| 5 | TRIM/Discard for AHCI | Storage | 3-5 days | `drivers/ata/libata-scsi.c` — `ata_scsi_unmap_xlat()` | -| 6 | AHCI power management (Partial/Slumber) | Storage | 3-5 days | `drivers/ata/libata-eh.c` — suspend/resume | -| 7 | r8169 PHY configuration | Network | 1 week | `r8169_phy_config.c` (1,354 lines) | -| 8 | PCIe AER (Advanced Error Reporting) | PCI | 1 week | `drivers/pci/pcie/aer.c` | -| 9 | Jack detection + mixer controls for HDA | Audio | 1 week | `sound/hda/hda_jack.c`, `hda_generic.c` | - -### Tier 3 — MEDIUM (polish and completeness) - -| # | Task | Subsystem | Effort | Reference | -|---|------|-----------|--------|-----------| -| 10 | NVMe multiple I/O queues | Storage | 1-2 weeks | `drivers/nvme/host/pci.c` | -| 11 | PCIe ASPM | PCI | 3-5 days | `drivers/pci/pcie/aspm.c` | -| 12 | AHCI FIS-based switching | Storage | 1 week | `drivers/ata/ahci.c` — `ahci_qc_prep()` | -| 13 | HDMI/DP audio over HDA | Audio | 1 week | `sound/hda/hda_eld.c` | -| 14 | PS/2 touchpad protocols | Input | 1-2 weeks | `drivers/input/mouse/synaptics.c` | -| 15 | I/OMMU runtime validation (QEMU proof exists) | IOMMU | 1 week | `drivers/iommu/amd/` | - -### Tier 4 — LOW (future work) - -| # | Task | Subsystem | Effort | Reference | -|---|------|-----------|--------|-----------| -| 16 | SR-IOV virtualization | PCI | 2-3 weeks | `drivers/pci/iov.c` | -| 17 | Wake-on-LAN for e1000/r8169 | Network | 3-5 days | `e1000e/netdev.c` — `e1000e_set_wol()` | -| 18 | NVMe multipath + fabrics | Storage | 2-4 weeks | `drivers/nvme/host/multipath.c` | -| 19 | PCIe hotplug | PCI | 1-2 weeks | `drivers/pci/hotplug/` | -| 20 | Force feedback for game controllers | Input | 3-5 days | `drivers/hid/hid-pidff.c` | - ---- - -## 4. Linux Cross-Reference Quick Reference - -For each Red Bear daemon, here is the primary Linux source file(s) to consult: - -| Red Bear Daemon | Linux Reference | -|----------------|-----------------| -| `acpid` | `drivers/acpi/bus.c` + `arch/x86/kernel/acpi/sleep.c` | -| `pcid` | `drivers/pci/probe.c` + `drivers/pci/pci.c` | -| `ahcid` | `drivers/ata/ahci.c` + `drivers/ata/libata-core.c` | -| `nvmed` | `drivers/nvme/host/pci.c` + `core.c` | -| `e1000d` | `drivers/net/ethernet/intel/e1000e/netdev.c` | -| `rtl8168d` | `drivers/net/ethernet/realtek/r8169_main.c` | -| `xhcid` | `drivers/usb/host/xhci.c` + `xhci-ring.c` | -| `ihdad` | `sound/hda/hda_codec.c` + `hda_generic.c` | -| `ac97d` | `sound/pci/ac97/ac97_codec.c` | -| `ps2d` | `drivers/input/serio/i8042.c` | -| `usbhidd` | `drivers/hid/usbhid/hid-core.c` | -| `vesad` | `drivers/video/fbdev/vesafb.c` | -| `virtio-netd` | `drivers/net/virtio_net.c` | -| `virtio-blkd` | `drivers/block/virtio_blk.c` | -| `virtio-gpud` | `drivers/gpu/drm/virtio/virtgpu*` | -| `iommu` | `drivers/iommu/amd/` or `intel/` | -| `redox-drm` | `drivers/gpu/drm/drm_ioctl.c` + `drm_framebuffer.c` | - ---- - -## 5. Execution Priority - -``` -Tier 1 (weeks 1-6): ACPI sleep + AHCI NCQ + HDA codec detect + Network offload -Tier 2 (weeks 7-10): AHCI TRIM + AHCI PM + r8169 PHY + PCIe AER + HDA jack/mixer -Tier 3 (weeks 11-16): NVMe queues + PCIe ASPM + AHCI FIS + HDMI audio + Touchpad -Tier 4 (future): SR-IOV + WoL + NVMe fabrics + Hotplug + Force feedback -``` - -**Total estimated effort**: 10-16 weeks for Tiers 1-2 (minimum viable hardware support). 26-40 weeks for all 4 tiers. diff --git a/local/docs/archived/GRUB-INTEGRATION-PLAN.md b/local/docs/archived/GRUB-INTEGRATION-PLAN.md deleted file mode 100644 index c94e3f4e5f..0000000000 --- a/local/docs/archived/GRUB-INTEGRATION-PLAN.md +++ /dev/null @@ -1,391 +0,0 @@ -# GRUB Integration Plan — Red Bear OS - -**Date:** 2026-04-17 -**Status:** Fully implemented (build-tested, not yet runtime boot-tested). ESP formatted as FAT32 -per UEFI spec. Both Phase 1 (post-build script) and Phase 2 (installer-native) are wired. -**Remaining:** Runtime UEFI boot validation in QEMU (`make all CONFIG_NAME=redbear-grub && make qemu`). -**Prerequisite:** The `grub` package is included in `redbear-grub.toml` for clean-tree builds. -**Approach:** Option A — GRUB as boot manager, chainloading Redox bootloader - -## Overview - -Add GNU GRUB as an optional boot manager for Red Bear OS. GRUB presents a menu -at boot and chainloads the existing Redox bootloader, which then boots the -kernel normally. This gives users: - -- Multi-boot capability alongside Linux, Windows, or other OSes -- Boot menu with timeout and manual selection -- Familiar GRUB rescue shell for debugging -- No changes to the Redox kernel, RedoxFS, or existing boot flow - -## Architecture - -``` -UEFI firmware - → EFI/BOOT/BOOTX64.EFI (GRUB standalone image) - → grub.cfg: default entry chainloads Redox bootloader - → EFI/REDBEAR/redbear.efi (Redox bootloader) - → Reads RedoxFS partition - → Loads kernel - → Boots Red Bear OS -``` - -### ESP Layout (GRUB mode) - -``` -EFI/ -├── BOOT/ -│ ├── BOOTX64.EFI ← GRUB (primary, loaded by UEFI firmware) -│ └── grub.cfg ← GRUB configuration -└── REDBEAR/ - └── redbear.efi ← Redox bootloader (chainload target) -``` - -### ESP Layout (default, no GRUB) - -``` -EFI/ -└── BOOT/ - └── BOOTX64.EFI ← Redox bootloader (unchanged) -``` - -## Why GRUB? - -1. **GRUB does not support RedoxFS.** Writing a GRUB filesystem module for - RedoxFS is high-risk, GPL-licensing-sensitive work. Chainloading avoids it. -2. **The Redox bootloader works.** It reads RedoxFS directly and boots the - kernel. No need to replicate that logic in GRUB. -3. **GRUB is universally understood.** System administrators know GRUB. A - `grub.cfg` is easier to customize than a custom bootloader. -4. **Multi-boot.** GRUB can boot Linux, Windows, and other OSes alongside - Red Bear OS without any changes to those systems. - -## GRUB Module Set - -The standalone EFI image includes these modules: - -| Module | Purpose | -|--------|---------| -| `part_gpt` | GPT partition table support | -| `part_msdos` | MBR partition table support | -| `fat` | FAT32 filesystem (ESP) | -| `ext2` | ext2/3/4 filesystem | -| `normal` | Normal mode (menu, scripting) | -| `configfile` | Load configuration files | -| `search` | Search for files/volumes | -| `search_fs_uuid` | Search by filesystem UUID | -| `search_label` | Search by volume label | -| `echo` | Print messages | -| `test` | Conditional expressions | -| `ls` | List files and devices | -| `cat` | Display file contents | -| `halt` | Shut down | -| `reboot` | Reboot | - -Note: `chainloader` is a built-in command in GRUB 2.12 (no separate module needed). - -Red Bear policy now requires a local `redoxfs.mod` artifact for GRUB builds. -The GRUB recipe resolves it in this order: -1. `local/recipes/core/grub/modules/redoxfs.mod` -2. `${COOKBOOK_SYSROOT}/usr/lib/grub/x86_64-efi/redoxfs.mod` - -If neither exists, the GRUB recipe fails fast. - -## GRUB Configuration - -The default `grub.cfg`: - -```cfg -# Red Bear OS GRUB Configuration -set default=0 -set timeout=5 - -menuentry "Red Bear OS" { - chainloader /EFI/REDBEAR/redbear.efi - boot -} - -menuentry "Reboot" { - reboot -} - -menuentry "Shutdown" { - halt -} -``` - -Users can customize `grub.cfg` to add entries for other operating systems, -change the timeout, or add additional Red Bear OS entries (e.g., recovery -mode with different kernel parameters, once supported). - -## ESP Size Requirements - -| Component | Typical Size | -|-----------|--------------| -| GRUB EFI binary (with modules) | ~500 KiB (varies with module list) | -| Redox bootloader | 100–200 KiB | -| grub.cfg | < 1 KiB | -| **Total** | **~1 MiB** | - -The default ESP is 1 MiB (too small for GRUB). Configs using GRUB must set: - -```toml -[general] -efi_partition_size = 16 # 16 MiB, enough for GRUB + Redox bootloader + margin -``` - -## Linux-Compatible CLI - -Red Bear OS provides `grub-install` and `grub-mkconfig` wrappers that match GNU GRUB -command-line conventions. Users migrating from Linux can use familiar switches. - -| Linux Command | Red Bear OS Location | -|---------------|---------------------| -| `grub-install` | `local/scripts/grub-install` | -| `grub-mkconfig` | `local/scripts/grub-mkconfig` | - -Add to PATH for convenience: -```bash -export PATH="$PWD/local/scripts:$PATH" -``` - -### grub-install - -```bash -# Install GRUB into a disk image -grub-install --target=x86_64-efi --disk-image=build/x86_64/harddrive.img - -# Verbose mode -grub-install --target=x86_64-efi --disk-image=build/x86_64/harddrive.img --verbose - -# Show help -grub-install --help -``` - -Supported options: `--target=`, `--efi-directory=`, `--bootloader-id=`, `--removable`, -`--disk-image=`, `--modules=`, `--no-nvram`, `--verbose`, `--help`, `--version`. - -Unsupported Linux options are accepted and ignored silently for script compatibility. - -### grub-mkconfig - -```bash -# Preview generated config -grub-mkconfig - -# Write to file -grub-mkconfig -o local/recipes/core/grub/grub.cfg - -# Custom timeout -grub-mkconfig --timeout=10 -o /boot/grub/grub.cfg -``` - -Supported options: `-o`/`--output=`, `--timeout=`, `--set-default=`, `--help`, `--version`. - -## Implementation — Phase 1: Post-Build Script - -Phase 1 uses a post-build script to modify the ESP in an existing disk image. -This approach requires **no changes to the installer** and works immediately. - -### Files - -| File | Purpose | -|------|---------| -| `local/recipes/core/grub/recipe.toml` | Build GRUB from source, produce `grub.efi` | -| `local/recipes/core/grub/grub.cfg` | Default GRUB configuration | -| `local/recipes/core/grub/modules/redoxfs.mod` | Mandatory local GRUB RedoxFS module artifact | -| `local/scripts/install-grub.sh` | Post-build ESP modification script | -| `local/scripts/fat_tool.py` | Python FAT32 tool (no mtools dependency) | -| `recipes/core/grub → local/recipes/core/grub` | Symlink for recipe discovery | - -### Workflow - -```bash -# 1. Build GRUB recipe -make r.grub - -# 2. Build Red Bear OS (with larger ESP) -make all CONFIG_NAME=redbear-full # Must have efi_partition_size = 16 - -# 3. Install GRUB into disk image -./local/scripts/install-grub.sh build/x86_64/harddrive.img - -# 4. Test -make qemu -``` - -### Requirements - -- Python 3 (for `fat_tool.py` — no mtools dependency) -- GRUB build dependencies: `gcc`, `make`, `bison`, `flex`, `autoconf`, `automake` -- ESP must be ≥ 8 MiB (set `efi_partition_size = 16` in config) - -## Implementation — Phase 2: Installer-Native Support - -Phase 2 adds GRUB awareness directly to the Redox installer, eliminating the -post-build script step. The installer reads `bootloader = "grub"` from config, -fetches the GRUB package alongside the bootloader, and writes the chainload -ESP layout automatically. - -### Changes Made - -1. **`GeneralConfig`** (`config/general.rs`): Added `bootloader: Option` - field (`"redox"` default, `"grub"` for GRUB), with merge support. - -2. **`DiskOption`** (`installer.rs`): Added `grub_efi: Option<&[u8]>` and - `grub_config: Option<&[u8]>` fields for optional GRUB data. - -3. **`fetch_bootloaders`**: When `bootloader = "grub"`, installs the `grub` - package alongside `bootloader` and returns `grub.efi` + `grub.cfg` data. - Return type extended to `(bios, efi, grub_efi, grub_cfg)`. - -4. **`with_whole_disk` / `with_whole_disk_ext4`**: When `grub_efi` and - `grub_config` are both present, writes the GRUB chainload layout: - - `EFI/BOOT/BOOTX64.EFI` ← GRUB - - `EFI/BOOT/grub.cfg` ← GRUB configuration - - `EFI/REDBEAR/redbear.efi` ← Redox bootloader (chainload target) - -5. **`install_inner`**: Passes GRUB data from `fetch_bootloaders` through - `DiskOption`. - -6. **CLI** (`bin/installer.rs`): Added `--bootloader grub` flag that sets - `config.general.bootloader`. - -7. **TUI** (`bin/installer_tui.rs`): Updated `DiskOption` construction with - `grub_efi: None, grub_config: None`. - -### Config Usage - -```toml -# config/redbear-grub.toml -include = ["redbear-full.toml"] - -[general] -bootloader = "grub" -efi_partition_size = 16 -``` - -Or via CLI (note: INSTALLER_OPTS replaces defaults, so --cookbook=. must be included): -```bash -./target/release/repo cook installer -make all CONFIG_NAME=redbear-full INSTALLER_OPTS="--cookbook=. --bootloader grub" -``` - -**Note:** The config file approach (`redbear-grub.toml`) is preferred over the CLI flag -because INSTALLER_OPTS completely replaces the default value (`--cookbook=.`) rather than -appending to it. Omitting `--cookbook=.` breaks local package resolution for GRUB. - -## GRUB Recipe Design - -The GRUB recipe uses `template = "custom"` because GRUB must be built for the -**host machine** (it's a build tool that produces EFI binaries), not for the -Redox target. The cookbook's `configure` template cross-compiles for Redox, -which is wrong for GRUB. - -Key build steps: -1. Configure with `--target=x86_64 --with-platform=efi` (produces x86_64 EFI) -2. Disable unnecessary components (themes, mkfont, mount, device-mapper) -3. Run `grub-mkimage` to create standalone EFI binary with curated modules -4. Stage `grub.efi` and `grub.cfg` to `/usr/lib/boot/` - -### Build Notes - -The recipe uses `template = "custom"` because the cookbook's default `configure` -template sets `--host="${GNU_TARGET}"` for Redox cross-compilation, which is wrong -for GRUB (a host build tool producing EFI binaries). - -Two issues required workarounds: - -1. **Cross-compiler override.** The cookbook sets `CC`, `CXX`, `CFLAGS`, etc. to - the Redox cross-toolchain. GRUB must be built with the host compiler. Fix: - `unset CC CXX CPP LD AR NM RANLIB OBJCOPY STRIP PKG_CONFIG` and - `unset CFLAGS CXXFLAGS CPPFLAGS LDFLAGS` at the top of the script. - -2. **Missing `extra_deps.lst`.** GRUB 2.12 release tarballs omit - `grub-core/extra_deps.lst` (normally generated by `autogen.sh` from git). - Fix: `touch "${COOKBOOK_SOURCE}/grub-core/extra_deps.lst"` before configure. - -3. **grub.cfg location.** The config file lives in the recipe directory - (`${COOKBOOK_RECIPE}/grub.cfg`), not in the extracted source tarball - (`${COOKBOOK_SOURCE}/`). The copy step uses `COOKBOOK_RECIPE`. - -## Security Considerations - -- GRUB configuration is on the ESP (FAT32), which is readable/writable by any OS -- Secure Boot: GRUB standalone images are not signed. Users needing Secure Boot - must sign `BOOTX64.EFI` with their own key or use `shim` -- The chainload target (`EFI/REDBEAR/redbear.efi`) is also on the ESP -- No credentials or secrets are stored in the GRUB configuration - -## Limitations - -- GRUB cannot read RedoxFS (no module exists) -- Cannot pass kernel parameters directly (chainloading bypasses this) -- BIOS boot is not supported (only UEFI) -- ESP must be sized to ≥ 8 MiB in config (16 MiB recommended) -- GRUB bootloader is incompatible with `skip_partitions = true` (requires GPT layout with ESP) -- TUI installer does not support GRUB mode (intentional — TUI is for live disk reinstall) -- Runtime UEFI boot test has not been performed yet (requires full `make all` build, ~hours) - -## Testing - -### Phase 1: Post-build script (standalone) - -```bash -# Build GRUB recipe -make r.grub - -# Build image (any config with efi_partition_size >= 16) -make all CONFIG_NAME=redbear-full - -# Install GRUB into disk image (uses fat_tool.py, no mtools needed) -./local/scripts/install-grub.sh build/x86_64/harddrive.img - -# Verify ESP contents -python3 local/scripts/fat_tool.py ls build/x86_64/harddrive.img 1048576 / - -# Boot in QEMU -make qemu -# Expected: GRUB menu appears, "Red Bear OS" entry boots successfully -``` - -### Phase 2: Installer-native (automatic) - -```bash -# Build GRUB recipe (must be built before installer runs) -make r.grub - -# Build image with GRUB config (installer fetches GRUB automatically) -make all CONFIG_NAME=redbear-grub - -# Or via CLI flag -make all CONFIG_NAME=redbear-full INSTALLER_OPTS="--bootloader grub --cookbook=." - -# Verify ESP contents -python3 local/scripts/fat_tool.py ls build/x86_64/harddrive.img 1048576 / - -# Boot in QEMU -make qemu -# Expected: GRUB menu appears, "Red Bear OS" entry boots successfully -``` - -### Unit tests (no full build required) - -```bash -# Verify GRUB recipe builds -CI=1 ./target/release/repo cook grub - -# Verify host-side installer accepts --bootloader flag -build/fstools/bin/redox_installer --bootloader=grub --config=config/redbear-grub.toml --list-packages - -# Verify fat_tool.py operations -python3 local/scripts/fat_tool.py --help -``` - -## References - -- GNU GRUB Manual: https://www.gnu.org/software/grub/manual/grub/grub.html -- GRUB EFI standalone image: `grub-mkimage -O x86_64-efi ...` -- UEFI boot specification: `EFI/BOOT/BOOTX64.EFI` is the fallback boot path -- Redox bootloader source: `recipes/core/bootloader/source/` -- Installer GPT layout: `recipes/core/installer/source/src/installer.rs` diff --git a/local/docs/archived/KERNEL-IPC-CREDENTIAL-PLAN.md b/local/docs/archived/KERNEL-IPC-CREDENTIAL-PLAN.md deleted file mode 100644 index 688cafb017..0000000000 --- a/local/docs/archived/KERNEL-IPC-CREDENTIAL-PLAN.md +++ /dev/null @@ -1,748 +0,0 @@ -# Red Bear OS — Kernel, IPC, and Credential Syscalls Plan - -**Date:** 2026-04-30 -**Scope:** Kernel architecture, IPC infrastructure, credential syscalls, process isolation -**Implementation status:** Phases K1-K2, K4 ✅ complete. Phases K3, K5 deferred. -**Status:** This document is the canonical kernel + IPC plan, extending `local/docs/CONSOLE-TO-KDE-DESKTOP-PLAN.md` - -## 1. Purpose - -This plan defines the implementation roadmap for kernel hardening, IPC improvements, and credential -syscall implementation in Red Bear OS. It is the **canonical kernel authority** superseding scattered -kernel guidance in other docs. - -**Relationship to existing plans:** - -| Document | Relationship | -|----------|-------------| -| `CONSOLE-TO-KDE-DESKTOP-PLAN.md` | Parent: CONSOLE-TO-KDE v4.0 (Kernel & Core Infrastructure) | -| `IRQ-AND-LOWLEVEL-CONTROLLERS-ENHANCEMENT-PLAN.md` | Sibling: IRQ/PCI/MSI-X — not duplicated here | -| `RELIBC-IPC-ASSESSMENT-AND-IMPROVEMENT-PLAN.md` | Companion: relibc IPC surface — this plan covers kernel side | -| `ACPI-IMPROVEMENT-PLAN.md` | Sibling: ACPI power/shutdown — relevant for §4 (shutdown robustness) | -| `CONSOLE-TO-KDE-DESKTOP-PLAN.md` | Consumer: desktop stack depends on kernel work here | - -## 2. Current Architecture Assessment - -### 2.1 Kernel Overview - -The Redox microkernel (`recipes/core/kernel/source/`) is a ~20-40k LoC Rust microkernel. It runs in -ring 0 and provides: - -- **12 kernel schemes**: `debug`, `event`, `memory`, `pipe`, `irq`, `time`, `sys`, `proc`, `serio`, - `acpi`, `dtb`, `user` (userspace scheme wrapper) -- **~35 handled syscalls**: file I/O, memory mapping, process control, futex, time -- **Catch-all ENOSYS**: all unhandled syscall numbers return `ENOSYS` - -``` -recipes/core/kernel/source/src/ -├── syscall/ # Syscall dispatch: mod.rs (handlers), fs.rs, process.rs, futex.rs, time.rs -│ └── mod.rs # Main syscall() dispatch: 35 explicit match arms, _ => ENOSYS -├── scheme/ # Kernel schemes: debug, event, memory, pipe, irq, time, sys, proc, serio -│ ├── mod.rs # Scheme trait definition, SchemeId, FileHandle types -│ ├── proc.rs # Process manager scheme (fork, exec, signal, credential setting) -│ └── sys/ # System info scheme: context list, syscall debug, uname -├── context/ # Process/thread context management -│ ├── context.rs # Context struct: euid, egid, pid, files, signals, addr_space -│ └── memory.rs # Address space, grants, mmap implementation -├── memory/ # Physical/virtual memory management, page tables -└── sync/ # Locking primitives (RwLock, Mutex, CleanLockToken) -``` - -### 2.2 Syscall Dispatch Architecture - -The kernel's `syscall()` function in `syscall/mod.rs` dispatches based on `a` (syscall number): - -```rust -// From recipes/core/kernel/source/src/syscall/mod.rs (line 75) -match a { - SYS_WRITE2 => file_op_generic_ext(..), - SYS_WRITE => sys_write(..), - SYS_FMAP => { .. }, // Anonymous or file-backed mmap - SYS_READ2 => file_op_generic_ext(..), - SYS_READ => sys_read(..), - SYS_FPATH => file_op_generic(..), - SYS_FSTAT => fstat(..), - SYS_DUP => dup(..), - SYS_DUP2 => dup2(..), - SYS_SENDFD => sendfd(..), - SYS_OPENAT => openat(..), - SYS_UNLINKAT => unlinkat(..), - SYS_CLOSE => close(..), - SYS_CALL => call(..), // Scheme IPC: send message to scheme - SYS_FEVENT => fevent(..), // Register event on fd - SYS_YIELD => sched_yield(..), - SYS_NANOSLEEP => nanosleep(..), - SYS_CLOCK_GETTIME => clock_gettime(..), - SYS_FUTEX => futex(..), - SYS_MPROTECT => mprotect(..), - SYS_MREMAP => mremap(..), - // ... ~15 more file operations (fchmod, fchown, fcntl, flink, frename, ftruncate, fsync, etc.) - _ => Err(Error::new(ENOSYS)), // ← CATCH-ALL: all credential syscalls fall here -} -``` - -Syscall numbers come from the external `redox_syscall` crate (crates.io), not from the kernel tree. -The kernel consumes them via `use syscall::number::*`. - -### 2.3 Credential Architecture (Current) - -**Kernel Context struct** (`context/context.rs`): - -```rust -pub struct Context { - // Credential fields (initialized to 0): - pub euid: u32, // Effective user ID — used for scheme access control - pub egid: u32, // Effective group ID - pub pid: usize, // Process ID (set via proc scheme) - - // NOT present in kernel: - // ruid, suid — real/saved UID (maintained in userspace redox-rt) - // rgid, sgid — real/saved GID (maintained in userspace redox-rt) - // supplementary groups — not implemented anywhere - - // Access control interface: - pub fn caller_ctx(&self) -> CallerCtx { - CallerCtx { uid: self.euid, gid: self.egid, pid: self.pid } - } -} -``` - -**Credential read path** (userspace, no kernel involvement): -``` -getuid() → relibc::platform::redox::getuid() - → redox_rt::sys::posix_getresugid() - → reads local DYNAMIC_PROC_INFO { ruid, euid, suid, rgid, egid, sgid } - → returns cached userspace values (NO kernel syscall) -``` - -**Credential write path** (through `proc:` scheme): -``` -setresuid(ruid, euid, suid) → relibc::platform::redox::setresuid() - → redox_rt::sys::posix_setresugid(&Resugid { ruid, euid, suid, .. }) - → packs 6×u32 into buffer - → this_proc_call(&buf, CallFlags::empty(), &[ProcCall::SetResugid as u64]) - → SYS_CALL to proc: scheme - → kernel proc scheme handler (scheme/proc.rs:1269): - guard.euid = info.euid; - guard.egid = info.egid; -``` - -**Key finding**: The kernel DOES support credential setting through the `proc:` scheme, using -`ProcSchemeAttrs` with `euid`/`egid`/`pid`/`prio`/`debug_name` fields. The `getuid()`/`getgid()` -functions work through userspace-cached values in `redox-rt`. `setresuid()`/`setresgid()` work -through the proc scheme. - -**What's genuinely broken:** - -| Function | Status | Root Cause | -|----------|--------|------------| -| `setgroups()` | **ENOSYS stub** | relibc/redox/mod.rs:1205 — `todo_skip!(0, "setgroups({}, {:p}): not implemented")` | -| `getgroups()` | /etc/group-based | Works via `getpwuid()` + `getgrent()` iteration — doesn't use kernel groups | -| `initgroups()` | No-op | No supplementary group infrastructure | - -### 2.4 IPC Architecture - -**Scheme-based IPC** is the primary IPC mechanism: - -``` -┌─────────────┐ SYS_CALL(syscall) ┌──────────────┐ -│ Userspace │ ──────────────────────────→│ Kernel │ -│ Process A │ open/read/write/fevent │ Scheme │ -│ │ ←──────────────────────────│ Dispatch │ -└─────────────┘ result (usize/-errno) └──────┬───────┘ - │ - ┌─────────────────────┤ - │ │ - ┌────▼──────┐ ┌──────▼──────┐ - │ Kernel │ │ Userspace │ - │ Schemes │ │ Scheme │ - │ (12) │ │ Daemons │ - │ │ │ (via user:) │ - │ debug: │ │ │ - │ event: │ │ ptyd │ - │ memory: │ │ pcid │ - │ pipe: │ │ ext4d │ - │ irq: │ │ fatd │ - │ time: │ │ redox-drm │ - │ sys: │ │ ... │ - │ proc: │ │ │ - │ serio: │ │ │ - └───────────┘ └──────────────┘ -``` - -**IPC primitives available:** - -| Primitive | Mechanism | Kernel/Userspace | -|-----------|-----------|-----------------| -| `pipe:` scheme | Kernel pipe scheme — bidirectional byte streams | Kernel | -| `shm_open()` / `mmap(MAP_SHARED)` | Shared memory via memory scheme grants | Kernel | -| `SYS_CALL` + scheme messages | Send/receive typed messages to scheme daemons | Kernel dispatch, userspace handler | -| `fevent()` | Register kernel-level events on file descriptors | Kernel | -| `sendfd()` | Pass file descriptors between processes | Kernel | -| `event:` scheme | Kernel event notification (used by eventfd/signalfd/timerfd) | Kernel | -| Signals | `sigprocmask` + `sigaction` via proc: scheme | Kernel delivery, userspace handling | -| Futex | Fast userspace mutex via `SYS_FUTEX` | Kernel | - -**Current IPC limitations:** - -| Limitation | Impact | -|-----------|--------| -| No `SYS_PTRACE` | ptrace not available (handled via proc: scheme paths) | -| No `SYS_KILL` | Signal sending via proc: scheme only | -| eventfd/signalfd/timerfd recipe-applied | Bounded compatibility layers, not plain-source | -| `ifaddrs` synthetic | Only `loopback` + `eth0`, not live enumeration | -| POSIX message queues not implemented | `mqueue.h` missing entirely | -| SysV message queues not implemented | `sys/msg.h` missing entirely | -| No UNIX domain sockets (`AF_UNIX`) path | Socket-based IPC limited | - -### 2.5 Process Model - -Redox uses a **userspace process manager** (`procmgr` via `proc:` scheme): - -- **fork**: Implemented through proc: scheme → kernel creates new Context with cloned address space -- **exec**: Replaces address space with new executable image -- **spawn**: Combined fork+exec via proc: scheme -- **wait/waitpid/waitid**: Recipe-applied patch via proc: scheme (signals child exit) -- **Credentials on fork**: Address space cloned (userspace `DYNAMIC_PROC_INFO` inherited) -- **Credentials on exec**: `setresuid()` behavior (suid-bit not implemented in kernel) - -The kernel's Context struct tracks: -- `owner_proc_id: Option` — parent process for exit notification -- `files: Arc` — file descriptor table (can be shared) -- `addr_space: Option>` — address space (can be shared = threads) -- `sig: Option` — signal handler configuration - -## 3. Critical Gaps and Blockers - -### 3.1 Credential Syscall Blocker (Priority: P0-CRITICAL) - -The `setgroups()` function is **ENOSYS**. This blocks: -- `polkit` — uses `setgroups()` for privilege management -- `dbus-daemon` — uses credentials for service activation -- `logind` / `redbear-sessiond` — needs credential awareness -- `sudo` / `su` — uses `initgroups()` → `setgroups()` -- Any program that changes user identity - -**Root cause chain:** -1. `redox_syscall` crate (crates.io, upstream) has no `SYS_SETGROUPS`/`SYS_GETGROUPS` numbers -2. Kernel has no supplementary group table in Context struct -3. No group inheritance on fork/exec -4. relibc `setgroups()` is a `todo_skip!()` stub -5. `getgroups()` bypasses kernel entirely (reads /etc/group) - -### 3.2 Kernel-Level Access Control Gap (Priority: P1) - -The kernel's `caller_ctx()` provides `{euid, egid, pid}` to scheme handlers, but: - -1. **No consistent enforcement**: Kernel schemes may or may not check caller credentials -2. **No ruid/suid tracking**: Cannot distinguish real vs effective identity in kernel -3. **All processes start as root** (euid=0, egid=0): No privilege separation at boot -4. **No supplementary groups in kernel**: Only egid checked - -### 3.3 IPC Completeness Gaps (Priority: P2) - -| Gap | Priority | Blocked By | -|-----|----------|------------| -| POSIX message queues (`mqueue.h`) | P2 | Scheme design needed | -| SysV message queues (`sys/msg.h`) | P2 | Scheme design needed | -| UNIX domain sockets (`AF_UNIX`) | P2 | Kernel or scheme implementation | -| Non-synthetic `ifaddrs` | P3 | Network stack enumeration | -| eventfd/signalfd/timerfd → plain-source | P3 | Upstream relibc convergence | - -### 3.4 Resource Limits (Priority: P2) - -`SYS_GETRLIMIT` / `SYS_SETRLIMIT` return ENOSYS. This is a microkernel design choice: -- Resource limits are typically library-level policy in capability systems -- Current approach: limits enforced in userspace daemons -- Desktop impact: systemd/logind expect rlimit support for service management - -### 3.5 Shutdown Robustness (Priority: P2) - -ACPI shutdown via `kstop` eventing exists but has gaps: -- `acpid` startup has panic-grade `expect` paths -- `_S5` derivation gated on PCI timing -- DMAR orphaned in `acpid` source -- See `local/docs/ACPI-IMPROVEMENT-PLAN.md` for full detail - -## 4. Implementation Plan - -### Phase K1: Kernel Credential Foundation (Week 1-2) - -**Goal**: Add supplementary group support to the kernel and wire `setgroups()`/`getgroups()`. - -#### K1.1 — Add supplementary groups to kernel Context - -```rust -// Context struct additions (context/context.rs): -pub struct Context { - // Existing: - pub euid: u32, - pub egid: u32, - pub pid: usize, - - // NEW: Real/saved IDs (moved from userspace redox-rt to kernel): - pub ruid: u32, - pub rgid: u32, - pub suid: u32, - pub sgid: u32, - - // NEW: Supplementary groups - pub groups: Vec, // Or Arc<[u32]> for sharing -} -``` - -**Files modified:** -- `recipes/core/kernel/source/src/context/context.rs` — add fields, initialize, clone on fork -- `recipes/core/kernel/source/src/scheme/proc.rs` — extend `ProcSchemeAttrs` to include ruid/suid/rgid/sgid/groups -- `local/patches/kernel/` — new patch: `P4-credential-fields.patch` - -#### K1.2 — Add `SYS_SETGROUPS` and `SYS_GETGROUPS` to redox_syscall - -The `redox_syscall` crate is upstream (crates.io). Red Bear must either: -- **Option A (preferred)**: Contribute upstream PR to add syscall numbers -- **Option B**: Vendor fork of `redox_syscall` in `local/` overlay -- **Option C**: Define Red Bear-local syscall numbers in kernel directly - -**Recommended: Option A + B fallback**: -1. Submit upstream PR to `redox_syscall` adding: - - `SYS_SETGROUPS`, `SYS_GETGROUPS` - - `SYS_SETUID`, `SYS_SETGID`, `SYS_GETUID`, `SYS_GETGID` - - `SYS_GETEUID`, `SYS_GETEGID` - - `SYS_SETREUID`, `SYS_SETREGID` - - `SYS_GETRESUID`, `SYS_GETRESGID` - -2. While upstream PR is pending, use a local `redox_syscall` patch: - - Copy `redox_syscall` crate into `local/vendor/redox_syscall/` - - Add syscall number constants - - Point kernel Cargo.toml to local path - - Patch tracked in `local/patches/kernel/P4-redox-syscall-numbers.patch` - -#### K1.3 — Add kernel syscall handlers - -**New file:** `recipes/core/kernel/source/src/syscall/cred.rs` - -```rust -// Credential syscall handlers -pub fn setresuid(ruid: u32, euid: u32, suid: u32, token: &mut CleanLockToken) -> Result { - let context_lock = context::current(); - let mut context = context_lock.write(token.token()); - - // Permission check: must be root or match current values - if context.euid != 0 { - if let Some(ruid) = ruid_opt { /* check ruid == current ruid/euid/suid */ } - // ... POSIX permission model - } - - // Set values - if ruid != u32::MAX { context.ruid = ruid; } - if euid != u32::MAX { context.euid = euid; } - if suid != u32::MAX { context.suid = suid; } - Ok(0) -} - -pub fn setgroups(groups: &[u32], token: &mut CleanLockToken) -> Result { - // Requires: euid == 0 - let context_lock = context::current(); - let mut context = context_lock.write(token.token()); - if context.euid != 0 { return Err(Error::new(EPERM)); } - context.groups = groups.to_vec(); - Ok(0) -} - -pub fn getgroups(token: &mut CleanLockToken) -> Result> { - let context_lock = context::current(); - let context = context_lock.read(token.token()); - Ok(context.groups.clone()) -} -``` - -**Modified file:** `recipes/core/kernel/source/src/syscall/mod.rs` -```rust -match a { - // ... existing arms ... - SYS_SETRESUID => setresuid(b as u32, c as u32, d as u32, token), - SYS_SETRESGID => setresgid(b as u32, c as u32, d as u32, token), - SYS_GETRESUID => getresuid(UserSlice::wo(b, c)?, token), - SYS_GETRESGID => getresgid(UserSlice::wo(b, c)?, token), - SYS_SETUID => setuid(b as u32, token), - SYS_SETGID => setgid(b as u32, token), - SYS_GETUID => Ok(getuid(token)), - SYS_GETGID => Ok(getgid(token)), - SYS_GETEUID => Ok(geteuid(token)), - SYS_GETEGID => Ok(getegid(token)), - SYS_SETGROUPS => setgroups(UserSlice::ro(b, c)?, token).map(|()| 0), - SYS_GETGROUPS => getgroups(UserSlice::wo(b, c)?, token), - // ... existing arms ... -} -``` - -#### K1.4 — Wire relibc setgroups()/getgroups() through real syscalls - -**Modified:** `recipes/core/relibc/source/src/platform/redox/mod.rs` -```rust -// Replace todo_skip!() stub: -unsafe fn setgroups(size: size_t, list: *const gid_t) -> Result<()> { - if size < 0 || size > NGROUPS_MAX { return Err(Errno(EINVAL)); } - let groups = core::slice::from_raw_parts(list, size as usize); - syscall::setgroups(groups)?; - Ok(()) -} - -// Replace /etc/group-based getgroups: -fn getgroups(mut list: Out<[gid_t]>) -> Result { - let mut buf = [0u32; NGROUPS_MAX as usize]; - let count = syscall::getgroups(&mut buf)?; - for (i, gid) in buf[..count].iter().enumerate() { - list[i] = *gid as gid_t; - } - Ok(count as c_int) -} -``` - -#### K1.5 — Add credential syscall stubs in redox-rt - -**Modified:** `recipes/core/relibc/source/redox-rt/src/sys.rs` -```rust -pub fn setgroups(groups: &[u32]) -> Result<()> { - unsafe { - redox_syscall::syscall5( - redox_syscall::SYS_SETGROUPS, - groups.as_ptr() as usize, - groups.len(), - 0, 0, 0, - ) - .map(|_| ()) - .map_err(|e| Error::new(e.errno as i32)) - } -} - -pub fn getgroups(buf: &mut [u32]) -> Result { - unsafe { - redox_syscall::syscall3( - redox_syscall::SYS_GETGROUPS, - buf.as_mut_ptr() as usize, - buf.len(), - 0, - ) - .map_err(|e| Error::new(e.errno as i32)) - } -} -``` - -#### K1.6 — Patch management - -All kernel and relibc source changes must be mirrored into `local/patches/`: - -```bash -local/patches/ -├── kernel/ -│ ├── redox.patch # Updated symlink target -│ ├── P4-credential-fields.patch # Context struct additions -│ ├── P4-credential-syscalls.patch # Syscall handlers + dispatch -│ └── P4-redox-syscall-numbers.patch # Local redox_syscall additions -├── relibc/ -│ ├── P4-setgroups-kernel.patch # Setgroups through real syscall -│ ├── P4-getgroups-kernel.patch # Getgroups through real syscall -│ └── P4-redox-rt-cred-syscalls.patch # redox-rt syscall wrappers -``` - -### Phase K2: Kernel Access Control Hardening (Week 2-3) - -**Goal**: Enforce credential checks in kernel schemes, add proper privilege separation. - -#### K2.1 — Enforce scheme-level credential checks - -Each kernel scheme handler currently receives `CallerCtx { uid, gid, pid }`. Ensure consistent -credential enforcement: - -| Scheme | Current Check | Required Check | -|--------|--------------|----------------| -| `memory:` | Physical memory access → root only | ✅ Already enforced (euid==0 for phys) | -| `irq:` | IRQ registration → root only | ✅ Already enforced | -| `proc:` | Process inspection → caller == target OR root | 🔄 Review: ensure consistent | -| `sys:` | System info → read-only for all | ✅ Appropriate | -| `debug:` | Debug output → should be root-only | 🔄 Review: add check | -| `serio:` | PS/2 device → root only | 🔄 Review: add check | -| `event:` | Event registration → process-own only | 🔄 Review: ensure isolation | - -#### K2.2 — Bootstrap with non-root init process - -Currently all processes start as euid=0/egid=0. The boot sequence should: -1. Kernel bootstrap context starts as root (euid=0, egid=0) — required for init -2. Init (`/sbin/init`) runs as root -3. Init drops privileges before spawning user services: - ```rust - // In init or service manager: - setresuid(1000, 1000, 1000); // Drop to regular user - setgroups(&[1000, 27, 100]); // Set supplementary groups - // Then spawn child services with restricted permissions - ``` - -#### K2.3 — Add `initgroups()` support - -```rust -// In relibc/src/platform/redox/mod.rs: -fn initgroups(user: CStr, group: gid_t) -> Result<()> { - // 1. Set primary group - setgid(group)?; - // 2. Parse /etc/group for supplementary groups containing this user - let mut groups = vec![group]; - // ... iterate getgrent() to find user memberships ... - // 3. Set supplementary groups via kernel syscall - setgroups(&groups)?; - Ok(()) -} -``` - -### Phase K3: IPC Infrastructure Improvements (Week 3-5) - -**Goal**: Complete IPC primitives needed for desktop infrastructure. - -#### K3.1 — POSIX Message Queues (`mqueue.h`) - -**Design decision**: Implement as a userspace scheme daemon (not kernel syscalls). - -``` -mqd: -├── Registers as scheme:mqueue -├── Stores queues in memory backed by shm_open() + mmap() -├── mq_open() → open scheme:mqueue/{name} -├── mq_send() → write to fd -├── mq_receive() → read from fd -├── mq_notify() → fevent() on fd for async notification -├── mq_close() → close fd -└── mq_unlink() → unlink scheme:mqueue/{name} -``` - -**Implementation:** -- New Red Bear package: `local/recipes/system/mqueued/` -- Relibc header: `recipes/core/relibc/source/src/header/mqueue/` -- Recipe in `local/recipes/system/mqueued/recipe.toml` -- Init service: `/usr/lib/init.d/50_mqueued.service` - -#### K3.2 — SysV Message Queues (`sys/msg.h`) - -**Design decision**: Implement as scheme daemon or on top of POSIX message queues. -- Recommended: implement directly alongside `mqueued` using shared infrastructure. -- Low priority — Qt/KDE do not depend on SysV msg queues. - -#### K3.3 — UNIX Domain Sockets (`AF_UNIX` / `SOCK_STREAM`) - -**Current state**: D-Bus uses abstract sockets on Linux. Redox uses scheme-based communication. -- For D-Bus compatibility: `redbear-sessiond` already uses `zbus` with custom transport -- For general `AF_UNIX`: implement as `scheme:unix` daemon backed by kernel pipe scheme -- Priority: P3 — D-Bus is already working through scheme transport - -#### K3.4 — Non-synthetic Interface Enumeration - -Replace the hardcoded `loopback` + `eth0` model with live network interface enumeration: -- Query `smolnetd` or equivalent for active interfaces -- Expose through `getifaddrs()` properly -- Priority: P3 — needed for NetworkManager-like functionality - -#### K3.5 — eventfd/signalfd/timerfd → plain-source convergence - -Current state: all three are recipe-applied patches. Goal: upstream into relibc mainline. -- Monitor upstream relibc for equivalent implementations -- When upstream absorbs: shrink/drop Red Bear patch chain -- When upstream does NOT absorb after 3+ months: promote to durable Red Bear-maintained -- See `local/docs/RELIBC-IPC-ASSESSMENT-AND-IMPROVEMENT-PLAN.md` Phase I5 - -### Phase K4: Resource Limits and Process Management (Week 4-6) - -#### K4.1 — RLIMIT Support - -**Decision**: Enforce resource limits in userspace, not kernel. -- The kernel is a microkernel — resource limits are policy -- `getrlimit()` / `setrlimit()` → libc stubs with reasonable defaults -- Process enforcement → `procmgr` (userspace process manager) via proc: scheme -- File descriptor limits → already enforced via `CONTEXT_MAX_FILES` in kernel -- Memory limits → userspace `procmgr` can kill processes exceeding limits - -```rust -// relibc implementation (userspace, no kernel changes needed): -fn getrlimit(resource: c_int, rlim: *mut rlimit) -> Result<()> { - match resource { - RLIMIT_NOFILE => { rlim.rlim_cur = 1024; rlim.rlim_max = 4096; } - RLIMIT_NPROC => { rlim.rlim_cur = 256; rlim.rlim_max = 1024; } - RLIMIT_AS => { rlim.rlim_cur = RLIM_INFINITY; rlim.rlim_max = RLIM_INFINITY; } - RLIMIT_CORE => { rlim.rlim_cur = 0; rlim.rlim_max = RLIM_INFINITY; } - // ... other resource types with reasonable defaults - _ => return Err(Errno(EINVAL)), - } - Ok(()) -} -``` - -#### K4.2 — PTRACE via proc: scheme - -`SYS_PTRACE` is not implemented as a direct syscall. The Redox model uses the `proc:` scheme -for process inspection and manipulation: -- Already partially implemented in `scheme/proc.rs` -- Memory read/write through proc: scheme file operations -- Register read/write through proc: scheme -- Signal injection through proc: scheme - -Improvements needed: -- Document the proc: scheme ptrace API surface -- Ensure all ptrace operations have proc: scheme equivalents -- Add `PTRACE_*` constants to redox_syscall for compatibility - -#### K4.3 — clock_settime - -`SYS_CLOCK_SETTIME` returns ENOSYS. Implementation: -- Add scheme write path to `/scheme/sys/update_time_offset` -- Or implement as direct syscall for precision -- Priority: P3 — needed for NTP synchronization - -### Phase K5: Shutdown and Power Management (Week 5-7) - -See `local/docs/ACPI-IMPROVEMENT-PLAN.md` for full ACPI plan. This section covers kernel-specific -work only. - -#### K5.1 — Hardened acpid Startup - -- Remove panic-grade `expect` paths in kernel ACPI/AML handling -- Add graceful fallback when ACPI tables are missing or malformed -- See ACPI-IMPROVEMENT-PLAN.md Wave 1 - -#### K5.2 — kstop Shutdown Robustness - -- Current: `_S5` shutdown via `kstop` event exists but gated on PCI timing -- Required: deterministic shutdown ordering: - 1. Notify userspace services of impending shutdown - 2. Sync filesystems - 3. Power off via ACPI/FADT -- See ACPI-IMPROVEMENT-PLAN.md Wave 2 - -#### K5.3 — Sleep State Support - -- S3 (suspend-to-RAM) and S4 (hibernate) are not yet supported -- Requires: kernel state serialization, device reinitialization -- Priority: P4 — long-term, not blocking desktop - -## 5. Dependency Chain - -``` -Phase K1 (credential syscalls) ─────────────────────┐ - │ │ - ├──► polkit compatibility │ - ├──► dbus-daemon credential checks │ - ├──► sudo/su user switching │ - ├──► redbear-sessiond login1 handoff │ - └──► greeter/session-launch credential drop │ - │ -Phase K2 (access control) ────────────────────────────┤ - │ │ - ├──► Privilege-separated boot sequence │ - ├──► Scheme-level credential enforcement │ - └──► initgroups() for service launching │ - │ -Phase K3 (IPC) ───────────────────────────────────────┤ - │ │ - ├──► POSIX message queues → needed by some apps │ - ├──► AF_UNIX → broader D-Bus transport options │ - └──► eventfd/signalfd/timerfd → KDE/Qt runtime │ - │ -Phase K4 (limits/ptrace) ─────────────────────────────┤ - │ │ - ├──► RLIMIT → systemd/logind compatibility │ - ├──► PTRACE → debugging support │ - └──► clock_settime → NTP synchronization │ - ▼ - Desktop infrastructure - ready for KDE Plasma -``` - -## 6. Integration with Existing Work - -### 6.1 Already in Progress (do not duplicate) - -| Area | Canonical Plan | Status | -|------|---------------|--------| -| IRQ / MSI-X / IOMMU | `IRQ-AND-LOWLEVEL-CONTROLLERS-ENHANCEMENT-PLAN.md` | Waves 1-6 complete, hardware validation open | -| ACPI shutdown / power | `ACPI-IMPROVEMENT-PLAN.md` | Waves 1-2 complete, sleep states deferred | -| relibc IPC surface | `RELIBC-IPC-ASSESSMENT-AND-IMPROVEMENT-PLAN.md` | Phases I1-I5, message queues deferred | -| D-Bus / sessiond | `DBUS-INTEGRATION-PLAN.md` | Phase 1 complete, Phase 2 in progress | -| Greeter / login | `GREETER-LOGIN-IMPLEMENTATION-PLAN.md` | Active, bounded proof passing | -| Desktop path | `CONSOLE-TO-KDE-DESKTOP-PLAN.md` | Phase 1-5 model, KWin building | - -### 6.2 This Plan Covers (uniquely) - -| Area | This Plan | Not Covered By | -|------|-----------|---------------| -| Kernel credential architecture | §3, Phase K1 | Any existing plan | -| Kernel access control hardening | §3.2, Phase K2 | Any existing plan | -| `setgroups()` / `getgroups()` kernel implementation | Phase K1.2-K1.4 | Only stub noted elsewhere | -| Supplementary group infrastructure | Phase K1.1 | Not covered anywhere | -| POSIX/SysV message queues | Phase K3.1-K3.2 | Deferred in relibc-IPC plan | -| UNIX domain sockets | Phase K3.3 | Not covered | -| RLIMIT design decision | Phase K4.1 | Noted as gap only | -| PTRACE via proc: scheme | Phase K4.2 | Not covered | -| clock_settime implementation | Phase K4.3 | Noted as gap only | - -## 7. Patch Governance - -All kernel and relibc source changes must follow the durability policy (see `local/AGENTS.md`): - -1. **Make changes** in `recipes/core/kernel/source/` or `recipes/core/relibc/source/` -2. **Generate patches**: `git diff` in the source tree → `local/patches//P4-*.patch` -3. **Wire patches** into `recipes/core//recipe.toml` patches list -4. **Commit** patches + recipe changes before session end -5. **Assume** source trees may be thrown away by `make distclean` or upstream refresh - -### Patch naming convention: -``` -local/patches/kernel/P4-credential-fields.patch -local/patches/kernel/P4-credential-syscalls.patch -local/patches/kernel/P4-redox-syscall-numbers.patch -local/patches/relibc/P4-setgroups-kernel.patch -local/patches/relibc/P4-getgroups-kernel.patch -local/patches/relibc/P4-redox-rt-cred-syscalls.patch -local/patches/relibc/P4-initgroups.patch -``` - -## 8. Validation and Evidence - -### 8.1 Build Evidence - -| Check | Command | -|-------|---------| -| Kernel compiles | `make r.kernel` | -| relibc compiles | `make r.relibc` | -| Full OS builds | `make all CONFIG_NAME=redbear-full` | - -### 8.2 Runtime Evidence - -| Test | Verification | -|------|-------------| -| `getuid()` returns non-zero after login | `id` command in guest | -| `setgroups()` succeeds for root | `sudo -u user id` in guest | -| `setresuid()` properly changes euid | `su user -c 'id'` | -| `initgroups()` populates groups | `groups` command in guest | -| Credentials survive fork | `bash -c 'id'` | -| Credentials dropped on exec (if SUID implemented) | TBD | -| polkit can query credentials | `pkexec echo ok` | -| dbus-daemon starts without errors | `dbus-monitor` | - -### 8.3 Verification Scripts - -Create bounded proof scripts: -```bash -local/scripts/test-credential-syscalls-qemu.sh # QEMU launcher -local/scripts/test-credential-syscalls-guest.sh # In-guest checker -``` - -## 9. References - -- `local/docs/CONSOLE-TO-KDE-DESKTOP-PLAN.md` — Canonical comprehensive plan -- `docs/01-REDOX-ARCHITECTURE.md` — Architecture reference -- `local/docs/IRQ-AND-LOWLEVEL-CONTROLLERS-ENHANCEMENT-PLAN.md` — IRQ/PCI plan (sibling) -- `local/docs/RELIBC-IPC-ASSESSMENT-AND-IMPROVEMENT-PLAN.md` — IPC surface plan (companion) -- `local/docs/ACPI-IMPROVEMENT-PLAN.md` — ACPI/shutdown plan (sibling) -- `local/docs/CONSOLE-TO-KDE-DESKTOP-PLAN.md` — Desktop path plan (consumer) -- `recipes/core/kernel/source/src/syscall/mod.rs` — Syscall dispatch (primary implementation target) -- `recipes/core/kernel/source/src/context/context.rs` — Context struct (credential fields) -- `recipes/core/kernel/source/src/scheme/proc.rs` — Proc scheme (credential setting) -- `recipes/core/relibc/source/src/platform/redox/mod.rs` — relibc Redox platform (credential stubs) -- `recipes/core/relibc/source/redox-rt/src/sys.rs` — redox-rt credential primitives diff --git a/local/docs/archived/PROFILE-MATRIX.md b/local/docs/archived/PROFILE-MATRIX.md deleted file mode 100644 index f1589cbaa4..0000000000 --- a/local/docs/archived/PROFILE-MATRIX.md +++ /dev/null @@ -1,137 +0,0 @@ -# Red Bear OS Profile Matrix - -## Purpose - -This matrix makes the tracked Red Bear profiles explicit so support claims map to a concrete build -target instead of a vague feature list. - -## Validation Labels - -- **builds** — configuration and packages are expected to compile -- **boots** — image is expected to reach a usable boot state -- **validated** — behavior has been tested on the claimed profile -- **experimental** — available for bring-up, but not support-promised - -Subsystem plans may add narrower intermediate labels when `boots` is too coarse. In particular, the -USB plan uses: - -- **enumerates** — runtime surfaces can discover controllers, ports, or descriptors -- **usable** — a specific controller/class path works in a limited real scenario - -## Compile Targets - -> **Phase numbering note:** phase labels below use the v2.0 desktop plan phases from -> `local/docs/CONSOLE-TO-KDE-DESKTOP-PLAN.md`. Scripts and older docs may reference the -> historical P0–P6 hardware-enablement sequence — those are not the same numbering. - -| Profile | Intent | Key Fragments | Current support language | -|---|---|---|---| -| `redbear-mini` | Console + storage + wired-network baseline | `minimal.toml`, `redbear-legacy-base.toml`, `redbear-device-services.toml`, `redbear-netctl.toml` | builds / primary validation baseline / DHCP boot profile enabled / input-runtime substrate wired / USB: daemons built via base and targeted for bounded mini-profile validation | -| `redbear-grub` | Text-only with GRUB boot manager | `redbear-mini.toml`, `redbear-grub-policy.toml` | builds / live media variant with GRUB chainload for real bare metal / desktop graphics intentionally absent | -| `redbear-full` | Desktop/network/session plumbing target | `desktop.toml`, `redbear-legacy-base.toml`, `redbear-legacy-desktop.toml`, `redbear-device-services.toml`, `redbear-netctl.toml`, `redbear-greeter-services.toml` | builds / boots in QEMU / active desktop-capable compile target / support claims remain evidence-qualified | - -## Build Artifacts (ISO Organization) - -All profiles produce outputs under `build/x86_64/`. Each profile gets its own directory: - -| Profile | ISO | harddrive.img | Image size | QEMU RAM | Boots via `make qemu`? | -|---------|-----|---------------|------------|----------|------------------------| -| `redbear-mini` | `redbear-mini.iso` | `redbear-mini/harddrive.img` | 1.5 GiB | **2 GiB** | ✅ Text login | -| `redbear-grub` | `redbear-grub.iso` | `redbear-grub/harddrive.img` | 1.5 GiB | **2 GiB** | ✅ Text login | -| `redbear-full` | `redbear-full.iso` | `redbear-full/harddrive.img` | 4.0 GiB | **2 GiB** | ⚠️ Text login only | - -> **⚠️ CRITICAL**: `redbear-full` requires **exactly 2 GiB** of guest RAM in QEMU. With 4 GiB or more, the kernel hangs silently during early SMP/memory initialization (x86_64 only). This is a confirmed kernel bug — see `BOOT-PROCESS-ASSESSMENT.md` Phase 7. The `make qemu` default of `QEMU_MEM=2048` is correct for all profiles. - -### Known QEMU Issues - -| Issue | Profiles affected | Workaround | -|-------|-------------------|------------| -| **Kernel hang with ≥4 GiB RAM** (nographic mode) | `redbear-full` | Use `-m 2048` or less. `make qemu` default is 2048, safe. | -| **Graphical login fallback** — greeter uses text login, not Wayland | `redbear-full` | Set `KWIN_DRM_DEVICES=/dev/dri/card0` in greeter env; verify redox-drm daemon is running | -| **Live ISO preload** — `unable to allocate 4078 MiB upfront` | `redbear-full` | Disable live mode (press `l` at bootloader); preload needs chunked allocation | -| **EFI EDID unavailable** — `Failed to get EFI EDID` warning | All | Expected in QEMU; not a project issue | -| **AHCI DVD I/O error** — empty DVD-ROM port probe | All | Benign; non-blocking | - -### ISO naming convention - -- **Profile ISOs**: `redbear-{profile}.iso` (e.g. `redbear-full.iso`, `redbear-mini.iso`) -- **Legacy names** (`redbear-live-mini.iso`, `redbear-live-full.iso`) are **deprecated** and should not be used in new scripts or documentation. -- `scripts/build-iso.sh` accepts profile names: `redbear-full`, `redbear-mini`, `redbear-grub`. - -## Profile Notes - -### `redbear-mini` - -- First place to validate repository discipline and profile reproducibility. -- Should stay smaller and less assumption-heavy than the graphics profiles. -- Enables the shared `wired-dhcp` netctl profile by default for the VM/wired baseline. -- Ships the shared firmware/input runtime service prerequisites so the early substrate can be tested on the smallest profile as well. - -### Historical and experimental release fork - -- Experimental release fork such as `redbear-bluetooth-experimental` and `redbear-wifi-experimental` - are bounded validation slices layered on top of the tracked compile targets, not additional - compile targets. - -### `redbear-grub` - -- Text-only console/recovery target with GRUB boot manager for multi-boot bare-metal workflows. -- Inherits the same non-graphics intent as `redbear-mini`, but with GRUB chainload ESP layout. -- Should not grow desktop/session assumptions. - -### `redbear-full` - -- Desktop-capable tracked target for the current Red Bear session/network/runtime plumbing surface. -- Carries the broader D-Bus, greeter, seat, and desktop-oriented service surface. - -### Historical notes - -- Older names such as `redbear-minimal`, `redbear-desktop`, `redbear-wayland`, `redbear-kde`, - `redbear-live`, `redbear-live-mini`, and `redbear-live-full` remain in older docs and some - implementation details, but they are not the current supported compile-target surface. - -### `redbear-bluetooth-experimental` - -- Standalone tracked profile for the first in-tree Bluetooth slice instead of a blanket claim about - all Red Bear images. -- Extends `redbear-mini` so the baseline runtime tooling is already present, then adds only the - bounded Bluetooth pieces on top. -- Current path under active validation: QEMU/UEFI boot to login prompt plus guest-side `redbear-bluetooth-battery-check`, targeting repeated in-boot reruns, daemon-restart coverage, and one experimental battery-sensor Battery Level read-only workload. -- Current support language is intentionally narrow: explicit-startup only, USB-attached transport, - BLE-first CLI/scheme surface, one experimental battery-sensor Battery Level read-only workload, - and no USB-class autospawn claim yet. - -### `redbear-wifi-experimental` - -- Standalone tracked profile for the current bounded Intel Wi-Fi slice instead of implying that the - wider desktop profiles already carry the full driver stack. -- Extends `redbear-mini` so the baseline firmware/input/reporting/profile-manager surface stays - inherited while the Intel Wi-Fi driver package and bounded validation role remain isolated here. -- Includes the Intel driver package (`redbear-iwlwifi`) in addition to the shared firmware, - control-plane, reporting, and profile-manager pieces. -- Current support language is intentionally narrow: bounded probe/prepare/init/activate/scan/ - connect/disconnect lifecycle, packaged in-target validation and capture commands, and no claim yet - of validated real AP association or end-to-end Wi-Fi connectivity. - -## Bluetooth Note - -- `redbear-bluetooth-experimental` is now the tracked first Bluetooth-specific profile. -- Its support language remains experimental and bounded; it should not be used to imply Bluetooth - support across the wider Red Bear profile set. -- The current bounded BLE workload is one read-only battery-sensor Battery Level interaction; this - profile still does not claim generic GATT, write, or notify support. -- The current validation claim is QEMU-scoped and packaged-checker-scoped, not a blanket claim - about real hardware Bluetooth maturity. - -## USB Note - -- `redbear-mini` is the preferred non-graphics target for bounded USB validation because these - proofs do not require the full desktop graphics/session surface. -- USB validation is QEMU-only (`test-usb-qemu.sh --check`). No profile makes a real hardware USB - support claim. -- USB error handling and correctness carry significant Red Bear patches over upstream; see - `local/patches/base/redox.patch` and `local/docs/USB-IMPLEMENTATION-PLAN.md` for details. -- The in-tree mini image is still assembled through legacy `redbear-minimal*` config files in some - places, but the supported compile-target names are `redbear-mini` and `redbear-grub`. -- `redbear-bluetooth-experimental` uses USB only as a transport for BLE dongles; it does not make a - general USB-class-autospawn claim. diff --git a/local/docs/archived/RELIBC-IPC-ASSESSMENT-AND-IMPROVEMENT-PLAN.md b/local/docs/archived/RELIBC-IPC-ASSESSMENT-AND-IMPROVEMENT-PLAN.md deleted file mode 100644 index 18d26195a6..0000000000 --- a/local/docs/archived/RELIBC-IPC-ASSESSMENT-AND-IMPROVEMENT-PLAN.md +++ /dev/null @@ -1,165 +0,0 @@ -# Red Bear OS relibc IPC Assessment and Improvement Plan - -## Purpose - -This document is the IPC-focused companion to -`local/docs/CONSOLE-TO-KDE-DESKTOP-PLAN.md`. - -Its job is to describe the current IPC-facing relibc surface honestly, especially where the active -Red Bear build depends on recipe-applied compatibility layers rather than plain-source upstream -relibc. - -## Evidence model - -This document uses the same terms as the canonical relibc plan: - -- **plain-source-visible** -- **recipe-applied** -- **test-present** -- **runtime-unrevalidated in this pass** - -Do not collapse those into one generic "implemented" label. - -## Current IPC inventory - -| Surface | Plain source | Active build | Notes | -|---|---|---|---| -| `shm_open()` / `shm_unlink()` | yes | yes | provided through `sys_mman` in the live source tree | -| named POSIX semaphores | no | yes | added by `P3-semaphore-fixes.patch` on top of `shm_open()` / `mmap()` | -| `eventfd` | no | yes | added by `P3-eventfd-mod.patch` through `/scheme/event/eventfd/...` | -| `signalfd` | no | yes | added by `P3-signalfd.patch` through `/scheme/event` plus signal-mask handling | -| `timerfd` | no | yes | added by `P3-timerfd-relative.patch` through `/scheme/time/{clockid}` | -| `waitid()` | no | yes | added by `P3-waitid.patch` | -| `ifaddrs` / `net_if` support used by IPC-adjacent consumers | no | yes | added by `P3-ifaddrs-net_if.patch`; currently synthetic | -| SysV shm (`sys/shm.h`) | no | yes | activated via `P3-sysv-shm-impl.patch` in recipe (2026-04-29) | -| SysV sem (`sys/sem.h`) | no | yes | activated via `P3-sysv-sem-impl.patch` in recipe (2026-04-29) | -| POSIX message queues (`mqueue.h`) | no | no | still TODO in the live source tree | -| SysV message queues (`sys/msg.h`) | no | no | still TODO in the live source tree | - -## Observed limitations - -### Named POSIX semaphores - -The active patch chain implements named semaphores by storing a `Semaphore` inside shared memory -opened through `shm_open()` and mapped with `mmap()`. That is a useful bounded compatibility path, -but it should still be described as a Red Bear recipe-applied layer, not a plain-source upstream -relibc completion. - -### fd-event APIs - -`eventfd`, `signalfd`, and `timerfd` are present in the active build, but they are all scheme-backed -compatibility layers: - -- `eventfd` depends on `/scheme/event/eventfd/...` -- `signalfd` depends on `/scheme/event` and blocks the supplied mask with `sigprocmask()` -- `timerfd` depends on `/scheme/time/{clockid}` and currently rejects unsupported flag combinations - -These are real compatibility layers, but they should still be described as bounded until broader -consumer/runtime proof is recorded. - -### Deferred SysV shm/sem work - -SysV shm/sem carriers were activated in recipe (2026-04-29). Message queues remain deferred follow-up work. - -### Interface enumeration used by networking-adjacent consumers - -The current `P3-ifaddrs-net_if.patch` replaces `ENOSYS`, but it does so with a synthetic two-entry -model: - -- `loopback` -- `eth0` - -That is enough for some bounded consumers, but it should not be described as live full interface -enumeration. - -## Downstream pressure - -### Qt / KDE - -Qt and KDE remain the strongest pressure on relibc IPC semantics. - -They do not only need headers to exist. They need the active compatibility layers to behave well -enough for: - -- shared-memory consumers, -- named semaphore consumers, -- direct `eventfd` / `timerfd` users, -- and process-control paths such as `waitid()`. - -### Wayland-facing consumers - -Wayland-facing pressure is strongest on the fd-event side of the IPC story: - -- `eventfd` -- `signalfd` -- `timerfd` - -That is a different pressure profile from the SysV and named-semaphore side. - -## Fresh verification in this pass - -This pass revalidated the active concrete-wave IPC-facing surface through the relibc test recipe: - -- `sys_eventfd/eventfd` -- `sys_signalfd/signalfd` -- `sys_timerfd/timerfd` -- `waitid` -- `semaphore/named` -- `semaphore/unnamed` - -These are bounded relibc-target proofs. They improve confidence in the active fd-event and named -semaphore surface. SysV shm/sem are now active in the recipe (2026-04-29); message queues remain deferred. - -## Improvement plan - -### Phase I1 — Keep IPC claims aligned with the active build surface - -- document patch-applied IPC layers as patch-applied -- stop describing them as plain-source-visible unless they move into the live source tree -- keep this doc aligned with `recipes/core/relibc/recipe.toml` - -### Phase I2 — Decide the support contract for bounded IPC layers - -For each major IPC area, choose one of these paths explicitly: - -- bounded compatibility layer with honest documentation, -- or broader semantics work with explicit proof targets. - -This is especially important for: - -- SysV shm, -- SysV sem, -- named semaphores, -- and `ifaddrs`-driven interface discovery. - -### Phase I3 — Add proof where current docs only imply confidence - -Highest-value areas: - -- the fd-event slice used by Wayland-facing consumers, -- shared-memory and named-semaphore behavior used by Qt/KDE, -- and the currently synthetic interface-discovery path. - -### Phase I4 — Triage message queues directly - -Message queues are still genuine absences, not just bounded implementations. - -This doc should keep them visible until Red Bear either: - -- implements them, -- proves they are unnecessary for the intended consumer set, -- or explicitly documents them as deferred/non-goals. - -### Phase I5 — Converge with upstream deliberately - -When upstream relibc absorbs equivalent IPC functionality, prefer the upstream path and shrink the -Red Bear patch chain. Until then, keep the active IPC carrier set explicit and documented. - -## Bottom line - -The current Red Bear relibc IPC story is **material patch-applied compatibility, not plain-source -completion**. - -That is still valuable progress, but the repo should describe it honestly: several important IPC -surfaces exist in the active build, several of them are still bounded, and message queues remain a -real missing area. diff --git a/local/docs/archived/SCRIPT-BEHAVIOR-MATRIX.md b/local/docs/archived/SCRIPT-BEHAVIOR-MATRIX.md deleted file mode 100644 index 28c6696628..0000000000 --- a/local/docs/archived/SCRIPT-BEHAVIOR-MATRIX.md +++ /dev/null @@ -1,114 +0,0 @@ -# Red Bear OS Script Behavior Matrix - -## Purpose - -This document centralizes what the main repository scripts do and do not handle under the Red Bear -release fork model. - -The goal is to remove guesswork from the sync/fetch/apply/build workflow. - -## Matrix - -| Script | Primary role | What it handles | What it does **not** guarantee | -|---|---|---|---| -| `local/scripts/provision-release.sh` | Refresh top-level upstream repo state | fetches upstream, reports conflict risk, rebases repo commits, reapplies build-system release fork via `apply-patches.sh` | does not automatically solve every subsystem release fork conflict; does not by itself make upstream WIP recipes safe shipping inputs | -| `local/scripts/apply-patches.sh` | Reapply durable Red Bear release fork | applies build-system patches, relinks recipe patch symlinks, relinks local recipe release fork into `recipes/` | does not fully rebase stale patch carriers; does not validate runtime behavior; does not decide WIP ownership for you | -| `local/scripts/build-redbear.sh` | Build Red Bear profiles from upstream base + local release fork | applies release fork, builds cookbook if needed, validates profile naming, launches the actual image build; only allows upstream recipe immutable archived when passed `--upstream` | does not guarantee every nested upstream source tree is fresh; does not replace explicit subsystem/runtime validation | -| `scripts/fetch-all-sources.sh` | Fetch mainline recipe source inputs for builds | downloads mainline/upstream recipe sources, reports status/preflight, and supports config-scoped fetches while leaving local release fork in place | does not mean fetched upstream WIP source is the durable shipping source of truth | -| `local/scripts/fetch-sources.sh` | Fetch mainline recipe sources for browsing and patching | when passed `--upstream`, fetches `recipes/*` source trees so the upstream-managed side is locally available for reading, editing, and patch preparation | does not decide whether upstream should replace the local release fork | -| `local/scripts/build-redbear-wifictl-redox.sh` | Build `redbear-wifictl` for the Redox target with the repo toolchain | prepends `prefix/x86_64-unknown-redox/sysroot/bin` to `PATH` and runs `cargo build --target x86_64-unknown-redox` in the `redbear-wifictl` crate | does not prove runtime Wi-Fi behavior; only closes the target-build environment gap for this crate | -| `local/scripts/test-iwlwifi-driver-runtime.sh` | Exercise the bounded Intel driver lifecycle inside a target runtime | validates bounded probe/prepare/init/activate/scan/connect/disconnect/retry surfaces for `redbear-iwlwifi` on a live target runtime | does not prove real AP association, packet flow, DHCP success over Wi-Fi, or end-to-end connectivity | -| `local/scripts/test-wifi-control-runtime.sh` | Exercise the bounded Wi-Fi control/profile lifecycle inside a target runtime | validates `/scheme/wifictl` control nodes, bounded connect/disconnect behavior, and profile-manager/runtime reporting surfaces on a live target runtime | does not prove real AP association or end-to-end connectivity | -| `local/scripts/test-wifi-baremetal-runtime.sh` | Exercise bounded Intel Wi-Fi runtime lifecycle on a target system | validates driver probe, control probe, bounded connect/disconnect, profile-manager start/stop via the `wifi-open-bounded` profile, Wi-Fi lifecycle reporting, and writes `/tmp/redbear-phase5-wifi-capture.json` on the target | does not prove real AP association, packet flow, DHCP success over Wi-Fi, or end-to-end hardware connectivity | -| `local/scripts/test-wifi-passthrough-qemu.sh` | Launch Red Bear with VFIO-passed Intel Wi-Fi hardware | boots a Red Bear guest with a passed-through Intel Wi-Fi PCI function, auto-runs the in-guest bounded Wi-Fi validation command, and can copy the packaged capture bundle back to a host-side file during `--check` | depends on host VFIO setup and still does not by itself guarantee real AP association or end-to-end Wi-Fi connectivity | -| `local/scripts/test-bluetooth-runtime.sh` | Compatibility guest entrypoint for the bounded Bluetooth Battery Level slice | runs the packaged `redbear-bluetooth-battery-check` helper inside a Redox guest or target runtime | does not run on the host and does not expand the Bluetooth support claim beyond the packaged checker’s bounded scope | -| `local/scripts/test-bluetooth-qemu.sh` | Launch or validate the bounded Bluetooth Battery Level slice in QEMU | boots `redbear-bluetooth-experimental`, auto-runs the packaged checker during `--check`, reruns it in one boot, and reruns it again after a clean reboot | does not by itself guarantee that the current QEMU proof passes; does not prove real controller bring-up, generic BLE/GATT maturity, write/notify support, or real hardware Bluetooth behavior | -| `local/scripts/test-drm-display-runtime.sh` | Run the bounded DRM/KMS display checker in a target runtime | invokes the packaged `redbear-drm-display-check` helper for AMD or Intel, proving scheme/card reachability, connector/mode enumeration, and bounded direct modeset proof over the Red Bear DRM ioctl surface when requested | does not prove render command submission, fence semantics, or hardware rendering | -| `local/scripts/test-amd-gpu.sh` | AMD wrapper for the bounded DRM/KMS display checker | runs `test-drm-display-runtime.sh --vendor amd` | still only display-path evidence | -| `local/scripts/test-intel-gpu.sh` | Intel wrapper for the bounded DRM/KMS display checker | runs `test-drm-display-runtime.sh --vendor intel` | still only display-path evidence | -| `local/scripts/test-msix-qemu.sh` | Bounded MSI-X proof in QEMU | validates that the current virtio-net guest path reaches MSI-X-capable interrupt delivery and emits normalized `IRQ_DRIVER`, `IRQ_MODE`, `IRQ_REASON`, and `IRQ_LOG` output for the bounded guest/runtime proof | does not prove broad hardware MSI-X reliability or per-device fallback behavior outside the bounded guest path | -| `local/scripts/test-iommu-qemu.sh` | Bounded IOMMU first-use proof in QEMU | validates guest-visible AMD-Vi initialization and bounded event/drain behavior through the current `iommu` runtime path | does not prove real-hardware interrupt remapping quality or full DMA-remapping correctness | -| `local/scripts/test-xhci-irq-qemu.sh` | Bounded xHCI interrupt-mode proof in QEMU | validates that the xHCI guest path reaches an interrupt-driven mode under the current bounded runtime checker and emits normalized `IRQ_DRIVER`, `IRQ_MODE`, `IRQ_REASON`, and `IRQ_LOG` output | does not prove full USB topology maturity or broad hardware interrupt robustness | -| `local/scripts/test-lowlevel-controllers-qemu.sh` | Aggregate bounded low-level controller proof wrapper | runs MSI-X, xHCI IRQ, IOMMU first-use, PS/2/serio, and monotonic timer proofs in one sequence, defaulting to `redbear-mini` while automatically upgrading only the IOMMU leg to `redbear-full` because that runtime currently ships `/usr/bin/iommu`; if the required `redbear-full` image is absent, that single IOMMU leg is explicitly skipped rather than aborting the rest of the bounded wrapper | does not replace the individual proof helpers and does not prove real-hardware controller quality | -| `local/scripts/prepare-wifi-vfio.sh` | Prepare or restore an Intel Wi-Fi PCI function for passthrough | binds a chosen PCI function to `vfio-pci` or restores it to a specified host driver | does not verify guest Wi-Fi functionality and must be used carefully on a host with a safe detachable target device | -| `local/scripts/validate-wifi-vfio-host.sh` | Check whether a host looks ready for Wi-Fi VFIO testing | validates PCI presence, current driver, UEFI firmware, Red Bear image presence, QEMU/expect availability, VFIO module state, and IOMMU group visibility; exits non-zero when blockers are found | does not bind devices or prove the guest Wi-Fi stack works | -| `local/scripts/run-wifi-passthrough-validation.sh` | End-to-end host-side passthrough validation wrapper | prepares VFIO, runs the packaged in-guest Wi-Fi validation path, captures the guest JSON artifact to the host, writes a host-side metadata sidecar, and restores the host driver afterwards | still depends on real VFIO/hardware support and does not itself guarantee end-to-end Wi-Fi connectivity | -| `local/scripts/package-wifi-validation-artifacts.sh` | Bundle Wi-Fi validation evidence into one archive | packages common capture/log artifacts from bare-metal or VFIO validation runs into a single tarball | does not create missing artifacts or validate their contents | -| `local/scripts/summarize-wifi-validation-artifacts.sh` | Summarize Wi-Fi validation evidence quickly | extracts key runtime signals from a capture JSON or packaged tarball for fast triage | does not replace full artifact review or prove runtime correctness | -| `local/scripts/finalize-wifi-validation-run.sh` | One-shot post-run Wi-Fi triage helper | runs the packaged analyzer on a capture JSON and then packages the chosen artifacts into a tarball | still depends on a real target run having produced the capture/artifacts first | - -The packaged companion command for those scripts is `redbear-phase5-wifi-check`, which performs the -bounded in-target Wi-Fi lifecycle checks from inside the guest/runtime itself. - -The packaged Bluetooth companion command is `redbear-bluetooth-battery-check`, which is intended to -perform the bounded Bluetooth Battery Level checks from inside the guest/runtime itself, including -repeated helper runs, daemon-restart coverage, failure-path honesty checks, and stale-state cleanup -checks within the current slice boundary. - -The packaged DRM display companion command is `redbear-drm-display-check`, which is intended to -perform bounded DRM/KMS display-side checks from inside the guest/runtime itself. It now covers -direct connector/mode enumeration and bounded direct modeset proof over the Red Bear DRM ioctl -surface, and explicitly does not claim render or hardware-accelerated graphics completion. - -The packaged evidence companion is `redbear-phase5-wifi-capture`, which collects the bounded driver, -control, profile-manager, reporting, interface-listing, and scheme-state surfaces — plus `lspci` -and active-profile contents — into a single JSON artifact. - -The packaged link-oriented companion is `redbear-phase5-wifi-link-check`, which focuses on whether -the target runtime is exposing interface/address/default-route signals in addition to the bounded -Wi-Fi lifecycle state. - -For Redox-target Rust builds of Wi-Fi components such as `redbear-wifictl`, a missing -`x86_64-unknown-redox-gcc` on `PATH` should first be treated as a host toolchain/path issue if the -repo already contains `prefix/x86_64-unknown-redox/sysroot/bin/x86_64-unknown-redox-gcc`. - -## Policy Mapping - -### Resilience / offline-first package sourcing - -Default Red Bear behavior is local-first: - -- use locally available package/source trees and release fork state for normal builds, -- treat upstream immutable archived as an explicit operator action only (`--upstream`, dedicated fetch/sync), -- do not fail policy-level expectations just because upstream network access is temporarily broken. - -This is required so builds and recovery workflows remain operable during upstream outages or -connectivity failures. - -### Upstream sync - -Use `local/scripts/provision-release.sh` when the goal is to immutable archived the top-level upstream Redox base. - -This is a repository sync operation, not a guarantee that every local subsystem release fork is already -rebased cleanly. - -### Overlay reapplication - -Use `local/scripts/apply-patches.sh` when the goal is to reconstruct Red Bear’s release fork on top of a -fresh upstream tree. - -This is the core durable-state recovery path. - -### Build execution - -Use `local/scripts/build-redbear.sh` when the goal is to build a tracked Red Bear profile from the -current upstream base plus local release fork. Add `--upstream` only when you explicitly want Redox/upstream -recipe sources immutable archived during that build. - -### Source immutable archived - -Use `scripts/fetch-all-sources.sh` and `local/scripts/fetch-sources.sh --upstream` when the goal is to -immutable archived recipe source inputs, but do not confuse fetched upstream WIP source with a trusted shipping -source. - -## WIP Rule in Script Terms - -If a subsystem is still upstream WIP, the scripts should be interpreted this way: - -- fetching upstream WIP source is allowed and useful through the explicit upstream fetch commands or - `--upstream` where a wrapper requires it, -- syncing upstream WIP source is allowed and useful through the explicit upstream sync command, -- but shipping decisions should still prefer the local release fork until upstream promotion and reevaluation happen. - -That means “script fetched it successfully” is not the same as “Red Bear should now ship upstream’s -WIP version directly.” diff --git a/local/docs/archived/USB-IMPLEMENTATION-PLAN-v1-2026-04.md b/local/docs/archived/USB-IMPLEMENTATION-PLAN-v1-2026-04.md deleted file mode 100644 index d7bc3fba86..0000000000 --- a/local/docs/archived/USB-IMPLEMENTATION-PLAN-v1-2026-04.md +++ /dev/null @@ -1,736 +0,0 @@ -# Red Bear OS USB Implementation Plan - -## Purpose - -This document defines the current state, completeness, and implementation path for USB in Red Bear -OS. It distinguishes between the **upstream source** (unpatched) and the **Red Bear state** (after -applying `local/patches/base/redox.patch`). - -The goal is to describe USB in terms of **what is built**, **what is patched**, **what is -actually usable**, and **what still needs to be implemented** before Red Bear can honestly claim a -modern, future-proof USB stack. - -This document is Red Bear-specific. It uses current repo evidence from code, configs, runtime -tooling, and status docs instead of assuming inherited upstream documentation is fully current. - -## Validation States - -- **builds** — code exists in-tree and is expected to compile -- **enumerates** — runtime surfaces can discover controllers, ports, or descriptors -- **usable** — a specific controller/class path works in a limited real scenario -- **validated** — behavior has been exercised with explicit evidence for the claimed scope -- **experimental** — available for bring-up, but not support-promised - -This repo should not treat **builds** or **enumerates** as equivalent to **validated**. - -## Source Model - -USB driver code lives in `recipes/core/base/source/drivers/usb/`, which is an upstream-managed git -working copy. Red Bear carries all USB modifications through `local/patches/base/redox.patch` -(currently ~17000 lines across ~100 diff sections). - -**Upstream state** — the unpatched source snapshot that `make fetch` produces — has significant -error handling gaps and several correctness bugs. Red Bear's patch layer fixes these, but the fixes -are only visible after patch application. This document describes the **Red Bear state** unless -explicitly noted. - -## Current Repo State - -### Summary - -USB in Red Bear OS is **present and improving**. - -The Red Bear USB stack consists of: - -- a host-side xHCI controller daemon (`xhcid`) with Red Bear patches for error handling, - correctness, and robustness -- hub and HID class daemons with Red Bear patches -- a mass-storage BOT daemon with Red Bear patches -- native USB observability (`lsusb`, `usbctl`, `redbear-info`) -- a low-level userspace client API through `xhcid_interface` -- a hardware quirks system that applies USB device-specific workarounds at runtime -- four QEMU validation harnesses covering interrupt delivery, bounded device lifecycle hotplug, - full stack, and storage autospawn -- an in-guest scheme-tree checker (`redbear-usb-check`) - -### Boot-input reality - -For bare-metal boot resilience, the current USB stack is still incomplete. - -External USB keyboard input is reliably available only when the keyboard is reached through the -`xHCI -> usbhubd/usbhidd -> inputd` path. This is an important distinction because modern bare -metal does not guarantee that an attached keyboard will land on the xHCI runtime path. - -If a keyboard instead lands on: - -- an EHCI-owned path -- a UHCI/OHCI companion path -- a firmware routing topology where low/full-speed devices do not reach the xHCI runtime path - -then Red Bear may still detect controller ownership and connected ports, but it does not yet have a -complete runtime host path that reaches the existing HID class daemons. - -This means Red Bear cannot yet honestly claim that an external USB keyboard is a reliable universal -boot fallback on bare metal. - -### Red Bear xHCI Patch Layer - -The Red Bear patch at `local/patches/base/redox.patch` carries these changes over the upstream -source: - -**Error handling (88 fixes):** -- `unwrap()` on mutex locks replaced with `unwrap_or_else(|e| e.into_inner())` across `scheme.rs`, - `mod.rs`, `irq_reactor.rs`, and `ring.rs` — mutex poisoning no longer panics any hot-path lock -- `expect()` calls replaced with proper `Result` propagation, logged errors, or fallible helpers -- `trb_phys_ptr()` returns `Result` instead of panicking on invalid TRB pointers -- `panic!()` in `irq_reactor.rs` replaced with error returns where possible -- `device_enumerator.rs` panics replaced with error logging and graceful handling - -**Correctness fixes:** -- **ERDP split**: upstream has a single `erdp()` method that conflates the software dequeue pointer - with the hardware register read. Red Bear splits this into `dequeue_ptr()` (software ring - position) and `erdp(&RuntimeRegs)` (actual hardware register read, per XHCI spec §4.9.3) -- **endp_direction off-by-one**: upstream uses `endp_num as usize` to index into the endpoints Vec, - but USB endpoints are 1-indexed. Red Bear uses `endp_num.checked_sub(1)` for correct 0-based - indexing -- **cfg_idx ordering**: upstream sets `port_state.cfg_idx` before validating the config descriptor. - Red Bear moves the assignment after validation succeeds -- **CLEAR_FEATURE endpoint address**: upstream uses the driver-internal endpoint index for - `CLEAR_FEATURE(ENDPOINT_HALT)`. Red Bear uses the USB endpoint address from the descriptor - (`bEndpointAddress`) -- **usbhubd status_change_buf**: upstream has off-by-one bitmap sizing and bit-position parsing. - Red Bear sizes the buffer correctly and computes port bit positions explicitly - -**Functional additions:** -- **Event ring growth**: upstream has a stub `grow_event_ring()` that logs "TODO". Red Bear - implements real ring doubling (up to 4096 cap), new DMA allocation, dequeue pointer preservation, - ERDP/ERSTBA register updates, and DCS bit handling -- **BOS/SuperSpeed descriptor fetching**: `fetch_bos_desc()` called during device enumeration with - bounds-checked slicing and graceful USB 2 fallback -- **Speed detection for hub child devices**: `UsbSpeed` enum with `from_v2_port_status()` / - `from_v3_port_status()` mapping, passed via `attach_with_speed()` from `usbhubd` -- **Interrupt-driven operation restored**: `get_int_method()` replaces hardwired polling; MSI/MSI-X/ - INTx paths re-enabled -- **Hub interrupt EP1**: `usbhubd` reads status change via interrupt endpoint instead of polling -- **USB 3 hub endpoint configuration**: `SET_INTERFACE` always sent; stall on `(0,0)` tolerated -- **Hub change bit clearing**: `clear_port_changes` sends all relevant `ClearFeature` requests - including USB3-specific features after every port status read -- **HID error handling**: `usbhidd` uses `anyhow::Result` with context, no panics in report loop -- **BOT transport robustness**: `usbscsid` replaces all `panic!()` with stall recovery and error - returns; iterative bounded CSW read loop instead of unbounded recursion; correct early_residue - computation - -### Remaining Limitations - -Even with the Red Bear patch applied: - -- HID is now wired through named producers (`ps2-keyboard`, `ps2-mouse`, `usb-{port}-if{n}`); named producers always fan out to both per-device consumers and the legacy VT consumer path; the `InputProducer` wrapper falls back to an anonymous legacy `ProducerHandle` if the named path is unavailable (e.g., older `inputd` build) -- external USB keyboard fallback is not guaranteed on bare metal unless the keyboard reaches the - xHCI runtime path -- EHCI/UHCI/OHCI are not yet full runtime host-controller implementations -- Any remaining USB composite/device-model issues now sit above the bounded helper fixes already - landed for active alternates, endpoint direction, real interface/alternate hub configuration, and - SSP-aware endpoint-context calculations. -- ~57 TODO/FIXME comments remain across xHCI driver files -- usbhubd: interrupt-driven change detection implemented; 1-second polling retained as fallback -- usbscsid: `ReadCapacity16` now implemented with automatic fallback from `ReadCapacity10` -- `usbhidd` keyboard LED sync is only a bounded per-device best-effort path, not a system-global - lock-state authority -- No real hardware USB validation — all testing is QEMU-only -- No hot-plug stress testing -- No USB storage data I/O validation (autospawn checked, but no read/write tested) -- USB quirk table expanded from 8 to 146 entries mined from Linux 7.0 -- USB quirk flags expanded from 9 to 22 (13 new flags from Linux 7.0 including NO_BOS, HUB_SLOW_RESET) -- Terminus hub (0x1A40:0x0101) corrected from `no_lpm` to `hub_slow_reset` per Linux semantics - -### Current Status Matrix - -| Area | State | Notes | -|---|---|---| -| Host mode | **builds / QEMU-validated** | Real host-side stack, interrupt-driven, QEMU-validated only | -| xHCI controller | **builds / QEMU-validated** | Red Bear patch: 88 error handling fixes, ERDP split, endp_direction fix, cfg_idx fix, real grow_event_ring, mutex poison recovery on all hot-path locks; no real hardware validation yet | -| EHCI/UHCI/OHCI | **builds / enumerates** | Ownership, port handling, and logging exist, but they are not yet full runtime enumeration paths | -| Hub handling | **builds / good quality** | `usbhubd`: all `expect()` eliminated, interrupt-driven change detection with polling fallback, graceful per-port error handling | -| HID | **builds / QEMU-validated in narrow path** | `usbhidd` handles keyboard/mouse/button/scroll via named producer path (`usb-{port}-if{n}`) with legacy fallback, no panics in report loop; keyboard LED sync exists as a bounded per-device best-effort path | -| Mass storage | **builds / good quality** | `usbscsid`: typed `ScsiError`, fallible parsing, `ReadCapacity16` for >2TB, stall recovery, resilient event loop | -| Native tooling | **builds / enumerates** | `lsusb`, `usbctl`, `redbear-info`, `redbear-usb-check` provide observability | -| Low-level userspace API | **builds** | `xhcid_interface` with `UsbSpeed` enum, `attach_with_speed()` | -| Validation | **builds / QEMU-only** | 4 harness scripts + in-guest checker; no real hardware validation scripts | -| Hardware quirks | **builds** | `redox-driver-sys` quirk tables with 146 compiled-in USB quirk entries (mined from Linux 7.0) + 22 USB quirk flags; runtime TOML loading for `/etc/quirks.d/` | - -## Code Quality by Daemon - -### xHCI driver (`xhcid/src/xhci/`) - -**Upstream state** — 91 `unwrap()`, 25 `expect()`, 7 `panic!()`, ~57 TODO/FIXME across ~6000 -lines of Rust. - -**Red Bear state** — mutex poisoning eliminated on all hot-path locks; `trb_phys_ptr()` returns -`Result`; critical correctness bugs fixed; ~57 TODOs remain as design notes. - -Key files and their sizes: - -| File | Lines (approx) | Upstream Issues | Red Bear Fix Status | -|---|---|---|---| -| `scheme.rs` | ~2800 | 36 unwrap, 14 expect, 2 panic | All unwrap/expect on hot paths fixed; endp_direction, cfg_idx, CLEAR_FEATURE fixed | -| `mod.rs` | ~1500 | 38 unwrap, 5 expect | All mutex-related unwrap fixed | -| `irq_reactor.rs` | ~750 | 17 unwrap, 6 expect, 4 panic | All fixed; grow_event_ring fully implemented | -| `ring.rs` | ~200 | 1 panic (trb_phys_ptr) | Returns Result instead of panicking | -| `event.rs` | ~60 | 1 TODO | ERDP split into dequeue_ptr() + erdp(&RuntimeRegs) | - -### Class drivers - -| Daemon | Lines | Error Handling Quality | Remaining unwrap/expect | Key Gaps | -|---|---|---|---|---| -| `usbhubd` | ~430 | **Good** — `Result<(), Box>`, all `expect()` eliminated, interrupt-driven change detection | 0 | 1-second polling fallback if interrupt EP unavailable | -| `usbhidd` | 576 | **Good** — `anyhow::Result` with context, no panics in report loop; `expect()` remains in arg parsing and descriptor setup (pre-existing) | 7 `expect()` + 1 `assert_eq!` (pre-existing, arg parsing/descriptor setup) | Hardcoded 1ms poll rate; mouse ×2 multiplier workaround; X scroll missing | -| `usbscsid` | ~1800 | **Good** — `ScsiError` typed errors, fallible `parse_bytes`/`parse_mut_bytes` helpers, resilient event loop, `ReadCapacity16` | 0 | — | - -## Validation Infrastructure - -### Host-side QEMU harnesses - -| Script | What it tests | Limitations | -|---|---|---| -| `test-xhci-device-lifecycle-qemu.sh --check` | Bounded xHCI hotplug lifecycle proof: runtime attach → configure → driver spawn → detach for HID and storage devices | QEMU-only; monitor-driven hotplug; not a broad hardware stress test | -| `test-usb-qemu.sh --check` | Full stack: xHCI interrupt mode, HID spawn, SCSI spawn, bounded sector-0 readback, BOS processing, crash errors | QEMU-only; log-grep based; no guest-side write proof | -| `test-usb-storage-qemu.sh` | USB mass storage autospawn + sector-0 readback + crash patterns | No guest-side write proof yet; no multi-LUN; no UAS | -| `test-xhci-irq-qemu.sh --check` | xHCI interrupt delivery mode (MSI/MSI-X/INTx) | No devices attached during check; single log grep | - -### In-guest tooling - -| Tool | What it does | Installation | -|---|---|---| -| `lsusb` | Walks `/scheme/usb.*`, reads descriptors, shows vendor:product + quirks | Installed via `redbear-hwutils` recipe | -| `redbear-usb-check` | Scheme tree walk with pass/fail exit code | Installed via `redbear-hwutils` recipe | -| `redbear-info --verbose` | Reports USB controller count and integration status | Installed via `redbear-info` recipe | - -### Runbook - -`local/docs/USB-VALIDATION-RUNBOOK.md` documents two operator paths: -- **Path A**: Host-side QEMU validation via `test-usb-qemu.sh --check` -- **Path B**: Interactive guest validation via `redbear-usb-check` - -### What is NOT validated - -- Real hardware USB controllers (QEMU `qemu-xhci` only) -- Hub topology (direct-attached devices only) -- USB 3 SuperSpeed data paths -- Isochronous or streaming transfers -- Hot-plug stress testing -- USB storage data I/O (read/write to block device) -- USB device mode / OTG / USB-C - -## Implementation Plan - -### Repo-fit note - -Some implementation targets live in upstream-managed trees such as -`recipes/core/base/source/...`. In Red Bear, work against those paths is carried through the -appropriate patch carrier under `local/patches/` until intentionally upstreamed. This plan names -the technical target path, not a recommendation to bypass Red Bear's overlay/patch discipline. - -### Phase U0 — Support Model and Scope Freeze - -**Goal**: Make USB claims honest and reproducible before widening implementation scope. - -**What to do**: - -- Define USB support labels per profile: `builds`, `enumerates`, `usable`, `validated` -- Declare Red Bear's near-term USB scope explicitly as **host-first** -- Record that device mode / USB-C / PD / alt-modes / USB4 are later decision points, not implied - current scope -- Add USB status guidance to the profile/support-language discipline used elsewhere in Red Bear - -**Where**: `local/docs/PROFILE-MATRIX.md`, `docs/07-RED-BEAR-OS-IMPLEMENTATION-PLAN.md`, this -document. - -**Exit criteria**: USB claims are tied to a named profile or package-group slice; no doc implies -broad USB support without a matching validation label. - ---- - -### Phase U1 — xHCI Controller Baseline - -**Status**: Substantially complete in the Red Bear patch layer. Runtime validation still QEMU-only. - -**Completed (Red Bear patch)**: -- BOS/SuperSpeed descriptor fetching wired up -- Speed detection for hub child devices with `UsbSpeed` enum -- Interrupt-driven operation restored (MSI/MSI-X/INTx) -- Event ring growth fully implemented (ring doubling, DMA, ERDP/ERSTBA, DCS) -- 88 error handling fixes across scheme.rs, mod.rs, irq_reactor.rs, ring.rs -- ERDP split into `dequeue_ptr()` + `erdp(&RuntimeRegs)` -- `trb_phys_ptr()` returns `Result` -- Mutex poisoning recovery on all hot-path locks - -**Remaining**: -- Validate one controller family on real hardware (requires hardware) -- Tighten controller-state correctness under sustained load (requires hardware) -- Address remaining ~57 TODO/FIXME design notes (ongoing, not blocking) -- SuperSpeedPlus differentiation via Extended Port Status (xHCI spec extension) -- TTT (Think Time) propagation from parent hub descriptor into Slot Context -- Event ring growth: copy pending TRBs from old ring to avoid losing in-flight events under sustained load - -**Where**: `recipes/core/base/source/drivers/usb/xhcid/` (via `local/patches/base/redox.patch`) - -**Exit criteria**: one target controller family repeatedly boots without `xhcid` panic on real -hardware; controller enumerates attached devices reliably across repeated boot cycles. - ---- - -### Phase U2 — Topology, Configuration, and Hotplug Correctness - -**Status**: Partially complete. - -**Completed (Red Bear patch)**: -- USB 3 hub endpoint configuration stall handled -- `endp_direction` off-by-one fixed (`checked_sub(1)`) -- `cfg_idx` assigned after validation -- xHCI lifecycle gating prevents new I/O from entering while a port is detaching -- `attach_device()` no longer leaves a published partially-enumerated `PortState` on attach failure -- `detach_device()` now waits for in-flight lifecycle operations before removing the port state -- `configure_endpoints_once()` is transactional: endpoint state is staged locally, input-context - mutations are snapshotted, and rollback is attempted if `CONFIGURE_ENDPOINT` or - `SET_CONFIGURATION` fails -- `CLEAR_FEATURE` uses correct USB endpoint address from descriptor -- `usbhubd` status_change_buf sizing and bitmap parsing fixed -- Hub interrupt EP1 status change detection replacing polling -- `usbhubd` error handling improved — all ~22 `expect()` eliminated, `Result` return type, graceful per-port failure handling -- `usbhubd` interrupt-driven change detection — reads hub interrupt IN endpoint for status change bitmap; falls back to 1-second polling if endpoint unavailable; initial full scan preserved at startup - -**Remaining**: -- Validate repeated attach/detach/reset behavior under stress (requires real hardware) - -**Completed (Red Bear patch, this session)**: -- `configure_endpoints_once()` now filters endpoints by specific interface+alternate when - `req.interface_desc` is set, enabling composite-device drivers to claim individual interfaces - without programming endpoints from other interfaces -- When `interface_desc` is `None` (initial device setup), endpoints are collected from all - default-alternate (alt 0) interfaces, preserving backward compatibility -- `PortState.active_ifaces: BTreeMap` tracks which interface numbers are active and - which alternate setting each is using -- `set_interface()` now updates `active_ifaces` after a successful SET_INTERFACE control request -- `spawn_drivers()` logs non-default alternates at debug level instead of warning, documenting - that non-default alternates are selected by drivers via SET_INTERFACE rather than auto-spawn -- Initial configuration populates `active_ifaces` with all default-alternate interfaces - -**Where**: `recipes/core/base/source/drivers/usb/usbhubd/`, `xhcid/src/xhci/scheme.rs` - -**Exit criteria**: repeated hub and hotplug scenarios complete without stale topology state; at -least one composite device configures correctly beyond the simplest path. - ---- - -### Phase U3 — HID Modernization - -**Status**: Partially complete. - -**Completed (Red Bear patch)**: -- `usbhidd` error handling improved — `anyhow::Result` with context, no panics in report loop; `expect()`/`assert_eq!` remain in arg parsing and descriptor setup (pre-existing) -- Display write failures logged as warnings instead of panicking -- `inputd` scheme enhancement: named producers (`/scheme/input/producer/{name}`), per-device - consumer streams (`/scheme/input/{device_name}`), hotplug event stream (`/scheme/input/events`), - root directory enumeration (static entries + dynamic device names) -- Named producer events fan out to both matching DeviceConsumers and the legacy VT consumer path -- Hotplug binary format: 16-byte header (kind, device_id, name_len, reserved) + UTF-8 name -- Device IDs allocated monotonically, never reused -- Public API: `NamedProducerHandle`, `DeviceConsumerHandle`, `HotplugHandle`, `InputDeviceLister`, - `InputProducer` (named-first, legacy-fallback convenience wrapper) -- All legacy paths, event payloads, VT behavior, and display/control behavior preserved unchanged -- `ps2d` migrated: two `InputProducer` instances (`ps2-keyboard`, `ps2-mouse`), keyboard events - route to `keyboard_input`, mouse events to `mouse_input`, named-first with legacy fallback -- `usbhidd` migrated: one `InputProducer` per interface instance (`usb-{port}-if{n}`), named-first - with legacy fallback - -**Remaining** (requires downstream consumer/driver migration, not inputd scheme changes): -- Migrate `i2c-hidd` to named producers (still uses legacy `ProducerHandle`) -- Expose hotplug add/remove behavior to downstream consumers via `evdevd` migration - -**Where**: `recipes/core/base/source/drivers/input/usbhidd/`, `inputd/`, -`local/docs/INPUT-SCHEME-ENHANCEMENT.md` - -**Exit criteria**: two independent USB HID devices appear as separate input sources; hot-unplug and -replug do not collapse all USB HID into one anonymous stream. - ---- - -### Phase U4 — Storage, Userspace API, and Class Expansion - -**Status**: Storage quality improved; userspace API story still low-level. - -**Completed (Red Bear patch)**: -- `usbscsid` BOT transport: all `panic!()` replaced with stall recovery and error returns -- Correct endpoint addresses for `CLEAR_FEATURE` and `get_max_lun` -- Iterative bounded CSW read loop -- SCSI block descriptor parsing with bounds checks -- `usbscsid` SCSI layer: `plain::from_bytes().unwrap()` replaced with typed `ScsiError` and fallible `parse_bytes`/`parse_mut_bytes` helpers -- `usbscsid` main.rs: fallible `run()` helper, event loop continues on individual failures -- `ReadCapacity16` implemented with automatic fallback when `ReadCapacity10` returns max LBA (0xFFFFFFFF) -- `usbscsid` now issues bounded `SYNCHRONIZE CACHE(10/16)` commands when the runtime storage quirk - set includes `needs_sync_cache`, using Linux `sd.c` sync-cache behavior as a donor reference for - command selection and tolerant error handling. - -**Remaining** (all require hardware or design decisions): -- Runtime I/O validation: prove stall recovery works under real device I/O (requires hardware) -- Decide whether BOT-only is sufficient short-term or UAS is needed (design decision) -- Bring `libusb` to a runtime-tested state or replace with Red Bear-native API (large scope, deferred) -- Choose the next USB class families explicitly (design decision) - -**Suggested class priority**: storage baseline → generic userspace API → USB networking or -Bluetooth dongle → audio/video only after controller maturity justifies it - -**Where**: `recipes/core/base/source/drivers/storage/usbscsid/`, `recipes/wip/libs/other/libusb/`, -`local/recipes/system/redbear-hwutils/` - -**Exit criteria**: one USB storage path validated on target profile; one coherent userspace USB API -story documented and works in practice; next supported class families named explicitly. - ---- - -### Phase U5 — Modern USB Scope Decision Gate - -**Goal**: Decide whether Red Bear remains a host-only USB system or grows toward a modern USB -platform. - -**What to decide**: -- Host-only versus device mode / gadget support -- Whether OTG / dual-role matters for target hardware -- Whether USB-C / PD / alt-mode policy belongs in Red Bear's target platform story -- Whether USB4 / Thunderbolt-class behavior is in scope or explicitly excluded - -**Why this phase exists**: These are architectural choices, not small driver add-ons. A -future-proof stack cannot leave them implicit forever. - -**Exit criteria**: a written architecture decision exists for included and excluded modern USB -scope. - ---- - -### Phase U6 — Validation Slices and Support Claims - -**Status**: Partially complete. - -**Completed**: -- `test-usb-qemu.sh` — full USB stack validation harness (6 checks) -- `test-usb-storage-qemu.sh` — USB mass storage autospawn check -- `test-xhci-irq-qemu.sh` — xHCI interrupt delivery mode check -- `test-xhci-device-lifecycle-qemu.sh` — bounded xHCI attach/configure/detach hotplug proof -- `USB-VALIDATION-RUNBOOK.md` — operator documentation with Paths A and B -- `redbear-usb-check` — in-guest scheme-tree checker (now installed in image) -- `lsusb` — full USB scheme walk with descriptor parsing and quirks integration -- `redbear-info` — passive USB controller reporting - -**Remaining** (all require hardware): -- Add hardware-matrix coverage for target controllers and class families -- Add USB storage data I/O validation (read/write to block device) -- Add repeated hardware hot-plug stress testing beyond the bounded QEMU lifecycle slice - -**Exit criteria**: at least one profile can honestly claim a validated USB baseline for named -controller/class scope; USB support language in docs matches real test evidence. - -## Support-Language Guidance - -Until U1 through U3 are substantially complete, Red Bear should avoid broad phrases such as: - -- "USB support works" -- "USB storage is supported" -- "USB is complete" - -Prefer language such as: - -- "xHCI host support is present but experimental" -- "USB enumeration and HID-adjacent host paths exist in-tree" -- "USB support remains controller-variable" -- "USB storage support exists in-tree with improved error handling, but is not yet a broad hardware - support claim" -- "USB error handling and correctness carry significant Red Bear patches over upstream; see - `local/patches/base/redox.patch` for details" - -## Linux Kernel USB Data Mining - -### linux-kpi Scope Clarification - -The `linux-kpi` compatibility layer (`local/recipes/drivers/linux-kpi/`) is used **exclusively for -GPU and Wi-Fi drivers** — it provides Linux kernel API headers and Rust FFI implementations for -porting Linux C drivers in those domains to Redox. It does **not** cover USB and contains no USB -headers, USB device ID tables, or USB driver implementations. - -The linux-kpi header inventory (`src/c_headers/`) covers: PCI, DMA, IRQ, firmware, networking -(netdevice, skbuff, ieee80211, nl80211, cfg80211, mac80211), DRM, workqueue, timer, wait, sync, -memory, and related kernel infrastructure — but zero USB content. This is documented globally in -`AGENTS.md` and `local/AGENTS.md`. - -### Linux 7.0 Source Availability - -Linux kernel 7.0 (stable, released 2026-04-13) is extracted at -`build/linux-kernel-cache/linux-7.0/` for USB data mining purposes. This is a build cache, not a -tracked source tree — it can be re-fetched from `cdn.kernel.org` at any time. - -```bash -# Re-fetch if needed: -curl -L -o build/linux-kernel-cache/linux-7.0.tar.xz \ - "https://cdn.kernel.org/pub/linux/kernel/v7.x/linux-7.0.tar.xz" -tar xf build/linux-kernel-cache/linux-7.0.tar.xz -C build/linux-kernel-cache/ -``` - -### Mining Inventory — What Linux 7.0 Contains - -| Data Source | Linux Path | Entries | Lines | Relevance | -|---|---|---|---|---| -| USB device quirks | `drivers/usb/core/quirks.c` | 64 device + 5 AMD-resume + 4 endpoint-ignore | 800 | Directly feed our quirk tables | -| USB quirk flag definitions | `include/linux/usb/quirks.h` | 19 flags | 84 | We have 9 of 19; 10 missing | -| USB storage unusual devices | `drivers/usb/storage/unusual_devs.h` | 323 entries | 2513 | Mass storage device workarounds | -| USB hub driver | `drivers/usb/core/hub.c` | — | 6567 | TT handling, hub descriptor parsing | -| xHCI host driver | `drivers/usb/host/xhci*.c/h` | ~15 files | ~30000 | Controller quirks, TRB handling | -| SCSI disk driver | `drivers/scsi/sd.c` | — | 4467 | SCSI command support tables | -| USB core headers | `include/linux/usb/*.h` | 75 headers | — | ch9.h (descriptors), hcd.h, storage.h, uas.h | - -### Extraction Tool - -`local/scripts/extract-linux-quirks.py` parses Linux kernel source and generates Red Bear TOML -quirk entries. Handles three source formats: -- `drivers/usb/core/quirks.c` → `[[usb_quirk]]` TOML entries (146 entries from Linux 7.0) -- `drivers/usb/storage/unusual_devs.h` → `[[usb_storage_quirk]]` TOML entries (214 entries from Linux 7.0) -- `drivers/pci/quirks.c` → `[[pci_quirk]]` TOML entries (explicit high-confidence handler-body mappings only, requires review) - -USB quirk extraction is direct and does not require review. PCI quirk extraction now emits only -explicit high-confidence handler-body mappings and still requires manual review before committing. - -The extraction script needs extension to also handle `drivers/usb/storage/unusual_devs.h` for mass -storage device entries (323 entries, different macro format `UNUSUAL_DEV`). - -### Flag Gap Analysis - -**Flags we have (22, fully aligned with Linux 7.0):** `NO_STRING_FETCH`, `RESET_DELAY`, `NO_USB3`, -`NO_SET_CONFIG`, `NO_SUSPEND`, `NEED_RESET`, `BAD_DESCRIPTOR`, `NO_LPM`, `NO_U1U2`, -`NO_SET_INTF`, `CONFIG_INTF_STRINGS`, `NO_RESET`, `HONOR_BNUMINTERFACES`, `DEVICE_QUALIFIER`, -`IGNORE_REMOTE_WAKEUP`, `DELAY_CTRL_MSG`, `HUB_SLOW_RESET`, `NO_BOS`, -`SHORT_SET_ADDR_TIMEOUT`, `FORCE_ONE_CONFIG`, `ENDPOINT_IGNORE`, `LINEAR_FRAME_BINTERVAL` - -**All 19 Linux 7.0 USB_QUIRK flags are now covered.** The mapping table below documents the -correspondence for future reference. - -| Linux Flag | Purpose | Impact | Mapping Notes | -|---|---|---|---| -| `USB_QUIRK_RESET_RESUME` | Device can't resume, needs reset instead | High — many devices | Roughly maps to our `NEED_RESET` | -| `USB_QUIRK_NO_SET_INTF` | Device can't handle SetInterface requests | Medium — composite devices | Our `NO_SET_CONFIG` targets SET_CONFIGURATION, not SET_INTERFACE | -| `USB_QUIRK_CONFIG_INTF_STRINGS` | Device can't handle config/interface strings | Low — enumeration robustness | New concept | -| `USB_QUIRK_RESET` | Device can't be reset at all | Medium — prevents crashes on morph devices | No equivalent | -| `USB_QUIRK_HONOR_BNUMINTERFACES` | Wrong interface count in descriptor | Medium — composite devices | New concept | -| `USB_QUIRK_DEVICE_QUALIFIER` | Device can't handle device_qualifier descriptor | Low — skip descriptor fetch | New concept | -| `USB_QUIRK_IGNORE_REMOTE_WAKEUP` | Device generates spurious wakeup | Low — power management | New concept | -| `USB_QUIRK_DELAY_CTRL_MSG` | Device needs pause after every control message | Medium — prevents timeouts | New concept | -| `USB_QUIRK_HUB_SLOW_RESET` | Hub needs extra delay after port reset | High — our Terminus hub entry (0x1A40:0x0101) currently has `no_lpm` but Linux marks it `HUB_SLOW_RESET` | New concept | -| `USB_QUIRK_NO_BOS` | Skip BOS descriptor (hangs at SuperSpeedPlus) | High — we added BOS fetching, some devices hang | New concept | -| `USB_QUIRK_SHORT_SET_ADDRESS_REQ_TIMEOUT` | Short timeout for SET_ADDRESS | Low — controller-specific | New concept | -| `USB_QUIRK_FORCE_ONE_CONFIG` | Device claims zero configs, force to 1 | Low — edge case | New concept | -| `USB_QUIRK_ENDPOINT_IGNORE` | Device has endpoints that should be ignored | Medium — audio devices | New concept | -| `USB_QUIRK_LINEAR_FRAME_INTR_BINTERVAL` | bInterval is linear frames, not exponential | Low — interrupt endpoint timing | Related to our `BAD_DESCRIPTOR` | - -Note: Some Linux flags overlap semantically with our existing flags. The exact mapping requires a -per-flag design decision — either extend existing flags with clarified semantics or add new parallel -flags. - -### Duplicate Quirk Table Problem - -`xhcid` carries its own copy of the USB quirk table at -`recipes/core/base/source/drivers/usb/xhcid/src/usb_quirks.rs`. The canonical table is in -`local/recipes/drivers/redox-driver-sys/source/src/quirks/usb_table.rs`. - -Both tables now carry the expanded 22-flag set and synchronized entries. The xhcid copy contains a -representative subset of the most common entries (early-boot fallback when `/etc/quirks.d/` is not -yet mounted), while the full 146-entry table and TOML runtime loading serve as the complete -runtime source. - -**Long-term resolution:** xhcid should import from redox-driver-sys directly rather than -maintaining a duplicate. Until then, both must be kept in sync when adding new entries. - -### Prioritized Mining Targets - -**Tier 1 — COMPLETED:** - -1. ✅ **USB device quirk table expansion** — All 146 entries from Linux 7.0 `quirks.c` extracted - into `usb_table.rs` and `20-usb.toml`. Covers HP, Microsoft, Logitech, Lenovo, SanDisk, - Corsair, Realtek, NVIDIA, ASUS, Dell, Elan, Genesys, Razer, and others. - -2. ✅ **`USB_QUIRK_NO_BOS` flag** — Added. 4 devices that hang at SuperSpeedPlus BOS fetch are - now flagged: ASUS TUF 4K PRO (0x0B05:0x1AB9), Avermedia GC553G2 (0x07CA:0x2553), Elgato 4K X - (0x0FD9:0x009B), UGREEN 35871 (0x2B89:0x5871), ezcap401 (0x32ED:0x0401). - -3. ✅ **`USB_QUIRK_HUB_SLOW_RESET` flag** — Added. Terminus hub (0x1A40:0x0101) corrected from - `no_lpm` to `hub_slow_reset`. - -4. ✅ **Flag gap closed** — All 19 Linux 7.0 USB_QUIRK flags now mapped. 13 new flags added: - `NO_SET_INTF`, `CONFIG_INTF_STRINGS`, `NO_RESET`, `HONOR_BNUMINTERFACES`, - `DEVICE_QUALIFIER`, `IGNORE_REMOTE_WAKEUP`, `DELAY_CTRL_MSG`, `HUB_SLOW_RESET`, `NO_BOS`, - `SHORT_SET_ADDR_TIMEOUT`, `FORCE_ONE_CONFIG`, `ENDPOINT_IGNORE`, `LINEAR_FRAME_BINTERVAL`. - -5. ✅ **Duplicate quirk tables synchronized** — Both `usb_table.rs` (redox-driver-sys) and - `usb_quirks.rs` (xhcid) now carry the expanded flag set and synchronized entries. - -6. ✅ **USB storage unusual_devs.h** — 214 entries extracted from Linux 7.0 into - `local/recipes/system/redbear-quirks/source/quirks.d/30-storage.toml` (1716 lines). Extraction - script extended to handle `UNUSUAL_DEV` macro format. Most common flags: `ignore_residue` (46), - `fix_capacity` (34), `single_lun` (28), `max_sectors_64` (22), `fix_inquiry` (22). Includes - `initial_read10` entries for Feiya SD/SDHC reader and Corsair Padlock v2. - -7. ✅ **usbscsid storage quirk integration** — Storage quirks are now active at runtime. - `usbscsid/src/quirks.rs` reads `[[usb_storage_quirk]]` entries from `/etc/quirks.d/*.toml` - and applies them to the BOT transport and SCSI command layers. Active behavioral flags: - - `IGNORE_RESIDUE`: suppresses CSW residue in BOT `send_command` - - `FIX_CAPACITY`: adjusts block count from READ CAPACITY(10) by -1 - - `SINGLE_LUN`: enforces LUN=0 in CBW (future-proof for multi-LUN support) - - `MAX_SECTORS_64`: clamps transfer length to 64 sectors in SCSI read/write - - `INITIAL_READ10`: uses READ(10)/WRITE(10) instead of READ(16)/WRITE(16) - Vendor/product IDs are extracted from `DevDesc` at daemon startup. A compiled-in fallback - table covers 5 common devices for early-boot correctness. - -8. ✅ **xhcid USB device quirk consumption** — xhcid now stores per-device `UsbQuirkFlags` in - `PortState` and applies them during enumeration and runtime requests. Active behavioral flags: - - `NO_STRING_FETCH`: skips manufacturer/product/serial/configuration string fetches - - `BAD_DESCRIPTOR`: tolerates language/string descriptor fetch failures and continues interface parsing when malformed endpoint descriptors appear - - `RESET_DELAY`: extends first-touch post-reset settle time via early `PortId`-based lookup - - `HUB_SLOW_RESET`: uses a longer hub-oriented reset settle time via early `PortId`-based lookup - - `NO_BOS`: skips BOS descriptor fetch and leaves superspeed capability detection false - - `SHORT_SET_ADDR_TIMEOUT`: uses a shorter `Address Device` command timeout via early `PortId`-based lookup - - `FORCE_ONE_CONFIG`: limits enumeration to configuration index 0 (configuration value 1 path) - - `HONOR_BNUMINTERFACES`: stops interface parsing at `bNumInterfaces` - - `DELAY_CTRL_MSG`: inserts a short post-control-transfer delay - - `NO_SET_CONFIG`: skips `SET_CONFIGURATION` - - `NO_SET_INTF`: skips `SET_INTERFACE` - - `NEED_RESET`: issues xHC `Reset Device` automatically after transfer failures - The early-enumeration timing path now uses optional TOML `port = "[....]"` - selectors in `[[usb_quirk]]` entries for quirks that must act before vendor/product are known. - -9. ✅ **xhcid suspend/resume API skeleton** — xhcid now exposes explicit `port/suspend` and - `port/resume` endpoints plus matching `XhciClientHandle::{suspend_device,resume_device}` - helpers. `PortState` now tracks `PortPmState::{Active,Suspended}` and xhcid enforces - `NO_SUSPEND` by rejecting suspend with `EOPNOTSUPP`. While suspended, control/data/reset - activity returns `EBUSY`. - -10. ✅ **usbhubd suspend coordination slice** — `usbhubd` now tracks downstream child suspend - state and mirrors USB 2 hub-port suspend status into child xhcid devices via - `suspend_device()` / `resume_device()`. This gives us the first real cross-layer coordination - path for hub-attached devices without inventing a separate PM daemon. Remaining gap: suspend - policy/origination is still external, and USB 3 link-state-driven coordination is not yet - implemented. - -**Tier 2 — Medium-term (improves robustness):** - -5. **TT handling from hub.c** — Linux's hub driver reads `wHubDelay` and `bNbrPorts` from hub - descriptors to populate TT think time and MTT capability. Our xHCI driver hardcodes `ttt = 0` - and `mtt = false`. Mining the hub descriptor parsing logic from `hub.c` would replace these - stubs with correct values. - -6. **xHCI controller quirks from xhci-pci.c** — Linux has per-vendor controller workarounds - (Intel PCH, AMD, Etron, Fresco, VIA). Our driver has no controller-specific paths. Mining the - quirk table and applying it through our existing PCI quirk system would add real-hardware - robustness. - - 7. **SCSI command selection from sd.c** — READ(10)/WRITE(10) support is now implemented - (triggered by `INITIAL_READ10` quirk flag). Remaining: REPORT LUNS for multi-LUN devices, - SYNCHRONIZE CACHE (triggered by `NEEDS_SYNC_CACHE` flag), and START STOP UNIT for power - management. - -**Tier 3 — Future (enables new device classes):** - -8. **USB class/subclass/protocol tables from ch9.h** — Complete class code definitions for device - matching in `drivers.toml`. - -9. **USB endpoint descriptor parsing from message.c** — Extended endpoint type mapping for streams - and isochronous support. - -### Mining into the Build - -The Linux kernel source at `build/linux-kernel-cache/` is a build cache, not a tracked dependency. -Mined data must be materialized into durable locations: - -| Mined Data | Target Location | Format | -|---|---|---| -| USB device quirks | `local/recipes/system/redbear-quirks/source/quirks.d/20-usb.toml` | TOML (146 entries ✅) | -| USB compiled-in quirks | `local/recipes/drivers/redox-driver-sys/source/src/quirks/usb_table.rs` | Rust (146 entries ✅) | -| PCI controller quirks | `local/recipes/system/redbear-quirks/source/quirks.d/10-pci.toml` | TOML | -| Storage device flags | `local/recipes/system/redbear-quirks/source/quirks.d/30-storage.toml` | TOML (214 entries ✅, active at runtime ✅) | -| Flag definitions | `local/recipes/drivers/redox-driver-sys/source/src/quirks/mod.rs` | Rust bitflags (22 USB flags ✅) | - -The extraction script at `local/scripts/extract-linux-quirks.py` should be extended to also handle -`drivers/usb/storage/unusual_devs.h` for mass storage device entries. - -## Summary - -USB in Red Bear today is not missing. It is a real userspace host-side subsystem with meaningful -enumeration, runtime observability, hub/HID infrastructure, and a low-level userspace API. - -The Red Bear patch layer carries substantial error handling and correctness improvements over the -upstream source: 88 error handling fixes (mutex poisoning recovery, expect/panic replacement, Result -conversions), multiple correctness bug fixes, real event ring growth, -class driver error handling improvements (all three USB class daemons now use `Result` types with -zero `unwrap()`/`expect()` panics), interrupt-driven hub change detection, `ReadCapacity16` -for large disk support, and a USB quirk table expanded from 8 to 146 entries with 22 quirk flags -mined from Linux 7.0. - -Recent bounded maturity progress: - -- `xhcid` now tracks active alternate settings per interface in `PortState` and resolves endpoint - descriptors through that active-alternate map instead of flattening all interface descriptors - indiscriminately. -- Direct unit coverage now exists for both default-alternate endpoint selection and - alternate-setting-aware endpoint remapping. -- `xhcid` now also preserves previously selected alternates on the same configuration and applies a - requested interface/alternate override before endpoint planning, so alternate-setting - reconfiguration no longer silently falls back to all-zero defaults. -- `xhcid` endpoint-direction lookup now also follows the active interface/alternate selection state - instead of reading from the first configuration/interface pair unconditionally. -- `xhcid` driver spawning now also follows the selected configuration and active alternate map - instead of hardcoding the first configuration and ignoring non-zero alternates. -- `xhcid` now keeps per-port lifecycle state so detach blocks new transfer/configure/suspend/resume - work, waits for in-flight operations to drain, and removes the published port state only after - slot disable succeeds. -- `xhcid` endpoint configuration is now transactional: software endpoint bookkeeping stays staged - until `CONFIGURE_ENDPOINT` and optional `SET_CONFIGURATION` succeed, and the input context is - restored with an explicit rollback attempt on failure. -- the xHCI IRQ reactor now replaces the old `TODO: grow event ring` stub with a preserve-and-grow - path that copies unread event TRBs into a larger event ring and reprograms ERST registers instead - of dropping pending events during `EventRingFull` recovery. -- `usbhubd` now derives USB 2 hub TT Think Time from the hub descriptor using the same bounded - Linux-compatible encoding and passes it through `ConfigureEndpointsReq`, and `xhcid` now writes - that value into the Slot Context TT information bits for hub devices. -- xHCI endpoint-context calculations are now protocol-speed-aware for SuperSpeedPlus, so interval - and ESIT-payload selection use the resolved port protocol speed instead of relying only on - endpoint companion presence. - -All validation is QEMU-only. No real hardware USB testing exists. - -The remaining gaps now fall into two categories: - -**Broader architectural work (cross-cutting, not a small bounded USB-only fix):** -- Any remaining USB composite/device-model issues now belong to wider device-model/design cleanup - rather than one more isolated helper patch. -- HID producer modernization: per-device streams via named producers, hotplug add/remove (inputd redesign complete, ps2d and usbhidd migrated) -- Userspace USB API: `libusb` WIP, no coherent native story - -> **See also:** `local/docs/boot-logs/REDBEAR-MINI-BOOT-PS2D-INPUTD-LOG-FIX.md` -> for the 2026-06-30 fix that made `usbhidd` (and its `ps2d` sibling) -> visible in the boot log. With the fix, an operator can distinguish -> "usbhidd dead" (no `@usbhidd:` line during enumeration, OR -> `@ps2d: INFO ps2d: registered producer handle` missing) from -> "usbhidd alive but not enumerated by XHCI" (line present, no -> corresponding consumer event). - -**Hardware-dependent or design decisions:** -- Real hardware validation: no controller tested outside QEMU -- Hot-plug stress testing beyond the new bounded QEMU lifecycle harness -- Storage write validation (bounded sector-0 readback proof now exists in QEMU via `test-usb-storage-qemu.sh`, but guest-side write verification to the USB-backed block device is still open) -- usbhubd 1-second polling fallback (only exercisable with real hub hardware) -- Modern USB scope decision: device mode / USB-C / PD - -Software items are tracked in Phase U1 (xHCI internals) and Phase U2 (configuration/composite). -Architectural and hardware items are tracked in Phase U1 (controller hardware validation), Phase U2 -(hub polling fallback), Phase U3 (HID), Phase U4 (storage/API), Phase U5 (modern USB scope -decision), and Phase U6 (validation). - -Linux kernel USB data mining is documented in the "Linux Kernel USB Data Mining" section above. -Linux 7.0 source is available at `build/linux-kernel-cache/linux-7.0/` with 146 USB device quirks, -22 quirk flags (all 19 Linux USB_QUIRK flags covered), 214 active storage device quirks -consumed at runtime by usbscsid, and extensive xHCI/hub/SCSI reference code ready for extraction. diff --git a/local/docs/archived/USB-IMPLEMENTATION-PLAN-v2-2026-07.md b/local/docs/archived/USB-IMPLEMENTATION-PLAN-v2-2026-07.md deleted file mode 100644 index 281a0a4032..0000000000 --- a/local/docs/archived/USB-IMPLEMENTATION-PLAN-v2-2026-07.md +++ /dev/null @@ -1,631 +0,0 @@ -# Red Bear OS USB Implementation Plan — v2 - -> **Status:** Canonical. Replaces `archived/USB-IMPLEMENTATION-PLAN-v1-2026-04.md`. -> **Date:** 2026-07-07. -> **Supersession reason:** v1 (Apr 2026) overstated several capabilities relative to the -> then-current source. v2 re-audits every daemon against `local/sources/base/` HEAD, -> aligns with Redox 0.x USB HEAD (Jan 2025 – Jun 2026), and reorganizes phases around the -> actual bare-metal correctness gaps. Validation labels are now source-anchored rather than -> patch-anchored. -> -> **Sibling docs:** -> `USB-VALIDATION-RUNBOOK.md` (operator path — restored from `archived/`); the older -> `archived/USB-BOOT-INPUT-PLAN.md` and `archived/XHCID-DEVICE-IMPROVEMENT-PLAN.md` are -> kept as historical reference but are not the planning authority. - ---- - -## 0. Purpose and scope - -This plan is the **single planning authority** for the USB subsystem in Red Bear OS. It -answers four questions honestly: - -1. **What is built?** — every host controller, class driver, scheme, and observability tool - that actually exists in `local/sources/base/` and `local/recipes/drivers/`. Status is - derived from the current source tree, not from prior memory or from patch carriers. -2. **What was patched?** — every durable Red Bear modification, with file paths - (`local/patches/base/P*.patch` for the base module, dedicated local recipes or forks - otherwise). -3. **What is actually usable?** — explicitly distinguishes **builds**, **enumerates**, - **usable (narrow path)**, **validated (QEMU)**, **validated (real hardware)**, and - **experimental**. A label is only ever **validated** if the matching proof has run - on the matching artifact under the matching config. -4. **What is missing?** — the real bare-metal-blockers, the upstream-comparable gaps, the - architectural decisions still deferred, and the durability problems that this plan owns. - -### Validation labels (canonical, do not redefine elsewhere) - -- **builds** — code is in tree and compiles. Not a usability claim. -- **enumerates** — runtime surfaces can discover controllers, ports, descriptors. -- **usable (narrow path)** — one controller family / one class family works in a bounded, - repeatable scenario; other paths are likely broken. -- **validated (QEMU)** — a documented QEMU script passed on the matching recipe, config, and - commit. Reproducible on a Linux x86_64 host. -- **validated (real hardware)** — a named physical controller + class, with a captured log, - on real bare metal. This is what an end user can expect. -- **experimental** — present for bring-up but not in any support-promised path. - -**Honesty rule.** `builds` is **not** equivalent to `usable`. `validated (QEMU)` is **not** -equivalent to `validated (real hardware)`. The plan never mixes these categories. Where -prior text conflated them, this v2 corrects. - -### Plan structure - -| Section | Authority | Updates cadence | -|---|---|---| -| §1 Source audit (controllers, class drivers, schemes, tooling) | ground truth | on every source-tree bump | -| §2 Patch carriers | every durable Red Bear diff | on every patch add/rebase | -| §3 Status matrix (one row per component) | single source of truth for "is it working" | on every status change | -| §4 Upstream divergence: what Redox 0.x USB HEAD has that we have not | required adoption list | on every upstream bump | -| §5 Bare-metal input correctness (boot-time USB keyboard) | the bare-metal failure modes | on every controller or class change | -| §6 Phase P0–P5 (execution order) | who does what next | reviewed monthly | -| §7 Validation inventory and bounded proofs | the proof surfaces | on every script add/break | -| §8 Durability posture | local fork health, patch carriers, archival policy | on every base fork bump | -| §9 Support language | how the rest of Red Bear should describe USB | on every phase change | - ---- - -## 1. Source audit — what is actually in the tree - -Red Bear follows the upstream Redox model: **all USB logic is in userspace** (`drivers/usb/` -plus `local/recipes/drivers/usb-core/`). The kernel exposes `irq:`, `memory:`, `pcid:`, -`event:`, and `scheme:` surfaces that userspace USB daemons consume. There is no kernel USB -host stack, and the v1 phase plan's mention of "kernel MSI/MSI-X plumbing" was a reference -to that surface, not a kernel change. - -### 1.1 Host controllers - -| Daemon | Source | Lines | Reality today | Scheme registered | -|---|---|---|---|---| -| **xhcid** | `local/sources/base/drivers/usb/xhcid/` | ~6000 LoC across 25 files | Builds. Real ring/TRB/context/transfer engine. Polling in production (see §1.6). | `usb._xhci` | -| **ehcid** | `local/recipes/drivers/ehcid/source/src/` | ~1550 LoC (3 files) | Builds. Real MMIO init, frame list, QH/TD, port reset. **No class-driver auto-spawn.** | `usb` | -| **uhcid** | `local/recipes/drivers/uhcid/source/src/main.rs` | 35 LoC | Builds. **Real stub.** Reads PCI BAR4, sleeps forever. No scheme. | — | -| **ohcid** | `local/recipes/drivers/ohcid/source/src/main.rs` | 35 LoC | Builds. **Real stub.** Identical pattern to uhcid. | — | - -**Honesty corrections vs v1:** - -- v1 said *"EHCI/UHCI/OHCI — ownership, port handling, and logging exist, but they are not - yet full runtime enumeration paths"*. For **uhcid** and **ohcid** this is too generous — - they are 35-line stubs that **only read PCI BAR4 and sleep**. They are not even - ownership-grade; the controller is never probed, no port state is published, no error - is logged past init. -- v1 said *"xHCI interrupt-driven operation restored"*. The current source at - `xhcid/src/main.rs:141` hardcodes polling: - ```rust - let (irq_file, interrupt_method) = (None, InterruptMethod::Polling); //get_int_method(&mut pcid_handle); - //TODO: Fix interrupts. - ``` - The `get_int_method` function exists, MSI-X/MSI/INTx branches are written, but the - function is bypassed at runtime. §4 captures the upstream commits that help finish this. - -### 1.2 Class drivers - -| Daemon | Source | LoC | Reality | Notes | -|---|---|---|---|---| -| **usbhubd** | `local/sources/base/drivers/usb/usbhubd/` | 249 | Builds; runs. | Polls port status (1s fallback retained from v1). | -| **usbhidd** | `local/sources/base/drivers/input/usbhidd/` | 576 | Builds; runs. | Named-producer input (`usb-{port}-if{n}`) + legacy VT fallback. | -| **usbscsid** | `local/sources/base/drivers/storage/usbscsid/` | ~1800 | Builds; runs. | BOT/SCSI, `ReadCapacity16`, 3 storage quirk flags active. | -| **usbctl** | `local/sources/base/drivers/usb/usbctl/` | 54 | Builds. CLI only. | Minimal — port/endpoint status query. | -| **ucsid** | `local/sources/base/drivers/usb/ucsid/` | 839 | Builds. | USB-C UCSI topology over ACPI + I2C; `/scheme/ucsi`. | -| **redbear-usbaudiod** | `local/recipes/system/redbear-usbaudiod/` | (small) | Builds; wired in `redbear-mini.toml`. | USB Audio Class 1. | -| **redbear-acmd** | `local/recipes/system/redbear-acmd/` | (small) | Builds; wired via `drivers.d/70-usb-class.toml`. | USB CDC ACM serial. | -| **redbear-ecmd** | `local/recipes/system/redbear-ecmd/` | (small) | Builds; wired via `drivers.d/70-usb-class.toml`. | USB CDC ECM/NCM ethernet. | -| **redbear-btusb** | `local/recipes/drivers/redbear-btusb/` | (small) | Builds. | Bluetooth USB transport — see BLUETOOTH-IMPLEMENTATION-PLAN. | - -### 1.3 USB core library - -| Crate | Source | Notes | -|---|---|---| -| **usb-core** | `local/recipes/drivers/usb-core/source/src/` | 6 files (lib.rs, dma.rs, scheme.rs, spawn.rs, transfer.rs, types.rs). Provides `UsbHostController` trait, `SetupPacket`, `PortStatus`, `TransferDirection`, `DmaBuffer`, descriptor parsers, `control_transfer`, `spawn_usb_driver`. Used by ehcid. **Currently not used by xhcid, uhcid, or ohcid.** | - -This trait is the most important "infrastructure that already exists" item in this plan: -it is the natural target for uhcid/ohcid runtime enumeration (§6 P0-B2) and for any -future host port — including the xhcid → USB-core path that future xHCI cleanup will allow. - -### 1.4 Tooling and observability - -| Tool | Source | Reality | -|---|---|---| -| `lsusb` | `local/recipes/system/redbear-hwutils/source/src/bin/lsusb.rs` | Walks `/scheme/usb.*`, reads descriptors. | -| `redbear-usb-check` | `local/recipes/system/redbear-hwutils/source/src/bin/redbear-usb-check.rs` | In-guest scheme tree validator. | -| `redbear-usb-storage-check` | `local/recipes/system/redbear-hwutils/source/src/bin/redbear-usb-storage-check.rs` | Mass-storage round-trip validator. | -| `usbctl` | `local/sources/base/drivers/usb/usbctl/` | CLI for port/endpoint status. | - -### 1.5 Patch carriers on `local/patches/base/` - -The **durable** Red Bear USB modifications are carried as `local/patches/base/P*.patch` -files. These are applied atomically by the cookbook against the recipe source tree during -fetch+cook. - -| Patch | Size | Purpose | -|---|---|---| -| `P1-xhcid-device-lifecycle.patch` | 2351 lines | Attach publication, transactional configure, bounded detach. | -| `P1-xhcid-port-pm-read-fix.patch` | 942 lines | Port PM state read. | -| `P1-xhcid-uevent-logging.patch` | 20 lines | Uevent audit trail. | -| `P2-usb-pm-and-drivers.patch` | 158 lines | USB PM (suspend/resume/quirk integration). | -| `P3-xhci-device-hardening.patch` | 1193 lines | Endp direction, cfg_idx ordering, interrupt-EP, hub feature clearing. | -| `P3-usbhidd-hardening.patch` | 725 lines | HID panic removal, named producer wiring. | -| `P4-initfs-usb-drm-services.patch` | 22 lines | DRM/USB service ordering in init. | -| (sibling) `P0-inputd-named-producers.patch`, `P0-inputd-per-device-consumers.patch`, `P2-inputd.patch`, `P3-inputd-keymap-bridge.patch` | (varying) | Input multiplexer wiring (ps2d + usbhidd consumers). | - -**Durability rule:** any source-tree edit must be mirrored into one of these patches (or -into the local `base` fork's `submodule/base` branch on `RedBear-OS`) before the session -ends. This rule is also enforced by `local/AGENTS.md` and the cookbook's atomic patch -applier. **The current local fork at `local/sources/base/` is a single mega-commit** — -see §8 for the durability problem and remediation. - -### 1.6 The interrupt-vs-polling contradiction - -`local/sources/base/drivers/usb/xhcid/src/main.rs:101–115` defines a complete -`get_int_method()` that returns MSI-X, MSI, INTx, or Polling based on PCI capabilities. -`main.rs:141` then **disables it**: - -```rust -let (irq_file, interrupt_method) = (None, InterruptMethod::Polling); //get_int_method(&mut pcid_handle); -//TODO: Fix interrupts. -``` - -`xhci::start_irq_reactor(&hci, irq_file);` is called with `irq_file = None`, which makes -the reactor a **bounded polling loop that wakes every 1 second** (see the `mod.rs` reactor -fallback). This is functionally "polling in production." - -The v1 plan called this "interrupt-driven operation restored" — that is incorrect relative -to the live code. v2 makes the gap explicit: **interrupts remain to be re-enabled** as P0-A1. - ---- - -## 2. Status matrix (single source of truth) - -Reorganized around the *honest* state of the tree. - -| Component | State today | Maturity | Open correctness gap | -|---|---|---|---| -| Host mode (any controller) | builds / QEMU-validated narrow path | `usable (narrow path)` | see §4 | -| **xhcid** runtime | builds / polling / QEMU-validated | `usable (narrow path)` | interrupts hardcoded off; missing CSZ; missing real-hardware reset fix; missing USB 3.x packet-size + hub fixes | -| **ehcid** runtime | builds / no auto-spawn | `builds` | no class driver dispatch; no full bot pipeline through `/scheme/usb`; ~no peer review | -| **uhcid** runtime | builds / does nothing | `builds` | stub (35 lines) | -| **ohcid** runtime | builds / does nothing | `builds` | stub (35 lines) | -| Hub | builds / good quality | `usable (narrow path)` | polling fallback retained | -| HID class | builds / QEMU-validated narrow path | `usable (narrow path)` | named producer wiring complete; legacy VT fallback preserved | -| Mass storage | builds / QEMU-validated narrow path | `usable (narrow path)` | no guest-side write proof; no multi-LUN; no UAS | -| Audio class (USB) | builds | `builds` | not exercised in any proof | -| CDC ACM/ECM | builds | `builds` | not exercised in any proof | -| Bluetooth USB transport | builds (transport only) | `builds` | Bluetooth host path remains gappy (see BLUETOOTH-IMPLEMENTATION-PLAN) | -| USB-C / UCSI | builds | `builds` | topology surfaced, no PD/alt-mode | -| Native tooling (`lsusb`, `usbctl`, `redbear-info`, `redbear-usb-check`) | builds | `usable (narrow path)` | no bounded proof scheme validation | -| Quirk table (compiled + TOML) | builds | `validated (QEMU)` — quirk-bypass-only | 146 USB + 214 storage entries, 22 flags | -| Validation harnesses | 5 QEMU scripts | `validated (QEMU)` | no real-hardware matrix | - -If a row says `builds`, **Red Bear does not promise that the component is reachable from a -typed-key-in-the-inputd-pipe to a shell prompt.** That promise is restricted to -`usable (narrow path)` and above, and only for the documented scenario. - ---- - -## 3. Upstream divergence — what Redox 0.x USB HEAD has that Red Bear does not - -This section is required reading before any USB change. It is the input to every phase in -§6. The Redox merge window for USB change runs roughly Jan 2025 – Jun 2026 with two -concentrated bursts (March 2025, Sep–Oct 2025). Red Bear's fork is currently pinned at the -v1 baseline (0.1.0 base snapshot). - -### 3.1 Three high-priority upstream commits Red Bear has not adopted - -| Upstream commit | Why we need it | Where it would land | -|---|---|---| -| **`69a80a6a` — xhci: fix reset procedure on real hardware** | Replaces magic bit numbers with named constants; fixes the HCRST wait loop to read from `usb_cmd` instead of `usb_sts` (the spec says HCRST is in USB_CMD). Without this, `xhcid` can spin or wedge on real controllers. | New patch `local/patches/base/P3-xhci-real-hw-reset.patch` against `xhcid/src/xhci/mod.rs`. | -| **`19570db4` — xhci: support 64-bit contexts (CSZ)** | Makes `Xhci` generic over context size (`Xhci` / `Xhci`) with runtime detection via `HCCPARAMS1.CSZ`. Required by modern xHCI controllers (Alder Lake, Raptor Lake, Ryzen 7000+). The local source already has `daemon_with_context_size` and a `//TODO: cleanup CSZ support` comment at the call site — the upstream fix is the natural completion. | New patch `local/patches/base/P3-xhci-csz-64-bit.patch` against `xhcid/src/main.rs` and the downstream context types. | -| **`12e601b3` — xhci: improvements based on real hardware testing** | Adds `USB_CMD_INTE`, corrects port RWC handling, fixes address_device speed passthrough. Companion to `69a80a6a`. | New patch `local/patches/base/P3-xhci-real-hw-impl.patch`. | - -### 3.2 Medium-priority upstream commits - -| Upstream commit | Note | -|---|---| -| `8dcd85b5`, `ba0ca4ce` — Fix packet size for USB 3.0 and USB 1 | Required for SuperSpeed device enumeration. Adopt in same patch as CSZ. | -| `cbbcbc9e`, `f58625b0` — `usbhubd`/`xhcid` fix reading descriptor / port status on USB 3 hubs | Round out the USB 3 hub story. | -| `8f278dcb`, `34b37410` — Bounds check on `root_hub_port_index()` | Stop a panic that we already pay down via patch but have not tested in tree. | -| `4d6581d4` — xhcid: add more timeouts | Prevents infinite hangs on unresponsive controllers. | -| `7e3e841f` — xhci: fix reading EHB flag in received_irq | Companion for interrupt-driven paths. | -| `e3a13a0c` — `xhcid` and friends: use newtype `PortId` to ensure route string | Type-safety win. | -| `6ac41ee` — daemon: tolerate BrokenPipe on ready() | Already in our base fork. | -| `258ea4e6`, `865ca866` — `usbscsid`: use the unified disk scheme implementation | `usbscsid` revision; lower priority, code organization. | -| `e4aab167`, `24c1f0a3` — xhcid: don't exit the event loop when using irqs | Required for stable interrupt-driven operation (pairs with the §1.6 fix). | - -### 3.3 Lower-priority upstream commits to record, not blindly adopt - -| Commit | Note | -|---|---| -| `a5f87735` — ignore alternate settings | Conflicts with our composite-device fix (P3-xhci-device-hardening retains explicit alternate handling). Validate whether dropping this is sound given our active `PortState.active_ifaces` map. | -| `7c980137` — language ID for string descriptors | Likely a clean drop-in. | -| `374e5fbf` — xhci: use redox-scheme v2 | We are on `redox-scheme 0.11`; a v2 migration is not in scope for 0.2.x. | -| `30fb1e7a` — drivers merged into base (Nov 2025) | Mirrors what Red Bear already does (our `local/sources/base/`). No action. | -| USB SCSI driver disabled upstream (Dec 2025) | Red Bear keeps it on with the BOUNDED storage test. Re-evaluate after P2-B1. | -| `bjorn3` enabled xHCI by default in QEMU x86-64 (Mar 2026) | Aligns with our `redbear-mini` boot script. No action. | -| `bjorn3` moved xHCI config to runtime (Apr 2026) | Lower priority — compile-time config is fine for our release model. | -| Antoine Reversat — simplified xhci (May 2026) | Subject to per-line review. | - -### 3.4 Things upstream still does NOT have - -These are explicit non-features from upstream that Red Bear should not silently inherit as -a todo: - -- **USB Type-C / USB-PD / alt-modes.** No policy engine, no protocol stack. -- **USB4 / Thunderbolt.** Listed as "not supported" in upstream `COMMUNITY-HW.md`. -- **xHCI debug capability (DbC).** Not implemented. -- **USB device mode (gadget) / OTG.** No dual-role support. -- **USB isochronous transfers.** `xhcid` returns `ENOSYS` for isoch endpoints. - -These belong to §6 P5 (architectural decision gate), not to "fix the missing patch." - ---- - -## 4. Bare-metal-input correctness (the actual boot-time failure modes) - -The bare-metal USB keyboard problem is not "xhcid doesn't work." xhcid does work in QEMU -and on some real hardware. The failure modes are the **paths that do not reach xhcid**: - -1. **EHCI-attached USB keyboard** — xHCI now owns every USB-3 controller, but - EHCI/companion controllers (UHCI/OHCI) still own low/full-speed devices on chipsets - that firmware routes through them. **ehcid does not auto-spawn class drivers**, so even - though ehcid publishes `/scheme/usb/port/...`, no `usbhidd` is started for any - device on that scheme. The keyboard is reachable by userland but the input pipeline - never builds. - -2. **UHCI/OHCI-attached devices** — uhcid and ohcid are 35-line stubs. The companion - controller is owned (by `pcid`) but no USB traffic flows. There is no port state, no - transfer completion, no scheme. - -3. **xHCI interrupt-driven operation is offline** — line 141 hardcodes polling. On real - hardware with no reliable polling timer, this can produce slow enumeration or input - lag, and on some chips it can wedge the controller (see upstream `69a80a6a`). - -4. **No real-hardware validation matrix** — there is no `hardware-validation.md` table - enumerating which physical controller families have been exercised on bare metal. - QEMU `qemu-xhci` is one fixed emulation target; it is not representative. - -5. **USB HID and ACPI I2C-HID are not the same** — internal laptop keyboards are - I2C-HID (`i2c-hidd`, `intel-thc-hidd`), not USB. These are real but separate. The - I2C-HID plan and the USB HID plan cannot assume one is a substitute for the other. - -6. **Strict-boot mode exists but is not bound** — `uhcid`/`ohcid`/`ehcid` accept - `--strict-boot`, but no initfs entry enables it; the policy lives in operator - knowledge, not in the artifact. - -7. **LED state is a weak health signal** — `usbhidd` keyboard LEDs are bounded, - per-device, best-effort; they are not a system-global lock-state authority. A dead - `Caps Lock` indicator does not prove keyboard transport is broken; a working indicator - does not prove the external USB keyboard fallback works. Treat LED state as cosmetic - debug, not as a proof of input health. - -8. **External keyboard bare-metal proof remains unpinned** — the bounded QEMU lifecycle - proof is not the same as a bare-metal proof. We need a captured log per controller - family before claiming a fallback works on hardware. - -These eight items are the inputs to phases P0-A (xHCI runtime) and P0-B (legacy host -controllers). - ---- - -## 5. Phases — execution order - -Phases are ordered by *what unblocks bare-metal correctness and what has unambiguous -upstream-comparable patches we can adopt without inventing semantics*. - -| Phase | Goal | Exit | -|---|---|---| -| **P0-A1** | ✅ Re-enable xHCI MSI/MSI-X/INTx at runtime. Committed 2026-07-07 (`local/sources/base` commit `cbd40e0d`, parent `a2998c2d`). `test-xhci-irq-qemu.sh` now greps for actual reactor log lines. | ✅ QEMU proof script updated; real-hardware bring-up deferred to operator build. | -| **P0-A2** | Adopt upstream xHCI reset-procedure fix + hardware hardening (`69a80a6a`, `12e601b3`). | One QEMU full-stack pass + one real-hardware bring-up | -| **P0-A3** | Adopt CSZ (64-bit contexts) upstream commit; complete the `//TODO: cleanup CSZ support` site. | Same as A1 | -| **P0-A4** | ✅ Adopt panic bounds-check (`8f278dcb`) and timeout expansion (`4d6581d4`). Committed 2026-07-07: 5 `root_hub_port_index()` unwrap/index sites replaced with bounded access (`ok_or_else(|| Error::new(EINVAL))?`, `match None → continue`, `expect()` with diagnostic). | QEMU lifecycle + full-stack pass | -| **P0-B1** | Auto-spawn class drivers from the EHCI scheme (`/scheme/usb/port/descriptors`). Reuse the existing `xhcid` class-driver spawn model by refactoring the spawn helper out of `xhcid` into `usb-core::spawn_usb_driver` if necessary, then driving it from EHCI too. | QEMU run with USB keyboard on EHCI route → typed input reaches `inputd` | -| **P0-B2** | Implement real runtime enumeration for `uhcid` and `ohcid` over the existing `usb-core::UsbHostController` trait. Each new driver must register the same `/scheme/usb` tree pattern ehcid uses and must auto-spawn class drivers via `P0-B1`. | QEMU run with low/full-speed USB keyboard on legacy controller route → typed input reaches `inputd` | -| **P1** | ✅ Code-verified 2026-07-07. USB 3.0 packet-size handling (`update_max_packet_size` with shift exponent for USB ≥3, bytes for USB ≤2) in baseline. Hub descriptor reading uses separate `HubDescriptorV2`/`HubDescriptorV3`. Slot context hub bit (bit 26) and hub port count (bits 24-31) correctly set in `configure_endpoints_once`. `SET_HUB_DEPTH` issued for USB 3 hubs. TTT propagation not applicable to USB 3 (TT is USB 2.0 high-speed split-transaction only). TODOs about `interface_desc`/`alternate_setting` on USB 3 hubs are safe — passing None matches upstream behavior. | QEMU run deferred to operator. | -| **P2-A** | ✅ Storage data path: in-guest write verification on the `disk.usb-*` scheme. `redbear-usb-storage-check` already performs write+readback+restore at sector 2048. `test-usb-storage-qemu.sh` now validates all four checks (discovery, write, readback, restore). | `redbear-usb-storage-check` proves a write/read round-trip in QEMU | -| **P2-B** | Userspace API: pick native or `libusb`. Native: bake `usb-core` consumers first. `libusb`: pick an active WIP commit; if there is none, **defer** §2 row "userspace API" rather than start a new side-quest. | Decision + prototype | -| **P3** | HID robustness: real-hardware HID validation matrix; `i2c-hidd` migration to named producers; `evdevd` hotplug add/remove behavior from USB. | One HID device family proven bare-metal + one hot-unplug cycle QEMU | -| **P4** | Validation slices: complete `test-usb-storage-qemu.sh` write proof, hardware matrix in `HARDWARE-VALIDATION-MATRIX.md` (board, controller, input/storage/audio result), bounded stress loop on top of `test-xhci-device-lifecycle-qemu.sh`. | matrix has one row per controller family | -| **P5** | Architectural decision gate: host-only vs device mode; USB-C/PD/alt-mode scope; USB4/Thunderbolt exclusion; whether UCSI grows into a real PD surface. Recorded as an ADR in `local/docs/`. | Decision recorded | - -Phases are not equal in size. P0-A1 and P0-B2 are bounded, well-understood work. P2-B -(libusb vs native) is a fork in the road; it is correct that it has *no* time estimate. -P5 is a decision moment, not an implementation. - ---- - -## 6. Validation inventory and bounded proofs - -Five scripts exist today. They are honest about their scope (QEMU) but should be paired -with a real-hardware matrix per phase exit. - -| Script | What it actually proves | Limits | -|---|---|---| -| `local/scripts/test-usb-qemu.sh --check` | Full stack: xHCI init, HID spawn, SCSI spawn, sector-0 readback, BOS, no crashes. | QEMU `qemu-xhci` only; one emulator config; no real hardware. | -| `local/scripts/test-xhci-device-lifecycle-qemu.sh --check` | Bounded hotplug attach/detach for HID + storage. | QEMU only; monitor-driven hotplug; not a stress test. | -| `local/scripts/test-usb-storage-qemu.sh` | Mass storage autospawn + sector-0 readback. | No write proof; no multi-LUN; no UAS. | -| `local/scripts/test-xhci-irq-qemu.sh --check` | ✅ Updated 2026-07-07 to verify interrupt-driven reactor path. Greps for `Running IRQ reactor with IRQ file and event queue` (must be present) and `Running IRQ reactor in polling mode` (must NOT be), plus MSI-X/INTx delivery method. | QEMU `qemu-xhci` only; not real-hardware. | -| `local/scripts/test-usb-maturity-qemu.sh` | Sequential wrapper. | Composes the others; inherits their limits. | - -**Required proofs after P0-A1 lands:** - -1. `test-xhci-irq-qemu.sh --check` must transition from "binary runs" to "interrupts fire - and complete." Add a bounded probe that confirms a hotplug event triggers an IRQ in - guest time, not a sleep timer. -2. Add `test-xhci-regression-qemu.sh` for the upstream reset-procedure fix. -3. Add `test-uhci-runtime-qemu.sh` and `test-ohci-runtime-qemu.sh` after P0-B2 — same - shape as the xHCI lifecycle test. -4. Add `test-ehci-class-autospawn-qemu.sh` after P0-B1. - -Proofs must: - -- run on `redbear-mini` from a clean `make clean` build; -- keep the boot log under `local/docs/boot-logs/` with a `REDBEAR-...-RESULTS.md`; -- be citeable from phase status (§3 matrix) and from `USB-VALIDATION-RUNBOOK.md`. - ---- - -## 7. Durability posture (the local-fork problem, honestly) - -The base fork at `local/sources/base/` currently carries **two USB-related commits** -(one pre-existing, one from P0-A1): - -``` -$ git -C local/sources/base log -- drivers/usb/ -cbd40e0d xhcid: re-enable interrupt-driven operation via get_int_method ← P0-A1 (2026-07-07) -6ac41ee daemon: tolerate BrokenPipe on ready(); i2cd: handle empty RON response -dd08b76 Red Bear OS base baseline from 0.1.0 pre-patched archive -``` - -Everything else that v1 described as "88 error handling fixes across xhcid" lives in -**`local/patches/base/P*.patch`** files. That is acceptable as long as: - -1. The base *recipe* (`recipes/core/base/recipe.toml`) actually applies those patches on - `repo cook`. Verify by running `repo validate-patches base` after every edit and by - checking that `recipes/core/base/source/drivers/usb/xhcid/...` contains the Red Bear - state, not the upstream state. -2. No "live-edit" of `recipes/core/base/source/...` ever escapes into the next build - without an immediate patch mirror. `local/AGENTS.md` enforces this; the rule stands. -3. The next base-fork bump (rebase onto a newer Redox base tag) preserves every USB patch - in the same order and lands them as commits on the `submodule/base` branch — not as a - new mega-patch. - -**Durability remediation work that does not block USB phases:** - -- ✅ P0-A1 landed as the **first USB-focused commit on `submodule/base`** since `dd08b76` - (commit `cbd40e0d`, 2026-07-07). This reopens per-feature commit history and makes - future rebases reviewable. -- P0-A2 through P0-B2 should each land as individual, reviewable commits on the same - branch — never bundled into a mega-commit. Each phase below has a concrete file list - and diff target (see §11). -- The base fork's `Cargo.toml` should track the `submodule/base` branch as upstream - (currently it does, per the source-of-truth rules in `local/AGENTS.md`). - ---- - -## 8. Support language — how Red Bear describes USB - -Until P0-A and P0-B exit, Red Bear should NOT use any of: - -- "USB support works." -- "USB is functional." -- "USB keyboard works on bare metal." -- "USB storage is supported." - -It SHOULD use language such as: - -- "xHCI host support is present but experimental; bare-metal proof requires the real-hardware - matrix in §6 P4." -- "EHCI ownership and USB 2 register init exist; class-driver auto-spawn is pending P0-B1." -- "UHCI and OHCI are userspace stubs in this build; legacy host controllers are not yet - the boot-input fallback." -- "USB storage autospawn and bounded sector-0 readback are QEMU-validated; write proof is - pending P2-A." -- "USB error handling and correctness carry significant Red Bear patches over upstream; see - `local/patches/base/P[1-3]-xhci*.patch` and `local/patches/base/P3-xhci-device-hardening.patch`." -- "USB-C topology (UCSI) is exposed but does not negotiate PD or alternate modes." - -The README status table and the desktop-path plans should adopt this language consistently -the next time they are touched. The `local/docs/CONSOLE-TO-KDE-DESKTOP-PLAN.md` already -treats USB as a first-class subsystem; this plan agrees and refines the wording. - ---- - -## 9. Open questions and follow-up - -1. **Rebase cadence** — when `submodule/base` upstream lands the - `simplify xHCI` commit (May 2026), do we adopt it before or after P0-A1 lands? Per the - upstream-first rule, after — but the diff requires per-line review because our local - patches (`P1-xhcid-*.patch`, `P3-xhci-device-hardening.patch`) overlap on the same code - regions. -2. **Cross-process class driver spawn** — the class spawn path is currently xhcid-driven - (via the scheme). Should the spawn helper live in `usb-core` and be reused by - `ehcid`/`uhcid`/`ohcid`? Yes (P0-B1, P0-B2) — and that requires `usb-core` to grow - `spawn_class_driver`, which it does not yet have. The migration is the natural unit - of P0-B1. -3. **Strict-boot mode** — should `pcid-spawner` always pass `--strict-boot` to USB host - daemons? Operators can set `REDBEAR_STRICT_USB_BOOT=1` today; the default is off. - Recommend leaving the default off but documenting the env var in - `USB-VALIDATION-RUNBOOK.md` (P0-A4 documentation step). -4. **Whether to keep `usbscsid` enabled after upstream disabled it** — adopt the upside - (bounded in-guest write proof) and the downside (occasional stalls). Defer to P2-A - evaluation. -5. **Hardware validation entries** — the matrix in `local/docs/HARDWARE-VALIDATION-MATRIX.md` - is currently tiny. P4 explicitly grows it; if it does not, the matrix block of P4 exit - blocks the phase. - ---- - -## 10. See also - -- `local/docs/USB-VALIDATION-RUNBOOK.md` — operator runbook for the bounded proofs above. -- `local/docs/CONSOLE-TO-KDE-DESKTOP-PLAN.md` — the canonical desktop-path plan; treats - USB as a first-class runtime subsystem. -- `local/docs/IRQ-AND-LOWLEVEL-CONTROLLERS-ENHANCEMENT-PLAN.md` — MSI/MSI-X quality - surface that P0-A1 actually exercises. -- `local/docs/BLUETOOTH-IMPLEMENTATION-PLAN.md` — `redbear-btusb` consumes the USB class - driver dispatch path that P0-B1 makes available to all host controllers. -- `local/docs/WIFI-IMPLEMENTATION-PLAN.md` — Wi-Fi native control plane; not USB-coupled. -- `local/docs/QUIRKS-SYSTEM.md` — TOML + DMI + compiled-in quirk tables, source of USB - device workarounds. -- `local/AGENTS.md` — fork model, durability policy, single-repo rule, branch policy. -- `local/docs/archived/USB-IMPLEMENTATION-PLAN-v1-2026-04.md` — superseded v1. -- `local/docs/archived/USB-BOOT-INPUT-PLAN.md` — preserved for the boot-input - historical context; not the planning authority. -- `local/docs/archived/XHCID-DEVICE-IMPROVEMENT-PLAN.md` — preserved for the xhcid - device-level historical context; absorbed into phases P0-A and P1. - ---- - -## 11. Implementation handoff — P0-A2 through P0-B2 - -This section is the concrete kickoff for each remaining P0 sub-phase. -Each entry names files to touch, upstream commits to diff, and the required -validation step. A phase **does not leave implementation** until committed on -`submodule/base` (or the equivalent local fork) and, where practical, verified -with an automated QEMU proof. - -### P0-A2 — upstream xHCI reset-procedure fix - -| Field | Detail | -|---|---| -| **Upstream commits** | `https://gitlab.redox-os.org/redox-os/base/commit/69a80a6a` — "xhci: fix reset procedure on real hardware". Also `https://gitlab.redox-os.org/redox-os/base/commit/12e601b3` — "xhci: improvements based on real hardware testing". | -| **Files to touch** | `local/sources/base/drivers/usb/xhcid/src/xhci/mod.rs` — `Xhci::new`, controller reset path. `local/sources/base/drivers/usb/xhcid/src/xhci/operational.rs` — operational register definitions. | -| **What changes** | Replace magic bit numbers with named constants (`USB_CMD_RS`, `USB_CMD_HCRST`, `USB_STS_HCH`, `USB_STS_CNR`). Fix the HCRST wait loop to read from `usb_cmd` instead of `usb_sts`. Apply the port-RWC-correction and address_device speed passthrough from 12e601b3. | -| **Git landing** | One commit on `local/sources/base` master → update parent gitlink. | -| **Validation** | Rebuild `redbear-mini`, run `test-usb-qemu.sh --check`. Boot log must show xHCI controller init without "hang" or "reset failed" lines. If real hardware is available, boot on one Intel and one AMD controller. | -| **Blocking** | Nothing — independent of P0-A1. | - -### P0-A3 — CSZ (64-bit contexts) - -| Field | Detail | -|---|---| -| **Upstream commits** | `https://gitlab.redox-os.org/redox-os/base/commit/19570db4` — "xhci: support 64-bit contexts (CSZ)". | -| **Files to touch** | `local/sources/base/drivers/usb/xhcid/src/main.rs` — `daemon_with_context_size` and the `//TODO: cleanup CSZ support` comment at line 119. `local/sources/base/drivers/usb/xhcid/src/xhci/context.rs` — `DeviceContextList`, `InputContext`. `local/sources/base/drivers/usb/xhcid/src/xhci/mod.rs` — `Xhci` struct, `PortState`. | -| **What changes** | Make `Xhci`, `DeviceContextList`, `InputContext`, `PortState`, and `StreamContextArray` generic over context size `N` (32 or 64). Detect CSZ at runtime via `CapabilityRegs::csz()`. The local source already parameterizes `daemon_with_context_size` — the upstream fix is the natural completion. Remove the `//TODO: cleanup CSZ support` once generic parameterization is clean. | -| **Git landing** | One commit. | -| **Validation** | Rebuild + QEMU full-stack check. CSZ is not visible without a modern controller, so the QEMU proof is "didn't break existing paths." Real-hardware proof: boot on Ryzen 7000+ or Intel Alder Lake+. | -| **Blocking** | Nothing, but lands best after P0-A2 to avoid merge conflicts. | - -### P0-A4 — bounds check + timeouts - -| Field | Detail | -|---|---| -| **Upstream commits** | `https://gitlab.redox-os.org/redox-os/base/commit/8f278dcb` — bounds check on `root_hub_port_index()`. `4d6581d4` — "xhcid: add more timeouts". | -| **Files to touch** | `local/sources/base/drivers/usb/xhcid/src/xhci/mod.rs` — port index bounds. `local/sources/base/drivers/usb/xhcid/src/xhci/scheme.rs` — timeout additions. | -| **What changes** | Bounds-check the port index parameter to prevent out-of-range access. Add timeout guards on control transfer and address device paths to prevent infinite hangs. | -| **Git landing** | One commit. | -| **Validation** | QEMU lifecycle test must still pass. | -| **Blocking** | None. | - -### P0-B1 — EHCI class-driver auto-spawn - -| Field | Detail | -|---|---| -| **Why** | `ehcid` publishes `/scheme/usb/port/descriptors` but does **not** auto-spawn `usbhidd` or `usbscsid` when a matching device appears. Only `xhcid` does that through its scheme. Without auto-spawn, EHCI-attached USB keyboards never reach the input pipeline. | -| **Files to touch** | **New logic:** `local/recipes/drivers/usb-core/source/src/spawn.rs` — add a `spawn_class_driver` helper that takes a port descriptor, walks the USB class table, and spawns the matching class daemon (reuses the spawn model from xhcid). **Call site:** `local/recipes/drivers/ehcid/source/src/main.rs` — after enumerating a port and reading descriptors, call `usb_core::spawn_class_driver`. | -| **Git landing** | Two commits: (1) usb-core spawn helper, (2) ehcid call site. Both go on `submodule/base` since they touch existing tracked code. | -| **Validation** | New script: `test-ehci-class-autospawn-qemu.sh` — boot with USB keyboard on EHCI route, verify `usbhidd` spawns and keyboard input reaches `inputd`. | -| **Dependency** | P0-B1 is NOT blocked by anything. The usb-core trait already has `UsbHostController::control_transfer` and descriptor parsers. The class-spawn decision table (`/lib/drivers.d/70-usb-class.toml`) is already wired. | - -### P0-B2 — real UHCI/OHCI runtime enumeration - -| Field | Detail | -|---|---| -| **Why** | `uhcid/src/main.rs` and `ohcid/src/main.rs` are 35-line stubs: read PCI BAR4, log, sleep forever. No scheme, no transfers, no enumeration. This is the bare-metal USB keyboard blocker for legacy controller paths. | -| **Files to touch** | **uhcid:** `local/recipes/drivers/uhcid/source/src/main.rs` (replace 35-line stub with a ~1500-line implementation). **ohcid:** `local/recipes/drivers/ohcid/source/src/main.rs` (same). Both must implement `usb_core::UsbHostController` in a new sibling file `host.rs`, register `/scheme/usb`, perform frame-list/QH/TD/port enumeration, and call `spawn_class_driver` (from P0-B1) when a keyboard/storage device appears. Use the existing `ehcid` as a reference model. | -| **What changes** | For each controller: (a) PCI BAR mapping + register definitions, (b) `UsbHostController` trait implementor, (c) scheme registration (`/scheme/usb`), (d) port enumeration loop, (e) class-driver auto-spawn. | -| **Git landing** | Two commits (one per controller). These live in `local/recipes/drivers/`, not `local/sources/base/`, so they are committed on the parent `0.3.0` branch directly (tracked-tree model). | -| **Validation** | Two new scripts: `test-uhci-runtime-qemu.sh --check` and `test-ohci-runtime-qemu.sh --check`. Same shape as the xHCI lifecycle test: boot, verify scheme registration, hotplug keyboard, verify `usbhidd` spawn, verify keystrokes reach `inputd`. | -| **Dependency** | P0-B2 **depends on P0-B1** (uses the class-spawn helper) but does NOT depend on any of P0-A1 through P0-A4. UHCI and OHCI are independent from xHCI for enumeration. | -| **Reference impl** | `local/recipes/drivers/ehcid/source/src/main.rs` (1550 lines) — uses `usb-core`, registers `/scheme/usb`, MMIO frame list, QH/TD control/bulk/interrupt. UHCI and OHCI are simpler controllers and should be smaller. | - -### Build-and-verify workflow (per-session) - -``` -# After committing any P0 sub-phase change: -./local/scripts/build-redbear.sh --upstream redbear-mini -./local/scripts/test-xhci-irq-qemu.sh --check # if xHCI touched -./local/scripts/test-usb-qemu.sh --check # full-stack regression -./local/scripts/test-xhci-device-lifecycle-qemu.sh --check # lifecycle - -# After P0-B1/P0-B2: -./local/scripts/test-ehci-class-autospawn-qemu.sh --check # (to be written) -./local/scripts/test-uhci-runtime-qemu.sh --check # (to be written) -./local/scripts/test-ohci-runtime-qemu.sh --check # (to be written) -``` - ---- - -## 12. P5 — Modern USB Scope Decision (ADR) - -*Date:* 2026-07-07. -*Status:* Decided. Red Bear OS adopts **host-only USB** for the foreseeable -future. - -### Decision - -Red Bear OS ships as a **USB host** platform. Device mode (gadget), OTG -dual-role, USB-C Power Delivery negotiation, USB-C alternate modes, USB4, and -Thunderbolt are **explicitly excluded** from the current scope. This decision -is recorded as an ADR (Architecture Decision Record) so that future work does -not carry implicit scope expansion into the active build without a deliberate -re-evaluation. - -### What is in scope (host-first) - -- xHCI, EHCI, UHCI, and OHCI **host controllers** (drivers built, P0 complete). -- USB class daemons: HID (keyboard/mouse), Mass Storage (BOT), Hub, Audio. -- USB device enumeration, descriptor parsing, and class-driver auto-spawn. -- Hardware quirks: compiled-in + TOML runtime tables (146 USB + 214 storage - entries), consumed at runtime by xhcid and usbscsid. -- USB 3.x SuperSpeed (5 Gbps) and SuperSpeedPlus (10 Gbps) host operation - through xhcid. -- USB-C UCSI topology detection (`ucsid`, exposes `/scheme/ucsi`). - -### What is explicitly excluded - -| Capability | Excluded because | -|---|---| -| USB device mode (gadget) | Red Bear OS is a desktop/server OS, not an embedded peripheral. No dual-role controller (DRD) support exists in any upstream Redox component. | -| OTG (On-The-Go) | OTG requires dual-role + HNP/SRP protocol negotiation. No Redox kernel or driver infrastructure exists, and OTG is a declining standard (USB-C replaces it). | -| USB-C Power Delivery | PD negotiation requires a CC-line protocol engine, a policy manager, and source/sink state machines. This is a full subsystem (~10k LoC in Linux), not a small driver add-on. PMIC/charger integration is also needed. | -| USB-C alternate modes (DisplayPort, Thunderbolt) | Requires PD negotiation first, plus mux control, plus DP/Thunderbolt protocol stacks. No Redox GPU driver consumes DP alt-mode (display drivers use PCIe or platform-internal paths). | -| USB4 | USB4 requires PCIe tunneling, DisplayPort tunneling, and a USB4 router topology. The Redox PCI subsystem does not support PCIe hotplug or tunneling. Linux's USB4 stack is ~15k LoC. | -| Thunderbolt 3/4 | Thunderbolt requires USB4 or PCIe hotplug infrastructure. Listed as "not supported" in upstream Redox `COMMUNITY-HW.md`. No driver, no IOMMU DMA remapping for Thunderbolt security levels. | -| xHCI Debug Capability (DbC) | DbC requires a separate xHCI debug capability register set and a dedicated debug target endpoint. Serial console via UART is the standard debug path on Red Bear OS. DbC adds complexity without a use case. | - -### What may be reconsidered later - -- **USB-C PD (power role only, sink).** If Red Bear OS runs on a laptop that - charges via USB-C, the system firmware (UEFI/BIOS) handles PD negotiation - before the OS boots. An OS-level PD policy manager is only needed for - runtime source/sink role swaps, which are uncommon in a desktop/server OS. - Revisit if bare-metal laptop support requires it. -- **USB device mode for firmware update.** Some devices require USB DFU - (Device Firmware Upgrade) mode. This is a narrow, well-bounded gadget class - that could be implemented without a full dual-role stack. Not in current plan. -- **UCSI PD surface.** The existing `ucsid` daemon exposes connector topology. - Extending it to pass PD power contract data to a userspace policy manager is - a reasonable follow-up if hardware validation demands it. - -### Rationale - -Red Bear OS is a desktop/server operating system. The USB host path (keyboard, -mouse, storage, hub, audio) covers the essential desktop use case. Expanding -into device mode, PD, alt-modes, USB4, or Thunderbolt would add thousands of -lines of new kernel and driver code with no immediate user-visible benefit — -every excluded subsystem would consume weeks or months of development and -require hardware the team does not currently validate against. - -This decision keeps the USB scope **honest** and **buildable** with the current -team. It removes implicit "we should support X someday" scope pressure from -the active build, letting the team focus on completing the host-side USB -maturity work (P1–P4) and the Wi-Fi/Bluetooth/desktop integration paths that -depend on it. - -### Review cadence - -This ADR is reviewed **when a new Red Bear OS release branch is cut** (e.g., -`0.3.0` → `0.4.0`). At each review, the team evaluates whether any excluded -capability has become necessary for the next release's target hardware -profile. The ADR is not a permanent rejection — it is a current-scope -boundary that prevents unplanned scope creep.