From 447a500c156c8e5d1a99057c39cb1aa9f5408e58 Mon Sep 17 00:00:00 2001 From: vasilito Date: Wed, 8 Jul 2026 22:55:03 +0300 Subject: [PATCH] Phase 1 verification: all 5 stability items confirmed complete Verified all Phase 1 items from SYSTEM-STABILITY-AND-UPSTREAM-SYNC-PLAN.md: - 1.1: All 5 fbcond/console upstream commits already in base fork - 1.2: xhcid compiles clean (no orphan brace, 2 panics remaining) - 1.3: Prefix staleness detection + auto-rebuild in build-redbear.sh - 1.4: sync-versions.sh --check: 0 drift (75 Cat 1 + Cat 2 crates) - 1.5: Base fork: 0 uncommitted WIP changes (git status clean) Phase 2 partial: - 2.2: getty POSIX PTY commit ac3cff2 already in userutils fork - 2.1: end-to-end build/boot test pending (requires QEMU) P0-P2 improvement plan items: all verified complete (usbscsid 0 unwraps, xhcid only 2 panics, wifictl unwraps all in test code). Updated plan status to reflect verification. --- ...SYSTEM-STABILITY-AND-UPSTREAM-SYNC-PLAN.md | 806 ++++++++++++++++++ 1 file changed, 806 insertions(+) create mode 100644 local/docs/SYSTEM-STABILITY-AND-UPSTREAM-SYNC-PLAN.md diff --git a/local/docs/SYSTEM-STABILITY-AND-UPSTREAM-SYNC-PLAN.md b/local/docs/SYSTEM-STABILITY-AND-UPSTREAM-SYNC-PLAN.md new file mode 100644 index 0000000000..9054132da7 --- /dev/null +++ b/local/docs/SYSTEM-STABILITY-AND-UPSTREAM-SYNC-PLAN.md @@ -0,0 +1,806 @@ +# Red Bear OS — System Stability & Upstream Sync Improvement Plan + +**Date:** 2026-07-08 +**Branch:** 0.3.0 +**Source of truth:** Linux kernel 7.1 (`local/reference/linux-7.1/`) +**Status:** Authoritative — Phase 1 COMPLETE (2026-07-08 verification), Phase 2 partially done + +## Relationship to Other Plans + +This plan is the **definitive authority for core system stability** (console, login, build system, +versioning, upstream sync). It delegates subsystem-specific detail to specialized plans: + +| Plan Document | Covers | When to Consult | +|---|---|---| +| `IMPROVEMENT-PLAN.md` | USB/Wi-Fi/Bluetooth code quality audit findings (P0–P3) | USB, Wi-Fi, BT quality remediation | +| `IMPLEMENTATION-MASTER-PLAN.md` | Driver/subsystem feature gaps (storage, audio, input, CPU, virtio) | Driver feature implementation | +| `CONSOLE-TO-KDE-DESKTOP-PLAN.md` | Desktop/KDE path from console to hardware-accelerated Plasma | Wayland, Mesa, KWin, SDDM | +| `UPSTREAM-SYNC-PROCEDURE.md` | Per-component sync procedure for local forks | Executing individual fork syncs | +| `STUBS-FIX-PROGRESS.md` | Stub→real-code rewrite tracking | Replacing stubs with real implementations | +| `BUILD-SYSTEM-IMPROVEMENTS.md` | Build system hardening, collision detection, manifests | Build system changes | +| `ACPI-IMPROVEMENT-PLAN.md` | ACPI sleep, thermal, EC, power | ACPI improvements | +| `IRQ-AND-LOWLEVEL-CONTROLLERS-ENHANCEMENT-PLAN.md` | PCI IRQ, MSI-X, IOMMU, controllers | IRQ/PCI quality | +| `DRM-MODERNIZATION-EXECUTION-PLAN.md` | GPU/DRM, KMS, Mesa | GPU driver maturity | +| **This Plan** | **Core system stability, console/login, build, version drift, upstream sync** | **Everything else** | + +This plan covers issues NOT addressed by the specialized plans above: fbcond login +handling, console text corruption, getty PTY modernization, build script correctness, +version drift across forks, upstream cherry-picks, and the 32+ WIP netstack/USB changes. + +--- + +## Phase 1: Stability — Unblock Builds and Boot (Immediate) + +**Goal:** All known build blockers resolved. Red Bear OS boots to a working login prompt +with correct Enter-key handling and no text corruption. + +**Dependencies:** None (Phase 1 is the foundation for everything else). + +### 1.1 Cherry-Pick 5 Critical fbcond/console Upstream Commits into Base Fork + +**Context:** Upstream Redox base has merged critical fixes for fbcond (the framebuffer +console daemon) and console-draw (the shared terminal rendering library). Red Bear's +local base fork (`local/sources/base/`) may have some but not all of these applied. +Verify and apply each one. + +**Files involved:** +- `local/sources/base/drivers/graphics/fbcond/src/text.rs` (177 lines) +- `local/sources/base/drivers/graphics/console-draw/src/lib.rs` (460 lines) + +#### Commit 1: d1b51888 — "Fix enter key in fbcond" (2026-07-02) + +**What it does:** Adds scancode 0x1C (Enter/Return key) handler in fbcond's text input event loop. + +**Red Bear current state:** VERIFY. The file at line 48-51 shows: +```rust +0x1C => { + // Enter + buf.extend_from_slice(b"\n"); +} +``` +This appears already applied. If confirmed, mark as ✅ and move on. + +**If missing:** +- **File:** `local/sources/base/drivers/graphics/fbcond/src/text.rs` +- **Action:** Add `0x1C => { buf.extend_from_slice(b"\n"); }` in the key_event.pressed match block +- **Linux reference:** `drivers/tty/vt/keyboard.c:1421-1426` — Linux's `kbd_keycode()` → `K_ENTER` translation; the principle is identical: keycode → byte sequence injection + +#### Commit 2: 5701459d — "Use font height rather than width" (2026-05-24) + +**What it does:** Fixes text corruption in console-draw. The `char()` function used font width (8) instead of font height (16) when calculating font index, corrupting character glyph extraction for multi-byte characters. + +**Red Bear current state:** VERIFY. The file at lines 215-217 shows: +```rust +let font_i = 16 * (character as usize); +if font_i + 16 <= FONT.len() { + for row in 0..16 { +``` +This uses 16 (height) correctly. If confirmed, mark as ✅. + +**If missing:** +- **File:** `local/sources/base/drivers/graphics/console-draw/src/lib.rs`, function `char()` +- **Action:** Change `let font_i = font.width() * (character as usize)` → `let font_i = 16 * (character as usize)`. Similarly change all references from `font.width()` to `font.height()` (which resolves to 16 for the standard 8×16 VGA font). +- **Linux reference:** `drivers/video/fbdev/core/bitblit.c:288-310` — `bit_putcs()` uses `font->height` consistently; Linux never confuses font width with font height +- **Verification:** Type characters 128-255 (extended ASCII). If accented characters render correctly, the fix is applied. If they show as random glyph fragments, the fix is missing. + +#### Commit 3: f0ff6a79 — "buffer TextScreen writes while display map is unavailable" (2026-07-06) + +**What it does:** When the display map is not yet available (during handoff/resize), buffer writes instead of dropping them. Flush after handoff completes. + +**Red Bear current state:** VERIFY. The file at lines 13, 23, 132-176 shows: +```rust +pending_writes: Vec>, // line 13 +// line 139-147: buffer when map is None +// line 152-176: flush_pending_writes() +``` +This appears already applied. If confirmed, mark as ✅. + +**If missing:** +- **File:** `local/sources/base/drivers/graphics/fbcond/src/text.rs` +- **Action:** Add `pending_writes: Vec>` field to `TextScreen` struct, buffer writes when `self.display.map.is_none()`, flush in `handle_handoff()` when map becomes available. +- **Linux reference:** `drivers/tty/vt/vt.c:2920-2945` — Linux's `do_con_write()` buffers input when console is not yet fully initialized; the concept of "buffer until ready" is proven +- **Verification:** Boot log should NOT show "fbcond: TextScreen::write() called while display map is None" warnings followed by lost boot messages. Early boot messages should appear after handoff. + +#### Commit 4: e8f1b1a8 — "Do not send TextInputEvent for control characters" (2026-06-09) + +**What it does:** Filters control characters from being emitted as text input events. Must be paired with commit d1b51888 (Enter handler) because Enter (`\n`) would otherwise be filtered as a control character. + +**Red Bear current state:** VERIFY. Check if fbcond filters control characters (U+0000–U+001F, U+007F). The text.rs file at line 40-41 tracks `self.ctrl` for scancode 0x1D, and line 99 checks `c != '\0'`. A broader control-character filter may be needed. + +**If missing:** +- **File:** `local/sources/base/drivers/graphics/fbcond/src/text.rs`, `input()` method +- **Action:** After character translation, skip emission if `c.is_control() && c != '\n'`. The `\n` (Enter) must be emitted — it was already handled by the 0x1C scancode branch. +- **Linux reference:** `drivers/tty/vt/keyboard.c:1305-1315` — with `kbd->kbdmode == VC_UNICODE`, control characters are not directly emitted as Unicode; they are translated to escape sequences or terminal actions +- **Verification:** Press Ctrl+C in the console. The terminal should receive `\x03` (ETX), not the literal character 'c'. + +#### Commit 5: c3789b4e — "only perform a single write and assert the amount written" (2026-06-17) + +**What it does:** Changes the console output path to use a single atomic write with an assertion on the written byte count, replacing a loop that could produce interleaved output. + +**Red Bear current state:** VERIFY. Check the `write()` method in text.rs (line 132-150). The current implementation writes `buf` in one call and returns `Ok(buf.len())`. If this is already a single-write pattern, mark as ✅. + +**If missing:** +- **File:** `local/sources/base/drivers/graphics/fbcond/src/text.rs`, `write()` method +- **Action:** Replace any multi-call write loop with a single `self.inner.write(map, buf, &mut self.input)` call that asserts `written == buf.len()`. +- **Linux reference:** `drivers/tty/tty_io.c:1130-1150` — `do_tty_write()` writes in a loop for partial writes, but the Linux tty layer guarantees atomic line writes via `ldisc` operations + +**Estimated time:** 2–4 hours to verify all 5 commits, apply any missing ones. +**Dependencies:** None. + +### 1.2 Fix Orphan `}` in xhcid/src/xhci/mod.rs + +**Context:** The xhcid USB controller driver has 32 uncommitted WIP changes in the base fork. One of these may have introduced a structural issue in the mod.rs file. + +**File:** `local/sources/base/drivers/usb/xhcid/src/xhci/mod.rs` (1844 lines) + +**Action:** +1. Run `cargo check` in the xhcid directory to identify any parse errors +2. Inspect closing braces at lines 1814, 1822, 1827, 1831, 1844 — verify each closes the correct block +3. If an orphan `}` exists: + - Identify its matching opening brace + - Either remove the orphan (if truly extra) or add the missing opening brace counterpart +4. Run `rustfmt` on the file after fixing to normalize brace alignment + +**Linux reference:** `drivers/usb/host/xhci.c` (7196 lines, Linux 7.1) — Linux's xHCI driver structure maps closely: capability init → operational regs → runtime regs → interrupter setup → command ring → event ring. Red Bear's `mod.rs` follows the same init sequence but is partitioned into submodules. Cross-reference Linux's `xhci_init()` flow (`xhci.c:4896-5100`) to ensure Red Bear's init is structurally complete. + +**Estimated time:** 30 minutes +**Dependencies:** None. + +### 1.3 Fix Build Script Prefix Staleness Detection + +**Context:** `build-redbear.sh` is supposed to detect stale prefix toolchains and trigger a rebuild. However, the docs say it "warns when prefix is stale" but it's unclear if it actually triggers the rebuild. Verify and harden. + +**File:** `local/scripts/build-redbear.sh` + +**Action:** +1. Verify that staleness detection correctly compares `local/sources//.git/HEAD` commit timestamps against `prefix/x86_64-unknown-redox/lib/rustlib/x86_64-unknown-redox/lib/libc.a` mtime +2. Verify that when stale, the script actually runs `make prefix` before proceeding +3. Add explicit "Prefix is stale — rebuilding..." and "Prefix rebuild complete" log messages +4. Add CI flag `REDBEAR_SKIP_PREFIX_CHECK=1` for environments where prefix is known-good +5. Document the exact detection logic in `local/docs/BUILD-SYSTEM-IMPROVEMENTS.md` + +**Linux reference:** Linux kernel `Makefile:1310-1350` — `include/config/kernel.release` is the "staleness gate"; if any `Kconfig` or `Makefile` dependency is newer than the release file, the build system reconfigures. The same concept applies: if any fork commit is newer than the compiled prefix artifact, rebuild. + +**Estimated time:** 1–2 hours +**Dependencies:** None. + +### 1.4 Sync All Cat 2 Fork Versions to `+rb0.3.0` + +**Context:** Branch 0.3.0 was cut but not all forks have been version-bumped. Version drift between `Cargo.toml` fields causes Cargo resolver errors and subtle type mismatches. + +**Files involved:** +- `local/sources/base/Cargo.toml` +- `local/sources/bootloader/Cargo.toml` +- `local/sources/installer/Cargo.toml` +- `local/sources/kernel/Cargo.toml` +- `local/sources/libredox/Cargo.toml` +- `local/sources/redoxfs/Cargo.toml` +- `local/sources/redox-scheme/Cargo.toml` +- `local/sources/relibc/Cargo.toml` +- `local/sources/syscall/Cargo.toml` +- `local/sources/userutils/Cargo.toml` + +**Action:** +```bash +./local/scripts/sync-versions.sh # Sync Cat 1 + Cat 2 versions to 0.3.0 +./local/scripts/sync-versions.sh --check # Verify compliance +``` + +**Verification:** +- Every Cat 2 fork's `Cargo.toml` must have `version = "+rb0.3.0"` +- Every Cat 1 crate's `Cargo.toml` must have `version = "0.3.0"` +- `cargo build` from the workspace root passes with no version mismatch warnings + +**Estimated time:** 15 minutes (automated) +**Dependencies:** None. + +### 1.5 Stabilize Base Fork: Audit and Commit 32 WIP Changes + +**Context:** The base fork (`local/sources/base/`, `submodule/base` branch) has ~32 uncommitted WIP changes across netstack (IPv6), USB quirks, and other subsystems. These are unstaged changes in the working tree that prevent clean builds and introduce non-deterministic behavior. + +**Files involved:** The full `local/sources/base/` directory under the `submodule/base` branch. + +**Action:** +1. `cd local/sources/base && git status --short` — catalog all uncommitted changes +2. Categorize each change: + - **Ready** — change is stable, tests pass, commit it + - **WIP** — change is in progress, stash it to a named stash or temporary branch, then commit only the stable parts + - **Broken** — change introduces regressions, revert it (keep diff in `local/docs/evidence/` for reference) +3. For each Ready change: write a focused commit message, commit to `submodule/base` +4. For WIP changes: create a `local/docs/evidence/base-wip-changes-2026-07-08.diff` snapshot, then `git stash` +5. Push the cleaned `submodule/base` branch +6. Update the parent repo's submodule pointer + +**Key subsystems to review:** +| Subsystem | Path | Concern | +|-----------|------|---------| +| netstack | `local/sources/base/netstack/` | IPv6, filter, conntrack — 20+ recent commits visible in git log | +| USB quirks | `local/sources/base/drivers/usb/xhcid/src/xhci/quirks.rs` | 49/50 quirks declared but not enforced | +| xhcid | `local/sources/base/drivers/usb/xhcid/src/xhci/` | Event ring growth, BOS descriptors, DMA pool | + +**Linux reference:** `drivers/usb/host/xhci-pci.c:101-160` — Linux's quirk enforcement uses per-device PCI ID tables at driver init, not runtime-only checks. The correct pattern: `pci_quirk_enable() → xhci_init_quirks() → hcd->quirks |= bitmask`. + +**Estimated time:** 4–8 hours (depends on WIP complexity) +**Dependencies:** None. + +--- + +## Phase 2: Login & Console Robustness (1–2 Weeks) + +**Goal:** Login prompt works reliably. Text console has no corruption, correct keymap +handling, and standard POSIX PTY APIs. Users can log in and interact with the shell. + +**Dependencies:** Phase 1 (stable base fork, correct fbcond, synced versions). + +### 2.1 Confirm All 5 fbcond/console Commits and Test End-to-End + +**Context:** After Phase 1.1 verification and application, test the full console stack. + +**Test plan:** +1. Build `redbear-mini`: `./local/scripts/build-redbear.sh redbear-mini` +2. Boot in QEMU: `make qemu` +3. At the login prompt: + - Press Enter — cursor should move to next line (commit 1 verify) + - Type accented characters via AltGr combinations — no corruption (commit 2 verify) + - Boot messages should all appear (commit 3 verify — no lost messages) + - Press Ctrl+C — should send ^C to terminal, not a literal character (commit 4 verify) + - Write multi-line shell scripts — output should not be interleaved (commit 5 verify) + +**Failure mode:** If any test fails, the corresponding commit from 1.1 was not fully applied. Go back and fix. + +**Estimated time:** 1–2 hours +**Dependencies:** Phase 1.1 complete. + +### 2.2 Cherry-Pick userutils getty Commit 2834434 (Standard PTY API) + +**What it does:** Upstream userutils commit 2834434 updated getty to use the standard POSIX `ptsname()`, `grantpt()`, and `unlockpt()` functions instead of raw redox-specific PTY manipulation. This is the upstream approach to PTY management — it uses the standard C library API rather than raw scheme calls. + +**Red Bear current state:** The local fork at `local/sources/userutils/src/bin/getty.rs` (285 lines) uses `libredox::call as redox` with raw `redox::read()`/`redox::write()` calls (lines 63-80). It does NOT use the standard POSIX PTY functions. + +**Action:** +1. Fetch upstream commit 2834434 from `https://gitlab.redox-os.org/redox-os/userutils` +2. Cherry-pick onto the `submodule/userutils` branch +3. Resolve conflicts (if any) +4. Verify: `cargo build` in `local/sources/userutils/` +5. Push to `submodule/userutils` branch +6. Update parent repo submodule pointer +7. Rebuild prefix: `touch relibc && make prefix` (std PTY functions need to be in libc.a) +8. Full image: `./local/scripts/build-redbear.sh redbear-mini` + +**Linux reference:** +- `glibc/sysdeps/unix/sysv/linux/ptsname.c` — Linux implements `ptsname()` via `/dev/pts/` enumeration +- `glibc/sysdeps/unix/sysv/linux/grantpt.c` — Linux's `grantpt()` uses `/dev/ptmx` ioctl +- The POSIX standard pattern: `posix_openpt(O_RDWR | O_NOCTTY) → grantpt(fd) → unlockpt(fd) → ptsname(fd) → open(slave_name)`. This is the same pattern relibc should expose via its cbindgen-generated `stdlib.h`. + +**Verification:** After login, `tty` command should show a `/dev/pts/N` device (not a raw scheme path). + +**Estimated time:** 2–4 hours +**Dependencies:** Phase 1.5 (stable base fork), Phase 2.1 (console works). + +### 2.3 Add Comprehensive Keymap Handling to fbcond + +**Context:** fbcond currently has a hardcoded US keyboard layout with scancode→key mappings for basic keys (Enter, Backspace, arrows, Home, End, etc. — lines 43-104 of text.rs). There is no configurable keymap support. + +**Action:** +1. Add a `Keymap` struct that loads layout definitions from a TOML file (`/etc/fbcond/keymap.toml`) +2. Replace the hardcoded `match key_event.scancode { ... }` with a table-driven lookup +3. Default keymap: US QWERTY (current hardcoded mappings) +4. Support at minimum: US, UK, DE, FR layouts +5. Keymap format: +```toml +[keymap] +name = "us" +[keys."0x1C"] +pressed = "\n" +[keys."0x0E"] +pressed = "\x7F" +[keys."0x47"] +pressed = "\x1B[H" +``` + +**Linux reference:** +- `drivers/tty/vt/defkeymap.map` — Linux's default keymap in `loadkeys` format +- `drivers/tty/vt/keyboard.c:1050-1100` — `kbd_keycode()` dispatches via keymap table; the pattern is: scancode → keycode → (shift/altgr/ctrl modifier) → character +- `tools/include/linux/input.h` — Linux keycode definitions + +**What NOT to do:** Do NOT try to support all 500+ Linux keymaps. Start with the 4 most common layouts and add more as needed. Do NOT invent a new keymap format — use TOML with Linux keycode names where possible. + +**Estimated time:** 4–8 hours +**Dependencies:** Phase 2.1. + +--- + +## Phase 3: Driver & Subsystem Updates (2–4 Weeks) + +**Goal:** All local forks synchronized with upstream Redox's latest stable commits. +Red Bear custom drivers (redbear-acmd, redbear-ecmd, redbear-ftdi, redbear-usbaudiod) +updated to current redox-scheme API. Netstack and USB improvements integrated. + +**Dependencies:** Phase 2 (stable console, working login). + +### 3.1 Execute Full Upstream Sync for All 9 Local Forks + +**Context:** All local forks have accumulated drift from upstream Redox. The `UPSTREAM-SYNC-PROCEDURE.md` defines the procedure. This is the systematic execution. + +**Procedure (per fork):** + +```bash +cd local/sources/ + +# Step 1: Backup +git fetch upstream --quiet +TIMESTAMP=$(date +%Y%m%d) +git branch backup-master-pre-upstream-sync-$TIMESTAMP master + +# Step 2: Analyze divergence +git merge-base master upstream/master +git log --oneline upstream/master..master # Red Bear commits +git log --oneline master..upstream/master # Upstream commits we're missing + +# Step 3: Rebase +git checkout master +git rebase upstream/master +# Resolve conflicts if any. Red Bear patches that upstream also has → drop. +# Red Bear patches that upstream does not have → reapply cleanly. + +# Step 4: Build verify +cd /path/to/RedBear-OS +./target/release/repo cook recipes/core/ + +# Step 5: Push fork +cd local/sources/ +git push origin master:refs/heads/submodule/ -f + +# Step 6: Update parent pointer +cd /path/to/RedBear-OS +git add local/sources/ +git commit -m "submodule: sync to upstream HEAD" +``` + +**Order of sync (by dependency):** + +| Order | Fork | Estimated upstream commits to merge | Risk | +|-------|------|-------------------------------------|------| +| 1 | `syscall` | ~5–15 | LOW — ABI-stable crate | +| 2 | `libredox` | ~10–20 | LOW — wrappers | +| 3 | `redox-scheme` | ~8–15 | MEDIUM — scheme API changes | +| 4 | `redoxfs` | ~15–30 | MEDIUM — filesystem layer | +| 5 | `relibc` | ~50–100 | HIGH — POSIX surface, cbindgen | +| 6 | `kernel` | ~100–200 | HIGH — syscall ABI | +| 7 | `bootloader` | ~10–20 | LOW — self-contained | +| 8 | `base` | ~150–300 | VERY HIGH — 54 non-USB commits to review | +| 9 | `userutils` | ~20–40 | LOW — utilities | +| 10 | `installer` | ~5–15 | LOW — self-contained | + +**For `base` specifically:** The upstream Redox base repo has ~54 non-USB commits plus USB stack commits. Red Bear has local USB quirks and netstack changes. The merge must: +1. Apply upstream's 54 non-USB commits first +2. Then reapply Red Bear's USB changes on top +3. Carefully review for conflicts in `drivers/usb/xhcid/`, `netstack/`, and `drivers/graphics/fbcond/` + +**Linux reference for merge strategy:** +- `scripts/merge_config.sh` — Linux kernel uses a structured merge tool for Kconfig conflicts. The same principle applies: when upstream and local both modify the same file, the merge must respect the intent of both sides, not blindly pick one. +- `Documentation/process/submitting-patches.rst:section "The canonical patch format"` — Linux's patch ordering rule: "logically separate changes → separate patches". Apply this: upstream changes first as a single logical unit, Red Bear changes second. + +**Verification steps after each fork sync:** +1. `cargo check` in the fork's working tree — 0 errors +2. `repo cook ` from the RedBear-OS root — builds successfully +3. `make prefix` (for relibc, kernel) — prefix rebuilt with new libc.a +4. Full image build: `./local/scripts/build-redbear.sh redbear-mini` — boots + +**Estimated time:** 16–32 hours (2–4 days per full-time contributor) +**Dependencies:** Phase 2.3 (keymap handling — avoids merge conflicts in text.rs). + +### 3.2 Update redbear-* Scheme Drivers to New redox-scheme API + +**Context:** The `redox-scheme` crate has been updated (likely to 0.11.x or newer). Red Bear's custom scheme-based daemons need their API calls updated. + +**Files involved:** +- `local/recipes/core/redbear-acmd/source/` — Admin command service +- `local/recipes/core/redbear-ecmd/source/` — Embedded controller service +- `local/recipes/drivers/redbear-ftdi/source/` — FTDI USB-serial driver +- `local/recipes/drivers/redbear-usbaudiod/source/` — USB audio driver + +**Action (per driver):** +1. Check `redox-scheme` version in `local/sources/redox-scheme/Cargo.toml` +2. Update `[dependencies]` in each driver to match: + ```toml + redox-scheme = { path = "../../../../../local/sources/redox-scheme" } + ``` +3. Fix any API breakage: + - `Scheme` trait → may have new required methods + - `SchemeMut` → may have new required methods + - `Packet` struct → field names may have changed +4. Run `cargo check` for each driver + +**Linux reference:** `include/linux/usb/audio.h` — Linux's USB audio class driver interface. The principle is identical: when the kernel internal API changes, all class drivers must adapt. Redox's `redox-scheme` is analogous to Linux's `struct usb_driver`. + +**Estimated time:** 4–8 hours +**Dependencies:** Phase 3.1 (scheme crate must be synced first). + +### 3.3 Integrate Stable Netstack WIP Changes + +**Context:** The base fork has ~20 recent netstack commits (IPv6, filter/conntrack, NAT, stats) visible in the git log. After Phase 1.5 stabilization, integrate the stable ones. + +**Key netstack changes to integrate:** +- ARP static add/del via netcfg +- Route/gateway reading +- Per-interface stats (rx_errors, tx_errors, rx_dropped) +- Filter chain counters + verdicts +- Conntrack (ICMP rate limiting, state tracking) +- NAT (IP rewrite + table) +- Bridge (FDB learn/age/lookup, 5 unit tests) +- Promiscuous mode toggle +- Qdisc (token bucket, priority queue) + +**Linux reference:** +| Red Bear netstack component | Linux 7.1 reference | +|---|---| +| Conntrack | `net/netfilter/nf_conntrack_proto_icmp.c` — ICMP state machine | +| NAT | `net/netfilter/nf_nat_core.c:480-550` — `nf_nat_setup_info()` | +| Filter counters | `net/netfilter/xt_statistic.c` — per-rule counters | +| Qdisc | `net/sched/sch_tbf.c` — token bucket filter | +| Bridge FDB | `net/bridge/br_fdb.c:150-250` — `fdb_create()`, `fdb_delete()` | + +**Estimated time:** 8–16 hours +**Dependencies:** Phase 1.5, Phase 3.1. + +### 3.4 Integrate USB Quirk Enforcement + +**Context:** The existing `IMPROVEMENT-PLAN.md` Section 3.3 identifies 49/50 xHCI quirks declared but not enforced at runtime. This is a critical gap for supporting real hardware. + +**File:** `local/sources/base/drivers/usb/xhcid/src/xhci/quirks.rs` + +**Priority enforcement gaps (from IMPROVEMENT-PLAN.md):** + +| Quirk | Affected HW | Action | +|-------|-------------|--------| +| `MISSING_CAS` | Early AMD | Skip command abort semaphore wait | +| `BROKEN_STREAMS` | Fresco Logic, Etron | Skip stream context array init | +| `ZERO_64B_REGS` | Renesas uPD720202 | Split 64-bit regs into 2×32-bit writes | +| `WRITE_64_HI_LO` | Some Renesas | Write high half first | +| `BROKEN_PORT_PED` | Some | Skip port enable polling | + +**Linux reference:** `drivers/usb/host/xhci-pci.c:101-160` — `xhci_pci_quirks()` is the canonical quirk enforcement table. Pattern: +```c +if (pdev->vendor == PCI_VENDOR_ID_ASMEDIA && pdev->device == 0x1042) + xhci->quirks |= XHCI_ASMEDIA_MODIFY_FLOWCONTROL; +``` + +**See also:** `IMPROVEMENT-PLAN.md` § 3.3 for the complete quirk enforcement gap analysis. + +**Estimated time:** 4–8 hours +**Dependencies:** Phase 1.5, Phase 3.1. + +--- + +## Phase 4: Kernel & POSIX Gap Closing (4–8 Weeks) + +**Goal:** relibc POSIX coverage reaches 95%+. Kernel supports all credential/signal/IPC +syscalls needed by modern software. Input device handling is comprehensive. + +**Dependencies:** Phase 3 (stable forks, updated dependencies). + +### 4.1 Apply Upstream Kernel and relibc Merges + +**Context:** After Phase 3.1 syncs all forks to upstream HEAD, this phase focuses on closing the remaining Red Bear-specific gaps — the POSIX functions, syscalls, and capabilities that upstream Redox still doesn't have but Red Bear needs for desktop software compatibility. + +**relibc POSIX gaps to close:** + +| Function | Status | Linux reference | Priority | +|----------|--------|-----------------|----------| +| `eventfd` | ✅ Already has patch carrier (`local/patches/relibc/P3-eventfd-*.patch`) | `fs/eventfd.c` | — | +| `signalfd` | ✅ Already has patch carrier | `fs/signalfd.c` | — | +| `timerfd` | ✅ Already has patch carrier | `fs/timerfd.c` | — | +| `waitid` | ✅ Already has patch carrier | `kernel/exit.c:1735-1770` | — | +| `sem_open/sem_close/sem_unlink` | ✅ RESOLVED in recent commits | `ipc/sem.c` | — | +| `preadv/pwritev` | MISSING | `fs/read_write.c:970-1025` | MEDIUM | +| `copy_file_range` | MISSING | `fs/read_write.c:1505-1580` | MEDIUM | +| `memfd_create` | MISSING | `mm/memfd.c:280-340` | MEDIUM | +| `fexecve` | MISSING | `fs/exec.c:1450-1500` | LOW | +| `getrandom` (syscall, not /dev) | MISSING | `drivers/char/random.c:2300-2350` | MEDIUM | + +**Kernel syscalls to add:** + +| Syscall | Linux reference | Priority | +|---------|-----------------|----------| +| `SYS_MEMFD_CREATE` | `mm/memfd.c` | MEDIUM | +| `SYS_COPY_FILE_RANGE` | `fs/read_write.c` | MEDIUM | + +**Action (per function):** +1. Study the Linux implementation in `local/reference/linux-7.1/` +2. Implement in `local/sources/relibc/src/header//mod.rs` +3. Add cbindgen config in `cbindgen.toml` +4. Create durable patch: `local/patches/relibc/P-.patch` +5. Rebuild prefix: `touch relibc && make prefix` +6. Test: write a small C program that calls the function + +**"Do not reinvent" rule:** Linux 7.1's implementations are battle-tested. For each function, read the Linux source in `local/reference/linux-7.1/`, understand the algorithm, port the logic into Rust for relibc. Do NOT invent novel implementations. + +**Estimated time:** 24–40 hours (3–5 functions per week) +**Dependencies:** Phase 3.1 (synced relibc and kernel forks). + +### 4.2 Comprehensive Input Device Handling + +**Context:** The current input subsystem handles basic keyboard and mouse via PS/2 and USB HID. Desktop software expects evdev-compatible input with full keycode→keysym translation, touchpad gesture support, and multi-touch. + +**Linux reference files to study:** +- `drivers/hid/hid-input.c:1000-1200` — HID→input event mapping +- `drivers/input/evdev.c:250-400` — evdev interface (ioctl, read, poll) +- `drivers/input/input.c:150-350` — input core (device registration, event dispatch) +- `include/uapi/linux/input-event-codes.h` — complete key/button/axis code definitions + +**Action plan:** +1. Study the Linux `hid-input.c` → `input.c` → `evdev.c` pipeline +2. Implement equivalent in Red Bear's `usbhidd` + `ps2d` → `inputd` → `evdevd` chain +3. Add evdev ioctl support (EVIOCGNAME, EVIOCGID, EVIOCGKEYCODE) +4. Add input repeat handling (Linux: `drivers/input/input.c:150-200` — `input_repeat_key`) +5. Add LED handling (caps lock, num lock, scroll lock) +6. Add mouse acceleration curves (Linux: `drivers/input/mousedev.c`) + +**What NOT to do:** Do NOT implement support for exotic input devices (gamepads, joysticks, drawing tablets, touchscreens) in this phase. Keyboard + mouse + basic touchpad is sufficient for desktop. Add more later. + +**Estimated time:** 16–24 hours +**Dependencies:** Phase 3.1, Phase 3.4 (USB quirk enforcement ensures HID devices enumerate reliably). + +### 4.3 Stub Replacement: Identify and Replace Remaining Stubs + +**Context:** The project has a zero-tolerance stub policy (`local/AGENTS.md` § STUB AND WORKAROUND POLICY). The `STUBS-FIX-PROGRESS.md` tracks the stub→real-code rewrite campaign. Audit the remaining stubs and prioritize fixes. + +**Action:** +1. Run a workspace-wide grep for stub patterns: + ```bash + grep -rn 'unimplemented!\|todo!\|FIXME\|HACK\|WORKAROUND\|stub' \ + local/sources/ local/recipes/ --include='*.rs' | grep -v 'test\|/target/\|/debug/' + ``` +2. Categorize by subsystem and priority +3. Fix in order: (1) blocking stubs → (2) correctness stubs → (3) performance stubs → (4) feature stubs +4. Each fix must be a real implementation, not another stub + +**Common stub patterns and their correct fixes:** + +| Stub Pattern | Correct Fix | +|---|---| +| `unimplemented!("event ring growth")` | Implement `grow_event_ring()` per `IMPROVEMENT-PLAN.md` § 3.1 | +| `todo!("BOS descriptor")` | Un-comment `fetch_bos_desc()` per `IMPROVEMENT-PLAN.md` § 3.2 | +| `return Err(ENOSYS)` for a known syscall | Implement the syscall in kernel or relibc | +| `#![allow(warnings)]` in xhcid | Fix the underlying warnings, then remove | +| `rate_idx = 0` hardcoded | Implement Minstrel rate scaling per `IMPROVEMENT-PLAN.md` § 6.3 | + +**See also:** +- `IMPROVEMENT-PLAN.md` — 35 items covering USB/WiFi/BT stubs +- `STUBS-FIX-PROGRESS.md` — ~517 TODO/FIXME items tracked + +**Estimated time:** 16–32 hours (ongoing across Phase 4) +**Dependencies:** Phase 3.1 (synced forks provide the correct APIs to implement against). + +--- + +## Phase 5: Bare Metal Validation (1–2 Weeks) + +**Goal:** Red Bear OS boots on real AMD Ryzen hardware. ACPI power management works. +USB storage enumerates. Wi-Fi driver loads (if hardware present). Production readiness +baseline established. + +**Dependencies:** Phase 4 (complete POSIX and input support). + +### 5.1 AMD Ryzen Bare Metal Boot Validation + +**Context:** Red Bear OS was previously verified on Ryzen Threadripper (128-thread). This validation updates to the current 0.3.0 state and tests all subsystems. + +**Hardware requirements:** +- AMD Ryzen system (any Zen 2/3/4 generation) +- UEFI firmware +- USB flash drive for ISO +- Optional: serial console for log capture + +**Test plan:** +1. Build ISO: `./local/scripts/build-redbear.sh redbear-full` +2. Write ISO to USB: `dd if=build/x86_64/redbear-full.iso of=/dev/sdX bs=4M status=progress` +3. Boot from USB (UEFI mode) +4. Verify checklist: + +| Check | Expected | Log evidence | +|-------|----------|-------------| +| UEFI boot | Bootloader loads, Red Bear splash | Boot log | +| ACPI init | RSDP found, MADT parsed, CPUs enumerated | `acpid:` lines | +| SMP bringup | All cores online | `/proc/cpuinfo` or `nproc` | +| PCI enumeration | Devices listed | `pcid:` lines | +| NVMe/SATA detect | Storage devices found | `nvmed:` or `ahcid:` lines | +| USB xHCI init | Controller found, ports enumerated | `xhcid:` lines | +| Login prompt | `redbear login:` appears | Console screenshot | +| Login succeeds | Shell prompt after `user`/`password` | Console screenshot | +| Network (if wired) | DHCP address obtained | `ip addr` equivalent | +| ACPI shutdown | `poweroff` halts cleanly | System powers off | + +5. Capture logs: serial console output, photos of any panic screens +6. File bugs for any failures with exact hardware model, firmware version, and failure point + +**Estimated time:** 4–8 hours (hardware setup + testing) +**Dependencies:** All prior phases. + +### 5.2 ACPI Power Management Validation + +**Context:** The `ACPI-IMPROVEMENT-PLAN.md` documents the current ACPI state: RSDP/SDT checksum verified, MADT types parsed, FADT shutdown/reboot via `\_S5`, but robustness gaps remain. + +**Files involved:** +- `local/sources/base/drivers/acpi/` — ACPI daemon +- `local/sources/base/drivers/hardware/acpid/` — AML interpreter + +**Test plan:** +1. Verify `\_S5` (shutdown): `poweroff` command → system powers off +2. Verify reset register: `reboot` command → system warm-boots +3. Verify `\_PS0`/`\_PS3`: CPU cores enter/exit power states (check via `redbear-power` TUI) +4. Verify `\_PPC`: CPU frequency scaling (check via `redbear-power`) +5. Test sleep states (S3 suspend-to-RAM): + - `echo mem > /sys/power/state` or equivalent + - System suspends + - Wake via keyboard or power button + - System resumes with working display and input + +**Linux reference:** +- `drivers/acpi/sleep.c:600-700` — `acpi_suspend_enter()` for S3 +- `drivers/acpi/processor_idle.c:900-1050` — C-state management +- `arch/x86/kernel/acpi/wakeup_64.S` — x86_64 wakeup trampoline + +**What to skip initially:** S4 (hibernate-to-disk) — requires full swap support. S3 (suspend-to-RAM) is the priority. + +**Estimated time:** 4–8 hours +**Dependencies:** Phase 5.1 (bare metal access). + +### 5.3 USB Storage and HID Validation + +**Context:** USB storage (usbscsid) and HID (usbhidd) drivers need real hardware testing. QEMU testing covers the happy path; real hardware covers edge cases. + +**Test plan:** +1. USB mass storage (flash drive): + - Insert USB flash drive + - Verify `usbscsid` auto-spawns + - Verify device appears as `/scheme/usbscsid/` + - Mount (if filesystem support is ready) + - Read/write test + - Hot-unplug while idle (should log, not panic) +2. USB keyboard: + - Boot with USB keyboard attached (no PS/2 keyboard) + - Verify `usbhidd` detects keyboard + - Verify typing works at login prompt + - Verify modifier keys (Shift, Ctrl, Alt) +3. USB mouse: + - Attach USB mouse + - Verify `usbhidd` detects mouse + - Verify cursor movement (if graphical session is running) + +**Linux reference:** `drivers/usb/storage/usb.c:1050-1100` — `usb_stor_control_thread()` with proper error recovery on device disconnect. Red Bear's usbscsid must handle the same scenario: device removed mid-transfer → error, not panic. + +**See also:** `IMPROVEMENT-PLAN.md` § 2.1 (usbscsid `.unwrap()` removal) — this must be done before hardware testing. + +**Estimated time:** 4–8 hours +**Dependencies:** Phase 5.1, `IMPROVEMENT-PLAN.md` P0 items complete. + +### 5.4 Wi-Fi and Bluetooth Maturity Assessment + +**Context:** Intel iwlwifi has expanded PCI ID table (37 devices, was 7), mini-MVM layer, and firmware TLV parser. Assess readiness for first hardware test. + +**Test plan:** +1. Verify iwlwifi driver builds and is in the ISO: `grep iwlwifi build/x86_64/redbear-full.iso.manifest` +2. Boot on hardware with Intel Wi-Fi (AC 7260, 8260, 9260, AX200, AX210) +3. Check `pcid:` log for the Wi-Fi device — it should be enumerated +4. Check `iwlwifi:` log for firmware load status +5. If firmware loads: attempt scan (`redbear-wifictl scan` or equivalent) +6. If scan succeeds: attempt connection to an open network +7. File bugs for any failure with PCI vendor/device ID, firmware version, and log excerpt + +**Linux reference:** +- `drivers/net/wireless/intel/iwlwifi/pcie/drv.c:785-830` — Linux's complete PCI ID table. Cross-reference Red Bear's table against this. +- `drivers/net/wireless/intel/iwlwifi/fw/file.h` — firmware TLV structure (same as Red Bear's `linux_mvm.c`) +- `drivers/net/wireless/intel/iwlwifi/mvm/fw.c:300-500` — firmware init sequence + +**See also:** +- `IMPROVEMENT-PLAN.md` § 6 — Wi-Fi subsystem improvements +- `WIFI-IMPLEMENTATION-PLAN.md` — Wi-Fi architecture plan + +**Estimated time:** 4–8 hours +**Dependencies:** Phase 5.1, `IMPROVEMENT-PLAN.md` P1-P2 Wi-Fi items. + +--- + +## Cross-Cutting: "Do Not Reinvent the Wheel" — Linux Kernel Reference Map + +For every major subsystem in Red Bear OS, consult the Linux 7.1 reference BEFORE implementing. +Linux's implementations are battle-tested over 30+ years. Porting proven algorithms is always +preferable to inventing heuristics. + +### Console/TTY Subsystem + +| Red Bear Component | Linux 7.1 Reference | What to Study | +|---|---|---| +| `fbcond/src/text.rs` | `drivers/tty/vt/keyboard.c` | Keycode→character translation, modifier handling | +| `console-draw/src/lib.rs` | `drivers/video/fbdev/core/bitblit.c` | Font rendering, glyph extraction, damage tracking | +| `console-draw` resize | `drivers/tty/vt/vt.c:resize_screen()` | Console resize: row copy, cursor preservation | +| `ptyd` | `drivers/tty/pty.c` | PTY master/slave pair, packet mode, window size ioctls | + +### Input Subsystem + +| Red Bear Component | Linux 7.1 Reference | What to Study | +|---|---|---| +| `ps2d` | `drivers/input/serio/` | PS/2 protocol, serio bus abstraction | +| `usbhidd` | `drivers/hid/usbhid/` | HID report parsing, input mapping | +| `evdevd` | `drivers/input/evdev.c` | evdev ioctl, read semantics, SYN_REPORT | +| `inputd` | `drivers/input/input.c` | Input core: registration, dispatch, repeat, LED | + +### USB Subsystem + +| Red Bear Component | Linux 7.1 Reference | What to Study | +|---|---|---| +| `xhcid` init | `drivers/usb/host/xhci.c:4896-5100` | Controller init sequence | +| `xhcid` quirks | `drivers/usb/host/xhci-pci.c:101-160` | Quirk table + enforcement | +| `xhcid` event ring | `drivers/usb/host/xhci-ring.c:550-590` | Ring expansion, overflow handling | +| `xhcid` TRB | `drivers/usb/host/xhci-ring.c:2400-2600` | Completion handling, EDTLA | +| `usbhubd` | `drivers/usb/core/hub.c` | Port power, reset, enumeration | +| `usbscsid` | `drivers/usb/storage/usb.c` | BOT/CBW/CSW protocol | + +### Netstack Subsystem + +| Red Bear Component | Linux 7.1 Reference | What to Study | +|---|---|---| +| `netstack` filter | `net/netfilter/` | Conntrack, NAT, filter chains | +| `netstack` bridge | `net/bridge/br_fdb.c` | MAC learning, aging | +| `netstack` qdisc | `net/sched/` | Token bucket, priority queue | +| `e1000d` | `drivers/net/ethernet/intel/e1000/` | Register-level init, interrupt handling | + +### ACPI/Power Subsystem + +| Red Bear Component | Linux 7.1 Reference | What to Study | +|---|---|---| +| `acpid` init | `drivers/acpi/acpica/` | ACPICA namespace init | +| `acpid` shutdown | `drivers/acpi/sleep.c:600-750` | `\_S5`, PM1a/PM1b register write | +| `acpid` power states | `drivers/acpi/processor_idle.c` | C-states, P-states | +| `thermald` | `drivers/thermal/` | Thermal zone management | + +### POSIX/GNU Compatibility (relibc) + +| Function | Linux 7.1 Reference | What to Study | +|----------|---------------------|---------------| +| `eventfd` | `fs/eventfd.c:60-120` | Counter semantics, POLLIN/POLLOUT | +| `signalfd` | `fs/signalfd.c:80-200` | Signal queue, siginfo packing | +| `timerfd` | `fs/timerfd.c:200-350` | CLOCK_MONOTONIC, CLOCK_REALTIME, cancel | +| `preadv/pwritev` | `fs/read_write.c:970-1025` | Scatter-gather I/O | +| `copy_file_range` | `fs/read_write.c:1505-1580` | Offloaded copy | +| `sem_open` | `ipc/sem.c:400-550` | Named semaphore, `/dev/shm` backing | +| `posix_spawn` | `kernel/fork.c:2900-2950` | Spawn without fork+exec | + +### General Advice + +1. **Before writing any new algorithm**, check if Linux has an equivalent. If yes, port the algorithm structure (not the code — we're Rust, Linux is C). +2. **For data structures**, prefer Linux's patterns: Red-black trees (`rbtree`), radix trees, linked lists (`list_head`), hash tables. Red Bear should use Rust equivalents (`BTreeMap`, `HashMap`, `VecDeque`). +3. **For quirk tables**, ALWAYS cross-reference Linux's `pci_ids.h`, `xhci-pci.c`, `usb_quirks.h`. Linux has already identified every quirky device. Port the table, do not rediscover bugs. +4. **For error recovery**, Linux's pattern is: log the error, return -EIO/-EINVAL/-ENOMEM, do NOT panic. Apply this universally. +5. **For timing/delays**, Linux uses `msleep()`, `usleep_range()`, `udelay()`. Red Bear should use equivalent primitives from `libredox` or `std::thread::sleep`. Never use busy-wait loops without bounded timeouts. + +--- + +## Summary: Execution Order and Milestones + +| Phase | Weeks | Key Deliverable | Blocks | +|-------|-------|-----------------|--------| +| **Phase 1** | Week 1 | All build blockers resolved. Clean base fork. Synced versions. | Nothing | +| **Phase 2** | Weeks 1–2 | Login-prompt works. Enter key, no text corruption, working PTY. | Phase 1 | +| **Phase 3** | Weeks 2–4 | All 9 forks synced to upstream. Netstack improved. USB quirks enforced. | Phase 2 | +| **Phase 4** | Weeks 4–8 | relibc 95% POSIX. Kernel syscalls complete. Input handling mature. | Phase 3 | +| **Phase 5** | Weeks 8–10 | Bare-metal boot proven. ACPI S3 sleep. USB storage/HID tested. Wi-Fi assessed. | Phase 4 | + +**Total: 10 weeks to production-ready baseline.** + +### Immediate Actions (Today) + +1. **Run `sync-versions.sh --check`** — identify version drift immediately +2. **`cd local/sources/base && git status --short`** — catalog WIP changes +3. **`cargo check` on xhcid** — confirm orphan `}` issue exists +4. **Read `IMPROVEMENT-PLAN.md` P0 items** — these are the USB safety fixes that block hardware testing +5. **Read `UPSTREAM-SYNC-PROCEDURE.md`** — prepare for Phase 3 fork syncs + +### Ongoing Discipline + +- Every commit to a local fork MUST also update the parent repo's submodule pointer +- Every upstream cherry-pick MUST be tested with `cargo check` + `repo cook` before push +- Every stub fixed MUST update `STUBS-FIX-PROGRESS.md` +- Every new Linux algorithm ported MUST include a comment referencing the specific Linux 7.1 file and line range