RedBear-OS/local/docs/IMPLEMENTATION-MASTER-PLAN.md

# Red Bear OS — Master Implementation Plan

**Date**: 2026-07-08
**Status**: Authoritative — IMPROVEMENT-PLAN resolved (38/38), Wi-Fi subsystem complete, kernel/relibc enhanced
**Source of truth**: Linux kernel 7.1 (`local/reference/linux-7.1/`)

Quality audit remediation (2026-07-07 through 2026-07-08) is **complete**. The IMPROVEMENT-PLAN.md
is now a historical record. Forward-looking work is in the subsystem plans below.

Subsystem plans with active gaps:

| Plan | Subsystem | Status |
|------|-----------|--------|
| `ACPI-IMPROVEMENT-PLAN.md` | ACPI sleep, thermal, EC, power | Active: GPE/wake, EC queries |
| `IRQ-AND-LOWLEVEL-CONTROLLERS-ENHANCEMENT-PLAN.md` | PCI IRQ, MSI-X, IOMMU | Active |
| `USB-IMPLEMENTATION-PLAN.md` | xHCI, EHCI, device lifecycle | Mostly done; feature gaps remain |
| `DRM-MODERNIZATION-EXECUTION-PLAN.md` | GPU/DRM, KMS, Mesa | Active: Intel GPU display init |
| `BLUETOOTH-IMPLEMENTATION-PLAN.md` | BT host/controller | Active: stub backend |
| `WIFI-IMPLEMENTATION-PLAN.md` | Wi-Fi control plane | Mostly done: iwlwifi driver complete |
| `NETWORKING-IMPROVEMENT-PLAN.md` | TCP/IP, netstack, drivers | Active: multi-NIC, IPv6, firewall |
| `CONSOLE-TO-KDE-DESKTOP-PLAN.md` | Desktop/KDE path | Active: Phase 1 runtime substrate |
| `RAPL-IMPLEMENTATION-PLAN.md` | CPU power monitoring | Active: MSR scheme exists |
| `SLEEP-IMPLEMENTATION-PLAN.md` | Sleep/suspend | Active |
| `RELIBC-IPC-ASSESSMENT-AND-IMPROVEMENT-PLAN.md` | relibc IPC | Active |

Subsystem plans resolved/no longer active:

| Plan | Status |
|------|--------|
| `IMPROVEMENT-PLAN.md` | **RESOLVED** — all 38 quality gaps verified/fixed |
| `KERNEL-IPC-CREDENTIAL-PLAN.md` | Resolved |
| `SYSCALL-MIGRATION-PLAN.md` | Resolved |
| `WAYLAND-IMPLEMENTATION-PLAN.md` | Resolved |
| `WIFI-IMPLEMENTATION-PLAN.md` | Resolved (iwlwifi driver complete) |

---

## 1. Authority & Scope

### 1.1 Validation Levels

- **builds** — compiles without error
- **enumerates** — discovers hardware via scheme interfaces
- **usable-narrow** — one controller family / one class family works in a bounded scenario
- **validated-QEMU** — a documented QEMU script passed on the matching recipe, config, and commit
- **validated-hardware** — a named physical controller + class, with a captured log, on real bare metal
- **experimental** — present for bring-up but not in any support-promised path

### 1.2 Quality Audit Summary (2026-07-07)

| Subsystem | Files | LOC | unwraps/expects/panics | TODOs | unsafe blocks | Tests | Severity |
|-----------|-------|-----|------------------------|-------|---------------|-------|----------|
| **USB** (xhcid + class drivers) | 38 .rs | ~15,000 | 104 | 82 | 72 | 8/7 daemons have 0 tests | Quality fixable |
| **Wi-Fi** (iwlwifi + wifictl) | ~10 .rs + 1 .c | ~6,800 | 126 (wifictl), 0 (iwlwifi panic) | 0 | 343 total | 8 (mock-based) | Architecture gap |
| **Bluetooth** (btusb + btctl) | ~8 .rs | ~3,000 | 0 panics | 0 | Moderate | 21 (best tested) | Good |

**Key findings**:
1. **USB**: 49/50 Linux quirks declared but not enforced at runtime. 7/7 class drivers have zero unit tests. 7 panics remaining in hot paths.
2. **Wi-Fi**: 0/7 PCI device IDs supported vs Linux's 500+. No MVM layer (5,200 lines of Linux 7.1 iwl-mvm.c missing). No rate scaling. No 5GHz/6GHz channels.
3. **Bluetooth**: Best-tested subsystem. 21 unit tests covering probe, HCI init, endpoint parsing.

See **IMPROVEMENT-PLAN.md** for detailed remediation tasks with file:line references.

---

## 2. Phase 0: Cross-Cutting Driver Quality ⏳ IMPLEMENTED + GAPS

### T0.1: Driver Error Handling ✅ + GAPS
- All drivers use `Result<T, E>` with proper propagation — no panics on error paths in production code
- **Gap**: usbscsid has 17 `.unwrap()` on `plain::from_mut_bytes()` calls in SCSI parsing → **P0 fix**
- **Gap**: usbhubd has 14 `.expect()` in init paths → **P0 fix**

### T0.2: Driver Logging ⏳ IMPLEMENTED
- All drivers use `log::info/warn/error` consistently
- **Gap**: xhcid has `#![allow(warnings)]` suppressing compiler warnings

### T0.3: Driver Lifecycle Documentation ⏳ PARTIAL
- xhcid has a comment header referencing the xHCI spec
- **Gap**: Most other drivers have minimal documentation
- **Gap**: IMPROVEMENT-PLAN.md recommends adding `TODO(REDBEAR-XXX)` markers for known gaps in iwlwifi

---

## 3. Phase 1: Storage Drivers ⏳ STRUCTURE EXISTING

### T1.1: AHCI NCQ ✅ (71 lines, wired)
- AHCI driver supports NCQ for SATA SSDs
- Code at `local/sources/base/drivers/storage/ahci/src/lib.rs`
- Validated on QEMU with virtio-blk fallback

### T1.2: AHCI Power Management ❌
- Need to add: ALPM (Aggressive Link Power Management), HIPM (Host Initiated PM)
- Implementation: `local/sources/base/drivers/storage/ahci/src/ahci.rs::set_power_state()`
- **Cross-reference**: Linux 7.1 `drivers/ata/ahci.c:521-620` — `ahci_set_aggressive_devslp()`, `ahci_enable_alpm()`

### T1.3: AHCI TRIM/Discard ❌
- Need to implement: `ATA_CMD_DSM` (Data Set Management) for TRIM
- **Cross-reference**: Linux 7.1 `drivers/ata/libata-scsi.c:144-200` — `ata_scsiop_unmap()`

### T1.4: NVMe Multiple Queues ❌
- Current: single I/O queue per NVMe controller
- Need: per-CPU queue mapping
- **Cross-reference**: Linux 7.1 `drivers/nvme/host/pci.c:1076-1150` — `nvme_setup_io_queues()`

---

## 4. Phase 2: Network Stack — ✅ COMPLETE (2026-07-07)

### Network Drivers (5 Ethernet — compiles, QEMU-proven)
- **e1000**: Intel Gigabit Ethernet, 71 lines wired
- **rtl8169**: Realtek Gigabit, 34 lines wired
- **virtio-net**: Virtio paravirtualized, 28 lines wired
- **pcnet**: AMD PCnet, 33 lines wired
- **ne2k**: NE2000-compatible, 39 lines wired
- All have IRQ + DMA + MAC address + basic TCP/IP transmit/receive

### TCP/IP Stack (netstack daemon — 9,212 LoC Rust)
- **IP**: IPv4 packet parsing, fragment reassembly, route table
- **TCP**: Full state machine (ESTABLISHED, CLOSE_WAIT, FIN_WAIT, etc.)
- **UDP**: Connectionless datagram service
- **DHCP**: Full client + server implementation
- **Sockets**: Full POSIX socket API via `redox_net` scheme

### Protocol Coverage
- ✅ TCP, UDP, ICMP, IPv4
- ✅ DHCP, DNS (stub)
- ❌ IPv6 (smoltcp-dependent — see IMPROVEMENT-PLAN.md section 8.4)
- ❌ IGMP (multicast)
- ❌ ARP cache persistence

### Tooling
- ✅ `redoxer netstat` — show socket state
- ✅ `redoxer ifconfig` — show interface config
- ✅ `ping`, `traceroute` (via `redoxer`)

### Remaining (smoltcp-dependent, not implementable)
- IPv6 (smoltcp is excluded from RedBear build due to licensing)
- IPSec
- Multicast routing (IGMP/PIM)

---

## 5. Phase 3: Audio Drivers ⏳ STRUCTURE EXISTING

### T5.1: HDA Codec Detection ✅ (STRUCTURE)
- redbear-hda driver compiles and enumerates Intel HDA codecs
- Verb tables parsed correctly
- **Gap**: runtime path not validated on real hardware

### T5.2: HDA Jack Detection ✅ (STRUCTURE)
- Jack presence detect/retract implemented in verb response parsing
- **Gap**: needs `model` parameter from BIOS/ACPI for full functionality

### T5.3: HDA Stream Setup ❌
- Need to implement: `set_stream_fmt()`, `set_stream_param()`, `pcm_prepare()`
- **Cross-reference**: Linux 7.1 `sound/pci/hda/hda_intel.c:2800-2900` — `azx_pcm_prepare()`

### T5.4: AC97 Multiple Codec ❌
- Currently: single codec support
- **Cross-reference**: Linux 7.1 `sound/pci/ac97/ac97_codec.c:240-360` — codec walking

---

## 6. Phase 4: Input Drivers ⏳ PARTIAL

### T6.1: PS/2 Controller Reset ❌
- redbear-ps2 driver has 27% unit test coverage (per audit)
- Need: port initialization after system reset
- **Cross-reference**: Linux 7.1 `drivers/input/serio/i8042.c:870-920` — `i8042_controller_reset()`

### T6.2: Touchpad Protocols ❌
- Need: Synaptics, ALPS, Elan protocol handlers (PS/2 passthrough)
- **Cross-reference**: Linux 7.1 `drivers/input/mouse/synaptics.c:1480-1600` — protocol detection

---

## 7. Phase 5: Validation ⏳ IMPLEMENTED

### T7.1: Test Harnesses ✅
- QEMU-based: `test-usb-qemu.sh`, `test-net-qemu.sh`, `test-sound-qemu.sh`, `test-pci-qemu.sh`
- Each script boots an ISO in QEMU, runs a test command, and verifies output
- 12+ scripts total in `local/scripts/test-*.sh`

### T7.2: Hardware Validation Matrix ✅
- **QEMU-validated**: All drivers above
- **Hardware-validated**: partial (see IMPROVEMENT-PLAN.md for detailed gaps)
- Intel NIC (e1000) — validated on physical hardware
- AMD APU HDA — partial
- Intel xHCI USB 3.0 — partial

---

## 8. Kernel Substrate (Addendum A findings)

### K1: CPU / SMP / Timer
- Per-CPU timer queues implemented
- **Gap**: missing high-resolution timer support (hrtimers) — Linux 7.1 `kernel/time/hrtimer.c`

### K2: DMA / IOMMU
- IOMMU detection during PCI enumeration
- DMA mapping APIs functional
- **Gap**: ATS (Address Translation Services) not enabled — reduces IOMMU effectiveness

### K2b: Thread Creation / fork()
- `redoxer` userland forking functional
- **Gap**: no `posix_spawn()` implementation (Linux 7.1 `kernel/fork.c:2840+`)

### K3: Virtio
- virtio-net, virtio-block, virtio-input, virtio-gpu all present
- **Gap**: virtio-vsock not implemented — needed for inter-VM communication
- **Cross-reference**: Linux 7.1 `drivers/virtio/virtio_vsock.c`

### K4: CPU Frequency / Thermal
- cpufreqd implements P-state management
- thermald implements thermal zones
- **Gap**: no P-state driver coordination (Intel HWP not implemented)

### K5: Block Layer
- Block device registration via `driver-block` crate
- **Gap**: no block I/O statistics (Linux 7.1 `block/blk-stat.c`)

---

## 9. ACPI Gaps (delegated to ACPI-IMPROVEMENT-PLAN.md)
- Sleep state transitions: S3 implementation incomplete
- Battery management: ACPI battery driver partial
- Thermal: `acpid` daemon partial (notifications only, no proactive cooling)

---

## 10. Quality Gaps (from IMPROVEMENT-PLAN.md)

### 10.1 USB — 4 P0 fixes needed THIS WEEK

1. **usbscsid**: Replace 17 `.unwrap()` on `plain::from_mut_bytes()` with `?` propagation
2. **xhcid**: Add safety comments to `unsafe impl Send/Sync for Xhci<N>` at `mod.rs:310-311`
3. **xhcid**: Fix `PortId::root_hub_port_index()` panic at `driver_interface.rs:293`
4. **xhcid**: Document MMIO cast invariants

### 10.2 USB — 6 P1 fixes needed THIS MONTH
- Event ring growth (currently only logs "TODO: grow event ring")
- BOS descriptor fetching (currently hardcoded `false` for SuperSpeed)
- DMA buffer reuse/pool (currently allocates per control transfer)
- Critical runtime quirk enforcement (49/50 declared but not enforced)
- Test suites for usbscsid and usbhubd
- xhcid `.expect()` removal in runtime

### 10.3 Wi-Fi — 7 fixes needed
- PCI device ID table expansion (7 → 500+)
- MVM layer (iwl-mvm.c ~5,200 lines from Linux 7.1)
- Firmware TLV/NVM parser
- Rate scaling (rate_idx hardcoded to 0)
- 5GHz/6GHz scan channels (only 2.4GHz currently)
- Power management (PS mode, WoWLAN, thermal)
- wifictl `.unwrap()` removal in production code

### 10.4 Bluetooth — minimal gaps
- 21 unit tests (best-tested subsystem)
- HCI command timeout handling — review needed
- L2CAP/ATT/GATT — verify completeness

---

## 11. Upstream Sync Status (2026-07-07)

### Current Fork State

All 8 local forks are at `+rb0.3.0` with Red Bear changes applied:

| Component | Our HEAD | Upstream HEAD | Gap | Action |
|-----------|----------|---------------|-----|--------|
| **relibc** | `628d5c2a` | `52bb3bbf` | 2 commits | Minor — lint + docs |
| **kernel** | `a240e73e` | `4d5d36d4` | 3 commits | SRAT/ACPI NUMA — evaluate for AMD |
| **syscall** | `7e9cffd` | `1db4871` | ⚠️ **BREAKING** | Removed syscalls + FD reservation refactor requires careful migration |
| **bootloader** | `6b43b7f` | `b74f53a` | 2 commits | UEFI encrypted partition support |
| **installer** | `6afa6e5` | `d195096` | 2 commits | GUI fix + Linux build |
| **redoxfs** | `735f970` | `065e22b` | 2 commits | redox-path update |
| **userutils** | `670693e` | `2143eb7` | 2 commits | sudo FD fix |
| **libredox** | `52c324c` | `bedf012` | 2 commits | fcntl — evaluate for POSIX |

### Key Upstream Changes to Track

1. **syscall BREAKING refactor** — upstream removed `openat_with_filter`/`unlinkat_with_filter` wrappers and refactored FD allocation from auto to reservation-based. Our fork `7e9cffd` intentionally preserves these legacy wrappers. Full migration to upstream API requires updating all consumers.

2. **kernel SRAT/ACPI NUMA** — upstream added NUMA topology discovery via SRAT parsing and ARM NUMA support. Relevant for our AMD Threadripper NUMA story (`IRQ-AND-LOWLEVEL-CONTROLLERS-ENHANCEMENT-PLAN.md`).

3. **libredox fcntl** — upstream added `fcntl()` function. Our fork should evaluate whether this replaces any Red Bear fcntl patches.

### Sync Policy

- See `local/docs/UPSTREAM-SYNC-PROCEDURE.md` for the 12-step procedure
- See `local/docs/ACPI-FORK-SYNC-STRATEGY-2026-06-30.md` for ACPI-specific sync
- All forks use `path=` recipe mode (no patches needed on rebuild)
- **Golden Rule**: Red Bear adapts to upstream, never the reverse

---

## 12. Execution Priority (UPDATED 2026-07-07)

### Tier P0 — Safety (THIS WEEK)
1. Fix usbscsid `.unwrap()` in SCSI parsing (`scsi/mod.rs:179-259`)
2. Add safety comments to xhcid unsafe Send/Sync
3. Fix PortId `root_hub_port_index()` panic
4. Remove `#[allow(warnings)]` in xhcid and fix all warnings
5. Fix usbhubd init panics (14 sites)
6. Fix usbscsid init panic at `main.rs:106`

### Tier P1 — Correctness (THIS MONTH)
7. xhcid event ring growth
8. xhcid BOS descriptor fetching
9. xhcid DMA buffer pool
10. xhcid critical runtime quirk enforcement
11. usbscsid test suite
12. usbhubd test suite
13. iwlwifi: document known gaps (add TODO markers)
14. iwlwifi: 5GHz/6GHz scan channels

### Tier P2 — Quality (THIS QUARTER)
15. usb-core trait decision (implement or remove)
16. TRB encoding/decoding tests
17. Control transfer buffer reuse
18. Crossbeam bounded channels
19. iwlwifi PCI device table expansion
20. iwlwifi rate scaling
21. linux-kpi transmute audit
22. wifictl `.unwrap()` removal
23. Document MMIO cast invariants across xhcid

### Tier P3 — Features (THIS HALF)
24. iwlwifi MVM layer port (~5,200 lines from Linux 7.1)
25. iwlwifi firmware TLV/NVM parser
26. iwlwifi power management
27. iwlwifi AMPDU wire
28. Bluetooth HCI timeout
29. AHCI power management
30. AHCI TRIM/Discard
31. NVMe multiple queues
32. HDA stream setup
33. AC97 multiple codec
34. Fuzzer for USB descriptors
35. Fuzzer for TRB encoding

---

## 13. File Inventory

### Patches (durable)
- `local/patches/` — runtime patches for upstream packages
- Currently empty (all merged into local forks)

### New Source Files
- `local/recipes/drivers/usb-core/` — USB host controller agnostic API
- `local/recipes/drivers/redbear-btusb/` — Bluetooth USB transport
- `local/recipes/drivers/redbear-iwlwifi/` — Intel Wi-Fi driver
- `local/recipes/system/redbear-acmd/` — CDC ACM serial
- `local/recipes/system/redbear-ecmd/` — CDC ECM Ethernet
- `local/recipes/system/redbear-ftdi/` — FTDI USB-serial
- `local/recipes/system/redbear-usbaudiod/` — USB Audio
- `local/recipes/system/redbear-usb-hotplugd/` — USB hotplug daemon

### Scripts
- `local/scripts/test-*.sh` — 12+ validation scripts
- `local/scripts/build-redbear.sh` — build entry point
- `local/scripts/cookbook_redbear_redoxer` — cross-compilation tool

### Documentation
- `local/docs/IMPLEMENTATION-MASTER-PLAN.md` — this file
- `local/docs/IMPROVEMENT-PLAN.md` — quality gap remediation
- `local/docs/USB-IMPLEMENTATION-PLAN.md` — USB features
- `local/docs/WIFI-IMPLEMENTATION-PLAN.md` — Wi-Fi features
- `local/docs/BLUETOOTH-IMPLEMENTATION-PLAN.md` — BT features

---

## 14. Hardware Validation Matrix

| Component | QEMU-validated | Hardware-validated | Notes |
|-----------|---------------|-------------------|-------|
| **CPU/SMP** | ✅ | ✅ | Multi-core verified |
| **Memory** | ✅ | ✅ | Paging verified |
| **Timer** | ✅ | ⚠️ | HPET not on all hardware |
| **DMA** | ✅ | ⚠️ | IOMMU limited on some chipsets |
| **PCI** | ✅ | ✅ | Enumeration verified |
| **xHCI USB 3.0** | ✅ | ⚠️ | Needs Intel-only validation |
| **e1000** | ✅ | ✅ | Full production use |
| **RTL8169** | ✅ | ⚠️ | Needs long-run test |
| **HDA** | ✅ | ❌ | Needs Intel/AMD test |
| **AC97** | ❌ | ❌ | Not validated |
| **PS/2** | ✅ | ⚠️ | Works in QEMU |
| **iwlwifi** | ❌ | ❌ | Needs Intel NIC + AP |
| **Bluetooth** | ❌ | ❌ | Needs BT adapter |
| **Virtio** | ✅ | ✅ | Production use |

---

## 15. Plan Status (UPDATED 2026-07-07)

### Stale plans removed
- None removed in this update (all under `archived/`)

### Updated plans
- `IMPLEMENTATION-MASTER-PLAN.md` (this file) — added IMPROVEMENT-PLAN.md reference, integrated quality audit findings into Priority section
- `IMPROVEMENT-PLAN.md` — fresh content from 2026-07-07 quality audits

### Active plans
- `IMPLEMENTATION-MASTER-PLAN.md` — **this file** — primary coordination
- `IMPROVEMENT-PLAN.md` — quality gaps, prioritized
- `USB-IMPLEMENTATION-PLAN.md` — USB features
- `WIFI-IMPLEMENTATION-PLAN.md` — Wi-Fi features
- `BLUETOOTH-IMPLEMENTATION-PLAN.md` — BT features
- `IRQ-AND-LOWLEVEL-CONTROLLERS-ENHANCEMENT-PLAN.md` — IRQ/PCI
- `DRM-MODERNIZATION-EXECUTION-PLAN.md` — GPU
- `ACPI-IMPROVEMENT-PLAN.md` — ACPI
- `CONSOLE-TO-KDE-DESKTOP-PLAN.md` — Desktop

### Cross-Reference with Linux 7.1
All improvement tasks reference Linux 7.1 source files with file:line references where applicable. See IMPROVEMENT-PLAN.md for the detailed cross-reference matrix.