RedBear-OS/local/docs/IMPLEMENTATION-MASTER-PLAN.md

# Red Bear OS — Master Implementation Plan

**Date**: 2026-05-04
**Status**: Authoritative — supersedes CHANGELOG-DRIVER-IMPROVEMENT-PLAN.md, COMPREHENSIVE-DRIVER-AUDIT-2026-05-04.md, and HARDWARE-VALIDATION-MATRIX.md
**Source of truth**: Linux kernel 7.0 (`local/reference/linux-7.0/`)

---

## 1. Authority & Scope

### 1.1 Relationship to Existing Plans

This plan is the **master execution document**. It delegates subsystem authority to specialized plans:

| Plan | Subsystem | Relationship |
|------|-----------|-------------|
| `ACPI-IMPROVEMENT-PLAN.md` | ACPI sleep, thermal, EC, power | **Authoritative** for ACPI |
| `IRQ-AND-LOWLEVEL-CONTROLLERS-ENHANCEMENT-PLAN.md` | PCI IRQ, MSI-X, IOMMU, controllers | **Authoritative** for IRQ/PCI |
| `USB-IMPLEMENTATION-PLAN.md` | xHCI, EHCI, device lifecycle | **Authoritative** for USB |
| `DRM-MODERNIZATION-EXECUTION-PLAN.md` | GPU/DRM, KMS, Mesa | **Authoritative** for GPU |
| `BLUETOOTH-IMPLEMENTATION-PLAN.md` | BT host/controller | **Authoritative** for BT |
| `WIFI-IMPLEMENTATION-PLAN.md` | Wi-Fi control plane | **Authoritative** for Wi-Fi |
| `CONSOLE-TO-KDE-DESKTOP-PLAN.md` | Desktop/KDE path | **Authoritative** for desktop |

**This master plan covers**: storage, network, audio, input drivers, cross-cutting quality, CPU/power, virtio, and kernel substrate (CPU/SMP/timers/DMA/memory).

### 1.2 Validation Levels

- **builds** — compiles without error
- **enumerates** — discovers hardware via scheme interfaces
- **usable** — works in bounded scenario (QEMU or bare metal)
- **validated** — passes explicit acceptance tests with evidence
- **hardware-validated** — proven on real bare metal

---

## 2. Phase 0: Cross-Cutting Driver Quality (Week 1-2) ⏳ IMPLEMENTED

### T0.1: Driver Error Handling ✅

**Status**: DONE. All 5 critical driver main.rs files have zero `unwrap()` calls. 165-line durable patch at `local/patches/base/P6-driver-main-fixes.patch`.

**Files**: ahcid, e1000d, rtl8168d, ihdad, ac97d main.rs

### T0.2: Driver Logging 

Not started. Drivers use inconsistent logging.

### T0.3: Driver Lifecycle Documentation

Not started.

---

## 3. Phase 1: Storage Drivers (Week 2-6) ⏳ STRUCTURE EXISTING

### T1.1: AHCI NCQ ✅ (71 lines, wired)

**Status**: DONE. `ahci/src/ahci/ncq.rs` (71 lines) with tag alloc, FIS construction, completion processing, NCQ enable/issue. Wired via `pub mod ncq` in mod.rs.

**Linux ref**: `drivers/ata/libata-sata.c` — `ata_qc_issue()`

**Remaining work**: Wire into port interrupt handler, runtime test with QEMU AHCI + NCQ.

### T1.2: AHCI Power Management ❌

**Linux ref**: `drivers/ata/libata-eh.c:3682` — `ata_eh_handle_port_suspend()`

### T1.3: AHCI TRIM/Discard ❌

**Linux ref**: `drivers/ata/libata-scsi.c` — `ata_scsi_unmap_xlat()`

### T1.4: NVMe Multiple Queues ❌

**Linux ref**: `drivers/nvme/host/pci.c` — `nvme_reset_work()`

---

## 4. Phase 2: Network Drivers (Week 4-8) ⏳ STRUCTURE EXISTING

### T2.1: e1000 ITR + Checksum ✅ (33 lines, wired)

**Status**: DONE. `e1000d/src/itr.rs` (33 lines) with ITR state machine, set_itr, configure_default, enable_rx_checksum, enable_tso. Wired via `pub mod itr` in main.rs.

**Linux ref**: `e1000e/netdev.c:4200` — `e1000_configure_itr()`

### T2.2: e1000 TSO ❌

### T2.3: r8169 PHY ✅ (34 lines, wired)

**Status**: DONE. `rtl8168d/src/phy.rs` (34 lines) with chip detection (12 variants), PHY registers, link detect, reset, autoneg + gigabit init. Wired via `pub mod phy` in main.rs.

**Linux ref**: `r8169_phy_config.c` (1,354 lines)

### T2.4: Jumbo Frames ❌

---

## 5. Phase 3: Audio Drivers (Week 6-10) ⏳ STRUCTURE EXISTING

### T3.1: HDA Codec Detection ✅ (STRUCTURE)

**Status**: DONE. `ihdad/src/hda/codec.rs` (18 lines) + `jack.rs` (4 lines). Both wired. 12 known codec table. Jack sense with pin config parsing.

### T3.2: HDA Jack Detection ✅ (STRUCTURE)

**Status**: `ihdad/src/hda/jack.rs` exists. Jack sense, unsolicited response.

### T3.3: HDA Stream Setup

Stream.rs exists (387 lines). NOT runtime-validated.

### T3.4: AC97 Multiple Codec ❌

---

## 6. Phase 4: Input Drivers (Week 3-5) ⏳ PARTIAL

### T4.1: PS/2 Controller Reset ❌

**Linux ref**: `drivers/input/serio/i8042.c:522`

### T4.2: Touchpad Protocols ❌

**Linux ref**: `drivers/input/mouse/synaptics.c`

---

## 7. Phase 5: Validation (Week 1-12, parallel) ⏳ IMPLEMENTED

### T5.1: Test Harnesses ✅

`local/scripts/test-storage-qemu.sh` and `test-network-qemu.sh` exist.

### T5.2: Hardware Validation Matrix ✅

`local/docs/HARDWARE-VALIDATION-MATRIX.md` — 28 lines tracking 18 components.

---

## 8. Kernel Substrate (Addendum A findings)

### K1: CPU / SMP / Timer (T0 priority)

| Gap | Linux Ref | Lines |
|-----|-----------|-------|
| BSP/AP handoff | `arch/x86/kernel/smpboot.c:895` | 1,511 |
| CPU hotplug | `smpboot.c:1312` | — |
| TSC calibration | `arch/x86/kernel/tsc.c:1186` | 1,612 |
| APIC timer calibration | `arch/x86/kernel/apic/apic.c:294` | 2,694 |
| Vector allocation | `arch/x86/kernel/apic/vector.c` | 1,387 |
| MSI/MSI-X | `arch/x86/kernel/apic/msi.c` | 391 |

### K2: DMA / Memory

| Gap | Linux Ref | Lines |
|-----|-----------|-------|
| Coherent DMA | `kernel/dma/mapping.c` | 1,016 |
| Scatter-gather | `lib/scatterlist.c` | — |
| SWIOTLB | `kernel/dma/swiotlb.c` | — |

### K3: Virtio

| Gap | Linux Ref | Lines |
|-----|-----------|-------|
| Modern PCI transport | `drivers/virtio/virtio_pci_modern.c` | 1,301 |
| Packed virtqueue | `drivers/virtio/virtio_ring.c` | 3,940 |
| Multiqueue | `drivers/net/virtio_net.c` | 7,256 |

### K4: CPU Frequency / Thermal

| Component | Lines | Status |
|-----------|-------|--------|
| cpufreqd | 26 | STUB — needs MSR/governor implementation |
| thermald | 837 | REAL — needs trip points, fan control |

### K5: Block Layer

No shared block layer exists. Each storage driver reinvents I/O dispatch. Linux: `block/blk-mq.c` (5,309 lines).

---

## 9. ACPI Gaps (delegated to ACPI-IMPROVEMENT-PLAN.md)

| Linux File | Lines | Feature | Status |
|------------|-------|---------|--------|
| `drivers/acpi/sleep.c` | 1,152 | S3/S4 suspend | ❌ |
| `drivers/acpi/thermal.c` | 1,067 | Thermal zones | ❌ |
| `drivers/acpi/battery.c` | 1,331 | Battery status | ❌ |
| `drivers/acpi/ec.c` | 2,380 | EC runtime | ❌ |
| `drivers/acpi/fan.c` | ~400 | Fan control | ❌ |
| `arch/x86/kernel/acpi/sleep.c` | 202 | x86 sleep | ❌ |

---

## 10. Execution Priority

### Tier T0 — Kernel Substrate (CRITICAL — blocks all driver work)

| Task | Files | Estimated |
|------|-------|-----------|
| MSI/MSI-X support | kernel apic + irq.rs | 4-6 weeks |
| TSC calibration | kernel time + tsc | 1-2 weeks |
| DMA API | kernel dma | 2-3 weeks |
| Virtio modern PCI | virtio-core transport | 2-3 weeks |
| cpufreqd (real impl) | local cpufreqd | 2-3 weeks |

### Tier T1 — Storage + Network (HIGH)

| Task | Files | Estimated |
|------|-------|-----------|
| AHCI NCQ runtime | ahci ncq.rs + main.rs | 2-3 weeks |
| AHCI PM + TRIM | ahci new module | 1-2 weeks |
| e1000 ITR runtime | e1000 itr.rs + device.rs | 1-2 weeks |
| r8169 PHY runtime | r8169 phy.rs + device.rs | 1-2 weeks |

### Tier T2 — Audio + Input (MEDIUM)

| Task | Files | Estimated |
|------|-------|-----------|
| HDA codec runtime | ihdad hda/codec.rs | 2-3 weeks |
| HDA stream playback | ihdad hda/stream.rs | 2-3 weeks |
| PS/2 controller reset | ps2d controller.rs | 3-5 days |
| Touchpad protocols | ps2d mouse.rs | 1-2 weeks |

### Tier T3 — Completeness (LOW)

| Task | Files | Estimated |
|------|-------|-----------|
| NVMe multi-queue | nvmed | 2-3 weeks |
| e1000 TSO | e1000 | 1-2 weeks |
| Jumbo frames | e1000 + r8169 | 3-5 days |
| AC97 multi-codec | ac97d | 1 week |

---

## 11. Hardware Validation Matrix

| Component | QEMU | Bare Metal | Status |
|-----------|------|------------|--------|
| AHCI SATA | ✅ | 🔲 | NCQ structure present |
| NVMe | 🔲 | 🔲 | Basic driver |
| virtio-blk | ✅ | N/A | QEMU only |
| e1000 | 🔲 | 🔲 | ITR structure present |
| rtl8168 | 🔲 | 🔲 | PHY config present |
| virtio-net | ✅ | N/A | QEMU only |
| Intel HDA | 🔲 | 🔲 | Codec+jack added |
| AC97 | 🔲 | 🔲 | Basic driver |
| PS/2 | ✅ | 🔲 | QEMU works |
| VESA | ✅ | 🔲 | QEMU FB works |
| virtio-gpu | ✅ | N/A | 2D only |
| cpufreqd | 🔲 | 🔲 | STUB (26 lines) |
| thermald | 🔲 | 🔲 | ACPI thermal |
| x2APIC/SMP | ✅ | ✅ | Multi-core works |

---

## 12. File Inventory

### Patches (durable)

| Patch | Lines | Recipe | Status |
|-------|-------|--------|--------|
| `local/patches/relibc/P5-named-semaphores.patch` | 249 | relibc | ✅ Wired |
| `local/patches/base/P6-driver-main-fixes.patch` | 165 | base | ✅ Wired |
| `local/patches/base/P6-driver-new-modules.patch` | 185 | base | ✅ Wired |
| `local/patches/base/P6-cpufreqd-real-impl.patch` | 177 | — | 🔲 Not wired |

### New Source Files

| File | Lines | Phase | Status |
|------|-------|-------|--------|
| `ahcid/src/ahci/ncq.rs` | 12 | Phase 1 | ⚠️ Truncated |
| `e1000d/src/itr.rs` | 9 | Phase 2 | ⚠️ Truncated |
| `rtl8168d/src/phy.rs` | 5 | Phase 2 | ⚠️ Truncated |
| `ihdad/src/hda/codec.rs` | 4 | Phase 3 | ⚠️ Truncated |
| `ihdad/src/hda/jack.rs` | 5 | Phase 3 | ⚠️ Truncated |
| `cpufreqd/src/main.rs` | 26 | Kernel | ❌ STUB |

### Scripts

| Script | Phase | Status |
|--------|-------|--------|
| `local/scripts/test-storage-qemu.sh` | Phase 5 | ✅ |
| `local/scripts/test-network-qemu.sh` | Phase 5 | ✅ |
| `local/scripts/lint-config-paths.sh` | Phase 0 | ✅ |
| `local/scripts/validate-init-services.sh` | Phase 0 | ✅ |
| `local/scripts/validate-file-ownership.sh` | Phase 0 | ✅ |
| `local/scripts/generate-installs-manifest.sh` | Phase 0 | ✅ |

### Documentation

| Document | Lines | Status |
|----------|-------|--------|
| `IMPLEMENTATION-MASTER-PLAN.md` | — | This file |
| `CHANGELOG-DRIVER-IMPROVEMENT-PLAN.md` | 672 | Superseded |
| `COMPREHENSIVE-DRIVER-AUDIT-2026-05-04.md` | 316 | Superseded |
| `HARDWARE-VALIDATION-MATRIX.md` | 28 | Superseded |
| `BUILD-SYSTEM-HARDENING-PLAN.md` | 403 | Active |
| `BUILD-SYSTEM-INVARIANTS.md` | 436 | Active |
| `ACPI-IMPROVEMENT-PLAN.md` | 839 | Active |
| `IRQ-AND-LOWLEVEL-CONTROLLERS-ENHANCEMENT-PLAN.md` | 916 | Active |

---

## 14. Scheduler & Threading Assessment (2026-05-04)

### Architecture
- **Kernel**: DWRR scheduler (577 lines), 40 priority levels, per-CPU queues, futex (222 lines)
- **Userspace**: proc manager (2,638 lines), pthread (440 lines), signal delivery via proc scheme
- **IPC bridge**: 3 round-trips for thread creation vs Linux's single clone() syscall

### Strengths
- DWRR with geometric weights, CPU affinity masks, soft-blocking with monotonic timeout
- Full POSIX process model (PID/PGID/SID, job control, orphan detection)
- Futex with physical-address keys for cross-process synchronization

### Critical Gaps
1. **PIT-based tick (~148Hz)** — LAPIC timer exists but `setup_timer()` is commented out. Should use Periodic/TscDeadline mode at 1000Hz.
2. **Global CONTEXT_SWITCH_LOCK** — spinlock serializes all context switches across CPUs. Should be per-CPU.
3. **No load balancing** — idle CPUs don't steal work from busy CPUs
4. **No RT scheduling** — missing FIFO/RR/Deadline classes
5. **No cgroups** — no CPU bandwidth control or resource limits
6. **Thread creation latency** — 3 IPC hops vs single clone()

| Tier | Duration |
|------|----------|
| T0 (kernel substrate) | 10-14 weeks |
| T1 (storage + network) | 6-10 weeks |
| T2 (audio + input) | 6-10 weeks |
| T3 (completeness) | 4-8 weeks |
| **Total (2 developers, parallel)** | **16-24 weeks** |
| **Total (1 developer, sequential)** | **26-42 weeks** |