diff --git a/local/docs/NETWORKING-IMPROVEMENT-PLAN.md b/local/docs/NETWORKING-IMPROVEMENT-PLAN.md new file mode 100644 index 0000000000..2cce619580 --- /dev/null +++ b/local/docs/NETWORKING-IMPROVEMENT-PLAN.md @@ -0,0 +1,959 @@ +# Red Bear OS Networking and TCP/IP Stack Improvement Plan + +## Purpose + +This document assesses the current network and TCP/IP stack in Red Bear OS for completeness +and quality, cross-references it against the Linux 7.1 reference implementation +(`local/reference/linux-7.1/`) and upstream Redox OS activity (2024–2026), and defines the +next improvement plan in execution order. + +This is the **canonical current implementation plan** for the userspace TCP/IP stack +(`netstack` / `smolnetd`), the Ethernet driver family, the POSIX socket surface in relibc, +network configuration (`netcfg`, `dhcpd`, `redbear-netctl`), and the runtime properties of +the scheme-based packet path. + +When another document discusses the smolnetd daemon, `scheme:tcp` / `scheme:udp` / +`scheme:icmp` / `scheme:ip` / `scheme:netcfg`, the `driver-network` trait, the Ethernet +driver family (`e1000d`, `ixgbed`, `rtl8139d`, `rtl8168d`, `virtio-netd`), or TCP/IP +performance and protocol coverage, prefer this file for: + +- the current robustness judgment, +- the current implementation order, +- the current validation/proof expectations, +- and the current language for build-visible vs runtime-proven vs hardware-validated claims. + +It is grounded in the current repository state, especially: + +- `local/sources/base/netstack/` (the TCP/IP daemon sources) +- `local/sources/base/drivers/net/` (Ethernet driver family + `driver-network` library) +- `local/sources/base/dhcpd/` (DHCP client) +- `local/sources/relibc/src/header/sys_socket/` and `local/sources/relibc/src/platform/redox/socket.rs` +- `local/sources/libredox/src/lib.rs` (SocketCall enum) +- `local/sources/kernel/src/scheme/{mod,user}.rs` (scheme dispatch) +- `local/recipes/drivers/redbear-iwlwifi/` (Intel Wi-Fi transport) +- `config/redbear-{netctl,wifi-experimental,bluetooth-experimental,mini,full}.toml` + +Companion plans that share authority over networking-adjacent surfaces: + +- `local/docs/WIFI-IMPLEMENTATION-PLAN.md` — canonical for the Wi-Fi control plane, Intel + transport, `redbear-iwlwifi`, `linux-kpi` wireless layer, and `redbear-wifictl`. +- `local/docs/IRQ-AND-LOWLEVEL-CONTROLLERS-ENHANCEMENT-PLAN.md` — canonical for the PCI + interrupt plumbing, MSI/MSI-X delivery, and IOMMU work that NIC drivers depend on. +- `local/docs/BLUETOOTH-IMPLEMENTATION-PLAN.md` — canonical for the Bluetooth stack. + +This plan covers the **wired TCP/IP stack and the socket surface**. Where Wi-Fi intersects +with the IP datapath (for example, hooking a Wi-Fi link-layer into the `Router`), the +Wi-Fi plan owns the wireless-specific detail and this plan owns the integration contract. + +## Validation States + +| State | Meaning | +|---|---| +| **builds** | Compiles in-tree | +| **host-tested** | Tests pass on Linux host with synthesized fixtures | +| **qemu-proven** | Behavior confirmed in QEMU with a virtio-net or e1000 NIC | +| **validated** | Behavior confirmed on real bare-metal hardware with evidence | +| **experimental** | Available for bring-up, not support-promised | +| **missing** | No in-tree implementation | + +## Current State + +### Architecture at a Glance + +Red Bear OS, like upstream Redox, has **zero in-kernel networking**. The kernel provides +only the scheme dispatch surface (`scheme/user.rs` proxies `open`/`read`/`write`/`dup`/ +`call` to userspace daemons) and the global `event:` scheme for I/O readiness notification. +All TCP/IP logic lives in the userspace `netstack` daemon, built on **smoltcp 0.12.0**. + +``` +Application (POSIX socket, libstd TcpStream, curl, ssh, ...) + │ relibc sys_socket: socket(), connect(), send(), recv(), ... + │ → open("/scheme/tcp", ...) → dup(fd, "1.2.3.4:80") + ▼ +Kernel scheme router (kernel/src/scheme/user.rs) + │ forwards scheme calls to the registered daemon via SQE/CQE IPC + ▼ +netstack / smolnetd (local/sources/base/netstack/src/main.rs) + │ registers: tcp, udp, icmp, ip, netcfg + │ owns: smoltcp::iface::Interface + SocketSet + Router + DeviceList + ▼ +Router (netstack/src/router/mod.rs) — implements smoltcp::phy::Device + │ poll(): drains EthernetLink.recv() into rx_buffer + │ dispatch(): RouteTable longest-prefix lookup → EthernetLink.send() + ▼ +EthernetLink (netstack/src/link/ethernet.rs) + │ ARP cache (60s TTL), frame encode/decode, neighbor discovery + │ synchronous read/write of raw Ethernet frames via the network scheme fd + ▼ +Ethernet driver daemon (e1000d, rtl8168d, ixgbed, rtl8139d, virtio-netd) + │ implements driver-network::NetworkAdapter + │ exposes scheme:network._ with raw frame read/write + /mac + ▼ +Hardware NIC (PCI MMIO + IRQ + DMA rings) +``` + +### Status Matrix + +| Area | State | Detail | +|---|---|---| +| TCP/IP daemon (`netstack`/`smolnetd`) | **builds, qemu-proven** | Single-threaded event loop over `EventQueue`. Owns smoltcp 0.12.0 `Interface` + `SocketSet`. Registers `tcp`, `udp`, `icmp`, `ip`, `netcfg` schemes. Binary aliased as both `netstack` and `smolnetd` for compatibility. | +| smoltcp version | **builds** | 0.12.0 with features: `std`, `medium-ethernet`, `medium-ip`, `proto-ipv4`, `socket-raw`, `socket-icmp`, `socket-udp`, `socket-tcp`, `socket-tcp-cubic`, `iface-max-addr-count-8`. **IPv6 NOT enabled.** | +| TCP congestion control | **builds** | CUBIC only (smoltcp `socket-tcp-cubic` feature). No BBR, no DCTCP, no per-socket selection. | +| TCP feature surface | **partial** | Window scaling, MSS negotiation, RTT estimation, exponential backoff retransmission, keep-alive, Nagle, delayed ACKs (all from smoltcp). **Missing: SACK, TCP timestamps, ECN, TCP Fast Open, MPTCP, urgent pointer.** | +| UDP sockets | **builds, qemu-proven** | `scheme:udp`. Peek-and-filter model for connected UDP. Ephemeral port range 49152–65535. | +| ICMP | **builds** | `scheme:icmp`, root-only. | +| Raw IP | **builds** | `scheme:ip`, root-only. Wraps smoltcp `RawSocket`. | +| ARP | **builds** | Hand-implemented in `link/ethernet.rs`. Request/reply, neighbor cache, 60s TTL, 3 retries with 1s silence. IPv4-only (no NDP for IPv6). | +| Routing | **builds, qemu-proven** | `router/route_table.rs` — longest-prefix-match over a `Vec`. Static routes only, configured via `scheme:netcfg/route/{add,del,list}`. No FIB trie, no policy routing, no multipath, no route daemon. | +| Network configuration (`netcfg`) | **builds** | Filesystem-like config scheme: `/resolv/nameserver`, `/route/{list,add,del}`, per-interface `ip`, `mac`, `gateway`. Backed by `BTreeMap` nodes with read/write callbacks. | +| DHCP client (`dhcpd`) | **builds, qemu-proven** | Standalone daemon. Reads/writes `scheme:netcfg` to apply leases. Init service `10_dhcpd.service` runs `dhcpd -f`. | +| Profile orchestration (`redbear-netctl`) | **builds, host-tested** | Arch-netctl-style profiles in `/etc/netctl/`. Wired/wireless, DHCP/static/bounded IP modes. Init service `12_netctl.service` applies the active profile after `smolnetd`+`dhcpd`. | +| DNS resolver | **builds** | In-process inside `netstack` via `dns-parser` crate. `scheme:netcfg/resolv/nameserver` for the recursive resolver address. Quad9 (9.9.9.9) fallback. | +| Ethernet drivers | **builds, qemu-proven** | 5 drivers: `e1000d` (Intel 8254x), `ixgbed` (Intel 10GbE 82599), `rtl8139d`, `rtl8168d` (Realtek), `virtio-netd`. All implement `driver-network::NetworkAdapter`. | +| Driver packet I/O model | **builds** | Synchronous file read/write of raw Ethernet frames. `NetworkAdapter::{read_packet, write_packet}`. IRQ-driven via `scheme:irq/{}` + `EventQueue`. No NAPI, no batched polling, no interrupt mitigation. | +| Wi-Fi IP datapath | **missing** | `redbear-iwlwifi` is control-plane/transport only (see `WIFI-IMPLEMENTATION-PLAN.md`). No 802.11 frame-to-EthernetLink bridge into `netstack`. | +| POSIX socket API (relibc) | **builds, qemu-proven** | `socket()`, `bind()`, `connect()`, `listen()`, `accept()`, `send()`, `recv()`, `sendto()`, `recvfrom()`, `sendmsg()`, `recvmsg()`, `getsockopt()`, `setsockopt()`, `shutdown()`, `socketpair()`, `getsockname()`, `getpeername()`. `AF_INET` and `AF_UNIX` only. **No `AF_INET6`.** | +| Socket multiplexing | **builds, qemu-proven** | Kernel `scheme:event` (analogous to epoll). `EventQueue` in daemons. `O_NONBLOCK` + `EAGAIN`/`EWOULDBLOCK` semantics on sockets. No `epoll_pwait2`, no io_uring equivalent. | +| Bulk FD passing | **partial** | Upstream relibc gained bulk FD passing for `recvmsg`/`sendmsg` in early 2026 (commits `94b0cfc68`, `5e61f17a3`, `1978c1aa4`). **Sync status with our local relibc fork needs verification** — see Upstream Sync. | +| Firewall / packet filtering | **missing** | No netfilter, iptables, nftables, or BPF equivalent. No hooks in the IP path. | +| Connection tracking | **missing** | No conntrack. No NAT. | +| IPv6 | **missing** | smoltcp `proto-ipv6` feature not enabled. `link/ethernet.rs` is IPv4-only (no NDP). relibc socket backend has no `AF_INET6` branch. `route_table.rs` is `IpAddress`-generic but no IPv6 routes are configured. | +| Performance offloads | **missing** | No GRO, no GSO, no TSO, no checksum offload, no zero-copy send/recv, no `MSG_ZEROCOPY`, no `sendfile`, no batched packet submission. | +| Multi-NIC | **partial** | `Router` supports a `DeviceList` and dispatches per-route, but `main.rs:get_network_adapter()` picks the **first** adapter and warns on multiple (FIXME comment). True multi-homing not wired. | +| Virtual network devices | **missing** | No bridge, VLAN, bond, tunnel, VXLAN, tun/tap scheme daemons. | +| Traffic control (qdisc) | **missing** | No shaping, policing, QoS, or scheduling discipline. | +| Network namespaces | **partial** | Scheme namespaces exist (via `setrens`), used by `smolnetd` to enter the null namespace. Not exposed as a general network isolation primitive. | +| Hardware validation (bare metal) | **qemu-proven only** | NIC drivers build and pass QEMU smoke tests. No bare-metal NIC throughput, packet loss, or long-connection validation evidence. | + +### Packet Flow Detail (Receive Path) + +``` +1. NIC raises IRQ → driver's EventQueue wakes +2. Driver::handle_irq() → drains NIC RX ring → packet into driver-side buffer +3. NetworkScheme posts fevent(EVENT_READ) on the network scheme fd +4. smolnetd's EventQueue wakes on EventSource::Network +5. smolnetd::on_network_scheme_event() → EthernetLink.recv(now) +6. EthernetLink reads raw frame via network_file.read(), parses EthernetRepr +7. If IPv4: returns payload to Router.poll() which enqueues into rx_buffer +8. If ARP: processes locally (neighbor cache update, may send reply) +9. smolnetd calls smoltcp Interface.poll(now, &mut Router, &mut SocketSet) +10. smoltcp processes IP/TCP/UDP state machines, fills socket buffers +11. Socket handlers post fevent on tcp/udp scheme fds → applications wake +12. Application calls read(fd) → kernel forwards to TcpScheme::read_buf() +13. TcpScheme::read_buf() → smoltcp TcpSocket::recv_slice() +``` + +### Packet Flow Detail (Transmit Path) + +``` +1. Application calls write(fd, data) → kernel forwards to TcpScheme::write_buf() +2. TcpScheme::write_buf() → smoltcp TcpSocket::send_slice() +3. smolnetd event loop calls smoltcp Interface.poll(now, &mut Router, &mut SocketSet) +4. smoltcp decides to transmit → Router.transmit(now) returns TxToken +5. smoltcp builds IP packet into TxToken buffer → Router.tx_buffer.enqueue() +6. smolnetd calls Router.dispatch(now) +7. Router.dispatch() parses Ipv4Packet, looks up RouteTable rule +8. If source address mismatch: rewrites src + refills checksum +9. Calls EthernetLink.send(next_hop, packet, now) +10. EthernetLink.send(): ARP neighbor cache lookup + - Hit: builds Ethernet frame, writes to network_file + - Miss: enqueues into waiting_packets, sends ARP request +11. Driver receives write() on network scheme → pushes to NIC TX ring → DMA → wire +``` + +## Gap Analysis vs Linux 7.1 + +Linux 7.1 (`local/reference/linux-7.1/`) is the reference for production-grade networking. +The table below maps every major Linux networking subsystem to its Red Bear equivalent and +identifies the gap. + +| Linux 7.1 Subsystem | Linux Location | Red Bear Equivalent | Gap Severity | +|---|---|---|---| +| **In-kernel TCP/IP** | `net/ipv4/`, `net/ipv6/` (~80k LoC) | Userspace `netstack` daemon (smoltcp) | Architectural difference, not a gap per se. Microkernel design. | +| **`sk_buff` packet metadata** | `include/linux/skbuff.h` (5,462 LoC) | Raw `[u8]` buffers, no metadata struct | **High** — no zero-copy across layers, no per-packet metadata propagation. | +| **NAPI (interrupt→poll)** | `include/linux/netdevice.h:381`, `napi_schedule()` | IRQ-per-packet + `EventQueue` wake | **High** — interrupt storm risk under load. No batched polling. | +| **GRO / GSO / TSO** | `net/core/gro.c`, `net/core/gso.c`, NIC driver offload | None | **High** — every packet traverses the full stack individually. | +| **BIG TCP (512KB GSO/GRO)** | `net/ipv6/ip6_output.c`, `net/ipv4/ip_output.c` | None | Medium — relevant only at 100GbE+, not near-term. | +| **Pluggable congestion control** | `include/net/tcp.h:1325` `struct tcp_congestion_ops`, `net/ipv4/tcp_cubic.c`, `tcp_bbr.c`, `tcp_dctcp.c` | smoltcp `socket-tcp-cubic` feature (CUBIC only) | **High** — no BBR for WAN throughput, no DCTCP for datacenter, no per-socket selection. | +| **SACK (Selective ACK)** | `net/ipv4/tcp_input.c` | Not in smoltcp | Medium — hurts lossy links. | +| **TCP timestamps** | `net/ipv4/tcp_output.c` | Not in smoltcp | Low — RTT estimation works without them but less precisely. | +| **ECN (Explicit Congestion Notification)** | `net/ipv4/tcp_input.c` | Not in smoltcp | Medium — datacenter relevance. | +| **MSG_ZEROCOPY send** | `net/ipv4/tcp.c:1140-1326`, `net/core/skbuff.c:1719-1806` | None | **High** — every send copies user→kernel→daemon. | +| **sendfile / splice** | `net/core/skbuff.c:3257` `skb_splice_bits()` | None | Medium — relevant for static file servers. | +| **io_uring zero-copy receive** | `io_uring/zcrx.c`, `IORING_OP_RECV_ZC` (`include/uapi/linux/io_uring.h:314`) | None | Low (long-term) — Linux 7.x reaches 187 Gbps with this; not near-term for Red Bear. | +| **epoll** | `fs/eventpoll.c` (~3,000 LoC), `ep_poll_callback()` | Kernel `scheme:event` + `EventQueue` | Medium — functionally equivalent but less optimized; no ready-list integration with socket wakeup path. | +| **XDP / AF_XDP** | `net/xdp/xsk.c`, BPF programs in driver context | None | Low (long-term) — line-rate BPF filtering. | +| **eBPF everywhere** | `net/core/filter.c`, `bpf_tcp_ca.c`, XDP, TC, socket filter | None | Medium — no programmable packet processing. | +| **Netfilter (5 hooks)** | `include/uapi/linux/netfilter.h:43`, `struct nf_hook_ops` | None | **High** — no firewall, no NAT, no packet interception layer. | +| **Conntrack** | `net/netfilter/nf_conntrack_core.c` (2,790 LoC), `struct nf_conn` | None | **High** — no stateful firewall, no NAT. | +| **nftables / iptables** | `net/netfilter/nf_tables_api.c`, `xt_*.c` | None | High — no user-facing firewall CLI. | +| **FIB (LC-trie routing)** | `include/net/ip_fib.h`, `net/ipv4/fib_trie.c`, `fib_semantics.c` | `router/route_table.rs` — `Vec` linear scan | Medium — fine for tens of routes; inadequate for full BGP table. | +| **Policy routing / multipath** | `net/ipv4/fib_rules.c`, `fib_select_multipath()` | None | Low — niche outside routers. | +| **IPv6 (full stack)** | `net/ipv6/` (40+ .c files), built-in only in 7.1 | Not enabled (smoltcp `proto-ipv6` off, no NDP, no `AF_INET6`) | **Critical** — IPv6 is mandatory for modern networking. | +| **cfg80211 / mac80211 / nl80211** | `include/net/cfg80211.h` (10,995 LoC), `net/mac80211/` (60+ files), `include/uapi/linux/nl80211.h` | `linux-kpi` wireless shim (Rust), `redbear-iwlwifi` transport (C), `redbear-wifictl` scheme | See `WIFI-IMPLEMENTATION-PLAN.md`. **Control-plane only; no IP datapath bridge yet.** | +| **`net_device` + `net_device_ops`** | `include/linux/netdevice.h:2139` (5,737 LoC), ~50 ops | `driver-network::NetworkAdapter` trait (4 methods) | **High** — no features negotiation, no offload reporting, no statistics, no queue selection. | +| **Multi-queue / RSS / RPS / RFS / XPS** | `net/core/dev.c`, per-CPU queues | None | Medium — relevant for >1Gbps. | +| **`page_pool` memory recycling** | `net/core/page_pool.c` | None | Medium — allocator overhead on RX. | +| **Cacheline-aware structs** | `include/linux/netdevice.h` `_cacheline_group` annotations | None | Low — micro-optimization. | +| **Network namespaces** | 67 namespace-aware subdirs under `net/` | Scheme namespaces (partial) | Medium — relevant for container story. | +| **Bridge / VLAN / bond / tunnel** | `net/bridge/`, `net/8021q/`, `net/bonding/`, `net/ipv4/ip_tunnel.c` | None | Medium — virtualization/networking appliance relevance. | +| **Traffic control (qdisc)** | `net/sched/` (100+ files), `include/net/sch_generic.h` | None | Low — QoS is niche for a desktop OS. | +| **MPTCP** | `net/mptcp/` | Not in smoltcp | Low — niche. | +| **Hardware offloads (checksum, TSO, LRO, TLS, IPsec)** | NIC driver `features`, `include/linux/netdevice.h` NETIF_F_* | None | Medium — drivers don't even advertise capabilities. | +| **Routing daemons (BGP/OSPF/RIP)** | Userspace (FRRouting, bird) | None | Low — not a kernel/daemon gap. | +| **Drop reasons / observability** | `include/net/dropreason.h`, `skb->drop_reason` | Logging only | Low — debugging aid. | + +### Severity Summary + +- **Critical (blocks modern use):** IPv6, firewall/NAT (netfilter equivalent). +- **High (major performance or functionality gap):** `sk_buff`-equivalent metadata, NAPI-style polling, GRO/GSO/TSO, MSG_ZEROCOPY, pluggable congestion control, multi-NIC. +- **Medium (matters for specific workloads):** SACK, ECN, FIB scaling, multi-queue, virtual devices, observability. +- **Low (long-tail polish):** TCP timestamps, policy routing, MPTCP, io_uring ZC, qdisc. + +## Upstream Redox Sync Opportunities (2024–2026) + +Red Bear OS is forked from a frozen Redox snapshot. Upstream Redox has been actively evolving +its networking stack. The following upstream commits should be evaluated for import into the +Red Bear local forks. **Per the Golden Rule (Red Bear adapts to upstream, never the reverse), +these are imports to track and apply, not workarounds.** + +### Already in Red Bear (verified) + +- ✅ smoltcp 0.12.0 with `socket-tcp-cubic` (upstream `b92be2e7d`, `d7c128684`, 2025-03-09) +- ✅ Named network adapters via `scheme:network._` (upstream `f9b3170f0`, 2024-02-28) +- ✅ `netstack` merged into `base` recipe (upstream `c06e5b14e`, 2025-03-10) +- ✅ MAC address fetched from network scheme (upstream `674f5b6d7`, 2024-02-28) +- ✅ `dhcpd` moved into `base` (upstream `b68e5a685`, 2026-04-11) +- ✅ New scheme format, no legacy paths (upstream `bafdb3b66`, 2024-07-11) + +### To Evaluate for Import + +| Upstream Commit | Date | Description | Red Bear Action | +|---|---|---|---| +| `redox-os/drivers@ad9305bf9` | 2024-02-28 | Unified network drivers under `net/` with shared scheme | Verify our driver layout matches. | +| `redox-os/drivers@921c6b07f` | 2024-12-26 | `driver-network` migrated to `redox-scheme` crate | **Import** — modernizes the scheme protocol. | +| `redox-os/drivers@0f24975ff` | 2025-11-29 | PCI interrupts rework: dedup vector handling for RTL drivers | **Import** — IRQ quality (see IRQ plan). | +| `redox-os/drivers@dd41c4f13` | 2025-11-23 | Net: scheme created and daemon ready before device init | **Import** — fixes race on early opens. | +| `redox-os/drivers@407533201` | 2025-09-24 | Add ThinkPad T60 ethernet to `e1000d` PCI IDs | **Import** — broader hardware. | +| `redox-os/relibc@94b0cfc68` | 2026-02-08 | Reimplement `recvmsg`/`sendmsg` using bulk FD passing | **Verify + import** — check our relibc fork has this; critical for `SCM_RIGHTS`. | +| `redox-os/relibc@5e61f17a3` | 2026-02-08 | `MSG_CMSG_CLOEXEC` handling in `recvmsg` | **Import** alongside the above. | +| `redox-os/relibc@1978c1aa4` | 2026-03-25 | Full `recvmsg` implementation | **Import**. | +| `redox-os/relibc@cab002146` | 2026-02-28 | Move protocols into `libredox` | **Major refactor — evaluate carefully.** Affects SocketCall enum location. | +| `redox-os/relibc@443145fde` | 2025-11-07 | Use `recvmsg`/`sendmsg` when `recvfrom`/`sendto` has flags | **Import**. | +| `redox-os/relibc@9eaa9e82b` | 2025-11-08 | Pass through all `SOL_SOCKET` options, fix `getsockopt` option_len | **Import** — fixes option handling bugs. | +| `redox-os/relibc@6a455159a` | 2026-01-22 | Fix `getaddrinfo` infinite loop hang | **Critical import** — known crash. | +| `redox-os/relibc@726a0fb1a` | 2025-09-15 | `getaddrinfo` NULL nodename, `AI_PASSIVE`, `AI_NUMERICHOST` | **Import**. | +| `redox-os/relibc@5334455a2` | 2025-08-20 | `getnameinfo` + `getaddrinfo` loopback | **Import**. | +| `redox-os/relibc@d44010170` | 2025-07-18 | UDS `bind`/`connect` with RedoxFS integration | **Import** — fixes Unix domain sockets. | +| `redox-os/relibc@6dd10a1f1` | 2025-08-01 | Fix `connect` on Redox | **Import**. | +| `redox-os/relibc@9c6701802` | 2026-01-07 | Add `sys_socket` tests | **Import** — regression coverage. | +| `redox-os/syscall@7a1409a91` | 2026-05-27 | Multiple FDs variant for `call` and `std_fs_call` | **Import** — enables bulk FD passing. | +| `redox-os/syscall@178461f6f` | 2025-12-27 | Remove remnants of old packet format | **Import** — cleanup. | +| `redox-os/kernel@c089667ad` | 2025-12-13 | Remove legacy packet user schemes | **Import** — kernel side of the above. | +| `redox-os/kernel@6c3d5d28c` | 2026-07-01 | Move FD allocation logic into userspace | **Major refactor — evaluate carefully.** Most recent kernel change. | +| `redox-os/kernel@08ea1da2f` | 2025-07-05 | Trigger read event for user schemes on fd close | **Import** — fixes socket cleanup races. | +| `redox-os/kernel@4ff82ad8b` | 2025-11-27 | Fix user scheme deadlocks in `call_extended_inner` | **Critical import** — fixes hangs under load. | +| `redox-os/kernel@99ff55ee1` | 2026-02-08 | Demux results as soon as received from user scheme | **Import** — latency improvement. | +| `redox-os/redox@85b62fd85` | 2026-03-01 | Support networking in all configs | **Reference** — verify our configs cover networking in all targets. | +| `redox-os/redox@193e81897` | 2026-02-17 | ~115 WIP networking recipes (HTTP, FTP, SSH, VPN, P2P) | **Cherry-pick** — many of these are now relevant for `cub`. | +| `redox-os/netutils@6df5af955` | 2024-11-27 | Basic `ifconfig` command | **Reference** — `redbear-netstat` covers this, but compare APIs. | +| `redox-os/netutils@62f96a4b5` | 2026-05-22 | Fix `nc` listener exiting on stdin EOF | **Import if we ship nc.** | + +### Upstream Gaps Red Bear Fills Independently + +These exist in Red Bear but **not** in upstream Redox: + +- Wi-Fi driver support (`redbear-iwlwifi`, `linux-kpi` wireless layer, `redbear-wifictl`) +- `redbear-netctl` profile orchestration (Arch netctl-style) +- `redbear-netstat`, `redbear-mtr`, `redbear-nmap`, `redbear-traceroute` +- AMD GPU networking offload path (via `redox-drm`) + +## Improvement Plan + +The plan is organized into six phases. Each phase has a clear success criterion, dependencies, +and validation target. Phases are ordered by **dependency** (early phases unblock later ones) +and by **impact** (critical gaps first). + +### Phase 0: Upstream Sync and Foundation Hardening + +**Goal:** Bring the Red Bear local forks to current upstream networking state before adding +new features. This avoids building on stale foundations. + +**Duration:** 2–3 weeks. + +**Workstreams:** + +0.1 **relibc socket fixes.** Apply the upstream commits listed in "To Evaluate for Import" + that touch relibc's `sys_socket`, `getaddrinfo`, `getnameinfo`, `recvmsg`/`sendmsg`, + `connect`, `bind`, and `SOL_SOCKET` option pass-through. **Priority: the `getaddrinfo` + infinite loop fix (`6a455159a`).** Verify against the `sys_socket` test suite. + +0.2 **Kernel scheme dispatch fixes.** Apply `4ff82ad8b` (deadlock fix), `08ea1da2f` + (close event), `99ff55ee1` (latency). These are direct kernel-fork patches. + +0.3 **Driver-side modernization.** Apply `921c6b07f` (`driver-network` → `redox-scheme`), + `dd41c4f13` (scheme-before-device-init race fix), `0f24975ff` (PCI interrupts rework). + Test each driver in QEMU after applying. + +0.4 **Config audit.** Verify all `redbear-*.toml` targets that should have networking + actually pull in `redbear-netctl`, `smolnetd`, `dhcpd`, and a NIC driver. Reference + upstream `85b62fd85`. + +0.5 **Bulk FD passing verification.** Check whether the local relibc fork already has + commits `94b0cfc68` and `1978c1aa4`. If not, import them. Validate `SCM_RIGHTS` + works end-to-end with a test daemon. + +**Success Criteria:** +- All listed upstream commits either applied or documented as "intentionally skipped" with + a reason. +- `redbear-mini` boots in QEMU, acquires DHCP, and `curl http://example.com/` succeeds. +- `redbear-netstat` shows the TCP connection in `ESTABLISHED` state. +- `sys_socket` tests pass on the relibc fork. + +**Validation target:** qemu-proven. + +--- + +### Phase 1: Multi-NIC and Driver Feature Surface + +**Goal:** Remove the single-NIC FIXME and give the stack visibility into driver capabilities. + +**Duration:** 3–4 weeks. + +**Dependencies:** Phase 0 complete. + +**Workstreams:** + +1.1 **Multi-NIC in `netstack`.** Replace `get_network_adapter()` (which picks the first NIC + and warns) with proper multi-interface support. The `Router` already has a `DeviceList`; + the gap is in `main.rs` initialization and in `EthernetLink` instantiation per adapter. + Reference Linux's `net_device` list semantics. Each NIC gets its own `EthernetLink` in + the `DeviceList`, with its own ARP cache and MAC. + +1.2 **`NetworkAdapter` trait expansion.** The current trait is 4 methods: + ```rust + pub trait NetworkAdapter { + fn mac_address(&mut self) -> [u8; 6]; + fn available_for_read(&mut self) -> usize; + fn read_packet(&mut self, buf: &mut [u8]) -> Result>; + fn write_packet(&mut self, buf: &[u8]) -> Result; + } + ``` + Expand to expose capabilities and statistics, modeled on Linux's + `struct net_device_ops` and `ethtool_ops`: + ```rust + pub trait NetworkAdapter { + fn mac_address(&mut self) -> [u8; 6]; + fn available_for_read(&mut self) -> usize; + fn read_packet(&mut self, buf: &mut [u8]) -> Result>; + fn write_packet(&mut self, buf: &[u8]) -> Result; + + // New: capability negotiation + fn capabilities(&self) -> NicCapabilities; // MTU, offloads, max queues + fn mtu(&self) -> usize { 1500 } + fn link_state(&self) -> LinkState; // Up / Down / Unknown + fn statistics(&self) -> NicStatistics; // rx/tx bytes, packets, errors, drops + } + ``` + Wire `NicCapabilities` into `netstack` so `Router` knows the real MTU (currently + hardcoded to 1486 in `router/mod.rs:42`). + +1.3 **`scheme:netcfg` interface enumeration.** Expose `netcfg/ifaces` listing all + `DeviceList` entries with their MAC, IP, link state, statistics, and MTU. This gives + `redbear-netstat` and `redbear-netctl` a proper interface table. + +1.4 **Per-driver statistics.** Each driver (`e1000d`, `rtl8168d`, etc.) should expose + RX/TX byte/packet/error counters through the new `statistics()` method. Reference + Linux's `ndo_get_stats64`. + +**Success Criteria:** +- `redbear-netstat -i` lists all NICs present (in QEMU: just `virtio-net` or `e1000`; on + bare metal: all detected NICs). +- A second NIC can be brought up with a static IP via `redbear-netctl` and routes traffic. +- `Router::MTU` is no longer a hardcoded constant — it derives from the active NIC's + reported MTU. +- Driver statistics are non-zero after a ping flood. + +**Validation target:** qemu-proven for multi-NIC (two virtio-net interfaces); host-tested +for the trait expansion (unit tests on the new trait). + +--- + +### Phase 2: IPv6 + +**Goal:** Bring the stack to modern IP standards. IPv6 is no longer optional for a +general-purpose OS. + +**Duration:** 4–6 weeks. + +**Dependencies:** Phase 1.1 (multi-NIC) helpful but not strictly required. Phase 0 +(relibc freshness) required. + +**Workstreams:** + +2.1 **Enable smoltcp IPv6.** In `netstack/Cargo.toml`, add `"proto-ipv6"` to the smoltcp + features list. Audit the build for new warnings/errors — smoltcp's IPv6 surface is + large. Reference: smoltcp `wire` module already has full IPv6 parsing. + +2.2 **`link/ethernet.rs` IPv6 support.** Currently the link layer is IPv4-only: the ARP + cache uses `Ipv4Address`, `process_arp` only handles `ArpRepr::EthernetIpv4`, and + `send_to` only handles `IpAddress::Ipv4`. Add NDP (Neighbor Discovery Protocol, + RFC 4861) for IPv6 neighbor discovery, mirroring the ARP logic. Use smoltcp's + `wire::{Icmpv6Repr, Icmpv6Packet, NdiscRepr}` types. + +2.3 **`router/route_table.rs` IPv6.** The `Rule` struct already uses `IpCidr` and + `IpAddress`, which are version-generic. The gap is in `dispatch()` (which only does + `Ipv4Packet::new_checked`) and in `set_src_addr` checksum refill. Add an IPv6 branch. + IPv6 has no header checksum (unlike IPv4), so the checksum logic is simpler — only + transport-layer pseudo-header checksums need attention. + +2.4 **relibc `AF_INET6`.** In `local/sources/relibc/src/platform/redox/socket.rs`, add a + branch for `AF_INET6` that opens `/scheme/tcp` and `/scheme/udp` with IPv6-encoded + addresses. The scheme path encoding `dup(fd, "[2001:db8::1]:443")` should work once + `netstack` parses IPv6 endpoints (smoltcp's `parse_endpoint` already does). + +2.5 **`netcfg` IPv6 surfaces.** Add `netcfg/resolv/nameserver6`, per-interface IPv6 + address configuration, and IPv6 route add/del. Reference: Linux `addrconf.c` + (SLAAC) and `ndisc.c` (router advertisements). + +2.6 **DHCPv6.** Either extend `dhcpd` or add a sibling `dhcpv6d` daemon. DHCPv6 (RFC 8415) + is a separate protocol from DHCPv4 — UDP port 546 (client) / 547 (server), different + message format. Alternatively, implement SLAAC (RFC 4862) as the default IPv6 + autoconfiguration method. + +2.7 **`redbear-netctl` IPv6 profiles.** Extend the profile format to support `IP6=dhcp`, + `IP6=slaac`, `IP6=static`, `Address6=`, `Gateway6=`, `DNS6=`. + +2.8 **Validation.** Boot `redbear-mini` in QEMU with `qemu-system-x86_64 ... -netdev + user,id=n0,ipv6=on -device virtio-net,netdev=n0`. Verify `ping6 ::1`, `curl -6 + https://ipv6.google.com`, and a dual-stack SSH connection. + +**Success Criteria:** +- `socket(AF_INET6, SOCK_STREAM, 0)` returns a valid fd. +- `ping6 ` succeeds. +- A dual-stack TCP server accepts both IPv4 and IPv6 connections. +- `redbear-netstat -r` shows both IPv4 and IPv6 routes. + +**Validation target:** qemu-proven for basic IPv6; host-tested for NDP and DHCPv6 +protocol correctness. + +**Risk:** This is the largest single workstream. The `link/ethernet.rs` IPv6 work is +non-trivial because NDP is more complex than ARP (solicitation-node multicast, router +discovery, prefix information options). Consider implementing NDP as a separate +`link/ndp.rs` module rather than complicating `ethernet.rs`. + +--- + +### Phase 3: TCP Performance and Congestion Control + +**Goal:** Bring TCP throughput and latency into the same order of magnitude as Linux for +common workloads. + +**Duration:** 6–8 weeks. + +**Dependencies:** Phase 1 (capabilities surface) for any hardware offload wiring. + +**Workstreams:** + +3.1 **Buffer pool audit.** `netstack/src/buffer_pool.rs` exists but its usage should be + audited. TCP socket buffers are currently `vec![0; 0xffff]` (64KB) per socket in + `scheme/tcp.rs:80-83`. For high-bandwidth connections, this is a throughput cap. + Make the buffer size configurable via `setsockopt(SO_SNDBUF/SO_RCVBUF)` (the scheme + already has stubs at `tcp.rs:14-15` but `set_setting` returns `Ok(0)`). Reference + Linux's `tcp_rmem`/`tcp_wmem` sysctl defaults. + +3.2 **Pluggable congestion control interface.** smoltcp 0.12 has `socket-tcp-cubic` as a + feature flag but no runtime selection. Two options: + + **Option A (recommended):** Add `setsockopt(TCP_CONGESTION, "cubic")` support to the + TCP scheme, backed by a `congestion_control` field on `SocketFile`. Since smoltcp + compiles CC at build time via features, this requires either (a) compiling both + Reno and CUBIC and switching at runtime (if smoltcp supports it), or (b) forking + smoltcp to expose a `CongestionController` trait object. Check whether smoltcp 0.12's + `socket-tcp-cubic` is additive to the default Reno controller. + + **Option B:** Contribute a BBR-style controller upstream to smoltcp and enable it + via a Red Bear patch. This is higher-effort but higher-impact. + +3.3 **SACK (Selective Acknowledgment).** smoltcp does not implement SACK (RFC 2018). + On lossy links (wireless, WAN), SACK dramatically improves throughput by allowing the + receiver to acknowledge non-contiguous blocks. This requires either (a) upstreaming + SACK to smoltcp, or (b) carrying a Red Bear patch against smoltcp. Given the Golden + Rule (prefer upstream), the path is: contribute to smoltcp, carry a local patch + until merged. Reference: Linux `net/ipv4/tcp_sack.c`. + +3.4 **Receive batching (NAPI-equivalent).** The current receive path processes one packet + per `EventSource::Network` event. Under load, this means one scheme-IPC round-trip + per packet. Add a drain loop in `on_network_scheme_event()` that reads packets until + `available_for_read()` returns 0 (or a budget cap). This mirrors Linux's NAPI + `budget` parameter. Reference: Linux `napi_complete_done()` and the `budget` pattern + in `include/linux/netdevice.h`. + +3.5 **Transmit batching.** Similarly, `Router::dispatch()` already drains `tx_buffer` + in a loop, but each `dev.send()` is a synchronous write to the network scheme fd. + Batch multiple frames into a single write if the driver supports it (requires the + `capabilities()` expansion from Phase 1.2). + +3.6 **`MSG_ZEROCOPY` investigation.** True zero-copy requires shared memory between the + application and `netstack`. This is architecturally significant in a microkernel — + it means the scheme IPC must support passing buffer references, not just data copies. + This is a research workstream: evaluate whether the kernel's `scheme:memory` can be + used to establish a shared ring buffer between an application and `netstack`. + Reference: Linux `msg_zerocopy_alloc()` and the `ubuf_info` mechanism, and the + Redox "ring buffers" NLnet-funded project mentioned in the Development Priorities + 2025/26 document. **Do not implement in this phase — produce a design doc.** + +**Success Criteria:** +- `iperf3` (port the recipe) between two QEMU VMs shows ≥100 Mbps on virtio-net + (current baseline unknown — establish it first). +- `setsockopt(SO_RCVBUF, ...)` affects the actual receive buffer size (verify with + `getsockopt`). +- SACK enabled connections recover from induced packet loss (use `tc netem` in a + Linux-to-RedBear bridge, or a QEMU packet loss model). +- A design doc for zero-copy exists at `local/docs/NETWORK-ZEROCOPY-DESIGN.md`. + +**Validation target:** qemu-proven for throughput and buffer sizing; host-tested for +SACK protocol correctness. + +--- + +### Phase 4: Firewall and Packet Filtering + +**Goal:** Provide a firewall and NAT capability, respecting the microkernel +everything-is-a-scheme design. + +**Duration:** 6–10 weeks. + +**Dependencies:** Phase 1 (interface awareness). Phase 2 (IPv6) for dual-stack filtering. + +**Workstreams:** + +4.1 **Architecture decision.** In Linux, netfilter hooks live inside the kernel IP path. + In Red Bear, the IP path is in `netstack`'s `Router` and `EthernetLink`. The + microkernel-aligned approach is to add **filter hooks in `netstack`'s `Router`** + that consult a `scheme:firewall` (or `scheme:nf`) daemon. This keeps policy in + userspace and matches Redox's design philosophy. Two designs to evaluate: + + **Design A (in-process):** Add a `FirewallEngine` inside `netstack` that reads rules + from `scheme:netcfg/firewall/{rules,chains}`. Simpler, faster (no IPC per packet), + but couples policy to the stack daemon. + + **Design B (out-of-process):** `netstack` calls out to a `firewalld` daemon via + scheme IPC for each packet. Cleaner separation but adds latency per packet. + Mitigated by a rule cache in `netstack` with invalidation via fevent. + + **Recommendation:** Design A for the hot path (rule lookup), with rules managed by a + separate `scheme:firewall` daemon that writes to `netcfg/firewall/*`. This matches + how `route` and `resolv` already work (config via `netcfg`, enforced in `netstack`). + +4.2 **Hook points.** Define 5 hook points mirroring Linux netfilter, adapted to the + `Router`/`EthernetLink` boundary: + - `PRE_ROUTING` — after `EthernetLink::recv()`, before `Router` enqueue. + - `LOCAL_IN` — after `Router` decides packet is for this host. + - `FORWARD` — after `Router` decides packet is to be forwarded (requires IP forwarding + support, currently missing — see Phase 6). + - `LOCAL_OUT` — after `Router::dispatch()` dequeues, before `EthernetLink::send()`. + - `POST_ROUTING` — just before `EthernetLink::send()`. + +4.3 **Rule format.** Define a TOML or filesystem-based rule format under + `netcfg/firewall/`. Example: + ```toml + # /etc/firewall/rules.toml (or via scheme:netcfg/firewall/rules) + [[chain]] + name = "input" + [[chain.rule]] + action = "accept" + proto = "tcp" + dest_port = 22 + source = "192.168.0.0/16" + [[chain.rule]] + action = "drop" + proto = "tcp" + dest_port = 22 + ``` + +4.4 **Connection tracking (conntrack).** For stateful rules (`ACCEPT ESTABLISHED, + RELATED`), implement a connection tracker inside `netstack` (or a separate daemon). + Track `(src_ip, src_port, dst_ip, dst_port, proto, state)` tuples with timeouts. + Reference: Linux `struct nf_conn` (`include/net/netfilter/nf_conntrack.h:74`). + +4.5 **NAT.** Build on conntrack. Source NAT (masquerade) rewrites `src_addr` in + `Router::dispatch()` and records the mapping in conntrack. Destination NAT (port + forwarding) rewrites `dst_addr` in `PRE_ROUTING`. This requires the checksum refill + logic that already exists for IPv4 src rewrites — extend it to handle dst rewrites + and (eventually) port rewrites in the transport pseudo-header. + +4.6 **`redbear-firewall` CLI.** A `redbear-firewall` tool (or extend `redbear-netctl`) + that reads/writes `scheme:netcfg/firewall/*` and provides `iptables`-like or + `nft`-like ergonomics. Reference: Linux `iptables(8)` and `nft(8)`. + +4.7 **Validation.** Tests: (a) drop all inbound TCP except 22, verify only SSH connects; + (b) masquerade a Red Bear VM behind another Red Bear VM's NAT, verify outbound + connections work and inbound connections to the inner VM fail without port forward; + (c) port-forward 8080 to an inner VM's 80, verify a curl to the gateway:8080 reaches + the inner VM. + +**Success Criteria:** +- A basic ruleset (accept loopback, accept established, accept SSH, drop rest) can be + loaded via `redbear-firewall` and is enforced. +- SNAT masquerade works for outbound traffic from a NAT'd subnet. +- DNAT port-forwarding works for at least one TCP port. +- `redbear-netstat -t` shows conntrack entries for active connections. + +**Validation target:** qemu-proven for all three test scenarios; host-tested for +conntrack state machine correctness. + +--- + +### Phase 5: Driver Coverage and Hardware Validation + +**Goal:** Expand the Ethernet driver family to cover common bare-metal hardware and +validate the stack on real NICs. + +**Duration:** Ongoing (parallel with Phases 2–4). + +**Dependencies:** Phase 0 (driver modernization). Phase 1 (capabilities surface). + +**Workstreams:** + +5.1 **Import upstream `alxd`.** Upstream Redox has an `alxd` driver (Atheros L2 Fast + Ethernet) that Red Bear does not. Import it. Reference: `redox-os/drivers/net/alxd/`. + +5.2 **Common consumer NIC coverage.** The current family misses several common consumer + chipsets. Prioritize by deployment frequency: + - **`igb` / `igc`** (Intel I219/I225 on modern laptops — NUC, ThinkPad, Dell) — + **High priority.** The `e1000d` PCI ID list (`10ec:1004, 100e, 100f, 109a, 1503`) + does not cover the I219/I225 (PCI IDs `8086:15d8`, `8086:15f2`, etc.). Either + extend `e1000d` or add a new `igbd` driver. + - **`r8125`** (Realtek 2.5GbE on modern motherboards) — Medium priority. + `rtl8168d` covers 8168/8169 but not 8125 (`10ec:8125`). + - **`e1000e`** (Intel I219/I218 Gigabit) — the PCI IDs `8086:1502, 1503, 1559` + overlap with `e1000d` but the register interface differs. Verify coverage. + +5.3 **Bare-metal validation harness.** Establish a bare-metal test bench with: + - An Intel NIC (e1000 or igb) and a Realtek NIC (rtl8168) on the same machine. + - A Linux host on the other end of the link for `iperf3`, `tcpdump`, and packet + injection. + - Test matrix: boot, DHCP acquire, ping, iperf3 TCP throughput, iperf3 UDP packet + loss, long-running TCP connection (1 hour), rapid connection churn (1000 + connects/sec). + +5.4 **Wi-Fi IP datapath bridge.** Once `redbear-iwlwifi` reaches runtime validation + (per `WIFI-IMPLEMENTATION-PLAN.md`), add the datapath bridge: a Wi-Fi link-layer + adapter that implements `driver-network::NetworkAdapter` and feeds 802.11 frames + (after 802.11-to-Ethernet header conversion) into `netstack`'s `DeviceList`. This + makes Wi-Fi a first-class `Router` device. Coordinate with the Wi-Fi plan. + +**Success Criteria:** +- `alxd` builds and is packaged. +- At least one modern Intel NIC (I219 or I225) is supported and validated. +- Bare-metal iperf3 throughput is documented for at least one NIC. +- (Stretch) Wi-Fi TCP connection works end-to-end through `netstack`. + +**Validation target:** validated for at least one bare-metal NIC. + +--- + +### Phase 6: Forwarding, Virtual Devices, and Advanced Features + +**Goal:** Bring the stack to feature parity with a general-purpose Linux networking +appliance for the features that matter. + +**Duration:** 8–16 weeks (parallel, long-term). + +**Dependencies:** Phases 2 (IPv6), 4 (firewall/conntrack for NAT foundation). + +**Workstreams:** + +6.1 **IP forwarding.** When `netstack` receives a packet whose destination is not a local + IP, it currently drops it (no `FORWARD` hook in the `Router`). Add a forwarding path: + look up the destination in the route table, and if the next-hop is via a different + interface than the ingress interface, re-queue the packet for that interface. Decrement + the IPv4 TTL (or IPv6 Hop Limit) and drop at 0. Reference: Linux `ip_forward()` in + `net/ipv4/ip_forward.c`. Gate behind a `netcfg/forwarding` enable flag. + +6.2 **Bridge scheme daemon.** A `bridged` userspace daemon that implements an L2 learning + bridge (MAC learning table, STP optionally). Exposes `scheme:bridge/` and + connects multiple `scheme:network/*` devices. The bridge itself registers as a + `scheme:network/bridge0` so `netstack` treats it like any other NIC. Reference: + Linux `net/bridge/br_device.c` and `struct net_bridge`. + +6.3 **VLAN support.** Add 802.1Q VLAN tag parsing in `EthernetLink` (or a separate + `VlanLink` wrapper). Expose `scheme:network/.` virtual devices. Reference: + Linux `net/8021q/`. + +6.4 **Bonding / link aggregation.** A `bondd` daemon implementing LACP (802.3ad) or + active-backup. Higher-effort; defer until multi-NIC (Phase 1) is solid. + +6.5 **TUN/TAP scheme daemon.** A `tund`/`tapd` that exposes a virtual interface to + userspace — read/write packets from an application. Useful for VPN daemons (WireGuard, + OpenVPN). Reference: Linux `drivers/net/tun.c`. + +6.6 **Routing daemon support.** Once the FIB/route table supports sufficient scale, port + a routing daemon (bird or FRRouting) so Red Bear can participate in dynamic routing. + This is userspace-only work; no `netstack` changes needed beyond performance. + +6.7 **`sk_buff`-equivalent metadata struct.** Introduce a `Packet` struct in `netstack` + that carries metadata (ingress interface, VLAN tag, receive timestamp, mark/label for + firewall) alongside the raw bytes. This replaces the current `[u8]`-only model and + enables future features (QoS marking, policy routing on metadata, observability). + Reference: Linux `struct sk_buff` (`include/linux/skbuff.h`). **This is a significant + internal refactor and should be staged carefully.** + +**Success Criteria:** +- A Red Bear VM routes packets between two subnets at >100 Mbps. +- A bridge connects two VMs on the same L2 segment. +- VLAN-tagged traffic is correctly classified. +- (Stretch) WireGuard establishes a tunnel between two Red Bear VMs. + +**Validation target:** qemu-proven for forwarding, bridge, VLAN; experimental for bonding +and TUN/TAP. + +## Cross-Cutting Concerns + +### Observability + +Throughout all phases, maintain and extend the `redbear-netstat` tool to surface new +state: +- Phase 1: per-interface statistics table. +- Phase 2: IPv6 neighbor cache (`ndp -a` equivalent), IPv6 routes. +- Phase 3: TCP socket details (cwnd, rtt, retransmits — from smoltcp socket introspection). +- Phase 4: conntrack table (`conntrack -L` equivalent), firewall rule hit counters. +- Phase 6: bridge MAC table, VLAN mappings. + +Add a `redbear-tcpdump` or `scheme:netcfg/pcap` capability for packet capture. A +userspace packet capture daemon that taps the `PRE_ROUTING` hook (Phase 4) and writes +pcap files would be invaluable for debugging. + +### Testing + +- **Host-tested:** smoltcp already has extensive upstream tests. Add Red Bear-specific + tests for: NDP state machine, firewall rule evaluation, conntrack state transitions, + NAT mapping correctness. +- **QEMU-tested:** Establish a standard QEMU networking test image with two NICs, + configurable network topology, and a test runner that boots Red Bear, runs a series + of network commands, and checks output. Reference: Linux's `tools/testing/selftests/net/`. +- **Bare-metal validated:** The Phase 5 harness. Document results in + `local/docs/networking-validation-log.md`. + +### Documentation + +This plan should be updated when: +- A phase is started or completed (update the status matrix). +- An upstream commit is imported or intentionally skipped (update the sync table). +- A design decision is made (e.g., the zero-copy design doc from Phase 3.6). +- A validation milestone is reached. + +## Validation Evidence Requirements + +Per the project's evidence policy, claims in this plan are not complete without: + +| Claim Type | Required Evidence | +|---|---| +| "builds" | `repo cook recipes/core/base` succeeds; `netstack` binary present in stage. | +| "qemu-proven" | QEMU boot log showing the daemon start, the scheme registration, and the test command succeeding (e.g., `curl -v` output). | +| "host-tested" | `cargo test` output from the relevant crate (relibc, netstack, redbear-netctl). | +| "validated" | Bare-metal test report: hardware model, NIC model, kernel/driver versions, test commands, and results. Stored in `local/docs/networking-validation-log.md`. | + +## Risk Register + +| Risk | Likelihood | Impact | Mitigation | +|---|---|---|---| +| smoltcp 0.12 IPv6 surface incomplete | Medium | High (blocks Phase 2) | Spike first: enable `proto-ipv6`, audit compilation errors and missing APIs before committing to Phase 2. | +| SACK requires smoltcp fork | High | Medium (Phase 3.3) | Engage upstream smoltcp maintainers early. Carry a local patch as fallback. | +| Zero-copy incompatible with scheme IPC | Medium | High (Phase 3.6) | Treat as research only in Phase 3. Do not block other Phase 3 work on it. | +| Firewall in-process vs out-of-process debate stalls Phase 4 | Medium | High | Decision criterion: rule-lookup latency. Benchmark Design A vs Design B with a 1000-rule ruleset before committing. | +| Bare-metal NIC unavailable for Phase 5 | Medium | Medium | Prioritize QEMU validation. Document bare-metal as stretch. | +| Upstream relibc refactor (`cab002146` "move protocols into libredox") conflicts with local patches | Medium | High | Evaluate early in Phase 0. If the refactor is large, defer to a dedicated sync sprint rather than mixing with feature work. | + +## Execution Order Summary + +``` +Phase 0 (upstream sync, 2-3w) ──┬── Phase 1 (multi-NIC, capabilities, 3-4w) ──┬── Phase 3 (TCP perf, CC, 6-8w) + │ │ + │ ├── Phase 5 (drivers, HW validation, ongoing) + │ │ + └── Phase 2 (IPv6, 4-6w) ──────────────────────┤ + │ + └── Phase 4 (firewall, NAT, 6-10w) + │ + └── Phase 6 (forwarding, virtual devices, 8-16w) +``` + +**Critical path:** Phase 0 → Phase 1 → Phase 3 (TCP perf is the highest user-visible impact). +**Parallel track:** Phase 2 (IPv6) can proceed alongside Phase 3 once Phase 0 is done. +**Enabling track:** Phase 4 (firewall) unblocks Phase 6 (forwarding/NAT). +**Long tail:** Phase 5 (hardware validation) and Phase 6 (virtual devices) run continuously. + +## Appendix A: File Inventory + +### TCP/IP Stack Core (`local/sources/base/netstack/`) + +| File | Purpose | +|---|---| +| `src/main.rs` | Daemon entry: event loop, scheme fd setup, `Smolnetd::new()` | +| `src/scheme/mod.rs` | `Smolnetd` struct: owns `Interface`, `SocketSet`, `Router`, all scheme handlers | +| `src/scheme/socket.rs` | Generic `SocketScheme`: open/dup/read/write/fcntl/fevent | +| `src/scheme/tcp.rs` | `TcpScheme` over smoltcp `TcpSocket`. Port range 49152–65535. | +| `src/scheme/udp.rs` | `UdpScheme` over smoltcp `UdpSocket`. Connected-UDP peek filter. | +| `src/scheme/icmp.rs` | `IcmpScheme` over smoltcp `IcmpSocket`. Root-only. | +| `src/scheme/ip.rs` | `IpScheme` over smoltcp `RawSocket`. Root-only. | +| `src/scheme/netcfg/mod.rs` | `NetCfgScheme`: filesystem-like config (routes, DNS, interfaces). | +| `src/scheme/netcfg/nodes.rs` | `cfg_node!` macro for declaring config tree nodes. | +| `src/scheme/netcfg/notifier.rs` | fevent notification on config change. | +| `src/link/mod.rs` | `LinkDevice` trait + `DeviceList` container. | +| `src/link/ethernet.rs` | `EthernetLink`: ARP, neighbor cache (60s TTL), frame I/O. **IPv4-only.** | +| `src/link/loopback.rs` | Loopback `LinkDevice`. | +| `src/router/mod.rs` | `Router`: smoltcp `phy::Device` impl. RX/TX packet buffers. MTU=1486. | +| `src/router/route_table.rs` | `RouteTable`: `Vec` longest-prefix-match. | +| `src/buffer_pool.rs` | Buffer pool (audit usage). | +| `src/port_set.rs` | Ephemeral port allocation (49152–65535). | +| `src/error.rs` | Error types. | +| `src/logger.rs` | `redox_log` integration. Logger name: `"smolnetd"`. | +| `Cargo.toml` | smoltcp 0.12.0 features, redox-scheme/daemon/event deps. | + +### Driver Family (`local/sources/base/drivers/net/`) + +| File | Purpose | +|---|---| +| `driver-network/src/lib.rs` | `NetworkAdapter` trait (4 methods) + `NetworkScheme` wrapper. | +| `e1000d/src/{main,device}.rs` | Intel 8254x. PCI BAR0 MMIO. IRQ-driven. | +| `e1000d/config.toml` | PCI IDs: 8086:1004, 100e, 100f, 109a, 1503. | +| `ixgbed/src/{main,device}.rs` | Intel 82599 10GbE. | +| `rtl8139d/src/{main,device}.rs` | Realtek RTL8139. | +| `rtl8168d/src/{main,device}.rs` | Realtek RTL8168/8169. | +| `virtio-netd/src/{main,scheme}.rs` | VirtIO net. DMA-backed rx/tx queues. | + +### Socket Layer (`local/sources/relibc/`) + +| File | Purpose | +|---|---| +| `src/header/sys_socket/mod.rs` | POSIX socket API: socket/bind/connect/listen/accept/send/recv/... | +| `src/header/sys_socket/constants.rs` | SOCK_STREAM, SOCK_DGRAM, AF_INET, SOL_SOCKET, SO_* constants. | +| `src/platform/redox/socket.rs` | Redox backend: AF_INET→`/scheme/tcp`, dup-based connect/bind. | +| `src/header/netdb/mod.rs` | getaddrinfo, getnameinfo, gethostbyname, h_errno. | + +### Protocol Layer (`local/sources/libredox/`) + +| File | Purpose | +|---|---| +| `src/lib.rs:909` | `SocketCall` enum: Bind/Connect/SetSockOpt/GetSockOpt/SendMsg/RecvMsg/Unbind/GetToken/GetPeerName/Shutdown. | + +### Kernel Scheme Dispatch (`local/sources/kernel/`) + +| File | Purpose | +|---|---| +| `src/scheme/mod.rs` | Scheme registry: global schemes + UserScheme dispatch. | +| `src/scheme/user.rs` | `UserScheme`: forwards open/read/write/dup/call to userspace daemons. | + +### Configuration + +| File | Purpose | +|---|---| +| `config/redbear-netctl.toml` | Netctl profile examples (wired-DHCP, wired-static, wifi-dhcp, wifi-open). Init service `12_netctl.service`. | +| `config/redbear-wifi-experimental.toml` | Wi-Fi experimental target (includes `redbear-iwlwifi`). | +| `config/redbear-mini.toml` | Mini target: includes `redbear-netctl`, `redbear-netstat`, `redbear-wifictl`, `smolnetd` service. | +| `config/redbear-full.toml` | Full desktop target. | +| `/etc/init.d/10_smolnetd.service` | smolnetd (notify type). Requires `00_pcid-spawner`. | +| `/etc/init.d/10_dhcpd.service` | dhcpd (oneshot_async). Requires `10_smolnetd`. | + +### Custom Red Bear Networking Tools (`local/recipes/system/`) + +| Package | Purpose | +|---|---| +| `redbear-netctl` | Network profile orchestration (Arch netctl-style). | +| `redbear-netctl-console` | TUI frontend for `redbear-netctl`. | +| `redbear-netstat` | Socket/interface/routing statistics. | +| `redbear-wifictl` | Wi-Fi control daemon + scheme. | +| `redbear-mtr` | MTR (traceroute + ping). | +| `redbear-nmap` | Port scanner. | +| `redbear-traceroute` | Traceroute. | + +## Appendix B: Linux 7.1 Reference File Map + +For cross-referencing during implementation, the key Linux 7.1 source locations: + +| Subsystem | Linux Path | +|---|---| +| Socket layer | `net/socket.c` (3,822 LoC) | +| TCP | `net/ipv4/tcp.c` (5,382 LoC), `tcp_input.c`, `tcp_output.c` | +| UDP | `net/ipv4/udp.c` (3,892 LoC) | +| IP input/output | `net/ipv4/ip_input.c`, `ip_output.c` | +| IPv6 | `net/ipv6/` (40+ files), always built-in in 7.1 | +| FIB routing | `net/ipv4/fib_trie.c`, `fib_semantics.c`, `include/net/ip_fib.h` | +| Netfilter hooks | `include/uapi/linux/netfilter.h:43`, `struct nf_hook_ops` | +| Conntrack | `net/netfilter/nf_conntrack_core.c`, `include/net/netfilter/nf_conntrack.h:74` | +| nftables | `net/netfilter/nf_tables_api.c`, 57+ `nft_*.c` | +| Congestion control | `include/net/tcp.h:1325` `struct tcp_congestion_ops`, `tcp_cubic.c`, `tcp_bbr.c` | +| NAPI | `include/linux/netdevice.h:381` `struct napi_struct` | +| GRO/GSO | `net/core/gro.c`, `net/core/gso.c` | +| Zero-copy send | `net/ipv4/tcp.c:1140`, `net/core/skbuff.c:1719` | +| `sk_buff` | `include/linux/skbuff.h` (5,462 LoC) | +| `net_device` | `include/linux/netdevice.h:2139` (5,737 LoC) | +| `net_device_ops` | `include/linux/netdevice.h:1443` | +| Wireless (cfg80211) | `include/net/cfg80211.h` (10,995 LoC), `net/wireless/core.c` | +| Wireless (mac80211) | `net/mac80211/` (60+ files) | +| nl80211 | `include/uapi/linux/nl80211.h` (9,080 LoC) | +| epoll | `fs/eventpoll.c` (~3,000 LoC) | +| io_uring networking | `io_uring/net.c`, `io_uring/zcrx.c`, `IORING_OP_RECV_ZC` | + +## Appendix C: Upstream Redox Networking Activity (2024–2026) + +Selected high-relevance commits from `redox-os/*` repositories. Full list maintained in the +sync tracking issue. + +### Kernel + +- `6c3d5d28c` (2026-07-01) Move FD allocation logic into userspace +- `c089667ad` (2025-12-13) Remove legacy packet user schemes +- `4ff82ad8b` (2025-11-27) Fix user scheme deadlocks in `call_extended_inner` +- `08ea1da2f` (2025-07-05) Trigger read event for user schemes on fd close +- `99ff55ee1` (2026-02-08) Demux results as soon as received from user scheme + +### Netstack (now in `base`) + +- `b92be2e7d` (2025-03-09) Update to smoltcp 0.12 +- `d7c128684` (2025-03-09) Use CUBIC as TCP congestion controller +- `640e54897` (2024-09-04) Use 0.0.0.0 as default IP (DHCP fix) +- `49abe218a` (2024-11-26) ARP fix: respond only to device's broadcast/unicast +- `8e2d02232` (2024-03-18) Switch to libredox and redox-event +- `bafdb3b66` (2024-07-11) New scheme format, no legacy paths +- `f9b3170f0` (2024-02-28) Named network adapters +- `c06e5b14e` (2025-03-10) Netstack moved to base repo + +### Drivers + +- `ad9305bf9` (2024-02-28) Unify network drivers under `net/` +- `921c6b07f` (2024-12-26) driver-network migrated to redox-scheme +- `dd41c4f13` (2025-11-23) Net: scheme created before device init +- `0f24975ff` (2025-11-29) PCI interrupts rework5 (RTL dedup) +- `407533201` (2025-09-24) Add ThinkPad T60 ethernet to e1000d + +### relibc + +- `94b0cfc68` (2026-02-08) Reimplement recvmsg/sendmsg with bulk FD passing +- `1978c1aa4` (2026-03-25) Full recvmsg implementation +- `cab002146` (2026-02-28) Move protocols into libredox +- `6a455159a` (2026-01-22) Fix getaddrinfo infinite loop hang +- `726a0fb1a` (2025-09-15) getaddrinfo NULL nodename, AI_PASSIVE +- `5334455a2` (2025-08-20) getnameinfo + getaddrinfo loopback +- `9eaa9e82b` (2025-11-08) Pass through all SOL_SOCKET options +- `d44010170` (2025-07-18) UDS bind/connect with RedoxFS +- `9c6701802` (2026-01-07) Add sys_socket tests + +### syscall + +- `7a1409a91` (2026-05-27) Multiple FDs variant for call (bulk FD passing) +- `178461f6f` (2025-12-27) Remove remnants of old packet format + +### Build/config + +- `85b62fd85` (2026-03-01) Support networking in all configs +- `193e81897` (2026-02-17) ~115 WIP networking recipes +- `b68e5a685` (2026-04-11) Move dhcpd from netutils to base + +--- + +**Document version:** 1.0 (2026-07-07) +**Authority:** Canonical for TCP/IP stack, socket surface, Ethernet drivers, and network +configuration. See companion plans for Wi-Fi, IRQ, and Bluetooth. diff --git a/local/docs/USB-IMPLEMENTATION-PLAN.md b/local/docs/USB-IMPLEMENTATION-PLAN.md new file mode 100644 index 0000000000..144eee1c75 --- /dev/null +++ b/local/docs/USB-IMPLEMENTATION-PLAN.md @@ -0,0 +1,696 @@ +# Red Bear OS USB Plan — v3 + +> **Status:** Canonical. Replaces v1 (archived `archived/USB-IMPLEMENTATION-PLAN-v1-2026-04.md`) and v2 (archived `archived/USB-IMPLEMENTATION-PLAN-v2-2026-07.md`). +> **Date:** 2026-07-07. +> **Goal:** Make USB a first-class citizen in Red Bear OS — mature, reliable, feature-complete host stack matching the user-visible capability of Linux 7.1. + +## 0. Scope and method + +This plan is the comprehensive roadmap for Red Bear OS USB. It is built from three sources: + +1. **Code audit (v2 era, 2026-07-07):** every USB source file enumerated; TODOs, unwraps, and panic sites catalogued; maturity assessed per component. +2. **Linux 7.1 reference deep-dive:** `local/reference/linux-7.1/drivers/usb/` mapped feature-by-feature against Red Bear. Linux is treated as the implementation of excellence — structures, sequences, and edge-case handling are cross-referenced and reimplemented in Rust as needed. +3. **Upstream Redox commit scan:** 33 USB-relevant upstream commits (Jan 2025 – May 2026) tracked; 28 already adopted, 4 needing verification, 1 outstanding. + +### Validation labels + +- **builds** — code is in tree and compiles. +- **enumerates** — runtime surfaces can discover controllers, ports, descriptors. +- **usable-narrow** — one controller family / one class family works in a bounded scenario. +- **validated-QEMU** — a documented QEMU script passed on the matching recipe, config, and commit. +- **validated-hardware** — a named physical controller + class, with a captured log, on real bare metal. +- **experimental** — present for bring-up but not in any support-promised path. + +### Linux 7.1 as the implementation of excellence + +Where the v2 plan referenced Redox upstream commits, v3 references **Linux 7.1** as the canonical reference. When implementation detail is in doubt: + +1. Check Linux 7.1 first. +2. If Linux has the algorithm, port it to Rust line-by-line (preserving the data structure layouts). +3. If Linux has multiple approaches (e.g., kernel vs. userspace), use the userspace model since Red Bear's USB drivers are all userspace. +4. Cite the Linux source file:line in commit messages. + +## 1. Executive summary — gap analysis + +| Dimension | Red Bear | Linux 7.1 | Gap | +|---|---|---|---| +| Total USB source lines | ~13,000 | ~280,000+ | ~22× | +| Host controllers with real implementations | 1 (xhcid) | 4 (xHCI, EHCI, UHCI, OHCI) | 3 missing | +| Host controllers building | 3 (ehcid, uhcid, ohcid) | — | — | +| Storage protocols | 1 (BOT) | 5+ (BOT, UAS, SAT, UASP, multi-LUN) | 4 missing | +| HID class features | Boot protocol only | Full report descriptor parsing, multi-touch, quirks | ~80% gap | +| Class drivers | 3 working + 3 empty stubs | 80+ drivers | ~75 missing | +| Hub driver lines | 249 | 6,567 (hub.c) | ~95% gap | +| xHCI quirks | 0 | 51 quirks | **CRITICAL** | +| Error recovery | Stall only | Babble, transaction error, split error, clear-TT | ~90% gap | +| Power management | None | Runtime PM, U1/U2, LPM, autosuspend | All missing | +| Hardware validation matrix | 0 entries | ~200 known-good controllers | All missing | + +### P0 — already done (carried from v2) + +P0 is complete and committed: + +- **P0-A1** xHCI interrupts re-enabled (base fork `cbd40e0d`) +- **P0-A2** Reset fix verified in 0.1.0 baseline +- **P0-A3** CSZ cleanup (removed stale TODO, base fork `7cfed158`) +- **P0-A4** Bounds check on `root_hub_port_index()` (base fork `774a0ac1`) +- **P0-B1** EHCI auto-spawn + xhcid-compat scheme (ehcid + usb-core) +- **P0-B2** UHCI + OHCI real drivers (459 + 280 LoC, compile clean) + +### Architectural goal + +The single architectural goal of v3: **every host controller implements `usb_core::UsbHostController`** and registers under the same scheme namespace. Currently only `ehcid` implements the trait; `xhcid`, `uhcid`, `ohcid` each have ad-hoc scheme implementations. Unifying these is the prerequisite for every other improvement in v3. + +### Critical risks (unconditional stop-ship) + +Three risks make Red Bear USB unsupportable today. They MUST be fixed before any further phase: + +1. **usbscsid has 12 `panic!()` calls in hot path.** A USB stick disconnect during transfer crashes the entire system. Must convert all to `Result` returns. +2. **xhcid has ~125 panic sites / unwrap() / expect() calls.** Mutex poison panic = full kernel panic. Most are fixable with proper error propagation. +3. **3 class drivers are 32-line stubs** (`redbear-usbaudiod`, `redbear-acmd`, `redbear-ecmd`) that print "not yet implemented" and exit. They are wired in configs but do nothing — dead weight in every image. + +## 2. Phase roadmap + +| Phase | Goal | Exit criteria | +|---|---|---| +| **P1** | Trait unification + panic hardening | All 4 controllers implement `UsbHostController`; usbscsid 0 panic sites; xhcid <20 unwraps | +| **P2** | xHCI core completeness | Quirk table populated; HCCPARAMS2 parsed; error recovery for babble/transaction/stall/split | +| **P3** | Hub driver maturity | Full enumeration sequence; USB 3.0 SS hub support; port power/reset timing; wHubDelay accumulation | +| **P4** | Storage: UAS + multi-LUN | UAS protocol implementation; multi-LUN dispatch; SYNCHRONIZE_CACHE; WRITE SAME/UNMAP | +| **P5** | HID: report descriptor parser + input mapping | Full HID report descriptor parser; usage→evdev mapping; HID quirks; multi-touch support | +| **P6** | Class driver completeness | CDC ACM; CDC NCM; USB Audio; remove the 3 empty stubs | +| **P7** | Power management | USB 2.0 LPM; USB 3.0 U1/U2/U3; runtime PM autosuspend; xHCI link state management | +| **P8** | Validation: hardware matrix + QEMU harness expansion | Hardware-validation matrix with one row per controller; new QEMU tests for OHCI/UHCI/EHCI; error-injection tests | +| **P9** | Modern USB scope ADR (inherited from v2 §12) | Decided: host-only. Already written. | + +--- + +## 3. P1 — Trait unification and panic hardening + +**Why first:** Without a unified trait, every later phase has to write per-controller glue. Panic hardening is unconditional — the system cannot ship with mass-storage and xHCI crashes. + +### P1-A: All four controllers implement `usb_core::UsbHostController` + +**Reference:** `local/reference/linux-7.1/drivers/usb/core/hcd.c` (Host Controller Driver framework, 3,194 lines) and `drivers/usb/host/` (4 host drivers, all conforming to a shared `struct hc_driver`). + +Current state: + +| Controller | Trait | Scheme | Status | +|---|---|---|---| +| xhcid | ❌ | own scheme.rs | ad-hoc | +| ehcid | ✅ | redox-scheme::SchemeSync | good template | +| uhcid | ❌ | none | enum-only | +| ohcid | ❌ | none | enum-only | + +**Tasks:** + +1. **Define unified scheme path:** `usb-core::SCHEME_NAMESPACE = "usb._"` (xhcid-style). All controllers register under this namespace. +2. **Refactor xhcid's `scheme.rs`** to satisfy `UsbHostController` trait. The trait's `control_transfer` / `bulk_transfer` / `interrupt_transfer` delegate to existing xhcid scheme handlers. +3. **Add scheme registration to uhcid and ohcid** using the same pattern. Their existing in-guest enumeration stays; we add a `SchemeSync` server under the new namespace. +4. **Replace ad-hoc class spawn** in all four controllers with `usb-core::spawn::spawn_class_driver_for_port`. Class daemons open the namespace from the controller's `scheme_name`. + +**Files to touch:** +- `local/sources/base/drivers/usb/xhcid/src/xhci/scheme.rs` (refactor) +- `local/recipes/drivers/uhcid/source/src/main.rs` (add scheme) +- `local/recipes/drivers/ohcid/source/src/main.rs` (add scheme) +- `local/recipes/drivers/usb-core/source/src/scheme.rs` (extend trait if needed) + +**Linux 7.1 cross-reference:** `drivers/usb/core/hcd.c:1700` `usb_create_hcd()` — uniform scheme-name pattern `bus_name-hcd_name`. Adapt for Redox's scheme namespace. + +**Exit:** all four controllers implement the trait; class daemons can connect to all four via the same scheme. + +### P1-B: usbscsid panic hardening + +**Reference:** `local/reference/linux-7.1/drivers/usb/storage/transport.c` (1,462 lines) and `drivers/usb/core/urb.c` (1,021 lines). Linux USB storage never panics on stall — it returns `-EPIPE`, `-ETIME`, `-EIO`, etc. + +**Files to touch:** `local/sources/base/drivers/storage/usbscsid/src/` + +**Tasks:** + +1. Replace every `panic!()` in `protocol/bot.rs` (12 sites) with `return Err(...)` or `bail!()`. +2. Each panic site maps to a specific failure mode. Audit each one: + - CBW send failure → `Err(StorageError::CommandBlockError)` + - Data-phase stall → `Err(StorageError::Stall)` (don't kill the daemon) + - CSW parse error → `Err(StorageError::InvalidStatusWrapper)` + - Mass storage reset → call `usbscsid::reset_port()` instead of panicking +3. Replace `mod uas { // TODO }` stub with either a real UAS implementation (see P4) or a clean `unimplemented!()` that returns `Err(UasError::NotSupported)`. +4. Add `#[derive(Debug, thiserror::Error)]` for all error types — no string-based panic messages. + +**Exit:** `cargo check` passes; `grep -c panic!()` on the usbscsid tree returns 0. + +### P1-C: xhcid panic reduction + +**Reference:** `local/reference/linux-7.1/drivers/usb/host/xhci.c` — uses `dev_err()`, `dev_warn()`, and explicit `return -EINVAL` for every error path. Linux xhci never panics in normal operation. + +**Files to touch:** `local/sources/base/drivers/usb/xhcid/src/xhci/` + +**Tasks:** + +1. Replace each `.unwrap()` on Mutex locks with `.lock().unwrap_or_else(|e| e.into_inner())` — already done in some sites; complete across the tree. +2. Replace each `.expect()` with explicit error logging + fallible helper functions. +3. Remove the 5 explicit `panic!()` calls in hot paths: + - `irq_reactor.rs:31` — `Failed to received an enumeration request!` → log error + return + - `irq_reactor.rs:71` — `No XHCI controllers found` → log error + abort init cleanly + - `irq_reactor.rs:620` — `Polling finished EventTrbFuture again` → log warning + skip + - `irq_reactor.rs:672/703/731` — `Invalid TRB type` → log error + corrupt-ring recovery + - `scheme.rs:2011` — `unreachable` HandleKind → log error + return EBADF +4. Convert `mod.rs:694` `TODO handle the second unwrap` and similar known-fragile sites. + +**Target:** ≤20 unwrap/expect total in xhcid (down from ~125). + +**Exit:** `grep -c 'panic!\|\\.unwrap()\|\\.expect(' local/sources/base/drivers/usb/xhcid/src/xhci/*.rs` ≤20. + +### P1-D: Remove the 3 empty stubs + +**Files:** +- `local/recipes/system/redbear-usbaudiod/` (32 LoC stub) +- `local/recipes/system/redbear-acmd/` (32 LoC stub) +- `local/recipes/system/redbear-ecmd/` (32 LoC stub) + +**Task:** Either implement them properly (see P6 for class driver work) or remove them from the configs. Stub classes inflate image size and mislead operators about USB capability. + +--- + +## 4. P2 — xHCI core completeness + +**Why this matters:** The 51-entry xHCI quirk table in Linux is the difference between "works on QEMU" and "works on real hardware." Without quirks, xhcid will silently misbehave on Intel, AMD, NEC, ASMedia, Etron, Cadence, and Fresco Logic controllers. + +### P2-A: xHCI quirk table + +**Reference:** `local/reference/linux-7.1/drivers/usb/host/xhci.h:1587-1649` (51 quirk flags). + +**Tasks:** + +1. Define a `XhciQuirks` bitflags struct in `xhcid/src/xhci/mod.rs` matching the Linux 51-entry enum. +2. Implement a per-controller quirk probe: read `PCI_VENDOR_ID` + `PCI_DEVICE_ID`, look up in a quirk table keyed by `{vendor, device}`. +3. Implement the behavior change for each quirk. Reference: `drivers/usb/host/xhci-pci.c:600+` `xhci_pci_quirks()`. +4. Start with the highest-priority quirks: + - `XHCI_RESET_TO_DEFAULT` — all xhci drivers should program MaxSlotsEn = 1 on reset + - `XHCI_LPM_SUPPORT` — Intel Panther Point and later + - `XHCI_BROKEN_STREAMS` — disable streams on affected controllers + - `XHCI_WRITE_64_HI_LO` — write 64-bit registers high-then-low on AMD + - `XHCI_ZERO_64B_REGS` — read 32-bit only on old hardware + - `XHCI_DEFAULT_PM_RUNTIME_ALLOW` — enable runtime PM by default on Intel +5. Wire quirk flags into all relevant code paths: reset, run, MSI-X allocation, streams, ring sizing, error recovery. + +**Linux cross-reference:** `xhci-pci.c:617-900` lists the per-controller quirk table. Translate to a Rust match expression. + +**Files:** `xhcid/src/xhci/quirks.rs` (new), `xhcid/src/xhci/mod.rs`, `xhcid/src/xhci/pci_quirks.rs` (new). + +**Exit:** per-controller quirk probe running at init; quirk-aware behavior in 5+ paths. + +### P2-B: HCCPARAMS2 parsing (xHCI 1.1+ features) + +**Reference:** `local/reference/linux-7.1/drivers/usb/host/xhci-caps.h:94-119` + +13 HCCPARAMS2 bits enable U3 entry, extended TBC, eUSB2, etc. Current xhcid reads only HCCPARAMS1. + +**Tasks:** + +1. Read and cache `HCSPARAMS2` and `HCCPARAMS2` in `capability.rs`. +2. Define bitflags for each of the 13 capabilities. +3. Gate xHCI 1.1+ features on these bits: + - `HCC2_U3C` — U3 Entry Capability + - `HCC2_CMC` — Configure Endpoint Max Exit Latency Too Large + - `HCC2_LEC` — Large ESIT Payload (>48KB) + - `HCC2_CIC` — Configuration Information Capability (used for Set Latency Tolerance) +4. Extended port capabilities from `xhci-ext-caps.h:62-66`: `XHCI_L1C`, `XHCI_HLC`, `XHCI_BLC` for LPM. + +**Exit:** HCCPARAMS2 parsed; 3+ features gated on it. + +### P2-C: Error recovery + +**Reference:** `local/reference/linux-7.1/drivers/usb/host/xhci-ring.c` (4,472 lines), specifically `handle_tx_event()` for the 36 completion codes. + +**Tasks:** + +1. Extend `TrbCompletionCode` handling from 3 codes (Success, ShortPacket, Stall) to all 36. +2. For each error code, implement recovery: + - `COMP_STALL_ERROR` → endpoint halt + clear-stall control transfer + retry + - `COMP_DATA_BUFFER_ERROR` → re-queue transfer with new DMA mapping + - `COMP_BABBLE_DETECTED_ERROR` → reset endpoint + check port disable + - `COMP_USB_TRANSACTION_ERROR` → retry up to 3 (per `MAX_SOFT_RETRY`) + - `COMP_TRB_ERROR` → log + fatal to upper layer + - `COMP_RESOURCE_ERROR` → re-queue later + - `COMP_SPLIT_TRANSACTION_ERROR` → call Clear-TT-Buffer on parent hub +3. Track retry counts in `Urb` struct. Reference: `xhci-ring.c:1700+` `xhci_urb_request()`. + +**Linux cross-reference:** `xhci-ring.c:1700-1900` `xhci_handle_completion()` — the canonical error-handling function. + +**Exit:** 36 completion codes handled with appropriate recovery. + +--- + +## 5. P3 — Hub driver maturity + +**Why this matters:** The current usbhubd is 249 LoC doing basic port-change detection. Linux hub.c is 6,567 lines doing full enumeration. The gap is 95%. + +### P3-A: Full enumeration sequence + +**Reference:** `local/reference/linux-7.1/drivers/usb/core/hub.c:1449` `hub_configure()` and `hub_port_connect()`. + +**Tasks:** + +1. Read hub descriptor (already done — `HubDescriptorV2` / `HubDescriptorV3`). +2. Power-on delay: `bPwrOn2PwrGood * 2ms` (USB 2.0 spec §7.1.6.1). +3. USB 3.0 hub initial state: wait for polling → U0 (tPollingLFPSTimeout = 360ms, polled 10×36ms). +4. Per-port enumeration: + - Reset signaling (USB 2 ≥10ms, USB 3 warm reset with PORT_WR) + - Wait for reset completion + - Read device descriptor (8 bytes → full 18 bytes) + - Set address + - Read configuration descriptors + - Configure endpoints +5. Hub-specific class requests for USB 3 hubs: + - `HUB_SET_DEPTH` (request 0x0C) — set hub tier depth +6. `wHubDelay` accumulation through the TT path (for split-transaction scheduling). + +**Linux cross-reference:** `hub.c:1497-1521` for wHubDelay; `hub.c:3000-3300` for the full port-connect path. + +**Files:** `local/sources/base/drivers/usb/usbhubd/src/main.rs` + +**Exit:** complete hub enumeration including power timing, USB 3 hub handling, and TT accumulation. + +### P3-B: Hub interrupt-driven change detection + +**Reference:** `linux-7.1/drivers/usb/core/hub.c:1000+` `hub_irq()`. + +Current usbhubd polls at 100ms intervals. Linux uses hub interrupt endpoints (`USB_ENDPOINT_DIR_IN` + `USB_USAGE_TYPE_INTERRUPT`) for immediate change notification. + +**Tasks:** + +1. Add interrupt endpoint configuration to usbhubd (USB hub interrupt endpoint is always EP1). +2. Subscribe to interrupt IN transfers for status change bitmap. +3. Fall back to polling only when interrupt endpoint is unavailable. + +**Exit:** interrupt-driven on USB 2 hubs; polling fallback on USB 3 hubs (which lack interrupt endpoints). + +### P3-C: Port LED control + +**Reference:** `linux-7.1/drivers/usb/host/xhci-port.h:64-67` (PORT_LED_OFF/AMBER/GREEN). + +**Tasks:** + +1. Expose hub port LED control via `SET_HUB_FEATURE(PORT_INDICATOR)`. +2. Implement in usbhidd keyboard LED sync (cross-couple with P5-C). + +**Exit:** port LEDs controllable via scheme. + +--- + +## 6. P4 — Storage: UAS + multi-LUN + SAT + +**Why this matters:** BOT alone limits USB 3.0 storage to ~30-40 MB/s. UAS unlocks 350+ MB/s. Multi-LUN is needed for card readers. SYNCHRONIZE_CACHE / UNMAP are needed for SSD correctness and TRIM. + +### P4-A: UAS protocol + +**Reference:** `local/reference/linux-7.1/drivers/usb/storage/uas.c` (1,304 lines). + +UAS uses 4 endpoints (cmd BULK OUT, status BULK IN, data-in BULK IN, data-out BULK OUT) and Information Units (Command IU, Sense IU, Response IU). + +**Tasks:** + +1. Detect UAS interface (interface class 0x08, subclass 0x04, protocol 0x62 for UAS; 0x50 for BOT). +2. Implement the 4-endpoint setup in usbscsid. +3. Implement Command IU submission (32-byte structured). +4. Implement Sense IU / Response IU parsing. +5. Wire to xHCI streams (P2 prerequisite): UAS needs streams for true concurrent commands. + +**Files:** `local/sources/base/drivers/storage/usbscsid/src/protocol/uas.rs` (new). + +**Linux cross-reference:** `uas.c:35-74` struct definitions; `uas.c:600-900` command submission. + +**Exit:** UAS path functional, fallback to BOT for non-UAS devices. + +### P4-B: Multi-LUN dispatch + +**Reference:** `linux-7.1/drivers/usb/storage/usb.c:1258` (`usb_stor_scan_thread()`). + +**Tasks:** + +1. Extend SCSI dispatch to iterate LUN 0..max_lun. +2. REPORT_LUNS opcode handling — list of LUNs available. +3. Per-LUN device state — each LUN gets its own BlockDevice. + +**Files:** `local/sources/base/drivers/storage/usbscsid/src/scsi/cmds.rs`, `scsi/mod.rs`. + +**Exit:** 4-slot card reader enumerates as 4 separate block devices. + +### P4-C: SYNCHRONIZE_CACHE / WRITE SAME / UNMAP + +**Reference:** `linux-7.1/drivers/scsi/sd.c` (sd driver issues these). + +**Tasks:** + +1. Add `SYNCHRONIZE_CACHE(10)` and `SYNCHRONIZE_CACHE(16)` opcodes to `scsi/opcodes.rs`. +2. Add `WRITE_SAME(10/16)` and `UNMAP` opcodes. +3. Dispatch them in `scsi/cmds.rs` to the BOT transport. + +**Exit:** SSD TRIM works, data integrity on disconnect improves. + +### P4-D: Mass-storage quirks integration + +**Reference:** `linux-7.1/drivers/usb/storage/unusual_devs.h` (2,513 lines, 323 entries). + +**Current state:** Red Bear has 214 mass-storage quirks extracted from `unusual_devs.h`. Verify they're loaded at runtime and applied during enumeration. + +**Tasks:** + +1. Verify `usb-core::spawn::class_driver_for_usb_class()` for mass storage (0x08). +2. Add runtime quirk lookup: when usbscsid enumerates a device, look up its VID:PID in the quirk table, apply `IGNORE_RESIDUE`, `FIX_CAPACITY`, `SINGLE_LUN`, `MAX_SECTORS_64`, etc. +3. Implement the runtime application in `usbscsid/src/main.rs` at the `read_capacity()` and `read_write()` paths. + +**Exit:** quirks applied at enumeration; redbear-usb-storage-check verifies a quirk-affected device behaves correctly. + +--- + +## 7. P5 — HID: report descriptor parser + input mapping + +**Why this matters:** usbhidd at 566 LoC only handles boot-protocol keyboard/mouse. Linux's `hid-core.c` (3,228 LoC) + `hid-input.c` (2,454 LoC) together handle every HID device class. The gap blocks touchscreens, gamepads (partially), consumer keys, and every non-trivial mouse/keyboard. + +### P5-A: HID report descriptor parser + +**Reference:** `local/reference/linux-7.1/drivers/hid/hid-core.c:1273` `hid_open_report()`. + +**Tasks:** + +1. Implement a full HID Report Descriptor parser in a new crate `local/recipes/drivers/redbear-hid-core/source/src/lib.rs`: + - `ReportDescriptor` struct: tree of `UsagePage`, `Usage`, `Collection` (Application/Physical/Logical), `Field` (Input/Output/Feature with bit fields), `Report` (with report IDs and sizes). +2. Parse the main item types: `Usage Page`, `Usage`, `Logical Minimum/Maximum`, `Report Size`, `Report Count`, `Report ID`, `Push/Pop`, `Collection/End Collection`. +3. Output a tree that other HID drivers can walk. + +**Files:** new crate `local/recipes/drivers/redbear-hid-core/`. + +**Exit:** parses a Logitech mouse descriptor correctly into a structured tree. + +### P5-B: HID input mapping (usages → evdev) + +**Reference:** `local/reference/linux-7.1/drivers/hid/hid-input.c:1244` `hidinput_configure_usage()`. + +**Tasks:** + +1. Implement usage → input event mapping table: + - Generic Desktop 0x01: X/Y/Wheel → ABS_X/ABS_Y/REL_WHEEL + - Keyboard 0x07: a/A/Enter/etc → KEY_A/KEY_A/KEY_ENTER + - Button 0x09: mouse buttons → BTN_MOUSE/BTN_LEFT + - LED 0x08: keyboard LEDs + - Consumer 0x0C: Mute/Volume → KEY_MUTE/KEY_VOLUMEUP + - Digitizer 0x0D: Touch/Position → BTN_TOUCH/ABS_MT_POSITION_X +2. Wire to `redbear-inputd` via evdev events (or whatever input daemon we use). +3. Switch from Orbital KeyEvent protocol to evdev for unified input. + +**Linux cross-reference:** `hid-input.c:140-200` (the usage→keycode mapping table). + +**Exit:** media keys, gamepad buttons, and touchpad gestures work via USB HID. + +### P5-C: LED sync + +**Reference:** `local/reference/linux-7.1/drivers/hid/hid-input.c:317` `hidinput_output_event()` (Caps Lock → LED report). + +**Tasks:** + +1. usbhidd: after boot completion, send `SET_REPORT` to set Caps Lock / Num Lock / Scroll Lock LED state. +2. Wire to `redbear-inputd` LED state events. +3. For both USB HID boot protocol and report protocol. + +**Exit:** Caps Lock LED on a USB keyboard actually lights up when toggled. + +### P5-D: HID quirks + +**Reference:** `local/reference/linux-7.1/drivers/hid/hid-quirks.c` (1,397 lines, ~200 device quirks). + +**Tasks:** + +1. Add HID quirks table in `redbear-hid-core`. +2. Per-device VID:PID overrides: `HID_QUIRK_NOTOUCH`, `HID_QUIRK_NO_INIT_REPORTS`, `HID_QUIRK_NOGET`, `HID_QUIRK_MULTI_INPUT`, etc. +3. Apply at `usbhidd` enumeration time. + +**Exit:** Logitech Unifying receivers, Apple Magic Mouse, and Surface keyboards all work. + +### P5-E: Multi-touch + +**Reference:** `local/reference/linux-7.1/drivers/hid/hid-multitouch.c` (2,756 lines). + +**Tasks:** + +1. Detect multi-touch class (Usage Page 0x0D). +2. Implement slot-based protocol (Type B) and report-based protocol (Type A). +3. Forward ABS_MT_POSITION_X, ABS_MT_POSITION_Y, ABS_MT_SLOT, BTN_TOUCH to `inputd`. + +**Exit:** a USB touchscreen or touchpad produces touch events. + +--- + +## 8. P6 — Class driver completeness + +**Why this matters:** Red Bear has 6 class drivers, of which only 3 work (usbhubd, usbhidd, usbscsid). The other 3 are 32-line stubs. Plus we're missing CDC NCM (ethernet), USB Audio, USB-serial, and others. + +### P6-A: CDC ACM (remove the stub) + +**Reference:** `local/reference/linux-7.1/drivers/usb/class/cdc-acm.c` (2,186 lines). + +**Files:** `local/recipes/system/redbear-acmd/` (currently stub). + +**Tasks:** + +1. Implement a real CDC ACM class driver (`redbear-cdc-acmd` or just keep the name and add code). +2. Line coding: baud rate, parity, stop bits, data bits. +3. Flow control: RTS/CTS, DTR/DSR via control transfers. +4. Break signaling. +5. Multi-port modems (data + control + notification interfaces). +6. Expose `/scheme/tty` for userspace clients (replacing the stub `redbear-acmd` admin CLI). + +**Exit:** an FTDI USB-serial adapter or Arduino appears as a TTY device. + +### P6-B: CDC NCM (Ethernet over USB) + +**Reference:** `linux-7.1/drivers/net/usb/cdc_ncm.c` (in `drivers/net/usb/`). + +**Tasks:** + +1. Detect CDC NCM interface (subclass 0x0D). +2. NTB (Network Transfer Block) framing. +3. Aggregate frames into NTBs for efficiency. +4. Wire to the netstack as a new interface. + +**Exit:** a USB-Ethernet adapter (e.g., a phone-tethered device) works. + +### P6-C: USB Audio + +**Reference:** `local/reference/linux-7.1/sound/usb/` (USB audio class driver, ~5,000 lines). + +`redbear-usbaudiod/` is currently a 32-line stub. Replace with a real USB Audio Class 1.0 driver: + +**Tasks:** + +1. Detect AudioControl interface + AudioStreaming interface. +2. Parse Audio Control descriptors (unit IDs, terminals, mixer units). +3. Configure audio streaming endpoint (alternate settings, packet size, format). +4. Isochronous transfer ring setup (xHCI isoch — new code path). +5. PCM playback/capture path to `redbear-audiod`. + +**Exit:** USB headphones / headset produce audio. + +### P6-D: USB-serial (top 5 drivers) + +**Reference:** `linux-7.1/drivers/usb/serial/` (56 .c files). + +For v3, implement only the top 5 most common USB-serial chip families: + +1. **FTDI** (`ftdi_sio.c`, 3,176 lines) — FT232, FT2232, FT4232 +2. **CP210x** (`cp210x.c`, 753 lines) — Silicon Labs CP2102/CP2104 +3. **PL2303** (`pl2303.c`, 615 lines) — Prolific PL2303 +4. **CH341** (`ch341.c`, 583 lines) — QinHeng CH341 +5. **CDC-ACM subset** (covered by P6-A) + +**Exit:** common USB-serial adapters work. + +### P6-E: USB compliance test driver + +**Reference:** `linux-7.1/drivers/usb/misc/usbtest.c` (3,079 lines). + +A `redbear-usbtest` tool that exercises: +- Bulk IN/OUT loopback with pattern verification +- Control endpoint round-trip +- Interrupt endpoint reception +- Isochronous endpoint loopback +- Halt/reset recovery + +**Exit:** USB protocol compliance can be validated in-tree. + +--- + +## 9. P7 — Power management + +**Why this matters:** Without PM, USB devices draw full power forever, draining laptop batteries and overheating peripherals. Linux's PM framework handles U1/U2/U3/LPM/autosuspend. + +### P7-A: USB 2.0 LPM + +**Reference:** `linux-7.1/drivers/usb/host/xhci-port.h:135-173`, `xhci-hub.c`. + +**Tasks:** + +1. Parse `HCCPARAMS1.HLC` (Hardware LPM Capability). +2. Implement `usb_set_device_initiated_lpm()` (port hardware LPM control via PORTSC.PORT_L1DEV). +3. xhcid: implement `PORT_HLE` enabling per port. +4. ehcid: implement via USBCMD.HIRD (Host Initiated Resume Duration). + +**Exit:** USB 2.0 devices on HWA LPM-capable hubs enter L1 state when idle. + +### P7-B: USB 3.0 U1/U2/U3 + +**Reference:** `linux-7.1/drivers/usb/host/xhci-port.h:17-30`, `xhci.c:2800+` `xhci_stop_device()`. + +**Tasks:** + +1. Implement U1/U2 entry per port via `PORT_U1_TIMEOUT` / `PORT_U2_TIMEOUT` registers. +2. U3 (suspend) via `Port Link State Write` command TRB. +3. U3 → Resume (U0) detection on PORTSC.PLC. +4. xHC-initiated suspend/resume (device responds to `SetPortFeature(PORT_SUSPEND)`). + +**Exit:** USB 3.0 devices enter U1/U2/U3 when idle, resume on traffic. + +### P7-C: Runtime PM autosuspend + +**Reference:** `linux-7.1/drivers/usb/core/driver.c:2081` `usb_autosuspend()`. + +**Tasks:** + +1. Add per-device idle timer (default 2s). +2. Suspend device (U1/U2/U3) when timer expires. +3. Resume on URBs being submitted (auto-resume). +4. Disable autosuspend for devices that explicitly opt out. + +**Exit:** USB mice, keyboards, storage automatically suspend after idle, resume instantly on activity. + +--- + +## 10. P8 — Validation: hardware matrix + QEMU harness expansion + +**Why this matters:** All USB work so far is QEMU-only. We need to validate on real hardware. This phase builds the harness for that. + +### P8-A: Hardware validation matrix + +**Reference:** `local/docs/HARDWARE-VALIDATION-MATRIX.md` (existing, currently tiny). + +**Tasks:** + +1. Expand the matrix: one row per (controller, class, board) tuple. +2. Required test points per row: + - Controller PCI ID → driver recognizes + - Device enumeration succeeds + - Specific class test (HID keypress, storage read+write, audio playback) + - Hot-plug works (USB 3.x + USB 2.0) + - Suspend/resume works (if device supports) +3. Minimum viable matrix entries (10+): + - xhcid on AMD Ryzen (USB 3.2, multiple hubs) + - xhcid on Intel Tiger Lake (USB 4 / Thunderbolt 3 compatible) + - xhcid on Intel C620 server chipset + - ehcid on a USB 2.0-only motherboard + - uhcid on a Via-based legacy system + - ohcid on an Nvidia nForce board + - USB HID: Logitech MX Master, Apple Magic Mouse, Cherry keyboard + - USB Storage: Samsung T7 SSD (UAS-capable), SanDisk Ultra (BOT-only) + - USB Hub: Anker 7-port powered hub, USB 3.0 hub chain + +**Exit:** matrix has ≥10 rows; each row's last-tested date <6 months. + +### P8-B: New QEMU tests + +The v3 plan calls for QEMU test scripts that don't exist yet: + +**Tasks:** + +1. `test-uhci-runtime-qemu.sh --check` — boot with `-device piix3-usb-uhci`, verify enumeration. +2. `test-ohci-runtime-qemu.sh --check` — boot with `-device nec-usb-xhci` emulating OHCI compat, verify enumeration. +3. `test-ehci-class-autospawn-qemu.sh --check` — boot with USB keyboard on EHCI route, verify usbhidd spawns and keypress reaches inputd. +4. `test-usb-hub-qemu.sh --check` — boot with a USB hub attached, verify the hub enumerates and devices behind it enumerate. +5. `test-usb-error-recovery-qemu.sh --check` — hot-unplug during transfer, verify graceful error (not panic). +6. `test-usb-uas-qemu.sh --check` — boot with USB 3.0 storage device supporting UAS, verify UAS path active. + +**Exit:** all 6 scripts pass. + +### P8-C: Error injection tests + +**Reference:** Linux's `usbtest.c` includes error injection patterns. + +**Tasks:** + +1. Build a `redbear-usb-fuzz` tool that performs: + - Random disconnects mid-transfer + - Stall injection on specific endpoints + - Babble simulation + - CRC error injection (in conjunction with `usbtest` if available) +2. Verify usbscsid / usbhidd / usbaudiod handle errors without panic. + +**Exit:** no panics under 10,000 random disconnect events. + +--- + +## 11. Build-and-verify workflow (per-session) + +```bash +# After any P-phase change: +cd local/sources/base && git checkout +cargo check -p xhcid --offline + +cd local/recipes/drivers/uhcid/source && cargo check --offline +cd local/recipes/drivers/ohcid/source && cargo check --offline +cd local/recipes/drivers/ehcid/source && cargo check --offline +cd local/recipes/drivers/usb-core/source && cargo check --offline + +# Full build: +./local/scripts/build-redbear.sh --upstream redbear-mini + +# QEMU tests: +./local/scripts/test-xhci-irq-qemu.sh --check +./local/scripts/test-xhci-device-lifecycle-qemu.sh --check +./local/scripts/test-usb-storage-qemu.sh +./local/scripts/test-usb-qemu.sh +./local/scripts/test-usb-maturity-qemu.sh + +# After P1 completes: +./local/scripts/test-uhci-runtime-qemu.sh --check # new +./local/scripts/test-ohci-runtime-qemu.sh --check # new +./local/scripts/test-ehci-class-autospawn-qemu.sh --check # new +./local/scripts/test-usb-hub-qemu.sh --check # new +./local/scripts/test-usb-error-recovery-qemu.sh --check # new +``` + +## 12. Stale documentation cleared + +This v3 supersedes: + +- `local/docs/USB-IMPLEMENTATION-PLAN.md` (v2 active) → archived as `local/docs/archived/USB-IMPLEMENTATION-PLAN-v2-2026-07.md` +- `local/docs/USB-VALIDATION-RUNBOOK.md` (v2 active) → archived as `local/docs/archived/USB-VALIDATION-RUNBOOK-2026-07.md` + +`archived/README.md` updated to record the v1 → v2 → v3 supersession chain. + +All Linux 7.0 references in active docs migrated to Linux 7.1 (this happened earlier in the v2 era). + +## 13. Cross-reference summary + +For every feature in v3, the cross-reference is to Linux 7.1 at `local/reference/linux-7.1/`: + +| Feature | Linux 7.1 file | Lines | Implementation strategy | +|---|---|---|---| +| xHCI quirk table | `drivers/usb/host/xhci.h:1587-1649` | 63 | Reimplement as bitflags struct + per-controller lookup table | +| xHCI error recovery | `drivers/usb/host/xhci-ring.c:1700-1900` | 200+ | Port 36-code dispatch table; line-by-line Rust translation | +| USB 2.0 LPM | `drivers/usb/host/xhci-port.h:135-173` | 39 | Reimplement port-link-state transitions | +| Full hub enumeration | `drivers/usb/core/hub.c:1449-1521,3000-3300` | 300+ | Sequential port-and-hub init with timing-aware power-on | +| UAS protocol | `drivers/usb/storage/uas.c` | 1304 | Port full protocol, port 4-endpoint + IU framing | +| HID report descriptor parser | `drivers/hid/hid-core.c:1273` | 200+ | Token-based parser → tree of usage/collection/field | +| HID input mapping | `drivers/hid/hid-input.c:1244+` | 200+ | Usage page → keycode/rel/abs table | +| CDC ACM | `drivers/usb/class/cdc-acm.c` | 2186 | Line discipline + tty interface | +| Multi-touch HID | `drivers/hid/hid-multitouch.c` | 2756 | Slot-based protocol parser | + +For all of these: when implementation detail is unclear, **port line-by-line from Linux 7.1**, preserving the data structure layouts and edge-case handling. + +## 14. See also + +- `local/docs/archived/USB-IMPLEMENTATION-PLAN-v1-2026-04.md` — v1 (P0 host work) +- `local/docs/archived/USB-IMPLEMENTATION-PLAN-v2-2026-07.md` — v2 (P0 + P5 host-only ADR) +- `local/docs/IRQ-AND-LOWLEVEL-CONTROLLERS-ENHANCEMENT-PLAN.md` — IRQ/MSI-X quality surface that P2-A depends on +- `local/docs/WIFI-IMPLEMENTATION-PLAN.md` — Wi-Fi subsystem plan; sibling first-class-citizen effort +- `local/docs/BLUETOOTH-IMPLEMENTATION-PLAN.md` — Bluetooth plan; uses `redbear-btusb` (USB Bluetooth transport) +- `local/reference/linux-7.1/drivers/usb/` — Linux 7.1 reference (the implementation of excellence) \ No newline at end of file diff --git a/local/docs/USB-VALIDATION-RUNBOOK.md b/local/docs/USB-VALIDATION-RUNBOOK.md new file mode 100644 index 0000000000..3183cd8089 --- /dev/null +++ b/local/docs/USB-VALIDATION-RUNBOOK.md @@ -0,0 +1,155 @@ +# Red Bear OS USB Validation Runbook — v3 + +> Companion to `local/docs/USB-IMPLEMENTATION-PLAN.md` v3. +> This runbook tells operators how to validate USB on a Red Bear build. + +## Validation matrix + +| Script | What it validates | Maturity target | +|---|---|---| +| `test-xhci-irq-qemu.sh --check` | xHCI interrupt-driven reactor path (line 208 of `irq_reactor.rs`) | `validated-QEMU` | +| `test-xhci-device-lifecycle-qemu.sh --check` | bounded USB attach/detach for HID + storage | `validated-QEMU` | +| `test-usb-qemu.sh` | full USB stack (xHCI + keyboard + tablet + storage) | `validated-QEMU` | +| `test-usb-storage-qemu.sh` | usbscsid autospawn + sector write+readback+restore | `validated-QEMU` | +| `test-usb-runtime.sh` | guest + QEMU mode dispatch harness | harness only | +| `test-usb-maturity-qemu.sh` | aggregate runner — calls the five above in sequence | runner | +| `test-uhci-runtime-qemu.sh --check` *(P1-B)* | UHCI controller enumeration on legacy QEMU machine | `validated-QEMU` (P1-B) | +| `test-ohci-runtime-qemu.sh --check` *(P1-B)* | OHCI controller enumeration on legacy QEMU machine | `validated-QEMU` (P1-B) | +| `test-ehci-class-autospawn-qemu.sh --check` *(P1-A)* | USB keyboard on EHCI route reaches inputd | `validated-QEMU` (P1-A) | +| `test-usb-hub-qemu.sh --check` *(P3-A)* | USB hub enumeration including power timing + wHubDelay | `validated-QEMU` (P3-A) | +| `test-usb-error-recovery-qemu.sh --check` *(P8-C)* | hot-unplug mid-transfer — graceful error, no panic | `validated-QEMU` (P8-C) | +| `test-usb-uas-qemu.sh --check` *(P4-A)* | USB 3.0 storage UAS path active | `validated-QEMU` (P4-A) | + +A row's last-passed ISO date goes in `local/docs/HARDWARE-VALIDATION-MATRIX.md`. + +## Path A — Host-side QEMU validation + +### Pre-flight + +```bash +# Confirm the build is fresh +./local/scripts/build-redbear.sh --upstream redbear-mini +ls build/x86_64/redbear-mini/harddrive.img +``` + +### Run aggregate + +```bash +./local/scripts/test-usb-maturity-qemu.sh redbear-mini +``` + +This runs all five existing QEMU tests in sequence and reports a single pass/fail. + +### Run individual + +```bash +# Interrupt-mode proof (must show "Running IRQ reactor with IRQ file +# and event queue" in the boot log; must NOT show "in polling mode"). +./local/scripts/test-xhci-irq-qemu.sh --check + +# Storage write+readback+restore (sector 2048; expects +# [PASS] STORAGE_DISCOVERY + STORAGE_WRITE + STORAGE_READBACK + +# STORAGE_RESTORE). +./local/scripts/test-usb-storage-qemu.sh + +# Lifecycle proof (QEMU monitor-driven attach/detach). +./local/scripts/test-xhci-device-lifecycle-qemu.sh --check +``` + +### What it validates vs. claims + +| Claim | Evidence | +|---|---| +| `xhcid` runs interrupt-driven | `[RUNNING] IRQ reactor with IRQ file and event queue` in log | +| `usbscsid` auto-spawns on storage attach | `usbscsid: scheme event` or `[redbear-usb-storage-check] STORAGE_DISCOVERY: disk.usb-...` | +| Storage read+write works | `[PASS] STORAGE_WRITE:` + `[PASS] STORAGE_READBACK:` + `[PASS] STORAGE_RESTORE:` | +| HID auto-spawns on keyboard attach | `usbhidd: registered producer` in log | +| No panics | absence of `panic\|crash\|abort\|RUST_BACKTRACE` in log | + +## Path B — In-guest manual validation + +Boot the image and check: + +```bash +# Inside the guest: +ls /scheme/usb* # host controllers' scheme namespaces +lsusb # walker (in redbear-hwutils package) +redbear-usb-check # pass/fail scheme validator +``` + +A healthy USB stack shows: + +- `/scheme/usb.0000:XX:XX.X_xhci/` (xhcid port directory) +- `/scheme/usb/` (ehcid port directory) +- `/scheme/disk.usb-...+...-scsi/` (usbscsid disk) +- `/scheme/input/...` (usbhidd device) + +## Path C — Bare-metal hardware validation (P8-A) + +For each (controller, class, board) tuple in `HARDWARE-VALIDATION-MATRIX.md`: + +```bash +# On the bare-metal host: +./local/scripts/build-redbear.sh +dd if=build/x86_64//harddrive.img of=/dev/ bs=4M status=progress +# Boot; capture serial console +minicom -D /dev/ttyUSB0 -b 115200 -C capture.bin + +# In the captured console: +redbear-info --verbose +lsusb +redbear-usb-check +``` + +Required test points (per row in the matrix): +1. Controller PCI ID detected +2. Specific class test (HID keypress, storage read+write, etc.) +3. Hot-plug works +4. Suspend/resume works (if device supports) + +Update the matrix row: `last_tested = `, `result = pass|fail|partial`. + +## Operator runbook for failures + +If `test-xhci-irq-qemu.sh` reports polling-mode: + +```bash +# Check the boot log for the interrupt path: +grep "IRQ reactor" build/x86_64/redbear-mini/xhci-irq-check.log +# If "in polling mode" appears, xHCI interrupts are bypassed. +# Check the base fork commit: +git -C local/sources/base log --oneline drivers/usb/xhcid/src/main.rs | head -5 +# Confirm commit cbd40e0d (or later) is in the merge base. +``` + +If `test-usb-storage-qemu.sh` reports a panic: + +```bash +# Find the panic site: +grep -n 'panic\|unwrap\|expect' build/x86_64/redbear-mini/usb-storage-check.log +# Cross-reference to usbscsid: +grep -rn 'panic!' local/sources/base/drivers/storage/usbscsid/src/ +# After P1-B is complete, no panic sites should exist. +``` + +If a USB device is not auto-spawning: + +```bash +# Check if the class driver is wired in the configs: +grep -A2 "redbear-acmd\|redbear-ecmd\|redbear-usbaudiod" config/redbear-mini.toml +# Cross-check drivers.d: +cat local/config/drivers.d/70-usb-class.toml +# After P1-A, all four controllers should auto-spawn via the unified trait. +``` + +## Validation cadence + +- **On every release branch cut:** run all QEMU tests; update matrix with last-tested dates. +- **On every P-phase completion:** add the new test scripts to the matrix. +- **Monthly:** if any operator has bare-metal access, add one hardware row. + +## See also + +- `local/docs/USB-IMPLEMENTATION-PLAN.md` v3 — the plan this runbook validates +- `local/docs/HARDWARE-VALIDATION-MATRIX.md` — the matrix this runbook populates +- `local/scripts/test-usb-maturity-qemu.sh` — the aggregate entry point \ No newline at end of file diff --git a/local/docs/archived/README.md b/local/docs/archived/README.md index 24f3589032..1bf16a6fa7 100644 --- a/local/docs/archived/README.md +++ b/local/docs/archived/README.md @@ -17,7 +17,9 @@ current plans. They are kept for reference only. | `VFAT-IMPLEMENTATION-PLAN.md` | VFAT is fully implemented (fatd, fat-mkfs, fat-label, fat-check) | | `USB-BOOT-INPUT-PLAN.md` | Superseded — USB HID in initfs, USB storage in initfs (Phase B2). Boot-input analysis remains historically useful; the live-input priority lives in `USB-IMPLEMENTATION-PLAN.md` v2. | | `XHCID-DEVICE-IMPROVEMENT-PLAN.md` | Superseded by `USB-IMPLEMENTATION-PLAN.md` v2 — phases 1–7 absorbed into the new plan's P0–P3. | -| `USB-IMPLEMENTATION-PLAN-v1-2026-04.md` | Superseded by `USB-IMPLEMENTATION-PLAN.md` v2 (current). v1 overstated xHCI interrupt-driven operation and durability; v2 rebases onto current source state and Redox 0.x USB HEAD. | +| `USB-IMPLEMENTATION-PLAN-v1-2026-04.md` | Superseded by `USB-IMPLEMENTATION-PLAN-v2-2026-07.md` (archived). v1 overstated xHCI interrupt-driven operation; v2 rebased onto current source state and Redox 0.x USB HEAD. | +| `USB-IMPLEMENTATION-PLAN-v2-2026-07.md` | Superseded by `USB-IMPLEMENTATION-PLAN.md` v3 (current). v2 captured P0–P5 host-controller work; v3 expands to full USB first-class-citizen scope including UAS, error recovery, CDC ACM, HID report parsing, and hardware matrix. | +| `USB-VALIDATION-RUNBOOK-2026-07.md` | Historical validation runbook for P0 era. v3 validates against the expanded scope. | | `ZSH-PORTING-PLAN.md` | Deferred indefinitely — ion is the default shell | ## Date archived: 2026-05-03