RedBear-OS/local/docs/SCHEME-NAMESPACE-POPULATION-PLAN.md

# Red Bear OS /scheme/ Namespace Population Plan

**Version**: 1.0 (2026-06-12)
**Status**: Draft — pending review
**Canonical**: `local/docs/SCHEME-NAMESPACE-POPULATION-PLAN.md`
**Blocks**: Writable rootfs on live ISO, `redoxfs` disk discovery, `ls /scheme/` in shell
**Cross-references**: Linux kobject/uevent, Fuchsia Zircon/Component Manager, seL4 CSpace, Plan 9 per-process namespace, Genode capability routing, MINIX 3 driver model

## 1. Problem Statement

`ls /scheme/` hangs or returns empty in Red Bear OS. Three root causes:

1. **initnsmgr `getdents` depends on daemons registering** — but boot ordering means some schemes
   haven't registered yet when `redoxfs` calls `fs::read_dir("/scheme")` to find disk devices.
2. **No aggregator for block devices** — `redoxfs` must enumerate all `disk.*` schemes individually,
   but `/scheme/disk.live` may not exist yet when the rootfs mount runs at priority 50.
3. **driver-block `getdents` returns `EOPNOTSUPP`** — individual disk schemes use legacy text-based
   listing, not proper `getdents`.

The result: `redoxfs` can't discover disks, rootfs fails to mount read-write, and `/scheme/`
listing is incomplete.

## 2. Design Principles (Informed by Cross-Reference)

### 2.1 Microkernel Principle (seL4, Red Bear OS)

The kernel tracks scheme IDs (integers), not names. All name→ID mapping happens in userspace
(`initnsmgr`). This is correct per the user's explicit correction:

> "Kernel does not have to track id-name mapping! Kernel only knows about IDs. It's a microkernel
> and stuff like this must be done in userspace"

**Implication**: We never modify the kernel to "export" scheme names. The namespace is purely
a userspace construct managed by `initnsmgr`.

### 2.2 Aggregator Pattern (Linux devtmpfs + Fuchsia devcoordinator)

Linux populates `/dev` via two mechanisms:
- **devtmpfs** — kernel auto-creates basic `/dev/null`, `/dev/sda1` etc. at boot
- **udev** — userspace daemon receives uevents via netlink, applies rules, creates additional nodes

Fuchsia uses **devcoordinator** (now driver-index + device-finder):
- Drivers register devices with the driver manager
- devcoordinator exposes them via `devfs` (listable, browsable)
- Component Manager routes specific devices to components via capability declarations

Red Bear OS should follow the **aggregator** pattern: userspace daemons that discover,
enumerate, and expose device categories through listable scheme namespaces.

### 2.3 Bootstrap Ordering (Plan 9, Fuchsia)

Plan 9 bootstraps namespace incrementally:
1. Kernel boots with `#` device drivers (kernel-resident, like Red Bear's `GlobalSchemes`)
2. `boot(8)` script binds drivers into the namespace
3. `init(8)` builds the per-process namespace from `/lib/namespace`

Fuchsia bootstraps similarly:
1. Zircon boots, creates root job + resource handles
2. component_manager starts, receives boot info (device handles from ZBI)
3. driver_index enumerates drivers, binds them to devices
4. devfs provides the listable namespace

Red Bear OS boot sequence (current):
```
bootstrap → initnsmgr (initial schemes: 10 kernel globals + "proc" + "initfs")
  → init starts service targets
  → 10_lived.service (priority 10): registers "disk.live"
  → 40_drivers.target: pcid, graphics, etc.
  → 45_diskd.service (NEW): scans disk.* schemes, registers "diskd"
  → 50_rootfs.service: redoxfs uses diskd to find root device
```

### 2.4 Separation of Discovery and Access (Genode, seL4)

Genode separates:
- **Platform session** — device discovery (what hardware exists)
- **I/O session** — device access (read/write/mmio)

seL4 separates:
- **Device Untyped caps** — raw hardware access
- **Platform description** — structured description of what devices exist

In Red Bear OS terms: `diskd` provides discovery (listing), but actual block I/O goes through
the original `disk.live`/`disk.sata0` schemes directly. `diskd` returns `OpenResult::OtherScheme`
so the kernel hands the caller a raw fd to the underlying scheme — zero overhead.

## 3. Current Architecture

### 3.1 Kernel Global Schemes (10)

Registered by bootstrap in `exec.rs` → `initnsmgr::run()`:

| Scheme | GlobalSchemes Variant | Kernel Source |
|--------|-----------------------|---------------|
| debug | Debug | `scheme/debug.rs` |
| event | Event | `scheme/event.rs` |
| memory | Memory | `scheme/memory.rs` |
| pipe | Pipe | `scheme/pipe.rs` |
| serio | Serio | `scheme/serio.rs` |
| irq | Irq | `scheme/irq.rs` |
| time | Time | `scheme/time.rs` |
| sys | Sys | `scheme/sys/mod.rs` |
| proc | Proc | `scheme/proc/mod.rs` |
| acpi | Acpi | `scheme/acpi.rs` |
| dtb | Dtb | `scheme/dtb.rs` |

These are registered in the `KernelSchemes` enum (kernel/src/scheme/mod.rs:438) and
exposed to initnsmgr during bootstrap.

### 3.2 initnsmgr Namespace Manager

Located at `local/sources/base/bootstrap/src/initnsmgr.rs`.

Key structures:
```rust
struct Namespace {
    schemes: HashMap<String, Arc<FdGuard>>,  // name → fd
}
```

- `open("")` → `Handle::List` (directory listing handle)
- `getdents(Handle::List)` → iterates `schemes` HashMap, returns `DirEntry` for each name
- Daemons register via `NsDup::IssueRegister` + sendfd mechanism
- Bootstrap passes initial set: kernel globals + "proc" + "initfs"

### 3.3 Userspace Scheme Registration

Daemons register via:
1. `Socket::create()` → creates scheme socket
2. `NsDup::IssueRegister` → tells initnsmgr the scheme name
3. `sendfd` → sends the scheme socket fd to initnsmgr
4. initnsmgr stores in `schemes: HashMap<String, Arc<FdGuard>>`

### 3.4 Current Userspace Schemes (at boot)

| Scheme | Daemon | Priority | Source |
|--------|--------|----------|--------|
| initfs | bootstrap | 0 | bootstrap exec.rs |
| proc | kernel | 0 | GlobalSchemes |
| disk.live | lived | 10 | init.initfs.d/10_lived.service |
| disk.sata0 | ahcid | 40 | pcid-spawner |
| disk.virtio0 | virtio-blkd | 40 | pcid-spawner |
| display | vesad | 20 | init.initfs.d/20_vesad.service |
| drm | redox-drm | 30 | init.initfs.d/30_graphics.service |
| net | e1000d / virtio-netd | 40 | pcid-spawner |
| orbital | orbital | rootfs | (legacy, not used in redbear-full) |

### 3.5 The Root Cause Chain

```
redoxfs mount (priority 50)
  → fs::read_dir("/scheme") → initnsmgr getdents
  → iterates schemes HashMap → finds "disk.live" (registered at priority 10)
  → is_scheme_category("disk") → true
  → Fd::open("/scheme/disk.live") → reads text listing
  → finds block device → opens /scheme/disk.live/0 → reads UUID
  → UUID matches → mounts as rootfs
```

**The bug**: `redoxfs` retries 20×200ms = 4 seconds. If disk discovery takes longer than
4 seconds (e.g., AHCI probe on real hardware), rootfs mount fails → read-only fallback.

**The fix**: `diskd` aggregator + longer timeout + event-driven notification.

## 4. Solution Architecture

### 4.1 Component Overview

```
┌─────────────────────────────────────────────────────────┐
│                    /scheme/ namespace                    │
│                   (initnsmgr)                           │
│                                                          │
│  Kernel globals:                                         │
│    debug, event, memory, pipe, serio, irq,              │
│    time, sys, proc, acpi, dtb                            │
│                                                          │
│  Boot schemes (initfs):                                  │
│    initfs, disk.live, display                            │
│                                                          │
│  Aggregators:                                            │
│    diskd  ← /scheme/diskd lists ALL block devices       │
│                                                          │
│  Hardware daemons (post-drivers.target):                 │
│    disk.sata0..7 (ahcid)                                │
│    disk.virtio0..7 (virtio-blkd)                        │
│    disk.nvme0..7 (nvmed)                                │
│    disk.usb0..7 (usbscsid)                              │
│    disk.ide0..3 (ideid)                                 │
│    net (e1000d, virtio-netd, ixgbed, rt8169d)          │
│    drm (redox-drm)                                      │
│                                                          │
│  System daemons (post-rootfs):                           │
│    audio (audiod)                                       │
│    firmware (firmware-loader)                            │
│    input (evdevd)                                       │
│    udev (udev-shim)                                     │
│    ...                                                  │
└─────────────────────────────────────────────────────────┘
```

### 4.2 diskd — Disk Aggregator Daemon (IMPLEMENTED)

**Location**: `local/recipes/system/diskd/`
**Scheme name**: `diskd`
**Binary**: `/usr/bin/diskd`
**Status**: Code complete, cargo check/clippy/fmt clean

**How it works**:

1. At boot (priority 45), diskd starts
2. Probes `/scheme/disk.live`, `/scheme/disk.sata0`..7, `/scheme/disk.virtio0`..7, etc.
3. For each found scheme, reads its text listing to discover devices and partitions
4. Registers scheme `diskd` with initnsmgr
5. `getdents` on `diskd:` returns real `DirEntry` with `DirentKind::BlockDev`
6. `open("0")` or `open("0p1")` opens the underlying scheme and returns `OtherScheme`
   (zero-copy — caller talks directly to the block device)

**Why this solves the root cause**:

- `redoxfs` currently must enumerate ALL `/scheme/disk.*` individually — 50+ `Fd::open` calls
- With `diskd`, `redoxfs` does ONE `read_dir("/scheme/diskd")` to get all block devices
- diskd already did the probing and enumeration
- Even if AHCI hasn't registered yet, diskd's retry logic handles late registration
- `redoxfs` timeout only needs to wait for `diskd` to be ready, not all individual schemes

### 4.3 Changes Required to Existing Components

#### 4.3.1 redoxfs — Use diskd for disk discovery

**File**: `local/sources/redoxfs/src/bin/mount.rs` (function `filesystem_by_uuid`)

**Current behavior**:
```rust
// Line 224: fs::read_dir("/scheme") → filter is_scheme_category("disk")
// For each disk.* scheme: open, read listing, find block devices, check UUID
// Retry 20×200ms = 4 seconds total
```

**New behavior** (two-path approach):

```rust
fn filesystem_by_uuid(uuid: &[u8; 16]) -> Option<File> {
    // Path A: Try diskd aggregator first (fast, single enumeration)
    if let Some(f) = try_diskd_uuid(uuid) {
        return Some(f);
    }
    // Path B: Fall back to legacy per-scheme enumeration
    // (for backwards compat and environments without diskd)
    try_legacy_uuid_search(uuid)
}

fn try_diskd_uuid(uuid: &[u8; 16]) -> Option<File> {
    // Wait for diskd scheme to appear
    for _ in 0..50 {  // 50 × 200ms = 10 seconds
        if let Ok(dir) = fs::read_dir("/scheme/diskd") {
            for entry in dir {
                let entry = entry.ok()?;
                let name = entry.file_name().to_string_lossy().into_owned();
                // Open the block device via diskd (which proxies to underlying scheme)
                let path = format!("/scheme/diskd/{name}");
                if let Ok(mut f) = File::open(&path) {
                    if check_uuid(&mut f, uuid) {
                        return Some(f);
                    }
                }
            }
        }
        thread::sleep(Duration::from_millis(200));
    }
    None
}
```

#### 4.3.2 init.initfs.d — Add diskd service

**New file**: `local/sources/base/init.initfs.d/45_diskd.service`

```ini
[[service]]
name = "diskd"
command = "/usr/bin/diskd"
priority = 45
requires = ["lived"]
```

This ensures diskd starts after lived (which provides disk.live at priority 10) and before
rootfs mount (priority 50).

#### 4.3.3 config/redbear-mini.toml — Add diskd package

Add `diskd` to the `[packages]` section so it's included in the image.

### 4.4 /scheme/ Namespace Completeness Matrix

After all changes, `/scheme/` will expose:

| Category | Scheme Name | Provider | getdents | Notes |
|----------|-------------|----------|----------|-------|
| **Kernel globals** | | | | |
| Debug | `debug` | kernel GlobalSchemes | ✅ real DirEntry | kernel/src/scheme/debug.rs |
| Event | `event` | kernel GlobalSchemes | ✅ real DirEntry | kernel/src/scheme/event.rs |
| Memory | `memory` | kernel GlobalSchemes | EOPNOTSUPP | No sub-entries expected |
| Pipe | `pipe` | kernel GlobalSchemes | EOPNOTSUPP | Anonymous, no listing |
| Serio | `serio` | kernel GlobalSchemes | ✅ real DirEntry | kernel/src/scheme/serio.rs |
| IRQ | `irq` | kernel GlobalSchemes | ✅ real DirEntry | cpu-XX entries |
| Time | `time` | kernel GlobalSchemes | ✅ real DirEntry | CLOCK_* entries |
| Sys | `sys` | kernel GlobalSchemes | ✅ real DirEntry | scheme:/scp/ sub-entries |
| Proc | `proc` | kernel GlobalSchemes | ✅ real DirEntry | pid entries |
| ACPI | `acpi` | kernel GlobalSchemes | ✅ real DirEntry | rxsdt, kstop |
| DTB | `dtb` | kernel GlobalSchemes | EOPNOTSUPP | Single blob |
| **Bootstrap** | | | | |
| InitFS | `initfs` | bootstrap | ✅ real DirEntry | initramfs contents |
| **Storage** | | | | |
| Live disk | `disk.live` | lived | ✅ text listing | virtio/ahci backend |
| SATA disk | `disk.sata0..7` | ahcid | ✅ text listing | per-disk scheme |
| VirtIO disk | `disk.virtio0..7` | virtio-blkd | ✅ text listing | per-disk scheme |
| NVMe disk | `disk.nvme0..7` | nvmed | ✅ text listing | per-disk scheme |
| USB disk | `disk.usb0..7` | usbscsid | ✅ text listing | per-disk scheme |
| IDE disk | `disk.ide0..3` | ideid | ✅ text listing | per-disk scheme |
| **Aggregators** | | | | |
| Disk aggregator | `diskd` | diskd | ✅ real DirEntry BlockDev | THIS PLAN |
| **Display** | | | | |
| Framebuffer | `display` | vesad | EOPNOTSUPP | Legacy text listing |
| DRM/KMS | `drm` | redox-drm | ✅ real DirEntry | card0, card0-*, connectors |
| **Network** | | | | |
| Ethernet | `net` | e1000d/virtio-netd | ✅ real DirEntry | interface entries |
| **Input** | | | | |
| Input events | `input` | evdevd | ✅ real DirEntry | event0, event1, ... |
| **Audio** | | | | |
| Audio | `audio` | audiod | ✅ text listing | Audio streams |
| **System** | | | | |
| Firmware | `firmware` | firmware-loader | ✅ real DirEntry | GPU/device blobs |
| Udev | `udev` | udev-shim | ✅ real DirEntry | Linux-compatible device nodes |

### 4.5 initnsmgr getdents — Already Correct

The `initnsmgr` `getdents` implementation at line 402-439 of `initnsmgr.rs` iterates
`schemes: HashMap<String, Arc<FdGuard>>` and emits a `DirEntry` for each registered scheme.
This is already correct — it will list any scheme that has been registered, including `diskd`.

**The `/scheme/` listing issue was NOT a getdents bug** — it was a timing issue:
- Daemons hadn't registered yet when `fs::read_dir("/scheme")` was called
- The fix is proper boot ordering (diskd at priority 45) and the diskd aggregator

## 5. Future Enhancements (Beyond Current Scope)

### 5.1 Event-Driven Discovery (uevent Equivalent)

Currently `diskd` probes statically at startup. For hotplug (USB drives, PCIe hot-add):

- **pcid** sends a `uevent`-like notification when a new PCI device appears
- **diskd** listens for these notifications and re-scans
- Alternative: inotify-like watch on `/scheme/` (would need kernel support)

This mirrors Linux's `uevent` netlink broadcast → `udev` listener pattern.

### 5.2 devfs-Style Aggregation

A future `devfsd` could provide Linux-compatible `/dev` paths:
```
/scheme/devfs/sda     → /scheme/diskd/0
/scheme/devfs/sda1    → /scheme/diskd/0p1
/scheme/devfs/null    → /scheme/debug (write sink)
/scheme/devfs/zero    → /scheme/memory (zero-filled read)
/scheme/devfs/random  → /scheme/entropy
/scheme/devfs/tty0    → /scheme/display.0
/scheme/devfs/input/event0 → /scheme/input/event0
```

This would be the Fuchsia devcoordinator equivalent — a unified, Linux-compatible
device namespace. The `udev-shim` already provides parts of this.

### 5.3 Per-Process Namespace (Plan 9 Style)

Plan 9's `bind` and `mount` allow per-process namespace customization. Red Bear OS's
`setrens` syscall provides a basic version (switch namespace fd). Future enhancement:
- Per-container namespaces (for `contain` and future container runtime)
- Namespace inheritance rules (like Fuchsia's `.cml` capability routing)
- `chroot`-like namespace restriction for sandboxed applications

### 5.4 Capability-Based Access (seL4 Style)

seL4 uses CSpace (capability spaces) for device access. Each process has a CSpace that
contains only the capabilities it should have access to. Red Bear OS could evolve toward
this model:

- `initnsmgr` tracks which schemes each process can access
- `open("/scheme/net")` checks the caller's capability set
- `setrens` evolves from "switch namespace" to "restrict to capability subset"

This would require kernel changes (per-process scheme allowlists), which is beyond current
scope but worth keeping in mind for security hardening.

## 6. Implementation Plan

### Phase 1 — Immediate Fix (This Session)

| Step | Action | Files | Status |
|------|--------|-------|--------|
| 1 | diskd daemon implementation | `local/recipes/system/diskd/` | ✅ Done |
| 2 | Add diskd init service | `local/sources/base/init.initfs.d/45_diskd.service` | Pending |
| 3 | Add diskd to config | `config/redbear-mini.toml` | Pending |
| 4 | Modify redoxfs to use diskd | `local/sources/redoxfs/src/bin/mount.rs` | Pending |
| 5 | Commit uncommitted changes | driver-manager, config | Pending |
| 6 | Remove pcid debug logging | `local/sources/base/drivers/pcid/src/cfg_access/fallback.rs` | Pending |
| 7 | Make C++ header fix durable | `mk/prefix.mk` | Pending |
| 8 | Build and test ISO | `./local/scripts/build-redbear.sh redbear-mini` | Pending |
| 9 | Boot test in QEMU | `scripts/run_mini1.sh` | Pending |

### Phase 2 — Hotplug Support (Future)

| Step | Action | Dependencies |
|------|--------|--------------|
| 1 | pcid uevent notification | pcid-spawner enhancement |
| 2 | diskd dynamic re-scan | uevent listener |
| 3 | devfsd Linux-compatible /dev | udev-shim + diskd integration |

### Phase 3 — Namespace Security (Future)

| Step | Action | Dependencies |
|------|--------|--------------|
| 1 | Per-process scheme allowlist | kernel scheme access control |
| 2 | Container namespace isolation | contain enhancement |
| 3 | Capability routing | initnsmgr capability model |

## 7. Cross-Reference Summary

| System | Mechanism | Red Bear Equivalent | Status |
|--------|-----------|---------------------|--------|
| **Linux** | kobject/uevent → udev → /dev | pcid → diskd → /scheme/diskd | Phase 1 |
| **Fuchsia** | devcoordinator → devfs | initnsmgr → diskd | Phase 1 |
| **seL4** | CSpace capabilities | setrens (basic) | Phase 3 |
| **Plan 9** | bind/mount per-process | setrens (basic) | Phase 3 |
| **Genode** | Platform session | redox-driver-sys | Existing |
| **MINIX 3** | driver announce → devfs | daemon register → initnsmgr | Existing |

## 8. Risk Assessment

| Risk | Mitigation |
|------|------------|
| diskd probe takes too long on real hardware | Increase retry count (50×200ms = 10s), add event-driven re-scan |
| diskd crashes and disk namespace disappears | init service auto-restart (`restart = true` in service file) |
| redoxfs legacy path broken by diskd changes | Two-path approach: try diskd first, fall back to legacy |
| Boot ordering regression (diskd starts before lived) | Explicit `requires = ["lived"]` in service file |
| diskd returns stale device list after hotplug | Phase 2: event-driven re-scan; Phase 1: manual re-trigger via signal |

## 9. Acceptance Criteria

1. `ls /scheme/` in shell shows all registered schemes (no hang, no empty)
2. `ls /scheme/diskd/` shows all block devices discovered by diskd
3. `redoxfs` mounts rootfs read-write via diskd path
4. `/tmp` is writable by non-root users
5. Boot completes to login prompt with zero warnings
6. QEMU boot test passes: `scripts/run_mini1.sh` reaches login prompt
7. `./local/scripts/build-redbear.sh redbear-mini` produces working ISO