RedBear-OS/local/docs/WAYLAND-IMPLEMENTATION-PLAN.md

# Red Bear OS Wayland Implementation Plan
**Version:** 2.1 (2026-05-06)
**Status:** Canonical Wayland subsystem plan — **Wayland-only path, no framebuffer workarounds**

## Architecture Decision (2026-05-06)

**Wayland is the only supported display protocol for the desktop path.**

No framebuffer fallbacks. No `QT_QPA_PLATFORM=offscreen`. No `libqredox.so` Wayland shim
pretending to be a native platform. The Qt6 Wayland crash (page fault at null+8 during
`wl_proxy_add_listener`) is a bug that must be fixed at the source — in Qt6's auto-generated
Wayland wrappers or in the relibc/libwayland client stack.

**Reasoning:**
- Qt6, KF6, and KWin are Wayland-native components. Making them work through a framebuffer
  abstraction adds complexity without solving the real problem.
- The `redbear-compositor` is protocol-compliant and serves 8 Wayland globals. It does not
  crash. The crash is entirely client-side (Qt6 QPA plugin initialization).
- Every workaround (offscreen QPA, redox QPA shim, D-Bus environment overrides) defers the
  inevitable: Qt6 Wayland must work on Redox.

**Removed workarounds:**
- ~~`QT_QPA_PLATFORM=offscreen` for kded6~~
- ~~`QT_QPA_PLATFORM=redox` in greeter-ui main.cpp~~
- kded6 must use Wayland. greeter-ui must use Wayland.

## Current Blocker: Qt6 Wayland Client Crash

### Crash Signature

```
Page fault: 0000000000000008 US
RAX: 0x0000000000000000   (NULL base pointer)
RDX: 0x0000000000000008   (value being written)
```

This is `proxy->object.implementation = listener` in `wl_proxy_add_listener()`
where `proxy` is NULL. Offset 8 from NULL = `object.implementation` field
in `wl_object` struct, which is at offset 0 of `wl_proxy`.

### Ruled Out

| Hypothesis | Verdict | Evidence |
|-----------|---------|----------|
| relibc `calloc` doesn't zero | **FALSE** | `header/stdlib/mod.rs:276` — `ptr.write_bytes(0, size)` runs unconditionally |
| Compositor protocol bug | **FALSE** | Compositor doesn't crash; client crashes before any compositor events exchanged |
| `wl_display_connect()` returns NULL | **FALSE** | Qt checks: `if (mDisplay) setupConnection()` |
| `wl_display_get_registry()` returns NULL | **FALSE** | My null guard `if (!registry) _exit(1)` catches this |
| libwayland `proxy_create()` returns NULL | **UNLIKELY** | Only fails on OOM or `wl_map_insert_new` failure; Qt would catch NULL |

### Most Likely Cause

The crash is in `QWaylandDisplay::init(registry)` → `wl_registry_add_listener()`.
The `registry` pointer is valid (caught by null guard), but the **Qt6 object wrapping it**
may have internal NULL members. Specifically, the auto-generated `QtWayland::wl_registry`
wrapper stores the C `wl_registry*` in `m_wl_registry`. If `this` (the QWaylandDisplay
object) is partially constructed when `init_listener()` runs, `m_wl_registry` could be
uninitialized.

Alternatively: the crash may be in a **subsequent** proxy creation — Qt6 binds `wl_compositor`,
`wl_shm`, or other globals during initialization, and one of those proxy creations returns
NULL that Qt doesn't check.

## Implementation Plan

### Phase A: Diagnose Exact Crash Point (1-2 hours)

1. Add `fprintf(stderr, "registry=%p this=%p\n", registry, this)` before `init(registry)`
   in `qwaylanddisplay.cpp:setupConnection()`
2. Add `fprintf(stderr, "m_wl_registry=%p\n", m_wl_registry)` at entry of `init_listener()`
   in generated `qwayland-wayland.cpp`
3. Add null guards at every `wl_registry_add_listener` and `wl_proxy_add_listener` call site
4. Rebuild, boot, observe which fprintf appears before the crash

### Phase B: Add Null Safety to Qt6 Generated Wrappers (2-4 hours)

The `qtwaylandscanner` tool generates wrapper code from Wayland XML protocol definitions.
The generated `init()` and `init_listener()` functions assume non-null proxies.

1. Patch `qtwaylandscanner` to emit null-checks before each `wl_*_add_listener()` call
2. Or: patch the generated output directly in the build tree
3. Add `if (!m_wl_registry) { qWarning(...); return; }` guards

### Phase C: Audit libwayland Client Initialization (2-4 hours)

1. Add logging to `proxy_create()` — log every proxy creation with pointer value
2. Add logging to `wl_proxy_add_listener()` — log proxy pointer
3. Verify `factory->display` is never NULL at `proxy_create:483`
4. Check if `pthread_mutex_lock`/`pthread_mutex_unlock` works correctly on Redox
   (mutex corruption could cause `wl_map_insert_new` to return 0)

### Phase D: Fix Root Cause (4-8 hours)

Based on Phase A-B-C findings, implement the actual fix:
- If null proxy: fix the creation path
- If null `factory->display`: fix the display initialization
- If mutex issue: fix relibc pthread
- If Qt6 wrapper bug: upstream the null-safety patch

### Phase E: Remove Workarounds (1 hour)

Once Qt6 Wayland works:
1. Remove `QT_QPA_PLATFORM=redox` from `greeter-ui/main.cpp`
2. Remove `Environment=QT_QPA_PLATFORM=offscreen` from D-Bus kded6 service
3. Remove `QT_QPA_PLATFORM=offscreen` from `redbear-kde-session`
4. Let kded6 and greeter-ui use native Wayland

## Compositor Status

The `redbear-compositor` is **protocol-compliant** and **runtime-proven** (QEMU).

### Globals Served (8)

| Global | Version | Status |
|--------|---------|--------|
| `wl_compositor` | 4 | ✅ Implemented |
| `wl_shm` | 1 | ✅ Implemented (ARGB8888, XRGB8888) |
| `wl_shell` | 1 | ✅ Implemented |
| `wl_seat` | 5 | ✅ Name + pointer capability |
| `wl_output` | 3 | ✅ Geometry, mode, scale, done |
| `xdg_wm_base` | 1 | ✅ Get/surface/toplevel + configure |
| `wl_data_device_manager` | 3 | 🔧 Stub (accepts bind, no events) |
| `wl_subcompositor` | 1 | 🔧 Stub (accepts bind, no events) |

### Rendering

Framebuffer compositing via `/scheme/display` (vesad). Software only.
Hardware DRM/KMS rendering depends on `redox-drm` driver maturity.

## Evidence Model

| Class | Meaning |
|-------|---------|
| **builds** | Package compiles and stages |
| **boots** | Image reaches prompt or known runtime surface |
| **runtime-proven (QEMU)** | Works end-to-end in QEMU |
| **runtime-proven (bare metal)** | Works on real hardware |
| **validated** | Repeated proof on intended target class |

### Current Status

| Component | Class | Notes |
|-----------|-------|-------|
| `redbear-compositor` | **runtime-proven (QEMU)** | Accepts connections, composites framebuffer |
| `libwayland` client | **builds** | Upstream code; Redox eventfd patch for server only |
| `libwayland` server | **builds** | Not used — compositor is pure Rust |
| Qt6 Wayland QPA | **build-verified; runtime gated** | Crashes at null+8 during `wl_registry` init |
| KWin Wayland | **builds** | Real cmake build; blocked by Qt6Quick/QML |
| Wayland protocols | **builds** | `wayland-protocols` package builds |

## Next Milestone

**Qt6 Wayland client boots without crashing.**

Success criterion: `redbear-compositor: client 65536 connected` followed by
`redbear-compositor: dispatch` (not a page fault).

### Build Dependency Chain (Critical, discovered 2026-05-06)

When modifying libwayland source or `local/patches/libwayland/redox.patch`, the
full chain must be rebuilt:

```
libwayland recipe → repo/x86_64-unknown-redox/libwayland.pkgar
qtbase recipe     → copies pkgar into sysroot → links Qt6 Wayland QPA
kf6-kded6 recipe  → links against qtbase/sysroot/libwayland-client.a
```

**Failure mode**: Modifying only libwayland does NOT update qtbase's static copy
of `libwayland-client.a`. The old binary persists in qtbase's sysroot. All
three recipes must be force-rebuilt (delete `target/` directories) AND the
repo pkgar cache must be regenerated.

**Recovery**: If `repo/x86_64-unknown-redox/libwayland.pkgar` is deleted, a
backup exists at `packages/x86_64-unknown-redox/libwayland.pkgar`.