RedBear-OS/local/docs/RELIBC-IPC-ASSESSMENT-AND-IMPROVEMENT-PLAN.md

# Red Bear OS relibc IPC Assessment and Improvement Plan

## Purpose

This document assesses the current **IPC-related relibc surface** in Red Bear OS and turns that
assessment into a concrete improvement plan.

The focus here is narrower than the general relibc plan:

- POSIX shared memory and semaphores
- System V shared memory and semaphores
- missing System V / POSIX IPC areas such as message queues
- IPC-adjacent descriptor/event primitives that downstream software treats as part of the same
  coordination substrate: `eventfd`, `signalfd`, and `timerfd`
- the downstream subsystem pressure created by Qt, KDE, Wayland, and related userland

This is not a generic libc-compliance document. It is grounded in the current repository state.

## Evidence Model

This assessment distinguishes four evidence levels:

- **source-visible** — behavior exists in relibc source now
- **test-visible** — behavior is exercised by focused relibc tests
- **build-visible downstream** — real consumers compile/link against it
- **runtime-validated** — behavior has been exercised in real Redox or consumer runtime paths

The key IPC problem in the current tree is not simple absence. It is the gap between
**source-visible**, **bounded**, **build-proven**, and **runtime-trusted**.

## Upstream vs Red Bear separation

For this IPC work, keep the storage model explicit:

- the live implementation under `recipes/core/relibc/source/src/header/` is the working upstream
  tree used for builds and tests
- the durable Red Bear ownership boundary is `local/patches/relibc/` plus `local/docs/`

So the IPC implementation is only truly safe when:

1. the upstream-owned relibc source tree builds with the change now, and
2. the same delta is preserved in `local/patches/relibc/` so a fresh upstream refetch can recover it

This repo should be able to pull renewed upstream sources every day and still rebuild after
reapplying the local relibc patch carriers. That requirement is part of the IPC improvement plan,
not an afterthought.

The same section also implies an upstream-preference policy:

- when upstream relibc already provides the same IPC fix, prefer upstream
- keep Red Bear IPC patches only for gaps that upstream still does not solve adequately
- review patch carriers regularly and delete or shrink ones made obsolete by upstream evolution

## Current Implementation Note

This repo pass did not just assess the IPC surface; it also restored the missing relibc IPC modules
that the drafted Red Bear docs were already assuming existed in-tree.

The current tree now contains source-visible implementations for:

- `sys/eventfd.h` / `eventfd()` / `eventfd_read()` / `eventfd_write()`
- `sys/timerfd.h` / `timerfd_create()` / `timerfd_settime()` / `timerfd_gettime()`
- `sys/signalfd.h` / `signalfd()` / `signalfd4()`
- `open_memstream()`
- bounded `sys/ipc.h`, `sys/shm.h`, and `sys/sem.h` compatibility layers
- a bounded `waitid()` path sufficient to satisfy current Qt process-side linking

This pass also added focused relibc tests for:

- `stdio/open_memstream`
- `sys_sem/semget`
- `sys_timerfd/timerfd`
- `sys_signalfd/signalfd`

Current manual verification in this repo pass:

- `cargo check --target x86_64-unknown-linux-gnu` passes for relibc
- host-side focused IPC tests execute successfully for `open_memstream` and `semget`
- targeted Redox runtime execution now validates the `timerfd` and `signalfd` tests directly through the repaired `write-exec` path instead of relying on bounded host-side fallback behavior
- `CI=1 ./target/release/repo cook relibc` completes successfully after clearing a stale stage-dir collision
- `CI=1 ./target/release/repo cook qtbase` now succeeds after exporting `eventfd_t` and restoring a bounded `waitid()` path
- a fresh `repo unfetch relibc` → `repo fetch relibc` cycle plus reapplication of
  `local/patches/relibc/` again supports successful downstream `libwayland` and `qtbase` builds,
  which is the current proof that the relibc IPC overlay is recoverable from refreshed upstream
  source, not only from the previously edited working tree

In other words, the current relibc IPC work is no longer just “working in the checked-out source
tree”. It is now proven as an overlay workflow:

1. refresh upstream relibc source
2. reapply the local relibc compatibility overlays
3. rebuild relibc
4. rebuild real downstream consumers (`libwayland`, `qtbase`)

For the current tree, that overlay story now includes the tracked
`local/patches/relibc/redox.patch` carrier owning the bounded interface-enumeration and
resolver-header compatibility deltas (`ifaddrs` / `net_if`, `arpa/nameser.h`, `resolv.h`) rather
than leaving those as standalone transient patch files.

## Scope Map

### In scope in relibc today

| Area | State | Primary evidence |
|---|---|---|
| `shm_open()` / `shm_unlink()` | implemented | `recipes/core/relibc/source/src/header/sys_mman/mod.rs` |
| POSIX unnamed semaphores | implemented | `recipes/core/relibc/source/src/header/semaphore/mod.rs` |
| POSIX named semaphores | implemented but bounded | `recipes/core/relibc/source/src/header/semaphore/mod.rs` |
| SysV shared memory | implemented but bounded | `recipes/core/relibc/source/src/header/sys_shm/mod.rs` |
| SysV semaphores | implemented but bounded | `recipes/core/relibc/source/src/header/sys_sem/mod.rs` |
| `eventfd` | implemented; stronger than the other descriptor-event APIs | `recipes/core/relibc/source/src/header/sys_eventfd/mod.rs` |
| `signalfd` | implemented, but runtime-thin and not broadly Redox-runtime-trusted yet | `recipes/core/relibc/source/src/header/signal/signalfd.rs` |
| `timerfd` | implemented, but semantically narrow and not broadly Redox-runtime-trusted yet | `recipes/core/relibc/source/src/header/sys_timerfd/mod.rs` |

### Explicitly incomplete or absent

| Area | Current state | Evidence |
|---|---|---|
| POSIX message queues | absent | `recipes/core/relibc/source/src/header/mod.rs` still has `TODO: mqueue.h` |
| SysV message queues | absent | `recipes/core/relibc/source/src/header/mod.rs` still has `TODO: sys/msg.h` |
| `threads.h` / other broader libc completeness | outside this IPC focus, still incomplete | `recipes/core/relibc/source/src/header/mod.rs` |

## Current Implementation Assessment

### 1. Strong spots

The strongest IPC-related point is that relibc is no longer missing its core coordination substrate.
The current tree has real, source-visible implementations for POSIX shm, POSIX semaphores, SysV
shared memory, SysV semaphores, `eventfd`, `signalfd`, and `timerfd`. This is already enough to
move several downstreams from patch-side workarounds to actual libc usage.

`shm_open()` and `shm_unlink()` are cleanly tied to the Redox-native `/scheme/shm/` path in
`sys_mman/mod.rs`. That is a good architectural fit: Red Bear is not pretending to have a Linux
kernel IPC model under the hood, but it still exposes familiar libc entry points on top of Redox
schemes.

The second strong point is that the IPC work is not just source-visible anymore. The focused relibc
tests already cover `sem_open`, `shmget`, `open_memstream`, `semget`, `eventfd`, and the targeted
Redox-runtime `timerfd` / `signalfd` cases. The broader relibc plan also records successful
downstream builds for `libwayland`, `qtbase`, and `openssh`, which means real consumers are already
benefiting from this work, but those consumers do **not** all prove IPC depth equally.

### 2. Weak spots

The biggest weakness is **boundedness masquerading as compatibility**. The SysV layers exist, but
they are deliberately thin wrappers over `/scheme/shm/` and relibc-local bookkeeping, not a broad
Unix-complete implementation.

In `sys_shm/mod.rs`, `shmat()` rejects non-null attach addresses with `ENOSYS`, `SHM_RND` is
defined but not meaningfully implemented, and `shmctl()` only meaningfully supports `IPC_RMID` and
`IPC_STAT`. This is good enough for simple `IPC_PRIVATE` workflows and current compile-time
consumers, but it is not strong enough to claim general SysV shared-memory completeness.

In `sys_sem/mod.rs`, `semget()` rejects any `nsems != 1`, so the implementation is effectively a
single-semaphore set model rather than a full semaphore-set model. `semop()` supports multiple
operations in one call, but only for semaphore number 0, and there is no `semtimedop()` support.
`SEM_UNDO` is defined but not actually implemented. Compared with the standard `semop(2)` model,
this means the current layer matches only the narrowest downstream cases.

Named POSIX semaphores are also present but still bounded. `sem_open()` is implemented on top of
`shm_open()`, which is a practical Redox-native strategy, but the current code comments already mark
it as a bounded Redox path rather than a full Linux/glibc-equivalent semantic model.

The descriptor-event primitives are in a materially better state than before. `eventfd` now has a
real counter-style runtime path instead of only a source-visible wrapper, and the targeted Redox
runtime test harness now executes strict `eventfd`, `signalfd`, and `timerfd` test binaries
successfully through the repaired `write-exec` runner path. The older "unavailable is success"
fallbacks were removed from those focused tests, so these are now actual runtime checks rather than
mere launch proofs.

The preserved overlay story for those paths is now simpler than it was during the original bounded
bring-up. The current relibc tree already contains the fd-event implementations and focused tests
upstream, so the active Red Bear recipe replay no longer needs the old standalone
`P3-eventfd.patch`, `P3-signalfd.patch`, `P3-signalfd-header.patch`, `P3-timerfd.patch`, and
`P3-fd-event-tests.patch` carriers. In the current repo, `redox.patch` remains the active shared
Red Bear relibc delta, while the historical P3 files are legacy references rather than recipe inputs.

The remaining caution is semantic breadth, not whether the paths execute at all. `timerfd` is now
runtime-validated for the bounded relibc test harness, but downstream consumers such as KWin still
pressure Linux-oriented details like `TFD_TIMER_CANCEL_ON_SET`, so broad desktop/runtime trust
should still be described as narrower than full Linux equivalence.

### 3. Missing areas

The obvious missing IPC area is message queues. Both `mqueue.h` and `sys/msg.h` remain TODOs in the
header tree, which means relibc currently has no story at all for POSIX message queues or SysV
message queues. That is not necessarily today’s highest-value blocker, but it is still a real IPC
gap and should be named directly instead of being buried under generic TODO volume.

## Downstream Subsystem Assessment

### Qt / KDE

Qt and KDE are the clearest subsystem forcing IPC depth rather than just IPC surface area.

`local/docs/QT6-PORT-STATUS.md` already treats `QSharedMemory`, `QSystemSemaphore`, and `QProcess`
as moved from “missing libc surface” to “present, but still needs runtime validation”. That is the
right framing. The libc surface is no longer the primary blocker; confidence and semantics are.

The strongest concrete consumers in-tree are:

- `local/recipes/kde/kf6-kservice/source/src/sycoca/kmemfile.cpp` — heavy `QSharedMemory` usage
- `local/recipes/kde/kf6-solid/source/src/solid/devices/backends/udisks2/udisksopticaldisc.cpp` —
  `QSharedMemory` plus `QSystemSemaphore`
- `local/recipes/kde/kf6-kio/source/src/gui/previewjob.cpp` — direct SysV `shmget` / `shmat`
- `local/recipes/kde/kwin/source/src/utils/xcbutils.cpp` — direct `shmget`
- `local/recipes/kde/kwin/source/src/core/syncobjtimeline.cpp` and kio scoped-process code —
  `eventfd`
- `local/recipes/kde/kwin/source/src/plugins/nightlight/clockskewnotifierengine_linux.cpp` —
  `timerfd` with `TFD_TIMER_CANCEL_ON_SET`

This matters because it shows two different downstream classes:

1. **Qt abstractions** (`QSharedMemory`, `QSystemSemaphore`) that can tolerate bounded underlying
   libc behavior if their common paths work.
2. **Direct Unix/Linux-style callers** (KIO/KWin) that expose the places where the current relibc
   SysV and timerfd layers are still semantically narrower than software expects.

### Wayland stack

Wayland is less about classic shared-memory IPC completeness now and more about the descriptor-event
side of the same subsystem family. The repo’s existing docs correctly show that `signalfd`,
`timerfd`, `eventfd`, and `open_memstream` were the historical blockers and are now source-visible.
`libwayland` cooking successfully is strong build-side proof, but the remaining work is runtime
behavior under a compositor/session stack.

### Secondary consumers: OpenSSH / GLib / tmux

These are weaker IPC drivers and stronger networking/resolver drivers. They still matter because they
show a pattern: once relibc exports the needed surface, downstream recipes can drop fake fallbacks,
but runtime validation still trails source visibility. For an IPC-focused roadmap, they are useful
secondary evidence, not primary IPC blockers.

The downstream proof should therefore be read this way:

- `qtbase` is the strongest IPC-facing downstream because it directly pressures shared memory,
  semaphores, and process behavior.
- KDE consumers on top of Qt are the strongest subsystem evidence for where IPC semantics still need
  runtime trust.
- `libwayland` is strongest as descriptor-event proof (`signalfd`, `timerfd`, `eventfd`,
  `open_memstream`) rather than SysV IPC proof.
- `openssh`, `glib`, and `tmux` are useful proof that relibc header/export cleanup is helping real
  ports, but they should not be over-counted as core IPC validation.

## Main Blockers

### Blocker 1 — SysV layers are intentionally narrower than their API surface suggests

This is the highest-value blocker because it affects both direct consumers and Qt/KDE confidence.

Current examples:

- `semget()` only supports one semaphore per set
- `semop()` only supports semaphore number 0
- `SEM_UNDO` is not implemented
- `semtimedop()` is absent
- `shmat()` does not support non-null attach addresses
- `shmctl()` does not cover the broader control matrix
- SysV message queues are absent entirely

None of these invalidate the current build work. But together they mean “API present” is still not
the same as “subsystem-complete”.

### Blocker 2 — Runtime validation is still shallower than subsystem importance

The IPC surface is better-tested than before, but runtime validation still trails the subsystem’s
importance.

Current test story:

- host-side focused execution exists for `sem_open`, `shmget`, `open_memstream`, `semget`, and
  `eventfd`
- targeted Redox runtime execution now exists for `signalfd`, `timerfd`, and `eventfd` via
  `relibc-tests-bins` and the repaired `cookbook_redoxer write-exec` path, with strict pass/fail
  semantics rather than availability fallbacks
- downstream build evidence exists for `libwayland`, `qtbase`, and `openssh`

What is still missing is stronger Redox-target or consumer-runtime proof for Qt/KDE and Wayland
paths that actually exercise shared memory, semaphores, and timer/signal descriptor behavior in a
live session.

The strongest safe claim today is therefore:

- **source-visible** across the major IPC surfaces,
- **test-visible** for focused host-side and Redox-target fd-event cases,
- **build-visible downstream** for meaningful consumers,
- with **bounded runtime trust on Redox for the relibc fd-event harness**,
- but **not yet broad proof of full Linux-equivalent semantics for every desktop consumer path**.

### Blocker 3 — Descriptor-event semantics are still narrower than Linux-oriented callers expect

KWin’s timer code wants `TFD_TIMER_CANCEL_ON_SET`. The current bounded relibc timerfd layer does
not claim that full Linux cancel-on-clock-change semantic. The preserved test/runtime slice proves
one-shot behavior and successful `TFD_TIMER_ABSTIME` / bounded flag-surface handling, while broader
Linux-equivalent cancel-on-clock-change semantics remain an explicit downstream expectation gap.

Likewise, `signalfd` support is no longer merely visible/exported; it now passes the targeted
Redox-runtime relibc test path. The remaining question is broader consumer semantics and long-tail
desktop/runtime confidence, not basic availability.

### Blocker 4 — Message queues remain a completely open IPC front

`mqueue.h` and `sys/msg.h` are still absent. This is not the first blocker to fix for today’s
desktop stack, but it is the clearest “IPC truly not implemented yet” gap left in relibc.

## Current Non-Goals / Not Yet Claimed

The current tree should **not** be described as claiming any of the following:

- full SysV semaphore-set semantics
- full SysV shared-memory semantics
- full Linux-equivalent `timerfd` semantics
- broad Redox-runtime trust for `signalfd` or `timerfd`
- any POSIX message queue support
- any SysV message queue support

## Recommended Improvement Plan

### Phase I1 — Reclassify the IPC support language

**Goal:** Make subsystem docs accurately describe the current state.

**Do:**

- describe POSIX shm and semaphores as implemented
- describe SysV shm and semaphores as **bounded compatibility layers**, not comprehensive support
- describe `eventfd` as stronger than `signalfd` / `timerfd`
- describe message queues as still absent

**Exit criteria:** repo docs stop using broad phrases that imply complete IPC compatibility.

### Phase I2 — Harden the bounded SysV compatibility layers

**Goal:** Make the existing SysV support less misleading and more useful.

**Do:**

- decide whether Red Bear wants full semaphore-set support or an intentionally limited single-set model
- if limited, document that choice explicitly in relibc and subsystem docs
- otherwise extend `semget` / `semop` / `semctl` beyond the current semaphore-0-only model
- implement or explicitly reject `SEM_UNDO`
- add `semtimedop()` if downstreams need it
- expand `shmctl()` and `shmat()` support where real consumers need more than the current `IPC_PRIVATE`
  attach workflow

**Exit criteria:** the SysV shm/sem layers either become materially broader or are clearly documented
as intentionally bounded Redox compatibility shims.

### Phase I3 — Close the Qt/KDE runtime-proof gap

**Goal:** Move the IPC story from build-visible to desktop-visible.

**Do:**

- validate `QSharedMemory` under real Qt/KDE usage paths
- validate `QSystemSemaphore` in KDE consumers such as Solid
- validate KIO / KWin direct SysV shm paths
- record exactly which Qt/KDE IPC paths are now runtime-trusted versus merely build-capable

**Exit criteria:** Qt/KDE docs stop listing shared memory and semaphore support as unresolved relibc
confidence gaps.

### Phase I4 — Improve descriptor-event completeness for compositor/session code

**Goal:** Turn the current `eventfd` / `signalfd` / `timerfd` set into a more trustworthy runtime layer.

**Do:**

- keep `eventfd` on the current stable path
- validate `signalfd` in real event-loop style consumers
- extend `timerfd` semantics where current downstream code expects more than `TFD_TIMER_ABSTIME`
  (notably `TFD_TIMER_CANCEL_ON_SET`)
- build targeted Redox-target tests where host behavior is inherently not representative

**Exit criteria:** at least one meaningful compositor/session consumer is runtime-validated against
the current descriptor-event path.

### Phase I5 — Triage message queues explicitly

**Goal:** Stop leaving message queues as unprioritized TODOs.

**Do:**

- determine whether any current Red Bear subsystem actually needs POSIX or SysV message queues
- if not, mark them as lower-priority completeness debt
- if yes, create a dedicated implementation plan rather than burying them in generic header backlog

**Exit criteria:** `mqueue.h` and `sys/msg.h` are either on a concrete roadmap or explicitly treated
as non-blocking backlog.

## Recommended Order

The current best order is:

1. documentation cleanup and accurate IPC classification
2. SysV shm/sem hardening or explicit non-goal documentation
3. Qt/KDE runtime validation
4. descriptor-event runtime validation and timerfd semantic expansion
5. message queue triage

That order matches the current subsystem pressure better than a generic “finish all missing IPC
headers” strategy.

## Bottom Line

relibc IPC in Red Bear OS is no longer a story of missing primitives. It is now a story of **real
surface area with bounded compatibility depth**.

The strongest parts are POSIX shm, POSIX semaphores, `eventfd`, and the fact that major downstreams
already build. The weakest parts are the narrow SysV semantics, the lack of message queues, and the
runtime-proof gap for the desktop/session stack. The right next step is not random header work; it
is to harden and validate the IPC layers that current Qt/KDE and Wayland-adjacent consumers are
already trying to use.