# Red Bear OS Build Tools Porting Plan

**Status:** Phases 1-2 complete (2026-05-07)
**Goal:** Enable native compilation inside Red Bear OS — `./configure && make` producing
x86_64-unknown-redox binaries from within the target OS itself.

## Executive Summary

Red Bear OS currently has a **fully functional cross-compilation toolchain** (GCC 13.2.0,
LLVM 21, Rust nightly-2025-10-03) running on the Linux build host. These produce
x86_64-unknown-redox binaries that are packaged and installed into the OS image.

**There is no native build environment inside Red Bear OS.** You cannot run `./configure`,
`make`, `cmake`, or `cargo build` inside the target OS. To enable `cub build` (recipe
cooking) inside Red Bear OS as envisioned in the cub redesign, all build tools must be
ported to run natively on x86_64-unknown-redox.

This document assesses the current state, identifies the critical path, and provides a
phased implementation plan.

## Current State Inventory

### Cross-Compiler Toolchain (Host → Target)

```
prefix/x86_64-unknown-redox/
├── gcc-install/       ← GCC 13.2.0 cross-compiler (host → redox)
├── clang-install/     ← LLVM 21 cross-compiler
├── rust-install/      ← Rust nightly cross-compiler
├── relibc-install/    ← relibc headers + libraries
└── sysroot/           ← Target sysroot (/usr)
```

These compilers **run on the Linux host** and produce redox binaries. They are NOT
usable inside Red Bear OS itself.

### Build Tool Recipe Inventory

Of 47 build-tool recipes in the codebase:

| Status | Count | Description |
|--------|-------|-------------|
| ✅ Production | 25 | Build and work |
| 🚧 WIP/Partially tested | 6 | Build but not validated |
| ❌ TODO/Broken | 16 | Recipe exists but doesn't compile |

### What Already Exists (Production-Ready)

| Category | Tools |
|----------|-------|
| Shell | bash, zsh, dash, ion |
| Core utils | coreutils (Rust), findutils (Rust), ripgrep, gnu-grep, sed |
| File tools | patch, grep, sed |
| Archives | bzip2, xz, zstd, lz4 |
| Scripting | python312, lua54 |
| Build systems | gnu-make, cmake 4.0.3, autoconf, automake, pkg-config |
| Compilers (cross) | gcc13, llvm21, rust |
| VCS | git (v2.13.1, old) |

### What's Missing or Broken (Critical Gaps)

| Gap | Severity | Impact |
|-----|----------|--------|
| **No `tar`** | ⚠️ Critical | `./configure` scripts need tar extraction |
| **No `procps` (ps, kill)** | ⚠️ Critical | Build job control |
| **No `m4`** | ⚠️ Critical | Autotools macro processor |
| **No `meson`/`ninja`** | ⚠️ High | Qt, systemd, many libs use meson |
| **No `flex`/`bison`** | ⚠️ High | Parser generators for gcc, binutils, many pkgs |
| **`diffutils` suppressed** | Medium | gnulib/relibc header conflict in mini target |
| **`mkfifo` disabled** | Medium | `make -jN` parallel jobserver needs named pipes |
| **`perl5` WIP** | Medium | Autoconf/automake need perl for regeneration |
| **`texinfo` broken** | Low | Documentation generation |
| **`ruby` broken** | Low | Ruby ecosystem tools |

### POSIX Substrate Status (relibc)

Key build-tool-relevant POSIX functions:

| Function | Status | Impact |
|----------|--------|--------|
| `fork`/`exec` | ✅ Working | Process spawning |
| `pipe` | ✅ Working | IPC |
| `mmap` | ✅ Working | Memory mapping |
| `eventfd` | ✅ Implemented | Event notification |
| `signalfd` | 🚧 Partial | Signal delivery via fd (read path unverified) |
| `sem_open`/`close` | ✅ Implemented | Named semaphores |
| `shm_open` | ✅ Working | Shared memory |
| `waitid` | ✅ Implemented | Process reaping |
| `mkfifo` | ❌ Disabled | Named pipes — `make -j` jobserver blocked |
| `times()` | ❌ Missing | zsh `times` builtin stubbed |
| `getrlimit`/`setrlimit` | ✅ Implemented | Resource limits |

The POSIX substrate is **mostly adequate** for build tools. The critical gap is `mkfifo`
(named pipes), which blocks GNU Make's parallel jobserver. Single-threaded `make` works.

## Why Port Build Tools? (Motivation)

The cub package manager redesign envisions `cub build` running inside Red Bear OS:
```
User runs:  cub -S some-pkg        # Search AUR, fetch PKGBUILD
            cub -G some-pkg        # Convert to recipe.toml → ~/.cub/
            cub -B some-pkg        # BUILD inside Red Bear OS → install
```

Without native build tools, step 3 (`cub -B`) requires the host build toolchain, which
doesn't exist inside Red Bear OS. Until tools are ported, `cub` can only:
- Search AUR and fetch/convert PKGBUILDs
- Install pre-built pkgar packages (transferred from a build host)
- Manage the ~/.cub/ package database

Full `cub build` functionality requires native compilation capability.

## Dependency Graph

### Critical Path Chain (Bootstrap Order)

```
Level 0: Already available
  ├── bash, zsh, sed, grep, coreutils, findutils, patch, diffutils (in full)
  ├── python312, lua54
  ├── bzip2, xz, zstd, lz4
  └── pkg-config

Level 1: Prerequisite tools (need Level 0 to build)
  ├── m4        ← needs: configure (uses Level 0)
  ├── perl5     ← needs: configure + relibc siginfo fixes
  ├── tar       ← needs: cargo build (uutils-tar) or configure (GNU tar)
  ├── flex      ← needs: configure + m4 + bison (circular!)
  └── bison     ← needs: configure + m4 + flex (circular!)

Level 2: Build systems (need Level 0-1)
  ├── gnu-make  ← already production (needs mkfifo fix for -jN)
  ├── autoconf  ← already production
  ├── automake  ← already production
  ├── libtool   ← already builds (needs testing)
  ├── meson     ← needs: python312 + standalone script
  └── ninja     ← needs: cmake or python configure.py

Level 3: Native compilers (need Level 0-2 + cross-compiler bootstrap)
  ├── gcc-native  ← needs: cross-gcc bootstrap → native build
  ├── llvm-native ← needs: cross-clang bootstrap → native build  
  └── rust-native ← needs: gcc-native or llvm-native to build

Level 4: Full build environment
  └── All Level 0-3 → can ./configure && make inside Red Bear OS
```

### Circular Dependencies

**flex ↔ bison**: Both require each other to build. Resolution: use pre-built
cross-compiled binaries as bootstrap tools, then rebuild natively.

**GCC ↔ relibc**: GCC needs relibc headers to build. relibc needs GCC to compile.
Resolution: Already solved by the multi-stage bootstrap in `mk/prefix.mk`:
1. Build gcc-freestanding (no libc)
2. Build relibc with gcc-freestanding
3. Build full gcc with relibc sysroot

The same multi-stage approach works for native compilation.

## Implementation Plan

### Phase 1: Substrate Completion (Week 1-3)

**Goal**: All Level 0-1 tools available and working natively.

| Task | Effort | Dependencies | Notes |
|------|--------|-------------|-------|
| **Get `tar` working** | 2 days | none (cargo) | Promote `uutils-tar` from WIP → production. Uses `cargo` template. Should be straightforward — it's Rust, already has a recipe. |
| **Get `m4` working** | 1 day | none (configure) | Promote from WIP → production. Standard `./configure && make`. |
| **Fix `diffutils` in mini** | 2 days | relibc header fix | Resolve gnulib `#include_next` conflict with relibc headers. May require adjusting include order or adding a relibc wrapper header. |
| **Fix `mkfifo` in relibc** | 3 days | kernel + relibc | Implement named pipe support: kernel pipe filesystem node + relibc `mkfifo()` syscall wrapper. Unlocks `make -jN` parallel builds. |
| **Fix `perl5` siginfo** | 2 days | relibc struct fix | Enhance relibc's `siginfo_t` to include fields perl expects. Perl 5 already compiles — this fixes warnings/missing features. |

**Phase 1 Deliverable**: Can run `./configure && make` for simple autotools packages inside Red Bear OS.

### Phase 2: Parser Generators + Build Systems (Week 4-6)

**Goal**: flex, bison, meson, ninja available natively.

| Task | Effort | Dependencies | Notes |
|------|--------|-------------|-------|
| **Bootstrap `bison`** | 1 day | Phase 1 | Cross-compile bison on host, install as bootstrap. Then attempt native build. |
| **Bootstrap `flex`** | 1 day | bison bootstrap | Same pattern: cross-compile → install → native build attempt. |
| **Get `meson` working** | 1 day | python312 | Create standalone meson script (the TODO in the recipe). python312 already works. |
| **Get `ninja` working** | 1 day | cmake or python | ninja builds with cmake (which works) or configure.py (python). |
| **Validate `libtool`** | 1 day | Phase 1 | libtool builds but not tested. Run test suite, fix issues. |

**Phase 2 Deliverable**: meson+ninja build system available. Autotools regeneration (autoreconf) works natively.

### Phase 3: Native GCC Bootstrap (Week 7-12)

**Goal**: GCC 13.2.0 runs natively on Red Bear OS, producing x86_64-unknown-redox binaries.

This is the most complex phase — a multi-stage bootstrap:

```
Stage 1: Build gcc-freestanding (C compiler only, no libc)
  using: cross-compiler from host → native gcc
  result: native gcc that compiles C but can't link (no libc)

Stage 2: Build relibc with native gcc-freestanding
  result: libc.a, crt0.o, headers for the target

Stage 3: Build full gcc (C + C++ + libgcc + libstdc++)
  using: native gcc-freestanding + relibc sysroot
  result: full native GCC toolchain

Stage 4: Build binutils natively (optional)
  using: native GCC
  result: as, ld, ar, nm, strip, objdump native
```

| Task | Effort | Dependencies | Notes |
|------|--------|-------------|-------|
| **Create `gcc-native` recipe** | 3 days | Phase 1-2 | New recipe at `local/recipes/dev/gcc-native/`. Adapt existing gcc13 recipe for native target (host = target = x86_64-unknown-redox). |
| **Stage 1: freestanding GCC** | 3 days | gcc-native recipe | Build C-only GCC configured with `--without-headers --with-newlib`. Produces `xgcc` that compiles but can't link. |
| **Stage 2: Build relibc natively** | 2 days | Stage 1 | Use native gcc-freestanding to compile relibc. Similar to existing relibc-freestanding stage in prefix.mk but using native compiler. |
| **Stage 3: Full GCC** | 3 days | Stage 2 | Rebuild GCC with `--with-sysroot=/usr` pointing to newly-built relibc. Enables C++, libgcc, libstdc++. |
| **Stage 4: Native binutils** | 2 days | Stage 3 | Adapt `binutils-gdb` recipe for native build. |
| **Validation** | 3 days | Stage 3-4 | Build a known package (e.g., bash, sed) natively and verify the binary works. |

**Phase 3 Deliverable**: `gcc` and `g++` commands work inside Red Bear OS. `./configure && make` produces working redox binaries.

### Phase 4: LLVM/Clang Native (Week 13-16)

**Goal**: LLVM/Clang 21 runs natively, enabling Rust compilation.

| Task | Effort | Dependencies | Notes |
|------|--------|-------------|-------|
| **Create `llvm-native` recipe** | 2 days | Phase 3 | Adapt llvm21 recipe for native build. LLVM is cmake-based — once cmake works, LLVM is straightforward. |
| **Build clang native** | 2 days | llvm-native | Part of the same LLVM build tree. |
| **Build lld native** | 1 day | llvm-native | Linker — part of LLVM monorepo. |

**Phase 4 Deliverable**: `clang` and `clang++` work natively.

### Phase 5: Rust Native (Week 17-20)

**Goal**: `rustc` and `cargo` run natively inside Red Bear OS.

Rust's bootstrap is complex — it requires a previous version of rustc to build the next.
The approach:

1. Use the host cross-compiler to produce a native `rustc` and `cargo` binary
2. Use those as bootstrap to build a full native Rust toolchain
3. Or: download prebuilt Rust binaries (if Rust provides redox-native builds)

| Task | Effort | Dependencies | Notes |
|------|--------|-------------|-------|
| **Cross-compile rustc for redox** | 3 days | Phase 4 (llvm-native libs) | Use host rustc to cross-compile native rustc binary. Needs llvm-native libraries available as target deps. |
| **Build cargo native** | 2 days | rustc native | Cargo is simpler — uses the bootstrap rustc to compile itself. |
| **Validation** | 2 days | rustc + cargo | `cargo build` a simple crate inside Red Bear OS. |

**Phase 5 Deliverable**: `cargo build` works inside Red Bear OS. Rust packages can be compiled natively.

### Phase 6: cub Integration (Week 21-22)

**Goal**: `cub -B <pkg>` works fully inside Red Bear OS.

| Task | Effort | Dependencies | Notes |
|------|--------|-------------|-------|
| **Wire cook.rs to native tools** | 1 day | Phase 3+ | Update `cook.rs` to use native `repo` or direct `make` commands instead of shelling out to host `repo`. |
| **Validate cub build flow** | 2 days | Phase 3-5 | End-to-end: `cub -G <pkg>` (fetch AUR) → `cub -B <pkg>` (build natively) → install. |
| **Update cub docs** | 1 day | validation | Update CUB-PACKAGE-MANAGER.md with native build instructions. |

**Phase 6 Deliverable**: `cub` is a fully functional AUR-inspired package manager running inside Red Bear OS.

## Alternative Strategies

### Strategy A: Pre-Built Binary Toolchain (Faster)

Instead of bootstrapping GCC natively, download or cross-compile a pre-built native toolchain:

1. Use host cross-compiler to build GCC, binutils, make, etc. as **native redox binaries**
2. Package them as pkgar archives
3. Install into the Red Bear OS image
4. Users download pre-built toolchain packages via `cub -S build-essential`

**Advantage**: Skips the complex bootstrap. Weeks instead of months.
**Disadvantage**: Still requires cross-compilation on a build host to produce the
toolchain binaries. Not truly self-hosting. Updates require rebuild + repackage.

### Strategy B: Cross-Compilation as a Service (Hybrid)

1. `cub` running inside Red Bear OS detects a build request
2. Submits the build job to a build server (Linux host with cross-compiler)
3. Build server compiles, produces pkgar
4. `cub` downloads and installs the pkgar

**Advantage**: No native toolchain needed. Works immediately.
**Disadvantage**: Requires network + build server infrastructure. Not offline-capable.

### Strategy C: Phased Approach (Recommended)

1. **Phase 1-2 first** (substrate + build systems) — 6 weeks
2. **Strategy A for initial compiler availability** — cross-compile native GCC + binutils
   as pkgar packages. Skip the bootstrap. 2 weeks.
3. **Phase 5 for Rust** — once GCC native exists, bootstrap Rust. 4 weeks.
4. **Phase 6 for cub integration** — 2 weeks.
5. **Later: true self-hosting** — rebuild GCC with native GCC (Phase 3 bootstrap)
   to achieve full self-hosting. Deferred.

**Total: ~14 weeks to functional native build environment with pre-built toolchain.**
**Full self-hosting: +5 weeks for Phase 3 bootstrap.**

## Risk Assessment

| Risk | Likelihood | Impact | Mitigation |
|------|-----------|--------|------------|
| relibc POSIX gaps block GCC bootstrap | Medium | High | GCC is already ported as cross-compiler — the relibc surface GCC needs is known. Focus on `mkfifo` and any missing syscalls. |
| flex/bison circular dependency | High | Medium | Use cross-compiled bootstrap binaries. Standard practice in toolchain bootstrapping. |
| GCC native build is too large (memory/disk) | Medium | Medium | GCC is ~500MB source, ~2GB build. Red Bear OS images are 1.5-4GB. May need larger images or swap. |
| Make jobserver (`make -jN`) blocked by mkfifo | High | Low | Single-threaded `make` still works — just slower. Acceptable for initial porting. |
| Python312 module loading issues | Low | Medium | Dynamic loading of C modules works for main python312. May need fixes for specific modules meson uses. |
| LLVM native build too resource-intensive | Medium | High | LLVM is ~3GB source, ~20GB build. May need to build on host and install as pre-built pkgar. |

## Resource Estimates

| Phase | Calendar Time | Developer Effort | Key Deliverable |
|-------|--------------|-----------------|-----------------|
| 1: Substrate | 3 weeks | 10 dev-days | tar, m4, diffutils, mkfifo, perl5 |
| 2: Build Systems | 3 weeks | 6 dev-days | bison, flex, meson, ninja, libtool |
| 3: Native GCC | 6 weeks | 13 dev-days | gcc/g++ running natively |
| 4: Native LLVM | 4 weeks | 7 dev-days | clang/clang++ running natively |
| 5: Native Rust | 4 weeks | 7 dev-days | rustc/cargo running natively |
| 6: cub Integration | 2 weeks | 4 dev-days | cub build works end-to-end |
| **Total (full bootstrap)** | **22 weeks** | **47 dev-days** | Self-hosting Red Bear OS |
| **Total (pre-built strategy)** | **14 weeks** | **33 dev-days** | Native builds with pre-built toolchain |

Note: Developer effort assumes 1-2 developers working concurrently on independent tasks.
Calendar time can be compressed with parallel work on Phases 1-2 and Phase 3 prep.

## Recommendation

**Start with Strategy C (Phased + Pre-Built Toolchain).**

1. **Immediate (this week)**: Promote `tar` (`uutils-tar`) from WIP → production.
   This unblocks the entire autotools chain.
2. **Month 1**: Complete Phase 1-2 (substrate + build systems).
3. **Month 2**: Cross-compile native GCC + binutils as pkgar packages (Strategy A).
   Install into redbear-full image. Verify `./configure && make` works for a test
   package.
4. **Month 3**: Cross-compile native Rust toolchain. Verify `cargo build`.
5. **Month 4**: Wire cub to use native tools. Ship in `redbear-full`.

This gives a functional native build environment in ~4 months with ~1.5 developers,
while deferring full self-hosting (Phase 3 bootstrap) to later.

## Current Status (Pre-Work)

Before any porting work begins, these items should be verified:

- [ ] `uutils-tar` recipe — does it actually compile? (marked TODO, not tested)
- [ ] `m4` recipe — what's the compilation error? (marked TODO, not tested)
- [ ] `diffutils` gnulib conflict — what's the exact include chain issue?
- [ ] `mkfifo` kernel support — does the kernel have pipe filesystem nodes?
- [ ] `gcc13` recipe — does it already have a `--host=` flag that could target redox?
- [ ] Image size — can redbear-full image accommodate GCC (~500MB installed)?
- [ ] Memory — can QEMU allocate 4GB+ RAM for GCC builds?

## Related Documents

- `local/docs/CUB-PACKAGE-MANAGER.md` — cub package manager documentation
- `local/docs/RELIBC-AGAINST-GLIBC-ASSESSMENT.md` — relibc POSIX gap analysis
- `local/docs/CONSOLE-TO-KDE-DESKTOP-PLAN.md` — canonical desktop path plan
- `mk/prefix.mk` — cross-compiler toolchain build orchestration
- `recipes/dev/gcc13/recipe.toml` — GCC 13.2.0 cross-compiler recipe
- `recipes/groups/dev-essential/recipe.toml` — development essential packages group