RedBear-OS/local/docs/BUILD-TOOLS-PORTING-PLAN.md

Red Bear OS Build Tools Porting Plan

Status: Phases 1-2 complete (2026-05-07) Goal: Enable native compilation inside Red Bear OS — ./configure && make producing x86_64-unknown-redox binaries from within the target OS itself.

Executive Summary

Red Bear OS currently has a fully functional cross-compilation toolchain (GCC 13.2.0, LLVM 21, Rust nightly-2025-10-03) running on the Linux build host. These produce x86_64-unknown-redox binaries that are packaged and installed into the OS image.

There is no native build environment inside Red Bear OS. You cannot run ./configure, make, cmake, or cargo build inside the target OS. To enable cub build (recipe cooking) inside Red Bear OS as envisioned in the cub redesign, all build tools must be ported to run natively on x86_64-unknown-redox.

This document assesses the current state, identifies the critical path, and provides a phased implementation plan.

Current State Inventory

Cross-Compiler Toolchain (Host → Target)

prefix/x86_64-unknown-redox/
├── gcc-install/       ← GCC 13.2.0 cross-compiler (host → redox)
├── clang-install/     ← LLVM 21 cross-compiler
├── rust-install/      ← Rust nightly cross-compiler
├── relibc-install/    ← relibc headers + libraries
└── sysroot/           ← Target sysroot (/usr)

These compilers run on the Linux host and produce redox binaries. They are NOT usable inside Red Bear OS itself.

Build Tool Recipe Inventory

Of 47 build-tool recipes in the codebase:

Status	Count	Description
✅ Production	25	Build and work
🚧 WIP/Partially tested	6	Build but not validated
❌ TODO/Broken	16	Recipe exists but doesn't compile

What Already Exists (Production-Ready)

Category	Tools
Shell	bash, zsh, dash, ion
Core utils	coreutils (Rust), findutils (Rust), ripgrep, gnu-grep, sed
File tools	patch, grep, sed
Archives	bzip2, xz, zstd, lz4
Scripting	python312, lua54
Build systems	gnu-make, cmake 4.0.3, autoconf, automake, pkg-config
Compilers (cross)	gcc13, llvm21, rust
VCS	git (v2.13.1, old)

What's Missing or Broken (Critical Gaps)

Gap	Severity	Impact
No tar	⚠️ Critical	./configure scripts need tar extraction
No procps (ps, kill)	⚠️ Critical	Build job control
No m4	⚠️ Critical	Autotools macro processor
No meson/ninja	⚠️ High	Qt, systemd, many libs use meson
No flex/bison	⚠️ High	Parser generators for gcc, binutils, many pkgs
diffutils suppressed	Medium	gnulib/relibc header conflict in mini target
mkfifo disabled	Medium	make -jN parallel jobserver needs named pipes
perl5 WIP	Medium	Autoconf/automake need perl for regeneration
texinfo broken	Low	Documentation generation
ruby broken	Low	Ruby ecosystem tools

POSIX Substrate Status (relibc)

Key build-tool-relevant POSIX functions:

Function	Status	Impact
fork/exec	✅ Working	Process spawning
pipe	✅ Working	IPC
mmap	✅ Working	Memory mapping
eventfd	✅ Implemented	Event notification
signalfd	🚧 Partial	Signal delivery via fd (read path unverified)
sem_open/close	✅ Implemented	Named semaphores
shm_open	✅ Working	Shared memory
waitid	✅ Implemented	Process reaping
mkfifo	❌ Disabled	Named pipes — make -j jobserver blocked
times()	❌ Missing	zsh times builtin stubbed
getrlimit/setrlimit	✅ Implemented	Resource limits

The POSIX substrate is mostly adequate for build tools. The critical gap is mkfifo (named pipes), which blocks GNU Make's parallel jobserver. Single-threaded make works.

Why Port Build Tools? (Motivation)

The cub package manager redesign envisions cub build running inside Red Bear OS:

User runs:  cub -S some-pkg        # Search AUR, fetch PKGBUILD
            cub -G some-pkg        # Convert to recipe.toml → ~/.cub/
            cub -B some-pkg        # BUILD inside Red Bear OS → install

Without native build tools, step 3 (cub -B) requires the host build toolchain, which doesn't exist inside Red Bear OS. Until tools are ported, cub can only:

• Search AUR and fetch/convert PKGBUILDs
• Install pre-built pkgar packages (transferred from a build host)
• Manage the ~/.cub/ package database

Full cub build functionality requires native compilation capability.

Dependency Graph

Critical Path Chain (Bootstrap Order)

Level 0: Already available
  ├── bash, zsh, sed, grep, coreutils, findutils, patch, diffutils (in full)
  ├── python312, lua54
  ├── bzip2, xz, zstd, lz4
  └── pkg-config

Level 1: Prerequisite tools (need Level 0 to build)
  ├── m4        ← needs: configure (uses Level 0)
  ├── perl5     ← needs: configure + relibc siginfo fixes
  ├── tar       ← needs: cargo build (uutils-tar) or configure (GNU tar)
  ├── flex      ← needs: configure + m4 + bison (circular!)
  └── bison     ← needs: configure + m4 + flex (circular!)

Level 2: Build systems (need Level 0-1)
  ├── gnu-make  ← already production (needs mkfifo fix for -jN)
  ├── autoconf  ← already production
  ├── automake  ← already production
  ├── libtool   ← already builds (needs testing)
  ├── meson     ← needs: python312 + standalone script
  └── ninja     ← needs: cmake or python configure.py

Level 3: Native compilers (need Level 0-2 + cross-compiler bootstrap)
  ├── gcc-native  ← needs: cross-gcc bootstrap → native build
  ├── llvm-native ← needs: cross-clang bootstrap → native build  
  └── rust-native ← needs: gcc-native or llvm-native to build

Level 4: Full build environment
  └── All Level 0-3 → can ./configure && make inside Red Bear OS

Circular Dependencies

flex ↔ bison: Both require each other to build. Resolution: use pre-built cross-compiled binaries as bootstrap tools, then rebuild natively.

GCC ↔ relibc: GCC needs relibc headers to build. relibc needs GCC to compile. Resolution: Already solved by the multi-stage bootstrap in mk/prefix.mk:

1. Build gcc-freestanding (no libc)
2. Build relibc with gcc-freestanding
3. Build full gcc with relibc sysroot

The same multi-stage approach works for native compilation.

Implementation Plan

Phase 1: Substrate Completion (Week 1-3)

Goal: All Level 0-1 tools available and working natively.

Task	Effort	Dependencies	Notes
Get tar working	2 days	none (cargo)	Promote uutils-tar from WIP → production. Uses cargo template. Should be straightforward — it's Rust, already has a recipe.
Get m4 working	1 day	none (configure)	Promote from WIP → production. Standard ./configure && make.
Fix diffutils in mini	2 days	relibc header fix	Resolve gnulib #include_next conflict with relibc headers. May require adjusting include order or adding a relibc wrapper header.
Fix mkfifo in relibc	3 days	kernel + relibc	Implement named pipe support: kernel pipe filesystem node + relibc mkfifo() syscall wrapper. Unlocks make -jN parallel builds.
Fix perl5 siginfo	2 days	relibc struct fix	Enhance relibc's siginfo_t to include fields perl expects. Perl 5 already compiles — this fixes warnings/missing features.

Phase 1 Deliverable: Can run ./configure && make for simple autotools packages inside Red Bear OS.

Phase 2: Parser Generators + Build Systems (Week 4-6)

Goal: flex, bison, meson, ninja available natively.

Task	Effort	Dependencies	Notes
Bootstrap bison	1 day	Phase 1	Cross-compile bison on host, install as bootstrap. Then attempt native build.
Bootstrap flex	1 day	bison bootstrap	Same pattern: cross-compile → install → native build attempt.
Get meson working	1 day	python312	Create standalone meson script (the TODO in the recipe). python312 already works.
Get ninja working	1 day	cmake or python	ninja builds with cmake (which works) or configure.py (python).
Validate libtool	1 day	Phase 1	libtool builds but not tested. Run test suite, fix issues.

Phase 2 Deliverable: meson+ninja build system available. Autotools regeneration (autoreconf) works natively.

Phase 3: Native GCC Bootstrap (Week 7-12)

Goal: GCC 13.2.0 runs natively on Red Bear OS, producing x86_64-unknown-redox binaries.

This is the most complex phase — a multi-stage bootstrap:

Stage 1: Build gcc-freestanding (C compiler only, no libc)
  using: cross-compiler from host → native gcc
  result: native gcc that compiles C but can't link (no libc)

Stage 2: Build relibc with native gcc-freestanding
  result: libc.a, crt0.o, headers for the target

Stage 3: Build full gcc (C + C++ + libgcc + libstdc++)
  using: native gcc-freestanding + relibc sysroot
  result: full native GCC toolchain

Stage 4: Build binutils natively (optional)
  using: native GCC
  result: as, ld, ar, nm, strip, objdump native

Task	Effort	Dependencies	Notes
Create gcc-native recipe	3 days	Phase 1-2	New recipe at local/recipes/dev/gcc-native/. Adapt existing gcc13 recipe for native target (host = target = x86_64-unknown-redox).
Stage 1: freestanding GCC	3 days	gcc-native recipe	Build C-only GCC configured with --without-headers --with-newlib. Produces xgcc that compiles but can't link.
Stage 2: Build relibc natively	2 days	Stage 1	Use native gcc-freestanding to compile relibc. Similar to existing relibc-freestanding stage in prefix.mk but using native compiler.
Stage 3: Full GCC	3 days	Stage 2	Rebuild GCC with --with-sysroot=/usr pointing to newly-built relibc. Enables C++, libgcc, libstdc++.
Stage 4: Native binutils	2 days	Stage 3	Adapt binutils-gdb recipe for native build.
Validation	3 days	Stage 3-4	Build a known package (e.g., bash, sed) natively and verify the binary works.

Phase 3 Deliverable: gcc and g++ commands work inside Red Bear OS. ./configure && make produces working redox binaries.

Phase 4: LLVM/Clang Native (Week 13-16)

Goal: LLVM/Clang 21 runs natively, enabling Rust compilation.

Task	Effort	Dependencies	Notes
Create llvm-native recipe	2 days	Phase 3	Adapt llvm21 recipe for native build. LLVM is cmake-based — once cmake works, LLVM is straightforward.
Build clang native	2 days	llvm-native	Part of the same LLVM build tree.
Build lld native	1 day	llvm-native	Linker — part of LLVM monorepo.

Phase 4 Deliverable: clang and clang++ work natively.

Phase 5: Rust Native (Week 17-20)

Goal: rustc and cargo run natively inside Red Bear OS.

Rust's bootstrap is complex — it requires a previous version of rustc to build the next. The approach:

1. Use the host cross-compiler to produce a native rustc and cargo binary
2. Use those as bootstrap to build a full native Rust toolchain
3. Or: download prebuilt Rust binaries (if Rust provides redox-native builds)

Task	Effort	Dependencies	Notes
Cross-compile rustc for redox	3 days	Phase 4 (llvm-native libs)	Use host rustc to cross-compile native rustc binary. Needs llvm-native libraries available as target deps.
Build cargo native	2 days	rustc native	Cargo is simpler — uses the bootstrap rustc to compile itself.
Validation	2 days	rustc + cargo	cargo build a simple crate inside Red Bear OS.

Phase 5 Deliverable: cargo build works inside Red Bear OS. Rust packages can be compiled natively.

Phase 6: cub Integration (Week 21-22)

Goal: cub -B <pkg> works fully inside Red Bear OS.

Task	Effort	Dependencies	Notes
Wire cook.rs to native tools	1 day	Phase 3+	Update cook.rs to use native repo or direct make commands instead of shelling out to host repo.
Validate cub build flow	2 days	Phase 3-5	End-to-end: cub -G <pkg> (fetch AUR) → cub -B <pkg> (build natively) → install.
Update cub docs	1 day	validation	Update CUB-PACKAGE-MANAGER.md with native build instructions.

Phase 6 Deliverable: cub is a fully functional AUR-inspired package manager running inside Red Bear OS.

Alternative Strategies

Strategy A: Pre-Built Binary Toolchain (Faster)

Instead of bootstrapping GCC natively, download or cross-compile a pre-built native toolchain:

1. Use host cross-compiler to build GCC, binutils, make, etc. as native redox binaries
2. Package them as pkgar archives
3. Install into the Red Bear OS image
4. Users download pre-built toolchain packages via cub -S build-essential

Advantage: Skips the complex bootstrap. Weeks instead of months. Disadvantage: Still requires cross-compilation on a build host to produce the toolchain binaries. Not truly self-hosting. Updates require rebuild + repackage.

Strategy B: Cross-Compilation as a Service (Hybrid)

1. cub running inside Red Bear OS detects a build request
2. Submits the build job to a build server (Linux host with cross-compiler)
3. Build server compiles, produces pkgar
4. cub downloads and installs the pkgar

Advantage: No native toolchain needed. Works immediately. Disadvantage: Requires network + build server infrastructure. Not offline-capable.

Strategy C: Phased Approach (Recommended)

1. Phase 1-2 first (substrate + build systems) — 6 weeks
2. Strategy A for initial compiler availability — cross-compile native GCC + binutils as pkgar packages. Skip the bootstrap. 2 weeks.
3. Phase 5 for Rust — once GCC native exists, bootstrap Rust. 4 weeks.
4. Phase 6 for cub integration — 2 weeks.
5. Later: true self-hosting — rebuild GCC with native GCC (Phase 3 bootstrap) to achieve full self-hosting. Deferred.

Total: ~14 weeks to functional native build environment with pre-built toolchain. Full self-hosting: +5 weeks for Phase 3 bootstrap.

Risk Assessment

Risk	Likelihood	Impact	Mitigation
relibc POSIX gaps block GCC bootstrap	Medium	High	GCC is already ported as cross-compiler — the relibc surface GCC needs is known. Focus on mkfifo and any missing syscalls.
flex/bison circular dependency	High	Medium	Use cross-compiled bootstrap binaries. Standard practice in toolchain bootstrapping.
GCC native build is too large (memory/disk)	Medium	Medium	GCC is ~500MB source, ~2GB build. Red Bear OS images are 1.5-4GB. May need larger images or swap.
Make jobserver (make -jN) blocked by mkfifo	High	Low	Single-threaded make still works — just slower. Acceptable for initial porting.
Python312 module loading issues	Low	Medium	Dynamic loading of C modules works for main python312. May need fixes for specific modules meson uses.
LLVM native build too resource-intensive	Medium	High	LLVM is ~3GB source, ~20GB build. May need to build on host and install as pre-built pkgar.

Resource Estimates

Phase	Calendar Time	Developer Effort	Key Deliverable
1: Substrate	3 weeks	10 dev-days	tar, m4, diffutils, mkfifo, perl5
2: Build Systems	3 weeks	6 dev-days	bison, flex, meson, ninja, libtool
3: Native GCC	6 weeks	13 dev-days	gcc/g++ running natively
4: Native LLVM	4 weeks	7 dev-days	clang/clang++ running natively
5: Native Rust	4 weeks	7 dev-days	rustc/cargo running natively
6: cub Integration	2 weeks	4 dev-days	cub build works end-to-end
Total (full bootstrap)	22 weeks	47 dev-days	Self-hosting Red Bear OS
Total (pre-built strategy)	14 weeks	33 dev-days	Native builds with pre-built toolchain

Note: Developer effort assumes 1-2 developers working concurrently on independent tasks. Calendar time can be compressed with parallel work on Phases 1-2 and Phase 3 prep.

Recommendation

Start with Strategy C (Phased + Pre-Built Toolchain).

1. Immediate (this week): Promote tar (uutils-tar) from WIP → production. This unblocks the entire autotools chain.
2. Month 1: Complete Phase 1-2 (substrate + build systems).
3. Month 2: Cross-compile native GCC + binutils as pkgar packages (Strategy A). Install into redbear-full image. Verify ./configure && make works for a test package.
4. Month 3: Cross-compile native Rust toolchain. Verify cargo build.
5. Month 4: Wire cub to use native tools. Ship in redbear-full.

This gives a functional native build environment in ~4 months with ~1.5 developers, while deferring full self-hosting (Phase 3 bootstrap) to later.

Current Status (Pre-Work)

Before any porting work begins, these items should be verified:

• uutils-tar recipe — does it actually compile? (marked TODO, not tested)
• m4 recipe — what's the compilation error? (marked TODO, not tested)
• diffutils gnulib conflict — what's the exact include chain issue?
• mkfifo kernel support — does the kernel have pipe filesystem nodes?
• gcc13 recipe — does it already have a --host= flag that could target redox?
• Image size — can redbear-full image accommodate GCC (~500MB installed)?
• Memory — can QEMU allocate 4GB+ RAM for GCC builds?

Related Documents

• local/docs/CUB-PACKAGE-MANAGER.md — cub package manager documentation
• local/docs/RELIBC-AGAINST-GLIBC-ASSESSMENT.md — relibc POSIX gap analysis
• local/docs/CONSOLE-TO-KDE-DESKTOP-PLAN.md — canonical desktop path plan
• mk/prefix.mk — cross-compiler toolchain build orchestration
• recipes/dev/gcc13/recipe.toml — GCC 13.2.0 cross-compiler recipe
• recipes/groups/dev-essential/recipe.toml — development essential packages group