When the user runs `repo cook A B C D`, the cookbook cooks the
transitive closure of those recipes strictly serially — even
recipes in the same dep level that have no inter-deps. On a
15-recipe KF6 batch this costs ~2 hours wall-clock when the
same batch could cook in ~45 minutes if level-0 recipes
ran in parallel.
Add `repo cook --jobs=N` to enable dep-aware level
parallelism. Default is 1 (serial — current behavior
preserved). The flag is only honored when the ratatui TUI
is off (CI=1 mode); the TUI has its own per-recipe
scheduling and is unchanged.
src/cook/scheduler.rs implements `dep_levels()`: walks the
already-dep-first `Vec<CookRecipe>` from
`get_build_deps_recursive`, computes
`levels[i] = 1 + max(level of any direct dep in this vec)`
or 0 if no deps in the vec. Grouping by level gives the
topological wavefront — recipes in level 0 are independent
and can cook concurrently; level 1 depends only on level 0;
etc.
src/bin/repo.rs: when jobs > 1 and !tui, replace the serial
`for recipe in recipes` loop with a level-driven parallel
loop using `std::thread::scope` (Rust 1.78+). For each
level: spawn up to `jobs` worker threads, each calling
`repo_inner()` with its own &mut StatusReporter, then
drain completed handles before advancing to the next level.
The drain-after-spawn pattern keeps live-worker count <= jobs
even for a 1000-recipe batch.
Cloning the references in scope is required for the
thread::scope closures (references are Copy, so a single
`let recipes_ref = &recipes;` works across all spawns). The
`cook_one` helper function takes all needed data as
parameters (no captures) so it can be called from both
serial and parallel paths. Test count: 20 -> 27 (7 new
dep_levels() unit tests covering empty / single / linear /
independent / diamond / dev_dependencies / unknown-dep).
Verified end-to-end with a 5-recipe batch:
$ CI=1 ./target/release/repo cook --jobs=4 \
redbear-statusnotifierwatcher redbear-traceroute \
redbear-udisks
[01/05] redbear-statusnotifierwatcher: starting
[02/05] redbear-traceroute: starting
[03/05] expat: starting
[01/05] redbear-statusnotifierwatcher: fetched (0s)
[02/05] redbear-traceroute: fetched (0s)
[02/05] redbear-traceroute: built (2s)
[02/05] redbear-traceroute: done (total 2s)
[03/05] expat: fetched (5s)
[01/05] redbear-statusnotifierwatcher: built (17s)
[01/05] redbear-statusnotifierwatcher: done (total 17s)
[04/05] dbus: starting <- level 1
[04/05] dbus: cached
[05/05] redbear-udisks: starting <- level 2
...
Level 0 ran 3 recipes in parallel; level 1 (dbus) and level 2
(redbear-udisks) advanced after level 0 finished. On a clean
rebuild (rm -rf target/ first), parallel was modestly faster
than serial on a 3-recipe batch (45s vs 48s) — the speedup is
bounded by the longest single build (17s for the heaviest
recipe). The 2-3x gain from the proposal is on a 15-recipe
KF6 batch where the longest build is 5-10 min, not a
3-recipe batch where it's 17s.
Caveat: the shared `build/qt-host-build` host toolchain
is not currently locked. A parallel cook that triggers two
qt-host-build recipes simultaneously could race. Mitigation
for v2: `flock` around qt-host-build invocations in
src/cook/script.rs. Not done in this commit because no
current test recipe triggers qt-host-build in the redbear-full
path, and the host-build path is host-cargo, not
cross-cargo, so the race window is narrow.
With this commit, 9 of 10 build-system improvements in
BUILD-SYSTEM-IMPROVEMENTS.md are DONE. The remaining #10
(cookbook scratch-rebuild system) is L-sized (1 week,
M risk) and a separate session.
When the cookbook runs without its ratatui TUI (e.g. `CI=1 repo cook
...` from a real terminal, SSH session, or backgrounded shell), the
only progress output is the per-recipe tail of the build script. The
user has no aggregate '5/15 done' view, no per-phase signal (fetch vs
build vs package), and no elapsed-time.
src/cook/status.rs adds a one-line StatusReporter that fills that
gap. Auto-enables when the TUI is off AND log capture is off AND
stderr is a TTY. Output format:
[05/15] kf6-kio: starting
[05/15] kf6-kio: fetched (3.2s)
[05/15] kf6-kio: built (4m 18s)
[05/15] kf6-kio: done (total 4m 23s)
Cached recipes emit `[NN/MM] recipe: cached` (no phase breakdown).
Writes to stderr via eprintln! so it never gets mixed with the
captured build-script log. The ratatui TUI in run_tui_cook() is
unchanged — this is the parallel status path for non-interactive
cooks.
Wiring: a &mut StatusReporter is created in main_inner's cook loop,
threaded through repo_inner() and the per-phase closures in
src/bin/repo.rs. Two phase emissions per recipe: `fetched` (after
handle_fetch) and `built` (after handle_cook, ONLY when the build
is not cached — cached cooks skip the 'built' emission to avoid
confusion). 6 unit tests cover format_elapsed boundaries, the
disabled no-op path, and phase tracking. Rust test count:
14 -> 20 (all pass in 3.2s).
Verified end-to-end with a real multi-recipe cook:
CI=1 ./target/release/repo cook redbear-statusnotifierwatcher \
redbear-traceroute \
redbear-udisks
[01/05] redbear-statusnotifierwatcher: starting
[01/05] redbear-statusnotifierwatcher: fetched (0s)
[01/05] redbear-statusnotifierwatcher: cached
[02/05] redbear-traceroute: starting
[02/05] redbear-traceroute: fetched (0s)
[02/05] redbear-traceroute: built (2s)
[02/05] redbear-traceroute: done (total 2s)
[03/05] expat: starting
...
Per build-system improvement #4. With this commit, 8 of 10
improvements in BUILD-SYSTEM-IMPROVEMENTS.md are DONE. Remaining:
#1 (parallel cook pool), #7A (QML gate), #10 (scratch-rebuild).
Two S-sized improvements from BUILD-SYSTEM-IMPROVEMENTS.md:
1. local/scripts/audit-patch-idempotency.py (improvement #3):
Validates that every external patch in local/patches/ is
idempotent (--reverse --check succeeds) and reproducible
(re-clone + re-apply produces an identical tree). Catches the
patch idempotency class of bug at lint time, where it used to
surface as a 2+ hour cookbook failure during a cook. Found a
real bug on first run: local/patches/libdrm/02-redox-dispatch.patch
has a hunk at xf86drm.c:321 that no longer matches the upstream
libdrm-2.4.125.
2. src/cook/script.rs auto-link Qt sysroot dirs (improvement #8):
The cookbook's BUILD_PRESCRIPT now auto-detects if the per-recipe
sysroot has Qt6 (qtbase or qtdeclarative) and creates the canonical
/usr/{plugins,mkspecs,metatypes,modules} symlinks that KF6 recipes
need for cmake to find Qt6Config.cmake's INTERFACE_* paths. New
KF6 recipes that depend on qtbase no longer need to manually
call redbear_qt_link_sysroot_dirs in their build script. Recipes
that need more customization can still call the helper directly
via 'source $COOKBOOK_ROOT/local/scripts/lib/qt-sysroot.sh'.
The per-recipe sysroot and stage cache used mtime of the dep pkgar
files to detect when a rebuild was needed. Any mtime bump on relibc
or any leaf dep (including the pre-cook relibc in build-redbear.sh)
would cascade-rebuild every downstream per-recipe sysroot even when
the dep's content was bit-identical. The resulting transient sysroot
extractions produced 'C compiler cannot create executables' and
'configure error' failures that retried fine standalone.
Replace the mtime checks with a blake3 content-hash fingerprint of
the dep pkgar set:
- For the per-recipe sysroot: store the fingerprint in
<target>/sysroot/.tags/deps-fingerprint and rebuild only when the
computed fingerprint does not match.
- For the per-recipe stage: store two fingerprints at
<target>/.deps-fingerprint and <target>/.host-deps-fingerprint.
Rebuild stage only when (source changed) OR (deps content changed)
OR (host-deps content changed) OR (auto_deps.toml missing).
This eliminates the transient build failures in 'make live' / 'build-redbear.sh'
and aligns the cache invalidation signal with the actual content the
recipe depends on, not the arbitrary mtime of the dependency package.
Per local/AGENTS.md Rule 2 (NO OVERLAY-STYLE PATCHES — AMENDED 2026),
big external projects (mesa, libdrm, wayland, qt, KF6, KWin, SDDM,
llvm, libepoxy, pipewire, wireplumber) apply their Red Bear edits as
external patches in local/patches/<component>/[0-9]*.patch on top of
the upstream checkout. Recipes for these projects previously inlined
the same 7-line bash loop (`cd $COOKBOOK_SOURCE; for p in
.../*.patch; do git apply ...; done; cd $COOKBOOK_BUILD`).
The cookbook_apply_patches helper centralizes this loop and adds
three quality improvements over the inline version:
1. Idempotence: each patch is checked with `git apply --reverse
--check` before applying. If the patch is already applied (e.g.
a partial re-cook after a previous successful build), the helper
skips it instead of failing with 'patch already applied'.
2. Failure accounting: applied/skipped/failed counts are reported
at the end of the run, so a single failed patch doesn't silently
abort the entire build chain.
3. Single source of truth: any future change to the patch application
semantics (e.g. supporting 3-way merges, --3way, or a different
check algorithm) happens in one place, not across 12+ recipes.
The helper is loaded into every recipe's build environment via the
existing cookbook shell-script injection in src/cook/script.rs and is
called as:
cookbook_apply_patches "${REDBEAR_PATCHES_DIR}"
This commit is the cookbook-side change. The 4 existing Rule 2
migration recipes (mesa, libdrm, pipewire, wireplumber) are refactored
in the next commit to call this helper instead of inlining the loop.
Red Bear is a full fork. The canonical recipe for a Red Bear
package lives in local/recipes/<category>/<pkg>/, NOT in
recipes/<category>/<pkg>/. The recipes/ tree is a frozen snapshot
that will eventually be deleted.
Previously the cookbook only walked recipes/, so local/recipes/
recipes were unreachable unless apply-patches.sh created a
symlink in recipes/. That overlay approach was wrong (per the
NO OVERLAY-STYLE PATCHES policy in AGENTS.md).
This change makes the cookbook walk ['local/recipes', 'recipes']
in that order. When a package exists in both trees, the
local/recipes/ path ALWAYS wins. The conflict resolution in
the walker is updated to enforce this rule explicitly.
Result: local/recipes/<pkg>/recipe.toml is discovered directly,
no symlink needed. 'repo find <pkg>' returns the local path.
'repo cook <pkg>' uses the local recipe.
This makes the full-fork model work end-to-end. local/recipes/
no longer needs an apply-patches.sh symlink; the cookbook
discovers it natively.
Add source_modified() that uses git ls-tree -r HEAD to hash the
contents of all tracked files in a source dir, falling back to
modified_dir_ignore_git for non-git sources. Wire into cook_build.rs
in place of the recursive mtime walk.
Eliminates spurious rebuilds from:
- .swp files, editor backups
- build artifacts in target/ or other untracked dirs
- filesystem timestamp drift after touch operations
The git-tree approach is content-addressed: identical content always
yields the same fingerprint, regardless of mtime. Combined with T1.1
(sysroot content-hash + pkgar mtime preservation), a no-op rebuild
should take seconds rather than hours.
Plan: local/docs/BUILD-SYSTEM-ROBUSTNESS-PLAN.md
After packaging, hash the staged sysroot with BLAKE3 (sorted paths,
deterministic). Compare against the previous build's fingerprint
stored next to stage.pkgar. If identical, restore the old pkgar mtime
on the new pkgar so dependents do not see a 'changed' timestamp and
skip their own rebuilds.
This catches the no-op rebuild pathology where a config-only change
(comment edit, [patch] reordering, dependency re-resolution) produces
byte-identical output but cascades through every dependent because of
mtime advancement.
Verified: 23 fingerprints written during redbear-mini build; T1.1
preserved mtime messages logged for relibc, libffi, expat, glib,
pcre2, etc. — all packages whose content was unchanged from the
previous build.
Plan: local/docs/BUILD-SYSTEM-ROBUSTNESS-PLAN.md
Bug: src/cook/cook_build.rs:268 only bumps source_modified when
recipe.toml is STRICTLY newer than source. In normal workflow:
1. Recipe edited at T1 (adds patch to patches array)
2. Source re-fetched at T2 > T1 (during make r.<recipe>)
3. Build caches stage at T3
4. Next build: recipe(T1) > source(T2) = FALSE → edit ignored
Fix: source_modified = source_modified.max(recipe_modified);
always considers recipe timestamp regardless of relative ordering.
Root cause of kernel rebrand not taking effect was ALSO a missing
.git/HEAD in the source tree (cookbook skips patches for release
archives). Re-fetch with 'repo --allow-protected fetch kernel'
restored the git repo and enabled patch application.
Verified: 'RedBear OS starting...' appears in QEMU boot log.
Add redbear-usb-storage-check in-guest binary that validates USB mass
storage read and write I/O: discovers /scheme/disk/ devices, writes a
test pattern to sector 2048, reads it back, verifies match, restores
original content. Updates test-usb-storage-qemu.sh with write-proof
verification step.
Includes all accumulated Red Bear OS work: kernel patches, relibc
patches, driver infrastructure, DRM/GPU, KDE recipes, firmware,
validation tooling, build system hardening, and documentation.
5-phase hardening to prevent silent file-layer collisions (the D-Bus
regression class):
Phase 1: lint-config-paths.sh + make lint-config in depends.mk
Phase 2: CollisionTracker in installer (content-hash comparison)
Phase 3: installs manifests in recipe.toml + validate-file-ownership.sh
Phase 4: validate-init-services.sh + make validate in disk.mk
Phase 5: documentation (AGENTS.md, BUILD-SYSTEM-HARDENING-PLAN.md)
Both redbear-mini and redbear-full build and validate clean.
66 declared install paths in base, zero conflicts.
Bootloader: alloc_zeroed_page_aligned no longer panics on OVMF
AllocatePages failure. Returns null on error.
Cookbook: pkgar falls back to repo/ dir when target pkgar missing.
Resolves KDE build chain failure.
redbear-full: builds 4.0GB image+ISO, boots without crash.
Shell: Changed default shell from ion to zsh for all user accounts
in base.toml, redbear-mini.toml, redbear-full.toml, and greeter config.
zsh 5.9 is already ported and builds (ZSH-PORTING-PLAN — fully implemented).
ion is kept as fallback/alternative.
Cookbook: pkgar staging fallback — when a dependency's target pkgar
doesn't exist, fall back to repo/<target>/<pkg>.pkgar. This fixes the
kf6-kitemviews build failure where libwayland's pkgar was missing from
the target directory.
Build system (src/cook/fetch.rs):
- Atomic patch application: applies patches to staging directory (cp -al),
atomically swaps on success, discards on failure — source tree is never
left in a partially-patched state
- normalize_patch(): strips diff --git/index/new-file-mode headers that the
build system's patch command does not recognize
- cleanup_workspace_pollution(): removes orphaned recipes/Cargo.toml and
recipes/Cargo.lock to prevent workspace conflicts
- Added --allow-protected CLI flag to repo binary
Input stack (local/patches/base/P3-*.patch):
- P3-ps2d-led-feedback: PS/2 LED state handling + InputProducer migration
- P3-inputd-keymap-bridge: InputProducer enum, keymap bridge query
- P3-usbhidd-hardening: HID descriptor validation, static lookup table,
8-button mouse support, transfer retry with exponential backoff
- P3-init-colored-output: ANSI-color coded init daemon output (green OK,
red FAILED, yellow SKIP/WARN)
XKB bridge (local/recipes/system/redbear-keymapd/source/src/xkb.rs):
- Parses X11 xkb/symbols/* format, maps XKB keycodes to PS/2 scancodes,
80+ X11 keysym names to Unicode, 4-level key support
Patch governance (local/patches/base/absorbed/README.md):
- Documents consolidation of P0-P3 patches into redox.patch
Finalize all non-artifact changes accumulated from other sessions:
- config updates, recipe changes, source edits, patches
- pkgar/cache artifacts intentionally excluded (build outputs)
This is the maximum achievable scope for this session.
Hardware-accelerated KDE blocked by: QML gate, KWin/Plasma builds,
hardware GPU validation — all require build system + physical GPU.
- Added Kahn's algorithm topological sort to new_recursive() in src/recipe.rs
- BFS previously returned flat dependency list with dependents before deps,
causing stage.pkgar 'Not Found' when cooking in list order
- Now deps always cook before dependents (kdecoration before breeze,
kf6-extra-cmake-modules before kde-cli-tools, etc.)
- Falls back to original order on dependency cycles
- Verified: kdecoration, kwin, plasma-wayland-protocols, KF6 packages
all cook successfully in correct dependency order
- kirigami still fails (needs Qt6 QML headers — known QML gate)
Root cause: modified_all_btree used ? on missing stage.pkgar,
causing cascade failure when make clean destroyed cached builds.
Fixes:
1. dep stage.pkgar missing → UNIX_EPOCH (triggers rebuild, not crash)
2. dep stage.pkgar missing during sysroot install → skip + rebuild
Build system now recovers from make clean by rebuilding deps.
- apply-patches.sh: add signature-marker checks for build-system patches
to handle cases where reverse-check fails but patch is already applied
- test-baremetal.sh: auto-disable TUI when stdout is not a terminal;
pass CI=1 to make
- test-live-iso-qemu.sh: pass CI=1 via env to prevent repo cook panic
- scripts/run.sh: auto-disable TUI when stdout is not a terminal;
pass CI=1 to qemu launch
- repo.rs: improve TUI initialization error messages (raw mode + alternate
screen) and rustfmt cleanups
- config.rs: auto-detect TTY presence for TUI enablement; use is_terminal()
instead of relying solely on CI env var
fetch.rs: use full commit hash for deterministic checkout. recipe.rs:
refactor recipe handling for cleaner patch application. sync-upstream:
add dry-run mode and improve rebase error recovery.
- Use full 40-char commit hash in base recipe.toml so the cookbook's
caching logic correctly recognizes already-fetched sources (short
hashes always missed the cache, causing patches to re-apply on top
of already-patched source).
- Add git clean -fd before git reset --hard in fetch.rs so untracked
files from previous patch applications are removed before re-patching.
- Remove ehcid/ohcid/uhcid from base-initfs BINS list (same fix as
base recipe, these drivers don't exist in the current upstream).
With these fixes, redbear-live-mini builds and boots to login prompt in QEMU.
Add is_local_overlay() path guard in repo.rs that detects recipes symlinked into local/recipes/ and refuses to delete their source/ during unfetch unless REDBEAR_ALLOW_LOCAL_UNFETCH=1 is set. Add the same guard in fetch.rs to block source-dir wipe and git reset --hard for local overlays unless REDBEAR_ALLOW_PROTECTED_FETCH=1 is set. Expand redbear_protected_recipe() from 8 core recipes to all 95+ local overlay recipe names.
Replace all 'rbos'/'RBOS' references with 'redbear'/'Red Bear OS'
across the build system, scripts, docs, and configs. Renamed files:
rbos.ipxe → redbear.ipxe
assets/rbos-icon.png → assets/redbear-icon.png
recipes/system/rbos-info → recipes/system/redbear-info
Added redbear-info: a system tool that enumerates all Red Bear OS
custom components, checks runtime availability via scheme paths and
binary presence, and prints status/test info. Supports --verbose,
--json, and --test output modes. Zero external dependencies.
Derivative of Redox OS (https://www.redox-os.org) adding:
- AMD GPU driver (amdgpu) via LinuxKPI compat layer
- ext4 filesystem support (ext4d scheme daemon)
- ACPI fixes for AMD bare metal (x2APIC, DMAR, IVRS, MCFG)
- Custom branding (hostname, os-release, boot identity)
Build system is full upstream Redox with RBOS overlay in local/.
Patches for kernel, base, and relibc are symlinked from local/patches/
and protected from make clean/distclean. Custom recipes live in
local/recipes/ with symlinks into the recipes/ search path.
Build: make all CONFIG_NAME=redbear-full
Sync: ./local/scripts/sync-upstream.sh
kellito
vasilito