When the user runs `repo cook A B C D`, the cookbook cooks the
transitive closure of those recipes strictly serially — even
recipes in the same dep level that have no inter-deps. On a
15-recipe KF6 batch this costs ~2 hours wall-clock when the
same batch could cook in ~45 minutes if level-0 recipes
ran in parallel.
Add `repo cook --jobs=N` to enable dep-aware level
parallelism. Default is 1 (serial — current behavior
preserved). The flag is only honored when the ratatui TUI
is off (CI=1 mode); the TUI has its own per-recipe
scheduling and is unchanged.
src/cook/scheduler.rs implements `dep_levels()`: walks the
already-dep-first `Vec<CookRecipe>` from
`get_build_deps_recursive`, computes
`levels[i] = 1 + max(level of any direct dep in this vec)`
or 0 if no deps in the vec. Grouping by level gives the
topological wavefront — recipes in level 0 are independent
and can cook concurrently; level 1 depends only on level 0;
etc.
src/bin/repo.rs: when jobs > 1 and !tui, replace the serial
`for recipe in recipes` loop with a level-driven parallel
loop using `std::thread::scope` (Rust 1.78+). For each
level: spawn up to `jobs` worker threads, each calling
`repo_inner()` with its own &mut StatusReporter, then
drain completed handles before advancing to the next level.
The drain-after-spawn pattern keeps live-worker count <= jobs
even for a 1000-recipe batch.
Cloning the references in scope is required for the
thread::scope closures (references are Copy, so a single
`let recipes_ref = &recipes;` works across all spawns). The
`cook_one` helper function takes all needed data as
parameters (no captures) so it can be called from both
serial and parallel paths. Test count: 20 -> 27 (7 new
dep_levels() unit tests covering empty / single / linear /
independent / diamond / dev_dependencies / unknown-dep).
Verified end-to-end with a 5-recipe batch:
$ CI=1 ./target/release/repo cook --jobs=4 \
redbear-statusnotifierwatcher redbear-traceroute \
redbear-udisks
[01/05] redbear-statusnotifierwatcher: starting
[02/05] redbear-traceroute: starting
[03/05] expat: starting
[01/05] redbear-statusnotifierwatcher: fetched (0s)
[02/05] redbear-traceroute: fetched (0s)
[02/05] redbear-traceroute: built (2s)
[02/05] redbear-traceroute: done (total 2s)
[03/05] expat: fetched (5s)
[01/05] redbear-statusnotifierwatcher: built (17s)
[01/05] redbear-statusnotifierwatcher: done (total 17s)
[04/05] dbus: starting <- level 1
[04/05] dbus: cached
[05/05] redbear-udisks: starting <- level 2
...
Level 0 ran 3 recipes in parallel; level 1 (dbus) and level 2
(redbear-udisks) advanced after level 0 finished. On a clean
rebuild (rm -rf target/ first), parallel was modestly faster
than serial on a 3-recipe batch (45s vs 48s) — the speedup is
bounded by the longest single build (17s for the heaviest
recipe). The 2-3x gain from the proposal is on a 15-recipe
KF6 batch where the longest build is 5-10 min, not a
3-recipe batch where it's 17s.
Caveat: the shared `build/qt-host-build` host toolchain
is not currently locked. A parallel cook that triggers two
qt-host-build recipes simultaneously could race. Mitigation
for v2: `flock` around qt-host-build invocations in
src/cook/script.rs. Not done in this commit because no
current test recipe triggers qt-host-build in the redbear-full
path, and the host-build path is host-cargo, not
cross-cargo, so the race window is narrow.
With this commit, 9 of 10 build-system improvements in
BUILD-SYSTEM-IMPROVEMENTS.md are DONE. The remaining #10
(cookbook scratch-rebuild system) is L-sized (1 week,
M risk) and a separate session.
When the cookbook runs without its ratatui TUI (e.g. `CI=1 repo cook
...` from a real terminal, SSH session, or backgrounded shell), the
only progress output is the per-recipe tail of the build script. The
user has no aggregate '5/15 done' view, no per-phase signal (fetch vs
build vs package), and no elapsed-time.
src/cook/status.rs adds a one-line StatusReporter that fills that
gap. Auto-enables when the TUI is off AND log capture is off AND
stderr is a TTY. Output format:
[05/15] kf6-kio: starting
[05/15] kf6-kio: fetched (3.2s)
[05/15] kf6-kio: built (4m 18s)
[05/15] kf6-kio: done (total 4m 23s)
Cached recipes emit `[NN/MM] recipe: cached` (no phase breakdown).
Writes to stderr via eprintln! so it never gets mixed with the
captured build-script log. The ratatui TUI in run_tui_cook() is
unchanged — this is the parallel status path for non-interactive
cooks.
Wiring: a &mut StatusReporter is created in main_inner's cook loop,
threaded through repo_inner() and the per-phase closures in
src/bin/repo.rs. Two phase emissions per recipe: `fetched` (after
handle_fetch) and `built` (after handle_cook, ONLY when the build
is not cached — cached cooks skip the 'built' emission to avoid
confusion). 6 unit tests cover format_elapsed boundaries, the
disabled no-op path, and phase tracking. Rust test count:
14 -> 20 (all pass in 3.2s).
Verified end-to-end with a real multi-recipe cook:
CI=1 ./target/release/repo cook redbear-statusnotifierwatcher \
redbear-traceroute \
redbear-udisks
[01/05] redbear-statusnotifierwatcher: starting
[01/05] redbear-statusnotifierwatcher: fetched (0s)
[01/05] redbear-statusnotifierwatcher: cached
[02/05] redbear-traceroute: starting
[02/05] redbear-traceroute: fetched (0s)
[02/05] redbear-traceroute: built (2s)
[02/05] redbear-traceroute: done (total 2s)
[03/05] expat: starting
...
Per build-system improvement #4. With this commit, 8 of 10
improvements in BUILD-SYSTEM-IMPROVEMENTS.md are DONE. Remaining:
#1 (parallel cook pool), #7A (QML gate), #10 (scratch-rebuild).
Build system (src/cook/fetch.rs):
- Atomic patch application: applies patches to staging directory (cp -al),
atomically swaps on success, discards on failure — source tree is never
left in a partially-patched state
- normalize_patch(): strips diff --git/index/new-file-mode headers that the
build system's patch command does not recognize
- cleanup_workspace_pollution(): removes orphaned recipes/Cargo.toml and
recipes/Cargo.lock to prevent workspace conflicts
- Added --allow-protected CLI flag to repo binary
Input stack (local/patches/base/P3-*.patch):
- P3-ps2d-led-feedback: PS/2 LED state handling + InputProducer migration
- P3-inputd-keymap-bridge: InputProducer enum, keymap bridge query
- P3-usbhidd-hardening: HID descriptor validation, static lookup table,
8-button mouse support, transfer retry with exponential backoff
- P3-init-colored-output: ANSI-color coded init daemon output (green OK,
red FAILED, yellow SKIP/WARN)
XKB bridge (local/recipes/system/redbear-keymapd/source/src/xkb.rs):
- Parses X11 xkb/symbols/* format, maps XKB keycodes to PS/2 scancodes,
80+ X11 keysym names to Unicode, 4-level key support
Patch governance (local/patches/base/absorbed/README.md):
- Documents consolidation of P0-P3 patches into redox.patch
- apply-patches.sh: add signature-marker checks for build-system patches
to handle cases where reverse-check fails but patch is already applied
- test-baremetal.sh: auto-disable TUI when stdout is not a terminal;
pass CI=1 to make
- test-live-iso-qemu.sh: pass CI=1 via env to prevent repo cook panic
- scripts/run.sh: auto-disable TUI when stdout is not a terminal;
pass CI=1 to qemu launch
- repo.rs: improve TUI initialization error messages (raw mode + alternate
screen) and rustfmt cleanups
- config.rs: auto-detect TTY presence for TUI enablement; use is_terminal()
instead of relying solely on CI env var
Add is_local_overlay() path guard in repo.rs that detects recipes symlinked into local/recipes/ and refuses to delete their source/ during unfetch unless REDBEAR_ALLOW_LOCAL_UNFETCH=1 is set. Add the same guard in fetch.rs to block source-dir wipe and git reset --hard for local overlays unless REDBEAR_ALLOW_PROTECTED_FETCH=1 is set. Expand redbear_protected_recipe() from 8 core recipes to all 95+ local overlay recipe names.
Derivative of Redox OS (https://www.redox-os.org) adding:
- AMD GPU driver (amdgpu) via LinuxKPI compat layer
- ext4 filesystem support (ext4d scheme daemon)
- ACPI fixes for AMD bare metal (x2APIC, DMAR, IVRS, MCFG)
- Custom branding (hostname, os-release, boot identity)
Build system is full upstream Redox with RBOS overlay in local/.
Patches for kernel, base, and relibc are symlinked from local/patches/
and protected from make clean/distclean. Custom recipes live in
local/recipes/ with symlinks into the recipes/ search path.
Build: make all CONFIG_NAME=redbear-full
Sync: ./local/scripts/sync-upstream.sh
kellito
vasilito