Two follow-up items from the #10 PARTIAL commit (0f8ad8a50):
1. Added `make scratch-rebuild` target to the Makefile. The
v2 of scratch-rebuild.sh supports running without --dry-run
but there was no actual make wrapper for it. The new
target runs the script in non-dry-run mode (deletes
target/<arch>/{build,sysroot,stage.tmp}/ per recipe in
the closure and re-cooks in dep order). JOBS=N (default 4)
controls the parallel rebuild workers. Verified end-to-end:
the rebuild correctly deletes the 6-recipe closure's
build dirs and starts a parallel cook. m4 succeeds; bison
fails (missing host toolchain dep) — the failure is
correctly captured to the log without aborting the script.
2. Updated BUILD-SYSTEM-V6-HARDENING-POSTMORTEM.md to reflect
the 13-session / 9.5-DONE / 120-Python-test state:
- Added Session 13 entry covering #10 foundation + tests
+ the Python regex gotcha discovered during testing
(`^[[:space:]]*` vs `^[\s]*`)
- Updated test count: 99 -> 120 Python (now 7 test files
in local/scripts/tests/, was 4 at session 1)
- Updated scope line (12-session -> 13-session)
- Updated durability caveat (10 most recent commits -> 11
most recent commits; added `0f8ad8a50` and `9e5794ea7`)
- Updated 'What remains uncommitted' table
- Updated commit history table with rows for
`827895d32`, `9e5794ea7`, `0f8ad8a50`
- Added `test_scratch_rebuild.py` row to test coverage
table
BUILD-SYSTEM-IMPROVEMENTS.md was already updated in the
#10 commit (PARTIAL status, make target table, Implemented
#10 entry). This commit re-confirms those updates after
the postmortem rebalance.
Total state:
- 9.5/10 build-system improvements DONE (1 PARTIAL on #10)
- 120/120 Python tests + 27/27 Rust tests pass
- 10-job Gitea Actions pipeline
The build-system hardening arc is now as complete as a
single-session work scope allows. Further work requires
either the multi-day #10 full L-sized verification, the
multi-week #7A QML gate, or one of the larger blocked cooks
(sddm, KF6 dep chain).
L-sized improvement #10 (cookbook scratch-rebuild) is now
PARTIALLY shipped: the M-sized foundation is a runnable
script that does the right thing in the common case.
Verification against real cascades + integration with
rebuild-cascade.sh remains for a separate session.
local/scripts/scratch-rebuild.sh (190 lines, +x):
Step 1: discover autotools-using recipes by content regex
(aclocal|autoreconf|libtoolize|automake|autoconf|gettextize|./configure)
PLUS the AUTOTOOLS_CORE list (m4, autoconf, automake,
libtool, bison, flex, gettext) which are always-included
because they are autotools infrastructure even if they
don't directly invoke aclocal.
Step 2: compute transitive closure via BFS over the recipe
TOML dep graph, including both [build].dependencies and
[build].dev_dependencies. Found 6 autotools users in the
live tree: bison, diffutils, flex, grub, libtool, m4.
Step 3: for each recipe in the closure, delete
target/<arch>/{build,sysroot,stage.tmp}/ — PRESERVE source/
so we don't re-fetch the upstream tar.
Step 4: re-cook in dep order with --jobs=N (default 4) so
the rebuild itself runs in parallel via the dep-aware
scheduler (#1).
Cook errors during Step 4 do NOT abort the script with
exit 1 — a failed cook may indicate a missing upstream dep
(legitimate on a fresh checkout) rather than a real bug.
The user inspects the log and re-runs after addressing the
dep. This is documented in the header + Step 4 comment.
Supports --dry-run, --jobs=N, --help. Env overrides for
RECIPES_DIR + LOG_DIR (mirroring the migration script's
test escape hatch pattern, used by the test suite below).
21 unit tests in local/scripts/tests/test_scratch_rebuild.py:
TestAutotoolsCoreList (3) — m4, libtool, bison/flex
in AUTOTOOLS_CORE
TestAutotoolsContentRegex (8) — catches each canonical
autotools command; does
NOT match cmake/make/meson
TestRecipeDepParsing (4) — parses dependencies and
dev_dependencies; both;
neither
TestScriptHelp (1) — --help describes the
script
TestScriptStructure (5) — executable bit; uses
./target/release/repo;
PRESERVES source/; uses
--jobs=N; dry-run safe
Test count: 99 -> 120 (all in <1s).
The test file also surfaces a real Python regex gotcha:
`^[[:space:]]*` (POSIX char class with quantifier) silently
fails to match the empty string under Python's regex
engine, while `^[\s]*` (shorthand) works correctly. The
test regex uses the shorthand to avoid this.
Wired into:
make test-scratch-dry-run -> scratch-rebuild.sh --dry-run
Gitea Actions job scratch-dry-run (job 6 of 10, every PR)
With this commit, 9 of 10 build-system improvements in
BUILD-SYSTEM-IMPROVEMENTS.md are DONE (1 PARTIAL on #10);
the remaining 1 is #7A (QML gate, Qt6 engine fix, not a
cookbook improvement).
Verified: `./local/scripts/scratch-rebuild.sh --dry-run`
correctly discovers 6 autotools users and computes the
6-recipe closure. `make test-lint-scripts` still passes
120/120 tests in <1s. Gitea workflow YAML validates with
10 jobs total (was 9).
Commit 827895d32 added the C-7 KF6 sed migration script v2
and 13 unit tests, but didn't wire the new make target or
Gitea Actions job. This commit adds both so the migration
smoke test runs on every PR.
Makefile:
- New `make test-migration-dry-run` target. Runs
`migrate-kf6-seds-to-patches.sh --dry-run --limit=1`.
Discovers candidates, prints the per-recipe plan, exits 0
on success. Does NOT do any fetches, cooks, or patch
writes. <5s wall-clock. Added to `.PHONY:`.
- Picked up automatically by the existing
`make test-lint-scripts` discovery path (the new test
file is in local/scripts/tests/, so it's already covered
by the existing target — no change there).
Gitea Actions (`.gitea/workflows/build-system.yml`):
- New job `migration-dry-run` (job 5 of 9, depends on
`unit-tests`, runs on every PR + branch push + schedule).
Triggers `make test-migration-dry-run` and treats
exit 0 as success.
- Renumbered subsequent stage headers to 1f (was 1e).
- Updated unit-tests job description: '55 cases' -> '99
cases' (reflects the new 13 migration tests).
Docs:
- BUILD-SYSTEM-IMPROVEMENTS.md: added the new make
target to the Make targets table.
- BUILD-SYSTEM-V6-HARDENING-POSTMORTEM.md: Session 12
entry covers the v2 migration script + 13 tests + CI
integration. Updated test count (86 -> 99 Python),
scope line (11-session -> 12-session), C-7 finding
(now 'migration script v2 ... now runnable; per-recipe
execution + recipe rewrite still manual'), and
durability caveat (10 most recent commits now cover
the migration work + this postmortem itself).
- Added the test_migrate_kf6_seds.py row to the test
coverage table.
Verified:
- `make test-migration-dry-run` discovers 1 candidate
and exits 0 in <1s.
- `make test-lint-scripts` still passes 99/99 tests
in <1s.
- Gitea workflow YAML validates: 9 jobs total
(was 8).
The v6.0 build-system hardening arc lands 5 of the 10 improvements
proposed in local/docs/BUILD-SYSTEM-IMPROVEMENTS.md. All scripts
have unit tests (62 -> 86, all pass in <1s) and the new 'lint-recipe'
Gitea Actions job runs on every PR.
Per-recipe audit & lint scripts (catch R1/R2 violations BEFORE cook):
* audit-patch-idempotency.py — verifies external patches in
local/patches/ still apply against the upstream pinned rev.
Caught 1 real bug on first run: libdrm/02-redox-dispatch.patch
hunk at xf86drm.c:321 no longer matches libdrm-2.4.125.
* audit-kf6-deps.py — fetches upstream, scans for
find_package(KF6Xxx REQUIRED), compares to recipe deps. Catches
missing + dead dependencies in every kf6-* and qt* recipe.
* classify-cook-failure.py — 17-rule cook-failure classifier.
10-30s diagnosis vs 5-10min manual. exit code is intentionally
inverted (0=novel failure, 1=known fix) for CI signal.
* lint-recipe.py — 7-rule recipe lint: R1-NO-PATCH-FILE,
R1-PATH-SOURCE, R2-INLINE-SED, R2-PATCHES-DIR-UNUSED,
NO-LEGACY-MAKE, R1-LEGACY-APPLY-PATCHES, DEP-NOT-FOUND.
1.1s for 171 recipes (down from 60s+ in v1 via recipe-index
precomputation). Strict mode promotes warnings to errors.
Build-system convenience:
* repair-cook.sh — incremental-build optimizer.
Equivalent to 'repo cook <pkg>' but with a fast-path that
skips configure when CMakeCache.txt is newer than source AND
external patches haven't changed. 30-60s vs 5-10min on KF6
recipes. make repair.<pkg> / make clean-repair.<pkg> targets.
* migrate-kf6-seds-to-patches.sh — migration skeleton for
converting 56 inline 'sed -i' chains across the KF6 recipes
to durable external patches in local/patches/<name>/.
Gitea Actions (host-execution, no Docker):
* .gitea/workflows/build-system.yml — 8-job pipeline:
unit-tests, lint-offline, lint-network (nightly),
lint-recipe (NEW), lint-docs, build-mini, build-full,
smoke (QEMU boot).
* .gitea/RUNNER-SETUP.md — one-time Manjaro/Arch host setup.
Build script hardening:
* build-redbear.sh — when a low-level source (relibc,
kernel, base, bootloader, installer) is newer than its pkgar,
clean build/ and sysroot/ across all recipes too. Low-level
package changes leave autotools packages (pcre2, gettext,
libiconv, ...) with stale configure/libtool scripts referencing
the old runtime, causing 'libtool version mismatch' and
'not a valid libtool object' errors. Cleaning forces
re-configuration; stage/ and source/ are preserved so the
cookbook skips unchanged packages that don't use autotools.
* Makefile — wire lint-cook-failure,
lint-cook-failure-explain, lint-recipe, lint-recipe.%,
lint-recipe.strict, lint-recipe.%.strict, repair.%,
clean-repair.%, test-lint-scripts[-quiet]. Replace the
legacy 'validate-patches' target with a deprecation notice
pointing at validate-sources.
Documentation:
* BUILD-SYSTEM-IMPROVEMENTS.md — mark #2 and #5 DONE; full
implementation notes; updated Make-targets table.
* BUILD-SYSTEM-V6-HARDENING-POSTMORTEM.md (NEW) — 226-line durable
record of the 8-session arc: 32 findings categorized, 5 P0
audit-script bugs fixed, 6 over-broad multi-pattern rules
discovered + fixed, test coverage 86/86 in <1s, 7/10
improvements DONE.
* SCRIPT-BEHAVIOR-MATRIX.md — apply-patches.sh row marked
LEGACY/ARCHIVED; build-redbear.sh row no longer claims to
call it.
* boot-logs/README.md (NEW) — frozen-evidence policy:
'do not edit' rule for REDBEAR-FULL-BOOT-*-RESULTS.md files.
* libdrm/02-redox-dispatch.patch.README (NEW) — 8-step regen
procedure for the broken hunk.
Cleanup:
* local/cache/README.md deleted (1-line placeholder).
* legacy 'make validate-patches' target removed.
Per build-system improvement #5: lint-recipe.py's first run on
the live tree surfaced 1 broken-patch reference (redbear-sessiond),
1 dangling cookbook_apply_patches call (tc), 19 sed -i calls in
sddm (warning — cookbook_apply_patches present, drop-x11.py
migration in progress), 4 sed -i calls in qt6-wayland-smoke
(uncovers the same bug class the libwayland fix prevented).
5-phase hardening to prevent silent file-layer collisions (the D-Bus
regression class):
Phase 1: lint-config-paths.sh + make lint-config in depends.mk
Phase 2: CollisionTracker in installer (content-hash comparison)
Phase 3: installs manifests in recipe.toml + validate-file-ownership.sh
Phase 4: validate-init-services.sh + make validate in disk.mk
Phase 5: documentation (AGENTS.md, BUILD-SYSTEM-HARDENING-PLAN.md)
Both redbear-mini and redbear-full build and validate clean.
66 declared install paths in base, zero conflicts.
Packages/ is the canonical binary package repository for Red Bear OS.
Contains stage.pkgar copies of all built packages (91 files).
New scripts:
- local/scripts/sync-packages.sh: syncs built pkgar → Packages/
- make packages-sync: run sync
- make packages-list: list package count
Future: cache-auto will auto-sync to Packages/ after each build.
Every successful 'make all' now:
1. BEFORE: restores cache from git if needed
2. AFTER: syncs built packages to git-tracked cache
3. AFTER: auto-commits cache to git (with fallback if not in repo)
Flow: build → cache-sync → cache-commit
Cache survives: make clean, make distclean, git clone
This makes the build system fully resilient for a fork/overlay OS.
Red Bear is a fork/overlay on top of Redox. The upstream build
system wasn't designed for forks — it loses all cached stages on
make clean with no recovery path.
This commit adds a git-tracked build cache:
- local/cache/pkgar/{pkg}/stage.pkgar — per-package cache files
- cache-sync.sh: sync built packages → git-tracked cache
- cache-sync.sh --restore: restore cache → recipe targets
- cache-sync.sh --commit: sync + git commit
- Auto-restore before build, auto-sync after build
Cache survives: make clean, make distclean, git clone, upstream rebase.
Recovery from clean: seconds (restore from git) vs hours (full rebuild).
Adds comprehensive build cache snapshot and restore for overlay OS.
Problem: Upstream Redox build system is single-stream — make clean
destroys cached stage.pkgar files permanently. Build can't recover
without full from-scratch rebuild (2-4 hours).
Solution: Red Bear cache system provides:
- snapshot-cache.sh: Save all stage.pkgar to local/cache/
- restore-cache.sh: Restore from snapshot after make clean
- Auto-restore: Makefile auto-restores cache before build
- Essential cache: Pre-built caches for boot packages tracked in git
- Cookbook fixes: Missing deps trigger rebuild instead of crash
With cache restore, make clean recovery is measured in seconds,
not hours.
Gaps fixed in cookbook:
- modified_all_btree: missing dep → UNIX_EPOCH (rebuild trigger)
- sysroot install: missing dep → skip + rebuild
make distclean now documents that local/ is protected and will NOT be deleted. Add make distclean-nuclear as the only path that can touch local overlay sources (requires REDBEAR_ALLOW_LOCAL_UNFETCH=1). Add unfetch risk comments in mk/repo.mk for local overlay recipes.
Replace all 'rbos'/'RBOS' references with 'redbear'/'Red Bear OS'
across the build system, scripts, docs, and configs. Renamed files:
rbos.ipxe → redbear.ipxe
assets/rbos-icon.png → assets/redbear-icon.png
recipes/system/rbos-info → recipes/system/redbear-info
Added redbear-info: a system tool that enumerates all Red Bear OS
custom components, checks runtime availability via scheme paths and
binary presence, and prints status/test info. Supports --verbose,
--json, and --test output modes. Zero external dependencies.
Derivative of Redox OS (https://www.redox-os.org) adding:
- AMD GPU driver (amdgpu) via LinuxKPI compat layer
- ext4 filesystem support (ext4d scheme daemon)
- ACPI fixes for AMD bare metal (x2APIC, DMAR, IVRS, MCFG)
- Custom branding (hostname, os-release, boot identity)
Build system is full upstream Redox with RBOS overlay in local/.
Patches for kernel, base, and relibc are symlinked from local/patches/
and protected from make clean/distclean. Custom recipes live in
local/recipes/ with symlinks into the recipes/ search path.
Build: make all CONFIG_NAME=redbear-full
Sync: ./local/scripts/sync-upstream.sh
kellito
vasilito