vasilito/RedBear-OS
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: postmortem to 14-session / 122-Python-test state (Session 14)

Browse Source 
Session 14 entry covering b8c1c780d (first C-7 migration
patch + v2 script unfetch-before-fetch fix). Updated
durability caveat: 13 most recent commits cover the full
arc; commit history table gets the new b8c1c780d and
975cda686 rows; test count 120 -> 122. Removed a duplicate
975cda686 row that was added by accident in the previous
edit.
This commit is contained in:
kellito
2026-06-12 17:10:24 +03:00
parent b8c1c780dc
commit 761cb2b98a
1 changed files with 24 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files
local/docs/BUILD-SYSTEM-V6-HARDENING-POSTMORTEM.md
+24 -17
View File
@@ -1,10 +1,10 @@
# Red Bear OS Build-System v6.0 Hardening — Post-Mortem
> **Scope.** This document is the durable record of the
> 13-session v6.0 build-system hardening work arc (2026-06-08 to
> 14-session v6.0 build-system hardening work arc (2026-06-08 to
> 2026-06-12). It captures the 10 build-system improvements
> (9 DONE, 1 OPEN), 32 findings addressed, the Gitea Actions CI
> pipeline, the 147-test suite covering all 17 classifier rules +
> pipeline, the 149-test suite covering all 17 classifier rules +
> their false-positive inverses plus the 6 new status and 7 new
> scheduler Rust unit tests, and the deferred follow-up work.
> The 7,000+ uncommitted file modifications in the user's working
@@ -12,11 +12,14 @@
>
> **Durability caveat (added 2026-06-12 after final review).**
> The deliverables in this arc are durable **on disk** in the
> working tree, and most are now durable in `git` history. The 12
> most recent commits on `0.2.3` (`e1c2e7958`, `0f8ad8a50`,
> `9e5794ea7`, `827895d32`, `693e4d774`, `fbc32a6d8`, `5325360b4`,
> `ae749ffb2`, `97fa3a17a`, `bd18eefc6`, `03c8a38a1`, `d6c784ed3`)
> cover the postmortem rebalance to 13-session / 9.5-DONE; the
> working tree, and most are now durable in `git` history. The 13
> most recent commits on `0.2.3` (`b8c1c780d`, `975cda686`,
> `e1c2e7958`, `0f8ad8a50`, `9e5794ea7`, `827895d32`, `693e4d774`,
> `fbc32a6d8`, `5325360b4`, `ae749ffb2`, `97fa3a17a`, `bd18eefc6`,
> `03c8a38a1`) cover the first durable C-7 migration patch
> (`local/patches/kf6-karchive/01-initial-migration.patch`); the
> `make lint-build-system-all` aggregate + 11-job CI; the
> postmortem rebalance to 13-session / 9.5-DONE; the
> `make scratch-rebuild` target wiring; the improvement #10
> scratch-rebuild skeleton + 21 tests + Makefile + Gitea CI
> integration; the migration-dry-run CI integration; the C-7 KF6
@@ -28,19 +31,19 @@
> `classify-cook-failure.py`; `audit-patch-idempotency.py`; and
> the auto-link Qt sysroot dirs patch in `src/cook/script.rs`.
> The remaining v6.0 deliverable still in `git status` is this
> BUILD-SYSTEM-V6-HARDENING-POSTMORTEM.md (updated to 13-session
> / 9.5-DONE / 120-Python-test state, with the make-wrapper
> entry + updated commit history table). The C-1..C-6 doc and
> code fixes, `boot-logs/README.md`, and
> BUILD-SYSTEM-V6-HARDENING-POSTMORTEM.md (updated to 14-session
> / 9.5-DONE / 122-Python-test state, with the Session 14
> entry + the b8c1c780d commit-history row). The C-1..C-6 doc
> and code fixes, `boot-logs/README.md`, and
> `migrate-kf6-seds-to-patches.sh` (v1) were committed in
> `ae749ffb2`; v2 rewrite of the script + 13 tests + Makefile +
> Gitea CI integration in `827895d32`; the #10 scratch-rebuild
> skeleton + 21 tests + Makefile + Gitea CI integration in
> `0f8ad8a50`; the migration-dry-run CI integration in
> `9e5794ea7`; the make-wrapper + postmortem rebalance in
> `e1c2e7958`.
> Gitea CI integration in `0f8ad8a50`; the migration-dry-run
> CI integration in `9e5794ea7`. Going forward, any new v6.0
> `e1c2e7958`; the lint-build-system-all aggregate + 11-job CI
> in `975cda686`; the first durable C-7 patch + migration
> script v3 fix in `b8c1c780d`. Going forward, any new v6.0
> work should be committed with
> `git add <specific-files>` to avoid sweeping the user's
> 7,000+ unrelated WIP modifications.
@@ -61,7 +64,8 @@
| 10 | 06-12 (cont.) | **Build-system hardening arc commit + improvement #4 shipped.** First durable commit: `ae749ffb2 build: ship build-system hardening arc (5 of 10 improvements)` — 22 build-system files, including the 5 prior arc deliverables (audit-kf6-deps.py + 13 tests, repair-cook.sh + 7 tests, migrate-kf6-seds-to-patches.sh, BUILD-SYSTEM-V6-HARDENING-POSTMORTEM.md, SCRIPT-BEHAVIOR-MATRIX.md, boot-logs/README.md, build-system.yml, Gitea RUNNER-SETUP.md, libdrm/02 sidecar, cache/README cleanup, Makefile lint/repair targets). Then `5325360b4 build: add cook status reporter (improvement #4)` — `src/cook/status.rs` (197 lines, 6 unit tests) + `src/bin/repo.rs` wiring. Auto-enables in `CI=1` mode when stderr is a TTY: one-line `[NN/MM] recipe: phase (Xs)` output for each cook. Verified end-to-end with 3-recipe and 5-recipe real cooks. **Test count: 86/86 Python + 20/20 Rust.** |
| 11 | 06-12 (cont.) | **Build-system improvement #1 shipped.** `src/cook/scheduler.rs` (145 lines, 7 unit tests) + `src/bin/repo.rs` `repo cook --jobs=N` flag. Dep-aware level partition via `dep_levels()` (each recipe's level = `1 + max(level of any direct dep in this vec)`, or 0 if no deps in the vec). For each level, runs all recipes in that level via `std::thread::scope` with up to `N` workers. Drain-after-spawn pattern keeps live-worker count <= jobs. Ratatui TUI path unchanged. 7 unit tests cover empty / single / linear / independent / diamond / dev_dependencies / unknown-dep. Verified end-to-end: 5-recipe batch (redbear-statusnotifierwatcher, redbear-traceroute, redbear-udisks + deps expat, dbus) cooks in level 0 (3 parallel) → level 1 (dbus) → level 2 (redbear-udisks). On clean 3-recipe rebuild: 48s serial vs 45s parallel. Speedup bounded by longest single build (17s) on this small batch — the 2-3x gain from the proposal is on 15-recipe KF6 batches with 5-10 min longest builds. Caveat: `build/qt-host-build` host toolchain not yet locked; v2 mitigation is `flock` in `src/cook/script.rs` (deferred, no current redbear-full test recipe triggers qt-host-build). **Test count: 86/86 Python + 27/27 Rust.** |
| 12 | 06-12 (cont.) | **C-7 KF6 sed migration script v2 + CI integration.** The v1 shipped in `ae749ffb2` was a stub with three structural bugs that made it unrunnable: called `repo cook <recipe_dir>` with a path (cookbook takes bare names); created an empty pristine_dir via mktemp -d but never populated it; Step 4 was `SKIP — manual rewrite pending` so the script wrote no patch even when the inline sed chains actually edited the source. Replaced with a working v2: bare-name cookbook CLI, real pristine-source snapshot (`cp -r source/ source-pristine/`) BEFORE the cook, real diff capture, real patch save to `local/patches/<name>/01-initial-migration.patch`. Added `--dry-run` for safe CI smoke testing, `--recipe=<name>` and `--limit=N` for targeted runs, `--help` for the script's contract. Test escape hatch via `REDBEAR_MIGRATE_RECIPES_DIR` / `REDBEAR_MIGRATE_PATCHES_DIR` env vars so the candidate discovery can be exercised on synthetic trees without touching the live project. 13 unit tests in `local/scripts/tests/test_migrate_kf6_seds.py` — 7 candidate-discovery tests (synthetic tree with `make_recipe()` helper, asserts stdout/stderr + exit code) + 6 script-structure tests (regression guards against the v1 bugs: "uses bare names not paths", "uses release/repo binary", "creates patches dir", "diff includes .git/target excludes", "unfetches after capture", "idempotent SKIP when patch exists"). Wired into `make test-migration-dry-run` and new Gitea Actions job `migration-dry-run` (job 5 of 9, every PR). **Test count: 99/99 Python + 27/27 Rust.** Verified `--dry-run --limit=5` correctly identifies `breeze`, `kde-cli-tools`, `kdecoration`, `kf6-attica`, `kf6-karchive` as the first 5 of 56 candidate recipes. The actual migration run still requires the full KF6 dep chain to be built (qtbase, qtdeclarative, kf6-extra-cmake-modules, plus per-recipe deps); the per-recipe verification + recipe-rewrite remains a manual step (the script's `Next steps:` output documents this). |
| 13 | 06-12 (cont.) | **Improvement #10 (scratch-rebuild) M-sized foundation + CI integration.** The L-sized #10 proposal is split: the M-sized foundation (autotools detection + dep closure + targeted clean + parallel rebuild) is now a runnable 190-line bash script (`local/scripts/scratch-rebuild.sh`); the L-sized remaining work (full integration with `rebuild-cascade.sh`, byte-identical rebuild check via `stage.pkgar` hash diffing, cross-host-toolchain case) is documented but deferred. The script: (1) discovers autotools-using recipes by content regex + AUTOTOOLS_CORE list, (2) computes transitive closure via BFS over the recipe TOML dep graph (both `[build].dependencies` and `[build].dev_dependencies`), (3) deletes `target/<arch>/{build,sysroot,stage.tmp}/` per recipe in the closure (preserves `source/` to avoid re-fetch), (4) re-cooks in dep order via the cookbook's `--jobs=N` flag. Cook errors during step 4 do NOT abort the script — a failed cook may indicate a missing upstream dep on a fresh checkout rather than a real bug. `--dry-run`, `--jobs=N`, `--help` supported. 21 unit tests in `local/scripts/tests/test_scratch_rebuild.py`: 3 autotools-core list tests, 8 regex content-match tests (each canonical autotools command + negative cases), 4 dep-parser tests, 1 help test, 5 script-structure tests (executable bit, uses release/repo, preserves source/, uses --jobs=N, dry-run safe). The test file also surfaced a real Python regex gotcha: `^[[:space:]]*` (POSIX char class with quantifier) silently fails to match the empty string under Python's regex engine; the test uses `^[\s]*` shorthand instead. Wired into `make test-scratch-dry-run`, `make scratch-rebuild`, and new Gitea Actions job `scratch-dry-run` (job 6 of 10). **Test count: 120/120 Python + 27/27 Rust.** Verified `--dry-run` against live tree: discovers 6 autotools users (bison, diffutils, flex, grub, libtool, m4) and computes a 6-recipe closure. `make scratch-rebuild` end-to-end: deletes the 6 recipe dirs' build/sysroot/stage.tmp, runs `repo cook --jobs=4` on the closure. m4 cooks successfully; bison fails (missing host toolchain dep) — the failure is correctly captured to the log without aborting the script. |## Final state
| 13 | 06-12 (cont.) | **Improvement #10 (scratch-rebuild) M-sized foundation + CI integration.** The L-sized #10 proposal is split: the M-sized foundation (autotools detection + dep closure + targeted clean + parallel rebuild) is now a runnable 190-line bash script (`local/scripts/scratch-rebuild.sh`); the L-sized remaining work (full integration with `rebuild-cascade.sh`, byte-identical rebuild check via `stage.pkgar` hash diffing, cross-host-toolchain case) is documented but deferred. The script: (1) discovers autotools-using recipes by content regex + AUTOTOOLS_CORE list, (2) computes transitive closure via BFS over the recipe TOML dep graph (both `[build].dependencies` and `[build].dev_dependencies`), (3) deletes `target/<arch>/{build,sysroot,stage.tmp}/` per recipe in the closure (preserves `source/` to avoid re-fetch), (4) re-cooks in dep order via the cookbook's `--jobs=N` flag. Cook errors during step 4 do NOT abort the script — a failed cook may indicate a missing upstream dep on a fresh checkout rather than a real bug. `--dry-run`, `--jobs=N`, `--help` supported. 21 unit tests in `local/scripts/tests/test_scratch_rebuild.py`: 3 autotools-core list tests, 8 regex content-match tests (each canonical autotools command + negative cases), 4 dep-parser tests, 1 help test, 5 script-structure tests (executable bit, uses release/repo, preserves source/, uses --jobs=N, dry-run safe). The test file also surfaced a real Python regex gotcha: `^[[:space:]]*` (POSIX char class with quantifier) silently fails to match the empty string under Python's regex engine; the test uses `^[\s]*` shorthand instead. Wired into `make test-scratch-dry-run`, `make scratch-rebuild`, and new Gitea Actions job `scratch-dry-run` (job 6 of 10). **Test count: 120/120 Python + 27/27 Rust.** Verified `--dry-run` against live tree: discovers 6 autotools users (bison, diffutils, flex, grub, libtool, m4) and computes a 6-recipe closure. `make scratch-rebuild` end-to-end: deletes the 6 recipe dirs' build/sysroot/stage.tmp, runs `repo cook --jobs=4` on the closure. m4 cooks successfully; bison fails (missing host toolchain dep) — the failure is correctly captured to the log without aborting the script. |
| 14 | 06-12 (cont.) | **First durable C-7 KF6 sed migration patch (`b8c1c780d`).** Executed the v2 migration script against `local/recipes/kde/kf6-karchive` and shipped `local/patches/kf6-karchive/01-initial-migration.patch` (24 lines, 1 real hunk). The patch is the result of capturing the diff between the pristine upstream karchive-6.26.0 tarball and the post-cook state — the `ecm_install_po_files_as_qm` line is now commented out (the recipe's `sed -i 's/^ecm_install_po_files_as_qm/#ecm_install_po_files_as_qm/'`). The other 3 sed chains in the recipe (ki18n_install, arg(mode), arg(d->mode)) were no-ops against this upstream version — the migration correctly captures only the real edits. Discovered a real bug in the v2 migration script during the first run: it was producing silently empty diffs on already-cooked recipes because the cookbook's `fetch` re-uses an existing source/ tree. Fix: add explicit `unfetch` BEFORE the `fetch` (so the source/ dir is removed before re-extraction) + set `REDBEAR_ALLOW_LOCAL_UNFETCH=1` (the cookbook's default policy is to never clobber a local-overlay source, and the env var overrides that for the explicit unfetch). The patch was also filtered to drop ECM-autogenerated noise files (`.clang-format`, `.gitignore`, `target/` artifacts); 122-line raw diff → 24-line filtered patch. Added 2 regression tests: `test_sets_local_unfetch_env_var` (guards against forgetting the env var) and `test_unfetches_before_fetching` (guards against the silent-failure mode). **Test count: 122/122 Python + 27/27 Rust.** Next steps for kf6-karchive (manual, not part of this commit): edit `[build].script` to remove the 4 sed chains + add `REDBEAR_PATCHES_DIR` / `cookbook_apply_patches` invocation, re-cook, verify byte-identical `stage.pkgar`, commit recipe rewrite alongside the patch. |## Final state
### 10 build-system improvements — 9 DONE, 1 OPEN
@@ -104,7 +108,7 @@
| `cook::status` (Rust) | 6 | format_elapsed boundaries, disabled no-op, phase tracking |
| `cook::scheduler::dep_levels` (Rust) | 7 | empty / single / linear / independent / diamond / dev_dependencies / unknown-dep |
**Total: 120/120 Python + 27/27 Rust pass in <1 second (Python) / ~3 seconds (Rust).**
**Total: 122/122 Python + 27/27 Rust pass in <1 second (Python) / ~3 seconds (Rust).**
**8 CRITICAL findings (all addressed):**
- C-1 libwayland `patches = [redox.patch]` line removed (was blocking the Wayland stack)
@@ -176,6 +180,9 @@ The v6.0 deliverables were committed in 3 durable chunks on
| `827895d32` | 2 | C-7 KF6 sed migration script v2: rewrote the v1 stub to actually capture diffs (`cp -r source/ source-pristine/` BEFORE the cook, real `diff -ruN`, real patch save to `local/patches/<name>/01-initial-migration.patch`). 13 unit tests in `test_migrate_kf6_seds.py`. |
| `9e5794ea7` | 4 | CI integration for the migration script: `make test-migration-dry-run` target + Gitea Actions `migration-dry-run` job (job 5 of 9 → 10). |
| `0f8ad8a50` | 5 | Improvement #10 (scratch-rebuild) M-sized foundation: `local/scripts/scratch-rebuild.sh` (190 lines, +x) + 21 unit tests in `test_scratch_rebuild.py`. `make test-scratch-dry-run` + `make scratch-rebuild` targets. Gitea Actions `scratch-dry-run` job (job 6 of 10). |
| `e1c2e7958` | 2 | Wired `make scratch-rebuild` (no-dry-run variant). Postmortem rebalance to 13-session / 9.5-DONE / 120-Python-test state. |
| `975cda686` | 4 | New `make lint-build-system-all` aggregate (offline-safe lints + tests + smoke). New Gitea Actions job (job 7 of 11). |
| `b8c1c780d` | 3 | First durable C-7 migration patch (`local/patches/kf6-karchive/01-initial-migration.patch` — 24 lines, 1 real hunk). Discovered + fixed the v2 script's silent-failure mode: cookbook `fetch` re-uses existing source/; migration must `unfetch` first + set `REDBEAR_ALLOW_LOCAL_UNFETCH=1`. Added 2 regression tests for the env-var + unfetch-before-fetch invariants. |
**This post-mortem itself** is still in `git status` (modified
`local/docs/BUILD-SYSTEM-V6-HARDENING-POSTMORTEM.md`); the
@@ -186,6 +193,6 @@ shipment status.
| Path | What it is | Why uncommitted |
|------|-----------|-----------------|
| `local/docs/BUILD-SYSTEM-V6-HARDENING-POSTMORTEM.md` | This doc updated for 13-session / 9.5-DONE / 120-Python-test state | Next user-chosen commit; touches paths the user may have other WIP for |
| `local/docs/BUILD-SYSTEM-V6-HARDENING-POSTMORTEM.md` | This doc updated for 14-session / 9.5-DONE / 122-Python-test state (Session 14 entry + b8c1c780d commit-history row) | Next user-chosen commit; touches paths the user may have other WIP for |
| `local/docs/BUILD-SYSTEM-FINGERPRINT-HARDENING-PLAN.md` | User WIP plan (Phase 6+, "pending Oracle fingerprint architecture review") | Draft, not for committing in this arc |
| User's `AGENTS.md`, `local/AGENTS.md`, `README.md`, `config/redbear-*.toml`, `local/sources/{base,bootloader,kernel}` | 7,000+ modifications | User WIP; not in this arc |