vasilito/RedBear-OS
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,845 Commits 19 Branches 0 Tags
b8c1c780dc9f59c19feb94cfa5702667cb103a7e
Commit Graph

4 Commits

Author SHA1 Message Date
kellito b8c1c780dc build: ship first C-7 KF6 sed migration patch (kf6-karchive) 
First durable artifact from the C-7 KF6 sed migration: the
inline sed -i chains in local/recipes/kde/kf6-karchive's
[build].script have been captured as a durable external
patch in local/patches/kf6-karchive/01-initial-migration.patch.

This patch was generated by running the v2 migration
script (commit 827895d32) against the live kf6-karchive
recipe. The actual sed edits captured are:

  -ecm_install_po_files_as_qm(poqm)
  +#ecm_install_po_files_as_qm(poqm)

The other 3 sed chains in the recipe (ki18n_install(po),
.arg(mode), .arg(d->mode)) were no-ops against the karchive
6.26.0 upstream tar (the target lines either no longer
exist or are already in the desired state in this upstream
version). The migration script correctly captures only the
real edits; no-ops produce no patch hunks.

Script fix in this commit:

The migration script's v2 was producing silently empty
diffs on already-cooked recipes because the cookbook's
`fetch` re-uses an existing source/ tree if it finds one
(it does this to avoid re-extracting tars on every fetch).
For C-7 migration we need the truly pristine upstream
state. The fix:
  1. Add an explicit `unfetch` step BEFORE the `fetch`
     (so the source/ dir is removed before re-extraction)
  2. Set `REDBEAR_ALLOW_LOCAL_UNFETCH=1` because kf6-*
     and qt* recipes are local-overlay recipes under
     local/recipes/, and the cookbook's default policy is
     to never clobber a local-overlay source (the env var
     overrides that policy for the migration's explicit
     unfetch call only)
  3. Apply the same env var to the post-capture `unfetch`
     at the end of the script

The script header documents this cookbook behavior with
inline comments so a future contributor doesn't re-introduce
the silent-failure mode.

Patch filter:

The migration script's diff includes ECM-autogenerated
files like .clang-format that aren't real sed edits. The
captured patch was 122 lines, of which 95 were the
.clang-format autogeneration. The committed patch is the
filtered 24-line version that drops `.clang-format`,
`.gitignore`, and any `target/` artifacts. (A future
script improvement could do this filter inline.)

Test count: 120 -> 122 (2 new tests in test_migrate_kf6_seds.py):
  - test_sets_local_unfetch_env_var: regression guard
    against forgetting the env var
  - test_unfetches_before_fetching: regression guard
    against calling fetch before unfetch (silent-failure
    mode in v2)

Next steps for kf6-karchive specifically (manual, not part
of this commit):
  1. Edit local/recipes/kde/kf6-karchive/recipe.toml's
     [build].script to remove the 4 inline sed -i chains
     and add:
         REDBEAR_PATCHES_DIR="local/patches/kf6-karchive"
         cookbook_apply_patches "${REDBEAR_PATCHES_DIR}"
  2. Cook again to verify the patch + rewritten script
     produce a byte-identical stage.pkgar
  3. Commit the recipe rewrite + the patch together

Verified:
  - The migration ran end-to-end on the live tree
  - The patch applies cleanly to the pristine upstream
  - 122/122 Python tests pass
  - The new test_sets_local_unfetch_env_var and
    test_unfetches_before_fetching both pass

C-7 status: 1 of 56 KF6 sed-bearing recipes migrated.
55 remaining (next: kf6-attica has the smallest sed chain;
after that, breeze, kf6-syntaxhighlighting).
 2026-06-12 17:05:46 +03:00
kellito 0f8ad8a50d build: ship scratch-rebuild skeleton + 21 tests (improvement #10 partial) 
L-sized improvement #10 (cookbook scratch-rebuild) is now
PARTIALLY shipped: the M-sized foundation is a runnable
script that does the right thing in the common case.
Verification against real cascades + integration with
rebuild-cascade.sh remains for a separate session.

local/scripts/scratch-rebuild.sh (190 lines, +x):
  Step 1: discover autotools-using recipes by content regex
    (aclocal|autoreconf|libtoolize|automake|autoconf|gettextize|./configure)
    PLUS the AUTOTOOLS_CORE list (m4, autoconf, automake,
    libtool, bison, flex, gettext) which are always-included
    because they are autotools infrastructure even if they
    don't directly invoke aclocal.
  Step 2: compute transitive closure via BFS over the recipe
    TOML dep graph, including both [build].dependencies and
    [build].dev_dependencies. Found 6 autotools users in the
    live tree: bison, diffutils, flex, grub, libtool, m4.
  Step 3: for each recipe in the closure, delete
    target/<arch>/{build,sysroot,stage.tmp}/ — PRESERVE source/
    so we don't re-fetch the upstream tar.
  Step 4: re-cook in dep order with --jobs=N (default 4) so
    the rebuild itself runs in parallel via the dep-aware
    scheduler (#1).

Cook errors during Step 4 do NOT abort the script with
exit 1 — a failed cook may indicate a missing upstream dep
(legitimate on a fresh checkout) rather than a real bug.
The user inspects the log and re-runs after addressing the
dep. This is documented in the header + Step 4 comment.

Supports --dry-run, --jobs=N, --help. Env overrides for
RECIPES_DIR + LOG_DIR (mirroring the migration script's
test escape hatch pattern, used by the test suite below).

21 unit tests in local/scripts/tests/test_scratch_rebuild.py:
  TestAutotoolsCoreList (3)         — m4, libtool, bison/flex
                                     in AUTOTOOLS_CORE
  TestAutotoolsContentRegex (8)     — catches each canonical
                                     autotools command; does
                                     NOT match cmake/make/meson
  TestRecipeDepParsing (4)          — parses dependencies and
                                     dev_dependencies; both;
                                     neither
  TestScriptHelp (1)                — --help describes the
                                     script
  TestScriptStructure (5)           — executable bit; uses
                                     ./target/release/repo;
                                     PRESERVES source/; uses
                                     --jobs=N; dry-run safe

Test count: 99 -> 120 (all in <1s).

The test file also surfaces a real Python regex gotcha:
`^[[:space:]]*` (POSIX char class with quantifier) silently
fails to match the empty string under Python's regex
engine, while `^[\s]*` (shorthand) works correctly. The
test regex uses the shorthand to avoid this.

Wired into:
  make test-scratch-dry-run  ->  scratch-rebuild.sh --dry-run
  Gitea Actions job scratch-dry-run (job 6 of 10, every PR)

With this commit, 9 of 10 build-system improvements in
BUILD-SYSTEM-IMPROVEMENTS.md are DONE (1 PARTIAL on #10);
the remaining 1 is #7A (QML gate, Qt6 engine fix, not a
cookbook improvement).

Verified: `./local/scripts/scratch-rebuild.sh --dry-run`
correctly discovers 6 autotools users and computes the
6-recipe closure. `make test-lint-scripts` still passes
120/120 tests in <1s. Gitea workflow YAML validates with
10 jobs total (was 9).
 2026-06-12 16:12:49 +03:00
kellito 827895d32f build: rewrite C-7 KF6 sed migration script + add 13 tests 
The C-7 KF6 sed migration script shipped in commit ae749ffb2
was a stub with three structural problems that made it
unrunnable:
  1. Called 'repo cook $recipe_dir' with a path, but the
     cookbook CLI takes bare names — this would have failed
     with 'Package name invalid' on first run.
  2. Step 2 created an empty pristine_dir via mktemp -d but
     never populated it, so the diff was always empty
     (zero-byte output, 'no diff' branch taken, no patch
     written).
  3. Step 4 was 'SKIP — manual rewrite pending', so the
     script wrote no patch even when the inline sed chains
     actually edited the source.

Replace the stub with a working v2 that:
  - Uses 'repo cook $name' (bare names) throughout
  - Snapshots source/ → source-pristine/ BEFORE the cook
    so the pristine state is real, not empty
  - Runs the full cook (with -i || true so a build failure
    after the sed step doesn't abort the migration — we
    only need the post-sed source state)
  - diffs the real pristine vs post-cook tree, with
    --exclude='.git' and --exclude='target' so the diff
    is the actual sed edits
  - Saves the diff as
    local/patches/<name>/01-initial-migration.patch with
    a header explaining provenance and the cookbook_apply_patches
    invocation the recipe should use
  - Cleans up source-pristine/ + runs 'repo unfetch $name' so
    the next migration run starts from a clean slate

Add a --dry-run mode that lists candidates without fetching,
for safe CI / smoke testing. Add --recipe=<name> and
--limit=N for targeted runs. Add --help.

Add a test escape hatch via REDBEAR_MIGRATE_RECIPES_DIR and
REDBEAR_MIGRATE_PATCHES_DIR env vars so the candidate
discovery can be exercised on a synthetic tree without
touching the live project. Also gate the cookbook-binary
check on DRY_RUN != 1 so --dry-run doesn't require a
pre-built ./target/release/repo.

13 unit tests in local/scripts/tests/test_migrate_kf6_seds.py:
  TestCandidateDiscovery (7):
    - discovers sed+tar recipe
    - skips recipe without sed
    - skips recipe with git source (Rule 1 in-tree, not
      sed-migration candidates)
    - --limit=N caps results
    - --recipe=<name> filters
    - existing patch triggers SKIP branch (via static analysis)
    - --help output describes the script
  TestScriptStructure (6):
    - regression: uses bare names, not paths
    - uses release/repo binary
    - creates patches dir
    - diff includes .git/target excludes
    - unfetches after capture
    - idempotent SKIP when patch exists

Test count: 86/86 → 99/99 (all in <1s).

The actual migration run still requires the full KF6 dep
chain to be built (qtbase, qtdeclarative, kf6-extra-cmake-modules,
plus the recipe's own deps). The 56 recipes are now
discoverable + scriptable; the recipe-by-recipe verification
+ patch validity check remains a per-recipe manual step
(open the patch, confirm the diff matches the inline sed
chain, edit [build].script to call cookbook_apply_patches,
re-cook, byte-compare stage.pkgar).
 2026-06-12 15:37:58 +03:00
kellito ae749ffb23 build: ship build-system hardening arc (5 of 10 improvements) 
The v6.0 build-system hardening arc lands 5 of the 10 improvements
proposed in local/docs/BUILD-SYSTEM-IMPROVEMENTS.md. All scripts
have unit tests (62 -> 86, all pass in <1s) and the new 'lint-recipe'
Gitea Actions job runs on every PR.

Per-recipe audit & lint scripts (catch R1/R2 violations BEFORE cook):
  * audit-patch-idempotency.py  — verifies external patches in
    local/patches/ still apply against the upstream pinned rev.
    Caught 1 real bug on first run: libdrm/02-redox-dispatch.patch
    hunk at xf86drm.c:321 no longer matches libdrm-2.4.125.
  * audit-kf6-deps.py           — fetches upstream, scans for
    find_package(KF6Xxx REQUIRED), compares to recipe deps. Catches
    missing + dead dependencies in every kf6-* and qt* recipe.
  * classify-cook-failure.py    — 17-rule cook-failure classifier.
    10-30s diagnosis vs 5-10min manual. exit code is intentionally
    inverted (0=novel failure, 1=known fix) for CI signal.
  * lint-recipe.py              — 7-rule recipe lint: R1-NO-PATCH-FILE,
    R1-PATH-SOURCE, R2-INLINE-SED, R2-PATCHES-DIR-UNUSED,
    NO-LEGACY-MAKE, R1-LEGACY-APPLY-PATCHES, DEP-NOT-FOUND.
    1.1s for 171 recipes (down from 60s+ in v1 via recipe-index
    precomputation). Strict mode promotes warnings to errors.

Build-system convenience:
  * repair-cook.sh              — incremental-build optimizer.
    Equivalent to 'repo cook <pkg>' but with a fast-path that
    skips configure when CMakeCache.txt is newer than source AND
    external patches haven't changed. 30-60s vs 5-10min on KF6
    recipes. make repair.<pkg> / make clean-repair.<pkg> targets.
  * migrate-kf6-seds-to-patches.sh — migration skeleton for
    converting 56 inline 'sed -i' chains across the KF6 recipes
    to durable external patches in local/patches/<name>/.

Gitea Actions (host-execution, no Docker):
  * .gitea/workflows/build-system.yml — 8-job pipeline:
    unit-tests, lint-offline, lint-network (nightly),
    lint-recipe (NEW), lint-docs, build-mini, build-full,
    smoke (QEMU boot).
  * .gitea/RUNNER-SETUP.md — one-time Manjaro/Arch host setup.

Build script hardening:
  * build-redbear.sh            — when a low-level source (relibc,
    kernel, base, bootloader, installer) is newer than its pkgar,
    clean build/ and sysroot/ across all recipes too. Low-level
    package changes leave autotools packages (pcre2, gettext,
    libiconv, ...) with stale configure/libtool scripts referencing
    the old runtime, causing 'libtool version mismatch' and
    'not a valid libtool object' errors. Cleaning forces
    re-configuration; stage/ and source/ are preserved so the
    cookbook skips unchanged packages that don't use autotools.
  * Makefile                    — wire lint-cook-failure,
    lint-cook-failure-explain, lint-recipe, lint-recipe.%,
    lint-recipe.strict, lint-recipe.%.strict, repair.%,
    clean-repair.%, test-lint-scripts[-quiet]. Replace the
    legacy 'validate-patches' target with a deprecation notice
    pointing at validate-sources.

Documentation:
  * BUILD-SYSTEM-IMPROVEMENTS.md   — mark #2 and #5 DONE; full
    implementation notes; updated Make-targets table.
  * BUILD-SYSTEM-V6-HARDENING-POSTMORTEM.md (NEW) — 226-line durable
    record of the 8-session arc: 32 findings categorized, 5 P0
    audit-script bugs fixed, 6 over-broad multi-pattern rules
    discovered + fixed, test coverage 86/86 in <1s, 7/10
    improvements DONE.
  * SCRIPT-BEHAVIOR-MATRIX.md   — apply-patches.sh row marked
    LEGACY/ARCHIVED; build-redbear.sh row no longer claims to
    call it.
  * boot-logs/README.md (NEW)   — frozen-evidence policy:
    'do not edit' rule for REDBEAR-FULL-BOOT-*-RESULTS.md files.
  * libdrm/02-redox-dispatch.patch.README (NEW) — 8-step regen
    procedure for the broken hunk.

Cleanup:
  * local/cache/README.md deleted (1-line placeholder).
  * legacy 'make validate-patches' target removed.

Per build-system improvement #5: lint-recipe.py's first run on
the live tree surfaced 1 broken-patch reference (redbear-sessiond),
1 dangling cookbook_apply_patches call (tc), 19 sed -i calls in
sddm (warning — cookbook_apply_patches present, drop-x11.py
migration in progress), 4 sed -i calls in qt6-wayland-smoke
(uncovers the same bug class the libwayland fix prevented).
 2026-06-12 13:37:39 +03:00