a123bf1c5dc0f944196dd5be79d0d1d2e6d1f8cc
10 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 vasilito
|
ffbbf4935c |
C-7 cleanup: lint-recipe down from 13 to 4 errors
The 4 remaining errors are out of C-7 scope:
- sddm (19 seds): needs separate migration to
`local/patches/sddm/` (already partly done via
the drop-x11.py approach in the sddm 0.21.0 work)
- qtbase (2 seds): needs separate migration
- redbear-sessiond: missing patch file (pre-existing
user WIP, not introduced by C-7)
- libwayland: missing patch file (pre-existing user
WIP, this is a re-add of the line C-1 removed)
Changes in this commit:
1. `local/scripts/lint-recipe.py`: R2-INLINE-SED rule now
distinguishes upstream-source seds (target
`${COOKBOOK_SOURCE}/` or `find "${COOKBOOK_SOURCE}``)
from build-time seds (target `${COOKBOOK_STAGE}/`,
`${COOKBOOK_BUILD}/`, `${COOKBOOK_SYSROOT}/`, or
non-source paths). Build-time seds are exempt
because they're adjustments to staged artifacts,
not upstream source edits — the Rule 2 concern is
upstream-source durability.
This is a non-trivial refinement of the R2 rule
because the original implementation flagged every
`sed -i` regardless of target. Several recipes
(bison, m4, rust-native, kf6-kded6, kf6-kbookmarks)
had build-time seds mixed with upstream-source
seds; the previous rule would force a migration
for a build-time Makefile edit which doesn't
actually fit the rule's intent.
2. `local/scripts/tests/test_lint_recipe.py`: updated
the 2 R2 fixture tests to use the upstream-source
path (`${COOKBOOK_SOURCE}/file.c`) so they actually
trigger R2. Added 1 new test
(`test_build_time_seds_are_exempt`) that verifies
build-time seds targeting `${COOKBOOK_STAGE}`,
`${COOKBOOK_BUILD}`, `${COOKBOOK_SYSROOT}` are
correctly exempt. 25/25 lint tests pass.
3. `local/recipes/kde/breeze/recipe.toml`: deleted the
lone `sed -i '/include(ECMQmlModule)/s/^/#/'` line.
`ECMQmlModule` is not in upstream 6.6.5, so the sed
was a no-op (dead code per the zero-tolerance
policy on stubs and workarounds).
4. `local/recipes/kde/kde-cli-tools/recipe.toml`: deleted
the `sed -i 's/^add_subdirectory(kdesu/#...'` line.
The regex is BROKEN — it has `^add_subdirectory(kdesu/`
but the upstream line is ` add_subdirectory(kdesu)`
(with a `)`, not `/`). The sed was a no-op.
The kdesu subdir has been building all along.
5. `local/recipes/kde/kf6-kbookmarks/recipe.toml`: deleted
the 2 ecm/ki18n seds (NO-OPs — line not in upstream
6.26.0) and the broken `find_package(Qt6GuiPrivate)`
injection (regex typo: `Widgets)` requires a closing
paren but upstream has `Widgets Xml)`). Remaining
2 seds target `${COOKBOOK_SYSROOT}` (build-time,
exempt per the new R2 rule).
6. `local/scripts/cleanup-kf6-noop-seds-targeted.sh`:
added `kf6-kbookmarks` to the recipe list. It was
missed in the original 24-recipe cleanup (the
initial list was derived from the NO-OP classifier
but kf6-kbookmarks' 2 sysroot seds made the
classifier put it in the 'has-sysroot' bucket).
Now caught by the targeted cleanup.
|
||
|
vasilito
|
e3e1faece6 |
test-cookbook-apply-patches-e2e: 4 integration tests for the cookbook helper
Extracts the cookbook_apply_patches function from
src/cook/script.rs and runs it against the real
kf6-karchive source + migration patch. Verifies the
end-to-end flow that C-7 step 2 relies on:
1. First apply: helper applies the patch via
`git apply` from inside ${COOKBOOK_SOURCE},
reports `applying 01-initial-migration.patch`
and `applied=1 skipped=0 failed=0`.
2. Idempotency: running the helper a second time
detects the patch is already applied via
`git apply --reverse --check` and reports
`already applied, skipping`.
3. Post-patch source state: the helper actually
modifies the source — verifies that
`ecm_install_po_files_as_qm` is now commented
out (line starts with `#`) in the source after
the helper runs.
4. 4-level path resolution: verifies that the
`${COOKBOOK_RECIPE}/../../../../local/patches/<name>`
path used in the recipe's [build].script
resolves to the actual patches dir
`local/patches/kf6-karchive`.
These tests use a real pristine/source/patch fixture
(not mocks) and run the actual cookbook helper
extracted from src/cook/script.rs. Any change to
the helper's behavior (path handling, idempotency
check, git apply flags) is caught by these tests.
Makefile:
- New `test-cookbook-apply-patches-e2e` target
- Added to `lint-build-system-all` aggregate
Total: 4 new tests, 164 Python tests total (160 + 4).
All 10 test files pass in <1s.
|
||
|
vasilito
|
4243beb4ae |
test-edit-kf6-recipes: 11 unit tests for the edit script heredoc
Tests the python heredoc that is the meat of
`edit-kf6-recipes-for-patches.sh` — the script that
replaces every `sed -i ...` chain in a recipe's
[build].script with a single cookbook_apply_patches
call.
Test fixtures:
- Single-line sed
- Multi-line sed with `\\` continuation
- 3 separate sed chains (verifies cookbook_apply_patches
is inserted ONCE even when multiple seds are removed)
- Chained `&& cd ...` sed
- No-sed baseline (text unchanged)
- 4-level path verification
- Real kf6-karchive recipe (4 sed chains, all removed)
TestScriptStructure checks:
- Script exists and is executable
- Script targets all 29 recipes with migration patches
- Script uses 4-level path (../ x4) for KF6 recipes
- Script skips already-migrated recipes (idempotency)
Makefile:
- New `test-edit-kf6-recipes` target
- Added to `lint-build-system-all` aggregate
Total: 11 new tests, 160 Python tests total (149 +
11).
|
||
|
vasilito
|
9a3c380e2a |
test-cleanup-noop-seds: 9 unit tests for sed-chain cleanup heredoc
Validates the python heredoc inside `local/scripts/cleanup-kf6-noop-seds.sh`. The heredoc is the meat of the script — it walks each `sed -i ...` line plus any `\\` or `&& cd ...` continuations and deletes them as a single chain. The test fixtures cover: - single-line sed - multi-line sed with `\\` continuation - chained seds with `&& cd ...` continuations - no-sed baseline (text unchanged) - actual kf6-attica recipe excerpt (5 sed lines, all gone) Also adds TestScriptStructure checks: - script exists and is executable - script lists all 24 NO-OP recipes - script makes a timestamped backup - script handles `\\` continuations Makefile: - new `test-cleanup-noop-seds` target - added to `lint-build-system-all` aggregate - .PHONY target list updated 132 Python tests total (was 124, +8 new). All 8 test files pass in <1s. |
||
|
vasilito
|
bd3550840f |
kf6-kwindowsystem: C-7 migration patch (ecm_install_po_files_as_qm) + script exclude
Second durable C-7 migration patch. Captures the inline sed chain in kf6-kwindowsystem's [build].script that comments out the `ecm_install_po_files_as_qm(poqm)` line in CMakeLists.txt. The patch is a 16-line single-file edit (CMakeLists.txt only) with no autogenerated noise. Script change: the diff command now also excludes `--exclude='.clang-format'` and `--exclude='.gitignore'`, which ECM (Extra CMake Modules) writes on every cmake configure step. Without these excludes, the patch would be ~120 lines of ECM autogenerated noise with the real Red Bear edit buried in the middle. This is a regression fix for kf6-kwindowsystem which had a clean real diff hidden under 95+ lines of clang-format config. Adds test_diff_excludes_ecm_generated_files (124 Python tests total, 17 in this file). All 7 test files pass. Migration status: 2/56 KF6 recipes migrated to external patches (kf6-karchive, kf6-kwindowsystem). The remaining 54 recipes will be migrated as their cook+diff completes; the migration script is now runnable end-to-end with no manual filtering required. |
||
|
vasilito
|
07f924fe09 |
migrate-kf6-seds: add 600s timeout on per-recipe cook (C-7)
Several KF6 recipes (kf6-kauth, kf6-kconfig, kf6-kwidgetsaddons) use autotools and their `autoreconf` step can take 5+ minutes on a clean cook. Without a per-recipe timeout, a hung cook blocks the migration script indefinitely and leaves `source-pristine/` snapshots lingering on disk. The sed chain we care about runs in the recipe's [build].script BEFORE the configure step, so a 10-minute window is plenty. The snapshot at step 2 is already on disk, so even if the cook is killed by the timeout, the post-cook source state is still useful for the diff. Adds test_cook_has_timeout regression test (123 Python tests total). All 7 test files pass. |
||
 kellito
|
b8c1c780dc |
build: ship first C-7 KF6 sed migration patch (kf6-karchive)
First durable artifact from the C-7 KF6 sed migration: the
inline sed -i chains in local/recipes/kde/kf6-karchive's
[build].script have been captured as a durable external
patch in local/patches/kf6-karchive/01-initial-migration.patch.
This patch was generated by running the v2 migration
script (commit
|
||
|
kellito
|
0f8ad8a50d |
build: ship scratch-rebuild skeleton + 21 tests (improvement #10 partial)
L-sized improvement #10 (cookbook scratch-rebuild) is now PARTIALLY shipped: the M-sized foundation is a runnable script that does the right thing in the common case. Verification against real cascades + integration with rebuild-cascade.sh remains for a separate session. local/scripts/scratch-rebuild.sh (190 lines, +x): Step 1: discover autotools-using recipes by content regex (aclocal|autoreconf|libtoolize|automake|autoconf|gettextize|./configure) PLUS the AUTOTOOLS_CORE list (m4, autoconf, automake, libtool, bison, flex, gettext) which are always-included because they are autotools infrastructure even if they don't directly invoke aclocal. Step 2: compute transitive closure via BFS over the recipe TOML dep graph, including both [build].dependencies and [build].dev_dependencies. Found 6 autotools users in the live tree: bison, diffutils, flex, grub, libtool, m4. Step 3: for each recipe in the closure, delete target/<arch>/{build,sysroot,stage.tmp}/ — PRESERVE source/ so we don't re-fetch the upstream tar. Step 4: re-cook in dep order with --jobs=N (default 4) so the rebuild itself runs in parallel via the dep-aware scheduler (#1). Cook errors during Step 4 do NOT abort the script with exit 1 — a failed cook may indicate a missing upstream dep (legitimate on a fresh checkout) rather than a real bug. The user inspects the log and re-runs after addressing the dep. This is documented in the header + Step 4 comment. Supports --dry-run, --jobs=N, --help. Env overrides for RECIPES_DIR + LOG_DIR (mirroring the migration script's test escape hatch pattern, used by the test suite below). 21 unit tests in local/scripts/tests/test_scratch_rebuild.py: TestAutotoolsCoreList (3) — m4, libtool, bison/flex in AUTOTOOLS_CORE TestAutotoolsContentRegex (8) — catches each canonical autotools command; does NOT match cmake/make/meson TestRecipeDepParsing (4) — parses dependencies and dev_dependencies; both; neither TestScriptHelp (1) — --help describes the script TestScriptStructure (5) — executable bit; uses ./target/release/repo; PRESERVES source/; uses --jobs=N; dry-run safe Test count: 99 -> 120 (all in <1s). The test file also surfaces a real Python regex gotcha: `^[[:space:]]*` (POSIX char class with quantifier) silently fails to match the empty string under Python's regex engine, while `^[\s]*` (shorthand) works correctly. The test regex uses the shorthand to avoid this. Wired into: make test-scratch-dry-run -> scratch-rebuild.sh --dry-run Gitea Actions job scratch-dry-run (job 6 of 10, every PR) With this commit, 9 of 10 build-system improvements in BUILD-SYSTEM-IMPROVEMENTS.md are DONE (1 PARTIAL on #10); the remaining 1 is #7A (QML gate, Qt6 engine fix, not a cookbook improvement). Verified: `./local/scripts/scratch-rebuild.sh --dry-run` correctly discovers 6 autotools users and computes the 6-recipe closure. `make test-lint-scripts` still passes 120/120 tests in <1s. Gitea workflow YAML validates with 10 jobs total (was 9). |
||
|
kellito
|
827895d32f |
build: rewrite C-7 KF6 sed migration script + add 13 tests
The C-7 KF6 sed migration script shipped in commit
|
||
|
kellito
|
ae749ffb23 |
build: ship build-system hardening arc (5 of 10 improvements)
The v6.0 build-system hardening arc lands 5 of the 10 improvements
proposed in local/docs/BUILD-SYSTEM-IMPROVEMENTS.md. All scripts
have unit tests (62 -> 86, all pass in <1s) and the new 'lint-recipe'
Gitea Actions job runs on every PR.
Per-recipe audit & lint scripts (catch R1/R2 violations BEFORE cook):
* audit-patch-idempotency.py — verifies external patches in
local/patches/ still apply against the upstream pinned rev.
Caught 1 real bug on first run: libdrm/02-redox-dispatch.patch
hunk at xf86drm.c:321 no longer matches libdrm-2.4.125.
* audit-kf6-deps.py — fetches upstream, scans for
find_package(KF6Xxx REQUIRED), compares to recipe deps. Catches
missing + dead dependencies in every kf6-* and qt* recipe.
* classify-cook-failure.py — 17-rule cook-failure classifier.
10-30s diagnosis vs 5-10min manual. exit code is intentionally
inverted (0=novel failure, 1=known fix) for CI signal.
* lint-recipe.py — 7-rule recipe lint: R1-NO-PATCH-FILE,
R1-PATH-SOURCE, R2-INLINE-SED, R2-PATCHES-DIR-UNUSED,
NO-LEGACY-MAKE, R1-LEGACY-APPLY-PATCHES, DEP-NOT-FOUND.
1.1s for 171 recipes (down from 60s+ in v1 via recipe-index
precomputation). Strict mode promotes warnings to errors.
Build-system convenience:
* repair-cook.sh — incremental-build optimizer.
Equivalent to 'repo cook <pkg>' but with a fast-path that
skips configure when CMakeCache.txt is newer than source AND
external patches haven't changed. 30-60s vs 5-10min on KF6
recipes. make repair.<pkg> / make clean-repair.<pkg> targets.
* migrate-kf6-seds-to-patches.sh — migration skeleton for
converting 56 inline 'sed -i' chains across the KF6 recipes
to durable external patches in local/patches/<name>/.
Gitea Actions (host-execution, no Docker):
* .gitea/workflows/build-system.yml — 8-job pipeline:
unit-tests, lint-offline, lint-network (nightly),
lint-recipe (NEW), lint-docs, build-mini, build-full,
smoke (QEMU boot).
* .gitea/RUNNER-SETUP.md — one-time Manjaro/Arch host setup.
Build script hardening:
* build-redbear.sh — when a low-level source (relibc,
kernel, base, bootloader, installer) is newer than its pkgar,
clean build/ and sysroot/ across all recipes too. Low-level
package changes leave autotools packages (pcre2, gettext,
libiconv, ...) with stale configure/libtool scripts referencing
the old runtime, causing 'libtool version mismatch' and
'not a valid libtool object' errors. Cleaning forces
re-configuration; stage/ and source/ are preserved so the
cookbook skips unchanged packages that don't use autotools.
* Makefile — wire lint-cook-failure,
lint-cook-failure-explain, lint-recipe, lint-recipe.%,
lint-recipe.strict, lint-recipe.%.strict, repair.%,
clean-repair.%, test-lint-scripts[-quiet]. Replace the
legacy 'validate-patches' target with a deprecation notice
pointing at validate-sources.
Documentation:
* BUILD-SYSTEM-IMPROVEMENTS.md — mark #2 and #5 DONE; full
implementation notes; updated Make-targets table.
* BUILD-SYSTEM-V6-HARDENING-POSTMORTEM.md (NEW) — 226-line durable
record of the 8-session arc: 32 findings categorized, 5 P0
audit-script bugs fixed, 6 over-broad multi-pattern rules
discovered + fixed, test coverage 86/86 in <1s, 7/10
improvements DONE.
* SCRIPT-BEHAVIOR-MATRIX.md — apply-patches.sh row marked
LEGACY/ARCHIVED; build-redbear.sh row no longer claims to
call it.
* boot-logs/README.md (NEW) — frozen-evidence policy:
'do not edit' rule for REDBEAR-FULL-BOOT-*-RESULTS.md files.
* libdrm/02-redox-dispatch.patch.README (NEW) — 8-step regen
procedure for the broken hunk.
Cleanup:
* local/cache/README.md deleted (1-line placeholder).
* legacy 'make validate-patches' target removed.
Per build-system improvement #5: lint-recipe.py's first run on
the live tree surfaced 1 broken-patch reference (redbear-sessiond),
1 dangling cookbook_apply_patches call (tc), 19 sed -i calls in
sddm (warning — cookbook_apply_patches present, drop-x11.py
migration in progress), 4 sed -i calls in qt6-wayland-smoke
(uncovers the same bug class the libwayland fix prevented).
|