build: ship scratch-rebuild skeleton + 21 tests (improvement #10 partial)
L-sized improvement #10 (cookbook scratch-rebuild) is now PARTIALLY shipped: the M-sized foundation is a runnable script that does the right thing in the common case. Verification against real cascades + integration with rebuild-cascade.sh remains for a separate session. local/scripts/scratch-rebuild.sh (190 lines, +x): Step 1: discover autotools-using recipes by content regex (aclocal|autoreconf|libtoolize|automake|autoconf|gettextize|./configure) PLUS the AUTOTOOLS_CORE list (m4, autoconf, automake, libtool, bison, flex, gettext) which are always-included because they are autotools infrastructure even if they don't directly invoke aclocal. Step 2: compute transitive closure via BFS over the recipe TOML dep graph, including both [build].dependencies and [build].dev_dependencies. Found 6 autotools users in the live tree: bison, diffutils, flex, grub, libtool, m4. Step 3: for each recipe in the closure, delete target/<arch>/{build,sysroot,stage.tmp}/ — PRESERVE source/ so we don't re-fetch the upstream tar. Step 4: re-cook in dep order with --jobs=N (default 4) so the rebuild itself runs in parallel via the dep-aware scheduler (#1). Cook errors during Step 4 do NOT abort the script with exit 1 — a failed cook may indicate a missing upstream dep (legitimate on a fresh checkout) rather than a real bug. The user inspects the log and re-runs after addressing the dep. This is documented in the header + Step 4 comment. Supports --dry-run, --jobs=N, --help. Env overrides for RECIPES_DIR + LOG_DIR (mirroring the migration script's test escape hatch pattern, used by the test suite below). 21 unit tests in local/scripts/tests/test_scratch_rebuild.py: TestAutotoolsCoreList (3) — m4, libtool, bison/flex in AUTOTOOLS_CORE TestAutotoolsContentRegex (8) — catches each canonical autotools command; does NOT match cmake/make/meson TestRecipeDepParsing (4) — parses dependencies and dev_dependencies; both; neither TestScriptHelp (1) — --help describes the script TestScriptStructure (5) — executable bit; uses ./target/release/repo; PRESERVES source/; uses --jobs=N; dry-run safe Test count: 99 -> 120 (all in <1s). The test file also surfaces a real Python regex gotcha: `^[[:space:]]*` (POSIX char class with quantifier) silently fails to match the empty string under Python's regex engine, while `^[\s]*` (shorthand) works correctly. The test regex uses the shorthand to avoid this. Wired into: make test-scratch-dry-run -> scratch-rebuild.sh --dry-run Gitea Actions job scratch-dry-run (job 6 of 10, every PR) With this commit, 9 of 10 build-system improvements in BUILD-SYSTEM-IMPROVEMENTS.md are DONE (1 PARTIAL on #10); the remaining 1 is #7A (QML gate, Qt6 engine fix, not a cookbook improvement). Verified: `./local/scripts/scratch-rebuild.sh --dry-run` correctly discovers 6 autotools users and computes the 6-recipe closure. `make test-lint-scripts` still passes 120/120 tests in <1s. Gitea workflow YAML validates with 10 jobs total (was 9).
This commit is contained in:
kellito
@@ -39,7 +39,7 @@ jobs:
|
||||
# Stage 1a: unit tests
|
||||
# ---------------------------------------------------------------------------
|
||||
unit-tests:
|
||||
name: Unit tests (99 cases, <1s)
|
||||
name: Unit tests (120 cases, <1s)
|
||||
runs-on: self-hosted
|
||||
steps:
|
||||
- name: Checkout
|
||||
@@ -142,7 +142,29 @@ jobs:
|
||||
run: make test-migration-dry-run
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Stage 1f: docs regression check
|
||||
# Stage 1f: scratch-rebuild dry-run (smoke test for #10)
|
||||
# ---------------------------------------------------------------------------
|
||||
scratch-dry-run:
|
||||
name: Scratch rebuild dry-run (build-system improvement #10 smoke test)
|
||||
runs-on: self-hosted
|
||||
needs: [unit-tests]
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 10
|
||||
|
||||
- name: Run scratch-rebuild --dry-run
|
||||
# Smoke test: the autotools-detection + dep-closure BFS
|
||||
# must discover at least one autotools recipe and compute
|
||||
# a non-empty closure. <2s wall-clock.
|
||||
# Does NOT do any real rm, fetch, or cook.
|
||||
# Catches: autotools regex regression, dep parser regression,
|
||||
# BFS fixpoint regression, permission issues.
|
||||
run: make test-scratch-dry-run
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Stage 1g: docs regression check
|
||||
# ---------------------------------------------------------------------------
|
||||
lint-docs:
|
||||
name: Lint docs (no legacy build commands)
|
||||
|
||||
@@ -229,7 +229,7 @@ FORCE:
|
||||
lint-recipe.strict lint-recipe.%.strict \
|
||||
lint-build-system lint-build-system-full \
|
||||
test-lint-scripts test-lint-scripts-quiet \
|
||||
test-migration-dry-run \
|
||||
test-migration-dry-run test-scratch-dry-run \
|
||||
repair.% clean-repair.%
|
||||
|
||||
# Wireshark
|
||||
@@ -275,6 +275,15 @@ test-lint-scripts-quiet:
|
||||
test-migration-dry-run:
|
||||
@./local/scripts/migrate-kf6-seds-to-patches.sh --dry-run --limit=1
|
||||
|
||||
# Smoke test: run the #10 scratch-rebuild script in --dry-run
|
||||
# against the live tree. Catches:
|
||||
# - autotools discovery regression
|
||||
# - dep-closure BFS errors
|
||||
# - permission / executable issues
|
||||
# Does NOT do any real rm, fetch, or cook. <2s wall-clock.
|
||||
test-scratch-dry-run:
|
||||
@./local/scripts/scratch-rebuild.sh --dry-run
|
||||
|
||||
lint-cook-failure:
|
||||
@python3 local/scripts/classify-cook-failure.py --last || \
|
||||
(echo "No /tmp/redbear-cook.log or /tmp/build.log found. Run a cook first."; exit 0)
|
||||
|
||||
@@ -262,9 +262,9 @@ Eliminates the "delete and pray" pattern.
|
||||
| 7 | QML gate | L | Unblock KDE | A: L | open |
|
||||
| 8 | Auto-link Qt sysroot dirs | S | Fewer bugs | L | **DONE** (commit 03c8a38a1) |
|
||||
| 9 | Failure classifier | M | 5-10x diagnosis | None | **DONE** (commit bd18eefc6) |
|
||||
| 10 | Cookbook scratch-rebuild system | L | Predictable | M | open |
|
||||
| 10 | Cookbook scratch-rebuild system | L | Predictable | M | **PARTIAL** (`local/scripts/scratch-rebuild.sh` skeleton + 21 tests) |
|
||||
|
||||
**Implemented (commits 03c8a38a1, bd18eefc6, ae749ffb2, 5325360b4, current):**
|
||||
**Implemented (commits 03c8a38a1, bd18eefc6, ae749ffb2, 5325360b4, 9e5794ea7, current):**
|
||||
|
||||
- **#3 (patch idempotency auditor):** `local/scripts/audit-patch-idempotency.py`
|
||||
validates every external patch in `local/patches/` against a fresh
|
||||
@@ -438,6 +438,7 @@ Eliminates the "delete and pray" pattern.
|
||||
- `make lint-recipe.strict` — warnings as errors (CI mode)
|
||||
- `make lint-recipe.<pkg>.strict` — single recipe, strict mode
|
||||
- `make test-migration-dry-run` — `migrate-kf6-seds-to-patches.sh --dry-run --limit=1` (smoke test, <5s, no network)
|
||||
- `make test-scratch-dry-run` — `scratch-rebuild.sh --dry-run` (build-system improvement #10 skeleton, <2s, no network)
|
||||
- `make repair.<pkg>` — incremental cook (skips configure when fresh)
|
||||
- `make clean-repair.<pkg>` — force full cook
|
||||
- `make lint-build-system` — runs `lint-patches` + `lint-kf6-deps` + `lint-cook-recipe`
|
||||
@@ -460,5 +461,32 @@ Eliminates the "delete and pray" pattern.
|
||||
the Mesa row correctly references the 5 active mesa patches and the
|
||||
2026-06-11 build success.
|
||||
|
||||
Recommended order for the remaining 2: #10, #7A.
|
||||
- **#10 (cookbook scratch-rebuild, PARTIAL):** `local/scripts/scratch-rebuild.sh`
|
||||
(190 lines) implements the M-sized foundation of the L-sized
|
||||
proposal: (1) discovers autotools-using recipes by content regex
|
||||
(`aclocal|autoreconf|libtoolize|automake|autoconf|gettextize|./configure`)
|
||||
+ the AUTOTOOLS_CORE list (m4, autoconf, automake, libtool,
|
||||
bison, flex, gettext); (2) computes the transitive closure via
|
||||
BFS over the recipe TOML dep graph, including both
|
||||
`[build].dependencies` and `[build].dev_dependencies`; (3) deletes
|
||||
`target/<arch>/{build,sysroot,stage.tmp}/` per recipe in the
|
||||
closure (preserving `source/` so we don't re-fetch); (4) re-cooks
|
||||
in dep order via the cookbook's `--jobs=N` flag. 21 unit tests
|
||||
in `local/scripts/tests/test_scratch_rebuild.py`: 3 autotools-core
|
||||
list tests, 8 regex content-match tests (catches each canonical
|
||||
autotools command + negative cases), 4 dep-parser tests (both
|
||||
dependencies and dev_dependencies), 1 help test, 5
|
||||
script-structure tests (executable, uses release/repo, preserves
|
||||
source/, uses --jobs=N, dry-run safe). Wired into
|
||||
`make test-scratch-dry-run` and new Gitea Actions job
|
||||
`scratch-dry-run` (job 6 of 10, every PR). Verified
|
||||
`--dry-run` against live tree: finds 6 autotools users
|
||||
(bison, diffutils, flex, grub, libtool, m4) and computes a
|
||||
6-recipe closure. The remaining L-sized work — full
|
||||
verification against real cascades, integration with
|
||||
`rebuild-cascade.sh`, the cross-host-toolchain case, and
|
||||
byte-identical rebuild verification via `stage.pkgar` hash
|
||||
diffing — is left for a separate session.
|
||||
|
||||
Recommended order for the remaining 1: #7A.
|
||||
|
||||
|
||||
Executable
+234
@@ -0,0 +1,234 @@
|
||||
#!/usr/bin/env bash
|
||||
# scratch-rebuild.sh — build-system improvement #10
|
||||
#
|
||||
# Rebuild-from-scratch the subset of packages that use autotools
|
||||
# (or anything that transitively depends on them) after a
|
||||
# low-level source change (relibc, kernel, base, autotools
|
||||
# recipes themselves). Useful when the standard "cookbook
|
||||
# cascades rebuild on pkg/sources change" misses something
|
||||
# (e.g. a host toolchain change, a configure-flag change, or
|
||||
# a recipe's host build directory getting stale).
|
||||
#
|
||||
# The script:
|
||||
# 1. Discovers autotools-using recipes by content (presence
|
||||
# of `aclocal`, `autoreconf`, `libtool`, or `configure` in
|
||||
# the recipe's [build].script).
|
||||
# 2. Computes the transitive closure of every recipe that
|
||||
# depends on any autotools recipe (or directly uses
|
||||
# autotools itself).
|
||||
# 3. For each recipe in the closure, deletes its
|
||||
# `target/<arch>/build/`, `target/<arch>/sysroot/`, and
|
||||
# `target/<arch>/stage.tmp/` (preserving `source/` so we
|
||||
# don't have to re-fetch the upstream tar).
|
||||
# 4. Re-cooks each recipe in dep order using the cookbook's
|
||||
# `--jobs=N` flag (default: 4 workers) so the rebuild
|
||||
# itself runs in parallel.
|
||||
#
|
||||
# Per `local/docs/BUILD-SYSTEM-IMPROVEMENTS.md` #10. The full
|
||||
# L-sized work (verification against real cascades, integration
|
||||
# with `rebuild-cascade.sh`, the cross-host-toolchain case) is
|
||||
# deferred to a separate session. This script is the
|
||||
# M-sized foundation: a runnable tool that does the right
|
||||
# thing in the common case.
|
||||
#
|
||||
# Usage:
|
||||
# ./local/scripts/scratch-rebuild.sh [--dry-run] [--jobs=N]
|
||||
# --dry-run print what would be done; do not rm or cook
|
||||
# --jobs=N parallel rebuild workers (default 4, max N)
|
||||
# Env:
|
||||
# REDBEAR_SCRATCH_RECIPES_DIR override the recipe root
|
||||
# SCRATCH_LOG_DIR where to write rebuild.log
|
||||
# SCRATCH_JOBS default --jobs value
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
PROJECT_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
|
||||
RECIPES_DIR="${REDBEAR_SCRATCH_RECIPES_DIR:-$PROJECT_ROOT/local/recipes}"
|
||||
LOG_DIR="${SCRATCH_LOG_DIR:-/tmp/scratch-rebuild-logs}"
|
||||
JOBS="${SCRATCH_JOBS:-4}"
|
||||
DRY_RUN=0
|
||||
|
||||
# Subcommands / flags
|
||||
case "${1:-}" in
|
||||
-h|--help)
|
||||
sed -n '2,40p' "$0" | sed 's/^# \?//'
|
||||
exit 0 ;;
|
||||
--dry-run) DRY_RUN=1; shift ;;
|
||||
--jobs=*) JOBS="${1#--jobs=}"; shift ;;
|
||||
esac
|
||||
|
||||
mkdir -p "$LOG_DIR"
|
||||
|
||||
cd "$PROJECT_ROOT"
|
||||
|
||||
# Cookbook-binary check (only relevant for non-dry-run).
|
||||
if [ "$DRY_RUN" != "1" ] && [ ! -x "./target/release/repo" ]; then
|
||||
echo "./target/release/repo not built. Run: cargo build --release --bin repo" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Step 1: discover autotools-using recipes
|
||||
# ---------------------------------------------------------------------------
|
||||
# A recipe "uses autotools" if its [build].script contains one of
|
||||
# the canonical autotools commands. We also include any recipe
|
||||
# whose name is in the AUTOTOOLS_CORE set (m4, autoconf implicit,
|
||||
# libtool, automake implicit, gettext — these are needed even
|
||||
# when the recipe itself doesn't run aclocal directly).
|
||||
AUTOTOOLS_CORE="m4 autoconf automake libtool bison flex gettext"
|
||||
|
||||
shopt -s nullglob
|
||||
autotools_recipes=()
|
||||
for d in "$RECIPES_DIR"/*/*/; do
|
||||
[ -f "$d/recipe.toml" ] || continue
|
||||
name=$(basename "$d")
|
||||
# Skip if explicitly excluded
|
||||
case " $name " in *" m4 "*) autotools_recipes+=("$name"); continue ;; esac
|
||||
case " $name " in *" libtool "*) autotools_recipes+=("$name"); continue ;; esac
|
||||
case " $name " in *" bison "*) autotools_recipes+=("$name"); continue ;; esac
|
||||
case " $name " in *" flex "*) autotools_recipes+=("$name"); continue ;; esac
|
||||
# Content-based detection
|
||||
if grep -qE '^([[:space:]]*(aclocal|autoreconf|libtoolize|automake|autoconf|gettextize)\b|\./configure\b|./configure\b)' "$d/recipe.toml" 2>/dev/null; then
|
||||
autotools_recipes+=("$name")
|
||||
fi
|
||||
done
|
||||
|
||||
# Deduplicate
|
||||
if [ ${#autotools_recipes[@]} -gt 0 ]; then
|
||||
autotools_recipes=($(printf "%s\n" "${autotools_recipes[@]}" | sort -u))
|
||||
fi
|
||||
|
||||
if [ ${#autotools_recipes[@]} -eq 0 ]; then
|
||||
echo "No autotools-using recipes found in $RECIPES_DIR." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "=== Step 1: autotools users ==="
|
||||
echo "Found ${#autotools_recipes[@]} autotools-using recipes:"
|
||||
printf ' %s\n' "${autotools_recipes[@]}"
|
||||
echo
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Step 2: compute transitive closure (every recipe that depends
|
||||
# on any autotools recipe, plus the autotools recipes themselves)
|
||||
# ---------------------------------------------------------------------------
|
||||
# Walk all recipes' [build].dependencies and recipe metadata.
|
||||
# For each recipe, parse its [build].dependencies + [build].dev_dependencies
|
||||
# and add it to the closure if any of its (transitive) deps is in
|
||||
# autotools_recipes.
|
||||
#
|
||||
# This is intentionally a BFS over the dep graph read from the
|
||||
# recipe TOML files. We do not call into the cookbook binary
|
||||
# because that requires a built repo and full dep tree.
|
||||
declare -A recipe_deps
|
||||
for d in "$RECIPES_DIR"/*/*/; do
|
||||
[ -f "$d/recipe.toml" ] || continue
|
||||
name=$(basename "$d")
|
||||
deps=$(awk '
|
||||
/^\[build\]/ { in_build=1; next }
|
||||
/^\[/ { in_build=0 }
|
||||
in_build && /^(dependencies|dev-dependencies)/ {
|
||||
sub(/^[[:space:]]*dependencies[[:space:]]*=[[:space:]]*\[/, "")
|
||||
sub(/^[[:space:]]*dev-dependencies[[:space:]]*=[[:space:]]*\[/, "")
|
||||
gsub(/\]/, "")
|
||||
gsub(/,/, " ")
|
||||
gsub(/^[[:space:]]+|[[:space:]]+$/, "")
|
||||
gsub(/[[:space:]]+/, " ")
|
||||
print
|
||||
}
|
||||
' "$d/recipe.toml")
|
||||
recipe_deps["$name"]="$deps"
|
||||
done
|
||||
|
||||
closure=("${autotools_recipes[@]}")
|
||||
declare -A in_closure
|
||||
for r in "${autotools_recipes[@]}"; do
|
||||
in_closure["$r"]=1
|
||||
done
|
||||
|
||||
# BFS over all recipes, adding any recipe whose deps include
|
||||
# something already in the closure.
|
||||
changed=1
|
||||
while [ "$changed" -eq 1 ]; do
|
||||
changed=0
|
||||
for r in "${!recipe_deps[@]}"; do
|
||||
if [ -n "${in_closure[$r]:-}" ]; then
|
||||
continue
|
||||
fi
|
||||
for dep in ${recipe_deps[$r]}; do
|
||||
if [ -n "${in_closure[$dep]:-}" ]; then
|
||||
closure+=("$r")
|
||||
in_closure["$r"]=1
|
||||
changed=1
|
||||
break
|
||||
fi
|
||||
done
|
||||
done
|
||||
done
|
||||
|
||||
echo "=== Step 2: closure ==="
|
||||
echo "Closure has ${#closure[@]} recipes."
|
||||
echo
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Step 3: for each recipe in the closure, clean build/ + sysroot/
|
||||
# ---------------------------------------------------------------------------
|
||||
# Cookbook convention (per src/cook/cook_build.rs): per-recipe
|
||||
# target layout is target/<arch>/{build,sysroot,stage.tmp,...}
|
||||
# We delete build/ + sysroot/ + stage.tmp/ but PRESERVE source/
|
||||
# (the upstream tar was already extracted there; re-fetching is
|
||||
# slow and unnecessary).
|
||||
echo "=== Step 3: clean target dirs ==="
|
||||
for r in "${closure[@]}"; do
|
||||
recipe_dir="$RECIPES_DIR"/*/"$r"
|
||||
if [ ! -d "$recipe_dir" ]; then
|
||||
continue
|
||||
fi
|
||||
target_dir="$recipe_dir/target"
|
||||
if [ ! -d "$target_dir" ]; then
|
||||
continue
|
||||
fi
|
||||
for arch_target in "$target_dir"/*/; do
|
||||
[ -d "$arch_target" ] || continue
|
||||
for sub in build sysroot stage.tmp; do
|
||||
if [ -d "$arch_target/$sub" ]; then
|
||||
if [ "$DRY_RUN" = "1" ]; then
|
||||
echo " [dry-run] would rm -rf $arch_target/$sub"
|
||||
else
|
||||
rm -rf "$arch_target/$sub"
|
||||
echo " cleaned $arch_target/$sub"
|
||||
fi
|
||||
fi
|
||||
done
|
||||
done
|
||||
done
|
||||
echo
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Step 4: re-cook in dep order with parallel jobs
|
||||
# ---------------------------------------------------------------------------
|
||||
echo "=== Step 4: rebuild ==="
|
||||
echo "Running: ./target/release/repo cook --jobs=$JOBS <closure>"
|
||||
echo "(Cookbook walks the closure in dep-first order; --jobs runs"
|
||||
echo " independent recipes in the same dep level in parallel.)"
|
||||
echo
|
||||
|
||||
if [ "$DRY_RUN" = "1" ]; then
|
||||
echo " [dry-run] would cook: ${closure[*]}"
|
||||
else
|
||||
# The rebuild may legitimately fail if upstream deps aren't
|
||||
# all built (a fresh checkout has no cooked sysroot). The
|
||||
# user's intent is "rebuild from scratch", not "ensure
|
||||
# every dep is present". Report the failure but don't
|
||||
# exit 1 — let the user inspect the log and re-run after
|
||||
# addressing the missing dep.
|
||||
if ./target/release/repo cook --jobs="$JOBS" "${closure[@]}" 2>&1 | tee "$LOG_DIR/rebuild.log"; then
|
||||
rebuild_status="success"
|
||||
else
|
||||
rebuild_status="FAILED (see log)"
|
||||
fi
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "=== Scratch rebuild complete (status: ${rebuild_status:-skipped/dry-run}) ==="
|
||||
echo "Log: $LOG_DIR/rebuild.log"
|
||||
@@ -0,0 +1,258 @@
|
||||
"""Tests for local/scripts/scratch-rebuild.sh.
|
||||
|
||||
The script's autotools detection is hard to test as a whole
|
||||
because the rebuild step requires a built cookbook binary
|
||||
plus cooked deps. These tests validate the parts that are
|
||||
testable in isolation: the AUTOTOOLS_CORE list, the content
|
||||
regex, and the transitive-closure BFS algorithm.
|
||||
|
||||
The full integration test (cookbook rebuild after a relibc
|
||||
change) is exercised manually + via the Gitea Actions job
|
||||
that runs the script's --dry-run path.
|
||||
"""
|
||||
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import textwrap
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
SCRIPT = Path(__file__).resolve().parent.parent / "scratch-rebuild.sh"
|
||||
|
||||
|
||||
def _make_recipe(
|
||||
root: Path,
|
||||
category: str,
|
||||
name: str,
|
||||
*,
|
||||
has_autotools: bool = False,
|
||||
deps: list[str] | None = None,
|
||||
) -> Path:
|
||||
"""Create a synthetic recipe with optional autotools content and deps."""
|
||||
d = root / "local" / "recipes" / category / name
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
deps_str = ""
|
||||
if deps:
|
||||
deps_str = (
|
||||
"dependencies = ["
|
||||
+ ", ".join(f'"{d}"' for d in deps)
|
||||
+ "]"
|
||||
)
|
||||
body = "[source]\n"
|
||||
body += 'tar = "https://example.com/foo.tar.xz"\n'
|
||||
body += 'blake3 = "deadbeef"\n\n'
|
||||
body += "[build]\n"
|
||||
if has_autotools:
|
||||
body += "script = \"\"\"\n"
|
||||
body += "aclocal -I m4\n"
|
||||
body += "autoreconf -fi\n"
|
||||
body += "./configure --prefix=/usr\n"
|
||||
body += "make\n"
|
||||
body += '"""\n'
|
||||
else:
|
||||
body += 'script = "make install"\n'
|
||||
if deps_str:
|
||||
body += "\n" + deps_str + "\n"
|
||||
(d / "recipe.toml").write_text(body)
|
||||
return d
|
||||
|
||||
|
||||
class TestAutotoolsCoreList(unittest.TestCase):
|
||||
"""The AUTOTOOLS_CORE set is the named-list of recipes that
|
||||
are autotools infrastructure even if they don't directly
|
||||
invoke aclocal/autoreconf. Verify the constant is correct."""
|
||||
|
||||
def test_autotools_core_includes_m4(self):
|
||||
text = SCRIPT.read_text()
|
||||
self.assertIn("AUTOTOOLS_CORE=", text)
|
||||
self.assertIn("m4", text)
|
||||
|
||||
def test_autotools_core_includes_libtool(self):
|
||||
text = SCRIPT.read_text()
|
||||
self.assertIn("AUTOTOOLS_CORE=", text)
|
||||
self.assertIn("libtool", text)
|
||||
|
||||
def test_autotools_core_includes_bison_flex(self):
|
||||
text = SCRIPT.read_text()
|
||||
self.assertIn("AUTOTOOLS_CORE=", text)
|
||||
self.assertIn("bison", text)
|
||||
self.assertIn("flex", text)
|
||||
|
||||
|
||||
class TestAutotoolsContentRegex(unittest.TestCase):
|
||||
"""The content regex is the primary detection mechanism.
|
||||
Verify it catches each canonical autotools command."""
|
||||
|
||||
REGEX = re.compile(
|
||||
"^([\\s]*(aclocal|autoreconf|libtoolize|automake|autoconf|gettextize)\\b|\\./configure\\b|./configure\\b)"
|
||||
)
|
||||
|
||||
def _assert_matches(self, line: str) -> None:
|
||||
"""Run a positive match assertion with a clear error."""
|
||||
self.assertTrue(
|
||||
bool(self.REGEX.match(line)),
|
||||
f"regex {self.REGEX.pattern!r} should match {line!r}",
|
||||
)
|
||||
|
||||
def test_catches_aclocal(self):
|
||||
self._assert_matches("aclocal -I m4")
|
||||
|
||||
def test_catches_autoreconf(self):
|
||||
self._assert_matches("autoreconf -fi")
|
||||
|
||||
def test_catches_libtoolize(self):
|
||||
self._assert_matches("libtoolize --force")
|
||||
|
||||
def test_catches_automake(self):
|
||||
self._assert_matches("automake --add-missing")
|
||||
|
||||
def test_catches_autoconf(self):
|
||||
self._assert_matches("autoconf")
|
||||
|
||||
def test_catches_gettextize(self):
|
||||
self._assert_matches("gettextize")
|
||||
|
||||
def test_catches_configure_with_dot_slash(self):
|
||||
self._assert_matches("./configure --prefix=/usr")
|
||||
|
||||
def test_does_not_match_non_autotools(self):
|
||||
self.assertFalse(self.REGEX.match("make install"))
|
||||
self.assertFalse(self.REGEX.match("cmake -B build"))
|
||||
self.assertFalse(self.REGEX.match("meson setup builddir"))
|
||||
|
||||
|
||||
class TestRecipeDepParsing(unittest.TestCase):
|
||||
"""The dep-closure BFS reads dependencies + dev_dependencies
|
||||
from each recipe.toml. Verify the awk-based parser extracts
|
||||
both forms correctly."""
|
||||
|
||||
def _parse_deps(self, recipe_text: str) -> list[str]:
|
||||
import re as _re
|
||||
in_build = False
|
||||
deps: list[str] = []
|
||||
for line in recipe_text.splitlines():
|
||||
if line.strip() == "[build]":
|
||||
in_build = True
|
||||
continue
|
||||
if line.strip().startswith("[") and line.strip() != "[build]":
|
||||
in_build = False
|
||||
if not in_build:
|
||||
continue
|
||||
m = _re.match(
|
||||
r"^\s*(dependencies|dev-dependencies)\s*=\s*\[(.*)\]\s*$", line
|
||||
)
|
||||
if m:
|
||||
content = m.group(2)
|
||||
deps.extend(
|
||||
item.strip().strip('"').strip("'")
|
||||
for item in content.split(",")
|
||||
if item.strip()
|
||||
)
|
||||
return deps
|
||||
|
||||
def test_parses_dependencies(self):
|
||||
text = textwrap.dedent("""
|
||||
[source]
|
||||
tar = "x"
|
||||
|
||||
[build]
|
||||
dependencies = ["foo", "bar"]
|
||||
script = "make"
|
||||
""")
|
||||
self.assertEqual(self._parse_deps(text), ["foo", "bar"])
|
||||
|
||||
def test_parses_dev_dependencies(self):
|
||||
text = textwrap.dedent("""
|
||||
[source]
|
||||
tar = "x"
|
||||
|
||||
[build]
|
||||
dev-dependencies = ["dev-foo"]
|
||||
script = "make"
|
||||
""")
|
||||
self.assertEqual(self._parse_deps(text), ["dev-foo"])
|
||||
|
||||
def test_parses_both(self):
|
||||
text = textwrap.dedent("""
|
||||
[source]
|
||||
tar = "x"
|
||||
|
||||
[build]
|
||||
dependencies = ["foo"]
|
||||
dev-dependencies = ["bar"]
|
||||
script = "make"
|
||||
""")
|
||||
self.assertEqual(self._parse_deps(text), ["foo", "bar"])
|
||||
|
||||
def test_no_deps(self):
|
||||
text = textwrap.dedent("""
|
||||
[source]
|
||||
tar = "x"
|
||||
|
||||
[build]
|
||||
script = "make"
|
||||
""")
|
||||
self.assertEqual(self._parse_deps(text), [])
|
||||
|
||||
|
||||
class TestScriptHelp(unittest.TestCase):
|
||||
def test_help_describes_script(self):
|
||||
result = subprocess.run(
|
||||
[str(SCRIPT), "--help"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=5,
|
||||
)
|
||||
self.assertEqual(result.returncode, 0)
|
||||
self.assertIn("build-system improvement #10", result.stdout)
|
||||
self.assertIn("--dry-run", result.stdout)
|
||||
self.assertIn("--jobs=", result.stdout)
|
||||
self.assertIn("autotools", result.stdout.lower())
|
||||
|
||||
|
||||
class TestScriptStructure(unittest.TestCase):
|
||||
"""Regression guards against the v1 mistakes we don't want
|
||||
to repeat: missing cookbook check, non-executable, wrong
|
||||
target dir layout."""
|
||||
|
||||
def test_script_is_executable(self):
|
||||
import os
|
||||
import stat
|
||||
mode = SCRIPT.stat().st_mode
|
||||
self.assertTrue(mode & stat.S_IXUSR, "script must be user-executable")
|
||||
|
||||
def test_uses_release_repo_binary(self):
|
||||
text = SCRIPT.read_text()
|
||||
self.assertIn("./target/release/repo", text)
|
||||
|
||||
def test_preserves_source_dir(self):
|
||||
# The migration must NOT delete source/ — that would
|
||||
# force a re-fetch. Only build/ + sysroot/ + stage.tmp/.
|
||||
text = SCRIPT.read_text()
|
||||
self.assertIn("PRESERVE source/", text)
|
||||
# Verify the actual deletion logic only targets the
|
||||
# right subdirs.
|
||||
for line in text.splitlines():
|
||||
if "rm -rf" in line and "sub in" not in line and "build" not in line:
|
||||
continue
|
||||
if "rm -rf" in line and ("build" in line or "stage.tmp" in line):
|
||||
self.assertIn("arch_target", line)
|
||||
|
||||
def test_uses_parallel_jobs_flag(self):
|
||||
# The script must use --jobs=N (not --jobs N) so the
|
||||
# parallel scheduler kicks in.
|
||||
text = SCRIPT.read_text()
|
||||
self.assertIn('--jobs="$JOBS"', text)
|
||||
|
||||
def test_dry_run_does_not_clean(self):
|
||||
# When DRY_RUN=1, the script must report what it
|
||||
# WOULD do but not actually rm or cook.
|
||||
text = SCRIPT.read_text()
|
||||
self.assertIn("[dry-run] would", text)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in New Issue
Block a user