kellito
0f8ad8a50d
L-sized improvement #10 (cookbook scratch-rebuild) is now PARTIALLY shipped: the M-sized foundation is a runnable script that does the right thing in the common case. Verification against real cascades + integration with rebuild-cascade.sh remains for a separate session. local/scripts/scratch-rebuild.sh (190 lines, +x): Step 1: discover autotools-using recipes by content regex (aclocal|autoreconf|libtoolize|automake|autoconf|gettextize|./configure) PLUS the AUTOTOOLS_CORE list (m4, autoconf, automake, libtool, bison, flex, gettext) which are always-included because they are autotools infrastructure even if they don't directly invoke aclocal. Step 2: compute transitive closure via BFS over the recipe TOML dep graph, including both [build].dependencies and [build].dev_dependencies. Found 6 autotools users in the live tree: bison, diffutils, flex, grub, libtool, m4. Step 3: for each recipe in the closure, delete target/<arch>/{build,sysroot,stage.tmp}/ — PRESERVE source/ so we don't re-fetch the upstream tar. Step 4: re-cook in dep order with --jobs=N (default 4) so the rebuild itself runs in parallel via the dep-aware scheduler (#1). Cook errors during Step 4 do NOT abort the script with exit 1 — a failed cook may indicate a missing upstream dep (legitimate on a fresh checkout) rather than a real bug. The user inspects the log and re-runs after addressing the dep. This is documented in the header + Step 4 comment. Supports --dry-run, --jobs=N, --help. Env overrides for RECIPES_DIR + LOG_DIR (mirroring the migration script's test escape hatch pattern, used by the test suite below). 21 unit tests in local/scripts/tests/test_scratch_rebuild.py: TestAutotoolsCoreList (3) — m4, libtool, bison/flex in AUTOTOOLS_CORE TestAutotoolsContentRegex (8) — catches each canonical autotools command; does NOT match cmake/make/meson TestRecipeDepParsing (4) — parses dependencies and dev_dependencies; both; neither TestScriptHelp (1) — --help describes the script TestScriptStructure (5) — executable bit; uses ./target/release/repo; PRESERVES source/; uses --jobs=N; dry-run safe Test count: 99 -> 120 (all in <1s). The test file also surfaces a real Python regex gotcha: `^[[:space:]]*` (POSIX char class with quantifier) silently fails to match the empty string under Python's regex engine, while `^[\s]*` (shorthand) works correctly. The test regex uses the shorthand to avoid this. Wired into: make test-scratch-dry-run -> scratch-rebuild.sh --dry-run Gitea Actions job scratch-dry-run (job 6 of 10, every PR) With this commit, 9 of 10 build-system improvements in BUILD-SYSTEM-IMPROVEMENTS.md are DONE (1 PARTIAL on #10); the remaining 1 is #7A (QML gate, Qt6 engine fix, not a cookbook improvement). Verified: `./local/scripts/scratch-rebuild.sh --dry-run` correctly discovers 6 autotools users and computes the 6-recipe closure. `make test-lint-scripts` still passes 120/120 tests in <1s. Gitea workflow YAML validates with 10 jobs total (was 9).
336 lines
11 KiB
Makefile
336 lines
11 KiB
Makefile
# This file contains the build system commands configuration
|
|
# and environment variables
|
|
include mk/config.mk
|
|
|
|
# Build system dependencies
|
|
include mk/depends.mk
|
|
|
|
all: lint-config $(BUILD)/harddrive.img
|
|
|
|
# ── Red Bear OS Build Cache (OBLIGATORY) ─────────────────────────────────
|
|
# Cache sync is a mandatory part of every successful build.
|
|
# The git-tracked cache survives make clean, make distclean, and git clone.
|
|
#
|
|
# Flow:
|
|
# make all → cache-restore (if needed) → build → cache-sync → cache-commit
|
|
#
|
|
# Commands:
|
|
# make cache-sync Sync built → git cache (manual)
|
|
# make cache-commit Sync + git commit (manual)
|
|
# make cache-restore Restore from git cache
|
|
# make cache-status Compare cache vs build state
|
|
|
|
CACHE_SYNC = local/scripts/cache-sync.sh
|
|
CACHE_SAVE = local/scripts/snapshot-cache.sh
|
|
CACHE_RESTORE = local/scripts/restore-cache.sh
|
|
|
|
cache-sync:
|
|
@bash $(CACHE_SYNC)
|
|
|
|
cache-commit:
|
|
@bash $(CACHE_SYNC) --commit
|
|
|
|
cache-restore:
|
|
@echo "Red Bear: restoring from git-tracked cache..."
|
|
@bash $(CACHE_SYNC) --restore
|
|
@bash $(CACHE_RESTORE) 2>/dev/null || true
|
|
|
|
cache-save:
|
|
@bash $(CACHE_SAVE)
|
|
|
|
cache-save-essential:
|
|
@bash $(CACHE_SAVE) --essential
|
|
|
|
cache-verify:
|
|
@bash $(CACHE_RESTORE) --verify
|
|
|
|
cache-list:
|
|
@bash $(CACHE_SAVE) --list
|
|
|
|
cache-status:
|
|
@bash $(CACHE_SYNC) --status
|
|
|
|
# Obligatory cache pipeline — runs before AND after every build
|
|
cache-auto:
|
|
@# ── BEFORE BUILD: restore cache if target is empty ──
|
|
@if [ ! -f $(BUILD)/repo.tag ]; then \
|
|
if ls local/cache/pkgar/*/stage.pkgar >/dev/null 2>&1; then \
|
|
echo "Red Bear: restoring build cache..."; \
|
|
bash $(CACHE_SYNC) --restore; \
|
|
fi; \
|
|
fi
|
|
@# ── AFTER BUILD: sync cache back to git-tracked storage ──
|
|
@if [ -f $(BUILD)/harddrive.img ]; then \
|
|
echo "Red Bear: syncing build cache..."; \
|
|
bash $(CACHE_SYNC); \
|
|
echo "Red Bear: committing cache to git..."; \
|
|
bash $(CACHE_SYNC) --commit 2>/dev/null || echo "Red Bear: cache commit skipped (no changes or not in git repo)"; \
|
|
fi
|
|
|
|
$(BUILD)/harddrive.img: cache-auto
|
|
|
|
live:
|
|
-$(FUMOUNT) $(BUILD)/filesystem/ || true
|
|
-$(FUMOUNT) /tmp/redbear_installer/ || true
|
|
rm -f $(LIVE_ISO) $(LIVE_IMG) $(LIVE_BOOTLOADER) $(LIVE_IPXE)
|
|
$(MAKE) $(LIVE_ISO)
|
|
|
|
popsicle: $(LIVE_ISO)
|
|
popsicle-gtk $(LIVE_ISO)
|
|
|
|
image:
|
|
-$(FUMOUNT) $(BUILD)/filesystem/ || true
|
|
-$(FUMOUNT) /tmp/redbear_installer/ || true
|
|
rm -f $(BUILD)/harddrive.img $(LIVE_ISO) $(LIVE_IMG) $(LIVE_BOOTLOADER) $(LIVE_IPXE)
|
|
$(MAKE) all
|
|
|
|
rebuild:
|
|
-$(FUMOUNT) $(BUILD)/filesystem/ || true
|
|
-$(FUMOUNT) /tmp/redbear_installer/ || true
|
|
rm -rf $(BUILD)/repo.tag $(BUILD)/harddrive.img $(LIVE_ISO) $(LIVE_IMG) $(LIVE_BOOTLOADER) $(LIVE_IPXE)
|
|
$(MAKE) all
|
|
|
|
# To tell that it's not safe
|
|
# to execute the cookbook binary
|
|
NOT_ON_PODMAN?=0
|
|
|
|
clean:
|
|
ifeq ($(PODMAN_BUILD),1)
|
|
ifneq ("$(wildcard $(CONTAINER_TAG))","")
|
|
$(PODMAN_RUN) make $@
|
|
else
|
|
$(info will not run cookbook clean as container is not built)
|
|
$(MAKE) clean PODMAN_BUILD=0 NOT_ON_PODMAN=1 SKIP_CHECK_TOOLS=1
|
|
endif # CONTAINER_TAG
|
|
else
|
|
ifneq ($(NOT_ON_PODMAN),1)
|
|
$(MAKE) repo_clean
|
|
-$(FUMOUNT) $(BUILD)/filesystem/ || true
|
|
-$(FUMOUNT) /tmp/redbear_installer/ || true
|
|
endif # NOT_ON_PODMAN
|
|
rm -rf repo
|
|
rm -rf $(BUILD) $(PREFIX)
|
|
$(MAKE) fstools_clean
|
|
endif # PODMAN_BUILD
|
|
|
|
# distclean: removes build artifacts, toolchain, and upstream source trees.
|
|
# local/ overlay source trees are PROTECTED — the repo binary refuses to
|
|
# unfetch local overlay recipes unless REDBEAR_ALLOW_LOCAL_UNFETCH=1 is set.
|
|
# This is the safe default for Red Bear OS. local/ is NEVER deleted.
|
|
distclean:
|
|
ifneq ($(REDBEAR_RELEASE),)
|
|
$(error distclean is disabled in release mode (REDBEAR_RELEASE=$(REDBEAR_RELEASE)). Sources are immutable. Use: make clean (build artifacts only, safe))
|
|
endif
|
|
ifeq ($(PODMAN_BUILD),1)
|
|
ifneq ("$(wildcard $(CONTAINER_TAG))","")
|
|
$(PODMAN_RUN) make $@
|
|
else
|
|
$(info will not run cookbook unfetch as container is not built)
|
|
$(MAKE) distclean PODMAN_BUILD=0 NOT_ON_PODMAN=1 SKIP_CHECK_TOOLS=1
|
|
endif # CONTAINER_TAG
|
|
else
|
|
ifneq ($(NOT_ON_PODMAN),1)
|
|
$(info ==> distclean: cleaning build artifacts and upstream source trees)
|
|
$(info ==> local/ overlay sources are PROTECTED and will NOT be deleted)
|
|
$(MAKE) fetch_clean
|
|
endif # NOT_ON_PODMAN
|
|
$(MAKE) clean NOT_ON_PODMAN=1
|
|
endif # PODMAN_BUILD
|
|
|
|
# distclean-nuclear: DESTRUCTIVE — also deletes local/ overlay source trees.
|
|
# This is the OLD distclean behavior that can destroy Red Bear work.
|
|
# You must set REDBEAR_ALLOW_LOCAL_UNFETCH=1 for this to actually delete
|
|
# local overlay sources. Without it, the repo binary still protects them.
|
|
# Use ONLY when you are certain you want to discard local overlay source code.
|
|
distclean-nuclear:
|
|
ifeq ($(PODMAN_BUILD),1)
|
|
$(info distclean-nuclear is not supported in Podman mode; use native build)
|
|
else
|
|
$(warning !! distclean-nuclear will attempt to DELETE ALL source trees including local/ overlays)
|
|
$(warning !! This can destroy Red Bear OS work that is not committed to git)
|
|
$(warning !! The repo binary still protects local overlays unless REDBEAR_ALLOW_LOCAL_UNFETCH=1)
|
|
$(MAKE) fetch_clean REDBEAR_ALLOW_LOCAL_UNFETCH=1
|
|
$(MAKE) clean NOT_ON_PODMAN=1
|
|
endif # PODMAN_BUILD
|
|
|
|
pull:
|
|
git pull
|
|
rm -f $(FSTOOLS_TAG)
|
|
|
|
repo: $(BUILD)/repo.tag
|
|
|
|
repo_clean: c.--all
|
|
|
|
fetch_clean: u.--all
|
|
|
|
# Podman build recipes and vars
|
|
include mk/podman.mk
|
|
|
|
# Disk Imaging and Cookbook tools
|
|
include mk/fstools.mk
|
|
|
|
# Cross compiler recipes
|
|
include mk/prefix.mk
|
|
|
|
# Repository maintenance
|
|
include mk/repo.mk
|
|
|
|
# Disk images
|
|
include mk/disk.mk
|
|
|
|
# Emulation recipes
|
|
include mk/qemu.mk
|
|
include mk/virtualbox.mk
|
|
|
|
# CI
|
|
include mk/ci.mk
|
|
|
|
include mk/redbear.mk
|
|
|
|
# Ensure Red Bear OS integration runs before repo cook and disk image creation
|
|
$(BUILD)/harddrive.img: $(REDBEAR_TAG)
|
|
$(LIVE_ISO): $(REDBEAR_TAG)
|
|
$(REPO_TAG): $(REDBEAR_TAG)
|
|
|
|
env: prefix FORCE $(CONTAINER_TAG)
|
|
ifeq ($(PODMAN_BUILD),1)
|
|
$(PODMAN_RUN) make $@
|
|
else
|
|
export PATH="$(PREFIX_PATH):$$PATH" && \
|
|
bash
|
|
endif
|
|
|
|
setenv: FORCE
|
|
@echo export ARCH='$(ARCH)'
|
|
@echo export BOARD='$(BOARD)'
|
|
@echo export CONFIG_NAME='$(CONFIG_NAME)'
|
|
@echo BUILD='$(BUILD)'
|
|
|
|
export RUST_GDB=gdb-multiarch # Necessary when debugging for another architecture than the host
|
|
GDB_KERNEL_FILE=recipes/core/kernel/target/$(TARGET)/build/kernel.sym
|
|
gdb: FORCE
|
|
rust-gdb $(GDB_KERNEL_FILE) --eval-command="target remote :1234"
|
|
|
|
# This target allows debugging a userspace application without requiring gdbserver running inside
|
|
# the VM. Because gdb doesn't know when the userspace application is scheduled by the kernel and as
|
|
# it stops the entire VM rather than just the userspace application that the user wants to debug,
|
|
# connecting to a gdbserver running inside the VM is highly encouraged when possible. This target
|
|
# should only be used when the application to debug runs early during boot before the network stack
|
|
# has started or you need to debug the interaction between the application and the kernel.
|
|
# tl;dr: DO NOT USE THIS TARGET UNLESS YOU HAVE TO
|
|
gdb-userspace: FORCE
|
|
rust-gdb $(GDB_APP_FILE) --eval-command="add-symbol-file $(GDB_KERNEL_FILE)" --eval-command="target remote :1234"
|
|
|
|
# An empty target
|
|
FORCE:
|
|
|
|
.PHONY: lint-patches lint-patches-full lint-kf6-deps lint-cook-failure \
|
|
lint-cook-failure-explain lint-cook-recipe lint-recipe lint-recipe.% \
|
|
lint-recipe.strict lint-recipe.%.strict \
|
|
lint-build-system lint-build-system-full \
|
|
test-lint-scripts test-lint-scripts-quiet \
|
|
test-migration-dry-run test-scratch-dry-run \
|
|
repair.% clean-repair.%
|
|
|
|
# Wireshark
|
|
wireshark: FORCE
|
|
wireshark $(BUILD)/network.pcap
|
|
packages-sync: ; @bash local/scripts/sync-packages.sh
|
|
packages-list: ; @ls -la Packages/*.pkgar 2>/dev/null | wc -l && echo "pkgar files in Packages/"
|
|
validate-sources:
|
|
@for d in local/sources/kernel local/sources/relibc local/sources/base local/sources/bootloader local/sources/installer; do \
|
|
if [ -d "$$d/.git" ]; then \
|
|
echo "$$d: $(shell git -C $$d rev-list --count HEAD 2>/dev/null || echo 0) commits, $(shell git -C $$d ls-files 2>/dev/null | wc -l) files"; \
|
|
else \
|
|
echo "$$d: MISSING — run local/scripts/create-forks.sh"; \
|
|
fi; \
|
|
done
|
|
|
|
# v6.0 build-system lint targets. These run the three audit scripts
|
|
# that validate the v6.0 build system: idempotent patches, KF6 dep
|
|
# accuracy, and cook-failure classification. Each target exits non-zero
|
|
# on a real problem so CI can wire them up directly.
|
|
|
|
lint-patches:
|
|
@python3 local/scripts/audit-patch-idempotency.py --no-fetch
|
|
|
|
lint-patches-full:
|
|
@python3 local/scripts/audit-patch-idempotency.py
|
|
|
|
lint-kf6-deps:
|
|
@python3 local/scripts/audit-kf6-deps.py --no-fetch
|
|
|
|
test-lint-scripts:
|
|
@python3 -m unittest discover -s local/scripts/tests -v
|
|
|
|
test-lint-scripts-quiet:
|
|
@python3 -m unittest discover -s local/scripts/tests
|
|
|
|
# Smoke test: run the KF6 sed migration script in --dry-run
|
|
# against the live recipe tree. Catches:
|
|
# - recipe discovery regression (script can't find kf6-*)
|
|
# - path-argument vs name-argument mixup
|
|
# - permission / executable issues
|
|
# Does NOT do any real fetches, cooks, or patch writes.
|
|
test-migration-dry-run:
|
|
@./local/scripts/migrate-kf6-seds-to-patches.sh --dry-run --limit=1
|
|
|
|
# Smoke test: run the #10 scratch-rebuild script in --dry-run
|
|
# against the live tree. Catches:
|
|
# - autotools discovery regression
|
|
# - dep-closure BFS errors
|
|
# - permission / executable issues
|
|
# Does NOT do any real rm, fetch, or cook. <2s wall-clock.
|
|
test-scratch-dry-run:
|
|
@./local/scripts/scratch-rebuild.sh --dry-run
|
|
|
|
lint-cook-failure:
|
|
@python3 local/scripts/classify-cook-failure.py --last || \
|
|
(echo "No /tmp/redbear-cook.log or /tmp/build.log found. Run a cook first."; exit 0)
|
|
|
|
lint-cook-failure-explain:
|
|
@python3 local/scripts/classify-cook-failure.py --explain-rule qfloat16
|
|
|
|
# Per-recipe v6.0-policy lint. Catches R1/R2 violations BEFORE the
|
|
# slow cook starts. Per build-system improvement #5.
|
|
# Usage:
|
|
# make lint-recipe # all recipes in local/recipes/
|
|
# make lint-recipe.kf6-kimageformats # one recipe by bare name
|
|
# make lint-recipe.strict # all recipes, warnings as errors
|
|
# make lint-recipe.kf6-kimageformats.strict # one recipe, strict mode
|
|
lint-recipe:
|
|
@python3 local/scripts/lint-recipe.py --all
|
|
|
|
lint-recipe.%:
|
|
@python3 local/scripts/lint-recipe.py $*
|
|
|
|
lint-recipe.strict:
|
|
@python3 local/scripts/lint-recipe.py --all --strict
|
|
|
|
lint-recipe.%.strict:
|
|
@python3 local/scripts/lint-recipe.py $* --strict
|
|
|
|
lint-build-system: lint-patches lint-kf6-deps lint-cook-recipe
|
|
@echo "Build system lint complete."
|
|
|
|
lint-build-system-full: lint-patches-full lint-kf6-deps lint-cook-recipe
|
|
@echo "Full build system lint complete (with network)."
|
|
cascade.%: FORCE
|
|
@bash local/scripts/rebuild-cascade.sh $(basename $(subst cascade,, $*))
|
|
|
|
# Repair-cook wrapper: equivalent to `repo cook <recipe>` but with
|
|
# a fast-path that skips configure + compile if the existing build/
|
|
# is still valid. Per build-system improvement #2.
|
|
# Usage: make repair.qtbase (incremental, fast if cache fresh)
|
|
# make repair.qtbase CLEAN=1 (force full rebuild)
|
|
repair.%: FORCE
|
|
@if [ "$(CLEAN)" = "1" ]; then \
|
|
./local/scripts/repair-cook.sh $* --clean-build; \
|
|
else \
|
|
./local/scripts/repair-cook.sh $*; \
|
|
fi
|
|
# Use `make clean-repair.X` to force a clean rebuild
|
|
# (alias for the CLEAN=1 form above)
|
|
clean-repair.%: FORCE
|
|
@./local/scripts/repair-cook.sh $* --clean-build
|