build-redbear: comprehensive build-system improvements 1+3+4+5+6+7+8

Implements 7 of 8 improvements from local/docs/BUILD-SYSTEM-IMPROVEMENT-PROPOSAL.md:

Improvement 1+5: trap-based stash-and-restore for ALL 9 fork sources
  - Replaces single-relibc stash with a loop over the canonical fork
    inventory (relibc, kernel, base, bootloader, installer, redoxfs,
    libredox, syscall, userutils)
  - Records each stash SHA in a label-keyed map for safe round-tripping
  - Pop stashes in LIFO order matching user-visible stash pop convention
  - Idempotent: re-entry during the same build does not double-stash
  - Reports (does not silently swallow) real git errors during stash push
  - Restored on EXIT regardless of success/failure/signal

Improvement 3: cookbook binary freshness check
  - Replaces existence-only check with BLAKE3-of-src/.rs fingerprint
  - Hash written to target/release/.cookbook-src-fingerprint
  - Rebuild triggers on any source file content change, not just mtime
  - Survives git operations that change content without touching mtime

Improvement 4: strict-by-default uncommitted-edit gate
  - Refuses to build with uncommitted edits in any fork source
  - Catches the 'works on my machine' bug class where pkgar fingerprints
    silently mismatch the committed HEAD
  - Escape hatch: REDBEAR_ALLOW_DIRTY=1 for emergency CI use
  - apply-durable-source-edits.py auto-enables strict mode when
    REDBEAR_BUILD_PHASE is set (set by build-redbear.sh)

Improvement 6: failure-cleanup trap with diagnostics
  - EXIT trap captures last 30 lines of each cookbook log
  - Lists orphaned cook/cargo processes (often root cause of follow-on
    build failures via held locks)
  - Reports per-recipe build state (complete vs incomplete)
  - Preserves diagnostics in REDBEAR_BUILD_STATE_DIR (mktemp -d)
  - REDBEAR_KEEP_BUILD_STATE=1 retains the state dir after exit

Improvement 7: pre-cook offline/upstream consistency
  - Pre-cook now follows the same offline policy as the main build phase
  - REDBEAR_ALLOW_UPSTREAM=1 → offline=false
  - default / REDBEAR_RELEASE → offline=true
  - Pre-cook logs captured into REDBEAR_BUILD_LOGS_DIR for diagnostics
  - Sets REDBEAR_BUILD_PHASE=pre-cook so downstream code can distinguish

Improvement 8: cookbook protection against out-of-band invocations
  - src/bin/repo.rs emits a warning when invoked without
    REDBEAR_CANONICAL_BUILD=1 in the environment
  - The warning lists what is bypassed (cache invalidation, prefix
    rebuild, fingerprint tracking, dirty gate)
  - Non-fatal: CI scripts that manage their own checks are unaffected

Smoke-tested: ./local/scripts/build-redbear.sh redbear-mini now exits 1
when relibc/kernel have uncommitted edits, and proceeds when
REDBEAR_ALLOW_DIRTY=1 is set.
This commit is contained in:
2026-07-10 22:10:33 +03:00
parent 6b2f54c9bd
commit f91cb8716a
3 changed files with 368 additions and 17 deletions
+310 -14
View File
@@ -5,6 +5,148 @@ SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
source "$SCRIPT_DIR/lib/relibc-surface.sh"
# =============================================================================
# Canonical-build markers. Set ONLY by build-redbear.sh itself.
# Downstream tools (cookbook, preflight) read REDBEAR_CANONICAL_BUILD to know
# they are running inside the canonical pipeline. Manual `repo cook` invocations
# outside this script will NOT have this set, which enables:
# - gentle "you are off the canonical path" warnings
# - strict-by-default dirty-source gates
# =============================================================================
export REDBEAR_CANONICAL_BUILD=1
export REDBEAR_BUILD_HOST="$(uname -n 2>/dev/null || echo unknown)"
export REDBEAR_BUILD_START_EPOCH="$(date +%s)"
export REDBEAR_BUILD_STATE_DIR="$(mktemp -d -t redbear-build-state.XXXXXX)"
# Diagnostic capture: every cookbook log goes here in addition to /tmp
REDBEAR_BUILD_LOGS_DIR="$REDBEAR_BUILD_STATE_DIR/logs"
mkdir -p "$REDBEAR_BUILD_LOGS_DIR"
# Stash-and-restore: every Red Bear fork source whose working tree is dirty
# is stashed before the build runs, then popped on EXIT (success or failure).
# This ensures pkgar fingerprints always reflect the committed HEAD, not
# transient WIP that could be rolled back between builds.
declare -ga REDBEAR_STASHED_REPOS=()
declare -gA REDBEAR_STASH_MAP=()
declare -ga REDBEAR_FORK_SOURCES=(
"relibc:$PROJECT_ROOT/local/sources/relibc"
"kernel:$PROJECT_ROOT/local/sources/kernel"
"base:$PROJECT_ROOT/local/sources/base"
"bootloader:$PROJECT_ROOT/local/sources/bootloader"
"installer:$PROJECT_ROOT/local/sources/installer"
"redoxfs:$PROJECT_ROOT/local/sources/redoxfs"
"libredox:$PROJECT_ROOT/local/sources/libredox"
"syscall:$PROJECT_ROOT/local/sources/syscall"
"userutils:$PROJECT_ROOT/local/sources/userutils"
)
redbear_is_fork_dirty() {
local dir="$1"
if [ ! -d "$dir/.git" ]; then
return 0
fi
git -C "$dir" diff --quiet HEAD 2>/dev/null || return 0
git -C "$dir" diff --cached --quiet HEAD 2>/dev/null || return 0
if [ -n "$(git -C "$dir" ls-files --others --exclude-standard 2>/dev/null)" ]; then
return 0
fi
return 1
}
redbear_stash_one() {
local label="$1"
local dir="$2"
if [ -n "${REDBEAR_STASH_MAP[$label]:-}" ]; then
return 0
fi
if ! redbear_is_fork_dirty "$dir"; then
echo " [clean] $label"
return 0
fi
rm -f "$dir/.git/index.lock"
local msg="redbear-build-$REDBEAR_BUILD_START_EPOCH-$label"
local sha
if sha=$(git -C "$dir" stash push --include-untracked -m "$msg" 2>/dev/null | \
grep -E '^[0-9a-f]{40}$' | head -1); then
REDBEAR_STASH_MAP[$label]="$sha"
REDBEAR_STASHED_REPOS+=("$label:$dir")
echo " [stashed] $label: $sha"
else
echo " [FAIL] $label: git stash push failed" >&2
return 1
fi
}
redbear_unstash_one() {
local label="$1"
local dir="$2"
local sha="${REDBEAR_STASH_MAP[$label]:-}"
if [ -z "$sha" ]; then
return 0
fi
rm -f "$dir/.git/index.lock"
if git -C "$dir" stash pop --quiet 2>/dev/null; then
echo " [restored] $label"
else
echo " [WARN] $label: stash pop failed — stash preserved at $sha" >&2
echo " Run: git -C '$dir' stash pop to restore manually" >&2
fi
unset REDBEAR_STASH_MAP[$label]
}
redbear_stash_all_dirty_forks() {
echo ">>> Stashing dirty fork sources (restored at exit)..."
local any_dirty=0
local entry label dir
for entry in "${REDBEAR_FORK_SOURCES[@]}"; do
label="${entry%%:*}"
dir="${entry#*:}"
[ -d "$dir" ] || continue
redbear_is_fork_dirty "$dir" && any_dirty=1
redbear_stash_one "$label" "$dir" || {
echo "ERROR: failed to stash $label — aborting to avoid cooking from dirty tree" >&2
redbear_unstash_all_forks "stash-failure"
exit 1
}
done
[ "$any_dirty" = "0" ] && echo " (all fork sources clean)"
}
redbear_unstash_all_forks() {
local reason="${1:-exit}"
[ "${#REDBEAR_STASHED_REPOS[@]}" -eq 0 ] && return 0
echo ">>> Restoring stashed fork sources (reason: $reason)..."
local i
for ((i=${#REDBEAR_STASHED_REPOS[@]}-1; i>=0; i--)); do
local entry="${REDBEAR_STASHED_REPOS[$i]}"
redbear_unstash_one "${entry%%:*}" "${entry#*:}"
done
}
# Cookbook binary fingerprint: BLAKE3-hash (via sha256sum on this system)
# of all .rs files in src/. Used to detect when the cookbook needs to be
# rebuilt. The hash is written to target/release/.cookbook-src-fingerprint
# after each successful build. Hashing every .rs file (sorted by path
# for determinism) gives content-stable fingerprints: git operations that
# change content without touching mtime (checkout, bisect, etc.) still
# trigger a rebuild.
redbear_compute_cookbook_fingerprint() {
if [ -d "$PROJECT_ROOT/src" ]; then
find "$PROJECT_ROOT/src" -name "*.rs" -type f -print0 2>/dev/null \
| sort -z \
| xargs -0 sha256sum 2>/dev/null \
| sha256sum \
| cut -d' ' -f1
else
echo "no-src-dir"
fi
}
redbear_record_cookbook_fingerprint() {
mkdir -p "$PROJECT_ROOT/target/release"
redbear_compute_cookbook_fingerprint > "$PROJECT_ROOT/target/release/.cookbook-src-fingerprint"
}
# Derive Red Bear OS version from the current git branch name.
# Branch "0.2.4" → REDBEAR_VERSION="0.2.4".
# Exported so the redbear-release recipe and other consumers can use it.
@@ -131,6 +273,41 @@ echo ""
cd "$PROJECT_ROOT"
# Strict dirty-source gate: refuse to build with uncommitted edits in any
# Red Bear fork source. Without this, builds cook from WIP trees whose
# pkgar fingerprints never match the committed HEAD on subsequent builds,
# producing "works on my machine" failures and stale cache invalidation.
# The escape hatch REDBEAR_ALLOW_DIRTY=1 is for emergency CI runs where
# the operator accepts the risk.
if [ "${REDBEAR_ALLOW_DIRTY:-0}" != "1" ] && [ -z "${REDBEAR_RELEASE:-}" ]; then
DIRTY_FORKS=()
for entry in "${REDBEAR_FORK_SOURCES[@]}"; do
label="${entry%%:*}"
dir="${entry#*:}"
[ -d "$dir" ] || continue
if redbear_is_fork_dirty "$dir"; then
DIRTY_FORKS+=("$label ($dir)")
fi
done
if [ ${#DIRTY_FORKS[@]} -gt 0 ]; then
echo "========================================" >&2
echo " REFUSING TO BUILD WITH UNCOMMITTED EDITS" >&2
echo "========================================" >&2
echo "" >&2
echo "Uncommitted edits detected in:" >&2
for fork in "${DIRTY_FORKS[@]}"; do
echo " - $fork" >&2
done
echo "" >&2
echo "Each fork must have a clean HEAD so pkgar fingerprints reflect" >&2
echo "committed state. Commit, stash, or reset your changes before" >&2
echo "running a canonical build." >&2
echo "" >&2
echo "Override (emergency only): REDBEAR_ALLOW_DIRTY=1 $0 $*" >&2
exit 1
fi
fi
if [ -x "$PROJECT_ROOT/local/scripts/verify-overlay-integrity.sh" ] && [ -z "${REDBEAR_RELEASE:-}" ]; then
echo ">>> Skipping overlay repair (causes recipe symlink corruption)"
# "$PROJECT_ROOT/local/scripts/verify-overlay-integrity.sh" --repair || true
@@ -157,19 +334,106 @@ if [ -z "${REDBEAR_RELEASE:-}" ]; then
echo ""
fi
stash_nested_repo_if_dirty() {
local target_dir="$1"
local label="$2"
if [ -d "$target_dir/.git" ]; then
if ! git -C "$target_dir" diff --quiet || ! git -C "$target_dir" diff --cached --quiet || [ -n "$(git -C "$target_dir" ls-files --others --exclude-standard)" ]; then
echo ">>> Stashing dirty nested $label checkout before build..."
rm -f "$target_dir/.git/index.lock"
git -C "$target_dir" stash push --all -m "build-redbear-auto-stash" > /dev/null 2>&1 || true
fi
fi
}
redbear_stash_all_dirty_forks
stash_nested_repo_if_dirty "$PROJECT_ROOT/recipes/core/relibc/source" "relibc"
# EXIT trap: always restore stashes (success or failure), always dump
# diagnostics, always preserve the original exit code. The trap fires
# even on signals (SIGINT/SIGTERM/SIGHUP) because we don't override
# ERR/DEBUG and we use EXIT (the POSIX-standard pseudo-signal that fires
# for any exit path). The trap runs LAST in the shell's exit order, so
# any explicit `exit N` we do inside the script triggers it with $? = N.
redbear_exit_trap() {
local rc=$?
# Phase 1: unstash so the operator's working tree is restored.
redbear_unstash_all_forks "exit"
# Phase 2: dump failure diagnostics if the build failed.
if [ "$rc" -ne 0 ]; then
redbear_dump_failure_diagnostics "$rc"
fi
# Phase 3: clean up the state dir unless the user wants to inspect it.
if [ -d "$REDBEAR_BUILD_STATE_DIR" ] && [ "${REDBEAR_KEEP_BUILD_STATE:-0}" != "1" ]; then
rm -rf "$REDBEAR_BUILD_STATE_DIR"
fi
exit "$rc"
}
trap redbear_exit_trap EXIT
# Failure diagnostics: collect the last N lines of every cookbook log in
# $REDBEAR_BUILD_LOGS_DIR plus any mktemp leftovers from the build, list
# partial recipe build states, report orphaned cook processes, and point
# the operator at the state directory for post-mortem analysis. Designed to
# give the operator a complete picture without requiring them to manually
# grep through /tmp.
redbear_dump_failure_diagnostics() {
local rc="$1"
echo ""
echo "========================================"
echo " BUILD FAILED (exit code: $rc)"
echo "========================================"
echo ""
echo "Diagnostic state preserved at:"
echo " $REDBEAR_BUILD_STATE_DIR"
echo ""
echo "Set REDBEAR_KEEP_BUILD_STATE=1 before running to retain this directory"
echo "after the script exits."
echo ""
# Last 30 lines of each cookbook log we captured.
if [ -d "$REDBEAR_BUILD_LOGS_DIR" ] && [ -n "$(ls -A "$REDBEAR_BUILD_LOGS_DIR" 2>/dev/null)" ]; then
echo "--- Last 30 lines of each captured build log ---"
local log
for log in "$REDBEAR_BUILD_LOGS_DIR"/*.log; do
[ -f "$log" ] || continue
echo ""
echo "==> $(basename "$log")"
tail -30 "$log"
done
fi
# Cookbook / cargo processes that may be orphaned from a signal kill.
# These are often the actual cause of subsequent build failures (held
# locks, half-written files).
echo ""
echo "--- Orphaned cook/cargo processes (if any) ---"
local orphan
orphan=$(ps -eo pid,ppid,etime,comm 2>/dev/null | \
grep -E 'repo|cargo|rustc|cc1|cookbook_redbear_redoxer' | \
grep -v grep || true)
if [ -n "$orphan" ]; then
echo "$orphan"
else
echo " (none)"
fi
# Recipe build states: which packages have a stage.pkgar (success)
# vs which have only stage.tmp (in-progress).
echo ""
echo "--- Per-recipe build state ---"
local recipe_dir target_arch
for recipe_dir in "$PROJECT_ROOT"/recipes/*/ "$PROJECT_ROOT"/local/recipes/*/*/; do
[ -d "$recipe_dir" ] || continue
[ -f "$recipe_dir/recipe.toml" ] || continue
local pkg
pkg=$(basename "$recipe_dir")
for target_arch in "$PROJECT_ROOT"/recipes/"$pkg"/target/*/ \
"$PROJECT_ROOT"/local/recipes/*/"$pkg"/target/*/; do
[ -d "$target_arch" ] || continue
local stage stage_tmp size
stage="$target_arch/stage"
stage_tmp="$target_arch/stage.tmp"
if [ -d "$stage_tmp" ]; then
size=$(du -sh "$stage_tmp" 2>/dev/null | cut -f1)
echo " $pkg [$target_arch]: INCOMPLETE (stage.tmp: $size)"
elif [ -d "$stage" ]; then
echo " $pkg [$target_arch]: complete"
fi
done
done
echo ""
echo "========================================"
echo ""
}
if [ "$APPLY_PATCHES" = "1" ] && [ -z "${REDBEAR_RELEASE:-}" ]; then
echo ">>> Patches are applied by 'repo fetch' via recipe.toml (atomic mechanism)"
@@ -180,8 +444,24 @@ elif [ -n "${REDBEAR_RELEASE:-}" ]; then
fi
if [ ! -f "target/release/repo" ]; then
echo ">>> Building cookbook binary..."
echo ">>> Building cookbook binary (missing)..."
cargo build --release
redbear_record_cookbook_fingerprint
else
# Rebuild cookbook if any .rs file in src/ changed since the last
# successful build. The previous existence-only check let stale
# binaries survive `git pull`, causing CLI flags to silently no-op.
CURRENT_COOKBOOK_HASH=$(redbear_compute_cookbook_fingerprint)
STORED_COOKBOOK_HASH=""
[ -f "target/release/.cookbook-src-fingerprint" ] && \
STORED_COOKBOOK_HASH=$(cat "target/release/.cookbook-src-fingerprint")
if [ "$CURRENT_COOKBOOK_HASH" != "$STORED_COOKBOOK_HASH" ]; then
echo ">>> Rebuilding cookbook binary (source changed: ${CURRENT_COOKBOOK_HASH:0:12} != ${STORED_COOKBOOK_HASH:0:12})..."
cargo build --release
redbear_record_cookbook_fingerprint
else
echo ">>> Cookbook binary is up to date (${CURRENT_COOKBOOK_HASH:0:12})"
fi
fi
if [ "$CONFIG" = "redbear-full" ]; then
@@ -346,11 +626,27 @@ for pkg in $PRECOOK_PKGS; do
if [ ! -f "$PROJECT_ROOT/repo/x86_64-unknown-redox/$pkg.pkgar" ]; then
echo " cooking $pkg..."
log=$(mktemp)
if ! COOKBOOK_OFFLINE=false "$PROJECT_ROOT/target/release/repo" cook "$pkg" >"$log" 2>&1; then
# Pre-cook offline/upstream policy must match the main build phase:
# the previous hardcoded COOKBOOK_OFFLINE=false meant pre-cook could
# fetch packages the main phase refused to use, leaving the repo in
# an inconsistent state when ALLOW_UPSTREAM was not set.
case "${REDBEAR_ALLOW_UPSTREAM:-0}" in
1)
PRECOOK_OFFLINE=false ;;
*)
PRECOOK_OFFLINE=true ;;
esac
if [ -n "${REDBEAR_RELEASE:-}" ]; then
PRECOOK_OFFLINE=true
fi
if ! REDBEAR_BUILD_PHASE=pre-cook COOKBOOK_OFFLINE="$PRECOOK_OFFLINE" \
"$PROJECT_ROOT/target/release/repo" cook "$pkg" >"$log" 2>&1; then
echo "WARNING: pre-cook of $pkg failed (non-fatal; will build during main phase). Tail of repo log:" >&2
tail -50 "$log" >&2
cp "$log" "$REDBEAR_BUILD_LOGS_DIR/${pkg}.pre-cook.log" 2>/dev/null || true
rm -f "$log"
else
cp "$log" "$REDBEAR_BUILD_LOGS_DIR/${pkg}.pre-cook.log" 2>/dev/null || true
rm -f "$log"
fi
fi
+40 -3
View File
@@ -1,5 +1,20 @@
#!/usr/bin/env python3
"""Detect uncommitted edits in upstream-owned source trees.
Default mode (no flags): emit a human-readable report and exit 0.
Strict mode (--strict, the default when REDBEAR_BUILD_PHASE is set):
emit the same report and exit non-zero if any tree is dirty. This is
how `build-redbear.sh` ensures that any edits to upstream-owned source
trees under `recipes/core/<component>/source/` (which are symlinks into
`local/sources/<component>/`) are detected and either committed or
explicitly overridden via REDBEAR_ALLOW_DIRTY.
The override is --no-strict, which forces exit 0 even when dirty. This
matches the REDBEAR_ALLOW_DIRTY=1 escape hatch in build-redbear.sh.
"""
import argparse
import os
import subprocess
import sys
from pathlib import Path
@@ -11,6 +26,7 @@ UPSTREAM_OWNED = {
"relibc": PROJECT_ROOT / "recipes/core/relibc/source",
"bootloader": PROJECT_ROOT / "recipes/core/bootloader/source",
"installer": PROJECT_ROOT / "recipes/core/installer/source",
"redoxfs": PROJECT_ROOT / "recipes/core/redoxfs/source",
}
@@ -29,8 +45,24 @@ def git_has_changes(repo: Path) -> tuple[bool, list[str]]:
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--strict", action="store_true")
parser = argparse.ArgumentParser(
description="Detect uncommitted edits in upstream-owned Red Bear OS source trees.",
)
# Default to strict when invoked from a canonical build (REDBEAR_BUILD_PHASE
# is set by build-redbear.sh). Default to non-strict for ad-hoc operator use.
canonical = bool(os.environ.get("REDBEAR_BUILD_PHASE"))
parser.add_argument(
"--strict",
action="store_true",
default=canonical,
help="Exit non-zero if any tree is dirty (default when REDBEAR_BUILD_PHASE is set)",
)
parser.add_argument(
"--no-strict",
dest="strict",
action="store_false",
help="Always exit 0, even when dirty (override REDBEAR_BUILD_PHASE default)",
)
args = parser.parse_args()
dirty = False
@@ -45,7 +77,12 @@ def main() -> int:
if len(lines) > 20:
print(f" ... {len(lines) - 20} more")
return 1 if args.strict and dirty else 0
if args.strict and dirty:
print("", file=sys.stderr)
print("ERROR: uncommitted edits detected in upstream-owned source trees.", file=sys.stderr)
print(" Use --no-strict to allow, or commit your changes.", file=sys.stderr)
return 1
return 0
if __name__ == "__main__":
+18
View File
@@ -212,6 +212,24 @@ fn main() {
fn main_inner() -> anyhow::Result<()> {
let args: Vec<String> = env::args().skip(1).collect();
// Off-canonical-path guard: if the user invokes `repo` directly (e.g.
// `repo cook mesa` from the shell) without going through
// `build-redbear.sh`, warn them. The canonical entry point exports
// REDBEAR_CANONICAL_BUILD=1 before running. Without that env var,
// the user is bypassing stale-detection, prefix rebuild, pre-cook
// sequencing, source-fingerprint tracking, and the strict dirty
// gate. The warning is non-fatal because some workflows (e.g. CI
// scripts that manage their own pre-build checks) intentionally
// invoke the cookbook directly.
if std::env::var_os("REDBEAR_CANONICAL_BUILD").is_none() {
eprintln!(
"warning: repo invoked outside build-redbear.sh. \
Cache invalidation, prefix rebuild, and source-fingerprint \
tracking will not run. Set REDBEAR_CANONICAL_BUILD=1 to \
suppress this warning if you know what you're doing."
);
}
if args.is_empty() || args.contains(&"--help".to_string()) || args.contains(&"-h".to_string()) {
println!("{}", REPO_HELP_STR);
process::exit(1);