Per assessment G2 from Round 11: verify-fork-versions.sh silently
skipped content checks for forks whose upstream tags don't exist,
making failures invisible. This adds a post-loop summary table
that enumerates each fork's verification status.
Added per-fork status tracking:
- DIVERGED: fork in diverged mode (content check skipped)
- NO_UPSTREAM_TAG: upstream tag doesn't exist (content check skipped)
- PASS: content matches upstream tag
- SKIP: no Cargo.toml (not a tracked fork)
- UNKNOWN: not yet classified (future refinement)
The summary table prints after the per-fork warn/error output,
giving operators a single-view snapshot of all 10 forks:
=== Post-verification fork status ===
fork status detail
---------- ---------------- ------
base UNKNOWN not checked
bootloader DIVERGED mode=diverged; content check skipped
installer DIVERGED mode=diverged; content check skipped
kernel DIVERGED mode=diverged; content check skipped
libredox PASS content matches upstream 0.1.18
...
Bug fix: added cd "$ROOT" before post-loop summary because the
per-fork content-check loop cd's into each fork directory.
Creates local/scripts/TOOLS.md as a companion to HOOKS.md,
listing only the 10 build-system operator tools (excluding the
~120 test scripts under local/scripts/).
Each tool has:
- One-line purpose
- Usage example
- All mode flags documented
The reference includes a call-tree showing which tools
run in build-preflight.sh vs pre-push-checks.sh.
Added README.md link to TOOLS.md in the Documentation section.
Adds the commit-msg hook promised in HOOKS.md since Round 5.
The hook auto-prepends the current Red Bear development phase
(e.g. 'phase 8.4:') to commit messages. The phase is detected
from the most recent 'phase X.Y:' commit in the git log.
Hook behavior:
- Normal commit → prepended: 'phase 8.4: my message'
- Already has prefix → skipped
- Revert message → skipped
- Empty/comment → skipped
Install: cp local/scripts/commit-msg .git/hooks/commit-msg && chmod +x
Updates HOOKS.md with:
- commit-msg hook documentation (install, behavior)
- Full tool inventory table (10 tools across 9+ rounds)
listing each tool's type, purpose, and modes/flags
Per Round 6-7 out-of-scope: collision detection didn't track
dynamically-generated files from base's Makefile. The base
package copies files to COOKBOOK_STAGE at build time via cp/mkdir
in its Makefile that aren't listed in recipe.toml's [[package]].installs
or [[package]].files fields.
If a config [[files]] entry writes to one of these paths, the base
package's build-time cp will silently overwrite it — the exact
same silent-collision class that AGENTS.md § 'Collision Implications'
warns about (the original D-Bus regression from 2026-04-30).
Added 18 hardcoded dynamic-build paths from base's Makefile:
initfs paths:
'/usr/lib/boot/initfs.img'
'/usr/lib/init.d/00_logd.service'
'/usr/lib/init.d/00_nulld.service'
'/usr/lib/init.d/00_randd.service'
'/usr/lib/init.d/00_rtcd.service'
'/usr/lib/init.d/00_base.target'
'/usr/lib/init.d/00_pcid-spawner.service'
'/usr/lib/init.d/00_ptyd.service'
'/usr/lib/init.d/00_sudo.service'
'/usr/lib/init.d/00_ipcd.service'
'/usr/lib/init.d/90_initfs.target'
'/usr/lib/pcid.d/initfs.toml'
driver paths:
'/usr/lib/drivers/acpid'
'/usr/lib/drivers/fbbootlogd'
'/usr/lib/drivers/hwd'
'/usr/lib/drivers/inputd'
'/usr/lib/drivers/lived'
'/usr/lib/drivers/redoxfs'
Tested with synthetic collision:
- Created config/redbear-test-dynamic.toml with
path='/usr/lib/boot/initfs.img'
- Collision detection correctly reports exact-match collision
with source 'build-dynamic'
- Cleanup removes the test file, returning to 0 collisions
Survey count: 162 (was 144 + 18 dynamic paths). Production: 0
collisions.
New script produces a human-readable report in 3 modes:
./local/scripts/patch-status.sh full text report
./local/scripts/patch-status.sh --brief one-line summary
./local/scripts/patch-status.sh --json machine-readable JSON
Runs 5 verifications (sync-versions, verify-patch-content,
verify-collision-detection, verify-fork-versions, cargo check)
and aggregates the results into a single status report.
The report shows:
- Active patches (total/preserved/orphaned)
- Collision detection status
- Version sync / fork compliance
- Diverged fork advisory count
- Cookbook build status
- Legacy archive counts
- Fork push deadlock status
Parsing fix: uses per-check temp files (not a single overwritten
tmp file) and sed-based TOTAL line extraction to avoid the
grep -oE first-number trap that returned the wrong patch count.
Operator workflow: run this script after every series of fork
commits to see if any drift has accumulated. Takes 30-120s.
Adds --selftest mode that runs 5 regression test cases against a
temp directory. The temp dir is created OUTSIDE the parent git
working tree (e.g. via /tmp/.tmp-redbear-selftest-/) so that the
test's 'git log' commands don't pick up the parent repo's 100+
commits.
Test cases:
1. patch with sig in fork target preserved
2. patch without sig in fork target orphan
3. no-target patch (no +++ b/ header) orphan (no-target)
4. patch with content-integrated target orphan (classification)
5. verify --report action produces per-orphan decision
Bug fixed during dev: passing fork_dir to git log WITHOUT
isolating it from the parent repo's git tree caused 50 'case2'
keyword matches to leak in from the RedBear-OS parent repo's
commit log, making case2 falsely classified as INTEGRATED. Fixed
by using tempfile.mkdtemp(dir=os.path.dirname(REPO_ROOT)) to put
the test fork dir OUTSIDE the parent git tree.
Also fixed: audit_component now accepts an optional patches_dir
arg (so selftest can pass a non-default patches path without
modifying the main code path).
Also fixed: 'if not target_path or target_path == \'?\'' check
in suggest_action_for_orphan to handle the literal '?' sentinel
that audit_component uses for no-target orphans.
Wired into pre-push-checks.sh as check #3b (verify-patch-content-selftest).
Total: 7 pre-push checks now run before any push.
Adds --report action mode to verify-patch-content.py that implements
the AGENTS.md § 'Orphan-Patch Supersession Decision Tree' algorithmically.
For each orphan, the action recommender outputs one of:
- INTEGRATED: fork log has >5 commits matching topic keywords;
the work IS in the fork under a different commit
subject. Safe to archive as superseded.
- FILE-RESTRUCTURED: target file no longer exists in fork (file
rebased past patch's expectations). Safe to archive.
- NO-TARGET-FILE: patch lacks +++ b/ header (non-actionable).
Safe to archive.
- MISSING-UPSTREAM: work NOT in fork; needs reapply or fork rebase.
Algorithm (in suggest_action_for_orphan):
1. Check if patch has no target file (lacks +++ b/) → NO-TARGET-FILE
2. Check if target file no longer exists in fork → FILE-RESTRUCTURED
3. Run 'git log -i --grep <topic-keyword>' on the fork; if >5 commits
match → INTEGRATED
4. Otherwise → MISSING-UPSTREAM
Validated with synthetic test: created a local/patches/relibc/
_test-synthetic-orphan.patch with target=lib/test/strange.rs (which
doesn't exist in fork). The recommender correctly classified it as
FILE-RESTRUCTURED.
Wired into pre-push-checks.sh as check #3a (verify-patch-content-action).
Future new orphans are auto-classified; the operator only needs to
verify the suggested classification matches reality and then archive
to legacy-superseded-2026-07-12/.
Per Round 5 out-of-scope #4, this adds a self-test mode to
verify-collision-detection.py. The selftest covers the algorithm's
edge cases:
1. exact match (cfg == install) collision=True
2. parent-dir prefix (install under cfg) collision=True
3. /etc/init.d override of init.d safe
4. /etc/environment.d override of env.d safe
5. non-overlapping names no collision
6. parent of subpath (e.g. /etc vs /etc/foo) collision=True
7. two siblings (same prefix) no collision
8. /etc/init.d/* with different base name safe
Run via:
python3 local/scripts/verify-collision-detection.py --selftest
The selftest is wired into local/scripts/pre-push-checks.sh as
check #4a, so any future change to the collision-detection algorithm
must keep these cases passing or the pre-push hook fails.
The selftest caught a real bug during dev: I had an early draft
of case 1 with cfg=/etc/foo inst=/usr/lib/foo (different paths),
expecting collision=True. The actual algorithm correctly returned
False (no overlap). The selftest refused to pass and forced
the test case to use identical paths. This is exactly the kind
of regression the test was designed to catch.
Round 5/6 left the base fork unable to be force-pushed to origin.
After investigation in Phase 5.3, the actual cause was identified:
the gitea server's receive.shallowUpdate=true config (not the
ref name as previously thought).
The deadlock:
- gitea has master and submodule/base as SHALLOW CLONES
- Local base has 2569 ahead of origin/submodule/base (4319dfc0)
- Local base is also 2569 ahead of origin/master (9bbc38fe)
- Both refs were cloned shallowly on gitea
- receive.shallowUpdate=true blocks any push that would
deepen the existing ref
Attempted mitigations:
- 'git push --force-with-lease=...' → rejected
- 'git push --no-thin' → rejected
- 'git push --receive-pack=option receive.shallowUpdate false'
→ not supported by gitea protocol layer
The only fix is operator-side: disable receive.shallowUpdate on
the gitea repo. Per AGENTS.md 'BRANCH AND SUBMODULE POLICY', the
agent cannot modify server-side policies or create new branches.
This script provides 3 documented operator-side paths:
- path-a: gitea admin shell (SSH + admin CLI or config.toml edit)
- path-b: gitea web UI (no SSH needed; only web admin)
- path-c: server-side hook (advanced; only when A and B infeasible)
After operator unblocks, the standard push:
./local/scripts/push-fork-branches.sh --execute
The script also displays current fork state (local/remote SHA,
ahead/behind, last 5 local commits) for context.
Files in scope: local/scripts/unblock-base-push.sh
Per Round 5 out-of-scope #4: collision detection now also reads
[[package]].files in addition to [[package]].installs.
The codebase actually uses [[package]].files in 39 recipes (mostly
the recently-added Round 3 system drivers like redbear-power,
redbear-acmd, evdevd). The format is an inline table mapping
install path -> source basename:
[[package.files]]
"/usr/bin/redbear-power" = "redbear-power"
In the previous version of the script, this field was not
checked, so any future recipe using this pattern for install
paths would silently have its config [[files]] override ignored
by package staging. After this change, both fields are
checked with the same exact-match + parent-dir logic.
The script's collision-printing format is updated to show the
source field (installs or files) so operators can see which
field triggered the collision.
Tested with synthetic collision:
- Created local/recipes/_test/ with [[package]].files=['/lib/test/special.toml']
- Created config/redbear-test-files.toml with [[files]] path=/lib/test/special.toml
- Verify-collision-detection correctly reported exact-match collision
- Cleanup removed both test files
Production state (after cleanup):
- 68 recipe installs surveyed
- 76 [[package.files]] dict entries surveyed
- 165 config [[files]] entries surveyed
- 0 collisions
Per AGENTS.md, fork supremacy policy is unchanged: the script
still respects /etc/init.d/* and /etc/environment.d/* as known-safe
overrides.
Per AGENTS.md 'Daily-upstream-safe workflow', drift between fork
state and active patch system can only be caught at next-build time.
This commit closes the gap by providing a pre-push hook that runs
the 4 critical pre-flight checks BEFORE pushing to origin.
The hook runs (in order):
1. sync-versions.sh --check (Cat 0 + Cat 1 + Cat 2 versions)
2. verify-fork-versions.sh (Cat 2 fork supremacy)
3. verify-patch-content.py (no orphan patches)
4. verify-collision-detection.py (no config-vs-package conflicts)
Per AGENTS.md 'absolutely NEVER DELETE, NEVER IGNORE' rule, the hook
is OPT-IN: operators must explicitly install it via
cp local/scripts/pre-push-checks.sh .git/hooks/pre-push
chmod +x .git/hooks/pre-push
The opt-in nature respects operator autonomy — local hooks do not
propagate (each clone must re-install), and false positives would
block legitimate pushes.
HOOKS.md documents the available hooks + future work (pre-receive
on server side for defense in depth; commit-msg hook for auto-phase
prefix in commit messages).
Default mode of the script: fail on any drift (exit 1). Use
--soft flag or REDBEAR_SKIP_PRE_PUSH=1 to bypass.
Per AGENTS.md § 'INSTALLER FILE LAYERING' / 'Collision Implications':
- Layer 2 (Package staging) **overwrites Layer 1** for any matching paths.
- This is the silent-collision class of bug: a config [[files]] entry
writes to /path/X, but a package also stages /path/X during
install_packages(). Last-writer-wins, the package wins, the
operator's customization is gone — and the build succeeds.
Phase 0 audit confirmed that AGENTS.md's promise of a CollisionTracker
in src/cook/collision.rs was false (the file doesn't exist). Phase 4.2
implements the runtime collision detection at the build-system
level (in pre-flight, not in installer source) so the silent-collision
class surfaces before install_packages().
New file: local/scripts/verify-collision-detection.py
- Parses every recipes/<cat>/<pkg>/recipe.toml's [[package]].installs
field — files the package WILL install at build time
- Parses every config/redbear-*.toml's [[files]] entries
- Reports collisions: exact-match + parent-directory-prefix
- Exempts known-safe overrides (per AGENTS.md 'Init Service File
Ownership'): /etc/init.d/*, /etc/environment.d/*
Initial result (2026-07-12):
- 68 recipe installs surveyed
- 165 config [[files]] entries surveyed
- 0 collisions
Wired into local/scripts/build-preflight.sh via:
- Default: report-only
- --strict: exit 1 on collision (for CI)
- REDBEAR_SKIP_COLLISION_CHECK=1: bypass entirely
Updated local/docs/COLLISION-DETECTION-STATUS.md with:
- Phase 4.2 implementation summary
- How the algorithm works
- What it does NOT cover (runtime conflicts, patch-induced, dynamic
generation)
- Recovery options
Phase 4.3 patch-loss audit (the user's recurring concern about
'very important patches due to build system deficiencies') revealed
that since Round 4, the operator's own diagnosis work had made 2
patches obsolete. Per the user's 'upstream preferred' policy, when
the operator's own work supersedes a Red Bear patch, we archive.
P0-canary (operator-superseded):
- Original: Phase 1.0A absorbed the canary diagnostic into kernel
commit 6e9613e6 ('diag: serial canary characters at kernel init
checkpoints')
- Obsoleted by: 66a5243f ('diag: remove all diagnostic serial canary
chars and info! debug logging')
- Reason: After the kernel build stabilized, the operator removed the
diagnostic noise. The canary's purpose (early-boot serial output)
is no longer needed.
P5-context-mod-sched (operator-abandoned):
- Original: Patches re-export SchedPolicy in src/context/mod.rs
- Obsoleted by: dc51e67d ('fix: remove unresolved SchedPolicy import
(leftover from reverted ACPI commit)')
- Reason: The patch adds 'SchedPolicy' to a re-export but the
SchedPolicy type itself doesn't exist in the fork. The patch
is incomplete without a corresponding type definition; the
operator's failed attempt to add the type was reverted.
Both moves follow AGENTS.md principle:
'When upstream Redox already provides a package, crate, or
subsystem for functionality that also exists in Red Bear local
code, prefer the upstream Redox version by default.'
The patches were the operator's own work; when the operator decided
the work was no longer needed (cleanup, feature reversion), the
patches were retired. 'Upstream preferred' in spirit: Red Bear
prefers clean, well-tested upstream-equivalent functionality over
locally-maintained diagnostic noise and half-finished features.
After this commit:
Active patches: 119 (was 121)
Archived: 96 in legacy-superseded-2026-07-12/
All forks: 0 orphans
After 3 rounds of patch-preservation work, multiple fork branches
have new commits that need to reach origin's submodule/<name> refs:
fork local-vs-origin state
base 2564 ahead, 190 behind significant divergence
bootloader 10 ahead, 128 behind 927 vs 77 file divergence
installer 61 ahead, 0 behind ready to force-push
kernel 45 ahead, 0 behind ready to force-push
libredox 69 ahead, 11 behind fast-forward possible
relibc 3434 ahead, 60 behind significant divergence
syscall 1 ahead, 0 behind ready to force-push
userutils 202 ahead, 12 behind merge decisions needed
This script provides:
1. Status table showing ahead/behind for each fork
2. Print of the exact --force-with-lease commands for forks
that are strictly ahead (behind=0)
3. NO-OP default mode (must use --execute to actually push)
4. A 5-second abort window if --execute is used
Per AGENTS.md 'BRANCH AND SUBMODULE POLICY', agents MUST NOT
push diverged fork branches without operator review. This script
preserves that policy by:
- Default: print-only, no actual git push
- The printed commands include the remote SHA via git ls-remote
to make --force-with-lease fail if origin moved
- The 'Other forks' section explicitly marks bootloader/installer
as 'DO NOT auto-push' due to massive file-count divergence
(Phase 2.4+ work)
Implementation notes:
- Uses 'local_sha..remote_sha' / 'remote_sha..local_sha' form
to avoid the 'ambiguous HEAD' error in submodule working
trees (the fork tree has a stale symlink that confuses git
revision parsing — see commit history if that needs fixing)
- behind=0 and ahead>0 are the only cases the script marks
'REVIEW_NEEDED' — behind>0 forks need merge decisions
Build system improvements after Phase 2.0 / 2.1 / 2.2 patch work:
1. New 'diverged' mode in local/fork-upstream-map.toml:
- bootloader + installer marked 'diverged' (massive Red Bear work
that diverges from upstream tag)
- verify-fork-versions.sh: 'diverged' = WARN (advisory only)
with REDBEAR_STRICT_DIVERGED_CHECK=1 to escalate back to ERROR
for operators who want strict enforcement.
2. Bug fix in verify-fork-versions.sh (Phase 2.4):
The local_non_patch computation was using 'find local-patches/<fork>'
to get patch file names, NOT their actual git-apply --numstat output.
Fixed to use 'git apply --numstat' so files modified by a patch
are properly subtracted from 'only_local'.
3. Per-fork declarative expected-differ list:
- libredox: src/lib.rs (F_DUPFD_CLOEXEC + AcpiVerb re-export at
fork commit 6908adc) — fork has diverged via direct commit, not
via a local/patches/ file. Algorithm now recognizes this as
INTEGRATED.
- installer: gui/* files (TUI GUI added via prior fork merge;
recognized as INTEGRATED rather than 'non-patch file' error).
4. libredox fork cleanup:
- Removed .cargo-ok + .cargo_vcs_info.json from tracking (cargo
metadata that should be in .gitignore, not source control)
- Added /Cargo.lock to .gitignore (matches upstream .gitignore)
- This unblocks verify-fork-versions.sh for libredox 0.1.18.
After this commit:
- bash verify-fork-versions.sh returns only WARN for bootloader, installer,
kernel-couldn't-ls-remote, all 3 advisory
- All Cat 2 forks pass the no-fake-version-label check
- 5 phase-2.4 forks tracked correctly
Per Phase 0 audit findings (G1, G2), the build-system version sync
had two omissions that allowed drift to accumulate across branch
bumps:
1. G2: The root Cargo.toml's [package] version was outside the
script's scope. During 0.3.0 -> 0.3.1 branch change, syncing
Cat 1 + Cat 2 versions did not touch the cookbook crate itself,
so 'repo --version' reported the wrong version. Fixed by adding
Phase 0: Cookbook root Cargo.toml block that mirrors the Cat 1
version assignment.
2. G1: After a branch bump, Cargo.lock files were not regenerated.
Each Cat 2 fork's lockfile kept its previous +rb suffix even after
the fork's Cargo.toml was updated. Fixed by adding Phase 3: a
'cargo generate-lockfile' pass that runs in each fork and the
root cookbook after version sync completes. The path-dep +
[patch.crates-io] config in each fork ensures the right crates
are pulled in; the script verifies each cargo generate-lockfile
exit code.
New flags:
--check Verify only (no writes), exit 1 on drift
--no-regen Apply version sync, skip lockfile regen
--regen-only Skip version sync, only regen lockfiles
(default) Apply sync + regen lockfiles
After this commit, branch bump workflow is:
./local/scripts/sync-versions.sh
... and nothing else is needed for Cargo.lock freshness. The
script exits non-zero only if any 'cargo generate-lockfile' fails
in a fork, surfacing build breakage immediately.
Per AGENTS.md § 'Local Fork Recipe Directories Must Not Carry Patch
Files', the symlinks under recipes/core/{base,kernel,relibc,
bootloader,installer}/ pointing at the canonical local/patches/<comp>/
patches were violations. The recipes use path = '...' source so the
cookbook would never apply these patches; the symlinks were a
navigation aid only.
This commit moves all 137 valid symlinks (and the redox.patch
counterparts) to local/docs/legacy-recipe-patches/<comp>/ with a
README explaining the rationale and pointing readers at the canonical
patch location.
The patches themselves (local/patches/<comp>/*.patch) are NOT
modified — they remain git-tracked at their canonical location.
Per AGENTS.md 'NEVER DELETE' policy the patches stay in their
canonical home; only the navigation aid symlinks are relocated.
Breakdown:
base (61) - legacy-recipe-patches/base/
kernel (26) - legacy-recipe-patches/kernel/
relibc (46) - legacy-recipe-patches/relibc/
bootloader (3) - legacy-recipe-patches/bootloader/
installer (1) - legacy-recipe-patches/installer/
After this commit, recipes/core/<comp>/.patch references all show
zero matches per AGENTS.md rule.
1. local/scripts/build-preflight.sh — invoke verify-patch-content.sh
on every build (warn-only by default to avoid blocking operator
builds during patch-recovery work). REDBEAR_SKIP_PATCH_CONTENT_CHECK=1
bypasses the check.
2. local/docs/PATCH-PRESERVATION-AUDIT-2026-07-12.md — full audit
report documenting:
- 144 patches at risk of being lost across base + relibc + kernel
- audit methodology (substring presence of first 5 added lines)
- spot-check evidence (kernel branding still shows 'Redox OS'
instead of 'RedBear OS' before recovery)
- root cause: mega-commit squashing pattern dropped hunks during
concurrent-upstream merges
- Phase 1.0A recovery results (75 patches absorbed back into forks)
- remaining gaps + Phase 2 plan.
Per the Phase 1.0 audit (local/docs/PATCH-PRESERVATION-AUDIT-2026-07-12.md)
no part of the build system was checking that patches listed in
local/patches/<comp>/ had actually been absorbed into the
local/sources/<comp>/ fork's working tree.
This commit adds a tool that audits every Cat 2 fork and reports any
patches whose content is not present in the fork HEAD. The tool is
located at local/scripts/verify-patch-content.{sh,py} and follows the
existing local/scripts/ convention (shell wrapper delegating to
Python implementation).
Audit method:
1. For each .patch in local/patches/<comp>/:
- Extract target file from ---/+++ headers
- Extract first 5 added lines (+ markers)
- Check substring presence in fork HEAD version
2. If 0/5 added lines found, patch is 'orphaned'
Current state (after Phase 1.0A recovery):
- 251 patches total, 155 preserved, 96 still orphaned
- The remaining orphans are patches where the upstream file structure
has shifted too far for patch(1) to apply — those need
upstream-tracking upgrade, deferred to Phase 2.
Usage:
./local/scripts/verify-patch-content.sh (report)
./local/scripts/verify-patch-content.sh --strict (exit 1 on orphans)
./local/scripts/verify-patch-content.sh base kernel (specific forks)
Wired in via build-preflight.sh in the next commit (Phase 1.0B cont.).
This closes the 'silent patch loss' gap identified in the audit.
When the upstream commit hash of the claimed upstream tag has not changed
since the last successful verify, reuse the cached file list instead of
re-cloning and re-hashing from scratch. This makes the canonical preflight
fast on the common case where upstream didn't move.
Cache is stored in .redbear-fork-verify/<fork>-<tag>.fingerprint. Touched
only after a successful full content diff, so cache corruption is detected
on the next run (commit hash mismatch triggers fresh clone).
The verify-overlay-integrity.sh script is now active. It runs at the
start of canonical builds and auto-repairs via apply-patches.sh on
failure.
Restored missing symlinks for:
- recipes/core/base/redox.patch
- recipes/core/bootloader/redox.patch
- recipes/core/bootloader/P2-live-preload-guard.patch
- recipes/core/bootloader/P3-uefi-live-image-safe-read.patch
- recipes/wip/wayland/qt6-wayland-smoke (incorrect relative path)
Created empty stub at local/patches/base/redox.patch so the relibc/base
symlinks can be re-created by apply-patches.sh.
Overlay integrity now reports:
365 recipe symlinks, 0 broken
9 patch symlinks, 0 broken
9 critical patches, 0 missing
10 critical configs, 0 missing
CRITICAL: Add verify-fork-functions.sh — detects silent upstream code loss
from bad merges that file-level verification cannot catch. This is the
verification that would have caught the kfdwrite drop bug.
Wire it into build-preflight.sh (pre-build gate) and upgrade-forks.sh
(post-upgrade gate with automatic rollback on failure).
Fix upgrade-forks.sh:
- Fetch failures now abort instead of being silently swallowed
- Backup branch names include fork name + PID to prevent collisions
- Post-upgrade verification runs verify-fork-functions.sh and rolls
back if upstream functions are missing
Fix bump-fork.sh:
- Replace wrong 'git tag -e' with 'git rev-parse --verify refs/tags/'
- Clone failures now error explicitly instead of silent fallback
- Version sed uses proper regex escaping for + characters
- Atomic swap has rollback recovery if mv fails
- Instructions now mention fork-upstream-map.toml update
Fix sync-upstream.sh:
- Define PROJECT_ROOT before use (was crashing under set -u)
Fix build-preflight.sh:
- Add REDBEAR_SKIP_FUNCTION_CHECK gate for verify-fork-functions.sh
Kernel submodule pointer updated to 0.3.1 branch with kfdwrite restored.
Per user instruction — upgrade to latest. Redox mesa fork at redox-24.0
has no redox-26.0 branch; switched to tar-based upstream 26.1.4 with
Red Bear patches applied on top.
Patch dry-run results vs 26.1.4:
✅ 02-gbm-dumb-prime-export.patch
✅ 04-sys-ioccom-stub-header.patch
✅ 05-vk-sync-wchar-include.patch
🔴 01-virgl-redox-disk-cache.patch (hunk #1 FAIL at 1054)
🔴 03-platform-redox-gpu-probe.patch (8 hunks ignored)
🔴 06-redox-surface-image-fields.patch (hunk #1 FAIL at 333)
🔴 07-wayland-scanner-env-override.patch (hunk #1 FAIL at 1992)
Patches 01,03,06,07 require manual rebase — documented in recipe comment.
Added mesa to apply-patches.sh graphics symlinks for local fork tracking.
- check-fork-drift.sh: compares each submodule HEAD against upstream/master,
reports behind/ahead counts, exits 1 if any fork exceeds threshold
- upgrade-forks.sh: fetches latest upstream, saves RB commits, resets to
upstream, cherry-picks RB patches with conflict handling and backups
- build-preflight.sh: wired drift check into preflight (threshold=50,
suppressible via REDBEAR_SKIP_DRIFT_CHECK=1)
- bootloader submodule pointer updated to latest upstream rebase
Live upstream versions verified 2026-07-11:
- gnu-grep 3.12 (2025-04-10) -- https://ftp.gnu.org/gnu/grep/
- libsodium 1.0.22-stable (2026-07-08) -- https://download.libsodium.org/libsodium/releases/
- autoconf 2.73 (2026-03-20) -- https://ftp.gnu.org/gnu/autoconf/
Each recipe updated with new tar URL + BLAKE3 hash. Build-tool upgrades are
isolated from the fork content-verification system, so these are SAFE to
upgrade without the relibc-style risk.
diffutils was already at 3.12 (previously committed).
Also: fork-upstream-map.toml — bootloader flagged PENDING_REBASE
(2026-07-11 detection: 1.0.0 tag mismatch, fork based on 0.1.0 archive
not a true rebase — documented inline).
Context: A relibc fork check via cargo compare exposed that the fork
labeled 0.6.0 (matching a 2020-12-23 upstream tag) was on a
completely different codebase from upstream master. The fork was
imported as a snapshot. The verify-fork-versions.sh tool allowed
this because 'snapshot' mode skipped byte-for-byte content comparison.
Changes:
1. Remove 'snapshot = skip content check' behavior. All Cat 2 forks must
pass actual content comparison against their claimed upstream tag.
2. Add base to local/fork-upstream-map.toml so it's checked too (was missing).
Effect: Future 'fake version label' divergences (Cargo.toml says
version X but source content does not match upstream X) will be
caught by the preflight check instead of slipping through build.
Implements 7 of 8 improvements from local/docs/BUILD-SYSTEM-IMPROVEMENT-PROPOSAL.md:
Improvement 1+5: trap-based stash-and-restore for ALL 9 fork sources
- Replaces single-relibc stash with a loop over the canonical fork
inventory (relibc, kernel, base, bootloader, installer, redoxfs,
libredox, syscall, userutils)
- Records each stash SHA in a label-keyed map for safe round-tripping
- Pop stashes in LIFO order matching user-visible stash pop convention
- Idempotent: re-entry during the same build does not double-stash
- Reports (does not silently swallow) real git errors during stash push
- Restored on EXIT regardless of success/failure/signal
Improvement 3: cookbook binary freshness check
- Replaces existence-only check with BLAKE3-of-src/.rs fingerprint
- Hash written to target/release/.cookbook-src-fingerprint
- Rebuild triggers on any source file content change, not just mtime
- Survives git operations that change content without touching mtime
Improvement 4: strict-by-default uncommitted-edit gate
- Refuses to build with uncommitted edits in any fork source
- Catches the 'works on my machine' bug class where pkgar fingerprints
silently mismatch the committed HEAD
- Escape hatch: REDBEAR_ALLOW_DIRTY=1 for emergency CI use
- apply-durable-source-edits.py auto-enables strict mode when
REDBEAR_BUILD_PHASE is set (set by build-redbear.sh)
Improvement 6: failure-cleanup trap with diagnostics
- EXIT trap captures last 30 lines of each cookbook log
- Lists orphaned cook/cargo processes (often root cause of follow-on
build failures via held locks)
- Reports per-recipe build state (complete vs incomplete)
- Preserves diagnostics in REDBEAR_BUILD_STATE_DIR (mktemp -d)
- REDBEAR_KEEP_BUILD_STATE=1 retains the state dir after exit
Improvement 7: pre-cook offline/upstream consistency
- Pre-cook now follows the same offline policy as the main build phase
- REDBEAR_ALLOW_UPSTREAM=1 → offline=false
- default / REDBEAR_RELEASE → offline=true
- Pre-cook logs captured into REDBEAR_BUILD_LOGS_DIR for diagnostics
- Sets REDBEAR_BUILD_PHASE=pre-cook so downstream code can distinguish
Improvement 8: cookbook protection against out-of-band invocations
- src/bin/repo.rs emits a warning when invoked without
REDBEAR_CANONICAL_BUILD=1 in the environment
- The warning lists what is bypassed (cache invalidation, prefix
rebuild, fingerprint tracking, dirty gate)
- Non-fatal: CI scripts that manage their own checks are unaffected
Smoke-tested: ./local/scripts/build-redbear.sh redbear-mini now exits 1
when relibc/kernel have uncommitted edits, and proceeds when
REDBEAR_ALLOW_DIRTY=1 is set.
Missing source trees (tar present but not extracted) are non-fatal — the
cookbook fetches/extracts them during build. Also fixed the
--missing-paths-only branch (previous commit).
Missing source trees (e.g. plasma-desktop with recipe.toml but not yet
extracted source/) should not block the build. Sources are fetched/extracted
by the cookbook during the build phase. The validator now warns about
missing sources but returns exit code 0.
Changed QEMU device from -device virtio-gpu (2D only) to
-device virtio-vga-gl -display egl-headless for virgl 3D
acceleration testing. The virgl patches are already wired into
Mesa recipe.toml (6 patches); this enables the runtime probe
to select virgl instead of falling back to llvmpipe swrast.
Also updated test-phase4-wayland-qemu.sh (both expect and
smoke sections). Matches plan §5 Blocker Detail #3 fix.
Three Phase 1.3/2.4 fixes that have real runtime impact:
1. local/scripts/build-redbear.sh: Export ARCH and HOST_ARCH at the top
of the script (derived from uname -m when unset). This fixes the
'Unsupported ARCH for QEMU ""' error from the auto-rebuild-prefix
path introduced in Phase 1.3 — the env(1) wrapper was masking
the variables, and even without it mk/config.mk needed explicit
ARCH to be exported to subprocesses. The prefix rule now succeeds
when the fork source is newer than prefix/x86_64-unknown-redox/sysroot.
2. config/redbear-device-services.toml: Remove the three init.d service
files 16_redbear-{acmd,ecmd,usbaudiod}.service. These are USB device
daemons that take <scheme> <port> [<iface>] arguments and panic on
missing args. They are spawned dynamically by pcid-spawner (00_driver-
manager.service) when matching USB hardware is detected. Starting
them as init services caused the boot to fail with
'thread main panicked at redbear-acmd <scheme> <port> <iface>'.
The binaries are still installed via the [packages] section so
pcid-spawner can find and exec them. This matches the Linux model
where cdc_acm / cdc_eem / snd-usb-audio are kernel modules or
udev-spawned, not init services.
3. config/redbear-mini.toml: Add explicit '-K us' to
29_activate_console.service so the inputd daemon activates VT 2 with
a deterministic keymap. Combined with the Russian (ЙЦУКЕН) keymap
added in commit 75f5480f, all 7 layouts (US, GB, Dvorak, Azerty, Bepo,
IT, RU) are now selectable via 'inputd -K <layout>' or by editing
this service.
The old --check section looked for log strings that do not exist in the
xhcid codebase ("xhcid: using MSI/MSI-X interrupt delivery" etc.).
All six grep patterns were fictitious — the script was written ahead of
P0-A1 anticipating different logging.
Rewrite to match actual debug-level output from xhcid:
irq_reactor.rs:208 — "Running IRQ reactor with IRQ file and event queue"
(irq_file is Some — this is the main proof that interrupts fired)
irq_reactor.rs:125 — "Running IRQ reactor in polling mode."
(irq_file is None — must NOT appear in a passing run)
main.rs:88 — "Enabled MSI-X" (debug, MSI-X configured)
main.rs:95 — "Legacy IRQ <n>" (debug, INTx fallback)
main.rs:143 — "XHCI <pci_name>" (info, controller detected)
Also fails if both polling and IRQ-driven mode appear in the same boot.
- Update AGENTS.md single-repo rule to explicitly forbid creating any new
Gitea repositories and to require deleting per-component repos.
- Change version suffix policy from pre-release -rb to build-metadata +rb
in AGENTS.md, sync-versions.sh, apply-rb-suffix.sh, verify-fork-versions.sh.
- Update migration instructions to push to existing submodule/<component>
branches inside RedBear-OS, not create new repos.
- Update BUILD-SYSTEM-IMPROVEMENTS.md and SLEEP-IMPLEMENTATION-PLAN.md to
reference submodule/* branches instead of defunct per-component repos.
- Fix delete-per-component-repos.sh example to use a real historical repo.
The local bootloader fork claims to be upstream 1.0.0 + Red Bear
patches, but the patches in local/patches/bootloader/ do NOT
apply cleanly to upstream 1.0.0. The fork was originally a 0.1.0
baseline with substantial additions that pre-date the upstream-1.0.0
refactor.
Changes:
* local/fork-upstream-map.toml: set bootloader's upstream tag to
PENDING_REBASE. The verifier recognises this as a deliberate
state marker (a fork whose rebase is in progress) and refuses
the build with a clear error pointing the user to the rebase
procedure documented in the map.
* local/scripts/verify-fork-versions.sh: when a fork is marked
PENDING_REBASE in the map, the script reports a dedicated error
message instead of running the upstream content comparison (which
would always fail for a fork in this state).
The current state of the build is:
* 5 of 6 `-rb1` Cat 2 forks pass the no-fake-version-label
check (redoxfs, redox-scheme, kernel, installer, userutils).
* bootloader refuses to build until a real rebase onto a chosen
upstream tag is completed (the patches must apply cleanly with
--fuzz=0).
* installer has additional divergent content that may need a
rebase too (separate operational task).
This is the strict enforcement the user asked for. The build
cannot proceed silently with fake labels. The user must drive
the rebase work for bootloader (and installer) following the
procedure in fork-upstream-map.toml.
Per local/AGENTS.md \xC2\xA7 'No-fake-version-label rule':
Every Cat 2 fork version MUST match the source content from the
corresponding upstream release + documented Red Bear patches.
A `-rbN` label on stale content is a fake label and a policy
violation.
Changes:
* local/AGENTS.md: documented the no-fake-version-label rule,
including what counts as a fake label, the enforcement contract,
and what a real Red Bear fork looks like.
* local/fork-upstream-map.toml: authoritative mapping of each
Cat 2 fork (syscall, libredox, redoxfs, redox-scheme, relibc,
kernel, bootloader, installer, userutils) to its upstream Git
URL and release tag.
* local/scripts/refresh-fork-upstream-map.sh: auto-update the
fork-upstream-map by querying each upstream repo for the
current latest stable release tag.
* local/scripts/verify-fork-versions.sh: preflight enforcement
script. For each Cat 2 fork with a `-rbN` version field:
1. Compare fork's file list and content against the upstream
release tag from the map. Reject the build if files are
missing (would be a fake label).
2. Reject the build if files exist in local that don't exist
in upstream (must be moved to local/patches/<fork>/ as
documented Red Bear patches).
3. Reject the build if shared files diverge in content.
* local/scripts/apply-rb-suffix.sh: invokes
verify-fork-versions.sh after applying the `-rbN` label so the
build fails fast if the labelled content is fake.
* local/scripts/build-preflight.sh: invokes
verify-fork-versions.sh at the start of every build. Bypassed
only with REDBEAR_SKIP_FORK_VERIFY=1 (emergency only).
* local/patches/bottom/0001-ratui-0.30-braille-compat.patch: the
ratatui 0.30+ compatibility shim for bottom 0.11.2.
This is the structural enforcement that prevents fake labels from
ever reaching the build again. The current 6 forks with `-rbN`
labels are flagged by the verifier — they must be rebased onto
their actual upstream release before the build can succeed.
Add comprehensive policy documentation in AGENTS.md covering:
- local/ fork always takes precedence over recipes/ paths
- build system must ensure local fork is at latest available version
- all Red Bear patches must be applied cleanly on top of latest version
- automatic version bump + patch reapplication via bump-fork.sh
Create local/scripts/bump-fork.sh that implements automatic version bumping:
- Detects current local version vs required version from Cargo.lock
- Fetches upstream source at required version
- Applies all Red Bear patches atomically
- Updates version field and replaces local fork contents
Fix driver-manager Cargo.toml lockfile collision:
- Remove redundant syscall dependency (transitive via pcid_interface)
- Update all driver recipes to use local/sources/syscall and libredox paths
- This eliminates the redox_syscall lockfile collision between
local/sources/syscall and recipes/core/base/syscall (same dir, different paths)
relibc: fix unsafe call for Rust 2024 edition compatibility
Per local/AGENTS.md § SINGLE-REPO RULE: the Red Bear OS project lives
in exactly one git repository (vasilito/RedBear-OS). Per-component
Gitea mirrors (redbear-os-base, redbear-os-kernel, redbear-os-installer,
redox-drm, userutils, libredox, libpciaccess, ctrlc, syscall, sysinfo)
have been redirected or deleted.
For each per-component repo with source content, the working-tree HEAD
was pushed as a 'submodule/<component>' branch on RedBear-OS:
- submodule/base
- submodule/bootloader
- submodule/installer
- submodule/kernel
- submodule/libredox
- submodule/redoxfs
- submodule/relibc
- submodule/syscall
- submodule/userutils
The .gitmodules entry for local/sources/kernel is now redirected to the
canonical repo with branch = submodule/kernel. The other submodule
.gitmodules entries remain to be added in a follow-up.
Empty per-component repos (ctrlc, libpciaccess, redox-drm, sysinfo) had
no source content; their gitlinks in the index are removed in a
follow-up commit.
Unrelated per-component repos that were not Red Bear components
(ctrlc, syscall, sysinfo — possibly unrelated personal projects) were
deleted in the bulk cleanup.
Gitea state under vasilito/ is now exactly: RedBear-OS, hiperiso.
Adds:
- local/scripts/redirect-to-submodules.sh
- local/scripts/delete-per-component-repos.sh
Updates:
- .gitmodules (kernel → RedBear-OS#submodule/kernel)
- local/AGENTS.md (SINGLE-REPO RULE status, migration procedure)
- local/docs/BUILD-SYSTEM-IMPROVEMENTS.md §11 (resolved)
- local/docs/QUIRKS-AUDIT.md (drop dead links)
- local/docs/SLEEP-IMPLEMENTATION-PLAN.md (mark historical)
- CHANGELOG.md (mark historical references)
The new `check-cargo-patches.sh` script verifies that all
[patch.crates-io] and [patch.'<URL>'] sections in the local
sources' Cargo.toml files actually resolve to the expected
local fork paths. It does this by running `cargo metadata`
on each source's workspace and checking that the
resolved source URL (or manifest_path for path-deps)
matches the expected local fork path.
This is the Phase J / Improvement C verification step
that the user explicitly requested: 'Build system must
report complete when upstream have our patches applied.'
The script handles the known-large workspaces gracefully:
* relibc is explicitly skipped — its [patch] section is
only the cc-rs git branch override (no `path` patches),
and `cargo metadata` on relibc takes minutes (hundreds
of deps) which would hang the script.
* All other `cargo metadata` calls are wrapped in a
30-second timeout.
Hardware-agnostic: works on any Red Bear OS checkout
regardless of which OEMs are added to the local sources
(Phase I/II/J DMI matches).
The Qt6SvgTargets.cmake config contains absolute paths to qtsvg's own
sysroot/metatypes/ for INTERFACE_SOURCES. When the cookbook cleans
qtsvg's sysroot between recipe builds, these paths dangle and cause
CMake Generate to fail.
Extend redbear_qt_ensure_dep_sysroots to create symlinks for ALL Qt
directories (metatypes, plugins, mkspecs, modules, qml) — not just
include and lib. Add the call to qtdeclarative's recipe (was missing).
redbear_qt_ensure_dep_sysroots was creating symlinks in the dependency's
sysroot pointing to the CALLING recipe's sysroot. This caused CMake
'hidden library' warnings and corrupts the dependency's build when it
gets rebuilt later.
Fix: derive the dependency's stage path from its sysroot path
(dirname(sysroot)/stage/usr) and link to that instead. This way
qtdeclarative's sysroot include/lib point to qtdeclarative's OWN
stage, not to qt6-sensors' sysroot.
Also fix wayland-patch.sh: replace ${LIBXCB_LIBRARIES} with empty
string instead of a comment. The comment was eating the closing )
on the same line, causing 'Function missing ending paren' CMake
parse error in src/daemon/CMakeLists.txt:63.
Three build failures (qt6-sensors, sddm, kwin) all trace to
qtdeclarative's sysroot/ directory being missing after staging.
CMake config files reference absolute paths into that sysroot.
1. qt-sysroot.sh: add mkdir -p before ln -sf in
redbear_qt_ensure_dep_sysroots() so the parent directory is
created when the dependency sysroot was cleaned up.
2. sddm wayland-patch.sh: fix sed pattern to include LinguistTools
token that exists in the actual upstream CMakeLists.txt line.
Without it the sed silently fails to match, leaving
LinguistTools/Test/QuickTest as REQUIRED components.
3. qt6-sensors recipe.toml: source qt-sysroot.sh and call
redbear_qt_ensure_dep_sysroots() before cmake configure.
Previously the recipe only symlinked plugins/mkspecs/metatypes/
modules but never fixed up dependency sysroot paths, causing
Qt6::Qml INTERFACE_INCLUDE_DIRECTORIES to reference a
non-existent include path.
The stale-build check in build-redbear.sh compared HEAD commit hashes
against a stored fingerprint, which silently ignored uncommitted changes
in local/sources/{relibc,kernel,base,bootloader,installer}.
This meant dev iterations where a maintainer edited the working tree
without committing would not trigger a rebuild of the affected package.
The cookbook would then cook the binary from a fingerprint that
claims 'up to date' but is actually older than the working tree.
This commit extends the staleness test to also check
'git diff HEAD', 'git diff --cached HEAD', and
'git ls-files --others --exclude-standard'. The error message
distinguishes 'uncommitted changes' from 'new commits' so the
operator can tell which case triggered the rebuild.
Also adds local/scripts/lint-doc-comments.sh: a doc-comment hygiene
linter that flags agent-memo style comments (Note:, This implements...,
Changed from..., Added new..., Korean variants) so future commits
can be screened for the WHAT-not-WHY comment anti-pattern.
- recipes/dev/python312/recipe.toml: use COOKBOOK_TOOLCHAIN for
--with-build-python instead of /tmp/python312, which the build system
never stages. Add [ -x ] guard for clear failure on missing dev-dep.
- recipes/core/userutils/recipe.toml: switch from upstream git URL to
local fork (local/sources/userutils/) per the local fork model. The
upstream source opens /scheme/pty/ptmx which the ptyd scheme does not
recognize; the local fork opens /scheme/pty correctly and avoids the
getty PTY panic.
- local/scripts/build-preflight.sh: warn when a recipe build script
references /tmp/<known-package>/, since the cookbook does not stage
host dev-deps under /tmp/<name>. Points authors at COOKBOOK_TOOLCHAIN.
- local/scripts/build-redbear.sh: replace 'tail -1 || true' on pre-cook
failures with proper error capture, last-50-lines tail on failure, and
exit-1. Verify the pkgar exists after a successful cook.
Add local/scripts/cleanup-build.sh - a git-aware cleanup script that
uses 'git ls-files' to whitelist tracked files before deletion.
Prevents the class of cleanup disasters that deleted local recipe
sources and local fork sources.
Update AGENTS.md with the new safe cleanup procedure.
Update CONSOLE-TO-KDE-DESKTOP-PLAN.md to v5.8 with the qtdeclarative
qfeatures.h fix and the new safe cleanup script.
vasilito
vasilito