docs: add canonical BUILD-SYSTEM.md; update build docs for flags + no-stashing
Add local/docs/BUILD-SYSTEM.md as the single authoritative build-system reference (entry point, CLI flags, pipeline stages, offline/release, caching, no working-tree stashing, fork/vendored-upstream versioning, self-versioning). Update AGENTS.md (dirty-gate replaces the removed stash-and-restore; --allow-dirty; pointer), SCRIPT-BEHAVIOR-MATRIX.md (stash line -> dirty-source gate), README.md (flags + pointer), and add canonical-doc pointers to BUILD-SYSTEM-INVARIANTS.md and docs/06-BUILD-SYSTEM-SETUP.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -115,7 +115,7 @@ any package, or any target artifact with anything else. This is absolute:
|
||||
|---|---|
|
||||
| `cargo check` / `cargo build` / `cargo clippy` in any source tree | Bypasses the cross-compile target, sysroot, feature flags, and workspace orchestration. Produces wrong-target or mis-configured output and misleading "it compiles" claims. |
|
||||
| `cargo test --target x86_64-unknown-redox` (or any target) in a source tree | Same problem — compiles outside the cookbook's sysroot/prefix. |
|
||||
| `./target/release/repo cook <pkg>` **as a substitute for a build** | Diagnostic/diagnostic-only. Outside build-redbear.sh it runs without prefix-staleness detection, source-fingerprint tracking, the concurrency lock, branch gates, or the stash-and-restore guard. The cookbook itself warns: *"repo invoked outside build-redbear.sh"*. |
|
||||
| `./target/release/repo cook <pkg>` **as a substitute for a build** | Diagnostic/diagnostic-only. Outside build-redbear.sh it runs without prefix-staleness detection, source-fingerprint tracking, the concurrency lock, the branch gate, or the dirty-source gate. The cookbook itself warns: *"repo invoked outside build-redbear.sh"*. |
|
||||
| `make r.<pkg>` / `make cr.<pkg>` **as a substitute for a build** | Same — a thin wrapper over `repo cook`, with the same missing guarantees. |
|
||||
| `make all` / `make live` directly | build-redbear.sh wraps these with the required preflight (prefix rebuild, version sync, config parsing, protected-fetch policy). Calling them directly skips that preflight. |
|
||||
|
||||
@@ -135,15 +135,22 @@ produces a Red Bear target artifact.
|
||||
- **Prefix staleness detection + rebuild** — the #1 cause of "undefined reference" link
|
||||
errors after relibc/kernel/base fork changes is a stale prefix; only build-redbear.sh
|
||||
detects and rebuilds it before any recipe compiles.
|
||||
- **Stash-and-restore** — dirty fork sources (`local/sources/*`) are stashed before the
|
||||
build and popped on EXIT, so fingerprints reflect committed HEAD. **Uncommitted fork
|
||||
changes are NOT built** — commit to the `submodule/<name>` branch first.
|
||||
- **Dirty-source gate** — the build refuses if any fork (`local/sources/*`) has
|
||||
uncommitted edits, so fingerprints reflect committed HEAD. **Uncommitted fork changes
|
||||
are NOT built** — commit to the `submodule/<name>` branch first, or pass `--allow-dirty`
|
||||
to deliberately cook the working tree AS-IS. (The build never stashes your tree; see
|
||||
`local/docs/BUILD-SYSTEM.md` §6.)
|
||||
- **Branch topology gate** — every fork worktree must be on its canonical
|
||||
`submodule/<name>` branch; the build refuses otherwise.
|
||||
- **Source-fingerprint tracking + content-hash cache** — correct incremental rebuilds.
|
||||
- **`.config` parsing + protected-fetch policy + version sync + pre-cooking.**
|
||||
- Sets `REDBEAR_CANONICAL_BUILD=1` so the cookbook knows it is inside the pipeline.
|
||||
|
||||
**Full reference:** `local/docs/BUILD-SYSTEM.md` is the canonical build-system doc
|
||||
(flags, pipeline stages, offline/release, caching, versioning). `build-redbear.sh
|
||||
--help` lists the flags; the common knobs are flags (`--allow-dirty`,
|
||||
`-j/--jobs`, `--release`, …), rarer escape hatches remain `REDBEAR_*` env vars.
|
||||
|
||||
**Correct per-component verification flow (the ONLY flow):**
|
||||
1. Make your edits.
|
||||
2. Commit to the right branch: fork code → `submodule/<name>` in `local/sources/<name>`;
|
||||
|
||||
@@ -89,8 +89,9 @@ cd RedBear-OS
|
||||
git clone https://vasilito:${REDBEAR_GITEA_TOKEN}@gitea.redbearos.org/vasilito/RedBear-OS.git
|
||||
|
||||
# Canonical build entry point
|
||||
./local/scripts/build-redbear.sh redbear-mini # Text-only target
|
||||
./local/scripts/build-redbear.sh redbear-full # Desktop-capable target
|
||||
./local/scripts/build-redbear.sh redbear-mini # Text-only target
|
||||
./local/scripts/build-redbear.sh redbear-full # Desktop-capable target
|
||||
./local/scripts/build-redbear.sh -j 8 --allow-dirty redbear-mini # flags: see --help
|
||||
|
||||
# Boot in QEMU
|
||||
make qemu
|
||||
@@ -99,7 +100,9 @@ make qemu
|
||||
> `local/scripts/build-redbear.sh` is the **only supported build entry point**. It handles
|
||||
> `.config` parsing, prefix staleness detection, protected-recipe authorization, pre-cooking
|
||||
> critical packages, and source fingerprint tracking. Direct `make` invocations bypass these
|
||||
> gates. See [`AGENTS.md` § Build Commands](./AGENTS.md) for details.
|
||||
> gates. Run `build-redbear.sh --help` for flags. Full reference:
|
||||
> [`local/docs/BUILD-SYSTEM.md`](./local/docs/BUILD-SYSTEM.md); see also
|
||||
> [`AGENTS.md` § Build Commands](./AGENTS.md).
|
||||
|
||||
### Config Targets
|
||||
|
||||
|
||||
@@ -1,5 +1,9 @@
|
||||
# 06 — Build System Setup Guide
|
||||
|
||||
> **Canonical build-system reference:** `local/docs/BUILD-SYSTEM.md` (entry point,
|
||||
> flags, pipeline stages, offline/release, caching, versioning). This file covers
|
||||
> host prerequisites and general setup mechanics.
|
||||
>
|
||||
> **Status note (2026-04-15):** This file explains the mechanics of building the repository, but it
|
||||
> is not the canonical source for repository ownership policy or current execution order. For the
|
||||
> current repository model, use `README.md`, `AGENTS.md`, and
|
||||
|
||||
@@ -1,5 +1,9 @@
|
||||
# Build System Invariants
|
||||
|
||||
> Canonical overview: **[`BUILD-SYSTEM.md`](BUILD-SYSTEM.md)** (entry point, flags,
|
||||
> pipeline stages). This document is the deep-dive on authoritative surfaces and
|
||||
> propagation order.
|
||||
|
||||
## Purpose
|
||||
|
||||
This document defines the non-negotiable build-system invariants for Red Bear OS.
|
||||
|
||||
@@ -0,0 +1,159 @@
|
||||
# Red Bear OS — Build System (canonical reference)
|
||||
|
||||
This is the single authoritative description of how Red Bear OS is built. Deeper
|
||||
dives live in companion docs and are linked where relevant:
|
||||
`BUILD-SYSTEM-INVARIANTS.md` (surfaces & propagation order),
|
||||
`SCRIPT-BEHAVIOR-MATRIX.md` (per-script roles), `PATCH-GOVERNANCE.md`,
|
||||
`BUILD-CACHE-PLAN.md`, `RELEASE-BUMP-WORKFLOW.md`, and host setup in
|
||||
`../../docs/06-BUILD-SYSTEM-SETUP.md`.
|
||||
|
||||
---
|
||||
|
||||
## 1. The one entry point
|
||||
|
||||
```
|
||||
./local/scripts/build-redbear.sh [OPTIONS] [CONFIG]
|
||||
```
|
||||
|
||||
`build-redbear.sh` is the ONLY supported way to build. It sets
|
||||
`REDBEAR_CANONICAL_BUILD=1` and orchestrates every stage below. Do not invoke
|
||||
`repo cook`, `make live`, or `cargo build --target …` directly as a build
|
||||
substitute — they run outside the gates and produce off-pipeline artifacts
|
||||
(diagnostic use only).
|
||||
|
||||
`CONFIG` is one of:
|
||||
|
||||
| Config | What it builds |
|
||||
|--------|----------------|
|
||||
| `redbear-full` | Desktop/graphics target (Wayland/KDE/Qt/Mesa). **Default.** |
|
||||
| `redbear-mini` | Text-only console/recovery target |
|
||||
| `redbear-grub` | Text-only with the GRUB boot manager |
|
||||
| `redbear-bare` | Kernel + 7-daemon initfs + login shell (minimum bootable) |
|
||||
|
||||
The running script prints its own version banner (magenta) at start, e.g.
|
||||
`══════ build-redbear.sh v1.3 ══════`. `--version` prints full provenance
|
||||
(script version, OS version from the git branch, repo commit, host, arch).
|
||||
|
||||
## 2. Options (flags)
|
||||
|
||||
Flags are the canonical interface. Colored output is automatic on a TTY and
|
||||
suppressed when piped or when `NO_COLOR` is set.
|
||||
|
||||
**Common:**
|
||||
|
||||
| Flag | Effect |
|
||||
|------|--------|
|
||||
| `-j, --jobs N` | Parallel build jobs (default: `nproc`) |
|
||||
| `--upstream` | Allow upstream recipe source refresh (goes online) |
|
||||
| `--no-cache` | Force a clean rebuild, discarding cached packages |
|
||||
| `--release VER` | Build from the immutable release archive `VER` |
|
||||
| `--allow-dirty` | Build with uncommitted fork edits — **cooks the working tree AS-IS** |
|
||||
| `--keep-build-state` | Keep the diagnostic state dir after exit |
|
||||
| `-h, --help` / `-V, --version` | Help / version |
|
||||
|
||||
**Advanced escape hatches** stay as environment variables (rare; emergency/CI):
|
||||
`REDBEAR_ALLOW_CONCURRENT`, `REDBEAR_ALLOW_WRONG_BRANCH`,
|
||||
`REDBEAR_SKIP_PREFIX_REBUILD`, `REDBEAR_FORCE_TOOLCHAIN_RECOOK`,
|
||||
`REDBEAR_SKIP_FORK_VERIFY`, `REDBEAR_SKIP_DRIFT_CHECK`,
|
||||
`REDBEAR_SKIP_FUNCTION_CHECK`, `REDBEAR_SKIP_COLLISION_CHECK`,
|
||||
`REDBEAR_SKIP_PATCH_CONTENT_CHECK`, `REDBEAR_STRICT_DURABILITY`,
|
||||
`REDBEAR_STRICT_METADATA`, `REDBEAR_NO_VALIDATE`, `REDBEAR_COLLISIONS_WARN`.
|
||||
The common flags' old env vars (`REDBEAR_ALLOW_DIRTY`, `REDBEAR_KEEP_BUILD_STATE`,
|
||||
`REDBEAR_ALLOW_UPSTREAM`, `REDBEAR_RELEASE`, `JOBS`) are still honored as
|
||||
**deprecated fallbacks**; a flag always overrides the environment.
|
||||
|
||||
## 3. What a build does, in order
|
||||
|
||||
1. **Concurrency lock** — a `flock` refuses a second concurrent build
|
||||
(`REDBEAR_ALLOW_CONCURRENT=1` overrides).
|
||||
2. **Cookbook binary** — rebuilds `target/release/repo` if any `src/*.rs`
|
||||
changed (content fingerprint, not mtime).
|
||||
3. **Overlay integrity** — verifies `recipes/ → local/recipes/` symlinks; repairs
|
||||
via `guard-recipes.sh --fix` if needed.
|
||||
4. **Local-over-WIP** — `recipes/wip/*` that shadow a `local/recipes/` package
|
||||
are replaced with symlinks to the local version.
|
||||
5. **Dirty-source gate** — refuses to build if any `local/sources/<fork>` has
|
||||
uncommitted edits, unless `--allow-dirty` (then it cooks the tree as-is). The
|
||||
build does **not** stash your tree — see §6.
|
||||
6. **Branch gate** — each fork must be on its `submodule/<name>` branch.
|
||||
7. **Staleness invalidation** — if a fork changed, invalidate the right pkgars.
|
||||
relibc/base/etc. only relink the static initfs-critical binaries (base,
|
||||
redoxfs, userutils, bootstrap); the dynamically-linked desktop is untouched
|
||||
(shared `libc.so`, the glibc model). Host fstools are rebuilt if
|
||||
installer/redoxfs changed.
|
||||
8. **Prefix sysroot** — if relibc is newer than the prefix `libc.a`, the derived
|
||||
prefix markers are removed and `make prefix` genuinely re-cooks relibc into
|
||||
the sysroot (a no-op is reported honestly, never as success). kernel/base-only
|
||||
staleness does not touch the prefix.
|
||||
9. **Preflight** — `build-preflight.sh`: repairs broken recipe symlinks, fork
|
||||
version/drift/function verification, collision detection, source-tree
|
||||
validation. The fork-function check treats inert std-trait methods
|
||||
(`fmt`/`eq`/`clone`/…) as warnings; intentional divergences are listed in each
|
||||
fork's `.verify-fork-functions.exclude`.
|
||||
10. **Version sync check** — `sync-versions.sh --check` (see §7).
|
||||
11. **Pre-cook** — cooks a small ordered set of critical packages (`CI=1`,
|
||||
offline unless `--upstream`).
|
||||
12. **`make live`** — the cookbook builds the config's package graph and the
|
||||
installer assembles the image.
|
||||
13. **Critical-package gate** — for `redbear-full`, fails loudly if mesa/qt/…
|
||||
pkgars are missing (cookbook returns 0 even when recipes fail).
|
||||
14. **Fingerprints** — fork source fingerprints are recorded **only after** a
|
||||
successful build, so a failed build never marks forks as "built".
|
||||
|
||||
On failure the EXIT trap dumps diagnostics (last log lines, orphaned cook
|
||||
processes, and per-recipe state **scoped to this build** so a mini build never
|
||||
lists leftover graphical packages) and preserves the state dir under
|
||||
`--keep-build-state`.
|
||||
|
||||
## 4. Offline / release mode
|
||||
|
||||
Cooking is offline by default (`COOKBOOK_OFFLINE=true`, `REPO_OFFLINE=1`).
|
||||
`--upstream` goes online for non-protected recipes. `--release VER` builds from
|
||||
`sources/redbear-<VER>/` immutable archives and forces offline; new releases are
|
||||
created only via `provision-release.sh`. A stray `REDBEAR_RELEASE=` line in
|
||||
`.config` silently switches the whole build off the local forks — the script
|
||||
never auto-sets it. See `RELEASE-BUMP-WORKFLOW.md`.
|
||||
|
||||
## 5. Caching & staleness
|
||||
|
||||
Per-recipe caching is BLAKE3 content-hash based (`dep_hashes.toml` +
|
||||
`source_hash.txt`), not mtime. `--no-cache` wipes `repo/` and recipe `target/`
|
||||
dirs. Cargo path-dep crates (syscall/libredox/redox-scheme) have no pkgar, so
|
||||
`build-redbear.sh` tracks their source fingerprints and forces a userspace
|
||||
relink when they change. See `BUILD-CACHE-PLAN.md`.
|
||||
|
||||
## 6. No working-tree stashing (design)
|
||||
|
||||
The build **never stashes your working tree.** An earlier version stashed dirty
|
||||
forks and restored them on exit, but modern git's `stash push` does not print the
|
||||
stash SHA on stdout, so the stash was recorded incorrectly and never restored —
|
||||
stranding operator WIP in `git stash list` on every dirty build. The machinery
|
||||
was removed entirely. Today: a clean tree cooks committed HEAD; `--allow-dirty`
|
||||
cooks the tree as-is. A read-only startup advisory reports any leftover
|
||||
`redbear-build-*` stashes from the old code (recover them from
|
||||
`git stash list`; see `../recovered-stashes/README.md`).
|
||||
|
||||
## 7. Forks & versioning
|
||||
|
||||
`local/sources/<fork>/` (the 9 submodule forks) take precedence over
|
||||
`recipes/<fork>/`. Version policy (`sync-versions.sh`):
|
||||
|
||||
- **Cat 1** — in-house crates under `local/recipes/*/source/`: `version = <branch>`
|
||||
(e.g. `0.3.1`).
|
||||
- **Cat 2** — upstream forks under `local/sources/*/`:
|
||||
`version = <upstream-tag>+rb<branch>` (e.g. `0.9.0+rb0.3.1`; `+rb` build
|
||||
metadata, never `-rb`).
|
||||
- **Vendored upstream** under `local/recipes/` (e.g. `brush`, a Cargo workspace):
|
||||
keeps its **upstream** crate versions (internal `^` requirements). Mark it with
|
||||
a `.vendored-upstream` file in the recipe dir so `sync-versions.sh` skips it —
|
||||
do **not** stamp it with the branch version. `zbus`/`tlc` are excluded by name.
|
||||
|
||||
`sync-versions.sh --check` is a build-time gate; `sync-versions.sh` applies the
|
||||
sync. See `PATCH-GOVERNANCE.md` for the durability rules (source trees are
|
||||
disposable; only `local/` survives).
|
||||
|
||||
## 8. Self-versioning of this script
|
||||
|
||||
`BUILD_REDBEAR_VERSION` in `build-redbear.sh` starts at `1.0` and is auto-bumped
|
||||
(minor) by a `pre-commit` git hook (`bump-build-version.sh`) whenever the script
|
||||
is committed. Install hooks with `./local/scripts/install-git-hooks.sh`.
|
||||
@@ -25,7 +25,7 @@ The goal is to remove guesswork from the sync/fetch/apply/build workflow.
|
||||
|---|---|---|---|
|
||||
| `local/scripts/provision-release.sh` | Refresh top-level upstream repo state | fetches upstream, reports conflict risk, rebases repo commits. Under v6.0 the "release fork reapplication" step is no longer needed because `local/sources/`, `local/recipes/`, and `local/patches/<component>/` already live in the main repo (Rule 1 + Rule 2). | does not automatically solve every subsystem release fork conflict; does not by itself make upstream WIP recipes safe shipping inputs |
|
||||
| `local/scripts/apply-patches.sh` | **LEGACY / ARCHIVED** — historical overlay only | under v5.x, applied build-system patches and relinked recipe patch symlinks; under v6.0 this is a no-op for in-tree components (Rule 1 direct edits) and is replaced by `cookbook_apply_patches` for big external projects (Rule 2). See `local/AGENTS.md`. | do not invoke during a v6.0 build. The `local/scripts/build-redbear.sh <profile>` canonical entry point never calls it. |
|
||||
| `local/scripts/build-redbear.sh` | **Canonical build entry point** for Red Bear profiles | under v6.0 it does NOT call `apply-patches.sh` — the release fork is already in `local/`. It enforces: (1) local-over-WIP recipe priority, (2) overlay integrity verification, (3) submodule dirty-state stash, (4) firmware presence warning, (5) profile validation, (6) cookbook build if needed, (7) image build. `--upstream` triggers explicit source immutable archived for non-protected recipes. | does not guarantee every nested upstream source tree is fresh; does not replace explicit subsystem/runtime validation |
|
||||
| `local/scripts/build-redbear.sh` | **Canonical build entry point** for Red Bear profiles | under v6.0 it does NOT call `apply-patches.sh` — the release fork is already in `local/`. It enforces: (1) local-over-WIP recipe priority, (2) overlay integrity verification, (3) dirty-source gate (refuses uncommitted fork edits unless `--allow-dirty`; it does NOT stash the tree), (4) firmware presence warning, (5) profile validation, (6) cookbook build if needed, (7) image build. Canonical reference: `BUILD-SYSTEM.md`. `--upstream` triggers explicit source immutable archived for non-protected recipes. | does not guarantee every nested upstream source tree is fresh; does not replace explicit subsystem/runtime validation |
|
||||
| `scripts/fetch-all-sources.sh` | Fetch mainline recipe source inputs for builds | downloads mainline/upstream recipe sources, reports status/preflight, and supports config-scoped fetches while leaving local release fork in place | does not mean fetched upstream WIP source is the durable shipping source of truth |
|
||||
| `local/scripts/fetch-sources.sh` | Fetch mainline recipe sources for browsing and patching | when passed `--upstream`, fetches `recipes/*` source trees so the upstream-managed side is locally available for reading, editing, and patch preparation | does not decide whether upstream should replace the local release fork |
|
||||
| `local/scripts/build-redbear-wifictl-redox.sh` | Build `redbear-wifictl` for the Redox target with the repo toolchain | prepends `prefix/x86_64-unknown-redox/sysroot/bin` to `PATH` and runs `cargo build --target x86_64-unknown-redox` in the `redbear-wifictl` crate | does not prove runtime Wi-Fi behavior; only closes the target-build environment gap for this crate |
|
||||
|
||||
Reference in New Issue
Block a user