RedBear-OS/local/docs/BUILD-SYSTEM.md

# Red Bear OS — Build System (canonical reference)

This is the single authoritative description of how Red Bear OS is built. Deeper
dives live in companion docs and are linked where relevant:
`BUILD-SYSTEM-INVARIANTS.md` (surfaces & propagation order),
`SCRIPT-BEHAVIOR-MATRIX.md` (per-script roles), `PATCH-GOVERNANCE.md`,
`BUILD-CACHE-PLAN.md`, `RELEASE-BUMP-WORKFLOW.md`, and host setup in
`../../docs/06-BUILD-SYSTEM-SETUP.md`.

---

## 1. The one entry point

```
./local/scripts/build-redbear.sh [OPTIONS] [CONFIG]
```

`build-redbear.sh` is the ONLY supported way to build. It sets
`REDBEAR_CANONICAL_BUILD=1` and orchestrates every stage below. Do not invoke
`repo cook`, `make live`, or `cargo build --target …` directly as a build
substitute — they run outside the gates and produce off-pipeline artifacts
(diagnostic use only).

`CONFIG` is one of:

| Config | What it builds |
|--------|----------------|
| `redbear-full` | Desktop/graphics target (Wayland/KDE/Qt/Mesa). **Default.** |
| `redbear-mini` | Text-only console/recovery target |
| `redbear-grub` | Text-only with the GRUB boot manager |
| `redbear-bare` | Kernel + 7-daemon initfs + login shell (minimum bootable) |

The running script prints its own version banner (magenta) at start, e.g.
`══════ build-redbear.sh  v1.3 ══════`. `--version` prints full provenance
(script version, OS version from the git branch, repo commit, host, arch).

## 2. Options (flags)

Flags are the canonical interface. Colored output is automatic on a TTY and
suppressed when piped or when `NO_COLOR` is set.

**Common:**

| Flag | Effect |
|------|--------|
| `-j, --jobs N` | Parallel build jobs (default: `nproc`) |
| `--upstream` | Allow upstream recipe source refresh (goes online) |
| `--no-cache` | Force a clean rebuild, discarding cached packages |
| `--release VER` | Build from the immutable release archive `VER` |
| `--allow-dirty` | Build with uncommitted fork edits — **cooks the working tree AS-IS** |
| `--keep-build-state` | Keep the diagnostic state dir after exit |
| **`--check-sweep`** | **Pre-build `cargo check` on every fork + config recipe against the target triple; aborts on any error before the cook/prefix cycle runs. See §2.1 "Pre-build check sweep" below.** |
| `-h, --help` / `-V, --version` | Help / version |

**Advanced escape hatches** stay as environment variables (rare; emergency/CI):
`REDBEAR_ALLOW_CONCURRENT`, `REDBEAR_ALLOW_WRONG_BRANCH`,
`REDBEAR_SKIP_PREFIX_REBUILD`, `REDBEAR_FORCE_TOOLCHAIN_RECOOK`,
`REDBEAR_SKIP_FORK_VERIFY`, `REDBEAR_SKIP_DRIFT_CHECK`,
`REDBEAR_SKIP_FUNCTION_CHECK`, `REDBEAR_SKIP_COLLISION_CHECK`,
`REDBEAR_SKIP_PATCH_CONTENT_CHECK`, `REDBEAR_STRICT_DURABILITY`,
`REDBEAR_STRICT_METADATA`, `REDBEAR_NO_VALIDATE`, `REDBEAR_COLLISIONS_WARN`.
The common flags' old env vars (`REDBEAR_ALLOW_DIRTY`, `REDBEAR_KEEP_BUILD_STATE`,
`REDBEAR_ALLOW_UPSTREAM`, `REDBEAR_RELEASE`, `JOBS`) are still honored as
**deprecated fallbacks**; a flag always overrides the environment.

## 2.1 Pre-build check sweep (`--check-sweep`)

`--check-sweep` runs `cargo check --target <triple> --offline` over every
fork source under `local/sources/*/` and every Rust recipe referenced by
the target config (mini-build does not check graphics recipes; full-build
checks all). It runs **before** the cook/prefix cycle so that a
type/borrow error in any single fork is surfaced up-front, not at the
end of a multi-minute relibc rebuild.

Implementation: `redbear_check_sweep()` in `local/scripts/build-redbear.sh`.
It uses the redoxer toolchain's `cargo` (so it needs `~/.redoxer/x86_64-unknown-redox/toolchain/bin/cargo`
provisioned — run a full build once if it is missing). `cargo check` does
no linking, so neither `libc.a` nor a C linker is required; this is a
fast pure-Rust type/borrow pass.

Behavior:

- One `cargo check` invocation per fork and per config recipe. Output is
  tee'd to `$REDBEAR_BUILD_LOGS_DIR/check-sweep-<name>.log`.
- On any error, the sweep aborts with exit 1 and prints a summary listing
  the failing packages. Fix those before invoking the canonical build —
  the actual `make live` would otherwise surface the same errors one
  package at a time, deep inside a relibc rebuild.
- On success, the sweep prints `Check-sweep passed: all N package(s)
  type-check clean.` and exits 0. The build then proceeds with the
  cook/prefix cycle as normal.

Skip list: `bootloader` (bare-metal/UEFI, built for its own custom
targets under `targets/*.json`, not for the userspace triple). All other
forks are checked.

The current canonical evidence: 48/48 packages pass on
`redbear-mini` with `--check-sweep` after the 2026-07-28 round of
logging/error-handling/file-split refactors.

## 3. What a build does, in order

1. **Concurrency lock** — a `flock` refuses a second concurrent build
   (`REDBEAR_ALLOW_CONCURRENT=1` overrides).
2. **Cookbook binary** — rebuilds `target/release/repo` if any `src/*.rs`
   changed (content fingerprint, not mtime).
3. **Overlay integrity** — verifies `recipes/ → local/recipes/` symlinks; repairs
   via `guard-recipes.sh --fix` if needed.
4. **Local-over-WIP** — `recipes/wip/*` that shadow a `local/recipes/` package
   are replaced with symlinks to the local version.
5. **Dirty-source gate** — refuses to build if any `local/sources/<fork>` has
   uncommitted edits, unless `--allow-dirty` (then it cooks the tree as-is). The
   build does **not** stash your tree — see §6.
6. **Branch gate** — each fork must be on its `submodule/<name>` branch.
7. **Staleness invalidation** — if a fork changed, invalidate the right pkgars.
   relibc/base/etc. only relink the static initfs-critical binaries (base,
   redoxfs, userutils, bootstrap); the dynamically-linked desktop is untouched
   (shared `libc.so`, the glibc model). Host fstools are rebuilt if
   installer/redoxfs changed.
8. **Prefix sysroot** — if relibc is newer than the prefix `libc.a`, the derived
   prefix markers are removed and `make prefix` genuinely re-cooks relibc into
   the sysroot (a no-op is reported honestly, never as success). kernel/base-only
   staleness does not touch the prefix.
9. **Preflight** — `build-preflight.sh`: repairs broken recipe symlinks, fork
   version/drift/function verification, collision detection, source-tree
   validation. The fork-function check treats inert std-trait methods
   (`fmt`/`eq`/`clone`/…) as warnings; intentional divergences are listed in each
   fork's `.verify-fork-functions.exclude`.
10. **Version sync check** — `sync-versions.sh --check` (see §7).
11. **Pre-cook** — cooks a small ordered set of critical packages (`CI=1`,
    offline unless `--upstream`).
12. **`make live`** — the cookbook builds the config's package graph and the
    installer assembles the image.
13. **Critical-package gate** — for `redbear-full`, fails loudly if mesa/qt/…
    pkgars are missing (cookbook returns 0 even when recipes fail).
14. **Fingerprints** — fork source fingerprints are recorded **only after** a
    successful build, so a failed build never marks forks as "built".

On failure the EXIT trap dumps diagnostics (last log lines, orphaned cook
processes, and per-recipe state **scoped to this build** so a mini build never
lists leftover graphical packages) and preserves the state dir under
`--keep-build-state`.

## 4. Offline / release mode

Cooking is offline by default (`COOKBOOK_OFFLINE=true`, `REPO_OFFLINE=1`).
`--upstream` goes online for non-protected recipes. `--release VER` builds from
`sources/redbear-<VER>/` immutable archives and forces offline; new releases are
created only via `provision-release.sh`. A stray `REDBEAR_RELEASE=` line in
`.config` silently switches the whole build off the local forks — the script
never auto-sets it. See `RELEASE-BUMP-WORKFLOW.md`.

## 5. Caching & staleness

Per-recipe caching is BLAKE3 content-hash based (`dep_hashes.toml` +
`source_hash.txt`), not mtime. `--no-cache` wipes `repo/` and recipe `target/`
dirs. Cargo path-dep crates (syscall/libredox/redox-scheme) have no pkgar, so
`build-redbear.sh` tracks their source fingerprints and forces a userspace
relink when they change. See `BUILD-CACHE-PLAN.md`.

## 6. No working-tree stashing (design)

The build **never stashes your working tree.** An earlier version stashed dirty
forks and restored them on exit, but modern git's `stash push` does not print the
stash SHA on stdout, so the stash was recorded incorrectly and never restored —
stranding operator WIP in `git stash list` on every dirty build. The machinery
was removed entirely. Today: a clean tree cooks committed HEAD; `--allow-dirty`
cooks the tree as-is. A read-only startup advisory reports any leftover
`redbear-build-*` stashes from the old code (recover them from
`git stash list`; see `../recovered-stashes/README.md`).

## 7. Forks & versioning

`local/sources/<fork>/` (the 9 submodule forks) take precedence over
`recipes/<fork>/`. Version policy (`sync-versions.sh`):

- **Cat 1** — in-house crates under `local/recipes/*/source/`: `version = <branch>`
  (e.g. `0.3.1`).
- **Cat 2** — upstream forks under `local/sources/*/`:
  `version = <upstream-tag>+rb<branch>` (e.g. `0.9.0+rb0.3.1`; `+rb` build
  metadata, never `-rb`).
- **Vendored upstream** under `local/recipes/` (e.g. `brush`, a Cargo workspace):
  keeps its **upstream** crate versions (internal `^` requirements). Mark it with
  a `.vendored-upstream` file in the recipe dir so `sync-versions.sh` skips it —
  do **not** stamp it with the branch version. `zbus`/`tlc` are excluded by name.

`sync-versions.sh --check` is a build-time gate; `sync-versions.sh` applies the
sync. See `PATCH-GOVERNANCE.md` for the durability rules (source trees are
disposable; only `local/` survives).

## 8. Self-versioning of this script

`BUILD_REDBEAR_VERSION` in `build-redbear.sh` starts at `1.0` and is auto-bumped
(minor) by a `pre-commit` git hook (`bump-build-version.sh`) whenever the script
is committed. Install hooks with `./local/scripts/install-git-hooks.sh`.