0073f6e230
The build system had multiple authoritative docs claiming 'Red Bear OS 0.1.0 (baseline)' and 'branch 0.2.5' long after the working branch advanced to 0.3.1. Operator confusion was inevitable: - 'repo --version' showed 0.2.5 (root Cargo.toml drift, fixed G2) - AGENTS.md said rust-toolchain = nightly-2025-10-03 (actual nightly-2026-05-24) - local/AGENTS.md example versions all said 0.2.5 (should be 0.3.1) - README.md and docs/* claimed 'baseline 0.1.0' on the current branch (which has 0.3.1 forks as the source of truth) This commit syncs the docs to reality without rewriting historical content where the 0.1.0 reference is genuinely about the legacy release archive at sources/redbear-0.1.0/ (which DOES exist and serves the release-mode fallback per BUILD-SYSTEM-SETUP). The 0.1.0 release-archive references are preserved with explicit 'legacy' annotations because that archive IS the durable record of the old overlay-patch approach and is used by restore-sources.sh --release=0.1.0 in fallback build paths. Updated across: - AGENTS.md (root): toolchain, current baseline phrasing - local/AGENTS.md: example versions 0.2.5->0.3.1, RELEASE MODEL clarified that current source-of-truth is local/sources/ forks - README.md: branch refs 0.2.5->0.3.1 - docs/06-BUILD-SYSTEM-SETUP.md: baseline 0.1.0->0.3.1 - local/docs/UPSTREAM-SYNC-PROCEDURE.md: baseline clarified - local/docs/LOCAL-FORK-SUPREMACY-POLICY.md: snapshot phrasing - local/docs/CUB-PACKAGE-MANAGER.md: archive context clarified
771 lines
27 KiB
Markdown
771 lines
27 KiB
Markdown
# Upstream Sync Procedure for Local Forks
|
|
|
|
**Created:** 2026-06-18
|
|
**Scope:** relibc, kernel, bootloader, installer, redoxfs, userutils, base
|
|
**Baseline:** Red Bear OS 0.1.0 release archive (legacy); current development is on the 0.3.1 branch with sources from local/sources/ forks
|
|
|
|
## Purpose
|
|
|
|
Red Bear OS maintains local forks of core Redox components. Upstream Redox is
|
|
actively developed and regularly merges refactoring, bug fixes, and new
|
|
features. This document defines the **standard procedure** for updating each
|
|
local fork to the latest upstream HEAD and re-applying Red Bear-specific
|
|
changes on top.
|
|
|
|
This is a **repeatable, per-package operation**. Every base system package
|
|
must go through this process periodically to avoid accumulating fork drift.
|
|
|
|
## When to Sync
|
|
|
|
- A Rust nightly bump causes compile errors in the fork (unstable API changes).
|
|
- Upstream adds features we were manually patching (eliminates a Red Bear patch).
|
|
- Accumulated fork drift makes rebasing individual patches impractical.
|
|
- Before a new Red Bear OS release.
|
|
- When a bug in the fork is already fixed upstream.
|
|
|
|
## Prerequisites
|
|
|
|
```bash
|
|
# Ensure upstream remote exists in the fork repo
|
|
cd local/sources/<component>
|
|
git remote -v
|
|
# Should show: upstream -> https://gitlab.redox-os.org/redox-os/<component>.git
|
|
|
|
# If missing:
|
|
git remote add upstream https://gitlab.redox-os.org/redox-os/<component>.git
|
|
```
|
|
|
|
## Procedure
|
|
|
|
### Step 1: Backup Current State
|
|
|
|
**Never skip this.** The backup branch is your rollback point.
|
|
|
|
```bash
|
|
cd local/sources/<component>
|
|
git fetch upstream --quiet
|
|
|
|
# Create timestamped backup branch
|
|
git branch backup-master-pre-upstream-update-$(date +%Y%m%d) master
|
|
|
|
# Verify
|
|
git log --oneline -1 backup-master-pre-upstream-update-*
|
|
```
|
|
|
|
### Step 2: Analyze Divergence
|
|
|
|
Understand what you have vs what upstream has:
|
|
|
|
```bash
|
|
# Latest upstream HEAD
|
|
git log --oneline upstream/master -3
|
|
|
|
# Divergence point (merge-base)
|
|
git merge-base master upstream/master
|
|
|
|
# Your commits on top of upstream
|
|
git log --oneline upstream/master..master
|
|
|
|
# Files changed (scope assessment)
|
|
git diff upstream/master..master --stat | tail -5
|
|
```
|
|
|
|
Categorize each commit:
|
|
- **Feature port** — ported FROM upstream (e.g., posix_spawn). Upstream now
|
|
has it natively → **DROP**, use upstream's version.
|
|
- **Red Bear feature** — unique to Red Bear (e.g., eventfd, signalfd, timerfd,
|
|
__fseterr, getprogname). Not in upstream → **KEEP**.
|
|
- **Compatibility shim** — gnulib/glibc/musl compat (e.g., rpl_malloc,
|
|
__freadahead). → **KEEP** unless upstream added equivalent.
|
|
- **Toolchain fix** — adapts to Rust nightly changes (e.g., unsafe blocks,
|
|
VaList API). → **CHECK** if still needed against current upstream.
|
|
- **Revert/cleanup** — noise from previous iterations. → **DROP**.
|
|
|
|
### Step 3: Create Sync Branch
|
|
|
|
```bash
|
|
# Branch from latest upstream
|
|
git checkout -b redbear-upstream-sync-$(date +%Y%m%d) upstream/master
|
|
```
|
|
|
|
### Step 4: Squash-Merge Your Changes
|
|
|
|
```bash
|
|
# Squash-merge brings ALL your changes as staged, conflict-marked if needed
|
|
git merge --squash master
|
|
```
|
|
|
|
If untracked files block the merge (files that exist in your fork but not
|
|
upstream, left over from failed attempts):
|
|
|
|
```bash
|
|
git merge --abort
|
|
git reset --hard upstream/master
|
|
git clean -fd
|
|
git merge --squash master
|
|
```
|
|
|
|
### Step 5: Resolve Conflicts
|
|
|
|
This is the most time-consuming step. For each conflicted file:
|
|
|
|
```bash
|
|
# List conflicts
|
|
git diff --name-only --diff-filter=U
|
|
```
|
|
|
|
**Resolution strategy per file:**
|
|
|
|
| Conflict Type | Resolution |
|
|
|---|---|
|
|
| Add/Add (both added same file) | Check which is canonical. If upstream now has it (e.g., spawn), take upstream: `git checkout --ours <file>` |
|
|
| Feature you ported from upstream | Take upstream: `git checkout --ours <file>` |
|
|
| Red Bear unique feature | Take ours: `git checkout --theirs <file>` |
|
|
| Both sides modified same code | **Manual merge** — keep both sets of changes |
|
|
| Cargo.toml/Cargo.lock | Resolve Cargo.toml manually, regenerate Cargo.lock with `cargo update` |
|
|
|
|
```bash
|
|
# Take upstream (current branch = upstream):
|
|
git checkout --ours <file> && git add <file>
|
|
|
|
# Take ours (the merged branch = our fork):
|
|
git checkout --theirs <file> && git add <file>
|
|
|
|
# Manual resolution:
|
|
# Edit file, remove conflict markers, keep both sides' relevant changes
|
|
git add <file>
|
|
```
|
|
|
|
**Conflict count expectation:**
|
|
- relibc: 5-15 conflict files (heavy upstream refactoring)
|
|
- kernel: 10-30 conflict files
|
|
- base: 5-10 conflict files
|
|
- Others: 2-5 conflict files
|
|
|
|
### Step 6: Commit the Squash
|
|
|
|
```bash
|
|
git commit -m "sync: upstream <component> to $(git rev-parse --short upstream/master)
|
|
|
|
Re-applied Red Bear-specific changes on top of latest upstream.
|
|
Dropped commits for features now in upstream.
|
|
Backup: backup-master-pre-upstream-update-$(date +%Y%m%d)"
|
|
```
|
|
|
|
### Step 7: Compile Test
|
|
|
|
```bash
|
|
# Build the component via the cookbook
|
|
cd /home/kellito/Builds/RedBear-OS
|
|
rm -rf recipes/core/<component>/source recipes/core/<component>/target
|
|
export CI=1 COOKBOOK_OFFLINE=false REDBEAR_ALLOW_PROTECTED_FETCH=1
|
|
./target/release/repo cook recipes/core/<component>
|
|
```
|
|
|
|
Fix any compile errors. These are usually:
|
|
- **Unsafe block changes** (Rust 2024 edition) — wrap in `unsafe { }`.
|
|
- **API renames** — upstream renamed a function/type.
|
|
- **Module path changes** — upstream reorganized headers.
|
|
- **Trait signature changes** — upstream changed a trait method signature.
|
|
|
|
### Step 8: Verify Functionality
|
|
|
|
For relibc specifically:
|
|
```bash
|
|
# Check critical symbols are present
|
|
nm recipes/core/relibc/target/x86_64-unknown-redox/stage/x86_64-unknown-redox/lib/libc.a | \
|
|
grep -E "__fseterr|eventfd|signalfd|timerfd|open_memstream"
|
|
|
|
# Check static libs exist
|
|
ls -la recipes/core/relibc/target/x86_64-unknown-redox/stage/x86_64-unknown-redox/lib/*.a
|
|
```
|
|
|
|
### Step 9: Push to Gitea
|
|
|
|
```bash
|
|
cd local/sources/<component>
|
|
# Switch sync branch to master (or merge sync into master)
|
|
git checkout master
|
|
git reset --hard redbear-upstream-sync-$(date +%Y%m%d)
|
|
git push gitea master --force-with-lease
|
|
```
|
|
|
|
### Step 10: Update Recipe
|
|
|
|
If the recipe uses `git + patches` (not `path`):
|
|
```bash
|
|
# Update the pinned rev in recipe.toml to match new HEAD
|
|
# Remove patches that are now in upstream
|
|
```
|
|
|
|
If the recipe uses `path` (local fork):
|
|
```bash
|
|
# No recipe changes needed — the fork IS the source
|
|
```
|
|
|
|
### Step 11: Rebuild Dependent Components
|
|
|
|
After updating a base component, everything that depends on it must rebuild:
|
|
|
|
| Component Updated | Must Rebuild |
|
|
|---|---|
|
|
| relibc | prefix, then ALL C recipes (bison, flex, m4, etc.) |
|
|
| kernel | base, drivers, all recipes |
|
|
| base | everything in the image |
|
|
| bootloader | disk image creation |
|
|
| installer | disk image creation |
|
|
| redoxfs | disk image creation |
|
|
| userutils | base, login/session recipes |
|
|
|
|
### Step 12: Full Image Build
|
|
|
|
```bash
|
|
cd /home/kellito/Builds/RedBear-OS
|
|
pkill -9 -f "build-redbear\|repo cook"
|
|
rm -rf recipes/core/<component>/source recipes/core/<component>/target
|
|
|
|
export CI=1 REDBEAR_ALLOW_PROTECTED_FETCH=1
|
|
setsid nohup env CI=1 REDBEAR_ALLOW_PROTECTED_FETCH=1 \
|
|
./local/scripts/build-redbear.sh --upstream redbear-mini \
|
|
> /tmp/build-V<N>.log 2>&1 < /dev/null &
|
|
```
|
|
|
|
## Package-Specific Notes
|
|
|
|
### relibc
|
|
|
|
- **Upstream repo:** `https://gitlab.redox-os.org/redox-os/relibc.git`
|
|
- **Fork location:** `local/sources/relibc/`
|
|
- **Recipe:** `recipes/core/relibc/recipe.toml`
|
|
- **Red Bear features to preserve:**
|
|
- `eventfd` (eventfd_create, eventfd_read, eventfd_write)
|
|
- `signalfd` (signalfd_create/signalfd)
|
|
- `timerfd` (timerfd_create, timerfd_settime, timerfd_gettime)
|
|
- `__fseterr`, `__freadahead` (gnulib/m4/bison compatibility)
|
|
- `getprogname` (via `sys:exe` scheme)
|
|
- `getloadavg` (via `/proc/loadavg`)
|
|
- `clock_settime` (proper implementation)
|
|
- `getifaddrs`, `if_nameindex` (networking)
|
|
- `getrlimit`, `getrusage` (resource limits)
|
|
- `waitid` (POSIX waitid)
|
|
- `dup3`, `F_DUPFD_CLOEXEC` fallback
|
|
- `secure_getenv`, `getentropy`
|
|
- `clock_nanosleep`
|
|
- POSIX semaphores (`sem_open`/`sem_close`/`sem_unlink`)
|
|
- POSIX mqueue (`mq_*`)
|
|
- `DT_RPATH` support in ld.so
|
|
- `redox_syscall = "0.8.1"` (ABI migration)
|
|
- DRM ioctl definitions (GET_UNIQUE, SET_VERSION, MODE_ADD_FB2)
|
|
- `rpl_malloc`/`rpl_realloc` (gnulib compat)
|
|
- 8MB stack size
|
|
- **Features now in upstream (DROP from Red Bear):**
|
|
- `posix_spawn` (port from upstream → upstream now has it)
|
|
- Various doc/lint fixes (upstream applied)
|
|
- **Features that were assumed upstream-fixed but are NOT:**
|
|
- VaList API adaptation — upstream relibc may be fixed for the redoxer default
|
|
toolchain (1.98-dev), but the PREFIX toolchain (nightly-2025-11-15) still
|
|
has the older `VaList<'a, 'f: 'a>` (2 lifetimes). When building via `make`
|
|
(which sets `REDOXER_TOOLCHAIN=PREFIX`), relibc HEAD fails to compile.
|
|
Fix: override `RUSTUP_TOOLCHAIN` in the recipe (see PACKAGE-BUILD-QUIRKS.md).
|
|
- cbindgen `va_list` generation — cbindgen translates `VaList<'_>` to `...`
|
|
(variadic) in C headers instead of `va_list`. This affects all `v*printf`
|
|
functions and breaks C++ code that takes their address (e.g., Qt6).
|
|
Fix: post-generation `sed` in recipe build script (see PACKAGE-BUILD-QUIRKS.md).
|
|
- **Compile gotchas:**
|
|
- `core::arch::x86_64::__cpuid()` requires `unsafe { }` block (edition 2024)
|
|
- `Vec::into_raw_parts()` is unstable — use `mem::transmute` + `as_ptr`/`len`/`capacity`
|
|
- `-D unused_mut` in upstream Makefile flags
|
|
|
|
### kernel
|
|
|
|
- **Upstream repo:** `https://gitlab.redox-os.org/redox-os/kernel.git`
|
|
- **Fork location:** `local/sources/kernel/`
|
|
- **Red Bear features to preserve:**
|
|
- Any ACPI improvements specific to AMD
|
|
- IRQ routing changes
|
|
- MSI/MSI-X support
|
|
- IOMMU work
|
|
- **Check upstream first** — kernel is the fastest-moving Redox component
|
|
|
|
### bootloader
|
|
|
|
- **Upstream repo:** `https://gitlab.redox-os.org/redox-os/bootloader.git`
|
|
- **Fork location:** `local/sources/bootloader/`
|
|
- **Red Bear features to preserve:**
|
|
- Branding (Red Bear logo, colors)
|
|
- Any boot protocol changes
|
|
|
|
### installer
|
|
|
|
- **Upstream repo:** `https://gitlab.redox-os.org/redox-os/installer.git`
|
|
- **Fork location:** `local/sources/installer/`
|
|
- **Red Bear features to preserve:**
|
|
- ext4 filesystem support
|
|
- GRUB bootloader support
|
|
- CollisionTracker (file ownership conflict detection)
|
|
- Init service path validation
|
|
|
|
### redoxfs
|
|
|
|
- **Upstream repo:** `https://gitlab.redox-os.org/redox-os/redoxfs.git`
|
|
- **Fork location:** `local/sources/redoxfs/`
|
|
- **Red Bear features to preserve:**
|
|
- Any performance patches
|
|
- Any filesystem format changes
|
|
|
|
### userutils
|
|
|
|
- **Upstream repo:** `https://gitlab.redox-os.org/redox-os/userutils.git`
|
|
- **Fork location:** `local/sources/userutils/`
|
|
- **Red Bear features to preserve:**
|
|
- Custom `/etc/issue` (Red Bear login banner)
|
|
- Any Redox ABI changes for syscall 0.8.x
|
|
|
|
### base
|
|
|
|
- **Upstream repo:** Pinned via `recipes/core/base/recipe.toml` (path-based fork)
|
|
- **Fork location:** `local/sources/base/`
|
|
- **Red Bear features to preserve:**
|
|
- All Red Bear init.d services
|
|
- All Red Bear branding
|
|
- All Red Bear config overrides
|
|
- HID/I2C driver wiring
|
|
- D-Bus integration
|
|
- Session management (redbear-sessiond)
|
|
- ~100 individual P*.patch files in `local/patches/base/`
|
|
|
|
## Rollback Procedure
|
|
|
|
If the sync fails or produces a broken build:
|
|
|
|
```bash
|
|
cd local/sources/<component>
|
|
git checkout master
|
|
git reset --hard backup-master-pre-upstream-update-<date>
|
|
git push gitea master --force-with-lease
|
|
```
|
|
|
|
Then in the main repo:
|
|
```bash
|
|
git checkout -- recipes/core/<component>/recipe.toml
|
|
# Revert recipe changes if any
|
|
```
|
|
|
|
## Decision Matrix: Keep vs Drop vs Merge
|
|
|
|
For each Red Bear commit when syncing:
|
|
|
|
```
|
|
Is the feature in upstream now?
|
|
├── YES → Does upstream's version work for us?
|
|
│ ├── YES → DROP our commit, use upstream
|
|
│ └── NO → MERGE: take upstream's structure, re-apply our specific changes
|
|
└── NO → Is it a Red Bear unique feature?
|
|
├── YES → KEEP our commit
|
|
└── NO → Is it a toolchain/compat fix?
|
|
├── Still needed? → KEEP
|
|
└── Fixed upstream? → DROP
|
|
```
|
|
|
|
## Checklist Per Package
|
|
|
|
- [ ] Backup branch created
|
|
- [ ] Divergence analyzed (commits categorized)
|
|
- [ ] Sync branch created from latest upstream
|
|
- [ ] Squash-merge applied
|
|
- [ ] All conflicts resolved
|
|
- [ ] Commit made with sync message
|
|
- [ ] Compiles via `repo cook`
|
|
- [ ] Critical symbols verified (for relibc)
|
|
- [ ] Pushed to Gitea
|
|
- [ ] Recipe updated (if git+patches mode)
|
|
- [ ] Dependent components rebuilt
|
|
- [ ] Full image build succeeds
|
|
- [ ] QEMU boot test passes
|
|
|
|
## Known Issues and Solutions (2026-06-18 Sync)
|
|
|
|
### Kernel: `-Zjson-target-spec` Rejected by Cargo in Redoxer
|
|
|
|
**Problem:** The upstream kernel Makefile passes `-Zjson-target-spec` as a
|
|
cargo CLI flag. The redoxer build environment's cargo (at
|
|
`~/.redoxer/x86_64-unknown-redox/toolchain/bin/cargo`) lists this flag in
|
|
`cargo -Z help` but rejects it when invoked through `make` inside
|
|
`redoxer env bash -ex`.
|
|
|
|
**Root cause:** The `make` subprocess inside redoxer env does not properly
|
|
propagate all cargo unstable feature flags from the CLI. The env-var form
|
|
(`CARGO_UNSTABLE_<FEATURE>=true`) works reliably.
|
|
|
|
**Fix (applied to kernel Makefile):**
|
|
```makefile
|
|
# Replace -Zjson-target-spec CLI flag with env var
|
|
export CARGO_UNSTABLE_JSON_TARGET_SPEC=true
|
|
# Remove -Zjson-target-spec from the cargo rustc command line
|
|
```
|
|
|
|
### Base: `map_bar()` API Change
|
|
|
|
**Problem:** Upstream changed `pcid_handle.map_bar(N)` to
|
|
`pcid_handle.map_bar(N, MemoryType::Uncacheable)`.
|
|
|
|
**Fix:** Update all Red Bear drivers that call `map_bar()` to pass
|
|
`common::MemoryType::Uncacheable` as the second argument. Affected drivers:
|
|
`amd-mp2-i2cd`, `intel-thc-hidd`. The upstream drivers (`ahcid`, `nvmed`,
|
|
`xhcid`) already use the new API.
|
|
|
|
### Base: `if let` Guards in pcid
|
|
|
|
**Problem:** Upstream `drivers/pcid/src/cfg_access/mod.rs` uses experimental
|
|
`if let` guards in match arms.
|
|
|
|
**Fix:** Add `#![feature(if_let_guard)]` to `drivers/pcid/src/main.rs`.
|
|
|
|
### Base: XHCI Quirk Integration Dropped
|
|
|
|
**Problem:** The Red Bear xhci quirk integration (`XhciControllerQuirkFlags`,
|
|
`staged_port_states`, `ZERO_64B_REGS` handling) was deeply coupled with the
|
|
old xhci module code and conflicts with upstream's evolved API.
|
|
|
|
**Resolution:** Reverted `drivers/usb/xhcid/src/xhci/mod.rs` and
|
|
`irq_reactor.rs` to upstream. The quirk system will be re-integrated as
|
|
follow-up work on top of the new upstream xhci code.
|
|
|
|
### Base: Workspace Members Missing After Sync
|
|
|
|
**Problem:** Taking upstream's `Cargo.toml` drops Red Bear-specific workspace
|
|
members (I2C, GPIO, HID, thermald, ucsid, acpi-resource).
|
|
|
|
**Fix:** Add all 13 Red Bear workspace members back to the `members` list.
|
|
Also fix the `[patch]` section to point to `../../sources/relibc/` (not the
|
|
stale `../../relibc/source/` path).
|
|
|
|
### Systemic: `redox_syscall` Version Alignment
|
|
|
|
**Problem:** All Red Bear custom recipes used `redox_syscall = "0.7"`, but the
|
|
synced base and relibc use `0.8.x`.
|
|
|
|
**Fix:** Bulk-replaced `"0.7"` → `"0.8"` across all 23 Red Bear recipe
|
|
Cargo.toml files.
|
|
|
|
### Installer: Switch to Local Fork
|
|
|
|
**Problem:** The installer recipe used `git = "..."` + `patches = [...]`, but
|
|
the patch (`redox.patch`) fails against the synced upstream code (368 commits
|
|
of drift).
|
|
|
|
**Fix:** Switched recipe to `path = "../../../local/sources/installer"` (local
|
|
fork mode). The ext4/GRUB installer changes need to be re-applied to the fork
|
|
as follow-up work.
|
|
|
|
### Kernel: Switch to Local Fork
|
|
|
|
**Problem:** The kernel recipe used `git = "..."` + 5 patches, all of which
|
|
fail against the synced upstream code.
|
|
|
|
**Fix:** Switched recipe to `path = "../../../local/sources/kernel"` (local
|
|
fork mode). All patches were already squash-merged into the fork.
|
|
|
|
### Kernel: Compile Errors After Upstream Sync (31 errors)
|
|
|
|
After the squash-merge, the kernel had 31 compile errors across ~20 files.
|
|
These were caused by upstream API evolution interacting with Red Bear code
|
|
that survived in non-conflicting regions of the merge.
|
|
|
|
#### Feature Gates Required
|
|
|
|
Upstream code uses two experimental features that require feature gates:
|
|
- `#![feature(asm_cfg)]` — for `#[cfg(target_arch)]` on inline assembly
|
|
- `#![feature(if_let_guard)]` — for `if let` guards in match arms
|
|
|
|
Both must be added to `src/main.rs`.
|
|
|
|
#### Unsafe Blocks Required (Rust 2024 Edition)
|
|
|
|
Rust 2024 edition requires explicit `unsafe { }` blocks for unsafe operations
|
|
even inside `unsafe fn`. Affected files:
|
|
- `src/arch/x86_shared/cpuid.rs` — `__cpuid_count`
|
|
- `src/arch/x86_shared/device/tsc.rs` — `__cpuid`
|
|
- `src/arch/x86_shared/device/local_apic.rs` — `set_lvt_timer`
|
|
|
|
#### Removed Red Bear APIs
|
|
|
|
These Red Bear types/constants no longer exist after taking upstream:
|
|
- `SchedPolicy` — Red Bear's DWRR scheduler policy enum (upstream uses EEVDF)
|
|
- `RUN_QUEUE_COUNT` — Red Bear's per-NUMA run queue count constant
|
|
- `get_event_stat()` — Red Bear's event system statistics (upstream uses eventfd)
|
|
- `PIPES` — Red Bear's pipe storage (renamed to `HANDLES`)
|
|
|
|
Fixes:
|
|
- Remove `SchedPolicy` from the `pub use` in `context/mod.rs`
|
|
- Define `RUN_QUEUE_COUNT` locally in `percpu.rs` if still needed
|
|
- Implement `get_event_stat()` in `event.rs` using the upstream `REGISTRY` static
|
|
- Use `HANDLES` instead of `PIPES` in `scheme/pipe.rs`
|
|
|
|
#### `CONTEXT_SWITCH_LOCK` Missing
|
|
|
|
Upstream added `pub static CONTEXT_SWITCH_LOCK: AtomicBool` to
|
|
`src/context/arch/x86_64.rs` but our fork was missing it. Must be added
|
|
with the `core::sync::atomic::AtomicBool` import.
|
|
|
|
#### `new_addrsp_guard` Field Missing from PercpuBlock
|
|
|
|
Upstream added a separate `new_addrsp_guard: Cell<Option<AddrSpaceSwitchReadGuard>>`
|
|
field to `PercpuBlock` for storing the addr space read guard during context
|
|
switch. Our fork only had `new_addrsp_tmp` (for the `Arc<AddrSpaceWrapper>`).
|
|
Both fields must exist.
|
|
|
|
#### `switch.rs` Must Be Replaced Entirely
|
|
|
|
The Red Bear DWRR scheduler in `switch.rs` is fundamentally incompatible with
|
|
the upstream EEVDF scheduler data structures. The `RunContextData` changed from
|
|
`set: [VecDeque<WeakContextRef>; N]` to `queue: BTreeMap<(vd, rem_slice, ctxt_id), ...>`.
|
|
|
|
The entire `switch.rs` must be replaced with the upstream version:
|
|
```bash
|
|
git show upstream/master:src/context/switch.rs > src/context/switch.rs
|
|
```
|
|
|
|
#### `kcall` Trait Method Signature Changed
|
|
|
|
Upstream changed `id: usize` to `fds: &[usize]` in the `kcall` trait method.
|
|
Update the `ProcScheme` impl to extract the first element:
|
|
```rust
|
|
fn kcall(&self, fds: &[usize], ...) -> Result<usize> {
|
|
let id = *fds.first().ok_or(Error::new(EBADF))?;
|
|
```
|
|
|
|
#### `ProcSchemeVerb` New Variants
|
|
|
|
Upstream added new variants to `ProcSchemeVerb`: `RegsInt`, `RegsFloat`,
|
|
`RegsEnv`, `SchedAffinity`, `Start`. The match in `scheme/proc.rs` must
|
|
handle them (currently returns `ENOSYS`).
|
|
|
|
#### `notify_files` Type Change
|
|
|
|
`handle_notify_files()` now expects `UnmapVec` (`SmallVec<[UnmapResult; 16]>`)
|
|
instead of `Vec`. Import `UnmapVec` from `context::memory` and use
|
|
`UnmapVec::new()` instead of `Vec::new()`.
|
|
|
|
#### `Rsdp::get_rsdp` Signature Change
|
|
|
|
Upstream changed the parameter from `Option<NonNull<u8>>` to
|
|
`Option<*const u8>`. Convert at the call site:
|
|
```rust
|
|
Rsdp::get_rsdp(already_supplied_rsdp.map(|ptr| ptr.as_ptr() as *const u8))
|
|
```
|
|
|
|
#### Syntax Error in `acpi/madt/arch/x86.rs`
|
|
|
|
A `const` statement was accidentally placed inside a `use crate::{...}` block
|
|
during the merge. Must be moved outside.
|
|
|
|
#### `halt_boot` Scope Issue
|
|
|
|
`halt_boot` was defined inside the `impl KernelArgs` block, making it a method
|
|
rather than a free function. Must be moved outside the impl block.
|
|
|
|
### Cookbook Symlink Behavior for Local Forks
|
|
|
|
**Critical:** When a recipe uses `path = "..."` pointing inside
|
|
`/local/sources/` or `/local/recipes/`, the cookbook creates a **symlink**
|
|
(not a copy) from `recipes/<name>/source` → the local fork path. This means:
|
|
|
|
1. Build artifacts and `cargo` operations can modify files in the local fork
|
|
through the symlink.
|
|
2. After each build attempt, verify the fork's git status:
|
|
```bash
|
|
cd local/sources/<component>
|
|
git status --short
|
|
```
|
|
3. If the working tree is dirty, reset before rebuilding:
|
|
```bash
|
|
git checkout -- .
|
|
```
|
|
4. Always clean both `target/` and `source/` when restarting a kernel build:
|
|
```bash
|
|
rm -rf recipes/core/kernel/target recipes/core/kernel/source
|
|
cd local/sources/kernel && git checkout -- .
|
|
```
|
|
|
|
### m4: `getlocalename_l-unsafe.c` Platform Detection
|
|
|
|
**Problem:** m4's gnulib `getlocalename_l-unsafe.c` has a Redox case guarded
|
|
by `HAVE_GETLOCALENAME_L`, which relibc doesn't define.
|
|
|
|
**Fix:** Changed the guard from `__redox__ && HAVE_GETLOCALENAME_L` to just
|
|
`__redox__` and return `"C"` (Redox only supports the C/POSIX locale).
|
|
|
|
### m4: `rpl_realloc` Multiple Definition (CFLAGS)
|
|
|
|
**Problem:** GCC 10+ defaults to `-fno-common`, causing `rpl_realloc` (gnulib
|
|
replacement) to be multiply defined across translation units.
|
|
|
|
**Fix:** Add `-fcommon` to CFLAGS in the m4 recipe:
|
|
```bash
|
|
export CFLAGS="-fcommon ${CFLAGS}"
|
|
```
|
|
|
|
**Placement:** BEFORE `DYNAMIC_INIT` (works because `reexport_flags` re-reads
|
|
`$CFLAGS` at configure time).
|
|
|
|
### m4: `rpl_realloc` Multiple Definition (LDFLAGS)
|
|
|
|
**Problem:** Even with `-fcommon`, the linker encounters duplicate definitions
|
|
of `rpl_realloc` from static library archives.
|
|
|
|
**Fix:** Add `-Wl,--allow-multiple-definition` to LDFLAGS:
|
|
```bash
|
|
DYNAMIC_INIT
|
|
export LDFLAGS="-Wl,--allow-multiple-definition ${LDFLAGS}"
|
|
```
|
|
|
|
**CRITICAL Placement:** MUST be AFTER `DYNAMIC_INIT`. `DYNAMIC_INIT` overwrites
|
|
`LDFLAGS` entirely — setting it before has no effect. See
|
|
`local/docs/PACKAGE-BUILD-QUIRKS.md` § "DYNAMIC_INIT" for details.
|
|
|
|
### m4: Configure Cache Variables
|
|
|
|
**Problem:** m4's `configure` tests fail for functions that exist in relibc but
|
|
have different semantics, or for gnulib internal functions.
|
|
|
|
**Fix:** Pass cache variables to skip problematic tests:
|
|
```bash
|
|
COOKBOOK_CONFIGURE_FLAGS+=(
|
|
--disable-nls
|
|
ac_cv_func___freadahead=yes
|
|
ac_cv_have_decl___freadahead=yes
|
|
gl_cv_header_wchar_h_correct_inline=yes
|
|
gl_cv_func_btowc_nul=yes
|
|
gl_cv_func_btowc_consistent=yes
|
|
gl_cv_onwards_func___freadahead=yes
|
|
)
|
|
```
|
|
|
|
### ninja-build: `BUILD_TESTING=OFF` (Cross-Compilation)
|
|
|
|
**Problem:** ninja-build's CMake includes `CTest`, enabling `BUILD_TESTING=ON`
|
|
by default. Tests require Google Test (gtest) from the host system. Host gtest
|
|
headers transitively include host `/usr/include/stdlib.h`, which conflicts with
|
|
the Redox sysroot `stdlib.h` (type redefinitions: `div_t`, `ldiv_t`, `lldiv_t`,
|
|
`at_quick_exit`). This is a fundamental cross-compilation constraint, not a
|
|
Redox capability gap.
|
|
|
|
**Fix:** Disable tests via cmakeflags:
|
|
```toml
|
|
[build]
|
|
template = "cmake"
|
|
cmakeflags = ["-DBUILD_TESTING=OFF"]
|
|
```
|
|
|
|
See `local/docs/PACKAGE-BUILD-QUIRKS.md` § "General Cross-Compilation Patterns"
|
|
for the rationale.
|
|
|
|
### Cookbook: `DYNAMIC_INIT` Overwrites Environment Variables
|
|
|
|
**Problem:** `DYNAMIC_INIT` overwrites `CFLAGS`, `LDFLAGS`, `CXXFLAGS`, etc.
|
|
entirely. Custom flags set BEFORE `DYNAMIC_INIT` are lost.
|
|
|
|
**Fix:** Set ALL custom flags AFTER `DYNAMIC_INIT`:
|
|
```bash
|
|
DYNAMIC_INIT
|
|
export CFLAGS="-fcommon ${CFLAGS}"
|
|
export LDFLAGS="-Wl,--allow-multiple-definition ${LDFLAGS}"
|
|
```
|
|
|
|
**Exception:** `CFLAGS` set BEFORE `DYNAMIC_INIT` works because `reexport_flags`
|
|
(called inside `cookbook_configure`) re-reads `$CFLAGS`. However, this is fragile
|
|
and not recommended. See `local/docs/PACKAGE-BUILD-QUIRKS.md` for full details.
|
|
|
|
### Build System: Source Reversion via Patch Application
|
|
|
|
**Problem:** `build-redbear.sh` previously called `apply_patch_dir` to apply
|
|
patches from `local/patches/{kernel,base,relibc,installer}/` to
|
|
`recipes/core/<comp>/source/`. For `path=` recipes, `source/` is a symlink to
|
|
`local/sources/<comp>/`. Patches went through the symlink and modified the local
|
|
fork directly, re-adding old code on every build.
|
|
|
|
**Root cause:** `apply_patch_dir` operated on `recipes/core/<comp>/source/`
|
|
without checking if it was a symlink to a local fork.
|
|
|
|
**Fix (commit `760634376`):** Removed `apply_patch_dir` calls for kernel, base,
|
|
relibc, and installer from `build-redbear.sh`. Only bootloader (which uses
|
|
`git + patches`, not `path=`) still gets `apply_patch_dir`.
|
|
|
|
**Verification:** Kernel source hashes now identical before and after build.
|
|
|
|
---
|
|
|
|
## CRITICAL: Red Bear Package Preservation Rules
|
|
|
|
### Never Delete — Comment Out
|
|
|
|
When syncing with upstream removes or restructures packages, configs, or
|
|
services that are Red Bear in-house projects:
|
|
|
|
1. **NEVER delete** the package, config, or service entry.
|
|
2. **Comment it out** with a clear explanation of why.
|
|
3. **ASK THE USER** if uncertain whether to keep or remove.
|
|
|
|
This applies to:
|
|
- Packages in `[packages]` sections of config TOMLs
|
|
- Dependencies in `recipe.toml` files
|
|
- Init service files (`.service`, `.target`)
|
|
- Build system patches and scripts
|
|
- Any file under `local/` or `config/redbear-*`
|
|
|
|
### driver-manager: Upstream Not Ready
|
|
|
|
**Status:** Red Bear internal project. Upstream Redox does NOT have an equivalent.
|
|
|
|
`driver-manager` (`local/recipes/system/driver-manager/`) is a combined PCI
|
|
enumeration + driver matching + hotplug daemon that replaces the upstream
|
|
`pcid + pcid-spawner` pair. It was developed because upstream's PCI driver
|
|
launching was not ready for Red Bear's needs (advanced driver matching via
|
|
`/lib/drivers.d/*.toml`, ACPI device enumeration, hotplug event handling).
|
|
|
|
**Upstream may provide its own solution in the future.** If and when upstream
|
|
Redox offers a concrete, working replacement that matches or exceeds
|
|
driver-manager's capabilities, notify the user for a route decision.
|
|
|
|
Until then, `driver-manager` is the canonical PCI driver orchestration path.
|
|
|
|
### Systemic Sync Flaw (2026-06 Incident)
|
|
|
|
**Root cause:** The 0.2.3 → 0.2.4 upstream sync blindly overwrote tracked
|
|
config and recipe files. Red Bear in-house packages, services, and configs
|
|
were silently dropped because upstream doesn't have them.
|
|
|
|
**What was lost (partial list):**
|
|
- `driver-manager` removed from base-initfs dependencies and packages
|
|
- `config/protected-recipes.toml` deleted (99 lines of protection rules)
|
|
- `config/redbear-boot-stages.toml` deleted (109 lines of boot stage targets)
|
|
- `set -eo pipefail` removed from base-initfs build script
|
|
- `drivers.d/` initfs config directory creation removed
|
|
- Multiple patches lost (libdrm, mesa, pipewire, sddm, wireplumber)
|
|
- `local/recipes/AGENTS.md` deleted
|
|
|
|
**Prevention:** Before ANY upstream sync:
|
|
1. Run `git diff` between the pre-sync and post-sync tree.
|
|
2. Flag ALL removed files, especially under `config/`, `local/`, and `recipes/`.
|
|
3. Flag ALL removed entries from `[packages]` sections and dependency arrays.
|
|
4. Cross-reference with `config/protected-recipes.toml` — protected recipes
|
|
must NEVER be removed.
|
|
5. Any removal must be explicitly approved by the user.
|
|
|
|
### pcid + pcid-spawner Architecture
|
|
|
|
**Upstream model:**
|
|
1. `pcid` creates `/scheme/pci` (PCI bus enumeration)
|
|
2. `pcid-spawner` reads `/scheme/pci` and launches drivers per `pcid.d/*.toml`
|
|
|
|
**Red Bear model (0.2.3 and forward):**
|
|
1. `pcid` creates `/scheme/pci` (initfs service `35_pcid.service`)
|
|
2. `pcid-spawner --initfs` launches critical boot drivers (initfs)
|
|
3. `driver-manager --hotplug` handles full driver matching + hotplug (rootfs)
|
|
|
|
**Key learning:** Without `35_pcid.service` in initfs, `/scheme/pci` is never
|
|
created, and BOTH `pcid-spawner` (initfs and rootfs) fail with ENODEV. This
|
|
was masked in 0.2.3 because driver-manager does its own PCI enumeration via
|
|
`redox_driver_pci` and doesn't depend on `/scheme/pci`.
|