RedBear-OS/local/docs/UPSTREAM-SYNC-PROCEDURE.md

Upstream Sync Procedure for Local Forks

Created: 2026-06-18 Scope: relibc, kernel, bootloader, installer, redoxfs, userutils, base Baseline: Red Bear OS 0.1.0 (Redox snapshot)

Purpose

Red Bear OS maintains local forks of core Redox components. Upstream Redox is actively developed and regularly merges refactoring, bug fixes, and new features. This document defines the standard procedure for updating each local fork to the latest upstream HEAD and re-applying Red Bear-specific changes on top.

This is a repeatable, per-package operation. Every base system package must go through this process periodically to avoid accumulating fork drift.

When to Sync

• A Rust nightly bump causes compile errors in the fork (unstable API changes).
• Upstream adds features we were manually patching (eliminates a Red Bear patch).
• Accumulated fork drift makes rebasing individual patches impractical.
• Before a new Red Bear OS release.
• When a bug in the fork is already fixed upstream.

Prerequisites

# Ensure upstream remote exists in the fork repo
cd local/sources/<component>
git remote -v
# Should show: upstream -> https://gitlab.redox-os.org/redox-os/<component>.git

# If missing:
git remote add upstream https://gitlab.redox-os.org/redox-os/<component>.git

Procedure

Step 1: Backup Current State

Never skip this. The backup branch is your rollback point.

cd local/sources/<component>
git fetch upstream --quiet

# Create timestamped backup branch
git branch backup-master-pre-upstream-update-$(date +%Y%m%d) master

# Verify
git log --oneline -1 backup-master-pre-upstream-update-*

Step 2: Analyze Divergence

Understand what you have vs what upstream has:

# Latest upstream HEAD
git log --oneline upstream/master -3

# Divergence point (merge-base)
git merge-base master upstream/master

# Your commits on top of upstream
git log --oneline upstream/master..master

# Files changed (scope assessment)
git diff upstream/master..master --stat | tail -5

Categorize each commit:

• Feature port — ported FROM upstream (e.g., posix_spawn). Upstream now has it natively → DROP, use upstream's version.
• Red Bear feature — unique to Red Bear (e.g., eventfd, signalfd, timerfd, __fseterr, getprogname). Not in upstream → KEEP.
• Compatibility shim — gnulib/glibc/musl compat (e.g., rpl_malloc, __freadahead). → KEEP unless upstream added equivalent.
• Toolchain fix — adapts to Rust nightly changes (e.g., unsafe blocks, VaList API). → CHECK if still needed against current upstream.
• Revert/cleanup — noise from previous iterations. → DROP.

Step 3: Create Sync Branch

# Branch from latest upstream
git checkout -b redbear-upstream-sync-$(date +%Y%m%d) upstream/master

Step 4: Squash-Merge Your Changes

# Squash-merge brings ALL your changes as staged, conflict-marked if needed
git merge --squash master

If untracked files block the merge (files that exist in your fork but not upstream, left over from failed attempts):

git merge --abort
git reset --hard upstream/master
git clean -fd
git merge --squash master

Step 5: Resolve Conflicts

This is the most time-consuming step. For each conflicted file:

# List conflicts
git diff --name-only --diff-filter=U

Resolution strategy per file:

Conflict Type	Resolution
Add/Add (both added same file)	Check which is canonical. If upstream now has it (e.g., spawn), take upstream: git checkout --ours <file>
Feature you ported from upstream	Take upstream: git checkout --ours <file>
Red Bear unique feature	Take ours: git checkout --theirs <file>
Both sides modified same code	Manual merge — keep both sets of changes
Cargo.toml/Cargo.lock	Resolve Cargo.toml manually, regenerate Cargo.lock with cargo update

# Take upstream (current branch = upstream):
git checkout --ours <file> && git add <file>

# Take ours (the merged branch = our fork):
git checkout --theirs <file> && git add <file>

# Manual resolution:
# Edit file, remove conflict markers, keep both sides' relevant changes
git add <file>

Conflict count expectation:

• relibc: 5-15 conflict files (heavy upstream refactoring)
• kernel: 10-30 conflict files
• base: 5-10 conflict files
• Others: 2-5 conflict files

Step 6: Commit the Squash

git commit -m "sync: upstream <component> to $(git rev-parse --short upstream/master)

Re-applied Red Bear-specific changes on top of latest upstream.
Dropped commits for features now in upstream.
Backup: backup-master-pre-upstream-update-$(date +%Y%m%d)"

Step 7: Compile Test

# Build the component via the cookbook
cd /home/kellito/Builds/RedBear-OS
rm -rf recipes/core/<component>/source recipes/core/<component>/target
export CI=1 COOKBOOK_OFFLINE=false REDBEAR_ALLOW_PROTECTED_FETCH=1
./target/release/repo cook recipes/core/<component>

Fix any compile errors. These are usually:

• Unsafe block changes (Rust 2024 edition) — wrap in unsafe { }.
• API renames — upstream renamed a function/type.
• Module path changes — upstream reorganized headers.
• Trait signature changes — upstream changed a trait method signature.

Step 8: Verify Functionality

For relibc specifically:

# Check critical symbols are present
nm recipes/core/relibc/target/x86_64-unknown-redox/stage/x86_64-unknown-redox/lib/libc.a | \
  grep -E "__fseterr|eventfd|signalfd|timerfd|open_memstream"

# Check static libs exist
ls -la recipes/core/relibc/target/x86_64-unknown-redox/stage/x86_64-unknown-redox/lib/*.a

Step 9: Push to Gitea

cd local/sources/<component>
# Switch sync branch to master (or merge sync into master)
git checkout master
git reset --hard redbear-upstream-sync-$(date +%Y%m%d)
git push gitea master --force-with-lease

Step 10: Update Recipe

If the recipe uses git + patches (not path):

# Update the pinned rev in recipe.toml to match new HEAD
# Remove patches that are now in upstream

If the recipe uses path (local fork):

# No recipe changes needed — the fork IS the source

Step 11: Rebuild Dependent Components

After updating a base component, everything that depends on it must rebuild:

Component Updated	Must Rebuild
relibc	prefix, then ALL C recipes (bison, flex, m4, etc.)
kernel	base, drivers, all recipes
base	everything in the image
bootloader	disk image creation
installer	disk image creation
redoxfs	disk image creation
userutils	base, login/session recipes

Step 12: Full Image Build

cd /home/kellito/Builds/RedBear-OS
pkill -9 -f "build-redbear\|repo cook"
rm -rf recipes/core/<component>/source recipes/core/<component>/target

export CI=1 REDBEAR_ALLOW_PROTECTED_FETCH=1
setsid nohup env CI=1 REDBEAR_ALLOW_PROTECTED_FETCH=1 \
  ./local/scripts/build-redbear.sh --upstream redbear-mini \
  > /tmp/build-V<N>.log 2>&1 < /dev/null &

Package-Specific Notes

relibc

• Upstream repo: https://gitlab.redox-os.org/redox-os/relibc.git
• Fork location: local/sources/relibc/
• Recipe: recipes/core/relibc/recipe.toml
• Red Bear features to preserve:
  • eventfd (eventfd_create, eventfd_read, eventfd_write)
  • signalfd (signalfd_create/signalfd)
  • timerfd (timerfd_create, timerfd_settime, timerfd_gettime)
  • __fseterr, __freadahead (gnulib/m4/bison compatibility)
  • getprogname (via sys:exe scheme)
  • getloadavg (via /proc/loadavg)
  • clock_settime (proper implementation)
  • getifaddrs, if_nameindex (networking)
  • getrlimit, getrusage (resource limits)
  • waitid (POSIX waitid)
  • dup3, F_DUPFD_CLOEXEC fallback
  • secure_getenv, getentropy
  • clock_nanosleep
  • POSIX semaphores (sem_open/sem_close/sem_unlink)
  • POSIX mqueue (mq_*)
  • DT_RPATH support in ld.so
  • redox_syscall = "0.8.1" (ABI migration)
  • DRM ioctl definitions (GET_UNIQUE, SET_VERSION, MODE_ADD_FB2)
  • rpl_malloc/rpl_realloc (gnulib compat)
  • 8MB stack size
• Features now in upstream (DROP from Red Bear):
  • posix_spawn (port from upstream → upstream now has it)
  • VaList API adaptation (upstream fixed)
  • Various doc/lint fixes (upstream applied)
• Compile gotchas:
  • core::arch::x86_64::__cpuid() requires unsafe { } block (edition 2024)
  • Vec::into_raw_parts() is unstable — use mem::transmute + as_ptr/len/capacity
  • -D unused_mut in upstream Makefile flags

kernel

• Upstream repo: https://gitlab.redox-os.org/redox-os/kernel.git
• Fork location: local/sources/kernel/
• Red Bear features to preserve:
  • Any ACPI improvements specific to AMD
  • IRQ routing changes
  • MSI/MSI-X support
  • IOMMU work
• Check upstream first — kernel is the fastest-moving Redox component

bootloader

• Upstream repo: https://gitlab.redox-os.org/redox-os/bootloader.git
• Fork location: local/sources/bootloader/
• Red Bear features to preserve:
  • Branding (Red Bear logo, colors)
  • Any boot protocol changes

installer

• Upstream repo: https://gitlab.redox-os.org/redox-os/installer.git
• Fork location: local/sources/installer/
• Red Bear features to preserve:
  • ext4 filesystem support
  • GRUB bootloader support
  • CollisionTracker (file ownership conflict detection)
  • Init service path validation

redoxfs

• Upstream repo: https://gitlab.redox-os.org/redox-os/redoxfs.git
• Fork location: local/sources/redoxfs/
• Red Bear features to preserve:
  • Any performance patches
  • Any filesystem format changes

userutils

• Upstream repo: https://gitlab.redox-os.org/redox-os/userutils.git
• Fork location: local/sources/userutils/
• Red Bear features to preserve:
  • Custom /etc/issue (Red Bear login banner)
  • Any Redox ABI changes for syscall 0.8.x

base

• Upstream repo: Pinned via recipes/core/base/recipe.toml (path-based fork)
• Fork location: local/sources/base/
• Red Bear features to preserve:
  • All Red Bear init.d services
  • All Red Bear branding
  • All Red Bear config overrides
  • HID/I2C driver wiring
  • D-Bus integration
  • Session management (redbear-sessiond)
  • ~100 individual P*.patch files in local/patches/base/

Rollback Procedure

If the sync fails or produces a broken build:

cd local/sources/<component>
git checkout master
git reset --hard backup-master-pre-upstream-update-<date>
git push gitea master --force-with-lease

Then in the main repo:

git checkout -- recipes/core/<component>/recipe.toml
# Revert recipe changes if any

Decision Matrix: Keep vs Drop vs Merge

For each Red Bear commit when syncing:

Is the feature in upstream now?
├── YES → Does upstream's version work for us?
│   ├── YES → DROP our commit, use upstream
│   └── NO  → MERGE: take upstream's structure, re-apply our specific changes
└── NO  → Is it a Red Bear unique feature?
    ├── YES → KEEP our commit
    └── NO  → Is it a toolchain/compat fix?
        ├── Still needed? → KEEP
        └── Fixed upstream? → DROP

Checklist Per Package

• Backup branch created
• Divergence analyzed (commits categorized)
• Sync branch created from latest upstream
• Squash-merge applied
• All conflicts resolved
• Commit made with sync message
• Compiles via repo cook
• Critical symbols verified (for relibc)
• Pushed to Gitea
• Recipe updated (if git+patches mode)
• Dependent components rebuilt
• Full image build succeeds
• QEMU boot test passes

Known Issues and Solutions (2026-06-18 Sync)

Kernel: -Zjson-target-spec Rejected by Cargo in Redoxer

Problem: The upstream kernel Makefile passes -Zjson-target-spec as a cargo CLI flag. The redoxer build environment's cargo (at ~/.redoxer/x86_64-unknown-redox/toolchain/bin/cargo) lists this flag in cargo -Z help but rejects it when invoked through make inside redoxer env bash -ex.

Root cause: The make subprocess inside redoxer env does not properly propagate all cargo unstable feature flags from the CLI. The env-var form (CARGO_UNSTABLE_<FEATURE>=true) works reliably.

Fix (applied to kernel Makefile):

# Replace -Zjson-target-spec CLI flag with env var
export CARGO_UNSTABLE_JSON_TARGET_SPEC=true
# Remove -Zjson-target-spec from the cargo rustc command line

Base: map_bar() API Change

Problem: Upstream changed pcid_handle.map_bar(N) to pcid_handle.map_bar(N, MemoryType::Uncacheable).

Fix: Update all Red Bear drivers that call map_bar() to pass common::MemoryType::Uncacheable as the second argument. Affected drivers: amd-mp2-i2cd, intel-thc-hidd. The upstream drivers (ahcid, nvmed, xhcid) already use the new API.

Base: if let Guards in pcid

Problem: Upstream drivers/pcid/src/cfg_access/mod.rs uses experimental if let guards in match arms.

Fix: Add #![feature(if_let_guard)] to drivers/pcid/src/main.rs.

Base: XHCI Quirk Integration Dropped

Problem: The Red Bear xhci quirk integration (XhciControllerQuirkFlags, staged_port_states, ZERO_64B_REGS handling) was deeply coupled with the old xhci module code and conflicts with upstream's evolved API.

Resolution: Reverted drivers/usb/xhcid/src/xhci/mod.rs and irq_reactor.rs to upstream. The quirk system will be re-integrated as follow-up work on top of the new upstream xhci code.

Base: Workspace Members Missing After Sync

Problem: Taking upstream's Cargo.toml drops Red Bear-specific workspace members (I2C, GPIO, HID, thermald, ucsid, acpi-resource).

Fix: Add all 13 Red Bear workspace members back to the members list. Also fix the [patch] section to point to ../../sources/relibc/ (not the stale ../../relibc/source/ path).

Systemic: redox_syscall Version Alignment

Problem: All Red Bear custom recipes used redox_syscall = "0.7", but the synced base and relibc use 0.8.x.

Fix: Bulk-replaced "0.7" → "0.8" across all 23 Red Bear recipe Cargo.toml files.

Installer: Switch to Local Fork

Problem: The installer recipe used git = "..." + patches = [...], but the patch (redox.patch) fails against the synced upstream code (368 commits of drift).

Fix: Switched recipe to path = "../../../local/sources/installer" (local fork mode). The ext4/GRUB installer changes need to be re-applied to the fork as follow-up work.

Kernel: Switch to Local Fork

Problem: The kernel recipe used git = "..." + 5 patches, all of which fail against the synced upstream code.

Fix: Switched recipe to path = "../../../local/sources/kernel" (local fork mode). All patches were already squash-merged into the fork.

Kernel: Compile Errors After Upstream Sync (31 errors)

After the squash-merge, the kernel had 31 compile errors across ~20 files. These were caused by upstream API evolution interacting with Red Bear code that survived in non-conflicting regions of the merge.

Feature Gates Required

Upstream code uses two experimental features that require feature gates:

• #![feature(asm_cfg)] — for #[cfg(target_arch)] on inline assembly
• #![feature(if_let_guard)] — for if let guards in match arms

Both must be added to src/main.rs.

Unsafe Blocks Required (Rust 2024 Edition)

Rust 2024 edition requires explicit unsafe { } blocks for unsafe operations even inside unsafe fn. Affected files:

• src/arch/x86_shared/cpuid.rs — __cpuid_count
• src/arch/x86_shared/device/tsc.rs — __cpuid
• src/arch/x86_shared/device/local_apic.rs — set_lvt_timer

Removed Red Bear APIs

These Red Bear types/constants no longer exist after taking upstream:

• SchedPolicy — Red Bear's DWRR scheduler policy enum (upstream uses EEVDF)
• RUN_QUEUE_COUNT — Red Bear's per-NUMA run queue count constant
• get_event_stat() — Red Bear's event system statistics (upstream uses eventfd)
• PIPES — Red Bear's pipe storage (renamed to HANDLES)

Fixes:

• Remove SchedPolicy from the pub use in context/mod.rs
• Define RUN_QUEUE_COUNT locally in percpu.rs if still needed
• Implement get_event_stat() in event.rs using the upstream REGISTRY static
• Use HANDLES instead of PIPES in scheme/pipe.rs

CONTEXT_SWITCH_LOCK Missing

Upstream added pub static CONTEXT_SWITCH_LOCK: AtomicBool to src/context/arch/x86_64.rs but our fork was missing it. Must be added with the core::sync::atomic::AtomicBool import.

new_addrsp_guard Field Missing from PercpuBlock

Upstream added a separate new_addrsp_guard: Cell<Option<AddrSpaceSwitchReadGuard>> field to PercpuBlock for storing the addr space read guard during context switch. Our fork only had new_addrsp_tmp (for the Arc<AddrSpaceWrapper>). Both fields must exist.

switch.rs Must Be Replaced Entirely

The Red Bear DWRR scheduler in switch.rs is fundamentally incompatible with the upstream EEVDF scheduler data structures. The RunContextData changed from set: [VecDeque<WeakContextRef>; N] to queue: BTreeMap<(vd, rem_slice, ctxt_id), ...>.

The entire switch.rs must be replaced with the upstream version:

git show upstream/master:src/context/switch.rs > src/context/switch.rs

kcall Trait Method Signature Changed

Upstream changed id: usize to fds: &[usize] in the kcall trait method. Update the ProcScheme impl to extract the first element:

fn kcall(&self, fds: &[usize], ...) -> Result<usize> {
    let id = *fds.first().ok_or(Error::new(EBADF))?;

ProcSchemeVerb New Variants

Upstream added new variants to ProcSchemeVerb: RegsInt, RegsFloat, RegsEnv, SchedAffinity, Start. The match in scheme/proc.rs must handle them (currently returns ENOSYS).

notify_files Type Change

handle_notify_files() now expects UnmapVec (SmallVec<[UnmapResult; 16]>) instead of Vec. Import UnmapVec from context::memory and use UnmapVec::new() instead of Vec::new().

Rsdp::get_rsdp Signature Change

Upstream changed the parameter from Option<NonNull<u8>> to Option<*const u8>. Convert at the call site:

Rsdp::get_rsdp(already_supplied_rsdp.map(|ptr| ptr.as_ptr() as *const u8))

Syntax Error in acpi/madt/arch/x86.rs

A const statement was accidentally placed inside a use crate::{...} block during the merge. Must be moved outside.

halt_boot Scope Issue

halt_boot was defined inside the impl KernelArgs block, making it a method rather than a free function. Must be moved outside the impl block.

Cookbook Symlink Behavior for Local Forks

Critical: When a recipe uses path = "..." pointing inside /local/sources/ or /local/recipes/, the cookbook creates a symlink (not a copy) from recipes/<name>/source → the local fork path. This means:

1. Build artifacts and cargo operations can modify files in the local fork through the symlink.
2. After each build attempt, verify the fork's git status:

   cd local/sources/<component>
   git status --short
   

3. If the working tree is dirty, reset before rebuilding:

   git checkout -- .
   

4. Always clean both target/ and source/ when restarting a kernel build:

   rm -rf recipes/core/kernel/target recipes/core/kernel/source
   cd local/sources/kernel && git checkout -- .
   

m4: getlocalename_l-unsafe.c Platform Detection

Problem: m4's gnulib getlocalename_l-unsafe.c has a Redox case guarded by HAVE_GETLOCALENAME_L, which relibc doesn't define.

Fix: Changed the guard from __redox__ && HAVE_GETLOCALENAME_L to just __redox__ and return "C" (Redox only supports the C/POSIX locale).

m4: rpl_realloc Multiple Definition

Problem: GCC 10+ defaults to -fno-common, causing rpl_realloc (gnulib replacement) to be multiply defined across translation units.

Fix: Add -fcommon to CFLAGS in the m4 recipe:

script = """
DYNAMIC_INIT
export CFLAGS="-fcommon $CFLAGS"