cookbook: make every staging path collision-free; restore the native toolchain

Concurrency
-----------
Package variants of one recipe (llvm-native, llvm-native.dev,
llvm-native.runtime) share a single target/<triple>/ directory and are
cooked concurrently by the cook_jobs thread pool, but every staging path
under it had a fixed name -- stage.tmp and sysroot.tmp. The variants
therefore raced: one renamed or removed the shared directory while a
sibling was still writing into it.

  Canonicalize stage dir failed at '.../stage.tmp': No such file or directory
  Renaming failed from '.../sysroot.tmp' to '.../sysroot': File exists

llvm-native cooks clean on its own -- verified, zero errors, all three
variants published -- but failed in every parallel build for this
reason. It was never a compile error, which is what the earlier
'C++/pthread header gaps' note in the config misattributed.

Both stage.tmp sites and the sysroot site now go through one
unique_tmp_dir() helper. A pid alone cannot disambiguate (the pool is
threads in one process), so a process-wide counter supplies uniqueness.

Config
------
Restore gcc-native, llvm-native and rust-native to redbear-full.
gcc-native was set to "ignore" and the other two were commented out as
'not needed for greeter proof'. Both forms are forbidden by AGENTS.md
ABSOLUTE RULE -- NEVER DELETE, NEVER IGNORE, NEVER COMMENT OUT, and by
local/AGENTS.md 'No build-excluded until ported packages': a package
that does not build gets ported, not excluded. llvm-native is required,
not optional -- it supplies the host LLVM dev tree that libclc and
Mesa's iris/radeonsi CLC path need.
This commit is contained in:
2026-08-03 14:53:39 +03:00
parent 2f6e2d2b58
commit 4fac89eec4
2 changed files with 46 additions and 11 deletions
+9 -3
View File
@@ -262,10 +262,16 @@ redbear-meta = {}
redbear-power = {}
# Native build toolchain (Phase 3: GCC + binutils running on redox)
gcc-native = "ignore"
# All four are first-class members of this target. gcc-native was set to
# "ignore" and llvm-native/rust-native were commented out as "not needed for
# greeter proof"; both forms are forbidden by AGENTS.md § ABSOLUTE RULE —
# NEVER DELETE, NEVER IGNORE, NEVER COMMENT OUT and by local/AGENTS.md
# § "No 'build-excluded until ported' packages". A package that does not build
# gets ported, not excluded.
gcc-native = {}
binutils-native = {}
# llvm-native = {} # suppressed: Redox C++/pthread header gaps; not needed for greeter proof
# rust-native = {} # suppressed: depends on llvm-native; not needed for greeter proof
llvm-native = {}
rust-native = {}
# Desktop fonts and icons
dejavu = {}
+37 -8
View File
@@ -853,7 +853,7 @@ pub fn build(
let build_dir = get_sub_target_dir(target_dir, "build");
if !stage_dir.is_dir() {
// Create stage.tmp
let stage_dir_tmp = target_dir.join("stage.tmp");
let stage_dir_tmp = unique_tmp_dir(target_dir, "stage");
create_dir_clean(&stage_dir_tmp)?;
// Create build dir, if it does not exist
@@ -1139,18 +1139,21 @@ fn build_deps_dir(
deps_modified: SystemTime,
) -> Result<bool, String> {
// Per-call staging path. A single shared "<sysroot>.tmp" is unsafe:
//
// (see unique_tmp_dir() below for the general form of this problem)
// package variants of one recipe (llvm-native{,.dev,.runtime}) share this
// target directory and are cooked concurrently by the cook_jobs thread
// pool, so create_dir_clean() below would wipe a sibling's half-populated
// staging tree while it was still extracting into it. The counter (not a
// pid -- the pool is threads inside one process) keeps each cook's staging
// private; the rename at the end is what publishes it.
static SYSROOT_TMP_SEQ: AtomicUsize = AtomicUsize::new(0);
let deps_dir_tmp = deps_dir.with_added_extension(format!(
"tmp.{}.{}",
std::process::id(),
SYSROOT_TMP_SEQ.fetch_add(1, AtomicOrdering::Relaxed)
));
let deps_dir_tmp = unique_tmp_dir(
deps_dir.parent().unwrap_or(Path::new(".")),
&deps_dir
.file_name()
.map(|s| s.to_string_lossy().into_owned())
.unwrap_or_else(|| "sysroot".to_string()),
);
if deps_dir.is_dir() {
let tags_dir = deps_dir.join(".tags");
let sysroot_modified = modified_dir(&tags_dir).unwrap_or(SystemTime::UNIX_EPOCH);
@@ -1252,6 +1255,32 @@ fn build_deps_dir(
Ok(false)
}
/// Build a staging path that no other concurrent cook can collide with.
///
/// Several package variants of one recipe (e.g. `llvm-native`,
/// `llvm-native.dev`, `llvm-native.runtime`) share a single `target/<triple>/`
/// directory and are cooked concurrently by the `cook_jobs` thread pool. Every
/// staging path under that directory was a fixed name — `stage.tmp`,
/// `sysroot.tmp` — so the variants raced: one would rename or remove the
/// shared directory while a sibling was still writing into it, producing
///
/// Canonicalize stage dir failed at ".../stage.tmp": No such file or directory
/// Renaming failed from ".../sysroot.tmp" to ".../sysroot": File exists
///
/// which failed the whole recipe. `llvm-native` cooks clean on its own but
/// failed in every parallel build for exactly this reason.
///
/// A pid is not enough to disambiguate — the pool is threads inside one
/// process — so a process-wide counter supplies the unique component.
fn unique_tmp_dir(base: &Path, name: &str) -> PathBuf {
static TMP_SEQ: AtomicUsize = AtomicUsize::new(0);
base.join(format!(
"{name}.tmp.{}.{}",
std::process::id(),
TMP_SEQ.fetch_add(1, AtomicOrdering::Relaxed)
))
}
/// Calculate automatic dependencies
fn build_auto_deps(
recipe: &Recipe,
@@ -1311,7 +1340,7 @@ pub fn build_remote(
if !stage_dir.is_dir() {
let (_, source_pkgar, _) = package_source_paths(package, target_dir);
let stage_dir_tmp = target_dir.join("stage.tmp");
let stage_dir_tmp = unique_tmp_dir(target_dir, "stage");
pkgar::extract(source_pubkey, &source_pkgar, &stage_dir_tmp).map_err(|err| {
format!(
"failed to install '{}' in '{}': {:?}",