cookbook: serialise cook units that share a recipe directory
The unique-temp-name fix was treating symptoms. The actual defect is structural: run_parallel_cook documents the invariant 'each recipe builds in its own target/stage/sysroot' and that invariant is false. dep_levels() keys on the package NAME, so a recipe's optional-package variants -- llvm-native, llvm-native.dev, llvm-native.runtime -- are distinct units with no dependency between them. They therefore land in the same level and cook concurrently while sharing one target/<triple>/ directory, because build/, sysroot/ and the stage* dirs are all derived from the recipe directory, not the package name. create_dir_clean() on the shared build/ then wipes a sibling's tree mid-compile: ninja: error: failed recompaction: No such file or directory which is the error the earlier parallel builds kept producing and which unique temp names could never have fixed -- build/ is a real, shared, destructively re-created directory, not a staging path. It is also why llvm-native cooks clean in isolation but failed in every parallel build, and why that failure was misread as a porting gap serious enough to exclude the package from the config. Make the recipe DIRECTORY the unit of mutual exclusion. Distinct recipes still cook fully in parallel; only siblings serialise, and that costs almost nothing because one cook already populates every stage dir for the recipe -- the siblings that follow find their work done and return cached. It also stops LLVM being built three times over. Residual, not addressed here: recipes joined by [source] same_as (e.g. llvm-native -> llvm21) share a source tree. That is read-only during cook, but a concurrent fetch/patch of the shared source would be a separate race.
This commit is contained in:
@@ -375,6 +375,41 @@ fn print_cached(command: &CliCommand, recipe: &PackageName) {
|
||||
/// This is safe because each recipe builds in its own target/stage/sysroot and a
|
||||
/// recipe's sysroot is assembled from its dependencies' (lower-level, already
|
||||
/// complete) stage pkgars — read-only for concurrent siblings.
|
||||
/// Serialise cook units that share a recipe directory.
|
||||
///
|
||||
/// `dep_levels` keys on the package NAME, so a recipe's optional-package
|
||||
/// variants (`llvm-native`, `llvm-native.dev`, `llvm-native.runtime`) are
|
||||
/// distinct units with no dependency between them and therefore land in the
|
||||
/// same level and cook concurrently. They do NOT have independent workspaces:
|
||||
/// `build/`, `sysroot/` and the `stage*` dirs are all derived from the shared
|
||||
/// `target/<triple>/` of their common recipe directory. `create_dir_clean()`
|
||||
/// on `build/` then wipes a sibling's tree mid-compile:
|
||||
///
|
||||
/// ninja: error: failed recompaction: No such file or directory
|
||||
/// Canonicalize stage dir failed at ".../stage.tmp": No such file or directory
|
||||
///
|
||||
/// which is why `llvm-native` cooked clean on its own and failed in every
|
||||
/// parallel build. Unique temp-directory names alone cannot fix this, because
|
||||
/// `build/` is a real, shared, destructively re-created directory.
|
||||
///
|
||||
/// This restores the invariant `run_parallel_cook` documents -- "each recipe
|
||||
/// builds in its own target/stage/sysroot" -- by making the recipe DIRECTORY,
|
||||
/// not the package name, the unit of mutual exclusion. Distinct recipes still
|
||||
/// cook fully in parallel. Serialising siblings costs almost nothing: one cook
|
||||
/// already populates every stage dir for the recipe, so the siblings that
|
||||
/// follow find their work done and return cached instead of rebuilding (which
|
||||
/// also stops LLVM being built three times over).
|
||||
fn recipe_dir_lock(dir: &std::path::Path) -> std::sync::Arc<std::sync::Mutex<()>> {
|
||||
use std::collections::HashMap;
|
||||
use std::path::PathBuf;
|
||||
use std::sync::{Arc, Mutex, OnceLock};
|
||||
static LOCKS: OnceLock<Mutex<HashMap<PathBuf, Arc<Mutex<()>>>>> = OnceLock::new();
|
||||
let key = dir.canonicalize().unwrap_or_else(|_| dir.to_path_buf());
|
||||
let table = LOCKS.get_or_init(|| Mutex::new(HashMap::new()));
|
||||
let mut guard = table.lock().unwrap_or_else(|e| e.into_inner());
|
||||
Arc::clone(guard.entry(key).or_insert_with(|| Arc::new(Mutex::new(()))))
|
||||
}
|
||||
|
||||
fn run_parallel_cook(
|
||||
config: &CliConfig,
|
||||
command: &CliCommand,
|
||||
@@ -435,8 +470,14 @@ fn run_parallel_cook(
|
||||
break;
|
||||
}
|
||||
let recipe = level_recipes[i];
|
||||
// Siblings sharing this recipe directory must not cook
|
||||
// concurrently -- they share build/ and sysroot/.
|
||||
let dir_lock = recipe_dir_lock(&recipe.dir);
|
||||
let _dir_guard =
|
||||
dir_lock.lock().unwrap_or_else(|e| e.into_inner());
|
||||
let r = repo_inner(level_config, command, recipe)
|
||||
.map_err(|e| format!("{e:#}"));
|
||||
drop(_dir_guard);
|
||||
results_ref.lock().unwrap().push((recipe.clone(), r));
|
||||
});
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user