vasilito/RedBear-OS
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(rust-native): build host bootstrap stages against a pure host LLVM

Browse Source 
rust-native failed with ~200 errors inside the HOST's own glibc and
libstdc++ headers while compiling rustc_llvm's llvm-wrapper/*.cpp for
x86_64-unknown-linux-gnu.

rustc_llvm's build script passes `llvm-config --includedir` straight to
the host c++. The redoxer toolchain's include/ is a MIXED directory,
holding relibc's TARGET headers (stdlib.h, features.h, fenv.h, ...)
beside the llvm/ and clang/ ones, so relibc's features.h shadowed
glibc's, __GLIBC_PREREQ was never defined, and every host header testing
it failed:

    /usr/include/c++/16/x86_64-pc-linux-gnu/bits/os_defines.h:44:19:
    error: missing binary operator before token '('

Reproducible in three lines with no rustc involved:

    printf '#include <cstdlib>\nint main(){}' > t.cpp
    c++ -std=c++17 -I${HOME}/.redoxer/x86_64-unknown-redox/toolchain/include -c t.cpp

This is the same leak recorded as gcc-native's stopping point in
local/docs/NATIVE-TOOLCHAIN-WORKSTREAM.md -- one root cause, both
recipes -- though in the opposite direction to what that note assumed:
target headers reaching a host compile, not the reverse.

Fixed with host:llvm-native (+ .dev/.runtime). Host deps stage into
COOKBOOK_TOOLCHAIN, a tree SEPARATE from the target sysroot, so the
GNU/Linux libLLVM.so.21.1 cannot collide with the Redox one llvm21
stages -- the collision this recipe's llvm21 comment already records.
Its include tree holds only llvm/, llvm-c/, clang/, clang-c/, lld/. An
unset COOKBOOK_TOOLCHAIN is now a hard error rather than a silent
fallback to the polluted path.

Second defect, exposed by the first fix: DYNAMIC_INIT exports
CXXFLAGS=-fno-hardened for the GCC 16 cross compiler, and the unset list
dropped CC/CXX/LDFLAGS but not CFLAGS/CXXFLAGS/CPPFLAGS. cc-rs falls
back to the generic CXXFLAGS when the target-specific one is empty and
handed it to the host c++, which now resolves through
COOKBOOK_TOOLCHAIN/bin to llvm-native's clang++:

    rustc_llvm@0.0.0: c++: error: unknown argument: '-fno-hardened'

A pre-existing leak: the cross recipes/dev/rust carries the same unset
list and never hit it only because nothing puts a clang on its PATH.
llvm-native's own recipe already unsets these three for this reason.

Target-side compilation is unaffected: config.toml pins cc/cxx/ar/linker
for the Redox triple, and the target RUSTFLAGS are captured into
CARGO_TARGET_<ARCH>_UNKNOWN_REDOX_RUSTFLAGS before the unset.

Also carries earlier uncommitted working-tree changes to this recipe
that the build depends on: the explicit relibc dependency, the
llvm-native -> llvm21 switch for the Redox-hosted rustc, and the
COOKBOOK_HOST_SYSROOT assignment, plus deny-warnings = false in
config.toml.

Verified: rustc, cargo, rustdoc, rustfmt, clippy-driver and
librustc_driver-*.so stage as Redox binaries -- ELF interpreter
/lib/ld64.so.1, OS/ABI System V, NEEDED libc.so.6 from relibc.
This commit is contained in:
vasilito
2026-08-05 15:41:10 +03:00
parent 5f7a2f2979
commit 75a33e0430
2 changed files with 135 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
local/recipes/dev/rust-native/config.toml
+21
View File
@@ -19,6 +19,17 @@ sysconfdir = "etc"
[rust]
backtrace = false
codegen-tests = false
# rustc's bootstrap defaults this to true, which puts -D warnings on every crate
# it compiles. That is rustc's own CI hygiene policy and it should not govern our
# dependencies -- but our libc fork reaches the build through [patch.crates-io]
# as a PATH dependency (local/patches/rust/02-libc-redox-fork.patch), and cargo
# applies --cap-lints allow only to REGISTRY dependencies. So the vendored copy
# gets linted where the real crates.io libc never would, and each inert lint is a
# hard error in turn: first `unreachable_pub` (5 sites), then `dead_code`
# (grantpt/posix_openpt/ptsname, 6 more). Fixing them one attribute at a time is
# whack-a-mole against upstream code we did not write. Turning the deny off is
# the knob bootstrap provides for exactly this, and is what distributions use.
deny-warnings = false
[target.COOKBOOK_TARGET]
cc = "COOKBOOK_GNU_TARGET-gcc"
@@ -27,3 +38,13 @@ ar = "COOKBOOK_GNU_TARGET-ar"
linker = "COOKBOOK_GNU_TARGET-gcc"
crt-static = false
llvm-config = "COOKBOOK_SYSROOT/bin/llvm-config"
# Host triples get the TOOLCHAIN's llvm-config, not the sysroot's. Bootstrap
# still builds host-side stages on the way to a Redox-hosted rustc, and without
# these it reaches for the target llvm-config for host work too. recipes/dev/rust
# carries the same two sections for the same reason; rust-native was missing them.
[target.x86_64-unknown-linux-gnu]
llvm-config = "COOKBOOK_TOOLCHAIN/bin/llvm-config"
[target.aarch64-unknown-linux-gnu]
llvm-config = "COOKBOOK_TOOLCHAIN/bin/llvm-config"
local/recipes/dev/rust-native/recipe.toml
+114 -8
View File
@@ -4,33 +4,139 @@ same_as = "../../../../recipes/dev/rust"
[build]
template = "custom"
dependencies = [
# relibc is explicit. Clearing the stale sysroot showed it had only ever
# arrived transitively: with it gone the sysroot had no libc headers at all
# and every C++ compile failed with
# error: 'abort' has not been declared in '::'
# error: 'abs' has not been declared in '::' (and ~40 more from <cstdlib>)
# Exactly the failure local/docs/NATIVE-TOOLCHAIN-WORKSTREAM.md records for
# gcc-native as finding #8 ('fenv_t' has not been declared), same cause.
"relibc",
"zlib",
"curl",
"openssl3",
"llvm-native",
"llvm-native.runtime",
# llvm21, NOT llvm-native. This recipe's config.toml sets both `host` and
# `target` to the Redox triple, so the rustc it produces runs on Redox and
# must link a REDOX libLLVM. llvm-native is a deliberate PURE HOST build
# (see its recipe header) -- depending on it staged a GNU/Linux
# libLLVM.so.21.1 into the target sysroot, and bootstrap then linked
# librustc_driver-*.so against it:
# .../sysroot/lib/libLLVM-21.so: undefined reference to `abort@GLIBC_2.2.5'
# (213 such references, every one a glibc symbol Redox does not have)
# llvm21 is the cross build and produces a Redox ELF; recipes/dev/rust --
# the equivalent cross recipe that works -- depends on exactly this.
"llvm21",
"llvm21.runtime",
# rustc_llvm compiles llvm-wrapper/*.cpp, which #include
# "llvm/Config/llvm-config.h". Those headers are staged by the .dev
# package, so without it the build fails with
# LLVMWrapper.h:6:10: fatal error: llvm/Config/llvm-config.h:
# No such file or directory
# recipes/dev/rust declares the equivalent llvm21.dev for the same reason.
"llvm-native.dev",
"llvm21.dev",
# host:llvm-native supplies the HOST llvm-config for the host-triple
# sections of config.toml. It must be a `host:` package, not a plain one:
# host deps stage into COOKBOOK_TOOLCHAIN (target/<t>/toolchain/), a tree
# SEPARATE from the target sysroot, so the GNU/Linux libLLVM.so.21.1 can
# never collide with the Redox one llvm21 stages -- which is exactly the
# breakage the llvm21 comment above records. Main goes in `dependencies`
# because the cook-tree (src/cook/tree.rs) only follows `dependencies` when
# resolving what to COOK; cooking it produces all three subpackage pkgars.
"host:llvm-native",
]
# Staged (not cooked) into COOKBOOK_TOOLCHAIN: .dev carries usr/include/llvm**
# plus the cmake exports, .runtime carries usr/bin/llvm-config and libLLVM*.so.
# Merged in one tree, llvm-config (which is relocatable -- it derives its prefix
# from its own location) then reports an includedir holding ONLY llvm/, llvm-c/,
# clang/, clang-c/, lld/. That is the whole point: see the script below.
dev-dependencies = [
"host:llvm-native.dev",
"host:llvm-native.runtime",
]
script = """
DYNAMIC_INIT
ARCH="${TARGET%%-*}"
export CARGO_TARGET_${ARCH^^}_UNKNOWN_REDOX_RUSTFLAGS="${RUSTFLAGS}"
cat "${COOKBOOK_ROOT}/bin/${TARGET}-llvm-config" > "${COOKBOOK_SYSROOT}/bin/llvm-config"
export LD_LIBRARY_PATH="${COOKBOOK_HOST_SYSROOT}/lib:${LD_LIBRARY_PATH}"
# The llvm-config just staged is the cross wrapper (bin/${TARGET}-llvm-config), a python
# script that reads COOKBOOK_HOST_SYSROOT, COOKBOOK_SYSROOT and TARGET from the
# ENVIRONMENT and exits 1 if any is missing. The cookbook passes COOKBOOK_SYSROOT
# and TARGET down, but COOKBOOK_HOST_SYSROOT is only a shell variable here -- so
# when rustc's bootstrap spawns `llvm-config --version` during the install step
# the wrapper sees nothing and the whole build dies at the very last phase:
# Error: COOKBOOK_HOST_SYSROOT or COOKBOOK_SYSROOT or TARGET not set
# Command `".../sysroot/bin/llvm-config" "--version"` failed with exit code 1
# The cookbook does NOT set COOKBOOK_HOST_SYSROOT -- nothing in src/ assigns it,
# and recipes that need it define it themselves (see local/recipes/libs/mesa,
# which sets exactly this path). It was empty here, which silently broke two
# lines that already used it: LD_LIBRARY_PATH resolved to a bare "/lib:", and
# the COOKBOOK_TOOLCHAIN substitution below wrote an empty toolchain path into
# config.toml. Exporting an unset variable fixed neither -- it has to be set.
export COOKBOOK_HOST_SYSROOT="${COOKBOOK_HOST_SYSROOT:-${HOME}/.redoxer/${TARGET}/toolchain}"
export COOKBOOK_SYSROOT TARGET
# The host-triple llvm-config must NOT be the redoxer toolchain's.
#
# ${HOME}/.redoxer/${TARGET}/toolchain/include is a MIXED directory: it holds
# relibc's TARGET headers (stdlib.h, features.h, fenv.h, ...) alongside the
# llvm/ and clang/ ones. rustc_llvm's build script passes
# `llvm-config --includedir` straight to the HOST c++ compiling
# llvm-wrapper/*.cpp, so relibc's features.h shadows glibc's, __GLIBC_PREREQ is
# never defined, and every host libstdc++/glibc header that tests it explodes:
# /usr/include/c++/16/x86_64-pc-linux-gnu/bits/os_defines.h:44:19:
# error: missing binary operator before token '('
# 44 | #if __GLIBC_PREREQ(2,15) && defined(_GNU_SOURCE)
# followed by ~200 more from /usr/include/stdlib.h, locale_classes.h,
# shared_ptr_base.h ('abort'/'calloc'/'__c_locale' has not been declared).
# Reproducible in three lines, with no rustc involved:
# printf '#include <cstdlib>\\nint main(){}' > t.cpp
# c++ -std=c++17 -I${HOME}/.redoxer/${TARGET}/toolchain/include -c t.cpp
# This is the SAME host/target header leak recorded as gcc-native's stopping
# point in local/docs/NATIVE-TOOLCHAIN-WORKSTREAM.md -- one root cause, both
# recipes.
#
# host:llvm-native is the fix: a pure HOST LLVM whose include tree contains
# ONLY llvm/, llvm-c/, clang/, clang-c/, lld/. The cookbook stages host deps
# into COOKBOOK_TOOLCHAIN, which is a different tree from COOKBOOK_SYSROOT, so
# the Redox llvm21 the target needs and the host LLVM the stage1 compiler needs
# coexist instead of overwriting each other.
if [ -z "${COOKBOOK_TOOLCHAIN}" ]; then
echo "rust-native: COOKBOOK_TOOLCHAIN is unset -- the host:llvm-native" >&2
echo "dependencies did not stage. Refusing to fall back to the redoxer" >&2
echo "toolchain: its include/ leaks relibc headers into host compiles." >&2
exit 1
fi
COOKBOOK_HOST_LLVM="${COOKBOOK_TOOLCHAIN}"
# Must come AFTER the assignments above: this previously expanded to a bare
# "/lib:" because the variable was empty at this point in the script.
# The host LLVM lib dir comes FIRST -- stage1 (host -> host) links against the
# host libLLVM.so.21.1, and must not pick up the redoxer toolchain's copy.
export LD_LIBRARY_PATH="${COOKBOOK_HOST_LLVM}/lib:${COOKBOOK_HOST_SYSROOT}/lib:${LD_LIBRARY_PATH}"
cat ${COOKBOOK_RECIPE}/config.toml > config.toml
sed -i "s|COOKBOOK_SYSROOT|${COOKBOOK_SYSROOT}|g" config.toml
sed -i "s|COOKBOOK_TOOLCHAIN|${COOKBOOK_HOST_SYSROOT}|g" config.toml
sed -i "s|COOKBOOK_TOOLCHAIN|${COOKBOOK_HOST_LLVM}|g" config.toml
sed -i "s|COOKBOOK_TARGET|${TARGET}|g" config.toml
sed -i "s|COOKBOOK_GNU_TARGET|${GNU_TARGET}|g" config.toml
unset AR AS CC CXX LD LDFLAGS NM OBJCOPY OBJDUMP RANLIB READELF RUSTFLAGS CARGO_ENCODED_RUSTFLAGS STRIP
# CFLAGS/CXXFLAGS/CPPFLAGS join this list: they are the CROSS flags DYNAMIC_INIT
# exported for the Redox target, and they poisoned the stage1 (host -> host)
# build. DYNAMIC_INIT sets CXXFLAGS=-fno-hardened for the GCC 16 cross compiler;
# cc-rs falls back to the generic CXXFLAGS when the target-specific
# CXXFLAGS_x86_64_unknown_linux_gnu is empty, and hands it to the host c++ --
# which resolves through COOKBOOK_TOOLCHAIN/bin to llvm-native's clang++, and
# clang has no such flag:
# rustc_llvm@0.0.0: c++: error: unknown argument: '-fno-hardened'
# The cross recipes/dev/rust never hit this only because nothing puts a clang on
# its PATH, so its host c++ stays GCC, which accepts the flag. local/recipes/dev/
# llvm-native already unsets exactly these three for exactly this reason
# ("Drop every cross-compile flag DYNAMIC_INIT exported").
# Target-side compilation is unaffected: config.toml pins cc/cxx/ar/linker for
# the Redox triple explicitly, and the target RUSTFLAGS were captured into
# CARGO_TARGET_<ARCH>_UNKNOWN_REDOX_RUSTFLAGS at the top of this script, before
# this unset.
unset AR AS CC CXX LD LDFLAGS CFLAGS CXXFLAGS CPPFLAGS NM OBJCOPY OBJDUMP RANLIB READELF RUSTFLAGS CARGO_ENCODED_RUSTFLAGS STRIP
python3 "${COOKBOOK_SOURCE}/x.py" install \
--config config.toml \
@@ -43,4 +149,4 @@ rm -rf "${COOKBOOK_STAGE}"/usr/lib/rustlib/*.log
[package]
description = "Native Rust toolchain for Red Bear OS (rustc + cargo running on redox)"
dependencies = ["llvm-native.runtime"]
dependencies = ["llvm21.runtime"]