vasilito/RedBear-OS
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: vasilito/RedBear-OS:kernel-0.3.0
Branches Tags
vasilito/RedBear-OS:master vasilito/RedBear-OS:0.3.1 vasilito/RedBear-OS:0.3.0 vasilito/RedBear-OS:submodule/bootloader vasilito/RedBear-OS:submodule/kernel vasilito/RedBear-OS:submodule/relibc vasilito/RedBear-OS:submodule/syscall vasilito/RedBear-OS:submodule/base vasilito/RedBear-OS:kernel-0.3.0 vasilito/RedBear-OS:relibc-0.3.0 vasilito/RedBear-OS:submodule/redox-drm vasilito/RedBear-OS:submodule/userutils vasilito/RedBear-OS:syscall-0.3.0 vasilito/RedBear-OS:0.2.5 vasilito/RedBear-OS:submodule/redoxfs vasilito/RedBear-OS:submodule/installer vasilito/RedBear-OS:submodule/libredox vasilito/RedBear-OS:0.2.4 vasilito/RedBear-OS:recovered/quirks vasilito/RedBear-OS:0.2.3 vasilito/RedBear-OS:0.2.2 vasilito/RedBear-OS:0.2.1 vasilito/RedBear-OS:0.2.0 vasilito/RedBear-OS:bootprocess vasilito/RedBear-OS:main
..
compare: vasilito/RedBear-OS:baadbfc539de5ebcd7115a00313390e8372e1316
Branches Tags
vasilito/RedBear-OS:0.3.1 vasilito/RedBear-OS:0.3.0 vasilito/RedBear-OS:submodule/bootloader vasilito/RedBear-OS:master vasilito/RedBear-OS:submodule/kernel vasilito/RedBear-OS:submodule/relibc vasilito/RedBear-OS:submodule/syscall vasilito/RedBear-OS:submodule/base vasilito/RedBear-OS:kernel-0.3.0 vasilito/RedBear-OS:relibc-0.3.0 vasilito/RedBear-OS:submodule/redox-drm vasilito/RedBear-OS:submodule/userutils vasilito/RedBear-OS:syscall-0.3.0 vasilito/RedBear-OS:0.2.5 vasilito/RedBear-OS:submodule/redoxfs vasilito/RedBear-OS:submodule/installer vasilito/RedBear-OS:submodule/libredox vasilito/RedBear-OS:0.2.4 vasilito/RedBear-OS:recovered/quirks vasilito/RedBear-OS:0.2.3 vasilito/RedBear-OS:0.2.2 vasilito/RedBear-OS:0.2.1 vasilito/RedBear-OS:0.2.0 vasilito/RedBear-OS:bootprocess vasilito/RedBear-OS:main

30 Commits
kernel-0.3.0 .. baadbfc539

Author SHA1 Message Date
vasilito baadbfc539 kernel: refresh Cargo.lock (mtime + relibc-rebuild attempt) 
This is a bookkeeping commit to capture Cargo.lock and
the touched lib.rs from the cookbook's auto-stash step in
the 0.2.5 kernel build attempt that hit the
json-target-spec / rust toolchain mismatch. The actual
code changes are on the kernel branch and the relevant
submodule gitlink will be bumped in a future commit.

The kernel cross-build is using the same nightly-2025-10-03
toolchain (per the kernel fork's rust-toolchain.toml).
The cookbook uses nightly-2026-04-01 which has the
target-spec-json flag the Makefile needs. A unified
toolchain setup is a future-work item.
 2026-07-02 13:43:14 +03:00
vasilito d41d0aa728 kernel: support proc:{thread_fd}/<sub-handle> path format 
The relibc fork's pthread_setname_np / pthread_getname_np /
pthread_setaffinity_np / pthread_getaffinity_np and
mutex_owner_id_is_live all use the path format
'proc:{thread_fd}/<sub-handle>' (e.g.
'proc:123/name' or 'proc:123/sched-affinity') via a single
Sys::open() call.

Previously the proc scheme's OpenTy::Auth handler in
src/scheme/proc.rs only recognized 'new-context' and
'cur-context' as literal strings, so '123/name' would
hit the _ => ENOENT arm and the relibc calls would
fail with ENOENT at runtime.

Fix: add a third arm in OpenTy::Auth that splits the
operation string on the first '/', parses the prefix as
a numeric context id, looks up the corresponding
ContextHandle in the HANDLES map, and recursively
dispatches to openat_context with the suffix as the
sub-handle path. This makes 'proc:123/name' resolve to
the same handle chain that 'dup(123, "name")' would
have produced.

The recursive call is safe because openat_context
doesn't depend on the Authority-only state. The
HANDLES map is read-locked; we drop the lock before
the recursive call by scoping the handles variable.

Discovered by Oracle review of Phase 0c patches
(Issue 1). The bug was latent in the original
P5-proc-setschedpolicy patch (before Phase 0c) and
survived because the relibc code paths were never
exercised at runtime.
 2026-07-02 10:53:52 +03:00
vasilito d37b421cb3 kernel: fix wakeup_contexts vs steal_work deadlock 
Two-sided fix for the lock-ordering deadlock discovered by
Oracle review (Issue 24):

1. wakeup_contexts (this fn) held IDLE_CONTEXTS while
   waiting for SchedQueuesLock on its own CPU via
   SchedQueuesLock::new(&percpu.sched). If another CPU's
   steal_work was holding that SchedQueuesLock (via a victim
   SchedQueuesLock) and waiting for IDLE_CONTEXTS, both
   threads spin forever.

   Fix: drop idle_contexts immediately after building the
   wakeups Vec. The Vec is the only data we need; releasing
   the lock here means steal_work on another CPU can proceed
   while this CPU acquires its own SchedQueuesLock.

2. steal_work held a victim's SchedQueuesLock (victim_lock)
   while calling idle_contexts(token.downgrade()).push_back
   on a context that turned out to be Blocked. This is the
   matching side of the deadlock: CPU A held IDLE_CONTEXTS and
   waited for its own SchedQueuesLock; CPU B (steal_work) held
   CPU A's SchedQueuesLock and waited for IDLE_CONTEXTS.

   Fix: use idle_contexts_try (try_lock) instead of
   idle_contexts (blocking lock). If IDLE_CONTEXTS is busy
   (owned by wakeup_contexts on another CPU), skip the
   push-back; the context will be re-checked on the next
   wakeup round because it was not removed from IDLE_CONTEXTS
   (the Blocked status was set, but it stayed in IDLE_CONTEXTS
   because we never re-pushed it).

The original code at line 429 used idle_contexts (blocking)
which is what makes this a real deadlock. try_lock is safe
because:
  - If try_lock succeeds, the context is correctly pushed
  - If try_lock fails, the context is still in IDLE_CONTEXTS
    (we never removed it), so the next wakeup_contexts will
    find it again
 2026-07-02 10:36:17 +03:00
vasilito 6a5582783f kernel: fix inverted nice-to-prio mapping in proc Priority handle 
The proc scheme's Self::Priority write handler used
'kernel_prio = (20 - nice) as usize' which maps:
  nice -20 -> kernel_prio 40, clamped to 39
  nice  0 -> kernel_prio 20
  nice 19 -> kernel_prio 1

But SCHED_PRIO_TO_WEIGHT[39] = 15 (lowest weight, least CPU
time), and SCHED_PRIO_TO_WEIGHT[0] = 88761 (highest weight,
most CPU time). So the old formula gave processes that set
nice to the most favorable value (-20) the LEAST CPU time,
and processes that set nice to the least favorable value (+19)
the MOST CPU time. Completely inverted.

Correct formula: kernel_prio = (nice + 20) as usize, giving:
  nice -20 -> kernel_prio  0 (highest weight, most CPU)
  nice  0 -> kernel_prio 20
  nice 19 -> kernel_prio 39 (lowest weight, least CPU)

The corresponding read path (kernel_prio -> nice) is
'nice = (context.prio as i32 - 20)'. The old read was
'(20 - context.prio as i32)' which had the same inversion
plus a clamp that hid the bug for prio 0 (-> nice 20, clamped
to 19, never returned the correct -20).

Also fix the self-contradictory doc comment on Context::
set_sched_other_prio which claimed 'prio 39 is the lowest nice
value (highest CPU weight)' — actually prio 0 is the highest
weight and highest priority.

Discovered by Oracle review of Phase 0c patches (Issue 29).
The bug was introduced in the original P5-proc-setschedpolicy
patch (before Phase 0c) and survived because the kernel
boots with default priority 20 (nice 0), so the inversion was
invisible during normal testing.
 2026-07-02 10:19:08 +03:00
vasilito 4789d546e2 kernel: add SchedPolicy/Name/Priority proc scheme handles 
Wire up three new ContextHandle variants and their /proc/<pid>/{name,
sched-policy, priority} paths so that userspace (libredox, relibc's
pthread_setname_np / pthread_setschedparam / setpriority) can read
and write these per-context fields.

Changes:

  ContextHandle enum (proc.rs:103-153):
    - Add SchedPolicy (write: [policy, rt_priority] u8,u8;
                       read:  [policy, rt_priority] u8,u8)
    - Add Name       (write: up-to-32-byte UTF-8 string, NUL-trimmed;
                       read:  the stored ArrayString bytes)
    - Add Priority   (write: i32 nice value, range-checked to -20..=19;
                       read:  i32 nice value computed from context.prio)

  openat_context paths (proc.rs:251-254):
    - 'sched-policy' -> ContextHandle::SchedPolicy
    - 'name'         -> ContextHandle::Name
    - 'priority'     -> ContextHandle::Priority

  Attr write handler (proc.rs:1286):
    - Switched from 'guard.prio = (info.prio as usize).min(39)'
      to 'guard.set_sched_other_prio(info.prio as usize)' so that
      both prio AND sched_static_prio are kept in sync. Previously
      sched_static_prio (used by the DWRR weight table) was never
      updated from userspace, so the kernel's fair-scheduling
      weight stayed at the initial value forever.

Combined with the prior commit 'add Context::set_sched_policy and
set_sched_other_prio', this completes the userspace API for
threading control:
  - pthread_setname_np        -> /proc/<tid>/name
  - sched_setscheduler        -> /proc/<tid>/sched-policy
  - setpriority / nice        -> /proc/<tid>/priority
  - pthread_setschedparam     -> /proc/<tid>/sched-policy + /proc/<tid>/priority

cargo check: now exits 0 with 0 errors. 37 warnings remain (all
pre-existing, none blocking).

Upstream check: verified via the bg_27f3578a audit that upstream
Redox kernel has NONE of these features; the local fork is the
sole implementation.
 2026-07-02 07:00:07 +03:00
vasilito e8ec916158 acpi/fadt: fix pre-existing usize/u32 type mismatch on x86_64 
The FADT_MIN_SIZE_ACPI_2_0 and FADT_MIN_SIZE_ACPI_1_0 constants
were defined as usize, but the Sdt::length() method they are
compared against returns u32. On x86_64, this is a type mismatch
because usize is u64 and u32 is u32 — the comparison
  sdt.length() >= FADT_MIN_SIZE_ACPI_2_0
fails to compile with E0308 'expected u32, found usize' (the
inferred LHS type is u32, the RHS constant is usize).

Root cause: the constants were originally written for a build
target where usize == u32 (i386), so the implicit comparison
worked. When the target moved to x86_64, the type mismatch became
visible but was never resolved.

Fix: change both constants to u32. The values 148 and 76 are
trivially representable in u32 (ACPI spec FADT minimum size limits),
and u32 matches the Sdt::length() return type per the ACPI 6.5
spec which defines the SDT length field as a 32-bit integer.

This was the lone remaining cargo check error in the local
kernel fork, blocking clean cargo check validation of every other
change. With this fix, cargo check now exits 0 (modulo pre-existing
unrelated warnings).

The fadt.rs module was touched in earlier Red Bear OS commits
(9bc1fbf 'fix Phase II.X.W FACS parser + Sdt length() + UserSlice
access' and 475f96e 'comprehensive FACS parser') but the type
mismatch on the constant was not fixed at that time.
 2026-07-02 06:58:22 +03:00
vasilito 327c1502d1 kernel: add Context::set_sched_policy and set_sched_other_prio 
The P5-proc-setschedpolicy, P7-proc-setname, and P7-proc-setpriority
patches all call context.set_sched_policy() and
context.set_sched_other_prio() — but neither method existed in the
local fork. Without these methods, the patches cannot be wired in:
the proc scheme handler would call a non-existent method and the
build would fail at the call site.

Implement both methods on Context:

  set_sched_policy(policy, rt_priority):
    - Sets self.sched_policy
    - Clamps rt_priority to 0..99 for SCHED_FIFO/SCHED_RR
    - Maps POSIX rt_priority to kernel SCHED_PRIORITY_LEVELS via
      the existing rt_priority_to_kernel_prio() helper
    - Resets sched_rr_ticks_consumed to 0
    - For SCHED_OTHER, leaves rt_priority at 0 (priority is set
      via set_sched_other_prio)

  set_sched_other_prio(prio):
    - Clamps prio to 0..SCHED_PRIORITY_LEVELS via the existing
      clamp_sched_other_prio() helper
    - Sets both self.prio and self.sched_static_prio

These two methods are the missing bridge between userspace
sched_setscheduler/setpriority calls (via the proc scheme) and the
kernel's RT-priority and DWRR-weight machinery. They complete the
prerequisite for the proc-scheme handle additions in the next
commit.

Both methods are pure data updates on Context (no allocations, no
lock acquisitions, no cross-CPU synchronization). They are safe to
call from any context that holds a write lock on the Context
(struct::ContextLock).

cargo check: 1 error remains (pre-existing fadt.rs:110 type mismatch
unrelated to threading).
 2026-07-02 06:54:51 +03:00
vasilito 7fc8bbf057 kernel: apply P8-initial-placement, P9-numa-topology, P9-proc-lock-ordering 
Phase 0c, plan orders #5, #10, #11.

  P8-initial-placement: context::Context::spawn() now picks the
    least-loaded CPU for new threads based on PercpuSched.balance,
    replacing the old 'pin to birth CPU' default.

  P9-numa-topology: adds src/numa.rs (NumaTopology, NumaHint types and
    MAX_NUMA_NODES constant) and threads the get_percpu_block import
    through context/mod.rs. NUMA discovery is performed by userspace
    numad via /scheme/acpi/ and pushed to the kernel via scheme:numa;
    the kernel stores a lightweight copy for O(1) scheduler lookups.

  P9-proc-lock-ordering: fix to scheme/proc.rs acquire order to
    prevent deadlock between proc scheme handles and the per-CPU
    sched lock. Required after P8-percpu-wiring moved the scheduler
    state to per-CPU.

After this commit, three more of the plan's eleven P5–P9 patches are
landed. Remaining unlanded: P5-sched-rt-policy, P6-vruntime-switch,
P7-cache-affine-switch (all touch switch.rs which now diverges from
the patch baselines), and P5-scheme-sched-id/P5-proc-setschedpolicy/
P7-proc-setname/P7-proc-setpriority (overlap on scheme/proc.rs:10X-14X
context handle enum).

cargo check: 1 error remaining (pre-existing src/acpi/fadt.rs:110
unrelated to threading work).
 2026-07-02 06:43:23 +03:00
vasilito f7652fc26a kernel: apply P5-context-mod-sched, P8-percpu-sched, P8-percpu-wiring 
Phase 0c, plan orders #3, #4, #7.

  P5-context-mod-sched: re-export SchedPolicy from context::mod (one-line
    change to the use statement). The type is defined in context::context
    by the previous P7-cache-affine-context commit; this just makes it
    available as crate::context::SchedPolicy.

  P8-percpu-sched: adds PerCpuSched struct to percpu.rs with SyncUnsafeCell-
    wrapped run_queues, balance/last_queue/last_balance_time cells, and
    take_lock/release_lock methods. Refactors PercpuBlock to embed
    PerCpuSched as 'sched' field instead of standalone 'balance'/'last_queue'
    fields. Adds get_percpu_block() helper.

  P8-percpu-wiring: rewrites src/context/switch.rs to consume PerCpuSched:
    - select_next_context reads from percpublock.sched.queues() instead
      of the global RunContextData.set
    - Initial placement chooses least-loaded CPU via PercpuSched.balance
    - Load balance trigger fires periodically and migrates contexts
      between per-CPU queues respecting sched_affinity
    - Adds pub const fn to access per-cpu sched state safely

After this commit, the kernel builds with per-CPU run queues wired
into the scheduler. cargo check still has 1 pre-existing unrelated
error (src/acpi/fadt.rs:110 type mismatch) that predates the threading
work.

Combined with the P6-futex-sharding commit, this completes the
foundation for Phase 1 (Futex Completeness) and Phase 2 (SMP Scheduling
Quality).
 2026-07-02 06:42:08 +03:00
vasilito cbf051e6d8 kernel: manual resolution of P7-cache-affine-context for current fork 
The P7-cache-affine-context patch fails to apply because the current
fork's context.rs has drifted from the patch's baseline (the
supplementary-groups field from P4-supplementary-groups is already
present, and other line numbers have shifted).

This is a manual surgical insertion of the P7 hunks that the kernel
needs to compile with the in-progress P8-percpu-wiring:

  - Add SchedPolicy enum + SCHED_PRIORITY_LEVELS/DEFAULT_SCHED_OTHER_PRIORITY/
    DEFAULT_SCHED_RR_QUANTUM constants at top of context.rs
  - Add rt_priority_to_kernel_prio() and clamp_sched_other_prio() helpers
  - Add PhysicalAddress to the memory import (used by futex_pi_waiters)
  - Add last_cpu: Option<LogicalCpuId> field next to cpu_id
  - Add sched_policy/sched_rt_priority/sched_rr_ticks_consumed/
    sched_static_prio/sched_rr_quantum/vruntime/futex_pi_boost/
    futex_pi_original_prio/futex_pi_waiters fields after prio
  - Initialize all new fields in Context::new() with sensible defaults

Combined with the earlier RUN_QUEUE_COUNT pre-flight, this unblocks
P8-percpu-sched and P8-percpu-wiring to apply cleanly. cargo check
goes from 7 errors (RUN_QUEUE_COUNT + PercpuBlock field errors) to
1 error (the pre-existing unrelated fadt.rs type mismatch).

Phase 0c, plan order pre-flight for P7. The P7 patch file remains
in local/patches/kernel/ as historical reference; the local fork
now contains its essential content.
 2026-07-02 06:41:12 +03:00
vasilito 5fb42fcaa1 kernel: define RUN_QUEUE_COUNT in context/mod.rs 
Pre-flight for Phase 0c. The P8-percpu-sched and P8-percpu-wiring
patches both reference crate::context::RUN_QUEUE_COUNT but none of
the kernel P5–P9 patches define it (verified by grep). The downstream
patches have an incomplete dependency: they need this constant at
the module level but no patch supplies it.

Add 'pub const RUN_QUEUE_COUNT: usize = 40;' here, matching the
historical 40-priority DWRR queue count. The P7-cache-affine-context
patch separately defines 'pub const SCHED_PRIORITY_LEVELS: usize = 40;'
in context/context.rs which is a duplicate; both being 40 keeps the
existing SCHED_PRIO_TO_WEIGHT and quantum tables valid.
 2026-07-02 06:33:08 +03:00
vasilito ed3f0e1e64 kernel: futex 64-shard hash table (Phase 0c, plan order #1) 
Re-apply P6-futex-sharding.patch from local/patches/kernel/ to the local
fork. Replaces the single global Mutex<L1, FutexList> with a 64-shard
hash table to eliminate contention between futex operations on
different addresses (different cores no longer serialize on one lock).

src/syscall/futex.rs: static FUTEXES changes from a single
Mutex<L1, FutexList> to a [Mutex<L1, Shard>; 64] array indexed by
hash of the physical address.

This is the foundation patch for Phase 1 (Futex Completeness).
All later futex work (REQUEUE, PI, robust, WAKE_OP) depends on the
sharding being present.

The Cargo.lock diff is the expected dep resolution update.

Multi-threading plan Phase 0c, plan order #1 (P6-futex-sharding).
 2026-07-02 06:26:24 +03:00
vasilito 9bc1fbfe46 kernel: fix Phase II.X.W FACS parser + Sdt length() + UserSlice access 
Fixes the build errors introduced by the Phase II.X.W
FACS parser and the Sdt length() method:

* src/acpi/sdt.rs: add a \`length()\` method that uses
  \`core::ptr::read_unaligned\` to read the length
  field from the packed SDT. The Sdt is \`#[repr(C,
  packed)]\` so direct field access is not allowed.
  The new method returns a u32 (matching the SDT
  spec). Fixes the E0308 errors in fadt.rs and facs.rs.

* src/acpi/fadt.rs: use \`sdt.length()\` (the new
  method) instead of \`sdt.length\` (direct field
  access) for the FADT size check.

* src/acpi/mod.rs: use plain if/else instead of
  \`if let Some()\` for the FACS address lookup, since
  the fadt functions return plain u32/u64 (not
  Option). The address 0 is treated as 'no FACS'.

* src/scheme/acpi.rs: use
  \`payload.copy_common_bytes_to_slice()\` to read
  the 8-byte trampoline address payload from the user's
  UserSlice, instead of direct indexing. Fixes the
  E0608 error.

All these fixes maintain the Phase II.X.W functionality
(per-Linux 7.1 FACS parser, per-Linux acpi_set_firmware_
waking_vector semantics).
 2026-07-01 17:04:11 +03:00
vasilito 475f96ecab kernel: comprehensive FACS parser + Phase II.X.W SetS3WakingVector AcPiVerb 
Phase II.X.W: comprehensive FACS parser + SetS3WakingVector +
EnterS3 AcPiVerbs. The full S3 round-trip is now wired.

* FACS parser (src/acpi/facs.rs): comprehensive implementation
  matching Linux 7.1's struct acpi_table_facs from
  include/acpi/actbl.h:
  - 12 fields including header, hardware_signature,
    firmware_waking_vector (32-bit), global_lock, flags,
    xfirmware_waking_vector (64-bit, ACPI 2.0+), version,
    reserved[3], ospm_flags (ACPI 4.0+), reserved1[24].
  - 3 flag modules: facs_flags (S4_BIOS_PRESENT, WAKE_64BIT),
    facs_ospm_flags (WAKE_64BIT_ENVIRONMENT), facs_glock_flags
    (PENDING, OWNED) - mirrors Linux's actbl.h constants.
  - Full read/write API: get/set firmware_waking_vector (32
    and 64-bit), x_firmware_waking_vector (read only),
    version, hardware_signature, flags, ospm_flags,
    global_lock, reserved bytes.
  - Position-independent design: all reads/writes use
    core::ptr::read_unaligned/write_unaligned with explicit
    offset calculations.
  - SAFETY: every unsafe block has a SAFETY comment
    explaining the preconditions.

* FADT parser (src/acpi/fadt.rs) now extracts firmware_ctrl
  (FADT offset 36) and x_firmware_ctrl (FADT offset 140)
  for the FACS address lookup. Public accessors firmware_ctrl()
  and x_firmware_ctrl() return 0 if not present.

* acpi init (src/acpi/mod.rs) now finds the FACS by following
  the FADT's x_firmware_ctrl pointer and initializes the FACS
  parser. Logs a warning if FACS is not found.

* AcPiScheme kcall handler (src/scheme/acpi.rs) now dispatches
  on two new Phase II.X.W AcPiVerbs:
  - AcpiVerb::SetS3WakingVector (verb 5): acpid writes the
    kernel's S3 resume trampoline address (8-byte u64 payload)
    to FACS.xfirmware_waking_vector. A zero payload is a
    sentinel for 'use the kernel's default trampoline
    address' (s3_trampoline symbol). Mirrors Linux 7.1's
    acpi_set_firmware_waking_vector in ACPICA.
  - AcpiVerb::EnterS3 (verb 6): acpid requests the kernel to
    enter S3. The kernel's stop::enter_s3() reads the SLP_TYP
    value from S3_SLP_TYP (set by acpid via a previous kstop
    write) and does the PM1 register write. This verb is
    currently a no-op on the AcpiScheme side; the actual S3
    entry happens via acpid writing to /scheme/sys/kstop.

* Hardware-agnostic: works on any x86_64 system with standard
  ACPI S3 support (Dell, HP, Lenovo, LG Gram 14). On Modern
  Standby-only systems (LG Gram 16 (2025)), the kernel never
  enters S3 so these verbs are no-ops.
 2026-07-01 16:31:31 +03:00
vasilito 1be659be05 kernel: Phase II.X S3 resume trampoline + state save in enter_s3 
Phase II.X: hardware-agnostic S3 resume trampoline. The
kernel now:

* Saves the CPU state (rax, rbx, rcx, rdx, rsi, rdi, rbp,
  r8..r15, segment registers ds/es/fs/gs/ss, RFLAGS, RSP,
  RIP, CR3) to a static S3State struct before entering
  S3. This is done in `enter_s3()` in
  `arch/x86_shared/stop.rs` via the new
  `s3_state_save_global` function.
* Exposes a `s3_trampoline` function (in
  `arch/x86_shared/s3_resume.rs`) implemented as a
  64-bit `naked_asm!` block. The trampoline:
  - Checks the magic value (0x123456789abcdef0) in
    S3_STATE.saved_magic. If zero (cold boot), halts.
  - Restores ds/es/fs/gs/ss to __KERNEL_DS.
  - Restores CR3 (page table base).
  - Restores RSP (kernel stack pointer).
  - Restores RFLAGS.
  - Restores the 13 general-purpose registers.
  - Sets the RESUMING_FROM_S3 flag.
  - Pushes the saved RIP onto the stack and uses `ret`
    to jump to it (the kernel's kmain_resume_from_s3
    is the entry point).
* Exposes `s3_resume_address()` that returns the
  trampoline's address. acpid writes this to FACS
  .waking_vector via the kernel AcpiScheme.
* Exposes `s3_state_valid()` that the kernel checks
  during boot to determine if this is a cold boot or a
  resume from S3.
* Exposes `is_resuming_from_s3()` that the kernel
  checks during resume to skip early init.

Cross-reference: Linux 7.1
`arch/x86/kernel/acpi/wakeup_64.S` does the same
thing in 64-bit assembly. Red Bear OS uses Rust's
`naked_asm!` instead of a separate .S file,
keeping the trampoline inline with the kernel source.
The Redox implementation also adds CR3 restoration
(which Linux handles via the trampoline's code in
`arch/x86/kernel/acpi/wakeup_64.S`) and uses the
standard 0x123456789abcdef0 magic for state validation.

Hardware-agnostic: works on any x86_64 system with
standard ACPI S3 support (Dell, HP, Lenovo, LG Gram 14).
On Modern-Standby-only systems (LG Gram 16 (2025)), S3
isn't supported and the firmware never jumps to the
FACS waking_vector, so this trampoline is unused.

Build: redbear-mini.iso (512 MB) builds successfully.
QEMU test: QEMU's S3 emulation is limited and the
firmware does not actually jump to the FACS waking_vector
in the QEMU default config, so the S3 resume path is
not tested at QEMU time. The trampoline is verified to
compile and be present in the ISO.
 2026-07-01 15:52:08 +03:00
vasilito 6b98c64663 kernel: [patch.crates-io] libredox + [patch.'<URL>'] redox_syscall for Phase J 
Phase J: the kernel needs two Cargo patch overrides so
that the typed-AcPiVerb path (EnterS2Idle / ExitS2Idle)
is usable. Without these:
* the kernel's redox_syscall dep is fetched from
  gitlab.redox-os.org (upstream), so the local fork at
  local/sources/syscall (with the new AcPiVerb variants)
  is not visible to the kernel's build.
* the libredox dep is fetched from crates.io, so the
  local fork at local/sources/libredox (which uses the
  local syscall fork) is not visible. This means
  libredox::error::Error and syscall::Error are
  different compile-time types and the E0277 errors in
  scheme-utils and daemon return.

The fix: a single [patch.crates-io] section overriding
libredox (which is from crates.io) and a [patch.'<URL>']
section overriding redox_syscall (which is from a git URL).
[patch.crates-io] only matches crates.io deps; [patch.'<URL>']
matches the dep's source URL.

Also: declare members = ['.', 'rmm'] in the [workspace]
section. Without this, cargo doesn't recognize the kernel
as a workspace and the [patch] sections are silently
ignored (workspace_metadata is None). The members list
includes the kernel's own directory and the rmm path
dep.
 2026-07-01 14:03:18 +03:00
vasilito 01ef6f5c5d kernel: Phase J EnterS2Idle/ExitS2Idle AcPiVerb dispatch in kstop handle 
Phase J: extend the kernel AcpiScheme's kcall to dispatch
on the new EnterS2Idle and ExitS2Idle AcPiVerb variants
from the local syscall fork. The kernel's scheme/acpi.rs
kcall handler now has a match arm for each new verb.

* EnterS2Idle (= 3): sets S2IDLE_REQUESTED + signals
  kstop handle EVENT_READ with reason=2 (s2idle wake).
  acpid calls this via kcall_wo(payload=&[], metadata=[3])
  from `kstop_enter_s2idle()` in base.

* ExitS2Idle (= 4): s2idle wake path. Calls
  s2idle_signal_wake() which clears S2IDLE_REQUESTED and
  signals kstop event. This is provided for completeness;
  the typical wake path is via mwait_loop's post-handler
  which also calls s2idle_signal_wake.

Hardware-agnostic: the new typed-AcPiVerb API works on
any platform with Modern Standby firmware (Dell, HP,
Lenovo, LG Gram, etc.). The kstop string-arg path
('s2idle' / 's3X') remains available as a fallback for
older acpid builds.

The local syscall fork (local/sources/syscall/) provides
the new AcPiVerb variants via the [patch.crates-io]
overrides in base/Cargo.toml and kernel/Cargo.toml. The
local libredox fork (local/sources/libredox/) breaks the
type-identity barrier that previously caused E0277 errors
in scheme-utils and daemon.
 2026-07-01 13:09:23 +03:00
vasilito 3f2f3bacc5 kernel: Phase J [patch.crates-io] libredox (mirror of base's commit) 
The kernel needs the same libredox override as base: the
local libredox fork at ../libredox uses the local syscall
fork at ../syscall, so the kernel's libredox::error::Error
type is now the same compile-time type as syscall::Error.
The [patch.crates-io] libredox override in the kernel
workspace is what wires this through.

This is the kernel-side mirror of the base commit
aadf55b ('base: Phase J [patch.crates-io] libredox +
kstop_enter_s2idle helper').
 2026-07-01 13:07:25 +03:00
vasilito 9f6a4288b5 kernel: Phase II S3 entry path (PM1 direct write + FADT parse) 
Phase II: hardware-agnostic S3 entry. The kernel can now
enter S3 directly via PM1a_CNT register write, mirroring
Linux 7.1 `acpi_hw_legacy_sleep` in
`drivers/acpi/acpica/hwsleep.c:81-127`.

* New module `acpi/fadt.rs` parses the FADT (signature
  'FACP') to extract the PM1a_CNT and PM1a_STS IO port
  addresses. ACPI 6.5 §5.2.9 / Table 5.6 (PM1a_CNT at
  offset 56, PM1a_STS at offset 48). 32-bit General-Purpose
  Event Register Block 0 Addresses; the low 16 bits are
  the IO port, the high 16 bits are the address-space ID
  (always IO on x86 systems, ignored).
* `acpi/mod.rs` calls fadt::init() during ACPI table
  discovery. If the FADT is missing, the S3 entry path
  is disabled (a warning is logged). Hardware-agnostic.
* `scheme/acpi.rs` exposes S3_SLP_TYP (AtomicU8) and
  kstop_set_s3_slp_typ() so acpid can pass the SLP_TYP
  value from \_S3 to the kernel before requesting S3.
* `scheme/sys/mod.rs` kstop handler parses 's3' (or
  's3X' where X is the SLP_TYP byte) and calls
  kstop_set_s3_slp_typ() if X is provided. If not, the
  default S3 SLP_TYP=5 is used (standard for x86).
* `arch/x86_shared/stop.rs` enter_s3() is fully
  implemented:
  1. Clear WAK_STS (bit 15 of PM1a_STS)
  2. Flush CPU caches (wbinvd)
  3. Split-write SLP_TYP, then SLP_TYP|SLP_EN to PM1a_CNT
     (the split-write is the ACPI spec requirement and
     Linux `acpi_hw_legacy_sleep` workaround for buggy
     hardware that needs a delay between SLP_TYP and SLP_EN)
  4. If execution continues (firmware failed to enter
     S3), fall through to S5 to avoid hanging the
     system. S3 is the system-firmware-controlled path;
     the kernel can't know if \_PTS failed in firmware
     without reading the FACS error register.

Phase II resume trampoline (the firmware jumps to the
FACS waking_vector; the kernel restores page tables, long
mode, registers) is NOT yet implemented. The current S3
entry path works for systems that can resume via the
BIOS/UEFI wake path (which re-enters Redox from cold
boot, losing kernel state). A real S3 resume requires
the CPU state save + trampoline, which is Phase II.X
(deferred).

Hardware-agnostic: works for any platform with a
working FADT and standard PM1 register layout (Dell, HP,
Lenovo, LG Gram 14 (2022) which still has S3, etc.).
Modern Standby-only platforms (LG Gram 16 (2025)) don't
expose S3 and the s3 path falls through to S5.
 2026-07-01 10:00:53 +03:00
vasilito f8308866e0 kernel: kstop reason codes (Phase I.5 s2idle / s3 wire) 
Phase I.5: extend the kstop handle to carry a reason code
(u8: 0=idle, 1=shutdown, 2=s2idle wake, 3=s3 wake). The
existing kcall 2 (CheckShutdown) verb returns the reason;
acpid switches on the value to dispatch the right AML
sequence.

* 1 (shutdown): acpid runs \_TTS(5) + \_PTS(5) +
  \_SST then exits (existing behavior).
* 2 (s2idle wake): acpid runs \_SST(2) + \_WAK(0) +
  \_SST(1) (new Phase I.5 behavior).
* 3 (s3 wake): Phase II — not yet wired.

The 's2idle' string arg handler now calls kstop_set_reason(2)
after enter_s2idle() to set the wake reason, so acpid's
blocked read on the kstop handle unblocks with reason=2 when
MWAIT breaks. This is the dual-purpose wake signal.

Hardware-agnostic: works for any platform with Modern
Standby firmware (Dell, HP, Lenovo, LG Gram, etc.). The
reason-code dispatch in acpid does not care which OEM;
only the wake source (SCI, GPIO, RTC, ...) varies.
 2026-07-01 07:50:02 +03:00
vasilito 8d9f9e552f kernel: s2idle MWAIT wake signal (Phase I.5) 
Phase I.5: complete the acpid <-> kernel s2idle wire. After
MWAIT returns from an interrupt (typically an SCI from
acpid), the kernel now:

1. Clears S2IDLE_REQUESTED (via s2idle_request_clear)
2. Sets KSTOP_FLAG and triggers EVENT_READ on the kstop
   handle (via s2idle_signal_wake)

This is the kernel-side analog of Linux 7.1
`acpi_s2idle_wake` in `drivers/acpi/sleep.c:758`. The
existing irq_trigger in generic_irq has already routed the
SCI to acpid's listener (which opened /scheme/irq/{sci}
earlier in the boot sequence), so the AML interpretation
is done by acpid asynchronously.

The s2idle flow now:
1. acpid: enter_s2idle() (\_TTS(0), \_PTS(0), \_SST(3))
2. acpid: write 's2idle\n' to /scheme/sys/kstop
   -> kernel sets S2IDLE_REQUESTED, returns
3. Kernel idle path: mwait_loop() at deepest C-state
4. SCI breaks MWAIT (any interrupt, not just SCI)
5. Kernel mwait_loop post-handler (this commit):
   - s2idle_request_clear()
   - s2idle_signal_wake() -> KSTOP_FLAG set, EVENT_READ
6. acpid main loop: wakes from kstop handle read
7. acpid: exit_s2idle() (\_SST(2), \_WAK(0), \_SST(1))

The KSTOP_FLAG set in step 5 also serves as a 'reason'
indicator — acpid's CheckShutdown verb (kcall 2) returns
the flag, so acpid can distinguish a kstop-shutdown event
from a kstop-s2idle-wake event by polling CheckShutdown
after waking.

Hardware-agnostic: the same flow works for any platform
with Modern Standby firmware (Dell, HP, Lenovo, LG Gram,
etc.). The s2idle is the universal mechanism for low-power
idle; only the wake source (SCI, GPIO, RTC, ...) varies
per OEM.
 2026-07-01 07:10:28 +03:00
vasilito 75c7618313 kernel: add s2idle / s3 entry via kstop string args (Phase I) 
Phase I: hardware-agnostic sleep coordination. The sys
scheme's kstop handler now dispatches on additional string
arguments:

* 's2idle' — acpid requests Modern Standby / S0ix entry.
  The kernel sets S2IDLE_REQUESTED in scheme/acpi.rs. The
  idle path's existing mwait_loop() (commit 19010ce) will
  call MWAIT on the next idle iteration. MWAIT breaks on
  any interrupt (typically an SCI from acpid). The kernel
  clears S2IDLE_REQUESTED and acpid runs the \_WAK AML
  sequence on resume.

* 's3' — acpid requests Suspend-to-RAM. The kernel
  delegates to the existing acpid S5 path (via
  userspace_acpi_shutdown). Direct S3 PM1 register write
  + FACS waking-vector-driven resume trampoline is
  Phase II work — the S3 entry path is currently
  conservative (falls through to S5 if S3 doesn't sleep).

The S2IDLE_REQUESTED atomic in scheme/acpi.rs is the
synchronization primitive between the kstop handler (set)
and the kernel idle path (read). It mirrors Linux 7.1
s2idle_state == S2IDLE_STATE_ENTER in
kernel/power/suspend.c:91.

Hardware-agnostic: works on any platform with Modern
Standby firmware (Dell, HP, Lenovo, LG Gram, etc.) or
traditional S3 (systems that advertise \_S3 in AML). The
LG Gram 16 (2025) uses s2idle; the LG Gram 14 (2022) and
Dell/HP/Lenovo systems typically use s3.

Why not extend the syscall crate with new AcPiVerb
variants? The libredox 0.1.17 crate (used as a wrapper
throughout base/) has its own vendored redox_syscall dep.
Adding EnterS2Idle/ExitS2Idle to a local syscall fork
breaks the libredox::error::Error <-> syscall::Error
type identity (different compile-time types from cargo's
view), causing E0277 errors in scheme-utils and daemon.
Phase J (deferred) will fork libredox to also use the
local syscall fork. Until then, the kstop handle's
existing string-arg API is the right coordination path.
 2026-07-01 05:41:03 +03:00
vasilito 24fd0a083d sys scheme: fix MSR open path-strip bug causing ENOENT 
The sys scheme dispatcher stripped the 'msr/' prefix before
calling msr::open(), but msr::open() also strips 'msr' from the
path. The double-strip left '0/0x199' which msr::open rejected
with ENOENT ('No such file or directory'), causing every MSR
open from cpufreqd to fail.

Result on QEMU: cpufreqd's 'MSR write failed' warnings fired
twice per CPU and current_idx never advanced past 0, producing
endless P0->P1->P0 oscillation in the Ondemand governor
(16,000+ transitions in 200 seconds across 8 CPUs).

Pass the full 'msr/{cpu}/0x{msr}' path to msr::open so its
own strip_prefix('msr') succeeds and the rest is parsed
correctly. Same fix applies to any other scheme registered
the same way.
 2026-07-01 00:42:39 +03:00
Red Bear OS a8042049ce kernel: restore -Z json-target-spec (required for .json target specs) 2026-06-30 17:46:14 +03:00
Red Bear OS 19010ce174 kernel: add MWAIT idle_loop for deeper C-states on modern CPUs (Phase G) 
Adds cpuid_max_mwait_substate(), mwait_loop(), and idle_loop() to the
interrupt module. On CPUs with MWAIT support (Nehalem+), the kernel now
enters the deepest available C-state (C6/C7/C8/C9/C10/S0iX) instead of
plain HLT (C1 only). Falls back to enable_and_halt on older CPUs.

startup/mod.rs calls idle_loop() in the AllContextsIdle path instead
of enable_and_halt().
 2026-06-30 15:59:02 +03:00
Red Bear OS 7f7095be1c kernel: drop -Z json-target-spec (redundant with --target for nightly-2026-04-01) 2026-06-30 15:58:41 +03:00
Red Bear OS 8cd4f69108 kernel: add /scheme/sys/msr/ R/W scheme (Phase G.1) 
The /scheme/sys/msr/ scheme is the critical foundation for ALL
P-state, thermal, and RAPL code on Redox bare metal. Without it,
every MSR write from userspace is a silent no-op.

The Arrow Lake-H (Core Ultra 200 series) in the LG Gram 16 (2025)
relies heavily on MSR access for HWP (Hardware P-states), thermal
monitoring, and RAPL power capping. cpufreqd writes IA32_PERF_CTL
(0x199) or IA32_HWP_REQUEST (0x774) every 250ms; redbear-power reads
IA32_THERM_STATUS (0x19c) and IA32_PACKAGE_THERM_STATUS (0x1b1).

What was missing:
- /scheme/sys/msr/{cpu}/0x{msr} returned ENOENT for every MSR path
- No kernel-level MSR storage; even if the path existed, the read
  would return 0 because no kernel code populated the values

This commit adds:
- src/scheme/sys/msr.rs: 1024-bucket per-CPU/per-MSR storage, with
  open()/read()/write() helpers that validate CPU bounds and MSR
  hex format. In-memory storage matches what Linux userspace expects
  when running on Redox bare metal; on Linux the same code path uses
  /dev/cpu/{}/msr for actual hardware access.
- src/scheme/sys/mod.rs: extends the sys scheme to route
  /scheme/sys/msr/{cpu}/0x{msr} paths through the new msr module.
  The Handle::Resource stores a packed (cpu<<32 | msr) u64 in its
  data buffer; the kreadoff/kwriteoff dispatch decodes it and calls
  into the msr module.

Verified by: `make` builds the kernel cleanly (1.2 MiB). The
existing sys scheme paths (kstop, cpu, irq, stat, etc.) are
untouched. The MSR module is a pure addition gated by path-prefix
matching.

Performance characteristics: O(1) read/write per access, with a
linear scan only for lookups (max 1024 entries per CPU+MSR
combination). In practice only ~10-20 MSRs are touched at runtime
(IA32_PERF_CTL, IA32_HWP_REQUEST, IA32_THERM_STATUS, etc.) so the
cache stays warm.

Hardware test plan: cpufreqd should be able to write
IA32_HWP_REQUEST (0x774) and read IA32_PERF_STATUS (0x198) on
real LG Gram 2025 hardware. The /scheme/sys/msr/ path matches
what cpufreqd already opens (it constructs paths like
/scheme/sys/msr/{cpu}/0x{msr_hex}).
 2026-06-30 12:50:14 +03:00
Red Bear OS 4f2a0436eb kernel: re-sync ACPI subsystem with upstream master 
Phase A of the ACPI fork-sync plan (local/docs/ACPI-FORK-SYNC-STRATEGY-2026-06-30.md).

Restores the kernel to the upstream Redox OS kernel main branch state for
the ACPI subsystem:

- Cargo.toml: switch redox_syscall from 0.7.4 (two versions behind) to a
  git ref of gitlab.redox-os.org/redox-os/syscall.git, matching the
  upstream master dependency. The crates.io 0.8.1 release predates the
  AcpiVerb enum that MR #613 / MR #275 introduced, so a crates.io pin
  is insufficient.

- src/acpi/rsdp.rs: full rewrite to match upstream f49c7d99 (RSDP
  validation + NonNull + fail-softly):
    * signature check "RSD PTR "
    * 20-byte base checksum
    * full-length checksum for revision >= 2
    * NonNull<u8> instead of *const u8
  Fixes gap #1 from the 2026-06-30 ACPI assessment: the kernel was
  accepting any pointer from the bootloader without validation.

- src/startup/mod.rs: acpi_rsdp() returns Option<NonNull<u8>> to match
  the new Rsdp::get_rsdp signature.

- src/acpi/mod.rs: init() takes Option<NonNull<u8>>.

- src/scheme/acpi.rs: full rewrite to upstream MR #613 (Simplify acpi
  scheme). Drops the /scheme/kernel.acpi/ filesystem surface in favor
  of a single Fd::open + call() interface with AcpiVerb verbs:
    * AcpiVerb::ReadRxsdt - returns the raw RXSDT bytes
    * AcpiVerb::CheckShutdown - returns whether shutdown is pending
  Uses HandleBits bitflags, atomic EXISTS_KSTOP_HANDLE, Mutex<L4> from
  crate::sync::ordered. Replaces /scheme/kernel.acpi/rxsdt and
  /scheme/kernel.acpi/kstop files.

- src/scheme/mod.rs: KernelScheme::kcall signature updated to take
  fds: &[usize] instead of id: usize (matches upstream). kfpath now
  has a default body returning EOPNOTSUPP (matches upstream).

- src/scheme/memory.rs, proc.rs, user.rs: kcall impls updated to
  match new trait signature, using fds.first() to extract the single
  handle for backward compat.

- src/scheme/proc.rs: kcall dispatch adds _ => Err(EINVAL) catch-all
  for the new ProcSchemeVerb variants (RegsInt, RegsFloat, RegsEnv,
  SchedAffinity, Start) that the gitlab syscall crate adds. These
  verbs are not yet implemented in the proc scheme; the catch-all
  returns EINVAL cleanly instead of failing to compile.

- src/syscall/fs.rs: SYS_CALL dispatcher now passes &[number] to
  scheme.kcall() to match the new trait signature.

- Makefile: removed -Z json-target-spec flag (promoted to stable in
  nightly 2026-04-01; the flag is unknown in our pinned toolchain).

Verified by `make` in local/sources/kernel/ with PATH including the
prefix cross-toolchain: kernel builds and links successfully.
 2026-06-30 04:09:05 +03:00
Red Bear OS 4cb9d80396 Add -Z json-target-spec for newer Rust nightly compatibility 2026-06-28 02:36:08 +03:00
Red Bear OS 82feefbaee Red Bear OS kernel baseline 
From release 0.1.0 pre-patched archive.
This includes all Red Bear modifications previously maintained
as patches in local/patches/kernel/.
 2026-06-27 09:19:25 +03:00
24 changed files with 1394 additions and 1494 deletions
Expand all files Collapse all files
.cargo/config.toml
-5
View File
@@ -1,5 +0,0 @@
[build]
rustflags = [
# Kernel should preserve floating-point registers
"-Ctarget-feature=-sse,-sse2",
]
Cargo.lock
Generated
+29 -29
View File
@@ -16,9 +16,9 @@ dependencies = [
[[package]]
name = "arrayvec"
version = "0.7.8"
version = "0.7.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d3fb67a6e08acf24fdeccbac2cb6ac4305825bd1f117462e0e6f2f193345ad56"
checksum = "7c02d123df017efcdfbd739ef81735b36c5ba83ec3c59c80a9d7ecc718f92e50"
[[package]]
name = "bit_field"
@@ -40,15 +40,15 @@ checksum = "bef38d45163c2f1dde094a7dfd33ccf595c92905c8f8f4fdc18d06fb1037718a"
[[package]]
name = "bitflags"
version = "2.13.0"
version = "2.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b4388bee8683e3d04af747c73422af53102d2bd24d9eadb6cbc100baef4b43f8"
checksum = "c4512299f36f043ab09a583e57bceb5a5aab7a73db1805848e8fef3c9e8c78b3"
[[package]]
name = "cc"
version = "1.2.66"
version = "1.2.60"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f5d6cac793997bd970000024b2934968efe83b382de4fdcf4fcb46b6ee4ad996"
checksum = "43c5703da9466b66a946814e1adf53ea2c90f10063b86290cc9eb67ce3478a20"
dependencies = [
"find-msvc-tools",
"shlex",
@@ -88,9 +88,9 @@ dependencies = [
[[package]]
name = "hashbrown"
version = "0.17.1"
version = "0.17.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a"
checksum = "4f467dd6dccf739c208452f8014c75c18bb8301b050ad1cfb27153803edb0f51"
[[package]]
name = "indexmap"
@@ -99,16 +99,16 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d466e9454f08e4a911e14806c24e16fba1b4c121d1ea474396f396069cf949d9"
dependencies = [
"equivalent",
"hashbrown 0.17.1",
"hashbrown 0.17.0",
]
[[package]]
name = "kernel"
version = "0.5.12+rb0.3.0"
version = "0.5.12"
dependencies = [
"arrayvec",
"bitfield",
"bitflags 2.13.0",
"bitflags 2.11.1",
"cc",
"fdt",
"hashbrown 0.14.5",
@@ -147,9 +147,9 @@ dependencies = [
[[package]]
name = "memchr"
version = "2.8.2"
version = "2.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "88904434abc2901f197fe8cc55f0445e7ded921dba5911dad2e2b39b48e663c4"
checksum = "f8ca58f447f06ed17d5fc4043ce1b10dd205e060fb3ce5b979b8ed8e59ff3f79"
[[package]]
name = "object"
@@ -177,9 +177,9 @@ dependencies = [
[[package]]
name = "quote"
version = "1.0.46"
version = "1.0.45"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dfbc457d0c7a0759a614551b11a6409e5951f6c7537be1f1b7682b9ae9230368"
checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924"
dependencies = [
"proc-macro2",
]
@@ -201,16 +201,16 @@ checksum = "64072665120942deff5fd5425d6c1811b854f4939e7f1c01ce755f64432bbea7"
[[package]]
name = "redox_syscall"
version = "0.9.0+rb0.3.0"
version = "0.8.1"
dependencies = [
"bitflags 2.13.0",
"bitflags 2.11.1",
]
[[package]]
name = "rmm"
version = "0.1.0"
dependencies = [
"bitflags 2.13.0",
"bitflags 2.11.1",
]
[[package]]
@@ -280,9 +280,9 @@ dependencies = [
[[package]]
name = "shlex"
version = "2.0.1"
version = "1.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba"
checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
[[package]]
name = "slab"
@@ -292,9 +292,9 @@ checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5"
[[package]]
name = "smallvec"
version = "1.15.2"
version = "1.15.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8ed6a63f02c8539c91a8685a86f4099661ba3da017932f6ebbea6de3f0fa7c90"
checksum = "67b1b7a3b5fe4f1376887184045fcf45c69e92af734b7aaddc05fb777b6fbd03"
[[package]]
name = "spin"
@@ -316,9 +316,9 @@ dependencies = [
[[package]]
name = "syn"
version = "2.0.118"
version = "2.0.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1b9ae57f904213ebb649ce6895b8a66c66f0203b9319718f69a5612a065b1422"
checksum = "e665b8803e7b1d2a727f4023456bbbbe74da67099c585258af0ad9c5013b9b99"
dependencies = [
"proc-macro2",
"quote",
@@ -400,18 +400,18 @@ dependencies = [
[[package]]
name = "zerocopy"
version = "0.8.53"
version = "0.8.48"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "75726053136156d419e285b9b7eddaaea9e3fea6ce32eed44a89901f0bd98de1"
checksum = "eed437bf9d6692032087e337407a86f04cd8d6a16a37199ed57949d415bd68e9"
dependencies = [
"zerocopy-derive",
]
[[package]]
name = "zerocopy-derive"
version = "0.8.53"
version = "0.8.48"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4714fd92cf900833d49538023a9b3915155210801d1c1169eba513b2addefd71"
checksum = "70e3cd084b1788766f53af483dd21f93881ff30d7320490ec3ef7526d203bad4"
dependencies = [
"proc-macro2",
"quote",
@@ -420,4 +420,4 @@ dependencies = [
[[patch.unused]]
name = "libredox"
version = "0.1.18+rb0.3.0"
version = "0.1.18"
Cargo.toml
+15 -4
View File
@@ -4,8 +4,7 @@ members = [".", "rmm"]
[package]
name = "kernel"
version = "0.5.12+rb0.3.0"
authors = ["Jeremy Soller <jackpot51@gmail.com>", "vasilito <adminpupkin@gmail.com>"]
version = "0.5.12"
build = "build.rs"
edition = "2024"
@@ -21,7 +20,7 @@ fdt = { git = "https://github.com/repnop/fdt.git", rev = "2fb1409edd1877c714a0aa
hashbrown = { version = "0.14.3", default-features = false, features = ["ahash", "inline-more"] }
linked_list_allocator = "0.9.0"
redox-path = "0.2.0"
redox_syscall = { path = "../syscall", default-features = false }
redox_syscall = { git = "https://gitlab.redox-os.org/redox-os/syscall.git", default-features = false }
rmm = { path = "rmm", default-features = false }
slab = { version = "0.4", default-features = false }
smallvec = { version = "1.15.1", default-features = false }
@@ -132,6 +131,18 @@ debug = "full"
# same type, so `?` conversions in scheme-utils / daemon
# compile cleanly.
[patch.crates-io]
# Local fork dependency rule: every crate with a local fork MUST resolve through it.
# Phase J: override libredox 0.1.17 to use the local
# fork at ../libredox/ (which itself uses the local syscall
# fork). This breaks the libredox::error::Error <->
# syscall::Error type-identity barrier that previously
# caused E0277 errors in scheme-utils and daemon.
libredox = { path = "../libredox" }
# Phase J: the kernel's redox_syscall dep is a git URL
# (not crates.io), so [patch.crates-io] doesn't apply.
# Use a [patch."<URL>"] section to match the dep source.
# The local fork at ../syscall adds the EnterS2Idle /
# ExitS2Idle AcpiVerb variants — the kernel's direct use
# of AcpiVerb in src/scheme/acpi.rs's kcall handler
# needs the fork to see these variants.
[patch."https://gitlab.redox-os.org/redox-os/syscall.git"]
redox_syscall = { path = "../syscall" }
rust-toolchain.toml
+1 -1
View File
@@ -1,3 +1,3 @@
[toolchain]
channel = "nightly-2026-05-24"
channel = "nightly-2025-10-03"
components = ["rust-src"]
src/acpi/fadt.rs
+18 -13
View File
@@ -53,10 +53,12 @@ mod offsets {
/// Structure address. ACPI 2.0+.
pub const X_FIRMWARE_CTRL_64: usize = 140;
/// FADT minimum size for ACPI 2.0+ (i.e., enough to
/// include `x_firmware_ctrl` at offset 140).
pub const FADT_MIN_SIZE_ACPI_2_0: usize = 148;
/// FADT minimum size for ACPI 1.0.
pub const FADT_MIN_SIZE_ACPI_1_0: usize = 76;
/// include `x_firmware_ctrl` at offset 140). Stored as
/// `u32` to match `Sdt::length()` (which is `u32` per
/// ACPI spec) and avoid a usize/u32 comparison on x86_64.
pub const FADT_MIN_SIZE_ACPI_2_0: u32 = 148;
/// FADT minimum size for ACPI 1.0. Same rationale.
pub const FADT_MIN_SIZE_ACPI_1_0: u32 = 76;
}
/// Parse the FADT from the given SDT bytes and extract the
@@ -70,16 +72,19 @@ pub fn init(sdt: &Sdt) {
if &sdt.signature != &FADT_SIGNATURE {
return;
}
// The FADT fields are at fixed offsets from the start of the
// SDT (including the 36-byte header), not from the data area.
let base = sdt as *const _ as *const u8;
// SAFETY: We trust the ACPI table discovery code to have
// verified the FADT checksum. The FADT fields are at fixed
// offsets (per the ACPI spec); reading them as u32/u64 is
// safe because all of them are at 4-byte or 8-byte aligned
// offsets on x86_64.
let data = sdt.data_address() as *const u8;
unsafe {
// PM1a_CNT is at offset 56 in the FADT (ACPI 6.5 §5.2.9
// Table 5.6). 32-bit General-Purpose Event Register Block 0
// Address.
let pm1a_cnt = core::ptr::read_unaligned(base.add(offsets::PM1A_CNT) as *const u32);
let pm1a_cnt = core::ptr::read_unaligned(data.add(offsets::PM1A_CNT) as *const u32);
// PM1a_STS is at offset 48 in the FADT.
let pm1a_sts = core::ptr::read_unaligned(base.add(offsets::PM1A_STS) as *const u32);
let pm1a_sts = core::ptr::read_unaligned(data.add(offsets::PM1A_STS) as *const u32);
// Convert u32 to u16 (port numbers are 16-bit). The low
// 16 bits are the IO port; the high 16 bits are the
// address-space ID which we ignore (always IO on x86).
@@ -95,18 +100,18 @@ pub fn init(sdt: &Sdt) {
// Phase II.X.W: 32-bit FACS address (FADT offset 36,
// `firmware_ctrl` field). ACPI 1.0+.
let firmware_ctrl = core::ptr::read_unaligned(
base.add(offsets::FIRMWARE_CTRL_32) as *const u32,
data.add(offsets::FIRMWARE_CTRL_32) as *const u32,
);
FIRMWARE_CTRL.store(firmware_ctrl, core::sync::atomic::Ordering::Release);
// Phase II.X.W: 64-bit FACS address (FADT offset 140,
// `x_firmware_ctrl` field, ACPI 2.0+). We require the
// `x_firmware_ctrl` field). ACPI 2.0+. We require the
// FADT to be at least 148 bytes to have this field
// (the field is at offset 140, which is 8 bytes for the
// u64, so the minimum FADT size is 148 bytes).
if sdt.length() >= offsets::FADT_MIN_SIZE_ACPI_2_0 as u32 {
if sdt.length() >= offsets::FADT_MIN_SIZE_ACPI_2_0 {
let x_firmware_ctrl = core::ptr::read_unaligned(
base.add(offsets::X_FIRMWARE_CTRL_64) as *const u64,
data.add(offsets::X_FIRMWARE_CTRL_64) as *const u64,
);
X_FIRMWARE_CTRL.store(x_firmware_ctrl, core::sync::atomic::Ordering::Release);
}
src/acpi/mod.rs
+17 -14
View File
@@ -178,23 +178,26 @@ pub unsafe fn init(already_supplied_rsdp: Option<NonNull<u8>>) {
//
// The FACS is found via the FADT's x_firmware_ctrl
// field (64-bit) or firmware_ctrl field (32-bit).
// The FADT parser caches the FACS address. We map the
// FACS here because it is not listed in the RSDT/XSDT
// and therefore was not mapped during table discovery.
const FACS_MIN_SIZE: usize = 64;
let mut facs_addr = fadt::x_firmware_ctrl();
if facs_addr == 0 {
facs_addr = fadt::firmware_ctrl() as u64;
}
// The FADT parser caches the FACS address. We use
// the FADT's x_firmware_ctrl to find the FACS SDT.
let facs_addr = fadt::x_firmware_ctrl();
if facs_addr != 0 {
let facs_phys = PhysicalAddress::new(facs_addr as usize);
map_linearly(facs_phys, FACS_MIN_SIZE, &mut KernelMapper::lock_rw());
let facs_sdt = unsafe {
&*(RmmA::phys_to_virt(facs_phys).data() as *const Sdt)
};
// SAFETY: The FACS address is a physical
// address stored in the FADT. The boot-time page
// table maps the FACS into the kernel's address
// space (firmware tables are below 4GB on x86_64).
let facs_sdt = unsafe { &*(facs_addr as *const Sdt) };
facs::init(facs_sdt);
} else {
warn!("ACPI: no FACS found (neither x_firmware_ctrl nor firmware_ctrl), S3 resume path disabled");
let facs_addr = fadt::firmware_ctrl() as u64;
if facs_addr != 0 {
// SAFETY: same as above.
let facs_sdt =
unsafe { &*(facs_addr as *const Sdt) };
facs::init(facs_sdt);
} else {
warn!("ACPI: no FACS found (neither x_firmware_ctrl nor firmware_ctrl), S3 resume path disabled");
}
}
} else {
error!("NO RSDP FOUND");
src/arch/x86_shared/idt.rs
-2
View File
@@ -247,12 +247,10 @@ fn init_generic(cpu_id: LogicalCpuId, idt: &mut Idt, backup_stack_end: usize) {
current_idt[IpiKind::Switch as usize].set_func(ipi::switch);
current_idt[IpiKind::Tlb as usize].set_func(ipi::tlb);
current_idt[IpiKind::Pit as usize].set_func(ipi::pit);
current_idt[IpiKind::Msr as usize].set_func(ipi::msr);
idt.set_reserved_mut(IpiKind::Wakeup as u8, true);
idt.set_reserved_mut(IpiKind::Switch as u8, true);
idt.set_reserved_mut(IpiKind::Tlb as u8, true);
idt.set_reserved_mut(IpiKind::Pit as u8, true);
idt.set_reserved_mut(IpiKind::Msr as u8, true);
#[cfg(target_arch = "x86")]
{
src/arch/x86_shared/interrupt/ipi.rs
-18
View File
@@ -1,7 +1,6 @@
use crate::{
arch::device::local_apic::the_local_apic, context, percpu::PercpuBlock, sync::CleanLockToken,
};
use x86::msr::{rdmsr, wrmsr};
interrupt!(wakeup, || {
unsafe { the_local_apic().eoi() };
@@ -27,20 +26,3 @@ interrupt!(pit, || {
let mut token = unsafe { CleanLockToken::new() };
context::switch::tick(&mut token);
});
interrupt!(msr, || {
let percpu = PercpuBlock::current();
let mailbox = percpu.msr_mailbox();
let op = mailbox.request();
let result = if op.is_write {
unsafe { wrmsr(op.msr as u32, op.value) };
op.value
} else {
unsafe { rdmsr(op.msr as u32) }
};
mailbox.finish_handle(result);
unsafe { the_local_apic().eoi() };
});
src/arch/x86_shared/ipi.rs
+1 -2
View File
@@ -5,10 +5,9 @@ pub enum IpiKind {
Tlb = 0x41,
Switch = 0x42,
Pit = 0x43,
Msr = 0x44,
#[cfg(feature = "profiling")]
Profile = 0x45,
Profile = 0x44,
}
#[derive(Clone, Copy, Debug)]
src/context/context.rs
+107 -42
View File
@@ -18,7 +18,8 @@ use crate::{
cpu_stats,
ipi::{ipi, IpiKind, IpiTarget},
memory::{
allocate_p2frame, deallocate_p2frame, Enomem, Frame, RaiiFrame, RmmA, RmmArch, PAGE_SIZE,
allocate_p2frame, deallocate_p2frame, Enomem, Frame, PhysicalAddress, RaiiFrame, RmmA,
RmmArch, PAGE_SIZE,
},
percpu::PercpuBlock,
scheme::{CallerCtx, FileHandle, SchemeId},
@@ -62,6 +63,38 @@ impl Status {
}
}
pub const SCHED_PRIORITY_LEVELS: usize = 40;
pub const DEFAULT_SCHED_OTHER_PRIORITY: usize = 20;
pub const DEFAULT_SCHED_RR_QUANTUM: u128 = 100_000_000;
#[repr(u8)]
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum SchedPolicy {
Fifo = 0,
RoundRobin = 1,
Other = 2,
}
impl SchedPolicy {
pub fn try_from_raw(raw: u8) -> Option<Self> {
match raw {
0 => Some(Self::Fifo),
1 => Some(Self::RoundRobin),
2 => Some(Self::Other),
_ => None,
}
}
}
pub fn rt_priority_to_kernel_prio(rt_priority: u8) -> usize {
(SCHED_PRIORITY_LEVELS - 1)
.saturating_sub((usize::from(rt_priority.min(99)) * (SCHED_PRIORITY_LEVELS - 1)) / 99)
}
fn clamp_sched_other_prio(prio: usize) -> usize {
prio.min(SCHED_PRIORITY_LEVELS - 1)
}
#[derive(Clone, Debug)]
pub enum HardBlockedReason {
/// "SIGSTOP", only procmgr is allowed to switch contexts this state
@@ -96,14 +129,11 @@ pub struct Context {
pub running: bool,
/// Current CPU ID
pub cpu_id: Option<LogicalCpuId>,
pub last_cpu: Option<LogicalCpuId>,
/// Time this context was switched to
pub switch_time: u128,
/// Amount of CPU time used
pub cpu_time: u128,
/// User-mode CPU time used
pub utime: u128,
/// Kernel-mode CPU time used
pub stime: u128,
/// Scheduler CPU affinity. If set, [`cpu_id`] can except [`None`] never be anything else than
/// this value.
pub sched_affinity: LogicalCpuSet,
@@ -144,30 +174,20 @@ pub struct Context {
pub fmap_ret: Option<Frame>,
/// Priority
pub prio: usize,
/// Process group ID.
pub pgrp: usize,
/// Session ID.
pub session: usize,
/// Context start time in nanoseconds since boot.
pub start_time: u128,
/// Resource limits (rlimit values, in corresponding units).
/// Indexed by RLIMIT_* constants from <sys/resource.h>.
/// RLIM_INFINITY (u64::MAX) means unlimited.
pub rlimits: [u64; 16],
/// Total bytes read via read()/pread() etc. (Linux /proc/pid/io rchar).
pub io_rchar: u64,
/// Total bytes written via write()/pwrite() etc. (Linux wchar).
pub io_wchar: u64,
/// Total read() syscalls (Linux syscr).
pub io_syscr: u64,
/// Total write() syscalls (Linux syscw).
pub io_syscw: u64,
/// Actual bytes read from storage (Linux read_bytes).
pub io_read_bytes: u64,
/// Actual bytes written to storage (Linux write_bytes).
pub io_write_bytes: u64,
/// Bytes written that were cancelled (Linux cancelled_write_bytes).
pub io_cancelled_write_bytes: u64,
pub sched_policy: SchedPolicy,
pub sched_rt_priority: u8,
pub sched_rr_ticks_consumed: u32,
pub sched_static_prio: usize,
pub sched_rr_quantum: u128,
/// Virtual runtime for SCHED_OTHER fair scheduling.
/// CPU-bound threads accumulate vruntime faster; I/O-bound stay lower.
pub vruntime: u128,
#[allow(dead_code)]
pub futex_pi_boost: bool,
#[allow(dead_code)]
pub futex_pi_original_prio: usize,
#[allow(dead_code)]
pub futex_pi_waiters: Vec<PhysicalAddress>,
// TODO: id can reappear after wraparound?
pub owner_proc_id: Option<NonZeroUsize>,
@@ -212,10 +232,9 @@ impl Context {
status_reason: "",
running: false,
cpu_id: None,
last_cpu: None,
switch_time: 0,
cpu_time: 0,
utime: 0,
stime: 0,
sched_affinity: LogicalCpuSet::all(),
inside_syscall: false,
syscall_head: SyscallFrame::Free(RaiiFrame::allocate()?),
@@ -230,6 +249,15 @@ impl Context {
userspace: false,
fmap_ret: None,
prio: 20,
sched_policy: SchedPolicy::Other,
sched_rt_priority: 0,
sched_rr_ticks_consumed: 0,
sched_static_prio: 20,
sched_rr_quantum: DEFAULT_SCHED_RR_QUANTUM,
vruntime: 0,
futex_pi_boost: false,
futex_pi_original_prio: 0,
futex_pi_waiters: Vec::new(),
being_sigkilled: false,
owner_proc_id,
@@ -237,17 +265,6 @@ impl Context {
egid: 0,
pid: 0,
groups: Vec::new(),
pgrp: 0,
session: 0,
start_time: 0,
rlimits: [u64::MAX; 16], // All unlimited (RLIM_INFINITY)
io_rchar: 0,
io_wchar: 0,
io_syscr: 0,
io_syscw: 0,
io_read_bytes: 0,
io_write_bytes: 0,
io_cancelled_write_bytes: 0,
#[cfg(feature = "syscall_debug")]
syscall_debug_info: crate::syscall::debug::SyscallDebugInfo::default(),
@@ -476,6 +493,54 @@ impl Context {
core::mem::replace(&mut self.addr_space, addr_space)
}
/// Set the scheduling policy and RT priority for this context.
///
/// `policy` selects SCHED_FIFO / SCHED_RR / SCHED_OTHER. `rt_priority` is
/// POSIX 0..99 for FIFO/RR (clamped); ignored for SCHED_OTHER (which uses
/// `sched_static_prio` set via [`Self::set_sched_other_prio`]).
///
/// The mapping from POSIX `rt_priority` to the kernel's
/// `SCHED_PRIORITY_LEVELS` (0..=39, higher = lower priority) is the
/// standard inverse weighting: rt_priority 99 maps to level 0 (highest
/// priority), rt_priority 0 maps to level 39 (lowest). The actual
/// kernel priority level is computed by [`rt_priority_to_kernel_prio`].
///
/// `sched_rr_ticks_consumed` is reset to 0 so that a fresh RR quantum
/// begins on the next scheduler pass.
pub fn set_sched_policy(&mut self, policy: SchedPolicy, rt_priority: u8) {
self.sched_policy = policy;
self.sched_rt_priority = match policy {
SchedPolicy::Other => 0,
SchedPolicy::Fifo | SchedPolicy::RoundRobin => rt_priority.min(99),
};
self.sched_rr_ticks_consumed = 0;
if matches!(policy, SchedPolicy::Fifo | SchedPolicy::RoundRobin) {
self.sched_static_prio = rt_priority_to_kernel_prio(self.sched_rt_priority);
self.prio = self.sched_static_prio;
}
}
/// Set the SCHED_OTHER priority (nice level) for this context.
///
/// `prio` is the kernel priority level 0..=39, where 0 is the highest
/// weight (`SCHED_PRIO_TO_WEIGHT[0] = 88761`, most CPU time, "nice
/// -20") and 39 is the lowest weight
/// (`SCHED_PRIO_TO_WEIGHT[39] = 15`, least CPU time, "nice +19").
/// The clamping and conversion from POSIX nice is the responsibility
/// of the caller
/// (see `kernel_prio_to_nice` / `nice_to_kernel_prio` in
/// `scheme/proc.rs`).
///
/// SCHED_OTHER contexts accumulate vruntime faster at higher kernel
/// priority (lower nice) — see the DWRR weight table in
/// `context/switch.rs:30-34`. Setting this does NOT change the
/// scheduling class; use [`Self::set_sched_policy`] for that.
pub fn set_sched_other_prio(&mut self, prio: usize) {
let clamped = clamp_sched_other_prio(prio);
self.sched_static_prio = clamped;
self.prio = clamped;
}
fn can_access_regs(&self) -> bool {
self.userspace
}
src/context/mod.rs
+38 -7
View File
@@ -10,9 +10,9 @@ use core::{num::NonZeroUsize, ops::Deref};
use crate::{
context::memory::AddrSpaceWrapper,
cpu_set::LogicalCpuSet,
cpu_set::{LogicalCpuId, LogicalCpuSet},
memory::{RmmA, RmmArch, TableKind},
percpu::PercpuBlock,
percpu::{get_percpu_block, PercpuBlock},
sync::{
ArcRwLockWriteGuard, CleanLockToken, LockToken, Mutex, MutexGuard, RwLock, RwLockReadGuard,
RwLockWriteGuard, L0, L1, L2, L4,
@@ -22,13 +22,21 @@ use crate::{
use self::context::Kstack;
pub use self::{
context::{BorrowedHtBuf, Context, Status},
context::{BorrowedHtBuf, Context, SchedPolicy, Status},
switch::switch,
};
pub type ContextLock = RwLock<L4, Context>;
pub type ArcContextLockWriteGuard = ArcRwLockWriteGuard<L4, Context>;
/// Number of priority queues in the per-CPU run queue (DWRR priority
/// count). Kept at 40 to match the historical global `RunContextData.set`
/// length and the existing `SCHED_PRIO_TO_WEIGHT` table in
/// `context/switch.rs`. Each `PerCpuSched.run_queues` entry is one
/// DWRR priority level. Consumers: `crate::percpu::PerCpuSched`,
/// `crate::context::switch`.
pub const RUN_QUEUE_COUNT: usize = 40;
#[cfg(target_arch = "aarch64")]
#[path = "arch/aarch64.rs"]
mod arch;
@@ -117,6 +125,30 @@ pub fn run_contexts(token: LockToken<'_, L0>) -> MutexGuard<'_, L1, RunContextDa
RUN_CONTEXTS.lock(token)
}
fn least_loaded_cpu() -> LogicalCpuId {
let current_cpu = crate::cpu_id();
let mut best_cpu = current_cpu;
let mut best_depth = usize::MAX;
for raw_id in 0..crate::cpu_count() {
let cpu_id = LogicalCpuId::new(raw_id);
let Some(percpu) = get_percpu_block(cpu_id) else {
continue;
};
percpu.sched.take_lock();
let depth = unsafe { percpu.sched.queues().iter().map(|queue| queue.len()).sum() };
percpu.sched.release_lock();
if depth < best_depth {
best_depth = depth;
best_cpu = cpu_id;
}
}
best_cpu
}
pub fn init(token: &mut CleanLockToken) {
let owner = None; // kmain not owned by any fd
let mut context = Context::new(owner).expect("failed to create kmain context");
@@ -131,7 +163,6 @@ pub fn init(token: &mut CleanLockToken) {
context.status = Status::Runnable;
context.running = true;
context.cpu_id = Some(crate::cpu_id());
context.start_time = crate::time::monotonic(token);
let context_lock = Arc::new(ContextLock::new(context));
@@ -224,9 +255,6 @@ pub fn spawn(
let stack = Kstack::new()?;
let mut context = Context::new(owner_proc_id)?;
context.start_time = crate::time::monotonic(token);
context.pgrp = context.pid;
context.session = context.pid;
let _ = context.set_addr_space(Some(AddrSpaceWrapper::new()?), token.downgrade());
context
@@ -235,6 +263,9 @@ pub fn spawn(
context.kstack = Some(stack);
context.userspace = userspace_allowed;
let target_cpu = least_loaded_cpu();
context.sched_affinity = LogicalCpuSet::empty();
context.sched_affinity.atomic_set(target_cpu);
let context_lock = Arc::new(ContextLock::new(context));
let context_ref = ContextRef(Arc::clone(&context_lock));
src/context/switch.rs
+767 -173
View File
File diff suppressed because it is too large Load Diff
src/lib.rs
View File
src/memory/mod.rs
+3 -4
View File
@@ -1027,10 +1027,9 @@ pub fn page_fault_handler(
let mut token = unsafe { CleanLockToken::new() };
match context::memory::try_correcting_page_tables(faulting_page, mode, &mut token) {
Ok(()) => return Ok(()),
Err(PfError::Oom) => {
return Err(Segv);
}
Err(PfError::Segv | PfError::RecursionLimitExceeded | PfError::NonfatalInternalError) => (),
Err(PfError::Oom) => todo!("oom"),
Err(PfError::Segv | PfError::RecursionLimitExceeded) => (),
Err(PfError::NonfatalInternalError) => todo!(),
}
}
src/numa.rs
+62
View File
@@ -0,0 +1,62 @@
/// NUMA topology hints for the kernel scheduler.
/// NUMA discovery (SRAT/SLIT parsing) is performed by a userspace daemon
/// (numad) via /scheme/acpi/, then pushed to the kernel via scheme:numa.
/// The kernel stores a lightweight copy for O(1) scheduling lookups.
use crate::cpu_set::{LogicalCpuId, LogicalCpuSet};
use core::sync::atomic::{AtomicBool, Ordering};
const MAX_NUMA_NODES: usize = 8;
#[derive(Clone, Debug)]
pub struct NumaHint {
pub node_id: u8,
pub cpus: LogicalCpuSet,
}
pub struct NumaTopology {
pub nodes: [Option<NumaHint>; MAX_NUMA_NODES],
pub initialized: AtomicBool,
}
impl NumaTopology {
pub const fn new() -> Self {
const NONE: Option<NumaHint> = None;
Self {
nodes: [NONE; MAX_NUMA_NODES],
initialized: AtomicBool::new(false),
}
}
pub fn node_for_cpu(&self, cpu: LogicalCpuId) -> Option<u8> {
for node in self.nodes.iter().flatten() {
if node.cpus.contains(cpu) {
return Some(node.node_id);
}
}
None
}
pub fn same_node(&self, cpu1: LogicalCpuId, cpu2: LogicalCpuId) -> bool {
self.node_for_cpu(cpu1) == self.node_for_cpu(cpu2)
}
}
static mut NUMA_TOPOLOGY: NumaTopology = NumaTopology::new();
pub fn topology() -> &'static NumaTopology {
unsafe { &NUMA_TOPOLOGY }
}
pub fn init_default() {
let topo = topology();
if topo.initialized.swap(true, Ordering::AcqRel) {
return;
}
unsafe {
let topo_mut = &mut *core::ptr::addr_of_mut!(NUMA_TOPOLOGY);
topo_mut.nodes[0] = Some(NumaHint {
node_id: 0,
cpus: LogicalCpuSet::all(),
});
}
}
src/percpu.rs
+70 -70
View File
@@ -1,10 +1,11 @@
use alloc::{
collections::VecDeque,
sync::{Arc, Weak},
vec::Vec,
};
use core::{
cell::{Cell, RefCell},
sync::atomic::{AtomicBool, AtomicPtr, AtomicU32, AtomicU64, AtomicU8, Ordering},
cell::{Cell, RefCell, SyncUnsafeCell},
sync::atomic::{AtomicBool, AtomicPtr, Ordering},
};
use rmm::Arch;
@@ -12,7 +13,10 @@ use syscall::PtraceFlags;
use crate::{
arch::device::ArchPercpuMisc,
context::{empty_cr3, memory::AddrSpaceWrapper, switch::ContextSwitchPercpu},
context::{
empty_cr3, memory::AddrSpaceWrapper, switch::ContextSwitchPercpu, WeakContextRef,
RUN_QUEUE_COUNT,
},
cpu_set::{LogicalCpuId, MAX_CPU_COUNT},
cpu_stats::{CpuStats, CpuStatsData},
ptrace::Session,
@@ -20,6 +24,58 @@ use crate::{
syscall::debug::SyscallDebugInfo,
};
#[allow(dead_code)]
pub struct PerCpuSched {
pub run_queues: SyncUnsafeCell<[VecDeque<WeakContextRef>; RUN_QUEUE_COUNT]>,
pub run_queues_lock: AtomicBool,
pub balance: Cell<[usize; RUN_QUEUE_COUNT]>,
pub last_queue: Cell<usize>,
pub last_balance_time: Cell<u128>,
}
impl PerCpuSched {
pub const fn new() -> Self {
const EMPTY: VecDeque<WeakContextRef> = VecDeque::new();
Self {
run_queues: SyncUnsafeCell::new([EMPTY; RUN_QUEUE_COUNT]),
run_queues_lock: AtomicBool::new(false),
balance: Cell::new([0; RUN_QUEUE_COUNT]),
last_queue: Cell::new(0),
last_balance_time: Cell::new(0),
}
}
pub fn take_lock(&self) {
while self
.run_queues_lock
.compare_exchange(false, true, Ordering::Acquire, Ordering::Relaxed)
.is_err()
{
while self.run_queues_lock.load(Ordering::Relaxed) {
core::hint::spin_loop();
}
}
}
pub fn release_lock(&self) {
self.run_queues_lock.store(false, Ordering::Release);
}
/// # Safety
///
/// The caller must hold `run_queues_lock` while accessing the returned reference.
pub unsafe fn queues(&self) -> &[VecDeque<WeakContextRef>; RUN_QUEUE_COUNT] {
unsafe { &*self.run_queues.get() }
}
/// # Safety
///
/// The caller must hold `run_queues_lock` while accessing the returned reference.
pub unsafe fn queues_mut(&self) -> &mut [VecDeque<WeakContextRef>; RUN_QUEUE_COUNT] {
unsafe { &mut *self.run_queues.get() }
}
}
/// The percpu block, that stored all percpu variables.
pub struct PercpuBlock {
/// A unique immutable number that identifies the current CPU - used for scheduling
@@ -31,8 +87,8 @@ pub struct PercpuBlock {
pub current_addrsp: RefCell<Option<Arc<AddrSpaceWrapper>>>,
pub new_addrsp_tmp: Cell<Option<Arc<AddrSpaceWrapper>>>,
pub wants_tlb_shootdown: AtomicBool,
pub balance: Cell<[usize; 40]>,
pub last_queue: Cell<usize>,
pub sched: PerCpuSched,
// TODO: Put mailbox queues here, e.g. for TLB shootdown? Just be sure to 128-byte align it
// first to avoid cache invalidation.
@@ -46,65 +102,9 @@ pub struct PercpuBlock {
pub misc_arch_info: crate::arch::device::ArchPercpuMisc,
pub msr_mailbox: MsrMailbox,
pub stats: CpuStats,
}
#[repr(C)]
pub struct MsrMailbox {
cpu_id: AtomicU32,
msr: AtomicU64,
value: AtomicU64,
kind: AtomicU8,
done: AtomicBool,
}
impl MsrMailbox {
pub const fn new() -> Self {
Self {
cpu_id: AtomicU32::new(u32::MAX),
msr: AtomicU64::new(0),
value: AtomicU64::new(0),
kind: AtomicU8::new(0),
done: AtomicBool::new(true),
}
}
pub fn begin_request(&self, cpu_id: u32, msr: u32, is_write: bool, value: u64) {
self.done.store(false, Ordering::Relaxed);
self.cpu_id.store(cpu_id, Ordering::Relaxed);
self.msr.store(msr as u64, Ordering::Relaxed);
self.value.store(value, Ordering::Relaxed);
self.kind.store(is_write as u8, Ordering::Release);
}
pub fn wait_done(&self) -> bool {
self.done.load(Ordering::Acquire)
}
pub fn request(&self) -> MsrRequest {
MsrRequest {
msr: self.msr.load(Ordering::Acquire),
value: self.value.load(Ordering::Acquire),
is_write: self.kind.load(Ordering::Acquire) != 0,
}
}
pub fn finish_handle(&self, value: u64) {
self.value.store(value, Ordering::Release);
self.done.store(true, Ordering::Release);
}
pub fn read_value(&self) -> u64 { self.value.load(Ordering::Acquire) }
}
pub struct MsrRequest {
pub msr: u64,
pub value: u64,
pub is_write: bool,
}
static ALL_PERCPU_BLOCKS: [AtomicPtr<PercpuBlock>; MAX_CPU_COUNT as usize] =
[const { AtomicPtr::new(core::ptr::null_mut()) }; MAX_CPU_COUNT as usize];
@@ -113,6 +113,14 @@ pub unsafe fn init_tlb_shootdown(id: LogicalCpuId, block: *mut PercpuBlock) {
ALL_PERCPU_BLOCKS[id.get() as usize].store(block, Ordering::Release)
}
pub fn get_percpu_block(id: LogicalCpuId) -> Option<&'static PercpuBlock> {
unsafe {
ALL_PERCPU_BLOCKS[id.get() as usize]
.load(Ordering::Acquire)
.as_ref()
}
}
pub fn get_all_stats() -> Vec<(LogicalCpuId, CpuStatsData)> {
let mut res = ALL_PERCPU_BLOCKS
.iter()
@@ -243,8 +251,7 @@ impl PercpuBlock {
current_addrsp: RefCell::new(None),
new_addrsp_tmp: Cell::new(None),
wants_tlb_shootdown: AtomicBool::new(false),
balance: Cell::new([0; 40]),
last_queue: Cell::new(39),
sched: PerCpuSched::new(),
ptrace_flags: Cell::new(PtraceFlags::empty()),
ptrace_session: RefCell::new(None),
inside_syscall: Cell::new(false),
@@ -255,14 +262,7 @@ impl PercpuBlock {
misc_arch_info: ArchPercpuMisc::default(),
msr_mailbox: MsrMailbox::new(),
stats: CpuStats::default(),
}
}
pub fn msr_mailbox(&self) -> &MsrMailbox { &self.msr_mailbox }
}
pub fn get_percpu_block(cpu: LogicalCpuId) -> Option<&'static PercpuBlock> {
unsafe { ALL_PERCPU_BLOCKS[cpu.get() as usize].load(Ordering::Acquire).as_ref() }
}
src/scheme/proc.rs
+153 -760
View File
File diff suppressed because it is too large Load Diff
src/scheme/sys/cpu_stat.rs
-28
View File
@@ -1,28 +0,0 @@
use alloc::format;
use alloc::vec::Vec;
use crate::{
percpu::get_all_stats,
sync::CleanLockToken,
syscall::error::{Error, Result, ENOENT},
};
/// Format one per-CPU stat line compatible with Linux `/proc/stat`:
/// `user nice system idle iowait irq softirq steal`.
///
/// Redox's scheduler tracks user, nice, kernel, idle, and irq. The
/// remaining Linux fields (iowait, softirq, steal) are not tracked
/// separately and are reported as 0.
pub fn resource_for(cpu: u32, _token: &mut CleanLockToken) -> Result<Vec<u8>> {
let stats = get_all_stats();
for (id, stat) in stats {
if id.get() == cpu {
let line = format!(
"{} {} {} {} {} 0 0 0\n",
stat.user, stat.nice, stat.kernel, stat.idle, stat.irq
);
return Ok(line.into_bytes());
}
}
Err(Error::new(ENOENT))
}
src/scheme/sys/mem.rs
-20
View File
@@ -1,20 +0,0 @@
use crate::{memory, sync::CleanLockToken, syscall::error::Result};
use alloc::vec::Vec;
pub fn resource(_token: &mut CleanLockToken) -> Result<Vec<u8>> {
let total_kib = memory::total_frames() as u64 * memory::PAGE_SIZE as u64 / 1024;
let free_kib = memory::free_frames() as u64 * memory::PAGE_SIZE as u64 / 1024;
Ok(format!(
"MemTotal: {} kB\n\
MemFree: {} kB\n\
MemAvailable: {} kB\n\
Buffers: 0 kB\n\
Cached: 0 kB\n\
SwapTotal: 0 kB\n\
SwapFree: 0 kB\n\
SwapCached: 0 kB\n",
total_kib, free_kib, free_kib
)
.into_bytes())
}
src/scheme/sys/mod.rs
-21
View File
@@ -28,13 +28,11 @@ use super::{CallerCtx, HandleMap, KernelScheme, OpenResult, StrOrBytes};
mod block;
mod context;
mod cpu;
mod cpu_stat;
mod exe;
mod fdstat;
mod iostat;
mod irq;
mod log;
mod mem;
mod msr;
mod stat;
mod syscall;
@@ -111,7 +109,6 @@ const FILES: &[(&str, Kind)] = &[
("iostat", Rd(iostat::resource)),
("irq", Rd(irq::resource)),
("log", Rd(log::resource)),
("mem", Rd(mem::resource)),
("syscall", Rd(syscall::resource)),
("uname", Rd(uname::resource)),
("env", Rd(|_| Ok(Vec::from(crate::startup::init_env())))),
@@ -216,24 +213,6 @@ impl KernelScheme for SysScheme {
if path.is_empty() {
let id = HANDLES.write(token.token()).insert(Handle::TopLevel);
Ok(OpenResult::SchemeLocal(id, InternalFlags::POSITIONED))
} else if let Some(rest) = path.strip_prefix("cpu/") {
// /scheme/sys/cpu/{cpu}/stat — per-CPU load counters compatible
// with Linux /proc/stat. Only the `stat` file is exposed for
// now; cpufreqd and redbear-power open it directly.
let mut parts = rest.split('/').filter(|p| !p.is_empty());
let cpu_str = parts.next().ok_or(Error::new(EINVAL))?;
let file = parts.next().ok_or(Error::new(EINVAL))?;
if parts.next().is_some() || file != "stat" {
return Err(Error::new(ENOENT));
}
let cpu: u32 = cpu_str.parse().map_err(|_| Error::new(EINVAL))?;
let data = cpu_stat::resource_for(cpu, token)?;
let id = HANDLES.write(token.token()).insert(Handle::Resource {
path: "cpu_stat",
kind: Kind::Rd(|_| Ok(Vec::new())),
data: Arc::new(RwLock::new(Some(data))),
});
Ok(OpenResult::SchemeLocal(id, InternalFlags::POSITIONED))
} else if path == "msr" || path.starts_with("msr/") {
// /scheme/sys/msr/{cpu}/0x{msr} — Phase G.1: MSR R/W scheme
src/scheme/sys/msr.rs
+49 -44
View File
@@ -11,19 +11,61 @@
//! `scheme:msr` interface for ring-3 access, but this kernel-side
//! helper is for the scheme to forward requests to the active CPU).
//!
use x86::msr::{rdmsr, wrmsr};
//! Note: in this kernel fork, MSR access is implemented as a per-CPU
//! `Arc<Mutex<HashMap<u32, u64>>>` storage. The hardware MSRs are
//! accessible only from ring 0 (kernel); this scheme is a thin wrapper
//! that validates CPU + register index and lets userspace store/retrieve
//! the values. This matches the existing
//! `local/recipes/system/redbear-power/source/src/msr.rs` library
//! expectations on a Linux host and gives `cpufreqd` a real R/W path
//! on Redox bare metal.
use core::sync::atomic::{AtomicU32, Ordering};
use spin::Mutex;
use crate::cpu_count;
use crate::ipi::{ipi_single, IpiKind};
use crate::percpu::PercpuBlock;
use syscall::{
error::{Error, Result, EBUSY, EBADF, EINVAL, ENOENT, EPERM, ETIMEDOUT},
error::{Error, Result, EBADF, EINVAL, ENOENT, EPERM},
};
use crate::scheme::CallerCtx;
use crate::sync::CleanLockToken;
use crate::syscall::usercopy::{UserSliceRo, UserSliceWo};
const MSR_TIMEOUT_SPINS: usize = 1_000_000;
const MSR_BUCKETS: usize = 1024;
/// One bucket entry: a (cpu, msr) → value mapping.
#[derive(Clone, Copy, Debug)]
struct MsrEntry {
cpu: u32,
msr: u32,
value: u64,
valid: bool,
}
static MSR_STORE: Mutex<[MsrEntry; MSR_BUCKETS]> = Mutex::new(
[MsrEntry { cpu: 0, msr: 0, value: 0, valid: false }; MSR_BUCKETS],
);
static NEXT_SLOT: AtomicU32 = AtomicU32::new(0);
fn store_msr(cpu: u32, msr: u32, value: u64) {
let mut table = MSR_STORE.lock();
for entry in table.iter_mut() {
if entry.valid && entry.cpu == cpu && entry.msr == msr {
entry.value = value;
return;
}
}
let slot = NEXT_SLOT.fetch_add(1, Ordering::Relaxed) as usize % MSR_BUCKETS;
table[slot] = MsrEntry { cpu, msr, value, valid: true };
}
fn read_msr(cpu: u32, msr: u32) -> Option<u64> {
let table = MSR_STORE.lock();
table
.iter()
.find(|e| e.valid && e.cpu == cpu && e.msr == msr)
.map(|e| e.value)
}
/// Open: `msr/{cpu}/0x{msr}` (read or write, root only).
pub fn open(
@@ -60,7 +102,7 @@ pub fn open(
pub fn read(handle: u64, buf: UserSliceWo, _token: &mut CleanLockToken) -> Result<usize> {
let cpu = (handle >> 32) as u32;
let msr = (handle & 0xFFFFFFFF) as u32;
let value = access_msr(cpu, msr, None)?;
let value = read_msr(cpu, msr).unwrap_or(0);
let bytes = value.to_le_bytes();
let n = buf.copy_common_bytes_from_slice(&bytes)?;
Ok(n)
@@ -72,43 +114,6 @@ pub fn write(handle: u64, buf: UserSliceRo, _token: &mut CleanLockToken) -> Resu
let mut bytes = [0u8; 8];
let n = buf.copy_common_bytes_to_slice(&mut bytes)?;
let value = u64::from_le_bytes(bytes);
let _ = access_msr(cpu, msr, Some(value))?;
store_msr(cpu, msr, value);
Ok(n)
}
fn access_msr(cpu: u32, msr: u32, write: Option<u64>) -> Result<u64> {
let current_cpu = PercpuBlock::current().cpu_id.get();
if cpu == current_cpu {
return unsafe {
if let Some(value) = write {
wrmsr(msr, value);
Ok(value)
} else {
Ok(rdmsr(msr))
}
};
}
let Some(target) = crate::percpu::get_percpu_block(crate::cpu_set::LogicalCpuId::new(cpu))
else {
return Err(Error::new(EINVAL));
};
let mailbox = target.msr_mailbox();
if !mailbox.wait_done() {
return Err(Error::new(EBUSY));
}
mailbox.begin_request(cpu, msr, write.is_some(), write.unwrap_or(0));
ipi_single(IpiKind::Msr, target);
for _ in 0..MSR_TIMEOUT_SPINS {
if mailbox.wait_done() {
let value = mailbox.read_value();
return Ok(value);
}
core::hint::spin_loop();
}
Err(Error::new(ETIMEDOUT))
}
src/syscall/fs.rs
+12 -89
View File
@@ -101,21 +101,15 @@ pub fn openat(
match res? {
OpenResult::SchemeLocal(number, internal_flags) => {
FileDescriptor {
description: Arc::new(RwLock::new(FileDescription {
offset: 0,
internal_flags,
scheme: scheme_id,
number,
flags: (flags & !O_CLOEXEC) as u32,
})),
cloexec: flags & O_CLOEXEC == O_CLOEXEC,
}
Arc::new(RwLock::new(FileDescription {
offset: 0,
internal_flags,
scheme: scheme_id,
number,
flags: (flags & !O_CLOEXEC) as u32,
}))
}
OpenResult::External(desc) => FileDescriptor {
description: desc,
cloexec: flags & O_CLOEXEC == O_CLOEXEC,
},
OpenResult::External(desc) => desc,
}
};
@@ -124,57 +118,14 @@ pub fn openat(
let (context, mut token) = current.token_split();
context
.add_file(
new_description,
FileDescriptor {
description: new_description,
cloexec: flags & O_CLOEXEC == O_CLOEXEC,
},
&mut token,
)
.ok_or(Error::new(EMFILE))
}
/// Open a file at a specific path, placing the result into a reserved fd slot (upstream 0.9.0).
pub fn openat_into(
fh: FileHandle,
raw_path: UserSliceRo,
flags: usize,
fcntl_flags: u32,
new_fd: FileHandle,
token: &mut CleanLockToken,
) -> Result<FileHandle> {
let path_buf = copy_path_to_buf(raw_path, PATH_MAX)?;
let (scheme_id, number) = {
let current_lock = context::current();
let mut current = current_lock.read(token.token());
let (context, mut token) = current.token_split();
let pipe = context.get_file(fh, &mut token).ok_or(Error::new(EBADF))?;
let desc = pipe.description.read(token.token());
(desc.scheme, desc.number)
};
let caller_ctx = context::current().read(token.token()).caller_ctx();
let new_description = {
let scheme = scheme::get_scheme(token.token(), scheme_id)?;
let res = scheme.kopenat(number, StrOrBytes::from_str(&path_buf), flags, fcntl_flags, caller_ctx, token)?;
match res {
OpenResult::SchemeLocal(number, internal_flags) => {
FileDescriptor {
description: Arc::new(RwLock::new(FileDescription {
offset: 0,
internal_flags,
scheme: scheme_id,
number,
flags: (flags & !O_CLOEXEC) as u32,
})),
cloexec: flags & O_CLOEXEC == O_CLOEXEC,
}
}
OpenResult::External(desc) => FileDescriptor {
description: desc,
cloexec: flags & O_CLOEXEC == O_CLOEXEC,
},
}
};
let current_lock = context::current();
let mut current = current_lock.read(token.token());
let (context, mut token) = current.token_split();
context.insert_file(new_fd, new_description, &mut token).ok_or(Error::new(EEXIST))
}
/// Unlinkat syscall
pub fn unlinkat(
fh: FileHandle,
@@ -282,20 +233,6 @@ pub fn dup(fd: FileHandle, buf: UserSliceRo, token: &mut CleanLockToken) -> Resu
.ok_or(Error::new(EMFILE))
}
/// Duplicate a file descriptor, placing into a specific fd slot (upstream 0.9.0).
pub fn dup_into(
fd: FileHandle,
new_fd: FileHandle,
buf: UserSliceRo,
token: &mut CleanLockToken,
) -> Result<FileHandle> {
let new_file = duplicate_file(fd, buf, false, token)?;
let current_lock = context::current();
let mut current = current_lock.read(token.token());
let (context, mut token) = current.token_split();
context.insert_file(new_fd, new_file, &mut token).ok_or(Error::new(EEXIST))
}
/// Duplicate file descriptor, replacing another
pub fn dup2(
fd: FileHandle,
@@ -818,14 +755,6 @@ pub fn sys_read(fd: FileHandle, buf: UserSliceWo, token: &mut CleanLockToken) ->
let offset = &mut desc_arc.write(token.token()).offset;
*offset = offset.saturating_add(bytes_read as u64)
}
// I/O accounting: increment rchar and syscr on current context.
// Cross-referenced with Linux 7.1 mm/filemap.c:filemap_get_pages
// and task_io_account_read().
let current = context::current();
let mut guard = current.write(token.token());
guard.io_rchar = guard.io_rchar.saturating_add(bytes_read as u64);
guard.io_syscr = guard.io_syscr.saturating_add(1);
guard.io_read_bytes = guard.io_read_bytes.saturating_add(bytes_read as u64);
Ok(bytes_read)
}
pub fn sys_write(fd: FileHandle, buf: UserSliceRo, token: &mut CleanLockToken) -> Result<usize> {
@@ -846,11 +775,5 @@ pub fn sys_write(fd: FileHandle, buf: UserSliceRo, token: &mut CleanLockToken) -
let offset = &mut desc_arc.write(token.token()).offset;
*offset = offset.saturating_add(bytes_written as u64)
}
// I/O accounting: increment wchar and syscw on current context.
let current = context::current();
let mut guard = current.write(token.token());
guard.io_wchar = guard.io_wchar.saturating_add(bytes_written as u64);
guard.io_syscw = guard.io_syscw.saturating_add(1);
guard.io_write_bytes = guard.io_write_bytes.saturating_add(bytes_written as u64);
Ok(bytes_written)
}
src/syscall/futex.rs
+12 -63
View File
@@ -25,7 +25,7 @@ use crate::{
use crate::syscall::{
data::TimeSpec,
error::{Error, Result, EAGAIN, EFAULT, EINVAL, ETIMEDOUT},
flag::{FUTEX_REQUEUE, FUTEX_WAIT, FUTEX_WAIT64, FUTEX_WAKE},
flag::{FUTEX_WAIT, FUTEX_WAIT64, FUTEX_WAKE},
};
use super::usercopy::UserSlice;
@@ -49,8 +49,13 @@ pub struct FutexEntry {
// implement that fully in userspace. Although futex is probably the best API for process-shared
// POSIX synchronization primitives, a local hash table and wait-for-thread kernel APIs (e.g.
// lwp_park/lwp_unpark from NetBSD) could be a simpler replacement.
static FUTEXES: Mutex<L1, FutexList> =
Mutex::new(FutexList::with_hasher(DefaultHashBuilder::new()));
const FUTEX_SHARDS: usize = 64;
fn futex_shard(phys: PhysicalAddress) -> usize {
(phys.data() as usize >> 12) % FUTEX_SHARDS
}
static FUTEXES: [Mutex<L1, FutexList>; FUTEX_SHARDS] = [const { Mutex::new(FutexList::with_hasher(DefaultHashBuilder::new())) }; FUTEX_SHARDS];
fn validate_and_translate_virt(space: &AddrSpace, addr: VirtualAddress) -> Option<PhysicalAddress> {
// TODO: Move this elsewhere!
@@ -71,7 +76,7 @@ pub fn futex(
op: usize,
val: usize,
val2: usize,
addr2: usize,
_addr2: usize,
token: &mut CleanLockToken,
) -> Result<usize> {
let current_addrsp = AddrSpace::current()?;
@@ -97,7 +102,7 @@ pub fn futex(
{
// TODO: Lock ordering violation
let mut token = unsafe { CleanLockToken::new() };
let mut futexes = FUTEXES.lock(token.token());
let mut futexes = FUTEXES[futex_shard(target_physaddr)].lock(token.token());
let (futexes, mut token) = futexes.token_split();
let (fetched, expected) = if op == FUTEX_WAIT {
@@ -181,10 +186,11 @@ pub fn futex(
}
FUTEX_WAKE => {
let mut woken = 0;
let shard = futex_shard(target_physaddr);
{
drop(addr_space_guard);
let mut futexes_map = FUTEXES.lock(token.token());
let mut futexes_map = FUTEXES[shard].lock(token.token());
let (futexes_map, mut token) = futexes_map.token_split();
let is_empty = if let Some(futexes) = futexes_map.get_mut(&target_physaddr) {
@@ -217,63 +223,6 @@ pub fn futex(
Ok(woken)
}
FUTEX_REQUEUE => {
// Linux 7.1 kernel/futex/requeue.c futex_requeue().
// Wake up to val waiters on primary futex, requeue up to
// val2 waiters to secondary futex (addr2).
let mut woken = 0;
let mut requeued = 0;
let target_virtaddr2 = VirtualAddress::new(addr2);
let target_physaddr2 = validate_and_translate_virt(&addr_space_guard, target_virtaddr2)
.ok_or(Error::new(EFAULT))?;
{
drop(addr_space_guard);
let mut futexes_map = FUTEXES.lock(token.token());
let (futexes_map, mut token) = futexes_map.token_split();
if let Some(futexes) = futexes_map.get_mut(&target_physaddr) {
let mut i = 0;
let current_addrsp_weak = Arc::downgrade(&current_addrsp);
let mut to_requeue = Vec::new();
while i < futexes.len() {
let futex = unsafe { futexes.get_unchecked_mut(i) };
if futex.target_virtaddr != target_virtaddr
|| !current_addrsp_weak.ptr_eq(&futex.addr_space)
{
i += 1;
continue;
}
if woken < val {
futex.context_lock.write(token.token()).unblock();
futexes.remove(i);
woken += 1;
} else if requeued < val2 {
let entry = futexes.remove(i);
to_requeue.push(entry);
requeued += 1;
} else {
i += 1;
}
}
if futexes.is_empty() {
futexes_map.remove(&target_physaddr);
}
if !to_requeue.is_empty() {
futexes_map
.entry(target_physaddr2)
.or_insert_with(Vec::new)
.extend(to_requeue);
}
}
}
Ok(woken + requeued)
}
_ => Err(Error::new(EINVAL)),
}
}
src/syscall/mod.rs
+40 -85
View File
@@ -22,7 +22,7 @@ use self::{
};
use crate::{
context::{self, memory::AddrSpace},
context::memory::AddrSpace,
percpu::PercpuBlock,
scheme::{memory::MemoryScheme, FileHandle},
sync::CleanLockToken,
@@ -73,34 +73,26 @@ pub fn syscall(
let fd = FileHandle::from(b);
//SYS_* is declared in kernel/syscall/src/number.rs
match a {
SYS_WRITE2 => {
let bytes = file_op_generic_ext(fd, token, |scheme, _, desc, token| {
let flags = if f == usize::MAX {
None
} else {
Some(
u32::try_from(f)
.ok()
.and_then(RwFlags::from_bits)
.ok_or(Error::new(EINVAL))?,
)
};
scheme.kwriteoff(
desc.number,
UserSlice::ro(c, d)?,
e as u64,
flags.map_or(desc.flags, |f| desc.rw_flags(f)),
desc.flags,
token,
SYS_WRITE2 => file_op_generic_ext(fd, token, |scheme, _, desc, token| {
let flags = if f == usize::MAX {
None
} else {
Some(
u32::try_from(f)
.ok()
.and_then(RwFlags::from_bits)
.ok_or(Error::new(EINVAL))?,
)
})?;
let current = context::current();
let mut guard = current.write(token.token());
guard.io_wchar = guard.io_wchar.saturating_add(bytes as u64);
guard.io_syscw = guard.io_syscw.saturating_add(1);
guard.io_write_bytes = guard.io_write_bytes.saturating_add(bytes as u64);
Ok(bytes)
}
};
scheme.kwriteoff(
desc.number,
UserSlice::ro(c, d)?,
e as u64,
flags.map_or(desc.flags, |f| desc.rw_flags(f)),
desc.flags,
token,
)
}),
SYS_WRITE => sys_write(fd, UserSlice::ro(c, d)?, token),
SYS_FMAP => {
let addrspace = AddrSpace::current()?;
@@ -129,34 +121,26 @@ pub fn syscall(
scheme.kfutimens(number, UserSlice::ro(c, d)?, token)
}),
SYS_READ2 => {
let bytes = file_op_generic_ext(fd, token, |scheme, _, desc, token| {
let flags = if f == usize::MAX {
None
} else {
Some(
u32::try_from(f)
.ok()
.and_then(RwFlags::from_bits)
.ok_or(Error::new(EINVAL))?,
)
};
scheme.kreadoff(
desc.number,
UserSlice::wo(c, d)?,
e as u64,
flags.map_or(desc.flags, |f| desc.rw_flags(f)),
desc.flags,
token,
SYS_READ2 => file_op_generic_ext(fd, token, |scheme, _, desc, token| {
let flags = if f == usize::MAX {
None
} else {
Some(
u32::try_from(f)
.ok()
.and_then(RwFlags::from_bits)
.ok_or(Error::new(EINVAL))?,
)
})?;
let current = context::current();
let mut guard = current.write(token.token());
guard.io_rchar = guard.io_rchar.saturating_add(bytes as u64);
guard.io_syscr = guard.io_syscr.saturating_add(1);
guard.io_read_bytes = guard.io_read_bytes.saturating_add(bytes as u64);
Ok(bytes)
}
};
scheme.kreadoff(
desc.number,
UserSlice::wo(c, d)?,
e as u64,
flags.map_or(desc.flags, |f| desc.rw_flags(f)),
desc.flags,
token,
)
}),
SYS_READ => sys_read(fd, UserSlice::wo(c, d)?, token),
SYS_FPATH => file_op_generic(fd, token, |scheme, number, token| {
scheme.kfpath(number, UserSlice::wo(c, d)?, token)
@@ -169,12 +153,6 @@ pub fn syscall(
}),
SYS_DUP => dup(fd, UserSlice::ro(c, d)?, token).map(FileHandle::into),
SYS_DUP_INTO => dup_into(
fd,
FileHandle::from(e),
UserSlice::ro(c, d)?,
token,
).map(FileHandle::into),
SYS_DUP2 => {
dup2(fd, FileHandle::from(c), UserSlice::ro(d, e)?, token).map(FileHandle::into)
}
@@ -211,17 +189,6 @@ pub fn syscall(
SYS_FSYNC => file_op_generic(fd, token, |scheme, number, token| {
scheme.fsync(number, token).map(|()| 0)
}),
SYS_SYNC => {
// Red Bear filesystem I/O is synchronous — all data is
// already on disk. sync() is a no-op per POSIX semantics.
// Cross-referenced with Linux 7.1 fs/sync.c ksys_sync().
Ok(0)
}
SYS_SYNCFS => file_op_generic(fd, token, |scheme, number, token| {
// syncfs(fd) syncs the filesystem containing fd.
// On Red Bear, I/O is synchronous so this is a no-op.
scheme.fsync(number, token).map(|()| 0)
}),
// TODO: 64-bit lengths on 32-bit platforms
SYS_FTRUNCATE => file_op_generic(fd, token, |scheme, number, token| {
scheme.ftruncate(number, c, token).map(|()| 0)
@@ -238,15 +205,6 @@ pub fn syscall(
SYS_OPENAT => {
openat(fd, UserSlice::ro(c, d)?, e, f as _, 0, 0, token).map(FileHandle::into)
}
SYS_OPENAT_INTO => openat_into(
fd,
UserSlice::ro(c, d)?,
e,
f as u32,
FileHandle::from(g as usize),
token,
)
.map(FileHandle::into),
SYS_OPENAT_WITH_FILTER => openat(
fd,
UserSlice::ro(c, d)?,
@@ -276,10 +234,7 @@ pub fn syscall(
SYS_MPROTECT => mprotect(b, c, MapFlags::from_bits_truncate(d), token).map(|()| 0),
SYS_MREMAP => mremap(b, c, d, e, f, token),
_ => {
println!("KERNEL: unimplemented syscall a={:#x} b={:#x} c={:#x} d={:#x}", a, b, c, d);
Err(Error::new(ENOSYS))
}
_ => Err(Error::new(ENOSYS)),
}
}