The addr_space_lock.mmap() result must be used or explicitly ignored.
Use let _ = ... to discard it — if mmap fails here, the process
crash will be more informative than a Result warning.
- syscall/futex.rs: scope the context write guard so futexes.swap_remove
can borrow mutably after the guard drops
- syscall/process.rs: add missing 'shared' arg to Grant::zeroed
The bump allocator previously had no way to track which frames are
free for reuse. Add a FreeNode struct that records (next, count,
phys) for freed regions, enabling frame recycling after initial
identity-mapping setup is complete.
The .map_err().?() pattern doesn't work in a function returning (),
because ? requires the enclosing function to return Result. Replace
with plain .expect() (these are bootstrap initialization failures —
if mmap fails here, the kernel cannot continue).
The initial bootstrap init (started directly by the kernel via
usermode_bootstrap) had RSP = 0 from InterruptStack::init(), which
does NOT set RSP. This caused any push/call in the binary to fault
on address 0.
The Rust panic handler caught the resulting page fault and generated
ud2 — visible as 'Invalid opcode fault' at a low RIP before main()
ever runs. This was the root cause of the init crash.
Fix:
- Map 8 pages (32 KiB) of user stack just above the initfs image
- Set RSP to the top of the stack
- Add debug log showing the mapped region
The stack is intentionally small (32 KiB) — enough for Rust _start
to run but small enough to detect stack overflow immediately. Once
init calls fexec_impl to spawn the real init, the real init gets its
own full 8 MB stack via the redox-rt exec path.
- rmm/src/arch/emulate.rs: replace unimplemented!() with actual TLB
invalidation (remove the address from the map). Without this the
emulate arch silently fails to invalidate TLB entries during
context switches, which can cause stale mappings.
- src/syscall/futex.rs: remove completed TODO marker for FUTEX_REQUEUE
(work done in a prior commit).
The sys_mkns and sys_setns functions added in 48fc2f1c reference
UserSliceRo and FileHandle but the imports were not updated, causing
the build to fail with 'cannot find type' errors. This commit adds
the missing imports.
Adds proper kernel-level namespace syscall handling:
- SYS_MKNS: dups the current namespace fd with the caller-supplied
buffer (NsDup::ForkNs + scheme names). The dup is handled by the
bootstrap's userspace namespace manager (initnsmgr) which creates
the new namespace and returns the fd.
- SYS_SETNS: switches the calling context's ns_fd to the given fd.
This makes the new namespace the default for all subsequent scheme
lookups by this context.
- Context gains an ns_fd: Option<usize> field, initialized to None,
meaning "use the global namespace" (default).
- Both syscalls are wired into the dispatch table in syscall/mod.rs.
Per local/docs/LOCAL-FORK-SUPREMACY-POLICY.md: the kernel fork must
be complete. Previously SYS_SETNS was undefined and SYS_MKNS returned
ENOSYS; init's setrens() panicked on bare metal.
sys_read and sys_write passed u64::MAX as the offset for non-positioned
file descriptors. Many scheme handlers (AddrSpace, proc:Start, etc.)
check offset != 0 or offset % word_size != 0, causing EINVAL when
receiving u64::MAX. Passing 0 instead fixes all non-positioned read/write
operations across all scheme handlers uniformly.
This replaces the per-handler u64::MAX workarounds with a root-cause fix.
sys_write passes u64::MAX as offset for non-positioned file descriptors.
ContextHandle::Start used an inline 'offset != 0' check which rejected
u64::MAX, causing EINVAL when bootstrap called start_fd.write(&[0]) during
fork_inner(). Replaced with require_zero_offset() which accepts both 0 and
u64::MAX (fixed in the previous commit for the same class of bug).
sys_write passes u64::MAX as the offset sentinel for non-positioned file
descriptors. The proc scheme's require_zero_offset rejected this value,
causing bootstrap's tcb_activate to crash with EINVAL when writing
EnvRegisters (fsbase) back to the regs/env handle.
This was the root cause of the 'Invalid opcode fault' during bootstrap
(PID 0) that prevented redbear-mini from reaching a login prompt.
Previously: PfError::Oom during page fault recovery caused the
entire kernel to panic via todo!("oom"). This means a single
OOM allocation in a user process would crash the whole OS.
Now: OOM is logged as a warning, the faulting process receives
SIGSEGV (same path as other non-fatal page faults), and the
kernel continues. The process may be killed, but the system
stays up. This is consistent with how Linux handles OOM during
page fault handling: send SIGSEGV/SIGKILL, don't panic.
The page fault handler for user-mode faults had a separate
arms for Segv, RecursionLimitExceeded, and NonfatalInternalError from
try_correcting_page_tables. The first two fell through to return
Segv to the process; NonfatalInternalError called todo!() which
causes a kernel panic. This is a kernel-level crash triggered by
a userspace page table correction attempt that reports an internal
(non-fatal) error.
Fix: collapse NonfatalInternalError into the same fall-through arm
as Segv and RecursionLimitExceeded. The error name says 'nonfatal'
— it should not crash the kernel. The userspace process receives
a segmentation fault signal instead.
Added println! to the syscall dispatch catch-all to log the
unknown syscall number and arguments when returning ENOSYS.
Previously any unrecognized syscall number silently returned
ENOSYS with no diagnostic, making it impossible to discover
missing syscall implementations without application-level
debugging.
Found by comprehensive disguised-stub audit (47 patterns, 37
actionable). This is the most impactful remaining fix from
that audit.
Added sync() and syncfs() syscall handlers. Both are no-ops
since Red Bear filesystem I/O is synchronous (data is always
on disk). syncfs(fd) delegates to scheme.fsync() for the
filesystem containing the fd.
Cross-referenced with Linux 7.1 fs/sync.c ksys_sync() and
do_syncfs(). SYS_SYNC uses number 119, SYS_SYNCFS uses 120.
Extended /proc/self resolution to handle sub-paths like
/proc/self/fd/0, /proc/self/stat, /proc/self/status, etc.
Previously only /proc/self (the directory itself) was resolved.
Now /proc/self/fd, /proc/self/stat, /proc/self/maps, and all
other sub-paths are resolved by replacing 'self' with the
current PID and delegating to the standard proc_open() parser.
Cross-referenced with Linux 7.1 fs/proc/self.c proc_self_get_link()
which creates a symlink to the current PID directory. This
completes the /proc/self implementation for all sub-entries.
The FADT fields (PM1a_CNT, PM1a_STS, FIRMWARE_CTRL) are at fixed
offsets from the START of the SDT (including the 36-byte ACPI header),
not from the data area. Previously using sdt.data_address() caused
reads from the wrong offset, making shutdown/reboot fail on real
hardware.
Fixed by reading from the SDT base pointer instead of data_address().
Cross-referenced with Linux 7.1 drivers/acpi/acpica/tbfadt.c which
reads FADT fields from the table base, accounting for the header.
Added FUTEX_REQUEUE (opcode 2) to the futex syscall implementation.
Cross-referenced with Linux 7.1 kernel/futex/requeue.c futex_requeue().
Operation: wake up to waiters on the primary futex, and
requeue up to waiters to a secondary futex at addr2.
This avoids the thundering herd problem in pthread condition
variable broadcast: instead of waking all waiters and having
them immediately contend, most are moved to a private futex
where they wake one at a time.
Previously the futex syscall would return EINVAL for FUTEX_REQUEUE,
breaking pthread_cond_broadcast on contended condition variables.
Added /proc/self resolution in open_inner(). When /proc/self
is opened, it resolves to /proc/<current-pid> by reading
context::current().pid and returning a ProcDir handle.
Cross-referenced with Linux 7.1 fs/proc/self.c proc_self_get_link()
which creates a symlink inode pointing to the current PID.
This enables 'ls /proc/self', 'cat /proc/self/status', and all
other tools that use /proc/self to access their own process info.
Added ProcFilesystems ContextHandle that lists supported filesystems
in Linux /proc/filesystems format. Lists sysfs, proc, devtmpfs,
and redoxfs as supported types.
Cross-referenced with Linux 7.1 fs/filesystems.c filesystems_proc_show().
'nodev' prefix indicates filesystems not requiring a block device.
Also added to ProcRoot directory listing.
Added ProcVersion ContextHandle that outputs the kernel version
string using CARGO_PKG_VERSION, TARGET, and COOKBOOK_SOURCE_IDENT
environment variables from the build system.
Cross-referenced with Linux 7.1 fs/proc/version.c version_proc_show().
Format: 'Red Bear OS version X.Y.Z (arch) source_ident'
Also added to ProcRoot directory listing.
Added ProcLoadavg ContextHandle that counts runnable contexts
using context::contexts() iterator and Status::is_runnable().
Outputs Linux /proc/loadavg format:
0.00 0.00 0.00 nr_runnable/total 0
Cross-referenced with Linux 7.1 fs/proc/loadavg.c loadavg_proc_show().
Load averages are 0.00 (Red Bear does not track 1/5/15-min EMA).
Last PID is 0 (no PID tracking yet).
Also added to ProcRoot directory listing.
Added ProcUptime ContextHandle that reads monotonic() time
from the kernel and outputs it as uptime_seconds idle_seconds
in Linux /proc/uptime format (two floats with newline).
Cross-referenced with Linux 7.1 fs/proc/uptime.c uptime_proc_show().
Idle time is 0.0 since Red Bear does not track per-CPU idle time.
Also added to ProcRoot directory listing.
Added ProcMeminfo ContextHandle that reads total_frames/free_frames
from the kernel memory subsystem and outputs them in Linux
/proc/meminfo format (MemTotal, MemFree, MemAvailable, Buffers,
Cached, SwapTotal, etc. all in kB).
Cross-referenced with Linux 7.1 fs/proc/meminfo.c meminfo_proc_show().
Also added to ProcRoot directory listing alongside cpuinfo.
Added ProcCpuinfo ContextHandle that calls the arch-specific
cpu_info() function to output processor information (model,
features, cache, topology). Appears as a regular file in
/proc/cpuinfo and is listed in the ProcRoot directory.
Cross-referenced with Linux 7.1 arch/x86/kernel/cpu/proc.c
show_cpuinfo() which formats /proc/cpuinfo from cpu_data.
Added ProcFdDir ContextHandle that lists open file descriptors
as symlink directory entries. Reads from posix_fdtbl and outputs
fd numbers (0, 1, 2, ...) as directory entries.
Cross-referenced with Linux 7.1 fs/proc/fd.c proc_fd_link()
and tid_fd_revalidate(). Listed fds are open (non-None) entries
from the process's POSIX file descriptor table.
Each entry appears as /proc/[pid]/fd/[n] for use by tools
like ls, lsof, and ps.
SYS_READ2 and SYS_WRITE2 are the positioned read/write syscalls
(equivalent to pread/pwrite in POSIX). Previously only SYS_READ and
SYS_WRITE were counted. Now all four I/O paths are tracked.
This makes /proc/[pid]/io accurate for programs that use
pread/pwrite (common in random-access file I/O, memory-mapped
I/O bypass, and async I/O libraries).
Cross-referenced with Linux 7.1 mm/filemap.c:rw_verify_area
which tracks these accounting fields for all read/write variants.
sys_read increments io_rchar, io_syscr, io_read_bytes.
sys_write increments io_wchar, io_syscw, io_write_bytes.
This makes /proc/[pid]/io show actual I/O activity.
Cross-referenced with Linux 7.1 mm/filemap.c:filemap_get_pages
and include/linux/task_io_accounting.h. The fields are saturated
additions to avoid overflow on long-running processes.
Added 7 I/O accounting fields to Context (rchar, wchar, syscr, syscw,
read_bytes, write_bytes, cancelled_write_bytes) and new ProcIo
ContextHandle that outputs them in Linux key=value format:
rchar: 0
wchar: 0
syscr: 0
syscw: 0
read_bytes: 0
write_bytes: 0
cancelled_write_bytes: 0
Cross-referenced with Linux 7.1 fs/proc/base.c do_task_io_accounting().
All values start at 0 and will be incremented as the syscall
interface is extended to track I/O operations.
Added rlimits: [u64; 16] to Context (initialized to RLIM_INFINITY)
and new ProcLimits ContextHandle that outputs in Linux format:
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
...
Cross-referenced with Linux 7.1 fs/proc/array.c proc_pid_limits().
All 16 RLIMIT_* resource limits are listed with their units.
New ProcStatm ContextHandle that reports the 7-field memory summary
matching Linux /proc/[pid]/statm format:
size resident shared text lib data dt
All values in pages. Cross-referenced with Linux 7.1
fs/proc/array.c:proc_pid_statm().
Resident count is approximated as the total physical grants
(GRANT_PHYS), since Red Bear has no paging/swapping and all
memory is resident. Other fields (shared, lib, dt) are 0 since
Red Bear's memory model doesn't distinguish these.
New ProcMaps ContextHandle that iterates over addr_space.grants
and formats each grant in Linux /proc/[pid]/maps format:
address_start-address_end perms 00000000 00:00 0 path
Where perms is rwxp based on GrantFlags (GRANT_READ/WRITE/EXEC),
and path is [file]/[vdso]/[heap] based on whether the grant is
file-backed and its address range.
Cross-referenced with Linux 7.1 fs/proc/task_mmu.c proc_pid_maps_op
and show_vma_header_prefix/show_map_vma().
- Add SYS_OPENAT/SYS_DUP aliases in local syscall fork for upstream 0.9.0 renamed constants.
- Cast FADT_MIN_SIZE_ACPI_2_0 to u32 to match sdt.length() return type.
- Keep Cargo.lock based on upstream master with only redox_syscall sourced from local fork.
- Rebase all Red Bear kernel changes onto upstream master (4d5d36d4).
- Update version to 0.5.12+rb0.3.0 and add Red Bear author attribution.
- Switch redox_syscall direct dependency to local fork path (../syscall).
- Bump rust-toolchain.toml to nightly-2026-05-24.
- Regenerate Cargo.lock for +rb0.3.0 suffixes and path deps.
* Remove lots of unsafe code by initialising in the closure passed to
`call_once`
* Remove `NUMBER_OF_DOMAINS` as size can always be inferred from the
hashmap's len
* `syscall::sendfd`, now, when called with a `ContextHandle::FileTable` adds the fd to the filetable removing it from the calling process's filetable
* Files can now be added to, removed from another process, and can be duplicated using `ProcScheme::kcall`
* Files of another process with the flag O_CLOEXEC can now be closed by using `kcall`
With this change, we use the recently-added `COOKBOOK_TIME_IDENT`
environment variable at compile time to get the "version" info for the
uname scheme. This field is unspecified by POSIX what it should have,
but most kernels (including Linux, XNU, NetBSD, etc.) put the build
time here.
According to Agner Fog's documentation
(https://agner.org/optimize/instruction_tables.pdf), the following information
is true:
These instructions have identical latency and throughput, except for Zen
5, which lists `test` as having reciprocal throughput of 0.25 cycles,
and `bt` having reciprocal throughput of 0.33 cycles. This rules out the
`bt` instruction for being the ideal instruction.
The other `test` instructions were removed as candidates because,
regardless of the size of the memory fetched at the address, at least 64
bytes will need to be fetched because it will be stored in the cache
line. The WORD and DWORD cases can be ruled out because we cannot assume
that `rsp + 16` or `rsp + 17` will not be on a 64-byte alignment
boundary, which would cause two cachelines to be essentially filled with
garbage we don't care about. The best case scenario is that we only need
to fill one cache line with garbage, which is what the BYTE version does
every time.
It has been broken since we moved namespaces into userspace. Fixing it
can be done entirely inside userspace. Either by introducing a SYS_CALL
for the root capability or by treating closing the last fd as unmount.
This commit fixes the following panic when running nodejs:
```
KERNEL PANIC: panicked at src/context/memory.rs:267:18:
grant cannot magically disappear while we hold the lock!
FP ffff80002021f650: PC ffffffff8005e45b
FFFFFFFF8005E2E0+017B
kernel::panic::panic_handler_inner
FP ffff80002021f730: PC ffffffff80057559
FP ffff80002021f740: PC ffffffff8009a0ff
FP ffff80002021f770: PC ffffffff8009c214
FP ffff80002021f7d0: PC ffffffff80010997
FFFFFFFF8000FDD0+0BC7
kernel::context::memory::AddrSpaceWrapper::mprotect
FP ffff80002021fd40: PC ffffffff8008cb0b
FFFFFFFF8008CA40+00CB
kernel::syscall::process::mprotect
FP ffff80002021fd70: PC ffffffff8006ce6c
FFFFFFFF8006CBE0+028C
kernel::syscall::syscall
FP ffff80002021fe90: PC ffffffff8008d3cf
FFFFFFFF8008D320+00AF
__inner_syscall_instruction
FP ffff80002021ff50: PC ffffffff800830c3
FFFFFFFF80083080+0043
kernel::arch::x86_64::interrupt::syscall::syscall_instruction
00007ffffffffaf0: GUARD PAGE
CPU #1, CID 0xffffff7f8015b910
NAME: /usr/bin/node, DEBUG ID: 74
SYSCALL: mprotect(0x203C0000, 262144, Some(MapFlags(0x0)))
HALT
```
The grant did not magically disappear. When going through the
`grant_span_res` regions, the function adds (and removes) grants to the
`self.grants` tree. The insertion and deletion functions also merge
adjacent grants together when possible. This is an issue since we can no
longer use the keys we established before we started iterating. This
commit modifies the places where `remove` is used in this fashion to use
`remove_containing` instead. The `remove_containing` function will
remove the grant that *contains* the page.
Should it be done this way (requires unstable feature `btree_cursors`)?
Signed-off-by: Anhad Singh <andypython@protonmail.com>
`Grant::remap` should not directly mark entries as writeable because the
entry could be private and requires Copy-On-Write (COW) handling. This
bug fixes the random panics that occur when running programs that
incorrectly mark private memory regions as writeable. This also
inadvertently caused other systems to crash. This is because The Zeroed
Frame is mapped instead of allocating a new zeroed memory region every
time. Only after the first write is the frame actually allocated. This
means that with this bug one could remap a freshly mmap'ed ANONYMOUS
region with the write flag and rewrite the contents of The Zeroed Frame,
which is a big problem. This means that any program that allocated
memory with the ANONYMOUS flag after this would not receive zeroed
memory.
Example of `make` hitting this bug when the shared `ld.so` patches are
applied:
```
....
[page at 0x48bd000] is mapped to the zeroed frame with PageFlags { present: true, write: false, executable: false, user: true, bits: 0x8000000000000005 }
[page at 0x49b1000] is mapped to the zeroed frame with PageFlags { present: true, write: true, executable: false, user: true, bits: 0x8000000000000007 }
[page at 0x49b2000] is mapped to the zeroed frame with PageFlags { present: true, write: true, executable: false, user: true, bits: 0x8000000000000007 }
[page at 0x49b3000] is mapped to the zeroed frame with PageFlags { present: true, write: true, executable: false, user: true, bits: 0x8000000000000007 }
[page at 0x49b4000] is mapped to the zeroed frame with PageFlags { present: true, write: true, executable: false, user: true, bits: 0x8000000000000007 }
0x43ff000..0x49b5000 :: MapFlags(PROT_WRITE | PROT_READ)
[page at 0x49b5000] is mapped to the zeroed frame with PageFlags { present: true, write: false, executable: false, user: true, bits: 0x8000000000000005 }
[page at 0x49b6000] is mapped to the zeroed frame with PageFlags { present: true, write: false, executable: false, user: true, bits: 0x8000000000000005 }
....
```
and causing the following crash:
```
Page fault: 0000000000001718 P | WR | US
RFLAG: 0000000000010202
CS: 000000000000002b
RIP: 00000000004125fd
RSP: 00007fffffffdd80
SS: 0000000000000023
FSBASE 00000000003f6000
GSBASE 0000000000000000
KGSBASE ffff80007a800000
RAX: 0000000000001718
RCX: 0000000000000000
RDX: ffffffffffffffff
RDI: 000000000043b5a0
RSI: 00000000004537e0
R8: 0000000000000030
R9: 000000000043b5a0
R10: 0000000000000030
R11: 00000000000000ff
RBX: 0000000000000000
RBP: 00007fffffffddb0
R12: 000000000043b5a0
R13: 00000000004537e0
R14: 00000000004537e0
R15: 000000000043b5a0
FP 00007fffffffddb0: PC 000000000042614a
FP 00007fffffffde40: PC 0000000000411ae9
FP 00007fffffffde80: PC 0000000000417187
FP 00007fffffffdee0: PC 0000000000411d39
FP 00007fffffffdf90: PC 000000000041207d
FP 00007fffffffe020: PC 000000000040bc4c
FP 00007fffffffe0c0: PC 000000000040c7c9
FP 00007fffffffe0f0: PC 00000000004276ba
FP 00007fffffffe1a0: PC 0000000000428087
FP 00007fffffffe210: PC 000000000041eb6a
FP 00007fffffffe400: PC 00000000004205c3
FP 00007fffffffe490: PC 00000000004208dd
FP 00007fffffffe4d0: PC 000000000040747e
FP 00007ffffffffd60: PC 000000000092111c
<Invalid next frame pointer 0x0000000000438490; stack walk ended>
FP ffff80001b5bfe80: PC ffffffff80046dad
FFFFFFFF80046BD0+01DD
kernel::arch::x86_shared::interrupt::exception::page::inner
FP ffff80001b5bff50: PC ffffffff8003ffbb
FFFFFFFF8003FF84+0037
kernel::arch::x86_shared::interrupt::exception::page
00007fffffffddb0: GUARD PAGE
kernel::context::signal:INFO -- UNHANDLED EXCEPTION, CPU #1, PID 6548, NAME /usr/bin/make, CONTEXT 0xffffff7f8013fdc0
```
Also the crash info would vary every time you run the program since the
contents of The Zeroed Frame would have been mutated. Sneaky bug :^)
This also fixes the crashes that happen when running node.js (and maybe
other programs): https://gitlab.redox-os.org/redox-os/relibc/-/issues/229
Signed-off-by: Anhad Singh <andypython@protonmail.com>
This commit modifies the transform function (`f`) argument of
`remap_with_full` to return an `Option`. This allows the caller to skip
remaps based on the previous frame and page flags. It can alternatively
be done by first translating the address and then remapping based on
that but that would mean we have to walk the page tables twice :|
Signed-off-by: Anhad Singh <andypython@protonmail.com>
During the snapshot shown, interrupts would be _disabled_ on both CPUs and both
would have have the _same_ address space (i.e.
`PercpuBlock::current().current_addrsp`).
```rs
// CPU 0 CPU 1 (inside `context::switch()`)
// ------------------------------------------------------------------------------------------------------------------------------------------------------------------------
// let guard = current_addrsp.acquire_write()
// let flusher = Flusher::with_cpu_set(&addrsp.used_by, ¤t_addrsp.tlb_ack);
// ...
// drop(flusher); // Flusher::flush()
// // send IPI to CPU 1 as it is in the `active_set`
// shootdown_tlb_ipi(Some(1));
// percpu::switch_arch_hook();
// // spin until TLB shootdown IPI has been acknowledged by CPU 1 // acquire_read() to remove the CPU from the `used_by` set in `prev_addrsp`
// while ackword < affected_cpu_count prev_addrsp.acquire_read()
// PercpuBlock::current().maybe_handle_tlb_shootdown();
// spin_loop();
// }
// drop(guard);
```
Now CPU 0 will spin _until_ CPU 1 has acknowledged the IPI. However, CPU 1 has
interrupts disabled and is in the middle of a context switch. To complete the
context switch it needs to remove itself from the `used_by` set in
`prev_addrsp` (which is the current address space atm). To do that it needs to
`acquire_read()` lock but it cannot since it is held by CPU 0 and won't release
it until CPU 1 has acknowledged the IPI; creating a deadlock.
Also note:
```rs
// src/context/memory.rs
// NOTE: Lock must be held, which must be guaranteed by the caller.
pub fn flush(&mut self) {
```
Here is some output from GDB to back this up:
```
pwndbg> info threads
Id Target Id Frame
1 Thread 1.1 (CPU#0 [running]) core::sync::atomic::atomic_compare_exchange_weak<usize> (dst=0xffffff7f801972f0, old=0, new=1, success=core::sync::atomic::Ordering::Acquire, failure=core::sync::atomic::Ordering::Relaxed)
at /mnt/redox/prefix/x86_64-unknown-redox/sysroot/lib/rustlib/src/rust/library/core/src/sync/atomic.rs:4100
2 Thread 1.2 (CPU#1 [running]) kernel::context::memory::Flusher::flush (self=0xffff80004212f4b8) at src/context/memory.rs:2827
3 Thread 1.3 (CPU#2 [running]) spin::rwlock::RwLock<kernel::context::context::Context, spin::relax::Spin>::write<kernel::context::context::Context, spin::relax::Spin> (self=0xffffff7f801a99c0)
at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/spin-0.9.8/src/rwlock.rs:239
* 4 Thread 1.4 (CPU#3 [running]) 0xffffffff80059cf9 in kernel::context::memory::AddrSpaceWrapper::acquire_read (self=0xffffff7f80188110)
at /mnt/redox/prefix/x86_64-unknown-redox/sysroot/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/x86/sse2.rs:25
pwndbg> p kernel::context::arch::CONTEXT_SWITCH_LOCK
$1 = core::sync::atomic::AtomicBool {
v: core::cell::UnsafeCell<u8> {
value: 1
}
}
pwndbg> p (*kernel::percpu::ALL_PERCPU_BLOCKS[0].p.value).wants_tlb_shootdown.v.value
$2 = 0
pwndbg> p (*kernel::percpu::ALL_PERCPU_BLOCKS[1].p.value).wants_tlb_shootdown.v.value
$3 = 0
pwndbg> p (*kernel::percpu::ALL_PERCPU_BLOCKS[2].p.value).wants_tlb_shootdown.v.value
$4 = 1
pwndbg> p (*kernel::percpu::ALL_PERCPU_BLOCKS[3].p.value).wants_tlb_shootdown.v.value
$5 = 0
pwndbg> p (*kernel::percpu::ALL_PERCPU_BLOCKS[3].p.value).current_addrsp
$6 = core::cell::RefCell<core::option::Option<alloc::sync::Arc<kernel::context::memory::AddrSpaceWrapper, alloc::alloc::Global>>> {
borrow: core::cell::Cell<isize> {
value: core::cell::UnsafeCell<isize> {
value: 0
}
},
value: core::cell::UnsafeCell<core::option::Option<alloc::sync::Arc<kernel::context::memory::AddrSpaceWrapper, alloc::alloc::Global>>> {
value: core::option::Option<alloc::sync::Arc<kernel::context::memory::AddrSpaceWrapper, alloc::alloc::Global>>::Some(alloc::sync::Arc<kernel::context::memory::AddrSpaceWrapper, alloc::alloc::Global> {
ptr: core::ptr::non_null::NonNull<alloc::sync::ArcInner<kernel::context::memory::AddrSpaceWrapper>> {
pointer: 0xffffff7f80188100
},
phantom: core::marker::PhantomData<alloc::sync::ArcInner<kernel::context::memory::AddrSpaceWrapper>>,
alloc: alloc::alloc::Global
})
}
}
pwndbg> p (*kernel::percpu::ALL_PERCPU_BLOCKS[2].p.value).current_addrsp
$7 = core::cell::RefCell<core::option::Option<alloc::sync::Arc<kernel::context::memory::AddrSpaceWrapper, alloc::alloc::Global>>> {
borrow: core::cell::Cell<isize> {
value: core::cell::UnsafeCell<isize> {
value: 0
}
},
value: core::cell::UnsafeCell<core::option::Option<alloc::sync::Arc<kernel::context::memory::AddrSpaceWrapper, alloc::alloc::Global>>> {
value: core::option::Option<alloc::sync::Arc<kernel::context::memory::AddrSpaceWrapper, alloc::alloc::Global>>::Some(alloc::sync::Arc<kernel::context::memory::AddrSpaceWrapper, alloc::alloc::Global> {
ptr: core::ptr::non_null::NonNull<alloc::sync::ArcInner<kernel::context::memory::AddrSpaceWrapper>> {
pointer: 0xffffff7f80188100
},
phantom: core::marker::PhantomData<alloc::sync::ArcInner<kernel::context::memory::AddrSpaceWrapper>>,
alloc: alloc::alloc::Global
})
}
}
pwndbg> p (*kernel::percpu::ALL_PERCPU_BLOCKS[1].p.value).current_addrsp
$8 = core::cell::RefCell<core::option::Option<alloc::sync::Arc<kernel::context::memory::AddrSpaceWrapper, alloc::alloc::Global>>> {
borrow: core::cell::Cell<isize> {
value: core::cell::UnsafeCell<isize> {
value: 0
}
},
value: core::cell::UnsafeCell<core::option::Option<alloc::sync::Arc<kernel::context::memory::AddrSpaceWrapper, alloc::alloc::Global>>> {
value: core::option::Option<alloc::sync::Arc<kernel::context::memory::AddrSpaceWrapper, alloc::alloc::Global>>::Some(alloc::sync::Arc<kernel::context::memory::AddrSpaceWrapper, alloc::alloc::Global> {
ptr: core::ptr::non_null::NonNull<alloc::sync::ArcInner<kernel::context::memory::AddrSpaceWrapper>> {
pointer: 0xffffff7f80188100
},
phantom: core::marker::PhantomData<alloc::sync::ArcInner<kernel::context::memory::AddrSpaceWrapper>>,
alloc: alloc::alloc::Global
})
}
}
pwndbg> p (*kernel::percpu::ALL_PERCPU_BLOCKS[0].p.value).current_addrsp
$9 = core::cell::RefCell<core::option::Option<alloc::sync::Arc<kernel::context::memory::AddrSpaceWrapper, alloc::alloc::Global>>> {
borrow: core::cell::Cell<isize> {
value: core::cell::UnsafeCell<isize> {
value: 0
}
},
value: core::cell::UnsafeCell<core::option::Option<alloc::sync::Arc<kernel::context::memory::AddrSpaceWrapper, alloc::alloc::Global>>> {
value: core::option::Option<alloc::sync::Arc<kernel::context::memory::AddrSpaceWrapper, alloc::alloc::Global>>::Some(alloc::sync::Arc<kernel::context::memory::AddrSpaceWrapper, alloc::alloc::Global> {
ptr: core::ptr::non_null::NonNull<alloc::sync::ArcInner<kernel::context::memory::AddrSpaceWrapper>> {
pointer: 0xffffff7f80133ac8
},
phantom: core::marker::PhantomData<alloc::sync::ArcInner<kernel::context::memory::AddrSpaceWrapper>>,
alloc: alloc::alloc::Global
})
}
}
```
```
pwndbg> thread 4
[Switching to thread 4 (Thread 1.4)]
25 in /mnt/redox/prefix/x86_64-unknown-redox/sysroot/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/x86/sse2.rs
pwndbg> info threads
Id Target Id Frame
1 Thread 1.1 (CPU#0 [running]) core::sync::atomic::atomic_compare_exchange_weak<usize> (dst=0xffffff7f801972f0, old=0, new=1, success=core::sync::atomic::Ordering::Acquire, failure=core::sync::atomic::Ordering::Relaxed)
at /mnt/redox/prefix/x86_64-unknown-redox/sysroot/lib/rustlib/src/rust/library/core/src/sync/atomic.rs:4100
2 Thread 1.2 (CPU#1 [running]) kernel::context::memory::Flusher::flush (self=0xffff80004212f4b8) at src/context/memory.rs:2827
3 Thread 1.3 (CPU#2 [running]) spin::rwlock::RwLock<kernel::context::context::Context, spin::relax::Spin>::write<kernel::context::context::Context, spin::relax::Spin> (self=0xffffff7f801a99c0)
at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/spin-0.9.8/src/rwlock.rs:239
* 4 Thread 1.4 (CPU#3 [running]) 0xffffffff80059cf9 in kernel::context::memory::AddrSpaceWrapper::acquire_read (self=0xffffff7f80188110)
at /mnt/redox/prefix/x86_64-unknown-redox/sysroot/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/x86/sse2.rs:25
pwndbg> thread 4
[Switching to thread 4 (Thread 1.4)]
25 in /mnt/redox/prefix/x86_64-unknown-redox/sysroot/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/x86/sse2.rs
pwndbg> bt
```
This is *after* everything freezes.
For relibc tests, if I run `while true; do
/root/relibc-tests/pthread/once; done` it eventually does freeze up and
does so at the same location:
```
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────[ THREADS (4 TOTAL) ]──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
► 1 "" stopped: 0xffffffff800750a2 <kernel::context::switch::switch+1282>
4 "" stopped: 0xffffffff80074d5a <kernel::context::switch::switch+442>
3 "" stopped: 0xffffffff80059cf9 <kernel::context::context::Context::set_addr_space+217>
2 "" stopped: 0xffffffff80020839 <kernel::context::memory::Flusher::flush+361>
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
```
Related: https://gitlab.redox-os.org/redox-os/kernel/-/issues/175
Signed-off-by: Anhad Singh <andypython@protonmail.com>
Previously, if a futex with a timeout was woken up (even via
`futex_wake`), it was treated as though the timeout had expired. When
the timeout expires, the scheduler sets `wake` to `None` and unblocks
the process. Hence if `wake` is `None` and if a timeout was given to
futex, it has expired. Otherwise the process was woken up by
`futex_wake`.
Signed-off-by: Anhad Singh <andypython@protonmail.com>
The `AddrSpaceWrapper::r#move` is used by `mremap` to move, expand or
shrink existing mappings.
`src_span.count`: Size of old region in pages
`new_page_count`: Size of new region in pages
Fixes the following panic when running `cargo`:
```
KERNEL PANIC: panicked at src/context/memory.rs:1969:9:
assertion failed: self.info.can_extract(false)
FP ffff80000cbbf2c0: PC ffffffff8004ced5
FFFFFFFF8004CD50+0185
kernel::panic::panic_handler_inner
FP ffff80000cbbf3a0: PC ffffffff8004ac99
FP ffff80000cbbf3b0: PC ffffffff800a34df
FP ffff80000cbbf3e0: PC ffffffff800a34b4
FP ffff80000cbbf430: PC ffffffff8001c029
FFFFFFFF8001BF10+0119
kernel::context::memory::Grant::extract
FP ffff80000cbbf4a0: PC ffffffff800181ac
FFFFFFFF80016D30+147C
kernel::context::memory::AddrSpaceWrapper::move
FP ffff80000cbbfd30: PC ffffffff80037c08
FFFFFFFF80035660+25A8
kernel::syscall::syscall
FP ffff80000cbbfe90: PC ffffffff8008efbf
FFFFFFFF8008EF10+00AF
__inner_syscall_instruction
FP ffff80000cbbff50: PC ffffffff80085d33
FFFFFFFF80085CF0+0043
kernel::arch::x86_64::interrupt::syscall::syscall_instruction
0000000000004455: GUARD PAGE
CPU #2, CID 0xffffff7f80133690
NAME: /scheme/initfs/bin/redoxfs, DEBUG ID: 43
SYSCALL: mremap(0x2177F000, 0x8000, 0x0, 0xC7000, 0x60000)
HALT
```
```
mremap_flags = MremapFlags(
0x0,
)
prot_flags = MapFlags(
PROT_WRITE | PROT_READ,
)
```
Signed-off-by: Anhad Singh <andypython@protonmail.com>
Instead of using a simple switch to determine if preemption is enabled
(`is_preemptable: bool`), a counter is used instead. This handles the
case where a function holding a `PreemptGuard` calls another function
that creates a new `PreemptGuard`.
Signed-off-by: Anhad Singh <andypython@protonmail.com>
This is a lot easier to parse in no_alloc scenarios than the current
textual format. This would help with moving the cloexec handling from
relibc to redox-rt.
The global allocator panics when out of memory rather than returning an
errors. In addition there are plenty of other places where we don't
handle allocation failure anyway. At some point in the future we should
systematically handle out of memory conditions, but until then let's not
pretend we do and get rid of the usage of the unstable allocator_api
feature that is unlikely to get stabilized any time soon.
This only adds a pointer worth of overhead per cpu core in the
PerCpuBlock struct.
Also fix a bunch of unsafe_op_in_unsafe_fn warnings in the profiling
code.
Rust-analyzer runs cargo check --all-targets which also checks the test
configuration where previously there would be a bunch of warnings for
unused things due to the panic handler being configured out.
This way we can choose our own size for the stack and don't have to
identity map it manually. Also this way the bootloader doesn't have to
change the stack pointer right before calling into the kernel (which it
currently does in an unsound way)
Inspired by how Haiku does printing for its kernel debugger, this commit
gets rid of the scrolling when the bootlog reaches the end of the screen
and instead wraps around to the start of the screen. Between the last
written line and the first visible written line there is always a blank
line to provide visual separation.
Getting rid of the scrolling significantly simplifies the implementation
of graphical debug and removes the need for double buffering for
performance as we no longer need to read back the framebuffer when
scrolling which is very expensive on write-combining memory like the
framebuffer.
This significantly reduces the amount of boot logs produced by the
kernel by default on x86. With this the full kernel boot logs now fit on
a single screen on my system.
Replaces the `Once<Vec<u8>>` with `Once<&'static [u8]>` to avoid an
unnecessary allocation.
Adds some basic documentation.
Signed-off-by: Elle Rhumsaa <elle@weathered-steel.dev>
This allows them to be immediately installed by kstart/kstart_ap without
having to wait for the page tables to be set up correctly. This removes
the initial IDT.
This allows them to be immediately installed by kstart_ap without having
to wait for the page tables to be set up correctly. This removes the
initial GDT.
This avoids the need to explicitly set a logger early during boot, which
reduces the amount of moving parts that could go wrong slightly. And it
cuts the kernel image size by 13kb.
According to the comments this previously wasn't done due to hashbrown
not having a const constructor. While it is true that HashMap::new() is
not const, HashMap::with_hasher() is const. HashMap::new() becoming
const is likely blocked on const traits.
This shrinks the kernel image by about 35kb.
It is only used for AlignedBox which can just directly implement Send
and Sync. Additionally make the Send and Sync impls more restrictive by
requiring the inner type to be Send cq Sync, previously AlignedBox was
always Send and Sync, which is not sound.
Previously the deallocation would be rounded to the next power of two
preventing partial deallocation. But more importantly previously trying
to free phys_contiguous frames while another processes still borrows
them. Now this should just cause the deallocation to be delayed.
SBI needs a physical address; it used to work before because of the buggy code by sheer chance.
Rather than trying to fix it the right way (find a mapping from virtual to physical both before
and after RMM) let's just throw it away. Serial logger works just fine for the early init logging.
A bunch of things are now printed a bit more compactly or without
interleaving of logs on a single line. Also a bunch of not that useful
logs are no longer printed by default at all.
Changed validate_region function to return PageSpan instead of a tuple. All the code
using validate_region function was updated to use PageSpan as well.
Chunk size of the scheme (`size_of::<ITimerScheme>() == 0`) was being
provided instead of `size_of::<ITimerSpec>()`. This resulted in a kernel
panic (division by zero) when kread was called for this scheme.
Signed-off-by: Anhad Singh <andypython@protonmail.com>
Before it was first add_ref'ed by `borrow_frame_enforce_rw_allocated`,
manually and then by `allocated_shared_one_page`.
Now it is only done by `borrow_frame_enforce_rw_allocated` and does not
get unref-ed as take() is called on the returned `RaiiFrame`.
Now the page is manually mapped and an `Allocated` type grant is
constructed (synonymous to `MAP_PRIVATE`). Before by using `allocated_shared_one_page`
an `AllocatedShared` provided grant was constructed (synonymous to
`MAP_SHARED`), which was wrong as the TCB would've not got CoW-ed
after fork(), making the Tcb malformed.
Signed-off-by: Anhad Singh <andypython@protonmail.com>
* Add documentation and more code comments to `src/context/switch.rs`
* Eliminate a `context.wake.expect(...)` where `context.wake.is_some()`
* Eliminate a TODO item about updating contexts' timestamps in `switch_to`
* Eliminate a dangling `else { continue }` at the end of the loop that iterates
on contexts
* Mention the need to have `nasm` available on the PATH in the README
* Replace the deprecated `hide_parse_errors` by `show_parse_errors` in `rustfmt.toml`
* Mark unused variables in `src/scheme/proc.rs`
The new helpers remove a lot of boilerplace. Unfortunately some rough edges still remain (in particular issue gh#12 which renders interrupts() helper useless)
RISC-V convention marks PDE with no read/write/execute, so we can't have none of this flags set there. Remove their setting from PDE handling code and instead set them as appropriate in arch-specific defaults.
Also enable both readonly and readwrite flags to be non-zero (as long as their intersection completely masks both of them), as required for RISC-V PTE handling.
Before this commit, RMM assumed base physical address was presented in PTE as is, i.e. physical page address was shifted exactly to PAGE_SHIFT, so physical address can be extracted from PTE by simply masking off some bits and can be placed in PTE by simple addition/OR.
This is not the case for RISC-V which has 4Kb base page so 12 bits PAGE_SHIFT, yet physical page address is only shifted 10 bits in PTE.
This commit removes this assumption.
NOTE: This commit changes meaning of constants:
* ENTRY_ADDRESS_SIZE from "total physical size in bytes" to "total physical size in PAGES"
* ENTRY_ADDRESS_MASK from "mask of physical bits in PTE" to "mask of physical bits starting at bit 0"
This is required for RISC-V. Privileged spec says:
> For non-leaf PTEs, the D, A, and U bits are reserved for future standard use.
> Until their use is defined by a standard extension, they must be cleared by software
> for forward compatibility.
QEMU fails address translation if it sees any of these flags set on non-leaf page entry.
I tested compiling the kernel with a more recent rustc, and this
triggered UB. Don't know for sure why there was an invalid entry passed
to the kernel, but it did.
This would allow logd to write all received log messages to debug:
without causing an infinite loop when it at the same time reads all
messages in sys:log.
Diagonal as in
Thread A Thread B
Send to B Send to A
Wait for B Wait for A.
I haven't reproduced this bug, but this should fix it, since the wait
steps now checks the local percpu block for pending TLB shootdowns onto
itself.
This is not a perfect solution, since the kernel technically already has
all the information it needs to synchronize this between the kill()
invocation and the time WaitCondition::wait is called, but it should not
have any noticeable performance impact.
All options are already set by ../recipe.toml, and removing this file
will make Cargo stop complaining about missing -Zbuild-std when for
example compiling RMM for the host arch.
This fixes a possible file table leak if a filetable contains an fd
referring (strongly) to the filetable itself. Now, it will automatically
be downgraded to a weak reference after it becomes a context's active
file table.
TODO: maybe support consuming a context's filetable to get a strong
reference back, provided it's exclusively owned.
This fixes a very rare (requires that an altreloc span two pages) kernel
crash where it writes past the region that was remapped as RWX, thus
triggering an in-kernel "wrote without write access" page fault.
Additionally, this fixes the logic for unavailable features. It fixes
the short-circuit that would forget to remap the memory back from RWX to
R-X. NOP bytes are also now replaced with multi-byte NOP regardless of
whether the CPU supports the given feature.
The code incorrectly documents and states that SS and RSP are
conditionally pushed, based on whether the privilege level changed. This
is correct on IA-32, but wrong on x86-64.
Previously trying to funmap an address after the start of the grant
would result in an overflow only caught when compiling the kernel with
debug assertions enabled.
This way the NULL page can stay unmapped and the bootstrap code can
avoid UB when reading the initfs header (which is at the start of the
bootstrap code.
Technically there should be an atomic and nonatomic set type, but perf
should be the same, as the nonatomic one would only be used when setting
the sched_affinity.
The spin crate considers it UB to call force_write_unlock while there is
a threads trying to obtain a read lock on the same rwlock.
This also switches to the spinning_lot crate for the context rwlock as
it has support for a write guard keeping a reference to the rwlock using
an Arc instead of a reference.
The memoffset crate requires const_refs_to_cell to work in const
contexts. This feature has some known issues around it's semantics. The
offset_of feature however is currently on track for stabilization.
Since these are stored in arrays that are typically 512 KiB each
(i.e. which in turn typically store metadata for 128 MiB), they will
certainly be a page-size multiple, and large contiguous allocations are
allowed at least at boot time. Not only is the linked-list allocator
less efficient for that, but this change will also help reduce TLB
overhead.
This might not be the most ideal solution, since a GiB grant can be
EBUSY blocked if a single page is used by an indefinitely-blocking
UserScheme. But, the alternatives would impose lots of additional
complexity, such as increasing the PageInfo size, adding more
refcounting arrays to grants, etc.
This enables proper CoW ^ shared enforcement, so that e.g. acquiring a
new CoW page from already shared memory, or vice versa, will enforce
that they aren't both simultaneously.
Since clone is no longer exists as a syscall, it makes little sense to
manage vfork in the kernel. Implementing vfork in userspace would still
be possible.
- Contexts for other CPUs will never be evaluated for switch
- Running contexts will never be evaluated for switch
- Arc::clone is only called for previous and next context when there is a context to switch to
- Lots of cleanup to the switch function
This fixes file descriptor leaks. Suppose relibc is just about to set
the address space. For this, it needs to write the address space fd to
the selection fd. To avoid having to close them in the kernel, it rather
memorizes what the file descriptors refer to internally, and then do the
actual operation when they are gone, i.e. when closing.
Turns out the problem all along was that the ActivePageTable was never
dropped in usermode_bootstrap. So as soon as any other hardware thread
tried to do page table business, it deadlocked!
This is so that any process can use pointers to ACPI tables, since they
now point to the universally-accessible KERNEL_OFFSET+physaddr virtual
addresses.
Currently, this uses a relatively naive method of simply scanning the
512 entries for the PRESENT flag. But, unless the optimizer cannot, it
can be reduced to calculating the bitwise OR of every entry and then
checking that.
If this turns out to be too slow, which it might be when unmapping lots
of pages, then we can (1) either fall back to using a counter like the
old paging code did, or even better (2) use the now-1:1 grants tree to
check if it became empty. Putting the grants code in RMM might be
suboptimal, so instead we can add "unmap_range" and have the kernel
paging code take the offset of the next grant, if any, and then possibly
unmap entire P1s/P2s/P3s -- whatever is in the page tables within that
range.
Note that I am fairly certain that method (1) was the cause of the
visually notorious orbital memory corruption bug.
This does the same as the previous MR, but fixes the issue where the
parent process got the mapping (and at the wrong address) and not the
child process.
Multi-core is slightly broken when using the latest version of spin
(0.9.2). I believe this is because Once used to do SeqCst loads/stores
everywhere, which might have made any possible data race much harder to
come by.
However, since not all platforms will allow the entire physical address
space to be simultaneously mapped to part of the virtual address space,
we may still require some dynamic mapping.
Namely, the global allocator API in Rust, actually only returns a null
pointer on failure, rather than wrapping it in a Result, which AllocRef
does. Since Box::from_raw(null) is direct UB, this can in theory lead to
very strange behavior.
Note that this is very preliminary, and I merely got my already freezing
kernel branch not to triple fault, but I would probably apply this patch
to upstream.
What is changed here, is that rather than relying on recursive mapping
for accessing page table frames, it now uses linear translation
(virt=phys+KERNEL_OFFSET). The only problem is that the paging code now
makes assumptions that the entire physical address space remains mapped,
which is not necessarily the case on x86_64 architecturally, even though
systems with RAM more than a PML4 are very rare. We'd probably lazily
(but linearly) map physical address space using huge pages.
Additionally, because it turned out to be infeasible to rely on
link-time constants in global_asm! code, I have also converted the
interrupt handlers to naked fns. This removes the proc-macro-reliant
"paste" dependency, but inserts a tiny ud2 at the end of every ISR.
Previously context::switch used compare_and_swap for acquiring the
global context switch lock, but given its deprecation in more recent
Rust versions, it has been replaced with compare_exchange_weak (which
can be further optimized on some architectures).
It also replaces panic!() with abort() in switch_finish_hook, because
unwinding from assembly is not that fun.
This also removes the need to do another semi-expensive remap when
cloning processes, since the KPCRs (for kernel TLS) are no longer stored
in the user PML4.
We may also want to do this with the MADT and the HPET tables, and let
user drivers specify what the tables mean independent of ACPI. That is,
adding an interface for registering new CPUs, and specifying the main
timer IRQ.
Currently, there are some things that need to be set up by userspace
that the kernel previously did. These include telling firmware when the
I/O APIC is used, and most importantly, shutting down the system.
The former is not particularly important, but for the latter I think
that we could implement this using a "shutdown pipe". Essentially it
will be a file that triggers an event shutting down, which would be used
to notify to acpid that the kernel is requesting a shutdown.
This allows schemes to avoid checking the length against zero before
constructing a slice from pointer+len that the kernel gave.
Additionally, the address is now non-canonical on x86, meaning that
userspace will fail instead of continuing with UB, if they would ever
forget to check the length.
This is done by making sure that when empty() is called on a context,
the grants Arc will be replaced with a new unused Arc, hence
decrementing the refcount. Previously this was only done when the
context was actually reaped, but since there is no guarantee as far as I
am aware about when this must happen, the grants could be completely
leaked, leading to the error.
This allows schemes to avoid checking the length against zero before
constructing a slice from pointer+len that the kernel gave.
Additionally, the address is now non-canonical on x86, meaning that
userspace will fail instead of continuing with UB, if they would ever
forget to check the length.
Previously, the kernel used the regular FS segment for Thread-Local
Storage. The problem however, is that userspace code also uses FS for
TLS, meaning that the kernel would have to switch the FS segment between
user and kernel, _upon every syscall_. This is obviously suboptimal for
performance (especially with fast syscalls such as futex, nanosleep, or
yield).
I had to search LLVM for hours, just to find out that the insertion of
the memory load with FS was actually done in the linker, so I added a
flag for that.
I haven't done any proper benchmarking, but the boot process seems to
have gotten much faster!
In order words, it swaps gs both directly at the start of the syscall
handler, then swaps it back, and the at the end of the syscall handler.
I cannot tell for sure why this is necessary, but probably since some
interrupt handler will execute swapgs in the wrong order or something.
The reason for these types of rewrites, is that more recent Rust
compilers have started to deprecate naked functions that consist of more
than only a single asm block, as they can trigger all sorts of UB.
Previously there was a triple fault, due to a combination of reasons
(e.g. rsp and rbp being ordered in the struct and in the assembly).
Now, the locks will be held __all the way until the new context__ has
been switched to, which completely eliminates any possibility that the
"pcid fault" originates here.
While I am unsure whether this will work, this could also be an
opportunity to be able to remove CONTEXT_SWITCH_LOCK fully.
This is due to a warning in more recent compilers, which forbid anything
but a single inline assembly block, in naked functions. It does
unfortunately triple fault right now, but I hope I may be able to fix it
soon.
So, when I first introduced io_uring, it was not compiled with the
`multi_core` kernel feature, mainly to make development easier (I
thought). However, since io_uring allows multiple simultaneous system
calls, we cannot longer make the in-kernel contexts block, for example
when receiving a message from a pipe, if there can be multiple such
requests simultaneously.
This has required me to change WaitCondition into allowing multiple
simultaneous tasks; although, it introduces a potential race condition:
since a future can only return Pending and not block directly before
releasing the lock (condvar logic), we need some way to make sure that
nothing happens after the context finds out that it has to wait, and the
actual waiting. If a message is pushed in between, and the waker is
called (Context::unblock), just before it was going to block itself,
then we miss the message, and potentially cause a deadlock.
Fortunately, in order to block and unblock contexts, we need to
exclusively lock the context. So, what we can do to ensure that waking
while running is no longer a no-op, is to introduce a "wake flag", which
is set only if the context is currently running, and Runnable.
But, this still caused all weird kinds of hard-to-debug problems, with
arbitrary CPU exceptions and possibly memory corruption. The reason for
this, is that the context switching logic uses really unsafe operations,
which is why context switching (at the moment) requires an exclusive
lock. Before this commit, it would modify the `running` field after the
lock had been released, which obviously can cause a data race, when the
regular context waker code that is run within a system call, locks the
context but not the global switching lock.
The solution was to make sure that the locks were held, all the way
until the actual switching, which was done in assembly. There can still
be a race condition here, since it modifies memory containing registers
after the lock has been released, even if it may be behind &mut on
another context, which can be UB, but it has not contributed to any
actual bugs... yet.
* I have not yet done that rigorous testing, but it appears to work well
enough, and I have not encountered the bug after like 10 tries.
This solves a bug, that allows processes in different address spaces to
be the target of a futex wakeup call, even though that process is in
another address space!
When mapping one (from) virtual address range to another (to) virtual
address range, be mindful of which mapper type to use for each range.
Before this, the same mapper type was used for both ranges. This meant
that if from and to were different (as in not both kernel virtual
addresses or user virtual addresses) then it would appear that either
from or to was not mapped previously and the kernel would panic.
Oddly, not specifying this or using aarch64-unknown-none (which would be
the default that cc gets from the TARGET environment variable) both
fail to invoke the appropriate compiler to build the asm code.
Using aarch64-unknown-redox works but shouldn't really be needed. This
is perhaps because of some odd arrangement of KTARGET, TARGET, the
installed prefix toolchain and the kernel target JSON spec.
The early_init asm code shall be replaced by a pure Rust bootloader
eventually so let's move with this for the moment.
This fixes a warning that may in the future become an error, about the
possibility for unaligned references, since the derive macros apparently
rely on creating references to fields. Unaligned references are direct
UB.
The cool thing here is that we're temporarily binary compatible with the
old stuff, so if anyone would use an old version of redox_syscall we can
easily find them with these prints.
Each architecture may have a different method to enable logging. Now
that can be customized with a function passed to the init_logger
function.
Also, provide a minimal x86_64 implementation.
This is the first commit where you can see logging coming from the log
crate.
Signed-off-by: Wren Turkal <wt@penguintechs.org>
This is the first step of integrating the log crate as the main way to
log messages from the kernel.
Also, reexport all log macros. This module should eventually be the
only logging API used in the kernel.
Signed-off-by: Wren Turkal <wt@penguintechs.org>
Because the way we were using inline assembly was technically incorrect
and breaking the laws of rust
This *finally* compiles. That doesn't mean it works!
This is safe because `Entry` is `#[repr(8)]` which is the minimum
alignment for qwords. Since the size of a qword is equal to that
alignment (8), they can also be borrowed from the array.
since this would require pcid to know the _PRT (PCI routing table) to
use and map the interrupt pins to the correct IRQs. xhcid is unaffected
by this though, since it uses MSI-X.
All ACPI handling will be done in userspace before the infrastructure
necessary would make sense (I don't think adding serde to the kernel
would be optimal, and how else would all of the ACPI namespace be
parsed?).
This patch fixes a deadlock in the sys: scheme that is triggered
when the iostat resource() is called in the same scope where the RwLock
protecting the scheme's handles is write-locked.
I believe this could cause a deadlock if a blocking I/O operation was
interrupted by a signal or otherwise, and decided to exit and close
all files. It's unlikely to happen, but it can happen nontheless. This
removes the mutex, but it's difficult to keep the code tidy. Hopefully
this is good enough.
Signals now cause an event, and there's a way to continue until the
next signal. I can see this being used for detection of `int3`
although I'm not entirely sure as it may prove being too late to stop
abortion of process.
This is a curious problem and it's really hard to solve it in a way
that doesn't feel hacky. On one hand, of course you want to be able to
modify and intercept what happens when you use a signal, right? On the
other hand, changes made to the context (especially singlestepping)
while a signal is handled (such as `SIGSTOP`) are not preserved since
the stack is restored after the signal handler was invoked.
I think what we have in this change makes sense anyway, as we don't
really want users modifying registers and other data in the default
signal behavior that occurs **in kernel mode**. Also trying to use
`PTRACE_SINGLESTEP` will set the singlestep flag only if in a
user-mode signal handler, else it will set it on the instruction after
the signal handling, which I guess makes sense since it can't affect
the kernel-mode code that runs the default handler.
I don't know. Help. Pls.
Since even a very basic ptrace can be nice to have, I thought I would split
the, perhaps rather big, ptrace project up in multiple PRs to make as few
changes as necessary in each. This PR contains the initial registry modifying
bits and only a very basic security measure. Letting this out to the community
should be good for spotting bugs and maybe getting some hype ;)
Most of this was generated by the absolutely extraordinary `cargo fix`
subcommand. There were still 2 errors and a few warnings to patch up,
but compared to the normal 600+ errors, I'd say the fixer did a damn
good job! I'm also amazed that I could still start the VM after this,
I half expected some kinds of runtime failure...
This allows booting with implementations that require them to be set separately.
Also, check for the availability of legacy-replacement mode and periodic interrupts before using hpet
I don't know if this was there for a reason, but it was making the dup()
fail with tcpd, and I don't seem this being handled specially in redoxfs
or anywhere else.
* Initial parser proof of concept
* Added better error handling to the parser
* Refactored into a better directory structure
* Parse package length
* Implemented named string, scope op
* Properly bounds checked namestring
* Fixed namestring regressions
* Started work parsing DefRegionOp. NB: As TermArg is not yet implemented, a bug is present parsing the address offset and length. Additionally, a bug was fixed in NameString
* Completed DefOpRegion implementation. TermArg remains unimplemented, stubbed out
* Implemented TermArg parsing
* Implemented integer parts of computational data
* Implemented defField, and associated FieldList. FieldElement still remains stubbed
* Implmenented FieldElement
* Implmenented named field
* Parsed DefMethod
* Parsed ToHexString
* Parsed ToBuffer
* Parsed both subtract and sizeof
* Fixed size bug in sizeof parsing
* Parsed Store, fixed a parse bug where Target should be a SuperName not a TermArg
* Parsed while
* Parsed LLess
* Parsed DerefOf
* Parse Index
* Parse increment
* Parse device
* Parse device
* Parsed create dword field
* Parsed if/else block
* Properly parsed Target, rendered an AST from existing parse code, and stubbed out MethodInvocation parser method
* Implemented deferred loading, and deferred method invocation parses
* Parsed Or
* Fixed a bunch of off-by-one errors. Shows what I get for copying code around
* Parsed Return
* Fixed a boolean logic error in the handling of the extended instruction namespace
* Added DefBuffer to ComputationalData
* Removed a temporary file
* Parsed ReservedField
* Parsed DefPackage, DefAnd, and ComputationalData::String
* Parsed DefMutex
* Parsed DefAlias and RevisionOp
* Parsed DebugObj
* Parsed DefRefOf
* Parsed type 6 opcodes
* Added ObjectReference and DDBHandle to DataRefObj parsing
* Parsed DefVarPackage, in both Type2OpCode, and in DataObj
* Parsed DefBankField
* Parsed AccessField
* Parsed ConnectField
* Parsed CreateBitField
* Parsed CreateByteField
* Parsed CreateWordField
* Parsed CreateQWordField
* Parsed CreateField
* Parsed DefDataRegion
* Parsed DefEvent
* Parsed IndexField
* Parsed DefPowerRes
* Parsed DefProcessor
* Parsed DefThermalZone:
* Parsed ExtendedAccessField
* Parsed DefBreak, DefBreakPoint, DefContinue and DefNoop (all type 1 opcodes with no parameters and one byte)
* Parsed DefFatal
* Parsed DefLoad
* Parsed DefNotify
* Parsed DefRelease
* Parsed DefReset
* Parsed DefSignal
* Parsed DefSleep
* Parsed DefStall
* Parsed DefUnload
* Parsed DefAcquire
* Parsed DefAnd
* Parsed DefConcat
* Parsed ConcatRes
* Switched Concat and ConcatRes opcodes
* Parsed CondRefOf
* Parsed DefDecrement and DefCopyObject
* Parsed DefDivide, fixed length calculation bug in a bunch of parse routines
* Parsed DefFindSetLeftBit
* Parsed DefFindSetRightBit
* Parsed DefFromBCD
* Parsed DefLAnd
* Parsed DefLGreater
* Parsed LNot
* Parsed DefLOr
* Parsed DefLoadTable
* Parsed DefMatch
* Parsed DefMid
* Parsed DefMod
* Parsed DefMultiply
* Parsed DefNAnd
* Parsed DefNOr
* Parsed DefNot
* Parsed DefObjectType
* Parsed DefShiftLeft and DefShiftRight
* Parsed DefTimer
* Parsed DefToBCD, DefToDecimalString, DefToInteger and DefToString
* Parsed DefXor
* Parsed DefWait
* Implemented a parser, abstract syntax tree, and basic infrastructure for the AML subsystem of the ACPI module. The entire AML grammar is parsed and placed into an abstract syntax tree, with one exception: method invocations, rather than parsing, defer the load until later on in the process, due to the way the grammar works.
Still to be done:
- Refactor the code: a lot of the parser is very repetitive, and could easily be refactored with the aid of macros. This would reduce the length and improve legibility, though not affect function.
- More rigorous testing of parser: the parser has, thus far, only been tested on the DSDT in QEMU. There may be bugs present that are hidden.
- Parse the SSDTs: the SSDTs should be parsed after the DSDT, and contain more AML bytecode to be treated as modifying the same namespace. Adding this would be simple, though not urgent.
- Transform the AST into a concrete executable tree: the CET is what will hold all information necessary to execute control methods and evaluate namespace objects. While this could be done in the AST, due to the way AML is laid out this would be very inefficient and require a lot of repetitive transformations every time something is to be executed. Therefore, perform the transformations upfront.
- Parse the deferred loads, and the method invocations contained within: Once the AST has been rendered into a CET, sufficient information will be present to parse method invocations and add those to the namespace.
- Bytecode interpreter: Once the CET has been finalized with method invocation parsing, it can then be called and executed.
- Control method executor: this should walk the namespace, locating the relevant control method, then calling the interpreter on it.
- Namespace enumerator: the executor shall use this to walk the namespace, and it should also be publicly accessible to allow outside code to determine what devices are present in the system.
- Memory accessor API: ACPI AML has a concept of memory access in certain device domains - for example, the PCI BAR registers. These are all device specific offsets, therefore device drivers, or more accurately bus drivers, should be capable of installing handlers to manage this memory access.
- CET concatenation: The DSDT and SSDTs all affect the same namespace, therefore concatenating the resulting trees should be possible.
- Type checking: some operations in AML are typed. This should be handled at tree transformation time or earlier, and could indeed done in the parse step with some modification to the parser. This is currently not the case.
* Initial parser proof of concept
* Added better error handling to the parser
* Refactored into a better directory structure
* Parse package length
* Implemented named string, scope op
* Properly bounds checked namestring
* Fixed namestring regressions
* Started work parsing DefRegionOp. NB: As TermArg is not yet implemented, a bug is present parsing the address offset and length. Additionally, a bug was fixed in NameString
* Completed DefOpRegion implementation. TermArg remains unimplemented, stubbed out
* Implemented TermArg parsing
* Implemented integer parts of computational data
* Implemented defField, and associated FieldList. FieldElement still remains stubbed
* Implmenented FieldElement
* Implmenented named field
* Parsed DefMethod
* Parsed ToHexString
* Parsed ToBuffer
* Parsed both subtract and sizeof
* Fixed size bug in sizeof parsing
* Parsed Store, fixed a parse bug where Target should be a SuperName not a TermArg
* Parsed while
* Parsed LLess
* Parsed DerefOf
* Parse Index
* Parse increment
* Parse device
* Parse device
* Parsed create dword field
* Parsed if/else block
* Properly parsed Target, rendered an AST from existing parse code, and stubbed out MethodInvocation parser method
* Implemented deferred loading, and deferred method invocation parses
* Parsed Or
* Fixed a bunch of off-by-one errors. Shows what I get for copying code around
* Parsed Return
* Fixed a boolean logic error in the handling of the extended instruction namespace
* Added DefBuffer to ComputationalData
* Removed a temporary file
* Parsed ReservedField
* Parsed DefPackage, DefAnd, and ComputationalData::String
* Parsed DefMutex
* Parsed DefAlias and RevisionOp
* Parsed DebugObj
* Parsed DefRefOf
* Parsed type 6 opcodes
* Added ObjectReference and DDBHandle to DataRefObj parsing
* Parsed DefVarPackage, in both Type2OpCode, and in DataObj
* Parsed DefBankField
* Parsed AccessField
* Parsed ConnectField
* Parsed CreateBitField
* Parsed CreateByteField
* Parsed CreateWordField
* Parsed CreateQWordField
* Parsed CreateField
* Parsed DefDataRegion
* Parsed DefEvent
* Parsed IndexField
* Parsed DefPowerRes
* Parsed DefProcessor
* Parsed DefThermalZone:
* Parsed ExtendedAccessField
* Parsed DefBreak, DefBreakPoint, DefContinue and DefNoop (all type 1 opcodes with no parameters and one byte)
* Parsed DefFatal
* Parsed DefLoad
* Parsed DefNotify
* Parsed DefRelease
* Parsed DefReset
* Parsed DefSignal
* Parsed DefSleep
* Parsed DefStall
* Parsed DefUnload
* Parsed DefAcquire
* Parsed DefAnd
* Parsed DefConcat
* Parsed ConcatRes
* Switched Concat and ConcatRes opcodes
* Parsed CondRefOf
* Parsed DefDecrement and DefCopyObject
* Parsed DefDivide, fixed length calculation bug in a bunch of parse routines
* Parsed DefFindSetLeftBit
* Parsed DefFindSetRightBit
* Parsed DefFromBCD
* Parsed DefLAnd
* Parsed DefLGreater
* Parsed LNot
* Parsed DefLOr
* Parsed DefLoadTable
* Parsed DefMatch
* Parsed DefMid
* Parsed DefMod
* Parsed DefMultiply
* Parsed DefNAnd
* Parsed DefNOr
* Parsed DefNot
* Parsed DefObjectType
* Parsed DefShiftLeft and DefShiftRight
* Parsed DefTimer
* Parsed DefToBCD, DefToDecimalString, DefToInteger and DefToString
* Parsed DefXor
* Parsed DefWait
* Implemented a parser, abstract syntax tree, and basic infrastructure for the AML subsystem of the ACPI module. The entire AML grammar is parsed and placed into an abstract syntax tree, with one exception: method invocations, rather than parsing, defer the load until later on in the process, due to the way the grammar works.
Still to be done:
- Refactor the code: a lot of the parser is very repetitive, and could easily be refactored with the aid of macros. This would reduce the length and improve legibility, though not affect function.
- More rigorous testing of parser: the parser has, thus far, only been tested on the DSDT in QEMU. There may be bugs present that are hidden.
- Parse the SSDTs: the SSDTs should be parsed after the DSDT, and contain more AML bytecode to be treated as modifying the same namespace. Adding this would be simple, though not urgent.
- Transform the AST into a concrete executable tree: the CET is what will hold all information necessary to execute control methods and evaluate namespace objects. While this could be done in the AST, due to the way AML is laid out this would be very inefficient and require a lot of repetitive transformations every time something is to be executed. Therefore, perform the transformations upfront.
- Parse the deferred loads, and the method invocations contained within: Once the AST has been rendered into a CET, sufficient information will be present to parse method invocations and add those to the namespace.
- Bytecode interpreter: Once the CET has been finalized with method invocation parsing, it can then be called and executed.
- Control method executor: this should walk the namespace, locating the relevant control method, then calling the interpreter on it.
- Namespace enumerator: the executor shall use this to walk the namespace, and it should also be publicly accessible to allow outside code to determine what devices are present in the system.
- Memory accessor API: ACPI AML has a concept of memory access in certain device domains - for example, the PCI BAR registers. These are all device specific offsets, therefore device drivers, or more accurately bus drivers, should be capable of installing handlers to manage this memory access.
- CET concatenation: The DSDT and SSDTs all affect the same namespace, therefore concatenating the resulting trees should be possible.
- Type checking: some operations in AML are typed. This should be handled at tree transformation time or earlier, and could indeed done in the parse step with some modification to the parser. This is currently not the case.
* Partial refactor of AML code
* Further refactoring
* Fully refactored type 2 opcode selector
* Refactored type 6 opcode selector
* Further refactored Type 2 opcode parsing
* Implemented basic infrastructure in order to render the AST down to a namespace object
* Resolved scopes into the namespace
* Put OpRegion into namespace
* Rendered field parsing to the namespace object
* Methods now placed in namespace
* Moved DefName into the namespace
* Moved packages into the namespace
* Converted shutdown sequence to use AML parser
* Moved shutdown over to use AML parsing fully
* Removed the no longer needed DSDT code
* Better messages on unmapping failure
* Disable preemption until paging bug is fixed
* Refactor kernel mapping so that symbol table is mapped
* Add symbol lookup (still very WIP)
* Improve method of getting symbol name
* Reenable preemption
* Demangle symbols
* Fix overallocation
* Remove tilde files
PIT interrupt should context switch or else all of redox crashes.
This will fix programs like the Snake game crashing all of Redox.
A global AtomicUSize counter was added, and a line to switch contexts
on every 10 PIT interrupts in irq.rs.
The default implementation of the memcpy, memmove, memset and memcmp
functions in the kernel file `extern.rs` uses a naive implementation
by copying, assigning or comparing bytes ony by one. This can be slow.
This commit proposes a reimplementation of those functions by copying,
assigning or comparing in group of 8 bytes by using the u64 type and
its respective pointers instead of u8. Alternative version for 32-bit
architectures are also supplied for future compatibility with x86.
Both version first copy whatever they can with wide word types. The
tail, i.e. the final few bytes that do not fit in a dword or qword
are then copied byte by byte.
Here is a comparison of copying 64kiB (65536 bytes) on stack:
x86_64-unknown-linux-gnu: (64-bit)
| naive (ns) | fast (ns) | speedup (x)
-------|------------|-----------|------------
memcpy | 204430 | 32994 | ~6.20
memmove| 202540 | 33186 | ~6.10
memset | 163391 | 23884 | ~6.84
memcmp | 205663 | 34385 | ~5.98
i686-unknown-linux-gnu: (32-bit)
| naive (ns) | fast (ns) | speedup (x)
-------|------------|-----------|------------
memcpy | 206297 | 66858 | ~3.09
memmove| 204576 | 70326 | ~2.91
memset | 165599 | 50227 | ~3.30
memcmp | 204262 | 70572 | ~2.89
Copying on the heap behaves simmilarly.
All tests performed on Intel i5 6600K (4x4.2GHz),
ArchLinux Kernel 4.8.12-3 x86_64.
There was a bug at the `initfs` generation which made the listing of the contents of the `initfs:`subdirectories impossible from the command line.
The subdirectory listing data had the full paths of the files (like `bin/ahcid\nbin/bgad\n...`) when it should just only be the names of the files (`ahcid\nbgad\n...`)
- Implemented a global variable, ACPI_TABLE, behind a mutex, which contains the ACPI information pertinent to the rest of the kernel, currently solely containing a pointer to the FADT.
- Split device initialization into two categories - "core" devices, such as the PIC and local APIC, necessary for initializing the rest of the kernel, and "non-core" devices such as serial and RTC, which are to be initialized last.
- Checked for the presence of the century register, and consequentially read from, in the RTC code, now factored into the date calculation. The location of the register is pulled from the "century" field in the FADT.
- Modified page unmapping in the ACPI code, such that any tables to be stored globally (currently only the FADT) are not unmapped after reading, such that they can be stored in globally accessible pointers without causing page faults.
A kernel panic occurs on some CPUs (notably qemu-system-x86_64) when
calling rust-cpuid's get_extended_function_info. This is fixed upstream
in commit c3ebfc553cdff98d19d29777fd85c4f9182bfb66 but has yet to make
it crates.io.
The panic can by triggered by running "ls sys:" from ion and causes
redox to become unresponsive.
* Fire up multiple processors
* Use IPIs to wake up secondary processors
* Much better exception information
* Modifications to show more information on fault
* WIP: Use real libstd
* Add TLS (not complete)
* Add random function, export getpid, cleanup
* Do not spin APs until new context
* Update rust
* Update rust
* Use rd/wrfsbase
* Implement TLS
* Implement compiler builtins and update rust
* Update rust
* Back to Redox libstd
* Update rust
* Rewriting network functions
* Add buffer to dup
Fix non-blocking handling by triggering once on enabling events to read to EOF
* Modifications for UDP API
* Implement TCP client side
* Add active close
* Add DMAR parser
* Implement basic TCP listening. Need to improve the state machine
* Reduce debugging
* Fixes for close procedure
* Updates to fix path processing in libstd
* Port previous ethernet scheme
* Add ipd
* Fix initfs rebuilds, use QEMU user networking addresses in ipd
* Add tcp/udp, netutils, dns, and network config
* Add fsync to network driver
* Add dns, router, subnet by default
* Fix e1000 driver. Make ethernet and IP non-blocking to avoid deadlocks
* Add orbital server, WIP
* Add futex
* Add orbutils and orbital
* Update libstd, orbutils, and orbital
Move ANSI key encoding to vesad
* Add orbital assets
* Update orbital
* Update to add login manager
* Add blocking primitives, block for most things except waitpid, update orbital
* Wait in waitpid and IRQ, improvements for other waits
* Fevent in root scheme
* WIP: Switch to using fevent
* Reorganize
* Event based e1000d driver
* Superuser-only access to some network schemes, display, and disk
* Superuser root and irq schemes
* Fix orbital
This minicommit introduces two newtpyes, `Physical` and `Virtual`,
respectively. These serves as a way to segregate the different forms of
addresses to avoid the issues we had in the old kernel.
# Porting the core Redox kernel to arm AArch64: An outline
## Intro
This document is [my](https://github.com/raw-bin) attempt at:
* Capturing thinking on the work needed for a core Redox kernel port
* Sharing progress with the community as things evolve
* Creating a template that can be used for ports to other architectures
Core Redox kernel means everything needed to get to a non-graphical console-only multi-user shell.
Only the 64-bit execution state (AArch64) with the 64-bit instruction set architecture (A64) shall be supported for the moment. For more background/context read [this](https://developer.arm.com/products/architecture/a-profile/docs/den0024/latest/introduction).
This document is intended to be kept *live*. It will be updated to reflect the current state of work and any feedback received.
It is hard~futile to come up with a strict sequence of work for such ports but this document is a reasonable template to follow.
## Intended target platform
The primary focus is on [qemu's virt machine platform emulation for the AArch64 architecture](https://github.com/qemu/qemu/blob/master/hw/arm/virt.c#L127).
Targeting a virtual platform is a convenient way to bring up the mechanics of architectural support and makes the jump to silicon easier. The preferred boot chain for AArch64 (explained later) is well supported on this platform and boot-over-tftp from localhost makes the debug cycle very efficient.
Once the core kernel port is complete a similar follow on document will be created that is dedicated to silicon bring-up.
| [Linux kernel boot protocol for AArch64](https://www.kernel.org/doc/Documentation/arm64/booting.txt) | The linked document describes assumptions made from the bootloader which are field tested and worthwhile to have for Redox an AArch64. <br/> The intent is to consider most of the document except anything tied to the Linux kernel itself. |
| [Flattened Device Tree](https://elinux.org/Device_Tree_Reference) | FDT binary blobs supplied by the bootloader shall provide the Redox kernel with misc platform \{memory, interrupt, devicemem} maps. Qemu's virt machine platform synthetically creates an FDT blob at a specific address which is very handy. |
| [ARM Trusted Firmware (TF-A)](https://github.com/ARM-software/arm-trusted-firmware) | TF-A is a de-facto standard reference firmware implementation and proven in the field. <br/> TF-A runs post power-on on Armv8-A implementations and eventually hands off to further stages of the boot flow.<br />For qemu's virt machine platform, it is essentially absent but I mean to rely on it heavily for silicon bring up hence mentioning it here. |
| [u-boot](https://www.denx.de/wiki/U-Boot) | u-boot will handle early console access, media access for fetching redox kernel images from non-volatile storage/misc disk subsystems/off the network. <br /> u-boot supports loading EFI applications. If EFI support to AArch64 Redox is added in the future that should essentially work out of the box. <br /> u-boot will load redox and FDT binary blobs into RAM and jump to the redox kernel. |
| Redox early-init stub | For AArch64, the redox kernel will contain an A64 assembly stub that will setup the MMU from scratch. This is akin to the [x86_64 redox bootloader](https://github.com/redox-os/bootloader/blob/master/x86_64/startup-x86_64.asm). <br /> This stub sets up identity maps for MMU initialization, maps the kernel image itself as well as the device memory for the UART console. At present this stub shall be a part of the kernel itself for simplicity. |
| Redox kstart entry | The early init stub hands off here. kstart will then re-init the MMU more comprehensively. |
## Supported devices
The following devices shall be supported. All necessary information specific to these devices will be provided to the redox kernel by the platform specific FDT binary blob.
| [Generic Interrupt Controller v2](https://developer.arm.com/products/architecture/a-profile/docs/ihi0048/b/arm-generic-interrupt-controller-architecture-version-20-architecture-specification) | The GIC is an Arm-v8A architectural element and is supported by all architecturally compliant processor implementations. GICv2 is supported by qemu's virt machine emulation and most subsequent GIC implementations are backward compatible to GICv2. |
| [Generic Timer](http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0500d/BGBBIJCB.html) | The Generic Timer Architecture is an Arm-v8A architectural element and is implemented by all compliant processor implementations. It is supported by qemu. |
| [PrimeCell UART PL011](http://infocenter.arm.com/help/topic/com.arm.doc.ddi0183f/DDI0183.pdf) | The PL011 UART is supported by qemu and most ARM systems. |
| Redox AArch64 toolchain | Create an usable redox AArch64 toolchain specification | Done | Using this JSON spec in isolated tests produces valid AArch64 soft float code |
| Stubbed kernel image | Stub out AArch64 kernel support using the existing x86_64 arch code as a template <br /> Modify redox kernel build glue and work iteratively to get a linkable (non-functional) image | Not done yet | |
| Boot flow | Create a self hosted u-boot -> redox kernel workflow <br /> Should obtain the stubbed image from a local TFTP server, load it into RAM and jump to it | Not done yet | |
| GDB Debug flow | Create a debug workflow centered around qemu's GDB stub <br /> This should allow connecting to qemu's GDB stub and debug u-boot/redox stub via a GDB client and single stepping through code | Not done yet | |
| Verify Redox entry | Verify that control reaches the redox kernel from u-boot | Not done yet | |
| AArch64 early init stub | Add support for raw asm code for early AArch64 init in the redox kernel <br /> Verify that this code is located appropriately in the link map and that control reaches this code from u-boot | Not done yet | |
| Basic DTB support | Integrate the [device_tree crate](https://mbr.github.io/device_tree-rs/device_tree/) <br /> Use the crate to access the qemu supplied DTB image and extract the memory map | Not done yet | |
| Basic UART support | Use the device_tree crate to get the UART address from the DTB image and set up the initial console <br /> This is a polling mode only setup | Not done yet | |
| Initial MMU support | Implement initial MMU support in the early init stub <br /> This forces the MMU into a clean state overriding any bootloader specific setup <br /> Create an identity map for MMU init <br /> Create a mapping for the kernel image <br /> Create a mapping for any devices needed at this stage (UART) | Not done yet | |
| kmain entry | Verify that kmain entry works post early MMU init | Not done yet | |
| Basic Redox MMU support | Get Redox to create a final set of mappings for everything <br /> Verify that this works as expected | Not done yet | |
| Basic libc support | Flesh out a basic set of libc calls as required for simple user-land apps | Not done yet | |
| userspace_init entry | Verify user-space entry and /sbin/init invocation | Not done yet | |
| Basic Interrupt controller support | Add a GIC driver <br /> Verify functionality | Not done yet | |
| Basic Timer support | Add a Generic Timer driver <br /> Verify functionality | Not done yet | |
| UART interrupt support | Add support for UART interrupts | Not done yet | |
| Task context switch support | Add context switching support <br /> Verify functionality | Not done yet | |
| Login shell | Iteratively add and verify multi-user login shell support | Not done yet | |
| Publish development branch on github | Work with the community to post work done after employer approval | Not done yet | |
| Break out the Bubbly | Drink copious quantities of alcohol to celebrate | Not done yet | |
| Silicon bring-up | Plan silicon bring-up | Not done yet | |
* [`nasm`](https://nasm.us/) needs to be available on the PATH at build time.
## Building The Documentation
Use this command:
```sh
cargo doc --open --target x86_64-unknown-none
```
## Debugging
### QEMU
Running [QEMU](https://www.qemu.org) with the `-s` flag will set up QEMU to listen on port `1234` for a GDB client to connect to it. To debug the redox kernel run.
```sh
make qemu gdb=yes
```
This will start a virtual machine with and listen on port `1234` for a GDB or LLDB client.
### GDB
If you are going to use [GDB](https://www.gnu.org/software/gdb/), run these commands to load debug symbols and connect to your running kernel:
```
(gdb) symbol-file build/kernel.sym
(gdb) target remote localhost:1234
```
### LLDB
If you are going to use [LLDB](https://lldb.llvm.org/), run these commands to start debugging:
After connecting to your kernel you can set some interesting breakpoints and `continue`
the process. See your debuggers man page for more information on useful commands to run.
## Notes
- Always use `foo.get(n)` instead of `foo[n]` and try to cover for the possibility of `Option::None`. Doing the regular way may work fine for applications, but never in the kernel. No possible panics should ever exist in kernel space, because then the whole OS would just stop working.
- If you receive a kernel panic in QEMU, use `pkill qemu-system` to kill the frozen QEMU process.
## How To Contribute
To learn how to contribute to this system component you need to read the following document:
To learn how to do development with this system component inside the Redox build system you need to read the [Build System](https://doc.redox-os.org/book/build-system-reference.html) and [Coding and Building](https://doc.redox-os.org/book/coding-and-building.html) pages.
### How To Build
To build this system component you need to download the Redox build system, you can learn how to do it on the [Building Redox](https://doc.redox-os.org/book/podman-build.html) page.
This is necessary because they only work with cross-compilation to a Redox virtual machine, but you can do some testing from Linux.
## Funding - _Unix-style Signals and Process Management_
This project is funded through [NGI Zero Core](https://nlnet.nl/core), a fund established by [NLnet](https://nlnet.nl) with financial support from the European Commission's [Next Generation Internet](https://ngi.eu) program. Learn more at the [NLnet project page](https://nlnet.nl/project/RedoxOS-Signals).
[<img src="https://nlnet.nl/logo/banner.png" alt="NLnet foundation logo" width="20%" />](https://nlnet.nl)
[<img src="https://nlnet.nl/image/logos/NGI0_tag.svg" alt="NGI Zero Logo" width="20%" />](https://nlnet.nl/core)
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
vasilito
R Aadarsh