From 1c469ac1edcb6dc9c6a4bed1dec0cec1341936d0 Mon Sep 17 00:00:00 2001 From: vasilito Date: Tue, 30 Jun 2026 14:28:44 +0300 Subject: [PATCH] Add documentation: README, architecture plan, config reference --- HIPERISO_PLAN.md | 1418 ++++++++++++++++++++++++++++++++++++++++++++++ README.md | 408 +++++++++++++ config/README.md | 104 ++++ 3 files changed, 1930 insertions(+) create mode 100644 HIPERISO_PLAN.md create mode 100644 README.md create mode 100644 config/README.md diff --git a/HIPERISO_PLAN.md b/HIPERISO_PLAN.md new file mode 100644 index 0000000..5d2541a --- /dev/null +++ b/HIPERISO_PLAN.md @@ -0,0 +1,1418 @@ +# hiperiso — Architecture & Implementation Plan (Full Ventoy Parity) + +> A hypervisor-based ISO boot tool with full bootlogging — **all Ventoy features, +> but using KVM+QEMU instead of OS-level injection.** + +--- + +## 1. Executive Summary + +**hiperiso** boots ISO files from a USB drive — like Ventoy — but instead of +injecting hooks into the booted OS, it runs a **thin hypervisor** (KVM + QEMU on +a minimal Linux kernel) that boots the ISO as an unmodified guest VM. The +hypervisor captures **full bootlogging**: serial console, disk I/O traces, port +I/O, PCI enumeration, and memory-mapped I/O — everything from the first firmware +instruction to userspace init. + +**FULL VENTOTY PARITY** is the requirement. hiperiso replicates every Ventoy +user-facing feature: + +- GUI installer (GTK3 + Qt5 auto-detection, like VentoyGUI) +- WebUI installer (local HTTP server, like VentoyWeb on port 24680) +- Plugin config tool (web-based, like VentoyPlugson on port 24681) +- Theme system (GRUB2 gfxmenu themes, per-arch, random selection) +- Plugin system (all 15 Ventoy plugins, full ventoy.json compatibility) +- WIM/VHD/VHDX boot support +- Secure Boot support (OVMF + shim + MOK enrollment) +- Persistence support (.dat files as QEMU drives) +- Auto-install support (kickstart/preseed via virtual floppy or fw_cfg) +- Password protection (boot-level + per-ISO) +- Language support (60+ languages, same languages.json format) +- Cross-platform installer (Linux + Windows) +- Non-destructive update (preserve data partition) +- LiveCD version (bootable hiperiso CD with kiosk GUI) +- All ISO types (Linux, Windows, BSD, WinPE, IMG, VHD, EFI, .vtoy) + +**The ONLY difference**: hiperiso uses a hypervisor and collects boot logs. +Everything else is identical to Ventoy's user experience. + +--- + +## 2. Ventoy Feature Assessment + +### 2.1 Source Analysis Summary + +Based on exhaustive source analysis of Ventoy (5 parallel research agents, all +completed). Key findings: + +| Component | LOC | Role | +|-----------|-----|------| +| `GRUB2/grub-core/ventoy/` | ~17,000 C | Menu, ISO enumeration, chain data, plugins | +| `ventoy_cmd.c` | 7,166 | All GRUB commands (boot, plugin, browser) | +| `ventoy_plugin.c` | 3,657 | JSON plugin framework (15 plugins) | +| `ventoy_windows.c` | 2,781 | Windows ISO boot (WIM patching, vtoyjump injection) | +| `ventoy_linux.c` | 2,120 | Linux ISO boot (initrd chain, kernel cmdline) | +| `ventoy_unix.c` | 1,245 | BSD/Unix boot (segment maps, ko replacement) | +| `ventoy_vhd.c` | 754 | VHD/VHDX boot | +| `VtoyTool/` | ~3,000 C | Runtime tools (vtoydm, vtoydump, vtoyloader) | +| `IMG/cpio/` | ~50 shell scripts | Distro-specific injection hooks (60+ distros) | +| `Ventoy2Disk/` (Windows) | ~5,000 C | Win32 installer (VDS/diskpart) | +| `LinuxGUI/` | ~5,000 C | GTK2/3 + Qt5 + WebUI installer | +| `Plugson/` | ~5,000 C | Plugin config web tool (87+ API endpoints) | +| `vtoyjump/` | ~3,200 C | Windows PE injection (ISO mount, Win11 bypass) | +| `LANGUAGES/languages.json` | 3,448 lines | 60+ languages | + +### 2.2 What hiperiso Eliminates vs Keeps + +| Ventoy Component | hiperiso Status | Reason | +|------------------|-----------------|--------| +| **GRUB2 ISO menu + enumeration** | ✅ KEEP | Menu UX is excellent | +| **Plugin framework (ventoy.json)** | ✅ KEEP ALL 15 | Full compatibility | +| **Theme system** | ✅ KEEP | GRUB2 themes work identically | +| **Menu alias/tip/class** | ✅ KEEP | Pure GRUB2 menu features | +| **Password protection** | ✅ KEEP | GRUB2-level feature | +| **Image list/blacklist** | ✅ KEEP | ISO filtering at menu level | +| **auto_install plugin** | ✅ ADAPT | Attach scripts via virtual floppy / fw_cfg | +| **persistence plugin** | ✅ ADAPT | .dat files as QEMU secondary drives | +| **dud plugin** | ✅ ADAPT | Attach DUD images as virtual floppy | +| **injection plugin** | ✅ ADAPT | Attach as virtual floppy / second CD | +| **conf_replace plugin** | ✅ ADAPT | Pre-process: create modified ISO copy | +| **WIM boot** | ✅ SIMPLIFY | QEMU CD-ROM emulation — no patching needed | +| **VHD/VHDX boot** | ✅ SIMPLIFY | QEMU native VHD support (`format=vpc`) | +| **Image chunk mapping** | ❌ ELIMINATE | ISO is a file opened by QEMU directly | +| **Chain head / os_param handoff** | ❌ ELIMINATE | No GRUB→OS memory handoff needed | +| **Linux initrd injection** | ❌ ELIMINATE | Guest boots natively from virtual CD-ROM | +| **Device-mapper setup (vtoydm)** | ❌ ELIMINATE | QEMU handles block device emulation | +| **60+ distro hook scripts** | ❌ ELIMINATE | OS is unmodified — no init patching | +| **vtloopex dm-mod.ko collection** | ❌ ELIMINATE | No device-mapper needed | +| **vtoyjump PE injection** | ❌ ELIMINATE | ISO is native virtual CD-ROM | +| **EFI Block I/O virtual disk** | ❌ ELIMINATE | QEMU provides real emulated hardware | + +### 2.3 The Fundamental Advantage + +Ventoy's entire complexity (30,000+ lines of injection code, thousands of binary +blobs) exists *solely* to reconstruct an ISO file as a virtual block device inside +a bare-metal-booted OS. hiperiso's hypervisor makes ALL of that unnecessary: + +- The ISO becomes a **native emulated AHCI CD-ROM** via QEMU +- The guest OS boots **completely unmodified** +- Every I/O operation is **visible to the hypervisor** for logging +- Compatibility is determined by QEMU's device model, not by injection code + +--- + +## 3. Architecture + +### 3.1 USB Partition Layout + +Identical concept to Ventoy — two partitions, data-preserving update: + +``` +┌──────────────────────────────────────────────────────────────────────┐ +│ USB DRIVE │ +│ │ +│ Partition 1 (ESP, FAT32, 256MB): │ +│ /EFI/BOOT/BOOTX64.EFI ← GRUB2 (hiperiso module) │ +│ /EFI/BOOT/grub.cfg ← GRUB2 menu config │ +│ /EFI/hiperiso/vmlinuz ← Host kernel (KVM built-in) │ +│ /EFI/hiperiso/initramfs.cpio.gz ← Host initramfs (QEMU+OVMF+init) │ +│ /EFI/hiperiso/OVMF_CODE.fd ← UEFI firmware code (read-only) │ +│ /EFI/hiperiso/OVMF_VARS.fd ← UEFI firmware variables (writable) │ +│ /EFI/hiperiso/trace/ ← Trace event files (standard/detailed/full) │ +│ /EFI/hiperiso/grub/themes/ ← Default GRUB2 themes │ +│ /EFI/hiperiso/grub/fonts/ ← .pf2 font files │ +│ /EFI/hiperiso/grub/icons/ ← Menu class icons │ +│ /EFI/hiperiso/lang/ ← Menu language files │ +│ /EFI/hiperiso/hiperiso-log ← Log analysis tool (static) │ +│ /EFI/hiperiso/version ← Version string │ +│ │ +│ Partition 2 (data, exFAT/NTFS): │ +│ /ISOs/*.iso ← User's ISO/WIM/IMG/VHD/VTOY files │ +│ /hiperiso/ │ +│ │ ├── hiperiso.json ← Main config (Ventoy-compatible) │ +│ │ ├── ventoy.json ← Ventoy-compatible alias (symlink) │ +│ │ ├── logs/ ← Boot logs (per-ISO directories) │ +│ │ ├── themes/ ← User-installed themes │ +│ │ ├── auto/ ← Auto-install scripts │ +│ │ ├── persistence/ ← .dat persistence files │ +│ │ ├── dud/ ← Driver Update Disk images │ +│ │ ├── injection/ ← Injection archives │ +│ │ ├── conf_replace/ ← Config replacement files │ +│ │ ├── fonts/ ← Custom .pf2 fonts │ +│ │ └── custom_boot/ ← .vcfg custom boot configs │ +│ /ventoy/ ← Ventoy-compatibility directory │ +│ │ ├── ventoy.json ← (symlink to hiperiso/hiperiso.json)│ +│ │ └── ... ← (symlinks for Ventoy path compat) │ +└──────────────────────────────────────────────────────────────────────┘ +``` + +**Key differences from Ventoy:** +- ESP is 256MB (Ventoy: 32MB) — needed for kernel + QEMU + OVMF +- ESP is visible/mountable (Ventoy hides VTOYEFI with GPT attribute) +- Config at `/hiperiso/hiperiso.json` with Ventoy-compatible symlink at `/ventoy/ventoy.json` +- Log directory on data partition ( Ventoy has no logs) + +### 3.2 Boot Chain + +``` +[1] Firmware (UEFI) boots from USB + │ + ▼ +[2] GRUB2 (BOOTX64.EFI) loads + │ - Parses hiperiso.json / ventoy.json + │ - Applies themes, aliases, menu classes, tips + │ - Enumerates ISO/WIM/IMG/VHD/VTOY from data partition + │ - Applies image_list/image_blacklist filters + │ - Displays menu with all plugin customizations + │ - User selects an ISO (or auto-selects via HISO_DEFAULT_IMAGE) + │ + ▼ +[3] GRUB2 checks password plugin (boot-level + per-ISO) + │ - If password required: prompt user + │ + ▼ +[4] GRUB2 resolves plugins for selected ISO: + │ - auto_install: find matching template path + │ - persistence: find matching .dat backend path(s) + │ - dud: find matching DUD image path(s) + │ - injection: find matching archive path + │ - conf_replace: find matching replacement files + │ - menu_tip: display tip if configured + │ - iso_overrides: get QEMU config overrides + │ + ▼ +[5] GRUB2 loads host kernel + initramfs: + │ linux /EFI/hiperiso/vmlinuz \ + │ hiperiso_iso="/ISOs/ubuntu-24.04.iso" \ + │ hiperiso_log="/hiperiso/logs/ubuntu-24.04/" \ + │ hiperiso_trace_level="standard" \ + │ hiperiso_ram="2048" hiperiso_cpus="2" \ + │ hiperiso_auto_install="/hiperiso/auto/ubuntu.seed" \ + │ hiperiso_persistence="/hiperiso/persistence/ubuntu.dat" \ + │ hiperiso_dud="/hiperiso/dud/driver.iso" \ + │ hiperiso_injection="/hiperiso/injection/inject.tar.gz" \ + │ hiperiso_conf_replace="/hiperiso/conf_replace/grub.cfg:org=/boot/grub/grub.cfg" \ + │ hiperiso_secure_boot="1" \ + │ hiperiso_tpm="0" \ + │ hiperiso_cpu_features="" \ + │ hiperiso_display="none" \ + │ hiperiso_vga="none" + │ initrd /EFI/hiperiso/initramfs.cpio.gz + │ + ▼ +[6] Host Linux kernel boots (KVM built-in) + │ - USB drivers init (XHCI/EHCI) + │ - Mount data partition at /mnt/usb + │ - KVM initializes + │ + ▼ +[7] Initramfs /init script: + │ a. Verify /dev/kvm exists + │ b. Verify ISO file exists + │ c. Create log directory + │ d. Process conf_replace (if any): create modified ISO copy in tmpfs + │ e. Process injection (if any): prepare virtual floppy image + │ f. Build QEMU command line with all plugin params + │ g. Launch QEMU in foreground + │ + ▼ +[8] QEMU boots with OVMF firmware + │ - OVMF UEFI initializes (Secure Boot if enabled) + │ - Discovers AHCI controller → emulated SATA CD-ROM + │ - CD-ROM backed by the ISO file (or modified copy) + │ - Virtual floppy with auto-install/DUD/injection (if configured) + │ - Secondary drive with persistence .dat (if configured) + │ - El Torito / UEFI boot from CD-ROM + │ + ▼ +[9] Guest OS boots (UNMODIFIED) + │ - Guest bootloader (GRUB2/Windows Boot Manager/etc.) + │ - Guest kernel/initrd loads from virtual CD-ROM + │ - All I/O trapped by KVM, optionally traced by QEMU + │ - Serial output → captured to log file + │ - Disk I/O → captured as trace events + │ + ▼ +[10] Guest running — FULL BOOTLOGGING ACTIVE + │ - Serial: /hiperiso/logs//serial.log + │ - Trace: /hiperiso/logs//trace.bin + │ - QEMU monitor socket for live introspection + │ + ▼ +[11] Guest shuts down → QEMU exits → flush logs → optional reboot to menu +``` + +### 3.3 QEMU Launch Configuration + +The initramfs builds a QEMU command line dynamically based on: +1. Default config from hiperiso.json `control` section +2. ISO-specific overrides from `iso_overrides` section +3. Plugin-resolved parameters (auto_install, persistence, dud, injection) +4. Secure Boot / TPM requirements + +**Base configuration (all ISOs):** + +```bash +qemu-system-x86_64 \ + -machine q35,accel=kvm \ + -cpu host \ + -m "${GUEST_RAM:-2048}" \ + -smp "${GUEST_CPUS:-2}" \ + \ + -drive file="${ISO_PATH}",if=none,id=cd0,format=raw,media=cdrom,readonly=on \ + -device ahci,id=ahci0 \ + -device ide-cd,drive=cd0,bus=ahci0.0,bootindex=1 \ + \ + -drive if=pflash,format=raw,readonly=on,file=OVMF_CODE.fd \ + -drive if=pflash,format=raw,file=OVMF_VARS.fd \ + \ + -serial "file:${LOG_PATH}/serial.log" \ + -trace events=${TRACE_EVENTS},file="${LOG_PATH}/trace.bin" \ + \ + -display "${DISPLAY_MODE:-none}" \ + -vga "${VGA_MODE:-none}" \ + -monitor "unix:${LOG_PATH}/monitor.sock,server,nowait" \ + \ + -nodefaults -no-reboot +``` + +**Conditional additions (added by plugin resolution):** + +| Plugin | QEMU Arguments Added | +|--------|---------------------| +| **persistence** | `-drive file=${DAT_PATH},if=none,id=disk0,format=raw -device virtio-blk-pci,drive=disk0` | +| **auto_install** | `-drive file=${FLOPPY_IMG},if=none,id=floppy0,format=raw -device isa-fdc,driveA=floppy0` OR `-fw_cfg name=opt/auto_install,file=${SCRIPT_PATH}` | +| **dud** | `-drive file=${DUD_IMG},if=none,id=floppy1,format=raw -device isa-fdc,driveB=floppy1` OR second CD-ROM | +| **injection** | `-drive file=${INJECT_IMG},if=none,id=floppy2,format=raw` (virtual floppy) | +| **secure_boot** | Use `OVMF_CODE.secboot.fd` + SMM: `-machine q35,accel=kvm,smm=on -global driver=cfi.pflash01,property=secure,value=on` | +| **tpm** | `-chardev socket,id=chrtpm,path=/tmp/swtpm-sock -tpmdev emulator,id=tpm0,chardev=chrtpm -device tpm-tis,tpmdev=tpm0` | +| **cpu_features** | `-cpu host,+vmx` (or other features) | +| **vhd boot** | `-drive file=${VHD_PATH},if=none,id=disk0,format=vpc -device virtio-blk-pci,drive=disk0` (replaces CD-ROM) | +| **display gtk** | `-display gtk -vga std` (for interactive use) | +| **display vnc** | `-vnc :0 -vga std` (for remote display) | +| **win11** | TPM + Secure Boot + 4GB RAM + cpu_features=vmx | +| **legacy bios** | `-bios bios-256k.bin` (SeaBIOS instead of OVMF) | + +### 3.4 Bootlogging System (hiperiso's Defining Feature) + +Four logging tiers (unchanged from MVP — this is hiperiso's unique value): + +#### Tier 1: Serial Console (always on) +- UART port 0x3F8 → `${LOG_PATH}/serial.log` +- Captures: bootloader, kernel printk, init, userspace output + +#### Tier 2: Disk I/O Tracing (always on) +- QEMU simpletrace events: `ide_sector_read`, `cd_read_sector`, `blk_co_preadv`, `virtio_blk_handle_read` +- Binary format → `${LOG_PATH}/trace.bin` → parsed by hiperiso-log + +#### Tier 3: Hardware Probes (optional: `hiperiso_trace_level=detailed`) +- Events: `cpu_in`, `cpu_out`, `pci_cfg_read`, `pci_cfg_write` + +#### Tier 4: Full MMIO (debug: `hiperiso_trace_level=full`) +- Events: `memory_region_ops_read`, `memory_region_ops_write` + +#### Log Analysis Tool: `hiperiso-log` +- Parse serial.log → boot stage timestamps +- Parse trace.bin → disk I/O heatmap, failure sectors +- Generate report.json + report.txt +- Compare boot traces across runs (regression detection) + +--- + +## 4. Plugin System — Full Ventoy Compatibility + +### 4.1 Config File + +**Location:** `/hiperiso/hiperiso.json` +**Compatibility:** `/ventoy/ventoy.json` (symlink to above) + +hiperiso reads BOTH paths. If `ventoy/ventoy.json` exists independently, it takes +precedence (Ventoy compatibility mode). Otherwise `hiperiso/hiperiso.json` is used. + +**Encoding:** UTF-8 (BOM handled), lowercase path required. + +**Platform suffixes:** All keys support per-platform variants: +- `_legacy` (BIOS), `_uefi` (x86_64 UEFI), `_ia32`, `_aa64`, `_mips` +- Lookup order: platform-specific first, then bare key + +### 4.2 All 15 Plugins + +#### Plugin 1: `control` (Array of objects) + +Maps to hiperiso environment variables + QEMU defaults: + +| Key | Type | Default | hiperiso Mapping | +|-----|------|---------|------------------| +| `HISO_DEFAULT_MENU_MODE` | int (0/1) | 0 | GRUB menu mode: 0=list, 1=tree | +| `VTOY_TREE_VIEW_MENU_STYLE` | int (0/1) | 0 | Tree sub-style | +| `VTOY_FILT_DOT_UNDERSCORE_FILE` | int (0/1) | 1 | Filter `._` files | +| `VTOY_SORT_CASE_SENSITIVE` | int (0/1) | 0 | Sort order | +| `VTOY_MAX_SEARCH_LEVEL` | int | -1 | Max dir depth (-1=unlimited) | +| `VTOY_VHD_NO_WARNING` | int (0/1) | 0 | Suppress VHD warning | +| `VTOY_FILE_FLT_ISO` | int (0/1) | 0 | Hide .iso files | +| `VTOY_FILE_FLT_WIM` | int (0/1) | 0 | Hide .wim files | +| `VTOY_FILE_FLT_EFI` | int (0/1) | 0 | Hide .efi files | +| `VTOY_FILE_FLT_IMG` | int (0/1) | 0 | Hide .img files | +| `VTOY_FILE_FLT_VHD` | int (0/1) | 0 | Hide .vhd files | +| `VTOY_FILE_FLT_VTOY` | int (0/1) | 0 | Hide .vtoy files | +| `VTOY_WIN11_BYPASS_CHECK` | int (0/1) | 1 | Win11: QEMU provides virtual hardware | +| `VTOY_WIN11_BYPASS_NRO` | int (0/1) | 1 | Win11: network bypass | +| `VTOY_LINUX_REMOUNT` | int (0/1) | 0 | N/A (hiperiso doesn't inject) | +| `VTOY_SECONDARY_BOOT_MENU` | int (0/1) | 1 | Show boot mode submenu | +| `VTOY_SHOW_PASSWORD_ASTERISK` | int (0/1) | 1 | Password display | +| `HISO_MENU_TIMEOUT` | int | 0 | Menu timeout (seconds) | +| `VTOY_WIN_UEFI_RES_LOCK` | int | 3 | Win UEFI resolution lock | +| `VTOY_SECURE_BOOT_POLICY` | int | 0 | 0=bypass, 1=enforce | +| `VTOY_SECONDARY_TIMEOUT` | int | 0 | Submenu timeout | +| `HISO_DEFAULT_KBD_LAYOUT` | string | "QWERTY_USA" | 20 layouts | +| `HISO_MENU_LANGUAGE` | string | "en_US" | 37 languages | +| `HISO_DEFAULT_SEARCH_ROOT` | string | "" | Restrict ISO search dir | +| `HISO_DEFAULT_IMAGE` | string | "" | Default ISO path | + +**hiperiso-specific additions** (in `control` or top-level): +| Key | Type | Default | Description | +|-----|------|---------|-------------| +| `default_ram` | int | 2048 | Default guest RAM (MB) | +| `default_cpus` | int | 2 | Default guest vCPUs | +| `default_trace_level` | string | "standard" | Trace tier | +| `default_display` | string | "none" | QEMU display mode | +| `default_vga` | string | "none" | QEMU VGA mode | +| `fallback_no_kvm` | bool | true | Fall back if no KVM | + +#### Plugin 2: `theme` (Object) + +Identical to Ventoy — GRUB2 gfxmenu themes: + +| Key | Type | Default | Description | +|-----|------|---------|-------------| +| `file` | string/array | — | Path(s) to theme.txt (relative = `/hiperiso/themes/` or `/ventoy/`) | +| `default_file` | int | 0 | Index (1-based, 0=random) | +| `random` | string | — | `"boot_second"`, `"boot_day"`, `"boot_month"` | +| `resolution_fit` | int (0/1) | 0 | Filter by resolution | +| `gfxmode` | string | "1024x768" | GRUB gfxmode ("max" = auto) | +| `display_mode` | string | — | `"GUI"`, `"CLI"`, `"serial"`, `"serial_console"` | +| `serial_param` | string | — | Serial port params | +| `hiperiso_left` | string | "5%" | Hotkey tip X | +| `hiperiso_top` | string | "95%" | Hotkey tip Y | +| `hiperiso_color` | string | "#0000ff" | Hotkey tip color | +| `fonts` | array | — | .pf2 font paths | + +Theme files use standard GRUB2 gfxmenu `.txt` format. Default theme shipped at +`/EFI/hiperiso/grub/themes/hiperiso/theme.txt`. + +Per-architecture themes: `theme`, `theme_legacy`, `theme_uefi`, `theme_ia32`, +`theme_aa64`, `theme_mips`. + +#### Plugin 3: `auto_install` (Array of objects) + +Maps auto-install scripts to ISOs. hiperiso attaches them as virtual media: + +| Key | Type | Description | +|-----|------|-------------| +| `image` | string | ISO path or glob | +| `parent` | string | Parent directory (alternative) | +| `template` | array | Script paths (kickstart/preseed/autoyast) | +| `autosel` | int | Auto-select template (1-based, 0=prompt) | +| `timeout` | int | Selection timeout (seconds) | + +**hiperiso implementation:** The selected template file is packaged into a +virtual floppy image (FAT12) and attached to QEMU via `-device isa-fdc`. +The guest OS reads it as `/dev/fd0`. Alternatively, for Linux guests, +`-fw_cfg name=opt/auto_install,file=script` passes the script via QEMU's +firmware config interface, and the guest's initramfs can read it from +`/sys/firmware/qemu_fw_cfg/`. + +#### Plugin 4: `persistence` (Array of objects) + +Persistent storage for live Linux ISOs: + +| Key | Type | Description | +|-----|------|-------------| +| `image` | string | ISO path or glob | +| `backend` | array | .dat file paths | +| `autosel` | int | Auto-select backend | +| `timeout` | int | Selection timeout | + +**hiperiso implementation:** The .dat file is attached as a secondary QEMU +drive: `-drive file=persistence.dat,if=none,id=persist,format=raw -device +virtio-blk-pci,drive=persist`. The guest OS discovers it as `/dev/vdb` and +mounts by label (e.g., `casper-rw`, `persistence`). + +**Persistence creation tool:** `hiperiso-create-persistence` (port of Ventoy's +`CreatePersistentImg.sh`): +``` +hiperiso-create-persistence [-s size_MB] [-t ext4|ext3|ext2|xfs] \ + [-l LABEL] [-c CONFIG] [-o output.dat] +``` + +#### Plugin 5: `menu_alias` (Array of objects) + +Identical to Ventoy — pure GRUB2 menu feature: + +| Key | Type | Description | +|-----|------|-------------| +| `image` | string | ISO path or glob | +| `dir` | string | Directory path | +| `alias` | string | Display name | + +#### Plugin 6: `menu_tip` (Object) + +Identical to Ventoy — tooltip messages in GRUB2 menu: + +| Key | Type | Default | Description | +|-----|------|---------|-------------| +| `left` | string | "10%" | Tooltip X | +| `top` | string | "81%" | Tooltip Y | +| `color` | string | "blue" | Tooltip color | +| `tips` | array | — | Each: `{image/dir, tip/tip1+tip2}` | + +#### Plugin 7: `menu_class` (Array of objects) + +Identical to Ventoy — GRUB2 icon class per ISO: + +| Key | Type | Description | +|-----|------|-------------| +| `key` | string | Substring match in filename | +| `parent` | string | Parent dir match | +| `dir` | string | Directory match | +| `class` | string | Icon class name | + +Default classes: `vtoyiso`, `vtoywim`, `vtoyefi`, `vtoyimg`, `vtoyvhd`, `vtoyvtoy`. + +#### Plugin 8: `injection` (Array of objects) + +Inject files into the booted OS. In Ventoy this modifies the initrd; in hiperiso +it attaches a virtual floppy: + +| Key | Type | Description | +|-----|------|-------------| +| `image` | string | ISO path or glob | +| `parent` | string | Parent dir (alternative) | +| `archive` | string | Path to tar.gz archive | + +**hiperiso implementation:** The archive is extracted and packaged into a +virtual floppy image (or small ISO) attached via QEMU. The guest OS can access +it as a removable device. For Linux guests, `-fw_cfg` can also be used. + +#### Plugin 9: `auto_memdisk` (Array of strings) + +ISOs that should be loaded entirely into RAM: + +**hiperiso implementation:** Instead of Ventoy's memdisk, hiperiso uses QEMU's +memory-backed CD-ROM: `-drive file=iso,if=none,id=cd0,media=cdrom,readonly=on` +is replaced with a RAM-backed copy for small ISOs. The initramfs copies the ISO +to tmpfs before passing to QEMU when `auto_memdisk` matches. + +#### Plugin 10: `image_list` (Array of strings) + +Whitelist — only show matching ISOs in menu. + +#### Plugin 11: `image_blacklist` (Array of strings) + +Blacklist — hide matching ISOs from menu. + +#### Plugin 12: `conf_replace` (Array of objects) + +Replace config files inside an ISO before booting: + +| Key | Type | Description | +|-----|------|-------------| +| `iso` | string | ISO path or glob | +| `org` | string | Original file path inside ISO | +| `new` | string | Replacement file on data partition | +| `img` | int | Which initrd (0=all) | + +**hiperiso implementation:** Since hiperiso can't modify a read-only CD-ROM at +runtime, the initramfs creates a modified copy of the ISO in tmpfs: +1. Copy original ISO to tmpfs +2. Use `xorriso` or `libisoburn` to replace the file(s) +3. Pass the modified ISO path to QEMU instead + +Max replacement file size: 1 MiB (same as Ventoy). + +#### Plugin 13: `dud` (Array of objects) + +Driver Update Disk images for specific ISOs: + +| Key | Type | Description | +|-----|------|-------------| +| `image` | string | ISO path or glob | +| `dud` | array | DUD image paths | + +**hiperiso implementation:** DUD images attached as virtual floppy or secondary +CD-ROM via QEMU. + +#### Plugin 14: `password` (Object) + +Boot-level and per-ISO password protection: + +| Key | Type | Description | +|-----|------|-------------| +| `bootpwd` | string | Boot password (`txt#`, `md5#`, `md5#salt#`) | +| `isopwd` | string | Password for .iso files | +| `wimpwd` | string | Password for .wim files | +| `vhdpwd` | string | Password for .vhd files | +| `imgpwd` | string | Password for .img files | +| `efipwd` | string | Password for .efi files | +| `vtoypwd` | string | Password for .vtoy files | +| `menupwd` | array | Per-ISO: `{file/parent, pwd}` | + +Password formats: `txt#plaintext`, `md5#32hexchars`, `md5#salt#hash`. + +Identical GRUB2-level implementation — prompt before booting. + +#### Plugin 15: `custom_boot` (Array of objects) + +Custom GRUB2 boot configuration per ISO: + +| Key | Type | Description | +|-----|------|-------------| +| `file` | string | ISO path | +| `dir` | string | Directory path | +| `vcfg` | string | Path to .vcfg GRUB2 config fragment | + +The .vcfg file is sourced by GRUB2 instead of the default boot path. + +### 4.3 ISO Matching Algorithm + +Identical to Ventoy: + +1. **Exact path match** (`image` key): Full path equality with `*` wildcard support +2. **Parent directory match** (`parent`/`dir` key): All ISOs in that directory +3. **Priority**: Exact matches checked first, then parent matches +4. **Wildcard**: `*` in filename portion (not across `/`) + +### 4.4 iso_overrides (hiperiso-specific extension) + +hiperiso adds an `iso_overrides` section for QEMU-specific configuration per ISO: + +```jsonc +{ + "iso_overrides": { + "*./windows11*.iso": { + "ram": 4096, + "cpus": 4, + "tpm": true, + "secure_boot": true, + "cpu_features": ["vmx"], + "trace_level": "detailed", + "display": "gtk", + "vga": "std", + "machine_type": "q35", + "bios": "ovmf" + }, + "*./memtest*.img": { + "ram": 512, + "cpus": 1, + "bios": "seabios", + "trace_level": "none" + }, + "*./freebsd*.iso": { + "ram": 2048, + "cpu_features": ["vmx"] + } + } +} +``` + +--- + +## 5. User-Facing Tools (Full Ventoy Parity) + +### 5.1 hiperisoGUI — Native GUI Installer + +**Architecture:** Same as VentoyGUI — smart launcher detects toolkit: + +``` +hiperiso_gui (C launcher) + ├── Parses /etc/ld.so.cache to detect GTK2/3/4, Qt4/5/6 + ├── Checks desktop environment (GNOME/KDE/XFCE) + ├── Falls back to pkexec for privilege escalation + ├── Supports --gtk3/--qt5 override flags + └── Execves: hiperiso_gui.gtk3 or hiperiso_gui.qt5 +``` + +**GTK3 variant** (port of `ventoy_gtk.c`): +- GtkBuilder UI (embedded XML, no external .glade file needed) +- Device combo box (USB disks, model/bus info) +- Install / Update / Clean buttons +- Secure Boot checkbox +- Partition style: MBR / GPT radio +- 4K alignment checkbox +- Reserve space spinner +- Filesystem selector: exFAT / NTFS / FAT32 / UDF +- Progress bar with percentage +- Language menu (60+ languages) +- About dialog with version + +**Qt5 variant** (port of `ventoy2diskwindow.cpp`): +- QMainWindow with same functionality +- Background thread for install/update operations +- Signal/slot wiring for progress updates + +**Build:** GTK3 via `pkg-config gtk+-3.0`, Qt5 via `qmake`. + +### 5.2 hiperisoWeb — Web-Based Installer + +**Architecture:** Embedded HTTP server (CivetWeb) on port 24680: + +``` +Browser (SPA) ←→ hiperiso_web (HTTP+JSON) ←→ /dev/sdX (direct I/O) +``` + +**API endpoints** (identical to VentoyWeb): + +| Method | Parameters | Response | +|--------|-----------|----------| +| `GET /sysinfo` | — | version, language, partstyle, busy, token | +| `POST /get_dev_list` | — | Array of disks (name, size, model, bus, ventoy status) | +| `POST /install` | disk, partstyle, secure_boot, align_4kb, reserve_space | Success/fail | +| `POST /update` | disk | Success/fail | +| `POST /clean` | disk | Success/fail | +| `GET /get_percent` | — | 0-100 progress | +| `POST /sel_language` | lang | OK | +| `POST /sel_partstyle` | mbr/gpt | OK | +| `POST /refresh_device` | — | OK | + +**Frontend:** Single-page HTML (AdminLTE + Bootstrap + jQuery). Served from +embedded data (compiled into binary as C arrays, same as Ventoy). + +**Launcher:** `hiperiso-web.sh` — detects arch, starts `hiperiso_web $HOST $PORT`, +opens browser to `http://127.0.0.1:24680`. + +### 5.3 hiperisoPlugson — Plugin Configuration Tool + +**Architecture:** Embedded HTTP server (CivetWeb) on port 24681: + +``` +Browser (SPA) ←→ hiperiso_plugson (HTTP+JSON) ←→ hiperiso.json on USB +``` + +Configures an already-installed hiperiso USB drive. Reads/writes +`hiperiso/hiperiso.json` (or `ventoy/ventoy.json`). + +**API endpoints** (87+ endpoints, same as VentoyPlugson): + +Each plugin type has CRUD endpoints: +- `GET /get_` — Read current config +- `POST /save_` — Save config +- `POST /add_` — Add entry +- `POST /del_` — Delete entry + +Plugin types: `control`, `theme`, `auto_install`, `persistence`, `menu_alias`, +`menu_tip`, `menu_class`, `injection`, `auto_memdisk`, `image_list`, +`image_blacklist`, `conf_replace`, `dud`, `password`, `custom_boot`. + +Plus: `sysinfo`, `handshake`, `device_info`, `check_path`, `fuzzy`. + +**Frontend:** Multi-page SPA with tabs for each plugin type. +AdminLTE + Bootstrap + DataTables + jQuery. + +**Launcher:** `hiperiso-plugson.sh` — validates hiperiso disk, mounts partition, +starts server, opens browser. + +### 5.4 hiperiso-install — CLI Installer + +**Shell-based** (port of Ventoy's `VentoyWorker.sh` pattern): + +```sh +Hiperiso2Disk.sh -i /dev/sdX [-g] [-s/-S] [-r SIZE] [-L label] [-n] +Hiperiso2Disk.sh -u /dev/sdX +Hiperiso2Disk.sh -l /dev/sdX +``` + +Options: +- `-i` / `-I`: Install (normal / force) +- `-u`: Update (preserve data partition) +- `-l`: List hiperiso version on disk +- `-g`: Use GPT (default), otherwise MBR +- `-s` / `-S`: Secure Boot on / off +- `-r SIZE`: Reserve space (MB) at disk end +- `-L label`: Data partition label +- `-n`: Non-destructive install (shrink existing partition) +- `-y`: Skip confirmation + +**Install flow:** +1. Check prerequisites (sgdisk, mkfs.vfat, mkfs.exfat) +2. Verify disk not mounted/swap +3. Detect existing hiperiso (refuse without -I) +4. Partition: create ESP (256MB FAT32) + data partition +5. Format: mkfs.vfat for ESP, mkfs.exfat for data +6. Copy payload: kernel, initramfs, OVMF, GRUB2, themes, tools +7. Install GRUB2: `grub-install --target=x86_64-efi` +8. Write hiperiso.json example +9. Create directory structure + +**Update flow:** +1. Read existing version +2. Preserve data partition entirely +3. Rewrite ESP: new kernel, initramfs, OVMF, GRUB2, themes +4. Preserve user config (hiperiso.json) + +### 5.5 hiperiso-log — Log Analysis Tool + +**Static binary** (already implemented, 9 C files): +- Parse QEMU simpletrace binary → I/O log +- Parse serial.log → boot stage timestamps +- Generate summary report (JSON + text) +- Compare boot traces across runs + +``` +hiperiso-log analyze /hiperiso/logs/ubuntu-24.04/ +hiperiso-log trace /hiperiso/logs/ubuntu-24.04/trace.bin --format json +hiperiso-log serial /hiperiso/logs/ubuntu-24.04/serial.log --stages +hiperiso-log compare /hiperiso/logs/ubuntu-24.04/ /hiperiso/logs/ubuntu-24.04.bak/ +``` + +### 5.6 hiperiso-create-persistence — Persistence Tool + +Port of Ventoy's `CreatePersistentImg.sh`: +```sh +hiperiso-create-persistence [-s 1024] [-t ext4] [-l casper-rw] [-c "/ union"] [-o persistence.dat] +hiperiso-extend-persistence persistence.dat [-s +2048] +``` + +### 5.7 Language Support / i18n + +**Format:** Same as Ventoy — single JSON file with all translations. + +**File:** `/EFI/hiperiso/lang/languages.json` (3,448 lines, 60+ languages). + +Structure: +```json +[{ + "name": "English", + "FontFamily": "Courier New", + "FontSize": 20, + "Author": "hiperiso", + "STR_INSTALL": "Install", + "STR_UPDATE": "Update", + "STR_ERROR": "Error", + ... +}] +``` + +**String IDs:** Same `STR_*` enum as Ventoy (Language.h). hiperiso adds: +- `STR_BOOTLOGGING`: "Boot logging active" +- `STR_HYPERVISOR`: "Hypervisor mode" +- `STR_KVM_MISSING`: "Hardware virtualization unavailable" +- `STR_FALLBACK_MODE`: "Fallback mode (no bootlogging)" +- `STR_LOG_SAVED`: "Boot logs saved to" + +**Runtime:** GRUB2 loads translations into memory, selects by `HISO_MENU_LANGUAGE`. +Installer GUI (GTK/Qt) uses same JSON. Web UI loads via JavaScript. + +--- + +## 6. ISO Type Support + +### 6.1 All Supported Types + +| Type | Extension | Boot Method | QEMU Handling | +|------|-----------|-------------|---------------| +| **Linux ISO** | .iso | UEFI CD-ROM boot | `-drive media=cdrom` | +| **Windows Install** | .iso | UEFI CD-ROM boot | Native CD-ROM (no WIM patching needed) | +| **WinPE** | .iso/.wim | UEFI CD-ROM boot | Native CD-ROM + optional wimboot | +| **VHD** | .vhd | Disk boot | `-drive format=vpc` | +| **VHDX** | .vhdx | Disk boot | `-drive format=vhdx` | +| **VDI** | .vdi | Disk boot | `-drive format=vdi` | +| **IMG** | .img | Memdisk / CD-ROM | RAM-backed or CD-ROM | +| **EFI** | .efi | Direct chainload | `-drive` with EFI application | +| **VTOY** | .vtoy | Custom GRUB2 config | User-provided .vcfg | +| **BSD** | .iso | UEFI CD-ROM boot | Native (no geom_ventoy.ko needed) | + +### 6.2 Windows ISO Handling + +Ventoy requires: BCD parsing, WIM patching, vtoyjump injection, winload.exe→.efi conversion. + +**hiperiso:** None of that. QEMU presents the ISO as a standard AHCI CD-ROM. +Windows Boot Manager reads BCD from the virtual CD-ROM, loads boot.wim, and +boots normally. No modification needed. + +**Win11 specifics:** +- TPM 2.0: `-tpmdev emulator` with swtpm +- Secure Boot: OVMF with enrolled keys +- 4GB+ RAM: `iso_overrides` sets `ram: 4096` +- CPU features: `-cpu host,+vmx` +- Win11 bypass check: `VTOY_WIN11_BYPASS_CHECK=1` → QEMU provides compliant hardware + +### 6.3 VHD/VHDX Handling + +Ventoy requires: footer parsing, BCD patching, partition GUID manipulation. + +**hiperiso:** QEMU natively supports VHD (`format=vpc`), VHDX (`format=vhdx`), +and VDI (`format=vdi`). Attach directly as a hard drive — the guest OS sees a +real disk with partitions. + +### 6.4 Per-ISO QEMU Templates + +For ISOs requiring special QEMU configuration, templates are stored in +`/hiperiso/iso_overrides/`: + +```jsonc +// windows11.json +{ + "ram": 4096, + "cpus": 4, + "tpm": true, + "secure_boot": true, + "cpu_features": ["vmx"], + "machine_type": "q35" +} +``` + +--- + +## 7. Component Map + +``` +hiperiso/ +├── grub2/ # GRUB2 hiperiso module +│ ├── hiperiso.c # Module init, arch detection +│ ├── hiperiso_def.h # Definitions, constants +│ ├── hiperiso_menu.c # ISO enumeration + menu generation +│ ├── hiperiso_boot.c # Boot command: load kernel+initramfs +│ ├── hiperiso_plugin.c # All 15 plugins (JSON parse + dispatch) +│ ├── hiperiso_json.c # JSON parser (from Ventoy, unchanged) +│ ├── hiperiso_json.h # JSON types +│ ├── hiperiso_theme.c # Theme system (from Ventoy) +│ ├── hiperiso_password.c # Password challenge (from Ventoy) +│ └── build_grub2.sh # GRUB2 build script +│ +├── host/ # Hypervisor host environment +│ ├── kernel/ +│ │ └── hiperiso_defconfig # Minimal kernel (KVM built-in) +│ ├── initramfs/ +│ │ ├── init # Main init (KVM detect, mount, launch QEMU) +│ │ ├── hiperiso-lib.sh # Shared functions +│ │ ├── kvm_check.sh # KVM detection +│ │ ├── qemu_launch.sh # QEMU arg builder + launcher +│ │ ├── log_flush.sh # Async log flusher +│ │ ├── conf_replace.sh # ISO modification for conf_replace +│ │ ├── make_floppy.sh # Virtual floppy creation for plugins +│ │ └── fallback_boot.sh # Fallback mode (no KVM) +│ └── qemu/ +│ └── configure_qemu.sh # QEMU build config +│ +├── firmware/ # Guest firmware +│ ├── build_ovmf.sh # OVMF build (regular + Secure Boot) +│ └── OVMF_VARS.template.fd # Template for per-boot variable copy +│ +├── logging/ # Bootlogging system +│ ├── trace-standard.events # Tier 1+2 trace events +│ ├── trace-detailed.events # Tier 3 trace events +│ ├── trace-full.events # Tier 4 trace events +│ └── hiperiso-log/ # Log analysis tool (C, static) +│ ├── main.c # CLI entry point +│ ├── simpletrace.h # QEMU trace binary format +│ ├── trace_parser.c # Parse trace.bin +│ ├── serial_parser.c # Parse serial.log, extract stages +│ ├── report.c # Generate report.json + report.txt +│ └── Makefile # Static build +│ +├── installer/ # CLI installer +│ ├── Hiperiso2Disk.sh # USB installer (port of VentoyWorker.sh) +│ └── partition_layout.h # Partition constants +│ +├── gui/ # GUI installer (GTK3 + Qt5) +│ ├── launcher/ +│ │ └── hiperiso_gui.c # Smart toolkit detection launcher +│ ├── gtk3/ +│ │ ├── hiperiso_gtk.c # GTK3 GUI +│ │ └── hiperiso_gtk.h # Widget handles +│ ├── qt5/ +│ │ ├── hiperiso_qt.cpp # Qt5 GUI +│ │ ├── hiperiso_qt.h # Qt5 headers +│ │ └── partcfg_dialog.cpp # Partition config dialog +│ └── core/ +│ ├── hiperiso_disk.c # Disk detection (sysfs scanning) +│ ├── hiperiso_disk.h # Disk types, device struct +│ └── hiperiso_install.c # Install/update/clean logic (shared) +│ +├── web/ # Web installer + Plugin config +│ ├── hiperiso_web/ # Web installer (port 24680) +│ │ ├── main.c # HTTP server entry +│ │ ├── http.c # CivetWeb API handlers +│ │ ├── http.h # API definitions +│ │ └── www/ +│ │ └── index.html # SPA frontend +│ └── hiperiso_plugson/ # Plugin config tool (port 24681) +│ ├── main.c # HTTP server entry +│ ├── http.c # 87+ API endpoints +│ ├── http.h # Plugin data structures +│ └── www/ +│ ├── index.html # Main SPA +│ ├── plugson_control.html +│ ├── plugson_theme.html +│ ├── plugson_auto_install.html +│ ├── plugson_persistence.html +│ ├── plugson_password.html +│ ├── plugson_image_list.html +│ ├── plugson_menu_alias.html +│ ├── plugson_menu_tip.html +│ ├── plugson_menu_class.html +│ ├── plugson_injection.html +│ ├── plugson_conf_replace.html +│ ├── plugson_dud.html +│ ├── plugson_auto_memdisk.html +│ ├── plugson_custom_boot.html +│ └── static/ # CSS, JS, fonts +│ +├── tools/ # Auxiliary tools +│ ├── hiperiso-create-persistence.sh # .dat creation +│ ├── hiperiso-extend-persistence.sh # .dat extension +│ └── hiperiso-vtoytool.c # Multi-call runtime tool +│ +├── config/ # Configuration +│ ├── hiperiso.json.example # Example config +│ ├── README.md # Config documentation +│ └── iso_overrides/ # Per-ISO QEMU templates +│ ├── windows11.json +│ └── default.json +│ +├── i18n/ # Internationalization +│ └── languages.json # 60+ languages (from Ventoy, adapted) +│ +├── theme/ # Default themes +│ └── hiperiso/ +│ ├── theme.txt # GRUB2 gfxmenu theme +│ └── *.png # Theme graphics +│ +├── scripts/ # Build system +│ ├── build_all.sh # Master build orchestrator +│ ├── download_sources.sh # Source downloader +│ ├── configure_qemu.sh # QEMU configure +│ └── build_initramfs.sh # Initramfs packer +│ +├── Makefile # Top-level build +├── INTERFACES.sh # Interface contracts (single source of truth) +├── HIPERISO_PLAN.md # This file +└── README.md # User documentation +``` + +--- + +## 8. Installer Architecture Details + +### 8.1 Partition Layout (MBR and GPT) + +``` +Sector 0: MBR (boot.img 446 bytes + partition table + 0x55AA) +Sector 1-33: GPT header (if GPT) or GRUB2 core.img start (if MBR) +Sector 34-2047: GRUB2 core image / GPT partition entries +Sector 2048: ─── PARTITION 1 START (1MB aligned) ─── + ESP (FAT32, 256MB) + MBR type: 0xEF | GPT type: C12A7328-F81F-11D2-BA4B-00A0C93EC93B + GPT name: "HIPERISO" | Label: "HIPERISO" + Contents: GRUB2, kernel, initramfs, OVMF, themes, tools + ─── PARTITION 2 START (4KB aligned) ─── + Data partition (exFAT default, also NTFS/FAT32/UDF) + MBR type: 0x07 | GPT type: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7 + GPT name: "HIPERISODATA" | Label: "HIPERISO" (configurable) + Contents: ISOs, logs, config, plugins +GPT only: Last 33 sectors: backup GPT header + partition table +``` + +### 8.2 Boot Code Written to Disk + +| What | Destination | +|------|-------------| +| **GRUB2 boot.img** (446 bytes) | Disk sector 0, bytes 0-445 | +| **GRUB2 core.img** | Sectors 1-2047 (MBR) or 34-2047 (GPT) | +| **ESP FAT32 image** | Partition 1 (formatted in-place) | +| **Disk UUID** | Disk offset 384 (16 bytes) | +| **Disk signature** | Disk offset 440 (4 bytes) | + +### 8.3 Install/Update/Clean Operations + +**Fresh install:** +1. Wipe first 64 sectors (destroy existing MBR/GPT) +2. Create partitions (sgdisk or fdisk) +3. Format ESP (mkfs.vfat -F32 -n HIPERISO) +4. Format data partition (mkfs.exfat -n HIPERISO) +5. Install GRUB2 to ESP +6. Copy all payload files to ESP +7. Create hiperiso.json example on data partition +8. Create directory structure + +**Update (data-preserving):** +1. Read existing version from ESP +2. Mount ESP +3. Replace all payload files on ESP +4. Preserve data partition entirely +5. Optionally migrate config format + +**Clean:** +1. Wipe MBR/GPT +2. Delete all partitions +(Disk is left blank) + +**Non-destructive install:** +1. Shrink existing data partition +2. Create ESP at end +3. Install boot code + +### 8.4 Device Detection (Linux) + +Scan `/sys/block/*`: +- Detect type via major number: SCSI, USB, IDE, NVMe, VirtIO +- Exclude: ram, zram, loop, dm-, sr*, partitions +- Get size via sysfs or BLKGETSIZE64 ioctl +- Get vendor/model from sysfs +- USB-only filter by default (override with `-U`) + +### 8.5 Cross-Platform + +| Platform | Installer | GUI | Web | Plugson | +|----------|-----------|-----|-----|---------| +| Linux x86_64 | Shell + C | GTK3/Qt5 | CivetWeb | CivetWeb | +| Linux aarch64 | Shell + C | GTK3/Qt5 | CivetWeb | CivetWeb | +| Windows x86_64 | C (Win32 API) | Win32 dialog | N/A | N/A | +| Windows ARM64 | C (Win32 API) | Win32 dialog | N/A | N/A | + +Windows installer uses VDS COM / diskpart for partition operations. +Same partition layout and boot images. + +--- + +## 9. Implementation Plan + +### Phase 1: MVP — Core Boot Chain (DONE ✅) + +All 6 components compiled from source: +- [x] Linux kernel 6.12 LTS (KVM built-in, 5.4MB bzImage) +- [x] QEMU 9.0.0 (x86_64-softmmu, KVM + simpletrace, 24MB) +- [x] OVMF (UEFI firmware, 4MB) +- [x] GRUB2 2.12 (hiperiso module → BOOTX64.EFI, 741KB) +- [x] Busybox 1.36.1 (static, 2.4MB) +- [x] hiperiso-log (static, 892KB) +- [x] Initramfs packed (7.9MB cpio.gz) +- [x] Payload assembled (19MB) + +### Phase 2: Enhanced GRUB2 Module (CURRENT) + +Expand the existing GRUB2 hiperiso module to full Ventoy parity: + +- [ ] Expand `hiperiso_plugin.c` to parse all 15 plugins from JSON +- [ ] Add `hiperiso_theme.c` — GRUB2 theme loading + application +- [ ] Add `hiperiso_password.c` — boot + per-ISO password challenges +- [ ] Expand `hiperiso_menu.c` — tree view mode, menu classes, image filtering +- [ ] Expand `hiperiso_boot.c` — pass all plugin params via kernel cmdline +- [ ] Add ISO matching algorithm (exact path + parent dir + wildcard) +- [ ] Add platform suffix support (`_legacy`, `_uefi`, etc.) +- [ ] Add secondary boot menu (normal/memdisk/grub2 modes) +- [ ] Add language support (load languages.json at boot) +- [ ] Add keyboard layout support (20 layouts) + +### Phase 3: Enhanced Initramfs + +Expand init scripts to handle all plugin parameters: + +- [ ] `conf_replace.sh` — Create modified ISO copy using xorriso +- [ ] `make_floppy.sh` — Create virtual floppy images for auto_install/dud/injection +- [ ] Expand `qemu_launch.sh` — Build QEMU args for all plugin types +- [ ] Add swtpm integration for TPM support +- [ ] Add Secure Boot OVMF variable management (copy template per boot) +- [ ] Expand `fallback_boot.sh` — Full Ventoy-compatible fallback + +### Phase 4: CLI Installer Enhancement + +Expand `Hiperiso2Disk.sh` to full Ventoy parity: + +- [ ] GPT + MBR dual support +- [ ] Secure Boot support (partition attributes) +- [ ] Non-destructive install (partition shrink) +- [ ] Reserve space option +- [ ] Filesystem selection (exFAT/NTFS/FAT32/UDF) +- [ ] Version detection and update mode +- [ ] 4K alignment detection +- [ ] USB device filtering + +### Phase 5: GUI Installer (GTK3 + Qt5) + +- [ ] `hiperiso_gui.c` — Smart launcher (toolkit detection) +- [ ] `hiperiso_gtk.c` — GTK3 GUI (port from ventoy_gtk.c) +- [ ] `hiperiso_qt.cpp` — Qt5 GUI (port from ventoy2diskwindow.cpp) +- [ ] `hiperiso_disk.c` — Disk detection (port from ventoy_disk_linux.c) +- [ ] `hiperiso_install.c` — Shared install/update/clean logic +- [ ] Language support in GUI (60+ languages) +- [ ] Kiosk mode for LiveCD + +### Phase 6: Web Installer (hiperisoWeb) + +- [ ] Port CivetWeb HTTP server integration +- [ ] Implement all 9 API endpoints +- [ ] Create SPA frontend (index.html) +- [ ] `hiperiso-web.sh` launcher script +- [ ] Device detection and enumeration +- [ ] Progress tracking +- [ ] Language support in web UI + +### Phase 7: Plugin Config Tool (hiperisoPlugson) + +- [ ] Port CivetWeb HTTP server +- [ ] Implement all 87+ API endpoints +- [ ] Create 15 plugin config pages (HTML) +- [ ] `hiperiso-plugson.sh` launcher script +- [ ] JSON validation and error handling +- [ ] Config backup/restore + +### Phase 8: Persistence + Auto-Install Tools + +- [ ] `hiperiso-create-persistence.sh` — .dat creation tool +- [ ] `hiperiso-extend-persistence.sh` — .dat extension tool +- [ ] Virtual floppy creation for auto-install scripts +- [ ] fw_cfg integration for Linux auto-install + +### Phase 9: Windows Installer + +- [ ] Port Ventoy2Disk C/Win32 code +- [ ] VDS/diskpart integration +- [ ] Win32 dialog GUI +- [ ] CLI mode (`hiperiso-install.exe VTOYCLI /I /PhyDrive:1`) + +### Phase 10: LiveCD + Polish + +- [ ] Build LiveCD ISO (bootable hiperiso with kiosk GUI) +- [ ] Default themes and icons +- [ ] Comprehensive documentation +- [ ] Cross-architecture builds (aarch64, i386) +- [ ] Automated testing framework + +--- + +## 10. Risk Analysis + +### Risk 1: KVM Unavailable (~15% of machines) +**Mitigation:** Fallback mode — detect `/dev/kvm`, fall back to direct boot +(no logging but still boots). Dual-mode strategy ensures ~100% hardware coverage. + +### Risk 2: Memory Pressure +**Mitigation:** QEMU streams ISO from USB. Pre-check available RAM. +Default 2GB, configurable per-ISO. Windows 11 → 4GB. + +### Risk 3: ISO Expects Specific Hardware +**Mitigation:** QEMU Q35 provides modern hardware. Per-ISO QEMU templates. +Accept some specialized ISOs won't work (document, same as Ventoy). + +### Risk 4: Secure Boot on Host +**Mitigation:** Build kernel as UKI (Unified Kernel Image). SHIM signing chain. +Standard Secure Boot Linux practice. + +### Risk 5: Plugin Compatibility +**Mitigation:** Accept ventoy.json directly. Test all 15 plugins against +real-world configurations. Maintain compatibility matrix. + +### Risk 6: conf_replace Performance +**Mitigation:** Creating modified ISO copy in tmpfs is fast for small files +(<1MB). Only triggered when conf_replace plugin is configured. + +--- + +## 11. Technology Stack + +| Component | Technology | Rationale | +|-----------|-----------|-----------| +| Hypervisor | KVM (in-kernel) | Ubiquitous, hardware-accelerated | +| VMM | QEMU 9.0.0 (stripped) | Comprehensive device model + tracing | +| Guest firmware | OVMF (edk2) | UEFI, SMM, Secure Boot ready | +| Host OS | Minimal Linux 6.12 LTS | KVM built-in, USB/FS drivers | +| Initramfs | Busybox 1.36.1 | Minimal footprint | +| Bootloader | GRUB2 2.12 (hiperiso module) | Menu UX, theme system | +| HTTP server | CivetWeb (embedded) | Same as Ventoy, proven | +| GUI | GTK3 + Qt5 | Same as Ventoy, broad compatibility | +| Web frontend | AdminLTE + Bootstrap + jQuery | Same as Ventoy | +| Config format | JSON (ventoy.json compatible) | Familiar, easy to parse | +| Log tool | C (statically linked) | Runs anywhere | +| i18n | JSON (languages.json) | Same as Ventoy | +| Installer | Shell (Linux) + C (Windows) | Same as Ventoy | + +--- + +## 12. Build System + +### 12.1 Build Targets + +```makefile +make all # Build everything +make kernel # Linux kernel bzImage +make qemu # QEMU system emulator (stripped) +make ovmf # OVMF firmware (regular + Secure Boot) +make grub2 # GRUB2 with hiperiso module → BOOTX64.EFI +make busybox # Busybox static +make initramfs # Pack initramfs.cpio.gz +make hiperiso-log # Log analysis tool (static) +make gui # GTK3 + Qt5 GUI installers +make web # Web installer + Plugson +make dist # Assemble build/payload/ +make clean # Remove build/ +make test # Run unit tests +``` + +### 12.2 Payload Output + +``` +build/payload/ +├── Hiperiso2Disk.sh # CLI installer +├── hiperiso-web.sh # Web installer launcher +├── hiperiso-plugson.sh # Plugin config launcher +├── hiperiso-gui # GUI launcher (smart detect) +├── config/ +│ ├── hiperiso.json.example +│ └── iso_overrides/ +├── theme/ +│ └── hiperiso/theme.txt +├── i18n/ +│ └── languages.json +├── tool/ +│ ├── x86_64/ +│ │ ├── hiperiso_web # Web server +│ │ ├── hiperiso_plugson # Plugin config server +│ │ ├── hiperiso_gui.gtk3 # GTK3 GUI +│ │ ├── hiperiso_gui.qt5 # Qt5 GUI +│ │ ├── hiperiso-create-persistence.sh +│ │ └── hiperiso-log # Log analysis tool +│ └── (other arches in future) +├── EFI/ +│ ├── BOOT/ +│ │ ├── BOOTX64.EFI # GRUB2 (hiperiso module) +│ │ └── grub.cfg # GRUB2 menu config +│ └── hiperiso/ +│ ├── vmlinuz # Host kernel +│ ├── initramfs.cpio.gz # Host initramfs +│ ├── OVMF_CODE.fd # UEFI firmware code +│ ├── OVMF_VARS.fd # UEFI firmware variables (template) +│ ├── hiperiso-log # Log tool (static) +│ ├── trace/ # Trace event files +│ │ ├── trace-standard.events +│ │ ├── trace-detailed.events +│ │ └── trace-full.events +│ ├── grub/ +│ │ ├── themes/ # Default themes +│ │ ├── fonts/ # Default fonts +│ │ └── icons/ # Menu class icons +│ └── lang/ # Language files +│ └── languages.json +└── version # Version string +``` + +--- + +## Appendix A: Ventoy Feature Parity Checklist + +### User-Facing Features +- [x] ISO menu with themes (GRUB2 gfxmenu) +- [ ] Tree view mode +- [ ] ISO browsing (directory navigation) +- [ ] Hotkey tips (F1-F6 shortcuts) +- [ ] Password protection (boot + per-ISO) +- [ ] Menu aliases +- [ ] Menu tooltips +- [ ] Menu class icons +- [ ] Language selection (60+ languages) +- [ ] Keyboard layout selection (20 layouts) +- [ ] Power off / reboot from menu +- [ ] Shell mode (GRUB2 shell) +- [ ] Theme selection at runtime + +### ISO Support +- [x] Linux ISO boot +- [ ] Windows ISO boot (native CD-ROM) +- [ ] WinPE boot +- [ ] WIM boot +- [ ] VHD/VHDX boot +- [ ] VDI boot +- [ ] IMG boot (memdisk mode) +- [ ] EFI executable boot +- [ ] .vtoy custom boot +- [ ] FreeBSD boot +- [ ] DragonFly BSD boot +- [ ] systemd-boot ISOs + +### Plugin System +- [x] JSON config parsing +- [ ] control plugin (all 25+ keys) +- [ ] theme plugin (all 10+ keys) +- [ ] auto_install plugin +- [ ] persistence plugin +- [ ] menu_alias plugin +- [ ] menu_tip plugin +- [ ] menu_class plugin +- [ ] injection plugin +- [ ] auto_memdisk plugin +- [ ] image_list plugin +- [ ] image_blacklist plugin +- [ ] conf_replace plugin +- [ ] dud plugin +- [ ] password plugin +- [ ] custom_boot plugin + +### Installer +- [x] CLI installer (basic) +- [ ] CLI installer (full: GPT+MBR, reserve, non-destructive) +- [ ] GTK3 GUI installer +- [ ] Qt5 GUI installer +- [ ] Web installer (port 24680) +- [ ] Plugin config tool (port 24681) +- [ ] Windows installer +- [ ] Update mode (data-preserving) +- [ ] Non-destructive install + +### Bootlogging (hiperiso unique) +- [x] Serial console capture +- [x] Disk I/O tracing (simpletrace) +- [x] Trace tiers (standard/detailed/full) +- [x] Log analysis tool (hiperiso-log) +- [ ] Boot stage timestamping +- [ ] Boot comparison/regression detection + +### Advanced +- [ ] Secure Boot (OVMF + shim + MOK) +- [ ] Virtual TPM (swtpm for Win11) +- [ ] Persistence (.dat creation + management) +- [ ] Auto-install script attachment +- [ ] Driver Update Disk support +- [ ] Config replacement (ISO modification) +- [ ] File injection +- [ ] LiveCD version (kiosk GUI) +- [ ] Cross-architecture (aarch64) + +--- + +## Appendix B: Ventoy Source Inventory + +Key reference files assessed: + +``` +reference/Ventoy/ +├── GRUB2/MOD_SRC/grub-2.04/grub-core/ventoy/ +│ ├── ventoy.c # Module init +│ ├── ventoy_def.h # All definitions +│ ├── ventoy_cmd.c (7166 lines) # All GRUB commands +│ ├── ventoy_linux.c (2120 lines) # Linux ISO boot chain +│ ├── ventoy_windows.c (2781) # Windows ISO boot +│ ├── ventoy_unix.c (1245) # BSD/Unix boot +│ ├── ventoy_vhd.c (754) # VHD/VHDX boot +│ ├── ventoy_plugin.c (3657) # Plugin framework (15 plugins) +│ ├── ventoy_browser.c # File browser +│ └── ventoy_json.c # JSON parser +├── Ventoy2Disk/Ventoy2Disk/ # Windows installer +├── LinuxGUI/Ventoy2Disk/ # GTK/Qt/Web installer +├── Plugson/ # Plugin config tool +├── VtoyTool/ # Runtime tools +├── IMG/cpio/ventoy/ # Linux injection hooks +├── LANGUAGES/languages.json # 60+ languages +├── INSTALL/ # Shell installer + tools +└── vtoyjump/ # Windows PE injection +``` + +--- + +## Appendix C: Research Sources + +- QEMU tracing: https://www.qemu.org/docs/master/devel/tracing.html +- OVMF Secure Boot: https://github.com/tianocore/tianocore.github.io/wiki/OVMF +- QEMU device model: https://www.qemu.org/docs/master/system/ +- swtpm (virtual TPM): https://github.com/stefanberger/swtpm +- openQA (ISO testing): https://open.qa/docs/ +- Ventoy documentation: https://www.ventoy.net/en/documentation.html diff --git a/README.md b/README.md new file mode 100644 index 0000000..909a78a --- /dev/null +++ b/README.md @@ -0,0 +1,408 @@ +# hiperiso + +A hypervisor-based ISO boot tool with full bootlogging — like Ventoy, but it +boots ISOs inside a **KVM + QEMU** virtual machine and captures the entire boot +(serial console, disk I/O, PCI/port I/O traces). The guest OS runs completely +unmodified. + +``` +USB → GRUB2 → minimal Linux kernel (KVM built-in) + initramfs → + QEMU (KVM accel) → OVMF UEFI → ISO boots as an emulated CD-ROM +``` + +## How it differs from Ventoy + +Ventoy injects hooks into the booted OS (initrd patching, device-mapper, WIM +patching, 60+ distro hooks). hiperiso instead presents the ISO as a native +emulated AHCI CD-ROM to a KVM guest, so **no guest modification is needed** and +the hypervisor has full visibility into every I/O operation. + +--- + +## Prerequisites + +### Build host + +- A Linux x86_64 machine with VT-x / AMD-V +- Build tools: `gcc`, `g++`, `make`, `bc`, `flex`, `bison`, `cpio`, `gzip`, + `wget`, `tar` +- EDK2 build deps: `nasm`, `iasl` (acpica), `python3` +- QEMU build deps: `ninja-build`, `pkg-config`, `libglib2.0-dev`, `libpixman-1-dev` +- Kernel build deps: `libelf-dev`, `libssl-dev` +- ~6 GB free disk for sources + build trees + +### Install host (where you prepare the USB) + +- `sgdisk` (package `gdisk` / `gptfdisk`) +- `dosfstools` (`mkfs.vfat`) +- `exfatprogs` (`mkfs.exfat`) **or** `ntfs-3g` (`mkntfs`) +- `grub2-efi-amd64` + `grub2-efi-amd64-modules` (provides `grub-install --target=x86_64-efi`) +- `util-linux` (`findmnt`, `lsblk`) + +### Target machine (the PC you boot from USB) + +- x86_64 with hardware virtualization (VT-x/AMD-V) **enabled in BIOS/UEFI** +- UEFI boot mode (a Legacy/CSM path is planned, not in the MVP) + +--- + +## Build + +### 1. One-shot full build + +```sh +./scripts/build_all.sh +``` + +This downloads sources, builds the kernel, QEMU, OVMF, GRUB2, the log tool, +packs the initramfs, and assembles a distributable payload in `build/payload/`. + +Re-run later without re-downloading: + +```sh +SKIP_DOWNLOAD=1 ./scripts/build_all.sh +``` + +### 2. Build individual components + +Each stage is a standalone script: + +```sh +./scripts/download_sources.sh # fetch + extract kernel, qemu, edk2, grub2, busybox +cd build/linux && cp ../../host/kernel/hiperiso_defconfig .config && make olddefconfig && make -j"$(nproc)" bzImage +cd build/qemu && ../../scripts/configure_qemu.sh && make -j"$(nproc)" qemu-system-x86_64 +cd build/edk2 && bash ../../firmware/build_ovmf.sh +``` + +Or use the provided Makefile: + +```sh +make all # kernel qemu ovmf grub2 initramfs hiperiso-log +make dist # assemble build/payload/ +make clean # remove build/ +``` + +### 3. Output layout + +``` +build/payload/ +├── Hiperiso2Disk.sh +├── config/hiperiso.json.example +└── EFI/ + ├── BOOT/ + │ ├── BOOTX64.EFI # GRUB2 (hiperiso module) + │ └── grub.cfg + └── hiperiso/ + ├── vmlinuz # host kernel (KVM built-in) + ├── initramfs.cpio.gz # busybox + qemu + init + ├── OVMF.fd # guest UEFI firmware (SMM + Secure Boot) + ├── hiperiso-log # log analysis tool + └── trace/ # trace-*.events (standard/detailed/full) +``` + +--- + +## Install to a USB drive + +> ⚠️ **This destroys all data on the target device.** Double-check the device name. + +```sh +lsblk # identify your USB, e.g. /dev/sdX +sudo bash build/payload/Hiperiso2Disk.sh -I -g /dev/sdX +``` + +`Hiperiso2Disk.sh` defaults to MBR. Use `-g` for GPT. The standard Ventoy-style +layout is: + +| # | Type | FS | Size | Contents | +|---|--------|--------|--------|--------------------------------------------| +| 1 | 0700 | exFAT | rest | Your ISOs + boot logs + config | +| 2 | EF00 | FAT16 | 32 MB | ESP: GRUB2, kernel, initramfs, OVMF | + +Copy your ISOs: + +```sh +cp my_os.iso /media/$USER/HIPERISO/ISOs/ +``` + +Options: + +```sh +sudo bash Hiperiso2Disk.sh -i /dev/sdX # install +sudo bash Hiperiso2Disk.sh -I -g /dev/sdX # force reinstall using GPT +sudo bash Hiperiso2Disk.sh -i -L MYUSB /dev/sdX # custom data-partition label +``` + +--- + +## Usage + +1. Boot the PC from the USB (pick the **UEFI** entry). +2. GRUB2 lists ISOs found under `/ISOs/`. Select one. +3. GRUB2 boots the host kernel; the initramfs detects `/dev/kvm` and launches + QEMU with the selected ISO attached as a CD-ROM. +4. The ISO's OS boots **unmodified** inside the VM. +5. On guest shutdown, logs are flushed to the data partition. + +### View boot logs + +Every boot gets its own timestamped session directory so logs are never +overwritten: + +``` +/hiperiso/logs/_/ +``` + +Mount the data partition on any machine and point the log tool at a session +directory: + +```sh +hiperiso-log analyze /hiperiso/logs/ubuntu-24.04_20260629T101530Z/ +hiperiso-log trace /hiperiso/logs/ubuntu-24.04_20260629T101530Z/trace.bin --format json +hiperiso-log serial /hiperiso/logs/ubuntu-24.04_20260629T101530Z/serial.log --stages +``` + +Three files are guaranteed in every session directory, even when the boot +fails before QEMU launches: + +``` +session.json # session manifest (structured metadata for agents) +status.json # live status, rewritten at every stage transition +env.txt # host environment snapshot (kernel, RAM, CPU, KVM, config) +``` + +The remaining files appear depending on how far the boot progressed: + +``` +serial.log # UART text (written by QEMU, Tier 1) +trace.bin # QEMU simpletrace (Tier 2+, only when trace_level != none) +qemu.cmdline # exact QEMU command line used +monitor.sock # QEMU monitor socket (live introspection while guest runs) +timing.dat # raw boot phase timing data (TSV: phaseuptimeiso8601) +timing.json # structured boot timing with per-phase deltas (agent-friendly) +hw/ # hardware inventory directory (see below) +network.pcap # network packet capture (only when net_dump is enabled) +report.json # generated analysis report (auto-run after QEMU exits) +report.txt # human-readable report +report.log # raw output from the analyze run +analysis.meta # flat KEY=VALUE derived analysis sidecar +HEARTBEAT # periodic liveness marker from the flush daemon +SESSION_COMPLETE # sentinel, present when the session finished (success or failure) +FAILURE.txt # human-readable failure summary (present only on failure) +``` + +### Hardware inventory (`hw/`) + +Every session collects a comprehensive hardware inventory from both the host +(initramfs kernel) and the guest (QEMU virtual machine). This data provides +ground truth about exactly what hardware the guest OS sees — invaluable for +OS development, driver writing, and agentic debugging. + +#### Host-side data (always present) + +Collected from `/proc` and `/sys` before QEMU launches. Describes the real +hardware the initramfs kernel sees. + +``` +hw/ +├── host_cpuinfo.txt # full /proc/cpuinfo +├── host_meminfo.txt # full /proc/meminfo +├── host_interrupts.txt # /proc/interrupts +├── host_iomem.txt # /proc/iomem (physical memory map) +├── host_ioports.txt # /proc/ioports +├── host_cmdline.txt # /proc/cmdline +├── host_version.txt # /proc/version +├── host_diskstats.txt # /proc/diskstats +├── host_pci_devices.txt # /proc/bus/pci/devices +├── host_cpu_cache.txt # CPU cache topology (levels, sizes, sharing) +├── host_cpu_topology.txt # CPU core/socket/thread IDs +├── host_numa.txt # NUMA node topology +├── host_iommu.txt # IOMMU groups (VT-d/AMD-Vi) +├── host_kvm.txt # /dev/kvm presence +├── host_block.txt # block device info +├── host_usb.txt # USB device tree (vendor/product IDs) +├── host_dmi.txt # DMI/SMBIOS identifiers (board, BIOS, product) +├── host_efi.txt # EFI firmware info (if UEFI boot) +├── host_dmesg.txt # host kernel boot messages +├── host_kernel_config.gz # kernel .config (if /proc/config.gz available) +├── acpi/ # host ACPI tables (binary: DSDT, FACP, APIC, MCFG...) +├── smbios/ # host DMI/SMBIOS raw tables +└── kvm_caps.json # KVM capabilities (module params: nested, ept, vpid...) +``` + +#### Guest-side data (present if QEMU ran ≥ 3 seconds) + +Collected via QEMU HMP monitor pipe. Describes the virtual hardware the +guest OS sees. Not all files may be present — depends on QEMU version and +configuration. + +``` +hw/ +├── qemu_version.txt # QEMU version string +├── qemu_qtree.txt # full QEMU device tree (buses, devices, properties) +├── qemu_pci.txt # PCI device list as seen by guest +├── qemu_cpuid.txt # CPUID leaves exposed to guest +├── qemu_memmap.txt # guest memory map (hierarchical view) +├── qemu_memmap_flat.txt # guest memory map (flat view, all regions) +├── qemu_ioapic.txt # IO-APIC interrupt routing state +├── qemu_lapic.txt # Local APIC state per vCPU +├── qemu_irq.txt # IRQ statistics (delivery counts) +├── qemu_registers.txt # CPU register snapshot (RAX-R15, RIP, flags, segments) +├── qemu_tlb.txt # TLB entries (TCG mode only) +├── qemu_numa.txt # guest NUMA topology +├── qemu_hpet.txt # HPET timer state +├── qemu_qomtree.txt # QOM object hierarchy +├── qemu_smbios.txt # SMBIOS table (if QEMU supports info smbios) +├── qemu_chardev.txt # QEMU character devices +├── qemu_block.txt # QEMU block devices (CD-ROM, disk, floppy) +├── qemu_net.txt # QEMU network devices +├── qemu_monitor_raw.txt # raw combined monitor output (all sections) +└── pci_summary.json # structured PCI device list (bus/dev/fn, vendor/device IDs) +``` + +Host files are always present. QEMU files appear only if QEMU launched +successfully and the guest ran long enough for the monitor commands to +execute (3-second delay after QEMU start). + +### Boot timing (`timing.json`) + +Records microsecond-precision timestamps at each boot phase transition +using `/proc/uptime` as a monotonic clock. Lets agents and developers +identify performance bottlenecks across the full hypervisor boot stack. + +```json +{ + "total_duration_s": 15.234, + "phases": [ + {"phase": "kernel_start", "uptime_s": 0.512, "timestamp_utc": "...", "delta_s": null}, + {"phase": "session_created", "uptime_s": 2.341, "timestamp_utc": "...", "delta_s": 1.829}, + {"phase": "env_written", "uptime_s": 2.520, "timestamp_utc": "...", "delta_s": 0.179}, + {"phase": "kvm_checked", "uptime_s": 2.890, "timestamp_utc": "...", "delta_s": 0.370}, + {"phase": "iso_verified", "uptime_s": 2.910, "timestamp_utc": "...", "delta_s": 0.020}, + {"phase": "qemu_launching", "uptime_s": 3.001, "timestamp_utc": "...", "delta_s": 0.091}, + {"phase": "qemu_started", "uptime_s": 3.102, "timestamp_utc": "...", "delta_s": 0.101}, + {"phase": "qemu_exited", "uptime_s": 14.567, "timestamp_utc": "...", "delta_s": 11.465}, + {"phase": "qemu_returned", "uptime_s": 14.570, "timestamp_utc": "...", "delta_s": 0.003}, + {"phase": "report_done", "uptime_s": 15.200, "timestamp_utc": "...", "delta_s": 0.630}, + {"phase": "session_finalized","uptime_s": 15.234, "timestamp_utc": "...", "delta_s": 0.034} + ] +} +``` + +`SESSION_COMPLETE` is the signal that the session finished and all logs were +flushed. If it is absent, check `status.json` for the last known stage and +`HEARTBEAT` to see whether the flush daemon was still alive. + +For cross-session ingestion, hiperiso also appends one compact JSON object per +session to: + +``` +/hiperiso/logs/index.jsonl +``` + +The session manifest (`session.json`) is written when the session starts and +rewritten at the end as the canonical final manifest. For the live in-progress +state, read `status.json`: + +```json +{ + "session_id": "ubuntu-24.04_20260629T101530Z", + "status": "started", + "iso_path": "/ISOs/ubuntu-24.04-desktop-amd64.iso", + "iso_basename": "ubuntu-24.04-desktop-amd64.iso", + "log_dir": "/hiperiso/logs/ubuntu-24.04_20260629T101530Z", + "start_time_utc": "2026-06-29T10:15:30Z", + "kvm": "available", + "trace_level": "standard", + "guest_ram_mb": 2048, + "guest_cpus": 2 +} +``` + +The final manifest (rewritten at session end) adds `end_time_utc`, `result`, +`qemu_exit_code`, `report_available`, `host` (CPU model, RAM, virt support), +`analysis` (boot result, failure domain, error count, timing), and +`hw_inventory` (list of collected hardware data files). + +### Fallback mode + +If `/dev/kvm` is absent (virtualization disabled) and `hiperiso_fallback` is +not set, hiperiso captures a failure session rather than launching a guest. +The session directory still gets `session.json`, `status.json`, and `env.txt`, +plus a `FAILURE.txt` explaining what went wrong, and the analysis report is +generated. Set `hiperiso_fallback=1` to force TCG software emulation instead: +the ISO boots (very slowly) with full logging enabled. + +--- + +## Configuration + +Edit `/hiperiso/hiperiso.json` on the **data partition** (copied from +`config/hiperiso.json.example`). See **[config/README.md](config/README.md)** +for every option. Highlights: + +```jsonc +{ + "control": { "default_ram": 2048, "default_trace_level": "standard" }, + "iso_overrides": { + "*./windows11*.iso": { "ram": 4096, "tpm": true, "cpu_features": ["vmx"] } + } +} +``` + +Trace tiers: `standard` (serial + disk I/O), `detailed` (+ PCI/port I/O), +`full` (+ every MMIO access — very slow, debug only). + +Network capture: set `"net_dump": true` in `control_ext` or per-ISO in +`iso_overrides` to capture all guest network traffic as a pcap file +(`network.pcap` in the session directory). Adds a virtio-net-pci device +to the guest and a QEMU filter-dump. + +--- + +## Troubleshooting + +**`/dev/kvm` missing → "Falling back to direct boot mode."** +Enable VT-x/AMD-V in the host BIOS/UEFI. hiperiso then uses hypervisor mode +with full logging. Disabling Secure Boot on the host may also be required. + +**Guest boots to OVMF shell or hangs.** +- Ensure the ISO is a valid hybrid/UEFI-bootable image. +- Try `trace_level: detailed` and inspect `serial.log` for the failure point. +- Windows 11 ISOs need `tpm: true` and a `swtpm` socket — see + `config/iso_overrides/windows11.json`. + +**USB not booting.** +- Boot in **UEFI mode** (not Legacy/CSM). +- Re-run the installer and confirm `EFI/BOOT/BOOTX64.EFI` exists on the ESP. +- Some firmums need the USB entry added manually via the boot menu (F12/F8). + +**`grub-install failed`.** +Install `grub2-efi-amd64` and `grub2-efi-amd64-modules` (Debian/Ubuntu) or +`grub2-efi-x64` + `grub2-efi-x64-modules` (Fedora/RHEL). + +**Build: OVMF step fails.** +Install `nasm`, `iasl` (acpica-tools), `python3`, and run from inside +`build/edk2`. The script builds EDK2 BaseTools first; ensure `make` and a +recent GCC (GCC5 toolchain = GCC 5+) are present. + +**QEMU configure complains about missing deps.** +Install `ninja-build`, `libglib2.0-dev`, `libpixman-1-dev`, `pkg-config`. + +--- + +## Project layout + +``` +hiperiso/ +├── scripts/ # build_all, download_sources, configure_qemu, build_initramfs +├── host/kernel/ # hiperiso_defconfig (minimal KVM kernel) +├── host/initramfs/ # init scripts (separate task) +├── firmware/ # build_ovmf.sh +├── grub2/ # GRUB2 hiperiso module (separate task) +├── logging/ # trace-*.events + hiperiso-log tool (separate task) +├── installer/ # Hiperiso2Disk.sh +├── config/ # hiperiso.json.example + per-ISO overrides +├── Makefile +└── INTERFACES.sh # single source of truth for all interfaces +``` diff --git a/config/README.md b/config/README.md new file mode 100644 index 0000000..e1c3729 --- /dev/null +++ b/config/README.md @@ -0,0 +1,104 @@ +# hiperiso configuration + +hiperiso is configured through a single JSON file placed on the **data +partition** of the USB at `/hiperiso/hiperiso.json`. The format is +[Ventoy](https://www.ventoy.net)-compatible so existing Ventoy configs need +only minor changes. Start from `hiperiso.json.example`. + +> JSON cannot contain comments, so every option is described here. + +## Location + +``` +/ +└── hiperiso/ + └── hiperiso.json ← global config (copied from hiperiso.json.example) +``` + +## Top-level sections + +| Section | Purpose | +|----------------|------------------------------------------------------------| +| `control` | Default hypervisor/guest settings applied to every boot. | +| `theme` | GRUB2 menu theme. | +| `menu_alias` | Rename ISO entries in the menu. | +| `menu_tip` | Show a one-line tip under a menu entry. | +| `iso_overrides`| Per-ISO settings keyed by glob pattern. | + +## `control` + +| Key | Type | Default | Meaning | +|-----------------------|---------|-------------|----------------------------------------------------| +| `default_ram` | number | `2048` | Guest RAM in MB. | +| `default_cpus` | number | `2` | Guest vCPUs. | +| `default_trace_level` | string | `standard` | `none` \| `standard` \| `detailed` \| `full`. | +| `default_display` | string | `none` | QEMU display: `none` \| `gtk` \| `vnc`. | +| `default_vga` | string | `none` | QEMU VGA: `none` \| `std` \| `virtio`. | +| `fallback_no_kvm` | boolean | `true` | Fall back to direct boot if `/dev/kvm` is missing. | +| `net_dump` | boolean | `false` | Capture guest network traffic to `network.pcap`. | +| `timeout` | number | `10` | GRUB2 menu auto-boot timeout in seconds. | + +These map directly to the kernel command-line parameters defined in +`INTERFACES.sh` (`HIPERISO_GUEST_RAM`, `HIPERISO_GUEST_CPUS`, +`HIPERISO_TRACE_LEVEL`, `HIPERISO_DISPLAY`, `HIPERISO_VGA`, +`HIPERISO_FALLBACK`). + +## `theme` + +| Key | Type | Meaning | +|--------------|--------|------------------------------------------| +| `file` | string | Path to a GRUB2 `theme.txt`. | +| `gfxmode` | string | GRUB2 graphics resolution. | +| `gfxpayload` | string | `keep` to preserve resolution for guest. | + +## `menu_alias` / `menu_tip` + +Both are arrays of objects with an `iso` path (relative to the data +partition) and a value: + +```json +[ + { "iso": "/ISOs/ubuntu-24.04.iso", "alias": "Ubuntu 24.04 LTS" } +] +``` + +## `iso_overrides` + +A map of **glob pattern → override object**. The first matching pattern wins. +Glob syntax matches Ventoy (anchored at the filename). + +| Key | Type | Meaning | +|------------------|-----------|------------------------------------------------------| +| `ram` | number | Override guest RAM (MB). | +| `cpus` | number | Override guest vCPUs. | +| `tpm` | boolean | Attach a virtual TPM (`swtpm`) for Windows 11. | +| `secure_boot` | boolean | Use Secure-Boot-enabled OVMF vars. | +| `cpu_features` | string[] | Extra CPU flags, e.g. `["vmx"]` for nested virt. | +| `trace_level` | string | Override trace tier for this ISO. | +| `display` | string | Override display mode. | +| `net_dump` | boolean | Capture network traffic for this ISO. | +| `qemu_extra_args`| string[] | Raw extra arguments appended to the QEMU command line.| + +```json +{ + "*./windows11*.iso": { "ram": 4096, "tpm": true, "cpu_features": ["vmx"] } +} +``` + +## Standalone override files + +`config/iso_overrides/.json` holds a single override (see +`windows11.json`). These are intended as reusable templates you can merge +into `iso_overrides` or ship alongside a curated ISO set. + +## Trace tiers (recap) + +| Tier | Events | Overhead | +|---------------|---------------------------------------------------|----------| +| `standard` | serial + basic disk/CD-ROM I/O | ~0% | +| `detailed` | + PCI config-space + port I/O | low | +| `full` | + every MMIO access | 10–100× | +| `none` | serial only (no trace) | 0% | + +The corresponding event lists live in `logging/trace-*.events` and are copied +to the ESP at `/EFI/hiperiso/trace/` during install.