# hiperiso

A hypervisor-based ISO boot tool with full bootlogging — like Ventoy, but it
boots ISOs inside a **KVM + QEMU** virtual machine and captures the entire boot
(serial console, disk I/O, PCI/port I/O traces). The guest OS runs completely
unmodified.

```
USB → GRUB2 → minimal Linux kernel (KVM built-in) + initramfs →
     QEMU (KVM accel) → OVMF UEFI → ISO boots as an emulated CD-ROM
```

## How it differs from Ventoy

Ventoy injects hooks into the booted OS (initrd patching, device-mapper, WIM
patching, 60+ distro hooks). hiperiso instead presents the ISO as a native
emulated AHCI CD-ROM to a KVM guest, so **no guest modification is needed** and
the hypervisor has full visibility into every I/O operation.

---

## Prerequisites

### Build host

- A Linux x86_64 machine with VT-x / AMD-V
- Build tools: `gcc`, `g++`, `make`, `bc`, `flex`, `bison`, `cpio`, `gzip`,
  `wget`, `tar`
- EDK2 build deps: `nasm`, `iasl` (acpica), `python3`
- QEMU build deps: `ninja-build`, `pkg-config`, `libglib2.0-dev`, `libpixman-1-dev`
- Kernel build deps: `libelf-dev`, `libssl-dev`
- ~6 GB free disk for sources + build trees

### Install host (where you prepare the USB)

- `sgdisk` (package `gdisk` / `gptfdisk`)
- `dosfstools` (`mkfs.vfat`)
- `exfatprogs` (`mkfs.exfat`) **or** `ntfs-3g` (`mkntfs`)
- `grub2-efi-amd64` + `grub2-efi-amd64-modules` (provides `grub-install --target=x86_64-efi`)
- `util-linux` (`findmnt`, `lsblk`)

### Target machine (the PC you boot from USB)

- x86_64 with hardware virtualization (VT-x/AMD-V) **enabled in BIOS/UEFI**
- UEFI boot mode (a Legacy/CSM path is planned, not in the MVP)

---

## Build

### 1. One-shot full build

```sh
./scripts/build_all.sh
```

This downloads sources, builds the kernel, QEMU, OVMF, GRUB2, the log tool,
packs the initramfs, and assembles a distributable payload in `build/payload/`.

Re-run later without re-downloading:

```sh
SKIP_DOWNLOAD=1 ./scripts/build_all.sh
```

### 2. Build individual components

Each stage is a standalone script:

```sh
./scripts/download_sources.sh              # fetch + extract kernel, qemu, edk2, grub2, busybox
cd build/linux && cp ../../host/kernel/hiperiso_defconfig .config && make olddefconfig && make -j"$(nproc)" bzImage
cd build/qemu  && ../../scripts/configure_qemu.sh && make -j"$(nproc)" qemu-system-x86_64
cd build/edk2  && bash ../../firmware/build_ovmf.sh
```

Or use the provided Makefile:

```sh
make all        # kernel qemu ovmf grub2 initramfs hiperiso-log
make dist       # assemble build/payload/
make clean      # remove build/
```

### 3. Output layout

```
build/payload/
├── Hiperiso2Disk.sh
├── config/hiperiso.json.example
└── EFI/
    ├── BOOT/
    │   ├── BOOTX64.EFI          # GRUB2 (hiperiso module)
    │   └── grub.cfg
    └── hiperiso/
        ├── vmlinuz              # host kernel (KVM built-in)
        ├── initramfs.cpio.gz    # busybox + qemu + init
        ├── OVMF.fd              # guest UEFI firmware (SMM + Secure Boot)
        ├── hiperiso-log         # log analysis tool
        └── trace/               # trace-*.events (standard/detailed/full)
```

---

## Install to a USB drive

> ⚠️ **This destroys all data on the target device.** Double-check the device name.

```sh
lsblk                              # identify your USB, e.g. /dev/sdX
sudo bash build/payload/Hiperiso2Disk.sh -I -g /dev/sdX
```

`Hiperiso2Disk.sh` defaults to MBR. Use `-g` for GPT. The standard Ventoy-style
layout is:

| # | Type   | FS     | Size   | Contents                                   |
|---|--------|--------|--------|--------------------------------------------|
| 1 | 0700   | exFAT  | rest   | Your ISOs + boot logs + config             |
| 2 | EF00   | FAT16  | 32 MB  | ESP: GRUB2, kernel, initramfs, OVMF        |

Copy your ISOs:

```sh
cp my_os.iso /media/$USER/HIPERISO/ISOs/
```

Options:

```sh
sudo bash Hiperiso2Disk.sh -i /dev/sdX          # install
sudo bash Hiperiso2Disk.sh -I -g /dev/sdX       # force reinstall using GPT
sudo bash Hiperiso2Disk.sh -i -L MYUSB /dev/sdX # custom data-partition label
```

---

## Usage

1. Boot the PC from the USB (pick the **UEFI** entry).
2. GRUB2 lists ISOs found under `/ISOs/`. Select one.
3. GRUB2 boots the host kernel; the initramfs detects `/dev/kvm` and launches
   QEMU with the selected ISO attached as a CD-ROM.
4. The ISO's OS boots **unmodified** inside the VM.
5. On guest shutdown, logs are flushed to the data partition.

### View boot logs

Every boot gets its own timestamped session directory so logs are never
overwritten:

```
<data partition>/hiperiso/logs/<iso-basename>_<UTC-timestamp>/
```

Mount the data partition on any machine and point the log tool at a session
directory:

```sh
hiperiso-log analyze /hiperiso/logs/ubuntu-24.04_20260629T101530Z/
hiperiso-log trace  /hiperiso/logs/ubuntu-24.04_20260629T101530Z/trace.bin --format json
hiperiso-log serial /hiperiso/logs/ubuntu-24.04_20260629T101530Z/serial.log --stages
```

Three files are guaranteed in every session directory, even when the boot
fails before QEMU launches:

```
session.json     # session manifest (structured metadata for agents)
status.json      # live status, rewritten at every stage transition
env.txt          # host environment snapshot (kernel, RAM, CPU, KVM, config)
```

The remaining files appear depending on how far the boot progressed:

```
serial.log       # UART text (written by QEMU, Tier 1)
trace.bin        # QEMU simpletrace (Tier 2+, only when trace_level != none)
qemu.cmdline     # exact QEMU command line used
monitor.sock     # QEMU monitor socket (live introspection while guest runs)
timing.dat       # raw boot phase timing data (TSV: phase<TAB>uptime<TAB>iso8601)
timing.json      # structured boot timing with per-phase deltas (agent-friendly)
hw/              # hardware inventory directory (see below)
network.pcap     # network packet capture (only when net_dump is enabled)
report.json      # generated analysis report (auto-run after QEMU exits)
report.txt       # human-readable report
report.log       # raw output from the analyze run
analysis.meta    # flat KEY=VALUE derived analysis sidecar
HEARTBEAT        # periodic liveness marker from the flush daemon
SESSION_COMPLETE # sentinel, present when the session finished (success or failure)
FAILURE.txt      # human-readable failure summary (present only on failure)
```

### Hardware inventory (`hw/`)

Every session collects a comprehensive hardware inventory from both the host
(initramfs kernel) and the guest (QEMU virtual machine). This data provides
ground truth about exactly what hardware the guest OS sees — invaluable for
OS development, driver writing, and agentic debugging.

#### Host-side data (always present)

Collected from `/proc` and `/sys` before QEMU launches. Describes the real
hardware the initramfs kernel sees.

```
hw/
├── host_cpuinfo.txt        # full /proc/cpuinfo
├── host_meminfo.txt        # full /proc/meminfo
├── host_interrupts.txt     # /proc/interrupts
├── host_iomem.txt          # /proc/iomem (physical memory map)
├── host_ioports.txt        # /proc/ioports
├── host_cmdline.txt        # /proc/cmdline
├── host_version.txt        # /proc/version
├── host_diskstats.txt      # /proc/diskstats
├── host_pci_devices.txt    # /proc/bus/pci/devices
├── host_cpu_cache.txt      # CPU cache topology (levels, sizes, sharing)
├── host_cpu_topology.txt   # CPU core/socket/thread IDs
├── host_numa.txt           # NUMA node topology
├── host_iommu.txt          # IOMMU groups (VT-d/AMD-Vi)
├── host_kvm.txt            # /dev/kvm presence
├── host_block.txt          # block device info
├── host_usb.txt            # USB device tree (vendor/product IDs)
├── host_dmi.txt            # DMI/SMBIOS identifiers (board, BIOS, product)
├── host_efi.txt            # EFI firmware info (if UEFI boot)
├── host_dmesg.txt          # host kernel boot messages
├── host_kernel_config.gz   # kernel .config (if /proc/config.gz available)
├── acpi/                   # host ACPI tables (binary: DSDT, FACP, APIC, MCFG...)
├── smbios/                 # host DMI/SMBIOS raw tables
└── kvm_caps.json           # KVM capabilities (module params: nested, ept, vpid...)
```

#### Guest-side data (present if QEMU ran ≥ 3 seconds)

Collected via QEMU HMP monitor pipe. Describes the virtual hardware the
guest OS sees. Not all files may be present — depends on QEMU version and
configuration.

```
hw/
├── qemu_version.txt        # QEMU version string
├── qemu_qtree.txt          # full QEMU device tree (buses, devices, properties)
├── qemu_pci.txt            # PCI device list as seen by guest
├── qemu_cpuid.txt          # CPUID leaves exposed to guest
├── qemu_memmap.txt         # guest memory map (hierarchical view)
├── qemu_memmap_flat.txt    # guest memory map (flat view, all regions)
├── qemu_ioapic.txt         # IO-APIC interrupt routing state
├── qemu_lapic.txt          # Local APIC state per vCPU
├── qemu_irq.txt            # IRQ statistics (delivery counts)
├── qemu_registers.txt      # CPU register snapshot (RAX-R15, RIP, flags, segments)
├── qemu_tlb.txt            # TLB entries (TCG mode only)
├── qemu_numa.txt           # guest NUMA topology
├── qemu_hpet.txt           # HPET timer state
├── qemu_qomtree.txt        # QOM object hierarchy
├── qemu_smbios.txt         # SMBIOS table (if QEMU supports info smbios)
├── qemu_chardev.txt        # QEMU character devices
├── qemu_block.txt          # QEMU block devices (CD-ROM, disk, floppy)
├── qemu_net.txt            # QEMU network devices
├── qemu_monitor_raw.txt    # raw combined monitor output (all sections)
└── pci_summary.json        # structured PCI device list (bus/dev/fn, vendor/device IDs)
```

Host files are always present. QEMU files appear only if QEMU launched
successfully and the guest ran long enough for the monitor commands to
execute (3-second delay after QEMU start).

### Boot timing (`timing.json`)

Records microsecond-precision timestamps at each boot phase transition
using `/proc/uptime` as a monotonic clock. Lets agents and developers
identify performance bottlenecks across the full hypervisor boot stack.

```json
{
  "total_duration_s": 15.234,
  "phases": [
    {"phase": "kernel_start",    "uptime_s": 0.512, "timestamp_utc": "...", "delta_s": null},
    {"phase": "session_created", "uptime_s": 2.341, "timestamp_utc": "...", "delta_s": 1.829},
    {"phase": "env_written",     "uptime_s": 2.520, "timestamp_utc": "...", "delta_s": 0.179},
    {"phase": "kvm_checked",     "uptime_s": 2.890, "timestamp_utc": "...", "delta_s": 0.370},
    {"phase": "iso_verified",    "uptime_s": 2.910, "timestamp_utc": "...", "delta_s": 0.020},
    {"phase": "qemu_launching",  "uptime_s": 3.001, "timestamp_utc": "...", "delta_s": 0.091},
    {"phase": "qemu_started",    "uptime_s": 3.102, "timestamp_utc": "...", "delta_s": 0.101},
    {"phase": "qemu_exited",     "uptime_s": 14.567, "timestamp_utc": "...", "delta_s": 11.465},
    {"phase": "qemu_returned",   "uptime_s": 14.570, "timestamp_utc": "...", "delta_s": 0.003},
    {"phase": "report_done",     "uptime_s": 15.200, "timestamp_utc": "...", "delta_s": 0.630},
    {"phase": "session_finalized","uptime_s": 15.234, "timestamp_utc": "...", "delta_s": 0.034}
  ]
}
```

`SESSION_COMPLETE` is the signal that the session finished and all logs were
flushed. If it is absent, check `status.json` for the last known stage and
`HEARTBEAT` to see whether the flush daemon was still alive.

For cross-session ingestion, hiperiso also appends one compact JSON object per
session to:

```
<data partition>/hiperiso/logs/index.jsonl
```

The session manifest (`session.json`) is written when the session starts and
rewritten at the end as the canonical final manifest. For the live in-progress
state, read `status.json`:

```json
{
  "session_id": "ubuntu-24.04_20260629T101530Z",
  "status": "started",
  "iso_path": "/ISOs/ubuntu-24.04-desktop-amd64.iso",
  "iso_basename": "ubuntu-24.04-desktop-amd64.iso",
  "log_dir": "/hiperiso/logs/ubuntu-24.04_20260629T101530Z",
  "start_time_utc": "2026-06-29T10:15:30Z",
  "kvm": "available",
  "trace_level": "standard",
  "guest_ram_mb": 2048,
  "guest_cpus": 2
}
```

The final manifest (rewritten at session end) adds `end_time_utc`, `result`,
`qemu_exit_code`, `report_available`, `host` (CPU model, RAM, virt support),
`analysis` (boot result, failure domain, error count, timing), and
`hw_inventory` (list of collected hardware data files).

### Fallback mode

If `/dev/kvm` is absent (virtualization disabled) and `hiperiso_fallback` is
not set, hiperiso captures a failure session rather than launching a guest.
The session directory still gets `session.json`, `status.json`, and `env.txt`,
plus a `FAILURE.txt` explaining what went wrong, and the analysis report is
generated. Set `hiperiso_fallback=1` to force TCG software emulation instead:
the ISO boots (very slowly) with full logging enabled.

---

## Configuration

Edit `/hiperiso/hiperiso.json` on the **data partition** (copied from
`config/hiperiso.json.example`). See **[config/README.md](config/README.md)**
for every option. Highlights:

```jsonc
{
  "control": { "default_ram": 2048, "default_trace_level": "standard" },
  "iso_overrides": {
    "*./windows11*.iso": { "ram": 4096, "tpm": true, "cpu_features": ["vmx"] }
  }
}
```

Trace tiers: `standard` (serial + disk I/O), `detailed` (+ PCI/port I/O),
`full` (+ every MMIO access — very slow, debug only).

Network capture: set `"net_dump": true` in `control_ext` or per-ISO in
`iso_overrides` to capture all guest network traffic as a pcap file
(`network.pcap` in the session directory). Adds a virtio-net-pci device
to the guest and a QEMU filter-dump.

---

## Troubleshooting

**`/dev/kvm` missing → "Falling back to direct boot mode."**
Enable VT-x/AMD-V in the host BIOS/UEFI. hiperiso then uses hypervisor mode
with full logging. Disabling Secure Boot on the host may also be required.

**Guest boots to OVMF shell or hangs.**
- Ensure the ISO is a valid hybrid/UEFI-bootable image.
- Try `trace_level: detailed` and inspect `serial.log` for the failure point.
- Windows 11 ISOs need `tpm: true` and a `swtpm` socket — see
  `config/iso_overrides/windows11.json`.

**USB not booting.**
- Boot in **UEFI mode** (not Legacy/CSM).
- Re-run the installer and confirm `EFI/BOOT/BOOTX64.EFI` exists on the ESP.
- Some firmums need the USB entry added manually via the boot menu (F12/F8).

**`grub-install failed`.**
Install `grub2-efi-amd64` and `grub2-efi-amd64-modules` (Debian/Ubuntu) or
`grub2-efi-x64` + `grub2-efi-x64-modules` (Fedora/RHEL).

**Build: OVMF step fails.**
Install `nasm`, `iasl` (acpica-tools), `python3`, and run from inside
`build/edk2`. The script builds EDK2 BaseTools first; ensure `make` and a
recent GCC (GCC5 toolchain = GCC 5+) are present.

**QEMU configure complains about missing deps.**
Install `ninja-build`, `libglib2.0-dev`, `libpixman-1-dev`, `pkg-config`.

---

## Project layout

```
hiperiso/
├── scripts/            # build_all, download_sources, configure_qemu, build_initramfs
├── host/kernel/        # hiperiso_defconfig (minimal KVM kernel)
├── host/initramfs/     # init scripts (separate task)
├── firmware/           # build_ovmf.sh
├── grub2/              # GRUB2 hiperiso module (separate task)
├── logging/            # trace-*.events + hiperiso-log tool (separate task)
├── installer/          # Hiperiso2Disk.sh
├── config/             # hiperiso.json.example + per-ISO overrides
├── Makefile
└── INTERFACES.sh       # single source of truth for all interfaces
```