Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

ctos (ctsOS)

ctos is a small operating-system kernel you can study. It is written in Rust for 64-bit ARM (AArch64). You run it in the QEMU emulator, not as a desktop or phone OS. The nickname ctsOS is only for display; the repo and crate stay ctos. Why Rust (and what it does not buy): Advantages — Why Rust.

This book is the website. The same markdown lives under docs/ in the GitHub repo. There is no second marketing copy.

It is not a product site. Do not say “secure OS” or “production ready.” Say “EL0 isolated” only as the accepted ADR-091 sentence, with its limits (one core, TCG default smoke machine; not speculative-execution / side-channel, not real hardware, not a certification). Status words need a probe.

What it is

ctos boots under QEMU’s virt machine and prints on a serial port (the PL011 UART). After that it grows one honest mile at a time: exceptions, paging, a heap, a tiny scheduler, then measured security and performance cuts.

Core principles drive. Roadmap tracks (A = freestanding hosting, B = Linux-compat research, N = network foundation — N1 first-frame ADR-066, N2 ICMP ping ADR-067, N3 UDP ADR-070, N3.x EL0 UDP SVC ADR-071, N4 EL0 net sample ADR-068, N5 thin TCP ADR-074, scope ADR-063) are subordinate. A track does not outrank a principle.

Primary path: QEMU virt + UART (ADR-003). Frozen requirement IDs: FR-01–FR-15 and NFR-01–NFR-14.

Driving principles

Everyday meaning first; frozen IDs second. Same list as the pillars hub. This landing is the visitor-facing source. docs/framework/ stays for deep links — do not keep a second marketing copy.

PrincipleIn everyday wordsFrozen ID
HonestyIf we did not run a check, we do not say it works. Unprobed stays Unknown.NFR-06 · ledger
AntifragilityThe same miss twice becomes an automated sensor, not another README paragraph.NFR-05 · Antifragility
SecurityWrite down what we fear, then prove a slice. Not a “secure OS” slogan.NFR-10 · Security
PerformanceMeasure a known path first. No invented benches.NFR-07 · Performance
Document-firstWrite the decision, then the code. One milestone → one branch → one PR.FR-14 / NFR-13
flowchart TD
  P["Core principles<br/>honesty · antifragility · security<br/>performance · document-first"]
  L["Three pillars<br/>antifragility · security · performance"]
  T["Tracks A / B / N<br/>hosting · Linux research · net first-frame<br/>subordinate — not the driver"]
  P --> L --> T

Principles sit above pillars. Tracks sit below both. A1–A4 are ABI + CRT + loader + standing-task miles only. Not a claim that Track A hosting is built.

What runs today

Rebuild recipes for samples that already have probes. Details: What can run today. In-tree index: user/README.md.

  1. Two kernel tasks that take turns — they print on the serial port and yield. Not preemptive. Not two CPUs.
  2. A serial echo gadget — one byte in, a line out. No terminal, no line editor.
  3. A short lower-privilege stub — a few instructions in the CPU’s user mode, including the A1 SVC ABI trip, an A2 libctos hello, an A3 guest PT_LOAD of that same hello, and an A4 standing task until exit (exit / uart_write / yield), then a call back into the kernel. Not a process. No libc. Freestanding catalog on FAT: /hello, /fsdemo (memfs VFS), /fatdemo (FAT /probe), /yldemo (yield rounds). Optional recipes: memfs named buffers and FAT16 on virtio-blk (read + write depth).

Cannot run: Linux programs, a shell, Python, network servers (Track N has virtio-net ARP + ICMP + minimal UDP + thin TCP plus tiny EL0 net/UDP SVC samples — ADR-066 / ADR-067 / ADR-070 / ADR-071 / ADR-068 / ADR-074; not BSD sockets / TCP product), POSIX filesystem apps, extra CPUs, or containers (guest OCI/Docker is a non-goal). An OS slot vs a separate app slot has an A9 first cut (FAT /hello) plus leftover cross-update on ba6541c (Verified miles); catalog also has /fsdemo / /fatdemo / /yldemo / /netdemo / /udpdemo. Product freestanding app hosting is Verified under ADR-048 / ADR-052 (not Linux/POSIX). Umbrella “EL0 isolated” is Verified only as the accepted ADR-091 sentence (M6, 2026-09-28; closure table ADR-060).

flowchart LR
  subgraph today ["Verified today"]
    W["Two UART workers"]
    E["Serial echo gadget"]
    S["Short user-mode stub"]
  end
  subgraph no ["Cannot run"]
    L["Linux binaries / shell / Python"]
    N["Network / files / extra CPUs"]
    C["Containers"]
  end
  today -.-> no

Left side matches existing smoke markers. Right side is out of scope. Do not say “apps run.”

How to build

You need nightly Rust and QEMU. Pages is not required for kernel work. Full list: Prerequisites.

rustup toolchain install nightly
rustup component add rust-src llvm-tools-preview
cargo build                 # ELF at target/aarch64-ctos/debug/ctos
./scripts/qemu-smoke.sh     # fail-closed serial + tests
flowchart LR
  R["rustup nightly"] --> B["cargo build"] --> Q["qemu-smoke"]

That rebuilds the kernel, including any in-tree code you add. It is not “port an app.”

If you want…Go here
What is in vs out (rebuild recipes)What can run today · Drawbacks / limits
Why POSIX does not portBuilding or porting
Files (memfs + FAT16)Filesystem
App host / containersHosting apps — freestanding product claim Verified (ADR-048 / ADR-052); containers: non-goal (ADR-029). Linux-compat frame: ADR-031 (not userspace).
How we measureKPIs
Why the repo is run this way · Why RustAdvantages · Why Rust
Deep divesVision · FR / NFR · Architecture · Roadmap · Ledger

Kernel build notes also live in the GitHub README.

URLs

URLHonesty
https://ctos.artof.linkVerified. HTTPS 200, cert for this name, landing shows Driving principles. Main deploy 34653046584 after #30.
https://artofdream.github.io/ctosRedirect. This path (no trailing slash) 301s to the custom domain. A trailing slash 404’d on the 2026-09-11 probe — do not treat github.io as a second live tree.

Publish mechanics: Docs website + DNS.

What can run today

Plain English. These are rebuild recipes for in-tree samples that already have probes. They are not third-party applications and not a product runtime.

Track A / A8 (issue #39) is this page: document the sample classes and how to rebuild them. A9 (ADR-030) adds two host artifacts + FAT /hello. Leftover (ADR-032): A2–A4 load that file (no production embed); same ELF on this OS and ba6541c (cross-update Verified). Product freestanding app hosting is Verified under ADR-048 / ADR-052 (checklist: hosting-apps.md). Not Linux/POSIX/containers.

Status of each probe: honesty ledger. Walkthroughs: apps-today.md. In-tree index: user/README.md. Umbrella “EL0 isolated” is Verified only as the accepted ADR-091 sentence (M6; one core, TCG default smoke machine; not speculative-execution / side-channel, not real hardware, not a certification). Do not say “apps,” “userspace,” or “secure OS” as if a general-purpose OS existed.

Freestanding sample catalog (glance)

Six published EL0 ELFs on one FAT16 volume. Same rebuild path. Not Linux/POSIX.

flowchart LR
  FAT["FAT16 volume"] --> H["/hello<br/>UART + one yield"]
  FAT --> F["/fsdemo<br/>memfs VFS"]
  FAT --> D["/fatdemo<br/>FAT /probe"]
  FAT --> Y["/yldemo<br/>yield rounds"]
  FAT --> N["/netdemo<br/>net SVCs"]
  FAT --> U["/udpdemo<br/>UDP DNS SVC"]
  FAT --> M["/mkdemo<br/>fs_mkdir SVC"]
  FAT --> T["/tcpdemo<br/>net_tcp_echo SVC"]
  H --> EL0["Standing EL0<br/>libctos"]
  F --> EL0
  D --> EL0
  Y --> EL0
  N --> EL0
  U --> EL0

/hello is the A9 product-slot sample. /fsdemo (ADR-059), /fatdemo (ADR-061), /yldemo (ADR-062), /netdemo (ADR-068), /udpdemo (ADR-071), /mkdemo (ADR-075), and /tcpdemo (ADR-076) deepen the catalog. Umbrella “EL0 isolated”: only the accepted ADR-091 sentence.

Privilege — where code runs

The CPU has privilege levels. EL1 is kernel privilege (where ctos runs). EL0 is lower privilege (user mode). A process would be a loaded program with its own address space, files, and a public ABI. That process does not exist yet.

flowchart TD
  EL1["EL1 — kernel privilege<br/>Verified: this is where ctos runs"]
  EL0["EL0 — lower privilege<br/>Verified: a short standing stub only"]
  PROC["A process / application<br/>Planned: not built"]
  EL1 --> EL0
  EL0 -.-> PROC

The stub is not a process. “EL0 isolated” means only the ADR-091 sentence.

A supervisor call (SVC) is the instruction the stub uses to ask the kernel for something. Reserved SVC #0–#2 stay test miles. A1 documented exit / uart_write / yield (syscall.md). A2 wraps those in libctos. That is not a process ABI.

One rebuild for every recipe

Every sample below rides the same hello path. There is no separate “run this app” command. You rebuild the kernel (and, for freestanding EL0 samples, build.rs publishes target/hello-libctos.elf + target/fs-libctos.elf + target/fat-libctos.elf + target/yield-libctos.elf + target/net-libctos.elf + target/udp-libctos.elf + target/mkdir-libctos.elf + target/tcp-libctos.elf; A2–A4 and A9 read FAT /hello; ADR-059 loads FAT /fsdemo; ADR-061 loads FAT /fatdemo; ADR-062 loads FAT /yldemo; ADR-068 loads FAT /netdemo; ADR-071 loads FAT /udpdemo; ADR-075 loads FAT /mkdemo; ADR-076 loads FAT /tcpdemo).

cargo build                 # aarch64-ctos.json; also builds user/hello-libctos + user/fs-libctos + user/fat-libctos + user/yield-libctos + user/net-libctos + user/udp-libctos
./scripts/qemu-smoke.sh     # fail-closed serial greps + cargo test + force-fail

qemu-smoke attaches the A7 FAT16 image (target/fat16.img) and injects UART 0x41. A machine that has not run that script (or Docker / GHA equivalent) stays Unknown for boot. File presence is not the probe.

flowchart LR
  R["edit in-tree source"] --> C["cargo build"] --> S["qemu-smoke"]
  S --> M["existing serial markers"]

Rebuild the guest. Do not drop in a foreign ELF.

Recipe 1 — Cooperative UART workers (EL1)

Two tasks on heap stacks that print a line and yield to each other. Same class as sched: task a / sched: task b / sched: ok (ADR-010, FR-11).

PiecePath
Workerssrc/sched.rs (task_a / task_b / run_two_tasks)
Probescripts/qemu-smoke.sh greps sched: ok and both task lines

Rebuild: edit those functions, then the one-rebuild commands above. A heartbeat / counter variant (print, yield, loop) is the natural next in-tree exercise. It is not in the tree and has no sched: beat marker — Planned until a probe greps one.

Not preemptive. Not two CPUs. Not EL0.

Recipe 2 — UART RX echo gadget

Read a byte from PL011 RX and print it. Same class as input: rx 0x41 (ADR-007, FR-08).

PiecePath
Guest pollsrc/uart.rs (observe_rx / observe_probe_byte)
Host injectscripts/qemu-serial-inject.py (default CTOS_INPUT_BYTE=0x41)

Rebuild: same cargo build + ./scripts/qemu-smoke.sh. The smoke host injects one byte. IRQs stay masked on this path. Do not wfi waiting for RX.

That is a byte in, a line out. No TTY, no line editor, no canonical mode, no virtio-keyboard.

Recipe 3 — Standing EL0 / libctos-loaded hello

A short payload in user mode (EL0) that uses the documented ABI, then returns. Miles stacked on FAT /hello:

MileSerial (do not invent extras)Decision
Standing stubel0: standing / el0: restoredADR-013
SVC ABIsvc: yield / svc: user-hi / svc: uart / svc: exit / svc: okADR-021
libctos hellolibctos: hi / libctos: ok / libctos: linkedADR-022
Guest PT_LOADloader: mapped / loader: okADR-023
Standing taskel0: task-enter / el0: task-active / el0: task-exit / el0: task-okADR-024
PiecePath
Hello sourceuser/hello-libctos/ (recipe README)
CRT / wrapperslibctos/
Host publishbuild.rs writes target/hello-libctos.elf; guest reads FAT /hello
Guest mapsrc/loader.rs

Rebuild the hello (optional, standalone):

cargo build --release \
  --manifest-path user/hello-libctos/Cargo.toml \
  --target user/hello-libctos/aarch64-ctos-user.json

A kernel cargo build already does that via build.rs and publishes the result. Then ./scripts/qemu-smoke.sh. There is no exec of a file on disk. The kernel ELF no longer embeds the hello bytes.

This is not a process. No libc, no argv, no loader for a foreign ELF. “EL0 isolated” means only the accepted ADR-091 sentence.

libctos wraps fs_create / fs_open / fs_read / fs_write / fs_close (SVC 19–23). The hello payload does not call them (UART + one yield). A second freestanding sample does: see Recipe 3b (fs-libctos, ADR-059). A third reads FAT via the same wrappers: see Recipe 3c (fat-libctos, ADR-061). A fourth exercises several cooperative yields: see Recipe 3d (yield-libctos, ADR-062). A fifth exercises EL0 net SVCs: see Recipe 3e (net-libctos, ADR-068). A sixth exercises EL0 UDP DNS SVC: see Recipe 3f (udp-libctos, ADR-071). The kernel /eprobe trampoline (fs: el0) remains a separate Verified EL0 VFS trip.

Recipe 3b — Standing EL0 / libctos VFS sample (fs-libctos)

Second freestanding EL0 payload that calls the VFS wrappers (ADR-059). Same class as libctos: fs-hi / libctos: fs-ok / fsdemo: ok.

PiecePath
Sampleuser/fs-libctos/
In-app path/memdemo (ADR-058: /mem prefix → memfs; flat grammar — no /mem/...)
FAT slot/fsdemo (target/fs-libctos.elf via mkfat16.py --app2)
Kernel probesrc/fsdemo.rs

Rebuild: same one-rebuild commands. Keep /hello + slot: ok. Not POSIX. Not getdents. Not a reopen of product app hosting (ADR-048/052).

Recipe 3c — Standing EL0 / libctos FAT sample (fat-libctos)

Third freestanding EL0 payload that opens/reads a FAT path through thin VFS (ADR-061). Same class as libctos: fat-hi / libctos: fat-ok / fatdemo: ok. Deeper than Recipe 3b (memfs only).

PiecePath
Sampleuser/fat-libctos/
In-app path/probe (ADR-058: / → FAT16; existing A7 payload fat-hi)
FAT slot/fatdemo (target/fat-libctos.elf via mkfat16.py --app3)
Kernel probesrc/fatdemo.rs

Rebuild: same one-rebuild commands. Keep /hello + /fsdemo + slot: ok + fsdemo: ok. Not POSIX. Not getdents. Not a reopen of product app hosting (ADR-048/052).

Recipe 3d — Standing EL0 / libctos yield rounds (yield-libctos)

Fourth freestanding EL0 payload that issues several cooperative yield_now() rounds (ADR-062). Same class as libctos: yld-hi / libctos: beat / libctos: yld-ok / yldemo: ok. Deeper than hello’s single yield. Not preemption. Not multi-task EL0. Not a process table.

PiecePath
Sampleuser/yield-libctos/
FAT slot/yldemo (target/yield-libctos.elf via mkfat16.py --app4)
Kernel probesrc/yldemo.rs

Rebuild: same one-rebuild commands. Keep /hello + /fsdemo + /fatdemo + slot: ok + fsdemo: ok + fatdemo: ok. Not POSIX. Not preemption. Not a reopen of product app hosting (ADR-048/052).

Recipe 3e — Standing EL0 / libctos net SVCs (net-libctos)

Fifth freestanding EL0 payload that exercises EL0 net SVCs (ADR-068). Same class as libctos: net-hi / libctos: net-mac / libctos: net-ok / netdemo: ok. Kernel owns virtio-net.

PiecePath
Sampleuser/net-libctos/
FAT slot/netdemo (target/net-libctos.elf via mkfat16.py --app5)
Kernel probesrc/netdemo.rs

Rebuild: same one-rebuild commands. Keep prior samples + N1/N2 markers. Not TCP product. Not sockets. Not “has networking.” Not a reopen of product app hosting (ADR-048/052).

Recipe 3f — Standing EL0 / libctos UDP DNS SVC (udp-libctos)

Sixth freestanding EL0 payload that exercises EL0 net_udp_dns (ADR-071). Same class as libctos: udp-hi / libctos: udp-ok / udpdemo: ok. DNS is probe bait only.

PiecePath
Sampleuser/udp-libctos/
FAT slot/udpdemo (target/udp-libctos.elf via mkfat16.py --app6)
Kernel probesrc/udpdemo.rs

Rebuild: same one-rebuild commands. Keep /hello + /fsdemo + /fatdemo + /yldemo + /netdemo + N1/N2/N3 markers. Not TCP product. Not sockets. Not a DNS product. Not “has networking.” Not a reopen of product app hosting (ADR-048/052).

Standing EL0 / libctos fs_mkdir sample

Seventh freestanding EL0 payload that exercises EL0 fs_mkdir (ADR-075). Same class as libctos: mkdir-hi / libctos: mkdir-ok / mkdemo: ok. Creates FAT root /edir only — not POSIX mkdir. One-level nested mkdir is a kernel/VFS mile (ADR-077), not this sample.

PieceWhere
FAT slot/mkdemo (target/mkdir-libctos.elf via mkfat16.py --app7)
Kernel probesrc/mkdemo.rs

Standing EL0 / libctos TCP echo sample

Eighth freestanding EL0 payload that exercises EL0 net_tcp_echo (ADR-076). Same class as libctos: tcp-hi / libctos: tcp-ok / tcpdemo: ok. Quiet thin TCP guestfwd echo — not BSD sockets, not listen/accept.

PieceDetail
FAT slot/tcpdemo (target/tcp-libctos.elf via mkfat16.py --app8)
SVCnet_tcp_echo (28)
Markerslibctos: tcp-hi / libctos: tcp-ok / tcpdemo: ok (keep net: tcp-ok)

Recipe 4 — memfs named-buffer probe (optional)

In-RAM named buffers behind the thin VFS (ADR-027). Same class as fs: create / fs: write / fs: read / fs: el0 / fs: ok.

PiecePath
VFS + memfssrc/vfs.rs
EL0 trampolinesrc/syscall.rs (/eprobe writes memfs-el0)
Wrapperslibctos fs_create / fs_open / fs_read / fs_write / fs_close

Rebuild: same one-rebuild commands. Not POSIX. Not a volume. Not app hosting.

Recipe 5 — FAT16 /probe on virtio-blk (optional)

FAT16 on QEMU virtio-mmio block (ADR-028, write depth ADR-050, multi-cluster grow ADR-064). Same VFS open / write as memfs. Same class as blk: ok / fat: mount / fat: read / fat: write / fat: ok. Guest /probe must read fat-hi; write probe restores it; /fwr holds fat-nw.

PiecePath
Host imagescripts/mkfat16.py → target/fat16.img
Guest virtiosrc/virtio.rs
FAT + VFSsrc/fat.rs, src/vfs.rs

qemu-smoke builds the image and attaches:

-drive if=none,file=target/fat16.img,format=raw,id=hd0 -device virtio-blk-device,drive=hd0

Host -drive without the guest serial is not the probe. Not POSIX. Not FAT32. Do not say “supports FAT” as a product; say the guest wrote FAT16 bytes when fat: write / fat: create pass, and grew across a cluster boundary when fat: grow / libctos: fat-grow pass. Same-boot CNTPCT pairs vs memfs: write ADR-051 (perf: fat-write / perf: memfs-write / perf: fs-write-delta) and read ADR-065 (perf: fat-read / perf: memfs-read / perf: fs-read-delta) — KPIs; measurement only, not a latency SLA.

Recipe 6 — OS image vs app slot (A9 first cut)

Two host artifacts and a FAT load path (ADR-030). Same class as slot: fat / slot: mapped / slot: ok / perf: app-load.

PiecePath
OS imagetarget/aarch64-ctos/debug/ctos (QEMU -kernel)
App payloadtarget/hello-libctos.elf (published by build.rs)
FAT slotscripts/mkfat16.py --app → /hello
Guest loadsrc/slot.rs + A3 src/loader.rs

A2–A4 load the same FAT file (no embed). Cross-update (same ELF on this OS and ba6541c) is the leftover host smoke, not this recipe. Not OTA. Product freestanding claim is Verified under ADR-048/052 (this recipe page is not that claim by itself).

What cannot run

flowchart TD
  Q{"Want to run it on ctos today?"}
  Q -->|UART worker / echo / stub / memfs / FAT probe| Y["Yes — extend the kernel in-tree"]
  Q -->|Linux binary, shell, Python| N1["No"]
  Q -->|Network server or Linux disk apps| N2["No — ARP/ICMP/UDP probe only, no sockets / no POSIX FS"]
  Q -->|Docker / OCI container| N3["No — not a goal"]

“Yes” means rebuild the kernel. It does not mean drop in an app.

Do not imply these work:

  • Linux binaries (no Linux ABI, no ELF loader for third-party programs)
  • A shell
  • Python (or any hosted language runtime)
  • Network servers (virtio-net ARP + ICMP + minimal UDP + thin TCP probe + EL0 UDP DNS SVC + EL0 TCP echo SVC samples only — ADR-066 / ADR-067 / ADR-070 / ADR-071 / ADR-074 / ADR-076; no BSD sockets / TCP product)
  • POSIX / Linux filesystem apps (memfs + FAT16 read/write miles are not that — Filesystem)
  • Extra-CPU workloads (one CPU, cooperative yield only)
  • Product freestanding app hosting (Verified under ADR-048 / ADR-052; not Linux/POSIX/containers/OTA — issue #48)

Also not claimed: POSIX, GPU, Raspberry Pi, certified security, “production ready,” or containers (non-goal, ADR-029; Hosting apps / containers).

How you would add something in-tree (and why Linux apps do not port): Building or porting.

Prerequisites

What you need to build and run the kernel. Publishing this docs site is optional. GitHub Pages is not required for kernel work.

Kernel (required)

NeedWhy
Nightly Rust + rust-src + llvm-tools-previewFreestanding build-std for the custom aarch64-ctos.json target. Pin is rust-toolchain.toml.
qemu-system-aarch64Primary guest is QEMU -machine virt (ADR-003). Debian/Ubuntu package is often qemu-system-arm.
A host that can run those toolsWindows, macOS, or Linux. The kernel target stays AArch64.
rustup toolchain install nightly
rustup component add rust-src llvm-tools-preview
# Debian/Ubuntu: sudo apt-get install qemu-system-arm

cargo build          # ELF at target/aarch64-ctos/debug/ctos
cargo run            # qemu-system-aarch64 -machine virt
./scripts/qemu-smoke.sh
flowchart LR
  R["rustup nightly + rust-src"] --> B["cargo build"]
  B --> Q["qemu-smoke / cargo run"]

That rebuilds the kernel, including any in-tree no_std code you add. It is not a port of a Linux app. See Building or porting.

Commands and honesty notes: GitHub README. A successful cargo build on your machine is not a copied Verified boot from another host.

Optional

ExtraWhen
Docker linux/arm64the arm64 workstation (Windows ARM64) path: ./scripts/docker-smoke.sh. Do not pass --platform linux/amd64.
Docker on the amd64 workstation (Alpine WSL2)Install bash (apk add bash); use wsl -d Alpine -u <user> -- bash (not WindowsApps bash). Docker Desktop WSL integration not enabled for Alpine — use docker.exe on PATH (…/DockerDesktop/resources/bin) or cmd docker build/run. scripts/docker-smoke.sh is #!/bin/sh.
mdBook 0.5.4 + mdbook-mermaid 0.17.1Local docs site only: ./scripts/docs-build.sh. See Docs website + DNS.

Not required

  • GitHub Pages, a custom domain, or ctos.artof.link (the site is live; you still do not need it to hack the kernel)
  • Raspberry Pi hardware, an x86_64 boot path, or VGA
  • Obsidian (.obsidian/ is gitignored)
  • AWS / Route 53 credentials (DNS is a publish concern, not a kernel build concern)

Building or porting something to ctos

Plain English. There is no easy POSIX port today. ctos is a kernel you extend in-tree, not a host you drop apps onto.

What already runs (rebuild recipes): What can run today. In-tree index: user/README.md. How to build the kernel: prerequisites. Probes: honesty ledger.

Do not claim an “easy port” path that does not exist.

Today (honest)

Nothing POSIX ports easily.

There is no C library (libc), no dynamic linker, and no POSIX filesystem. Thin VFS + memfs + FAT16 are not Linux open. A1 documented a kernel SVC ABI (exit / uart_write / yield — syscall.md, ADR-021). A2 adds a freestanding libctos CRT you can link in-tree (ADR-022). A3 parses that in-tree ELF on the guest and maps PT_LOAD (ADR-023). That is not an application porting target for Linux binaries. There is no PT_INTERP / glibc path that would run a foreign user-mode binary if you built one elsewhere.

A Linux, musl, or glibc program is a different contract. Recompiling it “for AArch64” does not make it a ctos program.

Easiest today

Write in-tree no_std Rust and ship it as part of the kernel image. Language discipline notes: Why Rust.

Typical shape (A8 recipes — what-can-run.md):

  • A cooperative kernel task (same class as sched: task a/b) or a small kernel module
  • Use UART / println! for output; yield to other tasks
  • Optional: serial receive for a byte-in gadget
  • Optional: link libctos like user/hello-libctos (published + FAT /hello; not a second -kernel)

Rebuild the whole guest with the existing target:

cargo build                 # aarch64-ctos.json
./scripts/qemu-smoke.sh     # or ./scripts/docker-smoke.sh on linux/arm64

That is rebuild the kernel, not “port an app.” The new code lives in src/ and links into target/aarch64-ctos/debug/ctos.

flowchart LR
  R["rustup nightly"] --> C["cargo build"] --> S["qemu-smoke"]
  S --> E["OS ELF + published app ELF<br/>A9 first cut"]

Two host artifacts. QEMU still -kernels the OS ELF. FAT /hello is the app slot. A2–A4 load that file. Cross-update is the leftover two-boot smoke.

Not easy

  • Recompile Linux / C / Rust apps against glibc or musl
  • Drop in a userspace ELF (ET_DYN or a Linux ET_EXEC)
  • Expect std, files, sockets, threads, or a process table

Those need a loader and a userspace that ctos does not have. The standing user-mode stub is a test mile plus an A1 ABI trip and an A2 libctos hello, not that runtime (el0.md, syscall.md).

Track A miles that already have recipes

A1–A7 are probed. A8 is docs: the recipes on what-can-run.md. Still not “applications port to ctos.” You extend the kernel or link libctos in-tree.

  1. A stable SVC ABI — A1 (exit / uart_write / yield).
  2. A freestanding CRT / libctos — A2.
  3. Guest PT_LOAD of that in-tree ELF — A3. Image is FAT /hello.
  4. Standing EL0 as normal until exit — A4 / ADR-024.
  5. Isolation cut (identity .rodata + PAN ID-field) — A5. PAN is enabled on the default -cpu cortex-a76 (ADR-080; PAN enable was a non-goal on the historical -cpu cortex-a57, ADR-047). Leftover identity RAM after the heap is torn (ADR-049); _start stays.
  6. Thin VFS + memfs — A6. Recipe 4.
  7. virtio-blk + FAT16 read — A7. Recipe 5. FAT16 write depth — ADR-050.
  8. OS/app slot first cut — A9 / ADR-030. Recipe 6. Leftover cross-update — ADR-032 (Verified host smoke).

A real userspace stays Planned. “EL0 isolated” is Verified only as the accepted ADR-091 sentence (one core, TCG default smoke machine; limits there). Gaps before hosting, and why containers are a non-goal: Hosting apps / containers.

A POSIX filesystem is the same story: not present. Thin VFS + memfs + FAT16 are not Linux open. Direction: Filesystem: new vs extend.

Product hosting still Planned (ADR-048)

An OS image vs app payload first cut and leftover cross-update already exist (A9 #48, ADR-030, ADR-032): same hello-libctos.elf on this OS and documented prior OS ba6541c. Product freestanding “hosting done” is Verified under ADR-048 / ADR-052 (not Linux/POSIX). See hosting-apps.md, overview.md, and immutability.md.

Filesystem: new vs extend

Today: thin VFS with two backends and a prefix mount table. In-RAM memfs (A6 / ADR-027) and FAT16 on virtio-blk (A7 / ADR-028). Same open / read / write / close. Path prefixes route to a backend (ADR-058): /mem (+ A6 probe names) → memfs; / → FAT16 (/probe, plus freestanding samples /hello / /fsdemo / /fatdemo / /yldemo). FAT16 write is a depth mile (ADR-050). Multi-cluster grow is a depth mile (ADR-064). FAT16 root listing is a depth mile (ADR-056). FAT16 root delete is a depth mile (ADR-057). Not POSIX open / mount / getdents / unlink, not Linux VFS, not FAT32, not xv6. Track B B5 compares the concepts: ADR-034. Not claiming a Linux filesystem.

Do not say “supports FAT” as a product. Say the guest read a known FAT16 file when the ledger has fat: ok, wrote FAT16 bytes when it has fat: write / fat: create, grew a file across a cluster boundary when it has fat: grow, listed root entries when it has fat: readdir / fat: entries, deleted a root file when it has fat: delete, created a root directory when it has fat: mkdir, created a one-level nested directory when it has fat: nested, and removed an empty directory when it has fat: rmdir. Hub: honesty ledger, What can run today. Extra stance: filesystem.md.

flowchart TD
  CALL["EL0 / EL1 callers"] --> VFS["Thin VFS<br/>open · read · write · close · readdir · unlink"]
  VFS --> MT["Prefix mount table<br/>ADR-058"]
  MT -->|"/mem · A6 names"| MEM["memfs<br/>in-RAM buffers"]
  MT -->|"/"| FAT["FAT16<br/>read · write · root readdir · unlink"]
  FAT --> BLK["virtio-blk<br/>host fat16.img"]

Same thin API, two backends, prefix mounts. Not POSIX open / mount / getdents / unlink. Do not say “supports FAT” as a product.

New vs extend

Prefer implementing a known small filesystem behind this thin VFS over inventing a novel on-disk format.

FitWhatWhy
LandedRamdisk / memfsNamed heap buffers. Create / lookup / read / write without DMA.
Landedvirtio-blk + FAT16 readHost-visible raw image (scripts/mkfat16.py). xv6-like was rejected so the host can inspect the volume.
LandedFAT16 writeGuest vfs::write on an open FAT handle + backend create of a small file (ADR-050).
LandedFAT16 multi-cluster growGuest write allocates a FAT chain and crosses a cluster boundary (ADR-064).
LandedFAT16 readdirGuest vfs::readdir lists FAT16 root paths (ADR-056). Not POSIX getdents.
LandedFAT16 deleteGuest vfs::unlink deletes a FAT16 root file (ADR-057). Not POSIX unlink.
LandedFAT16 mkdirGuest vfs::mkdir creates a FAT16 root directory with . / .. (ADR-073). Not POSIX mkdir.
LandedEL0 fs_mkdirSVC 27 + freestanding /mkdemo (ADR-075). Exists/BadPath fail-closed. Root create from EL0.
LandedFAT16 nested mkdir + empty rmdirOne-level /parent/child + empty vfs::rmdir (ADR-077). Not POSIX. Not arbitrary depth.
Landedctos-specific virtual mountsPrefix mounts behind the thin VFS (ADR-058). Not a new magic format. Not mount(2). Tree mounts (extra /) stay later.

Avoid early

Do not start with ext4, btrfs, ZFS, or NTFS. Those are large, journaled or feature-heavy, and hide the block mile. They are not a first cut.

Host QEMU -drive without guest virtio + a VFS read is not a filesystem.

Roadmap order

One milestone → one branch → one PR.

flowchart LR
  V["1. VFS ADR<br/>ADR-027"] --> M["2. memfs<br/>A6"]
  M --> B["3. virtio-blk<br/>A7"]
  B --> F["4. FAT16 read<br/>A7"]
  F --> H["5. host image + guest read<br/>A7/A9"]
  H --> W["6. FAT16 write<br/>ADR-050"]
  W --> P["7. FAT vs memfs write CNTPCT<br/>ADR-051"]
  P --> PR["7b. FAT vs memfs read CNTPCT<br/>ADR-065"]
  PR --> R["8. FAT16 readdir<br/>ADR-056"]
  R --> D["9. FAT16 delete<br/>ADR-057"]
  D --> V2["10. Prefix mounts<br/>ADR-058"]

A7 is virtio-blk + FAT16 read. ADR-050 is FAT16 write depth. ADR-051 / ADR-065 are same-boot measurement pairs (not benches). ADR-056 is FAT16 root listing. ADR-057 is FAT16 root delete. ADR-058 is prefix mounts. A8 is documented recipes. A9 / ADR-059 / ADR-061 / ADR-062 use the same volume for /hello / /fsdemo / /fatdemo / /yldemo. Do not say “supports FAT” as a product.

  1. VFS ADR — thin interface (create / open / read / write / close of a path).
  2. memfs — in-RAM named buffers; serial fs: ok.
  3. virtio-blk — virtqueues + sector I/O on QEMU virt. Serial blk: ok.
  4. On-disk FS — FAT16 (ADR-028). Serial fat: ok.
  5. Host-checkable image — scripts/mkfat16.py writes target/fat16.img; smoke attaches -drive if=none,file=…,id=hd0 -device virtio-blk-device,drive=hd0. Guest vfs::open("/probe") must read fat-hi. A9 adds --app so /hello is the published app ELF; ADR-059/061/062 add --app2 / --app3 / --app4 for /fsdemo / /fatdemo / /yldemo.
  6. FAT16 write — guest vfs::write on an open FAT handle + backend create of /fwr (ADR-050). Serial fat: write / fat: create; host --check-write.
  7. FAT vs memfs write CNTPCT — same-boot pair perf: fat-write / perf: memfs-write / perf: fs-write-delta (ADR-051). QEMU TCG lab — not a percent or latency SLA. 7b. FAT vs memfs read CNTPCT — same-boot pair perf: fat-read / perf: memfs-read / perf: fs-read-delta (ADR-065). QEMU TCG lab — not a percent or latency SLA.
  8. FAT16 readdir — guest vfs::readdir lists root thin-VFS paths (ADR-056). Serial fat: readdir / fat: entries. Not POSIX getdents.
  9. FAT16 delete — guest vfs::unlink deletes a root file (ADR-057). Serial fat: delete. Not POSIX unlink.
  10. Prefix mounts — mount table routes path prefixes to memfs or FAT16 (ADR-058). Serial vfs: mount / vfs: mounts. Not POSIX mount(2).

Honesty

ClaimProbeStatus
Thin VFS + memfs create/write/read/closeSerial fs: ok + #[test_case]Verified (A6; honesty ledger)
virtio-blk sector R/WSerial blk: ok + #[test_case]Verified on this tip when the ledger has the probe
FAT16 /probe via the same openSerial fat: ok + #[test_case]Verified on this tip when the ledger has the probe
FAT16 write + small createSerial fat: write / fat: rewrite / fat: create + host --check-writeVerified on this tip when the ledger has the probe (ADR-050)
FAT vs memfs write CNTPCT pairSerial perf: fat-write / perf: memfs-write / perf: fs-write-deltaVerified on this tip when the ledger has the probe (ADR-051). Not a bench.
FAT vs memfs read CNTPCT pairSerial perf: fat-read / perf: memfs-read / perf: fs-read-deltaVerified on this tip when the ledger has the probe (ADR-065). Not a bench.
FAT16 root readdirSerial fat: readdir / fat: entries + #[test_case]Verified on this tip when the ledger has the probe (ADR-056). Not POSIX.
FAT16 root deleteSerial fat: delete + #[test_case]Verified on this tip when the ledger has the probe (ADR-057). Not POSIX.
FAT16 root mkdirSerial fat: mkdir + #[test_case]Verified on this tip when the ledger has the probe (ADR-073). Not POSIX.
EL0 fs_mkdir sampleSerial libctos: mkdir-ok / mkdemo: okVerified on this tip when the ledger has the probe (ADR-075). Not POSIX.
FAT16 nested mkdir + empty rmdirSerial fat: nested / fat: rmdir + #[test_case]Verified on this tip when the ledger has the probe (ADR-077). Not POSIX. One nest level only.
Thin VFS prefix mountsSerial vfs: mount / vfs: mounts + #[test_case]Verified on this tip when the ledger has the probe (ADR-058). Not mount(2).
FAT16 /hello app slot (A9)Serial slot: ok + #[test_case]First cut when the ledger has the probe

Do not claim compatibility with anyone’s existing disk.

Hosting applications — gaps, and containers

What ctos can host today under the scoped Track A product claim (a loaded freestanding user-mode binary with a stable ABI on an OS/app slot), what is still missing for stronger isolation, and whether it can host containers.

Today you also extend the kernel (Building or porting). Samples that exist: What can run today. Isolation notes: el0.md. Kernel SVC numbers: syscall.md.

Do not say ctos hosts Linux programs, POSIX apps, or containers. The scoped freestanding slot-hosting claim is Verified under ADR-048 / ADR-052.

Today vs the slot split (A9 + product claim)

Today (Verified miles + product claim): QEMU -kernel still loads one OS ELF. Published freestanding samples live on FAT /hello (A9), /fsdemo (ADR-059), /fatdemo (ADR-061), and /yldemo (ADR-062). A2–A4 load /hello (no production embed). The same hello ELF boots on this OS and on ba6541c (ADR-032). Under ADR-048 criteria Met×5 (ADR-052), the product sentence “apps run independently of the OS” / “app hosting is done” is Verified for that freestanding slot path. It is not containers, not OTA, and not Linux/POSIX.

Product claim (Verified): “apps run independently / hosting done” is Verified under ADR-048 (checklist below). Scope stays freestanding slots — not Linux userspace.

flowchart LR
  OS["OS ELF<br/>QEMU -kernel"] --> FAT["FAT16 volume"]
  H["/hello"] --> FAT
  F["/fsdemo"] --> FAT
  D["/fatdemo"] --> FAT
  Y["/yldemo"] --> FAT
  FAT --> LOAD["Guest PT_LOAD<br/>standing EL0"]
  LOAD --> CLAIM["Product hosting<br/>Verified — ADR-048/052"]

Freestanding slot-hosting is Verified (Met×5). Catalog samples deepen recipes; they do not reopen the claim. Do not say Linux hosting, containers or OTA. Say “EL0 isolated” only as the accepted ADR-091 sentence. Runtime cost vs embed is a lab measurement pair (ADR-046), not a marketing delta.

Gaps before hosting applications

These are missing pieces for richer hosting / isolation, not a schedule. Rows without a probe stay Planned or unbuilt. “Later” is not a promise.

A supervisor call (SVC) is how user-mode code asks the kernel for help. Reserved SVC #0–#2 stay test miles. A1 documented exit / uart_write / yield (syscall.md). That is not a CRT or loader.

GapWhy it mattersStatus
Stable SVC ABIA1 documented exit / uart_write / yield (ADR-021). That is a kernel contract, not a loader.ABI mile — Verified only when the ledger has svc: ok on this tip. Mile ≠ whole product claim.
libctos / CRTA2: libctos wrappers + _start CRT. Hello is host-built and published for the slot path (ADR-022).CRT mile — Verified only when the ledger has libctos: ok on this tip. Not a guest loader.
ELF / user loaderA3: guest ELF64 PT_LOAD into user TTBR0, then ERET to e_entry (ADR-023). A2–A4 load FAT /hello (no production embed, ADR-032). Not a Linux ELF ABI.Loader mile — Verified only when the ledger has loader: ok on this tip. Not a Linux ABI.
OS vs app slotsA9: host kernel ELF + hello-libctos.elf; guest FAT /hello (ADR-030). Same ELF on this OS and ba6541c (ADR-032).Slot first cut + leftover — Verified only when the ledger has slot: ok and the cross-update host lines on this tip. Product claim uses this mile under ADR-048.
Standing user mode as normalA4: loaded image stands until exit; fail-closed restore (ADR-024). Not a process table.Standing-task mile — Verified only when the ledger has el0: task-ok on this tip. Not isolation.
Stronger isolationUmbrella “EL0 isolated” accepted by the sponsor 2026-09-28; revision 4 accepted 2026-10-02 (ADR-091; checklist ADR-055 all Met; sponsor closure ADR-060); PAN enabled (ADR-080); _start stub page stays the one identity exception (ADR-089); stock-QEMU taken SError still parks (ADR-053)Verified only as the ADR-091 sentence: one core, the TCG default smoke machine; not a speculative-execution or side-channel claim, not a real-hardware claim, not a certification; app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN); no multi-core race story
VFS + memfsThin VFS + in-RAM named buffers (ADR-027); see Filesystemmemfs mile — Verified only when the ledger has fs: ok on this tip. Not POSIX. Not FAT.
On-disk FSvirtio-blk + FAT16 behind the same VFS (ADR-028); write depth (ADR-050)block + FAT mile — Verified only when the ledger has blk: ok / fat: ok (and fat: write / fat: create for write). Not POSIX. Not “supports FAT” as a product.
Richer I/OUART byte in/out only; no TTY, disk beyond FAT, or socketsUART probed; the rest unbuilt
Preemption / extra CPUs / netCooperative one-CPU yield; no NICLater — not required for the ADR-048 claim

“Host an application independently of the OS” / “app hosting is done” is Verified under ADR-048 (documented recipes + slot cross-update + no production embed + honesty about missing Linux/POSIX/containers + sponsor accept / ADR-052). That block is Track A. A1–A9 first cuts alone were never enough; the checklist Met×5 is. A5: PAN ID-field Verified; enable is non-goal on default cortex-a57 (ADR-047). A8 documented rebuild recipes (what-can-run.md). A9 + leftover: FAT /hello, no production embed, cross-update on ba6541c. See Immutability.

Containers

Non-goal. ctos cannot host OCI / Docker / Kubernetes workloads, and it is not aiming to.

Those need Linux kernel features (namespaces, cgroups, a Linux ABI, usually overlay or equivalent, a rich syscall surface) that this learning kernel does not have. Decision: ADR-029. Track B B7 (#47).

Today the arrow is the other way: Docker on a host (arm64 workstation linux/arm64) runs the ctos smoke image. That is “Docker hosts ctos,” not “ctos hosts containers.” See the README Docker notes and the Docker rows in the ledger.

flowchart LR
  HOST["Linux host + Docker"] -->|"builds and QEMU-smokes"| CTOS["ctos guest"]
  CTOS -.->|"does not host"| OCI["OCI / Docker / k8s"]

Host harness ≠ guest runtime. File presence of this page is not a container probe.

Container support is not Planned on this page. Do not add a Planned row. Reopen only with a new GitHub epic plus a new ADR — not a Track B win, and not a new FR/NFR ID.

Claim criteria (ADR-048)

Product “apps run independently / hosting done” is Verified when all of the following are ledger-Verified and sponsor-accepted (ADR-048 / ADR-052):

  1. Documented freestanding-app recipes (existing markers only).
  2. Slot cross-update (same published ELF on this OS and a documented prior OS).
  3. No production hello embed (slot: embed never appears on the production path).
  4. Docs still refuse Linux ABI / POSIX / guest containers (ADR-029).
  5. Explicit sponsor accept on the flip PR (no self-merge).

Progress vs tip main (honesty checklist)

Tip audited: 190ef5a (Merge #106 / docs polish; or newer main with ADR-062). Product row Verified — criteria 1–5 Met. Catalog samples /fsdemo / /fatdemo / /yldemo deepen recipes; they do not reopen ADR-048/052.

#CriterionStatusTip evidence (cite ledger / tip)
1Documented recipes (A8)MetLedger “Documented sample apps (Track A A8)” Verified; hub what-can-run.md / apps-today.md / user/README.md.
2Slot cross-updateMetLedger “A9 cross-update …” Verified: cross-update prior-os=ba6541c… slot:ok + cross-update this-os=… slot:ok (ADR-032). Re-probed on later tips (incl. ADR-046 / amd64 workstation Docker rows).
3No production embedMetLedger “A2–A4 hello without include_bytes! (ADR-032)” Verified; smoke rejects slot: embed. Probe-only include_bytes! in src/slot.rs only (ADR-046).
4Honesty: no Linux/POSIX/containersMetContainers non-goal (ADR-029); this page + Track B frame refuse Linux userspace / POSIX / guest OCI.
5Explicit sponsor acceptMetSponsor: “address 1. app hosting” after still-Planned #5-only review. ADR-052. Flip PR; no self-merge.

A1–A9 first cuts alone remain insufficient without the checklist. Matching ledger section: honesty-ledger.md — ADR-048 progress.

Honesty

ClaimStatus
ctos hosts freestanding slot apps / “hosting done” (ADR-048 scope)Yes — Verified (ADR-048 / ADR-052)
ctos hosts Linux programs / POSIX / glibc/muslNo
ctos hosts OCI/Docker containersNo — non-goal (ADR-029)
Linux-compat research (Track B)Frame (ADR-031) + B2 gap map (linux-aarch64-syscall-gap.md) + B3 process-model stance (ADR-035) + B4 ELF/auxv gap (ADR-033) + B5 VFS compare (ADR-034). Not claiming Linux userspace yet. Not claiming dynamic Linux ELF. Not claiming a Linux filesystem.
Host Docker runs ctos-smokeSeparate ledger row (host tool, not a guest runtime)

KPIs / how we measure

Plain English. These are the probes this repo actually runs. They are not a product dashboard, a latency promise, or invented KPIs. Numbers live in the honesty ledger as one environment and one revision.

The three first-class pillars are antifragility (NFR-05), security (NFR-10), and performance (NFR-07). “Application support” below is an honest scope statement, not a new requirement ID.

Performance

What we measure today on QEMU virt (serial markers + tests, fail-closed in scripts/qemu-smoke.sh).

CNTPCT is the CPU’s physical cycle counter — a hardware clock we read to see that time moved, not a published bench.

ProbeWhat it isWhat it is not
Cycle-counter loop (perf: cntpct)The counter is readable and moves over a fixed trivial loopA microsecond budget, interrupt latency, or “faster than X”
IRQ-to-handler delta (perf: irq-delta)Counter minus the timer’s compare value; min / max / spread on several ticksA latency SLA, SPEC, or a comparison to other kernels
Boot-delta (perf: boot-delta)Cycle count from after paging init to after Hello World!QEMU process start time, a boot budget, or a published bench
Host ELF size (perf: elf-size)Byte size of the debug ctos ELF after cargo buildA size budget or “smaller is better”
App-load (perf: app-load)Cycle count around FAT /hello read + A3 map + ERET (A9)A budget, or “slots are free”
Embed-load (perf: embed-load)Probe-only CNTPCT around map + ERET of the same linked-in hello ELF (ADR-046)Production slot path, slot: embed, or a bench
Slot delta (perf: slot-delta)Raw app=<a> embed=<b> tick pair on one bootA percent, “faster/slower,” or SPEC
FAT write (perf: fat-write)CNTPCT around one FAT VFS write of the small /probe rewrite payload (ADR-051)A write latency SLA or “disk cost”
Memfs write (perf: memfs-write)CNTPCT around one memfs VFS write of the same bytes on the same bootA memfs SLA or published bench
FS write delta (perf: fs-write-delta)Raw fat=<a> memfs=<b> tick pair on one bootA percent, “faster/slower,” or SPEC
FAT read (perf: fat-read)CNTPCT around one FAT VFS read of /probe (fat-hi) (ADR-065)A read latency SLA or “disk cost”
Memfs read (perf: memfs-read)CNTPCT around one memfs VFS read of the same bytes on the same bootA memfs SLA or published bench
FS read delta (perf: fs-read-delta)Raw fat=<a> memfs=<b> tick pair on one bootA percent, “faster/slower,” or SPEC
Net-ping (perf: net-ping)CNTPCT around EL0 net_ping quiet ARP+ICMP vs SLIRP (el0_icmp_ping) (ADR-069)A ping latency SLA, “has networking,” or SPEC
UDP DNS (perf: udp-dns)CNTPCT around EL0 net_udp_dns quiet ARP+UDP DNS vs SLIRP (el0_udp_dns) (ADR-072)A DNS latency SLA, “has networking,” or SPEC
TCP echo (perf: tcp-echo)CNTPCT around EL0 net_tcp_echo quiet ARP+thin TCP vs guestfwd (el0_tcp_echo) (ADR-078)A TCP latency SLA, sockets, “has networking,” or SPEC

QEMU virt is one guest. It is not Raspberry Pi, not real silicon, and not SPEC. Optimize only after a probe shows a cost. Details: performance.md.

OS slot vs app slot (performance)

ADR-030 first cut exists: FAT /hello + perf: app-load. ADR-032 removes the A2–A4 production embed. ADR-046 adds a probe-only linked-in trip and perf: slot-delta app=<a> embed=<b> on the same boot. Immutability means disconnect OS update from apps. That can cost at runtime or be neutral. We do not invent a percentage, a budget, or “faster than linking the app into the kernel.” The pair is a QEMU TCG lab measurement.

KindHonest claim shapeWhat it is not
Compared pairperf: app-load and perf: embed-load ticks plus perf: slot-delta app=<a> embed=<b> on one guestA percent, SLA, or “slots are free”
Possible costExtra VFS/FAT read before the shared parse/map/ERET workMarketing “slower than embed” without naming the tip + ticks
Possible neutralSteady-state UART print / yield after the app is mappedA win claimed without a probe
Build-time, not a benchTwo host artifacts; FAT slot can change without a kernel rebuildperf: elf-size is still one image’s byte count. Kernel may include_bytes! only in src/slot.rs for the probe.

FAT write vs memfs write (performance)

ADR-050 is the write mile. ADR-051 adds a same-boot CNTPCT pair: perf: fat-write vs perf: memfs-write plus perf: fs-write-delta fat=<a> memfs=<b>. That can show a cost shape for going through virtio-blk/FAT versus in-RAM memfs — or be noisy under TCG. We do not invent a percentage, a budget, or “FAT write is X× slower.”

KindHonest claim shapeWhat it is not
Compared pairRaw tick counts for the same small payload on one guestA percent, SLA, or product KPI
Possible costExtra block/FAT work vs memfs memcpyMarketing “slower” without naming the tip + ticks
GateKeep fat: write / fat: ok; fail closed on missing pair markersCriterion / invented benches

Measure first (NFR-07). Do not tune a loader “for speed” on a hunch. Do not copy QEMU ticks into a product slide.

FAT read vs memfs read (performance)

ADR-065 adds the symmetric same-boot CNTPCT pair for read: perf: fat-read vs perf: memfs-read plus perf: fs-read-delta fat=<a> memfs=<b>. Same honesty rules as the write pair — measurement only, not a latency SLA.

KindHonest claim shapeWhat it is not
Compared pairRaw tick counts for the same small /probe payload on one guestA percent, SLA, or product KPI
Possible costExtra block/FAT work vs memfs memcpyMarketing “slower” without naming the tip + ticks
GateKeep fat: read / fat: ok / ADR-051 write markers; fail closed on missing pair markersCriterion / invented benches

Net-ping path (performance)

ADR-069 times the EL0 net_ping quiet path: perf: net-ping ticks=<n>. Smoke checks the prefix only — tick values vary under QEMU TCG. Optional N2 crumbs perf: net-icmp-tx / perf: net-icmp-rx remain lab-only (ADR-067).

KindHonest claim shapeWhat it is not
Single-path sampleRaw tick count for quiet ARP+ICMP as SYS_NET_PING does itA percent, SLA, or product KPI
GatePrefix perf: net-ping present; reject faster/slower/percentExact tick count; criterion / invented benches
Non-claim—TCP/UDP, sockets, “has networking,” Wi‑Fi

UDP DNS path (performance)

ADR-072 times the EL0 net_udp_dns quiet path: perf: udp-dns ticks=<n>. Smoke checks the prefix only — tick values vary under QEMU TCG. Window = quiet ARP + UDP DNS as SYS_NET_UDP_DNS / el0_udp_dns (same pattern as ADR-069).

KindHonest claim shapeWhat it is not
Single-path sampleRaw tick count for quiet ARP+UDP DNS as SYS_NET_UDP_DNS does itA percent, SLA, or product KPI
GatePrefix perf: udp-dns present; reject faster/slower/percentExact tick count; criterion / invented benches
Non-claim—TCP, sockets, DNS product, “has networking,” Wi‑Fi

Thin TCP echo path (performance)

ADR-078 times the EL0 net_tcp_echo quiet path: perf: tcp-echo ticks=<n>. Smoke checks the prefix only — tick values vary under QEMU TCG. Window = quiet ARP + thin TCP guestfwd echo as SYS_NET_TCP_ECHO / el0_tcp_echo (same pattern as ADR-069/072). Kernel N5 net: tcp-ok stays untimed this mile.

KindHonest claim shapeWhat it is not
Single-path sampleRaw tick count for quiet ARP+TCP as SYS_NET_TCP_ECHO does itA percent, SLA, or product KPI
GatePrefix perf: tcp-echo present; reject faster/slower/percentExact tick count; criterion / invented benches
Non-claim—BSD sockets, listen/accept, TLS/HTTP, “has networking,” Wi‑Fi

Stability / antifragility

We do not publish an uptime KPI. We measure whether sensors stay fail-closed and whether a repeated miss becomes a ratchet (NFR-05, antifragility.md).

MechanismWhat it proves
scripts/qemu-smoke.shBuild + required serial strings + tests exit 0. Missing a marker is a fail, not a skip.
cargo test --features force-failThe panic path is fail-closed (host/QEMU exit 1). A green suite that cannot fail is not a sensor.
Honesty ledgerEvery status word is a claim. Verified needs a probe. Unknown = unprobed. Planned = not built yet. Failed = the probe ran and lost. File presence is not QEMU boot.
Host ratchetsWhen a machine finds what CI missed, we keep the Failed row and add a sensor. Example: arm64 workstation Docker on main b2bbb99 missed paging: ok after a larger nightly layout; GHA had been green.

A green GitHub Actions run is one pair of runners. It is not “every host.” Docker on the arm64 workstation is a separate row.

Application support

Scope only — not a new requirement ID. Concrete examples and the cannot-run list: What can run today.

Security work is probed (heap not-executable, stack guards, read-only code / not-executable data, user-mode miles) — that is not a product security KPI. See security.md. Do not say “production ready.”

Advantages

These are properties of how the repo is run, not a product pitch. They are useful if you want a small AArch64 kernel you can study without inflated status.

The landing Driving principles block (honesty, antifragility, security, performance, document-first) is the force; Track A/B stay subordinate. Visitor-facing source: landing. Deep pillar notes: pillars.md.

Document-first

Vision → architecture → ADR → roadmap stay ahead of code for each loop (document-first — FR-14, NFR-13). Frozen IDs are FR-01–FR-15 and NFR-01–NFR-14. New IDs are not invented in chat.

Probed claims

Status words need a command, a serial capture, a gh run URL, or a file read that matches the claim (honesty — NFR-06, honesty ledger). Unprobed stays Unknown. That is slower to write and harder to game than a green README badge.

QEMU virt learning scope

Primary ISA is AArch64 on QEMU virt + serial UART (ADR-003). The scope is small on purpose: UART, exceptions, paging, heap, a cooperative scheduler, then pillar cuts. It is not a distro and not a board bring-up until a board probe exists.

Why Rust (for this kernel)

Rust is the language of this kernel on purpose. It is not a slogan. It is a tools-and-rules choice for a small AArch64 guest you can study.

  1. Memory safety by default. Ownership and borrowing catch many C-class bugs (use-after-free, double-free, many buffer overruns) at compile time, before the guest boots.
  2. No garbage collector, still systems-friendly. You keep control of layout and when things free. There is no stop-the-world collector in the way of a freestanding kernel.
  3. Safer concurrency relative to unchecked shared mutable state. In safe Rust, Send / Sync make “who can share this across threads” a type-system question. That does not invent a scheduler; it reduces a class of data races in safe code.
  4. Zero-cost abstractions and fail-closed paths. You can keep thin wrappers without paying for them at runtime. Result / Option push “did this fail?” into the type, which fits probed, fail-closed sensors better than silent null.
  5. Modern freestanding tooling. cargo, no_std, and crates like in-tree libctos are the everyday build path (prerequisites, porting).
flowchart LR
  S["Safe Rust"] --> M["Ownership / borrowing"]
  S --> C["Send / Sync"]
  S --> F["Result / Option"]
  U["unsafe still required"] --> H["page tables · MMIO · asm"]

Safe Rust helps discipline. unsafe is still how you talk to the machine.

Honesty caveats

  • unsafe still exists for page tables, MMIO, and assembly. The type system does not cover those edges.
  • Rust does not invent a correct architecture by itself. Wrong maps, wrong EL, and wrong ABI stay wrong.
  • This does not mean “secure OS.” “EL0 isolated” is Verified only as the accepted ADR-091 sentence: one core, TCG default smoke machine; not speculative-execution / side-channel, not real hardware, not a certification. Its clauses (PAN, taken SError, …) each have their own probe rows (limits, honesty ledger).
  • Status words still need probes / the honesty ledger. Language choice is not a substitute for a serial capture or a gh run URL.

Pair with Drawbacks / limits: Rust helps discipline; it does not remove those limits.

Three pillars as requirements

After the cooperative scheduler (M9), antifragility, security, and performance are first-class (ADR-011, pillars.md). They still need probes. “Secure” and “fast” are not default adjectives.

Author ≠ merger

ADR-002: the author does not merge their own PR. artofdream vs cursor[bot]. Same-login Approve is still self-review.

Fail-closed sensors

scripts/qemu-smoke.sh and force-fail are meant to break when a marker disappears. Repeated host misses become ratchets, not extra paragraphs of advice (antifragility.md).

Immutability

Scoped yes. Absolute no. Two different sentences:

Page-table scope (probed today). Some kernel memory is read-only and executable, other memory is writable and not executable (W^X on that image slice — write or execute, not both). Smoke markers: ro: ok (ADR-015), identity .text / .rodata / .data / heap tears, and leftover identity frame RAM after the heap (ident: ram*, ADR-049) on this QEMU virt guest. _start stays (ident: start-stay). Not “the kernel is immutable,” “W^X everywhere,” or full identity teardown. Claim only with a ledger probe.

OS slot vs app slot (first cut + leftover cross-update). The useful product meaning is: apps run independently of the OS on a separate OS slot (the kernel image you -kernel) and app slot (a loaded freestanding user-mode binary). ADR-030 is the first cut: host ctos ELF + hello-libctos.elf, guest FAT /hello, A3 PT_LOAD map. ADR-032 loads A2–A4 from that FAT file (no embed) and boots the same published ELF on this OS and on ba6541c. Product freestanding hosting is Verified under ADR-048 / ADR-052. Do not say OTA / “immutable OS” marketing / Linux hosting.

This is not containers (ADR-029 — non-goal, not later), and not OTA / A-B firmware updates (later and unclaimed). Do not say “immutable OS updates.”

Runtime cost vs neutral is unmeasured. Measure first; no invented numbers. KPIs — OS slot vs app slot.

Limits of this shape: Drawbacks / limits.

Drawbacks / limits

What this project is not, and what is still unfinished. Pair with KPIs / how we measure and product vision.

Learning kernel, not a product

ctos is a research / teaching AArch64 kernel. It is not production-ready, not a desktop, not POSIX, not a container host (non-goal, ADR-029), and not a “secure OS.” Do not treat a green QEMU smoke as certification. Rust helps discipline; it does not remove these limits (Why Rust).

Boot contract still starts at 0x4008_0000

QEMU -kernel and _start stay at 0x4008_0000. High-address work (kernel page-table alias, live code / .rodata / .data / heap / leftover frame-RAM tears) does not mean the kernel moved. Leftover identity RAM after the heap is unmapped (ident: ram*, ADR-049); smoke rejects ident: ram-stay. _start stays mapped by decision (ident: start-stay, ADR-047). Do not claim full identity teardown. See el0.md.

Privileged Access Never — enable Verified on cortex-a76 (ADR-080)

PAN stops the kernel from casually reading user-accessible pages when PSTATE.PAN is set. Default smoke CPU is -cpu cortex-a76 (ADR-079 / #125): ID-field Verified (pan: present) and enable + EL1-vs-EL0 fault Verified (pan: enabled / pan: el1-fault, ADR-080). Historical -cpu cortex-a57 remains pan: absent (ADR-026). Say “EL0 isolated” only as the accepted ADR-091 sentence (below). Do not silent -cpu.

Network is virtio-net ARP + ICMP + minimal UDP + thin TCP (+ tiny EL0 net/UDP SVCs); disk is virtio-blk + FAT16

N1 programs virtio-mmio virtio-net enough to TX/RX one raw Ethernet frame (ARP vs QEMU SLIRP gateway) — ADR-066. N2 adds a kernel-path ICMP echo to 10.0.2.2 (net: ping-ok) — ADR-067. N3 adds a kernel-path UDP datagram to SLIRP DNS 10.0.2.3:53 (net: udp-ok) — ADR-070 (DNS is probe bait, not a DNS product). N5 adds a kernel-path thin TCP active-open + one payload echo via QEMU guestfwd 10.0.2.4:7 (net: tcp-ok) — ADR-074 (not a sockets product). N4 adds a tiny EL0 net SVC (net_mac / net_ping) and freestanding /netdemo — ADR-068. N3.x adds EL0 net_udp_dns and /udpdemo — ADR-071 / track-n.md. That is not BSD sockets, TCP product, DHCP/DNS product, Wi‑Fi, or “has networking / sockets OS.” A7 programs virtio-mmio block and reads a host-built FAT16 image through the thin VFS; ADR-050 adds a guest FAT16 write / small create mile; ADR-051 / ADR-065 add same-boot FAT vs memfs write and read CNTPCT pairs (lab measurements, not benches) (Filesystem, KPIs). That is not a general DMA API, not virtio-pci, not POSIX, and not a Linux rootfs. Input on virt is serial receive. Timer is the virt interrupt controller plus the generic timer.

No real userspace apps

The standing user-mode stub is a mile, not an application runtime. No ELF loader for third-party programs, no libc, no extra CPUs, no GPU. Rebuild recipes: What can run today — four freestanding samples on FAT (/hello, /fsdemo, /fatdemo, /yldemo). How to extend the kernel (not port POSIX): Building or porting. OS vs app slot has an A9 first cut (FAT /hello) plus leftover cross-update on ba6541c (Verified miles). Product freestanding app hosting is Verified under ADR-048 / ADR-052 (not Linux/POSIX). Umbrella “EL0 isolated” is Verified only as the accepted ADR-091 sentence (M6); the other isolation locks stay (ADR-060).

Docs URL

https://ctos.artof.link serves this book (HTTPS Verified after #30 — website.md). A trailing-slash github.io/ctos/ path 404’d on the 2026-09-11 probe; use the custom domain. Publishing mechanics can still lag a new commit until the next main Pages deploy.

Immutability is not absolute

Read-only code / not-executable data, and torn identity ranges (including leftover frame RAM), are scoped probes. Do not upgrade them to “immutable kernel,” “W^X everywhere,” or “OS updates without touching apps.” The A9 OS vs app slot first cut exists; the freestanding product hosting claim is Verified (ADR-048 / ADR-052). Not OTA or containers. Details: Advantages — Immutability.

Other honest gaps

  • Umbrella user-mode isolation: Verified only as the accepted ADR-091 sentence (M6, 2026-09-28; checklist ADR-055 all Met; sponsor closure ADR-060). The sentence, verbatim (revision 4, accepted 2026-10-02):

    “On the ctos default smoke machine (QEMU virt, -cpu cortex-a76, TCG, one core), EL0 is isolated from the kernel at the architectural page-table / exception-level boundary: each EL0 task runs on its own TTBR0 with its own ASID; EL0 faults when it reads, writes or executes kernel memory; in every translation table (kernel, user and ASID-B TTBR0, and TTBR1) the only EL0-reachable pages are five allowlisted user slots backed by non-kernel frames, checked fail-closed while a user app runs and at rest; PAN is enabled, including while the kernel handles exceptions from EL0, and EL1 faults on EL0-readable memory (the execute-only app text page is not covered: this CPU has no FEAT_EPAN); lower-EL IRQ and FIQ are taken while EL0 stands; taken lower-EL SError is evidenced by the ADR-090 evidence class; the only identity (VA = PA) mapping left in any TTBR0 is the documented EL1-only _start stub page (ADR-089); and a system call copies from or to an EL0 pointer only if EL0 itself may read (for a copy from EL0) or write (for a copy to EL0) every page in the range. It is not a speculative-execution or side-channel claim, not a real-hardware claim, and not a certification.” Limits that stay with it: one core; the TCG default smoke machine; not speculative-execution / side-channel; not real hardware; not a certification; app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN) (ADR-097, accepted as ADR-091 revision 4 on 2026-10-02; before that EL0 could read its own code bytes, ADR-094); no multi-core race story. Also outside it: task-vs-task isolation beyond ASIDs, DMA / IOMMU / EL2, and taken SError on stock QEMU (still parks, ADR-053). _start stays (ADR-089).

  • x86_64 is not a primary path.
  • Raspberry Pi / other boards: unprobed.
  • CI on GitHub is a separate ledger row from a cloud-VM qemu-smoke.

Archify diagrams

Same-origin views on this site (Documented aid — not Live product status; snapshot from before M6; for “EL0 isolated” see the accepted ADR-091 sentence).

DiagramOpen
QEMU NMI pin vs stock park (architecture)HTML · JSON
Taken SError inject sequenceHTML · JSON

Honesty

  • Taken el0: serror is Verified opt-in (CTOS_QEMU + CTOS_REQUIRE_TAKEN_SERROR=1); see ADR-083.
  • Stock distro QEMU still parks (el0: serror-park).
  • Not FEAT_NMI / FIQ / BRK as SError. The pin alone is not the umbrella; since M6, “EL0 isolated” is only the accepted ADR-091 sentence.

Source folder: docs/archify/ (copied into the published book/archify/ on Pages). Generated with Archify v3.2.0; the former guided chapters are now cards under each drawing, and the viewer’s Route and Lens controls replace guided views. Regenerate steps (pinned to v3.2.0, ARCHIFY_UPDATE_CHECK_DISABLED=1) are in the folder README.

Chronify — evolution rail

Dated milestones for ctos (ctsOS). Sister to Archify diagrams.

ArtifactOpen
Pages rail (SVG)
Interactive HTMLctos-evolution.timeline.html
IR (source of truth)ctos-evolution.timeline.json

Honesty

Documented aid until ctos.artof.link Pages probe this session. Not Live product status. Snapshot from before M6; for “EL0 isolated” see the accepted ADR-091 sentence.

Regenerate with chronify v0.1.0:

node bin/chronify.mjs deliver timeline docs/chronify/ctos-evolution.timeline.json \
  docs/chronify/ctos-evolution-rail.svg --surface pages-rail

Product vision

Core principles are the driving force for ctos. Tracks and nicknames are subordinate. Non-negotiable: honesty ledger, antifragility (fail-closed + ratchets), security (threat model, probed mitigations only), performance (measure first), document-first / one-PR loops. principles.md.

What ctos is

ctos is a learning and research bare-metal OS kernel written in Rust for AArch64 (arm64 primary; ADR-003). The near-term product is a QEMU virt guest that owns the machine after -kernel load: UART text, then exceptions, paging, a heap, and a tiny scheduler — in that order.

It exists so we can study kernel mechanics with a document-first harness: claims stay honest, failures become sensors, and agents do not merge their own work. After M9 that harness sits on three first-class pillars — antifragility, security, and performance (ADR-011). Security and speed are not afterthoughts; they still need probes before anyone writes “secure” or “fast.”

What ctos is not

  • Not a production operating system, desktop, or app runtime. Probed guest samples: apps-today.md.
  • Not a Linux distro, not POSIX, not a container host (non-goal, ADR-029). Linux-compat is a research frame (ADR-031); not claiming Linux userspace yet. Porting stance: building-or-porting.md. No filesystem today: filesystem.md. Gaps to host apps: host-apps.md.
  • Not a florist / commerce platform and not a clone of any shop case study.
  • Not a Raspberry Pi (or other board) port until a board probe exists.
  • Not a claim that QEMU boot, CI, hardware bring-up, a “secure OS,” an “immutable OS,” or a published bench is finished until a probe says so. Scoped immutability: immutability.md.
  • Not an x86_64-primary kernel. x86_64 may become a secondary target later; it is not implemented now.

Success (current horizon)

A new session can read the vision, principles, overview, what can run today, building or porting, host-app gaps, architecture, honesty ledger, and latest daily brief, then take one roadmap milestone to a GitHub PR without inventing status.

ctos — Functional & Non-Functional Requirements

Learning/research AArch64 (arm64) Rust bare-metal kernel (not a general-purpose desktop OS).

ISA revision: ADR-003 (2026-09-08) replaced x86_64 / VGA / bootimage wording with AArch64 / UART / QEMU virt. IDs are unchanged. Sponsor authorized the text revision; do not mint new FR/NFR IDs here. The sponsor brief called this “ADR-002”; that number was already the identity-split ADR on the harness tip.

FR-06 stage note (2026-09-09): Stage moved Next → Now when M3 landed VBAR_EL1 + resumable BRK (ADR-004). ID unchanged.

FR-07 stage note (2026-09-09): Stage moved Next → Now when M4 landed the dedicated exception / fatal stacks (ADR-005). ID unchanged.

FR-08 stage note (2026-09-09): Stage moved Next → Now when M5 landed GICv2 + the EL1 physical timer (ADR-006). ID unchanged. FR-08 “later input” is M6 / ADR-007 (PL011 RX on virt). No new FR ID.

FR-09 stage note (2026-09-09): Stage moved Next → Now when M7 landed the EL1 identity map + bump frame allocator (ADR-008). ID unchanged. DTB parse remains staged (Unknown). Heap is M8 / FR-10.

FR-10 stage note (2026-09-09): Stage moved Next → Now when M8 landed GlobalAlloc on a first-fit heap backed by identity-mapped frames (ADR-009). ID unchanged. Growing / slab heaps remain later.

FR-11 stage note (2026-09-09): Stage moved Later → Now when M9 landed cooperative round-robin yield on EL1 (ADR-010). ID unchanged. Preemption / SMP remain later. A documented EL0 SVC ABI is ADR-021 / Track A A1 — not a new FR and not app hosting.

NFR-10 note (2026-09-17, ADR-078): EL0 net_tcp_echo quiet-path CNTPCT (perf: tcp-echo ticks=<n>, ADR-078). Threat-model v1.53. Measure-first only — not a latency SLA. Not BSD sockets. Not listen/accept. Keep net: tcp-ok / libctos: tcp-ok. IDs unchanged.

NFR-10 note (2026-09-17, ADR-077): FAT16 one-level nested mkdir + empty rmdir behind the thin VFS (fat: nested / fat: rmdir, ADR-077). Threat-model v1.52. Not POSIX mkdir/rmdir. Not arbitrary-depth trees. No new SVC. Not TCP CNTPCT. Keep fat: mkdir. IDs unchanged.

NFR-10 note (2026-09-17, ADR-075): EL0 fs_mkdir (SVC 27) + freestanding /mkdemo (libctos: mkdir-ok / mkdemo: ok, ADR-075). Threat-model v1.50. Not POSIX mkdir. Not a directory tree. Not EL0 TCP. Keep fat: mkdir. IDs unchanged.

NFR-10 note (2026-09-16, ADR-073): FAT16 root mkdir behind the thin VFS (fat: mkdir, ADR-073). Threat-model v1.48. Not POSIX mkdir. Not a directory tree. No new SVC this mile. Track N / TCP untouched. IDs unchanged.

NFR-10 note (2026-09-16, ADR-064): FAT16 multi-cluster grow behind the thin VFS (fat: grow / libctos: fat-grow, ADR-064). Threat-model v1.39. Not POSIX write API. Not exFAT. Track N untouched. IDs unchanged.

NFR pillars note (2026-09-09): ADR-011 revises NFR-05, NFR-07, and NFR-10 text in place (antifragility, performance, security as first-class pillars). IDs unchanged. Do not mint NFR-15+.

NFR-10 W^X note (2026-09-10): ADR-012 revises NFR-10 text in place for the heap / coop-stack NX cut. ADR-013 records EL0 direction; the first mile is enter/return + UXN fetch, not isolation. ADR-014 adds unmapped linker-stack guards. IDs unchanged.

NFR-10 / NFR-08 note (2026-09-11): ADR-015 revises NFR-10 text in place for the RO+NX text/data cut (SCTLR.WXN on). ADR-013 adds a user-TTBR0 read mile, an ASID-tagged TLB mile, and a standing EL0 context; ADR-016 adds a TTBR1 private-page first cut; ADR-017 adds EL1 fetch from the TTBR1 RAM alias; ADR-018 adds the first identity-tear cut (split tables + one torn text page); ADR-019 jumps the post-MMU continuation to high VA and unmaps a 16 KiB dedicated identity text range; ADR-020 rewrites rustc vtables to high aliases and unmaps live identity .text after the boot stub (.rodata/.data/heap stay). ADR-021 adds a documented EL0 SVC ABI (exit / uart_write / yield). ADR-022 adds a freestanding libctos CRT that wraps those numbers. ADR-023 adds a guest ELF64 PT_LOAD loader into user TTBR0. ADR-024 promotes standing EL0 so a loaded image runs until exit with fail-closed restore. Those are ABI + CRT + loader + standing-task miles, not app hosting. ADR-025 unmaps identity .rodata after a pointer rewrite (.data/heap stay). ADR-026 reads ID_AA64MMFR1_EL1.PAN on -cpu cortex-a57 (enable stays Planned). ADR-027 adds a thin VFS + in-RAM memfs (create / open / read / write / close; public SVC 19–23). ADR-028 adds virtio-mmio block + FAT16 behind that same VFS (/probe read). ADR-050 deepens FAT16 with guest write + small create. Those are memfs / block-FS miles, not POSIX and not app hosting. The umbrella isolation row stays Planned/non-claim (ADR-047). NFR-08 boot-to-ready CNTPCT is a measurement, not a budget. IDs unchanged.

Status legend: Now = hello-UART / QEMU virt / smoke sensors · Next = near roadmap · Later = aspirational.

Tracker is GitHub (gh). New IDs go through a GitHub issue plus an ADR/docs change — not chat.

Scope

In: freestanding no_std Rust, QEMU -kernel → _start, UART/earlycon output, exceptions/interrupts, paging, heap, basic multitasking, reproducible build/QEMU probes, docs-first harness (honesty, three pillars — antifragility / security / performance — second brain, thin multi-agent).

Out (for now): userspace processes, POSIX, networking stack, GPU, SMP, real-hardware certification, secure-boot productization, Raspberry Pi or other board bring-up (unprobed), x86_64 as a primary path.

IDs below are frozen. Do not invent new FR/NFR IDs in chat; add via issue + ADR/docs change.

Functional requirements

IDRequirementPriorityStage
FR-01Kernel builds as a freestanding no_std / no_main binary for a custom aarch64-*-none target.MustNow
FR-02QEMU (-kernel on virt) loads the kernel ELF and transfers control to a stable _start entry.MustNow
FR-03Kernel can write text to the virt PL011 UART (MMIO, or later earlycon) via print! / println!. Not VGA 0xb8000.MustNow
FR-04Panic handler prints a message (best-effort) and halts without unwinding.MustNow
FR-05Developer can produce a runnable AArch64 kernel ELF and boot it under qemu-system-aarch64 (-machine virt).MustNow
FR-06Synchronous exceptions are handled via VBAR_EL1 vectors (at least a breakpoint / fault path).MustNow
FR-07A fatal exception uses a dedicated stack so overflow does not silently lock the VM.MustNow
FR-08Hardware interrupts: timer via the virt GIC (M5). Input on QEMU virt is PL011 UART RX (M6). The x86-era keyboard wording is this UART path; virtio-keyboard remains later.ShouldNow
FR-09Kernel reads the firmware/QEMU memory map (DTB when probed) and establishes paging / virtual memory. M7: virt RAM convention + linker __kernel_end + EL1 identity map (ADR-008). DTB walk is staged.MustNow
FR-10GlobalAlloc heap so alloc types (Box, Vec) work in kernel. M8: first-fit list on a 64 KiB identity-mapped frame run (ADR-009).ShouldNow
FR-11Cooperative or simple round-robin task switching (threads or async tasks). M9: EL1 yield of AAPCS64 callee-saved GPRs; two heap-backed workers (ADR-010).ShouldNow
FR-12Serial remains usable for headless/CI logs (PL011 or extra earlycon), including later test-exit telemetry.ShouldNext
FR-13Integration tests that boot in QEMU virt and exit with a deterministic success/fail code (ARM semihosting SYS_EXIT, not x86 isa-debug-exit).MustNow
FR-14Documented milestone path (vision → architecture → ADR → roadmap) stays ahead of code for each stage.MustNow
FR-15Honesty ledger records each major capability as claim + probe + status (Verified / Unknown / Planned).MustNow

NFR-10 note (2026-09-12): ADR-039 revises NFR-10 text in place for the EL0 trampoline without TLBI VMALLE1 cut (after ADR-037/038). Threat-model v1.19. IDs unchanged.

NFR-10 note (2026-09-13): ADR-040 revises NFR-10 text in place for taken lower-EL IRQ while standing. Threat-model v1.20. FIQ/SError lower-EL still park. IDs unchanged.

NFR-10 note (2026-09-13, ADR-041): ADR-041 revises NFR-10 text in place for IRQ-unmasked default standing/task ERET. Threat-model v1.21. Short non-standing trampoline probes stay masked. FIQ/SError lower-EL still park. IDs unchanged.

NFR-10 note (2026-09-13, ADR-049): ADR-049 revises NFR-10 text in place for leftover identity RAM tear (ident: ram / ident: ram-fault / ident: ram-high; ident: ram-stay superseded). Threat-model v1.27. Umbrella isolation / PAN enable / taken SError / yank _start stay decided as in ADR-047. IDs unchanged.

NFR-10 note (2026-09-13, ADR-042): ADR-042 revises NFR-10 text in place for the isolation leftover honesty wrap (ident: start-stay / ident: ram-stay). Threat-model v1.22. Umbrella isolation / PAN enable / FIQ-SError / full _start teardown stay Planned. IDs unchanged.

NFR-10 note (2026-09-13, ADR-043): ADR-043 revises NFR-10 text in place for taken lower-EL FIQ while standing (el0: fiq) and SError park honesty (el0: serror-park). Threat-model v1.23. Taken SError / PAN enable / umbrella isolation stay Planned. IDs unchanged.

NFR-10 note (2026-09-13, ADR-047 / ADR-048): ADR-047 decides isolation leftovers (taken SError deferred/non-goal on this smoke machine; PAN enable non-goal on default cortex-a57; never yank _start; leftover RAM optional Planned; umbrella Planned/non-claim). ADR-048 writes product app-hosting claim criteria (still Planned at that note). Threat-model v1.26. IDs unchanged.

NFR-10 note (2026-09-13, ADR-048 / ADR-052): Product app-hosting claim is Verified under ADR-048 criteria Met×5; sponsor accept recorded in ADR-052. Threat-model v1.29. Still not Linux/POSIX/containers/“EL0 isolated.” IDs unchanged.

NFR-10 note (2026-09-14, ADR-053 / ADR-054 / ADR-055): Isolation-gap locks — taken SError hard-stopped (ADR-053); PAN enable locked non-goal on default cortex-a57 (ADR-054); umbrella “EL0 isolated” non-claim until checklist (ADR-055). Threat-model v1.30. IDs unchanged.

NFR-10 note (2026-09-14, ADR-056): FAT16 root listing behind the thin VFS (fat: readdir / fat: entries, ADR-056). Threat-model v1.31. Not POSIX getdents. IDs unchanged.

NFR-10 note (2026-09-14, ADR-057): FAT16 root delete behind the thin VFS (fat: delete, ADR-057). Threat-model v1.32. Not POSIX unlink. IDs unchanged.

NFR-10 note (2026-09-14, ADR-058): Thin VFS prefix mounts (vfs: mount / vfs: mounts, ADR-058). Threat-model v1.33. Not POSIX mount(2). IDs unchanged.

NFR-10 note (2026-09-28, ADR-091 / M6): the sponsor accepted the ADR-091 revision-3 sentence and scope in writing (B3, 2026-09-28 16:41 CEST). Umbrella “EL0 isolated” (P-SEC-3l) is Verified, only as that sentence: one core, the TCG default smoke machine; not a speculative-execution or side-channel claim, not a real-hardware claim, not a certification; EL0 can read its own code bytes; no multi-core race story. Checklist ADR-055 all Met. Threat-model v1.70. Earlier notes that say the umbrella is Planned / non-claim are as of their dates. IDs unchanged.

NFR-10 note (2026-10-02, ADR-091 revision 4): the sponsor accepted ADR-091 revision 4 in writing (“I accept ADR-091 rev 4”). Only the PAN clause and the code-bytes limit changed: the limit is now “app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN)” (was “EL0 can read its own code bytes”). The umbrella “EL0 isolated” (P-SEC-3l) stays Verified, only as the revision-4 sentence in ADR-091. Threat-model v1.74. The 2026-09-28 note above records revision 3 as of its date. IDs unchanged.

Non-functional requirements

IDRequirementPriorityStage
NFR-01 SafetyPrefer safe Rust; unsafe only at hardware/FFI boundaries, minimized and justified.MustNow
NFR-02 ReproducibilityToolchain pinned (rust-toolchain.toml); target JSON + .cargo/config.toml checked in.MustNow
NFR-03 Portability (dev)Build+QEMU path works on Windows/macOS/Linux (host tools); kernel target remains freestanding AArch64.MustNow
NFR-04 CI / sensorsPR gate runs scripts/qemu-smoke.sh (build + UART hello string + cargo test exit codes). GitHub Actions + optional Docker.ShouldNow
NFR-05 AntifragilityFirst-class pillar (ADR-011). Repeated failures become ratchets (#[test_case], scripts/qemu-smoke.sh grep, Dockerfile pin, GHA) — not README-only fixes. Fail-closed sensors stay on the strongest layer.MustNow
NFR-06 HonestyStatus words in docs/PRs need a probe; unprobed stays Unknown; no self-merge as “verified.”MustNow
NFR-07 PerformanceFirst-class pillar (ADR-011). Performance claims need a measurable probe (CNTPCT delta around a known path; IRQ-to-handler CNTPCT−CVAL min/max/spread). No fake benches and no “faster than X.” Optimize only after a probe shows a cost. Not a hard latency budget.ShouldNow
NFR-08 FootprintDebug image size and boot time tracked once measurable; no premature optimization.CouldLater
NFR-09 MaintainabilityClear module layout; ADRs for boot-path, console, or allocator choices.MustNow
NFR-10 SecurityFirst-class pillar (ADR-011). No secrets in repo. Do not claim “secure OS” / “hardened” / “the kernel is W^X” without a written threat model and a probe. Prefer minimize unsafe (NFR-01). Heap and heap-backed cooperative stacks are PXN (ADR-012). Linker-stack downward overflow hits an unmapped 4 KiB guard (ADR-014). .text/.rodata are RO+X and .data/.bss/live linker stacks are RW+NX; SCTLR.WXN is on (ADR-015). That scoped image cut is not a “secure OS” sentence. Device MMIO XN. Least privilege on IRQ paths. EL0 first mile (enter/return + cannot execute kernel data), the user-TTBR0 read mile, the ASID-tagged TLB mile, and the standing EL0 context are probes (ADR-013). TTBR1 private-page first cut is a probe (ADR-016). EL1 fetch from the TTBR1 RAM alias (plus high VBAR_EL1) is a probe (ADR-017). The first identity-tear cut (split TTBR1 RAM tables + one unmapped identity text page) is a probe (ADR-018). A 16 KiB dedicated identity text range plus a high-VA continuation is a probe (ADR-019). A high-VA vtable / fn-pointer rewrite plus live identity .text tear after the boot stub is a probe (ADR-020); identity -kernel stub stays. An identity .rodata tear after a pointer rewrite is a probe (ADR-025); .data/heap stay; full identity teardown stays Planned. A PAN ID-field read on -cpu cortex-a57 is a probe (ADR-026); PAN enable stays Planned. A documented EL0 SVC ABI (exit / uart_write / yield) is a probe (ADR-021). A freestanding libctos CRT wrapping those numbers is a probe (ADR-022). A guest ELF64 PT_LOAD loader into user TTBR0 is a probe (ADR-023); not a Linux ABI. Standing EL0 as normal mode for a loaded image until exit (fail-closed restore) is a probe (ADR-024). A thin VFS + in-RAM memfs (create / open / read / write / close; SVC 19–23) is a probe (ADR-027); not POSIX (product freestanding hosting is Verified under ADR-048/052 separately). virtio-mmio block + FAT16 behind that same VFS (/probe read) is a probe (ADR-028); FAT16 write + small create is a probe (ADR-050); FAT16 root listing is a probe (ADR-056); FAT16 root delete is a probe (ADR-057); thin VFS prefix mounts are a probe (ADR-058); host -drive without guest code is not that probe. An OS-image vs app-payload slot first cut (published hello-libctos.elf + FAT /hello + A3 map) is a probe (ADR-030). A2–A4 load that FAT file (no hello include_bytes!); the same published ELF on this OS and prior ba6541c is a probe (ADR-032). Identity .data/stacks and heap are torn (ident: data / ident: heap, ADR-037 / ADR-038); standing/EL0 trampoline switches without TLBI VMALLE1 (el0: no-vmalle1, ADR-039); taken lower-EL IRQ while standing returns to EL0 (el0: irq, ADR-040); default standing/task ERET clears IRQ mask (el0: irq-default, ADR-041); taken lower-EL FIQ while standing (el0: fiq, ADR-043); SError park honesty (el0: serror-park, ADR-045 A-clear + QMP attempt); short non-standing trampoline probes stay masked; _start stays (ident: start-stay); leftover identity frames after the heap are torn (ident: ram*, ADR-049); taken SError is hard-stopped/deferred (ADR-053); PAN enable is locked non-goal on default cortex-a57 (ADR-054); never yank _start; umbrella “EL0 isolated” is Verified only as the accepted ADR-091 sentence since M6 (2026-09-28; checklist ADR-055 / ADR-047); product “app hosting is done” is Verified under ADR-048 / ADR-052 (freestanding slots; not Linux/POSIX/containers). Do not claim PAN on -cpu cortex-a57.MustNow
NFR-11 ObservabilityQEMU UART + explicit test exit codes are first telemetry; host Grafana out of scope.ShouldNext
NFR-12 Multi-agentThin roles (engineer / knowledge / coherence / MRC); implementer ≠ merge approver.ShouldNow
NFR-13 Document-firstWritten milestone acceptance criteria match what code/CI actually prove.MustNow
NFR-14 Second brainSession memory + daily briefs in git so agents inherit state without chat archaeology.ShouldNow

Acceptance sketch (Now)

  • cargo build succeeds with pinned nightly for aarch64-ctos.json.
  • qemu-system-aarch64 -machine virt shows the hello line on serial or honesty ledger says Unknown until probed.
  • scripts/qemu-smoke.sh fails closed if the hello string is missing or cargo test does not exit 0.
  • cargo test uses ARM semihosting so QEMU exits 0 on pass and 1 on panic (force-fail).
  • Current-EL BRK is handled via VBAR_EL1 (serial string and/or #[test_case]) or the honesty ledger says Unknown until probed.
  • Nested / fatal exception prints a serial marker from a dedicated stack (FR-07) or the honesty ledger says Unknown until probed.
  • A timer tick is observable on serial and/or #[test_case] via the virt GIC (FR-08 / M5) or the honesty ledger says Unknown until probed.
  • A received PL011 byte is observable on serial (FR-08 input / M6) or the honesty ledger says Unknown until probed.
  • MMU-on + allocate-frame / map-unmap is observable on serial and/or #[test_case] (FR-09 / M7) or the honesty ledger says Unknown until probed.
  • Box / Vec on the kernel heap is observable on serial and/or #[test_case] (FR-10 / M8) or the honesty ledger says Unknown until probed.
  • Two cooperative tasks are observable on serial and/or #[test_case] (FR-11 / M9) or the honesty ledger says Unknown until probed.
  • A CNTPCT baseline delta is observable on serial and/or #[test_case] (NFR-07) or the honesty ledger says Unknown / Planned until probed.
  • An IRQ-to-handler CNTPCT delta (min/max/spread) is observable on serial and/or #[test_case] (NFR-07) or the honesty ledger says Unknown until probed.
  • Threat-model v1.1 exists under docs/framework/security.md; “secure OS” stays unclaimed (NFR-10).
  • Heap NX / execute-from-heap caught is observable on serial and/or #[test_case] (NFR-10 / ADR-012) or the honesty ledger says Unknown until probed.
  • Linker-stack guard store faults observably (NFR-10 / ADR-014) or the honesty ledger says Unknown until probed.
  • EL0 first mile (SVC return and/or UXN IABORT) is observable on serial and/or #[test_case] (NFR-10 / ADR-013) or the honesty ledger says Unknown / Planned until probed.
  • Host debug ELF size is printed by scripts/qemu-smoke.sh (NFR-08) or the honesty ledger says Unknown until probed.
  • Boot-to-ready CNTPCT (perf: boot-delta) is observable on serial and/or #[test_case] (NFR-08) or the honesty ledger says Unknown until probed.
  • RO+NX text/data (execute-from-.data and write-to-RO-text) is observable on serial and/or #[test_case] (NFR-10 / ADR-015) or the honesty ledger says Unknown until probed.
  • EL0 cannot-read-kernel-data (user TTBR0) is observable on serial and/or #[test_case] (NFR-10 / ADR-013) or the honesty ledger says Unknown / Planned until probed.
  • ASID isolation (dual ASID without TLBI VMALLE1 + conflict/stale-entry probe) is observable on serial and/or #[test_case] (NFR-10 / ADR-013) or the honesty ledger says Unknown / Planned until probed.
  • Standing EL0 (el0: standing / el0: restored, is_active() flips) is observable on serial and/or #[test_case] (NFR-10 / ADR-013) or the honesty ledger says Unknown / Planned until probed.
  • TTBR1 private page (ttbr1: el1 / ttbr1: no el0 / ttbr1: ok) is observable on serial and/or #[test_case] (NFR-10 / ADR-016) or the honesty ledger says Unknown / Planned until probed.
  • EL1 fetch from the TTBR1 RAM alias (ttbr1: el1 exec / ttbr1: vbar) is observable on serial and/or #[test_case] (NFR-10 / ADR-017) or the honesty ledger says Unknown / Planned until probed. Identity -kernel stub at 0x40080000 stays.
  • Identity-tear first cut (ident: split / ident: fault / ident: high / ident: no el0 / ident: ok) is observable on serial and/or #[test_case] (NFR-10 / ADR-018) or the honesty ledger says Unknown / Planned until probed. High-VA continuation plus a 16 KiB dedicated identity text range (ident: jump / ident: range / ident: text) is observable on serial and/or #[test_case] (NFR-10 / ADR-019) or the honesty ledger says Unknown / Planned until probed. High-VA vtable rewrite plus live identity .text tear (ident: reloc / ident: live) is observable on serial and/or #[test_case] (NFR-10 / ADR-020) or the honesty ledger says Unknown / Planned until probed. Identity .rodata tear (ident: rodata / ident: rodata-fault / ident: rodata-high) is observable on serial and/or #[test_case] (NFR-10 / ADR-025) or the honesty ledger says Unknown / Planned until probed. PAN ID field (pan: id= / pan: absent) is observable on serial and/or #[test_case] (NFR-10 / ADR-026) or the honesty ledger says Unknown / Planned until probed. Identity .data/stacks tear (ident: data / ident: data-fault / ident: data-high) is observable (NFR-10 / ADR-037) or the honesty ledger says Unknown / Planned until probed. Identity heap tear (ident: heap-reloc / ident: heap / ident: heap-fault / ident: heap-high) is observable (NFR-10 / ADR-038) or the honesty ledger says Unknown / Planned until probed. EL0 entry without TLBI VMALLE1 (el0: no-vmalle1) is observable (NFR-10 / ADR-039) or the honesty ledger says Unknown / Planned until probed. Lower-EL IRQ while standing (el0: irq) is observable (NFR-10 / ADR-040) or the honesty ledger says Unknown / Planned until probed. Lower-EL FIQ while standing (el0: fiq) and SError park honesty (el0: serror-park) are observable (NFR-10 / ADR-043 / ADR-045) or the honesty ledger says Unknown / Planned until probed. Yank _start decided never; leftover identity RAM after the heap is torn (ADR-049). Taken lower-EL SError is deferred/non-goal on this smoke machine. PAN is enabled on default cortex-a76 (ADR-080; was a non-goal on the historical cortex-a57). Umbrella “EL0 isolated”: see the M6 note above (ADR-091 sentence only). Do not claim PAN.
  • EL0 SVC ABI (svc: yield / svc: user-hi / svc: uart / svc: exit / svc: ok) is observable on serial and/or #[test_case] (NFR-10 / ADR-021 / Track A A1) or the honesty ledger says Unknown / Planned until probed. This mile alone ≠ the ADR-048 product claim (Verified separately under ADR-048/052). Not a Linux ABI.
  • libctos CRT (libctos: hi / libctos: ok / libctos: linked) is observable on serial and/or #[test_case] (NFR-10 / ADR-022 / Track A A2) or the honesty ledger says Unknown / Planned until probed. This mile alone ≠ the ADR-048 product claim (Verified separately under ADR-048/052).
  • Guest ELF PT_LOAD loader (loader: mapped / loader: ok) is observable on serial and/or #[test_case] (NFR-10 / ADR-023 / Track A A3) or the honesty ledger says Unknown / Planned until probed. Not a Linux ABI. This mile alone ≠ the ADR-048 product claim (Verified separately under ADR-048/052).
  • Standing EL0 as normal mode (el0: task-enter / el0: task-active / el0: task-exit / el0: task-restored / el0: restore-fail / el0: task-ok) is observable on serial and/or #[test_case] (NFR-10 / ADR-024 / Track A A4) or the honesty ledger says Unknown / Planned until probed. Not isolation. This mile alone ≠ the ADR-048 product claim (Verified separately under ADR-048/052).
  • Thin VFS + memfs (fs: create / fs: write / fs: read / fs: el0 / fs: ok) is observable on serial and/or #[test_case] (NFR-10 / ADR-027 / Track A A6) or the honesty ledger says Unknown / Planned until probed. Not POSIX. This mile alone ≠ the ADR-048 product claim (Verified separately under ADR-048/052).
  • virtio-blk + FAT16 (blk: virtio / blk: cap / blk: rw / blk: ok / fat: mount / fat: read / fat: ok) is observable on serial and/or #[test_case] (NFR-10 / ADR-028 / Track A A7) or the honesty ledger says Unknown / Planned until probed. Same VFS open. Not POSIX. Not FAT32. This mile alone ≠ the ADR-048 product claim (Verified separately under ADR-048/052).
  • FAT16 write depth (fat: write / fat: rewrite / fat: create / host --check-write) is observable on serial and/or #[test_case] (NFR-10 / ADR-050) or the honesty ledger says Unknown / Planned until probed. Same VFS write. Not POSIX. Not FAT32. This mile alone ≠ the ADR-048 product claim (Verified separately under ADR-048/052).
  • FAT16 root readdir (fat: readdir / fat: entries) is observable on serial and/or #[test_case] (NFR-10 / ADR-056) or the honesty ledger says Unknown / Planned until probed. Same thin VFS. Not POSIX getdents. Not FAT32. This mile alone ≠ the ADR-048 product claim (Verified separately under ADR-048/052).
  • FAT16 root delete (fat: delete) is observable on serial and/or #[test_case] (NFR-10 / ADR-057) or the honesty ledger says Unknown / Planned until probed. Same thin VFS. Not POSIX unlink. Not FAT32. This mile alone ≠ the ADR-048 product claim (Verified separately under ADR-048/052).
  • Thin VFS prefix mounts (vfs: mount / vfs: mounts) are observable on serial and/or #[test_case] (NFR-10 / ADR-058) or the honesty ledger says Unknown / Planned until probed. Same thin VFS. Not POSIX mount(2). Not a new disk FS. This mile alone ≠ the ADR-048 product claim (Verified separately under ADR-048/052).
  • Panic path compiles and is reachable in principle.
  • This FR/NFR file + honesty ledger live under docs/.

Traceability

Map roadmap milestones and PRs to FR/NFR IDs in PR descriptions when touching behavior.

Taken lower-EL SError on this smoke machine is deferred / non-goal (hard-stopped) (ADR-053 / ADR-047); research ADR-044; attempt ADR-045. Park honesty stays ADR-043. Product app hosting claim gate: ADR-048.

Technical architecture

ctos is a #![no_std] #![no_main] binary. There is no Rust standard library and no OS underneath. The crate builds for a custom target (aarch64-ctos.json): os: none, panic abort, red zone off, static relocation, soft-float (no early SIMD/FP). See ADR-003. Guest samples (not “apps”): apps-today.md. Porting stance: building-or-porting.md.

Boot path

  1. qemu-system-aarch64 -machine virt -cpu cortex-a57 (or max; gic-version=3 is allowed).
  2. -kernel loads the kernel ELF into virt RAM (linker base 0x40080000).
  3. Entry is _start (assembly in src/main.rs): set SP, zero BSS, call kernel_main.
  4. Early output is the virt PL011 UART at 0x0900_0000 (src/uart.rs).
  5. kernel_main installs VBAR_EL1 (src/exception.rs, ADR-004) after UART init, then the EL1 identity map plus a TTBR1 private page and a cloned TTBR1 RAM alias (ADR-008, W^X split ADR-012, stack guards ADR-014, RO+NX ADR-015, TTBR1 first cut ADR-016, EL1 fetch mile ADR-017, identity-tear first cut ADR-018, identity .text range tear ADR-019, live .text tear ADR-020, identity .rodata tear ADR-025), then the frame pool after MMU + SCTLR.C (pre-MMU .bss stores can vanish), then the first-fit heap (ADR-009), then the cooperative scheduler (ADR-010), then GICv2 + CNTP (ADR-006). After Hello World! it proves map/unmap, Box/Vec, two tasks, heap NX, linker-stack guards, the EL0 first mile + standing context, ASID isolation, the TTBR1 private page and high-VA EL1 fetch, torn identity .text and .rodata, the PAN ID field, CNTPCT + IRQ-delta, several timer ticks, and polls PL011 RX (ADR-007).
flowchart LR
  Q["QEMU virt<br/>cortex-a57"] --> K["-kernel ELF<br/>@ 0x40080000"]
  K --> S["_start<br/>SP + BSS"]
  S --> U["PL011 UART"]
  U --> M["kernel_main"]

QEMU loads one OS ELF. Early print is UART; everything else starts in kernel_main. No bootloader crate, no VGA.

There is no bootloader 0.9 crate and no VGA buffer. The x86_64 phil-opp path was deleted when this ADR landed.

.cargo/config.toml still needs json-target-spec = true on rustc 1.100 nightly. The AArch64 JSON is taken from aarch64-unknown-none-softfloat plus os: none / numeric widths. That nightly rejected the file until both "abi": "softfloat" and "rustc-abi": "softfloat" were set. Behavior is bare-metal AArch64, abort, rust-lld.

This architecture does not claim Raspberry Pi or other SoC support.

Privilege: EL1 kernel vs standing EL0

The kernel runs at EL1. A freestanding image can stand at EL0 and ask the kernel for help with SVC (exit / uart_write / yield / thin-VFS numbers). That is the ctos process model (ADR-035) — not Linux fork / execve. Many isolation miles are Verified; the umbrella “EL0 isolated” is Verified only as the accepted ADR-091 sentence (M6, 2026-09-28; one core, TCG default smoke machine; not speculative-execution / side-channel, not real hardware, not a certification) (ADR-055; sponsor leftovers closure ADR-060). Details: el0.md.

flowchart TD
  EL1["EL1 — kernel<br/>page tables · SVC handler · VFS"]
  EL0["EL0 — standing task<br/>loaded freestanding image"]
  SVC["SVC<br/>exit · uart_write · yield · fs_*"]
  UMB["Umbrella “EL0 isolated”<br/>Verified: ADR-091 sentence only — M6"]
  EL0 -->|"asks"| SVC --> EL1
  EL1 -->|"ERET back"| EL0
  EL0 -.-> UMB

Standing EL0 + SVC is real. “EL0 isolated” is claimed only as the accepted ADR-091 sentence (one core, TCG; not side-channel, not real hardware, not a certification). Stock-QEMU taken SError still parks, and _start is never yanked (ADR-089 exception).

Memory sketch (identity / TTBR0 / TTBR1)

Simplified view of what the guest maps after bring-up. Live identity .text / .rodata / .data / heap / leftover RAM tears are Verified miles; the QEMU -kernel stub at _start stays mapped (ident: start-stay). High twins live under TTBR1. User TTBR0 is what standing EL0 walks.

flowchart TD
  subgraph ttbr0k ["TTBR0 kernel (EL1)"]
    ID["Identity map<br/>torn live ranges"]
    ST["_start stub stays<br/>0x40080000"]
  end
  subgraph ttbr0u ["TTBR0 user (EL0)"]
    UW["User window<br/>text/rodata + trampoline<br/>omits kernel .data/heap"]
  end
  subgraph ttbr1 ["TTBR1 high"]
    HI["High RAM alias + VBAR<br/>EL1 fetch twin"]
    PRIV["TTBR1_PRIV<br/>EL1-only page"]
  end
  ID --> ST
  ID -.->|"high twin"| HI
  UW -->|"SVC / ERET"| ID

_start stays while live identity ranges are torn. User TTBR0 is not a full process mm. Do not say “the kernel moved.”

OS vs app slots

QEMU still -kernels one OS ELF. Published freestanding EL0 samples live on the same FAT16 volume: /hello (A9 / UART+one yield), /fsdemo (ADR-059 / memfs via libctos), /fatdemo (ADR-061 / FAT /probe via libctos), /yldemo (ADR-062 / cooperative yield rounds). A2–A4 load /hello (no production embed). Cross-update is the same hello ELF on this OS and prior OS ba6541c. Product freestanding “app hosting done” is Verified under ADR-048 / ADR-052 — not Linux/POSIX/containers/OTA. Recipes: what-can-run.md. Companion: hosting-apps.md.

flowchart LR
  OS["OS ELF<br/>QEMU -kernel"] --> FAT["FAT16 volume<br/>virtio-blk"]
  H["/hello<br/>UART + yield"] --> FAT
  F["/fsdemo<br/>memfs VFS"] --> FAT
  D["/fatdemo<br/>FAT /probe"] --> FAT
  Y["/yldemo<br/>yield rounds"] --> FAT
  FAT --> LOAD["Guest PT_LOAD<br/>→ standing EL0"]

One guest volume, three freestanding samples. Slots disconnect OS rebuilds from the published ELFs. Not a container runtime. Not POSIX.

Thin VFS stack

One thin VFS surface (open / read / write / close / readdir / unlink) with two backends and a prefix mount table: in-RAM memfs and FAT16 on virtio-blk. /mem (+ A6 names) → memfs; / → FAT16. Read, write, root listing, delete, and mounts are depth miles — not POSIX. Stance: filesystem.md. ADR-058.

flowchart TD
  APP2["Standing EL0 / EL1 callers"] --> VFS["Thin VFS<br/>open · read · write · close · readdir · unlink"]
  VFS --> MT["Prefix mount table<br/>ADR-058"]
  MT -->|"/mem · A6 names"| MEM["memfs<br/>in-RAM named buffers"]
  MT -->|"/"| FAT2["FAT16<br/>read · write · root readdir · unlink"]
  FAT2 --> BLK["virtio-blk<br/>host fat16.img"]

Same API, two backends, prefix mounts. Say “routed path prefixes through a mount table” when vfs: mounts passes — not mount(2) or “supports FAT” as a product.

Track N — virtio-net path (ARP + ICMP + UDP + EL0 net/UDP SVC)

QEMU user netdev (SLIRP) plus guest virtio-net-mmio. N1 exchanges one ARP request/reply with gateway 10.0.2.2 (ADR-066). N2 sends an ICMP echo and expects a reply (ADR-067). N3 sends one UDP datagram to SLIRP DNS 10.0.2.3:53 and expects a UDP reply (ADR-070) — DNS is probe bait, not a DNS product. N4 exposes tiny EL0 SVCs (net_mac / net_ping) and FAT /netdemo (ADR-068). N3.x adds net_udp_dns and FAT /udpdemo (ADR-071) — kernel still owns virtio-net. No BSD sockets / TCP product. Spec/ADR wins on conflict with this sketch.

flowchart LR
  QN["QEMU user netdev<br/>SLIRP 10.0.2.2 / 10.0.2.3"] --> VN["virtio-net-device<br/>mmio"]
  VN --> GTX["Guest TX<br/>ARP / ICMP / UDP"]
  VN --> GRX["Guest RX<br/>ARP / ICMP / UDP reply"]
  GTX --> VN
  GRX --> OK["Markers<br/>net: ok · net: ping-ok · net: udp-ok"]
  OK --> EL0["EL0 SVCs<br/>net_mac / net_ping<br/>/netdemo"]
  OK --> EL0U["EL0 SVC<br/>net_udp_dns<br/>/udpdemo"]

Host flags alone are not a probe. Do not say “has networking” or “has sockets.” Cite ADR-063 / ADR-066 / ADR-067 / ADR-070 / ADR-071 / ADR-068 / ADR-071.

Current stage (UART hello + M2–M9 + ADR-011 pillars)

  • Pl011 writer with TX-full wait and \n → \r\n; try_recv on UARTFR.RXFE / UARTDR
  • print! / println! via spin::Mutex; fatal / unhandled / IRQ paths write the PL011 without the mutex
  • After EL1 + VBAR, a bump frame allocator (src/frame.rs) and identity map (src/paging.rs) turn the MMU on (ADR-008); RAM after __kernel_end is PXN (ADR-012); 4 KiB holes sit under the linker stacks (ADR-014)
  • First-fit GlobalAlloc (src/heap.rs) on a 64 KiB identity-mapped frame run (ADR-009); build-std includes alloc
  • Cooperative round-robin (src/sched.rs): idle on the linker thread stack, two heap-backed workers, AAPCS64 callee-saved yield (ADR-010)
  • kernel_main prints Hello World!, proves map/unmap (serial paging: ok), proves Box/Vec (serial heap: ok), proves two tasks (serial sched: task a / sched: task b / sched: ok), proves heap NX (serial wx: ok), proves linker-stack guards (serial guard: ok), proves the EL0 first mile + standing context (serial el0: ok / el0: standing / el0: restored), proves the SVC ABI (serial svc: yield / svc: user-hi / svc: uart / svc: exit / svc: ok), proves the libctos CRT (serial libctos: hi / libctos: ok / libctos: linked), proves the guest ELF PT_LOAD loader (serial loader: mapped / loader: ok), proves standing EL0 as a loaded task until exit (serial el0: task-enter / el0: task-active / el0: task-exit / el0: task-restored / el0: restore-fail / el0: task-ok), proves ASID isolation (serial asid: ok), proves the TTBR1 private page and high-VA EL1 fetch (serial ttbr1: ok / ttbr1: el1 exec / ttbr1: vbar), proves the identity-tear first cut, .text range tear, vtable rewrite, live .text tear, and identity .rodata tear (serial ident: ok / ident: jump / ident: reloc / ident: range / ident: live / ident: rodata / ident: rodata-fault / ident: rodata-high / ident: text / ident: split / ident: fault / ident: high / ident: no el0), proves the PAN ID field (serial pan: id= / pan: absent), proves CNTPCT advances (serial perf: cntpct delta=…), observes CNTP ticks (serial timer: tick) plus IRQ-to-handler deltas (serial perf: irq-delta), polls one host-injected RX byte (serial input: rx 0x41), fires one healthy-stack BRK #0 (serial exception: sync BRK), then the FR-07 nest probe (serial exception: fatal nested)
  • VBAR_EL1 vector table (high alias after MMU, ADR-017); kernel runs on SP_EL0; first-level current-EL sync (SP_EL0 bank) handles AArch64 BRK, heap NX, guard-page data aborts, and the identity-tear IABORT; first-level IRQ handles GICv2 PPI 30; lower-EL AArch64 sync handles the EL0 SVC / UXN IABORT / standing dual-SVC / public SVC ABI / TTBR1 DABORT / torn-page DABORT; nested current-EL (SP_ELx bank) switches to the fatal stack (ADR-004, ADR-005, ADR-006, ADR-007, ADR-013, ADR-016, ADR-017, ADR-018, ADR-019, ADR-020, ADR-021)
  • cargo test uses #![feature(custom_test_frameworks)] and #[test_case] (including VBAR, BRK, SPSel, stack ranges, guards, GIC TYPER, CNTFRQ, timer tick, IRQ-delta samples, empty UART RX FIFO, MMU on, frames, map/unmap, heap Box/Vec, two-task yield, CNTPCT loop, W^X flags + execute-from-heap, EL0 first mile + standing enter/leave, SVC ABI yield/uart/exit + kernel-.data / TTBR1-alias reject, libctos hello trip, guest ELF loader trip, standing-task until exit + fail-closed fault restore, ASID isolation, TTBR1 private page + high-VA EL1 fetch, identity-tear first cut + .text range tear + vtable rewrite + live .text tear; is_active() is false at rest)
  • QEMU exit is ARM semihosting SYS_EXIT / hlt #0xf000 (src/qemu.rs), not isa-debug-exit. Needs -semihosting on the QEMU line (scripts/qemu-aarch64.sh).
  • Host smoke: scripts/qemu-smoke.sh (hello + paging + heap + two-task sched + W^X + guards + EL0 first mile + standing + SVC ABI + libctos CRT + guest ELF loader + standing-task (A4) + ASID isolation + TTBR1 private page + high-VA EL1 fetch + identity-tear first cut + identity .text range tear + vtable rewrite + live .text tear + CNTPCT baseline + IRQ-delta + host ELF size + timer tick + injected UART RX + BRK + fatal nested strings + tests + force-fail must be non-zero)
  • Docker: Dockerfile / scripts/docker-smoke.sh (linux/arm64-friendly; do not pin amd64)
  • GHA: .github/workflows/smoke.yml (ubuntu-24.04-arm and ubuntu-24.04)

Source + local smoke were first probed on 2026-09-08 (see the honesty ledger). Current idle tip is main ≈ 190ef5a (Merge PR #106 / docs polish after ADR-061). Recent docs/miles: ADR-058–061 plus ADR-062 yield-libctos (this PR when merged). Earlier Docker Verified on tip e80dc93 (Merge PR #28 / ADR-020; 50 tests, ident: reloc n=12, live pages=37, force-fail ok) and prior SHAs 24d94e6 (ADR-019), b0f0ee5 (#24–#26), 71ee15f (layout L3). Keep the b2bbb99 Failed row. GHA merge-commit 34651404108 on e80dc93 grepped that ADR-020 class of markers. A Route 53 CNAME ctos.artof.link → artofdream.github.io. exists; https://ctos.artof.link HTTPS is Verified (2026-09-11 after #30). Do not claim “secure OS” or “the kernel moved”; say “EL0 isolated” only as the ADR-091 sentence. Tip smoke / Docker / GHA on 97a8559 itself: see the honesty ledger (do not invent Verified from this docs edit).

Planned stages

StageDomain work
Custom test frameworkLanded (M2): #[test_case], semihosting exit, UART
CPU exceptionsM3: VBAR_EL1, resumable BRK. M4: dedicated exception + fatal stacks (FR-07) — cloud qemu-smoke Verified (honesty ledger); GHA Unknown until a run URL
Hardware interruptsM5: GICv2 + CNTP tick (FR-08) — cloud qemu-smoke + GHA Verified (honesty ledger). M6: PL011 UART RX (FR-08 input / ADR-007)
PagingM7: EL1 identity map + bump frames (FR-09 / ADR-008) — probe status in the honesty ledger. DTB walk staged.
HeapM8: first-fit GlobalAlloc on identity-mapped frames (FR-10 / ADR-009) — probe status in the honesty ledger.
SchedulerM9: cooperative EL1 yield (FR-11 / ADR-010) — probe status in the honesty ledger. Not preemptive.
PillarsADR-011: antifragility / security / performance. Threat-model v1.37 (security.md). Heap NX (ADR-012). Linker-stack guards (ADR-014). EL0 first mile + standing + ASID TLB mile; TTBR1 first cut (ADR-016); EL1 high-VA fetch (ADR-017); identity-tear first cut (ADR-018); identity .text range tear (ADR-019); live .text tear (ADR-020); SVC ABI (ADR-021); libctos CRT (ADR-022); guest ELF PT_LOAD loader (ADR-023); standing EL0 as normal mode (ADR-024); identity .rodata tear (ADR-025); PAN capability (ADR-026, enable locked non-goal on default a57 per ADR-054); thin VFS + memfs (ADR-027); virtio-blk + FAT16 (ADR-028); FAT16 write (ADR-050); FAT16 readdir (ADR-056); OS/app slot first cut (ADR-030); leftover cross-update + FAT-only hello (ADR-032); freestanding sample catalog /hello + /fsdemo + /fatdemo + /yldemo (ADR-059 / ADR-061 / ADR-062); umbrella “EL0 isolated” Verified only as the ADR-091 sentence since M6 (ADR-013 / ADR-047 / ADR-055; sponsor leftovers closure ADR-060). Product freestanding app hosting Verified under ADR-048 / ADR-052 (Track A #31; not Linux/POSIX/containers).
FilesystemThin VFS + memfs (ADR-027) + virtio-blk / FAT16 read (ADR-028) + write (ADR-050) + root readdir (ADR-056) + root delete (ADR-057) + prefix mounts (ADR-058). FAT also carries freestanding samples /hello / /fsdemo / /fatdemo / /yldemo (catalog, not a second FS). Stance: filesystem.md.
Host apps / containersGaps to Linux/shell/Python: host-apps.md. Linux-compat is a research frame (ADR-031); not claiming Linux userspace yet. Guest container runtime is a non-goal (ADR-029). Host docker-smoke is unrelated. A1 is a kernel SVC ABI mile. A2 is a libctos CRT mile. A3 is a guest ELF PT_LOAD loader mile. A4 is standing EL0 as normal mode for a loaded image. A6 is thin VFS + memfs. A7 is virtio-blk + FAT16. A8/A9 recipes + slot; freestanding catalog /hello + /fsdemo + /fatdemo + /yldemo (ADR-059 / ADR-061 / ADR-062).
ImmutabilityScoped RO only (immutability.md). Absolute “immutable OS” is incompatible. Track A #31 / Track B #40.

Each stage is one loop unit on the roadmap.

x86_64 remains a possible future secondary ISA. It is not a current tree.

Harness mapping (short)

Hardware and QEMU are the domain. Docs, ADRs, and this architecture note are shared understanding. Guides, sensors, the one-PR loop, second-brain vaults, merge permissions, and the honesty ledger are the outer harness. Details: formula.md.

ADR-001 — Apply a honesty/harness practice to ctos

  • Status: Accepted
  • Date: 2026-09-08

Context

ctos is a bare-metal kernel. Agents and humans will extend it across sessions. Status words drift: “it boots” gets written because source exists, or because a README says cargo run. Without a probe, that is fiction.

Harness-engineering practice (guides, sensors, a tight loop, persistent memory, no self-merge, observability of claims) is useful here. Prior art: architecture.artof.link. This ADR does not adopt that site’s product, florist domain, or stakeholder roster.

Decision

Apply a ctos-native honesty/harness practice:

  1. Honesty — every status word is a claim; the honesty ledger records the probe. Unprobed = Unknown. Never round Unknown up to Verified.
  2. Document-first — vision, architecture, ADRs, and roadmap land before more kernel features.
  3. Second brain — four vaults under research/ (procedure via skills, correction via constraints/rules, relationship via doc links, daily brief / session memory).
  4. Thin roles — Knowledge Guardian, Coherence Guardian, Kernel Engineer, MR Coordinator (ctos-* skills only).
  5. Loop — one milestone → one branch → one GitHub PR. The author does not self-approve or merge (see ADR-002).
  6. Sensors over vibes — when a failure repeats, strengthen a sensor or gate (see antifragility.md).

Tracker and reviews stay on GitHub (gh). No GitLab workflow is part of this repo.

Consequences

  • README and docs may teach cargo run without claiming QEMU boot is Verified.
  • New kernel work cites a roadmap milestone and updates the ledger when something is actually probed.
  • Role names stay ctos-*. Do not introduce aea-* hats or shop case-study content.

ADR-002 — PR author ≠ merger (two GitHub identities)

  • Status: Accepted
  • Date: 2026-09-08
  • Amended: 2026-09-12 (Track A historical exception + going-forward restatement)

Context

NFR-12 says implementer ≠ merge approver. On a solo GitHub login, a second Cursor agent cannot APPROVE a PR that login opened (GitHub self-APPROVE rule). Enabling “authors can approve their own PRs” would count same-login review as a gate. That is theater.

Café Fausse (artofdream/aea-interactive-design, .cursor/skills/pr-coordinator/SKILL.md) already probed this: cursor[bot] is the distinct second identity. Same principle applies here. This ADR does not import that repo’s restaurant/SRS content.

Cloud / Cursor agents on this repo often open PRs as artofdream. Commits commonly list cursoragent plus Co-authored-by: artofdream. That is still an owner-opened PR. A trailer does not change the GitHub author login and does not authorize a same-login merge.

Decision

Reuse that split on ctos:

  1. Do not enable GitHub author self-APPROVE. Do not add a required-approval ruleset until a second human exists.
  2. Author does not merge their own PR.
  3. Identities: artofdream (owner) and cursor[bot] (Cursor GitHub App, already used for cloud PRs). Do not install another App or mint a second personal account for this.
  4. Written MRC (ctos-mr-coordinator) is the review-in-the-room. Prefer COMMENT / REQUEST_CHANGES. Record who authored / who reviewed / who merges on the PR. Name those logins from gh pr view --json author,mergedBy (or the GitHub PR header), not from a daily-brief guess.
  5. If cursor[bot] REST APPROVE returns 403 (probed on Café Fausse #27; unprobed on ctos), do not block a valid cursor[bot] merge of an artofdream-authored PR after this-run green checks (when CI exists) and Bugbot resolved-or-declined. Missing Approve is not a reason to treat that merge as invalid.
  6. Owner PAT / artofdream still must not APPROVE or merge an artofdream-authored PR.

Going-forward (2026-09-12 restatement)

The GitHub PR author login is the identity that matters.

Who GitHub shows as authorWho writes the reviewWho merges
artofdreamNew MRC session (COMMENT)cursor[bot] after this-run green checks (when CI exists) and Bugbot resolved-or-declined. Owner must not Approve or merge.
cursor[bot] (cloud / this agent when GitHub actually attributes the PR to the App)Owner, optionally plus MRC COMMENTartofdream. The bot must not merge.

No self-merge. Owner-opened → bot merges. Bot-opened → owner merges.

Historical exception — Track A (A1–A9)

Named exception. Do not rewrite those PR pages. Sponsor directed this cleanup after Track A closed its first cuts.

  • When: 2026-09-11–2026-09-12
  • Scope: Track A first-cut PRs A1–A9: #49 (A1), #52 (A2), #53 (A3), #54 (A4), #55 (A5), #56 (A6), #57 (A7), #59 (A8), #60 (A9)
  • What gh showed (2026-09-12): each of those PRs had author=artofdream and mergedBy=artofdream. Commits were typically cursoragent with Co-authored-by: artofdream. That is same-login merge. It does not satisfy this ADR.
  • What this is not: not a claim those kernel or docs miles are invalid (their probes stay on their own ledger rows); not a rewrite of git history; not permission to repeat the pattern.
  • Earlier miles: many pre-Track-A merged PRs show the same GitHub author/mergedBy pair. This note names Track A as the sponsor-directed exception. It does not relitigate those older PR pages.

After this amendment, a same-login merge is a process miss, not an undocumented habit.

Consequences

  • Probe the GitHub author field before naming the merge hat. Chat “expected cursor[bot]” is not a probe.
  • #2: gh on 2026-09-12 shows author=artofdream, mergedBy=artofdream. The 2026-09-08 sentence that called it cursor[bot]-authored is withdrawn.
  • Smoke CI exists and has been probed on this repo (honesty ledger CI rows). “Green checks” on a new PR still need that PR’s run. Missing this-run green is not green.
  • cursor[bot] merge and App APPROVE on this repo stay Unknown until a distinct-identity merge is probed here. Do not copy Café Fausse #16/#27 as a ctos probe. Track A same-login merges are not that probe.

ADR-003 — Primary ISA is AArch64

  • Status: Accepted
  • Date: 2026-09-08

The sponsor brief named this decision “ADR-002.” On the harness tip, ADR-002 is already the author ≠ merger identity split. This file is ADR-003 so that accepted decision stays intact. Frozen FR/NFR IDs are unchanged; ISA-specific text is revised under this ADR.

Context

The sponsor host (arm64 workstation) is Windows ARM64 with Docker linux/arm64. Native AArch64 QEMU is the right guest. Keeping x86_64 as primary would mean a translated or foreign ISA on that machine.

The previous primary path (phil-opp VGA text at 0xb8000, bootloader 0.9, bootimage, qemu-system-x86_64) is superseded. It was a tutorial-era lift, not a host-ISA decision.

x86_64 may be noted later as a secondary target. It is not implemented in the change that lands this ADR.

Decision

  1. Primary ISA is AArch64 (arm64). The checked-in custom target is aarch64-ctos.json (aarch64-unknown-none style: os: none, panic abort, static relocation, soft-float so early boot does not require enabling SIMD/FP in CPACR_EL1).
  2. Boot path is QEMU -kernel of the kernel ELF on -machine virt (or virt,gic-version=3) with -cpu cortex-a57 or max. Entry is _start at a linker-script address in virt RAM. No bootloader 0.9 crate and no bootimage disk as the primary path.
  3. Console is the virt PL011 UART (MMIO 0x0900_0000) via print! / println!. Not VGA 0xb8000. earlycon remains an allowed later probe; this ADR does not claim it.
  4. Do not claim Raspberry Pi or other board support until a board-specific probe exists. virt is the supported guest.
  5. Delete the x86-only primary tree (x86_64-ctos.json, src/vga_buffer.rs) rather than leave a broken dual tree.

Consequences

  • New target JSON, linker script, and UART writer replace the VGA / bootloader 0.9 stack.
  • .cargo/config.toml default target is aarch64-ctos.json. json-target-spec stays for current nightly.
  • Roadmap M0/M1 probes are UART + qemu-system-aarch64, not a VGA dump at 0xb8e60.
  • FR-01, FR-02, FR-03, FR-05 (and other x86-specific wording) are revised in place. No new FR/NFR IDs.
  • Later exception/interrupt work is VBAR / GIC, not IDT / PIC / TSS. M3 / ADR-004 is the VBAR_EL1 + BRK path.
  • Honesty ledger rows for the x86 VGA path are historical. They do not verify the AArch64 path.

ADR-004 — EL1 VBAR and resumable BRK

  • Status: Accepted
  • Date: 2026-09-09

Context

FR-06 requires synchronous exceptions via VBAR_EL1 (at least a breakpoint / fault path). Roadmap M3 is that path. M4 (FR-07) is a dedicated fatal stack. M5 (FR-08) is GIC IRQs.

QEMU virt -kernel usually starts the guest at EL1. -machine virt,virtualization=on starts at EL2. Exceptions taken to the current EL use that EL’s VBAR_*. Staying at EL2 and programming only VBAR_EL2 would not satisfy FR-06.

Decision

  1. Live at EL1. exception::init calls ensure_el1: if CurrentEL is EL2, set HCR_EL2.RW, copy SP to SP_EL1, and ERET to EL1h (DAIF masked). If the EL is not 1 after that, print and park. Do not treat VBAR_EL2 as the primary table.
  2. One 2 KiB-aligned table at exception_vectors, written to VBAR_EL1. All sixteen AArch64 slots exist. Current EL / SP_EL0 / synchronous saves a frame and may ERET. Current EL / SP_EL0 / IRQ is live as of ADR-006. Every other slot parks (UART line + wfe, or semihosting fail under cargo test / force-fail).
  3. Context format (current-EL sync only): x0–x29, x30, ELR_EL1, SPSR_EL1, ESR_EL1. No SIMD/FP save (soft-float, ADR-003).
  4. BRK is resumable. ESR exception class 0x3C (AArch64 BRK) increments a counter, prints exception: sync BRK, adds 4 to ELR_EL1, and returns. Other synchronous exceptions are fatal for this milestone (print + park). That is not FR-07’s dedicated overflow stack.

Consequences

  • #[test_case] can execute brk #0 and continue. The hello kernel fires one BRK so scripts/qemu-smoke.sh can require the handler string on serial.
  • Lower-EL and FIQ/SError stubs are parks. Current-EL IRQ is M5 / ADR-006, not a syscall ABI.
  • A nested fault while println! holds the UART mutex can deadlock. ADR-005 (M4 / FR-07) splits SP_EL0 / SP_EL1, adds a fatal stack, and uses a raw UART write on that path.
  • This ADR does not claim Raspberry Pi, EL0, or a taken lower-EL exception.

ADR-005 — Dedicated exception and fatal stacks

  • Status: Accepted
  • Date: 2026-09-09

Context

FR-07 requires a fatal exception to use a dedicated stack so overflow does not silently lock the VM. Roadmap M4 is that path. ADR-004 already notes that a nested fault while println! holds the UART mutex can deadlock.

M3 ran the kernel and first-level current-EL exceptions on the same SP_EL1 (SPSel = 1). A nested sync exception would store another 272-byte frame on that stack. Without paging a downward overflow is not a hardware fault on QEMU virt — it smashes .bss / code with no serial evidence. M7 identity-maps RAM as one Normal block, so this is still true (no guard pages).

A data abort to an unused physical hole is QEMU-map-dependent. A nested BRK from the first-level handler is a real current-EL exception we already know how to take (FR-06).

Decision

  1. Thread stack on SP_EL0. After VBAR_EL1 is installed, copy the _start stack into SP_EL0 (MSR SP_EL0 is legal at EL1), write __exc_stack_top into SP while SPSel is still 1 (that is SP_EL1 — MSR SP_EL1 at EL1 is UNDEF), then msr spsel, #0. Normal kernel code uses the 64 KiB thread stack.
  2. Exception stack on SP_EL1. First-level current-EL exceptions (vector bank “Current EL, SP_EL0”) use the 16 KiB __exc_stack_* region automatically. The live sync slot moves from offset 0x200 (M3 / SP_ELx) to 0x000.
  3. Fatal stack before any nested store. Current-EL / SP_ELx slots (0x200–0x380) load SP from __fatal_stack_top (8 KiB) in asm, then call handle_fatal_exception. Do not push a frame on the exception stack that may already be exhausted.
  4. Raw UART on fatal / unhandled paths. Write the PL011 without spin::Mutex so a nest during println! still produces serial.
  5. Probe is nested BRK after a near-empty thread SP. The hello kernel fires a healthy-stack BRK (M3 string), then sets a flag, moves SP_EL0 to __stack_bottom + 64 (less than the 272-byte frame), and BRKs again. The first-level handler prints exception: sync BRK from SP_EL1, then BRKs while SPSel = 1. That nest must print exception: fatal nested and park. This is not an MMU stack-overflow fault and not a GIC path (M5 / FR-08).

Consequences

  • #[test_case] can still brk #0 and return. Tests do not set the nest flag.
  • scripts/qemu-smoke.sh requires the fatal marker in addition to hello + BRK. A fatal probe missed line is fail-closed.
  • Lower-EL and first-level FIQ/SError stubs still park. First-level IRQ is M5 / ADR-006. Nested IRQ/FIQ/SError still use the fatal stack. Taking a nested IRQ remains unprobed.
  • This ADR does not claim Raspberry Pi or x86. Unmapped linker-stack guard pages are ADR-014 (a later cut; the nested-BRK probe here still stands).

ADR-006 — GICv2 and the EL1 physical timer

  • Status: Accepted
  • Date: 2026-09-09

Context

FR-08 requires hardware interrupts via the virt GIC, with a timer tick observable. Roadmap M5 is that path. ADR-004 installed VBAR_EL1; ADR-005 split stacks. First-level IRQ was still a park.

QEMU -machine virt can expose GICv2 or GICv3 (gic-version=2|3|4|host|max). The current smoke line is -machine virt with no gic-version override. QEMU 8.2’s virt default is GICv2 (distributor at 0x0800_0000, CPU interface at 0x0801_0000). GICv3 uses a redistributor at 0x080A_0000 instead of that CPU interface.

The ARM generic timer is already in the core (CNTP_*_EL0 / CNTV_*_EL0). No extra virtio device is required.

Decision

  1. Program GICv2, matching default -machine virt. Do not add a GICv3 redistributor driver in this milestone. If a later QEMU default flips to v3, pin gic-version=2 on the smoke line or add a v3 driver in a new ADR.
  2. Use the non-secure EL1 physical timer (CNTP_CTL_EL0 / CNTP_TVAL_EL0 / CNTPCT_EL0) and PPI 30. If ensure_el1 drops from EL2, set CNTHCTL_EL2.EL1PCTEN|EL1PCEN and zero CNTVOFF_EL2 so EL1 can use the counter and physical timer.
  3. Take IRQs on the current-EL / SP_EL0 bank (vector offset 0x080). The kernel runs with SPSel = 0, so first-level IRQs use SP_EL1 (the M4 exception stack). Save the same GPR + ELR/SPSR/ESR frame as sync, then EOI.
  4. Keep DAIF.I masked except for an observe window. Hello and #[test_case] unmask, wait for one tick (or a CNTPCT timeout), then remask and stop the timer so M3 BRK / M4 nested fatal are not interrupted.
  5. Raw UART on the IRQ path. The handler must not take spin::Mutex (same reason as ADR-005). First tick prints timer: tick.

Consequences

  • scripts/qemu-smoke.sh requires timer: tick and rejects timer: tick missed, then still requires the M3/M4 strings.
  • #[test_case] can wait for a tick with IRQs unmasked and continue.
  • FIQ, SError, lower-EL, and unexpected IRQ IDs stay parks / raw markers. UART input is M6 / ADR-007.
  • This ADR does not claim Raspberry Pi, GICv3, virtualization=on as the primary path, or a taken FIQ.

ADR-007 — PL011 UART RX as virt input

  • Status: Accepted
  • Date: 2026-09-09

Context

FR-08 already covers hardware interrupts (timer via the virt GIC, M5 / ADR-006) and named “later input” as the rest of that ID. Roadmap M6 is that input path. The x86-era reading of FR-08 was a keyboard. Frozen IDs stay; the arm64 text is revised here.

Options on QEMU virt:

  1. PL011 UART RX at 0x0900_0000 (already the console).
  2. virtio-input / virtio-keyboard (virtio-mmio, virtqueues, DMA). No paging or heap yet (M7/M8). Too much device stack for one milestone.
  3. PL011 UARTCR.LBE loopback as a self-test. QEMU 8.2 (Ubuntu 24.04 / this repo’s GHA and cloud probe) does not implement LBE — hw/char/pl011.c still says the loopback bit is unimplemented. LBE landed in later QEMU. A loopback-only probe would be a false pass on newer hosts and a false fail on 8.2.

Host stdin on -serial stdio does reach pl011_receive → the RX FIFO. UARTLCR_H.FEN toggles reset that FIFO, so a byte sent before uart::init is lost. Inject after Hello World! (init already ran).

Decision

  1. M6 input is PL011 RX poll, not virtio-keyboard and not a UART RX IRQ. FR-08’s “via the virt GIC” remains the M5 timer. Input does not take a new FR ID.
  2. Prove a received byte with a host inject: scripts/qemu-serial-inject.py writes 0x41 ('A') to QEMU stdin after it sees Hello World!. The hello kernel polls UARTFR.RXFE / UARTDR (DAIF.I still masked after the timer window) and prints input: rx 0x41.
  3. Fail closed. Missing marker or input: rx missed fails scripts/qemu-smoke.sh. #[test_case] only asserts the FIFO is empty when cargo test does not inject — the character proof is the serial smoke.
  4. Do not claim Raspberry Pi UART, virtio-input, GICv3 UART SPI, or QEMU LBE.

Consequences

  • scripts/qemu-smoke.sh needs python3 to drive the inject (Dockerfile installs it).
  • cargo test / scripts/qemu-aarch64.sh stay inject-free so M2–M5 cases do not wait on stdin.
  • A later virtio-keyboard or UART-RX-via-GIC path needs a new ADR; it is not this milestone.

ADR-008 — Identity map and bump frame allocator

  • Status: Accepted
  • Date: 2026-09-09

Context

FR-09 requires the kernel to read a firmware/QEMU memory map (DTB when probed) and establish paging / virtual memory. Roadmap M7 is that path. Heap GlobalAlloc is M8; the scheduler is M9.

QEMU -machine virt memory (see hw/arm/virt.c):

RegionPhysical
flash / boot ROM0x0000_0000
GICv2 distributor / CPU interface0x0800_0000 / 0x0801_0000
PL011 UART0x0900_0000
RAM (VIRT_MEM)0x4000_0000, default 128 MiB
Kernel (-kernel TEXT_OFFSET 0x80000)0x4008_0000
DTB (typical -kernel placement)RAM base 0x4000_0000, below the kernel

_start currently overwrites x0 (the firmware DTB pointer) to set SP. A full FDT memory-node walk is extra surface for one milestone and is not required to prove MMU-on + allocate-frame on this machine.

Options:

  1. 1 GiB L1 identity blocks (39-bit VA, T0SZ=25, 4 KiB granule) plus a dedicated 4 KiB map window.
  2. Fine-grained 4 KiB identity of every used page (more tables, same probe).
  3. Higher-half kernel (breaks the current linker / VBAR / UART absolute addresses).
  4. Parse the DTB before enabling the MMU (correct long-term; not the M7 fail-closed proof).

Decision

  1. Enable the MMU at EL1 after exception::init (ensure_el1). Program MAIR_EL1 (Attr0 Device-nGnRnE, Attr1 Normal WB), TCR_EL1 (TTBR0 only, 40-bit IPS, inner-shareable WB), TTBR0_EL1, then set SCTLR_EL1.M|C|I|SA. Identity: VA == PA for everything the kernel already touches.
  2. L1 blocks, not a full 4 KiB identity. Entry 0 (0x0–1 GiB) is Device-nGnRnE + XN (UART, GIC, flash). Entry 1 (0x4000_0000–0x7FFF_FFFF) was a Normal WB executable 1 GiB RAM block in M7. ADR-012 supersedes that RAM block: L1 slot 1 is now an L2 table, only 128 MiB is mapped, and __kernel_end…RAM-end is PXN. The allocator still must not hand out frames past RAM_BASE + 128 MiB.
  3. Frame pool from linker + virt convention. __kernel_end (after image + stacks, 4 KiB-aligned) through 0x4000_0000 + 128 MiB. Bump cursor plus a 16-entry free stack. Do not allocate 0x4000_0000–0x4008_0000 (DTB / TEXT_OFFSET hole). Smoke pins -m 128M so the pool matches the VM.
  4. Map/unmap probe uses a third GiB window (0x8000_0000) via one L2 + L3 table, not by splitting the RAM block. Serial marker paging: ok. Do not load an unmapped VA (that would take an unhandled data abort).
  5. DTB parse is staged. M7 does not walk FDT. FR-09’s “DTB when probed” stays Unknown until a later ADR. The probed map is the virt table above plus __kernel_end.
  6. No GlobalAlloc. Frames are physical pages only.

Consequences

  • scripts/qemu-smoke.sh requires paging: ok and rejects paging: probe missed, then still requires M2–M6 strings.
  • #[test_case] can assert SCTLR_EL1.M, distinct aligned frames, and a map/unmap write-through.
  • Stack overflow was not a translation fault in M7 (RAM was one Normal block). Guard pages are ADR-014, not this milestone.
  • A later DTB walker or higher-half map needs a new ADR; it is not M8’s heap.
  • This ADR does not claim Raspberry Pi, GICv3, EL0 user maps, or ASID isolation. Heap NX is ADR-012, not M7.

ADR-009 — First-fit heap on identity-mapped frames

  • Status: Accepted
  • Date: 2026-09-09

Context

FR-10 requires a GlobalAlloc heap so alloc types (Box, Vec) work in the kernel. Roadmap M8 is that path. The scheduler is M9.

M7 already identity-maps virt RAM and hands out 4 KiB frames from __kernel_end through 128 MiB (ADR-008). Frames are physical pages, not a byte allocator.

Options:

  1. Bump-only heap (no dealloc). Box/Vec construct, but drop leaks. Reuse is unproved.
  2. linked_list_allocator crate. Common rust-osdev choice; extra dependency for one milestone.
  3. First-fit free list on a fixed contiguous frame run taken from the bump pool after the MMU is on.
  4. Grow-on-demand via the M7 map window (0x8000_0000). Extra paging surface; not needed to prove FR-10.
  5. Buddy / slab. Overkill for a Box/Vec smoke.

Decision

  1. Fixed 64 KiB heap (16 frames) via frame::alloc_contiguous after paging::init. Identity VA == PA. Do not map through the M7 probe window.
  2. First-fit + address-sorted coalesce in src/heap.rs. dealloc is real; the hello probe requires a freed Box pointer to be reused.
  3. #[global_allocator] + extern crate alloc. .cargo/config.toml build-std includes alloc. OOM returns a null pointer; #[alloc_error_handler] prints heap: oom on the raw UART and parks (or SYS_EXIT 1 under test). The handler must not format — formatting an OOM can allocate again.
  4. Serial marker heap: ok. Fail closed on heap: probe missed. #[test_case] covers pool bounds, Box reuse, and Vec growth.
  5. Not a growing heap, not userspace, not the scheduler. A later slab / grow-on-demand / higher-half heap needs a new ADR.

Consequences

  • scripts/qemu-smoke.sh requires heap: ok after paging: ok, then still requires M2–M7 strings.
  • The frame pool shrinks by 64 KiB at boot. M7 frame/map tests still have the rest of the 128 MiB guest.
  • This ADR does not claim Raspberry Pi or a production allocator. M9 task stacks on this heap are ADR-010.
  • ADR-038 keeps the first-fit algorithm and moves the pool VAs to the TTBR1 alias, then unmaps identity heap. The 64 KiB size is unchanged.

ADR-010 — Cooperative round-robin on EL1

  • Status: Accepted
  • Date: 2026-09-09

Context

FR-11 requires cooperative or simple round-robin task switching. Roadmap M9 is that path. The heap already exists (ADR-009); paging is an identity map (ADR-008). The kernel runs at EL1 with SPSel = 0 so SP is SP_EL0 (ADR-005).

Options:

  1. Async/await executor. Extra Future infrastructure for two markers.
  2. Timer preemption. Would steal the M5 CNTP path and needs IRQ-safe yield. Not required to prove FR-11.
  3. Cooperative yield that saves AAPCS64 callee-saved GPRs on a per-task stack and switches SP.
  4. SMP / EL0 processes. Out of scope (vision Out list).

The callee-saved layout is not obvious next to the exception frame in ADR-004 (that frame is x0–x30 + ELR/SPSR/ESR on SP_EL1). A yield must not touch the exception stacks.

Decision

  1. Cooperative only. Tasks call sched::yield_now(). No timer slice. DAIF.I stays masked during the hello/test probe (M5 still remasks after its own tick).
  2. AAPCS64 callee-saved switch in context_switch: save/restore x19–x28, x29, x30 on the task stack; store SP in the task slot; load the next SP. No SIMD/FP (ADR-003 soft-float). Not the exception context.
  3. Idle slot 0 is kernel_main / the test runner on the linker thread stack. Two workers get 8 KiB stacks from the M8 heap (Vec<u8>, not a stack-allocated [u8; N] that would overflow SP_EL0 during Box::new). Identity VA == PA.
  4. Round-robin among Ready slots. A worker that returns is Done (trampoline). Idle stays Ready so control returns to the caller of yield_now. Unlock the scheduler mutex before context_switch.
  5. Serial markers sched: task a, sched: task b, then sched: ok after both SPs land on distinct heap stacks. Fail closed on sched: probe missed.
  6. Not preemptive, not SMP, not EL0, not async. A later preemptive or process ADR is a new file.

Consequences

  • scripts/qemu-smoke.sh requires the three sched: strings after heap: ok, then still requires M2–M8 strings.
  • Two 8 KiB worker stacks come out of the 64 KiB heap. M8 Box/Vec probes drop before spawn.
  • Exception / fatal stacks are unchanged. A yield never runs from an IRQ handler.
  • This ADR does not claim Raspberry Pi, preemption, or userspace.

ADR-011 — Three pillars: antifragility, security, performance

  • Status: Accepted
  • Date: 2026-09-09

Frozen FR/NFR IDs are unchanged. This ADR revises NFR-05, NFR-07, and NFR-10 text in place. Do not mint FR-16+ or NFR-15+.

Context

Bring-up M0–M9 is on main (UART through a cooperative EL1 scheduler). The first-class practice so far was the honesty/harness loop (ADR-001): probes, ratchets, fail-closed CI, no self-merge.

Sponsor direction after M9: antifragility is not enough by itself. Security and performance are first-class pillars too — not afterthoughts bolted on when the kernel “feels done.”

Risks if we leave the NFR text as-is:

  • NFR-07 is still “Could / Later / learning-only.” That invites fake benches or silent “it’s fast enough” claims.
  • NFR-10 is mostly “no secrets in repo.” That is necessary and not a threat model.
  • NFR-05 already names ratchets, but it reads like a Next-stage SOP rather than a standing pillar.

This ADR does not clone florist / AEA vocabulary. The three pillars are ctos-native names for existing IDs.

Decision

  1. Three first-class pillars, same weight, different sensors:

    • Antifragility — keep the existing harness: honesty ledger, fail-closed scripts/qemu-smoke.sh, GHA smoke.yml, Docker smoke, #[test_case]. Repeated failures become ratchets, not README-only advice (antifragility.md).
    • Security — no “secure OS” / “hardened” / “W^X done” claim without a written threat model and a probe. Prefer minimize unsafe (already NFR-01). Prefer W^X / NX stacks and heap when paging can express it. No execute-from-writable heap by default once maps can mark NX. Least privilege on IRQ paths (no heap alloc, no sched::yield_now from an IRQ). Future EL0 isolation is Planned.
    • Performance — no fake benches and no invented latency numbers. Add honest measurable probes (CNTPCT delta around a known path; later timer-tick jitter). Optimize only after a probe shows a cost.
  2. Revise frozen NFR text in place (IDs stay NFR-05 / NFR-07 / NFR-10):

    • NFR-05: Must / Now. Ratchets stay the mechanism.
    • NFR-07: Should / Now. Probe language; not a hard latency budget.
    • NFR-10: Must / Now. Threat-model stub + claim gate, not only “no secrets.”
  3. W^X is Planned on this identity map. ADR-008 maps virt RAM as one executable 1 GiB L1 Normal block. The heap (ADR-009) and cooperative stacks (ADR-010) live in that block, so they are W+X today. Forbidding execute-from-heap needs an L2/L3 split (or a later higher-half map). Device MMIO is already XN. Do not claim heap NX because the L3 map window can set PXN — that window is not the heap.

  4. One small code ratchet may land with this docs PR. This change lands a baseline CNTPCT loop delta (serial perf: cntpct + #[test_case]). It does not split the L1 RAM block to fake W^X. Honesty over heroics.

  5. Post-M9 work is a pillars section, not a second bring-up stack on an open PR. Threat-model v1, W^X / NX heap+stacks, and further perf probes were listed as later loop units. The sponsor later asked for one coherent follow-up (ADR-012, ADR-013, irq-delta) rather than conflicting parallel branches.

Consequences

  • README, AGENTS.md, the roadmap, and the honesty ledger point at pillars.md.
  • A PR that says “secure” or “faster” without a ledger row is a coherence fail (NFR-06 / NFR-13).
  • The CNTPCT probe (when landed) is a baseline that the counter advances. It is not a published benchmark and not a comparison to other kernels.
  • PAN, full higher-half identity teardown, and preemption remain later. Standing EL0 and the ASID-tagged TLB mile are ADR-013. TTBR1 private page is ADR-016. EL1 high-VA fetch is ADR-017. The first identity-tear cut is ADR-018. The identity .text range tear is ADR-019. The live .text tear after a high-VA vtable rewrite is ADR-020. The SVC ABI mile is ADR-021. The libctos CRT mile is ADR-022. The guest ELF loader mile is ADR-023. Standing EL0 as normal mode is ADR-024 (app hosting Planned). Umbrella isolation still Planned. RO+NX text/data is ADR-015. Stack guard pages are ADR-014. EL0 first mile + user-TTBR0 read mile are also ADR-013.

ADR-012 — W^X: NX heap and cooperative stacks

  • Status: Accepted
  • Date: 2026-09-10

Context

NFR-10 and ADR-011 require a probe before any W^X claim. ADR-008 mapped virt RAM as one executable 1 GiB L1 Normal block. The heap (ADR-009) and cooperative stacks (ADR-010) live in that block, so they were W+X. Setting PXN only on the M7 map window (0x8000_0000) would have been a fake “NX heap” claim.

Device MMIO (L1 block 0) was already XN. The missing cut is: heap and heap-backed stacks NX, kernel text still executable.

SCTLR_EL1.WXN is not usable while kernel text pages are writable — it would make text NX too.

Decision

  1. Replace the RAM L1 block with an L2 table (still 39-bit / 4 KiB / TTBR0). Map only the 128 MiB guest (0x4000_0000…+128 MiB). The rest of that GiB stays invalid.
  2. 2 MiB L2 blocks where a block is entirely kernel-image or entirely frame-pool. One L3 table per mixed 2 MiB (originally the single block that straddled __kernel_end; ADR-015 splits on __data_start and KERNEL_TEXT, which can be two blocks).
  3. Executable: pages in [0x4008_0000, align_4k(__kernel_end)) (text, rodata, data, BSS, linker stacks). PXN+UXN: [__kernel_end, RAM end) (frame pool, heap, cooperative stacks) and the DTB hole below 0x4008_0000 when that hole is in the straddling L3. Device L1 stays XN. The M7 map window L3 pages are PXN (data-only).
  4. Do not claim the linker stacks NX. SP_EL0 / SP_EL1 / fatal stacks sit in the executable image. Overflow there is still not a translation fault (ADR-005).
  5. Fail-closed probe: walk PXN on kernel text (clear) and heap (set), then write RET on the heap, blr to it, and catch a current-EL permission instruction abort. Serial wx: nx heap + wx: ok. scripts/qemu-smoke.sh greps wx: ok and rejects wx: probe missed.
  6. NFR-10 text is revised in place (ID unchanged): heap + coop-stack NX is the scoped W^X claim, not “the kernel is W^X.”

Consequences

  • src/paging.rs grows L2_RAM and a kernel L3 pool (one table per mixed 2 MiB). Identity VA == PA is unchanged.
  • Linker-stack guard pages are ADR-014. The RO+NX text/data split and SCTLR.WXN are ADR-015.
  • EL0 isolation is ADR-013 (Planned). UXN is already set on RAM so a future EL0 cannot fetch kernel or heap by accident; that is not an EL0 probe.
  • This ADR does not claim a secure OS, Raspberry Pi, or side-channel resistance.

ADR-013 — EL0 isolation direction (first mile)

  • Status: Accepted (direction + first mile + user-TTBR0 read mile + ASID isolation mile + standing EL0; umbrella isolation still Planned)
  • Follow-on (2026-09-28, M6): B3 met: the sponsor accepted the ADR-091 revision-3 sentence and scope in writing (2026-09-28 16:41 CEST). ADR-091 is Accepted. “EL0 isolated” is Verified only as that sentence, with its limits. Text in this ADR that says the umbrella is Planned / non-claim is as of this ADR’s date.
  • Date: 2026-09-10
  • Updated: 2026-09-11 (standing EL0 dual-SVC; TTBR1 first cut is ADR-016; isolation still Planned)

Context

Vision and FR-11 leave userspace / EL0 as later work. NFR-10 names EL0 isolation as Planned. ADR-011 and the threat model (security.md) treat “malicious EL0” as a named adversary, not a standing userspace.

The first revision of this ADR recorded direction only (is_active() == false, lower-EL slots parked). The post-#18 deepen implements the smallest honest mile toward the closing probe. It does not invent a POSIX process model.

Decision

  1. First mile (this tree). Deliberate ERET to EL0t (DAIF masked) on one map-window page that is UXN-clear and PXN (EL1 must not fetch it). Two paths:
    • SVC #0 taken on the lower-EL AArch64 sync slot; handler prints el0: svc and ERETs back to EL1t.
    • BR X0 to a kernel .data bait; UXN on the identity image must take a lower-EL permission IABORT; handler prints el0: nx kernel and returns to EL1t. Serial el0: ok after the first-mile pair and the later miles on this path. The first mile itself does not install a standing context.
  2. Closing probe for “cannot execute kernel data.” The UXN / translation IABORT is that instruction-fetch probe. Mark that mile Verified when the serial / #[test_case] pass.
  3. User TTBR0 window (this tree, post-#19). A second L1 (L1_USER) is installed on ERET to EL0 (ASID=1 in TTBR0[63:48]). It maps kernel .text/.rodata and the exception stack so the lower-EL handler can restore kernel TTBR0 (kept in TPIDR_EL1), plus the shared map-window L2 for the trampoline. Coverage is every 2 MiB that holds those ranges, not only the first RAM 2 MiB. It omits .data / .bss / heap. An EL0 LDR from kernel .data is a lower-EL translation (or permission) DABORT (el0: no kernel read). Historically the EL0 trampoline still TLBI VMALLE1 because kernel .data leaves were global (nG=0); dropping that flush while identity .data stayed mapped would leak a cached .data translation into the user ASID. After ADR-037 / ADR-038, ADR-039 drops trampoline VMALLE1 (el0: no-vmalle1). Programming ASID=1 on that path is still not the isolation mile.
  4. ASID isolation mile (this tree). Two EL1 TTBR0 values (kernel L1 + ASID 1 vs a clone L1 + ASID 2) map one window VA to different PAs with nG=1, and a second VA only under ASID 1. The switch is MSR TTBR0 + ISB — no TLBI VMALLE1. Dual read must see the active ASID’s magic (asid: dual). The ASID-1-only VA must take a current-EL translation DABORT under ASID 2 (asid: conflict). Seeing ASID 1’s magic under ASID 2 is Failed (asid: stale). This is TLB ASID isolation on this virt guest, not “EL0 isolated.”
  5. Standing EL0 (this tree). A bounded user context on the user TTBR0: SVC #1 prints el0: standing and ERETs back to EL0; the user MOVZ must run; SVC #2 prints el0: restored and returns to EL1t. is_active() is true only between install and teardown (saved user PC / SP / TTBR0). A trampoline-only flag is not this mile. Lower-EL IRQ / FIQ / SError still park (not exercised). Not a POSIX process. Not preemption.
  6. Isolation remains Planned. Missing: PAN enable (typically unimplemented on -cpu cortex-a57 / ARMv8.0 — ADR-026 reads ID_AA64MMFR1_EL1.PAN and records absent; do not silent -cpu switch), full TTBR1 / higher-half identity teardown (ADR-016 is the private-page cut; ADR-017 is the EL1-fetch cut; ADR-018 is the first identity-tear cut; ADR-019 jumps the post-MMU continuation high and unmaps a 16 KiB dedicated text range; ADR-020 rewrites rustc vtables and unmaps live identity .text; ADR-025 unmaps identity .rodata; ADR-037 / ADR-038 unmap .data/stacks/heap — leftover identity frames / _start stay), EL0 entry without a full TLBI (superseded by ADR-039). A documented SVC ABI is ADR-021 (not isolation, not app hosting). The libctos CRT is ADR-022 (not isolation, not app hosting). Do not move FR IDs. Do not mint FR-16+. Do not claim PAN.
  7. Lower-EL slots. AArch64 sync is live for SVC / IABORT / DABORT (including standing dual-SVC, the public ABI numbers 16–18, and the TTBR1 private-page load). Lower-EL IRQ / FIQ / SError and all AArch32 slots stay parks (ADR-004).
  8. Honesty. Docs and PRs may say “EL0 entered and returned,” “EL0 cannot execute kernel data,” “EL0 cannot read kernel .data,” “user TTBR0 omits kernel data,” “standing EL0 context,” or “ASID isolation” when those probes pass. The TTBR1 private-page mile is ADR-016. The EL1 high-VA fetch mile is ADR-017. The identity-tear first cut is ADR-018. The identity .text range tear is ADR-019. The live .text tear after a high-VA vtable rewrite is ADR-020. The SVC ABI mile is ADR-021. The libctos CRT mile is ADR-022. The guest ELF loader mile is ADR-023. Standing EL0 as normal mode (loaded task until exit, fail-closed restore) is ADR-024. The identity .rodata tear is ADR-025. The PAN ID-field probe is ADR-026. They may not say “EL0 works,” “userspace,” “EL0 isolated,” “app hosting,” or “the kernel moved.”

Consequences

  • el0.md splits first-mile vs isolation rows.
  • Preemption and SMP remain separate later ADRs (ADR-010).
  • This ADR does not claim Raspberry Pi or a POSIX process model.

ADR-014 — Unmapped guard pages under linker stacks

  • Status: Accepted
  • Date: 2026-09-10

Context

ADR-012 made the heap and heap-backed cooperative stacks PXN. It explicitly left SP_EL0 / SP_EL1 / fatal stacks in the executable kernel image and named stack guard pages as a later ADR. ADR-005 still mitigates overflow with a dedicated fatal stack and a nested-BRK probe — that is not a translation fault.

A downward overflow of a linker stack today smashes the previous image bytes (.bss or the stack above). QEMU virt will not notice.

Options:

  1. Unmapped 4 KiB guard page(s) below each linker stack. Overflow becomes a current-EL data abort (translation). Observable serial + #[test_case].
  2. PXN on stack pages after splitting them from text. Stops execute-from-stack; overflow still smashes mapped data unless we also unmap something.
  3. SCTLR_EL1.WXN — still unusable while kernel text pages are writable.

Decision

  1. Option 1. Insert a 4 KiB linker hole below the thread, exception, and fatal stacks (__stack_guard, __exc_stack_guard, __fatal_stack_guard). After fill_ram_wx, split any 2 MiB L2 block that contains a guard into L3 and clear that L3 slot (invalid). Identity VA == PA is unchanged.
  2. Fail-closed probe: arm a current-EL translation data-abort catch, store to the thread-stack guard, resume at LR. Serial guard: fault + guard: ok. scripts/qemu-smoke.sh greps guard: ok and rejects guard: probe missed.
  3. Guard pages are holes, not NX stacks. This ADR does not by itself make live stack pages NX. ADR-015 maps those live pages RW+NX with .data. Do not claim “the kernel is W^X” from guard holes alone.
  4. NFR-10 text is revised in place (ID unchanged): mention the guard-page overflow cut. Do not mint NFR-15+.
  5. The ADR-005 nested-BRK fatal probe stays. It does not write below __stack_bottom.

Consequences

  • src/paging.rs may add one or two extra L3 tables to split an executable L2 block that is not the __kernel_end straddle.
  • Overflow of a heap-backed cooperative stack is still not this probe (those stacks live in PXN frame-pool RAM with no guard holes).
  • This ADR does not claim a secure OS, Raspberry Pi, or ASAN-quality stack checking.

ADR-015 — RO+NX text / data split

  • Status: Accepted
  • Date: 2026-09-11

Context

ADR-012 made the heap and heap-backed cooperative stacks PXN. ADR-014 punched unmapped holes under the linker stacks. Both left .text / .rodata / .data / .bss / live linker-stack pages in one executable, writable image mapping. SCTLR_EL1.WXN was unusable while text was writable.

NFR-10 still forbids saying “the kernel is W^X” without a scoped probe. This ADR is the next cut: split RO+X from RW+NX.

Decision

  1. Page-align __data_start in linker.ld so .text/.rodata never share a 4 KiB page with .data.
  2. Identity flags (still 39-bit / 4 KiB / TTBR0):
    • [KERNEL_TEXT, __data_start) — RO+X (AP[2]=1, PXN clear, UXN set)
    • [__data_start, RAM end) — RW+NX (including .data, .bss, live linker stacks, frame pool / heap)
    • Device L1 stays XN. Guard holes stay invalid (ADR-014).
  3. SCTLR_EL1.WXN on once text is RO. Writable pages are treated as XN even if a descriptor forgets PXN.
  4. Fail-closed probe: execute-from-.data (blr to a RET bait) is a current-EL permission IABORT (ro: nx data). Store to RO text is a current-EL permission DABORT (ro: write fault). Serial ro: ok. scripts/qemu-smoke.sh greps those strings and rejects ro: probe missed. #[test_case] covers flags + both faults.
  5. NFR-10 text is revised in place (ID unchanged): mention the RO+NX image cut. Do not mint NFR-15+.
  6. One L3 per mixed 2 MiB (2026-09-11 follow-up). ADR-012 assumed a single L3 for the 2 MiB that straddles __kernel_end. After this ADR the mixed block is the one that contains __data_start and the first RAM 2 MiB that contains KERNEL_TEXT (0x4008_0000). Those are different blocks once .text/.rodata (or a linker ratchet) crosses 0x4020_0000. Reusing one L3_RAM overwrites the kernel-text walk. The kernel L3 pool hands out a fresh table per straddle / guard split. The user TTBR0 maps every 2 MiB that holds text or the exception stack, not only the first RAM block.
  7. Post-MMU .bss publishes (same follow-up). USER_MAP_OK and frame::init run after SCTLR.C. A pre-MMU store to .bss can be invisible to a later cached read (boot-delta on the PR #20 test image; arm64 workstation Docker hello on b2bbb99 lost the frame pool / user-map ready flag while guard/ro still passed). Do not pin an old nightly for this.

Honesty — is this “the kernel is W^X”?

On this QEMU virt guest, the identity image plus heap is W^X: text/rodata are RO+X, data/bss/live linker stacks/heap are RW+NX, WXN is on. That is not:

  • a “secure OS” / “hardened” claim
  • a promise that every future mapping (DMA, new windows) stays W^X
  • Raspberry Pi or a second ISA
  • a reason to drop the heap / guard probes

Say “identity image is W^X on this virt guest (ADR-015 probe)” only when the serial / tests pass. Do not say “the kernel is W^X” as a product sentence.

Consequences

  • src/paging.rs sets AP[2] on text and PXN on data/stacks. src/ro.rs owns the two faults.
  • Live linker stacks are NX here. If a later change must execute from a linker stack, this ADR is the one to revisit.
  • EL0 user TTBR0 (ADR-013 mile) still maps kernel text so the lower-EL handler can restore kernel TTBR0; it omits .data. Coverage follows the image across L2 blocks. That is isolation-adjacent, not this ADR.
  • linker.ld parks __data_start at 0x4020_1000 so GHA / Docker / this cloud always exercise two kernel L3s. File presence of that address is not a boot probe.
  • This ADR does not claim PAN, ASID isolation, or a higher-half kernel.

ADR-016 — TTBR1 kernel-private page (first cut)

  • Status: Accepted (first cut; identity teardown Planned)
  • Date: 2026-09-11

Context

ADR-013 left TTBR1 / higher-half as a missing isolation mile. The kernel still runs from the QEMU virt identity map at 0x4008_0000 (TTBR0, T0SZ=25). TCR.EPD1 was set, so TTBR1 walks were disabled. EL0 and EL1 share the same translation regime: a high mapping is not automatically invisible to EL0 — access is an AP / UXN question.

A full higher-half relocate (kernel fetch, VBAR_EL1, stacks, UART MMIO, then tearing down the identity image) is too large for one honest PR. This ADR is the smallest Verified cut: enable TTBR1 and prove one kernel-private high page.

Decision

  1. Enable TTBR1 walks. Clear TCR.EPD1. Set T1SZ=25 (same 39-bit window as T0SZ) with WBWA / inner-shareable attrs on TTBR1. ASID still comes from TTBR0 (TCR.A1 = 0).
  2. High-half base. The TTBR1 region is 0xFFFF_FF80_0000_0000…0xFFFF_FFFF_FFFF_FFFF. One private page lives at TTBR1_PRIV = 0xFFFF_FF80_0000_0000 (src/paging.rs).
  3. EL1-only leaf. The page is Normal, RW, PXN, UXN, AP[2:1]=00 (EL1 RW, EL0 no data access). Backing store is a frame-pool page, also identity-mapped for EL1. User TTBR0 does not describe this VA.
  4. Fail-closed probe. EL1 store/load via the high VA prints ttbr1: el1. An EL0 LDR of TTBR1_PRIV is a lower-EL permission (or translation) DABORT (ttbr1: no el0). Serial ttbr1: ok. scripts/qemu-smoke.sh greps those strings and rejects ttbr1: probe missed / ttbr1: leaked. #[test_case] covers the same path.
  5. Identity map stays. _start, vectors, linker stacks, PL011 0x0900_0000, and the multi-L3 / layout ratchets from #21 keep using TTBR0 identity VAs. Full higher-half + identity teardown is Planned.
  6. NFR-10 text is revised in place (ID unchanged). Do not mint NFR-15+. Do not claim “the kernel runs in the high half” or “EL0 isolated.”

Honesty

Say “TTBR1 maps a kernel-private page EL0 cannot access” only when the serial / tests pass. Do not say:

  • the kernel has moved to a high VA
  • identity mappings were torn down
  • “secure OS” / “hardened” / “EL0 isolated”
  • PAN (still unclaimed on -cpu cortex-a57)

Consequences

  • paging::init programs TTBR1_EL1 and leaves EPD1 clear. src/ttbr1.rs owns the probe.
  • Standing EL0 (ADR-013) is a separate mile. Umbrella isolation stays Planned while PAN is unclaimed.
  • ADR-017 aliases identity RAM in TTBR1 and fetches a real EL1 path (plus high VBAR_EL1). ADR-018 splits those tables and unmaps one identity text page. ADR-019 unmaps a 16 KiB dedicated text range after a high-VA jump. ADR-020 unmaps live identity .text after a vtable rewrite. Full identity teardown (.rodata/.data/heap) is still Planned.

ADR-017 — EL1 fetch from the TTBR1 RAM alias

  • Status: Accepted (exec mile; first identity-tear cut is ADR-018)
  • Date: 2026-09-11

Context

ADR-016 enabled TTBR1 walks and proved one EL1-only data page at TTBR1_PRIV. The kernel still fetched instructions from the QEMU virt identity map at 0x4008_0000. Full higher-half relocate (every pointer, every fn item, then tearing down identity) is still too large for one honest PR: rustc emits link-time identity addresses (relocation-model: static), and -kernel loads _start at TEXT_OFFSET.

This ADR is the largest Verified cut that still keeps the boot stub: alias identity RAM in TTBR1 and run a real EL1 code path (instruction fetch, not a private store) at high VA.

Decision

  1. RAM alias. L1_HIGH[1] points at the same L2_RAM used by identity TTBR0. High VA is identity + TTBR1_BASE (0x4008_0000 → 0xFFFF_FF80_4008_0000). Permissions match the identity image (text RO+X, data/stacks/heap RW+NX, guards unmapped).
  2. Shared tables (superseded for the tear mile). This ADR reused L2_RAM. ADR-018 clones RAM tables so one identity text page can be unmapped without dropping the high twin. Full identity teardown is still Planned.
  3. EL1 exec probe. ttbr1_high_el1_path is invoked through its high VA (BLR / fn pointer). It captures PC with ADR, prints ttbr1: el1 exec from that path, and returns the PC. The caller checks PC is in the TTBR1 window and near the high entry. UART MMIO stays the absolute identity 0x0900_0000.
  4. VBAR_EL1 high. After the MMU is on, VBAR_EL1 is programmed to the high alias of exception_vectors (ttbr1: vbar). Later BRK / IRQ / EL0 sync fetch the table via TTBR1. Identity exception::init still installs the link address first (pre-MMU).
  5. Identity boot stub stays. _start, QEMU -kernel load, and link-time fn items remain at 0x4008_0000. Most EL1 data access still uses identity VAs. Do not say the kernel moved.
  6. Still Planned. Full identity teardown (unmap all low .text/.data/heap after a complete high-VA jump); PAN on -cpu cortex-a57; umbrella EL0 isolation. ADR-018 is the first cut (one torn text page). User TTBR0 still maps most kernel text so a missed high-VBAR path can fetch. Do not change default -cpu.
  7. NFR-10 text is revised in place (ID unchanged). Do not mint NFR-15+.
  8. ADRP is not a PA. After VBAR is high, handler code that does addr_of!(table) / user_ttbr0() must mask to the 39-bit identity VA before programming TTBR0 or comparing FAR_EL1. First attempts Failed: guard FAR compared to a high __stack_guard, then standing stay_at_el0 programmed a high L1_USER as TTBR0. paging::identity_pa / linker_sym are the ratchet.

Honesty

Say “EL1 fetched a real path from a TTBR1 high VA” or “VBAR lives at the high alias” only when the serial / tests pass. Do not say:

  • the kernel has moved to the high half
  • identity mappings were torn down
  • “secure OS” / “hardened” / “EL0 isolated”
  • PAN (still unclaimed on -cpu cortex-a57)

Consequences

  • paging::init installs the RAM alias and relocates VBAR. src/ttbr1.rs owns both the private-page probe and the exec probe.
  • ADR-016 remains the private-page first cut. This ADR is the exec mile.
  • ADR-018 splits TTBR1 RAM tables and unmaps one identity text page. ADR-019 jumps the post-MMU continuation to high VA and unmaps a 16 KiB dedicated text range. ADR-020 unmaps live identity .text after a vtable rewrite. Unmapping .rodata/.data/heap is still later.

ADR-018 — Identity teardown first cut (split tables + one torn text page)

  • Status: Accepted (partial tear; full identity teardown still Planned)
  • Date: 2026-09-11

Context

ADR-017 proved EL1 can fetch a real path from the TTBR1 RAM alias and moved VBAR_EL1 there. The alias reused identity L2_RAM, so unmapping a low page also dropped the high twin. _start / QEMU -kernel still load at 0x4008_0000. rustc emits link-time identity addresses (relocation-model: static), so a complete high-VA jump of kernel_main plus unmap of all identity .text/.data/heap is still too large for one honest PR.

This ADR is the largest Verified cut that still boots on virt: split the RAM tables, then unmap one defined identity text page that EL1 no longer needs for fetch.

Decision

  1. Split RAM tables. After identity W^X + stack guards, clone L2_RAM (and every L3 it points at) into L2_HIGH_RAM / L3_HIGH_RAM. L1_HIGH[1] points at the clone. L2 block descriptors are copied; L3 tables are duplicated. Unmapping an identity page no longer unmaps the high twin.
  2. Dedicated tear page. The linker keeps a 4 KiB ident_tear section (__ident_tear_start … __ident_tear_end) in the RO+X image, after .rodata and before __data_start. It is not the 0x4008_0000 _start page.
  3. Unmap after MMU + high VBAR. paging::init turns the MMU on, programs the high VBAR, then clears the identity and user-TTBR0 L3 slots for that page and TLBI VAAE1s the low VA. The high twin stays PXN-clear.
  4. Fail-closed probe. Serial ident: split (clone + torn low / live high walks). EL1 BLR of the low VA is a current-EL translation IABORT (ident: fault). EL1 BLR of the high twin still runs ident_tear_el1_path (ident: high). EL0 LDR of the low VA is a lower-EL translation DABORT (ident: no el0). Serial ident: ok. scripts/qemu-smoke.sh greps those strings and rejects ident: probe missed / ident: leaked.
  5. Identity boot stub stays. _start, QEMU -kernel load, remaining identity .text/.data/heap, PL011 0x0900_0000, and later probes keep using TTBR0 identity VAs. Do not say the kernel moved.
  6. Still Planned. Full identity teardown (unmap all low .text/.data/heap after a complete high-VA jump); PAN on -cpu cortex-a57; umbrella EL0 isolation. Do not change default -cpu.
  7. NFR-10 text is revised in place (ID unchanged). Do not mint NFR-15+.

Honesty

Say “TTBR1 RAM tables are independent of identity” or “one identity text page was unmapped while EL1 still fetched the high twin” only when the serial / tests pass. Do not say:

  • the kernel has moved to the high half
  • identity mappings were fully torn down
  • “secure OS” / “hardened” / “EL0 isolated”
  • PAN (still unclaimed on -cpu cortex-a57)

Consequences

  • paging::init clones RAM tables and unmaps __ident_tear_*. src/teardown.rs owns the serial probe.
  • ADR-017 remains the exec mile. This ADR is the first identity-teardown cut.
  • ADR-019 jumps the post-MMU continuation to high VA and expands the dedicated tear to 16 KiB. ADR-020 rewrites rustc vtables and unmaps live identity .text. .rodata/.data/heap stay.

ADR-019 — High-VA continuation + dedicated identity text range

  • Status: Accepted (partial tear; live .text / .rodata / .data / heap still identity-mapped)
  • Date: 2026-09-11

Context

ADR-018 cloned TTBR1 RAM tables and unmapped one dedicated identity text page. rustc still emits link-time identity addresses (relocation-model: static). _start / QEMU -kernel still load at 0x4008_0000.

The preferred next cut was to unmap contiguous live identity .text after the boot stub once EL1 fetched from the high alias. A first attempt on this cloud VM Failed: after unmapping [0x4008_1000, __text_end), the first writeln! / println! took an unhandled current-EL sync. core::fmt::write takes &mut dyn Write; those vtable methods are identity fn pointers. Unmapping live .text is therefore blocked until fmt / dyn dispatch is proven high-only.

This ADR is the largest Verified cut that still boots: jump the post-MMU continuation to its TTBR1 alias, then unmap a 16 KiB dedicated identity text range (four pages), not one probe page.

Decision

  1. High-VA continuation. After paging::init (MMU on, high VBAR_EL1, ADR-018 first tear page), kernel_main BRs to the TTBR1 alias of kernel_main_high. Serial ident: jump. Direct BL stays in the high window (PC-relative). UART MMIO stays the absolute identity 0x0900_0000. Stacks / heap / .data stay identity VAs.
  2. Boot stub stays mapped. The 0x4008_0000 page (_start + exception_vectors) stays identity-mapped. Live .text after that page also stays — rustc fmt / dyn still needs those identity fn pointers.
  3. Dedicated 16 KiB tear range. Linker __ident_tear_start … __ident_tear_end is four pages: page 0 holds ident_tear_el1_path (ADR-018), page 1 holds ident_range_el1_path, pages 2–3 are aligned pad. After the high jump, unmap the whole range from kernel and user TTBR0 and TLBI VAAE1 each low VA. High twins stay PXN-clear.
  4. Keep live .text / .rodata / .data identity-mapped. Do not yank them here. A botched full .text yank is worse than a smaller Verified slice.
  5. Identity fn pointers used after the jump go high where we control them (scheduler trampoline / task entries; custom test runner dyn Testable method). That is preparation, not a claim that rustc fmt is high-only.
  6. Fail-closed probe. Serial ident: jump. ident: range lo=… hi=… pages=N with N >= 4. Existing ident: split / fault / high / no el0 / ok stay. ident: text is EL1 fetch of the second torn page via the high twin. scripts/qemu-smoke.sh greps those strings and rejects ident: probe missed / ident: leaked / ident: range missed.
  7. Identity boot stub stays. _start, QEMU -kernel load, live identity .text/.rodata/.data/heap, PL011 0x0900_0000. Do not say the kernel moved.
  8. Still Planned. Unmap live identity .text after fmt / dyn calls are proven high-only; then .rodata/.data/heap; PAN on -cpu cortex-a57; umbrella EL0 isolation. Do not change default -cpu.
  9. NFR-10 text is revised in place (ID unchanged). Do not mint NFR-15+.

Honesty

Say “the post-MMU continuation ran at a high VA” or “a 16 KiB dedicated identity text range was unmapped while EL1 still fetched the high twins” only when the serial / tests pass. Do not say:

  • the kernel has moved to the high half
  • identity .text after _start was fully torn down
  • identity mappings were fully torn down
  • “secure OS” / “hardened” / “EL0 isolated”
  • PAN (still unclaimed on -cpu cortex-a57)

Consequences

  • paging::jump_high / tear_identity_text_range own the cut. src/teardown.rs extends the serial probe.
  • ADR-018 remains the split-tables + first dedicated page cut.
  • ADR-020 rewrites rustc vtables to high aliases and unmaps live identity .text after the boot stub. .rodata / .data / heap stay.

ADR-020 — High-VA fn-pointer rewrite + live identity .text tear

  • Status: Accepted (live .text after the boot stub; .rodata / .data / heap still identity-mapped)
  • Date: 2026-09-11

Context

ADR-019 jumps the post-MMU continuation to its TTBR1 alias and unmaps a 16 KiB dedicated identity text range. A first attempt to unmap live .text after _start Failed: the first writeln! / println! took an unhandled current-EL sync.

This ADR investigates that failure and takes the largest honest Verified cut toward tearing live identity .text.

Probed root cause (this cloud VM, main 24d94e6 hello ELF, then this branch):

  • rustc dyn Write vtables live in .rodata. The Pl011 table is drop=0 / size=8 / align=8 / write_str / write_char / write_fmt with identity method addresses (write_char at 0x4008_1084, immediately after the vectors page).
  • The hello .rodata held a handful of other identity .text words (Debug::fmt, PadAdapter as Write). .data had none. .text had no 8-byte identity literal pools.
  • AArch64 codegen used ADRP (PC-relative), not MOVZ/MOVK of 0x4008. After the high-VA jump, new fn() values are already high; tables linked as R_AARCH64_ABS64 stay identity until rewritten. The ELF is ET_EXEC / relocation-model: static — no leftover reloc records to apply.
  • .text and .rodata shared page 0x400a4000 on the ADR-019 layout (vtable at 0x400a4338). Unmapping “all of .text” without a page split would also drop the vtable.

relocation-model: pic was considered and not taken: static + a post-jump patcher matches the existing -kernel / 0x4008_0000 contract and does not need a dynamic linker.

Decision

  1. Page-split .text / .rodata. Linker __text_end / __rodata_start are 4 KiB aligned so live .text and fmt tables do not share a page.
  2. High-VA fn-pointer rewrite. After ident: jump, walk .rodata (through __ident_tear_start) and .data. Every 8-byte word that is an identity address in .text or __ident_tear_* is rewritten to its TTBR1 alias. RO pages are made writable only for that store, then restored (SCTLR.WXN makes a writable page XN — they are not fetched). Serial ident: reloc n=N with N >= 1. A Pl011 as dyn Write fat-pointer read must see a high write_str.
  3. Live identity .text tear. Unmap [0x4008_1000, __text_end) from kernel and user TTBR0 (TLBI VAAE1 each low VA). High twins stay PXN-clear. The 0x4008_0000 page (_start + exception_vectors) stays identity-mapped. Serial ident: live lo=… hi=… pages=N with N >= 8.
  4. Keep .rodata / .data / heap identity-mapped. String literals and rewritten vtables still live there. Do not yank them here.
  5. Keep _start / QEMU -kernel at 0x4008_0000. Do not change default -cpu. Do not claim PAN.
  6. Fail-closed probe. Existing ident: jump / range / split / fault / high / text / no el0 / ok stay. New ident: reloc / ident: live. scripts/qemu-smoke.sh greps those strings and rejects ident: reloc missed / ident: live missed. #[test_case] covers the rewrite and the live unmap.
  7. Still Planned (this ADR). Unmap identity .rodata / .data / heap after those accesses are proven high-only; PAN on -cpu cortex-a57; umbrella EL0 isolation. ADR-025 takes the .rodata cut. ADR-026 probes the PAN ID field (enable stays Planned).
  8. NFR-10 text is revised in place (ID unchanged). Do not mint NFR-15+.

Honesty

Say “rustc vtable / fn-pointer words were rewritten to high aliases” or “live identity .text after the boot stub was unmapped while println! still ran” only when the serial / tests pass. Do not say:

  • the kernel has moved to the high half
  • identity mappings were fully torn down (.rodata / .data / heap stay)
  • “secure OS” / “hardened” / “EL0 isolated”
  • PAN (still unclaimed on -cpu cortex-a57)

Consequences

  • paging::rewrite_identity_fn_ptrs / paging::tear_live_identity_text own the cut. ADR-019 remains the high-VA jump + dedicated 16 KiB range.
  • A later ADR may unmap .rodata once string / table accesses are high-only, then .data / heap. That work is not this cut.

ADR-021 — Minimal EL0 SVC syscall ABI

  • Status: Accepted (ABI mile Verified when the serial / tests pass; Track A / app hosting still Planned)
  • Date: 2026-09-11

Context

Track A (issue #31) wants freestanding apps to call the kernel. Issue #32 is the first mile: a documented, probed SVC contract.

ADR-013 already takes lower-EL AArch64 SVC for probe immediates #0 (first-mile return), #1 (standing announce), and #2 (standing restore). Those numbers stay reserved. This ADR adds a tiny public set so a later CRT / libctos (A2) has numbers to call. It does not load ELF, host apps, or claim Linux/POSIX.

Options:

  1. Linux AArch64 (svc #0, x8 = number). Collides with the first-mile SVC #0 probe and invites a Linux-ABI claim.
  2. SVC immediate is the number, public set starts at 16. Leaves 0–2 for ADR-013 probes. Arguments in x0–x2, return in x0.
  3. Wait for a loader. Would block Track A on A3.

Decision

  1. Calling convention. AArch64 SVC #<n> where n is the syscall number. Arguments in x0, x1, x2. Return value in x0. DAIF stays masked on the existing EL0 path. Not Linux. Not POSIX.

  2. Reserved 0–2. Probe-only (ADR-013). Not part of the public ABI. A later CRT must not issue them.

  3. Public numbers (this tree).

    NumberNameArgsEffectReturn
    16exitx0 = status (recorded; not a process model)End the EL0 trip; ERET to the EL1 caller(does not return to EL0)
    17uart_writex0 = user pointer, x1 = lengthCopy up to 64 bytes from a user-mapped and kernel-mapped range; write to PL011 (\n → \r\n, same as the kernel console)bytes written, or 0 if rejected
    18yieldnoneRecord the call; ERET back to EL0 (next insn)0
  4. yield is dispatch-only. It does not call sched::yield_now (ADR-010: that switch is EL1-only and must not run from an exception frame). A later ADR may connect the two.

  5. uart_write is fail-closed. Length 0 or > 64, overflow, a non-canonical or TTBR1 alias, or a range that is not user-mapped or not kernel-mapped returns 0 and writes nothing. Kernel .data is rejected. Table walks mask to 39 bits — the pointer itself must already be a TTBR0 identity VA, or copy_user would load a tagged address. This is not a VFS.

  6. Unknown immediates stay unhandled (park). Do not silently succeed.

  7. Fail-closed probe. Hello serial svc: yield + user buffer svc: user-hi + svc: uart + svc: exit + svc: ok. scripts/qemu-smoke.sh greps those and rejects svc: probe missed. #[test_case] covers the success trip, a kernel-.data reject, and a TTBR1-alias reject. Existing el0: markers stay.

  8. Honesty. Say “EL0 issued the documented SVC ABI and the kernel performed the documented effect” only when the serial / tests pass. Do not say: app hosting is done, Linux ABI, POSIX, userspace, “EL0 isolated,” or “secure OS.” Standing-as-normal is ADR-024. Thin VFS numbers 19–23 are ADR-027. Track A A7–A9 stay Planned.

  9. NFR-10 text is revised in place (ID unchanged) to name this ABI mile. Do not mint FR-16+ or NFR-15+.

Consequences

  • Code: src/syscall.rs. Dispatch is hooked from handle_sync_lower_el after the ADR-013 probe immediates.
  • Docs: syscall.md, el0.md. Roadmap cites #32 / Track A #31.
  • A2 (libctos) wraps these three numbers (ADR-022). A3 (loader) is not that PR.

ADR-022 — Freestanding CRT / libctos

  • Status: Accepted (CRT mile Verified when the serial / tests pass; Track A / app hosting still Planned)
  • Date: 2026-09-12

Context

Track A (issue #31) wants a freestanding program to call the A1 ABI without glibc or POSIX. Issue #33 is this mile: a documented, probed CRT + libctos that wraps ADR-021 numbers 16–18.

A1 already proved the kernel performs exit / uart_write / yield for a hand-encoded trampoline. That is not a library a program can link. A3 (ELF loader) must not start until something exists to link against.

Options:

  1. Wait for a loader. Would block A2 on A3.
  2. C/asm static lib only. Fine later; this tree is Rust-first and already uses aarch64-ctos.
  3. no_std crate + asm stubs + a hello payload copied onto today’s standing EL0 page. Proves link + run without a guest ELF loader.

Decision

  1. libctos crate (libctos/). no_std wrappers exit / uart_write / yield_now over asm stubs (sys.S: ctos_exit / ctos_uart_write / ctos_yield). SVC immediates are 16 / 17 / 18. Reserved 0–2 stay ADR-013 probes — the CRT must not issue them.
  2. CRT is _start in libctos/src/crt0.S: kernel ERET already set SP_EL0; _start calls main, then ctos_exit. No argv, no environ, no libc init. Not glibc/musl crt0.
  3. Hello payload (user/hello-libctos). A no_std / no_main program links libctos, prints libctos: hi via uart_write, yields, prints libctos: ok, returns 0. Linked at paging::EL0_PAGE (0x80002000) with user/hello-libctos/linker.ld.
  4. Not a guest loader. build.rs compiles that program on the host and extracts PT_LOAD bytes at/after EL0_PAGE (size metadata + publish). The kernel copies a flatten of FAT /hello onto the standing EL0 page and ERETs — same style as A1. Host flatten in build.rs is not A3. Guest flatten of the FAT ELF (ADR-032) is still A2 memcpy, not the A3 map.
  5. Code page + stack page. The image must fit in 3584 bytes on EL0_PAGE (EL0-exec, no EL0 data — same AP as A1). Rust main saves x30 on SP; that store cannot land on the code page (SCTLR.WXN forbids making it W+X). A second map-window page is EL0-RW NX. rust-lld’s extra ELF-header PT_LOAD at 0x80000000 is discarded at extract time.
  6. Fail-closed probe. Hello serial libctos: hi + libctos: ok (from the payload via uart_write) + kernel libctos: linked. scripts/qemu-smoke.sh greps those and rejects libctos: probe missed. #[test_case] covers the trip and that the blob encodes SVC #16/#17/#18 and not #0/#1/#2. Existing svc: / el0: markers stay.
  7. Honesty. Say “a program linked against libctos issued the A1 ABI and the kernel performed the documented effect” only when the serial / tests pass. Do not say: app hosting is done, Linux ABI, POSIX, userspace, “EL0 isolated,” or “secure OS.” Track A A3–A9 stay Planned.
  8. NFR-10 text is revised in place (ID unchanged) to name this CRT mile. Do not mint FR-16+ or NFR-15+. Guest ELF load is ADR-023, not this ADR.

Consequences

  • Code: libctos/, user/hello-libctos/, build.rs, src/libctos.rs.
  • Docs: syscall.md, el0.md, building-or-porting.md. Roadmap cites #33 / Track A #31.
  • A3 (ELF / raw loader into user TTBR0) may load a separate image. It is not this PR.

ADR-023 — Guest ELF64 PT_LOAD loader into user TTBR0

  • Status: Accepted (loader mile Verified when the serial / tests pass; Track A / app hosting still Planned)
  • Date: 2026-09-12

Context

Track A (issue #31) wants a freestanding image loaded into user maps and entered with ERET. Issue #34 is this mile.

ADR-022 already links user/hello-libctos against libctos. That hello is host-built. build.rs extracts PT_LOAD and the kernel memcpys the blob onto paging::EL0_PAGE. That is not a guest loader.

A3 must parse and map on the guest. The bytes may still be embedded (no VFS yet — A6/A7). The point is map + load + ERET, not host flatten-and-copy onto a fixed probe page alone.

Existing A1 svc:* and A2 libctos:* markers stay. A4–A9 stay out of this PR.

Hypotheses

IDOptionNotes
H1 (chosen)Minimal ELF64 LE AArch64 ET_EXEC, PT_LOAD onlyThe in-tree payload is already that ELF. A2 already walks PT_LOAD on the host. Guest parse is the missing mile.
H2 (rejected)Custom raw image (base VA + entry + blob)Smaller parser, but the host would flatten it — same shape as A2. Would not prove guest ELF walk.
H3 (rejected)Linux ELF ABI / PT_INTERP / ET_DYN / TLS / glibcInflates the claim. Fail-closed: reject PT_INTERP. Not Track B.

Decision

  1. Guest parser in src/loader.rs. Accept ELF64 little-endian AArch64 ET_EXEC. Walk program headers. Load PT_LOAD only. Ignore PT_PHDR / PT_GNU_STACK / notes. Reject PT_INTERP. Reject a PT_LOAD that is W+X (segment flags or two segments that union to W+X on one page). SCTLR.WXN is already on (ADR-015); do not map a writable executable leaf.
  2. User TTBR0 maps. Each page of a PT_LOAD is a freshly allocated frame mapped in the shared map-window L3 (paging::MAP_WINDOW … + MAP_WINDOW_SIZE). That L3 is already in L1_USER, so EL0 and EL1 both see it. Permissions: PF_X → map_el0_exec (EL0 fetch, no EL0 data; EL1 may write the identity PA before / while mapping); PF_W → map_el0_rw; R-only → map_el0_ro. Page-granular union: .text RX and .rodata R on one 4 KiB page become Exec. Not a new user address space.
  3. Copy + BSS. File bytes are written to the frame’s identity PA. p_memsz > p_filesz is zeroed. I-cache is synced on executable pages. e_entry must land in an Exec page.
  4. Stack. A separate EL0-RW NX page at paging::LOADER_STACK_VA (0x80007000), not the A2 code page. Rust main still saves x30 on SP_EL0.
  5. Image source. First cut embedded the ELF (include_bytes!). ADR-032 reads the same bytes from FAT /hello. build.rs still compiles and publishes target/hello-libctos.elf. Not a second QEMU -kernel.
  6. ERET. el0::install_standing(e_entry, stack_top, user_ttbr0) then exception::eret_to_el0. SYS_EXIT returns to EL1 as today. A2’s memcpy path stays so libctos:* markers remain.
  7. Fail-closed probe. Hello serial loader: mapped + loader: ok after a successful trip. The loaded payload still prints libctos: hi / libctos: ok via uart_write. scripts/qemu-smoke.sh greps those and rejects loader: probe missed. #[test_case] covers parse, PT_INTERP reject, W+X reject (flags and overlapping pages), RX+R union on the hello page, and the full map + ERET trip. Existing svc: / libctos: / el0: markers stay.
  8. Honesty. Say “the guest parsed a freestanding ELF64, mapped PT_LOAD into user TTBR0, and ERETed to e_entry” only when the serial / tests pass. Do not say: Linux ELF ABI, PT_INTERP, glibc, app hosting is done, POSIX, userspace, “EL0 isolated,” or “secure OS.” Standing-as-normal is ADR-024. Track A A5–A9 stay Planned.
  9. NFR-10 text is revised in place (ID unchanged) to name this loader mile. Do not mint FR-16+ or NFR-15+.

Consequences

  • Code: src/loader.rs, paging::map_el0_ro / window_range_ok / LOADER_STACK_VA, build.rs emits the ELF, scripts/qemu-smoke.sh.
  • Docs: el0.md, syscall.md, building-or-porting.md. Roadmap cites #34 / Track A #31.
  • A4 (standing EL0 as normal mode) keeps this loader as the way a payload appears (ADR-024).

ADR-024 — Standing EL0 as normal mode

  • Status: Accepted (standing-task mile Verified when the serial / tests pass; Track A / app hosting still Planned)
  • Date: 2026-09-12

Context

Track A (issue #31) wants a loaded freestanding image to run at EL0, not only enter and leave. Issue #35 is this mile.

ADR-013 already has a bounded standing context: SVC #1 stays at EL0, a user MOVZ runs, SVC #2 restores EL1. is_active() is true only for that lifetime. That is a smoke probe, not the supported path for a loaded app.

ADR-021 / ADR-022 / ADR-023 already ERET to a payload and come back on SYS_EXIT. They reuse install_standing as a flag. A leftover is_active() after ERET fails the probe. That is still a one-shot trip, not “standing EL0 is how a loaded image runs.”

A4 keeps the A3 loader as the way a payload appears. It does not add VFS, slots, PAN, or a Linux exec. Existing svc:* / libctos:* / loader:* / el0: standing / el0: restored markers stay.

Hypotheses

IDOptionNotes
H1 (chosen)Distinguish probe vs task standing. A task is the supported path: load (A3) → is_active() true → stay at EL0 for yield / uart_write → exit restores. Unexpected lower-EL sync while a task is standing restores fail-closed instead of parking.Proves real-task is_active and a restore that cannot leave a dangling standing context.
H2 (rejected)Treat the ADR-013 dual-SVC as “normal mode.”Still a one-shot SVC #1 / #2 trampoline. Does not run a loaded app until exit.
H3 (rejected)Claim isolation / PAN / process table while promoting standing.Inflates the mile. Isolation is A5 (issue #36).

Decision

  1. Two standing kinds. install_standing remains the ADR-013 / A1–A3 probe (dual-SVC or ABI trampoline). install_task is the A4 task. is_active() is true for both. is_task() is true only for a task. Saved user PC / SP / TTBR0 live only while is_active().
  2. Supported path. The A3 guest ELF loader maps PT_LOAD from FAT /hello (ADR-032), then install_task(e_entry, stack_top, user_ttbr0) and ERET. The loaded image runs at EL0 until SYS_EXIT. SYS_YIELD / SYS_UART_WRITE stay at EL0 and must observe is_active() (el0: task-active). A2’s memcpy path stays so libctos:* markers remain (same FAT file; no embed).
  3. Fail-closed restore. SYS_EXIT on a task prints el0: task-exit / el0: task-restored and clears is_active(). An unexpected lower-EL sync (unhandled SVC, translation/permission abort, and anything that would otherwise park) while a task is standing prints el0: restore-fail, clears is_active(), and returns to the EL1 caller. It does not park. Probe standing still parks on unexpected sync (A1–A3 unchanged). After ERET returns, leftover is_active() is Failed (caller force-clears and fails the probe).
  4. Fail-closed probe. Hello serial el0: task-enter + el0: task-active + el0: task-exit + el0: task-restored + el0: restore-fail + el0: task-ok. scripts/qemu-smoke.sh greps those and rejects el0: task missed. #[test_case] covers the loaded-hello task trip and the fault-restore trip. Existing svc: / libctos: / loader: / el0: standing / el0: restored markers stay.
  5. Honesty. Say “a loaded freestanding image stood at EL0 until exit, and an unexpected fault restored fail-closed” only when the serial / tests pass. Do not say: app hosting is done, POSIX, userspace, “EL0 isolated,” Linux exec, or “secure OS.” Track A A5–A9 stay Planned.
  6. NFR-10 text is revised in place (ID unchanged) to name this standing-task mile. Do not mint FR-16+ or NFR-15+.

Consequences

  • Code: src/el0.rs (install_task, restore_exit, restore_fault), src/loader.rs (run_hello_as_task), src/exception.rs (task restore instead of park), src/syscall.rs (note_active_while_standing), scripts/qemu-smoke.sh.
  • Docs: el0.md, syscall.md, track-a.md. Roadmap cites #35 / Track A #31.
  • A5 (isolation completion — remaining identity .rodata / .data / heap tear; PAN only if the probe CPU implements it and an ADR says so) is not this PR.

ADR-025 — Identity .rodata tear after pointer rewrite

  • Status: Accepted (identity .rodata unmapped; .data / heap still identity-mapped)
  • Date: 2026-09-12

Context

Track A / A5 (issue #36, parent #31) continues identity teardown where safe. ADR-020 rewrote rustc vtables and unmapped live identity .text after the boot stub. .rodata / .data / heap stayed mapped: string literals and rewritten tables still lived there.

After the ADR-019 high-VA jump, new PC-relative accesses (ADRP) already reach the TTBR1 alias. R_AARCH64_ABS64 words that point at .rodata stay identity until rewritten — the same class as the ADR-020 fn-pointer tables.

.data / .bss / linker stacks / heap are not safe to unmap here. SP is still the identity __stack_top. GlobalAlloc still returns identity frame VAs. Those stay Planned.

Hypotheses

IDOptionNotes
H1 (chosen)Rewrite identity pointers to .rodata, then unmap identity .rodata from kernel and user TTBR0. High twins stay. .data / heap stay.Largest honest cut that does not relocate SP or change the allocator.
H2 (rejected)Also unmap identity .data / heap.SP and Box/Vec pointers are still identity VAs. A silent yank would fault.
H3 (rejected)Claim PAN / “EL0 isolated” while tearing .rodata.Isolation is not this mile. PAN is ADR-026.

Decision

  1. Rewrite identity .rodata pointers. After the live .text tear, walk .rodata and .data. Every 8-byte word that is an identity address in [__rodata_start, __ident_tear_start) is rewritten to its TTBR1 alias. RO pages are writable only for that store, then restored. Serial ident: ro-reloc n=N. N == 0 is allowed (no ABS64 hits); the tear still has to prove high-only access.
  2. Unmap identity .rodata. Unmap [__rodata_start, __ident_tear_start) from kernel and user TTBR0 (TLBI VAAE1 each low VA). High twins stay. Serial ident: rodata lo=… hi=… pages=N with N >= 1.
  3. Keep .data / heap / linker stacks identity-mapped. SP and the first-fit heap still use those VAs. Do not yank them here.
  4. Keep _start / QEMU -kernel at 0x4008_0000. Do not change default -cpu. Do not claim PAN.
  5. Fail-closed probe. Existing ident:* markers stay. New ident: ro-reloc / ident: rodata / ident: rodata-fault / ident: rodata-high. EL1 LDR of a torn identity .rodata VA is a current-EL translation DABORT. EL1 LDR of the high twin still returns a known .rodata magic. scripts/qemu-smoke.sh greps those strings and rejects ident: rodata missed / ident: ro-reloc missed. #[test_case] covers the unmap.
  6. Still Planned (at accept time). Identity .data / heap tear. Superseded in part by ADR-037 (.data/stacks) and ADR-038 (heap). PAN enable on -cpu cortex-a57; EL0 entry without TLBI VMALLE1; lower-EL IRQ while standing; umbrella EL0 isolation.
  7. NFR-10 text is revised in place (ID unchanged). Do not mint NFR-15+.

Honesty

Say “identity .rodata was unmapped while println! still ran from the high twin” only when the serial / tests pass. Do not say:

  • the kernel has moved to the high half
  • identity mappings were fully torn down (.data / heap stay)
  • “secure OS” / “hardened” / “EL0 isolated”
  • PAN (see ADR-026; unimplemented on -cpu cortex-a57)

Consequences

  • paging::rewrite_identity_rodata_ptrs / paging::tear_identity_rodata own the cut. ADR-020 remains the live .text tear.
  • A later ADR may relocate SP and return high heap VAs, then unmap identity .data / heap. That work is not this cut.

ADR-026 — PAN capability: probe the ID field, do not invent the CPU

  • Status: Accepted (capability probe). Historical a57: pan: absent. Default smoke CPU since ADR-079: -cpu cortex-a76 → pan: present. PAN enable + EL1-vs-EL0 fault: ADR-080 (ADR-054 gate items 3–4).
  • Date: 2026-09-12

Context

Track A / A5 (issue #36) may take Privileged Access Never only if ID_AA64MMFR1_EL1.PAN != 0 on the probe CPU and an ADR says so. The documented QEMU line is -cpu cortex-a57 (ARMv8.0). PAN is typically unimplemented there. Silently switching -cpu (for example to max) to make the field non-zero is not an honest probe.

ADR-013 already named PAN as a closing isolation probe: an EL1 access to an EL0-accessible page must fault when PSTATE.PAN is set. That enable + fault mile is not this ADR.

Hypotheses

IDOptionNotes
H1 (chosen)Read ID_AA64MMFR1_EL1.PAN on the default -cpu. Print the field. Enable PSTATE.PAN only if the field is non-zero and a later cut proves the EL1-vs-EL0 fault. On cortex-a57 record pan: absent.CPU evidence without inventing hardware.
H2 (rejected)Change default -cpu to max (or another v8.1+ model) so PAN appears.Silent CPU switch. Forbidden by A5 and by this ADR.
H3 (rejected)Claim PAN / “EL0 isolated” from the ID-field print alone.File presence / a register read is not an access-fault probe.

Decision

  1. Probe the ID field. src/pan.rs reads ID_AA64MMFR1_EL1 bits [23:20] and prints pan: id=<n>. It does not execute MSR PAN. An unimplemented MSR PAN is UNDEF on ARMv8.0.
  2. Default CPU at accept time was cortex-a57. Superseded for the default probe by ADR-079 (-cpu cortex-a76). Do not silent-edit without an ADR.
  3. When n == 0 (expected). Serial pan: absent. Ledger: “PAN enable on virt cortex-a57” stays Planned, with this ID-field print as the CPU evidence. scripts/qemu-smoke.sh greps pan: id= / pan: absent and rejects pan: enabled / pan: probe missed.
  4. When n != 0. Serial pan: present. ADR-080 enables PSTATE.PAN and proves EL1-vs-EL0 fault. Smoke expects pan: present + pan: enabled + pan: el1-fault on -cpu cortex-a76.
  5. Still Planned. PAN enable + EL1-vs-EL0 access fault; lower-EL IRQ while standing; EL0 entry without TLBI VMALLE1; umbrella EL0 isolation. ADR-032 re-states that leftover: do not MSR PAN while the ID field is 0. ADR-025 is the identity .rodata cut, not this.
  6. NFR-10 text is revised in place (ID unchanged). Do not mint NFR-15+.

Honesty

Say “ID_AA64MMFR1_EL1.PAN is 0 on -cpu cortex-a57” only when the serial / tests pass. Do not say:

  • PAN is enabled
  • EL1 cannot access EL0 pages
  • “secure OS” / “hardened” / “EL0 isolated”
  • the probe used a different -cpu than the scripts

Consequences

  • src/pan.rs owns the ID-field probe. Isolation docs cite this ADR as Planned for the enable mile.
  • ADR-047 records: PAN enable is a non-goal on the default probe CPU; ID-field probe stays Verified.
  • A future CPU change needs its own ADR + sponsor and must not be a silent script edit.
  • Enable lock: ADR-054 (2026-09-14) locks PAN enable as non-goal on default cortex-a57 and documents the reopen gate (sponsor CPU + FEAT_PAN + enable+fault ADR).

ADR-027 — Thin VFS + in-RAM memfs

  • Status: Accepted (memfs mile Verified when the serial / tests pass; Track A / app hosting still Planned)
  • Date: 2026-09-12

Context

Track A (issue #31) wants a freestanding app to read and write named buffers. Issue #37 is this mile: a thin VFS with one backend, an in-RAM memfs, plus a fail-closed create / write / read / close probe.

ADR-021 already has exit / uart_write / yield (16–18). There is no path, no handle, and no store. A later virtio-blk + on-disk FS (A7 / issue #38) must not invent a second “open” story. The VFS ADR lands first.

This is not Linux VFS. Not POSIX open / stat / mount. Not a disk. Not app hosting.

Hypotheses

IDOptionNotes
H1 (chosen)Tiny VfsOps (create / open / read / write / close) + one MemFs of heap Vec<u8> named files. New public SVC 19–23. Kernel probe + EL0 trampoline.Same “easiest in-tree” shape as a coop task. No DMA. One backend this mile.
H2 (rejected)Skip the trait and call memfs from SVC only.A7 would then invent a second interface. The VFS is the mile.
H3 (rejected)Ship virtio-blk or FAT in the same PR.A7. Host -drive without guest code is not a probe. Do not say “supports FAT.”
H4 (rejected)Linux/POSIX open flags, directories, cwd, O_CREAT.Inflates the mile and invites a POSIX claim.

Decision

  1. Thin VFS. src/vfs.rs defines VfsOps: create / open / read / write / close on a path + handle. One global backend this mile: MemFs. Later A7 may add a block FS behind the same trait. Not a superblock. Not a dentry cache.

  2. memfs. Flat namespace. Paths are / plus 1–15 bytes of [a-z0-9_-] (no extra /, no .). At most 8 files, 256 bytes each, 4 handles. File bytes live on the first-fit heap (Vec<u8>, ADR-009). Handles are 1..=4. Offset is per handle. create fails if the name exists. open fails if missing. write stores at the handle offset and grows up to FILE_MAX. read copies from the offset. close drops the handle; the file stays. Not a directory tree.

  3. Public SVC numbers (extends ADR-021). Same calling convention (SVC #<n>, x0–x2, return x0). Reserved 0–2 stay ADR-013 probes.

    NumberNameArgsEffectReturn
    19fs_createx0 = user path, x1 = lengthCreate an empty named file and a handlefd >= 1, or 0
    20fs_openx0 = user path, x1 = lengthOpen an existing namefd >= 1, or 0
    21fs_readx0 = fd, x1 = user buf, x2 = lengthCopy from the handle offset into a user-mapped and kernel-mapped rangebytes, or u64::MAX on reject (0 is a valid empty read)
    22fs_writex0 = fd, x1 = user buf, x2 = lengthCopy into the file at the handle offsetbytes, or 0
    23fs_closex0 = fdDrop the handle0 on success, u64::MAX on reject

    Path / I/O length is capped (path 16, I/O 64). Fail-closed: empty length, overflow, non-canonical / TTBR1 alias, or a range that is not user-mapped or not kernel-mapped returns the reject value and does not mutate the file. Kernel .data is rejected. Unknown immediates still park.

  4. libctos wrappers. fs_create / fs_open / fs_read / fs_write / fs_close over svc #19–#23. The A2 hello payload stays exit / uart_write / yield only so A2–A4 markers do not change.

  5. Fail-closed probe. Kernel create / write / read / close of /kprobe prints fs: create / fs: write / fs: read. An EL0 trampoline does the same for /eprobe and prints fs: el0. Then fs: ok. scripts/qemu-smoke.sh greps those and rejects fs: probe missed. #[test_case] covers the kernel round-trip, a missing open, the EL0 trip, and a kernel-.data write reject. Existing svc: / libctos: / loader: / el0: task-* / A5 ident: rodata* / pan:* markers stay.

  6. Honesty. Say “the guest created, wrote, and read a named in-RAM file behind a thin VFS” only when the serial / tests pass. Do not say: app hosting is done, POSIX, Linux VFS, FAT, virtio-blk, on-disk FS, userspace, “EL0 isolated,” or “secure OS.” Track A A7–A9 stay Planned.

  7. NFR-10 text is revised in place (ID unchanged) to name this memfs mile. Do not mint FR-16+ or NFR-15+.

Consequences

  • Code: src/vfs.rs, src/syscall.rs (19–23), libctos/ wrappers, scripts/qemu-smoke.sh.
  • Docs: filesystem.md, overview/filesystem.md, syscall.md, track-a.md. Roadmap cites #37 / Track A #31.
  • A7 (virtio-blk + FAT or xv6-like on that block device) is not this PR. Choose the on-disk format in that ADR.

ADR-028 — virtio-blk + FAT16 behind the thin VFS

  • Status: Accepted (block + FAT read mile Verified when the serial / tests pass; Track A / app hosting still Planned). Write depth: ADR-050.
  • Date: 2026-09-12

Context

Track A (issue #31) wants a freestanding app to read a named file on a disk, not only an in-RAM buffer. Issue #38 is this mile: QEMU virt virtio-mmio block (virtqueues, DMA, sector R/W) plus one on-disk FS, mounted through the A6 thin VFS (ADR-027).

Host QEMU -drive without guest code is not a probe. memfs stays one backend. This mile must not invent a second open story. Not POSIX. Not ext4. Not app hosting. A8–A9 stay Planned.

Hypotheses

IDOptionNotes
H1 (chosen)virtio-mmio blk (poll used ring) + FAT16 on a host-built raw image. Same VfsOps / vfs::open as memfs. Known file /probe holds fat-hi.Host can inspect the image (scripts/mkfat16.py --check). Guest DMA + VFS read is the probe.
H2 (rejected)Tiny xv6-like inode FS.Fine as a teaching layout, but the host cannot file / mount / check the image without a ctos-specific tool. Issue #38 prefers a host-visible image.
H3 (rejected)FAT32.Root is a cluster chain; more code for the same probe. A 4 MiB image is naturally FAT16 once cluster count ≥ 4085.
H4 (rejected)A second open / mount SVC, or a /d/ prefix that breaks the A6 path grammar.ADR-027 forbids a second open story. Paths stay / + [a-z0-9_-].
H5 (rejected this mile)virtio-blk-pci, virtio IRQ via GIC, or write-back FAT this mile.PCI and IRQs are extra surface. FAT stayed read-only here (guest write → FsError::ReadOnly). Sector R/W is proven on an unused last sector. Write-back depth is ADR-050.

Decision

  1. On-disk format is FAT16. Chosen because the smoke host can build and inspect a raw image without a ctos-specific decoder. xv6-like and FAT32 stay unchosen. Do not say “supports FAT” as a product; say “the guest read a known FAT16 file from a virtio-blk image.”
  2. virtio-mmio on QEMU virt. Scan transports at 0x0a000000 + n×0x200 (32 slots) for Device ID 2. QEMU 8.2 virt ships legacy version 1 on those transports (page-aligned QueuePFN); modern version 2 is also accepted. One split virtqueue. DMA addresses are identity PAs of a 4 KiB-aligned .bss buffer (.data/heap stay identity-mapped; ADR-025). Poll used.idx after QueueNotify. Cache maintenance (dc civac) covers the whole DMA object (desc/avail and used/req/status/data) — a page-0-only sync leaves a stale request on write-back D-cache. No virtio IRQ this mile. Device MMIO already sits in the Device-nGnRnE L1 block.
  3. Thin VFS, two backends. vfs::open / read / write / close stay the only story. memfs is tried first. If the name is missing there, FAT16 root (8.3, no extension → /probe for PROBE) is tried. create stays memfs and fails with Exists if FAT already has the name. FAT write / create return ReadOnly. Same path grammar as A6. No extra /. No new SVC numbers.
  4. Host image. scripts/mkfat16.py writes a 4 MiB raw FAT16 volume (cluster count ≥ 4085 so the type is FAT16, not FAT12) with one file PROBE / contents fat-hi. scripts/qemu-smoke.sh, scripts/qemu-aarch64.sh, and scripts/qemu-serial-inject.py attach it as: -drive if=none,file=<img>,format=raw,id=hd0 -device virtio-blk-device,drive=hd0
  5. Fail-closed probe. virtio capacity + last-sector write/read prints blk: virtio / blk: cap / blk: rw / blk: ok. VFS open/read of /probe prints fat: mount / fat: read / fat: ok. scripts/qemu-smoke.sh greps those and rejects blk: probe missed / fat: probe missed. Host --check of the image is extra, not a substitute for the guest. #[test_case] covers sector R/W, VFS /probe, and a missing FAT name. Existing svc:* / libctos:* / loader:* / el0: task-* / ident:* / pan:* / fs:* markers stay.
  6. Honesty. Say “the guest programmed virtio-blk and read a known FAT16 file through the thin VFS” only when the serial / tests pass. First hello on a modern-only (Version==2) scan was blk: probe missed — QEMU 8.2 virt exposes legacy version 1. After accepting v1 + QueuePFN, this cloud VM printed blk: ok and fat: ok. Do not say: app hosting is done, POSIX, Linux VFS, FAT32, xv6, writeable volumes, virtio-net, “EL0 isolated,” or “secure OS.” Track A A8–A9 stay Planned. Host -drive without guest virtio + VFS read is not a probe.
  7. NFR-10 text is revised in place (ID unchanged) to name this virtio-blk + FAT16 mile. Do not mint FR-16+ or NFR-15+.

Consequences

  • Code: src/virtio.rs, src/fat.rs, src/vfs.rs (dispatch + ReadOnly), scripts/mkfat16.py, QEMU -drive in the smoke / runner scripts.
  • Docs: filesystem.md, overview/filesystem.md, track-a.md, threat-model v1.15. Roadmap cites #38 / Track A #31.
  • A8 (documented sample apps, issue #39) is not this PR. Follow-up A8 documented rebuild recipes (what-can-run.md); it did not add a new libctos hello that calls fs_open. The Verified EL0 VFS trip stays /eprobe. Not a new ABI and not a second FS.

ADR-029 — Guest OCI/Docker containers are a non-goal

  • Status: Accepted (docs decision; no guest runtime)
  • Date: 2026-09-12

Context

Track B (issue #40) is Linux-compat research. Child B7 / #47 asks for an explicit claim: ctos does not host Docker/OCI; today Docker on a Linux host runs the ctos smoke image.

Site pages already said no (hosting-apps.md, extra host-apps.md). Track B still listed B7 as Planned and said containers “stay out or far-later.” That is an honesty gap: far-later reads like a later Planned feature.

OCI / Docker / Kubernetes guests need a Linux host: namespaces, cgroups, a Linux ABI, usually overlay or equivalent, and a rich syscall surface. This learning kernel does not have those. Host docker-smoke.sh is a harness (NFR-04 / NFR-05), not a guest runtime.

This ADR does not implement a container runtime. It does not mint FR-16+ or NFR-15+. B1 (Linux-compat goals) may cite it and must not reopen containers as Planned.

Decision

  1. Non-goal. The ctos guest is not a container host. OCI / Docker / Kubernetes workloads on the guest are not Planned. Do not write a container roadmap on Track B.
  2. Arrow today. Docker (or Podman) on a Linux host may build and QEMU-smoke this kernel. That is “Docker hosts ctos,” not “ctos hosts containers.” Ledger Docker rows stay host-harness probes.
  3. Reopen gate. Far-later only with a new GitHub epic plus a new ADR that names the missing Linux-host features and a probe. Track B B1–B6 must not treat containers as a win or a silent Planned row. Do not mint a new FR/NFR ID for this.
  4. Principles drive. Honesty (NFR-06): absence of OCI/runc/cgroup/namespace code in src/ is the probe. Antifragility keeps the host Docker ratchet. Security / performance claims about “container isolation” stay unwritten.

Consequences

  • Docs: track-b.md B7 becomes Documented. Site SoT: hosting-apps.md. Extra: host-apps.md. Ledger row stays Verified absence + non-goal.
  • Track A A8 (sample apps) is unrelated. Do not add an in-tree “container” sample.
  • “Run Alpine on ctos” / “k8s node” stays the same class of claim as “POSIX port is easy.” It is not.

ADR-030 — OS image vs app payload slots

  • Status: Accepted (first-cut two-artifact + FAT load path). Cross-update is ADR-032 (same published ELF on this OS and ba6541c).
  • Date: 2026-09-12

Context

Track A (issue #31) wants a freestanding app that is not rebuilt every time the kernel is. Issue #48 is this mile: disconnect OS image updates from app payloads.

The sponsor goal is scoped immutability, not an “immutable OS” product sentence (immutability.md). A1–A8 already landed ABI, libctos, a guest PT_LOAD loader, standing EL0, memfs, virtio-blk + FAT16, and sample recipes. Today the hello ELF is still include_bytes! into the kernel. That is one linked image.

This mile must not invent OTA, A-B flash, or OCI containers (ADR-029). No new FR/NFR IDs.

Hypotheses

IDOptionNotes
H1 (chosen)Two host artifacts (kernel ELF + hello-libctos.elf) plus a FAT16 app slot (/hello). Guest vfs::open + A3 PT_LOAD map. A2–A4 kept the embed on the first cut.Honest first cut. Storage is the existing memfs/raw-image story (A6/A7). Cross-update and embed removal are ADR-032.
H2 (first-cut reject; leftover accept)Remove include_bytes! and load A2–A4 from FAT only.Rejected on the first cut (PROBE-only image risk). Leftover mile takes this after smoke already requires --app.
H3 (rejected)Second QEMU -kernel, fw_cfg, or a custom raw slot format.Extra boot contract. FAT16 + VFS is already probed.
H4 (rejected)Claim cross-update Verified because the SVC numbers did not change.ABI stability is not a two-boot probe. File presence of this ADR is not that probe.
H5 (rejected)OTA / A-B flash / OCI layers.Out of scope. Not Android. Not a container host.

Decision

  1. Slots. The OS image is the kernel ELF QEMU -kernels (target/aarch64-ctos/debug/ctos). The app payload is a separate host ELF (target/hello-libctos.elf) built from user/hello-libctos. The guest app slot is FAT16 /hello (8.3 HELLO) on the existing virtio-blk volume. /probe stays the A7 known file. Same VFS open / read / close. No new SVC numbers.
  2. Build. build.rs compiles the hello and publishes the ELF to target/hello-libctos.elf (and still writes host flatten metadata). After ADR-032 the kernel does not include_bytes! that file. scripts/mkfat16.py --app writes that ELF as /hello (cluster chain; cap 64 KiB). Smoke and the QEMU runner fail closed if the app artifact is missing.
  3. Boot/load. Hello path: OS boots, A1–A8 probes run. After ADR-032, A2–A4 also read FAT /hello (no include_bytes!). Then A9 reads /hello through the thin VFS, parses ELF64 PT_LOAD with the A3 loader, and ERETs. No embed fallback on this path. Missing /hello prints slot: probe missed.
  4. Fail-closed probe. Serial slot: fat + slot: mapped + slot: ok after a successful FAT load + ERET. perf: app-load ticks=<n> is a CNTPCT measurement around that path (NFR-07) — not a bench and not a delta vs the embed. scripts/qemu-smoke.sh greps those and rejects slot: probe missed / slot: embed. Host smoke requires both artifacts and mkfat16.py --check --require-app. #[test_case] covers FAT /hello ELF magic + parse + the FAT load trip.
  5. Honesty. Say “the guest loaded a separate app ELF from the FAT slot into user TTBR0” only when the serial / tests pass. Do not say: apps update independently of the OS, immutable OS, OTA, A-B, containers, “EL0 isolated,” or “secure OS.” Cross-update is a host two-boot probe (ADR-032): same published hello-libctos.elf on this OS and on ba6541c. File presence of this ADR is not that probe. A kernel rebuild still compiles the payload (build.rs); it no longer embeds the bytes.
  6. Performance. Expected costs stay the documented shape (boot/load, SVC, ASID/TTBR). The new marker proves the path is measurable. A compared CNTPCT pair vs a probe-only linked-in trip is ADR-046. Do not invent a bench.
  7. NFR-10 text is revised in place (ID unchanged) to name this slot first cut. Do not mint FR-16+ or NFR-15+.

Consequences

  • Code: src/slot.rs, src/loader.rs (run_image), build.rs publish path, scripts/mkfat16.py --app, QEMU runner / smoke.
  • Docs: immutability.md, advantages.md, track-a.md, threat-model v1.16. Roadmap cites #48 / Track A #31.
  • Track A children A1–A9 now have a first cut. ADR-032 takes cross-update + embed-off + isolation honesty. Slot-load CNTPCT pair is ADR-046 (Verified measurement, not a percent). Remaining Planned / decided: PAN enable non-goal on default a57, never yank _start, leftover identity RAM optional, umbrella isolation Planned/non-claim (ADR-047); product “app hosting is done” (ADR-048).

ADR-031 — Linux-compat goals, non-goals, keep ctos specificity

  • Status: Accepted (docs frame; no Linux userspace claim)
  • Date: 2026-09-12

Context

Track B (issue #40) is Linux-compat research. Child B1 / #41 asks for the frame: what “Linux-compat” means here (ABI subset vs a full Linux kernel), what stays out, and which ctos invariants must not move.

B7 / ADR-029 already records that guest OCI / Docker / Kubernetes is a non-goal. This ADR cites that decision. It does not reopen containers as Planned or “far-later on this track.”

Track A (issue #31) has first cuts A1–A9: a tiny SVC ABI (ADR-021), libctos (ADR-022), a freestanding ELF64 PT_LOAD loader that rejects PT_INTERP (ADR-023), standing EL0 (ADR-024), thin VFS + memfs (ADR-027), virtio-blk + FAT16 (ADR-028), sample recipes, and an OS/app slot first cut (ADR-030). Those miles are freestanding. They are not a Linux ABI and not Linux userspace.

Issue #40’s reality check stands: a full Linux ABI (and containers) is a multi-year kernel project. Track B is an ADR / research ladder, not a promise to become Linux.

This ADR does not implement B2–B6. It does not mint FR-16+ or NFR-15+. File presence of this page is not a Linux userspace probe.

Hypotheses

IDOptionNotes
H1 (chosen)Frame Linux-compat as research into a possible ABI subset. Keep the freestanding Track A path. Containers stay ADR-029. Say “not claiming Linux userspace yet.”Matches #40 / #41. Leaves B2–B6 room to map gaps and then decide.
H2 (rejected)Promise a full Linux ABI, glibc/musl, or “runs Alpine / busybox.”Honesty fail (NFR-06). No probe exists.
H3 (rejected)Reopen guest containers as Planned or “far-later” on Track B.Conflicts with ADR-029. Reopen only with a new epic + ADR.
H4 (rejected)Replace Track A SVC / libctos with Linux AArch64 svc #0 / x8 numbers in this mile.Would break probed A1–A9 markers for a research frame. B2 maps the gap; it does not rewrite src/.
H5 (rejected)Treat A3 PT_LOAD + A9 FAT /hello as Linux userspace.The loader rejects PT_INTERP. Standing EL0 is not execve. File presence is not that claim.

Decision

  1. What Linux-compat means. On ctos it means: research whether a subset of the Linux AArch64 ABI could ever be useful for a named, probed workload, without dropping ctos specificity. It is not a promise to run a Linux distro, glibc, musl, a shell, Python, or “Linux userspace.” Full Linux ABI is out of scope for this track’s promise. B6 may later choose “never.”

  2. Goals (research ladder only). Later children stay document-first. This PR does not write them.

    LaterJobMust not do in that child
    B2 (#42)Map Linux aarch64 syscalls vs today’s ctos SVC (16–23 + reserved 0–2)Do not add Linux syscall numbers to src/
    B3 (#43, ADR-035)Process model vs Linux fork / exec / waitDo not add those syscalls. Standing EL0 stays the ctos model.
    B4 (#44)Linux ELF / auxv / PT_INTERP vs the freestanding loaderDo not accept PT_INTERP. Gap ADR: ADR-033.
    B5 (#45)Linux VFS concepts vs the thin ctos VFSDo not grow POSIX open flags / dentries
    B6 (#46)Decision: compat layer vs reimplement vs neverDo not treat “compat layer” as already chosen

    Implementation, if any, comes after B6 names a path, in a later epic, with a probe. “Never” is a valid B6 outcome.

  3. Non-goals.

    • Full Linux ABI / POSIX product / glibc or musl ports.
    • Guest OCI / Docker / Kubernetes — already ADR-029. Not Planned on Track B.
    • “Run Alpine on ctos,” “k8s node,” “Linux userspace works.”
    • Replacing the freestanding Track A path (A1–A9 first cuts stay).
    • Widening off QEMU virt / AArch64 without a new ISA ADR (ADR-003).
    • New FR/NFR IDs. Frozen set stays fr-nfr.md.
    • Author self-merge (ADR-002).
  4. Keep (ctos specificity). Track B is subordinate to principles.md. Linux-compat research must not override:

    InvariantWhy it stays
    Honesty (NFR-06 / ADR-001)Status words need a probe. Unprobed stays Unknown. This ADR is not Verified Linux.
    Antifragility (NFR-05)Fail-closed qemu-smoke and host Docker/arm64 workstation ratchets stay. Host docker-smoke.sh is still “Docker hosts ctos,” not a guest Linux ABI.
    Security (NFR-10)No “secure OS,” “Linux-compat isolation,” or “the kernel is W^X” from a research frame.
    Performance (NFR-07)No invented benches for a compat layer that does not exist. Measure first if a later mile adds a path.
    Document-first / one-PR loopsOne child → one branch → one GitHub PR.
    Freestanding Track A pathIn-tree apps keep SVC #n, libctos, PT_LOAD without interpreter, thin VFS. A research map must not silently retarget those numbers to Linux x8.
    Virt learning scopeQEMU virt + PL011 until an ADR widens it.
    ADR-gated memory / EL0Isolation, PAN enable, remaining identity RAM / _start stay their own rows.
  5. Relationship to ADR-029. Guest containers remain a non-goal. B1–B6 must not treat containers as a win, a silent Planned row, or a reason to grow namespaces / cgroups / overlay. Reopen only with a new GitHub epic plus a new ADR that names the missing Linux-host features and a probe. Do not mint an FR/NFR ID for that.

  6. Not claiming Linux userspace yet. Say that sentence in the roadmap and on the Track B page. Today:

    • A3 maps freestanding ELF64 ET_EXEC PT_LOAD and rejects PT_INTERP.
    • Standing EL0 is not a Linux process table.
    • Thin VFS + memfs / read-only FAT16 is not Linux VFS.
    • A9 FAT /hello is not execve.
    • The ledger row “Guest runs host apps (Linux ELF / shell / Python)” stays Planned. This ADR does not change that row to Verified.
  7. Principles drive. Do not override honesty, fail-closed sensors, or the three pillars for a “runs Linux” headline.

Consequences

  • Docs: track-b.md B1 becomes Documented. SUMMARY.md and roadmap.md link here. Site extra: hosting-apps.md / host-apps.md point at the frame without a userspace claim.
  • B2 is Documented (syscall gap note). B3 is Documented (ADR-035). B4 is Documented (ADR-033). B5 / B6 stay Planned research. No src/ change in this PR.
  • Track A remains the product path for in-tree freestanding apps. Cross-update and “app hosting is done” stay unclaimed.
  • Honesty: document inspection only. Do not add a Verified Linux-userspace ledger row because this file exists.

ADR-032 — Track A leftovers: cross-update, FAT-only hello, isolation honesty

  • Status: Accepted (leftover mile when the serial / host smoke / tests pass)
  • Date: 2026-09-12

Context

Track A children A1–A9 have first cuts. After ADR-030 merged on main ba6541c, three honesty gaps remained:

  1. A9 cross-update. Same published hello-libctos.elf on this OS and a documented prior OS. ABI stability is not that probe.
  2. Embed honesty. A2–A4 still include_bytes! a hello copy while A9 loaded FAT /hello. Dual path or remove the embed.
  3. Isolation leftovers. After A5: identity .data / heap tear if safe; PAN enable only if ID_AA64MMFR1_EL1.PAN != 0 on default -cpu cortex-a57. Do not claim “EL0 isolated.”

No new FR/NFR IDs. Author does not merge (ADR-002). The number is 032 because open #62 already claimed ADR-031 for Linux-compat goals.

Hypotheses

IDOptionNotes
H1 (chosen)Cross-update: build documented prior OS ba6541c (A9 merge — earliest main tip with the slot/FAT path) and boot the same published app ELF on that kernel and on this tip. Fail closed.Real SHA. No stored kernel blob.
H2 (rejected)Claim cross-update from unchanged SVC numbers, or invent a prior kernel artifact that was never built here.Invents evidence.
H3 (chosen)Remove include_bytes! of hello. A2 flattens FAT /hello PT_LOAD; A3/A4 parse the same file. build.rs still compiles and publishes the payload. Host smoke greps that src/ has no hello include_bytes!.Prefer (a). Markers stay if FAT /hello is present (smoke already requires it).
H4 (rejected)Keep the embed as a dual path and only document the gap.Allowed if (a) broke smoke. Not needed if FAT load stays green.
H5 (chosen)Isolation: do not enable PAN (pan: absent on cortex-a57). Do not unmap identity .data / heap (SP and GlobalAlloc are still identity VAs). Print ident: data-stay / ident: heap-stay. Umbrella isolation stays Planned.Advance the honest mile. A silent tear would fault.
H6 (rejected)Switch -cpu so PAN appears, or yank .data/heap without relocating SP/allocator.Forbidden by ADR-026 / ADR-025.

Decision

  1. Prior OS is ba6541c. That is the A9 merge commit (A9: Disconnect OS image from app payloads (ADR-030) (#60)). No earlier main SHA has the slot/FAT path. Do not store a prior kernel ELF in git. scripts/qemu-smoke.sh builds that commit in a git worktree and QEMU -kernels it with the same app bytes this tip published (sha256 pin via sha256sum, shasum -a 256, or openssl dgst -sha256). Docker smoke needs git in the image and .git objects in the build context (not host .git/worktrees registrations). Smoke prunes stale worktrees before add. Do not invent a stored kernel blob. Both boots must print slot: fat / slot: mapped / slot: ok and must not print slot: embed / slot: probe missed. This OS SHA must differ from ba6541c.
  2. No hello embed. src/libctos.rs and src/loader.rs read FAT /hello through fat::read_file (same VFS open as A7). A2 still memcpy-flattens PT_LOAD at/after EL0_PAGE (not a guest ELF map). A3/A4 still parse and ERET. Size must match HELLO_ELF_LEN / HELLO_LEN from this build.rs so a swapped slot cannot silently satisfy those miles. Host smoke rejects include_bytes!.*hello-libctos under src/.
  3. Isolation honesty. Serial ident: data-stay / ident: heap-stay after walking those identity pages. #[test_case] identity_data_and_heap_still_mapped. PAN enable stays Planned (pan: id=0 / pan: absent; smoke still rejects pan: enabled). Do not claim “EL0 isolated.”
  4. Still Planned (at accept time). Identity .data / heap tear (needs SP relocate + high allocator VAs). Superseded in part by ADR-037 and ADR-038. PAN enable on -cpu cortex-a57; lower-EL IRQ while standing; EL0 entry without TLBI VMALLE1; umbrella isolation; A9 slot-disconnect delta vs the old embed (later ADR-046); product “app hosting is done.”
  5. NFR-10 text is revised in place (ID unchanged). Threat-model v1.17. Do not mint FR-16+ or NFR-15+.

Honesty

Say “the same published hello-libctos.elf loaded on this OS and on ba6541c” only when host smoke printed both cross-update … slot:ok lines and the app sha256 pin. Say “A2–A4 no longer embed the hello” only when src/ has no hello include_bytes! and the FAT markers still pass. Say “identity .data / heap are still mapped” when ident: data-stay / ident: heap-stay print. Do not say:

  • apps update independently / app hosting is done / OTA / A-B / containers
  • EL0 isolated / PAN enabled / the kernel moved / identity mappings fully torn down
  • a prior OS that does not exist in git
  • Verified without this cloud (or GHA) qemu-smoke

Consequences

  • ADR-037 supersedes the .data stay marker. ADR-038 supersedes ident: heap-stay with an honest heap tear (ident: heap-reloc / ident: heap).

  • Code: fat::read_file, loader/libctos FAT path, teardown stay markers, scripts/qemu-smoke.sh embed + cross-update + stay greps.

  • Docs: this ADR, ADR-030, A2–A4 ADRs (payload source), ADR-025 / ADR-026 (still Planned), ledger, Track A, threat-model v1.17.

ADR-033 — Linux ELF / auxv / PT_INTERP vs the freestanding loader

  • Status: Accepted (gap ADR; no Linux ELF / dynamic-linker claim)
  • Date: 2026-09-12

Context

Track B child B4 / #44 (parent #40) asks what would be needed to load a musl-linked binary versus today’s freestanding image, and to keep the Track A loader reusable.

Frame: ADR-031. That frame says B4 must not accept PT_INTERP, must not claim Linux userspace, and must not replace the freestanding Track A path. Syscall convention is already a gap (linux-aarch64-syscall-gap.md / B2). Process execve is B3 / #43. This ADR does not write B3, B5, or B6.

Track A A3 (ADR-023) already parses ELF64 little-endian AArch64 ET_EXEC, maps PT_LOAD only, rejects PT_INTERP, rejects W+X, and ERETs to e_entry. A2–A4 and A9 reuse that parser on FAT /hello (ADR-030, ADR-032). The in-tree payload is libctos, not musl or glibc.

This mile is docs only. It does not change src/loader.rs. File presence of this page is not a Linux ELF probe and not dynamic-linker support.

Probe (what Verified means here)

ClaimProbeStatus
Linux ELF types / PT_*Read Linux v6.10 include/uapi/linux/elf.h (ET_EXEC 2, ET_DYN 3, PT_LOAD 1, PT_DYNAMIC 2, PT_INTERP 3, PT_PHDR 6, PT_TLS 7)Document inspection
Linux auxv keysRead Linux v6.10 include/uapi/linux/auxvec.h (AT_NULL … AT_EXECFN, AT_MINSIGSTKSZ)Document inspection
ctos freestanding loaderRead src/loader.rs (parse_elf64, PT_INTERP reject, LOADER_STACK_VA) + ADR-023 on this treeDocument inspection
Dynamic Linux ELF / musl / glibc on the guestNo such serial markerNot claimed. Ledger “Guest runs host apps” stays Planned.

A later kernel qemu-smoke is a different row. This mile did not edit src/.

Hypotheses

IDOptionNotes
H1 (chosen)Document the Linux vs ctos gap. Keep the ADR-023 loader reusable as the freestanding contract. Do not accept PT_INTERP. Do not claim musl/glibc/dynamic ELF.Matches #44 / ADR-031. Leaves B6 room to choose never.
H2 (rejected)Accept PT_INTERP / ET_DYN / auxv in this mile, or rewrite parse_elf64 as a Linux binfmt_elf.ADR-031 forbids accepting PT_INTERP here. Would break A3 Interp tests and A2–A9 markers.
H3 (rejected)Treat A3 PT_LOAD + A9 FAT /hello as Linux execve / dynamic linking.The loader rejects PT_INTERP. Standing EL0 is not execve. Honesty fail (NFR-06).
H4 (rejected)Promise a musl or glibc port, “runs Alpine,” or a guest ld-linux / ld-musl.ADR-031 H2. No probe exists.
H5 (rejected)Fork a second incompatible parser this mile so Track A and “Linux ELF” diverge silently.#44 asked to keep the Track A loader reusable. A later Linux path, if B6 ever names one, is a new layer that may call the PT_LOAD helpers — not a silent retarget.

Decision

  1. What Linux exec of an ELF does (inspection). Linux fs/binfmt_elf.c (not copied here) loads ET_EXEC or ET_DYN (PIE). If a PT_INTERP string is present, it also maps that interpreter (typically glibc ld-linux-aarch64.so.1 or musl ld-musl-aarch64.so.1) and transfers control to the interpreter entry, not the binary e_entry. The kernel builds a user stack:

    argc → argv[] → NULL → envp[] → NULL → auxv[] (AT_* key/value pairs) → AT_NULL.

    The auxiliary vector (Linux v6.10 auxvec.h) is how the dynamic linker and libc CRT find program headers, page size, interpreter base, the real entry, and entropy. Typical keys a musl/glibc _start reads: AT_PHDR / AT_PHENT / AT_PHNUM, AT_PAGESZ, AT_BASE, AT_ENTRY, AT_UID/AT_EUID/AT_GID/AT_EGID, AT_SECURE, AT_RANDOM (16 bytes), AT_HWCAP / AT_HWCAP2, AT_CLKTCK, AT_PLATFORM, AT_EXECFN, often AT_SYSINFO_EHDR (vDSO) and AT_MINSIGSTKSZ.

    After that, the interpreter walks PT_DYNAMIC (DT_NEEDED, DT_SYMTAB, DT_RELA / DT_REL, DT_INIT_ARRAY), maps shared objects, applies relocations, sets up PT_TLS, then jumps to AT_ENTRY. That is dynamic linking. It is not PT_LOAD copy.

  2. What the ctos loader does (inspection). src/loader.rs / ADR-023:

    Itemctos today
    File typeELF64 LE AArch64 ET_EXEC only (NotExec on anything else, including ET_DYN)
    SegmentsPT_LOAD only. PT_PHDR / PT_GNU_STACK / notes ignored.
    PT_INTERPParseError::Interp. Fail-closed. Smoke/tests require the reject.
    W+XReject (segment flags or page-union). SCTLR.WXN is on (ADR-015).
    VA windowpaging::MAP_WINDOW 0x8000_0000 … + MAP_WINDOW_SIZE (512 × 4 KiB). Caps: 8 PT_LOADs, 8 pages.
    StackOne zeroed EL0-RW NX page at LOADER_STACK_VA (0x80007000). No argc / argv / envp / auxv.
    EntryERET to the binary e_entry. No interpreter.
    CRTlibctos wrappers + _start. Does not read auxv. Calls SVC #<n> (ADR-021).
    Image sourceFAT /hello (A9 / ADR-032). Not execve.

    No Linux ELF feature in the gap table below is present. PT_LOAD of a freestanding ET_EXEC is partial (related map, different contract).

  3. Status words (same idea as B2).

    StatusMeans
    presentSame Linux ELF/auxv/PT_INTERP meaning, so a musl/glibc AArch64 binary could start with no translation.
    partialTrack A has a related PT_LOAD / stack / entry behavior. Type, stack, or control flow still differ.
    absentNo analog. B6 may still discuss a named subset. Not a promise to implement.
    stay-rejectedA3 fail-closed reject. This ADR does not flip it. Reopen only after B6 names a path, in a later epic.
    never-per-ADR-031Would imply a Linux distro / glibc-musl product / guest interpreter that ADR-031 already ruled out as a Track B promise.
  4. Gap table.

    Image type and program headers

    Linux / ELF itemctos statusNotes
    ELF64 LE AArch64 magicpartialSame header walk. Not a Linux ABI.
    ET_EXEC PT_LOAD map + BSS zeropartialA3 does this inside the 2 MiB window, 8 segments / 8 pages.
    ET_DYN / PIE / loader-chosen baseabsentparse_elf64 returns NotExec. Linux PIE is ET_DYN.
    PT_INTERPstay-rejectedA3 tests require Interp. ADR-031: do not accept it in this child.
    Load the interpreter ELFabsentNo second image, no AT_BASE.
    Transfer to interpreter entryabsentERET is always binary e_entry.
    PT_DYNAMIC / DT_NEEDEDabsentIgnored (not PT_LOAD). No shared-object search.
    Relocations (DT_RELA / REL / RELR / IFUNC)absentFreestanding image is already linked.
    PT_TLS / TPIDR_EL0absent
    PT_GNU_STACK exec-stackabsentIgnored. Stack is NX. A Linux exec-stack request would fight W^X.
    PT_GNU_RELROabsent
    PT_PHDR usable at runtimeabsentNo AT_PHDR. Headers are not guaranteed mapped at a libc-known VA.

    Stack and auxv (Linux v6.10 auxvec.h)

    Itemctos statusNotes
    argc / argv[] / envp[]absentStack page is zeros. libctos does not read SP.
    AT_NULL terminator + key/value pairsabsent
    AT_PHDR / AT_PHENT / AT_PHNUMabsentmusl/glibc ldso needs these.
    AT_PAGESZabsentGuest pages are 4 KiB; nothing publishes that to EL0.
    AT_BASEabsentInterpreter load base.
    AT_ENTRYabsentReal program entry after ldso init.
    AT_RANDOM (16 bytes)absentTypical stack canary / SSP. No getrandom either (B2).
    AT_UID / AT_EUID / AT_GID / AT_EGID / AT_SECUREabsentNo uid model.
    AT_HWCAP / AT_HWCAP2 / AT_PLATFORMabsent
    AT_EXECFNabsentA9 path is FAT /hello, not a Linux pathname.
    AT_SYSINFO_EHDR (vDSO)absentNo vDSO mapping.
    AT_MINSIGSTKSZabsentNo POSIX signals (B2).

    musl-linked vs freestanding (the #44 question)

    WorkloadWhat it still needs on ctosHonesty
    In-tree libctos ET_EXECAlready the A3/A9 path. Keep it.Not Linux.
    Static musl (-static, often no PT_INTERP)Linux stack + auxv; Linux svc #0/x8 (exit_group, set_tid_address, brk/mmap, write on fd 1 — B2); often ET_DYN static-PIE; VAs usually not in 0x8000_0000…+2 MiB; image larger than 8 pages.Not a small subset. ADR-031 rejected “runs musl” as a promise.
    Dynamic musl (PT_INTERP → ld-musl-aarch64.so.1)Everything static needs, plus accept PT_INTERP, load the interpreter, AT_BASE/AT_PHDR/AT_ENTRY, PT_DYNAMIC + DT_NEEDED files on a real FS, relocations, TLS.stay-rejected + absent. Do not claim dynamic Linux ELF.
    glibc dynamicSame class as dynamic musl, usually heavier (ld-linux, vDSO, more CRT syscalls).Same non-claim.
  5. Keep the Track A loader reusable. A2 flatten, A3 map+ERET, A4 standing task, and A9 FAT /hello all call parse_elf64 / run_image. This ADR does not widen that function. A later Linux ELF path — only if B6 / #46 chooses a compat layer, and only with a probe — would be a new module that may reuse page-map helpers. It must not silently retarget parse_elf64 to accept PT_INTERP or ET_DYN. Track A tests that reject Interp stay.

  6. Not claiming dynamic Linux ELF support. Say that sentence on the Track B page. Today:

    • A3 maps freestanding ELF64 ET_EXEC PT_LOAD and rejects PT_INTERP.
    • The stack has no auxv.
    • ERET is not interpreter start.
    • A9 FAT /hello is not execve.
    • The ledger row “Guest runs host apps (Linux ELF / shell / Python)” stays Planned.
  7. If a subset is ever useful (not a decision). Research hint for B6, not a backlog:

    1. In-tree libctos apps need no Linux ELF features. Keep ADR-023.
    2. Static musl is already a libc-shaped surface (B2) plus auxv/stack/ET_DYN/VA-window work. That is not a B4 implementation item.
    3. Dynamic musl/glibc starts with accepting PT_INTERP. This track’s frame forbids that in B4. B6 may choose never.
    4. Do not start with relocations, TLS, or a guest ld.so.
  8. Non-goals. Full Linux ELF ABI; glibc/musl ports; accepting PT_INTERP in src/; rewriting Track A probes; guest containers (ADR-029); new FR/NFR IDs; author self-merge (ADR-002).

Consequences

  • Docs: this ADR. track-b.md B4 becomes Documented. SUMMARY.md and roadmap.md link here. Site extras point at the gap without a userspace claim.
  • B3 / B5 / B6 stay Planned research. No src/ change in this PR.
  • Track A A3 remains the product loader for in-tree freestanding apps. PT_INTERP stays rejected.
  • Honesty: document inspection only. Do not add a Verified Linux-ELF or dynamic-linker ledger row because this file exists.

ADR-034 — Linux VFS concepts vs thin ctos VFS

  • Status: Accepted (docs compare; no Linux FS claim)
  • Date: 2026-09-12
  • Number: ADR-034 is frozen for this mile. Main holds ADR-033 (B4 ELF / auxv / PT_INTERP, #68 / 774c672) and ADR-035 (B3 process model, #67 / d81a53a). Do not yield this number again.

Context

Track B (issue #40) is Linux-compat research. Child B5 / #45 asks how ctos VFS / memfs / FAT stays thin while optionally mapping to Linux-like semantics later.

ADR-031 already framed Linux-compat as ABI-subset research and told this child: compare Linux VFS concepts vs the thin ctos VFS; do not grow POSIX open flags / dentries. B2 already mapped the file-descriptor slice: Linux AArch64 openat / read / write / close are partial against ctos fs_create / fs_open / fs_read / fs_write / fs_close (SVC 19–23). Linux mount / umount2 / pivot_root stay never-per-ADR-031.

Track A first cuts stay freestanding:

  • Thin VFS + in-RAM memfs (ADR-027): VfsOps is create / open / read / write / close. Flat path / + 1–15 of [a-z0-9_-]. At most 8 files, 256 bytes, 4 handles.
  • virtio-blk + FAT16 behind the same vfs::open (ADR-028); write depth ADR-050. Known files /probe and A9 /hello; guest may create /fwr. Serial fat: mount is a probe marker, not mount(2).

This ADR does not implement B3, B4, or B6. It does not mint FR-16+ or NFR-15+. It does not add POSIX flags, dentries, cwd, or a second open story to src/. File presence of this page is not a Linux filesystem.

Hypotheses

IDOptionNotes
H1 (chosen)Compare Linux VFS objects (dentry / inode / path / mount) to today’s thin VFS. Stay thin. Document gaps. Optional later mapping is translation after B6, not a dentry cache this mile.Matches #45 / ADR-031. Leaves B6 room to choose never.
H2 (rejected)Grow POSIX open flags, dentries, cwd, or O_CREAT in this PR.ADR-031 forbids it. Inflates Track A probes and invites a POSIX claim.
H3 (rejected)Say “ctos has Linux VFS” because memfs + FAT16 exist.Honesty fail (NFR-06). Serial fs: ok / fat: ok are thin-VFS probes, not Linux VFS.
H4 (rejected)Add mount(2), overlay, or a mount namespace as Planned on Track B.Conflicts with ADR-031 / ADR-029. A7 FAT attach is not a Linux mount tree.
H5 (rejected)Decide compat layer vs reimplement vs never here.That is B6 / #46.

Decision

  1. Stay thin. VfsOps remains five calls on a path + handle. One open story. memfs first; if the name is missing, FAT16 root (8.3) is tried. FAT fds are tagged so the router does not invent a second interface. Not a superblock. Not a dentry cache. Not POSIX open.

  2. What Linux VFS is (concepts, not a port). Linux path lookup is a walk of cached dentries that bind a name to an inode, under a vfsmount, on a super_block. An open file is a separate file object (f_pos, f_op). A path is the pair (mnt, dentry). Hard links share an inode across dentries. . / .. / extra / / relative names / AT_FDCWD / symlinks are part of that walk. This paragraph is a teaching sketch of Linux, not a claim that ctos implements any of it.

  3. Concept map. Status words match B2: present means the Linux object exists here with comparable semantics; partial means a related ctos object exists but identity or walk still differ; absent means no analog; never-per-ADR-031 means a Linux mount / overlay / namespace class already ruled out.

    No Linux VFS object is present.

    Linux conceptctos todayStatusWhy not present
    super_block (FS instance, s_op, root)Two backends behind one router (MemFs + Fat16)absentNo superblock, no FS type registration, no s_root.
    inode (identity, i_ino, i_mode, i_op)memfs File (name + Vec<u8>) or FAT 8.3 dirent + clusterabsentNo inode number, mode, nlink, or inode ops. Close does not destroy the memfs bytes; that is not an inode lifetime.
    dentry (cached name → inode, d_parent)Exact string match on a flat tableabsentNo dentry cache, no parent, no hash walk, no RCU-walk. ADR-027 rejected a dentry cache.
    path (vfsmount + dentry)Canonical byte string / + [a-z0-9_-]partialA path exists as a name. It is not a (mnt, dentry) pair.
    file (open instance, f_pos, f_op)Handle 1..=4 with a per-handle offsetpartialRelated effect to an fd + offset. Four handles. No f_op, no fd 0/1/2, no dup. FAT handles use a tag bit, not a Linux fd table.
    vfsmount / mount treeSerial fat: mount after virtio + FAT attachnever-per-ADR-031Not mount(2). No mount namespace, no overlay, no umount2 / pivot_root. B2 already marked those never-per-ADR-031.
    Directory / getdents64Flat namespace; FAT root is a lookup table, not a listed dirabsentNo mkdirat, no cwd.
    stat / statx / inode metadataNoneabsentB2: newfstatat / fstat / statx absent.
    Symlink / hard linkNoneabsentOne name per memfs slot; FAT 8.3 one dirent.
    POSIX path walk (., .., //, relative, AT_FDCWD)valid_path rejects extra /, ., and anything outside [a-z0-9_-]absentSee path grammar below.
  4. POSIX path grammar vs ctos names.

    ItemLinux / POSIXctos (valid_path)
    ShapeAbsolute or relative; / separators; empty components collapseMust start with /; no extra /
    . / ..Current / parentRejected
    LengthPATH_MAX (typically 4096)2–16 bytes total (/ + 1–15)
    AlphabetAlmost any byte except NUL and /[a-z0-9_-] after the first /
    cwd / chdir / getcwdYesNo. B2: chdir / getcwd absent
    dirfd / AT_FDCWDopenat first argumentNo. fs_open takes a user path pointer + length
    O_* / modeopenat flags + modeNo. Create is a separate SVC (fs_create 19). Existing name is fs_open 20.
    FAT 8.3vfat long names + short namesGuest path /probe maps to PROBE; /hello to HELLO. Not VFAT.
  5. Backends are not Linux FS types. memfs is named heap buffers, not tmpfs. FAT16 on virtio-blk is a host-built 4 MiB image with two known files, not Linux vfat / msdos. Do not say “supports FAT” as a product. Say “the guest read a known FAT16 file through the thin VFS” only when the ledger has fat: ok.

  6. Syscall slice stays on B2. This ADR does not retarget SVC 19–23 to Linux x8. The B2 rows still stand: openat partial, read partial, write partial (uart_write or fs_write, neither is write(1, …)), close partial, lseek / pread64 / getdents64 / ioctl absent, mount never-per-ADR-031.

  7. Optional later mapping (not a decision). If B6 ever names a compat path, the smallest file slice would still be convention translation of openat / read / write / close onto the existing thin VFS — not a Linux dentry cache, not POSIX flags, not cwd, not mount(2). Growing those objects needs a later epic + ADR after B6. Never remains valid.

  8. Not claiming a Linux filesystem. Say that sentence on the Track B page. Today:

    • Thin VFS + memfs / read-only FAT16 is not Linux VFS.
    • fat: mount is not mount(2).
    • A9 FAT /hello is not a Linux rootfs.
    • The ledger row “Guest runs host apps (Linux ELF / shell / Python)” stays Planned. This ADR does not change that row to Verified.
  9. Keep (ctos specificity). Honesty (NFR-06): this file is not Verified Linux VFS. Antifragility keeps fs: / blk: / fat: smoke strings. Security: no “Linux-compat isolation” from a compare. Performance: no invented VFS bench. Freestanding Track A path stays. Guest containers stay ADR-029.

Consequences

  • Docs: track-b.md B5 becomes Documented. SUMMARY.md and roadmap.md link here. Site extra: filesystem.md / filesystem.md point at the compare without a Linux FS claim.
  • B6 stays Planned research. B3 is Documented (ADR-035). B4 is Documented (ADR-033). No src/ change in this PR.
  • Track A A6/A7 probes stay the product path for named files. Do not grow VfsOps to look like Linux so a research ADR can say present.
  • Honesty: document inspection of this ADR + src/vfs.rs / src/fat.rs + the B2 gap table. Do not add a Verified Linux-filesystem ledger row because this file exists.

ADR-035 — Process model: Linux fork/exec/wait vs ctos standing EL0

  • Status: Accepted (docs frame; no Linux process-model claim)
  • Date: 2026-09-12

Context

Track B (issue #40) is Linux-compat research. Child B3 / #43 asks to compare the Linux process model (fork / exec / wait) to ctos standing EL0 and the EL1 cooperative scheduler, and to decide an approach without abandoning probes.

ADR-031 already frames Linux-compat as ABI-subset research. It is not claiming Linux userspace. Guest OCI / Docker / Kubernetes stays a non-goal (ADR-029). This ADR cites those decisions. It does not reopen containers, mint a process table, or add syscalls.

B2 mapped numbers only. Process and lifetime rows there are absent or partial:

Linux AArch64 (v6.10)ctos today
No fork syscall — userspace fork() is clone (220) / clone3 (435)absent
execve (221) / execveat (281)absent — A3 maps freestanding PT_LOAD and rejects PT_INTERP. A9 FAT /hello is an EL1 load path.
wait4 (260) / waitid (95)absent — no child to wait for
exit (93) / exit_group (94)partial — SYS_EXIT 16 ends the standing EL0 trip
getpid / gettid / set_tid_address / killabsent — no pid, tid, or signals
clone + CLONE_NEW*never-per-ADR-031 (namespaces; see ADR-029)

Track A first cuts that this mile must not rewrite:

  • Standing EL0 as normal mode for one loaded image (ADR-024).
  • Freestanding ELF64 PT_LOAD loader (ADR-023).
  • EL1 cooperative yield of two heap-backed workers (ADR-010).
  • Tiny SVC ABI (exit / uart_write / yield, then memfs 19–23) (ADR-021).

This ADR does not implement B4–B6. It does not add fork / clone / execve / wait* to src/. It does not mint FR-16+ or NFR-15+. File presence of this page is not a Linux process table.

Hypotheses

IDOptionNotes
H1 (chosen)Keep standing EL0 (one loaded trip) plus EL1 cooperative yield as the ctos process model. Document Linux fork / exec / wait as a gap. Do not add those syscalls. Do not rewrite A1–A9 probes. Any Linux-shaped process table is a later epic after B6 names a path, or never.Matches #43 (“decide approach without abandoning probes”) and ADR-031. Leaves B4–B6 room.
H2 (rejected)Promise Linux fork / execve / wait, a pid table, or “runs a shell.”Honesty fail (NFR-06). No probe exists.
H3 (rejected)Treat A4 standing EL0 or A9 FAT /hello as execve.The loader is EL1-driven. It rejects PT_INTERP. SYS_EXIT returns to the EL1 caller, not a parent pid.
H4 (rejected)Treat M9 EL1 coop workers as Linux threads / clone(CLONE_THREAD).Those tasks are kernel EL1. SYS_YIELD does not call sched::yield_now.
H5 (rejected)Add clone / wait4 as Planned implementation on this track, or reopen CLONE_NEW*.Would turn a research compare into a backlog. Namespace flags stay never-per-ADR-031. B6 may still choose never.

Decision

  1. ctos process model stays standing EL0. One freestanding image is loaded by EL1 (A3 / A9), installed as a standing task (install_task), and ERETed. It runs at EL0 until SYS_EXIT or an unexpected lower-EL sync. Restore is fail-closed (el0: task-restored / el0: restore-fail). is_active() / is_task() are true only for that lifetime. There is no process table, pid, ppid, tid, or parent/child wait queue.

  2. EL1 cooperative yield stays a different machine. ADR-010 switches two heap-backed EL1 workers. It is not preemptive, not SMP, and not EL0. SYS_YIELD 18 is a dispatch-only hint that returns to EL0. Do not describe those workers as Linux threads.

  3. Linux fork / exec / wait is a documented gap, not a present analog.

    Linux ideaWhat it needsWhat ctos has
    fork / cloneChild pid, optional new mm, COW or share, parent continuesOne standing context. Shared user TTBR0 window. No child.
    execveReplace the current image; keep pid / fds / cwd; load interpreter via PT_INTERP / auxvEL1 maps a freestanding ET_EXEC and ERETs. Rejects PT_INTERP. No fd table to keep. B4 is the ELF/auxv compare.
    wait / waitidParent blocks or polls; reaps a child statusSYS_EXIT returns to the EL1 hello path that launched the trip. No waiter.
    exit vs exit_groupThread vs process-group teardownOne trip ends. No thread group. musl/glibc CRT usually hits exit_group.
    Signals / killDelivery + waitidLower-EL IRQ still parks. No POSIX signals.

    On Linux AArch64, say clone + execve + wait4, not a fork syscall. Userspace fork() is a libc wrapper.

  4. Goals (research stance only).

    GoalWhy
    Name the two models so later children cannot blur themHonesty (NFR-06). File presence is not execve.
    Keep A1–A9 serial markers and fail-closed smokeAntifragility (NFR-05). This mile must not retarget SVC 16–23 or drop el0: task-ok.
    Leave B6 free to choose compat layer vs reimplement vs neverADR-031. A process table is not pre-chosen.
    Keep CLONE_NEW* / mount / guest OCI outADR-029 / ADR-031 never.
  5. Non-goals. Same family as ADR-031:

    • Implementing clone / clone3 / execve / execveat / wait4 / waitid / getpid / kill in src/.
    • A pid / tid / process-group table.
    • COW address spaces, CLONE_VM / CLONE_THREAD / CLONE_FILES.
    • Treating A3 / A4 / A9 as Linux exec.
    • Treating SYS_EXIT as exit_group.
    • Full Linux ABI / POSIX product / glibc or musl ports / “runs a shell.”
    • Guest OCI / Docker / Kubernetes (ADR-029).
    • Replacing the freestanding Track A path.
    • B4 (ELF / auxv / PT_INTERP) and B5 (Linux VFS concepts) — those stay their own children.
    • New FR/NFR IDs. Frozen set stays fr-nfr.md.
    • Author self-merge (ADR-002).
  6. Approach if B6 later names a path (not a decision). A hint only:

    1. In-tree libctos apps need no Linux process model. Sequential standing trips (EL1 loads the next FAT image after exit) would still be ctos-native, not fork/wait.
    2. A static musl _start is not a small subset. Typical CRT wants set_tid_address, exit_group, and often a pid. clone/execve/wait4 are larger than convention translation + write on a console fd (B2’s smallest named slice).
    3. If B6 chooses a compat path, a later epic would still need: a process table, a child mm (copy or COW), a wait queue, and an EL0-driven replace-image path. That epic needs its own probes. It must not silently retarget today’s SYS_EXIT or drop standing-task markers.
    4. Never remains valid. Standing EL0 can stay the model indefinitely.
  7. Keep (ctos specificity). Same invariants as ADR-031: honesty, antifragility, security, performance, document-first / one-PR loops, freestanding Track A path, QEMU virt / AArch64 (ADR-003), ADR-gated EL0 / identity teardown. Do not claim “EL0 isolated,” “secure OS,” or a bench for a process table that does not exist.

  8. Not claiming Linux userspace yet. Say that sentence on the Track B page. The ledger row “Guest runs host apps (Linux ELF / shell / Python)” stays Planned. This ADR does not change that row to Verified.

flowchart TD
  LNX["Linux: clone then execve then wait4"]
  CTOS["ctos: EL1 load + standing EL0 until exit"]
  LNX -->|"needs pid, child mm, wait queue"| GAP["Documented gap — not present"]
  CTOS -->|"A3 / A4 / A9 probes stay"| KEEP["Keep — do not rewrite"]
  GAP -.->|"B6 may choose never"| KEEP

Consequences

  • Docs: this ADR. track-b.md B3 becomes Documented. SUMMARY.md and roadmap.md link here. linux-aarch64-syscall-gap.md points process rows at this file.
  • B4 is Documented on main (ADR-033). B5 / B6 stay Planned research. No src/ change in this PR.
  • Track A remains the product path for in-tree freestanding apps. Standing EL0 is not a Linux process. “App hosting is done” stays unclaimed.
  • Honesty: document inspection only. Do not add a Verified Linux-process or execve ledger row because this file exists.

ADR-036 — Track B decision: compat layer vs reimplement vs never

  • Status: Accepted (never — research ladder complete; no Linux-compat implementation track)
  • Date: 2026-09-12

Context

Track B child B6 / #46 (parent #40) asks for an explicit decision after the research ladder:

  1. thin Linux-compat layer on top of ctos,
  2. reimplement Linux-like surfaces in ctos identity, or
  3. never (stop at research docs).

Frame and evidence already on main:

MileArtifactHonest readout
B1ADR-031ABI subset research only; not claiming Linux userspace; containers stay ADR-029
B2linux-aarch64-syscall-gap.mdNo Linux syscall is present (convention + number space miss); Track A SVCs are partial
B3ADR-035Standing EL0 ≠ clone/execve/wait4; do not add those syscalls
B4ADR-033Freestanding loader stays; PT_INTERP stay-rejected; dynamic musl/glibc not a small subset
B5ADR-034Thin VfsOps ≠ Linux VFS; mount/overlay never-per-ADR-031
B7ADR-029Guest OCI/Docker already never

Track A A1–A9 (+ leftovers ADR-032) is the probed freestanding product path. Core principles (principles.md) outrank Track B.

This mile is docs only. It does not change src/. File presence of this ADR is not a Linux userspace probe.

Hypotheses

IDOptionNotes
H1 (rejected)Thin Linux-compat layer (translate Linux AArch64 svc #0 / x8 onto ctos, grow auxv/PT_INTERP/process table behind a flag)B2–B5 show the gap is wide (syscalls, process model, dynamic ELF, VFS). A layer would become a second ABI to honesty-ledger and fail-closed smoke. No named probed workload justified the cost.
H2 (rejected)Reimplement Linux-like surfaces as native ctos (Linux numbers/names in src/, still “ctos”)Same gap surface as H1, plus a silent retarget of Track A SVC #n / libctos / reject-PT_INTERP markers. Conflicts with ADR-031 “keep freestanding Track A.”
H3 (chosen)Never — Track B stops as a completed research ladder. No Linux-compat implementation epic from these ADRs.Matches honesty (NFR-06), antifragility (do not grow an unprobed ABI), and “principles drive.” Leaves the door open only via a new epic + ADR with a named workload and probe — not by treating B2–B5 as a backlog.

Decision

  1. Choose never. ctos will not start a Linux-compat layer or a Linux-like reimplementation from Track B B1–B5. Track B’s job was to map the gap honestly. That job is done.

  2. What “never” means here.

    • Do not treat the B2 syscall table, B3 process gap, B4 ELF/auxv gap, or B5 VFS compare as an implementation backlog.
    • Do not accept PT_INTERP, Linux x8 numbers, clone/execve/wait*, POSIX VFS, or guest containers “later on Track B.”
    • Do not change the ledger row “Guest runs host apps (Linux ELF / shell / Python)” from Planned to a softer promise. It stays a distant possible future only if a new epic says so with a probe.
    • Track A freestanding apps (SVC / libctos / PT_LOAD / standing EL0 / thin VFS / FAT slots) remain the supported path.
  3. What remains allowed.

    • Keep and cite B1–B5 / B7 docs as the honesty record of why Linux userspace is out.
    • Learn from Linux ideas in future freestanding miles only when they fit ctos pillars and get their own ADR + probe (not “compat”).
    • Reopen Linux-compat only with a new GitHub epic, a new ADR that names a concrete workload + probe, and an explicit override of this decision. Do not mint FR/NFR IDs here.
  4. Relationship to ADR-029 / ADR-031. Containers stay a non-goal. ADR-031’s frame (“B6 may choose never”) is exercised by this ADR. Principles still drive over tracks.

Consequences

  • track-b.md marks B6 Decided (never) and the research ladder complete.
  • Plan issue #40 may close once this ADR merges (children B1–B7 documented/decided).
  • No src/ change. No new FR/NFR IDs. No self-merge (ADR-002).
  • Future “runs musl/Alpine/Linux ELF” headlines without a new epic + probe are honesty failures.

Probe (what Verified means here)

ClaimProbeStatus
B1–B5 / B7 evidence cited above exists on this tipRead ADR-029/031/033/034/035 + B2 gap note + track-b on mainDocument inspection
This ADR records an explicit never decisionRead this file’s Decision sectionDocument inspection
Linux userspace / compat layer on the guestNo serial marker; not claimedNot claimed

ADR-037 — Identity .data / .bss / linker-stack tear after SP relocate

  • Status: Accepted (identity .data/.bss/linker stacks unmapped; heap still identity-mapped)
  • Date: 2026-09-12

Context

Track A / A5 continues identity teardown where safe. ADR-025 unmapped identity .rodata after a pointer rewrite. .data / .bss / linker stacks / heap stayed mapped: SP was still the identity __stack_top, page-table software walked tables via identity PAs that live in .bss, and GlobalAlloc still returned identity frame VAs.

After the ADR-019 high-VA jump, PC-relative accesses (ADRP) already reach the TTBR1 alias of statics. Absolute words that point into .data/stacks stay identity until rewritten. SP and SP_EL1 must move to the high twin of the same physical pages before those identity pages can be unmapped. Heap tear needs a high allocator and is not this cut.

Hypotheses

IDOptionNotes
H1 (chosen)Rewrite identity pointers into .data/stacks, relocate SP/SP_EL1 to high twins, switch page-table CPU accessors to the high twin, then unmap identity [__data_start, __kernel_end) from kernel+user TTBR0. High twins stay. Heap stays.Largest honest cut that does not change GlobalAlloc.
H2 (rejected)Also unmap identity heap.Allocator still returns identity VAs. A silent yank would fault.
H3 (rejected)Claim PAN / “EL0 isolated” / “kernel moved” while tearing .data.Isolation is not this mile. PAN is ADR-026. _start stays at 0x4008_0000.
H4 (rejected)Unmap .data without relocating SP or fixing PA-as-pointer table walks.Would fault on the next store or l3_write.

Decision

  1. Rewrite identity .data pointers first. After the live .text tear and before the .rodata tear (so identity .rodata is still walkable), walk .rodata and [__data_start, __kernel_end). Every 8-byte-aligned word that is an identity address in that data/stack range is rewritten to its TTBR1 alias. Valid page-table descriptors are excluded (low bits set). Serial ident: data-reloc n=N. N == 0 is allowed.
  2. Keep the ADR-025 .rodata cut. ident: ro-reloc / ident: rodata unchanged.
  3. Relocate SP. Move the current SP (SP_EL0) and SP_EL1 to to_high_va of the same physical stack pages. High guard holes already exist in L2_HIGH_RAM (cloned after identity guards). Serial path continues only if SP is high.
  4. Page-table CPU VA. After the high split, desc_at / l3_write / flag publish-load use the TTBR1 twin of table/flag PAs so identity .bss can be torn while the hardware walk still uses PAs.
  5. Unmap identity .data/.bss/linker stacks. Unmap [__data_start, page_align_up(__kernel_end)) from kernel and user TTBR0 (TLBI VAAE1 each low VA). High twins stay. Serial ident: data lo=… hi=… pages=N with N >= 1.
  6. Keep heap identity-mapped. Frames at/after __kernel_end stay. Print ident: heap-stay. Do not claim a high allocator.
  7. Keep _start / QEMU -kernel at 0x4008_0000. Do not change default -cpu. Do not claim PAN.
  8. Fail-closed probe. New ident: data-reloc / ident: data / ident: data-fault / ident: data-high. EL1 LDR of a torn identity .data VA is a current-EL translation DABORT. EL1 LDR of the high twin still returns a known .data magic. scripts/qemu-smoke.sh greps those strings and rejects missed markers. Replace ident: data-stay. Split #[test_case] into heap-still-mapped vs data-torn.
  9. Still Planned (at accept time). Identity heap tear (high allocator VAs). Superseded by ADR-038. PAN enable on -cpu cortex-a57; EL0 entry without TLBI VMALLE1; lower-EL IRQ while standing; umbrella EL0 isolation.
  10. NFR-10 text is revised in place (ID unchanged). Do not mint NFR-15+.

Honesty

Say “identity .data/.bss/linker stacks were unmapped while the high twin and heap stayed” only when the serial / tests pass. Do not say:

  • the kernel has moved to the high half
  • identity mappings were fully torn down (heap stays; _start stays)
  • “secure OS” / “hardened” / “EL0 isolated”
  • PAN (see ADR-026; unimplemented on -cpu cortex-a57)

Consequences

  • paging::rewrite_identity_data_ptrs / paging::relocate_stacks_high / paging::tear_identity_data own the cut. ADR-025 remains the .rodata tear. ADR-032 stay markers for .data are superseded by this tear; heap stay markers remain.
  • ADR-038 returns high heap VAs and unmaps identity heap. That work is not this cut.

ADR-038 — Identity heap tear after high GlobalAlloc VAs

  • Status: Accepted (identity heap unmapped; GlobalAlloc returns TTBR1 VAs) when the serial / tests pass
  • Date: 2026-09-12

Context

Track A / A5 continues identity teardown where safe. ADR-037 unmapped identity .data/.bss/linker stacks after SP relocate. Heap stayed identity-mapped: frame::alloc_contiguous handed out PAs and GlobalAlloc returned those identity VAs (ident: heap-stay).

Heap init already runs after the high-VA jump and the .data tear. High twins of the frame pool exist (L2_HIGH_RAM clone). The next honest cut is to install the first-fit pool at the TTBR1 alias, rewrite leftover identity heap pointers, then unmap the identity heap range. PAN enable and umbrella EL0 isolation are not this cut.

Hypotheses

IDOptionNotes
H1 (chosen)Install the heap free list at to_high_va(pa), rewrite leftover identity heap words, then unmap identity [heap_pa, heap_pa+HEAP_SIZE) from kernel+user TTBR0. High twins stay. Remaining frames after the heap stay identity-mapped.Smallest honest tear that does not yank live identity VAs.
H2 (rejected)Unmap identity heap while GlobalAlloc still returns identity VAs.Silent yank would fault.
H3 (rejected)Claim PAN / “EL0 isolated” / “kernel moved” while tearing the heap.Isolation is not this mile. PAN is ADR-026. _start stays at 0x4008_0000.
H4 (rejected)Remap only new allocations high and leave the existing identity pool mapped.Would print a stay marker forever.

Decision

  1. High allocator first. After frame::init, heap::init takes the same 16-frame run and installs the free list at the TTBR1 alias. GlobalAlloc returns high VAs. Serial later prints ident: heap-reloc n=N. N == 0 is allowed (init already high).
  2. Rewrite leftover identity heap pointers. Walk the heap pool via the high twin (do not walk live stacks or page-table .bss). Every 8-byte-aligned word that is an identity address in the heap PA range is rewritten to its TTBR1 alias. heap::init already installed high VAs, so n == 0 is the expected path. A leaked Box plants ident magic for the high-twin load.
  3. Unmap identity heap. Unmap [heap_pa, heap_pa+HEAP_SIZE) from kernel TTBR0 (TLBI VAAE1 each low VA). User TTBR0 already omits heap; clearing a present slot is fine. High twins stay. Frames after the heap stay identity-mapped. Serial ident: heap lo=… hi=… pages=N with N >= 1.
  4. Keep _start / QEMU -kernel at 0x4008_0000. Do not change default -cpu. Do not claim PAN.
  5. Fail-closed probe. New ident: heap-reloc / ident: heap / ident: heap-fault / ident: heap-high. EL1 LDR of a torn identity heap VA is a current-EL translation DABORT. EL1 LDR of the high twin still returns the planted free-list header size. scripts/qemu-smoke.sh greps those strings and rejects ident: heap-stay. Replace the ADR-032 / ADR-037 stay marker.
  6. Still Planned (at accept time). PAN enable on -cpu cortex-a57; EL0 entry without TLBI VMALLE1; lower-EL IRQ while standing; umbrella EL0 isolation. Remaining identity RAM after the heap (frame bump pool) is not this cut. EL0 entry without VMALLE1 is superseded by ADR-039.
  7. NFR-10 text is revised in place (ID unchanged). Threat-model v1.18 at accept (v1.19 after ADR-039). Do not mint NFR-15+.

Honesty

Say “identity heap was unmapped while the high twin stayed and GlobalAlloc returned TTBR1 VAs” only when the serial / tests pass. Do not say:

  • the kernel has moved to the high half
  • identity mappings were fully torn down (_start stays; frame pool after the heap stays identity-mapped)
  • “secure OS” / “hardened” / “EL0 isolated”
  • PAN (see ADR-026; unimplemented on -cpu cortex-a57)

Consequences

  • heap::init / paging::rewrite_identity_heap_ptrs / paging::tear_identity_heap own the cut. ADR-009 stays the first-fit algorithm; VAs are now high. ADR-037 stay marker for the heap is superseded.
  • A later ADR may tear remaining identity frames or enable PAN. That work is not this cut.

ADR-039 — EL0 entry / ASID switch without TLBI VMALLE1

  • Status: Accepted (standing/EL0 trampoline path is MSR TTBR0 + ISB only)
  • Date: 2026-09-12

Context

Track A / A5 isolation leftovers. Dual-ASID EL1 switches already avoided TLBI VMALLE1 (ADR-013 P-SEC-3c). The standing/EL0 trampoline still flushed the whole EL1 TLB on every ERET to EL0, every stay-at-EL0 after a standing SVC, and every lower-EL sync restore of kernel TTBR0 — because kernel .data leaves were global (nG=0). Dropping that flush while identity .data stayed mapped would leak a cached translation into the user ASID and break el0: no kernel read.

ADR-037 and ADR-038 unmap identity .data/.bss/stacks and the identity heap (TLBI VAAE1 each low VA). EL0 probes run after those tears. The historical global-.data reason for trampoline VMALLE1 is gone for those ranges. PAN enable and umbrella EL0 isolation are not this cut.

Hypotheses

IDOptionNotes
H1 (chosen)Remove TLBI VMALLE1 from eret_to_el0, stay_at_el0, and sync_lower_el TTBR0 restore. Keep MSR TTBR0 + ISB. Keep TLBI VMALLE1 only at MMU enable (paging::init) and keep TLBI VAAE1 on unmap/setup.Smallest honest mile after ADR-037/038.
H2 (rejected)Mark all kernel leaves nG and keep a full flush “just in case.”Larger surface; not required once identity .data/heap are torn + VAAE1’d.
H3 (rejected)Claim “EL0 isolated” / PAN / “kernel moved” because the trampoline no longer full-flushes.Isolation umbrella stays Planned. PAN is ADR-026.
H4 (fallback)If H1 regresses el0: no kernel read / standing / task smoke, ship a probe documenting where VMALLE1 still fires and why — do not regress smoke.Honesty over the mile.

Decision

  1. Trampoline path without VMALLE1. exception::eret_to_el0, stay_at_el0, and sync_lower_el (kernel TTBR0 restore from TPIDR_EL1) use MSR TTBR0_EL1 + ISB only.
  2. What TLBI remains. paging::init still TLBI VMALLE1 once at MMU enable. Unmaps / map-window / identity tears still TLBI VAAE1 (or equivalent per-VA invalidate). ASID dual probe stays no-full-flush (ADR-013).
  3. Fail-closed probe. Serial el0: no-vmalle1 after the existing EL0 first-mile + read + standing markers, only when identity .data and heap tears are ready. scripts/qemu-smoke.sh greps that string and rejects tlbi vmalle1 in src/exception.rs. #[test_case] el0_entry_without_vmalle1.
  4. Keep _start / QEMU -kernel at 0x4008_0000. Do not change default -cpu. Do not claim PAN.
  5. Still Planned. PAN enable on -cpu cortex-a57; remaining identity RAM / _start teardown; umbrella EL0 isolation. Lower-EL IRQ while standing is ADR-040.
  6. NFR-10 text is revised in place (ID unchanged). Threat-model v1.19. Do not mint NFR-15+.

Honesty

Say “standing/EL0 trampoline switches TTBR0 without TLBI VMALLE1 after identity .data/heap tear” only when the serial / tests pass. Do not say:

  • “EL0 isolated”
  • PAN (see ADR-026; unimplemented on -cpu cortex-a57)
  • the kernel has moved / identity mappings were fully torn down (_start and leftover identity frames stay)
  • “secure OS” / “hardened”

Consequences

  • src/exception.rs owns the trampoline TLBI drop. ADR-013 remains the isolation direction; this ADR is the EL0-entry flush cut.
  • A later ADR may enable PAN or tear remaining identity RAM. That work is not this cut.

ADR-040 — Lower-EL IRQ while standing EL0

  • Status: Accepted (taken lower-EL IRQ while standing is a supported/probed path when the serial / tests pass; FIQ/SError lower-EL still park; umbrella isolation Planned)
  • Date: 2026-09-13

Context

Track A / A5 isolation leftovers. Lower-EL AArch64 sync has long been live for SVC / IABORT / DABORT. The lower-EL IRQ / FIQ / SError vector slots historically parked (ADR-004). Standing EL0 (ADR-013 / ADR-024) entered with DAIF IRQ masked in SPSR, so a timer IRQ from EL0 was never taken.

After ADR-039, the standing/EL0 trampoline restores kernel TTBR0 with MSR TTBR0 + ISB only. The same restore is required on a lower-EL IRQ path before GIC / .data access (user TTBR0 omits MMIO and kernel data).

PAN enable and umbrella EL0 isolation are not this cut. Do not switch default -cpu cortex-a57. Keep _start at 0x4008_0000.

Hypotheses

IDOptionNotes
H1 (chosen)Wire lower-EL IRQ (irq_lower_el): restore kernel TTBR0, gic::handle_irq, if el0::is_active() put user TTBR0 back and ERET to EL0. Probe: standing dual-SVC with WFI and SPSR IRQ-unmasked; serial el0: irq. Default ERET to EL0 stays IRQ-masked. FIQ/SError stay parked.Largest honest first cut that does not race existing EL0 probes.
H2 (rejected)Unmask IRQ on every standing/task ERET and claim “IRQ while standing is normal for all tasks.”Larger surface; timer mid-task may interact with A4/A1–A3 probes before this mile is proven.
H3 (rejected)Claim “EL0 isolated” / PAN / “secure OS” because IRQ returns to EL0.Isolation umbrella stays Planned.
H4 (fallback)If H1 regresses smoke, ship a scaffold documenting the park and why — do not regress smoke.Honesty over the mile.

Decision

  1. Lower-EL IRQ live while standing. Vector slot 0x480 calls irq_lower_el (same frame + TTBR0 restore as sync_lower_el). handle_irq_lower_el runs gic::handle_irq, then stay_at_el0 when is_active(). Non-standing lower-EL IRQ still fails closed (park).
  2. Probe. Hello path: arm standing + expect IRQ, timer::arm_soon, ERET with SPSR 0x340 (EL0t, I clear), payload SVC #1 / WFI / MOVZ / SVC #2. Serial el0: irq when the IRQ was taken while standing. Existing el0: standing / el0: restored still print on that trip.
  3. Default ERET stays masked. eret_to_el0 keeps SPSR 0x3c0. Only the IRQ probe uses eret_to_el0_irq_enabled. Enabling IRQ for all standing tasks is a later cut.
  4. Fail-closed sensors. scripts/qemu-smoke.sh greps el0: irq. #[test_case] lower_el_irq_while_standing.
  5. Still Planned. Lower-EL FIQ/SError; IRQ-unmasked default standing/task ERET; PAN enable on -cpu cortex-a57; remaining identity RAM / _start teardown; umbrella EL0 isolation.
  6. NFR-10 text is revised in place (ID unchanged). Threat-model v1.20. Do not mint NFR-15+.

Honesty

Say “taken lower-EL IRQ while standing returned to EL0” only when the serial / tests pass. Do not say:

  • “EL0 isolated”
  • PAN (see ADR-026)
  • lower-EL FIQ/SError are handled
  • every standing task takes IRQs by default (default ERET still masks I)
  • “secure OS” / “hardened”

Consequences

  • src/exception.rs owns irq_lower_el / handle_irq_lower_el. src/el0.rs owns the WFI standing probe. src/timer.rs owns arm_soon.
  • ADR-013 remains the isolation direction; this ADR is the lower-EL IRQ-while-standing cut.
  • A later ADR may unmask IRQ for standing tasks by default, or take FIQ/SError. That work is not this cut.

ADR-041 — IRQ-unmasked default standing/task ERET

  • Status: Accepted (default ERET to EL0 clears SPSR.I when the serial / tests pass; short non-standing trampoline probes stay masked; FIQ/SError lower-EL still park; umbrella isolation Planned)
  • Date: 2026-09-13

Context

ADR-040 made taken lower-EL IRQ while standing a live path, but only via a dedicated probe SPSR (eret_to_el0_irq_enabled, I clear). Default eret_to_el0 still used SPSR 0x3c0 (I set), so standing/task entry remained IRQ-masked unless that special trampoline was used.

This cut makes standing EL0 and the loaded-task / standing-payload paths enter with IRQ unmasked by default, and proves a timer tick can arrive without the special probe helper. PAN enable and umbrella EL0 isolation are not this cut. Do not switch default -cpu cortex-a57. Keep _start at 0x4008_0000.

Hypotheses

IDOptionNotes
H1 (chosen)Default eret_to_el0 uses SPSR 0x340 (EL0t, I clear). Short non-standing trampoline probes (SVC / IABORT / DABORT without is_active()) use eret_to_el0_masked. Standing IRQ probe uses default eret_to_el0 + WFI + arm_soon (no special SPSR helper). Serial el0: irq + el0: irq-default. Task/loader/syscall/libctos standing paths inherit the default. FIQ/SError stay parked.Largest honest cut that matches the deferred ADR-040 item.
H2 (rejected)Unmask every ERET including short non-standing probes and claim “IRQ mid-SVC is always fine.”Non-standing lower-EL IRQ still parks (handle_irq_lower_el); a mid-probe tick would fail-closed.
H3 (rejected)Claim “EL0 isolated” / PAN / “secure OS” because default SPSR clears I.Isolation umbrella stays Planned.
H4 (fallback)If H1 regresses smoke, keep default masked and document the park — do not regress smoke.Honesty over the mile.

Decision

  1. Default ERET IRQ-unmasked. eret_to_el0 writes SPSR 0x340. eret_to_el0_irq_enabled becomes an alias (or is removed). eret_to_el0_masked keeps SPSR 0x3c0 for short non-standing probes.
  2. Probe. Hello path: same standing dual-SVC + WFI as ADR-040, but enter via default eret_to_el0. Serial still el0: irq; also print el0: irq-default after that trip succeeds. Existing el0: standing / el0: restored still print.
  3. Task path. Loaded standing task / A1–A3 standing payloads use eret_to_el0 and therefore inherit I-clear. No separate task WFI probe required for this cut; IRQ while is_active() already returns via stay_at_el0.
  4. Fail-closed sensors. scripts/qemu-smoke.sh greps el0: irq-default. #[test_case] lower_el_irq_default_eret (or extend lower_el_irq_while_standing to assert default path).
  5. Still Planned. Lower-EL FIQ/SError; PAN enable on -cpu cortex-a57; remaining identity RAM / _start teardown; umbrella EL0 isolation.
  6. NFR-10 text is revised in place (ID unchanged). Threat-model v1.21. Do not mint NFR-15+.

Honesty

Say “default standing/task ERET enters with IRQ unmasked” only when the serial / tests pass. Do not say:

  • “EL0 isolated”
  • PAN (see ADR-026)
  • lower-EL FIQ/SError are handled
  • short non-standing trampoline probes take IRQs (those stay masked)
  • “secure OS” / “hardened”

Consequences

  • src/exception.rs owns SPSR defaults. src/el0.rs owns the default-path IRQ probe. Short probes in el0 / ttbr1 / teardown call the masked helper.
  • ADR-013 remains the isolation direction; this ADR closes the ADR-040 “default ERET still masks I” park.
  • A later ADR may take FIQ/SError or enable PAN. That work is not this cut.

ADR-042 — Isolation leftover honesty wrap (start-stay + status)

  • Status: Accepted (serial / tests prove _start stays while live image + heap are torn; umbrella isolation stays Planned)
  • Date: 2026-09-13
  • Follow-on (2026-09-26): ident: ram-stay was superseded by ADR-049. The remaining identity (low RAM, padding tail, pre-heap frame, MMIO) was torn by ADR-087 / ADR-088. ident: start-stay remains, and the stub page is the one documented identity exception (ADR-089, sponsor D2).

Context

Track A / A5 closed a deep isolation ladder: identity .data (ADR-037), heap (ADR-038), EL0 trampoline without TLBI VMALLE1 (ADR-039), lower-EL IRQ while standing (ADR-040), and IRQ-unmasked default standing/task ERET (ADR-041). Former stay markers (ident: data-stay / ident: heap-stay) were superseded by those tears.

What remains Planned must stay fail-closed in the ledger and threat model: PAN enable on -cpu cortex-a57, lower-EL FIQ/SError, full identity teardown including _start, and the umbrella “EL0 isolated” row. QEMU -kernel still needs _start at 0x4008_0000 — do not yank it. This ADR is an honesty wrap of those leftovers, not a claim that isolation closed.

Hypotheses

IDOptionNotes
H1 (chosen)Print ident: start-stay (and ident: ram-stay for leftover identity frames after the heap) + #[test_case] that 0x4008_0000 stays mapped while live .text / .rodata / .data / heap are torn. Document Verified ladder ADR-037…041 vs still-Planned leftovers in this ADR; bump threat-model; keep P-SEC-3l Planned.Honest wrap. Do not switch -cpu. Do not claim “EL0 isolated.”
H2 (rejected)Unmap _start / claim “kernel moved” / claim umbrella isolation because the ladder exists.Would break -kernel and round up.
H3 (rejected)Switch default -cpu to enable PAN and call isolation done.Forbidden by ADR-026 / roadmap.
H4 (deferred)Take lower-EL FIQ/SError while standing.Not cheap on this wrap; leave Planned.

Decision

  1. Start-stay marker. After the ADR-038 heap tear proves live image ranges are unmapped, assert identity _start (KERNEL_TEXT / 0x4008_0000) is still mapped+executable and print ident: start-stay lo=… hi=….
  2. Remaining-identity-RAM note. Print ident: ram-stay lo=… hi=… for the still-mapped identity frame bump after the torn heap (heap_pa_end…pool_end).
  3. Fail-closed probe. scripts/qemu-smoke.sh greps ident: start-stay and ident: ram-stay. #[test_case] identity_boot_stub_stays_while_live_torn.
  4. Status table (this wrap).
ItemStatusNotes
ADR-037 identity .data/stacks tearVerifiedident: data*
ADR-038 identity heap tearVerifiedident: heap*
ADR-039 EL0 entry without VMALLE1Verifiedel0: no-vmalle1
ADR-040 lower-EL IRQ while standingVerifiedel0: irq
ADR-041 IRQ-unmasked default ERETVerifiedel0: irq-default
_start stay at 0x4008_0000Verified (honesty)ident: start-stay; not a teardown
Remaining identity RAM after heapVerified (honesty)ident: ram-stay; tear still Planned
PAN enable on -cpu cortex-a57Plannedpan: absent (ADR-026). Do not switch -cpu.
Lower-EL FIQ/SError while standingSuperseded by ADR-043FIQ Verified / SError park honesty on ADR-043.
Full identity teardown including _startPlannedDo not yank _start for QEMU -kernel.
Umbrella “EL0 isolated” (P-SEC-3l)PlannedLadder miles are not this row.
  1. Docs. Update el0.md, security.md (threat-model v1.22), honesty ledger, roadmap P-SEC-3l, NFR-10 text in place. Do not mint NFR-15+.
  2. Keep _start at 0x4008_0000. Do not change default -cpu. Do not claim PAN or “EL0 isolated.”

Honesty

Say “_start / boot-stub page stays mapped while live .text/.rodata/.data/heap are torn” only when the serial / tests pass. Do not say:

  • “EL0 isolated”
  • the kernel has moved to the high half
  • identity mappings were fully torn down
  • PAN enable / lower-EL FIQ/SError are handled
  • “secure OS” / “hardened”

Consequences

  • src/teardown.rs prints the stay markers; smoke and the new #[test_case] ratchet them.
  • ADR-013 remains the isolation direction; P-SEC-3l stays Planned.
  • ADR-043 takes FIQ and documents SError park honesty. ADR-047 decides: never yank _start while -kernel needs it; PAN enable non-goal on default a57; taken SError deferred/non-goal on this smoke machine; leftover RAM optional; umbrella stays Planned/non-claim. A later ADR may reopen with sponsor scope (honest inject, new CPU, boot redesign).

ADR-043 — Lower-EL FIQ while standing (+ SError park honesty)

  • Status: Accepted (taken lower-EL FIQ while standing is a supported/probed path when the serial / tests pass; SError has no safe trigger on this virt/cortex-a57 cut — park honesty; umbrella isolation Planned)
  • Date: 2026-09-13

Context

ADR-040 / ADR-041 made taken lower-EL IRQ while standing a live path. Lower-EL FIQ and SError still parked (ADR-004 / ADR-042 H4).

On QEMU virt + GICv2 + -cpu cortex-a57, Group 0 interrupts signal as IRQ by default (GICC_CTLR.FIQEn clear). Setting FIQEn routes Group 0 (including the CNTP PPI) as FIQ without changing the default -cpu. True asynchronous SError has no safe, deterministic inject on this guest without exotic QEMU/monitor setup — do not fake a taken path.

PAN enable and umbrella EL0 isolation are not this cut. Do not switch default -cpu cortex-a57. Keep _start at 0x4008_0000.

Hypotheses

IDOptionNotes
H1 (chosen)Wire fiq_lower_el (same TTBR0 restore as IRQ). Probe: temporarily GICC_CTLR.FIQEn, standing dual-SVC + WFI, SPSR 0x380 (F clear, I set), serial el0: fiq; restore Group-0 → IRQ after the trip. SError: wire a defensive standing-aware stub + print el0: serror-park (Planned / no safe trigger).Largest honest first cut mirroring IRQ.
H2 (rejected)Claim both FIQ and SError are Verified taken paths without a safe SError inject.Would round up.
H3 (rejected)Switch -cpu / enable PAN / claim “EL0 isolated” because FIQ returns to EL0.Forbidden.
H4 (fallback)If FIQEn does not deliver a lower-EL FIQ on this virt guest, ship scaffold + park markers and document why — do not fake Verified.Honesty over the mile.

Decision

  1. Lower-EL FIQ live while standing. Vector slot 0x500 calls fiq_lower_el. handle_fiq_lower_el runs gic::handle_fiq (same IAR/EOI as IRQ for Group 0), then stay_at_el0 when is_active(). Non-standing lower-EL FIQ still fails closed (park).
  2. Probe. Hello path: gic::route_group0_as_fiq, timer::arm_soon, eret_to_el0_fiq (SPSR 0x380), payload SVC #1 / WFI / MOVZ / SVC #2. Serial el0: fiq. Always gic::route_group0_as_irq after the trip so the IRQ path stays default.
  3. SError honesty. Vector slot 0x580 calls serror_lower_el (defensive: if standing, stay at EL0 and print el0: serror). No taken-path probe on this cut — hello prints el0: serror-park. Taken SError stays Planned.
  4. Fail-closed sensors. scripts/qemu-smoke.sh greps el0: fiq and el0: serror-park. #[test_case] lower_el_fiq_while_standing.
  5. Still Planned. Taken lower-EL SError; PAN enable on -cpu cortex-a57; remaining identity RAM / yank _start; umbrella EL0 isolation.
  6. NFR-10 text is revised in place (ID unchanged). Threat-model v1.23. Do not mint NFR-15+.

Honesty

Say “taken lower-EL FIQ while standing returned to EL0” only when the serial / tests pass. Do not say:

  • “EL0 isolated”
  • PAN (see ADR-026)
  • lower-EL SError is taken / Verified on this guest
  • default standing/task ERET unmasks FIQ (default SPSR still sets F; only the FIQ probe clears F and arms FIQEn)
  • “secure OS” / “hardened”

Consequences

  • src/gic.rs owns FIQEn route helpers. src/exception.rs owns fiq_lower_el / serror_lower_el. src/el0.rs owns the standing FIQ probe and the SError park marker.
  • ADR-013 remains the isolation direction; this ADR closes the FIQ half of ADR-042 H4.
  • A later ADR may take SError safely or enable PAN on a different CPU story. That work is not this cut.

ADR-044 — Taken lower-EL SError research (no Verified taken path yet)

  • Status: Accepted as research — taken lower-EL SError while standing stays Planned; el0: serror-park remains the Verified honesty marker (ADR-043)
  • Date: 2026-09-13

Context

ADR-043 wired a defensive standing-aware serror_lower_el stub and prints el0: serror-park because this cut had no safe, deterministic in-guest inject on QEMU virt + TCG + -cpu cortex-a57. Sponsor asked for a research mile (Grok/docs) before claiming a taken path.

Default standing/task SPSR still masks SError (PSTATE.A set; ADR-041 SPSR 0x340 is D+A+F with I clear). A taken probe must clear A (and keep the guest standing) the same way the FIQ probe clears F.

Do not switch default -cpu. Do not claim “EL0 isolated.” Keep _start at 0x4008_0000.

Options surveyed

IDOptionFeasibility on ctos smokeHonesty
H1 (recommended next mile)Host-driven inject via QEMU HMP/QMP nmi, which on virt raises the CPU SError line (ARM_CPU_SERROR / TCG CPU_INTERRUPT_SERROR). Run smoke with a QMP socket; while standing EL0 (A clear + WFI), issue {"execute":"inject-nmi"} / HMP nmi. Serial el0: serror (taken) + existing park marker only when not probing.Feasible with scripted QMP; needs smoke harness change (-qmp / unix socket), timing race around standing window, and fail-closed if QMP unavailable. Not an in-guest SVC.Honest: host inject is how virt simulates NMI→SError.
H2 (rejected for Verified)Guest-only RAS / imprecise abort / poisoned load to force async SError.Unreliable on -cpu cortex-a57 TCG virt without RAS story; easy to confuse with sync aborts.Would round up.
H3 (rejected)Treat a sync abort / BRK / FIQ as “SError.”Easy, wrong EC.Forbidden.
H4 (status quo)Keep el0: serror-park Verified; taken path Planned until H1 ships.Current main (65cf515).Correct until H1 probes.
H5 (out of scope)KVM KVM_SET_VCPU_EVENTS serror_pending or plugin arm_cpu_inject_exception.ctos smoke is TCG user-mode style -kernel on virt, not KVM guest tooling in CI.Useful notes only.

Vector / mask notes

  • Lower-EL AArch64 SError vector is VBAR_EL1 + 0x580 (already wired).
  • Taken SError requires PSTATE.A clear at EL0. Add eret_to_el0_serror (e.g. SPSR with A clear, I/F set as needed) for the probe only — do not unmask A on default standing ERET in the research mile.
  • After a taken trip, restore A-masked default so normal smoke stays stable.

Smoke harness sketch (not implemented here)

  1. qemu-system-aarch64 … -qmp unix:./qmp.sock,server,nowait (or stdio QMP in a side channel).
  2. Hello path: arm EXPECT_EL0_SERROR, eret_to_el0_serror, payload SVC #1 / short spin or WFI.
  3. Host: wait for a UART cue (or fixed delay after el0: standing), then QMP inject-nmi.
  4. Grep el0: serror (taken); reject claiming Verified if only park printed.
  5. GHA matrix must have a QMP-capable qemu-system-aarch64 (already true on ubuntu runners once flags are added).

Decision

  1. This ADR is research only. No taken-path Verified claim. No src/ change required to accept it.
  2. Recommended follow-up implementation is H1 (QMP/nmi while standing with A clear), as a separate ADR (e.g. ADR-045) with probes + qemu-smoke greps.
  3. Keep H4 until that follow-up lands: el0: serror-park stays Verified; taken lower-EL SError stays Planned.
  4. Still Planned (unchanged honesty): PAN enable on -cpu cortex-a57 (ADR-026); yank _start / leftover identity RAM (ADR-042); umbrella “EL0 isolated.”
  5. Threat-model text may note “taken SError research recorded; inject path identified as QMP nmi” without minting NFR-15+.

Honesty

Say “we know how we would take lower-EL SError on virt” (QMP nmi + A clear). Do not say:

  • taken SError is Verified on this guest
  • “EL0 isolated” / PAN / “secure OS”
  • default standing ERET unmasks SError
  • guest-only RAS inject works on cortex-a57 TCG

Consequences

  • ADR-043 remains the FIQ Verified + SError park cut.
  • Implementers should open a new ADR for H1, not widen this file into a silent Verified.
  • Follow-up: ADR-045 attempted H1; QEMU 10 virt+cortex-a57 has no working inject-nmi→SError. Taken stays Planned.
  • Docker/(amd64 workstation) smoke does not need re-probe for this docs-only research PR.

ADR-045 — Taken lower-EL SError via QMP/nmi (honest partial)

  • Status: Accepted as implementation attempt — guest A-clear standing path is wired; host QMP inject-nmi on QEMU 10 virt + -cpu cortex-a57 does not deliver SError (machine does not provide NMIs). Taken lower-EL SError while standing stays Planned. el0: serror-park remains the Verified honesty marker (ADR-043 / ADR-044).
  • Date: 2026-09-13

Context

ADR-044 recommended H1: host QMP/inject-nmi while standing with PSTATE.A clear, expecting virt → ARM_CPU_SERROR. That recommendation tracked a 2020 qemu-devel series; upstream QEMU 10 does not implement that path on this smoke machine:

  • inject-nmi / HMP nmi → machine does not provide NMIs (no TYPE_NMI on virt for cortex-a57 + default GICv2).
  • Modern virt “NMI” support is FEAT_NMI (GICv3 + a CPU with aa64_nmi), not async SError, and requires switching -cpu / GIC — forbidden here.
  • TCG has virtual SError (HCR_EL2.VSE / EXCP_VSERR) for EL2→EL1, not a host inject into our EL1/EL0 standing guest.
  • Guest-only RAS inject remains rejected for Verified (ADR-044 H2).

Do not switch default -cpu cortex-a57. Do not claim “EL0 isolated.” Keep _start at 0x4008_0000. Do not unmask A on default standing/task ERET (SPSR 0x340 keeps A set).

Decision

  1. Guest plumbing (shipped). eret_to_el0_serror uses SPSR 0x2c0 (D+I+F set, A clear). Hello arms EXPECT_EL0_SERROR, prints UART cue el0: serror-arm, enters standing dual-SVC with a bounded spin (not WFI — I/F stay masked, so a failed inject must not hang). Handler serror_lower_el prints el0: serror only when EXPECT is armed, then stay_at_el0.
  2. Host smoke (attempted). scripts/qemu-serial-inject.py opens -qmp unix:…,server,nowait and, after el0: serror-arm, issues {"execute":"inject-nmi"}, logging the result. On this QEMU it errors; guest prints el0: serror-park. qemu-smoke.sh still requires el0: serror-park and must not require el0: serror.
  3. No #[test_case] for the taken path — it needs a host inject. Smoke-only honesty.
  4. Taken path stays Planned. Do not mark Verified. Park marker stays Verified.
  5. Still Planned (unchanged honesty): PAN enable on -cpu cortex-a57 (ADR-026); yank _start / leftover identity RAM (ADR-042); umbrella “EL0 isolated.”
  6. NFR-10 text revised in place (ID unchanged). Threat-model v1.25. Do not mint NFR-15+.

Honesty

Say “guest A-clear standing SError path is wired; QMP inject-nmi does not take SError on virt/cortex-a57.” Do not say:

  • taken lower-EL SError is Verified on this guest
  • “EL0 isolated” / PAN / “secure OS”
  • default standing ERET unmasks SError
  • ADR-044 H1 landed as a working inject on QEMU 10
  • guest-only RAS inject works on cortex-a57 TCG

Follow-on

ADR-083 (2026-09-19): pinned QEMU opt-in Verifies taken el0: serror; stock park path unchanged.

Consequences

  • src/exception.rs / src/el0.rs own the A-clear probe + EXPECT. scripts/qemu-serial-inject.py owns the QMP attempt. el0: serror-park stays the fail-closed serial when inject does not land.
  • ADR-047 records the sponsor decision: taken path is deferred / non-goal on this smoke machine until an honest inject exists; park stays Verified; A-clear plumbing stays as dormant prep.
  • A later ADR may find a different honest inject (new QEMU feature, or a constrained CPU story with sponsor approval). That work is not this cut.
  • Hard-stop lock: ADR-053 (2026-09-14) re-probed inject-nmi / HMP nmi on QEMU 10.0.13 virt+cortex-a57 — same machine does not provide NMIs. Taken path is locked deferred/non-goal, not a fuzzy Planned mile.
  • Reopen re-probe: ADR-081 / ADR-082 (2026-09-19) on default -cpu cortex-a76 + GICv3/virtualization=on/max — still machine does not provide NMIs. Dormant prep unchanged.
  • Docker/(amd64 workstation) re-probe is optional; agent-box smoke must show park + QMP error log, not a fake el0: serror.

ADR-046 — A9 slot-disconnect performance delta (probe-only embed)

  • Status: Accepted (compared CNTPCT pair on one boot — QEMU TCG lab measurement, not a published bench).
  • Date: 2026-09-13

Context

A9 / ADR-030 prints perf: app-load ticks=<n> around the FAT /hello read + A3 map + ERET. ADR-032 removed the production A2–A4 / A9 include_bytes! path (FAT-only; smoke rejects slot: embed). The honesty ledger still had a Planned row: compare that FAT trip to the old embedded A3 trip on the same guest.

Do not restore production embed. Do not invent a percent or “faster/slower” claim. Do not switch -cpu. Keep _start at 0x4008_0000. QEMU TCG jitter is one lab — not criterion, not SPEC.

Hypotheses

IDOptionNotes
H1 (chosen)Probe-only include_bytes! of the same OUT_DIR/hello-libctos.elf in src/slot.rs. Time map+ERET with CNTPCT → perf: embed-load ticks=<n>. Keep FAT perf: app-load. Print perf: slot-delta app=<a> embed=<b> (raw ticks). Never print slot: embed.Honest compared pair. Production slot stays FAT-only.
H2 (rejected)Restore A3 embed as a production fallback when FAT is missing.Violates ADR-032. Smoke must keep rejecting slot: embed.
H3 (rejected)Close the Planned row as non-comparable (embed removed) without measuring.Prefer H1 when feasible. Docs-only fallback only if dual-path is unsafe.
H4 (rejected)Publish a percent / “slots are free” / criterion bench.NFR-07 forbids invented benches.

Decision

  1. Production path unchanged. A9 still loads FAT /hello only. Serial still prints slot: fat / slot: mapped / slot: ok / perf: app-load ticks=<n>. Missing /hello is still slot: probe missed. slot: embed must never appear.
  2. Probe-only embed. src/slot.rs may include_bytes!(concat!(env!("OUT_DIR"), "/hello-libctos.elf")) for measurement. Bytes must match FAT /hello / HELLO_ELF_LEN. A2–A4 (src/libctos.rs, src/loader.rs) must not embed.
  3. Markers. After a successful FAT trip on the same boot: perf: embed-load ticks=<n> then perf: slot-delta app=<a> embed=<b>. Fail closed on perf: embed-load missed. No ratio percent; no “faster/slower” words on the serial line.
  4. Smoke. scripts/qemu-smoke.sh greps the new markers, still rejects slot: embed, rejects A2/A3 hello include_bytes!, requires the slot.rs probe include, and rejects slot-delta lines that claim percent/faster/slower.
  5. Honesty. This is a measurement pair on QEMU virt TCG. It is not a latency budget, not a product KPI, and not “slots are free.” Record Verified only with serial evidence on a named tip.

Consequences

  • Code: src/slot.rs (probe include + markers); smoke greps; #[test_case] that embed CNTPCT advanced and bytes match FAT.
  • Docs: performance.md, measure.md, honesty ledger, track-a / roadmap as needed.
  • Kernel debug ELF grows by the hello ELF size (~8 KiB). That is expected for the probe.
  • Isolation leftovers (PAN enable, taken SError, yank _start, umbrella) stay Planned — unchanged by this mile.

ADR-047 — Isolation leftovers decisions (SError / PAN / _start / umbrella)

  • Status: Accepted (docs decision). Does not mint Verified for taken SError, PAN enable, yank _start, or “EL0 isolated.”
  • Date: 2026-09-13

Context

Track A / A5 closed a deep isolation ladder through ADR-042 (start-stay honesty), ADR-043 (FIQ + SError park), ADR-044 / ADR-045 (taken SError research + QMP attempt), and ADR-026 (PAN ID-field). Four leftovers still read as open Planned work in the ledger and roadmap even though sponsor intent is to decide them honestly rather than leave them as pending miles that invite a fake Verified.

This ADR records those decisions. It does not change guest code. CloudAgent HELD; (amd64 workstation) Docker re-smoke is optional for this docs-only PR.

Hypotheses

IDOptionNotes
H1 (chosen)Decide each leftover: taken SError = non-goal/deferred on this smoke machine; PAN enable = non-goal on default -cpu cortex-a57; never yank _start while -kernel needs 0x4008_0000; leftover frame RAM optional Planned; umbrella “EL0 isolated” stays Planned/non-claim.Honest close of the leftover decision without inventing probes.
H2 (rejected)Mark taken SError / PAN enable / “EL0 isolated” Verified from existing miles.Round-up. Forbidden.
H3 (rejected)Silent -cpu switch or yank _start to “finish” isolation.Breaks honesty / QEMU -kernel.

Decision

1. Taken lower-EL SError — non-goal / deferred on this smoke machine

  • QMP inject-nmi fails on QEMU 10 virt + -cpu cortex-a57 (machine does not provide NMIs). There is no honest host inject on this probe today (ADR-045).
  • Decision: taken lower-EL SError while standing is a non-goal / deferred on this smoke machine until an honest inject exists (new QEMU feature, accepted different machine story, or sponsor-approved CPU/GIC change with its own ADR).
  • Keep el0: serror-park Verified (park honesty).
  • Keep guest A-clear standing plumbing (el0: serror-arm / eret_to_el0_serror / EXPECT) as dormant prep — do not delete; do not require el0: serror in smoke.
  • Do not claim taken SError Verified.

2. PAN enable on default -cpu cortex-a57 — non-goal on the default probe CPU

  • ID field is 0 (pan: absent). Enable + EL1-vs-EL0 fault is impossible without inventing hardware (ADR-026).
  • Decision: PAN enable is a non-goal on the default probe CPU. Do not silent -cpu switch.
  • ID-field probe (pan: id= / pan: absent) stays Verified.
  • An optional future CPU story (different default -cpu, FEAT_PAN present, enable + fault probe) needs its own ADR + sponsor approval. Until then, do not claim PAN.

3. Yank _start / leftover identity RAM

Amended 2026-09-26 by ADR-089 (sponsor D2): “never yank _start” stands. Leftover identity RAM was torn (ADR-049, ADR-087), and the MMIO identity block was torn too (ADR-088). The _start stub page [0x4008_0000, 0x4008_1000) is the one documented identity exception, and a fail-closed TTBR0 inventory enforces it. The original text below is kept for history.

  • QEMU -kernel still needs _start at 0x4008_0000.
  • Decision: never yank _start while that boot contract holds. ident: start-stay / ident: ram-stay stay Verified honesty (ADR-042).
  • Leftover identity frame RAM after the heap may stay Planned as an optional later unmap. It is not required to close the umbrella isolation row under current constraints.
  • Do not claim “the kernel moved” or full identity teardown.

4. Umbrella “EL0 isolated” (P-SEC-3l) — Planned / non-claim

  • Decision: remains Planned / non-claim under current constraints (no PAN enable on a57, _start stays, taken SError deferred).
  • Specific Verified miles (IRQ/FIQ/tears/start-stay/PAN ID-field/…) are not this row. Do not round them up to “EL0 isolated.”
  • Follow-on (2026-09-28, M6): every ADR-055 §B row is Met and the sponsor accepted the umbrella sentence in writing (B3). This row is Verified, only as the accepted ADR-091 sentence. The decision above is as of 2026-09-14.

Status table (after this ADR)

ItemStatusNotes
el0: serror-parkVerifiedPark honesty
Guest A-clear SError prep (ADR-045)Dormant prepNot a taken probe
Taken lower-EL SError on this smoke machineDeferred / non-goal (locked)ADR-053
PAN ID-field on cortex-a57Verifiedpan: absent
PAN enable on default cortex-a57Non-goal (locked)ADR-054 reopen gate
ident: start-stay / never yank _startVerified (honesty) / decidedKeep 0x4008_0000
Leftover identity RAM after heapVerified (optional) via ADR-049Not umbrella-required
Umbrella “EL0 isolated”Planned / non-claim until checklistADR-055; constraints above

Honesty

Say the decisions above only as documentation. Do not say:

  • taken lower-EL SError is Verified
  • PAN is enabled / “EL0 isolated”
  • _start was yanked / the kernel moved
  • app hosting is done (see ADR-048)

Consequences

  • Docs: el0.md, security.md (threat-model v1.26), honesty ledger (Planned→decided wording), roadmap P-SEC-3k/3l/3r + leftover blurb, NFR-10 text in place, SUMMARY.
  • Code: unchanged. Smoke still requires park / start-stay / pan: absent; still rejects el0: serror as a required marker and rejects pan: enabled.
  • A later ADR may reopen taken SError (honest inject), PAN (new CPU story), or boot redesign so _start can leave identity — each needs sponsor scope.
  • 2026-09-14 locks: ADR-053 hard-stops taken SError with dated QMP evidence; ADR-054 locks PAN enable + reopen gate; ADR-055 publishes the umbrella non-claim checklist. Decisions in this ADR stand; those ADRs remove fuzzy Planned wording.
  • 2026-09-15 sponsor closure: ADR-060 is the sponsor-facing one-pager (locks + reopen gates). Does not rewrite decisions here.

ADR-048 — App hosting “done” claim criteria

  • Status: Accepted (claim gate). Product “apps run independently / hosting done” is Verified when criteria 1–5 are Met (ADR-052 records sponsor accept for #5). Do not claim Linux userspace, POSIX, containers, or “EL0 isolated.”
  • Date: 2026-09-13

Context

Track A children A1–A9 have first cuts (ABI, CRT, loader, standing-as-normal, isolation miles, memfs, FAT, recipes, OS/app slots + cross-update / embed-off). The honesty ledger and roadmap correctly refuse to round those miles up to “app hosting is done.” Sponsors still need explicit criteria so a Verified claim is fail-closed rather than mood.

Related: ADR-030, ADR-032, ADR-029, ADR-052 (sponsor accept), hosting-apps.md, ADR-047 (isolation leftovers are separate — closing them is not required to meet these criteria, and meeting these criteria does not make “EL0 isolated” true).

Decision

A1–A9 first cuts

Treat A1–A9 first cuts as Verified when the honesty ledger has matching rows for this tip (or cited merge tips). That is not product app hosting by itself.

Product claim Verified only when all criteria below are met

The sentence “apps run independently of the OS” / “app hosting is done” may move from Planned to Verified only when every criterion holds, with named probes in the honesty ledger:

  1. Documented recipes — Rebuild / run recipes for freestanding apps cite only existing serial markers and stay buildable (./scripts/docs-build.sh + A8 hub). No invented runtime.
  2. Slot cross-update — Same published app ELF loads on this OS and a documented prior OS (ba6541c or a later sponsor-pinned prior) with fail-closed host smoke (cross-update … slot:ok both boots). ADR-032.
  3. No production embed — A2–A4 / production slot path must not include_bytes! the hello app. FAT /hello (or an equivalent published slot file) is the production payload. Probe-only embeds (e.g. ADR-046 measurement) must never print slot: embed and must stay out of the production path.
  4. Honesty about missing Linux / POSIX / containers — Docs and PRs must still say: not a Linux ABI, not POSIX, not glibc/musl, guest OCI/Docker/k8s remains non-goal (ADR-029). Meeting (1)–(3) does not authorize those claims.
  5. Explicit sponsor accept — A PR that flips the product row to Verified must cite this ADR’s checklist and must not self-merge (ADR-002). Sponsor accept for the 2026-09-13 flip is recorded in ADR-052.

What does not satisfy the claim

  • A1–A9 first cuts alone
  • Isolation ladder miles / ADR-047 decisions
  • Host docker-smoke.sh (Docker hosts ctos, not the reverse)
  • Slot CNTPCT pairs / “slots are free” marketing (ADR-046)
  • Linux-compat research (Track B) or any “runs Alpine” sentence

Accepted claim (amendment)

On tip main @ be789c5+ with ADR-052: criteria 1–5 Met. Product row Verified for the scoped freestanding slot-hosting sentence. Still refuse Linux/POSIX/containers/“EL0 isolated”/OTA/immutable-OS marketing.

Honesty

Say “A1–A9 first cuts exist” when the ledger rows say so. Say the product claim is Verified under ADR-048 only with the checklist Met×5 and sponsor accept cited. Do not say “apps update independently” as OTA, or “container host” — containers stay non-goal regardless.

Consequences

  • Ledger row “Track A app hosting (slots)” is Verified and cites this ADR + ADR-052.
  • hosting-apps.md, Track A / roadmap, NFR-10 text point here.
  • No new FR/NFR IDs. Do not mint NFR-15+.

Progress checklist

Living Met/Partial/Blocked table (tip evidence): hosting-apps.md — Claim criteria and honesty-ledger.md — ADR-048 progress. Product claim is Verified when criterion 5 is Met (ADR-052).

ADR-049 — Leftover identity RAM tear after the heap

  • Status: Accepted (leftover identity frames after the heap unmapped; high twins stay; _start stays) when the serial / tests pass
  • Date: 2026-09-13

Context

ADR-038 unmapped identity heap while GlobalAlloc returned TTBR1 VAs. ADR-042 printed honesty stay markers: ident: start-stay for _start at 0x4008_0000 and ident: ram-stay for leftover identity frames in [heap_pa_end, pool_end). ADR-047 decided never yank _start while QEMU -kernel needs that address, and left leftover frame RAM as an optional Planned mile (not umbrella-required).

This ADR takes that optional mile: unmap leftover identity frame RAM after the torn heap, keep high twins, keep _start mapped+executable, and fail-closed on serial / tests / smoke. It does not claim “EL0 isolated,” “kernel moved,” PAN enable, or taken SError Verified. Umbrella P-SEC-3l stays Planned/non-claim.

Hypotheses

IDOptionNotes
H1 (chosen)Unmap identity frames in [heap_pa_end, pool_end). Keep boot-stub pages needed for -kernel. Print ident: ram / ident: ram-fault / ident: ram-high. Access newly allocated frames via the TTBR1 twin (frame_cpu_va).Smallest honest optional tear.
H2 (rejected)Yank _start / claim full higher-half / claim “EL0 isolated.”Forbidden by ADR-047 / roadmap.
H3 (rejected)Leave ident: ram-stay forever and mark the optional mile Verified without a tear.Would invent closure.

Decision

  1. Unmap leftover identity RAM. After tear_identity_heap, tear_identity_ram unmaps [heap_pa_end, pool_end) from kernel TTBR0. Full 2 MiB L2 blocks are cleared as one slot; partial edge blocks are punched page-by-page. User TTBR0 already omits that range. High twins stay (L2_HIGH_RAM). One boot-time TLBI VMALLE1 after the bulk unmap.
  2. Plant + probe. Before unmap, plant a magic word via the high twin. Serial ident: ram lo=… hi=… pages=N with N >= 1. EL1 LDR of a torn identity leftover VA is a current-EL translation DABORT (ident: ram-fault). EL1 LDR of the high twin still returns the planted magic (ident: ram-high).
  3. Keep _start. ident: start-stay must still pass. Do not change default -cpu. Do not claim PAN / “EL0 isolated” / “kernel moved.”
  4. Frame CPU VA. Call sites that used a frame PA as an identity VA (paging::observe_probe, map_unmap_roundtrip, loader, asid, ttbr1 private-page plant) switch to paging::frame_cpu_va (TTBR1 twin after the high split).
  5. Fail-closed smoke. scripts/qemu-smoke.sh requires ident: ram lo= / ident: ram-fault / ident: ram-high and rejects ident: ram-stay. Keep requiring ident: start-stay.
  6. Still Planned / decided. Umbrella “EL0 isolated” (P-SEC-3l) stays Planned/non-claim. PAN enable non-goal on default a57. Taken SError deferred/non-goal on this smoke machine. Never yank _start.
  7. NFR-10 text is revised in place (ID unchanged). Threat-model v1.27. Do not mint NFR-15+.

Honesty

Say “leftover identity frame RAM after the heap was unmapped while the high twin stayed and _start stayed mapped” only when the serial / tests pass. Do not say:

  • the kernel has moved to the high half
  • identity mappings were fully torn down (_start stays)
  • “secure OS” / “hardened” / “EL0 isolated”
  • PAN enable / taken SError Verified

Consequences

  • paging::tear_identity_ram / frame_cpu_va / teardown probes own the cut. ADR-042 ident: ram-stay is superseded as a stay marker; ident: start-stay remains.
  • ADR-047 leftover-RAM row becomes Verified (optional mile taken); umbrella stays Planned/non-claim.

ADR-050 — FAT16 write behind the thin VFS

  • Status: Accepted (FAT16 write mile Verified when the serial / tests / host-check pass; Track A / app hosting still Planned)
  • Date: 2026-09-13

Context

ADR-028 landed virtio-blk + read-only FAT16 behind the thin VFS (ADR-027). Guest write on a FAT handle returned ReadOnly. Sector R/W was proven on an unused last sector, not by mutating the volume.

Docs “new vs extend” prefer deepening the known small FS (FAT16) over inventing a format or jumping to ext4 / FAT32 / xv6. This mile is Future slice 3: guest FAT16 write (and a small create) on the virtio-blk volume, same thin VFS, fail-closed serial + tests + optional host check.

Not POSIX open/write. Not FAT32. Not “supports FAT” as a product. Not app hosting. A9 /hello load path must keep working.

Hypotheses

IDOptionNotes
H1 (chosen)Guest FAT16 write on an open handle via the same vfs::write, plus FAT backend create of a small single-cluster file. Serial fat: write / fat: rewrite / fat: create / keep fat: ok. Host --check-write sees /fwr.Deepens ADR-028. Same path grammar. Caps: one cluster, FILE_MAX (256).
H2 (alt, rejected this PR)Directory listing / readdir-style thin VFS mile only.Write is in scope and fits one PR. Listing is a later depth mile.
H3 (rejected)FAT32, ext4, xv6 FS, or a Linux VFS clone.Forbidden by overview + ADR-028.
H4 (rejected)Route vfs::create to FAT when mounted.Would put A6 /kprobe / /eprobe on the volume; vfs::reset cannot clear them; second create → Exists. vfs::create stays memfs-first. FAT create is fat::create (backend), then confirmed with vfs::open / read.
H5 (rejected)Claim POSIX write or “writeable FAT volumes” as a product.Honesty: say the guest wrote FAT16 bytes when markers pass.

Decision

  1. FAT write on an open handle. Updates the file’s first (only) cluster and the root dirent size. Offset starts at 0 on open. Empty write → TooBig (same shape as memfs). Refuse multi-cluster grow and sizes above FILE_MAX / one cluster this mile.
  2. Probe rewrite of /probe. vfs::open("/probe") → write fat-wr → read back → write fat-hi restore → fat: write / fat: rewrite. Keeps later /probe readers and tests stable.
  3. FAT backend create of /fwr. fat::create("/fwr") allocates one free cluster + root dirent, writes fat-nw, then vfs::open / read confirms. Serial fat: create. Left on the image for host --check-write.
  4. vfs::create stays memfs-first (A6). Do not invent a second open story; do not put memfs probe names on the volume.
  5. Fail-closed smoke. Grep fat: write / fat: rewrite / fat: create / fat: ok. Reject fat: probe missed. Keep A9 slot: ok / cross-update. Host scripts/mkfat16.py --check-write requires /fwr=fat-nw and restored /probe=fat-hi.
  6. Honesty. Say “the guest wrote FAT16 bytes through the thin VFS (and created a small FAT16 file)” only when serial / tests / host-check pass. Do not say: POSIX, FAT32, “supports FAT,” app hosting done, EL0 isolated, PAN, taken SError.
  7. NFR-10 text is revised in place (ID unchanged). Threat-model v1.28. Do not mint NFR-15+.

Consequences

  • Code: src/fat.rs (write / create / restore probe), scripts/mkfat16.py --check-write, scripts/qemu-smoke.sh greps.
  • Docs: this ADR, filesystem.md, framework/filesystem.md, honesty ledger, roadmap note, threat-model v1.28.
  • Follow-ups (not this PR): multi-cluster grow, dirent shrink helpers, vfs::create → FAT without breaking A6. Root readdir: ADR-056. Delete: ADR-057.

ADR-051 — FAT16 vs memfs write CNTPCT pair (measure-first)

  • Status: Accepted (compared CNTPCT pair on one boot — QEMU TCG lab measurement, not a published bench).
  • Date: 2026-09-13

Context

ADR-050 landed guest FAT16 write behind the thin VFS (fat: write / fat: rewrite / fat: create / fat: ok). Existing performance probes cover loop CNTPCT, IRQ delta, boot-delta, ELF size, and the A9 slot pair (ADR-046). Future slice 4 asks for a measure-first CNTPCT sample around the new FAT write path, compared on the same boot to a memfs write of the same small payload.

Do not invent a latency SLA, percent, or “faster/slower” claim. Do not add a criterion crate. Keep fat: write / fat: ok / A9 slot markers. QEMU TCG jitter is one lab — not SPEC.

Hypotheses

IDOptionNotes
H1 (chosen)Time the FAT VFS write of the /probe rewrite payload (fat-wr) with CNTPCT → perf: fat-write ticks=<n>. On the same boot, time a memfs write of the same bytes → perf: memfs-write ticks=<n>. Print perf: fs-write-delta fat=<a> memfs=<b> (raw ticks).Honest compared pair. Not a bench.
H2 (rejected)Publish a percent / “FAT is slower than memfs” / criterion bench.NFR-07 forbids invented benches.
H3 (rejected)Time the whole open/read/restore/create path as one number and call it “write latency.”Too wide; pair the write of the same payload.
H4 (rejected)Skip the memfs compare and only print FAT ticks.Prefer a same-boot pair when feasible (same shape as ADR-046).

Decision

  1. FAT write sample. During the ADR-050 /probe rewrite, sample CNTPCT around the single vfs::write of WRITE_BYTES (fat-wr). Store ticks; fail closed if the counter does not advance.
  2. Memfs compare. After the FAT write/create probes succeed, vfs::create/open a memfs path (/mwprobe) and sample CNTPCT around vfs::write of the same bytes. Fail closed if ticks are zero.
  3. Markers. Serial: perf: fat-write ticks=<n>, perf: memfs-write ticks=<n>, perf: fs-write-delta fat=<a> memfs=<b>. No ratio percent; no “faster/slower” words on the serial line. Missed samples print perf: fat-write missed / perf: memfs-write missed.
  4. Existing markers unchanged. Keep fat: write / fat: rewrite / fat: create / fat: ok and A9 slot:* / ADR-046 pair.
  5. Smoke. scripts/qemu-smoke.sh greps the new markers and rejects fs-write-delta lines that claim percent/faster/slower.
  6. Honesty. This is a measurement pair on QEMU virt TCG. It is not a latency budget, not a product KPI, and not “FAT write cost is X.” Record Verified only with serial evidence on a named tip. Do not claim app hosting done, EL0 isolated, POSIX, or FAT32.

Consequences

  • Code: src/fat.rs (CNTPCT around FAT write + memfs compare + markers); smoke greps; #[test_case] that both samples are > 0.
  • Docs: this ADR, performance.md, measure.md, honesty ledger, roadmap P-PERF-6.
  • Follow-ups (not this PR): multi-cluster write cost, create-path timing, host-side FS benches — only after a probe shows a need.

ADR-052 — Sponsor accept: product app-hosting claim (ADR-048 #5)

  • Status: Accepted (docs). Product “apps run independently of the OS” / “app hosting is done” is Verified under ADR-048 criteria 1–5. Not Linux userspace. Not POSIX. Not containers. Not “EL0 isolated.”
  • Date: 2026-09-13

Context

ADR-048 gated the product Track A app-hosting sentence on five criteria. On tip main @ be789c5 (and newer), criteria 1–4 were already Met in the honesty ledger (documented A8 recipes; A9 cross-update on this OS + ba6541c; no production hello embed — probe-only include_bytes! in src/slot.rs for ADR-046; honesty that Linux/POSIX/guest OCI remain refused / non-goal per ADR-029). Criterion 5 (explicit sponsor accept on the flip PR, no self-merge) was the remaining Blocked gate.

Sponsor signal (user): after a still-Planned review that listed product app hosting as blocked only on criterion #5, the sponsor said “address 1. app hosting.” Treat that as explicit sponsor accept for ADR-048.

Related: ADR-048, ADR-030, ADR-032, ADR-029, ADR-002, hosting-apps.md, honesty-ledger.md.

Decision

  1. Accept the product claim. Flip the honesty-ledger row “Track A app hosting (slots)” from Planned → Verified for the scoped sentence: freestanding apps load from a separate OS/app slot (FAT /hello), rebuild recipes exist, the same published app ELF cross-updates on this OS and documented prior ba6541c, and production does not embed the hello payload.
  2. Cite this ADR + ADR-048 checklist on the flip PR. Do not self-merge (ADR-002).
  3. Keep the non-claims. Meeting ADR-048 does not authorize: Linux userspace, POSIX, glibc/musl, guest OCI/Docker/k8s, “EL0 isolated,” PAN enabled, taken SError Verified, “immutable OS” marketing, or OTA.
  4. Threat-model patch. security.md may record the Verified product claim under ADR-048/052 and bump the patch version; it still must refuse the non-claims above.

Honesty

Say the product Track A app-hosting claim is Verified under ADR-048 (criteria Met×5, sponsor accept cited). Do not say ctos hosts Linux programs, POSIX apps, or containers. Do not round A1–A9 miles alone into the product claim without this accept record.

Consequences

  • Ledger product row + ADR-048 progress checklist criterion #5 → Met / product Verified.
  • Present-tense docs (hosting-apps.md, Track A / roadmap, syscall.md, NFR-10 note, companions) align without Linux/container/isolation round-ups.
  • ADR-048 status amended: claim gate Accepted; product claim Verified via this ADR.
  • No src/ change required for this accept. No new FR/NFR IDs.

ADR-053 — Taken lower-EL SError hard-stop (no honest inject on smoke machine)

  • Status: Accepted (docs / evidence lock). Taken lower-EL SError while standing stays deferred / non-goal and is locked. Reopen re-probe on default -cpu cortex-a76: ADR-081 / ADR-082 (2026-09-19) — still blocked. (not a fuzzy Planned mile). el0: serror-park remains Verified. Guest A-clear plumbing remains dormant prep. Do not mark taken SError Verified.
  • Date: 2026-09-14
  • Supersedes fuzzy Planned wording for this row; does not reopen ADR-047 decision #1 — it hard-stops it with dated re-probe evidence.

Context

ADR-044 recommended host QMP/inject-nmi (H1). ADR-045 wired the guest A-clear standing path and attempted H1; QEMU 10 virt + -cpu cortex-a57 returned machine does not provide NMIs. ADR-047 decided deferred/non-goal. Sponsor / DSO asked to secure this gap before Guest Linux / containers / immutable-OS marketing: re-check whether any honest inject exists; if not, ADR hard-stop with dated evidence so the taken path is not a fuzzy Planned mile.

Do not silent -cpu switch. Do not claim “EL0 isolated.” Keep _start at 0x4008_0000. Do not unmask A on default standing/task ERET. CloudAgent HELD; docs-only (no new guest inject code — no honest host path exists).

Dated re-probe (2026-09-14, agent box)

Smoke machine (same class as ADR-045 / ledger):

  • Host: agent box
  • qemu-system-aarch64 10.0.13 (Debian 1:10.0.13+ds-0+deb13u1)
  • -machine virt + -cpu cortex-a57
  • QMP unix socket; guest paused with -S (machine/CPU identity only — inject availability does not need the ctos kernel)

Commands tried (documented H1 path only; no invented inject):

AttemptResult
QMP {"execute":"inject-nmi"}error class=GenericError desc=machine does not provide NMIs
QMP human-monitor-command / HMP nmiError: machine does not provide NMIs

query-commands on this QEMU shows inject-nmi present, but the virt machine does not implement TYPE_NMI for this CPU/GIC story. Other *-inject-* commands are CXL-only and are not an ARM SError path for ctos. ADR-044 alternatives remain rejected or out of scope:

IDOption2026-09-14 status
H1QMP/HMP nmi → ARM_CPU_SERRORImpossible on this smoke machine (dated error text above). Matches ADR-045 / amd64 workstation 3f7d3c5 smoke log.
H2Guest-only RAS / poisoned loadStill rejected for Verified (ADR-044).
H3Treat sync/BRK/FIQ as SErrorStill forbidden.
H5KVM serror_pending / plugin injectStill out of scope for TCG virt -kernel smoke.

Modern virt “NMI” / FEAT_NMI (GICv3 + aa64_nmi) is a different CPU/GIC story and would require a silent or sponsor-approved -cpu / GIC change — not an honest inject on the default probe.

Decision

  1. Hard-stop. Taken lower-EL SError while standing on QEMU 10 virt + -cpu cortex-a57 is deferred / non-goal and locked. It is not a Planned implementation mile awaiting a “try harder” cut on this machine.
  2. Park stays Verified. Smoke must keep requiring el0: serror-park. Do not require el0: serror.
  3. A-clear plumbing stays dormant prep. Keep el0: serror-arm / eret_to_el0_serror / EXPECT from ADR-045. Do not delete; do not claim taken.
  4. Reopen gate (only). A later cut may reopen taken SError only with: (a) an honest host inject that actually delivers SError on the accepted smoke machine, or a sponsor-approved machine/CPU/GIC change with its own ADR; (b) serial el0: serror under EXPECT; (c) a new ADR that cites working evidence. Until then, do not mint Verified. 2026-09-19 re-probe: ADR-081 / ADR-082 — gate unmet on virt+cortex-a76 (and GICv3 / virtualization / max variants); explicit unlock = ADR-081 Decision §4 B1/B2/B3 (B1 research: ADR-082).
  5. Still non-claims: PAN enable on default a57 (ADR-054); umbrella “EL0 isolated” (ADR-055); never yank _start.

Honesty

Say “taken lower-EL SError is hard-stopped / deferred on this smoke machine; QMP inject-nmi still returns machine does not provide NMIs (re-probed 2026-09-14).” Do not say:

  • taken lower-EL SError is Verified
  • “EL0 isolated” / PAN enabled / “secure OS”
  • a different -cpu was used to make inject work
  • guest-only RAS inject is Verified
  • the taken path is still a near-term Planned mile on virt+a57

Follow-on

ADR-083 (2026-09-19): pinned QEMU opt-in Verifies taken el0: serror; stock park path unchanged.

Consequences

  • Ledger / roadmap P-SEC-3r: cite this ADR as the hard-stop lock (park Verified; taken locked deferred/non-goal).
  • ADR-045 remains the implementation attempt; this ADR is the dated impossibility lock.
  • ADR-047 decision #1 stands; wording tightens from “until an honest inject exists” (open-ended) to “locked until reopen gate in Decision §4.”
  • Threat-model patch version bump (security.md). No src/ change. No new FR/NFR IDs.
  • Reopen attempt: ADR-081 / ADR-082 (2026-09-19) — still blocked; park Verified.
  • B2 spike: ADR-084 (2026-09-25) — free hosted runners have no arm64 KVM/HVF/WHPX; B2 Planned / blocked.
  • FAT readdir and other product cuts stay out of this PR.

ADR-054 — PAN enable lock on default -cpu cortex-a57 (reopen gate)

  • Status: Accepted (docs / evidence lock). Reopen gate items 1–4 are met: ADR-079 (CPU + pan: present) and ADR-080 (enable + EL1-vs-EL0 fault). Historical a57 ID-field stays Verified (pan: absent, ADR-026). Do not claim “EL0 isolated.”
  • Date: 2026-09-14
  • Locks ADR-047 decision #2 with an explicit reopen gate. Does not change guest code or default -cpu.

Context

ADR-026 Verified that ID_AA64MMFR1_EL1.PAN == 0 on -cpu cortex-a57 (serial pan: id=0 / pan: absent). Enable + EL1-vs-EL0 access fault is impossible without inventing hardware or silently switching -cpu. ADR-047 already called enable a non-goal on the default probe CPU. Sponsor / DSO asked to secure this gap before Guest Linux / containers / immutable-OS marketing: ADR+evidence lock so enable is not a fuzzy Planned mile, with an explicit reopen gate.

Do not silent -cpu switch. Do not execute MSR PAN while the ID field is 0. Do not claim “EL0 isolated.” CloudAgent HELD; docs-only.

Evidence (locked)

EvidenceStatusCite
Default smoke CPU is -cpu cortex-a57Scripts contractscripts/qemu-aarch64.sh, scripts/qemu-serial-inject.py
ID_AA64MMFR1_EL1.PAN == 0 → pan: absentVerifiedADR-026; honesty-ledger PAN ID-field rows (e.g. 2026-09-12 pan: id=0 / pan: absent)
Smoke rejects pan: enabledVerified harnessscripts/qemu-smoke.sh
QEMU CPU model props do not expose a free “turn on PAN” on cortex-a57 without a model changeAgent-box QMP 2026-09-14query-cpu-model-expansion type=full model.name=cortex-a57 — no pan prop to flip on this model

Decision

  1. Lock. PAN enable (PSTATE.PAN / MSR PAN + EL1 load of an EL0-accessible page must fault) is a non-goal on the default probe CPU -cpu cortex-a57. Roadmap P-SEC-3k stays Non-goal (default probe CPU), not Planned.
  2. ID-field stays Verified. Keep printing pan: id= / pan: absent. Keep #[test_case] pan_unimplemented_on_probe_cpu. Smoke must still reject pan: enabled / pan: probe missed.
  3. Explicit reopen gate. PAN enable may be reopened only when all of the following hold:
    1. Sponsor-approved new default probe CPU (or documented FEAT_PAN-present CPU story) — recorded in a new ADR, not a silent script edit.
    2. Serial (or equivalent) evidence that ID_AA64MMFR1_EL1.PAN != 0 on that accepted CPU (pan: present).
    3. A follow-up ADR that enables PSTATE.PAN and proves an EL1-vs-EL0 access fault (fail-closed if the fault does not land).
    4. Honesty ledger + smoke greps updated for the new CPU; old a57 pan: absent rows remain historical truth for a57.
  4. Until the reopen gate is met, do not claim PAN, “privileged access never,” or “EL0 isolated.”

Reopen progress (2026-09-19)

ADR-079 / #125: sponsor unlock + default probe CPU -cpu cortex-a76 + serial pan: present (gate items 1–2). ADR-080: enable + EL1-vs-EL0 fault (gate items 3–4). a57 pan: absent remains historical truth for that CPU. Umbrella “EL0 isolated” still non-claim (ADR-055).

Honesty

Say “PAN enable is locked non-goal on default -cpu cortex-a57; ID-field Verified pan: absent (ADR-026).” Do not say:

  • PAN is enabled
  • EL1 cannot access EL0 pages (PAN mile)
  • the probe used -cpu max / another model without a sponsor ADR
  • “EL0 isolated” / “secure OS”

Consequences

  • Ledger / roadmap P-SEC-3k cite this ADR as the enable lock + reopen gate.
  • ADR-026 remains the capability probe; this ADR is the enable hard-lock.
  • ADR-055 lists PAN enable as an unmet umbrella requirement.
  • Threat-model patch bump. No src/ change. No silent -cpu edit.

ADR-055 — Umbrella “EL0 isolated” non-claim checklist

  • Status: Accepted (docs). Since M6 (2026-09-28) every §B row is Met and the umbrella is Verified only as the accepted ADR-091 sentence. Original status text (2026-09-14): Umbrella “EL0 isolated” stays Planned / non-claim and cannot become Verified under current constraints. This ADR tightens the non-claim with an evidence checklist; it does not invent Verified.
  • Date: 2026-09-14
  • Follow-on (2026-09-28, M6): the sponsor accepted the ADR-091 revision-3 sentence and scope in writing (B3, 2026-09-28 16:41 CEST; sponsor wrote it in the DSO agent chat, DSO relayed it to ctos). Row 3 is Met. ADR-091 is Accepted. The umbrella row flips to Verified, scoped to that sentence and its limits.
  • Follow-on (2026-09-27, ADR-092): sponsor option (b) on ADR-091. G1 (EL0-reachability walk) and G2 (EL0 write fault) closed as §A miles below; new gap G4 (syscall copy helpers skip the EL0 permission check) recorded. Row 3 still not Met (ADR-091 revision 2 is Draft).
  • Follow-on (2026-09-26, M5): the umbrella sentence and scope are drafted in ADR-091 (Draft / pending sponsor accept, D4). Row 3 stays not Met until the sponsor has read the draft and accepted it in writing. Nothing is flipped.
  • Follow-on (2026-09-26, M4): sponsor D1 accepted. The taken-SError evidence class for the umbrella is the B1 pin job (now unconditional on every push/PR) plus the B2-P one-off KVM run (ADR-090). Row 1 is reworded and Met on that class. Stock QEMU still parks. The umbrella still needs the sponsor’s written accept (M5 draft → ADR-091).
  • Audit (2026-09-26): ADR-086 re-probed every row this session. All §A miles are Verified. All §B rows are still not Met: taken SError is Verified only on opt-in paths (B1 pin / B2-P one-off). The identity inventory still leaves five identity ranges (_start is one of them, by decision). No umbrella accept exists. §A marker el0: no data is stale wording: the code uses el0: no kernel read.
  • Locks ADR-047 decision #4. Optional ledger clarity: “non-claim until checklist.”

Context

Many Track A / A5 miles are Verified (standing EL0, IRQ/FIQ while standing, identity tears, PAN ID-field, SError park, start-stay, leftover RAM tear, …). Rounding those miles up to “EL0 isolated” is forbidden. Sponsor / DSO asked to secure the umbrella gap before Guest Linux / containers / immutable-OS marketing: publish a checklist of Verified miles vs umbrella requirements still unmet, given:

CloudAgent HELD; docs-only.

Decision

  1. (Superseded by M6, 2026-09-28: all §B rows Met; the umbrella is Verified as the ADR-091 sentence only.) Umbrella stays Planned / non-claim. Ledger wording may say “non-claim until checklist” (this ADR). Do not flip to Verified.
  2. Checklist is the gate. “EL0 isolated” may be reconsidered only when every Unmet row below is Met under its own ADR + probes — not by summing existing miles.
  3. Specific Verified miles are not the umbrella. Cite them as themselves.

Evidence checklist

A. Verified miles (not sufficient for the umbrella)

MileEvidence (serial / test)ADR
EL0 first mile + NX kernel datael0: ok / el0: nx kernelADR-013
User TTBR0 read mileel0: no kernel read (wording fixed 2026-09-26, ADR-087; earlier text said el0: no data, which no code prints)ADR-013
Standing EL0 + standing taskel0: standing / el0: task-okADR-013 / ADR-024
ASID isolationasid: okADR-013
TTBR1 private page + high EL1 fetchttbr1: ok / ttbr1: el1 execADR-016 / ADR-017
Identity tears (text/rodata/data/heap/ram)ident: * / ident: okADR-018…020 / 025 / 037 / 038 / 049
EL0 entry without VMALLE1el0: no-vmalle1ADR-039
Lower-EL IRQ while standing + default I clearel0: irq / el0: irq-defaultADR-040 / ADR-041
Lower-EL FIQ while standingel0: fiqADR-043
SError park honestyel0: serror-parkADR-043 / ADR-045
PAN ID-field on default probe CPUpan: present on cortex-a76 (ADR-079); historical a57 pan: absentADR-026 / ADR-079
PAN enable + EL1-vs-EL0 faultpan: enabled / pan: el1-faultADR-080
EL0 store to kernel data (TTBR1) / EL0-RO page → permission fault (2026-09-27)el0: write-fault kernel … ec=0x24 wnr=1 dfsc=perm-l3 / el0: write-fault user-ro … / el0: write-ok kernel,user-roADR-092
Fail-closed EL0-reachability walk (k/u/ASID-B TTBR0 + TTBR1; allowlist = five user slots) (2026-09-27)el0-reach: live k=3 u=3 a=0 h=0 pages=6 leaks=0 / el0-reach: steady … pages=0 leaks=0 / el0-reach: neg … caught / el0-reach: ok allow=…ADR-092
_start stay honestyident: start-stayADR-042 / ADR-047

B. Umbrella requirements (were Unmet; all Met since M6, 2026-09-28)

RequirementCurrent statusWhy unmetReopen / lock ADR
Taken lower-EL SError while standing, on the D1 evidence class (reworded 2026-09-26 per sponsor D1, ADR-090): B1 qemu-nmi-pin green, unconditional on every push/PR, plus B2-P ADR-085 run 4Met (D1 evidence class; not the umbrella). B1 bare el0: serror fail-closed in CI; B2-P el0: serror + b2: taken under Graviton3 KVM (one-off, at 159b178). Stock virt TCG itself: still deferred / non-goal (hard-stopped), el0: serror-parkStock QEMU has no honest inject on virt TCG (machine does not provide NMIs) — re-probed a76 + GICv3/virt-on/max 2026-09-19. B1 opt-in pin Verified (ADR-083); B2 free-runner spike blocked 2026-09-25 (ADR-084)ADR-053 / ADR-081 / ADR-082 / ADR-083 / ADR-084
Full identity teardown except the documented _start stub page (reworded 2026-09-26 per sponsor D2, ADR-089; was “including yank _start”)Met (this session; not the umbrella). The fail-closed inventory reports only the stub page: ident: inv k=1 u=1 a=1 leaks=0 / ident: inv-ok allow=stubM1 ADR-087 + M2 ADR-088. _start never yanked (ident: start-stay)ADR-042 / ADR-047 (amended) / ADR-089
Written sponsor accept that the umbrella sentence is in scopeMet (2026-09-28 16:41 CEST, B3). Verbatim: “I accept the ADR-091 revision-3 sentence and scope as written. After M6, have ctos propose a short costed list of next milestones.” Relay path: sponsor wrote it in the DSO agent chat, DSO relayed it to ctos.Was absent until 2026-09-28 (draft since 2026-09-26, D4; revisions 2 and 3 after G1/G2/G4).ADR-091 (Accepted) → M6 flip

§B is cleared (M6, 2026-09-28): the umbrella row is Verified, scoped to the accepted ADR-091 sentence and its limits. Original text: Until §B is cleared under honest probes (and _start policy is redesigned with sponsor scope if ever), the umbrella row stays non-claim.

Honesty

Since M6, say “EL0 isolated” only as the accepted ADR-091 sentence, with its limits. Many isolation miles are Verified; one mile alone is not the umbrella. (Before M6 this read: “umbrella EL0 isolation is Planned / non-claim until the ADR-055 checklist; many isolation miles are Verified but that is not ‘EL0 isolated.’”) Do not say:

  • “EL0 isolated” wider than the ADR-091 sentence / “secure OS” / “hardened isolation complete”
  • “EL0 isolated” from PAN alone / taken SError Verified / _start yanked
  • Guest Linux / containers / immutable-OS marketing as if isolation were closed

Consequences

  • Honesty ledger umbrella row cites this checklist; optional label “non-claim until checklist.”
  • el0.md, roadmap P-SEC-3l, security.md, limits.md point here.
  • Sponsor-facing one-pager (locks + reopen gates table): ADR-060.
  • No src/ change. No Verified invent. Threat-model patch bump only.

ADR-056 — FAT16 readdir behind the thin VFS

  • Status: Accepted (FAT16 root listing mile Verified when the serial / tests pass; Track A / app hosting still Planned for Linux/POSIX claims)
  • Date: 2026-09-14

Context

ADR-028 landed virtio-blk + FAT16 read. ADR-050 deepened write / small create and deferred directory listing. Docs “new vs extend” prefer deepening the known small FS (FAT16) behind the thin VFS (ADR-027).

This mile is guest root directory listing: walk FAT16 root dirents and return thin-VFS paths (/probe, /hello, /fwr). Same path grammar. Fail-closed serial + #[test_case].

Not POSIX getdents / opendir / readdir(3). Not a directory tree. Not delete/unlink. Not virtual mounts. Keep write, /hello, cross-update, slot:ok.

Hypotheses

IDOptionNotes
H1 (chosen)Kernel + thin VFS vfs::readdir lists FAT16 root file names as /… paths. Serial fat: readdir / fat: entries n=<k>. Expect /probe, /hello, /fwr after the write-mile create. No new SVC this mile.Deepens ADR-028/050. Same path grammar. Caps: READDIR_MAX (16).
H2 (rejected)Claim POSIX getdents / opendir or add Linux dirent ABI.Forbidden honesty.
H3 (rejected)New SVC / libctos wrapper this PR.Write mile used existing vfs::write. Listing is a kernel/VFS depth probe; EL0 SVC can follow later.
H4 (rejected)Subdirectory trees, LFN, delete/unlink, or virtual mounts.Out of scope; follow-ups.
H5 (rejected)Route memfs through the same listing as a union namespace.memfs stays flat named buffers; this mile is FAT16 root only.

Decision

  1. fat::readdir / vfs::readdir. Walk the FAT16 root, skip deleted / volume / LFN / directory entries, map 8.3 (blank extension) to thin-VFS paths via the inverse of path_to_83. Cap READDIR_MAX. Overflow → FsError::Full (fail-closed).
  2. Probe after create. observe_probe lists the root after /fwr exists so the write-mile name is visible. Serial fat: readdir then fat: entries n=<k>. Require /probe, /hello, /fwr. Keep fat: ok / write / A9 slot markers.
  3. No new SVC numbers. Public ABI stays 16–23. Not a POSIX product claim.
  4. Fail-closed smoke. Grep fat: readdir / fat: entries / keep fat: ok. #[test_case] covers listing + cap Full.
  5. Honesty. Say “the guest listed FAT16 root directory entries through the thin VFS” only when serial / tests pass. Do not say: POSIX getdents/opendir, FAT32, “supports FAT,” EL0 isolated, PAN, taken SError, Linux, containers.
  6. NFR-10 text is revised in place (ID unchanged). Threat-model v1.31. Do not mint NFR-15+.

Consequences

  • Code: src/fat.rs (list_root / readdir / probe), src/vfs.rs (vfs::readdir), scripts/qemu-smoke.sh greps.
  • Docs: this ADR, filesystem.md, framework/filesystem.md, honesty ledger, roadmap note, threat-model v1.31, SUMMARY.
  • Follow-ups (not this PR): EL0 fs_readdir SVC, LFN, subdirectories, virtual mounts, memfs listing. Delete/unlink: ADR-057.

ADR-057 — FAT16 delete behind the thin VFS

  • Status: Accepted (FAT16 root delete mile Verified when the serial / tests pass; Track A / app hosting still Planned for Linux/POSIX claims)
  • Date: 2026-09-14

Context

ADR-028 landed virtio-blk + FAT16 read. ADR-050 deepened write / small create. ADR-056 deepened root listing. Docs “new vs extend” prefer deepening the known small FS (FAT16) behind the thin VFS (ADR-027).

This mile is guest root file delete: mark a FAT16 root dirent deleted (0xE5), free its cluster chain, and expose the same path grammar through vfs::unlink. Fail-closed serial + #[test_case].

Not POSIX unlink / remove / rm. Not a directory tree. Not virtual mounts. Keep write, readdir, /hello, cross-update, slot: ok.

Hypotheses

IDOptionNotes
H1 (chosen)Kernel + thin VFS vfs::unlink deletes a FAT16 root file. Serial fat: delete / keep fat: ok. Probe creates /fdel, deletes it, confirms Missing + absent from readdir. No new SVC this mile.Deepens ADR-028/050/056. Same path grammar.
H2 (rejected)Claim POSIX unlink / remove or add a Linux unlink ABI.Forbidden honesty.
H3 (rejected)New SVC / libctos wrapper this PR.Write/readdir miles used existing VFS; delete is a kernel/VFS depth probe; EL0 SVC can follow later.
H4 (rejected)Subdirectory trees, LFN, multi-cluster shrink helpers, or virtual mounts.Out of scope; follow-ups.
H5 (rejected)Route only memfs through vfs::unlink this mile.Product-depth train is FAT16; memfs unlink is incidental for the same entry point.

Decision

  1. fat::unlink / vfs::unlink. Look up the 8.3 root dirent, mark first byte 0xE5, free the cluster chain (walk to EOC; single-cluster files this mile), drop open handles to that name. Missing → FsError::Missing (fail-closed).
  2. Probe after readdir. Create /fdel, write fat-dl, vfs::unlink("/fdel"), confirm open/has_name Missing and root listing still has /probe, /hello, /fwr without /fdel. Serial fat: delete. Keep fat: ok / write / readdir / A9 slot markers.
  3. No new SVC numbers. Public ABI stays 16–23. Not a POSIX product claim.
  4. Fail-closed smoke. Grep fat: delete / keep fat: ok / readdir / write. #[test_case] covers delete + missing.
  5. Honesty. Say “the guest deleted a FAT16 directory entry through the thin VFS” only when serial / tests pass. Do not say: POSIX unlink/remove, FAT32, “supports FAT,” EL0 isolated, PAN, taken SError, Linux, containers.
  6. NFR-10 text is revised in place (ID unchanged). Threat-model v1.32. Do not mint NFR-15+.

Consequences

  • Code: src/fat.rs (mark deleted / free chain / probe), src/vfs.rs (vfs::unlink), scripts/qemu-smoke.sh greps.
  • Docs: this ADR, filesystem.md, framework/filesystem.md, honesty ledger, roadmap note, threat-model v1.32, SUMMARY.
  • Follow-ups (not this PR): EL0 fs_unlink SVC, LFN, subdirectories, virtual mounts, richer samples.

ADR-058 — Thin VFS prefix mounts (virtual mounts)

  • Status: Accepted (prefix-mount mile Verified when the serial / tests pass; Track A / app hosting still Planned for Linux/POSIX claims)
  • Date: 2026-09-14

Context

ADR-027 landed a thin VFS + memfs. ADR-028 added FAT16 behind the same open, with memfs-first then FAT fallback. Write / readdir / delete deepened that FAT backend (ADR-050 / ADR-056 / ADR-057).

Docs “new vs extend” list ctos-specific virtual mounts (prefix / tree mounts; not a new on-disk format) as a later optional depth mile (overview/filesystem.md). Routing today is an implicit overlay, not a mount table.

This mile is a small mount table that routes path prefixes to a backend. Keep A9 /hello, FAT write/readdir/delete, memfs create/write/read. Flat path grammar unchanged (/ + [a-z0-9_-], no extra /).

Not Linux VFS. Not POSIX mount(2) / mount namespaces / overlays. Not a new disk FS format.

Hypotheses

IDOptionNotes
H1 (chosen)Static prefix mount table; longest prefix wins. /mem (+ pinned A6 probe names) → memfs; / → FAT16. Serial vfs: mount / vfs: mounts. #[test_case] + fail-closed smoke.Matches /hello / /probe on FAT and A6 memfs paths. No new SVC.
H2 (rejected)Remap everything under /fat/… and /mem/… (tree mounts with an extra /).Breaks flat path grammar and A9 /hello.
H3 (rejected)Claim Linux mount(2) / vfsmount / mount namespaces.Forbidden honesty (ADR-034 / ADR-031).
H4 (rejected)Invent a new on-disk FS format for “virtual mounts.”Overview forbids novel magic formats this mile.
H5 (rejected)Keep memfs-first overlay only; document it as mounts without a table.Not a real first cut; product-depth train wants an honest router.

Decision

  1. Mount table. vfs::MOUNTS: (prefix, backend) entries. vfs::resolve(path) picks the longest matching prefix. First cut:
    • /kprobe, /eprobe, /mwprobe, /mem → memfs (A6 habits + documented /mem home)
    • / → FAT16 (/probe, /hello, /fwr, /fdel, …)
  2. Route create / open / unlink by mount. No memfs→FAT fallback overlay. read / write / close still key off the FAT fd tag. readdir stays FAT-root only (not a union listing).
  3. Probe. Hello prints each vfs: mount <prefix> -> <memfs|fat> then vfs: mounts, then the existing fs:* memfs trip. Keep fat:* / slot: ok.
  4. No new SVC numbers. Public ABI stays 16–23. Not POSIX mount.
  5. Fail-closed smoke. Grep vfs: mount / vfs: mounts / keep fs: ok / fat: ok / slot: ok. #[test_case] covers resolve + memfs-mount create off the FAT volume.
  6. Honesty. Say “the guest routed path prefixes through a thin VFS mount table” only when serial / tests pass. Do not say: Linux VFS, mount(2), containers, POSIX, “supports FAT,” EL0 isolated, PAN, taken SError.
  7. NFR-10 text is revised in place (ID unchanged). Threat-model v1.33. Do not mint NFR-15+.

Consequences

  • Code: src/vfs.rs (mount table, resolve, routed create/open/unlink, probe), scripts/qemu-smoke.sh greps.
  • Docs: this ADR, filesystem.md, framework/filesystem.md, honesty ledger, roadmap note, threat-model v1.33, SUMMARY, architecture glance.
  • Follow-ups (not this PR): tree mounts with an extra /, runtime mount/unmount SVC, memfs readdir, stripping the mount prefix inside backends.

ADR-059 — Second freestanding sample (fs-libctos) + catalog expansion

  • Status: Accepted (sample + smoke Verified when serial / tests pass; product freestanding app hosting remains Verified under ADR-048/052 — do not reopen)
  • Date: 2026-09-14

Context

Track A / A8 documented rebuild recipes for in-tree samples (what-can-run.md, user/README.md). Today only one freestanding EL0 sample exists: user/hello-libctos (UART + yield). Docs called out the gap: a libctos hello that calls fs_open was not in the tree. Wrappers exist (SVC 16–23); the Verified EL0 VFS trip was the kernel /eprobe trampoline.

After ADR-058, longest-prefix mounts route /mem (+ A6 names) → memfs and / → FAT16. Flat path grammar unchanged (/ + [a-z0-9_-], no extra /).

This mile adds a second freestanding EL0 sample that exercises libctos VFS, publishes it on FAT beside /hello, expands the catalog, and fail-closes smoke on new markers. Honesty: not POSIX, not Linux, not getdents from EL0.

Hypotheses

IDOptionNotes
H1 (chosen)New crate user/fs-libctos mirroring hello target/linker/build-std. Create/write/read/close /memdemo (memfs via /mem prefix). FAT slot /fsdemo (8.3 FSDEMO). Kernel loads after A9 hello. Serial libctos: fs-hi / libctos: fs-ok + fsdemo: ok. ADR + catalog docs.Cleanest fit; keeps /hello intact.
H2 (rejected)Teach hello-libctos to call fs_open in-place.Collapses two recipes into one; muddies UART-only vs VFS samples.
H3 (rejected)Use /mem/... tree path or claim POSIX open.Breaks flat grammar / honesty.
H4 (rejected)Replace /hello with the VFS sample.Breaks A9 cross-update and existing markers.
H5 (rejected)Add EL0 getdents this mile.Not wired; do not invent.

Decision

  1. Sample crate. user/fs-libctos: same aarch64-ctos-user / linker.ld / build-std as hello. main prints libctos: fs-hi, yields, then fs_create / fs_write / fs_close / fs_open / fs_read / fs_close on /memdemo, then libctos: fs-ok. Fail path prints libctos: fs-fail and exits non-zero.
  2. Path choice (hunch → verify). /memdemo longest-prefix-matches /mem → memfs under ADR-058. Flat grammar forbids /mem/foo. The ELF lives on FAT /fsdemo; the in-app file is memfs.
  3. Publish. build.rs builds and publishes target/fs-libctos.elf. scripts/mkfat16.py --app2 writes 8.3 FSDEMO. Keep /hello + slot: ok.
  4. Kernel probe. src/fsdemo.rs loads FAT /fsdemo, ERETs via loader::run_image_expecting (last uart len 15). Serial fsdemo: fat / fsdemo: mapped / fsdemo: ok. Runs after the A9 slot probe.
  5. Smoke. Fail-closed greps for the new markers; preserve all existing greps (hello, FAT write/readdir/delete, mounts, A9 cross-update, …).
  6. Docs. This ADR; honesty ledger evidence-only; user/README.md; what-can-run.md; apps-today.md; SUMMARY. Threat-model v1.34 (NFR-10 text in place).
  7. EL0 exec pages. RX user pages are fetch-only for EL0 (existing l3_page_el0_exec). The sample may pass .rodata pointers to SVCs (EL1 copy) but must not LDR them at EL0; read-back compare uses immediates.
  8. Honesty. Do not claim: EL0 isolated, PAN enable, taken SError, Linux/POSIX/containers, immutable-OS marketing, or reopen “app hosting is done.”

Consequences

  • Code: user/fs-libctos/, build.rs, scripts/mkfat16.py, scripts/qemu-aarch64.sh, scripts/qemu-smoke.sh, src/fsdemo.rs, src/loader.rs (run_image_expecting), src/main.rs.
  • Docs: this ADR + catalog / ledger / threat-model v1.34.
  • Follow-ups (not this PR): richer samples, EL0 readdir when wired, tree mounts with an extra /.

ADR-060 — Isolation leftovers closure checklist for sponsor

  • Status: Accepted (docs). Updated 2026-09-19 for ADR-079 / ADR-080 PAN CPU reopen + enable, and ADR-081 / ADR-082 taken-SError re-probe (still blocked); 2026-09-25 ADR-084 B2 free-runner spike (blocked). Sponsor-facing closure checklist for isolation leftovers locked by ADR-047, ADR-053, ADR-054, ADR-055, and #98. Does not invent Verified for taken SError or “EL0 isolated.” Does not reopen Guest Linux / containers / immutable-OS marketing.
  • Date: 2026-09-15
  • Follow-on (2026-09-28, M6): B3 met: the sponsor accepted the ADR-091 revision-3 sentence and scope in writing (2026-09-28 16:41 CEST). ADR-091 is Accepted. The umbrella row below is Verified, only as that sentence. The other rows and locks are unchanged: stock TCG still parks SError, and _start is never yanked.
  • Follow-on (2026-09-27, ADR-092): sponsor chose option (b) on ADR-091. G1 (fail-closed EL0-reachability walk) and G2 (EL0 write permission fault) are closed; new gap G4 (syscall copy helpers skip the EL0 permission check) is recorded. ADR-091 revision 2 goes back to the sponsor. The umbrella row below is unchanged: Planned / non-claim.
  • Follow-on (2026-09-26, M5): draft accept ADR-091 (Draft / pending sponsor accept) proposes the exact umbrella sentence and scope, plus gaps G1–G3 for the sponsor to weigh. The umbrella row below is unchanged: Planned / non-claim.
  • Follow-on (2026-09-26, M4): sponsor D1 accepted. The taken-SError evidence class for the umbrella is the B1 pin job (now unconditional on every push/PR) plus the B2-P one-off KVM run (ADR-090). The stock-TCG hard-stop in the row below still holds for stock QEMU.
  • Follow-on (2026-09-26, M3): sponsor D2 accepted. The umbrella’s identity row now reads “full identity teardown except the documented _start stub page” (ADR-089) and is Met on this session’s evidence (ADR-087/088). Taken SError (D1 → ADR-090) and the sponsor accept (D4 → ADR-091 draft) are still open.
  • Follow-on (2026-09-26): B2-P taken SError Verified opt-in on Graviton3 KVM (ADR-085). Checklist audit + gap list + milestone plan + sponsor decisions D1–D5: ADR-086. The umbrella stays non-claim, and the locks below are unchanged.
  • Numbering note: next free after ADR-059 on main @ 404d74c. FAT/fs-libctos sample already took ADR-059; this ADR is 060 (no collision with a parallel FAT-sample ADR-060).

Context

Sponsor / DSO asked for a single clear table that closes the leftover decision surface after the isolation-gap locks (#98 / ADR-053–055): what is locked, what would be required for an honest Verified, what is blocked on the default smoke machine (virt + -cpu cortex-a57), and the reopen gates. This ADR cites those decisions; it does not rewrite them.

CloudAgent HELD. Docs-only. Prefer the amd64 workstation / agent-box for docs build. Do not self-merge (ADR-002).

Decision (cite locks; do not invent Verified)

The sponsor closure checklist is the table below. Status labels are locks and reopen gates only.

LeftoverLocked statusWhat’s required for honest VerifiedWhat’s blocked on a57+virt todayReopen gate
Taken lower-EL SError while standingDeferred / non-goal (hard-stopped) (ADR-053 / ADR-081 / ADR-082 / ADR-047 §1). el0: serror-park stays Verified. A-clear path = dormant prep.Honest host inject that delivers async SError on the accepted smoke machine; serial el0: serror under EXPECT; new ADR citing working evidence.QMP inject-nmi / HMP nmi → machine does not provide NMIs on QEMU 10 virt + -cpu cortex-a76 (re-probed 2026-09-19; also a57 / GICv3 / virtualization=on / max). No silent invent. FEAT_NMI ≠ SError. KVM out of scope for TCG smoke.ADR-081 Decision §4 B1/B2/B3: (B1) upstream/pinned QEMU virt TYPE_NMI→ARM_CPU_SERROR — research ADR-082: no shippable pin 2026-09-19; wait upstream or reviewed local pin; (B2) sponsor smoke-machine ADR with CI-runnable honest inject — spike ADR-084 2026-09-25: blocked on free hosted runners (no arm64 KVM/HVF/WHPX); needs sponsor choice (self-hosted arm64 KVM / paid bare metal / wait); (B3) permanent non-goal. B1 opt-in pin Verified (ADR-083).
PAN enable (PSTATE.PAN + EL1-vs-EL0 fault)Verified (ADR-080 / #125 mile 2). Default -cpu cortex-a76 → pan: present + pan: enabled + pan: el1-fault. Historical a57 pan: absent (ADR-026 / ADR-054 / ADR-079).Met: EL1 load of EL0-accessible page faults with SPSR.PAN.a57 alone cannot enable (historical).ADR-054 gate 1–4 met (079+080).
Umbrella “EL0 isolated” (P-SEC-3l)Verified only as the accepted ADR-091 sentence (M6, 2026-09-28; B3 met 16:41 CEST; limits: one core, TCG default smoke machine, not side-channel, not real hardware, not a certification). Was: Planned / non-claim until checklist (ADR-055 / ADR-047 §4 / #98). Specific Verified miles ≠ this row.Every ADR-055 §B Unmet row Met under its own ADR + probes, plus written sponsor accept that the umbrella sentence is in scope (ADR-048-style — not this file).Under current constraints: no PAN enable on a57; _start stays; taken SError hard-stopped; umbrella sponsor accept absent until 2026-09-28 (now met, B3). Remaining miles after #98 = those §B blockers (not more Track A ladder work on virt+a57).Path to Verified: clear §B under honest probes (and redesign boot if _start ever leaves identity — sponsor scope). Path to permanent non-claim: leave locks as-is; do not round Verified miles into the umbrella.
Yank _start (2026-09-26: the stub page is the one documented identity exception, ADR-089; all other identity is torn, see ADR-087 / ADR-088)Decided: never while QEMU -kernel needs 0x4008_0000 (ADR-042 / ADR-047 §3). ident: start-stay Verified honesty.N/A under current boot contract — not a Verified target.Boot stub must stay mapped at entry.Only a future sponsor-scoped boot redesign ADR. Never yank _start in the meantime.
Guest Linux / containers / immutable-OS marketingStill out (ADR-029 / ADR-036 / ADR-055 honesty).N/A — out of scope for this checklist.Do not reopen via isolation docs.Separate product ADR + sponsor scope if ever.

Explicit non-actions (this ADR)

  • Do not invent Verified for taken SError / “EL0 isolated.”
  • Do not silent -cpu switch (ADR-079 documented the cortex-a76 change).
  • Do not yank _start.
  • Do not reopen Guest Linux / containers / immutable-OS marketing.
  • Do not require el0: serror in smoke. Smoke does require pan: enabled / pan: el1-fault (ADR-080).

Verified ladder vs unmet §B (glance)

flowchart TD
  V["Verified miles<br/>standing · tears · IRQ/FIQ<br/>PAN ID · SError park · start-stay"]
  B1["§A Met: PAN enable<br/>ADR-079 + ADR-080"]
  B2["§B Met: taken SError<br/>ADR-090 evidence class"]
  B3["§B Met: identity except _start stub<br/>ADR-089"]
  B4["§B Met: sponsor umbrella accept<br/>B3, 2026-09-28"]
  U["Umbrella EL0 isolated<br/>Verified: ADR-091 sentence only"]
  V -.-> U
  B1 --> U
  B2 --> U
  B3 --> U
  B4 --> U

≤12 nodes. Miles alone are not the umbrella. All §B rows are Met (M6, 2026-09-28); the claim is only the ADR-091 sentence.

Honesty

Say: “isolation leftovers are closed as locks + reopen gates (ADR-060); PAN enable is Verified (ADR-080); taken SError stays blocked after ADR-081 re-probe on stock QEMU; “EL0 isolated” is Verified only as the accepted ADR-091 sentence (M6).” Do not say:

  • taken lower-EL SError is Verified
  • “EL0 isolated” from PAN alone
  • “EL0 isolated” wider than the ADR-091 sentence / “secure OS” / “hardened isolation complete”
  • _start was yanked / the kernel moved
  • Guest Linux / containers / immutable-OS marketing as if isolation closed

Consequences

ADR-061 — Third freestanding sample (fat-libctos) — FAT via thin VFS

  • Status: Accepted (sample + smoke Verified when serial / tests pass; product freestanding app hosting remains Verified under ADR-048/052 — do not reopen)
  • Date: 2026-09-15

Context

ADR-059 added a second freestanding EL0 sample that exercises libctos VFS on memfs (/memdemo via ADR-058 /mem prefix). Product gap remains: no freestanding sample that opens/reads a FAT path from user mode through the same thin VFS (/ → FAT16). Kernel /probe (fat-hi) write/readdir/delete are already Verified at the EL1 path.

This mile adds a third freestanding EL0 sample that reads FAT /probe via libctos, publishes it on FAT beside /hello and /fsdemo, expands the catalog, and fail-closes smoke on new markers. Honesty: not POSIX, not Linux, not “EL0 isolated.”

Hypotheses

IDOptionNotes
H1 (chosen)New crate user/fat-libctos. Open/read/close FAT /probe expecting fat-hi. FAT slot /fatdemo (8.3 FATDEMO). Kernel loads after ADR-059 fsdemo. Serial libctos: fat-hi / libctos: fat-ok + fatdemo: ok. ADR + catalog docs.Cleanest deeper product; keeps /hello + /fsdemo intact.
H2 (rejected)Teach fs-libctos to also open /probe.Collapses memfs vs FAT recipes; muddies ADR-059 markers.
H3 (rejected)EL0 create/write/read a new FAT root name this mile.Prefer existing /probe (already restored to fat-hi before this probe). Write-to-FAT from EL0 is already routed but not required for the product claim.
H4 (rejected)Replace /fsdemo or /hello.Breaks A9 / ADR-059 cross-update and markers.
H5 (rejected)Claim POSIX open / Linux VFS / “EL0 isolated.”Honesty forbid.

Decision

  1. Sample crate. user/fat-libctos: same aarch64-ctos-user / linker.ld / build-std as hello/fs. main prints libctos: fat-hi, yields, then fs_open / fs_read / fs_close on /probe, then libctos: fat-ok. Fail path prints libctos: fat-fail and exits non-zero.
  2. Path choice. /probe resolves to FAT16 under ADR-058 (/ → FAT). Payload bytes are the existing A7 fat-hi (restored by fat::observe_probe before this slot runs).
  3. Publish. build.rs builds and publishes target/fat-libctos.elf. scripts/mkfat16.py --app3 writes 8.3 FATDEMO. Keep /hello + /fsdemo + slot: ok + fsdemo: ok.
  4. Kernel probe. src/fatdemo.rs loads FAT /fatdemo, ERETs via loader::run_image_expecting (last uart len 16). Serial fatdemo: fat / fatdemo: mapped / fatdemo: ok. Runs after the ADR-059 fsdemo probe.
  5. Smoke. Fail-closed greps for the new markers; reject libctos: fat-fail; preserve all existing greps.
  6. Docs. This ADR; honesty ledger evidence-only; user/README.md; what-can-run.md; apps-today.md; SUMMARY. Threat-model v1.36 (NFR-10 text in place).
  7. EL0 exec pages. RX user pages are fetch-only for EL0. The sample may pass .rodata pointers to SVCs (EL1 copy) but must not LDR them at EL0; read-back compare uses immediates.
  8. Honesty. Do not claim: EL0 isolated, PAN enable, taken SError, Linux/POSIX/containers, immutable-OS marketing, or reopen “app hosting is done.” Never yank _start. No silent -cpu.

Consequences

  • Code: user/fat-libctos/, build.rs, scripts/mkfat16.py, scripts/qemu-aarch64.sh, scripts/qemu-smoke.sh, src/fatdemo.rs, src/main.rs.
  • Docs: this ADR + catalog / ledger / threat-model v1.36.
  • Follow-ups (not this PR): EL0 FAT create/write sample, EL0 readdir when wired, tree mounts with an extra /.

ADR-062 — Fourth freestanding sample (yield-libctos) — cooperative yield rounds

  • Status: Accepted (sample + smoke Verified when serial / tests pass; product freestanding app hosting remains Verified under ADR-048/052 — do not reopen)
  • Date: 2026-09-16

Context

ADR-022 / hello prints libctos: hi, yields once, then libctos: ok. ADR-059 and ADR-061 deepened the catalog on VFS/FAT. Product gap remains: no freestanding sample that exercises several cooperative yield_now() rounds from standing EL0 (still single-task; not preemption).

This mile adds a fourth freestanding EL0 sample that yields across multiple uart beats, publishes it on FAT beside /hello / /fsdemo / /fatdemo, expands the catalog, and fail-closes smoke on new markers. Honesty: not POSIX, not Linux, not “EL0 isolated,” not preemption, not a process table.

Hypotheses

IDOptionNotes
H1 (chosen)New crate user/yield-libctos. Print libctos: yld-hi, then several libctos: beat across yield_now() rounds, then libctos: yld-ok. FAT slot /yldemo (8.3 YLDEMO). Kernel loads after ADR-061 fatdemo. Serial greps + yldemo: ok. ADR + catalog docs.Cleanest distinct fourth product; keeps prior three intact.
H2 (rejected)Teach hello-libctos to loop yields.Collapses A2 markers; muddies hello smoke greps.
H3 (rejected)Preemptive timer slice / multi-task EL0 / process table this mile.Out of honesty scope; cooperative only.
H4 (rejected)Replace /hello or another slot.Breaks A9 / ADR-059 / ADR-061 cross-update and markers.
H5 (rejected)Claim POSIX / Linux / “EL0 isolated” / preemption.Honesty forbid.

Decision

  1. Sample crate. user/yield-libctos: same aarch64-ctos-user / linker.ld / build-std as hello/fs/fat. main prints libctos: yld-hi, then ≥3 libctos: beat lines each after yield_now(), then one more yield and libctos: yld-ok.
  2. Publish. build.rs builds and publishes target/yield-libctos.elf. scripts/mkfat16.py --app4 writes 8.3 YLDEMO. Keep /hello + /fsdemo + /fatdemo + their *: ok markers.
  3. Kernel probe. src/yldemo.rs loads FAT /yldemo, ERETs via loader::run_image_expecting (last uart len 16 for libctos: yld-ok\n), and requires syscall::yield_count() >= 3. Serial yldemo: fat / yldemo: mapped / yldemo: ok. Runs after the ADR-061 fatdemo probe.
  4. Smoke. Fail-closed greps for the new markers (≥3 libctos: beat); preserve all existing greps.
  5. Docs. This ADR; honesty ledger evidence-only; user/README.md; what-can-run.md; apps-today.md; SUMMARY. Threat-model v1.37 (NFR-10 text in place).
  6. Honesty. Do not claim: EL0 isolated, PAN enable, taken SError, Linux/POSIX/containers, preemption, multi-task EL0, process table, immutable-OS marketing, or reopen “app hosting is done.” Never yank _start. No silent -cpu.

Consequences

  • Code: user/yield-libctos/, build.rs, scripts/mkfat16.py, scripts/qemu-aarch64.sh, scripts/qemu-smoke.sh, src/yldemo.rs, src/main.rs.
  • Docs: this ADR + catalog / ledger / threat-model v1.37.
  • Follow-ups (not this PR): preemptive scheduling (separate ADR), multi-task EL0, EL0 FAT create/write sample.

ADR-063 — Track N — network foundation scope (virtio-net first frame)

  • Status: Accepted (docs scope; Planned until N1 probes — do not invent Verified)
  • Date: 2026-09-16

Context

Sponsor treats networking as a new track (Track N), not a side edit on filesystem / freestanding-sample PRs. Today the guest has virtio-mmio + virtio-blk (ADR-028, src/virtio.rs) and no NIC / no virtio-net (limits.md). Track A freestanding hosting is Verified under ADR-048 / ADR-052. Track B Linux-compat research ended never (ADR-036).

This mile (N0) is document-first: name the track, lock foundation scope and non-goals, publish a mile ladder, and wire SUMMARY / roadmap / honesty ledger / limits. No virtio-net driver code in this PR unless trivial scaffolding is needed for honesty — prefer docs-only. File presence of this ADR is not a network probe.

Hypotheses

IDOptionNotes
H1 (chosen)Open Track N (network) as a separate ladder from Track A/B. N0 = this ADR (scope). Foundation target: QEMU virt + virtio-net (mmio); discover device; TX/RX one raw Ethernet frame; serial markers + smoke + QEMU -netdev / -device virtio-net-device; measure-first; fail-closed.Matches sponsor “new track” + honesty. Reuses virtio-mmio patterns from blk.
H2 (rejected)Fold networking into Track A FS / sample PRs as a side edit.Sponsor forbids; muddies A7/FAT/smoke ownership.
H3 (rejected)Ship a TCP/UDP stack, BSD sockets, DHCP/DNS product, or “has networking” marketing this mile.Honesty fail. Locked non-goals until a new ADR + sponsor.
H4 (rejected)Claim Verified for link bring-up or first frame from this docs PR.No guest probe yet. Status stays Planned.
H5 (rejected)virtio-pci-only or Wi‑Fi stories as the foundation path.Keep learning scope on QEMU virt + virtio-mmio (same as blk).

Decision

  1. Track name. Track N (network). Separate from Track A (freestanding app hosting) and Track B (Linux-compat research). Subordinate to principles.md. Hub: track-n.md.

  2. In scope for foundation (N1 and immediate follow-ons under this ADR’s intent).

    • QEMU virt + virtio-net over virtio-mmio (scan transports like blk; Device ID for net).
    • Discover the device; program queues enough to TX and RX one raw Ethernet frame.
    • Serial markers + fail-closed scripts/qemu-smoke.sh greps.
    • Host QEMU flags: -netdev + -device virtio-net-device (exact args named in the N1 PR).
    • Measure-first (NFR-07): CNTPCT or similar only when a path exists; no invented benches.
    • Fail-closed: missing markers / probe missed fail smoke. Do not break existing FAT / blk / slot / sample greps.
  3. Explicit non-goals (locked until a new ADR + sponsor).

    • TCP/UDP stack as product.
    • BSD sockets API as product.
    • DHCP / DNS as product features.
    • Wi‑Fi.
    • virtio-pci-only stories as the foundation path.
    • Linux network stack / “runs Linux networking.”
    • Marketing “has networking” / “network-ready OS.”
    • EL0 net SVC / ABI until after link bring-up is Verified (N1+) — unlocked by ADR-068.
  4. Mile ladder.

    IDWorkProbe that closes itStatus
    N0This ADR — scope, non-goals, wiringDocs-build; ledger Planned row; no Verified inventAccepted (docs) — this PR
    N1Link bring-up / first frameSerial markers for discover + TX and/or RX one raw frame; smoke greps; QEMU -netdev/-device; keep blk/FAT smoke greenVerified via ADR-066 when tip prints net: ok
    N2ARP + ICMP pingNamed markers + fail-closed smoke; still no sockets productVerified via ADR-067 when tip prints net: ping-ok
    N3Transport / socketsOnly with new sponsor scope + new ADRLocked out until that ADR
    N4Freestanding sample using net SVCsAfter N1+ Verified; catalog honesty like ADR-059/061/062Verified via ADR-068 when tip prints libctos: net-ok / netdemo: ok
  5. Reuse. Reuse virtio-mmio patterns from blk (ADR-028, src/virtio.rs): transport scan, split virtqueue, poll used ring, identity-PA DMA + cache maintenance. Do not break FAT/blk smoke or A9/slot/sample markers. Prefer extending virtio carefully over a second transport story.

  6. Honesty.

    • Do not invent Verified for networking from this PR.
    • Ledger status for Track N foundation / first frame stays Planned until N1 probes pass.
    • Do not claim: TCP/UDP, sockets, DHCP/DNS product, Wi‑Fi, Linux net stack, “has networking,” EL0 net ABI, “secure OS,” or reopen Track B Linux-compat.
    • No silent -cpu. Never yank _start. Author does not self-merge (ADR-002).
    • CloudAgent remains HELD; N1 evidence tags agent-box or the amd64 workstation as available.
  7. Docs wiring (this PR). SUMMARY + ADR list; roadmap Track N section; track-n.md; honesty-ledger Planned row; limits.md (“Track N Planned — see ADR”); index one-liner; threat-model light bump (security.md v1.38 — I/O surface names Track N Planned, still no stack).

  8. No new FR/NFR IDs. Frozen set stays fr-nfr.md. Cite existing NFR-06 / NFR-07 / NFR-10 / FR-14 as needed.

Consequences

  • Docs: this ADR; track-n.md; SUMMARY; roadmap; honesty ledger; limits; index; threat-model v1.38.
  • Code: none in N0 (docs-only). N1 may add virtio-net guest code + QEMU netdev in a later PR.
  • Follow-ups: N1 link bring-up / first frame PR; optional N2 ARP+ICMP; N3+ only with new sponsor ADR.

ADR-064 — FAT16 multi-cluster grow (write depth)

  • Status: Accepted (multi-cluster grow mile Verified when the serial / tests / EL0 markers pass; Track A product claims unchanged; Track N untouched)
  • Date: 2026-09-16

Context

ADR-050 landed guest FAT16 write + small create, explicitly refusing multi-cluster grow (cap: one cluster / FILE_MAX 256). ADR-057 frees a cluster chain (already walked to EOC). ADR-061 added an EL0 FAT read sample on /probe. Reads of multi-cluster host ELFs (/hello, …) already walked the FAT chain; guest write still stopped at one cluster.

Docs “new vs extend” prefer deepening the known small FS (FAT16) behind the thin VFS. This mile is honest multi-cluster file grow: allocate a FAT chain, update dirent size, write across a cluster boundary, prove with kernel serial + unit tests + EL0 chunked SVC write/read-back.

Not exFAT. Not FAT32. Not a POSIX full write API. Not Track N / virtio-net (ADR-063 landed on main as docs-only N0 — do not collide; this mile is 064 only).

Hypotheses

IDOptionNotes
H1 (chosen)Extend fat::write_at to allocate additional clusters when end crosses a cluster boundary; raise FILE_MAX to 1024 (enough for two 512-byte clusters). Kernel probe /fgrow (600 bytes). EL0 fat-libctos creates /egrow and chunked-writes via existing SVC 19–23 (FS_IO_MAX 64). Serial fat: grow / libctos: fat-grow.Deepens ADR-050. Same path grammar. Fail-closed.
H2 (rejected)Claim POSIX write / pwrite / sparse files / exFAT.Honesty forbid.
H3 (rejected)New SVC numbers or a second write ABI.Existing fs_create/fs_write/fs_read suffice.
H4 (rejected)Only kernel probe; no EL0 proof.Product-depth train wants EL0 via existing SVCs.
H5 (rejected)Touch Track N / virtio-net.Separate ladder; out of scope.

Decision

  1. FILE_MAX = 1024. Shared memfs/FAT cap. Was 256 (below one 512-byte cluster); multi-cluster grow is impossible under that ceiling. Still small; not a product volume claim.
  2. extend_chain / multi-cluster write_at. Walk the existing chain; allocate free clusters, link prior→new→EOC, zero new clusters; write payload across cluster boundaries. Refuse end > FILE_MAX and empty writes (TooBig). No sparse holes (off > size → BadHandle). Still assumes spc=1 (one sector per cluster) this mile.
  3. Kernel probe /fgrow. After delete probe: create, write 600 patterned bytes, confirm FAT chain_len ≥ 2, VFS read-back, boundary bytes differ. Serial fat: grow. Keep fat: ok / write / readdir / delete / A9.
  4. EL0 via existing SVCs. Extend user/fat-libctos: after /probe read, create-or-open /egrow, write 600 bytes in 64-byte SVC chunks, read-back boundary bytes with immediates. Serial libctos: fat-grow then libctos: fat-ok. Fail → libctos: fat-fail.
  5. Fail-closed smoke + tests. Grep fat: grow / libctos: fat-grow; reject missing. #[test_case] fat16_multi_cluster_grow + over-FILE_MAX refuse.
  6. Honesty. Say “the guest grew a FAT16 file across a cluster boundary through the thin VFS” only when markers pass. Do not say: POSIX write API, exFAT/FAT32, “supports FAT,” app hosting reopen, EL0 isolated, PAN, taken SError, networking.
  7. NFR-10 text revised in place (ID unchanged). Threat-model v1.39 (slice after #109 / ADR-063 threat-model v1.38). Do not mint NFR-15+.

Consequences

  • Code: src/fat.rs (extend_chain / write_at / grow probe), src/vfs.rs (FILE_MAX), user/fat-libctos/, scripts/qemu-smoke.sh.
  • Docs: this ADR, filesystem overview/stance, honesty ledger, SUMMARY, threat-model v1.39, light fr-nfr NFR-10 note.
  • Follow-ups (not this PR): spc>1, shrink-on-truncate, exFAT, POSIX write API claim, Track N N1.

ADR-065 — FAT16 vs memfs read CNTPCT pair (measure-first)

  • Status: Accepted (compared CNTPCT pair on one boot — QEMU TCG lab measurement, not a published bench).
  • Date: 2026-09-16

Context

ADR-051 landed a same-boot FAT vs memfs write CNTPCT pair (perf: fat-write / perf: memfs-write / perf: fs-write-delta). Existing performance probes also cover loop CNTPCT, IRQ delta, boot-delta, ELF size, and the A9 slot pair (ADR-046). Perf slice 3 asks for another measure-first compared pair around a known path. The natural symmetric cut is the small FAT /probe read versus a memfs read of the same bytes on the same boot.

Do not invent a latency SLA, percent, or “faster/slower” claim. Do not add a criterion crate. Keep fat: read / fat: write / fat: ok / A9 slot markers and the ADR-051 write pair. QEMU TCG jitter is one lab — not SPEC. Do not touch Track N / virtio-net.

Hypotheses

IDOptionNotes
H1 (chosen)Time the FAT VFS read of /probe (fat-hi) with CNTPCT → perf: fat-read ticks=<n>. On the same boot, seed memfs /mrprobe with the same bytes and time vfs::read → perf: memfs-read ticks=<n>. Print perf: fs-read-delta fat=<a> memfs=<b> (raw ticks).Honest compared pair. Not a bench.
H2 (rejected)Publish a percent / “FAT is slower than memfs” / criterion bench.NFR-07 forbids invented benches.
H3 (rejected)Time open+read+close as one number and call it “read latency.”Too wide; pair the read of the same payload (open outside the window).
H4 (rejected)Only print FAT ticks without a memfs compare.Prefer a same-boot pair when feasible (same shape as ADR-046 / ADR-051).
H5 (rejected)Time SVC yield / memcpy instead of FAT read this mile.Useful later; FAT read is the direct follow-on to ADR-051.

Decision

  1. FAT read sample. During the existing /probe VFS read proof, sample CNTPCT around the single vfs::read that expects PROBE_BYTES (fat-hi). Store ticks; fail closed if the counter does not advance.
  2. Memfs compare. After FAT probes that store the FAT sample, vfs::create/open a memfs path (/mrprobe — added to the ADR-058 prefix table so it does not fall through to FAT /), write the same bytes outside the timed window, reopen, and sample CNTPCT around vfs::read. Fail closed if ticks are zero or bytes mismatch.
  3. Markers. Serial: perf: fat-read ticks=<n>, perf: memfs-read ticks=<n>, perf: fs-read-delta fat=<a> memfs=<b>. No ratio percent; no “faster/slower” words on the serial line. Missed samples print perf: fat-read missed / perf: memfs-read missed.
  4. Existing markers unchanged. Keep fat: read / fat: write / ADR-051 write pair / A9 slot:* / ADR-046 pair. Grow / readdir / delete / Track N untouched.
  5. Smoke. scripts/qemu-smoke.sh greps the new markers and rejects fs-read-delta lines that claim percent/faster/slower.
  6. Honesty. This is a measurement pair on QEMU virt TCG. It is not a latency budget, not a product KPI, and not “FAT read cost is X.” Record Verified only with serial evidence on a named tip. Do not claim app hosting done, EL0 isolated, POSIX, or FAT32.

Consequences

  • Code: src/fat.rs (CNTPCT around FAT read + memfs compare + markers); smoke greps; #[test_case] that both samples are > 0.
  • Docs: this ADR, performance.md, measure.md, honesty ledger, roadmap P-PERF-7, threat-model light bump.
  • Follow-ups (not this PR): yield/SVC CNTPCT pair, memcpy baseline, multi-cluster read cost — only after a probe shows a need.

ADR-066 — Track N / N1 — virtio-net first Ethernet frame

  • Status: Accepted (N1 runtime — Verified only when this tip’s qemu-smoke prints net: ok; else Documented/partial)
  • Date: 2026-09-16

Context

ADR-063 opened Track N (docs N0). This mile is N1: discover virtio-net on virtio-mmio, program RX+TX queues enough to send and receive one raw Ethernet frame, with serial markers, fail-closed smoke, and host QEMU -netdev / -device virtio-net-device. Reuse virtio-mmio patterns from blk (ADR-028, src/virtio.rs). Do not break FAT/blk/slot/sample/perf greps.

Hypotheses

IDOptionNotes
H1 (chosen)Discover Device ID 1 on the same mmio scan as blk. Setup queue 0 (RX) + queue 1 (TX). Post one RX buffer. TX a broadcast ARP request for QEMU SLIRP gateway 10.0.2.2 from guest 10.0.2.15. Wait for TX used.idx, then RX ARP reply from SLIRP. Markers + smoke. Host: -netdev user,id=net0 -device virtio-net-device,netdev=net0.Deterministic on GHA ubuntu (no external network). Proves both directions.
H2 (rejected as primary)TX-only: prove used.idx after TX without RX.Allowed as a fallback minimum by ADR-063, but SLIRP ARP reply is available and stronger. Prefer both.
H3 (rejected)Depend on DHCP/DNS product, outbound Internet, or a second peer VM.Flaky / product creep. Fail-closed CI must not need external net.
H4 (rejected)Inject a crafted RX via QMP / filter without guest TX.Extra host machinery; ARP round-trip is enough and exercises TX+RX.
H5 (rejected)virtio-pci-only, Wi‑Fi, TCP/UDP stack, BSD sockets, EL0 net SVC this mile.Locked non-goals (ADR-063).
H6 (rejected)Invent Verified without net: ok on a named tip.Honesty.

Decision

  1. Transport. Same virtio-mmio scan (0x0a000000 + n×0x200, 32 slots). Device ID 1 = net. Accept legacy v1 (QueuePFN) and modern v2 (desc/avail/used + QueueReady), matching blk. Identity-PA DMA + whole-object dc civac cache maintenance. Poll used.idx. No virtio IRQ this mile.

  2. Queues. Queue 0 = receiveq, queue 1 = transmitq. Separate 4 KiB-aligned .bss DMA objects (legacy page layout). 10-byte virtio_net_hdr (no VIRTIO_NET_F_MRG_RXBUF). Negotiate VIRTIO_NET_F_MAC when offered; read MAC from config bytes 0–5.

  3. Honest probe (CI-deterministic).

    • Post one RX WRITE buffer (hdr+frame).
    • TX Ethernet+ARP request: who-has 10.0.2.2 tell 10.0.2.15, src = guest MAC, dst = broadcast.
    • QEMU user netdev SLIRP answers ARP for its gateway locally — no Internet, no DHCP client, no second VM.
    • Require TX completion and an ARP reply whose spa is 10.0.2.2.
  4. Host QEMU args (exact). Wired into scripts/qemu-aarch64.sh and scripts/qemu-serial-inject.py (hello smoke path):

    -netdev user,id=net0
    -device virtio-net-device,netdev=net0
    

    Kept alongside existing -drive … -device virtio-blk-device,drive=hd0. Host flags without guest markers are not a probe.

  5. Serial markers.

    • net: virtio — discovered + queues programmed
    • net: mac xx:xx:xx:xx:xx:xx — config MAC
    • net: tx — TX used ring advanced for the ARP request
    • net: rx — RX used ring delivered an ARP reply from 10.0.2.2
    • net: ok — both directions proven
    • Miss → net: probe missed (fail-closed)
    • Optional cheap CNTPCT: perf: net-tx ticks=<n> / perf: net-rx ticks=<n> when the counter advances (lab only, not a bench)
  6. Smoke. scripts/qemu-smoke.sh greps the markers above and rejects net: probe missed. Must keep existing blk:* / fat:* / slot / sample / perf greps green.

  7. Non-goals restated (locked). TCP/UDP product; BSD sockets; DHCP/DNS product; Wi‑Fi; virtio-pci-only foundation; Linux net stack; “has networking” marketing; EL0 net SVC/ABI. N2 ICMP is ADR-067; N3+ needs new sponsor ADR.

  8. Honesty. Say “the guest programmed virtio-net and exchanged one ARP request/reply with QEMU SLIRP” only when serial/qemu-smoke print net: ok on a named tip. Do not say the OS “has networking.” CloudAgent HELD; tag agent-box / amd64 workstation. No self-merge (ADR-002).

Consequences

  • Code: extend src/virtio.rs (net discover/setup/TX/RX + observe_net_probe); src/main.rs calls init_net + probe; QEMU runners + smoke greps; #[test_case] virtio_net_discover_and_arp.
  • Docs: this ADR; track-n.md; SUMMARY; roadmap; honesty ledger; limits.md; threat-model bump; index one-liner.
  • Follow-ups: N2 ICMP ping — ADR-067; N3/N4 locked out until new ADR + sponsor.

ADR-067 — Track N / N2 — ICMP ping after ARP (kernel-path)

  • Status: Accepted (N2 runtime — Verified only when this tip’s qemu-smoke prints net: ping-ok; else Documented/partial)
  • Date: 2026-09-16

Context

ADR-063 opened Track N. ADR-066 closed N1: virtio-net discover + ARP request/reply vs QEMU SLIRP gateway 10.0.2.2 (net: ok). This mile is N2: after (or with) that ARP exchange, send an ICMP echo request from guest 10.0.2.15 to gateway 10.0.2.2 and receive the echo reply — still on the kernel path only. Reuse N1 queues/DMA. Do not break blk/FAT/slot/net: ok greps.

Hypotheses

IDOptionNotes
H1 (chosen)Keep N1 ARP. Capture gateway MAC from ARP sha. Post a new RX buffer. TX Ethernet+IPv4+ICMP echo request (type 8) to 10.0.2.2. Wait for ICMP echo reply (type 0) with matching id/seq. Markers + fail-closed smoke. QEMU args unchanged: -netdev user,id=net0 -device virtio-net-device,netdev=net0.Deterministic on GHA/agent-box (SLIRP answers ICMP locally). Proves L3 echo without a TCP/UDP stack.
H2 (rejected)Add an EL0 net SVC / freestanding ping sample this mile.N4 territory; prefer kernel-path ping only (sponsor).
H3 (rejected)Depend on DHCP/DNS product, outbound Internet, TCP/UDP, or sockets.Locked non-goals (ADR-063).
H4 (rejected)Invent Verified without net: ping-ok on a named tip.Honesty.
H5 (rejected)Drop or weaken N1 ARP markers.Keep net: virtio / net: mac / net: tx / net: rx / net: ok.

Decision

  1. Order. Run N1 ARP first (learn gateway MAC from ARP sha). Then ICMP echo TX/RX on the same virtio-net device. Same poll-used.idx / identity-PA DMA pattern as N1.

  2. Packet. Ethernet (dst = gateway MAC, src = guest MAC, ethertype IPv4) + IPv4 (src 10.0.2.15, dst 10.0.2.2, proto ICMP, TTL 64) + ICMP echo request (type 8, code 0, fixed id/seq, small payload ctos-n2). Ones-complement checksums for IP header and ICMP message. No IP options. No TCP/UDP.

  3. Honest probe. Require ICMP echo reply (type 0) from 10.0.2.2 to 10.0.2.15 with matching id/seq. Host -netdev without guest markers is not a probe. QEMU user-netdev args stay as in ADR-066 unless a future ADR names a change (none needed here).

  4. Serial markers (N2).

    • net: icmp-tx — TX used ring advanced for the ICMP echo request
    • net: icmp-rx — RX delivered a matching ICMP echo reply
    • net: ping-ok — N2 closed
    • Keep N1 markers including net: ok
    • Miss → net: probe missed (fail-closed)
    • Optional cheap CNTPCT: perf: net-icmp-tx ticks=<n> / perf: net-icmp-rx ticks=<n> (lab only)
  5. Smoke. scripts/qemu-smoke.sh greps N1 markers and net: icmp-tx / net: icmp-rx / net: ping-ok. Must keep blk:* / fat:* / slot / sample / perf greps green.

  6. Non-goals restated (locked). TCP/UDP product; BSD sockets; DHCP/DNS product; Wi‑Fi; virtio-pci-only foundation; Linux net stack; “has networking” marketing; EL0 net SVC/ABI this PR (N4 later). N3+ needs new sponsor ADR.

  7. Honesty. Say “the guest ARP’d the SLIRP gateway and completed one ICMP echo request/reply” only when serial/qemu-smoke print net: ping-ok on a named tip. Do not say the OS “has networking.” CloudAgent HELD; tag agent-box / amd64 workstation. No self-merge (ADR-002).

Consequences

  • Code: extend src/virtio.rs (fill_icmp_echo_request, ICMP RX match, observe_net_probe after ARP); smoke greps; #[test_case] virtio_net_icmp_echo_ping (+ keep ARP test).
  • Docs: this ADR; track-n.md; SUMMARY; roadmap; honesty ledger; limits.md; threat-model bump; plain overview/architecture mermaid; index one-liner.
  • Follow-ups: N3 sockets / N4 freestanding net sample locked out until new ADR + sponsor.

ADR-068 — Track N / N4 — EL0 net SVC + freestanding net-libctos sample

  • Status: Accepted (sample + smoke Verified when serial / tests pass; product freestanding app hosting remains Verified under ADR-048/052 — do not reopen)
  • Date: 2026-09-16

Context

ADR-063 locked EL0 net ABI until after link bring-up Verified. ADR-066 closed N1 (net: ok). ADR-067 closed N2 (net: ping-ok). Catalog freestanding samples exist for hello / VFS / FAT / yield (ADR-059 / ADR-061 / ADR-062).

This mile (N4) unlocks a tiny EL0 net SVC surface and a fifth freestanding sample that proves net via SVCs only. Kernel still owns virtio-net. Do not break blk/FAT/slot/N1/N2/sample greps.

Hypotheses

IDOptionNotes
H1 (chosen)Add SYS_NET_MAC (24) + SYS_NET_PING (25). New crate user/net-libctos on FAT /netdemo. Print libctos: net-hi / libctos: net-mac / libctos: net-ok. Kernel netdemo: ok. ADR + catalog + smoke.Minimal ABI; proves MAC read + kernel-path ICMP from EL0 without a guest driver.
H2 (rejected)Expose raw Ethernet frame TX/RX buffers to EL0 this mile.Larger surface; prefer ping-as-proof.
H3 (rejected)Ship BSD sockets / TCP/UDP / DHCP/DNS product.Locked non-goals (ADR-063).
H4 (rejected)Teach EL0 to program virtio-net MMIO.Honesty: kernel owns the NIC; SVCs only.
H5 (rejected)Replace /hello or another slot.Breaks A9 / catalog greps.

Decision

  1. SVC surface (document in syscall.md).

    • 24 net_mac — x0 = user buf, x1 = length (≥6). Copy 6-byte guest MAC. Return 6, or 0 if rejected / net not ready.
    • 25 net_ping — no args. Kernel does quiet ARP + ICMP echo vs SLIRP 10.0.2.2 (reuse N1/N2 helpers). Return 0 on success, u64::MAX on fail.
    • Unknown SVCs still park. Not sockets. Not a process ABI.
  2. Sample crate. user/net-libctos: same aarch64-ctos-user / linker.ld / build-std as prior samples. main prints libctos: net-hi, calls net_mac then libctos: net-mac, calls net_ping, then libctos: net-ok.

  3. Publish. build.rs builds and publishes target/net-libctos.elf. scripts/mkfat16.py --app5 writes 8.3 NETDEMO. Keep /hello + /fsdemo + /fatdemo + /yldemo + their *: ok markers.

  4. Kernel probe. src/netdemo.rs loads FAT /netdemo, ERETs via loader::run_image_expecting (last uart len 16 for libctos: net-ok\n). Serial netdemo: fat / netdemo: mapped / netdemo: ok. Runs after the ADR-062 yldemo probe. Requires N1/N2 virtio-net already brought up (init_net).

  5. Smoke. Fail-closed greps for the new markers; preserve all existing greps (blk/fat/slot/net/ping/samples).

  6. Docs. This ADR; honesty ledger evidence-only; user/README.md; what-can-run.md; apps-today.md; track-n.md; SUMMARY; threat-model v1.43.

  7. Honesty. Do not claim: TCP/UDP product, BSD sockets, DHCP/DNS, Wi‑Fi, “has networking,” EL0 virtio driver, Linux/POSIX, EL0 isolated, or reopen ADR-048/052. CloudAgent HELD; tag agent-box / amd64 workstation. No self-merge (ADR-002).

Consequences

  • Code: libctos + sys.S; src/syscall.rs; src/virtio.rs (guest_mac / el0_icmp_ping); user/net-libctos/; src/netdemo.rs; build.rs; scripts/mkfat16.py; scripts/qemu-aarch64.sh; scripts/qemu-smoke.sh; src/main.rs.
  • Docs: this ADR + catalog / ledger / threat-model v1.43 / track-n N4.
  • Follow-ups (not this PR): N3 sockets (locked); richer EL0 frame I/O only with new ADR + sponsor.

ADR-069 — Net-ping path CNTPCT (measure-first)

  • Status: Accepted (lab CNTPCT on QEMU TCG — not a published bench).
  • Date: 2026-09-16

Context

ADR-065 landed a same-boot FAT vs memfs read CNTPCT pair. ADR-067 closed N2 kernel-path ICMP (net: ping-ok) with optional crumbs perf: net-icmp-tx / perf: net-icmp-rx. ADR-068 closed N4 with EL0 SYS_NET_PING (25) and freestanding /netdemo.

Perf slice next asks for another measure-first probe around a known path. The natural cut is the EL0 net_ping quiet path (kernel-owned ARP + ICMP echo vs SLIRP 10.0.2.2) that ADR-068 already exercises.

Do not invent a latency SLA, percent, or “faster/slower” claim. Do not add a criterion crate. Do not unlock N3 sockets / TCP/UDP. Keep N1/N2/N4 markers and all prior smoke greps.

Hypotheses

IDOptionNotes
H1 (chosen)Sample CNTPCT around the full quiet el0_icmp_ping body (ARP + ICMP echo RTT as SYS_NET_PING does it). Print perf: net-ping ticks=<n>. Fail closed if the counter does not advance after a successful ping. Smoke greps the prefix perf: net-ping (not an exact tick count).Honest single-path measurement. Values vary under TCG.
H2 (rejected)Publish a percent / “ping is X µs” / criterion bench / SLA.NFR-07 forbids invented benches.
H3 (rejected)Only time the kernel N2 icmp-tx or icmp-rx crumb and rename it net-ping.Those optional crumbs stay as ADR-067 lab notes; ADR-069 names the EL0 SVC round-trip.
H4 (rejected)Require an exact tick count in smoke.Flaky under QEMU TCG jitter.
H5 (rejected)Add TCP/UDP/sockets or widen the EL0 net ABI this mile.N3 locked (ADR-063).

Decision

  1. Window. In virtio::el0_icmp_ping (called from SYS_NET_PING), sample CNTPCT from before the quiet ARP post through a matching ICMP echo reply. That is the EL0 net_ping round-trip the sample already runs — not “network latency” as a product KPI.
  2. Marker (exact form). On success with a non-zero delta: perf: net-ping ticks=<n> (decimal ticks, no units, no percent). If the ping succeeds but the counter did not advance: perf: net-ping missed and treat the probe as failed. Related existing optional crumbs (unchanged): perf: net-icmp-tx ticks=<n> / perf: net-icmp-rx ticks=<n> on the kernel N2 probe path.
  3. Smoke. scripts/qemu-smoke.sh greps for the prefix perf: net-ping (presence). Rejects lines that claim faster/slower/percent. Does not assert a numeric tick value. Keep all prior greps (blk/fat/slot/net/ping/samples/ADR-065 pair).
  4. Test. #[test_case] el0_net_ping_cntpct_advances asserts a successful quiet ping and net_ping_ticks() > 0.
  5. Honesty. QEMU virt TCG lab only. Not a latency budget, not SPEC, not “has networking,” not sockets. Record Verified only with serial evidence on a named tip. CloudAgent HELD; tag agent-box / amd64 workstation. No self-merge (ADR-002).

Consequences

  • Code: src/virtio.rs (el0_icmp_ping CNTPCT + perf: net-ping + net_ping_ticks); smoke greps; #[test_case].
  • Docs: this ADR, performance.md, measure.md, honesty ledger, roadmap P-PERF-8, SUMMARY, threat-model light bump (v1.44).
  • Follow-ups (not this PR): yield/SVC CNTPCT pair, memcpy baseline — only after a probe shows a need. N3 sockets stay locked.

ADR-070 — Track N / N3 — UDP transport unlock (minimal IPv4 datagram)

  • Status: Accepted (N3 runtime — Verified only when this tip’s qemu-smoke prints net: udp-ok; else Documented/partial)
  • Date: 2026-09-16

Context

ADR-063 opened Track N and left N3 (transport / sockets) locked until a new sponsor scope ADR. N1 (ADR-066) Verified ARP first-frame (net: ok). N2 (ADR-067) Verified ICMP echo (net: ping-ok). N4 (ADR-068) Verified tiny EL0 net_mac / net_ping + /netdemo. Perf slice ADR-069 measures EL0 net_ping CNTPCT.

Sponsor unlocked N3: proceed with a real transport mile, not docs-only forever. Prefer UDP datagram first (measurable, fail-closed on SLIRP) over a fake POSIX sockets layer. Do not ship a full BSD sockets / TCP product this PR.

Hypotheses

IDOptionNotes
H1 (chosen)After N1 ARP + N2 ICMP on the same virtio-net path: TX one IPv4 UDP datagram to QEMU SLIRP DNS 10.0.2.3:53 (minimal DNS query as payload), RX any matching DNS response (QR=1, same TXID). Markers net: udp-tx / net: udp-rx / net: udp-ok. QEMU args unchanged.Deterministic on GHA/agent-box (SLIRP answers UDP DNS locally). Proves L4 datagram without sockets/TCP. DNS here is probe bait only, not a DNS product.
H2 (rejected)QEMU guestfwd UDP echo this mile.Extra host plumbing; H1 is enough if green. Revisit only if SLIRP DNS proves flaky.
H3 (rejected)Ship a BSD sockets ABI / TCP stack / “has networking / sockets OS” marketing.Honesty fail; locked product non-goals.
H4 (rejected)Docs-only unlock with no guest probe.Sponsor asked for a real transport mile.
H5 (rejected)Require EL0 UDP SVCs this mile.Optional later (N3.x); kernel-path probe matches N2 style. Keep existing net_mac/net_ping.

Decision

  1. Unlock N3 under this ADR. Ladder status moves from Locked → Verified when this tip’s qemu-smoke prints net: udp-ok (with N1/N2 markers preserved).

  2. In scope this mile.

    • Minimal UDP TX/RX (IPv4) on existing virtio-net (reuse ARP/IP helpers from the ICMP path).
    • Honest CI probe: UDP DNS query from guest 10.0.2.15 to SLIRP DNS 10.0.2.3:53; accept any DNS response with matching transaction id and QR=1 (any RCODE).
    • Serial markers + fail-closed smoke greps.
    • Optional thin EL0 UDP SVC(s): not this PR (natural follow-up only with another ADR / N3.x).
  3. Still not product (locked).

    • Full BSD sockets API.
    • TCP stack as product.
    • DHCP / DNS as product features (the probe may use SLIRP DNS as a peer; that is not a guest DNS resolver product).
    • Wi‑Fi; Linux net stack; marketing “has networking / sockets OS.”
  4. Packet. Ethernet (dst = gateway MAC from N1 ARP sha) + IPv4 (src 10.0.2.15, dst 10.0.2.3, proto UDP, TTL 64) + UDP (src ephemeral 0x6333, dst 53, checksum 0 allowed for IPv4) + minimal DNS A query for label ctos (TXID 0x6333). No IP options. No TCP. No sockets ABI.

  5. Serial markers (N3).

    • net: udp-tx — TX used ring advanced for the UDP datagram
    • net: udp-rx — RX delivered a matching UDP DNS response
    • net: udp-ok — N3 closed
    • Keep N1 (net: ok) and N2 (net: ping-ok) markers
    • Miss → net: probe missed (fail-closed)
  6. Smoke. scripts/qemu-smoke.sh greps N1 + N2 and net: udp-tx / net: udp-rx / net: udp-ok. Must keep blk:* / fat:* / slot / sample / perf greps green.

  7. Future N3.x. TCP and/or richer socket ABI only with another ADR + sponsor. Do not silently grow into a sockets product.

  8. Honesty. Say “the guest sent one UDP datagram and received a UDP reply via SLIRP” only when serial/qemu-smoke print net: udp-ok on a named tip. Do not say the OS “has networking,” “has sockets,” or “has DNS.” CloudAgent HELD; tag agent-box / amd64 workstation. No self-merge (ADR-002).

Consequences

  • Code: extend src/virtio.rs (fill_udp_dns_query, UDP RX match, observe_net_probe after ICMP); smoke greps; #[test_case] virtio_net_udp_dns_probe.
  • Docs: this ADR; track-n.md; SUMMARY; roadmap; honesty ledger; limits.md; threat-model bump (v1.45); plain overview/architecture; index one-liner.
  • Follow-ups: EL0 UDP SVC / TCP / sockets ABI — new ADR only.

ADR-071 — Track N / N3.x — EL0 UDP SVC + freestanding udp-libctos sample

  • Status: Accepted (sample + smoke Verified when serial / tests pass; product freestanding app hosting remains Verified under ADR-048/052 — do not reopen)
  • Date: 2026-09-16

Context

ADR-070 closed N3 kernel-path UDP (net: udp-ok) and deferred EL0 UDP SVCs to N3.x. N4 (ADR-068) already Verified tiny EL0 net_mac (24) / net_ping (25) + FAT /netdemo. Catalog samples exist for hello / VFS / FAT / yield / net.

Sponsor: proceed as proposed proactively (N3.x). Keep the surface tiny and honest. Do not ship BSD sockets, TCP product, or DHCP/DNS as product. Kernel still owns virtio-net.

Hypotheses

IDOptionNotes
H1 (chosen)One combined SYS_NET_UDP_DNS (26) that mirrors the kernel N3 UDP DNS probe (quiet ARP + UDP query to SLIRP 10.0.2.3:53 + matching QR=1 reply). New crate user/udp-libctos on FAT /udpdemo. Print libctos: udp-hi / libctos: udp-ok. Kernel udpdemo: ok.Tiny ABI; same probe bait as ADR-070; no send/recv buffers this mile.
H2 (rejected)Pair net_udp_send / net_udp_recv with user buffers this mile.Larger surface; H1 is enough to prove EL0 can trigger the existing kernel UDP path.
H3 (rejected)Ship BSD sockets / TCP / DHCP/DNS product.Locked non-goals (ADR-063 / ADR-070).
H4 (rejected)Teach EL0 to program virtio-net MMIO or copy raw frames.Honesty: kernel owns the NIC; SVCs only.
H5 (rejected)Replace /netdemo or another slot.Breaks N4 / catalog greps.

Decision

  1. SVC surface (document in syscall.md).

    • Keep 24 net_mac and 25 net_ping (ADR-068).
    • 26 net_udp_dns — no args. Kernel does quiet ARP + one UDP DNS query vs SLIRP 10.0.2.3:53 (reuse N3 helpers). Return 0 on success, u64::MAX on fail.
    • Unknown SVCs still park. Not sockets. Not a process ABI. DNS remains probe bait, not a guest resolver.
  2. Sample crate. user/udp-libctos: same aarch64-ctos-user / linker.ld / build-std as prior samples. main prints libctos: udp-hi, calls net_udp_dns, then libctos: udp-ok.

  3. Publish. build.rs builds and publishes target/udp-libctos.elf. scripts/mkfat16.py --app6 writes 8.3 UDPDEMO. Keep /hello + /fsdemo + /fatdemo + /yldemo + /netdemo + their *: ok markers.

  4. Kernel probe. src/udpdemo.rs loads FAT /udpdemo, ERETs via loader::run_image_expecting (last uart len 16 for libctos: udp-ok\n). Serial udpdemo: fat / udpdemo: mapped / udpdemo: ok. Runs after the ADR-068 netdemo probe. Requires virtio-net already brought up (init_net).

  5. Smoke. Fail-closed greps for the new markers; preserve all existing greps (N1–N4, udp-ok, blk/FAT/samples/perf).

  6. Docs. This ADR; honesty ledger evidence-only; user/README.md; what-can-run.md; apps-today.md; track-n.md; SUMMARY; threat-model v1.46.

  7. Honesty. Do not claim: BSD sockets, TCP product, DHCP/DNS as product, Wi‑Fi, “has networking / sockets OS,” EL0 virtio driver, Linux/POSIX, EL0 isolated, or reopen ADR-048/052. CloudAgent HELD; tag agent-box / amd64 workstation. No self-merge (ADR-002).

Consequences

  • Code: libctos + sys.S; src/syscall.rs; src/virtio.rs (el0_udp_dns); user/udp-libctos/; src/udpdemo.rs; build.rs; scripts/mkfat16.py; scripts/qemu-aarch64.sh; scripts/qemu-smoke.sh; src/main.rs.
  • Docs: this ADR + catalog / ledger / threat-model v1.46 / track-n N3.x.
  • Follow-ups (not this PR): TCP / richer socket ABI / generic UDP send/recv — new ADR + sponsor only.

ADR-072 — UDP DNS path CNTPCT (measure-first)

  • Status: Accepted (lab CNTPCT on QEMU TCG — not a published bench).
  • Date: 2026-09-16

Context

ADR-069 landed CNTPCT around the EL0 net_ping quiet path. ADR-070 closed N3 kernel UDP (net: udp-ok). ADR-071 closed N3.x with EL0 SYS_NET_UDP_DNS (26) and freestanding /udpdemo.

Perf slice next asks for another measure-first probe around a known path. The natural cut is the EL0 net_udp_dns quiet path (kernel-owned ARP + UDP DNS query vs SLIRP 10.0.2.3:53) that ADR-071 already exercises — same pattern as ADR-069 on net_ping.

Do not invent a latency SLA, percent, or “faster/slower” claim. Do not add a criterion crate. Do not implement FAT mkdir or TCP. Keep N1/N2/N3/N3.x/N4 markers and all prior smoke greps (including ADR-069 perf: net-ping).

Hypotheses

IDOptionNotes
H1 (chosen)Sample CNTPCT around the full quiet el0_udp_dns body (ARP + UDP DNS RTT as SYS_NET_UDP_DNS / /udpdemo does it). Print perf: udp-dns ticks=<n>. Fail closed if the counter does not advance after a successful probe. Smoke greps the prefix perf: udp-dns (not an exact tick count).Honest single-path measurement. Values vary under TCG. Matches ADR-069.
H2 (rejected)Publish a percent / “DNS is X µs” / criterion bench / SLA.NFR-07 forbids invented benches.
H3 (rejected)Only time the kernel N3 net: udp-tx/udp-rx path and rename it udp-dns.Kernel N3 markers stay as ADR-070 transport proof; ADR-072 names the EL0 SVC round-trip (same choice as ADR-069 H3).
H4 (rejected)Require an exact tick count in smoke.Flaky under QEMU TCG jitter.
H5 (rejected)Add TCP / FAT mkdir / sockets or widen the EL0 net ABI this mile.Out of scope; sponsor mile (2) is FAT mkdir — not this PR.

Decision

  1. Window. In virtio::el0_udp_dns (called from SYS_NET_UDP_DNS / FAT /udpdemo), sample CNTPCT from before the quiet ARP post through a matching UDP DNS reply. That is the EL0 net_udp_dns round-trip the sample already runs — not “DNS latency” as a product KPI. Chosen surface: EL0 net_udp_dns (not a separate kernel-only probe).
  2. Marker (exact form). On success with a non-zero delta: perf: udp-dns ticks=<n> (decimal ticks, no units, no percent). If the probe succeeds but the counter did not advance: perf: udp-dns missed and treat the probe as failed.
  3. Smoke. scripts/qemu-smoke.sh greps for the prefix perf: udp-dns (presence). Rejects lines that claim faster/slower/percent. Does not assert a numeric tick value. Keep all prior greps (blk/fat/slot/net/ping/udp/samples/ADR-069/ADR-071).
  4. Test. #[test_case] el0_udp_dns_cntpct_advances asserts a successful quiet UDP DNS probe and udp_dns_ticks() > 0.
  5. Honesty. QEMU virt TCG lab only. Not a latency budget, not SPEC, not “has networking,” not sockets, not a DNS product. Record Verified only with serial evidence on a named tip. CloudAgent HELD; tag agent-box / amd64 workstation. No self-merge (ADR-002).

Consequences

  • Code: src/virtio.rs (el0_udp_dns CNTPCT + perf: udp-dns + udp_dns_ticks); smoke greps; #[test_case].
  • Docs: this ADR, performance.md, measure.md, honesty ledger, roadmap P-PERF-9, SUMMARY, threat-model light bump (v1.47).
  • Follow-ups (not this PR): FAT mkdir (sponsor mile 2), TCP — new ADR + sponsor only.

ADR-073 — FAT16 mkdir (directory create depth)

  • Status: Accepted (FAT16 root mkdir mile Verified when the serial / tests pass; Track A / app hosting still Planned for Linux/POSIX claims; Track N untouched)
  • Date: 2026-09-16

Context

ADR-028 landed virtio-blk + FAT16 read. ADR-050 deepened write / small create. ADR-056 deepened root listing. ADR-057 deepened root file delete. ADR-064 deepened multi-cluster file grow. Docs “new vs extend” prefer deepening the known small FS (FAT16) behind the thin VFS (ADR-027).

Sponsor train mile (2) is honest FAT16 directory create (mkdir): allocate a root dirent with ATTR_DIR, allocate an empty directory cluster, write . / .., update the parent (root). Fail-closed serial + #[test_case].

Not POSIX mkdir / mkdirat. Not a nested path tree (thin-VFS path grammar stays flat /name). Not exFAT / FAT32. Not recursive rm. Do not implement thin TCP (sponsor mile 3) in this PR.

Hypotheses

IDOptionNotes
H1 (chosen)Kernel + thin VFS vfs::mkdir creates a FAT16 root directory. Serial fat: mkdir / keep fat: ok. Probe creates /fdir, confirms dirent + ./.. cluster, refuses second create (Exists). No new SVC this mile.Deepens ADR-028/050/056/057/064. Same flat path grammar.
H2 (rejected)Claim POSIX mkdir / mkdirat or a Linux mkdir ABI.Forbidden honesty.
H3 (rejected)New SVC / libctos wrapper this PR.Delete/readdir/grow miles used existing VFS; mkdir is a kernel/VFS depth probe; EL0 SVC can follow later.
H4 (rejected)Nested subdirectory trees, LFN, recursive rm, or exFAT.Out of scope; follow-ups.
H5 (rejected)Thin TCP / Track N mile 3 in this PR.Separate sponsor mile; out of scope.

Decision

  1. fat::mkdir / vfs::mkdir. Allocate a free cluster (EOC), zero it, write . (self cluster) and .. (parent = FAT16 root → cluster 0), allocate a free root dirent with ATTR_DIR + size 0. Name collision with a file or directory → FsError::Exists (fail-closed). Missing mount / bad path → fail-closed.
  2. Probe after grow. Create /fdir (leftover empty dir: rmdir_empty then recreate). Confirm has_dir, refuse open as a file (Missing), confirm ./.. bytes, second mkdir → Exists. Serial fat: mkdir. Keep fat: ok / write / readdir / delete / grow / A9 / net markers.
  3. No new SVC numbers. Public ABI stays as today (FS 19–23, net 24–26). Not a POSIX product claim. EL0/libctos mkdir is an explicit follow-up.
  4. Fail-closed smoke. Grep fat: mkdir / keep fat: ok / grow / delete / net / samples. #[test_case] covers mkdir + exists/bad-path + create-on-dir-name refuse.
  5. Honesty. Say “the guest created a FAT16 root directory through the thin VFS” only when serial / tests pass. Do not say: POSIX mkdir, directory trees, exFAT/FAT32, “supports FAT,” EL0 isolated, PAN, taken SError, Linux, containers, TCP.
  6. NFR-10 text revised in place (ID unchanged). Threat-model v1.48. Do not mint NFR-15+.

Consequences

  • Code: src/fat.rs (mkdir / dot entries / probe), src/vfs.rs (vfs::mkdir), scripts/qemu-smoke.sh greps.
  • Docs: this ADR, filesystem overview/stance, honesty ledger, roadmap note, threat-model v1.48, SUMMARY, light fr-nfr NFR-10 note.
  • Follow-ups: EL0 fs_mkdir SVC + sample → ADR-075. Nested dirs + empty rmdir → ADR-077. Recursive rm (non-goal unless sponsored) remains open. Thin TCP closed in ADR-074.

ADR-074 — Track N / N5 — thin TCP unlock (active open + one payload)

  • Status: Accepted (N5 runtime — Verified only when this tip’s qemu-smoke prints net: tcp-ok; else Documented/partial)
  • Date: 2026-09-16

Context

ADR-063 opened Track N. N1–N4 + N3.x are Verified (ADR-066 / ADR-067 / ADR-070 / ADR-071 / ADR-068). FAT mkdir depth is ADR-073. Sponsor train mile (3) unlocks a thin / honest TCP probe — not a BSD sockets product.

Prior miles left TCP as a locked non-goal. This ADR opens a single-connection kernel-path mile only.

Hypotheses

IDOptionNotes
H1 (chosen)After N1 ARP + N2 ICMP + N3 UDP on the same virtio-net path: active-open TCP to a QEMU guestfwd echo at 10.0.2.4:7 (cmd: → scripts/tcp-echo-stdio.sh), send one payload ctos-tcp\n, require matching echo. Markers net: tcp-syn / net: tcp-est / net: tcp-tx / net: tcp-rx / net: tcp-ok.CI-deterministic (no external net). More honest than hoping SLIRP HTTP exists. Documents exact -netdev change.
H2 (rejected)Rely on SLIRP built-in HTTP / external TCP peer.Not fail-closed on GHA/agent-box; external net forbidden for smoke.
H3 (rejected)Ship a BSD sockets ABI, listen/accept product, TLS, HTTP stack, or “has TCP sockets OS” marketing.Honesty fail; locked product non-goals.
H4 (rejected)Docs-only unlock with no guest probe.Sponsor asked for a real thin TCP mile.
H5 (rejected)Require EL0 TCP SVC this mile.Optional follow-up ADR; kernel-path proof matches N2/N3 style.

Decision

  1. Unlock N5 under this ADR. Ladder status moves from Locked → Verified when this tip’s qemu-smoke prints net: tcp-ok (with N1–N3 markers preserved).

  2. In scope this mile.

    • Minimal TCP active open (SYN → SYN-ACK → ACK) + one payload send/recv on existing virtio-net (reuse ARP/IP helpers).
    • One connection; no socket table; no listen/accept.
    • Honest CI probe via QEMU user-netdev guestfwd:
      • -netdev user,id=net0,guestfwd=tcp:10.0.2.4:7-cmd:<repo>/scripts/tcp-echo-stdio.sh
      • -device virtio-net-device,netdev=net0 (unchanged device line)
    • Host peer script echoes exactly nine bytes ctos-tcp\n then exits (TCP FIN). No commas in the -netdev path.
    • Serial markers + fail-closed smoke greps.
    • EL0 TCP SVC: not this PR (follow-up ADR only).
  3. Still not product (locked).

    • Full BSD sockets API; listen / accept product; multi-connection socket table.
    • TLS; HTTP stack; DHCP / DNS as product features.
    • Wi‑Fi; Linux net stack; marketing “has TCP / sockets OS.”
  4. Packet / state (tiny).

    • Ethernet dst = gateway MAC from N1 ARP (SLIRP L2 for 10.0.2.4).
    • IPv4: src 10.0.2.15, dst 10.0.2.4, proto TCP, TTL 64.
    • TCP: src ephemeral 0x6334, dst 7, fixed ISN 0x10000000, no options, window 8192, real TCP checksum (pseudo-header).
    • Payload ctos-tcp\n (9 bytes) with PSH+ACK after established.
    • Accept any matching peer segment that carries the echoed payload.
  5. Serial markers (N5).

    • net: tcp-syn — SYN TX used-ring advanced
    • net: tcp-est — SYN-ACK seen + ACK TX’d
    • net: tcp-tx — payload PSH+ACK TX’d
    • net: tcp-rx — matching echo payload RX’d
    • net: tcp-ok — N5 closed
    • Keep N1 (net: ok) / N2 (net: ping-ok) / N3 (net: udp-ok)
    • Miss → net: probe missed (fail-closed)
  6. Smoke. scripts/qemu-smoke.sh greps N1–N3 and the N5 markers. Must keep blk:* / fat:* / slot / sample / perf greps green. Runners that attach netdev (qemu-aarch64.sh, qemu-serial-inject.py) must include the guestfwd clause. Default CTOS_QEMU_TIMEOUT raised 12 → 20 so the longer N1–N5 + sample path still finishes under the inject capture window.

  7. Follow-ups. EL0 TCP SVC / richer socket ABI / listen-accept only with another ADR + sponsor.

  8. Honesty. Say “the guest completed one TCP active open and echoed one payload via QEMU guestfwd” only when serial/qemu-smoke print net: tcp-ok on a named tip. Do not say the OS “has networking,” “has sockets,” or “has TCP.” CloudAgent HELD; tag agent-box / amd64 workstation. No self-merge (ADR-002).

Consequences

  • Code: extend src/virtio.rs (thin TCP TX/RX + observe_net_probe after UDP); scripts/tcp-echo-stdio.sh + .py; QEMU netdev guestfwd; smoke greps; #[test_case] virtio_net_tcp_echo_probe.
  • Docs: this ADR; track-n.md; SUMMARY; roadmap; honesty ledger; limits.md; threat-model bump (v1.49); index / overview one-liners.
  • Follow-ups: EL0 TCP SVC — new ADR only.

ADR-075 — EL0 fs_mkdir SVC + freestanding mkdir-libctos sample

  • Status: Accepted (EL0 mkdir mile Verified when the serial / tests pass; POSIX mkdir / nested trees / Track N EL0 TCP still non-goals)
  • Date: 2026-09-17

Context

ADR-073 landed kernel fat::mkdir / vfs::mkdir with serial fat: mkdir and deferred the public EL0 SVC + sample. Public ABI today: FS SVCs 19–23, net 24–26 (syscall.md). Catalog samples exist for hello / VFS / FAT / yield / net / udp.

Sponsor follow-up unlocks EL0 directory create through the thin VFS — same fail-closed Exists / BadPath honesty as ADR-073. Do not implement EL0 TCP or nested directory trees in this PR.

FAT 8.3 path grammar caps the sample slot at eight characters. /mkdirdemo (9) is rejected by path_to_83; the slot is /mkdemo.

Hypotheses

IDOptionNotes
H1 (chosen)Add SYS_FS_MKDIR (27). Path ABI matches create/open (x0 ptr, x1 len). Map Ok→0, Exists→1, BadPath→2, other→u64::MAX. New crate user/mkdir-libctos on FAT /mkdemo. Print libctos: mkdir-hi / libctos: mkdir-ok. Kernel mkdemo: ok.Minimal ABI; proves EL0 mkdir + Exists/BadPath without nested trees.
H2 (rejected)Claim POSIX mkdir / mkdirat or Linux mkdir ABI.Forbidden honesty.
H3 (rejected)Extend fat-libctos instead of a seventh catalog slot.Prior miles used a dedicated sample per capability; keep that style.
H4 (rejected)Nested subdirectory trees / richer path grammar / recursive create.Out of scope; no nested claim unless proven (not this mile).
H5 (rejected)EL0 TCP SVC in this PR.Separate follow-up; out of scope.

Decision

  1. SYS_FS_MKDIR = 27. Document in syscall.md. Args: x0 = user path pointer, x1 = length. Kernel copies the path and calls vfs::mkdir. Return:
    • 0 (FS_MKDIR_OK) on success
    • 1 (FS_MKDIR_EXISTS) on FsError::Exists
    • 2 (FS_MKDIR_BAD_PATH) on FsError::BadPath (including failed user-path copy / invalid UTF-8 path)
    • u64::MAX (FS_ERR) for other failures (Missing mount, Full, …) Unknown SVCs still park. Not POSIX. Not a process ABI.
  2. libctos. ctos_fs_mkdir / fs_mkdir + SYS_FS_MKDIR / FS_MKDIR_* constants. CRT must not issue reserved 0–2.
  3. Freestanding sample. user/mkdir-libctos on FAT /mkdemo (8.3 MKDEMO). Creates /edir (separate from kernel /fdir), requires second create → Exists, nested /bad/nest → BadPath. Markers: libctos: mkdir-hi / libctos: mkdir-ok / mkdemo: ok. Keep prior six samples + fat: mkdir + all net greps.
  4. Fail-closed smoke. Grep the new markers; keep fat: mkdir / fat: ok / grow / delete / net / samples. #[test_case] on the slot module.
  5. Honesty. Say “EL0 created a FAT16 root directory through fs_mkdir” only when serial / tests pass. Do not say: POSIX mkdir, directory trees, exFAT/FAT32, “supports FAT,” EL0 isolated, PAN, taken SError, Linux, containers, EL0 TCP.
  6. NFR-10 text revised in place (ID unchanged). Threat-model v1.50. Do not mint NFR-15+.

Consequences

  • Code: src/syscall.rs, libctos/, user/mkdir-libctos/, src/mkdemo.rs, build.rs, scripts/mkfat16.py, scripts/qemu-smoke.sh.
  • Docs: this ADR, syscall.md, honesty ledger, filesystem overview/stance, what-can-run, SUMMARY, roadmap note, threat-model v1.50, light fr-nfr NFR-10 note, user/README.md.
  • Follow-ups: nested dirs + empty rmdir → ADR-077. EL0 TCP SVC → ADR-076.

ADR-076 — EL0 thin TCP SVC + freestanding tcp-libctos sample

  • Status: Accepted (EL0 TCP mile Verified when the serial / tests pass; BSD sockets / listen-accept / TLS / multi-connection table still non-goals)
  • Date: 2026-09-17

Context

ADR-074 closed N5 kernel-path thin TCP (net: tcp-ok via QEMU guestfwd 10.0.2.4:7) and deferred the EL0 TCP SVC + sample. Public ABI today: FS SVCs 19–23 + 27, net 24–26 (syscall.md). Catalog samples exist for hello / VFS / FAT / yield / net / udp / mkdir.

Sponsor follow-up unlocks EL0 through the same quiet echo probe — one SVC, return 0 / fail. Do not implement BSD sockets, listen/accept, TLS, multi-connection tables, nested FAT/rmdir, or TCP CNTPCT in this PR.

FAT 8.3 path grammar: /tcpdemo fits (7 chars under the 8.3 name).

Hypotheses

IDOptionNotes
H1 (chosen)Add SYS_NET_TCP_ECHO (28). No args. Kernel runs quiet ARP + thin TCP active-open + one payload vs guestfwd 10.0.2.4:7 (reuse ADR-074 helpers; distinct ephemeral port from N5). Return 0 on success, u64::MAX on fail. New crate user/tcp-libctos on FAT /tcpdemo. Print libctos: tcp-hi / libctos: tcp-ok. Kernel tcpdemo: ok. Keep kernel net: tcp-ok.Tiny ABI; proves EL0 can trigger the existing thin TCP path.
H2 (rejected)Ship BSD sockets / connect/send/recv / listen-accept this mile.Honesty fail; locked product non-goals.
H3 (rejected)Teach EL0 to program virtio-net MMIO or copy raw frames.Kernel owns the NIC; SVCs only.
H4 (rejected)Replace /netdemo / /udpdemo or another slot.Breaks prior catalog greps.
H5 (rejected)TCP CNTPCT / perf: tcp-* this PR.Separate follow-up; out of scope.

Decision

  1. SYS_NET_TCP_ECHO = 28. Document in syscall.md. No args. Kernel virtio::el0_tcp_echo does quiet ARP + SYN/est/tx/rx vs guestfwd (no serial net: tcp-* markers on this quiet path). Return 0 on success, u64::MAX (NET_ERR) on fail. Unknown SVCs still park. Not sockets. Not a process ABI.
  2. libctos. ctos_net_tcp_echo / net_tcp_echo + SYS_NET_TCP_ECHO. CRT must not issue reserved 0–2.
  3. Freestanding sample. user/tcp-libctos on FAT /tcpdemo (8.3 TCPDEMO). Markers: libctos: tcp-hi / libctos: tcp-ok / tcpdemo: ok. Keep prior seven samples + kernel net: tcp-ok + all other smoke.
  4. Fail-closed smoke. Grep the new markers; keep N1–N5 / N3.x / N4 / FAT / mkdir / samples. #[test_case] on the slot module + el0_tcp_echo_probe. Default CTOS_FORCE_FAIL_TIMEOUT is 180s (was 120): ubuntu-24.04-arm force-fail rebuild+boot with eight FAT samples exceeded 120s; force-fail must still exit non-zero.
  5. Honesty. Say “EL0 triggered the thin TCP guestfwd echo through net_tcp_echo” only when serial / tests pass. Do not say: BSD sockets, listen/accept, TLS, HTTP, “has networking / sockets OS,” EL0 virtio driver, Linux/POSIX, EL0 isolated, PAN, taken SError.
  6. NFR-10 text revised in place (ID unchanged). Threat-model v1.51. Do not mint NFR-15+.

Consequences

  • Code: src/syscall.rs, src/virtio.rs (el0_tcp_echo), libctos/, user/tcp-libctos/, src/tcpdemo.rs, build.rs, scripts/mkfat16.py, scripts/qemu-smoke.sh.
  • Docs: this ADR, syscall.md, honesty ledger, track-n, SUMMARY, what-can-run, apps-today, roadmap note, threat-model v1.51, user/README.md.
  • Follow-ups (not this PR): TCP CNTPCT, richer socket ABI, listen-accept — new ADR + sponsor only.

ADR-077 — FAT16 nested mkdir + empty rmdir

  • Status: Accepted (nested mkdir + empty rmdir mile Verified when the serial / tests pass; Track A / app hosting still Planned for Linux/POSIX claims; Track N / TCP CNTPCT untouched)
  • Date: 2026-09-17

Context

ADR-073 landed FAT16 root mkdir (fat: mkdir). ADR-075 added EL0 fs_mkdir (27) for root /edir. Nested subdirectory trees and empty rmdir were explicit follow-ups. ADR-057 deletes files only.

Sponsor follow-up: one-level nested directory create under an existing FAT dir, plus empty rmdir fail-closed. Do not implement TCP CNTPCT or a new EL0 SVC in this PR (kernel/VFS proof is enough this mile).

Path grammar (honest)

FormExampleNotes
Flat/fdirSame as ADR-073: / + 1..=8 of [a-z0-9_-]
One nested/fdir/nest/ + parent + / + child; each component 1..=8 of [a-z0-9_-]; whole path ≤ PATH_MAX (16)
Deeper/a/b/cBadPath (fail-closed)
File ops/probecreate/open/unlink/readdir stay flat (vfs::valid_path); nested file paths → BadPath

Missing parent on a nested mkdir → Missing. Parent name occupied by a file → BadPath. Name collision → Exists.

Hypotheses

IDOptionNotes
H1 (chosen)Extend fat::mkdir / vfs::mkdir for one nested level; expose fat::rmdir / vfs::rmdir for empty dirs. Serial fat: nested / fat: rmdir. Keep fat: mkdir. No new SVC.Deepens ADR-073; documents grammar.
H2 (rejected)Claim POSIX mkdir / rmdir / mkdirat or a Linux directory ABI.Forbidden honesty.
H3 (rejected)New SYS_FS_RMDIR / EL0 sample this PR.Optional; kernel/VFS proof is enough; EL0 can follow.
H4 (rejected)Arbitrary-depth trees, LFN, recursive rm, multi-cluster directories, exFAT.Out of scope.
H5 (rejected)TCP CNTPCT in this PR.Separate follow-up; out of scope.

Decision

  1. fat::mkdir. Flat root unchanged. Nested /parent/child: look up parent dirent, allocate child cluster with . / .. (.. = parent cluster id), install ATTR_DIR dirent in the parent cluster (spc=1). Fail-closed Exists / Missing / BadPath as above.
  2. fat::rmdir / vfs::rmdir. Empty only (.`` / ..+ rest zero). Mark dirent0xE5, free cluster chain. Non-empty → BadHandle. Missing → Missing. File path → BadPath`. Not recursive rm.
  3. Probe after root mkdir. Create /fdir/nest, confirm dirent + dots, Exists on second create, BadPath on /fdir/nest/x, Missing on /nope/x. Empty-rmdir nest; refuse rmdir /fdir while nest exists; refuse rmdir /probe. Serial fat: nested then fat: rmdir. Keep fat: mkdir / grow / delete / net / samples.
  4. No new SVC numbers. Public ABI stays as today (FS 19–23 + 27, net 24–26 + 28). EL0 mkdir-libctos BadPath bait becomes /a/b/c (three-level) so Exists/BadPath proof still holds under the new grammar.
  5. Fail-closed smoke. Grep fat: nested / fat: rmdir; keep fat: mkdir / fat: ok / net / samples. #[test_case] covers nested+rmdir + path grammar.
  6. Honesty. Say “the guest created a one-level nested FAT16 directory and removed an empty one through the thin VFS” only when serial / tests pass. Do not say: POSIX mkdir/rmdir, arbitrary directory trees, recursive rm, exFAT/FAT32, “supports FAT,” EL0 isolated, PAN, taken SError, Linux, containers, TCP CNTPCT.
  7. NFR-10 text revised in place (ID unchanged). Threat-model v1.52. Do not mint NFR-15+.

Consequences

  • Code: src/fat.rs (parse_dir_path / nested mkdir / rmdir / probe), src/vfs.rs (vfs::rmdir + mkdir routing), scripts/qemu-smoke.sh greps, light user/mkdir-libctos BadPath bait update.
  • Docs: this ADR, filesystem overview/stance, honesty ledger, roadmap note, threat-model v1.52, SUMMARY, light fr-nfr NFR-10 note.
  • Follow-ups (not this PR): EL0 fs_rmdir SVC + sample, deeper path trees, multi-cluster directories, recursive rm (non-goal unless sponsored), TCP CNTPCT.

ADR-078 — Thin TCP echo path CNTPCT (measure-first)

  • Status: Accepted (lab CNTPCT on QEMU TCG — not a published bench).
  • Date: 2026-09-17

Context

ADR-069 timed EL0 net_ping. ADR-072 timed EL0 net_udp_dns. ADR-074 closed N5 kernel thin TCP (net: tcp-ok). ADR-076 closed N5.x with EL0 SYS_NET_TCP_ECHO (28) and freestanding /tcpdemo, and deferred TCP CNTPCT.

Perf slice next asks for another measure-first probe around a known path. The natural cut is the EL0 net_tcp_echo quiet path (kernel-owned ARP + thin TCP active-open + one payload vs QEMU guestfwd 10.0.2.4:7) that ADR-076 already exercises — same pattern as ADR-069 / ADR-072.

Do not invent a latency SLA, percent, or “faster/slower” claim. Do not add a criterion crate. Do not implement BSD sockets, listen/accept, TLS, or widen the EL0 net ABI. Keep N1–N5 / N5.x / N3.x / N4 markers, ADR-069/072 prefixes, FAT nested/rmdir, and all prior smoke greps. Force-fail timeout stays 180s.

Hypotheses

IDOptionNotes
H1 (chosen)Sample CNTPCT around the full quiet el0_tcp_echo body (ARP + TCP SYN/est/tx/rx as SYS_NET_TCP_ECHO / /tcpdemo does it). Print perf: tcp-echo ticks=<n>. Fail closed if the counter does not advance after a successful probe. Smoke greps the prefix perf: tcp-echo (not an exact tick count).Honest single-path measurement. Values vary under TCG. Matches ADR-069/072. Chosen surface: EL0 net_tcp_echo (not a separate kernel-only N5 timer).
H2 (rejected)Publish a percent / “TCP is X µs” / criterion bench / SLA.NFR-07 forbids invented benches.
H3 (rejected)Only time the kernel N5 net: tcp-* path and rename it tcp-echo.Kernel N5 markers stay as ADR-074 transport proof; ADR-078 names the EL0 SVC round-trip (same choice as ADR-069/072 H3).
H4 (rejected)Require an exact tick count in smoke.Flaky under QEMU TCG jitter.
H5 (rejected)Add listen/accept / sockets / TLS or widen the EL0 net ABI this mile.Out of scope; honesty fail.

Decision

  1. Window. In virtio::el0_tcp_echo (called from SYS_NET_TCP_ECHO / FAT /tcpdemo), sample CNTPCT from before the quiet ARP post through a matching TCP echo payload. That is the EL0 net_tcp_echo round-trip the sample already runs — not “TCP latency” as a product KPI. Chosen surface: EL0 net_tcp_echo (document explicitly; kernel net: tcp-ok path unchanged / untimed this mile).
  2. Marker (exact form). On success with a non-zero delta: perf: tcp-echo ticks=<n> (decimal ticks, no units, no percent). If the probe succeeds but the counter did not advance: perf: tcp-echo missed and treat the probe as failed.
  3. Smoke. scripts/qemu-smoke.sh greps for the prefix perf: tcp-echo (presence). Rejects lines that claim faster/slower/percent. Does not assert a numeric tick value. Keep all prior greps (blk/fat/slot/net/ping/udp/tcp/samples/ADR-069/072/076/077). Default CTOS_FORCE_FAIL_TIMEOUT remains 180s.
  4. Test. #[test_case] el0_tcp_echo_cntpct_advances asserts a successful quiet TCP echo and tcp_echo_ticks() > 0.
  5. Honesty. QEMU virt TCG lab only. Not a latency budget, not SPEC, not “has networking,” not sockets. Record Verified only with serial evidence on a named tip. CloudAgent HELD; tag agent-box / amd64 workstation. No self-merge (ADR-002).

Consequences

  • Code: src/virtio.rs (el0_tcp_echo CNTPCT + perf: tcp-echo + tcp_echo_ticks); smoke greps; #[test_case].
  • Docs: this ADR, performance.md, measure.md, honesty ledger, roadmap P-PERF-10, SUMMARY, threat-model light bump (v1.53), NFR-10 note.
  • Follow-ups (not this PR): richer socket ABI, listen-accept, TLS — new ADR + sponsor only.

ADR-079 — PAN CPU reopen foundation (sponsor unlock; default -cpu cortex-a76)

  • Status: Accepted (CPU reopen foundation). Meets ADR-054 reopen gate items 1–2: sponsor-approved FEAT_PAN default probe CPU + serial pan: present. Enable+fault is ADR-080 (Accepted). Taken SError reopen is ADR-081 (follow-up). Does not claim “EL0 isolated.”
  • Date: 2026-09-19
  • Tracks #125. Sponsor unlock Sat 19 Sep 2026 via DSO. CloudAgent HELD — agent-box / amd64 workstation / GHA only.

Context

ADR-054 locked PAN enable as non-goal on default -cpu cortex-a57 (pan: absent, ADR-026). Reopen required: (1) sponsor-approved FEAT_PAN CPU + new ADR; (2) ID_AA64MMFR1_EL1.PAN != 0 / pan: present; (3) later enable+fault ADR; (4) ledger/smoke update; keep a57 pan: absent historical.

Sponsor / DSO unlocked mile (1): research honest QEMU virt CPUs, pick one CI-viable default, switch smoke scripts with this ADR cite, print pan: present. Do not implement enable or taken SError Verified in this PR. ADR-053 stays hard-stopped until ADR-081. Umbrella “EL0 isolated” stays non-claim (ADR-055 / ADR-060). Never yank _start. No silent -cpu without this ADR.

Research (agent-box, 2026-09-19 CEST)

Host: agent-box. qemu-system-aarch64 10.0.13 (Debian 1:10.0.13+ds-0+deb13u1). Machine: -machine virt. Short freestanding guest probe (MRS ID_AA64MMFR1_EL1, PL011 serial) — not a silent host invent.

-cpuID_AA64MMFR1_EL1.PAN [23:20]Serial
cortex-a570pan: id=0 / pan: absent (historical)
cortex-a720pan: id=0 / pan: absent
cortex-a552pan: id=2 / pan: present
cortex-a762pan: id=2 / pan: present (chosen default)
cortex-a7102pan: id=2 / pan: present
neoverse-n12pan: id=2 / pan: present (UDP not CI-viable here)
neoverse-n22pan: id=2 / pan: present
neoverse-v12pan: id=2 / pan: present
max3pan: id=3 / pan: present (moving target — rejected as default)

CI / smoke viability (agent-box, same tip ELF, inject runner)

-cpupan:N1–N5 netFAT / samples
cortex-a57absentokok (historical)
neoverse-n1presentUDP miss (net: udp-tx then net: probe missed)ok after miss
cortex-a76presentok (udp+tcp)ok
cortex-a55presentokok
maxpresent (id=3)okok (rejected as default — moving target)

Chosen default: cortex-a76. FEAT_PAN id=2, full existing smoke (net/FAT/samples) green on agent-box QEMU 10.0.13. neoverse-n1 is FEAT_PAN-present but not CI-viable for this tree’s UDP path without a separate timing/net ADR.

inject-nmi note (SError mile later)

QMP inject-nmi on the same host still returns machine does not provide NMIs for cortex-a57, cortex-a76, neoverse-n1, and max (re-probed 2026-09-19). CPU switch alone does not unlock taken SError — ADR-081 documents the blocker.

Hypotheses

IDOptionNotes
H1 (chosen)Default probe CPU → -cpu cortex-a76; smoke expects pan: present; cite this ADR; keep a57 rows historical; no MSR PAN.FEAT_PAN id=2; full net/FAT/samples smoke green on agent-box; not max; neoverse-n1 UDP-fails on this tree.
H2 (rejected)Stay on cortex-a57 and only document research.Sponsor asked for concrete CPU switch evidence when a FEAT_PAN CPU works honestly.
H3 (rejected)Default to -cpu max.Field value drifts with QEMU; ADR-026 already rejected silent max.
H4 (rejected)Enable PSTATE.PAN / claim Verified enable this PR.Mile 2 = ADR-080.
H5 (rejected)Fake pan: present on a57 or invent Verified SError.Honesty fail.

Decision

  1. Sponsor unlock recorded. Isolation leftovers reopen mile (1) is authorized (#125). ADR-054 gate items 1–2 are met by this ADR + evidence below. Items 3–4 landed in ADR-080.

  2. Default probe CPU. scripts/qemu-aarch64.sh and scripts/qemu-serial-inject.py use -cpu cortex-a76 with this ADR cite. Do not silent-edit without an ADR.

  3. Serial contract (this mile). Guest reads ID_AA64MMFR1_EL1.PAN (ADR-026 probe). On the new default: pan: id=<n> with n != 0 and pan: present. ADR-080 adds pan: enabled / pan: el1-fault. Historical a57 pan: absent rows remain truth for a57.

  4. Probe success semantics. pan::observe_probe returns success for both absent and present (ID field printed). Fail-closed only if the probe did not publish. No MSR PAN in this mile.

  5. Still non-claim. Taken SError until ADR-081. Umbrella “EL0 isolated” (ADR-055 / ADR-060). Never yank _start. No Guest Linux / containers marketing.

  6. Toolchain pin. rust-toolchain.toml pins nightly-2026-09-12 (0fc141305) — same tip as ADR-078 ledger evidence. Floating nightly @ 2026-09-18 failed freestanding cargo test with E0463 (can't find crate for test). Smoke uses cargo (pin) not cargo +nightly.

  7. Follow-ups. ADR-080 (Accepted): enable + EL1-vs-EL0 fault. ADR-081 (Accepted): taken SError reopen re-probe — still blocked; sponsor unlock B1/B2/B3.

Honesty

Say: “default smoke CPU is -cpu cortex-a76 (ADR-079); ID_AA64MMFR1_EL1.PAN != 0 → pan: present; enable still locked pending ADR-080; a57 pan: absent remains historical.” Do not say:

  • PAN is enabled / “privileged access never”
  • “EL0 isolated” / “secure OS”
  • taken lower-EL SError is Verified
  • the probe silently used a different -cpu than the scripts
  • Guest Linux / containers / immutable-OS

Consequences

  • Scripts + smoke + src/pan.rs / #[test_case] updated for cortex-a76 / pan: present.
  • Docs: this ADR, ADR-054 reopen note, ADR-060, honesty ledger, roadmap P-SEC-3k, threat-model light bump, SUMMARY, limits / el0 light touch.
  • Preserve all existing smoke greps (net/FAT/samples). CloudAgent HELD. Do not self-merge (ADR-002).

ADR-080 — PAN enable + EL1-vs-EL0 access fault (ADR-054 reopen gate 3–4)

  • Status: Accepted (enable + fault Verified on default -cpu cortex-a76). Meets ADR-054 reopen gate items 3–4: enable PSTATE.PAN / MSR PAN and prove an EL1 load of an EL0-accessible page faults (fail-closed); honesty ledger + smoke updated. Does not claim “EL0 isolated.” Taken SError reopen is ADR-081 (Accepted — still blocked).
  • Date: 2026-09-19
  • Tracks #125 mile (2). Builds on ADR-079 (gate items 1–2). CloudAgent HELD — agent-box / amd64 workstation / GHA only.

Context

ADR-054 locked PAN enable until a follow-up ADR after FEAT_PAN CPU reopen. ADR-079 met gate items 1–2 (-cpu cortex-a76, pan: present). Gate item 3 required this ADR: enable PSTATE.PAN and prove EL1-vs-EL0 fault. Gate item 4: ledger + smoke for enable; keep a57 pan: absent historical.

ADR-013 already named the enable mile: with PSTATE.PAN set, an EL1 data access to a page that is unprivileged-accessible must take a permission fault. Umbrella “EL0 isolated” stays non-claim (ADR-055 / ADR-060). Never yank _start.

Hypotheses

IDOptionNotes
H1 (chosen)After ID-field pan: present, MSR PAN, #1; map an EL0-RO page; EL1 LDR must permission-fault with SPSR.PAN set; print pan: enabled / pan: el1-fault; leave PAN set; syscall uaccess clears PAN around copies.Fail-closed if fault missing or SPSR.PAN clear. Preserves net/FAT/samples.
H2 (rejected)Enable for the probe only, then clear PAN for the rest of boot.Weaker than “enable”; uaccess wrapping is small and honest.
H3 (rejected)Claim “EL0 isolated” from this mile alone.ADR-055 §B still unmet (taken SError, _start stay, umbrella sponsor accept).
H4 (rejected)Invent Verified without a caught fault / silent -cpu.Honesty fail.

Decision

  1. Enable. src/pan.rs observe_enable (after frame::init) executes MSR PAN, #1 on FEAT_PAN CPUs (ID_AA64MMFR1_EL1.PAN != 0). Serial pan: enabled.
  2. EL1-vs-EL0 fault. Map an EL0-accessible RO page (map_el0_ro); with PAN clear, EL1 preload must succeed; with PAN set, EL1 LDR must take a current-EL permission DABORT. Handler prints pan: el1-fault and requires SPSR.PAN (bit 22) set at the fault (fail-closed otherwise). Note: under QEMU 10.0.13 TCG, MRS PAN may read 0 even while SPSR.PAN and the permission check prove enable — SPSR is the probe of record here.
  3. Leave PAN set. After a successful probe, PSTATE.PAN stays 1. pan::with_user_access clears/restores around syscall copy_user / copy_to_user so intentional EL0-accessible copies keep working.
  4. Serial + smoke. Keep pan: id= / pan: present (ADR-079). Require pan: enabled / pan: el1-fault. Reject pan: enable missed / unexpected pan: absent. Historical a57 pan: absent rows remain historical.
  5. Target feature. aarch64-ctos.json adds +pan so the assembler accepts the pan PSTATE name (still -cpu cortex-a76 in scripts).
  6. Still non-claim. Umbrella “EL0 isolated” unchanged. Taken SError → ADR-081 (blocked). Never yank _start. No Guest Linux / containers marketing.

Honesty

Say: “on -cpu cortex-a76 (ADR-079), PSTATE.PAN is enabled (ADR-080); EL1 load of an EL0-accessible page faults (pan: el1-fault); umbrella EL0 isolation stays non-claim.” Do not say:

  • “EL0 isolated” / “secure OS”
  • taken lower-EL SError is Verified
  • PAN worked on cortex-a57 without a CPU change
  • Guest Linux / containers / immutable-OS

Consequences

  • src/pan.rs, src/exception.rs (arm + SPSR.PAN check), src/syscall.rs (uaccess), src/main.rs, scripts/qemu-smoke.sh, aarch64-ctos.json.
  • Docs: this ADR; ADR-054 / ADR-055 / ADR-060 / ADR-026 / ADR-079 cross-links; honesty ledger; roadmap P-SEC-3k; threat-model v1.55; SUMMARY; el0 / limits / security light touch.
  • Preserve all existing net/FAT/sample smoke greps. CloudAgent HELD. Do not self-merge (ADR-002).

Follow-on note (2026-09-29, ADR-098)

ADR-098 found that PAN, once enabled here, was not held while the kernel handled exceptions from EL0 or after it returned from an EL0 trip: SCTLR_EL1.SPAN was left at 1 (so exception entry from EL0 did not set PSTATE.PAN), and return_from_el0 resumed EL1 with SPSR.PAN clear. The pan: el1-fault probe above runs once at boot, before any EL0 trip, so it could not see this. ADR-098 clears SPAN after a successful enable, sets SPSR.PAN on return_from_el0, and adds fail-closed live checks with a negative build. In that work MRS PAN read 1 whenever a fault proved PAN set, so the “MRS PAN may read 0” note in Decision 2 is not reproduced there. This ADR’s text is kept as the record of its time.

ADR-081 — Taken lower-EL SError reopen (ADR-053 hard-stop re-probe; still blocked)

  • Status: Accepted (docs / evidence). #125 mile (3). Dated re-probe on default -cpu cortex-a76 + documented virt options: no honest inject delivers ARM async SError to standing EL0 on QEMU 10 virt TCG. Taken path stays deferred / non-goal (locked). el0: serror-park stays Verified. Guest A-clear / EXPECT plumbing stays dormant prep. Do not mark taken SError Verified.
  • Date: 2026-09-19
  • Tracks #125. Builds on ADR-053 reopen gate. CloudAgent HELD — agent-box / amd64 workstation / GHA only.

Context

ADR-053 hard-stopped taken lower-EL SError on QEMU 10 virt + -cpu cortex-a57 after inject-nmi / HMP nmi → machine does not provide NMIs. Reopen gate §4: (a) honest host inject on the accepted smoke machine or sponsor-approved machine/CPU/GIC change with its own ADR; (b) serial el0: serror under EXPECT; (c) new ADR with working evidence.

ADR-079 / ADR-080 changed the default probe CPU to cortex-a76 and Verified PAN enable — not an SError unlock. Sponsor / DSO unlocked mile (3): re-research any honest path; if none, document the blocker explicitly rather than invent Verified.

Do not invent FIQ / BRK / FEAT_NMI as SError. Do not silent machine invent. Keep _start at 0x4008_0000. Umbrella “EL0 isolated” stays non-claim (ADR-055 / ADR-060).

Research (agent-box, 2026-09-19 CEST)

Host: agent-box. qemu-system-aarch64 10.0.13 (Debian 1:10.0.13+ds-0+deb13u1). Accel: TCG only (query-kvm → present=false; -accel kvm → invalid accelerator kvm on this box). Full brief: research/daily-briefs/2026-09-19-adr-081-serror-inject-probe.md.

ConfigQMP inject-nmiNotes
virt + cortex-a76 (default smoke)error machine does not provide NMIsHMP nmi same text
virt + cortex-a57 (historical)sameMatches ADR-053 / ADR-045
virt,gic-version=3 + cortex-a76sameGICv3 alone ≠ TYPE_NMI
virt,gic-version=3 + maxsame
virt,virtualization=on + cortex-a76sameEL2 virt ≠ SError inject
virt,gic-version=3,virtualization=on + cortex-a76same
virt,gic-version=3,secure=on + cortex-a76same
-cpu max,nmi=onstartup failProperty 'max-arm-cpu.nmi' not found

query-commands lists inject-nmi, but virt does not implement TYPE_NMI. CXL *-inject-* commands are not an ARM SError path. Live smoke still arms el0: serror-arm, logs the QMP error, and prints el0: serror-park (no el0: serror).

Options (re-evaluated)

IDOption2026-09-19 status
H1QMP/HMP nmi → ARM_CPU_SERRORStill impossible on virt TCG (all rows above). 2020 qemu-devel “Simulate NMI Injection” series did not land in QEMU 10.0.13.
H2Guest-only RAS / poisoned loadStill rejected for Verified (ADR-044).
H3Treat sync/BRK/FIQ / FEAT_NMI as SErrorStill forbidden. FEAT_NMI is a GICv3 interrupt story, not async SError.
H5KVM serror_pending / plugin injectStill out of scope for TCG virt -kernel smoke; this agent-box has no KVM accel.

Decision

  1. Hard-stop stands on the new default CPU. Taken lower-EL SError while standing on QEMU 10 virt TCG + -cpu cortex-a76 (and the documented GICv3 / virtualization=on / secure=on / max variants probed above) remains deferred / non-goal and locked. ADR-053 reopen gate §4 is not met.
  2. Park stays Verified. Smoke must keep requiring el0: serror-park. Do not require el0: serror.
  3. A-clear plumbing stays dormant prep. Keep el0: serror-arm / eret_to_el0_serror / EXPECT / QMP attempt from ADR-045. Do not delete; do not claim taken.
  4. Explicit blocker (what would unlock). Sponsor must approve one of:
    • B1 — Upstream / pinned QEMU: virt implements TYPE_NMI such that inject-nmi raises async SError (ARM_CPU_SERROR) on the accepted smoke CPU/GIC; re-probe dated; serial el0: serror under EXPECT; new ADR citing working evidence; or
    • B2 — Sponsor smoke-machine ADR: a documented, CI-runnable machine/accel/tooling change that honestly delivers async SError (not FIQ/BRK/FEAT_NMI-as-SError; not silent invent); same EXPECT + evidence ADR; or
    • B3 — Permanent non-goal: sponsor records that taken SError will not be pursued further under virt TCG (locks remain; park stays Verified).
  5. Still non-claims: umbrella “EL0 isolated” (ADR-055); never yank _start. PAN enable remains Verified separately (ADR-080).

Honesty

Say: “taken lower-EL SError stays hard-stopped on virt TCG after ADR-081 re-probe on cortex-a76; inject-nmi still returns machine does not provide NMIs; unlock needs sponsor B1/B2/B3.” Do not say:

  • taken lower-EL SError is Verified
  • “EL0 isolated” / “secure OS”
  • GICv3 / virtualization=on / FEAT_NMI made inject work
  • guest-only RAS inject is Verified
  • KVM was used on this CI smoke path

Follow-on (sponsor B1)

Sponsor chose B1 (2026-09-19 ~08:58 CEST). Follow-on research: ADR-082 — no shippable upstream/apt pin this session; virt through QEMU master still lacks TYPE_NMI→async SError. Park stays Verified; taken stays blocked. Not B2/B3.

Follow-on

ADR-083 (2026-09-19): pinned QEMU opt-in Verifies taken el0: serror; stock park path unchanged.

Follow-on (sponsor B2)

Sponsor chose B2 (2026-09-25 23:44 CEST). ADR-084: free GitHub-hosted runners expose no arm64 KVM/HVF/WHPX; stock TCG there still says machine does not provide NMIs. B2 stays Planned / blocked on a sponsor choice (self-hosted arm64 KVM, paid bare metal, or wait). B1 opt-in pin unchanged; B1 ≠ B2.

Consequences

  • Docs: this ADR; ADR-053 / ADR-045 / ADR-055 / ADR-060 / ADR-079 cross-links; honesty ledger; roadmap P-SEC-3r; threat-model v1.56; SUMMARY; el0 / limits / security light touch; research brief under research/daily-briefs/.
  • Smoke / qemu-serial-inject.py: keep QMP attempt + park requirement; log cites ADR-081. No src/ change required (dormant prep already present).
  • Preserve all existing smoke greps. CloudAgent HELD. Do not self-merge (ADR-002).

ADR-082 — B1 follow-on: upstream/pinned QEMU TYPE_NMI → async SError (blocker)

  • Status: Accepted (docs / research). Sponsor chose B1 (2026-09-19 ~08:58 CEST) after ADR-081. Dated upstream + agent-box research: no shippable QEMU pin this session makes inject-nmi deliver real taken SError on virt TCG. Taken path stays deferred / non-goal (locked). el0: serror-park stays Verified. Do not mark el0: serror Verified. Not B2/B3.
  • Date: 2026-09-19
  • Tracks #125. CloudAgent HELD — agent-box / amd64 workstation / GHA only.

Context

ADR-081 re-probed taken lower-EL SError on default -cpu cortex-a76 and recorded unlock options B1/B2/B3. Sponsor selected B1 — Upstream / pinned QEMU so virt implements TYPE_NMI such that QMP inject-nmi / HMP nmi raises async SError (ARM_CPU_SERROR / taken path → serial el0: serror under EXPECT).

Do not invent FIQ / BRK / FEAT_NMI as SError. Do not silent machine invent. Keep _start at 0x4008_0000. Umbrella “EL0 isolated” stays non-claim (ADR-055 / ADR-060).

Research (agent-box, 2026-09-19 CEST)

Host: agent-box (<agent-vm-hostname>). qemu-system-aarch64 10.0.13 (Debian 1:10.0.13+ds-0+deb13u1). Accel: TCG only. Default smoke CPU: -cpu cortex-a76 (ADR-079).

Live inject matrix (paused -S; QMP only)

Brief: research/daily-briefs/2026-09-19-adr-082-b1-qemu-nmi.md. Full log: research/daily-briefs/2026-09-19-adr-082-b1-qemu-nmi-probe-script.txt. Repro: ./scripts/qemu-nmi-probe.py.

ConfigQMP inject-nmi
virt + cortex-a76 (default smoke)error machine does not provide NMIs
virt + cortex-a57 (historical)same
virt,gic-version=3 + cortex-a76same
virt,gic-version=3 + maxsame
virt,virtualization=on + cortex-a76same
virt,gic-version=3,virtualization=on + cortex-a76same
virt,gic-version=3,secure=on + cortex-a76same

Same failure class as ADR-081. No machine/CPU flag on this package exposes NMI.

Upstream QEMU (source inspection, 2026-09-19)

Treehw/arm/virt.c implements TYPE_NMI?Notes
Debian / agent-box 10.0.13Noinject-nmi → machine does not provide NMIs
Upstream tag v10.0.0Norg TYPE_NMI empty in virt.c
Upstream tag v10.1.0Nosame
Upstream master (fetched 2026-09-19)NoFEAT_NMI / GICv3 has-nmi wires ARM_CPU_NMI (architectural interrupt NMI), not async SError, and does not register TYPE_NMI for QMP inject-nmi

Historical series that would have satisfied B1:

  • 2019–2020 qemu-devel “Simulate NMI Injection” / “Support SError injection” (Gavin Shan et al.): proposed TYPE_NMI on virt → raise ARM_CPU_SERROR. Never merged into virt.
  • 2024 FEAT_NMI / FEAT_GICv3_NMI series: different architecture (PPI/SPI NMI interrupts + PSTATE.ALLINT). Not a substitute for async SError Verified.
  • 2026-08 qemu-devel nmi_inject API cleanup: still “machine does not provide NMIs” when no TYPE_NMI object exists.

CPU_INTERRUPT_VSERR / EXCP_VSERR in current target/arm is the virtual SError (EL2→EL1) path — not a host QMP inject into our EL1/EL0 standing TCG guest.

CI pin options evaluated

OptionCI-runnable?Delivers el0: serror?2026-09-19 verdict
Apt pin newer Debian qemuYes (GHA apt-get)No — 10.0.x / published 10.1 packages still lack virt TYPE_NMI→SErrorRejected as unlock
Build upstream master from source + cachePossible (slow; multi-min)No — master also lacks the wiringRejected as unlock
Local QEMU patch (draft under research/qemu-nmi/) + build-from-sourcePossible next sessionWould, if patch verified under EXPECTNot shipped this session — needs sponsor-reviewed patch + reproducible Dockerfile/GHA cache before smoke can require el0: serror
Machine flag / -cpu *,nmi=on aloneN/ANo — Property 'max-arm-cpu.nmi' not found on 10.0.13; FEAT_NMI ≠ SErrorForbidden as Verified

Draft local-patch sketch (research only, not CI-wired): research/qemu-nmi/README.md + research/qemu-nmi/0001-hw-arm-virt-TYPE_NMI-raise-SError.patch.

Decision

  1. B1 chosen; B1 not met this session. Sponsor selected B1. Upstream and packaged QEMU (through master as of 2026-09-19) do not implement virt TYPE_NMI → async SError. No apt/git pin alone unlocks taken SError.
  2. Blocker (explicit). Unlock remains: wait for upstream virt TYPE_NMI→ARM_CPU_SERROR (or equivalent honest QMP), or a sponsor-approved pinned QEMU build (Dockerfile / GHA cache) that carries a reviewed local patch proving inject-nmi → serial el0: serror under EXPECT on agent-box + GHA. Until then, hard-stop from ADR-053/081 stands.
  3. Park stays Verified. Smoke keeps requiring el0: serror-park. Do not require el0: serror.
  4. A-clear plumbing stays dormant prep. Keep ADR-045 EXPECT / el0: serror-arm / QMP attempt.
  5. Not B2/B3. This ADR does not approve a different smoke machine (B2) and does not declare permanent non-goal (B3).
  6. Still non-claims: umbrella “EL0 isolated”; never yank _start; FEAT_NMI / FIQ / BRK are not SError.

Honesty

Say: “sponsor chose B1; ADR-082 research finds no upstream/packaged QEMU pin that makes inject-nmi deliver taken SError on virt TCG; park stays Verified; taken stays blocked pending upstream or a reviewed pinned build.” Do not say:

  • taken lower-EL SError is Verified / el0: serror is required
  • a newer apt qemu “should” unlock without probe
  • FEAT_NMI / GICv3 NMI is the taken-SError path
  • the research patch is CI-wired or proven
  • “EL0 isolated” / “secure OS”

Consequences

  • Docs: this ADR; cross-links on ADR-053/081/045/055/060; honesty ledger; roadmap P-SEC-3r; threat-model v1.57; SUMMARY; el0 / limits / security light touch; research brief + probe script + draft patch under research/qemu-nmi/.
  • Smoke: unchanged fail-closed park requirement. No src/ change.
  • Follow-on: ADR-083 ships the pinned build + opt-in el0: serror Verified (2026-09-19).
  • CloudAgent HELD. Do not self-merge (ADR-002).

ADR-083 — B1 pinned QEMU: inject-nmi → taken el0: serror (Verified opt-in)

  • Status: Accepted (implementation + evidence). Sponsor B1 (ADR-081/082) met via pinned QEMU build (not upstream merge). Dated proof: QMP inject-nmi succeeds and guest serial shows el0: serror under EXPECT. Stock distro QEMU path unchanged (park Verified). Umbrella “EL0 isolated” stays non-claim.
  • Date: 2026-09-19
  • Follow-on (2026-09-26): ADR-090 (sponsor D1/D5) made qemu-nmi-pin unconditional (repo-variable gate removed, job renamed “QEMU NMI pin smoke (taken SError)”). This reverses §6’s “do not force the compile on every PR”. It is part of the umbrella’s taken-SError evidence class together with B2-P.
  • Tracks #125. CloudAgent HELD — agent-box proof; amd64 workstation preferred for heavy build but offline this session (<machine-id> intent). Do not self-merge (ADR-002).

Context

ADR-082 found no upstream/packaged QEMU pin: virt through master lacks TYPE_NMI→async SError. DSO (2026-09-19) preferred a pinned custom QEMU over waiting upstream. Draft sketch lived under research/qemu-nmi/.

Decision

  1. Pin QEMU v10.0.0 (7c949c53e936aa3a658d84ab53bae5cadaa5d59c) + reviewed local patch research/qemu-nmi/0001-hw-arm-virt-TYPE_NMI-raise-SError.patch.
  2. Patch semantics (honest): add TCG EXCP_SERROR / CPU_INTERRUPT_SERROR; virt implements TYPE_NMI so inject-nmi raises that line. Masked by PSTATE.A. Not FEAT_NMI / FIQ / BRK.
  3. Install prefix: tools/qemu-nmi/ via ./scripts/build-qemu-nmi.sh (bin/share gitignored).
  4. Opt-in smoke: CTOS_QEMU=…/qemu-system-aarch64 + CTOS_REQUIRE_TAKEN_SERROR=1 fail-closed requires bare el0: serror. Default (unset) still requires el0: serror-park.
  5. Host inject timing: qemu-serial-inject.py issues QMP stop → inject-nmi → cont on el0: serror-arm so TCG cannot race past the A-clear window.
  6. CI/Docker: default stays distro QEMU. Dockerfile ARG CTOS_BUILD_QEMU_NMI=0 opt-in. GHA job snippet in docs/dev/qemu-nmi-pin-ci.md (OAuth push lacks workflow scope this session — paste when available). Do not force multi-minute QEMU compile on every PR.
  7. Still non-claims: umbrella EL0 isolated; never yank _start; park remains Verified on stock QEMU.

Evidence (2026-09-19 CEST, agent-box)

ProbeResult
CTOS_QEMU=tools/qemu-nmi/bin/… ./scripts/qemu-nmi-probe.pyinject-nmi ok on virt + cortex-a76 / a57 / GICv3 / virt-on / secure / max
Stock qemu-system-aarch64 10.0.13 probestill machine does not provide NMIs
CTOS_QEMU=… CTOS_REQUIRE_TAKEN_SERROR=1 ./scripts/qemu-smoke.shel0: serror present; qemu-smoke: ok
Stock inject (no pin)inject-nmi error + el0: serror-park

Machine tag: (agent-box); (amd64 workstation) intent only (CloudAgent HELD / arm64 workstation offline).

Honesty

Say: “B1 met with a pinned QEMU build; taken el0: serror Verified under opt-in CTOS_QEMU + CTOS_REQUIRE_TAKEN_SERROR=1; stock QEMU still parks.” Do not say taken is Verified on distro QEMU, or that FEAT_NMI is the path, or “EL0 isolated.”

Consequences

  • Docs: this ADR; ADR-082/081/053/045 cross-links; honesty ledger; SUMMARY; threat-model bump; research/qemu-nmi/ + tools/qemu-nmi/PIN.txt.
  • Smoke/scripts: CTOS_QEMU honored; taken gate opt-in; stock park preserved.
  • Dockerfile: CTOS_BUILD_QEMU_NMI ARG. GHA: see docs/dev/qemu-nmi-pin-ci.md (workflow file unchanged this PR).

ADR-084 — B2 spike: taken SError on a different free GitHub-hosted runner (blocked — no arm64 KVM/HVF/WHPX)

  • Status: Accepted (docs / evidence). Sponsor chose B2 (“Go with B2 to unlock 1 and 2”, DSO relay 2026-09-25 23:44 CEST). ADR-081 B2 spike on free GitHub-hosted runners: impossible this session. No free hosted runner exposes a hardware accelerator that can run an AArch64 guest (/dev/kvm absent on arm64 Linux, HVF HV_UNSUPPORTED on macOS arm64, no VT/WHPX on Windows arm64). Stock QEMU TCG on those runners still returns machine does not provide NMIs. The B2 (non-B1) path for P-SEC-3r stays Planned / blocked pending a sponsor decision. No new Verified. B1 opt-in pin (ADR-083) is unchanged and is not B2.
  • Date: 2026-09-25
  • Follow-on (2026-09-26): sponsor chose B2-P. ADR-085 ran ctos on an AWS Graviton3 c7g.metal under KVM and Verified a taken lower-EL SError (opt-in b2-serror profile, one-off paid run, not CI). This ADR’s free-runner finding stands.
  • Tracks P-SEC-3r (taken lower-EL SError). Builds on ADR-081 §4 B2, ADR-053 reopen gate, ADR-083. CloudAgent not used (over cap until 2026-10-04). No self-hosted runner registered. No paid / larger runner used. Do not self-merge (ADR-002).

Context

ADR-081 §4 listed three unlocks for a taken lower-EL SError while standing: B1 pinned/upstream QEMU with TYPE_NMI→async SError, B2 a documented CI-runnable machine / accel / tooling change that honestly delivers async SError, B3 permanent non-goal. B1 landed as an opt-in pinned-QEMU job (qemu-nmi-pin, repo variable CTOS_BUILD_QEMU_NMI=1, ~12–13 min; ADR-082/083, PRs #129–#132). That is still stock-virt TCG with a patched emulator.

The sponsor asked for B2 as a genuinely different path: run on a different CI machine/runner/accelerator (for example real arm64 hardware + KVM) and reach a real taken SError there — not a re-run of the B1 pin. Hard rules for this spike: free runners only (flag before paid/larger runners), no self-hosted runner registration, no CloudAgents, and if B2 is impossible, record it with evidence and stop (no slide into B1/B3 variants).

B2 hypotheses (what a genuine B2 would need)

IDCandidate SError sourceNeedsNotes
K1KVM KVM_SET_VCPU_EVENTS with exception.serror_pending=1 (KVM sets HCR_EL2.VSE → guest takes a virtual SError, masked by PSTATE.A)arm64 host with /dev/kvm + a small VMM (or QEMU change) that injects on the el0: serror-arm cueStock QEMU only get/puts env->serror.pending for migration (target/arm/kvm.c, target/arm/machine.c); no QMP/HMP setter. So K1 is a custom VMM, not stock QEMU.
K2QEMU inject-nmi under -accel kvmsame hostinject-nmi is machine-level (hw/core/nmi.c walks for a TYPE_NMI object). No hw/arm/* implements TYPE_NMI on QEMU master (code search 2026-09-25: only hppa / i386 / m68k / ppc / s390x / xtensa / macio). So K2 fails regardless of accelerator.
K3Guest access to an unbacked / external-abort region reported asynchronouslyreal hardware or KVMUnder KVM, unbacked IPA → MMIO exit to the VMM; QEMU can only inject a synchronous external data abort (KVM_CAP_ARM_INJECT_EXT_DABT). Async reporting is implementation-defined and not reproducible on a CI VM. Also a guest-only source (ADR-044 H2 rejected for Verified).
H1’Stock TCG inject-nmi on a different runner (native arm64)any runnerSame QEMU code as the agent box → expected machine does not provide NMIs. Control row.

K1 is the only honest B2 SError source, and it requires an arm64 host that exposes a hypervisor to the job. So the spike’s first question is: does any free GitHub-hosted runner expose one?

Probe (GitHub Actions, 2026-09-25 CEST)

Repo artofdream/ctos is public. Standard GitHub-hosted runners (including ubuntu-24.04-arm, ubuntu-22.04-arm, macos-15, windows-11-arm) are free for public repos under GitHub’s Actions billing policy. No larger runner labels used. Temporary branch probe/adr-084-b2-kvm (workflow b2-serror-probe, research only, not a gate; copy kept at research/daily-briefs/2026-09-25-adr-084-b2-probe-workflow.yml.txt).

  • Run 1 (c198bf1, 23:47 CEST): 36193423604 — facts + KVM/HVF/Hyper-V checks (22.04-arm job failed only because the ADR-082 probe script races QEMU 6.2, which lacks cortex-a76; not a finding).
  • Run 2 (749cd32, 23:49 CEST): 36193613972 — all 5 jobs green; adds an all-machine TYPE_NMI sweep and an unconditional HVF boot attempt.
Runner (image)HostAccelerator result (exact log line)Stock TCG inject-nmi
ubuntu-24.04-arm (20260920.129.1) jobAzure Hyper-V VM, Neoverse-N2 (Cobalt 100), kernel 6.17.0-1022-azuredmesg kvm [1]: HYP mode not available; b2-probe: /dev/kvm=absent; QEMU 8.2.2 -accel kvm: Could not access KVM kernel module: No such file or directory / failed to initialize kvmvirt × a76/a57/GICv3/max/virt-on/secure: all machine does not provide NMIs; sweep machines-probed=118 inject-nmi-ok=0
ubuntu-22.04-arm (20260920.137.1) jobsame class, kernel 6.8.0-1064-azurekvm [1]: HYP mode not available; /dev/kvm=absent; QEMU 6.2 -accel kvm same failuresweep machines-probed=101 inject-nmi-ok=0 (QEMU 6.2 wording this feature or command is not currently supported)
macos-15 arm64 (20260907.0337.1) jobApple M1 (Virtual), Darwin 24.6.0 RELEASE_ARM64_VMAPPLEsysctl: unknown oid 'kern.hv_support'; kern.hv_vmm_present: 1; QEMU 11.1.1 -accel hvf: Error: ret = HV_UNSUPPORTED (0xfae9400f, at ../target/arm/hvf/hvf.c:1269)n/a (HVF has no SError inject API anyway)
windows-11-arm (20260920.164.1) jobCobalt 100 Hyper-V guestVirtualizationFirmwareEnabled : False, VMMonitorModeExtensions : False, HypervisorPlatform (WHPX) state = Disabled (enabling needs a reboot; no nested virt)n/a
ubuntu-24.04 x86_64 (control) jobAMD EPYC (7763 run 1, 9V74 run 2), Hyper-V/dev/kvm=present (kvm_amd) but qemu-system-aarch64 -accel kvm: invalid accelerator kvm (x86 KVM cannot run an AArch64 guest)same machine does not provide NMIs; sweep machines-probed=117 inject-nmi-ok=0

Sweep note: on ubuntu-24.04-arm, 92 machine models answered machine does not provide NMIs and 26 did not start without board-specific -cpu/firmware (for example sbsa-ref, xlnx-zcu102); the QEMU source search above shows none of them implements TYPE_NMI either.

Consistent with public upstream reports: actions/runner-images#14062 (“KVM on ARM runners” — Azure arm64 VMs lack nested virtualization).

Decision

  1. B2 on free GitHub-hosted runners is impossible (2026-09-25). No free hosted runner can run ctos under a hardware accelerator (KVM/HVF/WHPX) on AArch64, so the only honest B2 SError source (K1, KVM virtual SError) cannot be reached. TCG on a different runner is the same emulator code and still has no NMI/SError path (H1’ control).
  2. Stop here. No B2 CI job, no new smoke marker, no src/ change. Do not slide into a B1 variant (patched QEMU on the arm runner) or B3. Do not relabel the B1 pin as B2.
  3. Status stays honest.
    • Stock QEMU TCG: taken SError deferred / non-goal (hard-stopped); smoke keeps requiring el0: serror-park (ADR-053/081).
    • B1 opt-in pin: taken el0: serror Verified (opt-in) only in qemu-nmi-pin (ADR-083). Unchanged.
    • B2 (different runner/accelerator): Planned / blocked — probe ran and lost on free runners (ledger row Failed for the free-runner probe).
  4. What would unlock B2 (sponsor decision required; none taken here):
    • B2-S — self-hosted arm64 runner with KVM on sponsor hardware (bare-metal arm64 Linux booted at EL2, e.g. an Ampere/Graviton-metal box, Raspberry Pi 5, or an Apple-silicon machine running Linux natively). Security change: registering a self-hosted runner on a public repo needs explicit sponsor go (fork PRs must not reach it). Not done.
    • B2-P — paid bare-metal arm64 (cloud *.metal or a third-party runner vendor that exposes arm64 KVM). Costs money; flagged, not used. (WarpBuild documents nested virt on x64 only.)
    • B2-W — wait for GitHub/Azure to offer nested virtualization on arm64 hosted runners (runner-images#14062), then re-run this probe (workflow_dispatch copy in the research brief). Any of those still needs K1 tooling: a small KVM VMM (or reviewed QEMU change) that loads the ctos ELF at 0x4008_0000, provides PL011 + GIC, and calls KVM_SET_VCPU_EVENTS (serror_pending=1, optional ESR via KVM_CAP_ARM_INJECT_SERROR_ESR) on el0: serror-arm; plus a fail-closed smoke that requires bare el0: serror and fails on absence. That would be a new milestone ADR.
  5. Still non-claims: umbrella “EL0 isolated” (P-SEC-3l; ADR-055 / ADR-060) — out of scope here; never yank _start (0x4008_0000).

Honesty

Say: “ADR-084 B2 spike: free GitHub-hosted runners expose no arm64 KVM/HVF/WHPX (runs 36193423604 / 36193613972, 2026-09-25); TCG there still says machine does not provide NMIs; B2 is Planned / blocked on a sponsor choice (self-hosted arm64 KVM, paid bare metal, or wait). Taken SError stays Verified only via the B1 opt-in pin.” Do not say:

  • B2 is Verified, or taken SError is Verified on a hosted arm64 runner
  • KVM was used anywhere on ctos CI
  • the B1 pin job is a B2 result
  • FEAT_NMI / FIQ / BRK / a synchronous external abort is an SError
  • “EL0 isolated” / “secure OS”

Consequences

  • Docs: this ADR; ADR-081 follow-on; ADR-053 / ADR-055 / ADR-060 cross-links; honesty ledger; roadmap P-SEC-3r; threat model v1.59; SUMMARY; research brief + raw log excerpt + probe workflow copy under research/daily-briefs/; session memory under research/random-thoughts/.
  • CI: .github/workflows/ unchanged on main by this PR. The temporary probe branch/workflow lived only on probe/adr-084-b2-kvm. Existing smoke (el0: serror-park required) and opt-in qemu-nmi-pin unchanged.
  • Cost: all probe minutes on free standard runners of a public repo. No paid, larger, or self-hosted runner.

ADR-085 — B2-P: taken lower-EL SError under KVM on real arm64 (AWS Graviton3 c7g.metal) — Verified (opt-in, one-off paid run)

  • Status: Accepted. Sponsor GO for B2-P (paid Graviton bare metal), DSO relay 2026-09-26 11:28 CEST. Follow-on to ADR-084 §4 (B2-P). Result: taken lower-EL SError Verified under KVM on real Graviton3 hardware in a one-off, opt-in b2-serror profile (run 4, 2026-09-26 13:26–13:33 CEST). Not a CI gate: no ctos CI job runs KVM. Stock-QEMU TCG smoke still parks; the B1 opt-in pin (ADR-083) is unchanged and is not B2.
  • Date: 2026-09-26
  • Tracks P-SEC-3r (taken lower-EL SError). Builds on ADR-081 §4 B2, ADR-084 K1, ADR-045 standing-EL0 A-clear window, ADR-053. CloudAgent not used. No self-hosted runner. Do not self-merge (ADR-002). Item 2 (umbrella “EL0 isolated”) not started. It stays gated.
  • Follow-on (2026-09-26): run 4 is the B2-P leg of the umbrella’s taken-SError evidence class (ADR-090, sponsor D1). It is historical (at 159b178, before the M1/M2 boot-path changes). A KVM re-verify is ≈ $0.2 and is a sponsor option, not required.

Context

ADR-084 showed that no free GitHub-hosted runner exposes an arm64 hypervisor. The only honest B2 SError source is K1: a KVM host sets serror_pending through KVM_SET_VCPU_EVENTS, KVM raises a virtual SError (HCR_EL2.VSE), and the guest takes it when PSTATE.A is clear. That path needs real arm64 hardware booted at EL2. The sponsor chose B2-P under these constraints: a paid AWS Graviton .metal box, cheapest type/region, Spot, tags project=ctos / component=b2-spike on every resource, a self-terminate timer, no standing resources, stop if the estimate exceeds ~$20, and a sanctioned credentials path only.

Design (what is different from B1)

B1 (ADR-083)B2-P (this ADR)
Where the guest runsQEMU TCG (emulated CPU)Real Neoverse-V1 (Graviton3), QEMU -accel kvm -cpu host
Who delivers the SErrorQEMU TCG raises an emulated async SError (patch 0001, TYPE_NMI on virt)KVM / hardware: patch 0002 turns inject-nmi under kvm_enabled() into env.serror.pending=1; QEMU’s resume path (kvm_arch_put_registers → kvm_put_vcpu_events) issues KVM_SET_VCPU_EVENTS{serror_pending=1}; KVM sets HCR_EL2.VSE; the CPU takes a virtual SError when EL0 runs with A clear
HostGitHub ubuntu-24.04-arm (free) or agent boxAWS c7g.metal, eu-north-1b, Spot
Gateopt-in CI job qemu-nmi-pinone-off paid run; no CI job

Pieces (branch isolation/adr-085-b2p-graviton-kvm-serror):

  • research/qemu-nmi/0002-hw-arm-virt-TYPE_NMI-KVM-serror-pending.patch, applied on top of B1’s 0001 on QEMU v10.0.0 via scripts/build-qemu-b2.sh (output tools/qemu-b2/, gitignored).
  • Opt-in Cargo feature b2-serror: after the normal paging/identity-teardown boot, it prints b2: boot, runs only the standing-EL0 SError window (el0::observe_b2_serror), then b2: done. It skips GIC/timer/virtio/FAT, because the host KVM exposes GICv3 only (kvm [1]: disabling GICv2 emulation). It also adds raw early markers b2: e0/e1/e2/p0/p1 and a longer outer spin (MOVZ X3,#0xffff) for native speed. The default build is unchanged apart from the TG1 fix below.
  • scripts/b2-kvm-smoke.py: fail-closed. Exit 0 only if a bare el0: serror line and b2: taken appear with no park marker. If the guest is silent for 15 s it dumps vCPU registers (QMP human-monitor-command info registers) as diagnostics. That dump is never a pass signal.
  • scripts/b2-metal-userdata.sh: no-SSH user-data (hard shutdown -h +80; InstanceInitiatedShutdownBehavior=terminate). It clones the branch, builds patched QEMU + the b2-serror guest natively, runs the smoke up to 3×, prints CTOSB2: lines to the serial console (read with ec2 get-console-output --latest), then powers off.

Runs (2026-09-26, eu-north-1, all tagged project=ctos, component=b2-spike)

Credentials path: the sanctioned user-Aws-mcp aws___run_script (IAM user <iam-user>, account <aws-account-id>). Security group ctos-b2-spike had no ingress. No key pair and no IAM role were created.

#InstanceType / marketLaunch (CEST)EndResult
0<instance-id-run0>m6g.metal (Graviton2) Spot12:22:26self-poweroff ≈12:28:50Launched by a batch RunInstances whose MCP response timed out; not noticed until flagged. HOME unset in cloud-init → cargo: command not found. Patched QEMU built with kvm on Graviton2; smoke failed closed (no ELF).
1<instance-id-run1>c7g.metal Spot12:23:50terminated by me 12:29:29Same HOME bug; terminated early (at most one metal box at a time).
2<instance-id-run2>c7g.metal Spot12:42:59self-poweroff ≈12:55:15Guest built. Under KVM: zero guest UART bytes in 3×90 s → FAIL (fail-closed).
3<instance-id-run3>c7g.metal Spot13:06:27self-poweroff ≈13:17:31Early markers reached b2: p1 mmu-on, then stalled. Register dump: PC=ffffff8040080a00 (high-alias VBAR 0xffffff8040080800 + 0x200), PSTATE=a00003c5 EL1h, identity LR → recursive abort on the vector fetch through TTBR1. A control keeping the pre-MMU spin locks behaved identically, so the “pre-MMU exclusives” hypothesis was refuted.
4<instance-id-run4>c7g.metal Spot13:26:02self-poweroff ≈13:33:33PASS — taken lower-EL SError under KVM (below).

Early tries at 12:1x CEST got InsufficientInstanceCapacity for c6g.metal Spot in eu-north-1a/b/c. Spot placement scores (read-only) then pointed to c7g.metal. ap-south-1 was cheaper but its vCPU quota is 5, so unusable.

Root cause found on real hardware (run 3) → fix

TCR_EL1 was programmed without TG1, so TG1 stayed 0b00. That encoding is reserved for TG1 (01=16K, 10=4K, 11=64K; TG0 differs). QEMU TCG sanitises a reserved TG1 to 4K, so the TTBR1 high alias (ADR-016…ADR-019) always worked there. On Graviton3 the reserved value selected a different granule, so every high-VA walk failed. Fix: TCR_TG1_4K = 0b10 << 30 in src/paging.rs (commit 159b178). It applies globally because it is a latent real-hardware bug, and it is identical on TCG: default smoke at 159b178 was green on both runners (run 36238549525).

Run 4 evidence (serial console, verbatim)

CTOSB2: cpu Vendor ID: ARM;BIOS Vendor ID: Amazon EC2;Model name: Neoverse-V1;BIOS Model name: AWS Graviton3 N/A CPU @ 2.6GHz;
CTOSB2: dmesg [    0.026567] CPU: All CPU(s) started at EL2
CTOSB2: dmesg [    0.228100] kvm [1]: VHE mode initialized successfully
CTOSB2: /dev/kvm=present
CTOSB2: head 159b17818fb6d22497de0544c0693844876571cc
CTOSB2: qemu build-qemu-b2: accelerators: Accelerators supported in QEMU binary: kvm tcg
CTOSB2: b2-kvm-smoke: cmd: /root/ctos/tools/qemu-b2/bin/qemu-system-aarch64 -accel kvm -machine virt,gic-version=3 -cpu host -m 128M ... -kernel target-b2/aarch64-ctos/debug/ctos
CTOSB2: b2: p1 mmu-on
CTOSB2: ident: jump
CTOSB2: pan: enabled
CTOSB2: pan: el1-fault
CTOSB2: ident: ram lo=0x4026a000 hi=0x48000000 pages=32150
CTOSB2: b2: boot (ADR-085 KVM SError profile)
CTOSB2: el0: serror-armb2-kvm-smoke: qmp stop=>ok inject-nmi=>ok cont=>ok (ADR-085: KVM_SET_VCPU_EVENTS serror_pending)
CTOSB2: el0: standing
CTOSB2: el0: serror
CTOSB2: el0: restored
CTOSB2: b2: taken
CTOSB2: b2-kvm-smoke: summary armed=True injected=True taken_marker=True b2_taken=True park=False kvm_err=False
CTOSB2: b2-kvm-smoke: PASS taken lower-EL SError observed under accel=kvm ('el0: serror' + 'b2: taken')
CTOSB2: smoke attempt 1 rc=0

(el0: serror-arm and the host’s QMP log line share one console line because guest serial and host log are tee’d to the same console.) Full excerpts: research/daily-briefs/2026-09-26-adr-085-b2p-console.txt.

The bare el0: serror line is printed only by the lower-EL SError vector handler when the standing-EL0 expectation is armed (src/exception.rs). Under KVM the guest runs natively, so the only SError delivery path is KVM’s virtual SError that the host requested via serror_pending. Limitation: no host-side strace of the ioctl was captured. The mechanism is QEMU’s documented kvm_put_vcpu_events path, and the fail-closed marker was observed.

Cost

Spot, per-second billing. Total metal time across runs 0–4 ≈ 43 min: c7g.metal ≈ 36.6 min at $0.2476/h plus m6g.metal ≈ 6.4 min at ≈$0.26/h. Prices come from DescribeSpotPriceHistory in eu-north-1b, 2026-09-26. Metal ≈ $0.18. Add gp3 30 GB and public IPv4 for ≈0.7 h (< $0.01). Total ≈ $0.19, far under the $20 gate.

Decision

  1. B2 (different accelerator: real arm64 + KVM): Verified, opt-in, one-off. It rests on run 4 above, a probe from this session. It is not a standing CI gate. Re-running needs another paid metal run (user-data script in-tree) or a self-hosted arm64 KVM runner (B2-S, still not registered).
  2. Stock QEMU TCG smoke: unchanged. It still requires el0: serror-park (ADR-053/081).
  3. B1 pin: unchanged, Verified (opt-in, TCG). Still not B2.
  4. TG1=4K is a real correctness fix for real hardware. The TTBR1 alias was only “Verified” on TCG before this ADR.
  5. Still non-claims: umbrella “EL0 isolated” (P-SEC-3l; ADR-055/060) is not claimed and not started. _start stays at 0x4008_0000. This is not “ctos runs on real hardware” in general: only the b2-serror profile ran, with no GIC/timer/virtio/FAT.

Honesty

Say: “ADR-085 B2-P: on an AWS Graviton3 c7g.metal (eu-north-1, Spot, 2026-09-26), QEMU -accel kvm with patch 0002 set serror_pending via KVM_SET_VCPU_EVENTS on the el0: serror-arm cue, and the ctos b2-serror profile took the lower-EL SError while standing at EL0 (el0: serror / b2: taken, fail-closed smoke PASS). One-off paid run, not CI. Found and fixed a latent TCR_EL1.TG1 bug that TCG masked.” Do not say:

  • taken SError is Verified on stock QEMU, on free runners, or in CI under KVM
  • the default ctos image boots on real hardware (only the b2 profile ran; GIC/timer/virtio untested under KVM)
  • the B1 pin job is B2 evidence
  • “EL0 isolated” / “secure OS”

Consequences

  • Code: opt-in b2-serror feature (src/main.rs, src/el0.rs, early markers in src/paging.rs); global TCR_TG1_4K; patch 0002; scripts/build-qemu-b2.sh, scripts/b2-kvm-smoke.py, scripts/b2-metal-userdata.sh; .gitignore /tools/qemu-b2/.
  • CI: .github/workflows/ unchanged. Default smoke and opt-in qemu-nmi-pin stay green (run 36238549525).
  • Docs: this ADR; ADR-084 follow-on; honesty ledger; roadmap P-SEC-3r; threat model v1.60; SUMMARY; research brief + console excerpt; session memory.
  • AWS: all resources deleted; tag audit in the research brief.

ADR-086 — Umbrella “EL0 isolated” checklist audit after B2-P (gap list + milestone plan)

  • Status: Accepted (docs / audit). The umbrella “EL0 isolated” (P-SEC-3l) stays Planned / non-claim. This ADR audits every ADR-055 / ADR-060 checklist item against evidence gathered this session (2026-09-26). It publishes a measured TTBR0 identity inventory, a gap list, and a milestone plan (one milestone per branch/PR). It mints no new Verified for the umbrella, taken SError on the accepted smoke machine, or full identity teardown.
  • Follow-on (2026-09-28, M6): B3 met: the sponsor accepted the ADR-091 revision-3 sentence and scope in writing (2026-09-28 16:41 CEST). ADR-091 is Accepted. “EL0 isolated” is Verified only as that sentence, with its limits. Text in this ADR that says the umbrella is Planned / non-claim is as of this ADR’s date.
  • Date: 2026-09-26
  • Trigger: sponsor relay (DSO, 2026-09-26 13:55 CEST): item 2 (umbrella EL0 isolated) is unlocked now that B2 has a result (ADR-085). “Claim it ONLY when every checklist item is Verified by a probe from this session.”
  • Base: stacked on PR #137 (915796f). #136 / #137 are not merged at the time of writing, and main @ 4ea871e has neither.
  • Numbering: next free after ADR-085.

Context

ADR-055 §B names three Unmet rows that block the umbrella: (1) taken lower-EL SError while standing, (2) full identity teardown including yank _start, (3) a written sponsor accept that the umbrella sentence is in scope. ADR-060 is the sponsor one-pager of the same locks. B1 (ADR-083) and B2-P (ADR-085) have since produced a taken el0: serror, but only on opt-in paths. The identity tears ADR-018…038 plus ADR-049 landed, while _start stays (ADR-042 / ADR-047 §3, ident: start-stay).

This ADR answers: which checklist items are Verified today, which are not, and what exactly is left.

Evidence gathered this session

ProbeWhen (CEST)WhatResult
Agent-box default smoke2026-09-26 13:59–14:04scripts/qemu-smoke.sh on 915796f, stock QEMU 10.0.13 (Debian), virt -cpu cortex-a76 TCG, rustc 1.100.0-nightly 0fc141305 (repo pin; corrected in ADR-087 — this line first said 5ceaf6608, the box’s +nightly)qemu-smoke: ok; force-fail exit 1 (fail-closed ok). All §A markers below present. inject-nmi=>error:machine does not provide NMIs → el0: serror-park.
GHA PR run 36239619910 (+ push run 36239617837)2026-09-26 13:41–13:53smoke.yml on 915796f: ubuntu-24.04-arm + ubuntu-24.04 default jobs; opt-in qemu-nmi-pin (B1)All success. Default: park. qemu-nmi-pin job 108397549350: inject-nmi=>ok → bare el0: serror → qemu-smoke: taken 'el0: serror' present (ADR-083 pinned QEMU).
B2-P run 4 (ADR-085)2026-09-26 13:26–13:33AWS c7g.metal KVM, opt-in b2-serror profile, one-off paidel0: serror / b2: taken / b2-kvm-smoke: PASS. Not CI.
Scratch TTBR0 inventory (not committed to src/)2026-09-26 ~14:10915796f + scratch walker patch: walks kernel and user TTBR0 L1→L3, coalesces leaves, prints just before Hello World! (after every tear). Same QEMU/rustc; smoke rc=0Output. Five identity (VA = PA) ranges remain in kernel TTBR0 and two in user TTBR0. See §Inventory.

Inventory: what is still identity-mapped after every tear

Kernel TTBR0 (the hello boot; the other two boots in the smoke show the same classes, with boundaries shifted by text size):

#RangeSizeAttrsWhat it isStatus vs “full identity teardown”
I10x0000_0000–0x4000_00001 GiB L1 blockDevice-nGnRnE, EL1 RW, PXN+UXNMMIO identity: GIC 0x0800_0000, PL011 0x0900_0000, virtio-mmio 0x0a00_0000 (all accessed through identity VAs)Not torn. Needs high MMIO alias in TTBR1 first.
I20x4000_0000–0x4008_0000512 KiBNormal, EL1 RW, XNRAM below the image (QEMU DTB hole). ctos never reads the DTB (frame.rs: “never handed out”)Not torn. Nothing uses it.
I30x4008_0000–0x4008_10001 pageNormal, EL1 RO+XBoot stub page: _start + exception_vectors (identity copy; VBAR is high)Stays by decision (ident: start-stay, ADR-042 / ADR-047 §3).
I4[.ident_tear end, 0x4020_1000)≈ 1.06 MiB (e.g. 0x400f1000–0x40201000)Normal, EL1 RO+XLinker padding between the last torn RO section and .data (pinned at 0x4020_1000). No live code.Not torn. The ADR-019/020/025 tears cover .text after the stub, .rodata, and .ident_tear only.
I5__kernel_end page (0x4025d000)1 pageNormal, EL1 RW, XNThe page between __fatal_stack_top / __kernel_end and the heap (ident: heap lo=0x4025e000)Not torn. Falls between the ADR-037 data tear (hi=__kernel_end) and the ADR-038 heap tear.

User TTBR0 (what EL0 runs on; all EL1-only, el0=false): I3 (boot stub, RO+X) and I4 (padding tail, RO+X). Kernel .text/.rodata/.data/heap/RAM are already absent from user TTBR0.

Everything else (live .text after the stub, .rodata, .ident_tear, .data/.bss/stacks, heap, and the frame pool up to 0x4800_0000) is torn, which matches the Verified tear rows. The inventory is a scratch probe: it is Documented evidence for this audit, not a committed fail-closed ratchet. That ratchet is milestone M1.

Checklist audit

Status words: Verified = backed by a probe from this session (box and/or CI above). CI-gated = scripts/qemu-smoke.sh fails closed without the marker on the default GHA jobs. Note that main has no branch protection (GET /branches/main/protection → 404), so “CI-gated” means the sponsor does not merge red. No check is GitHub-required.

ADR-055 §A — Verified miles (necessary, not sufficient)

#ItemStatusProbe / marker (this session)RecordedCI-gated?
A1EL0 first mile + NX kernel dataVerifiedel0: ok, el0: nx kernelLedger ADR-013 rows; box 14:04 + run 36239619910Yes
A2User TTBR0 omits kernel dataVerifiedel0: no kernel read (ADR-055 lists el0: no data, which is stale wording: the code and smoke use el0: no kernel read)Ledger ADR-013Yes
A3Standing EL0 + standing taskVerifiedel0: standing, el0: restored, el0: task-enter/active/exit/restored, el0: task-okLedger ADR-013/024Yes
A4ASID isolationVerifiedasid: dual / asid: conflict / asid: okLedger ASID rowYes
A5TTBR1 private page + EL1 high fetchVerifiedttbr1: el1 / ttbr1: no el0 / ttbr1: ok, ttbr1: el1 exec / ttbr1: vbarLedger ADR-016/017Yes
A6Identity tears text / rodata / data / heap / RAMVerified (as specific tears)ident: live/range/text, ident: rodata(+-fault/-high), ident: data(+…), ident: heap(+…), ident: ram(+-fault/-high), ident: ok; smoke rejects ident: heap-stay / ident: ram-stayLedger ADR-018…020/025/037/038/049Yes
A7EL0 entry without TLBI VMALLE1Verifiedel0: no-vmalle1 + smoke greps src/exception.rs for no tlbi vmalle1Ledger ADR-039Yes
A8Lower-EL IRQ while standing + default I clearVerifiedel0: irq, el0: irq-defaultLedger ADR-040/041Yes
A9Lower-EL FIQ while standingVerifiedel0: fiqLedger ADR-043Yes
A10SError park honesty (default machine)Verifiedel0: serror-arm → inject-nmi=>error:machine does not provide NMIs → el0: serror-parkLedger ADR-043/045/053Yes
A11PAN ID fieldVerifiedpan: id= / pan: present (cortex-a76)Ledger ADR-079Yes
A12PAN enable + EL1-vs-EL0 faultVerifiedpan: enabled, pan: el1-faultLedger ADR-080Yes
A13_start stay honestyVerified (honesty; not a teardown)ident: start-stay lo=0x40080000 hi=0x40081000Ledger ADR-042Yes

ADR-055 §B / ADR-060 — umbrella blockers

#ItemStatusEvidenceHonest readingCI-gated?
B1Taken lower-EL SError while standingUnmet as worded (Verified only on two opt-in, non-default paths)Default machine: park (A10, this session). B1 pin: GHA qemu-nmi-pin job 108397549350 (this session) inject-nmi=>ok → bare el0: serror. B2-P: ADR-085 run 4 (this session) el0: serror under KVM.ADR-060 requires “honest host inject that delivers async SError on the accepted smoke machine; serial el0: serror under EXPECT”. The accepted smoke machine is still stock distro QEMU virt TCG, which parks. B1 is CI-runnable but uses a locally patched emulator (0001 TYPE_NMI→SError). Its job runs only while repo variable CTOS_BUILD_QEMU_NMI=1, so it is opt-in, not a gate. B2-P is the only real-hardware source (KVM serror_pending), but it was one paid run on the b2 profile (no GIC/timer/virtio/FAT) and is not CI-runnable without paid metal or a self-hosted arm64 KVM runner. Neither meets the row as worded. It can count only if the sponsor redefines the accepted evidence (decision D1).Row wording implies yes (default smoke + EXPECT). B1 is CI but opt-in. B2-P: no.
B2Full identity teardown including yank _startUnmet, and unmeetable as wordedScratch inventory: I1, I2, I4, I5 still identity-mapped in kernel TTBR0; I3 + I4 in user TTBR0.The row demands yanking _start, while ADR-047 §3 decides “never yank _start” and the sponsor repeated “never yank _start at 0x4008_0000”. As written the row cannot be met. Decision D2 must redefine it. Independently of _start, four other identity ranges (I1, I2, I4, I5) are not torn, so even “full teardown except the stub” is Unmet today.Needs a new fail-closed ratchet (M1). None exists today.
B3Written sponsor accept that the umbrella sentence is in scopeUnmet (intent Documented)DSO relays 2026-09-25 23:44 (“Go with B2 to unlock 1 and 2”) and 2026-09-26 13:55 (“item 2 unlocked … claim ONLY when every item Verified”)This is direction, not an ADR-048/052-style accept that names the exact sentence and its scope (machine, CPU, profile). Decision D4.n/a (docs)
B4PAN enable (ADR-060 row)Verified= A12Met (ADR-054 gates 1–4 via ADR-079/080).Yes
B5Yank _start (ADR-060 row)Decided: never (not a Verified target)ident: start-stayConflicts with B2 wording. See D2.Yes (stay marker)
B6Guest Linux / containers / immutable-OS marketingNon-goalADR-029 / ADR-036Out of scope. Not required for the umbrella, and not unlocked by it.n/a

Roadmap P-SEC-3l also paraphrases the umbrella as “standing + PAN + full TTBR1 / identity teardown”: standing (A3) and PAN (A12) are Verified. “Full TTBR1 / identity teardown” = B2 (Unmet).

Bottom line: 13 of 13 §A miles plus PAN enable are Verified this session. All three §B blockers are not Met. The umbrella stays Planned / non-claim.

Gap list (ordered; identity teardown first)

GapWhatSizeNeeds sponsor?
G1No committed, fail-closed identity inventory ratchet. Today “what is left” is a scratch probe.Small: walker + allowlist + smoke line + #[test_case]No
G2I2 low RAM 0x4000_0000–0x4008_0000 (DTB hole) identity RWSmall (unused range)No
G3I4 padding tail [.ident_tear end, 0x4020_1000) identity RO+X in kernel and user TTBR0Small (no live code)No
G4I5 __kernel_end page identity RWSmallNo
G5I1 MMIO 1 GiB identity Device block: GIC / PL011 / virtio must move to a TTBR1 high alias before the tearMedium (UART is used pre-MMU and in fault paths; the b2 profile shares the UART)Scope confirm (D3)
G6I3 boot-stub page (_start + identity vectors) in kernel + user TTBR0Small code if allowed; policy-blockedYes (D2)
G7Taken SError on the accepted machinePolicy / CIYes (D1)
G8Umbrella accept ADR with the exact sentenceDocsYes (D4)
G9Doc drift: ADR-055 §A el0: no data ≠ code el0: no kernel readTrivialNo (noted here; fix in M1)

Milestone plan (one milestone per branch / PR; all TCG, no AWS)

MBranch (proposed)ScopeFail-closed markersGate
M0 (this PR)isolation/adr-086-el0-isolated-auditThis audit + inventory evidence + plan + sponsor decisions. Docs only.n/a—
M1isolation/adr-087-identity-inventory-ratchetCommit the TTBR0 walker as ident: inv + tear G2, G3, G4 (RAM-side leftovers, kernel and user TTBR0) + fix G9ident: inv k=<n> u=<n> + ident: inv-ok. Smoke rejects any identity leaf outside the allowlist {I1 MMIO block, I3 stub page} (ident: inv-leak …). ident: low / ident: tail / ident: kend tears with -fault probes. Keeps ident: start-stay.None
M2isolation/adr-088-mmio-high-aliasMap GIC / PL011 / virtio-mmio in TTBR1 (Device, EL1-only), switch base VAs after the high split, tear I1ident: mmio-high, ident: mmio-fault (identity MMIO VA → DABORT), allowlist shrinks to {I3}D3 (scope). Optional b2-profile KVM re-check is not required. If wanted: ≈ $0.2, report cost first.
M3isolation/adr-089-boot-stub-policyPer D2: (a) docs ADR that defines “full identity teardown” as all identity except the _start stub page (start-stay remains), or (b) code: after the high jump, unmap the identity mapping of the stub page (the image and _start stay physically at 0x4008_0000, and QEMU -kernel only needs that PA at entry), print ident: stub-torn, retire ident: start-stay(b): ident: stub-torn / ident: stub-fault, allowlist becomes {} → ident: inv k=0 u=0D2
M4isolation/adr-090-serror-accepted-machinePer D1: close ADR-055 §B row 1 under the sponsor-chosen evidence class (see D1). A workflow edit (e.g. making qemu-nmi-pin unconditional) is pushed from the amd64 workstation.Taken el0: serror required under the chosen gateD1
M5docs/adr-091-umbrella-acceptADR-048/052-style sponsor accept: exact sentence, machine/CPU/profile scope, the list of Met rowsn/aD4
M6isolation/adr-092-umbrella-flipOnly if M1–M5 are Met: fresh-session re-probe of every row, ledger/roadmap flip, sponsor mergesAll of the above in one runSponsor merge (ADR-002)

ADR numbers in M1–M6 are proposed. Each PR takes the next free number at open time.

  • D1 — Which taken-SError evidence counts for the umbrella? Options: (i) the B1 pinned-QEMU CI job as the accepted smoke machine for this row (make it unconditional; honest caveat: TCG emulator patched locally, not stock); (ii) the B2-P one-off real-hardware run as sufficient (not CI; b2 profile only); (iii) require a CI-gated real-KVM run (self-hosted arm64 KVM runner = security change, or paid metal per gate run ≈ $0.2 each, tagged + self-terminating); (iv) keep taken SError a non-goal → the umbrella is permanently non-claim. Agent recommendation: (i)+(ii) together as the evidence class, stated verbatim in the accept ADR, or (iv). Not a silent redefinition.
  • D2 — _start and “full identity teardown”. ADR-055 §B row 2 (yank _start) contradicts ADR-047 §3 (never yank). Options: (a) the umbrella accepts start-stay as the single documented identity exception; (b) authorize unmapping the stub’s identity mapping after boot while _start stays at 0x4008_0000 (no image move, no -kernel change). ADR-047 §3 would then read as “never move or remove _start”.
  • D3 — Is device MMIO in scope of identity teardown? Recommendation: yes (I1 is the largest identity range left).
  • D4 — Umbrella sentence + accept ADR. Exact wording and scope (e.g. “on QEMU virt -cpu cortex-a76 TCG smoke, plus [D1 evidence]”).
  • D5 — CI gating. main has no branch protection. Should the flip require GitHub-required checks, or is “sponsor does not merge red” enough?

Honesty

Say: “ADR-086 audited the ADR-055/060 checklist on 2026-09-26: every §A mile and PAN enable are Verified this session. Taken SError is Verified only on opt-in paths (B1 pin, B2-P one-off), not on the accepted smoke machine. Identity teardown still leaves five identity ranges (one is _start by decision). No umbrella accept exists. ‘EL0 isolated’ stays Planned / non-claim.”

Do not say:

  • “EL0 isolated” / “secure OS” / “hardened isolation complete”
  • taken lower-EL SError Verified on the default smoke machine
  • identity mappings fully torn down / _start yanked / the kernel moved
  • that the scratch inventory is a CI-gated probe (it is Documented research until M1)

Consequences

  • Ledger, roadmap P-SEC-3l, threat model v1.61, el0.md pointer, SUMMARY, second brain cite this audit.
  • ADR-055 / ADR-060 get a follow-on pointer. Their decisions are not rewritten.
  • No src/ change. No AWS. No CloudAgent. Do not self-merge.

ADR-087 — Identity inventory ratchet + RAM-side identity leftovers tear (M1)

  • Status: Accepted (serial / tests / smoke). M1 of the ADR-086 plan. It unmaps identity leftovers I2 / I4 / I5 and commits a fail-closed TTBR0 identity inventory. Allowed set = {I1 MMIO block, I3 _start stub page}. A planted identity page must be caught in-boot, and a dedicated inv-leak-probe kernel must fail the check. The umbrella “EL0 isolated” stays Planned / non-claim.
  • Date: 2026-09-26
  • Base: stacked on PR #138 (3f4de00). #136 / #137 / #138 are unmerged.
  • Sponsor decisions (DSO relay ~14:35 CEST) arrived while this ran: D2 keeps I3 as the one documented identity exception, and D3 puts MMIO in scope (M2). This ADR does not act on them beyond keeping I1 / I3 allowed for now.

Context

The ADR-086 scratch walker found five identity (VA = PA) ranges left in kernel TTBR0 after the ADR-018…049 tears: I1 MMIO 1 GiB Device block, I2 low RAM 0x4000_0000–0x4008_0000, I3 boot-stub page 0x4008_0000, I4 RO+X linker padding [.ident_tear end, 0x4020_1000), and I5 the pre-heap frame at __kernel_end. User TTBR0 still held I3 + I4. Those tears were section-driven, so nothing ratcheted “what is still identity-mapped”.

Decision

  1. Tear I2 / I4 / I5 (paging::tear_identity_leftovers, right after tear_identity_ram) from kernel and user TTBR0. ASID-B shares the kernel RAM L2. The ranges are paging::leftover_ranges(): [VIRT_RAM_BASE, KERNEL_TEXT), [boot_stub_end, data_start), and [data_tear_end, heap_pa). pages= counts only pages that were still mapped. High twins stay. One boot-time TLBI VMALLE1. _start is never touched: the function refuses any range overlapping the stub page.
  2. Fault probes. An EL1 LDR of a torn leftover VA must take a current-EL translation DABORT: ident: low-fault / ident: tail-fault / ident: kend-fault. is_torn_identity_va learns the three ranges.
  3. Inventory ratchet (paging::identity_inventory). It walks kernel, user and ASID-B TTBR0 (L1→L3), coalesces identity leaves, and prints each as ident: inv-range (allowed) or ident: inv-leak (not allowed, or EL0-accessible), then ident: inv k=<n> u=<n> a=<n> leaks=<n>. ident: ok now requires leaks=0.
  4. In-boot negative probe (inventory_negative_probe). It plants an identity page at 0x4000_0000 in kernel TTBR0 and requires the walk to flag it, then does the same in user TTBR0. It then removes the plant and requires a clean walk: ident: inv-neg k caught / ident: inv-neg u caught / ident: inv-neg clean → ident: inv-ok allow=mmio,stub.
  5. Smoke-level negative probe. Cargo feature inv-leak-probe leaves the plant in place. scripts/qemu-smoke.sh builds and boots it and fails unless the serial shows ident: inv-leak k lo=0x40000000 and no ident: inv-ok / ident: ok. This is CI-gated without a workflow edit.
  6. Smoke requires the markers in 1–4 plus leaks=0 and still requires ident: start-stay. It rejects ident: inv-leak, ident: inv missed, ident: inv-neg missed, and ident: left missed. #[test_case]s: identity_leftovers_torn_stub_stays, identity_inventory_allowlist_only_catches_plant.
  7. Doc drift fix: ADR-055 §A now says el0: no kernel read, which is what code and smoke use. The earlier text said el0: no data. ADR-086 also misstated the rustc on the box smoke: the repo pin is 1.100.0-nightly 0fc141305, not 5ceaf6608. This is corrected in place.

Evidence (this session)

Agent box, 2026-09-26 14:35–14:40 CEST, scripts/qemu-smoke.sh → qemu-smoke: ok (QEMU 10.0.13 Debian, virt -cpu cortex-a76 TCG, rustc 1.100.0-nightly 0fc141305). Hello boot:

ident: low lo=0x40000000 hi=0x40080000 pages=128 user=0
ident: tail lo=0x400f3000 hi=0x40201000 pages=270 user=270
ident: kend lo=0x4025d000 hi=0x4025e000 pages=1 user=0
ident: start-stay lo=0x40080000 hi=0x40081000
ident: low-fault va=0x40000000
ident: tail-fault va=0x40200000
ident: kend-fault va=0x4025d000
ident: inv-range k lo=0x0 hi=0x40000000 dev rw nx
ident: inv-range k lo=0x40080000 hi=0x40081000 mem ro x
ident: inv-range u lo=0x40080000 hi=0x40081000 mem ro x
ident: inv-range a lo=0x0 hi=0x40000000 dev rw nx
ident: inv-range a lo=0x40080000 hi=0x40081000 mem ro x
ident: inv k=2 u=1 a=2 leaks=0
ident: inv-neg k caught va=0x40000000
ident: inv-neg u caught va=0x40000000
ident: inv-neg clean
ident: inv-ok allow=mmio,stub
ident: ok

cargo test: identity_inventory_allowlist_only_catches_plant [ok], identity_leftovers_torn_stub_stays [ok]. inv-leak-probe kernel:

ident: inv-leak-probe planted va=0x40000000
ident: inv-leak k lo=0x40000000 hi=0x40001000 mem rw nx
ident: inv-leak a lo=0x40000000 hi=0x40001000 mem rw nx
ident: inv k=3 u=1 a=3 leaks=2
ident: inv missed
qemu-smoke: inv-leak-probe caught (fail-closed ok)

(ASID-B sees the plant because it shares the kernel RAM L2.)

Before → after (kernel TTBR0 identity ranges): 5 → 2 (I1 MMIO, I3 stub). User TTBR0: 2 → 1 (I3).

b2-serror profile under stock TCG (CTOS_B2_ACCEL=tcg): boots through ident: low/tail/kend to el0: serror-arm → el0: serror-park, the expected result without patch 0002. Not re-run on KVM: no metal was used in this phase.

Surprise: the pointer rewrite also rewrote the allowlist

The first run flagged the stub page as a leak. INV_ALLOW was promoted into .rodata, and the ADR-020/025 identity-pointer rewrite (ident: ro-reloc n=1418) treated the words 0x4008_0000 / 0x4008_1000 as identity text pointers and rewrote them to TTBR1 aliases. The check now compares identity_pa() of the loaded bounds. Lesson: any .rodata constant that looks like a kernel identity address is silently relocated. Security-relevant constants must be compared physically or encoded as immediates.

Honesty

Say: “RAM-side identity leftovers (low RAM, image padding tail, pre-heap frame) are unmapped from kernel and user TTBR0. A fail-closed walk proves only the MMIO block and the _start stub page remain identity-mapped, and a planted identity page is caught.” Do not say:

  • identity mappings are fully torn down (MMIO + stub remain)
  • _start was yanked / the kernel moved
  • “EL0 isolated” / “secure OS”
  • the b2 profile was re-verified on KVM after this change

Consequences

  • Allowed set is {I1, I3}. M2 (ADR-086 plan, D3 accepted) moves MMIO high and shrinks it to {I3}. M3 records I3 as the one documented exception (D2).
  • Ledger, roadmap P-SEC-3l, threat model v1.62, el0.md, SUMMARY, second brain updated. Do not self-merge.

ADR-088 — Device MMIO through a TTBR1 high alias; identity MMIO block torn (M2)

  • Status: Accepted (serial / tests / smoke). M2 of the ADR-086 plan. Sponsor D3 (DSO relay 2026-09-26 ~14:35 CEST): device MMIO is in scope for identity teardown. GIC / PL011 / virtio-mmio now use TTBR1 Device blocks. The identity MMIO 1 GiB L1 block (I1) is cleared. The identity inventory’s allowed set shrinks to {I3 _start stub page}. The umbrella “EL0 isolated” stays Planned / non-claim.
  • Date: 2026-09-26
  • Base: stacked on PR #139 (ADR-087, M1).

Decision

  1. TTBR1 Device alias. paging::init adds three 2 MiB Device-nGnRnE blocks (EL1 RW, PXN+UXN, no EL0 access) to L2_HIGH at TTBR1_BASE + pa: 0x0800_0000 (GICD + GICC), 0x0900_0000 (PL011), 0x0a00_0000 (virtio-mmio transports). That is 6 MiB instead of the 1 GiB identity hole. The block list is built from immediates, because the ADR-087 finding showed that .rodata tables of kernel-looking words get rewritten.
  2. Driver switch. paging::mmio_va(pa) returns the identity PA until enable_mmio_high() and the high alias afterwards. uart, gic and virtio register accessors all go through it. enable_mmio_high() runs first thing in kernel_main_high (after the high jump, before any tear) and prints ident: mmio-high gic=… uart=… virtio=…. The flag is a plain atomic load, so the UART is safe pre-MMU, in fatal paths, and while TABLES is held.
  3. Tear I1. tear_identity_mmio() runs after the ADR-087 leftovers tear. It clears kernel L1[0] and the ASID-B copy; user TTBR0 never had it. One boot-time TLBI VMALLE1, then it verifies the GIC / PL011 / virtio identity VAs are unmapped: ident: mmio lo=0x0 hi=0x40000000 l1=1. prepare_asid_b_tables no longer requires an identity MMIO slot.
  4. Fault probe. An EL1 LDR of the torn identity UARTFR (0x0900_0018) must take a translation DABORT before any device access: ident: mmio-fault va=0x9000018.
  5. Allowlist shrink. INV_ALLOW = {[0x4008_0000, 0x4008_1000)}. The smoke requires ident: inv k=1 u=1 a=1 leaks=0 and ident: inv-ok allow=stub, and rejects any ident: inv-range k|a lo=0x0. W^X keeps “MMIO is XN” via paging::mmio_xn, which checks whichever alias is live.
  6. #[test_case] identity_mmio_torn_high_alias_xn.

Evidence (this session)

Agent box 2026-09-26 ~14:43–14:44 CEST scripts/qemu-smoke.sh → qemu-smoke: ok (QEMU 10.0.13, cortex-a76 TCG, rustc 0fc141305):

ident: jump
ident: mmio-high gic=0xffffff8008000000 uart=0xffffff8009000000 virtio=0xffffff800a000000
ident: low lo=0x40000000 hi=0x40080000 pages=128 user=0
ident: tail lo=0x400f3000 hi=0x40201000 pages=270 user=270
ident: kend lo=0x4025d000 hi=0x4025e000 pages=1 user=0
ident: mmio lo=0x0 hi=0x40000000 l1=1
Hello World!
ident: start-stay lo=0x40080000 hi=0x40081000
ident: mmio-fault va=0x9000018
ident: inv-range k lo=0x40080000 hi=0x40081000 mem ro x
ident: inv-range u lo=0x40080000 hi=0x40081000 mem ro x
ident: inv-range a lo=0x40080000 hi=0x40081000 mem ro x
ident: inv k=1 u=1 a=1 leaks=0
ident: inv-neg k caught va=0x40000000
ident: inv-neg u caught va=0x40000000
ident: inv-neg clean
ident: inv-ok allow=stub
ident: ok

All UART output after ident: mmio-high went through the TTBR1 alias, as did the virtio-blk / virtio-net / FAT / GIC timer and IRQ / FIQ probes. wx: ok and cargo test identity_mmio_torn_high_alias_xn [ok]. The inv-leak-probe kernel is still caught.

Identity ranges: kernel TTBR0 2 → 1, ASID-B 2 → 1, user 1 (unchanged). Only the _start stub page remains.

The b2-serror profile under stock TCG boots through ident: mmio-high / ident: mmio to el0: serror-arm → park, the expected result without patch 0002. It was not re-run on KVM. The ADR-085 B2-P evidence was taken at 159b178, before M1/M2 changed the boot path (tears + MMIO alias). A KVM re-run would cost about $0.2 (≈ 45 min of c7g.metal Spot). Not done in this phase.

Honesty

Say: “device MMIO is reached only through EL1-only TTBR1 Device blocks; the identity MMIO block is unmapped; the only identity mapping left in any TTBR0 is the _start stub page, and a fail-closed walk proves it.” Do not say:

  • identity teardown is “complete”. I3 remains, as the documented exception (D2 / M3).
  • _start was yanked / the kernel moved
  • “EL0 isolated” / “secure OS”
  • the b2 profile was re-verified on KVM after M1/M2

Consequences

  • ADR-055 §B row 2 (as reworded by M3 per D2) has its evidence: identity = {stub} only. M3 records the exception; the umbrella still needs M4 (D1 evidence), M5 (draft accept) + sponsor accept, and M6.
  • Ledger, roadmap P-SEC-3l, threat model v1.63, el0.md, SUMMARY, second brain. Do not self-merge.

ADR-089 — _start stub page is the one documented identity exception (M3, sponsor D2)

  • Status: Accepted (docs). Records sponsor decision D2 (DSO relay 2026-09-26 ~14:35 CEST): keep the _start boot-stub page (I3) as the one documented identity exception, never yank _start, and reword ADR-047 §3 plus the checklist’s identity-teardown row to match. No src/ change. The umbrella “EL0 isolated” stays Planned / non-claim.
  • Follow-on (2026-09-28, M6): B3 met: the sponsor accepted the ADR-091 revision-3 sentence and scope in writing (2026-09-28 16:41 CEST). ADR-091 is Accepted. “EL0 isolated” is Verified only as that sentence, with its limits. Text in this ADR that says the umbrella is Planned / non-claim is as of this ADR’s date.
  • Date: 2026-09-26
  • Base: stacked on PR #140 (ADR-088, M2).

Context

ADR-055 §B row 2 read “Full identity teardown including yank _start”, while ADR-047 §3 decided “never yank _start”. ADR-086 flagged the row as unmeetable as worded and asked the sponsor (D2). After ADR-087 (M1) and ADR-088 (M2), a fail-closed walk shows that the stub page is the only identity mapping left in any TTBR0.

Decision

  1. The exception. Exactly one identity (VA = PA) mapping may remain: the page [0x4008_0000, 0x4008_1000) that holds _start and the identity copy of exception_vectors (VBAR is high). It must be EL1 read-only + executable, never EL0-accessible, and present in kernel, user and ASID-B TTBR0 as it is today. paging::INV_ALLOW encodes exactly this page. Any other identity leaf fails scripts/qemu-smoke.sh (ident: inv-leak).
  2. Never yank or move _start. QEMU -kernel enters at 0x4008_0000; the image stays there. (ADR-086 D2 option (b), unmapping the stub’s identity mapping after boot, was not chosen.)
  3. Reworded requirement (ADR-055 §B row 2, ADR-060 “Yank _start” / umbrella rows): “Full identity teardown except the documented _start stub page (ADR-089): the fail-closed TTBR0 identity inventory reports only that page (ident: inv k=1 u=1 a=1 leaks=0 + ident: inv-ok allow=stub) on the default smoke.”
  4. ADR-047 §3 amended in place (history kept): “never yank _start” stands; leftover identity RAM is no longer “optional”, since ADR-049 + ADR-087 tore it; the stub page is the one documented exception.
  5. Row status after this ADR: reworded row 2 = Met on evidence from this session. That evidence is the ADR-088 box smoke 2026-09-26 ~14:43 CEST, with CI on #140 recorded in the ledger once green. That is not the umbrella: rows 1 (taken SError → M4 / D1) and 3 (sponsor accept → M5 draft, then the sponsor’s own accept) remain.

Future narrowing (not in scope, not decided)

The stub page is in user TTBR0 only because fill_user_map maps [KERNEL_TEXT, data) as EL1 text. VBAR is high, so EL0 exceptions do not fetch it through TTBR0. Removing it from user TTBR0 alone would leave kernel TTBR0 and _start untouched. That would need a sponsor call; this ADR does not act on it.

Honesty

Say: “the only identity mapping left is the documented _start stub page (EL1 RO+X), ratcheted fail-closed.” Do not say “identity fully torn down”, “_start yanked”, “kernel moved”, or “EL0 isolated”.

Consequences

  • ADR-047 §3, ADR-055 §B, ADR-060 and ADR-042 honesty wording point here. Ledger, roadmap P-SEC-3l, threat model v1.64, el0.md, SUMMARY, MOC.
  • Next: M4 (D1 evidence set), M5 (draft accept). Stop before M6.

ADR-090 — Taken-SError evidence class for the umbrella = B1 pin + B2-P (M4, sponsor D1)

  • Status: Accepted (docs + CI workflow). Records sponsor decision D1 (DSO relay 2026-09-26 ~14:35 CEST). The taken lower-EL SError evidence that counts for the umbrella is the B1 pinned-QEMU CI job plus the B2-P run, stated verbatim below and copied into the draft accept ADR (ADR-091). Makes the qemu-nmi-pin job unconditional. No src/ logic change; one stale M2 doc comment in src/teardown.rs is fixed (the allowlist has been stub-only since ADR-088). The umbrella “EL0 isolated” stays Planned / non-claim: the sponsor’s written accept (B3) is still absent.
  • Follow-on (2026-09-28, M6): B3 met: the sponsor accepted the ADR-091 revision-3 sentence and scope in writing (2026-09-28 16:41 CEST). ADR-091 is Accepted. “EL0 isolated” is Verified only as that sentence, with its limits. Text in this ADR that says the umbrella is Planned / non-claim is as of this ADR’s date.
  • Date: 2026-09-26
  • Base: stacked on PR #141 (ADR-089, M3).

Context

ADR-055 §B row 1 required “honest host inject that delivers async SError on the accepted smoke machine” (ADR-060). ADR-086 found two taken-SError sources, neither on that machine as worded. B1 (ADR-083) is a CI job on a locally patched QEMU, gated by repo variable CTOS_BUILD_QEMU_NMI=1, so it is opt-in. B2-P (ADR-085) is one paid real-hardware KVM run, not CI. ADR-086 asked D1: which evidence counts? The sponsor accepted the recommendation (i)+(ii): both together, stated verbatim, with no silent redefinition.

Decision

  1. D1 evidence class (verbatim; ADR-091 must quote this paragraph unchanged):

    Taken lower-EL SError while standing at EL0 is evidenced by two sources together. (i) B1: the GitHub Actions job qemu-nmi-pin (“QEMU NMI pin smoke (taken SError)”) in .github/workflows/smoke.yml. It builds QEMU v10.0.0 (7c949c53e936aa3a658d84ab53bae5cadaa5d59c) with reviewed local patch research/qemu-nmi/0001-hw-arm-virt-TYPE_NMI-raise-SError.patch (TCG: virt TYPE_NMI → async SError, masked by PSTATE.A), boots the default image on virt -cpu cortex-a76 TCG, issues QMP stop → inject-nmi → cont on the guest’s el0: serror-arm cue, and with CTOS_REQUIRE_TAKEN_SERROR=1 fails closed unless the bare el0: serror line (printed only by the lower-EL SError vector while the standing-EL0 expectation is armed) appears. It runs on every push and pull request. (ii) B2-P: ADR-085 run 4, 2026-09-26 13:26–13:33 CEST, AWS Graviton3 c7g.metal (eu-north-1b, Spot), ctos commit 159b178, QEMU -accel kvm with patch 0002 (inject-nmi → KVM_SET_VCPU_EVENTS{serror_pending=1} → HCR_EL2.VSE), opt-in b2-serror profile: el0: serror + b2: taken, fail-closed b2-kvm-smoke.py PASS. It is a one-off real-hardware run, not CI. Stock distro QEMU (the default smoke) cannot inject and continues to require el0: serror-park.

  2. Make B1 unconditional. Remove if: ${{ vars.CTOS_BUILD_QEMU_NMI == '1' }} from qemu-nmi-pin and rename the job from “(opt-in)” to “QEMU NMI pin smoke (taken SError)”. Why: D5 makes the smoke a GitHub-required check before the flip. GitHub treats a job skipped by if: as passing a required check, so a variable-gated job would let a PR pass without taking an SError. This reverses ADR-083 §6 (“do not force multi-minute QEMU compile on every PR”) on purpose. Cost: ≈ 10–15 min on a free GitHub-hosted ubuntu-24.04 runner per push/PR. The repo is public, so standard runners are free. The repo variable becomes unused and can be deleted by the sponsor at any time; its value no longer matters.

  3. Reworded requirement (ADR-055 §B row 1, ADR-060 taken-SError row): “Taken lower-EL SError while standing, on the D1 evidence class (ADR-090): B1 qemu-nmi-pin green, unconditional on every push/PR, plus the B2-P one-off KVM run (ADR-085 run 4).”

  4. Row status: reworded row 1 = Met (D1 evidence class). The B1 leg is green after the boot-path changes of this session: #139 (post-M1) pin jobs 108406107228 / 108406127586 and #140 (post-M2) pin job 108408283187 (inject-nmi=>ok → bare el0: serror, 2026-09-26 14:56–15:10 CEST). The first unconditional run is on this PR’s CI. This is not the umbrella: row 3 (sponsor accept) remains, see M5 → ADR-091.

  5. Not changed: the default smoke on stock QEMU still requires el0: serror-park (ADR-053/081). ADR-053’s hard-stop stays true for stock virt TCG. Dockerfile ARG CTOS_BUILD_QEMU_NMI=0 stays opt-in (local images only). No KVM in CI.

Caveats stated with the evidence (must travel with any quote of §1)

  • B1 is a locally patched emulator. The SError is raised by QEMU TCG code this repo carries (patch 0001), not by stock upstream QEMU and not by hardware. It proves that the guest’s lower-EL SError path takes the exception; it does not prove any CPU’s RAS behaviour.
  • B2-P is historical. It ran once at 159b178, before this session’s M1/M2 boot-path changes (identity leftover tears, MMIO high alias). The b2-serror profile still TCG-boots through its markers after M2 (agent box, ~14:44 CEST), but it was not re-run under KVM. A re-verify is one more paid c7g.metal Spot run, ≈ $0.2 (ADR-085 cost basis). D1 does not require it; it is a sponsor option.
  • B2-P used a reduced profile (no GIC / timer / virtio / FAT), per ADR-085.
  • Neither leg is real hardware running the full default image.

Honesty

Say: “per sponsor D1 (ADR-090), taken lower-EL SError is evidenced by the B1 pinned-QEMU CI job, which is unconditional and fail-closed on every push/PR, plus the one-off B2-P Graviton3 KVM run (ADR-085 run 4). Stock QEMU parks.” Do not say “taken SError on stock QEMU”, “taken SError Verified on real hardware in CI”, “SError hard-stop lifted” (it stands for stock TCG), or “EL0 isolated”.

Consequences

  • .github/workflows/smoke.yml: qemu-nmi-pin unconditional and renamed. docs/dev/qemu-nmi-pin-ci.md updated.
  • D5 / M6 required-check names (branch protection is not changed here; the sponsor/DSO sets it at M6): QEMU aarch64 smoke (ubuntu-24.04-arm), QEMU aarch64 smoke (ubuntu-24.04), QEMU NMI pin smoke (taken SError).
  • ADR-055 §B row 1, ADR-060 and ADR-083 follow-ons; ledger, roadmap P-SEC-3r/3l, threat model v1.65, el0.md, SUMMARY, MOC.
  • Next: M5 (ADR-091 draft accept). Stop before M6.

ADR-091 — Umbrella “EL0 isolated” accept (sentence + scope) — Accepted (M5 draft, sponsor D4; B3 met 2026-09-28; M6; revision 4 accepted 2026-10-02)

  • Revision 4 (2026-10-02): accepted. The sponsor accepted ADR-091 revision 4 in writing (“I accept ADR-091 rev 4”, record below). Revision 4 changes exactly two things in the accepted text: the PAN clause of the sentence, and the limit “EL0 can read its own code bytes”, now “app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN)”. Every other clause, the scope table and the non-scope list are unchanged. The wording is ADR-097’s rev 4 proposal, copied character for character. Everything below that says the sentence is “unchanged from revision 3” describes the state before 2026-10-02 and is kept as history.
  • Status (as of revision 3, kept for history): Accepted (2026-09-28, M6). The sponsor accepted the revision-3 sentence and scope in writing: B3 is met (record below). “EL0 isolated” is Verified only as the sentence below, read together with the non-scope list below. The sentence and the scope are unchanged from revision 3.
  • Draft status (2026-09-26 to 2026-09-28, kept for history): Draft / pending sponsor accept. Not accepted. Not a claim. This file proposes the exact umbrella sentence and its scope for the sponsor to read (decision D4, DSO relay 2026-09-26 ~14:35 CEST). Checklist row B3 (written sponsor accept, ADR-055 §B row 3) is not Met and must not be counted until the sponsor has read this draft and accepted it in writing. Until then the umbrella stays Planned / non-claim. Nothing is flipped here: no ledger status change, no roadmap Verified. The flip is M6 and is out of scope for this PR.
  • Date: 2026-09-26 (draft); revision 2: 2026-09-27 after sponsor option (b) (2026-09-27 00:09 CEST: “Close G1 and G2 first … then bring the sentence back to me”). ADR-092 closed G1 and G2 with CI-gated fail-closed probes and found G4 (below). Revision 3: 2026-09-27. The sponsor skipped the G4 question, so the DSO applied the standing order (close G4 before bringing the sentence back). ADR-094 closed G4 with CI-gated fail-closed probes. The “direct EL0 memory access only” carve-out is dropped. G3 is the only remaining limit, and it is optional. The sentence below is restated for the sponsor. Still Draft; B3 still not Met; nothing flipped.
  • Limits update (2026-09-28, not a revision of the sentence): the sponsor asked for G3. ADR-095 re-ran the ADR-085 B2-P procedure on main 8107578 on an AWS Graviton3 c7g.metal under KVM: PASS (el0: serror / b2: taken, fail-closed smoke, attempt 1). G3 is done. The revision-3 sentence and scope are unchanged and need no change. Still Draft; B3 still not Met; nothing flipped.
  • Correction (2026-09-29, not a revision of the sentence): the clause “PAN is enabled” held at boot but, from the ADR-080 enable and so also from the M6 accept until ADR-098, not while the kernel handled EL0 exceptions (SCTLR_EL1.SPAN was 1) and not after EL0 trips (return_from_el0 resumed with SPSR.PAN clear). The ADR-080 boot probe missed it because it ran before any EL0 trip. ADR-098 fixes both and adds fail-closed live checks (pan: live-syscall pstate-pan=1 sctlr-span=0, pan: after-el0 …, pan: ok held …). With the fix every clause is literally true again. The sentence and scope are unchanged.
  • Base: stacked on PR #142 (ADR-090, M4).
  • Merge: sponsor only (ADR-002). Merging this draft does not constitute acceptance; acceptance is a separate written sponsor statement recorded by a follow-up edit (M6).

B3: the sponsor’s written accept (verbatim; met 2026-09-28)

  • Quote (verbatim): “I accept the ADR-091 revision-3 sentence and scope as written. After M6, have ctos propose a short costed list of next milestones.”
  • Date and time: 2026-09-28 16:41 CEST (Europe/Oslo).
  • Relay path: sponsor wrote it in the DSO agent chat, DSO relayed it to ctos.
  • Question answered: “Do you accept the ADR-091 revision-3 sentence and scope as written?” The DSO asked it after showing the sponsor the revision-3 sentence verbatim and the list of what it does not cover.
  • B3 (ADR-055 §B row 3): Met. The accept covers the revision-3 sentence and the scope and non-scope below exactly as written. This edit does not change either.
  • The second sentence of the quote is a request to ctos for after M6. It is not part of the claim.

B3 rev 4: the sponsor’s written accept (verbatim; 2026-10-02)

  • Quote (verbatim): “I accept ADR-091 rev 4”
  • Date: 2026-10-02 (Europe/Oslo). The relay did not give a time of day.
  • Relay path: the sponsor wrote it in the DSO chat, DSO recorded it in the DSO vault, and DSO relayed it to ctos.
  • Question answered: the exact wording proposed in ADR-097 (section “Revision-4 wording”, formerly “Proposed revision-4 wording for the sponsor”): the new PAN clause and the new code-bytes limit, everything else unchanged.
  • What the accept covers: the two changes above, exactly as worded. It does not widen the claim, add a scope item or change the non-scope list.
  • Implementing change: the XO app text change, PR #150 / ADR-097, merged as 8344797 (2026-10-02, DSO merged it after the accept). The PAN-held-during-EL0-exceptions fix that the new PAN clause relies on is PR #151 / ADR-098, on main as 3c47c0c.
  • Rev 3’s accept (B3, above) stays on record for revision 3. This follow-up edit is docs only.

Accepted umbrella sentence (verbatim, revision 4; accepted 2026-10-02)

Revision 3 (accepted 2026-09-28) differed only in the PAN clause. Its text is kept in the History section.

“On the ctos default smoke machine (QEMU virt, -cpu cortex-a76, TCG, one core), EL0 is isolated from the kernel at the architectural page-table / exception-level boundary: each EL0 task runs on its own TTBR0 with its own ASID; EL0 faults when it reads, writes or executes kernel memory; in every translation table (kernel, user and ASID-B TTBR0, and TTBR1) the only EL0-reachable pages are five allowlisted user slots backed by non-kernel frames, checked fail-closed while a user app runs and at rest; PAN is enabled, including while the kernel handles exceptions from EL0, and EL1 faults on EL0-readable memory (the execute-only app text page is not covered: this CPU has no FEAT_EPAN); lower-EL IRQ and FIQ are taken while EL0 stands; taken lower-EL SError is evidenced by the ADR-090 evidence class; the only identity (VA = PA) mapping left in any TTBR0 is the documented EL1-only _start stub page (ADR-089); and a system call copies from or to an EL0 pointer only if EL0 itself may read (for a copy from EL0) or write (for a copy to EL0) every page in the range. It is not a speculative-execution or side-channel claim, not a real-hardware claim, and not a certification.”

Scope: what the sentence covers, and the evidence for each clause

#ClauseEvidence (all CI-gated by scripts/qemu-smoke.sh unless noted)ADR
S1Default smoke machine.github/workflows/smoke.yml jobs QEMU aarch64 smoke (ubuntu-24.04-arm) / (ubuntu-24.04): distro QEMU virt -cpu cortex-a76 TCG, no -smp (one core)ADR-079
S2Own TTBR0 + ASID per EL0 taskel0: standing, el0: task-ok, asid: ok; EL0 entry without VMALLE1 (el0: no-vmalle1)ADR-013 / 024 / 039
S3EL0 faults on kernel read / write / executeRead: el0: no kernel read (EL0 load of the torn identity alias of kernel .data → translation fault) and ttbr1: no el0 (EL0 load of the mapped TTBR1 private page → data abort). Write (G2, closed): el0: write-fault kernel va=0xffffff80… esr=0x9200004f far=<same> ec=0x24 wnr=1 dfsc=perm-l3 (store to kernel .data via TTBR1), el0: write-fault user-ro va=0x80009000 esr=0x9200004f far=0x80009000 ec=0x24 wnr=1 dfsc=perm-l3, el0: write-ok kernel,user-ro; the smoke rejects el0: write-succeeded / el0: write-bad, and the write-leak-probe build must report el0: write-succeeded. Execute: el0: nx kernel (EL0 fetch of kernel data → instruction abort)ADR-013 / 016 / 092
S8Every EL0-reachable page is an allowlisted user slot (G1, closed)Fail-closed walk of kernel / user / ASID-B TTBR0 and TTBR1 (EL0-reachable = AP[1] set or UXN clear; rules: allowlisted VA, pool frame outside heap, not Device, not EL0 W+X, nothing in TTBR1): el0-reach: live k=3 u=3 a=0 h=0 pages=6 leaks=0 (hello-libctos standing at EL0), el0-reach: steady k=0 u=0 a=0 h=0 pages=0 leaks=0, el0-reach: neg {kernel-pa,va,ttbr1,el0-wx} caught …, el0-reach: neg clean, el0-reach: ok allow=app-hdr[0x80000000,0x80001000),app-text[0x80002000,0x80003000),crt-stack-pan[0x80003000,0x80004000),user-stack[0x80007000,0x80008000),store-ro[0x80009000,0x8000a000); the reach-leak-probe build must report el0-reach: leak … why=kernel-paADR-092
S4PAN enabled, EL1-vs-EL0 faultpan: present, pan: enabled, pan: el1-fault. Added for revision 4: PAN held while the kernel handles EL0 exceptions and after EL0 trips (pan: live-syscall pstate-pan=1 sctlr-span=0, pan: after-el0 …, pan: ok held …); EL1 faults on EL0-readable memory (xo: el1-pan control app-hdr … fault=yes spsr-pan=1); the execute-only text page is not covered, pinned as a fact (xo: el1-pan text … fault=no … mmfr1-pan=2 epan=no)ADR-079 / 080 / 097 / 098
S5Lower-EL IRQ / FIQ while standingel0: irq, el0: irq-default, el0: fiqADR-040 / 041 / 043
S6Taken lower-EL SErrorD1 evidence class, quoted verbatim from ADR-090 §1 (below). The default smoke itself parks (el0: serror-park).ADR-083 / 085 / 090
S9Syscall pointers need EL0 permission on every page (G4, closed)el0: sys-ptr control rw-data n=6; el0: sys-ptr denied kernel-stub (SYS_UART_WRITE of the _start stub page); el0: sys-ptr denied kernel-window (kernel-only map-window page); el0: sys-ptr denied xo-text and el0: sys-ptr denied write-ro (SYS_NET_MAC copy_to_user into EL0’s own execute-only text / an EL0-RO page, target unchanged); el0: sys-ptr denied straddle; seven lines el0: sys-ptr denied sys=<name> nr=<n> … (every pointer-taking syscall: 17, 19, 20, 21, 22, 24, 27); el0: sys-ptr sweep syscalls=7 denied=7; el0: sys-ptr ok. The smoke rejects el0: sys-ptr leaked / bad / skip / probe missed. The sysptr-leak-probe build must report el0: sys-ptr leaked kernel-stub n=8 etc. and withhold okADR-094
S7Only identity mapping = _start stub pageFail-closed TTBR0 inventory: ident: inv k=1 u=1 a=1 leaks=0, ident: inv-ok allow=stub, negative probes ident: inv-neg … caught, inv-leak-probe build caught; leftover/MMIO fault probes ident: {low,tail,kend,mmio}-fault. The stub page is EL1 RO+X with UXN set, so EL0 cannot fetch itADR-087 / 088 / 089

S6: D1 evidence paragraph (verbatim from ADR-090 §1)

Taken lower-EL SError while standing at EL0 is evidenced by two sources together. (i) B1: the GitHub Actions job qemu-nmi-pin (“QEMU NMI pin smoke (taken SError)”) in .github/workflows/smoke.yml. It builds QEMU v10.0.0 (7c949c53e936aa3a658d84ab53bae5cadaa5d59c) with reviewed local patch research/qemu-nmi/0001-hw-arm-virt-TYPE_NMI-raise-SError.patch (TCG: virt TYPE_NMI → async SError, masked by PSTATE.A), boots the default image on virt -cpu cortex-a76 TCG, issues QMP stop → inject-nmi → cont on the guest’s el0: serror-arm cue, and with CTOS_REQUIRE_TAKEN_SERROR=1 fails closed unless the bare el0: serror line (printed only by the lower-EL SError vector while the standing-EL0 expectation is armed) appears. It runs on every push and pull request. (ii) B2-P: ADR-085 run 4, 2026-09-26 13:26–13:33 CEST, AWS Graviton3 c7g.metal (eu-north-1b, Spot), ctos commit 159b178, QEMU -accel kvm with patch 0002 (inject-nmi → KVM_SET_VCPU_EVENTS{serror_pending=1} → HCR_EL2.VSE), opt-in b2-serror profile: el0: serror + b2: taken, fail-closed b2-kvm-smoke.py PASS. It is a one-off real-hardware run, not CI. Stock distro QEMU (the default smoke) cannot inject and continues to require el0: serror-park.

The ADR-090 caveats travel with it: B1 is a locally patched emulator; B2-P is historical (at 159b178, before M1/M2) on a reduced profile; a KVM re-verify ≈ $0.2 is a sponsor option. Update 2026-09-28: the KVM re-verify was done (ADR-095): the B2-P leg passed again on main 8107578 (post-M1/M2, ADR-092, ADR-094), still on the reduced b2-serror profile and still a one-off, not CI.

Explicitly not in scope (the sentence must never be read as covering these)

  • Speculative execution / side channels. No KPTI: the TTBR1 kernel mapping stays present (EL1-only) while EL0 runs. No Spectre/Meltdown/cache/timing analysis; QEMU TCG is the wrong lab (threat model: “not side-channel complete”).
  • Real hardware. Only the emulated default machine, plus the one-off B2-P KVM run on the reduced b2-serror profile. No board, no Graviton CI. (The B2-P run was repeated once on current main for G3, ADR-095; still reduced profile, still not CI.)
  • Multi-core. One core only. No SMP TLB-shootdown or cross-core isolation story.
  • EL0-to-EL0 (task-vs-task) isolation beyond the asid: ok TLB mile and per-task TTBR0. No probe that one EL0 task cannot read another task’s pages.
  • Hypervisor / EL2 / secure world, DMA / IOMMU, physical attacks. The virtio device can DMA anywhere; there is no SMMU.
  • Certification, formal verification, or “secure OS”. Not a guest-Linux, container or immutable-OS claim (ADR-029 non-goals stand).
  • Stock-QEMU taken SError. Stock TCG still parks. The ADR-053 hard-stop for stock virt TCG stands.
  • Check-to-copy races on SMP. S9 is a software check on one core. Nothing changes the calling task’s user page tables during a syscall. No AT S1E0R/W, no LDTR/STTR copy, no fault-fixup copy (ADR-094).

Gaps (revision 3, 2026-09-27)

  • G1: exhaustive EL0-permission walk. CLOSED by ADR-092 (S8 markers above). Correction to revision 1: its code-reading note missed l3_page_el0_exec (EL0 execute-only, UXN clear). The walk covers both AP[1] and UXN, which supersedes that note.
  • G2: EL0 write probe. CLOSED by ADR-092 (S3 write markers above).
  • G3: B2-P re-verify on the post-M2 boot path. DONE (2026-09-28) by ADR-095: AWS Graviton3 c7g.metal Spot, QEMU v10.0.0 + 0001 + 0002 -accel kvm, ctos main 8107578: el0: serror + b2: taken, b2-kvm-smoke: PASS, attempt 1, ≈ $0.04. Still the reduced b2-serror profile and a one-off, not CI. Revision-3 text, kept for history: “(≈ $0.2 paid run). Open, optional, not required by D1. This is the only remaining limit.” No known limit remains open; B3 (the sponsor’s written accept) is the only step left before M6.
  • G4: syscall copy helpers skip the EL0 permission check. CLOSED by ADR-094 (S9 markers above): user_range_ok_max now requires EL0-readable (reads) / EL0-writable (writes) on every page, and the loader maps app text+rodata EL0 read-only + executable. The G1 allowlist is unchanged (five slots). Revision-2 note, kept for history: found by ADR-092; documented by code reading; not probed, not fixed at that time. syscall::user_range_ok_max accepts any page with a valid leaf in user TTBR0, and copy_user then clears PAN. So EL0 could ask SYS_UART_WRITE to copy bytes of an EL1-only page mapped in user TTBR0: the _start stub page, or an EL1-only map-window page present while the app runs. copy_to_user could also write the app’s own EL0 execute-only text. Fix: require EL0 read/write permission on the leaf (or AT S1E0R/W). The loader must then map app rodata EL0-readable (today it shares the execute-only text page). Revision-4 note (2026-10-02): ADR-097 later moved rodata to its own EL0-readable page and made app text EL0 execute-only again (still five slots).

The sponsor can:

  • (a) accept the revision-3 sentence as it stands, with G3 (optional B2-P re-verify) as the only known limit;
  • (b) require G3 (the ≈ $0.2 KVM re-verify) first; (done 2026-09-28, ADR-095: PASS; so (a) and (b) now lead to the same sentence)
  • (c) narrow the sentence further.

Revision-2 options were (a) accept with G3/G4 as limits, (b) fix G4 first, (c) narrow. The sponsor skipped that question. The DSO applied the standing order, which is (b), done in ADR-094.

History

Revision 1 (2026-09-26) proposed: “… EL0 faults when it reads or executes kernel memory; PAN is enabled … (ADR-089). This is not a speculative-execution or side-channel claim, not a real-hardware claim, and not a certification.” Revision 2 adds “writes”, the all-tables EL0-reachability clause (S8) and the direct-access-only limit (G4). Revision 2’s sentence ended its clause list with “… _start stub page (ADR-089). This covers direct EL0 memory access only, not data the kernel copies on EL0’s behalf in system calls. It is not …”. Revision 3 replaces that carve-out with the S9 syscall-pointer clause (G4 closed by ADR-094).

Revision 4 (2026-10-02, accepted) changes only the PAN clause and one limit; the sponsor accepted it in writing (“I accept ADR-091 rev 4”, record above). Revision-3 text, kept for history:

  • PAN clause, before (revision 3): “PAN is enabled and EL1 faults on EL0 memory”
  • PAN clause, after (revision 4): “PAN is enabled, including while the kernel handles exceptions from EL0, and EL1 faults on EL0-readable memory (the execute-only app text page is not covered: this CPU has no FEAT_EPAN)”
  • Limit, before (revision 3): “EL0 can read its own code bytes” (app text and rodata were one EL0 read-only + executable page, ADR-094)
  • Limit, after (revision 4): “app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN)”

Why: ADR-097 (PR #150) makes app text EL0 execute-only. This CPU has no FEAT_EPAN, so with PAN set an EL1 load of that page does not fault, which made the literal revision-3 clause false for it. ADR-098 (PR #151, 3c47c0c) separately fixed PAN being clear while the kernel handled EL0 exceptions and after EL0 trips, which is why the new clause can say “including while the kernel handles exceptions from EL0”. All other clauses of the sentence are identical to revision 3.

M6 prerequisites (all met 2026-09-28; the flip is the M6 PR, branch isolation/m6-el0-isolated-flip)

  1. B3: the sponsor’s written accept of the revision-3 sentence and scope (option a/b/c above), recorded verbatim with date and relay path. Merging this PR is not acceptance. Met: 2026-09-28 16:41 CEST, recorded above.
  2. D5: before the flip, the sponsor/DSO sets branch protection on main to require QEMU aarch64 smoke (ubuntu-24.04-arm), QEMU aarch64 smoke (ubuntu-24.04) and QEMU NMI pin smoke (taken SError). The agent does not change branch protection. Met: the three checks are required on main (set by the sponsor/DSO).
  3. The stack merged in order: #136–#138 are already merged (2026-09-26); #139 → #140 → #141 → #142 → #143 (this draft) → #144 (ADR-092) → ADR-094’s PR remain, with all three checks green on the merge tip. Met: #139–#147 are merged; main at 68d4c8f (merge of #147, ADR-095) is green on all three checks.
  4. The flip PR changes ADR-055 §B row 3, ADR-060, the ledger P-SEC-3l row, roadmap, el0.md, SUMMARY and the threat model only to the accepted wording, and cites this ADR (status → Accepted). Done in the M6 PR.

Honesty (Accepted, M6; revision 4 since 2026-10-02)

Say “EL0 isolated” only as the sentence above (revision 4): quote it verbatim or point to it, and never widen it. Keep its limits next to it: one core; the TCG default smoke machine; not a speculative-execution or side-channel claim; not a real-hardware claim; not a certification; app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN) (ADR-097); no multi-core / SMP race story. Do not read it as covering anything in the non-scope list above.

Honesty text while Draft (kept for history): Say: “ADR-091 is a draft of the umbrella sentence and scope, pending the sponsor’s written accept; ‘EL0 isolated’ stays Planned / non-claim.” Do not quote the proposed sentence as a statement of fact, count B3 as Met, or flip any status.

ADR-092 — G1 EL0-reachability walk + G2 EL0 write permission fault (sponsor option (b) on ADR-091)

  • Status: Accepted (implementation + evidence). Closes ADR-091 gaps G1 and G2 with fail-closed probes that CI gates. Found a new gap, G4 (syscall copy helpers; below), which is recorded, not fixed. The umbrella “EL0 isolated” stays Planned / non-claim. ADR-091 stays Draft; B3 (the sponsor’s written accept) is not Met.
  • Follow-on (2026-09-28, M6): B3 met: the sponsor accepted the ADR-091 revision-3 sentence and scope in writing (2026-09-28 16:41 CEST). ADR-091 is Accepted. “EL0 isolated” is Verified only as that sentence, with its limits. Text in this ADR that says the umbrella is Planned / non-claim is as of this ADR’s date.
  • Date: 2026-09-27
  • Sponsor decision: 2026-09-27 00:09 CEST, option (b) on ADR-091: “Close G1 and G2 first (full EL0-reachable walk plus an EL0 write test), then bring the sentence back to me.”
  • Base: stacked on PR #143 (ADR-091 draft).
  • _start (0x4008_0000) untouched: no probe targets it, and the stub page stays the ADR-089 exception.

Context

ADR-091 rested its clause “EL0 faults when it reads or executes kernel memory” on two single-address probes plus a code-reading note. It listed G1 (no exhaustive EL0-permission walk) and G2 (no EL0 write probe). The sponsor asked for both before re-reading the sentence.

Decision

G1 — fail-closed EL0-reachability walk (paging::el0_reach, src/reach.rs)

  1. What is walked: every valid leaf of four translation roots:

    • k: kernel TTBR0
    • u: user TTBR0
    • a: ASID-B TTBR0
    • h: TTBR1, added beyond the request because EL0 can walk TTBR1 while TCR_EL1.EPD1 = 0
  2. EL0-reachable = AP[1] set (EL0 load/store) or UXN clear (EL0 fetch). Table-level APTable/UXNTable bits are ignored. That can only over-count, and this kernel never sets them.

  3. Rules per coalesced run. Any failure is a leak, printed el0-reach: leak … why=<rule>:

    • ttbr1: no EL0-reachable leaf in TTBR1 at all.
    • device: never Device memory.
    • kernel-pa: backed only by frame-pool frames, [pool_start, pool_end). Never the image, linker stacks or static page tables below pool_start, and never outside RAM.
    • heap-pa: never the kernel heap run.
    • el0-wx: never EL0-writable and EL0-executable at once.
    • va: inside one EL0_ALLOW range.
  4. EL0_ALLOW: the five 4 KiB map-window slots this kernel hands to EL0, taken from every map_el0_* call site:

    NameVAUsed by
    app-hdr[0x8000_0000, 0x8000_1000)loader: rust-lld ELF-header PT_LOAD (EL0-RO)
    app-text[0x8000_2000, 0x8000_3000)EL0_PAGE: loaded app text + rodata (EL0 execute-only), EL0 probe trampolines
    crt-stack-pan[0x8000_3000, 0x8000_4000)libctos CRT stack; PAN probe page
    user-stack[0x8000_7000, 0x8000_8000)loader user stack (EL0-RW, NX)
    store-ro[0x8000_9000, 0x8000_a000)G2 read-only store target

    The rest of the 2 MiB window is not allowed. A larger guest image needs a new ADR, not a silent widening.

  5. Three views:

    • Live: armed in the loader probe and taken on hello-libctos’s first SYS_YIELD, while the app stands at EL0 with every user page mapped.
    • Steady: after the boot probes. Nothing may be EL0-reachable.
    • Negative: four plants (never accessed; removed, then TLBI VAAE1). Each must be caught in the expected roots under the expected rule, and a clean walk must follow:
      • kernel-pa: EL0-RO mapping of a kernel .data page at the allowlisted 0x8000_7000
      • va: pool frame at off-allowlist 0x8019_0000
      • ttbr1: pool frame at TTBR1_BASE + 1 MiB
      • el0-wx: EL0 RW+X pool frame at 0x8000_2000
  6. Leak build (--features reach-leak-probe): a persistent EL0-RO mapping of kernel .data at 0x8000_7000 before the steady walk. It must print el0-reach: leak k|u … why=kernel-pa and withhold el0-reach: ok.

G2 — EL0 write permission fault (el0::el0_write_faults)

  1. EL0 runs MOVZ X1, #magic; STR X1, [X0]; BRK #0 with X0 = target. The store must take a lower-EL data abort with:

    • EC = 0x24
    • WnR = 1
    • DFSC = permission fault, level 1, 2 or 3
    • FAR = target
    • ELR = the STR
    • the target unchanged afterwards

    Reaching the BRK means the store completed, and the probe prints el0: write-succeeded. Any other exception prints el0: write-bad. Either one withholds el0: write-ok, and the smoke rejects both.

  2. Targets:

    • (a) kernel .data through its TTBR1 alias: mapped EL1-RW / EL0-none, and TTBR1 walks stay live while EL0 runs.
    • (b) an EL0-read-only user page at 0x8000_9000, filled through its TTBR1 RAM alias because PAN blocks EL1 access through the EL0 mapping.
    • _start is deliberately not a target.
  3. Negative build (--features write-leak-probe): target (b) is mapped EL0-RW, so the store completes. The probe must print el0: write-succeeded user-ro and withhold el0: write-ok. The smoke builds both leak features in one kernel (target/el0-leak-probe).

Evidence (agent box, 2026-09-27 00:20–00:23 CEST; QEMU 10.0.13 Debian, virt -cpu cortex-a76 TCG, rustc 1.100.0-nightly 0fc141305): qemu-smoke: ok

el0: write-fault kernel va=0xffffff8040201008 esr=0x9200004f far=0xffffff8040201008 ec=0x24 wnr=1 dfsc=perm-l3
el0: write-fault user-ro va=0x80009000 esr=0x9200004f far=0x80009000 ec=0x24 wnr=1 dfsc=perm-l3
el0: write-ok kernel,user-ro
el0-reach: range u lo=0x80000000 hi=0x80001000 pa=0x4025d000 mem el0-ro el0-nx
el0-reach: range u lo=0x80002000 hi=0x80003000 pa=0x4026e000 mem el0-none el0-x
el0-reach: range u lo=0x80007000 hi=0x80008000 pa=0x4026f000 mem el0-rw el0-nx
el0-reach: live k=3 u=3 a=0 h=0 pages=6 leaks=0
el0-reach: steady k=0 u=0 a=0 h=0 pages=0 leaks=0
el0-reach: neg kernel-pa caught va=0x80007000 roots=k,u why=kernel-pa
el0-reach: neg va caught va=0x80190000 roots=k,u why=va
el0-reach: neg ttbr1 caught va=0xffffff8000100000 roots=h why=ttbr1
el0-reach: neg el0-wx caught va=0x80002000 roots=k,u why=el0-wx
el0-reach: neg clean
el0-reach: ok allow=app-hdr[0x80000000,0x80001000),app-text[0x80002000,0x80003000),crt-stack-pan[0x80003000,0x80004000),user-stack[0x80007000,0x80008000),store-ro[0x80009000,0x8000a000)

(The live walk also prints the same three range k … lines; k and u share the map-window L3.) Leak build: el0: write-succeeded user-ro va=0x80009000 esr=0xf2000000 (the BRK), el0-reach: leak-probe planted va=0x80007000, el0-reach: leak k|u lo=0x80007000 hi=0x80008000 pa=0x40204000 mem el0-ro el0-nx why=kernel-pa, el0-reach: steady k=1 u=1 a=0 h=0 pages=2 leaks=2 → qemu-smoke: el0-leak-probe caught (fail-closed ok). cargo test: new cases el0_store_to_kernel_and_ro_page_faults and el0_reach_rest_clean_and_plants_caught pass (146 [ok]). CI: see the ledger row for this PR.

Surprises / findings

  1. Nothing is EL0-reachable at rest. Every probe and loader run unmaps its user pages, so a steady-state walk alone would never exercise the allowlist. Hence the live walk from inside a standing app.
  2. Only one existing read probe was a permission fault. el0: no kernel read targets the torn identity alias of kernel .data, so it is a translation fault. ttbr1: no el0 (an EL0 load of the TTBR1 private page) was already a permission-class check on a mapped kernel page. G2(a) is the first store on a mapped kernel page.
  3. ADR-091’s code-reading note was incomplete. It said only l3_page_el0_rw / l3_page_el0_ro set EL0 access and missed l3_page_el0_exec (AP = 00, UXN clear: EL0 execute-only). The walk covers both AP[1] and UXN, so the note is superseded.
  4. App rodata sits on an EL0 execute-only page. The loader unions hello’s R segment into the RX page, so EL0 cannot load its own rodata directly. The apps still work because the kernel reads their strings in syscalls (G4).
  5. a = 0 and h = 0: ASID-B’s window pages are EL1-only nG probe pages. The kernel TTBR0 carries the same EL0 pages as user TTBR0 (shared window L3), and EL0 only ever runs on user TTBR0.

New gap G4 (Documented by code reading; not probed, not fixed)

syscall::user_range_ok_max (used by copy_user / copy_to_user / path_from_user) accepts a user pointer if paging::user_mapped(page) && paging::is_mapped(page). user_mapped accepts any valid leaf in user TTBR0, whatever its AP bits. The copy then clears PSTATE.PAN. Consequences:

  • SYS_UART_WRITE(0x4008_0000, n) from EL0 would copy bytes of the EL1-only _start stub page to the UART.
  • Any EL1-only map-window page that exists while an app runs would be readable the same way, because the window L3 is shared by kernel and user TTBR0.
  • copy_to_user could write an EL0 execute-only page (the app’s own text).

The direct EL0 access path is closed (G1/G2); the kernel-mediated path is not. The fix is to require EL0 read/write permission on the user leaf (or use AT S1E0R/W). It needs the loader to map app rodata EL0-readable (AP = 11 with UXN clear, i.e. EL0 R+X), which changes loader and trampoline permissions, so it is out of this milestone’s scope. ADR-091 lists G4 for the sponsor.

Honesty

Say: “ADR-092: a fail-closed walk of kernel, user and ASID-B TTBR0 and TTBR1 finds EL0-reachable pages only in five allowlisted user slots backed by non-kernel frames (live: 3 per shared window root while an app stands; at rest: none), and four planted leaks are caught; EL0 stores to kernel data and to an EL0-RO page take permission faults (ESR/FAR checked). Syscall copy helpers do not yet check EL0 permission (G4).” Do not say “EL0 isolated”, “EL0 cannot obtain kernel bytes by any path”, or “exhaustively verified” beyond these four roots on this machine.

Consequences

  • Code:
    • src/paging.rs: el0_reach, EL0_ALLOW, plant helpers, EL0_STORE_RO_VA.
    • src/reach.rs (new).
    • src/exception.rs: store-probe capture.
    • src/el0.rs: G2.
    • src/syscall.rs / src/loader.rs: live-walk hook.
    • src/main.rs.
    • Cargo.toml: reach-leak-probe, write-leak-probe.
    • scripts/qemu-smoke.sh: required markers, rejects, el0-leak build.
  • Docs: ADR-091 revision 2 (G1/G2 closed, G4 added, sentence restated), ADR-055 §A rows, ledger, roadmap, threat model v1.67, el0.md, SUMMARY, MOC, daily brief, session memory.

ADR-093 — Public-identifier scrub + fail-closed guard

  • Status: Accepted (security / compliance guardrail). Not a kernel change. No src/ change.
  • Date: 2026-09-27
  • Trigger: sponsor-approved task relayed by DSO (2026-09-27): remove the main AWS account ID and other identifying data from the current tree of this public repo; add a cheap fail-closed guard if it fits.
  • Base: branched from main (c1d98c7), not stacked on the open EL0 stack (#139–#144).
  • Numbering: ADR-087–092 are taken by the open EL0 stack; this ADR takes 093 to avoid a collision on rebase.

Context

This repository is public. A read-only scan found a cloud account identifier and related cloud resource identifiers written into docs, research notes and one Route 53 recovery batch file. Most came from the 2026-09-11 docs/DNS session and the 2026-09-26 B2-P paid spike (ADR-085). A second scan found a local OS user name inside host paths and a WSL command line. No workflow, script logic or kernel code uses any of these values.

Decision

  1. Replace, don’t delete. Each value becomes a typed placeholder so the prose keeps its meaning:

    • prose / tables: <aws-account-id>, <route53-zone-id>, <instance-id-run0> … <instance-id-run4>, <spot-request-id-run0> … <spot-request-id-run4>, <security-group-id>, <iam-user>, <user> (local OS user in paths). Run indices keep the ADR-085 tables cross-referenced.
    • copy-paste shell snippets: "${AWS_ACCOUNT_ID:?set AWS_ACCOUNT_ID locally}" and "${ROUTE53_ZONE_ID:?set ROUTE53_ZONE_ID locally}". The real values stay in the operator’s local config. Nothing in CI needs them, so no new secret or variable is needed.
  2. Counts (types only, no values). Current tree on main:

    Identifier typeFilesOccurrencesWhere
    AWS account ID11144 docs (incl. README), 6 research notes, 1 script data file (Route 53 JSON comment); 0 workflows
    Route 53 hosted-zone ID12175 docs, 6 research notes, 1 script data file
    EC2 instance IDs (5 distinct)4272 docs, 2 research notes
    Spot request IDs (5 distinct)151 research console log
    Security-group ID121 research note
    IAM user name111 ADR
    Local OS user in paths / WSL -u463 docs, 1 research note

    17 content files changed in total (several carry more than one type).

  3. Fail-closed guard. scripts/check-public-ids.sh scans every git-tracked text file for identifier shapes: 12-digit account IDs next to an account/owner word, in an ARN or in an ECR host; AWS access key IDs; EC2/VPC resource IDs; Spot request IDs; Route 53 hosted-zone IDs (after a zone word, or bare Z + digit form). It holds generic patterns only, never a real value. Its allowlist holds public AWS-documentation example values only. On a hit it prints file:line class and not the matched text, because CI logs are public. --self-test plants synthetic tokens of every class in a throwaway repo (built at runtime) and fails if any class is missed or if an allowlisted example or a context-free number is flagged.

  4. Wired into qemu-smoke.sh as its first step (self-test, then scan). That script already runs in both default smoke.yml jobs and in Docker smoke, so the guard is CI-gated without touching .github/workflows/. Marker lines: public-ids: self-test ok classes=7, public-ids: ok files=N hits=0.

  5. History is not rewritten. The replaced values remain in earlier commits and in already-merged PR diffs. Rewriting history or force-pushing needs a separate sponsor decision after a risk evaluation. This ADR does not decide that.

Evidence

  • Guard on this branch: public-ids: self-test ok classes=7, public-ids: ok files=… hits=0 (bash and dash).
  • Guard on unmodified main (script copied into a scratch worktree): FAIL hits=77 (raw matches). Distinct file:line class pairs: account-id-ctx 12, route53-zone-id 16, ec2-resource-id 26, spot-request-id 5. Every line that carried one of the values above was flagged, with no extra lines.
  • CI: see the honesty ledger row for this PR.

Limits (non-claims)

  • A bare 12-digit number with no account/ARN/ECR context is not flagged. GitHub Actions job IDs have the same shape and appear throughout the ledger. The guard is a ratchet for the common shapes, not a DLP product.
  • It does not scan git history, PR descriptions, issue comments or CI logs.
  • Public, intended identifiers stay: the GitHub owner name in repo URLs, the public docs domain, git author metadata, public AWS region / instance-type names, GitHub run/job IDs, QEMU user-net addresses (10.0.2.x) and loopback.
  • Not a “secure” or “compliant” claim.

Consequences

  • The open EL0 stack PRs (#139–#144) must pass this guard after they rebase on a main that contains it.
  • New cloud work must write placeholders or env vars, never literal account or resource IDs.

ADR-094 — G4: syscall user pointers require EL0 permission on every page

  • Status: Accepted (implementation + evidence). Closes ADR-091 gap G4 with fail-closed probes that CI gates. The umbrella “EL0 isolated” stays Planned / non-claim. ADR-091 stays Draft; B3 (the sponsor’s written accept) is not Met.
  • Follow-on (2026-09-28, M6): B3 met: the sponsor accepted the ADR-091 revision-3 sentence and scope in writing (2026-09-28 16:41 CEST). ADR-091 is Accepted. “EL0 isolated” is Verified only as that sentence, with its limits. Text in this ADR that says the umbrella is Planned / non-claim is as of this ADR’s date.
  • Date: 2026-09-27
  • Decision path: the sponsor did not pick among ADR-091 revision-2 options (a)/(b)/(c). The DSO applied the standing order “close gaps before bringing the sentence back” (relay 2026-09-27 ~12:06 CEST), i.e. option (b): fix G4 first.
  • Base: stacked on PR #144 (ADR-092). ADR-093 is the public-identifier guard (PR #145).
  • _start (0x4008_0000) untouched. The stub page is only the target of a refused read. No code maps, writes or changes it.

Context

ADR-092 found G4 by code reading. syscall::user_range_ok_max accepted any page with a valid leaf in the user TTBR0. copy_user then cleared PAN and loaded the bytes. EL0 could therefore ask the kernel to do what EL0 itself cannot:

  • Read: SYS_UART_WRITE could copy the EL1-only _start stub page, or an EL1-only map-window page present while the app runs, to the UART.
  • Write: copy_to_user (SYS_FS_READ, SYS_NET_MAC) could write EL0’s own execute-only text.

That is a confused-deputy path around the page-table boundary. So ADR-091 revision 2 limited its sentence to direct EL0 access.

Decision

1. The check (src/syscall.rs)

  • user_range_ok_max(ptr, len, max, access) gains an access kind:
    • UAccess::Read: the kernel copies from EL0 (copy_user_max: uart_write, fs_write, and paths for fs_create / fs_open / fs_mkdir).
    • UAccess::Write: the kernel copies to EL0 (copy_to_user, plus the pre-checks in SYS_FS_READ and SYS_NET_MAC).
  • The existing guards stay: length cap, overflow, identity/canonical, and “mapped in user TTBR0 and kernel TTBR0”.
  • New, for every 4 KiB page in the range: paging::user_el0_access(page) reads the user-root leaf’s AP bits.
    • A read needs EL0-readable: AP[1]=1.
    • A write needs EL0-writable: AP[1]=1 and AP[2]=0.
    • Execute-only (AP[2:1]=00, UXN clear), kernel-only and unmapped pages fail.
  • On failure the syscall returns its existing error value, and no byte moves:
    • uart_write / fs_write / net_mac: 0
    • fs_create / fs_open: fd 0
    • fs_mkdir: FS_MKDIR_BAD_PATH
    • fs_read: FS_ERR
  • The map-window L3 is shared by the kernel and user roots. So the leaf the check reads is the leaf the copy uses under kernel TTBR0.
  • A counter, SYSPTR_DENIED, counts refusals made by the permission check itself. The probes require a delta of exactly 1 per call. So a refusal cannot come from some other guard (length, unmapped) by accident.

2. Loader: app rodata EL0-readable (src/loader.rs, src/libctos.rs, src/paging.rs)

  • Apps link .text and .rodata into one R+X PT_LOAD page. Perm::Exec used to map it EL0 execute-only (l3_page_el0_exec, AP=00). The app’s own strings therefore failed the new read rule.
  • Perm::Exec now maps with the new paging::map_el0_text / l3_page_el0_text:
    • AP[2:1]=11: EL0 read-only, EL1 read-only
    • UXN clear, PXN set
    • so EL0 can read and execute the page but not write it.
  • Bytes are written through the frame’s TTBR1 alias, as before. Cache maintenance by the EL0 VA runs inside pan::with_user_access, since the page is EL0-accessible now.
  • The A2 libctos::run_hello path does the same.
  • No new slot, no split page, no app ELF change. The app image and its sha256 are byte-identical, so the A9 cross-update to the prior OS still boots the same ELF.
  • The G1 walk still passes. The page was already EL0-reachable (UXN clear), and it is not EL0 W+X.
  • Kernel-built probe payloads keep execute-only code pages. Their strings and buffers move to a new EL0-RW NX data page, syscall::EL0_DATA_VA (window slot 3, the allowlisted crt-stack-pan slot).

3. Probes (syscall::observe_sys_ptr_probe, called from kernel_main_high after svc: ok)

LineWhat EL0 asksMust be
el0: sys-ptr control rw-data n=6SYS_NET_MAC into the EL0-RW data page6 bytes land, equal to the guest MAC (positive control: the NIC works, so a later 0 is the check)
el0: sys-ptr denied kernel-stubSYS_UART_WRITE(0x40080000, 8): the _start stub pagereturns 0, nothing printed, 1 check refusal
el0: sys-ptr denied kernel-windowSYS_UART_WRITE of a kernel-only map-window page (map_page, EL1-RW, EL0-none, slot 11)returns 0, nothing printed, 1 check refusal
el0: sys-ptr denied xo-textSYS_NET_MAC writing its own execute-only code page (+0x800, sentinel)returns 0, sentinel unchanged, 1 check refusal
el0: sys-ptr denied write-roSYS_NET_MAC writing an EL0-RO page (the store-ro slot, sentinel set via alias)returns 0, sentinel unchanged, 1 check refusal
el0: sys-ptr denied straddleSYS_UART_WRITE(data+4092, 8): EL0-RW page, then a kernel-only pagereturns 0, nothing printed, 1 check refusal (the second page is checked)
el0: sys-ptr denied sys=<name> nr=<n> {read kernel-stub | write xo-text} ×7every pointer-taking syscall: uart_write 17, fs_create 19, fs_open 20, fs_mkdir 27, fs_write 22 (reads of the stub page); fs_read 21, net_mac 24 (writes of own xo text)each returns its error value, 1 check refusal
el0: sys-ptr sweep syscalls=7 denied=7summaryall 7
el0: sys-ptr okall of the aboveonly then
  • Fail-closed: the smoke requires all 15 lines. It rejects el0: sys-ptr leaked, el0: sys-ptr bad, el0: sys-ptr skip and el0: sys-ptr probe missed.

  • Leak-probe build: the sysptr-leak-probe feature (built together with reach-leak-probe,write-leak-probe into target/el0-leak-probe) makes the permission check always pass, keeping only the old guards. That kernel must print:

    • el0: sys-ptr leaked kernel-stub n=8: 8 bytes of the stub page go to the UART
    • el0: sys-ptr leaked kernel-window n=8
    • el0: sys-ptr leaked xo-text n=6 intact=false checks=0: the kernel wrote EL0’s execute-only text
    • el0: sys-ptr skip write-ro: an EL1 store to an AP[2]=1 page would take an EL1 permission fault, so the probe stops there
    • no el0: sys-ptr ok and no el0: sys-ptr denied

    The smoke requires those and fails otherwise.

  • cargo test adds sys_ptr_requires_el0_permission and abi_success_from_el0_data_page.

4. G1 allowlist

Unchanged: five slots, fail-closed (app-hdr, app-text, crt-stack-pan, user-stack, store-ro). No new EL0-reachable slot:

  • The EL0 data page reuses crt-stack-pan.
  • The write-ro target reuses store-ro.
  • The kernel-window (slot 11) and straddle (slot 4) pages are EL0-none, so they are not EL0-reachable.
  • All probe pages are unmapped before the walk’s steady pass.

el0-reach: live k=3 u=3 a=0 h=0 pages=6 leaks=0 and el0-reach: ok allow=… are byte-identical to ADR-092.

Consequences

  • G4 is closed on the default smoke machine: no syscall that takes a user pointer can read a page EL0 cannot read, or write a page EL0 cannot write. Probes cover each such syscall. The ADR-091 draft drops its “direct EL0 memory access only” limit (revision 3). Only G3 (the optional B2-P re-verify) is left.
  • App text is now EL0-readable as well as executable. That is the usual Unix-like model for a shared text+rodata page, and still not writable. An EL0 app can read its own code bytes. That is not a kernel-isolation property.
    • Follow-on (2026-09-28, proposed): ADR-097 moves app .rodata onto the app-hdr page and maps .text EL0 execute-only again, so an EL0 pointer into text is now refused for copies from EL0 (xo: sys-read text denied … checks=1). Merge requires a sponsor rev 4 accept of the ADR-091 PAN clause (no FEAT_EPAN). This ADR’s text is kept as the record of its time. Update 2026-10-02: the sponsor accepted ADR-091 revision 4 and ADR-097 was merged as 8344797; app text is now EL0 execute-only and the limit “EL0 can read its own code bytes” no longer applies.
  • Not covered (stated, not gaps in G4’s scope):
    • TOCTOU between check and copy. Today there is one core, and nothing changes the calling task’s user page tables during a syscall. On SMP this would need AT S1E0R/W or a fault-fixup copy.
    • AT-based checks.
    • Real hardware. LDTR/STTR unprivileged loads/stores would enforce the same rule in hardware; that is a possible later hardening.
  • Still not “EL0 isolated”, not side-channel, not real hardware, not certified.

Honesty

Say: “ADR-094 closes G4: syscall user pointers need EL0 read/write permission on every page, proven by refused kernel-stub / kernel-window reads, refused xo-text / RO writes, a straddle case, a seven-syscall sweep, and a leak-probe build that must fail.” Do not say “EL0 isolated”, “secure” or “B3 met”.

ADR-095 — G3: B2-P KVM taken-SError re-run on current main (AWS Graviton3 c7g.metal) — PASS

  • Status: Accepted (evidence record, docs only). The sponsor asked for G3, the optional B2-P KVM re-run listed as the only remaining limit in ADR-091 revision 3 (DSO relay 2026-09-28). Result: PASS on the first smoke attempt. The taken lower-EL SError still happens under KVM on real Graviton3 hardware on current main, the same way ADR-085 run 4 showed it at 159b178. ADR-091 stays Draft; B3 (the sponsor’s written accept) is not Met; the umbrella “EL0 isolated” stays Planned / non-claim. Nothing is flipped here.
  • Follow-on (2026-09-28, M6): B3 met: the sponsor accepted the ADR-091 revision-3 sentence and scope in writing (2026-09-28 16:41 CEST). ADR-091 is Accepted. “EL0 isolated” is Verified only as that sentence, with its limits. Text in this ADR that says the umbrella is Planned / non-claim is as of this ADR’s date.
  • Date: 2026-09-28 (launch 13:50:32 CEST, smoke PASS before 13:58:13 CEST, teardown verified ≈ 14:06 CEST).
  • Tested commit: main at 8107578a6c715a89c9e5861ebb4102460ba20e80 (merge of PR #146, ADR-094). It contains the whole EL0-isolation stack: ADR-087 … ADR-092 and ADR-094, including the ADR-090 evidence class and the ADR-091 draft.
  • main moved during review (2026-09-28): PR #145 (ADR-093) merged as e1ee4a8 after this run, and this branch merged it. Between the tested 8107578 and e1ee4a8 there is no change to src/, user/, libctos/, Cargo.*, build.rs, linker.ld, the target JSON, rust-toolchain.toml, research/qemu-nmi/ or the three B2 scripts (empty git diff), so the guest and QEMU tested here are byte-identical in source to the current tip. ADR-093 changed only docs, scripts/check-public-ids.sh and a guard hook in scripts/qemu-smoke.sh.
  • No src/, scripts/ or .github/workflows/ change. The in-tree scripts/b2-metal-userdata.sh is unchanged; the run used a copy with the small delta listed below. _start (0x4008_0000) untouched. CloudAgent not used. Do not self-merge (ADR-002).
  • Public-identifier rule (ADR-093, PR #145): no account, instance, Spot request, security group, volume, network, image or user identifiers appear in this record. They are described in words.

Context

ADR-085 run 4 (2026-09-26) is the B2-P leg of the ADR-090 taken-SError evidence class. It ran at 159b178, before the M1/M2 boot-path changes (ADR-087 RAM-side identity tears, ADR-088 TTBR1 MMIO high alias + identity MMIO tear) and before ADR-092 / ADR-094. ADR-090 therefore carries the caveat “B2-P is historical”, and ADR-091 revision 3 lists G3: B2-P re-verify on the post-M2 boot path (≈ $0.2 paid run) as its only remaining limit. Since then the b2-serror profile was only checked under stock TCG (ADR-087 / ADR-088 / ADR-090), where it parks as expected.

What was run

Same procedure as ADR-085 run 4, adapted to current main:

ADR-085 run 4ADR-095 G3 (this ADR)
ctos commit159b178 (branch head)8107578 (main, detached checkout of the full SHA)
HostAWS c7g.metal (Graviton3), eu-north-1b, one-time Spot, max price $0.60same
OS imageUbuntu 24.04 arm64Canonical Ubuntu 24.04 LTS arm64 server (gp3), build 20260923
Root volumegp3 30 GB, delete on terminationsame
Accessno SSH, no key pair, no IAM role; security group with no ingress; serial console onlysame; IMDSv2 required
Tags (every resource)project=ctos, component=b2-spikeproject=ctos, component=g3-kvm-recheck on the instance, root volume, network interface, Spot request and security group
Hard stopshutdown -h +80 + shutdown behaviour terminateshutdown -h +60 + shutdown behaviour terminate; the user-data powers off right after the smoke
QEMUv10.0.0 7c949c53 + patches 0001 + 0002 via scripts/build-qemu-b2.shsame script, same pin
Guestcargo build --features b2-serror --target-dir target-b2same
Smokescripts/b2-kvm-smoke.py (fail-closed), CTOS_B2_TIMEOUT=90, CTOS_B2_STALL=15, up to 3 attemptssame

Credentials path: the sanctioned AWS MCP run_script tool with the ctos-scoped IAM user (no other project’s credentials). Each mutating call ran in its own script, and every launch re-checked that no project=ctos instance was pending, running, stopping, stopped or shutting down (ADR-085 lesson: never two metal boxes).

AWS call sequence (eu-north-1):

  1. Read-only: DescribeInstances (guard: 0 active ctos instances), DescribeSpotPriceHistory (c7g.metal eu-north-1b $0.2476/h, unchanged from ADR-085), DescribeImages (latest Canonical Ubuntu 24.04 arm64), DescribeSubnets / DescribeVpcs (default VPC), DescribeSecurityGroups (no leftover ctos group).
  2. CreateSecurityGroup ctos-g3-kvm-recheck in the default VPC, tagged at create. No ingress rule was added (default egress only).
  3. DescribeInstances guard, then one RunInstances: c7g.metal, one-time Spot, user-data below, tags on instance / volume / network interface / Spot request. Launch 13:50:32 CEST.
  4. Read-only polling: GetConsoleOutput (Latest=true) and DescribeInstances.
  5. After the instance self-terminated: DeleteSecurityGroup, then the verification describes in Teardown.

User-data = scripts/b2-metal-userdata.sh with this delta (nothing else changed):

- shutdown -h +80 "ctos b2 hard timer" || true
+ shutdown -h +60 "ctos g3 hard timer" || true
- echo "CTOSB2: uname $(uname -a)"
+ echo "CTOSB2: uname $(uname -srm)"                      # no hostname on the console
+ echo "CTOSB2: kvm-api $(python3 -c "import fcntl,os;print(fcntl.ioctl(os.open('/dev/kvm',os.O_RDWR),0xAE00))" 2>&1)"   # KVM_GET_API_VERSION
- git clone --depth 1 --branch "__BRANCH__" https://github.com/artofdream/ctos.git ctos
+ git clone --branch main https://github.com/artofdream/ctos.git ctos && git -C ctos checkout --detach 8107578a6c715a89c9e5861ebb4102460ba20e80
+ echo "CTOSB2: qemu-version $(tools/qemu-b2/bin/qemu-system-aarch64 --version | head -1)"

Pre-flight (free, agent box, before paying): cargo build --features b2-serror at 8107578, then CTOS_B2_ACCEL=tcg python3 scripts/b2-kvm-smoke.py with stock QEMU 10.0.13. The guest booted through ident: mmio-high … ident: mmio to b2: boot → el0: serror-arm; stock QEMU answered inject-nmi=>error:machine does not provide NMIs, so the guest printed el0: serror-park / b2: not taken and the smoke failed closed, which is the expected stock-TCG result. It showed the profile still boots on current main; it is not SError evidence.

Versions (from the serial console)

  • CPU: Neoverse-V1 / AWS Graviton3 … @ 2.6GHz; CPU: All CPU(s) started at EL2; kvm [1]: VHE mode initialized successfully.
  • Host kernel: Linux 7.0.0-1013-aws aarch64 (Ubuntu 24.04 AWS kernel). KVM is the in-kernel module of that kernel; KVM_GET_API_VERSION = 12; vGIC is GICv3 only (disabling GICv2 emulation), IPA Size Limit: 48 bits.
  • QEMU: QEMU emulator version 10.0.0 (v10.0.0-dirty) (dirty = patches 0001 + 0002 applied), Accelerators supported in QEMU binary: kvm tcg. Guest command: -accel kvm -machine virt,gic-version=3 -cpu host -m 128M ….
  • Rust: nightly-2026-09-12 from rust-toolchain.toml, auto-installed by rustup 1.29.1. Honest note: the CTOSB2: rustc line captured rustup’s “syncing channel updates for nightly-2026-09-12-aarch64-unknown-linux-gnu” message instead of a rustc --version string; the toolchain path in the cargo warning confirms the same pin. cargo rc=0, qemu-build rc=0.

Evidence (serial console, verbatim)

CTOSB2: uname Linux 7.0.0-1013-aws aarch64
CTOSB2: cpu Vendor ID: ARM;BIOS Vendor ID: Amazon EC2;Model name: Neoverse-V1;BIOS Model name: AWS Graviton3 N/A CPU @ 2.6GHz;
CTOSB2: dmesg [    0.026519] CPU: All CPU(s) started at EL2
CTOSB2: dmesg [    0.227753] kvm [1]: VHE mode initialized successfully
CTOSB2: /dev/kvm=present
CTOSB2: kvm-api 12
CTOSB2: head 8107578a6c715a89c9e5861ebb4102460ba20e80
CTOSB2: qemu-version QEMU emulator version 10.0.0 (v10.0.0-dirty)
CTOSB2: qemu build-qemu-b2: accelerators: Accelerators supported in QEMU binary: kvm tcg
CTOSB2: ===== smoke attempt 1 =====
CTOSB2: b2-kvm-smoke: cmd: /root/ctos/tools/qemu-b2/bin/qemu-system-aarch64 -accel kvm -machine virt,gic-version=3 -cpu host -m 128M -display none -serial stdio -monitor none -qmp unix:/tmp/ctos-b2-yyvqz4_3/qmp.sock,server,nowait -kernel target-b2/aarch64-ctos/debug/ctos
CTOSB2: b2: p1 mmu-on
CTOSB2: ident: jump
CTOSB2: ident: mmio-high gic=0xffffff8008000000 uart=0xffffff8009000000 virtio=0xffffff800a000000
CTOSB2: pan: id=2
CTOSB2: pan: present
CTOSB2: pan: enabled
CTOSB2: pan: el1-fault
CTOSB2: ident: ram lo=0x4026b000 hi=0x48000000 pages=32149
CTOSB2: ident: low lo=0x40000000 hi=0x40080000 pages=128 user=0
CTOSB2: ident: tail lo=0x400ca000 hi=0x40201000 pages=311 user=311
CTOSB2: ident: kend lo=0x4025a000 hi=0x4025b000 pages=1 user=0
CTOSB2: ident: mmio lo=0x0 hi=0x40000000 l1=1
CTOSB2: b2: boot (ADR-085 KVM SError profile)
CTOSB2: el0: serror-armb2-kvm-smoke: qmp stop=>ok inject-nmi=>ok cont=>ok (ADR-085: KVM_SET_VCPU_EVENTS serror_pending)
CTOSB2: 
CTOSB2: el0: standing
CTOSB2: el0: serror
CTOSB2: el0: restored
CTOSB2: b2: taken
CTOSB2: b2: doneb2-kvm-smoke: summary armed=True injected=True taken_marker=True b2_taken=True park=False kvm_err=False
CTOSB2: b2-kvm-smoke: PASS taken lower-EL SError observed under accel=kvm ('el0: serror' + 'b2: taken')
CTOSB2: smoke attempt 1 rc=0
CTOSB2: end 2026-09-28T11:58:13Z

(el0: serror-arm + host QMP line and b2: done + summary share console lines because guest serial and host log are tee’d to the same console, as in ADR-085.) Full CTOSB2: excerpt (66 lines, including the early markers b2: e0 … p1 and every ident: line): research/daily-briefs/2026-09-28-adr-095-g3-console.txt.

Reading: the guest ran natively on Graviton3 under KVM through the current boot path: ident: mmio-high (ADR-088 TTBR1 Device alias), ident: low / tail / kend (ADR-087 RAM-side tears), ident: mmio … l1=1 (ADR-088 identity MMIO tear), PAN enable and EL1 fault. On the el0: serror-arm cue QEMU (patch 0002) turned inject-nmi into KVM_SET_VCPU_EVENTS{serror_pending=1}; KVM raised a virtual SError (HCR_EL2.VSE), and the guest took it at the lower-EL SError vector while standing at EL0: the bare el0: serror line is printed only by that handler with the standing-EL0 expectation armed (src/exception.rs). No el0: serror-park, no b2: not taken, fail-closed smoke PASS on attempt 1.

Difference from TCG: Graviton3 reports pan: id=2 (ID_AA64MMFR1_EL1.PAN = 2); PAN enable and the EL1-vs-EL0 fault still pass. ident: ram … pages=32149 vs 32150 in run 4 follows the image growth since 159b178.

Cost

One c7g.metal Spot instance, per-second billing, $0.2476/h (DescribeSpotPriceHistory, eu-north-1b, flat on 2026-09-28). Running 13:50:32 → self-poweroff ≈ 13:58:35 CEST (console end 11:58:13Z + 20 s sleep) ≈ 8.1 min ≈ $0.033. The instance then sat in shutting-down and was terminated by ≈ 14:06 CEST; even billing that whole 15.5-minute window at the Spot rate gives only ≈ $0.064. gp3 30 GB and one public IPv4 for the same window: < $0.01. Total ≈ $0.04 (upper bound ≈ $0.07), under the ≈ $0.2 budget and far under the $20 stop gate.

Teardown

  • The instance powered itself off after the smoke and terminated (Client.InstanceInitiatedShutdown; shutdown behaviour terminate). The root volume was delete-on-termination; the network interface went with the instance; the one-time Spot request closed.
  • The security group was deleted after the instance reached terminated.
  • Verification (≈ 14:06 CEST, read-only describes, eu-north-1): project=ctos instances pending / running / stopping / stopped / shutting-down → 0; the run’s instance → terminated; project=ctos volumes → 0, and the run’s root volume by ID → InvalidVolume.NotFound; tagged network interfaces → 0; project=ctos security groups → 0, and the run’s group by ID → InvalidGroup.NotFound; tagged key pairs → 0; tagged snapshots → 0; tagged Elastic IPs → 0; the one component=g3-kvm-recheck Spot request → closed / instance-terminated-by-user (a closed one-time request is not billable and ages out by itself).
  • No key pair, IAM role, instance profile, snapshot, AMI or Elastic IP was created.

Decision

  1. G3: done, PASS. The B2-P leg of the ADR-090 evidence class is re-verified on current main (8107578) under KVM on real Graviton3 hardware, same host class, same QEMU pin and patches, same fail-closed smoke. ADR-091 gap G3 is closed with a pointer here.
  2. ADR-090 §1 is unchanged. It is the sponsor’s verbatim D1 paragraph and still names ADR-085 run 4. This ADR is a supplementary re-run of that leg on the post-M2 boot path. The ADR-090 caveat “B2-P is historical (at 159b178)” is answered by this record; the caveats “reduced profile” and “not real hardware running the full default image” still stand.
  3. ADR-091 sentence: no change needed. The revision-3 sentence says taken lower-EL SError “is evidenced by the ADR-090 evidence class” and that the sentence is “not a real-hardware claim”. Both stay true and complete. G3 was a limit on the evidence, not on the wording.
  4. Not changed: ADR-091 stays Draft; B3 not Met; umbrella Planned / non-claim; no CI job runs KVM; the default smoke on stock QEMU still requires el0: serror-park; the B1 pin is unchanged.

What this does not show

  • Reduced profile only. b2-serror skips GIC / timer / virtio / FAT, and it does not run the identity inventory (ident: inv), the EL0-reachability walk (ADR-092), the EL0 write-fault probes or the syscall-pointer probes (ADR-094). Those are CI-gated on TCG only.
  • One-off, not CI. Re-running needs another paid metal run or a self-hosted arm64 KVM runner (B2-S, not registered).
  • No host-side strace of the ioctl (same limitation as ADR-085); the mechanism is QEMU’s kvm_put_vcpu_events path and the fail-closed marker was observed.
  • Not “the default ctos image runs on real hardware”, not “EL0 isolated”, not “secure OS”.

Honesty

Say: “ADR-095 (G3): on 2026-09-28 an AWS Graviton3 c7g.metal (eu-north-1b, Spot) re-ran the ADR-085 B2-P procedure on ctos main 8107578; QEMU -accel kvm with patch 0002 set serror_pending via KVM_SET_VCPU_EVENTS on the el0: serror-arm cue, and the b2-serror profile took the lower-EL SError while standing at EL0 (el0: serror / b2: taken, fail-closed smoke PASS, attempt 1). One-off paid run, not CI, reduced profile. ADR-091 G3 is closed; ADR-091 is still a draft.” Do not say:

  • taken SError is Verified in CI under KVM, on stock QEMU, or on free runners
  • the default image boots on real hardware
  • “EL0 isolated”, “B3 met”, or “ADR-091 accepted”

Consequences

  • Docs: this ADR; ADR-091 gap G3 marked done (pointer here), sentence untouched, status untouched; honesty-ledger rows; threat model v1.69 note; SUMMARY; research MOC; console excerpt research/daily-briefs/2026-09-28-adr-095-g3-console.txt.
  • Code / scripts / CI: unchanged. CI on PR #147: honesty-ledger row “CI on GitHub (ADR-095 / PR #147)”.
  • AWS: everything created for the run is deleted and verified (above).

ADR-096 — Task-to-task EL0 isolation probe (task A cannot read or write task B’s page)

  • Status: Accepted (implementation + evidence). Adds a fail-closed probe that CI gates: an EL0 task on one TTBR0/ASID cannot read or write another EL0 task’s private page at the same VA, including right after the other task ran. This is additional evidence recorded next to the accepted ADR-091 sentence. It does not change that sentence or its scope. Task-vs-task isolation stays in ADR-091’s non-scope list. Any wording that adds it (a possible revision 4) goes to the sponsor in writing first.
  • Date: 2026-09-28
  • Decision path: sponsor-approved milestone (DSO relay, 2026-09-28 21:21 CEST): “task-to-task isolation probe”.
  • Base: main at 9a1c2bb (M6 flip, PR #148).
  • _start (0x4008_0000) untouched. No probe maps, reads or writes it.

Context

ADR-091’s non-scope says: “EL0-to-EL0 (task-vs-task) isolation beyond the asid: ok TLB mile and per-task TTBR0. No probe that one EL0 task cannot read another task’s pages.” The asid: ok mile (ADR-013) runs at EL1: it switches TTBR0 between the kernel L1 tagged ASID 1 and the ASID-B L1 tagged ASID 2 and loads through each. No EL0 code runs in it.

The tree already has a second address space: the ASID-B root (L1_ASID_B → L2_ASID_B → its own map-window L3_ASID_B, ASID 2). Every loaded app uses the user root (L1_USER, ASID 1), which shares the kernel’s map-window L3.

Decision

1. Two EL0 tasks on two roots (src/t2t.rs)

Task ATask B
TTBR0user root L1_USER, ASID 1 (paging::user_ttbr0())ASID-B root L1_ASID_B, ASID 2 (paging::asid_b_ttbr0())
CodeEL0 execute-only page at 0x80002000 (app-text slot), shared window L3own frame, EL0 execute-only, nG, at 0x80002000 in L3_ASID_B
Private dataround 1: nothing at 0x80007000; round 2: own frame, EL0-RW NX, nGown frame, EL0-RW NX, nG, at 0x80007000 (user-stack slot)
  • Both tasks run at EL0 through a masked ERET. B uses the new exception::eret_to_el0_masked_ttbr (an explicit TTBR0 value, ASID included); A uses the existing path. The existing eret_to_el0_* helpers now call the same asm with paging::user_ttbr0(), so their behaviour does not change.
  • A one-shot capture (exception::arm_el0_capture / el0_capture_result) records ESR, FAR, ELR and x1 of the task’s first lower-EL sync exception (a data abort, or the BRK #0 that ends the payload) and returns to EL1.
  • Payloads are kernel-built: LDR X1, [X0] (read) or MOVZ/MOVK X1, #val; STR X1, [X0]; LDR X1, [X0] (write), each ending in BRK #0, with X0 = 0x80007000.
  • Frames are written and checked through their TTBR1 alias (PAN blocks EL1 access through the EL0 VA).
  • Sentinels: B 0x7a5cb0b0; A 0x7a5ca0a0, then 0x7a5ca2a2; A’s attempted store into B’s page 0x7a5ceeee.

2. Probe steps (called from kernel_main_high after asid: ok)

Round 1 (A has no mapping at the VA):

  1. Walk: walk every valid leaf of A’s root and B’s root for B’s frame. A’s root must have no leaf at all that outputs to B’s frame; B’s root must have exactly one EL0-reachable leaf for it (control). Marker: t2t: walk b-frame u-refs=0 u-el0=0 a-el0=1.
  2. B runs (ASID 2): stores its sentinel, reloads it (BRK, x1 = sentinel, frame = sentinel). Marker: t2t: b-own ok asid=2 va=0x80007000 val=0x7a5cb0b0.
  3. A reads (ASID 1) the same VA right after B, with no TLBI of 0x80007000 in between (A’s code-page map/unmap invalidates only 0x80002000). A must take a lower-EL translation fault: EC 0x24, WnR 0, FAR = the VA, ELR = the LDR, x1 still 0. Marker: t2t: a-read fault asid=1 va=0x80007000 esr=0x92000007 far=0x80007000 ec=0x24 wnr=0 dfsc=trans-l3 got=0x0 after-b no-tlbi.
  4. A writes the same VA: translation fault with WnR 1, FAR = the VA, ELR = the STR; B’s frame still holds B’s sentinel. Marker: t2t: a-write fault asid=1 va=0x80007000 esr=0x92000047 far=0x80007000 ec=0x24 wnr=1 dfsc=trans-l3 b-intact=true.

Round 2 (same VA, A gets its own frame):

  1. Walk: A’s frame is EL0-reachable only in A’s root, B’s frame only in B’s root. Marker: t2t: walk same-va a-frame u-el0=1 a-el0=0 b-frame u-el0=0 a-el0=1.
  2. B runs again first. Then A reads the VA and must see its own sentinel, never B’s: t2t: same-va a-sees-own asid=1 got=0x7a5ca0a0 b=0x7a5cb0b0.
  3. A writes the VA: the value lands in A’s frame only; B’s frame is unchanged. t2t: same-va a-write-own got=0x7a5ca2a2 b-intact=true.
  4. B reads the VA after A’s trips and sees its own value; A’s frame is unchanged. t2t: same-va b-sees-own asid=2 got=0x7a5cb0b0 a-intact=true.
  5. Summary: t2t: ok asid-a=1 asid-b=2 va=0x80007000 read,write,same-va.

Setup checks before step 1: the two TTBR0 values carry ASIDs 1 and 2 and different table bases; both window slots are empty in both roots; B’s leaf has nG. All probe pages are unmapped (and TLBI VAAE1, all ASIDs) before the ADR-092 steady walk. Both VAs are ADR-092 allowlisted slots, so the G1 allowlist does not grow; the steady walk still reports pages=0 leaks=0.

3. Fail-closed wiring (scripts/qemu-smoke.sh)

  • The default smoke requires all nine t2t: lines above (regex in the script: ESR low bits and table level may vary within “translation fault”). It rejects t2t: leak, t2t: bad and t2t: probe missed, and any task-A line that shows B’s sentinel (got=0x7a5cb0b0).
  • Leak-probe build: new feature t2t-leak-probe (built together with reach-leak-probe,write-leak-probe,sysptr-leak-probe into target/el0-leak-probe) maps B’s frame into A’s root at the same VA (a shared-frame bug). That kernel must print t2t: leak-probe planted b-frame in a-ttbr0 va=0x80007000, t2t: leak walk b-frame u-refs=1 u-el0=1, t2t: leak read asid=1 va=0x80007000 got=0x7a5cb0b0 and t2t: leak write asid=1 va=0x80007000 b=0x7a5ceeee, and must not print t2t: ok. The smoke requires this.
  • cargo test adds el0_task_cannot_read_or_write_other_task.
  • The three required checks run scripts/qemu-smoke.sh, so they enforce all of the above.
  • Checker negative test (local, 2026-09-28): the ADR-096 check block from scripts/qemu-smoke.sh was run against a passing serial log (it passed) and against five tampered logs: t2t: ok deleted, B’s sentinel on the task-A same-VA line, got= on the read-fault line changed to B’s sentinel, t2t: probe missed appended, and the real leak-build serial. It exited 1 on each, naming the missing or forbidden line. A sixth log added an extra task-A line with B’s sentinel while keeping all required lines; it was rejected by the sentinel guard alone.

What this proves

On the default smoke machine (QEMU virt, -cpu cortex-a76, TCG, one core): an EL0 task on the user root (ASID 1) cannot read or write the private page of an EL0 task on the ASID-B root (ASID 2) at the same VA. This holds when A has nothing mapped there (architectural translation fault, B’s data never observed, B’s frame unchanged), and when A has its own page there (each task sees and changes only its own frame). A’s root has no translation of any kind to B’s frame. The kernel does not issue a TLB invalidate of that VA between B’s trip and A’s trip.

What this does not prove

  • Not part of the ADR-091 sentence. The accepted sentence is unchanged; task-vs-task stays in its non-scope. A revision 4 would need the sponsor’s written accept.
  • Stale-TLB angle on hardware. QEMU TCG flushes its own software TLB whenever a TTBR0 write changes the ASID (vmsa_ttbr_write in QEMU target/arm/helper.c). So “no TLBI after B” shows only that the kernel does not rely on a flush it issues itself. It says nothing about how a hardware ASID-tagged TLB would behave. The asid: ok mile has the same limit.
  • Two loaded apps. Both tasks are kernel-built payloads. There is no scheduler that switches between two loaded apps; every loaded app still runs on the one user root (ASID 1), one at a time. The probe checks the page-table and ASID mechanism that such a switch would use, not a multi-app scheduler.
  • One core. No SMP, no cross-core TLB shootdown.
  • Shared kernel-mapped pages. The user root shares the kernel’s map-window L3 and maps the kernel image text EL1-only; those are covered by the ADR-092 walk (nothing else is EL0-reachable), not by this probe.
  • Not side-channel, not real hardware, not DMA/IOMMU, not a certification.

Consequences

  • Honesty ledger row (Verified, scoped as above), threat model v1.71, daily brief, SUMMARY and MOC entries.
  • el0.md points to this ADR as evidence next to the ADR-091 claim. ADR-091 itself is not edited: its sentence, scope table and non-scope list stay exactly as the sponsor accepted them.

ADR-097 — EL0 app text execute-only (rodata moved to the app-hdr slot)

Accepted (2026-10-02). The sponsor accepted ADR-091 revision 4 in writing: “I accept ADR-091 rev 4”, given in answer to the exact wording in Revision-4 wording below. Written by the sponsor in the DSO chat, recorded in the DSO vault, relayed by DSO. This PR (#150) was merged as 8344797 after the accept. ADR-091 was then edited to revision 4 (PAN clause and code-bytes limit only) by a docs-only follow-up PR; the record is in ADR-091 (“B3 rev 4”).

Original notice, 2026-09-28 (kept as history; the condition it set was met on 2026-10-02): Merge requires sponsor rev 4 written accept. This change makes one clause of the accepted ADR-091 revision-3 sentence false as literally worded: “PAN is enabled and EL1 faults on EL0 memory”. The execute-only app text page is EL0 memory (EL0 can execute it), and on the default smoke machine (no FEAT_EPAN) an EL1 load of it does not fault with PAN set. ADR-091 is not edited here. Its sentence and scope stay exactly as accepted. A candidate revision-4 wording is proposed below for the sponsor to read. Until the sponsor accepts a revision 4 in writing, this PR must not be merged.

  • Status: Accepted (2026-10-02). The sponsor accepted ADR-091 revision 4 in writing (“I accept ADR-091 rev 4”); merged as 8344797. Status as of 2026-09-28, kept for history: Proposed (implementation + evidence; merge blocked on sponsor rev 4 written accept).
  • Date: 2026-09-28
  • Decision path: sponsor-approved milestone 3 (DSO relay, 2026-09-28): split the app’s read-only data out of the app code page so EL0 app text becomes execute-only; $0 (local QEMU + existing GitHub Actions checks).
  • Also merges PR #151 (ADR-098, isolation/pan-span-fix, based on main) for the PAN fix.
  • Base: stacked on PR #149 (ADR-096, branch isolation/t2t-probe @ e6e755a). It reuses #149’s EL0 capture helpers (arm_el0_capture / eret_to_el0_masked / el0_capture_result) and extends the same scripts/qemu-smoke.sh leak-build block. Merge #149 first.
  • _start (0x4008_0000) untouched.

Context

ADR-094 closed G4 by making every syscall pointer check require EL0 permission. To keep app string literals printable through uart_write, it mapped the app code page (text and rodata, one page at 0x80002000) EL0 read-only + executable and said so openly: “An EL0 app can read its own code bytes.” ADR-091’s honesty section carries that as a limit.

Decision

Layout (every app, same five slots)

Slot (label unchanged)VAContentsStage-1 leaf
app-hdr0x80000000ELF headers + .rodata (all string literals and constants)AP=11 (EL1 RO, EL0 RO), UXN=1, PXN=1 (unchanged: map_el0_ro)
app-text0x80002000.text onlyAP=10 (EL1 RO, EL0 none), UXN=0, PXN=1 (new map_el0_xo); was AP=11
crt-stack-pan, user-stack, store-rounchangedunchangedunchanged
  • All eight user/*/linker.ld place .rodata at 0x80000000 + SIZEOF_HEADERS and start .text alone at 0x80002000. Each app ELF now has exactly two PT_LOADs: R at 0x80000000 and R E at 0x80002000. The asm fallback hello.S moved its strings to .rodata.
  • No sixth slot. The allowlist, its labels and the at-rest / live inventory lines are byte-identical: el0-reach: live k=3 u=3 a=0 h=0 pages=6 leaks=0 and el0-reach: ok allow=app-hdr[0x80000000,0x80001000),app-text[0x80002000,0x80003000),crt-stack-pan[…),user-stack[…),store-ro[…).
  • build.rs enforces it at build time: loads below the text page must be read-only and inside the 0x80000000 page; each app’s marker strings must be in that page and absent from the text segment.
  • src/loader.rs enforces it at load time (host unit tests + boot): Ro may only map at 0x80000000, Exec only at 0x80002000, Rw never; a page that would combine Exec + Ro is rejected (ParseError::RoOnText, was a union to EL0 RX); anything else is ParseError::OffSlot. Exec pages map through map_el0_xo.
  • The A2 libctos path maps the hello ro page at the map window with map_el0_ro and its text with map_el0_xo.

EL0 reachability (inventory unchanged, XO page still counted)

The ADR-092 walk already defines EL0-reachable as AP[1] set or UXN clear, so the XO page (AP[1]=0, UXN=0) is counted and must match the app-text allowlist entry. paging::leaf_view reports reachable from the same rule, and a new probe plants an XO leaf (EL0 none, UXN=0) at an unallowlisted VA (0x80190000) and requires the walk to catch it (why=va ap-el0=0 uxn=0). So the inventory cannot pass by only looking at AP.

Syscall pointers (ADR-094, strengthened)

user_range_ok_max(…, Read) already requires AP[1] on every page. A pointer into the XO text now fails it: uart_write(0x80002000, 8) from EL0 returns 0 with one permission-check denial, while uart_write of the rodata message in app-hdr still prints.

PAN / EPAN facts (probed, not assumed)

  • ID_AA64MMFR1_EL1.PAN = 2 on QEMU 10.0.13 -cpu cortex-a76 (FEAT_PAN2, no FEAT_EPAN) on the box and in every CI smoke job (pan: id=2).
  • Without FEAT_EPAN (and SCTLR_EL1.EPAN), PAN only blocks EL1 data access to pages that are EL0 data-accessible (AP[1]=1). The XO page has AP[1]=0, so PAN does not apply. Probe: with PAN set (MRS PAN=1, and a control load of the EL0-RO app-hdr page faulting with SPSR.PAN=1 after EL0 trips), an EL1 load of the XO text VA does not fault and returns the code word (xo: el1-pan text va=0x80002000 fault=no match=true mmfr1-pan=2 epan=no). An EL1 store to it faults, but because AP[2]=1 (read-only), not because of PAN (xo: el1-write text … fault=yes intact=true).
  • Consequence for ADR-091: the clause “PAN is enabled and EL1 faults on EL0 memory” is not literally true for the XO app text page. Before this change the only standing EL0 pages were AP[1]=1 pages, so PAN covered all of them (kernel-built probe payload pages mapped EL0-execute-only via map_el0_exec are transient and already had the same property). This is why merge needs a rev 4 accept.

Pre-existing PAN gap: split out to ADR-098

While probing the above, this work found that PAN was not set while the kernel handled EL0 exceptions (SCTLR_EL1.SPAN was 1) or after it returned from an EL0 trip (return_from_el0 resumed with SPSR.PAN clear). ADR-080’s boot probe runs before any EL0 trip, so it never saw this. Because that made the accepted ADR-091 clause “PAN is enabled” false on main while handling EL0 exceptions, the fix was split out (DSO, 2026-09-28 23:54 CEST) into ADR-098 (PR #151, based on main, no rev 4 needed). This branch merges that branch (no force-push) and no longer carries its own copy of the fix: SPAN=0, SPSR.PAN on return, the quiet EL1 fault catcher, pan::pstate_pan / pan::sctlr_span and the pan-keep-leak-probe feature all come from ADR-098. The xo: pan live-syscall … line is kept; it now echoes the reading taken by ADR-098’s pan::on_live_syscall in the same handler (no second implementation). The xo: el1-pan control … after-el0-trips line stays as the xo image probe’s own calibration of that reading.

Probes and markers (all fail-closed in scripts/qemu-smoke.sh)

Live (real hello-libctos, inside its SYS_YIELD handler, called from the ADR-092 live walk):

  • xo: pan live-syscall pstate-pan=1 sctlr-span=0
  • xo: live app=hello text va=0x80002000 ap=2 uxn=0 pxn=1 el0=x reach=1 hdr va=0x80000000 ap=3 uxn=1 el0=r rodata=0x80000120 in-text=0 sys-read text=deny rodata=allow (the app’s own libctos: hi string is found in the hdr frame and not in the text frame; the ADR-094 read check denies the text VA and allows the rodata VA)

Image probe (an app-shaped ELF built in-kernel, loaded by the real loader, run at EL0 on the user TTBR0):

  • xo: layout loader app-hdr va=0x80000000 ap=3 uxn=1 el0=r app-text va=0x80002000 ap=2 uxn=0 pxn=1 el0=x reach=uxn — the inventory reports the new layout
  • xo: el0-exec ok entry=0x80002000 el0-rodata va=0x80000800 got=0x584f524441544131 — EL0 executes its text and loads a rodata sentinel from its new location
  • xo: el0-read-text fault va=0x80002000 esr=0x9200000f far=0x80002000 ec=0x24 wnr=0 dfsc=perm-l3 got=0x0 — an EL0 load of its own text takes a lower-EL permission fault
  • xo: el0-uart rodata-str ok — printed by EL0 through uart_write from a rodata string
  • xo: sys-read rodata ok va=0x80000810 n=27
  • xo: sys-read text denied sys=uart_write nr=17 va=0x80002000 n=0 checks=1 — a syscall given a text pointer is refused
  • xo: el1-pan control app-hdr va=0x80000800 mrs-pan=1 fault=yes spsr-pan=1 after-el0-trips
  • xo: el1-pan text va=0x80002000 fault=no match=true mmfr1-pan=2 epan=no — fact, not a pass condition in the security sense; the smoke pins it so a CPU-model change forces a re-read
  • xo: el1-write text va=0x80002f00 fault=yes intact=true
  • xo: reach xo-plant caught va=0x80190000 roots=k,u why=va ap-el0=0 uxn=0
  • xo: ok text=el0-xo rodata=app-hdr el0-read=fault sys-read=deny slots=5

The smoke requires all thirteen lines (ESR low bits / fault level flexible within permission faults), rejects xo: leak / xo: bad / xo: probe missed / xo: live missed, and rejects any xo: el0-read-text line with a nonzero got=. The same probe runs again as #[test_case] el0_app_text_is_execute_only under cargo test (without the live part). New host unit tests: loader_plans_hello_as_ro_hdr_and_xo_text, loader_rejects_ro_on_text_page, loader_rejects_pages_outside_app_slots.

Negative tests

  • Build: the existing el0-leak-probe smoke build now also enables xo-leak-probe (text mapped EL0 RX again: AP=11, UXN=0) and ADR-098’s pan-keep-leak-probe (the pre-fix PAN behaviour; its own pan: lost … lines are checked by the ADR-098 block). The smoke requires xo: bad pan live-syscall pstate-pan=0 sctlr-span=1, xo: leak live text ap=3 el0=rx sys-read=allow, xo: leak layout text ap=3 el0=rx, xo: leak el0-read-text va=0x80002000 got=<nonzero code word>, xo: leak sys-read text n=8 checks=0, xo: bad el1-pan control pan=true mrs-pan=0 fault=false spsr-pan=false and xo: probe missed, and fails if xo: ok, xo: el0-read-text fault or xo: sys-read text denied appears. All the ADR-092/094/096 leak lines are still required from the same build.
  • Tampered logs (the ADR-097 smoke block run against edited copies of a good serial log; each must fail): xo: ok deleted; the read-text fault replaced by a leak line; a translation fault instead of a permission fault; live text shown as ap=3 el0=rx; the inventory showing reach=none; xo: probe missed appended; the text-pointer syscall with checks=0; an EPAN CPU (fault=yes … mmfr1-pan=3 epan=yes); PAN clear in the handler; and the real leak-build serial. All ten failed with a named missing / forbidden line; the unedited log passed. Results are in the PR body and the honesty ledger.

ADR-091 clause check (sentence not edited)

State of 2026-09-28 (this PR did not edit ADR-091). On 2026-10-02, after the sponsor’s accept, ADR-091 was edited to revision 4; the table below is the check that led to it.

ClauseStill literally true?
Default smoke machine (QEMU virt, -cpu cortex-a76, TCG, one core)Yes (unchanged).
Own TTBR0 + own ASID per EL0 taskYes (unchanged; asid: ok, el0: standing).
EL0 faults when it reads, writes or executes kernel memoryYes (unchanged markers).
Only EL0-reachable pages are five allowlisted user slots backed by non-kernel frames, checked live and at restYes. Still five slots, same labels; the XO page is counted as reachable (UXN=0) and allowlisted; live/at-rest lines byte-identical.
PAN is enabled and EL1 faults on EL0 memoryNo, not literally, for the XO app text page (no FEAT_EPAN: EL1 loads of it do not fault). PAN is enabled (and, with ADR-098 merged here, also during syscall handling and after EL0 trips); EL1 still faults on every EL0 data-accessible page.
Lower-EL IRQ and FIQ taken while EL0 standsYes (unchanged).
Taken lower-EL SError evidenced by the ADR-090 classYes (unchanged).
Only identity mapping left in any TTBR0 is the EL1-only _start stub pageYes (unchanged; ident: lines).
A syscall copies from/to an EL0 pointer only if EL0 may read/write every pageYes, and stronger for text: EL0 can no longer read its text, so text pointers are refused for copies from EL0.
Not speculative / real-hardware / certificationUnchanged.

Revision-4 wording (proposed 2026-09-28; accepted 2026-10-02)

Accepted 2026-10-02: the sponsor wrote “I accept ADR-091 rev 4” in answer to exactly this wording. ADR-091 now carries it, character for character (PAN clause and code-bytes limit; nothing else changed). As proposed on 2026-09-28:

Change only the PAN clause, from “PAN is enabled and EL1 faults on EL0 memory” to:

“PAN is enabled, including while the kernel handles exceptions from EL0, and EL1 faults on EL0-readable memory (the execute-only app text page is not covered: this CPU has no FEAT_EPAN)”

and replace the limit “EL0 can read its own code bytes” with “app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN)”. Everything else unchanged. (2026-09-28 text, history:) This is a proposal for the sponsor to read, not an edit of ADR-091. If the sponsor declines, the alternative is to close this PR (or keep only the separable PAN fix).

Limits

  • One core, the TCG default smoke machine; not real hardware (no KVM re-run of this change), not side-channel, not a certification.
  • Execute-only is enforced for EL0 only. The kernel can still read app text (no FEAT_EPAN, and PAN would not cover it anyway on this CPU). A future CPU with FEAT_EPAN would need SCTLR_EL1.EPAN set; the smoke pins epan=no so that change is noticed.
  • The live check covers hello-libctos; the other seven apps are covered by the build-time checks (build.rs), the loader rules (host tests) and the fact that they all run in the smoke with the same loader path. The image probe uses an in-kernel app-shaped ELF, not a separate user crate.
  • The A9 cross-update to the prior OS image still boots the prior loader; that loader maps R segments with map_el0_ro and R E as execute-only, so the new two-segment ELFs load there too (smoke green), but the prior OS has no ADR-097 probe.
  • Writable (Rw) app segments are now refused at load time: no allowlisted slot takes them. None of the eight apps has one (their .data / .bss are empty), so nothing changes today, but an app that needs writable globals would need a design change (it would need a slot, which is an ADR-091 question).
  • The PAN fix (ADR-098) is proven by the live MRS PAN read and the post-trip checks, not by an EL1 load inside a handler (fatal by design).

ADR-098 — PAN held while the kernel handles EL0 exceptions (correction of a live-claim defect)

  • Status: Accepted (implementation + evidence). Fixes a defect in the accepted ADR-091 claim. It does not change the ADR-091 sentence or scope.
  • Date: 2026-09-29
  • Decision path: DSO relay (2026-09-28 23:54 CEST). The defect was found while working on ADR-097 (PR #150, execute-only app text, which waits for a sponsor rev 4 accept). It is split out here so the fix can merge without waiting for that accept.
  • Base: main at 9a1c2bb (M6 flip, PR #148). Not stacked on #149 or #150.
  • _start (0x4008_0000) untouched.

Context: the defect

ADR-091’s accepted sentence says “PAN is enabled and EL1 faults on EL0 memory”. The evidence is ADR-080’s boot probe (pan: enabled / pan: el1-fault). That probe runs once, early in boot, before any EL0 trip. Two things in the exception path undid PAN once EL0 had run:

  1. SCTLR_EL1.SPAN was 1 (the QEMU reset value; the kernel never cleared it). With SPAN=1, taking an exception to EL1 leaves PSTATE.PAN unchanged. EL0 was entered by ERET with an SPSR whose PAN bit (22) is clear, so every exception from EL0 (SVC, IRQ, FIQ, abort) started its EL1 handler with PAN clear. PAN came back only if the handler called a copy helper (pan::with_user_access restores it after the copy).
  2. return_from_el0 resumed the kernel with SPSR 0x3C4 (PAN clear). After any EL0 trip that ended by returning to the kernel continuation, the kernel kept running with PAN clear until the next with_user_access set it again.

Measured before the fix (reproduced by the pan-keep-leak-probe build below): in the loaded app’s SYS_YIELD handler MRS PAN reads 0 and SCTLR_EL1.SPAN is 1; after the app exits, MRS PAN reads 0 and an EL1 load of an EL0-readable page does not fault. The boot probe still prints pan: enabled and pan: el1-fault in that same kernel.

Consequence for the claim: from the ADR-080 enable (2026-09-19), and so also from the M6 accept (2026-09-28 16:41 CEST) until this fix, the clause “PAN is enabled” held at boot, but not while the kernel handled EL0 exceptions, and not after EL0 trips until a copy helper ran. The boot probe missed this because it ran before any EL0 trip. EL0 itself could not exploit it directly (EL0 still could not reach kernel memory, and the syscall pointer checks of ADR-094 do not depend on PAN), but PAN’s purpose is to catch kernel bugs that dereference user pointers, and that net was down exactly when the kernel handles EL0 requests.

Decision

  1. After a successful ADR-080 enable, clear SCTLR_EL1.SPAN (bit 23). Every exception taken to EL1 then sets PSTATE.PAN.
  2. return_from_el0 sets SPSR.PAN (bit 22) when PAN is enabled, so the kernel resumes with PAN set after an EL0 trip.
  3. Copy helpers are unchanged: with_user_access still clears PAN around an intentional copy and restores it.

Probes (fail-closed in scripts/qemu-smoke.sh)

  • Live, during an EL0 exception: called from the ADR-092 live walk in the SYS_YIELD handler of the loaded hello-libctos app: pan: live-syscall pstate-pan=1 sctlr-span=0. An EL1 load cannot be the test here: once PAN is really set it takes a current-EL permission fault inside an exception handler, which the kernel treats as fatal by design. So the handler reads MRS PAN.
  • After EL0 trips: right after the loaded app exits: pan: after-el0 trips=<n> mrs-pan=1 el1-load va=0x80003000 fault=yes spsr-pan=1 (n ≥ 1 returns from EL0 so far; an EL0-readable page is mapped at the transient crt-stack-pan slot VA, loaded with the ADR-080 catcher armed silently, then unmapped).
  • Summary: pan: ok held boot,live-syscall,after-el0, printed only if both checks passed.
  • Calibration of MRS PAN: it reads 1 in the after-EL0 check, where the real EL1 load faults with SPSR.PAN set; it reads 0 in the negative build, where the load does not fault. (ADR-080 noted that MRS PAN “may read 0” under QEMU 10.0.13 TCG; in this build, whenever a fault proved PAN set, MRS PAN read 1.)
  • #[test_case] pan_span_cleared_after_enable (SPAN = 0 and MRS PAN = 1 in kernel context under cargo test).
  • The smoke requires the three lines and rejects pan: lost, pan: bad after-el0, pan: held missed, pan: held probe missed. Existing pan: id= / pan: present / pan: enabled / pan: el1-fault checks are unchanged.

Negative tests

  • Build: the existing el0-leak-probe smoke build also enables pan-keep-leak-probe, which undoes both parts of the fix (SPAN stays 1; return_from_el0 resumes with SPSR.PAN clear). The smoke requires, from that kernel: pan: enabled and pan: el1-fault (the boot probe still passes — the blind spot), pan: lost live-syscall pstate-pan=0 sctlr-span=1, pan: lost after-el0 trips=<n> mrs-pan=0 el1-load va=0x80003000 fault=no spsr-pan=0, pan: held missed live-taken=true live-ok=false after-el0=false; and fails if pan: ok held, pan: live-syscall pstate-pan=1 or pan: after-el0 appears.
  • Tampered logs (the ADR-098 block against edited copies of a good box serial; each must fail): pan: ok held deleted; live line with pstate-pan=0 sctlr-span=1; after-EL0 line with fault=no spsr-pan=0; trips=0; live line deleted; a pan: lost line appended; and the real negative-build serial. All seven failed with a named missing / forbidden line; the unedited log passed. The leak-build section passes on the negative-build serial and fails on the good serial.

ADR-091 clause check after this fix

Every clause of the accepted revision-3 sentence is literally true again on the default smoke machine, and now continuously for the PAN clause: PAN is enabled at boot, in EL0 exception handlers, and after EL0 trips; EL1 faults on EL0-readable memory (boot probe + after-EL0 check). The sentence is not edited. A dated correction note is added to ADR-091’s status list (the same place as its 2026-09-28 limits update), and a correction row to the honesty ledger.

Limits

  • One core, QEMU TCG default smoke machine; not real hardware (no KVM re-run of this change), not side-channel, not a certification.
  • In handlers the check is MRS PAN (calibrated), not an EL1 load; the live check samples one exception type (SVC SYS_YIELD). IRQ/FIQ/abort entries get the same SPAN=0 behaviour architecturally but are not sampled separately.
  • The after-EL0 check samples one point (after the loaded app exits).
  • Kernel paths that deliberately touch EL0 pages must keep using with_user_access; with PAN now held, one that doesn’t would fault (and in a handler, park). The full smoke (all EL0 apps, FAT / net / TCP samples, A9 cross-update, cargo test) is green with the fix.

ADR-099 — Milestone item 2: the KVM re-check in CI via GitHub OIDC (design sketch, inert)

  • Status: Proposed — design only; since 2026-10-13 also the exact spec for ONE supervised pilot run (sponsor decision 2026-10-13 20:49 CEST, DSO relay). Not recurring: manual dispatch only, no schedule. See Pilot spec.
  • Original status line: Nothing in this ADR is implemented. No AWS call was made and no AWS resource was created or changed to write it. The workflow it adds, .github/workflows/kvm-recheck.yml, is inert (workflow_dispatch only, not scheduled, not a required check, fails closed while its role variable is unset). The sponsor and DSO decide; do not self-merge (ADR-002).
  • Date: 2026-10-02 (CEST).
  • Follows: ADR-085 (first KVM run), ADR-090 (evidence class), ADR-091 (Accepted; its sentence is not touched), ADR-093 (no identifiers in public text), ADR-095 (G3: the manual run this automates).
  • Not in this ADR: no src/, user/, libctos/ or Cargo.* change; _start (0x4008_0000) untouched; smoke.yml, pages.yml and the in-tree scripts/b2-metal-userdata.sh untouched; the three required checks are unchanged; CloudAgent not used.
  • No threat-model bump. security.md on main was v1.72 when this ADR was written (ADR-098, PR #151, merged by DSO on the sponsor’s instruction on 2026-10-02 13:01 CEST); #149 and #150 have merged since (v1.71, v1.73), and the next free number is taken by the ADR-091 revision 4 PR (#153). A bump here would collide with those numbers. The risk table below is written so DSO can attach it to the sponsor record; folding it into security.md is a follow-up after those PRs merge.
  • Public-identifier rule (ADR-093): every account, role, instance, image, subnet, security-group and zone identifier is a <PLACEHOLDER> here and in the workflow. Real values belong in repository variables set by the sponsor-approved path, never in repo text, commits, PR bodies or logs.

Context

ADR-095 proved by hand (one c7g.metal Graviton3 Spot instance, about $0.04, reduced b2-serror profile, markers el0: serror + b2: taken) that the taken lower-EL SError still happens under KVM. That run is a point in time. Milestone item 2 asks whether the same check can run from CI so the claim can be re-checked on a later commit without a person at an AWS console.

GitHub-hosted runners cannot run KVM for this profile on Graviton metal, so CI would have to start a short-lived AWS instance. That needs AWS credentials in CI. The credential rule (rule 4) forbids stored keys. The sponsor approved one exception, tracked as PX0: a GitHub OIDC role with no stored keys (DSO relay, 2026-09-28 21:23 CEST). The exception is conditional. Before the role is created or used:

  • (a) a CloudTrail trail is on;
  • (b) the ctos branch protection on main is enforced for admins;
  • (c) how test results leave the instance is decided before asking for any extra permission such as iam:PassRole.

Condition (c) is answered in this ADR (section 2). The conditions above are taken from the DSO relay text. The decision record in the DSO vault (01-second-brain/decisions/2026-09-28-ctos-ci-oidc-exception.md) was not read while writing this ADR: the file exists on the DSO machines, but the executor’s file reader refused a path outside its allowed root, and that refusal was not worked around. Nothing here quotes or paraphrases the record, and the text of credential rule 4 is not reproduced. DSO must reconcile this ADR with the record before it goes to the sponsor.

Observations made while writing (GitHub read-only, 2026-10-02 about 13:00 CEST; no AWS)

ItemObservedMeaning
Branch protection on main, enforce for adminsOnCondition (b) is verified today. Re-verify at role-creation time.
Required checks on mainThe three smoke checks (ubuntu-24.04-arm, ubuntu-24.04, NMI pin)The new workflow is not one of them and must not become one.
Required approving reviews on mainNone (only status checks are required); strict is offA PR that passes the three checks can be merged by anyone with merge rights. In practice the sponsor merges (ADR-002), but the control is by convention, not by GitHub. Listed as a residual risk below.
Force-push and branch deletion on mainNot allowedGood.
Actions environmentsOnly copilot and github-pages. No KVM environment exists.A workflow that names an environment that does not exist makes GitHub create it with no protection. The environment must be created (reviewers, main-only branch policy) before the role exists.
Repository variablesOnly one build variable; no KVM_* variableThe workflow is inert: its guard fails on the unset role variable.
OIDC sub claim customisationThe repository uses the default template with an immutable subject; the subject prefix carries the numeric owner and repository IDsThe sub GitHub emits is probably not the plain repo:<owner>/<repo>:… form that most AWS examples show. A trust policy written from an example would never match (fails closed, but also never works). The exact string must come from the documentation and the customisation setting at role-creation time, and be confirmed in the supervised pilot from the CloudTrail AssumeRoleWithWebIdentity event. Not verified here.
Repository visibilityPublicActions logs are public. See “Identifier leakage” in the risk table.

Decision (proposed)

Add an opt-in, manual, inert workflow that, once the sponsor path enables it, assumes a least-privilege OIDC role, launches one Spot c7g.metal instance, reads the result from the serial console only, and terminates the instance in an always() step, with three independent teardown layers and a hard cost cap. Only a result produced by a probe on the instance is ever reported as PASS.

1. Trust policy and permission policy (placeholders only)

1.1 Trust policy shape

StringEquals only. No StringLike, no wildcard. The audience is fixed, the subject names this repository and exactly one of: the main branch or one named environment. The environment form is recommended, because the environment can also require a human reviewer and a main-only branch policy.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "GitHubOidcOnlyThisRepoOnlyThisEnvironment",
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::<ACCOUNT_ID>:oidc-provider/token.actions.githubusercontent.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "token.actions.githubusercontent.com:aud": "sts.amazonaws.com",
          "token.actions.githubusercontent.com:sub": "repo:artofdream@<OWNER_ID>/ctos@<REPO_ID>:environment:kvm-pilot"
        }
      }
    }
  ]
}

<SUB_FOR_THIS_REPO_AND_ENVIRONMENT> is one exact string. Two shapes, by the claim template in force:

Template in forceEnvironment form (recommended)Branch form (alternative)
Classicrepo:<OWNER>/<REPO>:environment:<ENV_NAME>repo:<OWNER>/<REPO>:ref:refs/heads/main
Immutable subject (what this repository reports) [unverified format]repo:<OWNER>@<OWNER_ID>/<REPO>@<REPO_ID>:environment:<ENV_NAME>repo:<OWNER>@<OWNER_ID>/<REPO>@<REPO_ID>:ref:refs/heads/main

<OWNER_ID> and <REPO_ID> are GitHub’s numeric IDs, read from the repository settings by the person creating the role, and not written to the repo. Do not “fix” a mismatch by widening to StringLike or *. A mismatch is a fail-closed AccessDenied; fix the string.

Other properties: role maximum session 3600 s; the OIDC identity provider for token.actions.githubusercontent.com must already exist in the account (precondition P4); the audience is the default that aws-actions/configure-aws-credentials requests. Optional hardening: customise the subject or add a job_workflow_ref condition so only kvm-recheck.yml on main matches [verify the claim and the customisation in the AWS and GitHub docs before relying on it].

Why the environment form: a pull request from a fork, or from any branch, produces a different sub (a pull_request subject, or another ref) and does not match. A workflow_dispatch from main bound to the environment matches only after the environment’s protection rules pass.

1.2 Permission policy (least privilege)

Placeholders (region is eu-north-1, the one used in ADR-095; formerly <REGION>), <ACCOUNT_ID>, <AMI_ID> (pinned Ubuntu 24.04 arm64 image), <SUBNET_ID_1> … <SUBNET_ID_n> (one subnet per AZ), <SG_ID> (a security group pre-created by DSO: tagged, no ingress).

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "RunInstancesNewInstanceOnly",
      "Effect": "Allow",
      "Action": "ec2:RunInstances",
      "Resource": "arn:aws:ec2:eu-north-1:<ACCOUNT_ID>:instance/*",
      "Condition": {
        "StringEquals": {
          "ec2:InstanceType": "c7g.metal",
          "ec2:InstanceMarketType": "spot",
          "ec2:MetadataHttpTokens": "required",
          "aws:RequestTag/project": "ctos",
          "aws:RequestTag/component": "kvm-ci-pilot"
        },
        "NumericLessThanEquals": { "ec2:MetadataHttpPutResponseHopLimit": 1 },
        "ForAllValues:StringEquals": { "aws:TagKeys": ["project", "component", "run"] }
      }
    },
    {
      "Sid": "RunInstancesNewChildResourcesMustBeTagged",
      "Effect": "Allow",
      "Action": "ec2:RunInstances",
      "Resource": [
        "arn:aws:ec2:eu-north-1:<ACCOUNT_ID>:volume/*",
        "arn:aws:ec2:eu-north-1:<ACCOUNT_ID>:network-interface/*",
        "arn:aws:ec2:eu-north-1:<ACCOUNT_ID>:spot-instances-request/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:RequestTag/project": "ctos",
          "aws:RequestTag/component": "kvm-ci-pilot"
        }
      }
    },
    {
      "Sid": "RunInstancesPinnedInputsOnly",
      "Effect": "Allow",
      "Action": "ec2:RunInstances",
      "Resource": [
        "arn:aws:ec2:eu-north-1::image/<AMI_ID>",
        "arn:aws:ec2:eu-north-1:<ACCOUNT_ID>:subnet/<SUBNET_ID_1>",
        "arn:aws:ec2:eu-north-1:<ACCOUNT_ID>:subnet/<SUBNET_ID_n>"
      ]
    },
    {
      "Sid": "RunInstancesTaggedNoIngressSecurityGroupOnly",
      "Effect": "Allow",
      "Action": "ec2:RunInstances",
      "Resource": "arn:aws:ec2:eu-north-1:<ACCOUNT_ID>:security-group/<SG_ID>",
      "Condition": { "StringEquals": { "ec2:ResourceTag/project": "ctos" } }
    },
    {
      "Sid": "TagOnCreateOnly",
      "Effect": "Allow",
      "Action": "ec2:CreateTags",
      "Resource": [
        "arn:aws:ec2:eu-north-1:<ACCOUNT_ID>:instance/*",
        "arn:aws:ec2:eu-north-1:<ACCOUNT_ID>:volume/*",
        "arn:aws:ec2:eu-north-1:<ACCOUNT_ID>:network-interface/*",
        "arn:aws:ec2:eu-north-1:<ACCOUNT_ID>:spot-instances-request/*"
      ],
      "Condition": { "StringEquals": { "ec2:CreateAction": "RunInstances" } }
    },
    {
      "Sid": "TerminateAndReadConsoleOfOurTaggedInstancesOnly",
      "Effect": "Allow",
      "Action": ["ec2:TerminateInstances", "ec2:GetConsoleOutput"],
      "Resource": "arn:aws:ec2:eu-north-1:<ACCOUNT_ID>:instance/*",
      "Condition": {
        "StringEquals": {
          "ec2:ResourceTag/project": "ctos",
          "ec2:ResourceTag/component": "kvm-ci-pilot"
        }
      }
    },
    {
      "Sid": "DescribeInstancesForPollAndSweep",
      "Effect": "Allow",
      "Action": "ec2:DescribeInstances",
      "Resource": "*",
      "Condition": { "StringEquals": { "aws:RequestedRegion": "eu-north-1" } }
    },
    {
      "Sid": "DenyOtherRegions",
      "Effect": "Deny",
      "Action": "ec2:*",
      "Resource": "*",
      "Condition": { "StringNotEquals": { "aws:RequestedRegion": "eu-north-1" } }
    },
    {
      "Sid": "DenyIamS3SsmExplicitly",
      "Effect": "Deny",
      "Action": ["iam:*", "s3:*", "ssm:*"],
      "Resource": "*"
    }
  ]
}

(The last Deny is belt and braces: nothing here allows those services, but an explicit Deny keeps a later, careless policy attachment from widening the role. It deliberately does not name kms:*: if account-default EBS encryption uses a customer-managed key, the launch may need KMS permissions, and that is a decision for the sponsor, not a hidden Deny.)

Exact actions to launch, tag-scope and terminate one Spot instance:

ActionNeeded forCan the project=ctos tag restrict it?
ec2:RunInstanceslaunch the one instance (with Spot market options, in one call)Partly. On instance, volume, network-interface, spot-instances-request through aws:RequestTag/project (and component), i.e. the request must tag what it creates. On the security group through ec2:ResourceTag/project. Not on the image or the subnets (they are not tagged by us; pinned by ARN instead).
ec2:CreateTagstag at creationScoped, not by the tag value: ec2:CreateAction = RunInstances means tags can only be written during a launch, never onto an existing resource.
ec2:TerminateInstancesteardownYes, ec2:ResourceTag/project + component on the instance.
ec2:GetConsoleOutputthe result channelYes, same condition.
ec2:DescribeInstancespoll state, Spot interruption reason, sweeperNo. Describe* has no resource-level scope; it is limited by region only. It leaks nothing the account’s EC2 console would not already show the role holder.
iam:PassRolean instance profileNot requested. There is no instance profile.
iam:CreateServiceLinkedRolefirst-ever Spot useNot requested. The Spot service-linked role must pre-exist (P8, unverified).
ec2:CreateSecurityGroup, ec2:Authorize*, key pairs, SSM, S3, KMS—Not requested. The group is pre-created.
ec2:DescribeImages, DescribeSubnets, DescribeSpotPriceHistory—Not requested. Image and subnets are pinned; MaxPrice is the cap.

Service-reference check (public AWS documentation, fetched 2026-10-02, not an AWS API call): TerminateInstances and GetConsoleOutput act on the instance resource type, which supports ec2:ResourceTag/*; RunInstances touches the instance, volume, network-interface, spot-instances-request, security-group, subnet and image types; Describe* have no resource types. IAM condition behaviour is not exercised until a supervised pilot (P13); expect one or two adjustments, each of which goes back to the sponsor as a change to this ADR.

Unverified, flagged for the pilot: if the account’s default EBS encryption uses a customer-managed key, RunInstances may need extra KMS permissions; if so, that is a new permission request and goes to the sponsor under condition (c) logic, it is not added silently.

2. Results-out decision (condition (c))

The question: how does “the smoke passed” leave a box that has no credentials, no key pair and no inbound port?

OptionWhat it needsWhat it avoids / costsVerdict
A. Serial console, ec2:GetConsoleOutputOne tag-scoped Get on the instance (already in the policy). No instance profile.Avoids iam:PassRole, any instance profile, any bucket, any agent, any credential on the instance. The instance cannot send anything except text to its own console. Cost: nothing. Limits: last 64 KiB, may lag, text only, boot noise on the console can carry the host name or addresses (so the workflow prints only ^CTOSB2: lines).Recommended. It is exactly what ADR-085 and ADR-095 used.
B. SSM (Run Command / session output)An instance profile with SSM core permissions, hence iam:PassRole, plus ssm:SendCommand, ssm:GetCommandInvocation, an agent on the box and an output path.Adds a new principal on the instance, a new service, a bigger role.Rejected. It buys a bigger channel than a one-line verdict needs.
C. S3 bucket (instance PUTs the result)Either an instance profile (iam:PassRole) or a pre-signed PUT URL minted by the role (no PassRole, but s3:PutObject and a bucket).A new resource class (bucket, lifecycle, public-block settings, cost), a URL that is a bearer secret on the instance and in user-data, a read path.Not now. Kept as a fallback if console output proves unreliable in the pilot; that would be a new sponsor decision under (c).
D. Instance pushes to GitHub (status, artefact, comment, self-hosted runner)A GitHub token (or runner registration token) on the instance.Violates the credential rule’s intent: a long-lived or minted secret on a machine that downloads and builds third-party code.Rejected.

Sponsor accept (2026-10-02): the sponsor accepted this section, results out via tag-scoped ec2:GetConsoleOutput, no iam:PassRole, no instance profile, in writing on 2026-10-02. It is recorded in the DSO vault and was relayed by DSO. Only this section is accepted; the rest of the ADR is still a proposal.

Recommendation: A, console output. It needs the least (two tag-scoped EC2 read/terminate actions the role needs anyway), avoids the most (PassRole, instance profile, S3, SSM, any credential on the instance), and was already proven by hand twice.

How the workflow stays honest with a text-only channel:

  • Result contract. The user-data prints CTOSB2: result smoke-rc=<rc> sha=<commit> nonce=<random> last (rc starts at 99 = “never ran”; a setup failure prints 98). The nonce is generated by the workflow and substituted into the user-data, so a stale console or another run’s text cannot be mistaken for this run.
  • PASS needs all of: the result line with rc=0, the exact commit and nonce; an anchored CTOSB2: b2-kvm-smoke: PASS taken lower-EL SError observed under accel=kvm line (anchored, so a set -x echo of the same words does not count); /dev/kvm=present; uname reporting aarch64; the KVM API-version line. Anything else is FAIL. No result line before the deadline is INCONCLUSIVE.
  • Live vs after termination (unverified). It is not verified here whether the console is readable live or only after the instance stops. The design handles both: the instance lingers about 90 s after printing the result; the workflow polls live, and if the instance is already terminated it makes a few more reads before giving up as INCONCLUSIVE.
  • Local test of the parser (no AWS): the parse block was run against the real ADR-095 console excerpt plus synthetic result lines. PASS only with the right commit, nonce, smoke line and markers; wrong nonce, wrong commit, rc≠0, missing smoke line, KVM absent, an unanchored echo and a setup failure all give FAIL; no result gives INCONCLUSIVE. This tests the parser, not AWS.
  • Public logs. The workflow prints only the filtered marker lines and AWS error codes; it never prints raw console text or raw AWS stderr. Instance IDs are masked with ::add-mask:: as soon as they exist.

3. Fail-safe teardown and cost cap

Four layers, so no single failure leaves a metal instance running:

  1. Pre-flight refusal. If any project=ctos instance is active, the run stops with REFUSED-active-instance before spending anything. Concurrency group kvm-recheck (cancel-in-progress: false) serialises runs.
  2. On the instance: shutdown -h +<LIFETIME_MIN> is the first line of the user-data (default 45 min; the guard refuses anything over 60), shutdown behaviour is terminate, and the script powers off after a 90 s linger. This works even if the runner dies.
  3. Always-run terminate step. if: always(), with fresh credentials (a second configure-aws-credentials, guarded by the first one’s success, because the first session may have expired or been cancelled). It finds the run’s instances by tag (run=<run id>), so a launch that died after RunInstances but before returning an ID is still cleaned up, terminates them, and waits for terminated.
  4. Tag-based sweeper check (same step). After polling up to 20 min for terminated, counts the instances this run launched (project=ctos, component=kvm-ci-pilot, run=<run id>). Any still pending/running/stopping/stopped gives LEAK; any still shutting-down gives TEARDOWN-SLOW (red, not LEAK). Other active project=ctos instances (for example the one a REFUSED-active-instance pre-flight found) only raise a warning; they never turn this run’s verdict into LEAK (that informational count cannot fail the step) (Bugbot finding, fixed 2026-10-13). This is a check inside the run, not a scheduled sweeper: a schedule would use the role with no human present and needs its own sponsor decision.

Job timeout-minutes is 60; the poll limit is 40 min.

Hard cost cap per run. Spot is billed at the market price, never above MaxPrice. The guard computes cap = MAX_PRICE_USD_HR × (LIFETIME_MIN + 5) / 60 and refuses to run if it exceeds RUN_COST_CAP_USD, or if the price is above the ceiling the sketch allows. Defaults: MaxPrice $0.40/h (the observed Spot price in ADR-095 was $0.2476/h, the manual run used a $0.60 ceiling), lifetime 45 min, cap $0.50; the computed worst case is about $0.33 per run [estimate]. EBS (30 GB gp3, minutes) adds under $0.01 [estimate]. A price above MaxPrice interrupts the instance, which ends the spend and gives INCONCLUSIVE, not PASS.

Typical cost [estimate]: $0.03–0.07 per run (ADR-095: about 8 minutes of metal at about $0.25/h, about $0.04; build time varies with Spot price and apt mirrors).

ScenarioRuns / monthMonthly cost [estimate]
Manual, as needed4–8about $0.12–0.56
Weekly schedule (not enabled, not proposed here)about 4–5about $0.12–0.35
After every merge to main (not proposed)about 20about $0.6–1.4
Worst case: every run hits the full capnabout $0.34 × n

CloudTrail cost for one management-events trail is small [unverified]. An AWS Budget alarm at a low monthly figure is recommended as a last backstop (P9, optional, unverified).

Residual: if the GitHub runner dies hard and the user-data failed to start, the instance runs until MaxPrice interrupts it or someone notices (up to about $0.40 per hour). The in-instance timer, the pre-flight refusal on the next run, and the budget alarm are the mitigations.

4. Spot capacity-failure story

KVM_SUBNET_IDS is a space-separated list, one subnet per AZ, tried in order. On each failed RunInstances the workflow reads only the AWS error code:

Code classExamplesAction
CapacityInsufficientInstanceCapacity, SpotMaxPriceTooLow, capacity-not-availabletry the next AZ
Quota / limitMaxSpotInstanceCountExceeded, VcpuLimitExceededstop: SKIPPED-no-capacity (reason: quota)
Anything else (for example AccessDenied)FAIL immediately; a permissions problem is not “no capacity”
All AZs exhaustedSKIPPED-no-capacity

A skip is never green. The job exits non-zero with a step summary that says “not a pass”. There is no on-demand fallback: a metal on-demand instance would break the cost cap. A mid-run Spot interruption (StateReason of Spot termination) is INCONCLUSIVE and is not retried automatically. The ledger records the last PASS commit and date; a skipped or inconclusive run never updates it.

Result words: PASS (the only exit 0), FAIL, INCONCLUSIVE, SKIPPED-no-capacity, REFUSED-active-instance, LEAK, TEARDOWN-SLOW (this run’s instance still shutting down when the teardown wait ended), ABORTED-after-aws-call (cancelled or crashed after the role was requested or an instance launched, with no result written; teardown still runs), NOT-RUN (guard refused or an earlier step failed; no AWS call was attempted).

5. Honesty: what a PASS proves

A PASS proves only this: at the printed commit on main, on one Graviton3 .metal host with /dev/kvm, under QEMU v10.0.0 + patches 0001/0002 with accel=kvm, the reduced b2-serror profile printed el0: serror and b2: taken (the lower-EL SError was taken).

It does not prove: the default image boots or behaves the same under KVM; the identity inventory, reach walk or syscall-pointer probes; anything about real hardware beyond this one profile; or that “EL0 isolated” is wider than the accepted ADR-091 sentence (that sentence is not changed here). It is also not a security claim. Status words in the summary and the ledger come from the probe’s result line only, never from the workflow’s own opinion. The code under test controls its own console output; that is inherent to the method, so every run records the commit it tested.

6. Precondition checklist (DSO and sponsor)

Nothing below is done by this PR. “State” is the state in this PR’s evidence; all items must be re-checked at the time they matter.

#PreconditionOwnerState (2026-10-02)
P1(a) A CloudTrail trail is on (management events, all regions or at least eu-north-1)DSOMET (verified by DSO from the arm64 workstation, 2026-10-13 about 20:56 CEST: a multi-region trail, home region eu-north-1; the agent made no AWS call)
P2(b) Branch protection on main enforced for adminsDSO / sponsorVerified via the GitHub API. Note: no required approving review; strict is off.
P3(c) Results-out decided before any extra permissionSponsor accepts this ADRDecided in this ADR (console output). Section 2 was accepted by the sponsor in writing on 2026-10-02, recorded in the DSO vault (relayed by DSO; the verbatim wording is in the vault and is not reproduced here). No PassRole requested.
P4The GitHub OIDC identity provider exists in the accountDSOExists (DSO, 2026-10-13): already present and shared; managed by another project’s Terraform, not by ctos.
P5A GitHub environment (name kvm-pilot since 2026-10-13; was placeholder kvm-recheck) with required reviewers and a main-only deployment branch policy, created before the roleSponsor / DSOCreated by DSO 2026-10-13 22:12-22:16 CEST (sponsor approval 22:11). Sponsor decision 22:15: sole required reviewer = the sponsor, prevent_self_review off, deployments from main only, admin bypass off.
P6The role created via a sponsor-approved path with exactly the trust and permission policies above, adjusted for the immutable subjectSponsor / DSOCreated by DSO 2026-10-13 22:12-22:16 CEST on the sponsor’s 22:11 approval (supersedes the earlier “not by an agent” wording). Tagged project=ctos, component=kvm-ci-pilot; inline policy follows this ADR incl. the hop-limit fix (NumericLessThanEquals). Not verified by this agent (no AWS call).
P7Pinned inputs set as repository variables (never repo text): image, no-ingress tagged security group, one subnet per AZ, region, role ARNDSONot done (all unset)
P8Spot service-linked role and Spot vCPU quota for one c7g.metalDSONOT VERIFIED
P9AWS Budget alarm (optional backstop)DSOSet by DSO 2026-10-13: budget ctos-monthly-5, USD 5/month, filtered to project=ctos, alerts at 80% actual and 100% forecast.
P10Credential rule 4 and the PX0 exception record referenced in the PR that enables the workflowDSORecord not read for this ADR; rule text not reproduced here
P11KVM_ROLE_ARN is set lastDSOUnset, so the workflow is inert (verified)
P12Third-party actions pinned by full commit SHAAuthor, before enablingNot done (tags in the sketch)
P13A supervised pilot run, then DSO reviews CloudTrail (who assumed what, which calls, which denies)DSO + sponsorNot done
P14The exact immutable-subject sub string confirmedDSONOT VERIFIED
P15Public-log hygiene confirmed on the pilot (no identifier in the log or summary)DSONot done

7. Risk and impact evaluation (for the sponsor record)

Impact is “what an attacker or a mistake could cost”. Residual is after the listed mitigations, assuming P1–P15 are met.

#ThreatLikelihood × impact (before)MitigationsResidual risk
1Another repo, branch, fork or pull request assumes the roleLow × HighExact aud and sub (no wildcard); environment with required reviewers and main-only branch policy; fork PRs get a different sub and no id-token for secretsLow. Depends on the exact sub being right (P14) and the environment existing first (P5).
2A PR edits the workflow to abuse the roleMedium × HighDispatch runs only the default-branch file; guard refuses any ref but main; environment gate; enforce_admins is onLow–Medium. main requires no approving review (observed), so the control is the sponsor merging by convention. DSO may ask the sponsor to require one review on main or a code-owner rule for .github/workflows/.
3Role is over-permissioned (miner, wrong instance type, other region)Medium × HighConditions on instance type, Spot market, IMDSv2, tags and region; pinned image/subnets/SG; 1 h session; no IAM, no PassRole; Spot quota; budget alarmLow. IAM conditions are untested until the pilot (P13).
4Tag spoofing (a resource tagged project=ctos to dodge the sweeper, or the terminate scope abused)Low × LowTags only at creation (CreateAction); terminate only matches tags the role itself applied; the sweeper counts the tag broadlyLow. A person with other credentials could still tag things; out of scope for this role.
5Privilege escalation through the instanceLow × HighNo instance profile, no iam:PassRole, explicit Deny on iam:*; IMDSv2 required, hop limit 1; no credentials on the boxLow.
6Secret in user-dataLow × MediumUser-data holds only a commit, a nonce and a timer; no keys, tokens or URLs with secretsVery low.
7Identifier leakage in public logs (instance ID, account ID, host name, IP)Medium × Low–MediumAccount-ID masking; instance IDs masked at creation; only ^CTOSB2: lines and error codes are printed; no raw console or stderr; identifier guard on repo textLow. Confirm on the pilot (P15).
8Leaked running instance (cost, exposure)Medium × LowPre-flight refusal; in-instance shutdown -h +45 and terminate-on-shutdown; always-run terminate with fresh credentials, by tag; wait for terminated; sweeper gives LEAK; budget alarmLow. Worst case about $0.40/h until noticed if the runner and the user-data both fail.
9Console text is forged or replayedLow × MediumPer-run nonce and commit in the result line; anchored PASS line; KVM and arch markers; terminate-by-tag so one run’s console is read onceLow. The code under test can print its own PASS (inherent; the commit is recorded and the tested commit must be on main).
10Silent green on a skip or timeoutMedium × High (trust)Only PASS exits 0; SKIPPED, INCONCLUSIVE, REFUSED, LEAK, TEARDOWN-SLOW, ABORTED-after-aws-call, NOT-RUN are red; guard fails (not skips) on an unset variableLow.
11Spot capacity or interruptionHigh × LowMulti-AZ retry; honest SKIPPED / INCONCLUSIVE; no on-demand fallbackAccepted: the check is sometimes unavailable.
12Third-party action supply chain (configure-aws-credentials, checkout)Low × HighPin by full commit SHA before enabling (P12); permissions: {} at top level; the job only gets id-token: write and contents: readLow–Medium until pinned.
13Supply chain on the instance (apt, the rustup install script, git clone, QEMU source)Medium × LowQEMU is built from the pinned v10.0.0 source plus the repo’s patches; the repo is cloned at a commit already on main; the instance holds no credentials, has no ingress, IMDSv2 hop limit 1, and lives under an hourLow. Open egress remains (needed for apt/rustup/git).
14Repo rename or transfer (subject reuse by a new owner)Low × HighThe immutable subject (owner and repo IDs) is what the trust policy pinsLow once P14 is confirmed.
15Cost runawayLow × LowCost-cap guard; MaxPrice; lifetime; concurrency 1; budget alarmLow; about $0.33 worst case per run [estimate].
16CloudTrail off, so no audit trailUnknown × HighCondition (a) is a hard precondition before the role is created or usedP1 met 2026-10-13 (multi-region trail, DSO-verified).
17Unprotected environment auto-created by the workflow’s first referenceMedium × HighP5: create the environment with protections first; order mattersLow if the order is followed.
18Unknown account facts (Spot service-linked role, quota, EBS default key)Medium × LowP8; pilot; any new permission goes to the sponsor, not added silentlyLow.
19Credential rule 4 is only relaxed for this roleLow × MediumOne role, one repository, one environment, no stored key, 1 h tokens; the exception record is referenced in the enabling PR (P10)Low. Any second role or a schedule needs its own approval.

8. The inert workflow

.github/workflows/kvm-recheck.yml and scripts/kvm-recheck-userdata.sh are the sketch.

  • Triggers: workflow_dispatch only (one optional input, the commit to test, which must be on main). No schedule. It is not, and must not become, a required check.
  • Fails closed: the first step makes no AWS call and fails (red, summary “NOT-RUN”) unless the repository, the ref (refs/heads/main), the role variable (and image, group, subnets, region variables), the role ARN shape and the cost arithmetic are all in order. While KVM_ROLE_ARN is unset, it cannot do anything else.
  • Permissions: permissions: {} at the top; the job gets id-token: write and contents: read only.
  • Pinned inputs come from variables, not repo text.
  • Pre-run checks done locally (no AWS): YAML parse, actionlint with shellcheck, bash -n and shellcheck on the user-data, the guard run with simulated variables (unset, wrong ref, wrong repo, bad ARN, over-cap, hostile input), and the console parser run on the real ADR-095 excerpt (see section 2).
  • Sketch status: the workflow has never run; the ubuntu-24.04 runner’s AWS CLI behaviour, the aws-actions inputs, and every AWS call are untested.

User-data is derived from the ADR-095 variant of scripts/b2-metal-userdata.sh: same QEMU build, same cargo build --features b2-serror, same scripts/b2-kvm-smoke.py with CTOS_B2_TIMEOUT=90 and CTOS_B2_STALL=15, up to three attempts; plus the timer first, the uname -srm (no host name), KVM API line, CTOSB2: result line with commit and nonce, and the linger-then-poweroff.

Consequences

  • If accepted and later enabled: a re-check of the reduced profile can run on demand without a stored AWS key, with the role limited to one region, one instance type, one pinned image, tagged resources and no IAM, at roughly $0.03–0.07 per run [estimate].
  • Now: nothing changes. The workflow is a file that fails closed. The honesty ledger gets a Planned row (documented sketch, not implemented), not Implemented.
  • Cost of the choice: the console channel is narrow and text only; Spot capacity makes the check sometimes unavailable; the immutable-subject format and several account facts have to be verified by DSO before the first run.
  • Not decided here: a schedule, a per-merge run, making it a required check, an S3 or SSM result channel, an on-demand fallback, a standalone sweeper. Each needs its own sponsor decision.

Pilot spec (one-off, supervised; 2026-10-13)

Sponsor decision 2026-10-13 20:49 CEST (DSO relay): prepare this PR as the exact spec for one supervised pilot run. Not recurring: the workflow stays workflow_dispatch only, with no schedule, and it is not a required check. The PR stays draft. Nothing here creates an AWS or GitHub resource.

Order of operations (DSO / sponsor; the agent does none of these):

  1. CloudTrail on (P1): met, verified by DSO from the arm64 workstation, 2026-10-13 about 20:56 CEST: a multi-region trail, home region eu-north-1.
  2. Create the GitHub environment kvm-pilot (below), before the role.
  3. Create the OIDC provider (if absent) and the role through the sponsor-approved path, with exactly the trust policy and permission policy in section 1 and role MaxSessionDuration = 3600 s.
  4. Set the repository variables KVM_AMI_ID, KVM_SG_ID (pre-created, tagged project=ctos, no ingress), KVM_SUBNET_IDS, and last KVM_ROLE_ARN.
  5. One supervised dispatch from main with pilot_ack = supervised-one-off-pilot; a reviewer approves the environment deployment; DSO watches the run.
  6. DSO reviews CloudTrail for the session, then unsets KVM_ROLE_ARN (back to inert) unless the sponsor decides otherwise.

Exact sub (environment-scoped, immutable-subject form): repo:artofdream@<OWNER_ID>/ctos@<REPO_ID>:environment:kvm-pilot. The repository reports use_default: true, use_immutable_subject: true (read-only GitHub API, 2026-10-13); its subject prefix carries the numeric owner and repository IDs. <OWNER_ID> and <REPO_ID> are those numeric IDs: values supplied to DSO out of band, not written here. StringEquals, no wildcard.

Environment kvm-pilot (GitHub setting, $0; to be created by DSO / the sponsor):

SettingValue
Namekvm-pilot (the workflow’s environment: and the sub must match exactly)
Required reviewersthe sponsor (and/or DSO’s account); at least one; “Prevent self-review” on
Wait timer0
Deployment branches and tagsSelected branches and tags, one rule: branch main (no tags)
Allow administrators to bypassoff
Environment secrets / variablesnone (no stored keys; repository variables above are enough)

Pinned actions (full commit SHA, version in the comment; looked up from the release tags 2026-10-13):

ActionTagCommit SHA
actions/checkoutv7.0.13d3c42e5aac5ba805825da76410c181273ba90b1
aws-actions/configure-aws-credentialsv6.3.0e1253824e5c10ff9df46874f81ed3ec929e19cfd

Region: eu-north-1 (hard-coded in the workflow and in the policies; not an identifier value).

Tags: project=ctos, component=kvm-ci-pilot, run=<run id> on the instance, volume, network interface and Spot request (RunInstances request-tag conditions); TerminateInstances and GetConsoleOutput require ec2:ResourceTag/project=ctos and ec2:ResourceTag/component=kvm-ci-pilot; CreateTags only with ec2:CreateAction=RunInstances.

Limits enforced by IAM condition keys: ec2:InstanceType=c7g.metal, ec2:InstanceMarketType=spot, ec2:MetadataHttpTokens=required, ec2:MetadataHttpPutResponseHopLimit ≤ 1 (NumericLessThanEquals), region via aws:RequestedRegion. Not enforceable by IAM: the Spot price. EC2 has no condition key for the Spot max price (AWS service reference, checked 2026-10-13), so the price cap is the workflow’s MaxPrice (MAX_PRICE_USD_HR, guard refuses above $0.60/h and above the per-run cap), plus the in-instance timer, plus an AWS Budget alarm (P9). Session: role MaxSessionDuration 3600 s (a role property, not a policy condition) and role-duration-seconds: 3600 in the workflow.

Results out: tag-scoped ec2:GetConsoleOutput only; no iam:PassRole, no instance profile (sponsor accepted 2026-10-02, recorded in the DSO vault).

CloudTrail (P1): met (verified by DSO from the arm64 workstation, 2026-10-13 about 20:56 CEST: a multi-region trail, home region eu-north-1). Still unverified: OIDC provider, Spot service-linked role and quota, EBS default key, live-vs-terminated console readability, IAM condition behaviour (first exercised in the pilot).

Setup recorded 2026-10-13 (DSO, sponsor-approved; docs only)

Relayed by DSO; this agent made no AWS call and set no repo variable. Text only, no identifier values.

  • Who/when: on the sponsor’s approval at 22:11 CEST, DSO created the kvm-pilot environment and the OIDC role between 22:12 and 22:16 CEST on 2026-10-13. This supersedes P6’s earlier “not by an agent” line.
  • Role: tagged project=ctos and component=kvm-ci-pilot; inline policy follows this ADR, including the hop-limit fix.
  • OIDC provider: already existed; shared, managed by another project’s Terraform.
  • Image: Ubuntu 24.04 arm64 AMI dated 20261004 - not the image ADR-095 used, so a pilot result is not a byte-identical re-run of ADR-095.
  • Security group: no ingress.
  • Environment protection (sponsor decision 22:15): sole reviewer is the sponsor, prevent_self_review off, deployments from main only, admin bypass off.
  • Budget: ctos-monthly-5, USD 5/month, filtered to project=ctos, alerts at 80% actual and 100% forecast.
  • Still open: P7/P11 repository variables unset (workflow inert), P8, P10, P12-P15.

Pilot run 2026-10-13 (relayed by DSO)

  • The sponsor set the repository variables and dispatched the one supervised pilot, workflow run 38468490030, at main 069fe83.
  • Smoke: PASS under accel=kvm on c7g.metal (el0: serror + b2: taken; run nonce 685780b675853a95).
  • The run’s final verdict was red, from a false LEAK: teardown’s 5-minute wait for terminated timed out while the metal instance was still shutting down, and the sweep counted it. DSO verified the instance reached terminated. Fixed by the slow-termination change below (item 4).
  • RunInstances succeeded with the policy as written (no snapshot resource).
  • Relayed facts; this agent made no AWS call. No instance or account identifiers here.

Review fixes 2026-10-13 (Cursor Bugbot findings on PR #152, plus pilot findings)

  1. High (Bugbot) - snapshot resource for RunInstances: kept out. AWS docs list snapshot as a RunInstances resource type (Service Authorization Reference; account-less ARN form), but the live pilot run launched without it (the AMI is public and its snapshot is not in this account). The policy is unchanged. Add a snapshot resource only if a concrete UnauthorizedOperation denial on a snapshot appears.
  2. Medium - a pre-flight refusal was reported as LEAK. The sweeper counted every active project=ctos instance, so the instance that triggered REFUSED-active-instance made teardown fail and the verdict became LEAK. The sweeper now counts only instances this run launched (tag run=<run id>); others only warn.
  3. Slow metal termination (pilot finding). The pilot’s teardown waited only 5 minutes for terminated; a c7g.metal was still shutting down, and the sweep reported a false LEAK. Teardown now polls up to TERM_WAIT_MIN (20 min; job timeout 75 min). At the end: a run-tagged instance still pending/running/stopping/stopped is LEAK; one still shutting-down is TEARDOWN-SLOW (red, not a pass, not LEAK; the smoke result stays in the summary). A later run’s pre-flight still refuses on any instance that never terminates.
  4. Subnet list parsing. KVM_SUBNET_IDS may be separated by spaces, commas, or both; an empty list after parsing fails the launch step.
  5. Low - NOT-RUN after an AWS call. The verdict defaulted to NOT-RUN whenever no result file existed, including a cancel or crash after assume-role or launch. New word ABORTED-after-aws-call: used when the credential step ran (success, failure or cancelled) and no result was written. NOT-RUN now means no AWS call was attempted. Teardown is unchanged (if: always() after credentials succeeded).

No identifier values. No AWS call, no repository variable set, no dispatch by the agent.

Open reconciliation points (for the enabling PR)

  1. CloudTrail (P1, condition (a)): met — verified by DSO from the arm64 workstation, 2026-10-13 about 20:56 CEST: a multi-region trail, home region eu-north-1.

DSO’s reconciliation notes, 2026-10-02. They are open points for the PR that would enable this workflow, not done here. This sketch does not close any of them.

  1. Exact immutable-subject string (P14): stated 2026-10-13 in the Pilot spec, with the numeric IDs supplied to DSO out of band; still to be confirmed against the pilot’s CloudTrail event. This repository’s sub claim uses the immutable-subject form (numeric owner and repository IDs), so the trust policy’s sub must be copied from what GitHub actually emits, then confirmed in the supervised pilot from the CloudTrail event. Never widened to a wildcard.
  2. The protected environment must be created first (P5): required reviewers and a main-only deployment branch policy, before the role and its trust policy exist. Today no such environment exists.
  3. Actions must be SHA-pinned (P12): done 2026-10-13 (see Pilot spec). Re-check against the tags before the run.
  4. Region: eu-north-1. Since 2026-10-13 the policies and the workflow state it directly; the region is eu-north-1, the one the G3 re-check (ADR-095) used. The region name is not an identifier value. AZ, subnet, image and security-group values stay in repository variables.
  5. Results-out (section 2) accepted. The sponsor accepted section 2 in writing on 2026-10-02 (results out via tag-scoped ec2:GetConsoleOutput, no iam:PassRole, no instance profile), recorded in the DSO vault, relayed by DSO. This satisfies condition (c) for that section only. Anything beyond it (S3, SSM, a PassRole) needs a new decision.
  6. Still open from the checklist and not touched by DSO’s notes: the OIDC provider (P4), the Spot service-linked role and quota (P8), the DSO vault decision record (P10; not read here), the supervised pilot (P13).

Merge notes

This PR is draft and must stay a draft until DSO and the sponsor decide. It was cut from main at 3c47c0c, then merged with main (a normal merge, no force-push) to stay mergeable after #149 and #150 merged. #149, #150 and #151 were merged by DSO on the sponsor’s instruction. The ADR-091 revision 4 PR (#153) also appends to docs/SUMMARY.md, research/moc.md and docs/framework/honesty-ledger.md; if it merges first, expect trivial append-only conflicts there: keep both sides.

Reproducibility

# Syntax and lint (no AWS)
python3 -c "import yaml;yaml.safe_load(open('.github/workflows/kvm-recheck.yml'))"
actionlint .github/workflows/kvm-recheck.yml    # needs shellcheck on PATH
bash -n scripts/kvm-recheck-userdata.sh
sh scripts/check-public-ids.sh --self-test && sh scripts/check-public-ids.sh   # hits=0

ADR-100 — The SMP race story: plan, global-state inventory, and S1 secondary bring-up smoke

  • Status: Proposed (plan) + S1 implemented as an opt-in smoke in the same PR. The plan does not change any claim.
  • Date: 2026-10-13
  • Decision path: milestone item 5 (“the SMP race story”), sponsor approval 2026-10-13, relayed by DSO. $0 work: one local machine plus the existing GitHub Actions checks.
  • Base: main at af5a9d1 (ADR-091 rev 4 recorded, PR #153).
  • _start (0x4008_0000) untouched. The secondary entry is a separate symbol, _secondary_entry, in the same boot-stub page.

Context

ctos runs on one core. The accepted claim, ADR-091 revision 4, scopes “EL0 isolated” to -cpu cortex-a76, TCG, one core. That sentence does not change in this ADR or in any PR of this story. A multi-core claim of any kind needs a new, separate written sponsor accept.

Every piece of kernel code was written on the one-core assumption: “nothing else runs while I hold this”. On two cores that is false. This ADR writes down, before any code relies on it, where the kernel keeps global mutable state, what each piece assumes, what goes wrong on a second core, and the order in which the gaps get closed, each with a probe that fails closed.

Inventory of global mutable kernel state (main af5a9d1)

Method: rg for static / static mut items under src/ (excluding the new src/smp.rs), then reading each owner module. 262 static items:

KindCountWhere
static mut (no lock at all)25paging.rs 19 (page-table storage L1/L2/L3/L2_RAM/L3_KERNEL[], *_USER, *_ASID_B, *_HIGH, *_HIGH_RAM, and three “used” bump counters); virtio.rs 3 (DMA, NET_RX, NET_TX); el0.rs, ro.rs, teardown.rs 1 each (bait / magic words)
spin::Mutex-wrapped10UART, heap ALLOCATOR, frame ALLOC, SCHED, VFS FS + LAST_READ, FAT, virtio DEV + NET, and paging::TABLES: Mutex<()> (a convention lock: 44 TABLES.lock() sites guard the static mut tables, but the type system does not force it)
Atomic flags / counters208mostly probe bookkeeping (EXPECT_* / *_CAUGHT in exception.rs 61, paging.rs, syscall.rs, el0.rs, pan.rs, sched.rs, timer.rs)
Immutable / linker symbols19__stack_top etc., embedded ELF, magic constants

Plus per-CPU hardware state that the kernel treats as global: one boot stack (__stack_top), one exception stack, one fatal stack (linker symbols), VBAR_EL1, TTBR0/1_EL1, TCR, SCTLR (incl. SPAN), PSTATE.PAN, TPIDR_EL1 (holds the saved kernel TTBR0 for the EL0 trampoline), DAIF, the generic timer, the GIC CPU interface.

What each piece assumes and how it races on two cores

#StateOne-core assumptionRace on two coresRisk
R1Page-table storage + bump counters (static mut, paging.rs)TABLES is taken by convention (44 sites); not audited that every writer (early boot, tears, relocations, leak-probe plants) holds it; L3_*_USED = n + 1 is a plain read-modify-writeAny writer outside TABLES races a builder on the other core: same L3 table handed out twice, lost or crossed mappings, an EL0-visible leaf in the wrong rootHigh (isolation)
R2TLB maintenance (tlbi vmalle1, tlbi vaae1, no IS suffix anywhere; switch_ttbr0_no_tlbi)Invalidating the local TLB is enoughA torn / unmapped page (identity tears, unmap, ASID-B private page) stays cached in the other core’s TLB; EL0 or EL1 on that core still reaches itHigh (isolation; every “torn” claim)
R3ASIDs are fixed constants (1 = user root, 2 = ASID-B), no allocator, no generationOnly one task holds an ASID at a time on one TLBTwo cores running different roots under one ASID share TLB tags (stale translations across tasks); ADR-096 t2t “no-tlbi” argument holds per core onlyHigh
R4TPIDR_EL1 = saved kernel TTBR0; one exception / fatal stackOne CPU, one nesting chainEach core needs its own; sharing one exception stack is memory corruption on the first simultaneous exceptionHigh (correctness)
R5el0.rs ACTIVE / USER_PC / USER_SP / USER_TTBR / KIND; exception.rs EXPECT_* / *_CAUGHT“The” EL0 context and “the” expected faultA fault on core B is accepted as core A’s expected probe fault (false pass) or the restore path resumes the wrong contextHigh (probe integrity)
R6SCHED (spin::Mutex) + TASK_A_SP / TASK_B_SP; unlock before context_switchLock is only against re-entry; one runnerBoth cores pick and run the same task on one stackHigh
R7Heap ALLOCATOR, frame ALLOC (spin::Mutex)No IRQ path allocates; contention never happensMutual exclusion holds across cores (spin locks are SMP-safe), but not IRQ-safe: a lock taken in IRQ context on the same core deadlocks; frame free/alloc vs. page-table use needs TLB ordering (R2)Medium
R8virtio DMA / NET_RX / NET_TX (static mut) and the descriptor ringsSingle driver user; polled completionTwo cores post to one ring; double-used descriptors, corrupted DMA buffersMedium (no isolation claim depends on it)
R9VFS FS, FAT, LAST_READ (spin::Mutex)Whole-op serialisation by one callerLock-correct per call, but multi-call sequences (open → read → close, mkdir) are not atomicLow/Medium
R10UART (spin::Mutex) + write_str_raw / raw() (no lock, used by fatal/exception paths and probes)Raw writes never interleaveInterleaved bytes from two cores break line-based markers; a fatal path on one core while the other holds UARTLow for safety, High for evidence (smoke grep)
R11GIC distributor / timer init, timer tick countersOne CPU interface, one timerSecondary needs its own GICC + timer init; SPI routing; tick counts are per-coreMedium
R12PSTATE.PAN, SCTLR.SPAN, TCR, VBARSet once at boot on the one coreA secondary that does not repeat ADR-080/098 setup runs with PAN off / wrong vectorsHigh (the ADR-091 PAN clause)

Riskiest: R2 (local-only TLBI), R1 (unlocked page-table builders), R3 (fixed ASIDs), R5 (global “expected fault” flags), R12 (per-core PAN/SPAN/VBAR setup). Any of these alone would make a multi-core “EL0 isolated” claim false.

Plan

Locking / ownership

  • Page tables (R1): make TABLES the only way in: move the tables and bump counters into the Mutex payload (type-enforced), mask IRQs while held, audit every current static mut writer. No lock-free page-table writes.
  • TLB (R2): every unmap, permission downgrade or tear that may be cached on another core uses the Inner-Shareable forms (tlbi vae1is / vaae1is / vmalle1is / aside1is) followed by dsb ish; isb. Local forms stay only where the mapping was never visible to another core (documented per site). No IPI-based shootdown is needed for the ARMv8 broadcast case on QEMU virt; an IPI path is only planned if a probe shows broadcast is insufficient.
  • ASIDs (R3): a small allocator with a generation counter; on roll-over, tlbi vmalle1is and bump generation; each core checks its current task’s generation on switch. Until then: secondaries never run EL0 (S1–S3).
  • Per-CPU (R4, R5, R11, R12): a PerCpu block per core (stacks, saved TTBR0, EL0 context, expected-fault flags, timer state), pointed to by TPIDR_EL1; the trampoline’s current use of TPIDR_EL1 moves into that block. Probe “expect/caught” flags move to per-CPU.
  • Scheduler (R6): one run-queue lock, task state Running(cpu); a task is never picked while running elsewhere.
  • Heap / frames (R7): keep spin mutexes; mask IRQs while held; forbid allocation in IRQ handlers (documented check).
  • Drivers / FS (R8, R9): stay owned by CPU 0 (other cores reach them only via a syscall that is serviced on CPU 0) until a probe needs more.
  • UART (R10): one console lock that also covers raw() writes in non-fatal paths; per-line atomic writes; fatal path may break the lock (documented).

Secondary bring-up

PSCI CPU_ON (function 0xC400_0003) via HVC #0 — QEMU virt with -kernel and no EL2/EL3 firmware exposes PSCI on the HVC conduit, with secondaries powered off. Entry is a physical address; the secondary starts at EL1 MMU off. Later slices give it its own stack, VBAR, TCR/MAIR, TTBR1 (shared kernel high map), SCTLR (incl. SPAN=0), PAN set, GICC + timer, then a park/run loop.

Isolation probes that must be re-proved per core (S4)

On each core: ADR-080/098 PAN enabled + held; ADR-013/039 EL0 cannot read/execute kernel; ADR-092 EL0-reach walk (shared tables, but walk from each core’s live TTBR0); ADR-094 syscall pointer checks; ADR-096 task-to-task; ADR-097 execute-only text; ADR-087/088 identity inventory (each core’s TTBR0); plus a new cross-core TLB probe: unmap on core 0, access on core 1 must fault (negative build: local-only TLBI must be caught).

PR sequence (each with a fail-closed acceptance probe)

SliceContentAcceptance probe (fail-closed)Negative test
S1 (this PR)Plan + opt-in smp2-probe: CPU_ON one secondary, MMU off, print MPIDR, store it to a flag, parksmp: secondary up mpidr=0x0000000080000001 and smp: ok cpu1 … parked under -smp 2; default smoke must show no smp: lineSame ELF under -smp 1 → rc=-2, smp: cpu1 missed; checker must fail
S2Secondary MMU on: shared TTBR1, own stack/VBAR/SPAN=0/PAN; PerCpu via TPIDR_EL1; console locksmp: cpu1 mmu-on pan=1 span=0 vbar=highbuild that skips SPAN clear on cpu1 must print smp: cpu1 pan lost
S3Type-enforced TABLES + Inner-Shareable TLBI everywhere + cross-core unmap probesmp: xtlb unmap cpu0 → cpu1 faultlocal-tlbi-leak-probe build (old local TLBI) must report smp: xtlb leak
S4ASID allocator with generation; per-CPU EL0 context + expect flags; re-run each isolation probe on cpu1 (still kernel-driven, one EL0 task per core)each probe’s existing … ok marker tagged cpu=1existing leak builds run on cpu1 must still be caught
S5Scheduler run-queue lock, EL0 tasks on both cores; docs-only proposal of a multi-core sentence for the sponsorall of the above in one boot, under -smp 2 CI job—

The -smp 2 CI job stays non-required until the sponsor accepts a multi-core claim.

What will NOT be claimed

  • Not “SMP support”, not “multi-core safe”, not “EL0 isolated on multiple cores”. ADR-091 rev 4 stays one core.
  • S1 does not run EL0, the scheduler, the heap, page-table code, drivers or interrupts on the secondary. The secondary runs MMU off, prints, writes one word, parks.
  • No claim about real hardware, KVM, other -cpu models, more than two cores, or PSCI firmware other than QEMU’s built-in HVC conduit.
  • No memory-ordering / data-race freedom claim for any existing module. The inventory is by code reading; it may miss state reached through raw pointers.
  • No performance or scaling claim.

S1 decision (this PR)

  • Cargo feature smp2-probe (opt-in, default off). After the identity tears and before GIC/virtio (same spot as b2-serror), the primary prints smp: cpu0 mpidr=…, cleans the flag line, calls PSCI CPU_ON(target=1, entry=PA(_secondary_entry), context=PA(flag)), waits up to 0.5 s polling the flag (with dc civac), then prints smp: cpu_on target=0x1 rc=… and smp: ok cpu1 mpidr=… parked or smp: cpu1 missed seen=…, then parks.
  • _secondary_entry (in .text.boot, after _start, PA 0x4008_0048 on this build): MMU off, raw PL011 writes of smp: secondary up mpidr=0x<16 hex>, store MPIDR to the context PA, dsb sy; sev, wfe loop. No stack, no Rust.
  • scripts/qemu-smoke.sh: smp2_check + smp2_smoke. CTOS_SMP2_SMOKE=1 adds it after the default run; CTOS_SMP2_ONLY=1 runs only the id guard + this. Positive under -smp 2, negative under -smp 1 (checker must fail and smp: cpu1 missed seen=0x0 must appear). The default run gains a check that no smp: line appears.
  • CI: a new non-required job QEMU -smp 2 bring-up (opt-in, ADR-100; not required) runs CTOS_SMP2_ONLY=1. The three required checks are unchanged.

Consequences

  • Default kernel binary: no code from src/smp.rs (module is #[cfg(feature = "smp2-probe")]). The default smoke proves it still passes and has no smp: marker.
  • _secondary_entry exists only in the smp2-probe build (it lives in src/smp.rs), so the default boot-stub page is byte-identical in layout; the ADR-087 inventory on the default run still reports the one stub page (k=1 u=1 a=1).
  • Next: S2. Done as ADR-102 (opt-in; PerCpu via TPIDR_EL1 moved to S4). S3: ADR-103. S4: ADR-104. S5: ADR-105. Claim sentence (proposed): ADR-106.

ADR-101 — Forward-only scrub of host and user names + guard classes

  • Status: Accepted on merge (security / compliance guardrail, sponsor decision 2026-10-13 20:49 CEST, relayed by DSO). Not a kernel change.
  • Date: 2026-10-13
  • Follows: ADR-093 (cloud identifiers + scripts/check-public-ids.sh). ADR-099 is taken by the open draft PR #152; this ADR takes 100.
  • No history rewrite (sponsor: forward-only). The replaced names remain in earlier commits and merged PR diffs.
  • _start (0x4008_0000) untouched. No kernel logic change: src/ and linker.ld change in comments only.

Context

ADR-093 scrubbed cloud identifiers and the local OS user name in host paths. It did not classify machine names (the nicknames of the two developer workstations, used in docs as if they were roles), host names that tools printed into committed logs (an agent VM host name, an EC2 private host name, GitHub-hosted runner host names inside uname -a lines) or a machine ID (a UUID, in full once and as an 8-hex prefix four times). This repository is public. Two scripts would also print a host name into future logs: scripts/b2-metal-userdata.sh (uname -a) and scripts/qemu-nmi-probe.py (socket.gethostname()).

Decision

  1. Replace with role names or placeholders (values are not listed here):
    • the amd64 developer workstation → “amd64 workstation”; the arm64 developer workstation → “arm64 workstation” (with “the” where the grammar needs it);
    • agent VM host name → <agent-vm-hostname>; EC2 private host name → <ec2-private-hostname>; GitHub runner host names → <gha-runner-hostname>; machine ID → <machine-id>. “agent-box” stays: it is a role name, not a host name.
  2. Stop new leaks at the source: b2-metal-userdata.sh prints uname -srm (as ADR-095 already did by hand); qemu-nmi-probe.py prints host-os=<sysname> <machine> instead of the host name.
  3. Guard classes (fail closed, CI-gated through scripts/qemu-smoke.sh as before). scripts/check-public-ids.sh gains eight classes: vm-hostname, ec2-private-hostname, uname-hostname (a host-name-shaped token between Linux/Darwin and a version), gha-runner-hostname, user-home-path (C:\Users\<name>, /Users/<name>, /home/<name>; /home/runner and <placeholder> allowed), wsl-user (wsl … -u <name>; root and <placeholder> allowed), uuid, and machine-name. Machine names are matched by SHA-256 of the lower-cased token, so the guard does not spell them out. That keeps them out of plain-text search; a hash of a short name can be brute-forced, so it is not secrecy. On a hit the guard still prints only path:line class. --self-test now plants a synthetic token for all 15 classes (including a runtime-hashed fake machine name in two cases) plus a line of allowed forms that must not be flagged; marker public-ids: self-test ok classes=15.

Counts (by class, no values)

Guard of this branch run on unmodified main af5a9d1 (distinct path:line class pairs) and a plain grep for occurrences:

ClassBefore (lines)Before (occurrences)After
machine-name (two workstation names)129148 (88 + 60)0
vm-hostname770
ec2-private-hostname110
uname-hostname (overlaps the two rows above and below)550
gha-runner-hostname340
uuid (machine ID, runner host)220
machine-ID 8-hex prefix (not a guard class; grep)440
hostname-printing code2 scripts20

New guard on main: FAIL hits=151. On this branch: public-ids: ok … hits=0 (bash and dash).

Limits (non-claims)

  • Shapes and a short hashed name list, not DLP. A new machine name needs its hash added. Short role-like words are not flagged.
  • Git history, PR descriptions, issue comments and CI logs are not scanned or changed.
  • Git author metadata and the GitHub owner name in repo URLs stay (public and intended).
  • Not a “secure” or “compliant” claim.

ADR-102 — SMP slice S2: secondary core MMU on, own stack and vectors, PAN/SPAN, console lock

  • Status: Accepted for implementation as an opt-in probe (sponsor go for S2, 2026-10-13 about 22:45 CEST, relayed by DSO). Changes no claim.
  • Date: 2026-10-13
  • Follows: ADR-100 (plan + S1). Implements its slice S2, with the deviations listed below.
  • Base: main at a6cb41e (#154 merged).
  • _start (0x4008_0000) untouched. The secondary still enters at _secondary_entry in the boot-stub page.
  • $0: one local machine plus the existing GitHub Actions checks. No cloud agent.

Scope (from the sponsor go)

Bring core 1 up properly: MMU on with the shared kernel tables, its own stack, its own VBAR_EL1 vectors, PAN/SPAN set the same way as core 0 (ADR-080 / ADR-098), and a console lock so the two cores do not interleave UART output. A fail-closed probe proves it; negative builds must be caught.

ADR-091 revision 4 is unchanged: “EL0 isolated” stays scoped to -cpu cortex-a76, TCG, one core. No SMP safety claim of any kind.

Decision

All of it stays behind the opt-in Cargo feature smp2-probe (default build: no src/smp.rs code; the default smoke still fails on any smp: line).

  1. Parameter block. Core 0 fills a 64-byte-aligned block with the values it is running on: MAIR_EL1, TCR_EL1, the live TTBR0_EL1 (kernel root) and TTBR1_EL1 (shared kernel high map), and SCTLR_EL1 with SPAN forced back to its reset value 1 (so core 1 must clear it itself). Plus the high-VA top of a dedicated 16 KiB secondary stack, the high VA of cpu1_main, and the block’s own high VA. Core 0 cleans the block to the point of coherency (dc civac), because core 1 reads it with the MMU off.
  2. _secondary_entry (MMU off). Unchanged S1 part: raw MPIDR line, MPIDR stored to the block’s first word, dsb sy; sev. New: program MAIR/TCR/TTBR0/TTBR1, tlbi vmalle1 (local; core 1’s TLB has seen nothing), write SCTLR (MMU, caches, WXN on), switch sp to the secondary stack, branch to cpu1_main at the TTBR1 alias. The instruction after the SCTLR write runs from the boot-stub page, which stays identity-mapped executable in the kernel TTBR0 (the same exception _start already relies on).
  3. cpu1_main (MMU on, high alias, own stack). Installs its own 2 KiB vector table ctos_cpu1_vectors (not core 0’s, so no exception on core 1 can land on core 0’s exception / fatal stacks, ADR-100 R4). Clears SCTLR_EL1.SPAN, sets PSTATE.PAN. Reads back SCTLR, PSTATE.PAN, VBAR_EL1, sp and pc and prints them.
  4. Deliberate exception. Only if MMU, VBAR and PAN read back correctly. Core 1 clears PAN, executes brk #0x5102, and its own current-EL SPx sync handler (assembly, on core 1’s stack) records ESR_EL1, ELR_EL1, PSTATE.PAN at entry and an entry count, steps ELR past the brk and erets. Core 1 then sets PAN again. Pass requires: one entry, ESR = 0xf2005102 (EC=0x3C BRK, IL=1, imm=0x5102), ELR = the brk address, PAN=1 inside the handler (SPAN=0 sets it on entry, the ADR-098 property), PAN=1 after. Any other vector slot records which slot fired and parks the core, so core 0 times out and reports a miss.
  5. Console lock. After its raw MMU-off line, every smp: line from either core goes through the existing uart::UART spin lock, held for one whole line. Core 0 also stops using raw writes once core 1 may be printing (smp: done included). After core 1’s checks, core 1 waits for a go word, then both cores print four smp: tick cpuN i lines at the same time.
  6. Verdict by core 0. Core 0 waits up to 2 s for core 1’s verdict word, re-reads what core 1’s handler recorded, and prints smp: ok cpu1 s2 mmu-on pan=1 span=0 vbar=own exc=handled console=locked only if the verdict is OK, no stray vector fired, four ticks were counted and the recorded ESR / count match. Otherwise smp: cpu1 s2 missed verdict=… bad=… ticks=….

Negative builds (each implies smp2-probe; checker must fail and the miss line must appear)

FeatureWhat it breaks on core 1Caught by
smp2-neg-panskips msr pan, #1smp: cpu1 pan lost (pstate pan=0), verdict 3, no exception taken
smp2-neg-spanskips the SPAN clearhandler sees PAN=0: smp: cpu1 pan lost (handler pan=0 span=1), verdict 4
smp2-neg-vbarinstalls core 0’s vectors instead of its ownsmp: cpu1 vbar mismatch, verdict 5; the exception is not taken (would hit core 0’s stacks)

Plus the S1 negative: the same S2 ELF under -smp 1 (rc=-2, smp: cpu1 missed seen=0x0).

Checker (scripts/qemu-smoke.sh, smp2_check)

Requires every S1/S2 marker line exactly, each of the eight tick lines exactly once, rejects any smp: line that is not one whole known line (a torn or interleaved line fails closed), and rejects any miss line. CTOS_SMP2_ONLY=1 (the existing non-required CI job) runs: positive -smp 2, -smp 1 negative, then the three negative builds. The job name and required checks are unchanged; the job stays not required.

Deviations from ADR-100’s S2 row

  • PerCpu via TPIDR_EL1 is not done here. Core 1 does not use TPIDR_EL1 (it never enters the EL0 trampoline). Moved to S4, where the per-CPU EL0 context and expect flags need it.
  • The planned marker smp: cpu1 mmu-on pan=1 span=0 vbar=high became the more specific smp: cpu1 regs … + smp: ok cpu1 s2 … lines; the planned single negative (“skip SPAN clear → pan lost”) is smp2-neg-span, plus two more.
  • S1’s smp: ok cpu1 … parked became smp: ok cpu1 … up (core 1 no longer parks MMU off).

What the probe proves (QEMU virt, TCG, -cpu cortex-a76, -smp 2)

  • Core 1 runs with SCTLR_EL1.M=1, from the TTBR1 high alias, on its own stack, with the same MAIR/TCR/TTBR0/TTBR1 values as core 0 at that moment.
  • Core 1’s VBAR_EL1 is its own table; one synchronous exception taken on core 1 was handled by that table on core 1 and returned.
  • Core 1 has PSTATE.PAN=1 and SCTLR_EL1.SPAN=0; PAN was set by hardware on exception entry. Each of the three settings, when skipped, is caught.
  • With the lock, the observed output lines are whole on this run.

What it does not prove

  • Not SMP support, not multi-core safety, not “EL0 isolated” on two cores. Core 1 never runs EL0, the scheduler, the heap, page-table builders, drivers, the GIC or the timer.
  • Nothing about TLB coherence between cores (S3), ASIDs (S4), concurrent page-table writes (R1), or that the tables core 1 sees stay valid when core 0 changes them.
  • The console lock is not shown to be necessary: there is no deterministic negative for interleaving (it depends on host thread timing), so only the positive “lines are whole” is checked. Fatal / raw paths still bypass the lock (ADR-100 R10).
  • Core 1’s stack has no guard page. Core 1’s vectors handle exactly one synchronous slot; IRQ/FIQ/SError on core 1 park it.
  • No claim about real hardware, KVM, other -cpu models, more than two cores, or other PSCI firmware.

Proposed S3 (for sponsor go; not started)

Type-enforced TABLES (tables and bump counters inside the Mutex payload, IRQs masked while held) and Inner-Shareable TLB maintenance (tlbi …is + dsb ish; isb) at every unmap / downgrade / tear site, with a cross-core probe: core 1 (MMU on, as in S2) reads a probe page; core 0 unmaps it with the broadcast form; core 1 must fault on its own vectors (smp: xtlb unmap cpu0 → cpu1 fault). Negative build local-tlbi-leak-probe uses the old local tlbi and must report smp: xtlb leak (core 1 still reads the page through its stale TLB entry). Still no EL0 on core 1; ADR-091 rev 4 unchanged.

Consequences

  • Default kernel and smoke unchanged (no smp: line; required checks unchanged).
  • The opt-in CI job now builds four ELFs (positive + three negatives); it stays within its 30-minute timeout.

ADR-103 — SMP slice S3: type-enforced page-table lock, broadcast TLB invalidation, cross-core unmap probe

  • Status: Accepted for implementation (sponsor go for S3, 2026-10-13 about 23:37 CEST, effective on the #156 merge at 23:38 CEST; relayed by DSO). Changes no claim.
  • Date: 2026-10-13
  • Follows: ADR-100 (plan, R1/R2), ADR-102 (S2). Implements ADR-100’s slice S3 exactly as proposed in ADR-102.
  • Base: main at bb3f0ee (#156 merged).
  • _start untouched; the secondary still enters at _secondary_entry. Core 1 still runs no EL0.
  • $0: the Linux box plus the existing GitHub Actions checks. No cloud agent.

ADR-091 revision 4 is unchanged (one core). No SMP safety claim.

Decision

1. TABLES is enforced by the type system (R1)

  • All 19 page-table statics (L1…L3_HIGH_RAM and the three bump counters) move into a private module paging::tables. Nothing outside it can name them.
  • TABLES is now Mutex<Token>; Token has a private field, so only that module can build one. lock_tables() masks IRQs (DAIF.I), then takes the spin lock, and returns a TablesGuard that restores DAIF on drop.
  • Every function that hands out a writable slot, allocates an L3, clears a root or writes a descriptor takes &mut TablesGuard: the 13 slot accessors, 3 allocators, 5 clear helpers, l3_write, and (because they call those) fill_ram_wx, fill_user_map, install_stack_guards, clone_ram_tables_for_high, l3_pa_for_ram_block, l3_pa_for_user_block, unmap_identity_page, unmap_user_ram_page, set_identity_l3_writable, reach_plant_slot. Forgetting the lock is now a compile error, not a convention. 45 lock sites.
  • Taking a table’s physical address needs no guard (it writes nothing).

Limits: reads still go through desc_at / walkers without the lock (racy reads are only a problem once another core writes; core 1 writes no tables). A slot pointer returned under the guard is a raw pointer; it is used only inside the guard’s scope by reading, not enforced by a lifetime. Code outside paging.rs (for example frame allocation) is not covered.

2. Broadcast TLB invalidation at every unmap / tear site (R2)

All TLB maintenance in paging.rs goes through two helpers (24 call sites: every unmap, permission downgrade, identity tear, map-window change and the boot tlbi_all). Both now do dsb ishst → tlbi vmalle1is / tlbi vaae1is → dsb ish → isb. The dsb ishst before the TLBI is new: it makes the descriptor write visible to other cores’ table walkers before the invalidate.

The default single-core build therefore runs the Inner-Shareable forms. On one core they act like the local forms; the default smoke and every existing probe pass unchanged. Not changed: switch_ttbr0_no_tlbi (ADR-096 t2t “no TLBI” by design, ASID-tagged) and core 1’s own boot-time tlbi vmalle1 in _secondary_entry (its TLB has never been visible to anyone).

3. Cross-core unmap probe (opt-in smp2-probe, after S2 passes)

  1. Core 0 writes a magic word to a dedicated 4 KiB kernel page and maps it in the shared kernel TTBR0 map window (paging::map_page, global, under the guard).
  2. Core 1 reads it through that VA (its TLB now holds the translation) and signals.
  3. Core 0 unmaps it (paging::unmap_page: descriptor cleared under the guard, broadcast TLBI) and signals.
  4. Core 1 executes one ldr from the same VA. Its own sync handler (S2 vectors) records ESR / ELR / FAR and skips the load.
  5. Pass: exactly one new exception, EC=0x25 (data abort, current EL), DFSC a translation fault, FAR = the VA, the load’s register still holds its sentinel. Core 1 prints smp: xtlb cpu1 fault ec=0x25 dfsc=trans-l3 far=window own-vectors; core 0 prints smp: xtlb unmap cpu0 → cpu1 fault. If core 1 instead reads the old magic without a fault: smp: xtlb leak.

4. Negative build local-tlbi-leak-probe

Same as smp2-probe but the two helpers keep the old local-only forms (tlbi vmalle1, tlbi vaae1). It must print smp: xtlb leak and the checker must fail. The smoke also still runs the S1/S2 negatives (-smp 1, smp2-neg-pan, smp2-neg-span, smp2-neg-vbar).

TCG timing (the honesty question)

Under TCG a missing broadcast does not have to leave a stale entry: QEMU’s per-vCPU soft TLB could be evicted or flushed for other reasons. Observed on the Linux box (QEMU TCG, -cpu cortex-a76, -smp 2), 2026-10-13: the leak build printed smp: xtlb leak on 27 of 27 runs and the broadcast build printed smp: xtlb unmap cpu0 → cpu1 fault on 27 of 27 runs. So on this QEMU the negative is reliable, and the smoke treats it as required.

This is a property of QEMU’s TLB model, not of any CPU. If a future QEMU stops reproducing the leak, the smoke will fail loudly (the negative will not print smp: xtlb leak) rather than pass silently. The fallback then is to show the leak on KVM on real arm64 (the ADR-099 pilot path, with sponsor approval), or to have core 1 read the page in a tight loop while core 0 unmaps it and report whether any read succeeded after the unmap completed.

What this proves

  • No code can write page tables, allocate tables or clear roots without holding TABLES with IRQs masked: the compiler rejects it.
  • On QEMU TCG -smp 2: after core 0 unmaps a page with the broadcast TLBI, core 1 faults on its next access (its own vectors handle it); with local-only TLBI, core 1 keeps reading the unmapped page. So the old local TLBI was a real cross-core hole here, and the broadcast form closes it for this case.

What this does not prove

  • Not SMP support or safety; not “EL0 isolated” on two cores. Core 1 runs no EL0, scheduler, heap, drivers or interrupts, and writes no page tables.
  • One VA, one page, one unmap direction (core 0 → core 1). Not every tear site is exercised cross-core; the others use the same helper.
  • Nothing about ASIDs / non-global entries (S4), lock-free reads of tables, or ordering on real hardware or KVM.
  • Nothing about QEMU versions other than the one on the box and the CI runner.

Proposed S4 (for sponsor go; not started)

ASID allocator with a generation counter (roll-over: tlbi vmalle1is + generation bump); a PerCpu block via TPIDR_EL1 (deferred from S2) holding each core’s saved kernel TTBR0, EL0 context and expected-fault flags (R4/R5); then run EL0 on core 1 for the first time, kernel-driven, one task per core, and re-run each isolation probe on core 1 with its marker tagged cpu=1. The existing leak builds run on core 1 must still be caught. Still no scheduler across cores (S5).

Consequences

  • Default kernel: page-table code now needs the guard (compile-time); TLBI uses the IS forms; behaviour and smoke output unchanged on one core.
  • The opt-in CI job builds five ELFs (positive + four negatives); it stays not required and within its 30-minute timeout.

ADR-104 — SMP slice S4: PerCpu via TPIDR_EL1, ASID allocator with generations, first EL0 on core 1

  • Status: Accepted for implementation (sponsor “Go on S4”, relayed by DSO on 2026-10-19 at 17:31 CEST; the stop-all hold is lifted for this item only). Changes no claim.
  • Date: 2026-10-19
  • Follows: ADR-100 (plan, R3/R4/R5), ADR-102 (S2), ADR-103 (S3). Implements the “Proposed S4” section of ADR-103 exactly.
  • Base: main at 3234668 (#158 merged).
  • _start untouched; core 1 still enters at _secondary_entry. The default single-core build keeps its behaviour and its full smoke.
  • $0: the Linux box plus the existing GitHub Actions checks. No cloud agent. No workflow file changed.

ADR-091 revision 4 is unchanged (one core). No multi-core safety claim. Any multi-core sentence is S5 and needs a separate sponsor accept.

Decision

1. PerCpu via TPIDR_EL1 (R4, R5)

  • New src/percpu.rs: one 64-byte-aligned PerCpu block per core (NCPU = 2): saved kernel TTBR0 at offset 0, core index, last ASID generation seen. TPIDR_EL1 holds the block’s TTBR1 high VA.
  • TPIDR_EL1 used to hold the saved kernel TTBR0 directly. The four lower-EL trampolines (sync, IRQ, FIQ, SError) now do mrs x2, tpidr_el1; cbz; ldr x2, [x2]; cbz; msr ttbr0_el1, x2 (one extra load).
  • Core 0 installs its block in kernel_main_high after every identity tear, before the first EL0 entry. Installing it earlier failed: the ADR-037 pointer rewrite covers [__data_start, __kernel_end), and the saved TTBR0 is the kernel L1’s PA, which lies in .bss and looks like an identity pointer, so it was rewritten to a high VA. The boot then faulted on the first EL0 return. Until the install, TPIDR_EL1 is 0 and the trampolines skip the restore (no EL0 runs before then).
  • percpu::Banked<T> gives each core its own copy of a static and derefs to the reading core’s copy. Banked: the 11 EL0-context statics in el0.rs (ENTERED, ACTIVE, USER_PC, USER_SP, USER_TTBR, KIND, …) and the 60 expected-fault / caught / EL0-return statics in exception.rs (EXPECT_*, *_CAUGHT, EL0_CONT, EL0_KSP, …). Code that uses them is unchanged. On one core only slot 0 is used.

2. ASID allocator with a generation counter (R3), smp2-probe only

New src/asidgen.rs. 8-bit ASIDs; 0 (kernel), 1 and 2 (the fixed ADR-013 / ADR-096 probe ASIDs) are never issued, so 3..=255 (253 per generation). A task holds (generation, asid). Allocation runs under a spin lock. ASIDs are never freed one at a time. A retired task’s tables may be torn down without a TLBI, because its ASID is not reissued until the next rollover.

Rollover (bitmap exhausted): generation + 1, clear the bitmap, keep each core’s running ASID reserved into the new generation (so a task live on another core never shares its ASID with a new task), then dsb ishst; tlbi vmalle1is; dsb ish; isb before any ASID is reissued. switch_to revalidates a slot: same generation, keep it; older generation but still this core’s reserved running ASID, keep it under the new generation; otherwise allocate a fresh one. Each core records the generation it last switched under in PerCpu::asid_gen_seen.

Not wired into the default build: the default single-core kernel still uses the fixed ASIDs 1 and 2.

3. Core 1 in the kernel’s normal mode; the first EL0 on core 1 (smp2-probe)

After S1–S3 pass, core 0 runs vfs::init, virtio::init, init_net, fat::init, sched::init and gic::init (distributor and its own CPU interface; no timer on core 0), then signals core 1 and waits without printing. Core 1:

  1. installs PerCpu[1] in TPIDR_EL1;
  2. takes core 0’s SCTLR_EL1.SPAN (so a build that undoes the PAN fix behaves on core 1 as on core 0);
  3. sets VBAR_EL1 to the kernel vector table (no longer its S2 table), SP_EL1 to its own 16 KiB exception stack, and SP_EL0 to its own 64 KiB thread stack; then SPSel = 0 (EL1t, like core 0’s kernel threads);
  4. enables its own banked GIC CPU interface, timer PPI and CNTP (timer::init);
  5. runs the single-core boot’s EL0 isolation probes, in the same order, on core 1: el0 (SVC, NX, read / write faults, standing + IRQ + FIQ + SError-arm), svc, sys-ptr, libctos, loader (also arms the live reach and PAN checks), pan-held, asid, t2t, xo, ttbr1, ident (identity inventory), el0-reach, pan. After each one it prints smp: cpu1 probe <name> ok cpu=1 or … missed cpu=1; the probes print their own markers as on one core.

One task per core: core 0 runs no EL0 in this build. The standing loader task (observe_standing_task) and the non-isolation samples (FAT / net demos) are not part of S4.

4. ASID-generation probes (both cores)

  1. Concurrent allocation: both cores allocate 150 ASIDs at the same time (300 > 253, so at least one rollover). Core 0 checks that every (generation, asid) pair is unique, every ASID is in 3..=255, and that at least one rollover happened: smp: asidgen alloc cpu0+cpu1 n=300 unique range=3..=255 gens=2 rollovers=1.
  2. Reissue after rollover: core 0 maps a magic page (non-global) in the kernel map window. On core 1, task T1 gets ASID x (generation N), switches TTBR0 to the kernel L1 tagged x, and reads the page (core 1’s TLB now holds an x-tagged entry). T1 exits. Core 0 then clears the descriptor without any TLBI (the generation rule) and allocates until x is issued again in a later generation. That forces a rollover, which flushes with tlbi vmalle1is. Core 1 runs task T2 with that slot (same TTBR0 value) and does one ldr from the page. Pass: an EL1 translation fault on the kernel vectors, caught by a per-core expected-fault flag: smp: asidgen cpu1 fault ec=0x25 dfsc=trans-l3 gen-seen=3 cpu=1, then smp: asidgen reissue cpu0 rollover → cpu1 fault.
  3. Core 0 prints smp: ok cpu1 s4 percpu=tpidr el0=cpu1 probes=13 asidgen=unique,rollover-is only if core 1 entered kernel mode, all 13 probes passed on core 1, the allocation was unique, and the reissue faulted.

5. Negative build (one ELF, all leaks on core 1)

smp2-probe + the new asid-rollover-local-probe (rollover flush is the local tlbi vmalle1) + the existing leak features: inv-leak-probe, reach-leak-probe, write-leak-probe, sysptr-leak-probe, t2t-leak-probe, xo-leak-probe, pan-keep-leak-probe. The smoke runs the same log checks as the single-core leak builds (factored into inv_leak_check / el0_leak_check; the only change is that the xo miss line is core 1’s smp: cpu1 probe xo missed cpu=1 instead of main.rs’s xo: probe missed). It also requires:

  • smp: cpu1 probe {el0,sys-ptr,pan-held,t2t,xo,ident,el0-reach} missed cpu=1, smp: cpu1 probes missed failed=7 n=13;
  • … reissued (flush=local), smp: asidgen cpu1 read the retired page under the reissued asid cpu=1, smp: asidgen leak;
  • no smp: ok cpu1 s4, and the S1–S4 checker must fail.

6. Smoke

  • scripts/qemu-serial-inject.py: new opt-in CTOS_QEMU_SMP (adds -smp N) and CTOS_QEMU_STOP_AT (stop at a serial string). Both absent means the default runs exactly as before.
  • smp2_check additionally requires all S4 lines, the 13 probe tags exactly once each, and the probes’ own ok markers. The console-lock allow-list covers the new smp: lines.
  • The S4 positive run and the S4 leak run use the inject script (virtio-blk FAT image, virtio-net) with -smp 2. The -smp 1, smp2-neg-* and local-tlbi-leak-probe runs are unchanged (they stop before S4).
  • The opt-in CI job (CTOS_SMP2_ONLY=1) builds one more ELF (six in total). Every cargo build already publishes the app ELFs, so the workflow file is unchanged. Its comment “Does not run EL0 on the secondary” is now stale; fixing it needs a separate workflow-file push and is left for later.

TCG timing (the honesty question)

  • QEMU TCG does not model ASID-tagged TLB retention across an ASID change. On a TTBR0 write that changes the ASID, QEMU flushes that vCPU’s soft TLB. So under TCG the existing asid: ok (ADR-013) and the ASID-switch part of t2t pass trivially. For the same reason the reissue probe keeps core 1 on one TTBR0 value from T1 through T2 (T1 “exits” without switching back to ASID 0). On hardware the stale entry would also survive an ASID switch, so the probe still models the real hazard, but it does not show ASID tagging itself.
  • What the probe does show under TCG: the only thing between core 1’s cached entry and its next access is core 0’s rollover flush. With tlbi vmalle1is core 1 faults; with local tlbi vmalle1 core 1 reads the retired page.
  • Observed on the Linux box, QEMU 10.0.13 TCG, -cpu cortex-a76, -smp 2, 2026-10-19: the S4 positive build printed smp: ok cpu1 s4 … (13/13 probes ok on core 1, reissue fault) on 20 of 20 runs. The S4 leak build printed smp: asidgen leak and smp: cpu1 probes missed failed=7 n=13 on 20 of 20 runs. The S3 local-tlbi-leak-probe still printed smp: xtlb leak on 20 of 20 runs on this QEMU (S3’s 27/27 were measured on an earlier QEMU on the box). The full CTOS_SMP2_SMOKE=1 ./scripts/qemu-smoke.sh passed (default smoke, single-core leak builds, S1–S4 positive and all negatives).
  • If a future QEMU stops reproducing a leak, the negative fails loudly instead of passing silently. The fallback is the same as in ADR-103: KVM on real arm64 (ADR-099 pilot path, sponsor approval needed).

What this proves

  • Two cores each have their own TPIDR_EL1 → PerCpu block, own exception and thread stacks, and own copies of the EL0 context and expected-fault flags. The EL0 trampolines restore each core’s own kernel TTBR0 through it.
  • On QEMU TCG -smp 2, every EL0 isolation probe of the single-core boot also passes when run on core 1 (kernel-driven EL0, IRQ / FIQ from core 1’s own timer via the lower-EL trampolines). Every existing leak build is still caught there, with its ok marker withheld.
  • The ASID allocator hands out unique (generation, asid) pairs to two cores at once across a rollover. A rollover’s broadcast flush removes core 1’s stale entry for a reissued ASID; a local flush does not.

What this does not prove

  • Not SMP support or safety; not “EL0 isolated” on two cores. ADR-091 rev 4 stays one core.
  • The cores run probes one at a time. Core 0 waits while core 1 runs EL0. The globals that are not banked are the result / “ready” flags core 0 publishes at boot and the probe bookkeeping in syscall.rs, pan.rs, loader.rs, reach.rs, t2t.rs, xo.rs, teardown.rs. So are the shared user tables (user_ttbr0, ASID-B), the shared EL0 probe page, and the scheduler. Running EL0 on both cores at once would race on these.
  • The fatal / nested-exception stack is still one linker stack shared by both cores. Core 1’s stacks have no guard pages. The guard, wx, ro, sched, standing-task and sample probes were not run on core 1.
  • The allocator is not used by the default build or by the probes’ fixed ASIDs 1 and 2. There is no scheduler, so switch_to is only exercised by the probe. The “reserved across rollover” path is not exercised by a probe (no task runs on core 1 during core 0’s rollovers in the reissue probe).
  • One reissue scenario, one direction (core 0 rolls over, core 1 faults). Nothing about real hardware, KVM or other QEMU versions.
  • The SError-while-standing probe on core 1 parks (el0: serror-park), as on one core with this QEMU: inject-nmi is not available on stock QEMU. The taken-SError path was not shown on core 1.

Proposed S5 (for sponsor go; not started)

Done as ADR-105 (sponsor go 2026-10-19 21:26 CEST).

Scheduler and concurrency: a run-queue lock and per-core current-task pointers (R6); EL0 tasks on both cores at the same time, each with an allocator ASID and its own user tables (no shared EL0 probe page); bank or retire the remaining global probe flags; a per-core fatal stack and guard pages for core 1; a probe that runs t2t / reach / xo on both cores at once with a rollover in between, plus a negative that drops the reserved-running-ASID rule. Then, docs only and only with a written sponsor accept, a proposal for the multi-core sentence (scope, the TCG limits above, and what KVM evidence it would need).

Consequences

  • Default kernel: the TPIDR indirection costs one extra load per lower-EL exception. The 71 banked statics double in size. Behaviour and the full default smoke (including the single-core leak builds) are unchanged.
  • The opt-in CI job builds six ELFs (the positive, also run under -smp 1; three S2 negatives; the S3 leak; the S4 leak). It stays not required, within its 30-minute timeout.

ADR-105 — SMP slice S5: concurrent EL0 on both cores, run-queue lock, reserved ASID

  • Status: Accepted for implementation (sponsor go for S5, relayed by DSO on 2026-10-19 at 21:26 CEST). Changes no claim.
  • Date: 2026-10-19
  • Follows: ADR-100 (plan, R3/R4/R5/R6), ADR-104 (S4). Implements the “Proposed S5” section of ADR-104.
  • Base: branch smp/s4-asid-percpu-el0 (PR #160 / ADR-104), or main once #160 has merged.
  • _start untouched. The default single-core build keeps its behaviour and its full smoke.
  • $0: the Linux box plus the existing GitHub Actions checks. No cloud agent.

ADR-091 revision 4 is unchanged (one core). A separate docs-only PR proposes a multi-core claim sentence; it stays Proposed until the sponsor writes an accept.

Decision

1. Run-queue lock and per-core current task (R6)

New src/runq.rs (smp2-probe only): a spin-locked table of up to 8 tasks. pick returns the next runnable unowned task, marks this core as its owner and stores id+1 in PerCpu::current. put returns it (runnable or finished); an owner mismatch increments a violation counter. Round-robin, no preemption: each trip is one EL0 entry to SYS_EXIT.

2. Per-task address spaces

New paging::TaskSpace (smp2-probe only): three fresh frames (L1→L2→L3) mapping only the task’s own pages in the window — text (EL0 execute-only), data (EL0 RW), stack (EL0 RW), all nG. No kernel mapping in TTBR0: vectors, exception stacks and TPIDR_EL1→PerCpu are TTBR1. Writes take TABLES. task_space_walk collects every EL0-reachable leaf for the reach / t2t / xo check.

3. Remaining shared flags banked; per-core fatal stack; guard pages

  • Banked (on top of S4’s 71): the run-local SVC result flags in syscall.rs (18), the live PAN / xo / reach flags in pan.rs, xo.rs, reach.rs (3+2+3). Boot-published IDs (PAN_ID, …) stay global.
  • Core 1 gets its own fatal stack (8 KiB + 4 KiB guard). fatal_enter reads PerCpu::fatal_top (offset 24); 0 = the linker fatal stack (core 0 / default). On a secondary the fatal handler only records the stack check (no raw UART) so it cannot tear core 0’s locked lines.
  • Core 1’s thread / exception / fatal stacks each have a 4 KiB guard page, punched at boot with the linker-stack guards (before the TTBR1 RAM clone), so the high aliases have the same holes.

4. Concurrent EL0 + mid-run ASID rollover (smp2-probe, after S4)

  1. Core 0 builds two tasks (A, B), each with its own TaskSpace and an allocator ASID, checks reach/t2t/xo on both spaces and that core 1’s guard pages are unmapped, then signals.
  2. Both cores pick from the run queue and drive EL0 (s5_run_once: switch ASID, ERET, wait for SYS_EXIT, check the data magic).
  3. Core 1’s first pick holds its ASID (still marked running) while core 0 forces at least one ASID-generation rollover. The reserved-running-ASID rule (ADR-104) must keep core 1’s ASID out of the allocator. Core 1 then finishes its run and drains; core 0 drains the rest.
  4. Live xo: each task’s EL0 load of its own text must fault (permission L3).
  5. Core 1 triggers the nested-BRK fatal probe; it must land on its own fatal stack.
  6. Pass line: smp: ok cpu0+cpu1 s5 runq=locked asid=reserved concurrent=el0 reach,t2t,xo.

5. Negative build asid-reserve-drop-probe

Same as smp2-probe but the ASID rollover no longer keeps running ASIDs reserved (and switch_to will not keep an older generation’s running ASID). The probe must print smp: s5 asid reissued while running cpu=1 asid=… and must not print the S5 ok line; the checker must fail.

6. Smoke / CI

  • smp2_check requires the S5 lines. New S5 negative ELF in the opt-in smoke.
  • Workflow comment on the qemu-smp2 job updated (ADR-100..105; EL0 on the secondary is no longer denied). Job stays not required. Timeout stays 30 minutes.

TCG honesty

  • Same ASID-tagging limit as ADR-104: QEMU flushes a vCPU’s soft TLB on an ASID change. The reserved-ASID proof is that the allocator did not reissue core 1’s ASID during the hold (positive) / did reissue it (negative), not that a stale TLB entry survived a switch.
  • Concurrent EL0 on TCG is real (both vCPUs enter EL0 and take SVCs); scheduling quanta are TCG’s, not a wall-clock claim.
  • Observed on the Linux box, QEMU 10.0.13 TCG, -cpu cortex-a76, -smp 2, 2026-10-19: the S5 positive build printed smp: ok cpu0+cpu1 s5 … on 10 of 10 runs; asid-reserve-drop-probe printed smp: s5 asid reissued while running cpu=1 on 10 of 10 runs (no S5 ok line). Full CTOS_SMP2_SMOKE=1 smoke: qemu-smoke: ok (default smoke, single-core leaks, S1–S5 positive and all negatives).

What this proves

  • A locked run queue hands each task to one core at a time; owner mismatches stay at 0.
  • Two tasks with private tables and allocator ASIDs run EL0 on both cores in one session; each space has exactly the three expected EL0 leaves (xo text, RW data, RW stack) and neither data frame appears in the other’s walk; EL0 load of own text faults.
  • A mid-run ASID rollover keeps core 1’s running ASID reserved; dropping that rule reissues it and is caught.
  • Core 1’s fatal nested BRK runs on its own fatal stack; its three guard pages are unmapped (identity and high).

What this does not prove

  • Not a general-purpose SMP OS. No preemption, no IRQ-driven reschedule, no multi-priority queue, no user-facing API.
  • The default build still uses fixed ASIDs 1 and 2; the allocator and run queue are opt-in only.
  • Boot-published result flags, the shared user / ASID-B tables used by the single-core probes, virtio DMA rings and VFS locks are unchanged. The S5 tasks do not share the single-core EL0 probe page.
  • Nothing about real hardware, KVM or other QEMU versions. ADR-091 rev 4 stays one core.

Proposed next (not started)

  • Wire the allocator into the single-core loader path (retire fixed ASIDs 1/2 for real tasks).
  • IRQ-driven preemption across cores; per-core timer ticks counted separately.
  • Sponsor accept of the multi-core claim sentence (separate docs-only PR).

Consequences

  • Default kernel: core 1’s stacks add ~96 KiB to .bss in the opt-in build only (the Guarded statics live in smp.rs, which is compiled only with smp2-probe). Default build unchanged.
  • Opt-in CI job builds one more ELF (S5 reserve-drop negative). Still not required.

Fix-forward (CI)

The first CI run of the opt-in smp2 job failed: core 0’s unlocked el0: task-enter print tore core 1’s locked s5 cpu1 hold line. In the smp2-probe build the task-enter/active/exit/restored and svc: exit lines now go through the UART lock, one whole line at a time. The default build still uses the raw writer and is unchanged.

ADR-106 — Proposed multi-core claim sentence (docs only; not accepted)

  • Status: Proposed, not accepted; needs the sponsor’s written acceptance.
  • Date: 2026-10-19
  • Follows: ADR-091 rev 4 (the current accepted claim), ADR-100–ADR-105 (the SMP race story, opt-in probes only).
  • This PR changes no code and does not edit the ADR-091 rev 4 claim. Until the sponsor writes an accept, the accepted claim stays one core.

Proposed sentence

On QEMU TCG, -cpu cortex-a76, -smp 2, the opt-in smp2-probe build has demonstrated: (1) each core has its own TPIDR_EL1 → PerCpu block, exception / thread / fatal stacks and expected-fault flags; (2) broadcast TLBI (tlbi …is) removes a mapping from the other core’s TLB; (3) an ASID allocator with generations keeps a running core’s ASID reserved across a rollover (tlbi vmalle1is), and dropping that rule is caught; (4) two tasks with private TTBR0 tables and allocator ASIDs run EL0 concurrently on both cores, with a reach / task-to-task / execute-only check over both spaces. This is not a claim that “EL0 is isolated” on two cores in the ADR-091 sense: the default build is still one core; the probes are opt-in; TCG flushes a vCPU’s soft TLB on any ASID change (so ASID tagging itself is not shown); there is no preemptive cross-core scheduler; and no KVM or real-hardware evidence is attached.

Why this shape

  • Tight to what ADR-102..105 actually printed on the box and in CI.
  • Keeps ADR-091 rev 4 as the accepted product claim until a written accept replaces or extends it.
  • Names the TCG / ASID-tagging limit explicitly so a future reader cannot treat the rollover proof as a hardware ASID-tag proof.

Accept path

A written sponsor accept of this sentence (or a tightened revision of it). On accept: move this ADR to Accepted, add a one-line pointer from ADR-091 (without rewriting rev 4’s one-core sentence unless the accept says to), and add a ledger row. No code change is required for the accept itself.

ADR-107 — Track A — FAT32 read path beyond FAT16 (second virtio-blk)

  • Status: Accepted (runtime Verified only when this tip’s qemu-smoke prints fat32: ok and the broken-chain negative prints fat32: fail reason=bad-chain; otherwise Documented)
  • Date: 2026-10-23

Context

ADR-028 put FAT16 on virtio-blk behind the thin VFS, and later ADRs grew write, readdir, unlink, grow, mkdir and rmdir on it. fat.rs rejects any volume with 65525 or more clusters, so a FAT32 volume (the kind mkfs.fat makes for anything past a few hundred MiB) cannot be read at all. The sponsor asked for FAT32 support on the existing virtio-blk path, tested in QEMU against an image built by mkfs.fat -F 32, with a fail-closed probe and a negative.

FAT32 differs from FAT16 in three places that matter here:

  1. BPB. BPB_RootEntCnt and BPB_FATSz16 are 0; the FAT size is the 32-bit BPB_FATSz32, and the root directory’s first cluster is BPB_RootClus. BPB_FSInfo names the FSInfo sector, which carries three signatures and a free-cluster hint.
  2. FAT table. Entries are 32 bits wide, of which the low 28 are used. End-of-chain is >= 0x0FFFFFF8; 0x0FFFFFF7 is a bad cluster.
  3. Root directory. There is no fixed root region. The root is an ordinary cluster chain.

Hypotheses

IDOptionNotes
H1 (chosen)A separate read-only FAT32 reader (src/fat32.rs) on a second virtio-blk device holding a mkfs.fat -F 32 image. The FAT16 disk, fat.rs and the VFS stay unchanged.Smallest honest step. FAT16 regressions are impossible by construction, and the FAT16 smoke still runs on the same boot.
H2 (rejected)Widen fat.rs (u16 clusters everywhere) to FAT32 with write support in one PR.Touches every FAT16 write path at once. Too large to review and test honestly in one step.
H3 (rejected)Append the FAT32 volume to the FAT16 image at a fixed LBA.The FAT16 host checks (mkfat16.py --check) pin the image size, and a magic offset is a hidden contract.
H4 (rejected)Hand-roll the FAT32 image in Python.The sponsor asked for mkfs.fat -F 32. A real tool also catches reader assumptions a home-made writer would share.

Decision

  1. Disk. QEMU gets a second -drive …,readonly=on + -device virtio-blk-device with target/fat32.img, listed before the FAT16 disk. QEMU fills virtio-mmio slots from the top, so the FAT16 disk stays the guest’s first-found blk device, and older kernels (the cross-update prior OS) keep using it. virtio::init2() programs the second transport in scan order with its own queue memory; it exposes read_sector2 only (no write).

  2. Image. scripts/mkfat32.sh runs mkfs.fat -F 32 -s 1 -S 512 on 34 MiB (about 68k clusters, above the 65525 floor), then uses mcopy to add:

    • PROBE32.TXT (fat32-hi\n);
    • BIG.BIN (3000 deterministic bytes, 6 clusters);
    • 20 small files, so the root directory needs more than one cluster;
    • LAST.TXT (fat32-root-chain\n), which therefore lands in a later root cluster.

    mtools allocates forward, so the script then moves BIG.BIN’s last three clusters to a free run near cluster 40000 and relinks them in every FAT copy. fsck.fat -n must pass on the result (a host check, not a guest probe).

  3. Reader (fat32.rs). It does the following:

    • Mount checks the 0x55AA signature, bps == 512, a power-of-two spc, and RootEntCnt == 0 and FATSz16 == 0 (otherwise NotFat32). It requires a cluster count of at least 65525 (otherwise NotFat32), checks that the FAT fits every cluster index and that the root cluster is in range, and checks the FSInfo lead/struct/trail signatures plus a free-count bound.
    • Chain walk masks entries to 28 bits and ends at >= 0x0FFFFFF8. A free (0), reserved (1), bad (0x0FFFFFF7) or out-of-range link is BadLink. The walk is bounded by the cluster count, so a cycle is Cycle.
    • File read requires the chain length to equal ceil(size / cluster) exactly (otherwise SizeMismatch).
    • Directory scan walks the root chain, stops at the first 0x00 entry, and skips deleted, LFN and volume-label entries. It matches 8.3 names only.
  4. Probe (boot path, after the FAT16 probe). It prints:

    • fat32: mount clusters=N root=R fsinfo=ok
    • fat32: read /probe32.txt bytes=9
    • fat32: chain /big.bin clusters=6 jumps=J bytes=3000, where J ≥ 1 non-adjacent links and the content matches byte for byte
    • fat32: root-chain clusters=K last=found (K ≥ 2)
    • fat32: ok

    Any error prints fat32: fail reason=<reason> and the kernel prints fat32: probe missed.

  5. Smoke (fail closed). qemu-smoke.sh requires every marker above, plus clusters >= 65525, jumps >= 1 and root clusters >= 2, and rejects any fat32: fail or probe missed. Negative: a second boot of the same kernel with a copy of the image whose FAT[root] is set to free (mkfat32.sh --corrupt) must print fat32: fail reason=bad-chain, must not print fat32: ok, and must still print FAT16 fat: ok.

  6. cargo test (in QEMU). The tests are:

    • in-memory FAT32 images covering a good read through a two-cluster root, a free link, a cycle, bad and out-of-range links, a chain longer than the size, bad FSInfo, a FAT16-sized cluster count and a non-zero fixed root;
    • the real-disk probe;
    • a check that the FAT16 primary disk is rejected by the FAT32 mount (NotFat32).
  7. CI. dosfstools and mtools are added to the smoke jobs’ apt lines and to the Dockerfile.

Not claimed

  • No FAT32 write, create, delete or grow. No VFS mount of the FAT32 volume. No LFN decoding. No subdirectory walk on FAT32 (only the root chain). No FSInfo update. No FAT mirror comparison (copy 0 is read). No exFAT.
  • Not a performance claim. Not a real-hardware claim (QEMU virt only).
  • FAT16 behaviour is unchanged. The FAT16 VFS is still the only mounted FAT.

Consequences

  • QEMU runs (smoke, cargo test, cross-update) now need mkfs.fat, mcopy and fsck.fat on the host. mkfat32.sh fails loudly if they are missing.
  • A later ADR can mount FAT32 behind the VFS or add write support, starting from this reader.

ADR-108 — Taken-SError QEMU spike: what triggers it on TCG (stock and pinned)

  • Status: Accepted (spike / research). Sponsor go relayed by DSO on 2026-10-23 at 19:25 CEST (stop-all lifted for ctos only). One finding, one PR. No change to the kernel, _start, the default smoke or the ADR-091 claim.
  • Date: 2026-10-23
  • Base: main c420fba (includes #160, #161, #162).
  • Machine: amd64 workstation (bare-metal Ubuntu, TCG only; no KVM for an aarch64 guest). No cloud, no CloudAgent. Builds ran in an ubuntu:24.04 container on that host because the host lacks QEMU build deps and root.
  • Number: ADR-107 is taken by open PR #163, so this is ADR-108.

Question

Find a QEMU setup that can trigger and observe a taken SError, as groundwork for the “EL0 isolated” claim, and say exactly what is and isn’t shown today.

What is shown today (before this spike)

EvidenceWhereShowsDoes not show
B1 QEMU NMI pin smoke (taken SError) CI jobADR-083, ADR-090Pinned QEMU v10.0.0 + patch 0001; QMP inject-nmi → bare el0: serror while the full ctos image stands at EL0 on cpu0, -smp 1. Fail-closed (CTOS_REQUIRE_TAKEN_SERROR=1).Stock QEMU; cpu1 or any secondary; an SError source other than the repo’s own QEMU patch; hardware RAS.
B2-P (one-off KVM, Graviton3)ADR-085Historical run 4 at 159b178, reduced profile.Current boot path; repeatability; default image.
Default smoke on stock QEMUADR-053/081el0: serror-park: stock virt refuses inject-nmi (“machine does not provide NMIs”).Any taken SError.
SMP S4/S5ADR-104, ADR-105cpu1 runs EL0 probes.No SError probe is taken on cpu1; it parks on stock QEMU.

Options tried (exact versions, this session)

QEMUs:

  • stock A: distro qemu-system-aarch64 10.2.1 (1:10.2.1+ds-1ubuntu3.2) on the host.
  • stock B: Ubuntu 24.04 qemu-system-arm 8.2.2 (1:8.2.2+ds-0ubuntu1.18), the same version as the GHA runners.
  • upstream master: b6c4b7e (2026-10-23, VERSION 11.1.50), built from source.
  • pinned B1: v10.0.0 7c949c53 + research/qemu-nmi/0001-…patch, via ./scripts/build-qemu-nmi.sh.
RouteResult
QMP inject-nmi / HMP nmi on stock A, stock B, masterRefused: Error: machine does not provide NMIs. Master hw/arm/virt.c still has no TYPE_NMI.
Same on pinned B1Works, and targets a chosen vCPU: HMP nmi with QMP human-monitor-command cpu-index=1 raises the SError on cpu1 only.
gdbstub (-gdb, monitor nmi)It is the same HMP nmi (forwarded over qRcmd): refused on stock, taken on pinned. gdb has no SError injection of its own, and writing registers would be a fake. Rejected as a separate route.
Bad device access (EL0 load from unassigned PA 4 GiB)Synchronous external abort: vector 0x400, EC 0x24, DFSC 0x10. It is not an SError. Upstream target/arm/tcg/translate-a64.c says: “QEMU does not have a source of physical SErrors, so we are only concerned with virtual SErrors”; arm_cpu_do_transaction_failed raises a sync abort. Rejected (negative row kept).
Virtual SError (HCR_EL2.VSE + AMO, guest boots at EL2 with -M virt,virtualization=on)Works on every QEMU tried, including stock: taken at EL1 from EL0, vector VBAR_EL1+0x580, ESR_EL1 0xbe000000 (EC 0x2f), on cpu0 and on cpu1 (PSCI CPU_ON, -smp 2). Architectural, no QEMU patch.
RAS extension registers (ERX*) / FEAT_RAS error injectionNot tried in code: TCG has no physical SError source (see above), so there is nothing to drive. FEAT_NMI is an interrupt and does not count as an SError (ADR-082).
KVMNot available: the host is amd64.

Finding (one)

On TCG there are exactly two honest ways to get a taken SError: (a) the pinned B1 patch, which is physical and host-injected and can now target cpu1; and (b) the architectural virtual SError via HCR_EL2.VSE, which works on stock QEMU 8.2.2, 10.2.1 and master, on cpu0 and cpu1, but only if the guest owns EL2. Bad-PA accesses and the gdbstub add nothing new.

Evidence: research/serror-spike/. The matrix has 13 fail-closed rows per QEMU, 4 QEMUs, 52/52 ok. Log: research/serror-spike/2026-10-23-matrix-log.txt.

Negatives (all pass, on every QEMU):

  • VSE without AMO is not routed;
  • AMO without VSE gives no SError;
  • VSE with EL0 PSTATE.A set is masked;
  • with the pin, an nmi to cpu0 while cpu1 stands at EL0 does not reach cpu1, and an nmi with EL0 A-masked is not taken;
  • with stock QEMU, every inject row must show the refusal and no SError.

Decision

  1. Record the finding. Add the stand-alone spike image (research/serror-spike/vse), serror-matrix.py (fail-closed, exit 1 on any mismatch) and gdb-nmi.sh. No CI job is added, and the required checks are unchanged.
  2. No claim changes. The ADR-090 evidence class and the ADR-091 rev 4 sentence are untouched. This spike uses a stand-alone image, not the ctos kernel.
  3. Possible next steps, each needing a sponsor go:
    • N1 (pin, cpu1): extend B1 to -smp 2 with cpu-index=1 while ctos cpu1 stands at EL0 (S4/S5 path). Needs a cpu1 el0: serror arm/handler in ctos.
    • N2 (stock, virtual): a small EL2 shim, loaded with -device loader ahead of the kernel and not touching _start, that sets HCR_EL2.{RW,AMO}, ERETs to ctos at EL1, and sets VSE on an HVC from ctos’s serror-arm point. This would give a taken SError on stock QEMU with the full image. It is a virtual SError (EL2-injected), and its honesty wording must say so.

Honesty

Say: “On QEMU TCG, a taken lower-EL SError is produced only by (a) the repo’s pinned TYPE_NMI patch, which can target cpu1 too, or (b) an architectural virtual SError from EL2 (HCR_EL2.VSE), which works on stock QEMU; shown on a stand-alone test image with negatives.”

Do not say:

  • that ctos takes SErrors on cpu1;
  • that stock QEMU has a physical SError source;
  • that bad device accesses produce SErrors on QEMU;
  • that this changes the “EL0 isolated” claim.

Consequences

  • New: this ADR, research/serror-spike/ (README, vse image source, matrix script, gdb script, log), a brief, a ledger row, SUMMARY and moc entries.
  • No kernel, workflow, Dockerfile or smoke change. The default single-core build is unchanged.

ADR-109 — Track N / N6 — inbound TCP (one passive open over QEMU hostfwd)

  • Status: Accepted (runtime Verified only when this tip’s qemu-smoke prints net: tcp-in-ok and the host client prints hostfwd-client: neg=reset pos=ok rx=19 eof=yes; otherwise Documented)
  • Date: 2026-10-23

Context

ADR-074 (N5, PR #120) gave the kernel a thin active-open TCP: SYN to a QEMU guestfwd echo at 10.0.2.4:7, one payload, one echo. ADR-076 exposed it to EL0, and ADR-078 timed it. ADR-074 listed listen/accept as a locked non-goal. The sponsor now asks for inbound TCP: the host connects, data is exchanged, the connection closes, with fail-closed checks and a negative, over whatever networking the smoke already uses (QEMU user-net).

With QEMU user-net, hostfwd=tcp:127.0.0.1:<host port>-:<guest port> makes SLIRP accept on host loopback and open a TCP connection to the guest (source 10.0.2.2, destination 10.0.2.15:). SLIRP is a full TCP peer: it retransmits, ACKs and closes like a normal stack.

Hypotheses

IDOptionNotes
H1 (chosen)Kernel-path passive open on one port (8080) via hostfwd. The host client in qemu-serial-inject.py waits for the guest cue net: tcp-in-listen, then runs the negative and the positive.Deterministic in CI (host loopback only, no external net). It uses the same virtio-net path and SLIRP as N1–N5.
H2 (rejected)A sockets ABI (listen/accept/recv/send SVCs) with a socket table.Product-sized. An EL0 surface can follow in its own ADR.
H3 (rejected)TAP/bridge networking for a “real” inbound path.Needs host privileges in CI and changes the network the smoke uses.
H4 (rejected)Connect blindly on a timer without a guest cue.Racy: SLIRP would accept on the host before the guest polls, and the result would depend on retransmit timing.

Decision

  1. QEMU. The hello-boot runner (qemu-serial-inject.py) adds hostfwd=tcp:127.0.0.1:<p1>-:8080,hostfwd=tcp:127.0.0.1:<p2>-:8081 to the existing user netdev. p1 and p2 are free loopback ports chosen at run time; nothing binds outside loopback. Guest port 8081 has no listener: it is the negative.

  2. Guest (src/tcpin.rs, after the N1–N5 probe). Two small raw-frame helpers are added to virtio.rs: net_raw_tx and net_raw_rx. net_raw_rx reuses a still-posted RX buffer rather than posting twice. The guest then does the following:

    • Cue. Print net: tcp-in-listen port=8080 closed=8081, then poll for up to 15 s.

    • ARP. Answer ARP who-has 10.0.2.15.

    • Parse. Accept only IPv4/TCP to 10.0.2.15 with a valid TCP checksum. Bad-checksum segments are dropped and counted; fragments and other addresses are dropped.

    • Closed port. A SYN to 8081 gets an RFC 793 reset (RST|ACK, ack = seq + 1) and net: tcp-in-rst port=8081. Late segments of earlier outbound probes, which arrive at other ports, also get an RST but do not count as the negative.

    • Port 8080, one peer.

      1. SYN → SYN-ACK (net: tcp-in-syn; a retransmitted SYN gets the SYN-ACK again).
      2. The ACK of our ISS+1 → net: tcp-in-est.
      3. In-order data up to a newline → net: tcp-in-rx bytes=N. Out-of-order data is re-ACKed.
      4. Reply ctos-in:<line> (net: tcp-in-tx bytes=M), then the guest closes with FIN.
      5. The host’s ACK of our FIN plus the host’s FIN, which we ACK → net: tcp-in-fin closed.

      A second 4-tuple to 8080 gets an RST. A peer RST is a failure.

    • Result. net: tcp-in-ok requires both the completed exchange and the closed-port RST. Otherwise the guest prints net: tcp-in-fail reason=… and the kernel prints net: tcp-in probe missed.

  3. Host client. It starts on the cue and runs two steps:

    1. Negative. Connect to p2. It must read EOF or get a reset with zero bytes; a timeout or any data is a failure.
    2. Positive. Connect to p1, send ctos-hello\n, and read to EOF. The reply must be exactly ctos-in:ctos-hello\n (19 bytes). Then close.

    It prints hostfwd-client: neg=… pos=… rx=… eof=… after QEMU’s output.

  4. Smoke (fail closed). qemu-smoke.sh rejects any tcp-in-fail or probe missed. It requires all eight guest markers (including rx bytes=11, tx bytes=19 and rst port=8081) and the exact host line hostfwd-client: neg=reset pos=ok rx=19 eof=yes.

  5. cargo test (in QEMU). The tests cover:

    • parsing a SYN;
    • a corrupted segment being rejected as BadChecksum;
    • RFC 793 RST fields for a SYN and for an ACK, and never answering an RST;
    • reply checksums being valid with addresses and ports swapped;
    • fragments and foreign destinations being dropped.

    The runtime exchange itself is only in the boot smoke, which is where the host client exists.

Not claimed

  • No sockets API, no EL0 SVC, no backlog or socket table, no concurrent connections, no retransmission timer, no window management or options (MSS not sent), no urgent data, no IPv6, no TLS or HTTP.
  • No real-network claim: SLIRP on host loopback only, QEMU virt TCG only. Not a security boundary.
  • N5 outbound TCP (ADR-074/076/078) is unchanged.

Consequences

  • Every hello boot through qemu-serial-inject.py now runs the inbound exchange; the host client is always present there. cargo test boots (qemu-aarch64.sh) have no hostfwd and do not run this boot-path probe.
  • A later ADR can put an EL0 accept/recv/send surface on top of this state machine.

ADR-110 — N1: taken SError on core 1 at EL0 (pinned QEMU, -smp 2, cpu-index=1)

  • Status: Accepted (implementation + evidence, opt-in). Sponsor go relayed by DSO on 2026-10-23 at 20:17 CEST (N1 from ADR-108).
  • Date: 2026-10-23
  • Base: main c420fba. ADR-107 (#163, FAT32), ADR-108 (#165, the spike) and ADR-109 (#164, inbound TCP) are taken, so this is ADR-110.
  • Machine: amd64 workstation (TCG; builds in an ubuntu:24.04 container). CI: the QEMU NMI pin smoke (taken SError) job.

Context

The B1 pinned QEMU (v10.0.0 + patch 0001, ADR-083) gave a taken lower-EL SError only on core 0, at -smp 1. On core 1 (S4/S5, ADR-104/ADR-105) there was no SError probe. The ADR-109 spike showed that the pinned build’s HMP nmi honours cpu-index.

Decision

  1. Opt-in features:

    • smp2-serror-probe (implies smp2-probe): after core 1’s 13 S4 probes, core 1 prints smp: cpu1 serror-arm masked=no cpu=1 and ERETs to the same standing-EL0 spin as the single-core probe, with PSTATE.A clear and the banked per-core EXPECT flags.
    • smp2-serror-masked is the negative build: it ERETs with A set.
  2. Handler (fail-closed): in that build, core 1’s lower-EL SError handler prints its own locked line, smp: cpu1 el0 serror taken cpu=1. It never prints the core-0 el0: serror cue. The probe prints smp: cpu1 serror taken=yes … only if the handler ran and the standing trip restored. Otherwise it prints smp: cpu1 serror park taken=no ….

  3. Host: qemu-serial-inject.py, on the smp: cpu1 serror-arm cue and only when CTOS_QEMU_NMI_CPU1=<n> is set, does QMP stop → human-monitor-command nmi with cpu-index=<n> → cont. QMP inject-nmi has no CPU argument.

  4. Check: scripts/smp2-serror-pin.sh. Every row requires the host nmi=>ok, smp: ok cpu1 s4, and no unhandled exception.

    RowBuildnmi toMust seeMust not see
    Pprobecpu-index=1cpu1 el0 serror taken and taken=yespark
    N0probecpu-index=0park taken=no masked=nocpu1 el0 serror taken
    NMmaskedcpu-index=1park taken=no masked=yescpu1 el0 serror taken
  5. CI: a new step in qemu-nmi-pin after the unchanged single-core step (CTOS_REQUIRE_TAKEN_SERROR=1).

Evidence (2026-10-23, amd64 workstation, pinned QEMU 10.0.0)

  • smp2-serror-pin.sh: P ok (nmi(cpu-index=1)=>ok, smp: cpu1 el0 serror taken cpu=1), N0 ok, NM ok.
  • The full stock smoke (CTOS_SMP2_SMOKE=1) and the single-core pin smoke (CTOS_REQUIRE_TAKEN_SERROR=1) still pass.
  • CI results are on the PR.

Honesty

Say: “With the repo’s pinned QEMU, a host-injected SError aimed at core 1 (cpu-index=1) is taken by ctos core 1 while it stands at EL0, on -smp 2 TCG. Aimed at core 0, or with core 1’s A bit set, it is not taken.”

Do not say:

  • that stock QEMU does this;
  • that it is hardware RAS;
  • that the “EL0 isolated” sentence changes. A claim change needs a separate sponsor accept.

Also not shown:

  • whether core 0 takes the N0 SError later: it stays pending with A masked, and nothing after unmasks it;
  • concurrency with S5 (core 1 runs the probe before S5);
  • KVM.

Consequences

  • Cargo.toml gains two features.
  • Code: src/el0.rs cpu1_serror_probe, a cfg branch in src/exception.rs for the handler, one cfg call in src/smp.rs, and smp::line becomes pub(crate).
  • Scripts: scripts/qemu-serial-inject.py (hmp_nmi, env-gated) and scripts/smp2-serror-pin.sh.
  • Workflow: one step added to qemu-nmi-pin.
  • Docs: ledger row, SUMMARY, moc, brief.
  • The default build, default smoke and _start are unchanged.

ADR-111 — N2: EL2 shim → taken virtual SError at EL0 on stock QEMU (opt-in)

  • Status: Accepted (implementation + evidence, opt-in). Sponsor go relayed by DSO on 2026-10-23 at 20:17 CEST (N2 from ADR-108).
  • Date: 2026-10-23
  • Base: main c420fba. ADR-107 (#163), ADR-108 (#165, the spike), ADR-109 (#164) and ADR-110 (#166, N1) are taken, so this is ADR-111.
  • Machine: amd64 workstation (TCG; builds in an ubuntu:24.04 container).

Context

ADR-108 found that stock QEMU has no physical SError source, but it does model the architectural virtual SError (HCR_EL2.VSE) for a guest that owns EL2. ctos runs at EL1 and _start (0x4008_0000) must not change. So a tiny EL2 shim must sit in front of it.

Decision

  1. tools/el2-shim: a stand-alone ELF linked at PA 0, the virt flash window, away from RAM and the ctos image. It is loaded with -device loader,file=<shim>,cpu-num=0, which also points core 0’s reset PC at it; QEMU still loads the ctos ELF with -kernel.

    • At EL2 it sets HCR_EL2 = RW | AMO. IRQ/FIQ stay at EL1.
    • It also sets CNTHCTL_EL2 = 3, CNTVOFF_EL2 = 0, CPTR_EL2 = 0x33ff, MDCR_EL2 = 0, and VPIDR/VMPIDR mirroring MIDR/MPIDR.
    • It prints el2-shim: el1 and ERETs to 0x4008_0000 at EL1h with DAIF masked and x0..x3 as QEMU set them. SCTLR_EL1 keeps its reset value.
    • It uses no RAM; its only scratch is TPIDR_EL2.
    • HVC ABI (hvc #0xC7, only x0 changes): 0 = query (returns magic 0x5E2E1D00), 1 = set HCR_EL2.VSE, 2 = clear it. Any other HVC (ctos PSCI hvc #0) is forwarded as smc #0 to QEMU’s PSCI. Any other EL2 exception prints el2-shim: unexpected and parks.
  2. ctos feature el2-vserror (opt-in). In the standing-EL0 SError window, after el0: serror-arm:

    • It checks ID_AA64PFR0_EL1.EL2, and only then queries the shim. With no EL2 it issues no HVC.
    • It asks for VSE, prints el0: virtual serror ask el2-shim=… vse=… masked=…, and ERETs with A clear.
    • The handler counts the SError only if ESR EC is 0x2f, and prints el0: virtual serror taken (el2 vse) ec=0x2f esr=…. The default el0: serror cue is never printed in this build.
    • VSE is cleared after the trip.
    • el2-vserror-masked is the negative build: it ERETs with A set.
  3. qemu-serial-inject.py: opt-in CTOS_QEMU_MACHINE and CTOS_QEMU_EXTRA. When they are unset the argv is unchanged.

  4. Check: scripts/el2-vserror-smoke.sh (fail-closed). Every row must reach el0: ok with no unhandled exception.

    RowBuildQEMUMust seeMust not see
    Pel2-vserrorvirt,virtualization=on + shimask el2-shim=yes vse=set masked=no, virtual serror taken (el2 vse) ec=0x2fserror-park
    NAdefaultsame + shimel2-shim: el1, serror-parkvirtual serror
    NMel2-vserror-maskedsame + shimvse=set masked=yes, serror-parktaken
    NSel2-vserrorplain virt (no EL2), no shimel2-shim=no vse=no, serror-parktaken, el2-shim: el1
  5. CI: a new opt-in job, QEMU EL2 shim virtual SError (opt-in, ADR-111; not required), on stock Ubuntu 24.04 QEMU.

    • The default build, qemu-smoke.sh and the required checks are unchanged.
    • The NA row also shows the default image runs fully under the shim.

Evidence (2026-10-23, amd64 workstation)

  • Stock QEMU 8.2.2 (Ubuntu 24.04, container): P, NA, NM and NS all ok.
  • Stock QEMU 10.2.1 (host distro): P, NA, NM and NS all ok. P line: el0: virtual serror taken (el2 vse) ec=0x2f esr=0xbe000000.
  • CI results are on the PR.

Honesty

Say: “On stock QEMU (-M virt,virtualization=on) with the repo’s EL2 shim, the full ctos image takes a virtual SError (EL2-injected via HCR_EL2.VSE) at EL0, and it is not taken without the ask, with A masked, or without EL2.”

Do not say:

  • that this is a physical SError or hardware RAS;
  • that the default boot has an EL2;
  • that it covers cpu1 (-smp 1 only; the shim does not wrap PSCI CPU_ON);
  • that the “EL0 isolated” sentence changes. A claim change needs a separate sponsor accept.

Consequences

  • New: tools/el2-shim/ and src/el2.rs (cfg).
  • Changed, all opt-in: cfg branches in src/el0.rs and src/exception.rs, and two features in Cargo.toml.
  • Scripts: env-gated qemu-serial-inject.py, plus scripts/el2-vserror-smoke.sh.
  • Workflow: one new opt-in job.
  • Docs: ledger row, SUMMARY, moc, brief.
  • _start, the default build and the default smoke are unchanged.

CI for ADR-083 pinned QEMU (unconditional since ADR-090)

The default qemu-smoke jobs in .github/workflows/smoke.yml stay on distro QEMU (park path).

Job qemu-nmi-pin (“QEMU NMI pin smoke (taken SError)”) builds the pinned QEMU and runs the smoke with CTOS_REQUIRE_TAKEN_SERROR=1 on every push and pull request (ADR-090, sponsor D1/D5). Until ADR-090 it was gated by repo variable CTOS_BUILD_QEMU_NMI == '1'. That gate was removed because GitHub treats a job skipped by if: as passing a required check. The variable is now unused; the Dockerfile build-arg of the same name is unrelated and stays opt-in.

Workflow edits need a credential with the OAuth workflow scope (amd64 workstation gh login). The box OAuth app has gist, read:org, repo only.

Local / amd64 workstation (no GHA variable needed)

./scripts/build-qemu-nmi.sh
export CTOS_QEMU="$PWD/tools/qemu-nmi/bin/qemu-system-aarch64"
export CTOS_REQUIRE_TAKEN_SERROR=1
./scripts/qemu-smoke.sh

Dockerfile already supports --build-arg CTOS_BUILD_QEMU_NMI=1.

Job as landed (reference; see smoke.yml for the source of truth)

  qemu-nmi-pin:
    name: QEMU NMI pin smoke (taken SError)
    runs-on: ubuntu-24.04
    timeout-minutes: 90
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }
      - uses: dtolnay/rust-toolchain@nightly
        with: { components: rust-src, llvm-tools-preview }
      - run: |
          sudo apt-get update
          sudo apt-get install -y ninja-build meson pkg-config python3-venv flex bison \
            libglib2.0-dev libpixman-1-dev libfdt-dev libslirp-dev
      - run: ./scripts/build-qemu-nmi.sh
      - env:
          CTOS_QEMU: ${{ github.workspace }}/tools/qemu-nmi/bin/qemu-system-aarch64
          CTOS_REQUIRE_TAKEN_SERROR: "1"
        run: ./scripts/qemu-smoke.sh

Roadmap

Each row is one loop unit: one milestone, one branch, one GitHub PR. Do not stack a later stage onto an unclosed earlier PR. Cite frozen FR/NFR IDs in the PR when touching behavior; do not invent new IDs in chat.

Primary ISA is AArch64 (ADR-003). M0/M1 probes are UART + qemu-system-aarch64, not VGA 0xb8000.

IDMilestoneProbe that closes itStatus
M0Kernel tree + harness scaffold on GitHubFiles on the default-target PR; ledger startedIn harness PRs #2/#3 (source). Merge is a human/MRC job.
M1QEMU virt boots UART “Hello World!”qemu-system-aarch64 serial shows the string; see honesty ledgerVerified in the 2026-09-08 cloud probe (not CI). x86 VGA probe is historical only.
M2Integration test harnessQEMU virt ARM semihosting exit + #[test_case]; scripts/qemu-smoke.sh fail-closedVerified in the 2026-09-08 cloud probe (not CI). Sponsor stacked this on the ISA PR.
M3Exception vectors + breakpointTest or QEMU serial proof the handler runsVerified: 2026-09-09 cloud qemu-smoke + GHA smoke.yml on the M3 PR (see honesty ledger).
M4Fatal exception stackFatal path does not silently lock the VMVerified: 2026-09-09 cloud qemu-smoke (see honesty ledger). GHA on the M4 PR branch still Unknown.
M5Hardware interrupts (GIC + timer)Timer tick observable (serial or test)Verified: 2026-09-09 cloud qemu-smoke + GHA smoke.yml on the M5 PR (see honesty ledger).
M6InputInjected PL011 RX byte observable on serial (or test)Verified: 2026-09-09 cloud qemu-smoke (see honesty ledger). GHA on the M6 PR branch still Unknown.
M7Paging + frame allocatorMap/unmap or allocator testVerified: 2026-09-09 cloud qemu-smoke (see honesty ledger). GHA on the M7 PR branch still Unknown.
M8Heap (alloc)Box/vec smoke on the heapVerified: 2026-09-09 cloud qemu-smoke + GHA smoke.yml on the M8 PR (see honesty ledger).
M9Cooperative schedulerTwo tasks observed to runVerified: 2026-09-09 cloud qemu-smoke + GHA smoke.yml (see honesty ledger).

M0–M9 are on main (M9 = merge of PR #15 / FR-11). Pillar work through ADR-020 (#17–#28) plus docs #30/#51/#29, A1–A6 (ADR-021–ADR-027) are on main. This PR is Track A / A7 (issue #38, parent issue #31): ADR-028 virtio-blk + FAT16. Merge is still a human/MRC job (ADR-002).

Pillars (post-M9)

Bring-up M0–M9 stays one loop unit each. After M9, work is grouped under the three pillars.

IDWorkProbe that closes itStatus
P-SEC-1Threat-model v1.1 slice (NFR-10)Read security.md; no “secure OS” claimVerified (file + review). v1.1 adds linker stacks, guards, EL0 first mile, remaining W^X gaps.
P-SEC-2W^X / NX heap + coop stacksPage-table PXN on heap + caught execute-from-heap IABORT (wx: ok)Verified: 2026-09-10 cloud qemu-smoke (see honesty ledger). Map: ADR-012. Linker stack pages still X.
P-SEC-2bLinker-stack guard pagesUnmapped 4 KiB holes; store faults (guard: ok)Verified: 2026-09-10 cloud qemu-smoke (see honesty ledger). ADR-014. Not kernel W^X.
P-PERF-1Baseline CNTPCT probe (NFR-07)Serial perf: cntpct and/or #[test_case]; not a published benchVerified: 2026-09-09 cloud qemu-smoke + GHA (see honesty ledger).
P-PERF-2IRQ-to-handler CNTPCT deltaSerial perf: irq-delta + samples max >= min; not a latency budgetVerified: 2026-09-10 cloud qemu-smoke (see honesty ledger).
P-PERF-3Host debug ELF size (NFR-08)Host marker perf: elf-size bytes=<n>; not a budgetVerified: 2026-09-10 cloud qemu-smoke host print (see honesty ledger).
P-SEC-3EL0 first mileel0: svc + el0: nx kernel + el0: okFirst mile Verified: 2026-09-10 cloud qemu-smoke (see honesty ledger).
P-SEC-2cRO+NX text/data (ADR-015)ro: nx data + ro: write fault + ro: okVerified: 2026-09-11 cloud qemu-smoke (see honesty ledger). Identity image W^X on virt; not “secure OS.”
P-PERF-4Boot-to-ready CNTPCT (NFR-08)perf: boot-delta ticks=<n>; not a budgetVerified: 2026-09-11 cloud qemu-smoke (see honesty ledger).
P-PERF-5A9 slot-disconnect CNTPCT pair (ADR-046)perf: app-load + perf: embed-load + perf: slot-delta app=<a> embed=<b>; not a percentVerified when this tip’s qemu-smoke prints the pair (see honesty ledger). QEMU TCG lab. Production slot stays FAT-only.
P-PERF-6FAT16 vs memfs write CNTPCT pair (ADR-051)perf: fat-write + perf: memfs-write + perf: fs-write-delta fat=<a> memfs=<b>; not a percentVerified when this tip’s qemu-smoke prints the pair (see honesty ledger). QEMU TCG lab. Keep fat: write / fat: ok.
P-PERF-7FAT16 vs memfs read CNTPCT pair (ADR-065)perf: fat-read + perf: memfs-read + perf: fs-read-delta fat=<a> memfs=<b>; not a percentVerified when this tip’s qemu-smoke prints the pair (see honesty ledger). QEMU TCG lab. Keep fat: read / fat: ok / ADR-051 write pair.
P-PERF-8Net-ping path CNTPCT (ADR-069)perf: net-ping ticks=<n>; smoke greps prefix only; not a percentVerified when this tip’s qemu-smoke prints the prefix (see honesty ledger). QEMU TCG lab. Keep N1/N2/N4.
P-PERF-9UDP DNS path CNTPCT (ADR-072)perf: udp-dns ticks=<n>; smoke greps prefix only; not a percentVerified when this tip’s qemu-smoke prints the prefix (see honesty ledger). QEMU TCG lab. Keep N1/N2/N3/N3.x/N4 + ADR-069.
P-PERF-10Thin TCP echo path CNTPCT (ADR-078)perf: tcp-echo ticks=<n>; smoke greps prefix only; not a percentVerified when this tip’s qemu-smoke prints the prefix (see honesty ledger). QEMU TCG lab. Keep N1–N5 / N5.x + ADR-069/072.
P-SEC-3bUser TTBR0 + EL0 cannot read kernel .datael0: no kernel read; user table omits .dataVerified: 2026-09-11 cloud qemu-smoke (see honesty ledger). Isolation / PAN stay Planned.
P-SEC-3cASID-tagged TLB isolationasid: dual + asid: conflict + asid: ok; no TLBI VMALLE1 on the switchVerified: 2026-09-11 cloud qemu-smoke (see honesty ledger). Umbrella isolation / PAN stay Planned.
P-SEC-3dStanding EL0 contextel0: standing + el0: restored; is_active() true only while standingVerified: 2026-09-11 cloud qemu-smoke (see honesty ledger). Not POSIX. Default ERET clears IRQ mask (ADR-041).
P-SEC-3eTTBR1 kernel-private page (ADR-016 first cut)ttbr1: el1 + ttbr1: no el0 + ttbr1: okVerified: 2026-09-11 cloud qemu-smoke (see honesty ledger).
P-SEC-3fEL1 fetch from TTBR1 high VA (ADR-017)ttbr1: el1 exec + ttbr1: vbar + existing ttbr1: okVerified: 2026-09-11 cloud qemu-smoke (see honesty ledger). Identity boot stub stays.
P-SEC-3gIdentity-tear first cut (ADR-018)ident: split + ident: fault + ident: high + ident: no el0 + ident: okVerified: 2026-09-11 cloud qemu-smoke (see honesty ledger). Full identity teardown / PAN / umbrella isolation were Planned at this mile (umbrella: P-SEC-3l). Do not claim “the kernel moved.”
P-SEC-3hHigh-VA jump + 16 KiB identity text range (ADR-019)ident: jump + ident: range + ident: text + existing ident: okVerified: 2026-09-11 cloud qemu-smoke + arm64 workstation Docker on 24d94e6 (see honesty ledger). Do not claim “the kernel moved.”
P-SEC-3iHigh-VA vtable rewrite + live identity .text tear (ADR-020)ident: reloc + ident: live + existing ident: ok; println! after the tearVerified: 2026-09-11 cloud qemu-smoke + arm64 workstation Docker on e80dc93 + GHA merge-commit 34651404108 (see honesty ledger). .rodata / .data / heap stay. Do not claim “the kernel moved.”
P-SEC-3jIdentity .rodata / .data / heap tear.rodata + .data/stacks + heap unmapped + high-only access..rodata Verified (ADR-025). .data/stacks Verified (ADR-037). Heap Verified (ADR-038) on this branch qemu-smoke (ident: heap-reloc / ident: heap / ident: heap-fault / ident: heap-high; heap-stay gone). Do not claim “the kernel moved.”
P-SEC-3kPAN enable on virtID_AA64MMFR1_EL1.PAN != 0 and an EL1-vs-EL0 access faultVerified (ADR-080 / #125 mile 2): default -cpu cortex-a76, pan: present + pan: enabled + pan: el1-fault. Historical a57 pan: absent (ADR-026 / ADR-054 / ADR-079). One mile; alone it is not the ADR-091 umbrella claim.
P-SEC-3mEL0 entry without TLBI VMALLE1 (ADR-039)el0: no-vmalle1; src/exception.rs has no tlbi vmalle1Verified: 2026-09-12 agent-box qemu-smoke (see honesty ledger). One mile; alone it is not the ADR-091 umbrella claim. PAN enable was a non-goal on the then-default a57 (enabled on a76 since ADR-080); umbrella: P-SEC-3l.
P-SEC-3nLower-EL IRQ while standing (ADR-040)el0: irq; #[test_case] lower_el_irq_while_standingVerified when this tip’s qemu-smoke prints el0: irq (see honesty ledger). FIQ/SError still park. One mile; alone it is not the ADR-091 umbrella claim.
P-SEC-3oIRQ-unmasked default standing/task ERET (ADR-041)el0: irq-default; #[test_case] lower_el_irq_default_eretVerified when this tip’s qemu-smoke prints el0: irq-default (see honesty ledger). Short non-standing probes stay masked. FIQ/SError still park. One mile; alone it is not the ADR-091 umbrella claim.
P-SEC-3pIdentity _start stay honesty (ADR-042 / ADR-047)ident: start-stay; #[test_case] identity_boot_stub_stays_while_live_tornVerified honesty. Never yank _start. One mile; alone it is not the ADR-091 umbrella claim.
P-SEC-3sLeftover identity RAM tear (ADR-049)ident: ram / ident: ram-fault / ident: ram-high; reject ident: ram-stayVerified on this branch when smoke passes. Optional mile from ADR-047. One mile; alone it is not the ADR-091 umbrella claim. Not “kernel moved.”
P-SEC-3qLower-EL FIQ while standing + SError park (ADR-043)el0: fiq + el0: serror-park; #[test_case] lower_el_fiq_while_standingVerified (FIQ taken + park honesty). Taken SError = deferred/non-goal (ADR-047). One mile; alone it is not the ADR-091 umbrella claim.
P-SEC-3rTaken lower-EL SError QMP attempt (ADR-045) / hard-stop (ADR-053) / reopen (ADR-081) / B1 pin (ADR-083) / B2 spike (ADR-084) / B2-P KVM (ADR-085)Guest A-clear path + el0: serror-arm + QMP inject-nmi log; el0: serror-park if not taken; bare el0: serror under opt-in pinStock virt TCG: deferred / non-goal (hard-stopped) (ADR-053 / ADR-081 / ADR-047); park Verified; do not require el0: serror on default smoke. B1: Verified (opt-in pin only) — ADR-083 pinned QEMU TYPE_NMI→SError, GHA job qemu-nmi-pin (CTOS_BUILD_QEMU_NMI=1). B2 (different runner/accelerator): Planned / blocked — ADR-084 2026-09-25 free-runner spike: no arm64 KVM/HVF/WHPX on any free GitHub-hosted runner (runs 36193423604 / 36193613972); needs sponsor choice (self-hosted arm64 KVM, paid bare metal, or wait). B2-P: Verified (opt-in, one-off paid run) — ADR-085 2026-09-26: AWS Graviton3 c7g.metal, QEMU -accel kvm + 0002 KVM_SET_VCPU_EVENTS serror_pending → bare el0: serror + b2: taken (fail-closed smoke PASS, run 4). Not a CI gate. Also fixed TCR_EL1.TG1 (reserved → 4K). Evidence class (D1): ADR-090 2026-09-26: B1 pin job made unconditional (every push/PR; a skipped job would pass a required check) + B2-P run 4 = the umbrella’s taken-SError evidence. Stock TCG hard-stop unchanged. One mile; alone it is not the ADR-091 umbrella claim.
P-SEC-3lUmbrella EL0 isolationStanding + PAN + full TTBR1 / identity teardownVerified (scoped: the accepted ADR-091 sentence only; M6, 2026-09-28). History: was Planned / non-claim until checklist (ADR-055 / ADR-047 / sponsor closure ADR-060). Constraints: PAN enable Verified separately (ADR-080) but not the umbrella; _start stays; taken SError hard-stopped (ADR-081). Specific miles are not this row. 2026-09-26 audit ADR-086: all ADR-055 §A miles Verified this session; §B not Met (taken SError only opt-in B1/B2-P; five identity ranges remain in kernel TTBR0 incl. the _start stub; no umbrella accept). Milestones M1 (inventory ratchet + RAM-side tears) → M2 (MMIO high alias) → M3 (stub policy, D2) → M4 (SError evidence class, D1) → M5 (accept, D4) → M6 (flip). M1 done (ADR-087): I2/I4/I5 torn from kernel + user TTBR0; fail-closed ident: inv walk allows only {MMIO block, _start stub}; planted identity page caught (ident: inv-neg, inv-leak-probe). Sponsor D1–D5 answered 2026-09-26 ~14:35 CEST (D4 accept still pending the sponsor reading the draft). M2 done (ADR-088): GIC/PL011/virtio via TTBR1 Device blocks; identity MMIO block torn; inventory k=1 u=1 a=1 = _start stub only (ident: inv-ok allow=stub). M3 (ADR-089): sponsor D2: _start stub page = the one documented identity exception; checklist identity row reworded and Met (ADR-087/088 evidence). M4 (ADR-090): sponsor D1: taken-SError evidence class = B1 qemu-nmi-pin (now unconditional) + B2-P run 4; checklist row 1 reworded and Met on that class. M5 (ADR-091, Draft): proposed umbrella sentence + scope, pending the sponsor’s written accept (B3 not Met); M6 flip not started. ADR-092 (2026-09-27, sponsor option (b)): G1 EL0-reachability walk (k/u/ASID-B TTBR0 + TTBR1, allowlist = five user slots, fail-closed + leak build) and G2 EL0 write permission fault closed; new gap G4 (syscall copy helpers skip the EL0 permission check); ADR-091 revision 2 sentence back with the sponsor (B3 not Met). ADR-094 (2026-09-27): G4 closed; ADR-091 revision 3. ADR-095 (2026-09-28): G3 KVM re-check PASS. M6 (2026-09-28): sponsor’s written accept (B3, 16:41 CEST); ADR-091 Accepted; row flipped. Say “EL0 isolated” only as the ADR-091 sentence, with its limits (one core, the TCG default smoke machine; not a speculative-execution or side-channel claim, not a real-hardware claim, not a certification; app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN); no multi-core race story).

Hub: pillars.md. ABI contract: syscall.md.

Track A — freestanding app hosting (#31)

A1 is the SVC ABI mile. A2 is the CRT / libctos mile. A3 is the guest ELF PT_LOAD loader mile. A4 is standing EL0 as normal mode for a loaded image. A5 is isolation completion (identity .rodata / .data / heap tears + EL0 entry without VMALLE1 + PAN ID-field; PAN enable was a non-goal on the then-default a57 per ADR-047, enabled on a76 since ADR-080). A6 is thin VFS + in-RAM memfs. A7 is virtio-blk + FAT16. A8 is documented sample rebuild recipes (no new runtime). A9 is the OS/app slot first cut plus leftover cross-update / embed-off (ADR-032). Track A children A1–A9 have a first cut; sample catalog deepened with fs-libctos (ADR-059), fat-libctos (ADR-061), and yield-libctos (ADR-062); product “app hosting is done” is Verified under ADR-048 criteria Met×5 (ADR-052). Not Linux containers. Not glibc. One child issue → one PR. Do not round A1–A9 miles alone into Linux/POSIX claims.

IDWorkProbe that closes itStatus
A1Stable SVC ABI + docs (#32, ADR-021)svc: yield + svc: user-hi + svc: uart + svc: exit + svc: ok; #[test_case]ABI mile Verified: 2026-09-11 cloud qemu-smoke on 8846bc5 (onto aa46219 / #29). App hosting stays Planned.
A2Freestanding CRT / libctos (#33, ADR-022)libctos: hi + libctos: ok + libctos: linked; #[test_case]CRT mile Verified: 2026-09-12 cloud qemu-smoke on c9b292b. App hosting stays Planned.
A3ELF (or raw image) loader into user TTBR0 (#34, ADR-023)loader: mapped + loader: ok; payload libctos: hi / libctos: ok; #[test_case]Loader mile Verified: 2026-09-12 cloud qemu-smoke (see honesty ledger). Not a Linux ABI. App hosting stays Planned.
A4Standing EL0 as normal mode (#35, ADR-024)el0: task-enter + el0: task-active + el0: task-exit + el0: task-restored + el0: restore-fail + el0: task-ok; #[test_case]Standing-task mile Verified: 2026-09-12 cloud qemu-smoke (see honesty ledger). Not isolation. App hosting stays Planned.
A5Isolation completion (#36, ADR-025 / ADR-026 / ADR-032 / ADR-037 / ADR-038 / ADR-039 / ADR-040 / ADR-041 / ADR-042 / ADR-043 / ADR-044 / ADR-045 / ADR-047 / ADR-049 / ADR-053 / ADR-054 / ADR-055 / ADR-060 / ADR-079 / ADR-080 / ADR-081 / #125)ident: rodata + ident: data + ident: heap + ident: ram; el0: no-vmalle1; el0: irq + el0: irq-default + el0: fiq + el0: serror-park; pan: id= + pan: present + pan: enabled + pan: el1-fault (cortex-a76)Prior A5 + PAN enable (ADR-079/080). Locks: taken SError still hard-stopped after ADR-081 re-probe (ADR-053 / ADR-081 / ADR-082); umbrella: P-SEC-3l (ADR-055); never yank _start. One mile; alone it is not the ADR-091 umbrella claim.
A6Thin VFS + memfs (#37, ADR-027)fs: create + fs: write + fs: read + fs: el0 + fs: ok; #[test_case]memfs mile Verified (see honesty ledger). Not POSIX. App hosting stays Planned.
A7virtio-blk + FAT16 (#38, ADR-028)blk: virtio + blk: cap + blk: rw + blk: ok; fat: mount + fat: read + fat: ok; #[test_case]block + FAT mile Verified when this tip’s qemu-smoke passes (see honesty ledger). Same VFS open. Not POSIX. Not FAT32. App hosting stays Planned.
—FAT16 write depth (Future slice 3, ADR-050)fat: write + fat: rewrite + fat: create + host --check-write; keep fat: ok / slot: okVerified when this tip’s qemu-smoke passes (see honesty ledger). Not POSIX. Not FAT32. App hosting stays Planned.
—FAT vs memfs write CNTPCT (Future slice 4, ADR-051 / P-PERF-6)perf: fat-write + perf: memfs-write + perf: fs-write-delta; keep fat: ok / slot: okVerified when this tip’s qemu-smoke prints the pair (see honesty ledger). Not a bench. Not a percent. App hosting stays Planned.
—FAT vs memfs read CNTPCT (perf slice 3, ADR-065 / P-PERF-7)perf: fat-read + perf: memfs-read + perf: fs-read-delta; keep fat: ok / write pair / slot: okVerified when this tip’s qemu-smoke prints the pair (see honesty ledger). Not a bench. Not a percent. App hosting stays Planned.
—Net-ping CNTPCT (perf slice, ADR-069 / P-PERF-8)perf: net-ping ticks=<n>; prefix grep; keep N1/N2/N4Verified when this tip’s qemu-smoke prints the prefix (see honesty ledger). Not a bench. Not a percent. N3 UDP unlocked by ADR-070 (not BSD sockets).
—UDP DNS CNTPCT (perf slice, ADR-072 / P-PERF-9)perf: udp-dns ticks=<n>; prefix grep; keep N1/N2/N3/N3.x/N4 + ADR-069Verified when this tip’s qemu-smoke prints the prefix (see honesty ledger). Not a bench. Not a percent. Not TCP.
—Thin TCP echo CNTPCT (perf slice, ADR-078 / P-PERF-10)perf: tcp-echo ticks=<n>; prefix grep; keep N1–N5 / N5.x + ADR-069/072Verified when this tip’s qemu-smoke prints the prefix (see honesty ledger). Not a bench. Not a percent. Not sockets. EL0 surface (not kernel N5 timer).
—FAT16 mkdir depth (ADR-073 / sponsor mile 2)fat: mkdir; keep fat: ok / write / readdir / delete / grow / slot: okVerified when this tip’s qemu-smoke passes (see honesty ledger). Not POSIX mkdir. Not a directory tree. Not TCP (mile 3).
—Thin TCP (ADR-074 / N5 / sponsor mile 3)net: tcp-ok (+ tcp-syn/tcp-est/tcp-tx/tcp-rx); guestfwd 10.0.2.4:7; keep N1–N3 + blk/FATVerified when this tip’s qemu-smoke prints net: tcp-ok (see honesty ledger). Not BSD sockets. Not listen/accept. Not TLS/HTTP.
—FAT16 readdir depth (ADR-056)fat: readdir + fat: entries; keep fat: ok / write / slot: okVerified when this tip’s qemu-smoke passes (see honesty ledger). Not POSIX getdents. Not FAT32. Not Linux/containers.
—FAT16 delete depth (ADR-057)fat: delete; keep fat: ok / write / readdir / slot: okVerified when this tip’s qemu-smoke passes (see honesty ledger). Not POSIX unlink. Not FAT32. Not Linux/containers.
—VFS prefix mounts (ADR-058)vfs: mount + vfs: mounts; keep fs: ok / fat: ok / slot: okVerified when this tip’s qemu-smoke passes (see honesty ledger). Not Linux mount(2). Not a new disk FS.
A8Documented sample apps (#39)recipes cite only existing serial markers; docs-buildRecipes Verified (docs; smoke markers unchanged). Hub: what-can-run.md. No new ADR. Not app hosting.
A9OS–app slots (#48, ADR-030 / ADR-032)host OS ELF + hello-libctos.elf; FAT /hello; slot: ok; same ELF on this OS and ba6541cFirst cut + leftover when this tip’s qemu-smoke prints cross-update lines (see honesty ledger). A2–A4 load FAT (no embed). Product claim Verified under ADR-048/052 (this mile alone ≠ Linux hosting).
—Freestanding fs-libctos catalog (ADR-059)libctos: fs-ok + fsdemo: ok; keep slot: ok / /helloVerified when this tip’s qemu-smoke passes (see honesty ledger). Memfs via ADR-058 /mem. Not POSIX. Not a reopen of ADR-048/052.
—Freestanding fat-libctos catalog (ADR-061)libctos: fat-ok + fatdemo: ok; keep /hello / /fsdemoVerified when this tip’s qemu-smoke passes (see honesty ledger). FAT /probe via thin VFS. Not POSIX. Not a reopen of ADR-048/052.
—Freestanding yield-libctos catalog (ADR-062)libctos: yld-ok + ≥3× libctos: beat + yldemo: ok; keep prior threeVerified when this tip’s qemu-smoke passes (see honesty ledger). Cooperative yield rounds. Not preemption. Not a reopen of ADR-048/052.
—FAT16 nested mkdir + empty rmdir (ADR-077)fat: nested + fat: rmdir; keep fat: mkdir / grow / delete / net / samplesVerified when this tip’s qemu-smoke passes (see honesty ledger). Not POSIX. One nest level only. No new SVC. Not TCP CNTPCT.
—Freestanding mkdir-libctos catalog (ADR-075)libctos: mkdir-ok + mkdemo: ok; keep fat: mkdir / prior sixVerified when this tip’s qemu-smoke passes (see honesty ledger). EL0 fs_mkdir (27). Not POSIX. Not nested trees. Not EL0 TCP.
—Freestanding tcp-libctos catalog (ADR-076)libctos: tcp-ok + tcpdemo: ok; keep net: tcp-ok / prior sevenVerified when this tip’s qemu-smoke passes (see honesty ledger). EL0 net_tcp_echo (28). Not BSD sockets. Not listen/accept.

On-disk filesystem work started as A7 (FAT16 read, not xv6-like). FAT16 write depth is ADR-050. FAT16 root listing is ADR-056. FAT16 root delete is ADR-057. FAT16 root mkdir is ADR-073. Nested mkdir + empty rmdir is ADR-077. EL0 fs_mkdir is ADR-075. EL0 net_tcp_echo is ADR-076. Thin VFS prefix mounts are ADR-058. Freestanding catalog samples on the same volume: ADR-059 / ADR-061 / ADR-062 / ADR-068 / ADR-071 / ADR-075 / ADR-076. See Filesystem: new vs extend.

Tracks (subordinate to principles)

Track A (freestanding apps, #31), Track B (Linux-compat research, #40), and Track N (network foundation, ADR-063) do not override principles.md. A9 is after Track A ABI/loader. Track B B1: Linux-compat subset research frame (ADR-031) — not claiming Linux userspace yet. Track B B2: Linux AArch64 vs ctos SVC gap map (linux-aarch64-syscall-gap.md) — inspection only; no Linux numbers in src/. Track B B3: standing EL0 stays the ctos process model (ADR-035) — Linux clone/execve/wait4 is a gap; do not add those syscalls. Track B B4: Linux ELF / auxv / PT_INTERP vs the freestanding loader (ADR-033) — gap ADR; Track A loader stays reusable; not claiming dynamic Linux ELF. Track B B5: Linux VFS vs thin ctos VFS (ADR-034) — concept compare only; no POSIX flags / dentries. Track B B6: never Linux-compat implementation from this ladder (ADR-036). Track B B7: guest OCI/Docker is a non-goal (ADR-029).

Track N — network foundation (ADR-063)

Separate track from A/B. Hub: track-n.md. N0 scope: ADR-063. N1 first frame: ADR-066. N2 ICMP ping: ADR-067. N3 UDP transport: ADR-070. N3.x EL0 UDP SVC: ADR-071. N4 EL0 net sample: ADR-068. N5 thin TCP: ADR-074. Reuse virtio-mmio patterns from blk; do not break FAT/blk smoke. Still not product: BSD sockets, TCP stack as product / listen-accept, DHCP/DNS as product, Wi‑Fi, Linux net stack, “has networking / sockets OS” marketing.

IDWorkProbe that closes itStatus
N0Scope ADR + docs wiring (ADR-063)Docs-build; ledger Planned; no Verified inventAccepted (docs) — this PR
N1Link bring-up / first frame (virtio-net mmio, ADR-066)Discover + TX ARP + RX SLIRP gateway ARP reply; -netdev user,id=net0 -device virtio-net-device,netdev=net0; net: ok; keep blk/FAT greenVerified when this tip’s qemu-smoke prints net: ok
N2ARP + ICMP ping (kernel-path, ADR-067)After ARP: ICMP echo vs 10.0.2.2; net: icmp-tx / net: icmp-rx / net: ping-ok; QEMU args unchangedVerified when this tip’s qemu-smoke prints net: ping-ok
N3Minimal UDP transport (ADR-070)UDP DNS query/reply vs SLIRP 10.0.2.3:53; net: udp-tx / net: udp-rx / net: udp-ok; keep N1/N2Verified when this tip’s qemu-smoke prints net: udp-ok
N4Freestanding sample using net SVCs (ADR-068)libctos: net-ok / netdemo: ok; keep N1/N2/N3/samplesVerified when tip prints those markers
N3.xEL0 UDP DNS SVC + udpdemo (ADR-071)libctos: udp-ok / udpdemo: ok; keep N1/N2/N3/N4/samplesVerified when tip prints those markers
N5Thin TCP active-open + one payload (ADR-074)net: tcp-ok (+ syn/est/tx/rx); guestfwd 10.0.2.4:7; keep N1–N3Verified when tip prints net: tcp-ok

Docs website

mdBook + GitHub Pages (not a kernel milestone, not a new FR/NFR ID). https://ctos.artof.link HTTPS is Verified after #30 (deploy 34653046584 + HTTPS 200). Route 53 CNAME remains in place (zone <route53-zone-id>). Overview: what can run today. Publish notes: website.md.

Isolation leftovers (decided — ADR-047 / sponsor closure ADR-060)

  • Taken SError: deferred / non-goal (hard-stopped) on virt TCG (ADR-053 / ADR-081 / ADR-082). Re-probed 2026-09-19 on cortex-a76 — still machine does not provide NMIs. Unlock = ADR-081 B1/B2/B3; B1 research ADR-082, then B1 opt-in pin Verified (ADR-083, qemu-nmi-pin). B2 free-runner spike blocked (ADR-084, 2026-09-25: no arm64 KVM/HVF/WHPX on free hosted runners) — sponsor then chose B2-P: Verified (opt-in, one-off) on AWS Graviton3 under KVM (ADR-085, 2026-09-26). Park Verified; A-clear = dormant prep (ADR-045).
  • PAN: CPU reopen + enable Verified (ADR-079 / ADR-080; default -cpu cortex-a76). Historical a57 pan: absent.
  • Yank _start: never while QEMU -kernel needs 0x4008_0000. ident: start-stay Verified honesty. Leftover identity RAM after heap: Verified optional mile (ADR-049; not umbrella-required).
  • Umbrella “EL0 isolated”: Verified only as the accepted ADR-091 sentence (M6, 2026-09-28; revision 4 accepted 2026-10-02; checklist ADR-055 all Met). Limits: one core, the TCG default smoke machine; not a speculative-execution or side-channel claim, not a real-hardware claim, not a certification; app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN); no multi-core race story. Sponsor one-pager: ADR-060.
  • Product app hosting “done”: Verified under ADR-048 / ADR-052 (freestanding slots; not Linux/POSIX/containers).

Track A — freestanding app hosting

Tracker epic: issue #31. Child issues A1–A9.

Driving force (non-negotiable)

This track is subordinate to ctos core principles (principles.md). Do not skip a probe, drop a ratchet, or invent a “secure/fast app runtime” to finish a row faster.

  • Honesty ledger — Verified only with a probe
  • Antifragility — fail-closed smoke + ratchets
  • Security — threat model; probed mitigations only
  • Performance — measure first; no invented benches
  • Document-first / one milestone → one branch → one PR

A9 (OS/app slot disconnect) sits after A1–A4. It does not override the list above. The first cut plus ADR-032 leftover mile: cross-update on ba6541c, no hello embed. Product “app hosting is done” is Verified under ADR-048 / ADR-052 (not Linux/POSIX/containers).

Goal

A freestanding (non-POSIX) application can be loaded and run with honest probes. Not Linux containers. Not glibc.

Ordered work

One loop unit each. Each row needs a fail-closed ledger probe.

IDWorkStatus (2026-09-12)
A1Stable SVC / syscall ABI + docsABI mile Verified on 8846bc5 (#32, ADR-021, syscall.md). Not app hosting.
A2Freestanding CRT / libctosCRT mile Verified on c9b292b (#33, ADR-022). Not app hosting.
A3ELF (or raw) loader into user TTBR0Loader mile Verified (#34, ADR-023). Not a Linux ABI. Not app hosting.
A4Standing EL0 as normal modeStanding-task mile Verified on this tip (#35, ADR-024). Loaded app until exit; fail-closed restore. Not isolation. Not app hosting.
A5Isolation completion (remaining identity tear; PAN only if CPU + ADR)Identity tears Verified; PAN CPU reopen Accepted (ADR-079 / #125; -cpu cortex-a76, pan: present). PAN enable still locked (ADR-080). Umbrella “EL0 isolated”: Verified only as the accepted ADR-091 sentence (M6); this mile alone is not it.
A6Thin VFS + memfsmemfs mile Verified (#37, ADR-027). In-RAM named buffers; create / write / read / close. Not POSIX. Not app hosting.
A7virtio-blk + FAT16block + FAT mile Verified on this tip when the ledger has blk: ok / fat: ok (#38, ADR-028). Same VFS open. FAT write depth: ADR-050. Root listing: ADR-056. Root delete: ADR-057. Not POSIX. Not a Linux FS.
A8Documented sample appsRecipes Verified (docs + existing smoke markers) (#39). Rebuild recipes: what-can-run.md, apps-today.md, in-tree user/README.md. Coop UART, RX echo, standing EL0 / libctos hello; optional memfs + FAT16. No new ADR (docs-only). No new runtime markers. Not app hosting.
A9Disconnect OS image from app payloadsFirst cut (#48, ADR-030) + leftover (ADR-032): OS ELF + published hello-libctos.elf + FAT /hello. A2–A4 load that file (no production embed). Cross-update: same ELF on this OS and ba6541c when qemu-smoke prints both cross-update … slot:ok lines. Slot-load CNTPCT pair (ADR-046): perf: app-load + probe-only perf: embed-load + perf: slot-delta (QEMU TCG lab; not a percent). Not OTA. Product claim Verified under ADR-048 / ADR-052 (not Linux/containers).

Out of scope for Track A

OCI/Docker containers (non-goal, ADR-029), glibc/musl ports, SMP, networking (Track N — ADR-063; Planned until N1). See host-apps.md.

Samples today: site what-can-run.md (extra: apps-today.md). Porting: site porting.md (extra: building-or-porting.md).

Track B — Linux-compat research (keep ctos specificity)

Tracker epic: issue #40. Child issues B1–B7.

Driving force (non-negotiable)

This track is subordinate to ctos core principles (principles.md). Linux-compat research must not abandon honesty, fail-closed sensors, the three pillars, or virt learning scope. Do not override principles for a “runs Linux” headline.

  • Honesty ledger — Verified only with a probe
  • Antifragility — fail-closed smoke + ratchets
  • Security — threat model; probed mitigations only
  • Performance — measure first; no invented benches
  • Document-first / one milestone → one branch → one PR

Track A (track-a.md) should largely complete before deep Linux ABI work. Reuse A1–A9 where they help. Full Linux ABI is not a promise. Not claiming Linux userspace yet (ADR-031). Guest OCI/Docker/k8s is a non-goal (ADR-029) — not a later Planned row on this track.

Goal

Explore what Linux userspace / ABI-subset work would take without dropping ctos specificity. Frame: ADR-031. This is an ADR/research ladder, not a distro.

Keep

Honesty ledger, fail-closed smoke (including Docker/arm64 workstation ratchets), pillars (antifragility / security / performance), ADR-gated ISA/memory/EL0 decisions, QEMU virt until an ADR widens it.

Ordered research

IDWorkStatus (2026-09-12)
B1ADR: Linux-compat goals & non-goalsDocumented (#41, ADR-031). ABI subset research; keep ctos specificity; not claiming Linux userspace yet. Cites ADR-029; does not reopen containers as Planned.
B2Syscall surface map (Linux aarch64 vs ctos SVC)Documented (#42, linux-aarch64-syscall-gap.md). Inspection map only. No Linux numbers in src/. Not claiming Linux userspace.
B3Process model vs Linux (fork/exec/wait)Documented (#43, ADR-035). Standing EL0 stays the ctos model. Linux clone/execve/wait4 is a gap. Do not add those syscalls to src/. Not claiming Linux userspace.
B4Linux ELF / auxv / PT_INTERP vs freestanding loaderDocumented (#44, ADR-033). Gap ADR only. Track A loader stays reusable. Does not accept PT_INTERP. Not claiming dynamic Linux ELF.
B5Linux VFS concepts vs thin ctos VFSDocumented (#45, ADR-034). Concept compare only. No POSIX flags / dentries in src/. Not claiming a Linux filesystem.
B6Decision: compat layer vs reimplement vs neverDecided (never) (#46, ADR-036). Research ladder complete. No Linux-compat implementation epic from B1–B5. Track A freestanding path stays.
B7Containers remain a non-goal (OCI needs a Linux host)Documented (#47, ADR-029). Site: hosting-apps.md (extra: host-apps.md). Ledger: absence Verified; not Planned.

B1 — Linux-compat frame

ADR-031 is the Track B frame. Linux-compat here means research into a possible Linux AArch64 ABI subset, not a promise to run a distro, glibc, musl, a shell, or Linux userspace.

Keep: honesty ledger, fail-closed smoke (including host Docker/arm64 workstation ratchets), three pillars, the freestanding Track A path (A1–A9 first cuts), QEMU virt until an ADR widens it.

Out: full Linux ABI as a promise; guest containers (ADR-029); replacing SVC #n / libctos with Linux x8 numbers in a research mile; claiming userspace because A3 loads ELF64 or A9 reads FAT /hello.

B2 is Documented (syscall gap table). B3 is Documented (process-model stance). B4 is Documented (ELF / auxv / PT_INTERP gap). B5 is Documented (VFS concept compare). B6 is Decided (never) (ADR-036): no Linux-compat layer or reimplementation from this ladder. File presence of these ADRs is not a Linux userspace probe.

B2 — syscall gap map

linux-aarch64-syscall-gap.md maps a representative Linux AArch64 set (exit, write, read, openat, close, brk/mmap, clone/execve/wait4, ioctl, plus CRT/namespace neighbors) onto today’s ctos SVC surface (reserved 0–2, public 16–23).

Keep: no Linux svc #0 / x8 numbers in src/ this mile. Track A SVC #<n> stays. No row is present (convention + number space both miss). Related Track A SVCs are partial. Namespace/mount stay never-per-ADR-031 (ADR-029).

B3–B5 must not treat this table as an implementation backlog. B6 chose never (ADR-036).

B3 — process model vs standing EL0

ADR-035 compares Linux fork / exec / wait to ctos standing EL0 (ADR-024) and the EL1 cooperative scheduler (ADR-010).

Keep: standing EL0 is one loaded trip until exit. EL1 coop workers are not Linux threads. A1–A9 probes stay. Do not add clone / execve / wait* to src/ this mile.

Out: treating A4 or A9 FAT /hello as execve; treating SYS_EXIT as exit_group; a pid table; CLONE_NEW* (already never-per-ADR-031).

On Linux AArch64, userspace fork() is clone. There is no fork syscall. B4 is Documented (ADR-033). B5 is Documented (ADR-034). B6 is Decided (never) (ADR-036). File presence of ADR-035 is not a Linux process table.

B4 — Linux ELF / auxv / PT_INTERP vs the freestanding loader

ADR-033 compares Linux exec of an ELF (interpreter + auxv + PT_DYNAMIC) with today’s A3 loader (ADR-023).

Keep: the Track A parser reusable (ET_EXEC, PT_LOAD, reject PT_INTERP, reject W+X). In-tree libctos apps stay on that path.

Out: accepting PT_INTERP in src/; claiming musl/glibc or dynamic Linux ELF; treating A9 FAT /hello as execve.

Static musl still needs a Linux stack/auxv and the B2 syscall surface. Dynamic musl/glibc also need an interpreter. Not claiming dynamic Linux ELF. B3 is Documented. B5 is Documented. B6 is Decided (never) (ADR-036).

B5 — Linux VFS vs thin ctos VFS

ADR-034 compares Linux VFS objects (dentry / inode / path / mount) to today’s thin VFS (ADR-027 / ADR-028).

Keep: no POSIX open flags, dentries, cwd, or mount(2) in src/ this mile. Track A VfsOps stays five calls. No Linux VFS object is present. Path strings and handles are partial. Mount / overlay stay never-per-ADR-031. File-descriptor numbers stay on the B2 gap map (openat / read / write / close partial).

B6 chose never (ADR-036) — this compare is not an implementation backlog. B3 and B4 are Documented. Not claiming a Linux filesystem.

B7 — containers are a non-goal

ctos guests do not host containers. OCI / Docker / Kubernetes need a Linux kernel (namespaces, cgroups, a Linux ABI, usually overlay). This tree has none of that in src/.

Today the arrow is reversed: a Linux host may run Docker/docker-smoke.sh to build and QEMU-smoke the ctos image. That is a host harness (NFR-04 / NFR-05), not a guest runtime.

Do not claim a Docker/OCI host. Do not treat this as far-later work waiting on Track B. Reopen only with a new GitHub epic plus a new ADR — not a Track B “win,” and not a new FR/NFR ID.

B6 — decision (never)

ADR-036 records the Track B outcome: never start a Linux-compat layer or Linux-like reimplementation from B1–B5.

Keep: Track A freestanding path; B1–B5 / B7 as the honesty record of the gap; principles over tracks.

Out: treating the research tables as a backlog; accepting PT_INTERP / Linux x8 / clone/execve / POSIX VFS / guest containers “later on Track B.”

Reopen only with a new epic + ADR + named workload probe.

Track N — network foundation (virtio-net ARP + ICMP + UDP + thin TCP + EL0 net/UDP samples)

Scope ADR: ADR-063. Separate from Track A (freestanding apps) and Track B (Linux-compat research — never implementation).

Driving force (non-negotiable)

This track is subordinate to ctos core principles (principles.md). Networking must not abandon honesty, fail-closed sensors, the three pillars, or virt learning scope. Do not override principles for a “has networking” headline.

  • Honesty ledger — Verified only with a probe; N0 is Planned for runtime claims
  • Antifragility — fail-closed smoke + ratchets; do not break blk/FAT markers
  • Security — threat model names the I/O surface; no “secure OS”
  • Performance — measure first; no invented benches
  • Document-first / one milestone → one branch → one PR

Goal

Bring up a minimal guest network path on QEMU virt: virtio-mmio virtio-net, discover the device, TX/RX one raw Ethernet frame (N1), an ICMP echo to the SLIRP gateway (N2), a UDP datagram to SLIRP DNS (N3), then a thin TCP active-open + one payload vs QEMU guestfwd echo (N5), with serial markers and fail-closed smoke. Reuse virtio-mmio patterns from blk (ADR-028). N1: ADR-066. N2: ADR-067. N3: ADR-070. N3.x EL0 UDP SVC: ADR-071. N5 thin TCP: ADR-074. EL0 TCP SVC: ADR-076.

Keep

Honesty ledger, fail-closed qemu-smoke, existing virtio-blk + FAT16 + sample catalog markers, QEMU virt + AArch64 until an ADR widens it, Track A freestanding path.

Mile ladder

IDWorkProbe that closes itStatus
N0Scope ADR + docs wiringDocs-build; ledger Planned row; no Verified inventAccepted (docs) — ADR-063
N1Link bring-up / first frameDiscover + TX/RX one raw Ethernet frame; QEMU -netdev / -device virtio-net-device; smoke greps; keep blk/FAT greenVerified when this tip’s qemu-smoke prints net: ok (ADR-066)
N2ARP + ICMP ping (kernel-path)After ARP: ICMP echo request/reply vs 10.0.2.2; net: icmp-tx / net: icmp-rx / net: ping-ok; keep N1 + blk/FAT greenVerified when this tip’s qemu-smoke prints net: ping-ok (ADR-067)
N3Transport — minimal UDP (IPv4)UDP TX/RX vs SLIRP DNS 10.0.2.3:53; net: udp-tx / net: udp-rx / net: udp-ok; keep N1/N2 + blk/FAT greenVerified when this tip’s qemu-smoke prints net: udp-ok (ADR-070)
N4Freestanding sample using net SVCsAfter N1+ Verified; EL0 net_mac/net_ping + FAT /netdemo; smoke grepsVerified when this tip’s qemu-smoke prints libctos: net-ok / netdemo: ok (ADR-068)
N3.xEL0 UDP DNS SVC + freestanding sampleAfter N3 Verified; EL0 net_udp_dns + FAT /udpdemo; smoke grepsVerified when this tip’s qemu-smoke prints libctos: udp-ok / udpdemo: ok (ADR-071)
N5Thin TCP (active open + one payload)After N3; TCP SYN/est + one payload echo vs guestfwd 10.0.2.4:7; net: tcp-syn / net: tcp-est / net: tcp-tx / net: tcp-rx / net: tcp-ok; keep N1–N3 + blk/FAT greenVerified when this tip’s qemu-smoke prints net: tcp-ok (ADR-074)
N5.xEL0 thin TCP SVC + sampleAfter N5; SYS_NET_TCP_ECHO (28) quiet guestfwd echo; FAT /tcpdemo; libctos: tcp-ok / tcpdemo: ok; keep net: tcp-okVerified when this tip’s qemu-smoke prints libctos: tcp-ok / tcpdemo: ok (ADR-076)

Explicit non-goals (locked until new ADR + sponsor)

BSD sockets as product; TCP stack as product / listen/accept / multi-connection socket table (N5 is one active-open + one payload only — ADR-074); TLS; HTTP stack; DHCP/DNS as product features (N3/N3.x may use SLIRP DNS as probe bait only); Wi‑Fi; virtio-pci-only foundation stories; Linux net stack; “has networking / sockets OS” marketing. N3.x adds EL0 net_udp_dns only (ADR-071); N5.x adds EL0 net_tcp_echo only (ADR-076); general udp_send/udp_recv / listen-accept / sockets ABI need another ADR. N4 EL0 net ABI remains the tiny net_mac/net_ping surface (ADR-068).

Honesty

Do not say the guest “has networking” or “has sockets.” N0 is scope (ADR-063). N1 first-frame is ADR-066 (net: ok). N2 ICMP ping is ADR-067 — Verified only with net: ping-ok on a named tip. N3 UDP is ADR-070 — Verified only with net: udp-ok on a named tip. N3.x EL0 UDP SVC sample is ADR-071 — Verified only with libctos: udp-ok / udpdemo: ok on a named tip. N4 EL0 net sample is ADR-068 — Verified only with libctos: net-ok / netdemo: ok on a named tip. N5 thin TCP is ADR-074 — Verified only with net: tcp-ok on a named tip. N5.x EL0 TCP SVC sample is ADR-076 — Verified only with libctos: tcp-ok / tcpdemo: ok on a named tip. Still not BSD sockets / TCP product / DHCP/DNS product. CloudAgent HELD; evidence tags agent-box / amd64 workstation. No self-merge (ADR-002).

B2 — Linux AArch64 syscalls vs ctos SVC

Track B child issue #42 (parent #40). Frame: ADR-031. Freestanding ABI: ADR-021 / syscall.md.

This page is a gap map. It does not add Linux syscall numbers to src/. It does not decide compat layer vs reimplement vs never (B6 / #46). Not claiming Linux userspace. File presence of this note is not a Linux ABI.

Probe (what Verified means here)

ClaimProbeStatus
Linux AArch64 numbers in the tableRead include/uapi/asm-generic/unistd.h from Linux v6.10 (torvalds/linux). AArch64 uses that generic table (svc #0, number in x8).Document inspection
ctos public / reserved SVC surfaceRead src/syscall.rs (SYS_* 16–23) + ADR-021 + ADR-027 + reserved 0–2 in ADR-013 on this treeDocument inspection
Linux userspace / musl / Alpine on the guestNo such serial markerNot claimed. Ledger “Guest runs host apps” stays Planned.

A later kernel qemu-smoke is a different row. This mile did not edit src/.

Status words

StatusMeans
presentSame Linux AArch64 number and comparable semantics, callable by a Linux svc #0 / x8 binary with no translation.
partialA Track A SVC exists with a related effect. Number, calling convention, args, or meaning still differ.
absentNo ctos analog. B3–B6 may still discuss a named subset. Not a promise to implement.
never-per-ADR-031Would imply a full Linux host / distro / container promise that ADR-031 already ruled out (guest OCI also ADR-029).

No Linux AArch64 syscall is present. The convention and the number space both miss.

Calling convention (the first gap)

ItemLinux AArch64ctos (ADR-021)
Instructionsvc #0 (immediate is always 0)SVC #<n> — n is the number
Number registerx8Immediate (not x8)
Argsx0–x5x0, x1, x2 only
Returnx0; negative Linux errnox0; 0 or u64::MAX reject (not errno)
Unknown number-ENOSYSUnhandled immediate parks (fail-closed)

Linux svc #0 collides with the ADR-013 first-mile probe (SVC #0). ADR-021 rejected adopting the Linux convention for that reason. A translation shim would be a later epic after B6 — not this page.

Public ctos numbers 16–26 are also not the Linux meaning of those integers:

Immediate / x8Linux AArch64 (v6.10 generic)ctos
0io_setupADR-013 first-mile return (reserved)
1io_destroyADR-013 standing announce (reserved)
2io_cancelADR-013 standing restore (reserved)
16fremovexattrexit
17getcwduart_write
18lookup_dcookieyield
19eventfd2fs_create
20epoll_create1fs_open
21epoll_ctlfs_read
22epoll_pwaitfs_write
23dupfs_close
24dup3net_mac
25fcntlnet_ping
26inotify_init1net_udp_dns

You cannot “just accept Linux numbers” in the SVC immediate, and you cannot treat today’s 16–26 as Linux.

ctos surface (inspection)

Reserved 0–2 are probe-only. Public:

ctos #NameRelated Linux name(s)Why not present
16exitexit (93), exit_group (94)Ends the standing EL0 trip; no process table; not exit_group.
17uart_writewrite (64) to a TTYPL011 only; no fd; cap 64; \n → \r\n.
18yieldsched_yield (124)Dispatch-only hint; does not run EL1 sched::yield_now (ADR-010).
19fs_createopenat (56) + O_CREATFlat memfs name; no dirfd, flags, or mode.
20fs_openopenat (56)Existing name only; FAT /probe and /hello are the known on-disk files.
21fs_readread (63)Handle 1–4; cap 64; no stdin. UART RX is kernel-side, not this SVC.
22fs_writewrite (64)memfs only (FAT write is ReadOnly); cap 64; not fd 1/2.
23fs_closeclose (57)Drops the handle; the file stays. Four handles max.
24net_mac—Guest MAC copy; not Linux dup3 (24). Kernel owns NIC (ADR-068).
25net_ping—Quiet ICMP echo; not Linux fcntl (25).
26net_udp_dns—Quiet UDP DNS probe; not Linux inotify_init1 (26). DNS bait only (ADR-071).

Unknown immediates park. Not POSIX. Not Linux VFS.

Gap table (representative Linux AArch64 set)

Numbers from Linux v6.10 asm-generic/unistd.h. AArch64 has no fork, open, creat, dup2, pipe, poll, or select syscalls — those are clone / openat / dup3 / pipe2 / ppoll / pselect6. __NR_syscalls in that header is 463. This table is a slice, not the full list.

Process and lifetime (B3)

Linux #Namectos statusNotes
93exitpartialAnalog SYS_EXIT 16. Halts the EL0 trip and returns to the EL1 caller. Status is recorded. Not a thread/process exit.
94exit_groupabsentNo process group. musl/glibc CRT usually ends here.
—forkabsentNot in the Linux AArch64 table. Userspace fork() is clone. Stance: ADR-035 / B3 / #43.
220cloneabsentNo child address space, no CLONE_*. Standing EL0 (ADR-024) is one loaded trip, not clone (ADR-035). CLONE_NEW* is never-per-ADR-031 (see namespace rows).
435clone3absentSame class as clone.
221execveabsentA3 maps freestanding PT_LOAD and rejects PT_INTERP. A9 FAT /hello is an EL1 load path, not execve (ADR-035). ELF / auxv / interpreter gap: ADR-033 (B4).
281execveatabsentSame class as execve.
260wait4absentNo child to wait for. ADR-035.
95waitidabsentADR-035.
129killabsentNo pid / signal delivery.
172getpidabsentNo process table.
178gettidabsentNo tid.
96set_tid_addressabsentTypical musl/glibc _start requirement.

File descriptors and I/O (B5 concept compare: ADR-034)

Linux #Namectos statusNotes
56openatpartialfs_open 20 / fs_create 19. Path is / + [a-z0-9_-] (1–15). No dirfd, O_*, or mode. At most 8 files. ADR-027.
437openat2absentNo open_how.
57closepartialfs_close 23. File stays after close.
63readpartialfs_read 21. Cap 64. No fd 0 / line discipline.
64writepartialuart_write 17 (console, no fd) or fs_write 22 (memfs). Neither is POSIX write(1, …).
65 / 66readv / writevabsentNo iovec.
67pread64absentHandle offset exists inside VFS; no positioned SVC.
62lseekabsentNo seek SVC.
29ioctlabsentNo TTY (TCGETS / TIOCGWINSZ), no block ioctl. PL011 is kernel-owned.
25fcntlabsentNo F_GETFL / F_DUPFD.
23 / 24dup / dup3absentHandles 1–4 only; no dup. Linux 23 is dup; ctos 23 is fs_close.
59pipe2absentNo pipes.
61getdents64absentNo directory listing. Flat namespace.
79 / 80newfstatat / fstatabsentNo stat.
291statxabsentNo stat.
48faccessatabsent
34mkdiratabsentNo directories.
35unlinkatabsent
38renameatabsent
49chdirabsentNo cwd.
17getcwdabsentLinux 17 is getcwd; ctos 17 is uart_write.

Memory (loader / libc heap)

Linux #Namectos statusNotes
214brkabsentKernel first-fit heap is EL1 (ADR-009). No user program break.
222mmapabsentA3 maps PT_LOAD at load time into the user window. No mmap SVC. Anonymous / file maps are out.
215munmapabsent
226mprotectabsentImage W^X is a kernel policy (ADR-015), not an EL0 syscall.
233madviseabsent

Scheduling, time, sync, identity

Linux #Namectos statusNotes
124sched_yieldpartialAnalog yield 18 is a recorded hint and ERET to EL0. It does not switch EL1 tasks.
101nanosleepabsentCNTP tick is EL1.
113clock_gettimeabsentNo guest clock SVC. CNTPCT probes are kernel/NFR-07.
98futexabsentNo user futex.
99set_robust_listabsentTypical libc _start.
134 / 135 / 139rt_sigaction / rt_sigprocmask / rt_sigreturnabsentLower-EL IRQ still parks. No POSIX signals.
73ppollabsent
160unameabsent
167prctlabsent
163 / 261getrlimit / prlimit64absent
278getrandomabsent

Sockets (not a Track B promise)

Linux #Namectos statusNotes
198socketabsentNo virtio-net, no stack. Same class as the host-apps “sockets” gap. Not a B2 implementation item.

Namespace / mount — never on this track

Linux #Namectos statusNotes
97unsharenever-per-ADR-031Linux namespaces. Guest OCI is a non-goal (ADR-029).
268setnsnever-per-ADR-031Same.
40mountnever-per-ADR-031Linux mount/overlay class. A7 FAT is not mount(2). Concept compare: ADR-034 (B5). Must not grow POSIX mount.
39umount2never-per-ADR-031Same.
41pivot_rootnever-per-ADR-031Same.

clone with CLONE_NEWNS / CLONE_NEWPID / CLONE_NEWNET is the same never-per-ADR-031 class. A later clone subset, if B6 ever names one, would still exclude those flags.

If a subset is ever useful (not a decision)

Issue #42 asked to prioritize a minimal subset if any. This is a research hint for B6, not an implementation list.

  1. In-tree libctos apps need no Linux numbers. A1–A9 already have exit / uart_write / yield / memfs / FAT /hello. Keep that path.
  2. A static musl “hello” is not a small subset. Besides convention translation, typical CRT wants set_tid_address, exit_group, brk or mmap, often write on fd 1, uname / geteuid-class probes, and signal/prctl stubs. That is already a libc-shaped surface. ADR-031 rejected “runs musl / Alpine” as a Track B promise.
  3. The smallest named research slice (only if B6 chooses a compat path, and only with a probe) would still be: convention translation + exit/exit_group + write to a console fd. openat/read/close are the next file slice. brk/mmap are the next heap slice. clone/execve/wait4 are the process gap (ADR-035); they are not a small add-on.
  4. Do not start with ioctl, sockets, or mount. ioctl is a device encyclopedia. Sockets need a stack. mount/unshare stay never-per-ADR-031.

B3 is Documented (ADR-035). B4 is Documented (ADR-033). B5 is Documented (ADR-034). They must not add Linux numbers to src/. B6 may choose never.

Honesty

  • Analysis Verified means: this file was read against src/syscall.rs and Linux v6.10 unistd.h. It is not ABI Verified and not Linux userspace.
  • Do not say “ctos implements write(2)” because uart_write or fs_write exists.
  • Do not mint FR-16+ / NFR-15+. Frozen set: fr-nfr.md.
  • Guest containers stay non-goal. Host docker-smoke.sh is unrelated.

Roadmap: track-b.md. Extra stance: host-apps.md.

Core principles (driving force)

The project name is ctos. ctsOS is an optional display nickname only — use ctos in docs, issues, and code.

These principles are the driving force. They are non-negotiable. Track A, Track B, and A9 are subordinate: they never override principles for speed or marketing.

PrincipleWhat it meansHome
Honesty ledgerStatus words need a probe. Unprobed stays Unknown. No rounding up.honesty-ledger.md
AntifragilityFail-closed sensors. Repeated failures become ratchets, not README advice.antifragility.md / NFR-05
SecurityWritten threat model. Mitigations are claimed only when probed. No “secure OS.”security.md / NFR-10
PerformanceMeasure first. No invented benches. Optimize only after a probe.performance.md / NFR-07
Document-first / one-PR loopsOne milestone → one branch → one GitHub PR. Author ≠ merger.ADR-002, roadmap

Landing page (same list, site SoT): Driving principles. Hub: overview.md. Pillars: pillars.md. Vision: product-vision.md.

ctos harness map

A reliable kernel session is:

Shared understanding + domain + outer harness.

That three-part split is common harness-engineering language (prior art: architecture.artof.link). On ctos it is not a product name. It is a map for this repo.

Shared understanding

The session’s reviewable model of intent and state:

  • Vision, architecture, ADRs, roadmap
  • Honesty ledger (what we claim vs what we probed)
  • Daily briefs and random-thoughts (handoff, not source of truth)

If a claim is only in chat, it is not shared understanding.

After M9, shared understanding also names the three pillars (antifragility, security, performance). A “secure” or “faster” sentence without a ledger row is not shared understanding.

Domain

The thing that is allowed to decide what is true about the machine:

  • Kernel source and the custom AArch64 target
  • QEMU virt (and later real hardware, if probed)
  • Kernel ELF loaded by -kernel
  • CPU, UART, interrupt, and memory behavior

Agents interpret. The domain (a build, a QEMU probe, a test) decides. Do not “confirm boot” from a README sentence.

Outer harness (six layers, kernel-scaled)

LayerOn ctosPrevents
GuidesAGENTS.md, .cursor/rules/, .cursor/skills/ctos-*Out-of-scope work (wrong crate era, shop content, GitLab SOPs)
Sensorsscripts/qemu-smoke.sh, cargo test semihosting exit, GHA smoke.yml, optional DockerRounding “source exists” up to “it boots”
LoopOne milestone → one branch → one GitHub PRSprawling PRs that mix UART, paging, and docs rewrites
Memoryresearch/ vaultsSession amnesia; stuffing raw chat into the next prompt
PermissionsAuthor ≠ merger (artofdream vs cursor[bot]); MRC writes COMMENT; no GitHub self-APPROVE (ADR-002)Same-login stamp counted as a second review
ObservabilityHonesty ledger + PR text that lists probed vs UnknownStatus theater

Fail closed: if QEMU was not run, boot status is Unknown. Do not write Verified.

ctos honesty ledger

Status words are claims. Each row needs a probe. Unprobed = Unknown. Never round Unknown up to Verified because the file exists, a PR is open, or a README teaches cargo run.

Allowed flags: Verified (probe passed), Unknown (no probe or probe blocked), Planned (not built yet), Failed (probe ran and lost).

ClaimProbeStatusNotes
UART hello source present (println!("Hello World!"); + PL011 writer)Read src/main.rs and src/uart.rs on this branchVerifiedSource inspection. Boot is a separate row.
.cargo/config.toml present (build-std, aarch64-ctos.json, qemu runner, json-target-spec)Read .cargo/config.tomlVerifiedPresence. Build success is the next row.
cargo build for aarch64-ctos.jsoncargo +nightly build on 2026-09-08 (rustc 1.100.0-nightly cea272fa3)VerifiedFirst attempt failed: invalid aarch64 ABI combination until the JSON had both "abi": "softfloat" and "rustc-abi": "softfloat". Then the ELF built (target/aarch64-ctos/debug/ctos, AArch64, entry 0x40080000).
QEMU aarch64 serial shows Hello Worldqemu-system-aarch64 8.2.2, -machine virt -cpu cortex-a57 -display none -serial stdio -kernel target/aarch64-ctos/debug/ctos; timeout 4VerifiedSerial printed Hello World! then the VM was killed (exit 124). Same string on -machine virt,gic-version=3 and -cpu max. One cloud environment, not CI. Not a Raspberry Pi probe.
x86_64 VGA / bootimage pathHistorical probes on 2026-09-08 (PR #2)HistoricalPath removed by ADR-003. Those Verified rows do not apply to this tree.
CI on GitHubWorkflow file exists and a run is greenVerified.github/workflows/smoke.yml: push run 34285784458 success (ubuntu-24.04 52s, ubuntu-24.04-arm 1m9s). PR run 34285786641 success. ubuntu-24.04-arm label is available on this repo.
Docker smoke (Dockerfile / scripts/docker-smoke.sh)the arm64 workstation Docker Desktop linux/arm64, 2026-09-09: docker build -t ctos-smoke . && docker run --rm ctos-smoke → exit 0VerifiedAfter three Failed runs, ratchets landed in git and the full smoke passed on the arm64 workstation: serial Hello World! (hello QEMU timeout 124, expected); cargo test [ok]; force-fail exit 1; qemu-smoke: ok. Failures that were ratcheted: (1) CRLF shebang → exec ./scripts/qemu-smoke.sh: no such file or directory (.gitattributes *.sh/Dockerfile eol=lf, image sed, CMD bash); (2) linker cc not found on compiler_builtins (build-essential); (3) failed to find romfile "efi-virtio.rom" (qemu-efi-aarch64 + ipxe-qemu). One sponsor host, not CI and not this cloud VM (still no Docker engine here).
Second-brain vaults (research/)Paths exist; README explains Procedure / Correction / Relationship / Daily BriefVerifiedStructure present. Not a claim that vaults are richly filled. Optional Obsidian UI is structure-only (PR #3).
Thin ctos-* rolesAGENTS.md + .cursor/skills/ctos-*/SKILL.md existVerifiedFour roles. No aea-* names.
Frozen FR/NFR IDs (FR-01–FR-15, NFR-01–NFR-14)Read docs/02-requirements/fr-nfr.md; IDs present; ISA text revised under ADR-003VerifiedFrozen IDs. Text now AArch64/UART. Do not invent extra FR/NFR IDs in chat. File presence is not a claim that every Now row is implemented.
PR identity split (author ≠ merger; artofdream vs cursor[bot])Read ADR-002 (amended 2026-09-12), AGENTS.md, .cursor/rules/pr-identity-no-self-merge.mdcVerifiedDocs present. Principle reused from Café Fausse pr-coordinator (identity only). Going-forward rule: owner-opened → cursor[bot] merges; bot-opened → artofdream merges. File presence of the rule is not a practiced split. Distinct-identity merge on this repo (author ≠ mergedBy): Unknown until gh shows it on a post-amendment PR. cursor[bot] merge / App APPROVE on this repo: Unknown until probed on artofdream/ctos. Track A same-login merges are the Historical row below, not this probe.
Track A A1–A9 same-login merge (ADR-002 historical exception)gh pr view --json author,mergedBy on 2026-09-12 for #49, #52–#57, #59, #60HistoricalEach PR: author=artofdream and mergedBy=artofdream. Commits typically cursoragent + Co-authored-by: artofdream. Named exception (2026-09-11–2026-09-12); do not rewrite those PRs. Not a kernel-boot claim. Not permission to repeat.
Primary ISA is AArch64Read ADR-003; x86_64-ctos.json / src/vga_buffer.rs absentVerifiedDecision + file tree. QEMU boot is a separate row.
Integration tests / QEMU test exit (M2)cargo +nightly test → QEMU virt + -semihosting; #[test_case]; QEMU host exit 0Verified2026-09-08 cloud: Running 2 tests / [ok]. 2026-09-09 M3: 4 tests. 2026-09-09 M4 re-probe: Running 6 tests / all [ok] (adds SPSel + stack-range cases).
Fail-closed test paniccargo +nightly test --features force-failVerified2026-09-08 and 2026-09-09 cloud: panic force-fail, QEMU/host exit 1.
scripts/qemu-smoke.shRan on this cloud VMVerified2026-09-09 M4 (qemu-system-aarch64 8.2.2, rustc 1.100.0-nightly 4aa1fbcf4): hello (timeout 124), exception: sync BRK (esr=0xf2000000), exception: fatal nested (kind=0x200), cargo test 0 (6 cases), force-fail 1, qemu-smoke: ok. First hello run Failed: MSR SP_EL1 at EL1 is UNDEF (esr=0x2000000) before Hello World! — fixed by setting SP_EL1 via mov sp while SPSel=1 (ADR-005). Not Docker. GHA on this PR is a separate row.
VBAR_EL1 installed at 2 KiB-aligned exception_vectors (M3 / FR-06)cargo +nightly test vbar_el1_points_at_tableVerified2026-09-09 cloud: [ok]. CurrentEL==1 and mrs vbar_el1 equals exception_vectors. ELF also had the table at 0x40080800.
Current-EL sync BRK handler runs and returns (M3 / FR-06)cargo +nightly test breakpoint_from_current_el; hello-kernel serialVerified2026-09-09 cloud: test [ok]; hello serial exception: sync BRK esr=0xf2000000 elr=0x40081190 then timeout 124 (wfe loop). Other sync classes and lower-EL slots are parks — taking those is unprobed.
qemu-smoke requires BRK handler stringscripts/qemu-smoke.sh hello phase greps exception: sync BRKVerified2026-09-09 cloud: qemu-smoke: BRK handler string present. Extends the M2 sensor (FR-06 / NFR-04).
Lower-EL AArch64 sync is live (SVC / IABORT); IRQ while standing live (ADR-040/041); FIQ while standing live (ADR-043); SError parkSource read of vector table; taken path is the EL0 first-mile / ADR-040 / ADR-041 / ADR-043 rowsVerifiedSource: sync_lower_el + irq_lower_el + fiq_lower_el; default eret_to_el0 SPSR 0x340. SError lower-EL is standing-aware (ADR-045 EXPECT) but no working host inject on virt/a57; AArch32 still parks. Taken lower-EL FIQ: see ADR-043 row. Taken SError: see ADR-045 row.
CI on GitHub (M3 PR / this branch).github/workflows/smoke.yml on cursor/m3-vbar-el1-exceptions-bd73VerifiedPush 34389110709 success (ubuntu-24.04 + ubuntu-24.04-arm). PR 34389132580 success. Same qemu-smoke.sh as the cloud probe (hello + BRK string + 4 tests + force-fail). Commit 72ecd37.
Dedicated SP_EL1 exception stack + SPSel=0 thread stack (M4 / FR-07)cargo +nightly test spsel_uses_thread_stack + stacks_are_distinct_and_alignedVerified2026-09-09 cloud: both [ok]. SPSel==0 and current SP in __stack_*; three stack ranges 16-byte aligned and disjoint.
Nested current-EL exception uses fatal stack and is observable (M4 / FR-07)Hello-kernel serial exception: fatal nested via scripts/qemu-smoke.shVerified2026-09-09 cloud: after two exception: sync BRK lines, serial exception: fatal nested then exception: fatal esr=0xf2000000 elr=0x400816cc kind=0x200. Nested AArch64 BRK after near-empty thread SP (ADR-005). Not an MMU overflow fault. Not GIC (M5).
qemu-smoke requires fatal nested stringscripts/qemu-smoke.sh hello phase greps exception: fatal nested and rejects fatal probe missedVerified2026-09-09 cloud: qemu-smoke: fatal nested string present. Extends the M3 sensor (FR-07 / NFR-04).
CI on GitHub (M4 PR / this branch).github/workflows/smoke.yml on cursor/m4-fatal-exception-stack-c8b7UnknownNot a green GHA URL on the post-fix revision before merge. M4 merged as PR #7 (180dbf2).
GICv2 init + CNTP PPI 30 tick observable (M5 / FR-08)Hello-kernel serial timer: tick via scripts/qemu-smoke.sh; #[test_case] timer_tick_is_observableVerified2026-09-09 cloud (QEMU 8.2.2, rustc 1.100.0-nightly 4aa1fbcf4): after Hello World!, serial timer: tick, then M3/M4 BRK + fatal. cargo test Running 9 tests all [ok] including gicd_typer_readable, cntfrq_is_nonzero, timer_tick_is_observable. Not GICv3. Not UART input (M6).
qemu-smoke requires timer tick stringscripts/qemu-smoke.sh hello phase greps timer: tick and rejects timer: tick missedVerified2026-09-09 cloud: qemu-smoke: timer tick string present. Extends the M4 sensor (FR-08 / NFR-04).
CI on GitHub (M5 PR / this branch).github/workflows/smoke.yml on cursor/m5-hardware-interrupts-d5a6VerifiedPush/PR run 34391557761 success (ubuntu-24.04 + ubuntu-24.04-arm) on the cloud-probe commit. Implementation commit run 34391552019 also success. Same qemu-smoke.sh (hello + tick + BRK + fatal + 9 tests + force-fail). Commit e28e4bb.
PL011 RX of injected 0x41 observable (M6 / FR-08 input)Hello-kernel serial input: rx 0x41 via scripts/qemu-smoke.sh + qemu-serial-inject.pyVerified2026-09-09 cloud (QEMU 8.2.2, rustc 1.100.0-nightly 4aa1fbcf4): after Hello World! and timer: tick, serial input: rx 0x41, then M3/M4 BRK + fatal. Not virtio-keyboard. Not QEMU LBE (unimplemented on 8.2).
qemu-smoke requires UART RX stringscripts/qemu-smoke.sh hello phase greps input: rx 0x41 and rejects input: rx missedVerified2026-09-09 cloud: qemu-smoke: UART RX string present. Extends the M5 sensor (FR-08 input / NFR-04).
Empty RX FIFO under cargo test (M6)#[test_case] uart_rx_fifo_empty_without_host_byteVerified2026-09-09 cloud: cargo test Running 10 tests all [ok]. Character proof is the serial row.
CI on GitHub (M6 PR / this branch).github/workflows/smoke.yml on cursor/m6-uart-rx-input-7ef6UnknownNo green GHA URL on the cloud-probe commit yet.
EL1 identity map + MMU on (M7 / FR-09)Hello-kernel serial paging: ok via scripts/qemu-smoke.sh; #[test_case] mmu_is_enabledVerified2026-09-09 cloud (QEMU 8.2.2, rustc 1.100.0-nightly 4aa1fbcf4): after Hello World!, serial paging: ok, then M5/M6/M3/M4 markers. cargo test mmu_is_enabled [ok]. Identity 1 GiB blocks (ADR-008). Not a DTB walk.
Frame alloc + map/unmap window (M7 / FR-09)#[test_case] frame_alloc_aligned_and_distinct + map_unmap_roundtrip; serial marker from paging::observe_probeVerified2026-09-09 cloud: both [ok]. Write-through window VA 0x8000_0000 matches identity PA.
qemu-smoke requires paging stringscripts/qemu-smoke.sh hello phase greps paging: ok and rejects paging: probe missedVerified2026-09-09 cloud: qemu-smoke: paging string present. Extends the M6 sensor (FR-09 / NFR-04).
CI on GitHub (M7 PR / this branch).github/workflows/smoke.yml on cursor/m7-paging-frame-allocator-b567UnknownNo green GHA URL on the cloud-probe commit yet.
GlobalAlloc + Box/Vec on identity-mapped frames (M8 / FR-10)Hello-kernel serial heap: ok via scripts/qemu-smoke.sh; #[test_case] box_alloc_roundtrip + vec_grows + heap_lives_in_frame_poolVerified2026-09-09 cloud (QEMU 8.2.2, rustc 1.100.0-nightly 4aa1fbcf4): after paging: ok, serial heap: ok, then M5/M6/M3/M4 markers. cargo test Running 17 tests all [ok]. First-fit + coalesce (ADR-009). Not a growing heap. Not M9.
qemu-smoke requires heap stringscripts/qemu-smoke.sh hello phase greps heap: ok and rejects heap: probe missedVerified2026-09-09 cloud: qemu-smoke: heap string present. Extends the M7 sensor (FR-10 / NFR-04).
CI on GitHub (M8 PR / this branch).github/workflows/smoke.yml on cursor/m8-heap-globalalloc-0ee8VerifiedPush run 34394697056 success (ubuntu-24.04-arm 59s, ubuntu-24.04 1m36s) on the cloud-probe commit ba6096a. Same qemu-smoke.sh (hello + paging + heap + tick + RX + BRK + fatal + 17 tests + force-fail).
Cooperative two-task yield (M9 / FR-11)Hello-kernel serial sched: task a / sched: task b / sched: ok via scripts/qemu-smoke.sh; #[test_case] two_tasks_run_on_distinct_heap_stacks + yield_round_robin_resumes_bothVerified2026-09-09 cloud (QEMU 8.2.2, rustc 1.100.0-nightly 4aa1fbcf4): after heap: ok, serial sched: task a, sched: task b, sched: ok, then M5/M6/M3/M4 markers. cargo test Running 19 tests all [ok]. Cooperative EL1 yield (ADR-010). Not preemptive. Not SMP.
qemu-smoke requires scheduler stringsscripts/qemu-smoke.sh hello phase greps sched: ok and both task markers; rejects sched: probe missedVerified2026-09-09 cloud: qemu-smoke: scheduler strings present. Extends the M8 sensor (FR-11 / NFR-04).
qemu-smoke requires CNTPCT perf stringscripts/qemu-smoke.sh hello phase greps perf: cntpct and rejects perf: probe missedVerified2026-09-09 cloud: qemu-smoke: CNTPCT perf string present. Extends the M9 sensor (NFR-07 / NFR-04).
CI on GitHub (M9 PR / this branch).github/workflows/smoke.yml on cursor/m9-cooperative-scheduler-7da9VerifiedFolded from #16 (not re-probed here): push 34395786658 and PR 34395791288 success (ubuntu-24.04-arm + ubuntu-24.04) on 88a9305. Merge-commit push 34396135202 success on 4e3b732. Same qemu-smoke.sh (hello + paging + heap + two-task sched + tick + RX + BRK + fatal + 19 tests + force-fail).
Three pillars accepted (ADR-011); NFR-05 / NFR-07 / NFR-10 text revised in placeRead ADR-011 and fr-nfr.md; IDs still NFR-05/NFR-07/NFR-10VerifiedDocument inspection. Not a kernel boot claim.
Threat-model v1 exists (NFR-10)Read security.md (assets, adversaries, trust boundaries, non-goals, mitigations→probes)VerifiedFile + review (2026-09-10, #18). Replaces the ADR-011 stub. “Secure OS” / “hardened”: unclaimed.
Threat-model v1.1 slice (NFR-10)Read security.md (linker stacks, guards, EL0 first mile, remaining W^X gaps)VerifiedFile + review. v1.1 update, not “secure.”
Threat-model v1.5 slice (NFR-10)Read security.md (standing EL0, TTBR1 first cut, EL1 high-VA fetch, PAN/identity teardown still Planned)VerifiedFile + PR #25. v1.5 update, not “secure.”
Threat-model v1.6 slice (NFR-10)Read security.md (identity-tear first cut, PAN/full teardown still Planned)VerifiedFile + PR #26. v1.6 update, not “secure.”
Threat-model v1.7 slice (NFR-10)Read security.md (high-VA jump + 16 KiB dedicated identity text range, live .text stays, PAN/full teardown still Planned)VerifiedFile + PR #27 (on main 24d94e6). v1.7 update, not “secure.”
Threat-model v1.8 slice (NFR-10)Read security.md (high-VA vtable rewrite + live identity .text tear, .rodata/.data/heap stay, PAN/full teardown still Planned)VerifiedFile + PR #28 (on main e80dc93). v1.8 update, not “secure.”
W^X / NX heap + coop stacks (ADR-012)Hello-kernel serial wx: nx heap + wx: ok via scripts/qemu-smoke.sh; #[test_case] heap_and_mmio_are_pxn + kernel_text_is_executable + execute_from_heap_is_caughtVerified2026-09-10 cloud (QEMU 8.2.2, rustc 1.100.0-nightly a36d05efa): after sched: ok, serial wx: nx heap then wx: ok. cargo test Running 25 tests all [ok]. L2/L3 PXN on [__kernel_end, RAM end) (ADR-012). Linker SP_EL0 / SP_EL1 / fatal stacks stay in the executable image. Device MMIO L1 is XN. Not a “secure OS” or “the kernel is W^X” claim.
qemu-smoke requires W^X stringscripts/qemu-smoke.sh hello phase greps wx: ok and rejects wx: probe missedVerified2026-09-10 cloud: qemu-smoke: W^X string present. Extends the pillars sensor (NFR-10 / NFR-04).
Baseline CNTPCT loop probe (NFR-07)Hello-kernel serial perf: cntpct via scripts/qemu-smoke.sh; #[test_case] cntpct_advances_over_loopVerified2026-09-10 cloud re-probe (QEMU 8.2.2, rustc 1.100.0-nightly a36d05efa): after wx: ok, serial perf: cntpct delta=22968. Prior 2026-09-09 pillars probe also Verified. Counter-advances baseline. Not a published bench.
IRQ-to-handler CNTPCT delta (NFR-07)Hello-kernel serial perf: irq-delta via scripts/qemu-smoke.sh; #[test_case] irq_delta_samples_recordedVerified2026-09-10 cloud (QEMU 8.2.2): after timer: tick, serial perf: irq-delta min=12973 max=35391 spread=22418 n=8. cargo test irq_delta_samples_recorded [ok]. CNTPCT−CVAL spread on this QEMU virt guest. Not a latency budget. Not “faster than X.”
qemu-smoke requires IRQ-delta stringscripts/qemu-smoke.sh hello phase greps perf: irq-delta and rejects perf: irq-delta missedVerified2026-09-10 cloud: qemu-smoke: IRQ-delta perf string present. Extends the NFR-07 sensor (NFR-04).
Linker-stack guard pages (ADR-014)Hello-kernel serial guard: fault + guard: ok via scripts/qemu-smoke.sh; #[test_case] linker_stack_guards_unmapped + store_to_thread_guard_is_caughtVerified2026-09-10 cloud (QEMU 8.2.2, rustc 1.100.0-nightly a36d05efa): after wx: ok, serial guard: fault then guard: ok. cargo test both guard cases [ok]. Unmapped 4 KiB holes (ADR-014). Live stack pages stay executable. Not “the kernel is W^X.”
qemu-smoke requires guard stringscripts/qemu-smoke.sh hello phase greps guard: ok and rejects guard: probe missedVerified2026-09-10 cloud: qemu-smoke: stack-guard string present.
Host debug ELF size (NFR-08)scripts/qemu-smoke.sh prints perf: elf-size bytes=<n> and rejects < 4096Verified2026-09-10 cloud: host perf: elf-size bytes=3752056 after cargo +nightly build. Measurement only. Not a budget. Not a bench.
EL0 entered and returned (SVC)Hello-kernel serial el0: svc + el0: ok; #[test_case] el0_svc_roundtripVerified2026-09-10 cloud: after guard: ok, serial el0: svc then el0: ok. cargo test el0_svc_roundtrip [ok]. is_active() stays false. Not userspace.
EL0 cannot execute kernel dataHello-kernel serial el0: nx kernel; #[test_case] el0_cannot_execute_kernel_dataVerified2026-09-10 cloud: serial el0: nx kernel. cargo test [ok]. Lower-EL UXN IABORT on kernel .data. 2026-09-11 hello re-probe (this branch) still prints el0: nx kernel (now also a translation IABORT under user TTBR0). Not isolation.
EL0 isolation (P-SEC-3)Standing EL0 + PAN enable + full TTBR1/higher-half teardown; since M6: every ADR-055 §A/§B row Met + sponsor’s written accept (B3) + M6 PR CI re-probeVerified (scoped: the accepted ADR-091 sentence only)M6 (2026-09-28): ADR-091 Accepted; “EL0 isolated” means only that sentence, with its limits (one core, the TCG default smoke machine; not a speculative-execution or side-channel claim, not a real-hardware claim, not a certification; app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN); no multi-core race story). Revision 4 (2026-10-02): the sponsor accepted ADR-091 rev 4 in writing (PAN clause + that limit changed; see the revision-4 row near the end of this file). Earlier notes: Direction ADR-013; decisions ADR-047; checklist ADR-055; sponsor closure ADR-060. Specific miles are different rows. PAN enable Verified on a76 (ADR-080; historical a57 lock ADR-054); taken SError hard-stopped (ADR-053 / ADR-081); never yank _start. Do not widen “EL0 isolated” past the ADR-091 sentence.
RO+NX text/data (ADR-015)Hello-kernel serial ro: nx data + ro: write fault + ro: ok; #[test_case] execute-from-.data + write-to-RO-textVerified2026-09-11 cloud qemu-smoke (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e, -cpu cortex-a57): after guard: ok, serial ro: nx data then ro: write fault then ro: ok. cargo test both RO cases [ok]. Identity image W^X on this virt guest (text RO+X, data/stacks/heap RW+NX, SCTLR.WXN on). Not a “secure OS.”
qemu-smoke requires RO+NX stringsscripts/qemu-smoke.sh hello phase greps ro: ok / ro: nx data / ro: write faultVerified2026-09-11 cloud: qemu-smoke: RO+NX string present.
Boot-to-ready CNTPCT (NFR-08)Hello-kernel serial perf: boot-delta; #[test_case] boot_delta_sample_existsVerified2026-09-11 cloud qemu-smoke: perf: boot-delta ticks=111019 (after paging::init → after Hello World!). First attempt Failed: pre-MMU .bss store lost on the test image. Measurement only. Not a budget. Not a bench.
qemu-smoke requires boot-delta stringscripts/qemu-smoke.sh hello phase greps perf: boot-deltaVerified2026-09-11 cloud: qemu-smoke: boot-delta perf string present.
User TTBR0 omits kernel .data#[test_case] user_ttbr0_omits_kernel_data; walk L1_USERVerified2026-09-11 cloud cargo test: user_ttbr0_omits_kernel_data [ok]. ASID=1 programmed. Not ASID isolation.
EL0 cannot read kernel .dataHello-kernel serial el0: no kernel read; #[test_case] el0_cannot_read_kernel_dataVerified2026-09-11 cloud qemu-smoke: after el0: nx kernel, serial el0: no kernel read then el0: ok. cargo test [ok]. User TTBR0 omits .data. is_active() stays false. Not isolation. Not PAN.
PAN enable on virtID_AA64MMFR1_EL1.PAN != 0 and an EL1-vs-EL0 access faultVerified (ADR-080)Default -cpu cortex-a76 (ADR-079 / ADR-080 / #125): pan: present + pan: enabled + pan: el1-fault (SPSR.PAN). Historical a57 pan: absent (ADR-026). Not “EL0 isolated.”
ASID isolationHello-kernel serial asid: dual + asid: conflict + asid: ok; #[test_case] asid_isolation_without_vmalle1; no TLBI VMALLE1 on the switchVerified2026-09-11 cloud qemu-smoke (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e, -cpu cortex-a57): after el0: ok, serial asid: dual then asid: conflict then asid: ok. cargo test asid_isolation_without_vmalle1 [ok] (Running 39 tests). Dual TTBR0 (ASID 1 vs 2) with nG pages; switch is MSR TTBR0 + ISB. At probe time the EL0 trampoline still TLBI VMALLE1 (.data leaves were global); ADR-039 later drops trampoline VMALLE1. Not “EL0 isolated.” Not PAN.
Standing EL0 contextHello-kernel serial el0: standing + el0: restored; #[test_case] standing_el0_enter_leave; is_active() true only while standingVerified2026-09-11 cloud qemu-smoke (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e, -cpu cortex-a57): after el0: no kernel read, serial el0: standing then el0: restored then el0: ok. cargo test standing_el0_enter_leave [ok] (Running 42 tests). Dual-SVC on user TTBR0 (not a trampoline flag). Default ERET clears IRQ mask (ADR-041). Not POSIX. Not isolation.
TTBR1 kernel-private page (ADR-016 first cut)Hello-kernel serial ttbr1: el1 + ttbr1: no el0 + ttbr1: ok; #[test_case] ttbr1_el1_sees_priv_el0_does_notVerified2026-09-11 cloud qemu-smoke: after asid: ok, serial ttbr1: el1 then ttbr1: no el0 then ttbr1: ok. cargo test ttbr1_el1_sees_priv_el0_does_not [ok]. EL1-only high page. Identity teardown Planned. Not a relocated kernel. Not “EL0 isolated.”
qemu-smoke requires standing EL0 stringsscripts/qemu-smoke.sh hello phase greps el0: standing / el0: restoredVerified2026-09-11 cloud: qemu-smoke: EL0 first-mile + read-mile + standing strings present.
qemu-smoke requires TTBR1 stringsscripts/qemu-smoke.sh hello phase greps ttbr1: ok / ttbr1: el1 / ttbr1: no el0; rejects ttbr1: leaked and ttbr1: probe missedVerified2026-09-11 cloud: qemu-smoke: TTBR1 private-page strings present.
EL1 fetch from TTBR1 high VA (ADR-017)Hello-kernel serial ttbr1: el1 exec + ttbr1: vbar; #[test_case] el1_executes_from_ttbr1_high_va + high_alias_maps_kernel_textVerified2026-09-11 cloud qemu-smoke (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e, -cpu cortex-a57): after asid: ok, serial ttbr1: el1 exec then ttbr1: vbar then the ADR-016 private-page lines. cargo test both new cases [ok] (Running 44 tests). _start stays at 0x4008_0000. Nested fatal ELR can be the high alias (elr=0xffffff80400839e4). Not a relocated kernel. Not “EL0 isolated.”
qemu-smoke requires TTBR1 high-VA exec stringsscripts/qemu-smoke.sh hello phase greps ttbr1: el1 exec / ttbr1: vbar; rejects ttbr1: exec missedVerified2026-09-11 cloud: qemu-smoke: TTBR1 private-page + high-VA exec strings present.
Full higher-half / identity teardownRemaining identity RAM after the heap plus identity _start unmappedDecided partialADR-020/025/037/038/049 tear live image + leftover frame RAM. Never yank _start while -kernel needs 0x4008_0000 (ADR-047). ident: start-stay Verified honesty. Leftover frame RAM after heap: Verified optional mile (ident: ram*, ADR-049). Do not claim the kernel moved.
Identity .rodata tear (ADR-025)Serial ident: rodata / ident: rodata-fault / ident: rodata-highVerifiedSee earlier ADR-025 rows.
Identity .data / stack tear (ADR-037)Hello-kernel serial ident: data-reloc + ident: data + ident: data-fault + ident: data-high; heap stay; #[test_case] data-torn + heap-still-mappedVerified2026-09-12 agent-box qemu-smoke (QEMU, nightly): ident: data-reloc n=241 / ident: data lo=0x40201000 hi=0x40258000 pages=87 / ident: data-fault / ident: data-high / ident: heap-stay / ident: ok; Running 85 tests all [ok] incl. identity_data_torn_high_stays + identity_heap_still_mapped; qemu-smoke: ok. (amd64 workstation) checkout target noted; this probe ran on the agent box. Heap stay is Historical after ADR-038. Not “EL0 isolated.”
Identity heap still mappedHello-kernel serial ident: heap-stay; #[test_case] identity_heap_still_mappedVerifiedSame smoke as ADR-037 row. Honesty marker, not a tear. Historical: superseded by ADR-038 on later tips.
Identity heap tear (ADR-038)Hello-kernel serial ident: heap-reloc + ident: heap + ident: heap-fault + ident: heap-high; #[test_case] heap-tornVerified2026-09-12 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on ident-heap-tear after dde185f: hello ident: heap-reloc n=0 / ident: heap lo=0x40258000 hi=0x40268000 pages=16 / ident: heap-fault / ident: heap-high / ident: ok; no ident: heap-stay; Running 86 tests all [ok] incl. identity_heap_torn_high_stays + identity_heap_unmapped_high_stays; A9 cross-update both slot:ok; force-fail exit 1; qemu-smoke: ok. (amd64 workstation) checkout target noted; this probe ran on the agent box — Docker on the amd64 workstation not re-run (CloudAgent HELD). Not “EL0 isolated.” Not “the kernel moved.”
Threat-model v1.18 slice (NFR-10)Read security.md (identity heap tear; PAN enable Planned)VerifiedFile + this PR. v1.18 update, not “secure.”
Docs site at https://ctos.artof.linkHTTP(S) fetch of the published Pages site on that hostVerified2026-09-11 after #30: curl -sL https://ctos.artof.link/ → HTTP 200; HTML title/menu ctos (ctsOS); landing #driving-principles. This VM repeat: same 200 + title + heading. Also 200: overview/what-can-run.html, overview/porting.html, framework/honesty-ledger.html. Deploy cited on the published-site row. Not a kernel boot claim.
High-VA vtable rewrite + live identity .text tear (ADR-020)Hello-kernel serial ident: reloc + ident: live + existing ident: ok; #[test_case] identity_fn_ptrs_rewritten_high + live_identity_text_unmapped_boot_stub_staysVerified2026-09-11 cloud qemu-smoke (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e, -cpu cortex-a57): ident: jump then ident: reloc n=12 then ident: range lo=0x400ae000 hi=0x400b2000 pages=4 then ident: live lo=0x40081000 hi=0x400a6000 pages=37 then Hello World! then later ident: split / ident: fault / ident: high / ident: text / ident: no el0 / ident: ok. Hello BRK ELRs high (elr=0xffffff8040082990). cargo test Running 50 tests all [ok] (ident: reloc n=74, ident: live pages=45). Not a relocated kernel. Not “EL0 isolated.” .rodata/.data/heap stay. Full teardown Planned.
Identity text range tear (ADR-019)Hello-kernel serial ident: jump + ident: range + ident: text + existing ident: ok; #[test_case] identity_text_range_unmapped_boot_stub_staysVerified2026-09-11 cloud qemu-smoke (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e, -cpu cortex-a57): ident: jump then ident: range lo=0x400ac000 hi=0x400b0000 pages=4 then later ident: split / ident: fault / ident: high / ident: text / ident: no el0 / ident: ok. Hello BRK ELRs high (elr=0xffffff8040082758). cargo test identity_text_range_unmapped_boot_stub_stays [ok] (Running 48 tests). Live .text stays. First attempt to yank live .text after the stub Failed (unhandled sync on println! — rustc dyn Write vtables). Not a relocated kernel. Not “EL0 isolated.” Full teardown Planned.
Identity-tear first cut (ADR-018)Hello-kernel serial ident: split + ident: fault + ident: high + ident: no el0 + ident: ok; #[test_case] high_ram_tables_are_independent + identity_tear_page_unmapped_high_stays + identity_tear_el1_faults_high_staysVerified2026-09-11 cloud qemu-smoke (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e, -cpu cortex-a57): after ttbr1: ok, serial ident: split then ident: fault then ident: high then ident: no el0 then ident: ok. cargo test all three new cases [ok] (Running 47 tests). _start stays at 0x4008_0000. Nested fatal ELR can be the high alias (elr=0xffffff8040083bec). Not a relocated kernel. Not “EL0 isolated.” Full teardown Planned.
qemu-smoke requires identity-tear stringsscripts/qemu-smoke.sh hello phase greps ident: ok / ident: split / ident: fault / ident: high / ident: no el0; rejects ident: leaked and ident: probe missedVerified2026-09-11 cloud: qemu-smoke: identity-tear strings present.
Obsidian open-vault checklistRead research/obsidian-checklist.md; .gitignore has .obsidian/VerifiedStructure only. .obsidian/ not committed. Not a claim that Obsidian is installed or synced.
CI on GitHub (pillars PR / ADR-011).github/workflows/smoke.yml on cursor/pillars-adr-011-325fVerifiedCloud-probe commit cdf75eb: push 34396574483 and PR 34396590545 success. Ledger follow-up d716599: push 34396774022 and PR 34396779239 success. Same qemu-smoke.sh (hello + paging + heap + sched + CNTPCT + tick + RX + BRK + fatal + 20 tests + force-fail).
scripts/qemu-smoke.sh (pillars follow-up / #18)Ran on that cloud VMVerified2026-09-10 (QEMU 8.2.2, rustc 1.100.0-nightly a36d05efa): hello (timeout 124) with paging: ok, heap: ok, sched: ok, wx: nx heap / wx: ok, perf: cntpct delta=22968, timer: tick, perf: irq-delta min=12973 max=35391 spread=22418 n=8, input: rx 0x41, two exception: sync BRK, exception: fatal nested / kind=0x200. cargo test Running 25 tests all [ok]. force-fail exit 1. qemu-smoke: ok. Not Docker.
scripts/qemu-smoke.sh (pillar deepen / this branch)Ran on this cloud VMVerified2026-09-10 (QEMU 8.2.2, rustc 1.100.0-nightly a36d05efa): host perf: elf-size bytes=3752056; hello (timeout 124) with paging: ok, heap: ok, sched: ok, wx: nx heap / wx: ok, guard: fault / guard: ok, el0: svc / el0: nx kernel / el0: ok, perf: cntpct delta=20592, timer: tick, perf: irq-delta min=14477 max=24149 spread=9672 n=8, input: rx 0x41, two exception: sync BRK, exception: fatal nested / kind=0x200. cargo test Running 29 tests all [ok]. force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this PR is a separate row.
CI on GitHub (pillars follow-up PR / #18).github/workflows/smoke.yml on cursor/pillars-sec-perf-el0-9bc0VerifiedCloud-probe commit 8ae63bb: push 34541457222 and PR 34541473721 success (ubuntu-24.04 + ubuntu-24.04-arm). Ledger follow-up 4c610a2: push 34541534491 and PR 34541537733 success. Same qemu-smoke.sh (hello + paging + heap + sched + W^X + CNTPCT + IRQ-delta + tick + RX + BRK + fatal + 25 tests + force-fail).
CI on GitHub (pillar deepen PR / #19).github/workflows/smoke.yml on cursor/pillar-loops-deepen-48aeUnknownNo green GHA URL on the cloud-probe commit recorded here. Merged as #19.
scripts/qemu-smoke.sh (RO+NX / boot-delta / user TTBR0 / PR #20)Ran on that cloud VMVerified2026-09-11 (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e): host perf: elf-size bytes=3785520; hello (timeout 124) with paging: ok, heap: ok, sched: ok, wx: nx heap / wx: ok, guard: fault / guard: ok, ro: nx data / ro: write fault / ro: ok, el0: svc / el0: nx kernel / el0: no kernel read / el0: ok, perf: cntpct delta=20506, perf: boot-delta ticks=111019, timer: tick, perf: irq-delta min=4885 max=26301 spread=21416 n=8, input: rx 0x41, two exception: sync BRK, exception: fatal nested / kind=0x200. cargo test Running 36 tests all [ok]. force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on that PR is a separate row.
CI on GitHub (RO+NX / boot-delta / user TTBR0 / PR #20).github/workflows/smoke.yml on cursor/pillar-deepen-ro-boot-el0-aadcVerifiedCloud-probe commit 6606be8: push 34561716967 and PR 34561719542 success (ubuntu-24.04 + ubuntu-24.04-arm). Host ELF there ~3.69 MiB — smaller than the arm64 workstation Docker 3786384. Do not copy that Verified onto the Docker host.
Docker smoke on main b2bbb99 (post PR #20)the arm64 workstation Docker Desktop linux/arm64 ./scripts/docker-smoke.sh after sync to b2bbb99FailedSponsor serial: perf: elf-size bytes=3786384; Hello World!; paging: probe missed; heap: probe missed; sched: probe missed; wx: probe missed; guard: fault / guard: ok; ro: nx data / ro: write fault / ro: ok; el0: probe missed; then CNTPCT / boot-delta / tick / irq-delta / RX / BRK / fatal still printed. qemu-smoke: missing 'paging: ok' EXIT 1. Guard/ro are current-EL identity walks — they do not need frame::ALLOC or USER_MAP_OK. Same class as the PR #20 pre-MMU .bss miss, plus a single shared L3 / first-2-MiB user map that a larger nightly layout can also break. Antifragility: this host found what GHA missed. Not this cloud VM.
scripts/qemu-smoke.sh (layout L3 pool / post-MMU frame init / this branch)Ran on this cloud VMVerified2026-09-11 (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e): host perf: elf-size bytes=3837736; hello (timeout 124) with paging: layout data=0x40201000 end=0x4023c000 pool=0x4023c000 user=1, paging: ok, heap: ok, sched: ok, wx: nx heap / wx: ok, guard: fault / guard: ok, ro: nx data / ro: write fault / ro: ok, el0: svc / el0: nx kernel / el0: no kernel read / el0: ok, perf: cntpct delta=21094, perf: boot-delta ticks=115251, timer: tick, perf: irq-delta min=12053 max=24899 spread=12846 n=8, input: rx 0x41, two exception: sync BRK, exception: fatal nested / kind=0x200. cargo test Running 38 tests all [ok] (adds layout_stress_crosses_first_ram_l2 + frame_allocator_ready_after_mmu). force-fail exit 1. qemu-smoke: ok. Loaded layout larger than the arm64 workstation Docker’s 3.78 MiB class (__data_start in the second RAM 2 MiB). Not Docker. GHA on this PR is a separate row.
CI on GitHub (layout L3 pool / PR #21).github/workflows/smoke.yml on cursor/layout-l3-user-map-c915VerifiedCloud-probe commit c04b84b: push 34563006005 and PR 34563008516 success (ubuntu-24.04 + ubuntu-24.04-arm). Merge-commit push 34563104020 success on 71ee15f. MRC grepped both-matrix serial: paging: layout data=0x40201000 … user=1, paging: ok, heap: ok, sched: ok, wx: ok, el0: ok, 38 tests, force-fail fail-closed. Do not copy that Verified onto arm64 workstation Docker.
Docker smoke after the layout fix (main 71ee15f)the arm64 workstation Docker Desktop linux/arm64 ./scripts/docker-smoke.sh after #21 mergeVerifiedSponsor re-run on 71ee15f: perf: elf-size bytes=3838592; Hello World!; paging: layout data=0x40201000 end=0x4023c000 pool=0x4023c000 user=1; paging: ok; heap: ok; sched: ok; wx: ok; guard: ok; ro: ok; el0: svc / el0: nx kernel / el0: no kernel read / el0: ok; perf: cntpct …; perf: boot-delta ticks=217103; perf: irq-delta …; input: rx 0x41; exception: sync BRK … / fatal nested …; Running 38 tests all [ok]; force-fail exit 1; qemu-smoke: ok; EXIT 0. Keep the b2bbb99 Failed row (antifragility history). Not this cloud VM.
qemu-smoke requires ASID isolation stringsscripts/qemu-smoke.sh hello phase greps asid: ok / asid: dual / asid: conflict; rejects asid: stale and asid: probe missedVerified2026-09-11 cloud: qemu-smoke: ASID isolation strings present.
scripts/qemu-smoke.sh (ASID isolation / this branch)Ran on this cloud VMVerified2026-09-11 (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e): host perf: elf-size bytes=3853144; hello (timeout 124) with paging: layout data=0x40201000 end=0x4023f000 pool=0x4023f000 user=1, paging: ok, heap: ok, sched: ok, wx: nx heap / wx: ok, guard: fault / guard: ok, ro: nx data / ro: write fault / ro: ok, el0: svc / el0: nx kernel / el0: no kernel read / el0: ok, asid: dual / asid: conflict / asid: ok, perf: cntpct delta=21093, perf: boot-delta ticks=115983, timer: tick, perf: irq-delta min=9499 max=29721 spread=20222 n=8, input: rx 0x41, two exception: sync BRK, exception: fatal nested / kind=0x200. cargo test Running 39 tests all [ok]. force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this PR is a separate row.
CI on GitHub (ASID isolation / PR #23).github/workflows/smoke.yml on cursor/el0-asid-isolation-f98dUnknownNo green GHA URL on the cloud-probe commit recorded here. Merged as #23 (fb050b7).
scripts/qemu-smoke.sh (standing EL0 + TTBR1 / this branch)Ran on this cloud VMVerified2026-09-11 (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e): host perf: elf-size bytes=3872120; hello (timeout 124) with paging: layout data=0x40201000 end=0x40243000 pool=0x40243000 user=1, paging: ok, heap: ok, sched: ok, wx: nx heap / wx: ok, guard: fault / guard: ok, ro: nx data / ro: write fault / ro: ok, el0: svc / el0: nx kernel / el0: no kernel read / el0: standing / el0: restored / el0: ok, asid: dual / asid: conflict / asid: ok, ttbr1: el1 / ttbr1: no el0 / ttbr1: ok, perf: cntpct delta=23782, perf: boot-delta ticks=127971, timer: tick, perf: irq-delta min=11604 max=27616 spread=16012 n=8, input: rx 0x41, two exception: sync BRK, exception: fatal nested / kind=0x200. cargo test Running 42 tests all [ok]. force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this PR is a separate row.
CI on GitHub (standing EL0 + TTBR1 / PR #24).github/workflows/smoke.yml on cursor/el0-standing-ttbr1-5ce0UnknownNo green GHA URL on the cloud-probe commit recorded here. Merged as #24 (8745939).
scripts/qemu-smoke.sh (TTBR1 high-VA exec / this branch)Ran on this cloud VMVerified2026-09-11 (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e): host perf: elf-size bytes=3877712; hello (timeout 124) with paging: layout data=0x40201000 end=0x40243000 pool=0x40243000 user=1, paging: ok, heap: ok, sched: ok, wx: nx heap / wx: ok, guard: fault / guard: ok, ro: nx data / ro: write fault / ro: ok, el0: svc / el0: nx kernel / el0: no kernel read / el0: standing / el0: restored / el0: ok, asid: dual / asid: conflict / asid: ok, ttbr1: el1 exec / ttbr1: vbar / ttbr1: el1 / ttbr1: no el0 / ttbr1: ok, perf: cntpct delta=20622, perf: boot-delta ticks=117383, timer: tick, perf: irq-delta min=11983 max=21932 spread=9949 n=8, input: rx 0x41, two exception: sync BRK, exception: fatal nested / kind=0x200 (fatal elr=0xffffff80400839e4). cargo test Running 44 tests all [ok]. force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this PR is a separate row.
CI on GitHub (TTBR1 high-VA exec / PR #25).github/workflows/smoke.yml on cursor/ttbr1-high-el1-exec-136cUnknownNo green GHA URL on the cloud-probe commit recorded here. Merged as #25 (008f7c7).
scripts/qemu-smoke.sh (identity-tear first cut / this branch)Ran on this cloud VMVerified2026-09-11 (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e): host perf: elf-size bytes=3895632; hello (timeout 124) with paging: layout data=0x40201000 end=0x40255000 pool=0x40255000 user=1, paging: ok, heap: ok, sched: ok, wx: nx heap / wx: ok, guard: fault / guard: ok, ro: nx data / ro: write fault / ro: ok, el0: svc / el0: nx kernel / el0: no kernel read / el0: standing / el0: restored / el0: ok, asid: dual / asid: conflict / asid: ok, ttbr1: el1 exec / ttbr1: vbar / ttbr1: el1 / ttbr1: no el0 / ttbr1: ok, ident: split / ident: fault / ident: high / ident: no el0 / ident: ok, perf: cntpct delta=25355, perf: boot-delta ticks=130631, timer: tick, perf: irq-delta min=14217 max=34342 spread=20125 n=8, input: rx 0x41, two exception: sync BRK, exception: fatal nested / kind=0x200 (fatal elr=0xffffff8040083bec). cargo test Running 47 tests all [ok]. force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this PR is a separate row.
CI on GitHub (identity-tear first cut / PR #26).github/workflows/smoke.yml on cursor/identity-teardown-c0fdUnknownNo green GHA URL on the cloud-probe commit recorded here. Merged as #26 (b0f0ee5).
Docker smoke on main b0f0ee5 (post PR #26 / ADR-018)the arm64 workstation Docker Desktop linux/arm64 ./scripts/docker-smoke.sh after #26 mergeVerifiedSponsor re-run on b0f0ee5: 47 tests, ident: ok, fatal nested ELR high alias. Not this cloud VM. Keep prior Failed b2bbb99 row (antifragility history).
qemu-smoke requires identity .text range-tear stringsscripts/qemu-smoke.sh hello phase greps ident: jump / ident: range / ident: text; rejects ident: range missedVerified2026-09-11 cloud: qemu-smoke: identity-tear strings present (includes jump/range/text).
scripts/qemu-smoke.sh (identity text range tear / this branch)Ran on this cloud VMVerified2026-09-11 3e710e1 (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e): host perf: elf-size bytes=3905712; hello (timeout 124) with ident: jump, ident: range lo=0x400ac000 hi=0x400b0000 pages=4, paging: layout data=0x40201000 end=0x40255000 pool=0x40255000 user=1, paging: ok, heap: ok, sched: ok, wx: ok, guard: ok, ro: ok, el0: ok, asid: ok, ttbr1: ok, ident: split / ident: fault / ident: high / ident: text / ident: no el0 / ident: ok, perf: cntpct delta=20293, perf: boot-delta ticks=92466, timer: tick, perf: irq-delta min=12711 max=26328 spread=13617 n=8, input: rx 0x41, two exception: sync BRK (ELRs 0xffffff8040082758 / 0xffffff8040082730), exception: fatal nested / kind=0x200 (fatal elr=0xffffff8040083bf4). cargo test Running 48 tests all [ok]. force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this PR is a separate row.
CI on GitHub (identity text range tear / PR #27 ac3ad0b).github/workflows/smoke.yml on cursor/identity-tear-text-range-ea15 @ ac3ad0bFailedBoth matrices: hello printed ident: jump / ident: range then ident: probe missed (init still required a 4 KiB __ident_tear_*; ADR-019 is 16 KiB). Keep this Failed row.
CI on GitHub (identity text range tear / main 24d94e6).github/workflows/smoke.yml merge-commit push on 24d94e6VerifiedPush 34640667227 success (ubuntu-24.04 + ubuntu-24.04-arm). Grepped both jobs: hello ident: jump / ident: range lo=0x400ac000 hi=0x400b0000 pages=4 / ident: text / ident: ok; BRK/fatal ELRs high; Running 48 tests; force-fail exit 1; qemu-smoke: ok.
Docker smoke on main 24d94e6 (post PR #27 / ADR-019)the arm64 workstation Docker Desktop linux/arm64 ./scripts/docker-smoke.sh after #27 mergeVerifiedSponsor re-run on 24d94e6: ident: jump, 16 KiB range tear, ident: ok, BRK/fatal on high ELRs, 48 tests + force-fail. Not this cloud VM. Keep the b2bbb99 Failed row.
qemu-smoke requires identity reloc + live .text stringsscripts/qemu-smoke.sh hello phase greps ident: reloc / ident: live; rejects ident: reloc missed / ident: live missedVerified2026-09-11 cloud: qemu-smoke: identity-tear strings present (includes reloc/live).
scripts/qemu-smoke.sh (ADR-020 live .text tear / this branch)Ran on this cloud VMVerified2026-09-11 0fda690 (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e): host perf: elf-size bytes=3988408; hello (timeout 124) with ident: jump, ident: reloc n=12, ident: range lo=0x400ae000 hi=0x400b2000 pages=4, ident: live lo=0x40081000 hi=0x400a6000 pages=37, paging: layout data=0x40201000 end=0x40255000 pool=0x40255000 user=1, paging: ok, heap: ok, sched: ok, wx: nx heap / wx: ok, guard: fault / guard: ok, ro: nx data / ro: write fault / ro: ok, el0: svc / el0: nx kernel / el0: no kernel read / el0: standing / el0: restored / el0: ok, asid: dual / asid: conflict / asid: ok, ttbr1: el1 exec / ttbr1: vbar / ttbr1: el1 / ttbr1: no el0 / ttbr1: ok, ident: split / ident: fault / ident: high / ident: text / ident: no el0 / ident: ok, perf: cntpct delta=21002, perf: boot-delta ticks=94258, timer: tick, perf: irq-delta min=10686 max=27025 spread=16339 n=8, input: rx 0x41, two exception: sync BRK (ELRs 0xffffff8040082990 / 0xffffff8040082968), exception: fatal nested / kind=0x200 (fatal elr=0xffffff8040083e2c). cargo test Running 50 tests all [ok]. force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this PR is a separate row.
CI on GitHub (ADR-020 live .text tear / this branch).github/workflows/smoke.yml on cursor/adr-020-live-text-reloc-322cUnknownNo green GHA URL on the cloud-probe commit recorded here. Merged as #28 (e80dc93).
CI on GitHub (ADR-020 live .text tear / main e80dc93).github/workflows/smoke.yml merge-commit push on e80dc93VerifiedPush 34651404108 success (ubuntu-24.04 + ubuntu-24.04-arm). Grepped both jobs: hello ident: jump / ident: reloc n=12 / ident: live lo=0x40081000 hi=0x400a6000 pages=37 / ident: ok; Hello World! after the tear; Running 50 tests; force-fail exit 1; qemu-smoke: ok.
Docker smoke on main e80dc93 (post PR #28 / ADR-020)the arm64 workstation Docker Desktop linux/arm64 ./scripts/docker-smoke.sh after #28 mergeVerifiedSponsor re-run on e80dc93: 50 tests, ident: reloc n=12, live pages=37, force-fail ok. Not this cloud VM. Keep the earlier 24d94e6 Docker Verified row and the b2bbb99 Failed row.
mdBook docs site builds locally./scripts/docs-build.sh (mdBook 0.5.4 + mdbook-mermaid 0.17.1) writes book/ and book/CNAME is ctos.artof.linkVerified2026-09-11 this cloud VM (plain-English / mermaid branch): mdbook v0.5.4; mdbook-mermaid 0.17.1; docs-build: ok; mermaid class="mermaid" nodes on landing + overview + pillars. Generator only. Live URL is a separate row. Re-probe on this Darwin-pin rebase is a separate row.
mdBook PATH pin compare strips clap vPATH mdbook v0.5.4; ./scripts/docs-build.sh does not print the mismatch warningVerified2026-09-12 this cloud VM on 8d14d08 rebase tip: real binary mdbook --version = mdbook v0.5.4; ${have#v} = 0.5.4; script output had no using PATH binary anyway and no re-download. Same strip on a bare 0.5.4 stays 0.5.4. Not a Darwin install probe.
docs-build auto-install on Linux (this rebase)./scripts/docs-build.sh with no PATH mdbook / mdbook-mermaidVerified2026-09-12 this cloud VM (Linux/x86_64): downloaded x86_64-unknown-linux-gnu for mdBook 0.5.4 and mdbook-mermaid 0.17.1; docs-build: ok; mermaid class="mermaid" + book/CNAME = ctos.artof.link. Generator only.
docs-build auto-install on DarwinRun ./scripts/docs-build.sh with no PATH mdbook / mdbook-mermaid on macOS (uname -s = Darwin)UnknownSource maps Darwin:arm64 / Darwin:x86_64 to published aarch64-apple-darwin / x86_64-apple-darwin assets for mdBook 0.5.4 and mdbook-mermaid 0.17.1 (GitHub release API listed those tarballs). This VM is Linux; Darwin download/extract was not run.
mdBook docs site builds locally (B7 / this branch)./scripts/docs-build.sh on cursor/b7-containers-nongoal-6b58Verified2026-09-12 this cloud VM: mdbook v0.5.4; mdbook-mermaid 0.17.1; docs-build: ok; book/CNAME is ctos.artof.link; generated hosting-apps.html / track-b.html contain ADR-029 + non-goal. Generator only. Pages GHA on this PR is a separate row.
Pages workflow builds this PR (B7 containers non-goal).github/workflows/pages.yml on cursor/b7-containers-nongoal-6b58 @ 1dd0a4fVerifiedPR run 34688560928 success (mdBook build 7s). Deploy job skipped on pull_request (expected). Not a new live-site deploy. Later docs-only SHAs on this branch are not this probe.
Pages workflow file existsRead .github/workflows/pages.ymlVerifiedWorkflow present: PR builds the book; main uploads + actions/deploy-pages. Not a live URL.
Pages workflow builds this PR.github/workflows/pages.yml on cursor/docs-pages-site-c371VerifiedPR run 34651704287 success (mdBook build 7s). Deploy job skipped on pull_request (expected). Not a live site.
Pages workflow builds this PR (plain-English / mermaid).github/workflows/pages.yml on cursor/docs-plain-english-diagrams-e4a9VerifiedPR run 34654325467 success (mdBook build 5s). Deploy job skipped on pull_request (expected). Not a new live-site deploy.
Docs website published (GitHub Pages)Green pages workflow on main and an HTTPS fetch of https://ctos.artof.linkVerified2026-09-11 this cloud VM after #30: main deploy 34653046584 success (mdBook build + Deploy GitHub Pages). curl -sSI https://ctos.artof.link HTTP 200; body includes Driving principles. https://artofdream.github.io/ctos (no trailing slash) 301s to the custom domain; .../ctos/ 404’d — not a second live tree. From #51; not re-probed on this A1 rebase.
Public CNAME ctos.artof.link → artofdream.github.iodig CNAME ctos.artof.link +shortVerified2026-09-11 this cloud VM (repeat): artofdream.github.io. DNS is in place at the public resolver. Serving the book is the Verified docs-site / reachability rows. From #51; not re-probed here.
Route 53 ctos CNAME in account <aws-account-id> zone <route53-zone-id>Sponsor stated CREATE done (zone <route53-zone-id>); API list-resource-record-sets as that accountUnknownThis environment still has no AWS CLI / credentials. Sponsor named the zone and said the record exists. Do not CREATE again. Public dig is the Verified row above, not this API probe.
Custom domain ctos.artof.link reachabilityPages Settings/API lists the hostname and curl -sSI https://ctos.artof.link is HTTP 200 with a cert for that nameVerified2026-09-11 this cloud VM: Pages API cname=ctos.artof.link, https_enforced=true, cert state=approved (expires 2026-12-10). HTTPS 200; TLS CN=ctos.artof.link (Let’s Encrypt). Route 53 API LIST stays the Unknown row above. From #51; not re-probed here.
Guest filesystem (virtio-blk + FAT16)Hello-kernel serial blk: ok + fat: ok; VFS /probeVerifiedThis tip’s qemu-smoke (see A7 rows below). Host -drive alone is not the probe. memfs is a separate A6 row. Do not say “supports FAT” as a product.
Guest runs host apps (Linux ELF / shell / Python)No exec, libc, or app-load serial markerPlannedSite: docs/overview/hosting-apps.md. Extra: host-apps.md. Today: in-tree samples only. A1 is a kernel ABI mile, not this row.
Guest is a container host (OCI / Docker / k8s)Source: no OCI, runc, cgroup, or namespace code in src/VerifiedAbsence + non-goal (ADR-029, Track B B7 / #47). Site: hosting-apps.md. Host docker-smoke.sh is a build harness, not a guest runtime. Not Planned. Reopen only with a new epic + ADR.
Product “immutable OS”Marketing sentence vs immutability.mdPlannedNon-claim. Absolute immutability is incompatible (heap/PTEs/devices mutate). Scoped RO is ADR-015 / ADR-020. Site: advantages.md. File presence is not Verified.
OS image vs app payloads disconnected (A9 first cut)Host OS ELF + target/hello-libctos.elf; FAT /hello; hello serial slot: fat + slot: mapped + slot: ok; #[test_case] slot_load_from_fat_eretsVerified2026-09-12 this cloud VM (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305): host OS perf: elf-size bytes=4392040 + app 8624 bytes; FAT /hello; serial slot: fat bytes=8624 / slot: mapped segs=3 entry=0x80002000 / slot: ok / perf: app-load ticks=1134019. A1–A8 markers still present. First-cut note: A2–A4 still include_bytes! on that SHA. Leftover embed-off is a later row. Not “apps update independently.”
A9 cross-update (same app on OS n and n+1)Same published hello-libctos.elf (sha256sum) on this OS tip and documented prior OS ba6541c (A9 merge; earliest main tip with the slot/FAT path); scripts/qemu-smoke.sh greps slot: ok on both bootsVerified2026-09-12 this cloud VM (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305): app sha256=142dedf1c3f6452555fe3af86c8bfafa260f083fecfdc81159063c433872d865 bytes=8624; host cross-update prior-os=ba6541c8e17c75c89c42451cd721735eddb36c2f slot:ok + cross-update this-os=800f52d183d0d58839f4c7363ac9d069414e55fc slot:ok. Both boots slot: fat bytes=8624 / slot: mapped / slot: ok. Prior kernel built from that SHA (no stored blob). Not “apps update independently.”
A9 slot-disconnect performance delta (ADR-046)Hello serial perf: app-load ticks=<n> + perf: embed-load ticks=<n> + perf: slot-delta app=<a> embed=<b>; #[test_case] embed CNTPCT + bytes match FAT; smoke rejects slot: embed / percent marketingVerified2026-09-13 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on a9/slot-perf-delta bb0379d: hello slot: fat bytes=8624 / slot: mapped / slot: ok / perf: app-load ticks=1079247 / perf: embed-load ticks=62805 / perf: slot-delta app=1079247 embed=62805 (no slot: embed); Running 92 tests all [ok] incl. slot_embed_probe_bytes_match_fat; force-fail exit 1; A9 cross-update both slot:ok; qemu-smoke: ok. Probe-only include_bytes! in src/slot.rs only. QEMU TCG lab — not a published bench, not a percent. Isolation leftovers (PAN enable / taken SError / yank _start / umbrella) unchanged Planned. (amd64 workstation) intent; CloudAgent HELD — probe on agent box clone. ADR-046.
Threat-model v1.9 slice (NFR-10)Read security.md (SVC ABI mile; app hosting Planned)VerifiedFile + A1 PR. v1.9 update, not “secure.”
Threat-model v1.10 slice (NFR-10)Read security.md (libctos CRT mile; app hosting Planned)VerifiedFile + A2 PR. v1.10 update, not “secure.”
Threat-model v1.11 slice (NFR-10)Read security.md (guest ELF PT_LOAD loader mile; app hosting Planned)VerifiedFile + this PR. v1.11 update, not “secure.”
SVC ABI (exit / uart_write / yield) (ADR-021 / Track A A1)Hello-kernel serial svc: yield + svc: user-hi + svc: uart + svc: exit + svc: ok; #[test_case] el0_svc_abi_yield_uart_exit + el0_uart_write_rejects_kernel_data + el0_uart_write_rejects_high_aliasVerified2026-09-11 tip 09425c0 (onto aa46219 / #29). Serial below. App hosting stays Planned. Not Linux.
qemu-smoke requires SVC ABI stringsscripts/qemu-smoke.sh hello phase greps svc: ok / svc: yield / svc: user-hi / svc: uart / svc: exit; rejects svc: probe missedVerified2026-09-11 09425c0: qemu-smoke: SVC ABI strings present.
scripts/qemu-smoke.sh (SVC ABI / this branch)Ran on this cloud VMVerified2026-09-11 09425c0 (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e): host perf: elf-size bytes=4020520; hello (timeout 124) with ident: jump / ident: reloc n=12 / ident: live / paging: ok / heap: ok / sched: ok / wx: ok / guard: ok / ro: ok / el0: standing / el0: restored / el0: ok / svc: yield / svc: user-hi / svc: uart n=13 / svc: exit / svc: ok / asid: ok / ttbr1: ok / ident: ok / perf: cntpct delta=20326 / perf: boot-delta ticks=104297 / timer: tick / input: rx 0x41 / exception: fatal nested. cargo test Running 54 tests all [ok] (includes el0_uart_write_rejects_high_alias). force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this tip is a separate row.
CI on GitHub (SVC ABI / this branch).github/workflows/smoke.yml on cursor/svc-syscall-abi-0dc5UnknownPrior GHA is not this tip (09425c0).
Track A app hosting (slots)Product “apps run independently of the OS”VerifiedParent issue #31. Claim gate ADR-048 criteria 1–5 Met on tip be789c5+; sponsor accept ADR-052 (sponsor: “address 1. app hosting” after still-Planned #5-only review). Evidence: A8 recipes Verified; A9 cross-update cross-update … slot:ok on this OS + ba6541c; no production embed (probe-only include_bytes! in src/slot.rs / ADR-046; smoke rejects slot: embed); honesty — not Linux/POSIX/containers (ADR-029). Not Linux userspace, POSIX, glibc/musl, guest OCI, “EL0 isolated,” PAN enable, taken SError Verified, immutable-OS marketing, or OTA.
Documented sample apps (Track A A8)Read what-can-run.md, apps-today.md, in-tree user/README.md; recipes cite only existing serial markers; ./scripts/docs-build.shVerified2026-09-12 this cloud VM: file read of the recipes + docs-build after rebase onto main c63077e (B7 / ADR-029; next row). Coop UART (sched:*), RX echo (input: rx 0x41), standing EL0 / libctos hello (libctos:* / loader:* / el0: task-*), optional memfs (fs:*) + FAT16 (blk:* / fat:*). No new runtime markers. No ADR (docs-only). App hosting stays Planned. A9 first cut is a later row. Guest containers stay non-goal. Kernel qemu-smoke not re-run (no src/ edit).
scripts/docs-build.sh (A8 recipes / this branch)Ran on this cloud VMVerified2026-09-12 after rebase onto c63077e (B7 / ADR-029 + Darwin pin): mdbook v0.5.4; mdbook-mermaid 0.17.1; Linux auto-install x86_64-unknown-linux-gnu; docs-build: ok; book/CNAME is ctos.artof.link. Generator only. Live URL is a separate row. Kernel qemu-smoke not re-run (no src/ edit).
Standing EL0 as normal mode (ADR-024 / Track A A4)Hello-kernel serial el0: task-enter + el0: task-active + el0: task-exit + el0: task-restored + el0: restore-fail + el0: task-ok; #[test_case] standing_task_runs_until_exit + standing_task_fault_restoresVerified2026-09-12 this cloud VM (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305): after loader: ok, serial el0: task-enter / el0: task-active / payload libctos: hi / libctos: ok / svc: exit / el0: task-exit / el0: task-restored / second el0: task-enter / el0: restore-fail / el0: task-ok. A1–A3 markers still present. Not isolation. App hosting stays Planned.
qemu-smoke requires standing-task stringsscripts/qemu-smoke.sh hello phase greps el0: task-ok / el0: task-enter / el0: task-active / el0: task-exit / el0: task-restored / el0: restore-fail; rejects el0: task missedVerified2026-09-12 this cloud VM: qemu-smoke: standing-task (A4) strings present.
scripts/qemu-smoke.sh (standing-as-normal / this branch)Ran on this cloud VMVerified2026-09-12 this tip (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305): host perf: elf-size bytes=4108960; hello (timeout 124) with ident: jump / ident: reloc n=12 / ident: live lo=0x40081000 hi=0x400ac000 pages=43 / paging: ok / heap: ok / sched: ok / wx: ok / guard: ok / ro: ok / el0: standing / el0: restored / el0: ok / svc: yield / svc: user-hi / svc: uart n=13 / svc: exit / svc: ok / libctos: hi / libctos: ok / libctos: linked / loader: mapped segs=3 entry=0x80002000 / loader: ok / el0: task-enter / el0: task-active / el0: task-exit / el0: task-restored / el0: restore-fail / el0: task-ok / asid: ok / ttbr1: ok / ident: ok / perf: cntpct delta=20161 / perf: boot-delta ticks=94742 / timer: tick / input: rx 0x41 / exception: fatal nested. cargo test Running 66 tests all [ok] (adds standing_task_runs_until_exit + standing_task_fault_restores). force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this tip is a separate row.
CI on GitHub (standing-as-normal / this branch).github/workflows/smoke.yml on cursor/a4-standing-el0-normal-1e14UnknownNo green GHA URL on the cloud-probe commit recorded here.
Threat-model v1.12 slice (NFR-10)Read security.md (standing-as-normal mile; app hosting Planned)VerifiedFile + A4 PR. v1.12 update, not “secure.”
Threat-model v1.13 slice (NFR-10)Read security.md (identity .rodata tear + PAN ID-field; app hosting Planned)VerifiedFile + A5 PR. v1.13 update, not “secure.”
Threat-model v1.14 slice (NFR-10)Read security.md (thin VFS + memfs mile; app hosting Planned)VerifiedFile + this PR. v1.14 update, not “secure.”
Thin VFS + memfs (ADR-027 / Track A A6)Hello-kernel serial fs: create + fs: write + fs: read + fs: el0 + fs: ok; #[test_case] vfs_memfs_create_write_read_close + el0_fs_svc_roundtrip + el0_fs_write_rejects_kernel_dataVerified2026-09-12 this cloud VM (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57): after el0: task-ok, serial fs: create / fs: write / fs: read / svc: exit / fs: el0 / fs: ok. cargo test all three new cases [ok]. Not POSIX. Not FAT. App hosting stays Planned.
qemu-smoke requires memfs stringsscripts/qemu-smoke.sh hello phase greps fs: ok / fs: create / fs: write / fs: read / fs: el0; rejects fs: probe missedVerified2026-09-12 this cloud VM: qemu-smoke: thin VFS + memfs (A6) strings present.
scripts/qemu-smoke.sh (thin VFS + memfs / this branch)Ran on this cloud VMVerified2026-09-12 this tip (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305): host perf: elf-size bytes=4291384; hello (timeout 124) with ident: jump / ident: reloc n=12 / ident: live lo=0x40081000 hi=0x400b4000 pages=51 / ident: ro-reloc n=683 / ident: rodata lo=0x400b4000 hi=0x400c0000 pages=12 / pan: id=0 / pan: absent / paging: ok / heap: ok / sched: ok / wx: ok / guard: ok / ro: ok / el0: standing / el0: restored / el0: ok / svc: ok / libctos: ok / loader: ok / el0: task-ok / fs: create / fs: write / fs: read / fs: el0 / fs: ok / asid: ok / ttbr1: ok / ident: split / ident: fault / ident: high / ident: text / ident: no el0 / ident: rodata-fault / ident: rodata-high / ident: ok / perf: cntpct delta=21003 / perf: boot-delta ticks=104836 / timer: tick / input: rx 0x41 / exception: fatal nested. cargo test Running 73 tests all [ok] (adds vfs_memfs_create_write_read_close + vfs_open_missing_is_err + el0_fs_svc_roundtrip + el0_fs_write_rejects_kernel_data + vfs_paths_and_caps_are_documented). force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this tip is a separate row.
CI on GitHub (thin VFS + memfs / this branch).github/workflows/smoke.yml on cursor/a6-thin-vfs-memfs-766aUnknownNo green GHA URL on this tip yet.
Identity .rodata tear (ADR-025 / Track A A5)Hello-kernel serial ident: ro-reloc + ident: rodata + ident: rodata-fault + ident: rodata-high + existing ident: ok; #[test_case] identity_rodata_unmapped_high_staysVerified2026-09-12 this cloud VM (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57): after ident: live, serial ident: ro-reloc n=589 / ident: rodata lo=0x400ad000 hi=0x400b8000 pages=11 then later ident: rodata-fault / ident: rodata-high / ident: ok. Hello World! after the tear. cargo test identity_rodata_unmapped_high_stays [ok]. .data/heap stay. Not “EL0 isolated.”
PAN capability ID field (ADR-026 / Track A A5)Hello-kernel serial pan: id= + pan: absent; #[test_case] pan_unimplemented_on_probe_cpuVerified2026-09-12 this cloud VM: pan: id=0 / pan: absent. cargo test pan_unimplemented_on_probe_cpu [ok]. Enable stays Planned. Do not switch -cpu.
qemu-smoke requires identity .rodata + PAN stringsscripts/qemu-smoke.sh hello phase greps ident: rodata / ident: rodata-fault / ident: rodata-high / pan: id= / pan: absent; rejects ident: rodata missed / pan: probe missed / pan: enabledVerified2026-09-12 this cloud VM: qemu-smoke: identity-tear strings present + qemu-smoke: PAN capability strings present.
scripts/qemu-smoke.sh (A5 identity .rodata + PAN / this branch)Ran on this cloud VMVerified2026-09-12 this tip (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305): host perf: elf-size bytes=4121360; hello (timeout 124) with ident: jump / ident: reloc n=12 / ident: live lo=0x40081000 hi=0x400ad000 pages=44 / ident: ro-reloc n=589 / ident: rodata lo=0x400ad000 hi=0x400b8000 pages=11 / pan: id=0 / pan: absent / paging: ok / heap: ok / sched: ok / wx: ok / guard: ok / ro: ok / el0: standing / el0: restored / el0: ok / svc: ok / libctos: ok / loader: ok / el0: task-ok / asid: ok / ttbr1: ok / ident: split / ident: fault / ident: high / ident: text / ident: no el0 / ident: rodata-fault / ident: rodata-high / ident: ok / perf: cntpct delta=21378 / perf: boot-delta ticks=96439 / timer: tick / input: rx 0x41 / exception: fatal nested. cargo test Running 68 tests all [ok] (adds identity_rodata_unmapped_high_stays + pan_unimplemented_on_probe_cpu). force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this tip is a separate row.
CI on GitHub (A5 / this branch).github/workflows/smoke.yml on cursor/a5-identity-tear-pan-0355VerifiedPR run 34678808063 success on 726a4a2 (ubuntu-24.04 job 103513419325 + ubuntu-24.04-arm job 103513419189). Both matrices grepped: hello ident: ro-reloc n=588 / ident: rodata lo=0x400ad000 hi=0x400b8000 pages=11 / ident: rodata-fault / ident: rodata-high / pan: id=0 / pan: absent / ident: live / el0: task-ok / loader: ok / libctos: ok / svc: ok; Running 68 tests; qemu-smoke: identity-tear strings present / qemu-smoke: PAN capability strings present / qemu-smoke: ok. Push run 34678806604 success. mdBook 34678808079 success (Pages deploy skipped on PR). Cursor Bugbot pass on 726a4a2. Later docs-only SHAs on this branch are not this probe.
Guest ELF PT_LOAD loader (ADR-023 / Track A A3)Hello-kernel serial loader: mapped + loader: ok; payload libctos: hi / libctos: ok; #[test_case] loader_maps_hello_and_eretsVerified2026-09-12 this cloud VM (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305): loader: mapped segs=3 entry=0x80002000 then loader: ok. Payload printed libctos: hi / libctos: ok on the loaded trip (after the A2 memcpy trip). Not a Linux ABI. App hosting stays Planned.
qemu-smoke requires guest loader stringsscripts/qemu-smoke.sh hello phase greps loader: ok / loader: mapped; rejects loader: probe missedVerified2026-09-12 this cloud VM: qemu-smoke: guest ELF loader strings present.
scripts/qemu-smoke.sh (guest ELF loader / this branch)Ran on this cloud VMVerified2026-09-12 this tip (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305): host perf: elf-size bytes=4101056; hello (timeout 124) with ident: jump / ident: reloc n=12 / ident: live lo=0x40081000 hi=0x400ac000 pages=43 / paging: ok / heap: ok / sched: ok / wx: ok / guard: ok / ro: ok / el0: standing / el0: restored / el0: ok / svc: yield / svc: user-hi / svc: uart n=13 / svc: exit / svc: ok / libctos: hi / libctos: ok / libctos: linked / loader: mapped segs=3 entry=0x80002000 / loader: ok / asid: ok / ttbr1: ok / ident: ok / perf: cntpct delta=21608 / perf: boot-delta ticks=98798 / timer: tick / input: rx 0x41 / exception: fatal nested. cargo test Running 64 tests all [ok] (adds loader_maps_hello_and_erets + parse / PT_INTERP / W+X / RX+R union). force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this tip is a separate row.
CI on GitHub (guest ELF loader / this branch).github/workflows/smoke.yml on cursor/a3-elf-loader-ttbr0-49f0UnknownNo green GHA URL on the cloud-probe commit recorded here.
libctos CRT (exit / uart_write / yield wrappers) (ADR-022 / Track A A2)Hello-kernel serial libctos: hi + libctos: ok + libctos: linked; #[test_case] libctos_hello_yield_uart_exit + libctos_payload_encodes_public_svc_onlyVerified2026-09-12 tip c9b292b. Serial below. App hosting stays Planned. Not a guest ELF loader. Not Linux.
qemu-smoke requires libctos CRT stringsscripts/qemu-smoke.sh hello phase greps libctos: hi / libctos: ok / libctos: linked; rejects libctos: probe missedVerified2026-09-12 this cloud VM: qemu-smoke: libctos CRT strings present.
scripts/qemu-smoke.sh (libctos CRT / this branch)Ran on this cloud VMVerified2026-09-12 c9b292b (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305): host perf: elf-size bytes=4024936; hello (timeout 124) with ident: jump / ident: reloc n=12 / ident: live / paging: ok / heap: ok / sched: ok / wx: ok / guard: ok / ro: ok / el0: standing / el0: restored / el0: ok / svc: yield / svc: user-hi / svc: uart n=13 / svc: exit / svc: ok / libctos: hi / libctos: ok / libctos: linked / asid: ok / ttbr1: ok / ident: ok / perf: cntpct delta=38969 / perf: boot-delta ticks=153560 / timer: tick / input: rx 0x41 / exception: fatal nested. cargo test Running 57 tests all [ok] (adds libctos_hello_yield_uart_exit + payload SVC-only + ABI number match). force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this tip is a separate row.
CI on GitHub (libctos CRT / this branch).github/workflows/smoke.yml on cursor/libctos-crt-a2-590eUnknownNo green GHA URL on the cloud-probe commit recorded here.
Threat-model v1.15 slice (NFR-10)Read security.md (virtio-blk + FAT16 mile; app hosting Planned)VerifiedFile + A7 PR. v1.15 update, not “secure.”
Threat-model v1.16 slice (NFR-10)Read security.md (OS/app slot first cut; cross-update Planned)VerifiedFile + A9 PR. v1.16 update, not “secure.”
Threat-model v1.17 slice (NFR-10)Read security.md (leftover cross-update + FAT-only hello + stay markers; PAN enable Planned)VerifiedFile + this PR. v1.17 update, not “secure.”
A2–A4 hello without include_bytes! (ADR-032)src/ has no include_bytes!.*hello-libctos; hello serial libctos: linked / loader: ok / el0: task-ok still from FAT /hello; build.rs still publishes target/hello-libctos.elfVerified2026-09-12 this cloud VM: host qemu-smoke: kernel does not include_bytes! hello-libctos + kernel rebuild compiled app payload; serial libctos: linked / loader: ok / el0: task-ok / slot: ok. Same FAT /hello 8624 bytes.
virtio-blk (ADR-028 / Track A A7)Hello-kernel serial blk: virtio + blk: cap + blk: rw + blk: ok; #[test_case] virtio_blk_capacity_and_rwVerified2026-09-12 this cloud VM (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57): after fs: ok, serial blk: virtio / blk: cap sectors=8192 / blk: rw / blk: ok. cargo test virtio_blk_capacity_and_rw + virtio_blk_boot_sector_is_fat [ok]. QEMU virt virtio-mmio legacy v1. Host -drive is not this row.
FAT16 via thin VFS (ADR-028 / Track A A7)Hello-kernel serial fat: mount + fat: read + fat: ok; VFS /probe = fat-hi; #[test_case] fat16_vfs_open_probeVerified2026-09-12 this cloud VM: fat: mount / fat: read / fat: ok. Same vfs::open as memfs. cargo test fat16_vfs_open_probe + fat16_missing_and_readonly + vfs_fat_probe_is_same_open [ok]. Not POSIX. Not FAT32. Not writeable FAT.
qemu-smoke requires virtio-blk + FAT16 stringsscripts/qemu-smoke.sh hello phase greps blk: ok / fat: ok; rejects blk: probe missed / fat: probe missedVerified2026-09-12 this cloud VM: qemu-smoke: virtio-blk (A7) strings present + qemu-smoke: FAT16 (A7) strings present.
scripts/qemu-smoke.sh (virtio-blk + FAT16 / this branch)Ran on this cloud VMVerified2026-09-12 435bf2d cache-sync follow-up (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305): host perf: elf-size bytes=4379520; hello (timeout 124) with blk: virtio / blk: cap sectors=8192 / blk: rw / blk: ok / fat: mount / fat: read / fat: ok plus A1–A6 markers (svc: ok / libctos: ok / loader: ok / el0: task-ok / fs: ok / ident: ok / pan: id=0). cargo test Running 79 tests all [ok]. force-fail exit 1. qemu-smoke: ok. Prior b4903e8 cloud + GHA rows stay. Not Docker. GHA on 435bf2d is a separate row.
CI on GitHub (virtio-blk + FAT16 / b4903e8).github/workflows/smoke.yml on cursor/a7-virtio-blk-fat-a587 @ b4903e8VerifiedPR run 34687722316 success (ubuntu-24.04 job 103537544581 + ubuntu-24.04-arm job 103537544698). Both matrices grepped: hello blk: virtio / blk: cap sectors=8192 / blk: rw / blk: ok / fat: mount / fat: read / fat: ok / fs: ok / el0: task-ok / loader: ok / libctos: ok / svc: ok / ident: ok / pan: id=0 / pan: absent; qemu-smoke: virtio-blk (A7) strings present / qemu-smoke: FAT16 (A7) strings present / Running 79 tests / qemu-smoke: ok. Push run 34687720964 success. mdBook 34687722361 success (Pages deploy skipped on PR). First 79c6432 PR smoke 34687537221 Failed (modern-only / blk: probe missed).
CI on GitHub (virtio-blk + FAT16 / cache-sync tip).github/workflows/smoke.yml on cursor/a7-virtio-blk-fat-a587 @ 435bf2dUnknownCloud qemu-smoke Verified on this SHA (row above). No green gh run URL on 435bf2d yet.
OS/app slot first cut (ADR-030 / Track A A9)Host OS ELF + target/hello-libctos.elf; FAT /hello; hello serial slot: fat + slot: mapped + slot: ok + perf: app-load; #[test_case] slot_fat_hello_is_elf + slot_load_from_fat_eretsVerifiedSame 2026-09-12 cloud qemu-smoke as the row above. cargo test both new cases [ok]. A2–A4 still embed. Not cross-update. Not app hosting.
qemu-smoke requires A9 slot stringsscripts/qemu-smoke.sh greps slot: ok / slot: fat / slot: mapped / perf: app-load / perf: embed-load / perf: slot-delta; rejects slot: probe missed / slot: embed / percent marketingVerified2026-09-12 this cloud VM: qemu-smoke: OS/app slot (A9) strings present. ADR-046 extends greps on this PR tip.
scripts/qemu-smoke.sh (A9 slots / this branch)Ran on this cloud VMVerified2026-09-12 9ca7372 (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305): host perf: elf-size bytes=4392040; app target/hello-libctos.elf 8624 bytes; hello (timeout 124) with slot: fat bytes=8624 / slot: mapped segs=3 entry=0x80002000 / slot: ok / perf: app-load ticks=1134019 plus A1–A8 markers (svc: ok / libctos: ok / loader: ok / el0: task-ok / fs: ok / blk: ok / fat: ok / ident: ok / pan: id=0). cargo test Running 82 tests all [ok] (adds slot_fat_hello_is_elf + slot_load_from_fat_erets + slot_path_is_not_probe). force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this tip is a separate row.
CI on GitHub (A9 slots / this branch).github/workflows/smoke.yml on cursor/a9-os-app-slots-cf00UnknownCloud qemu-smoke Verified on 9ca7372. No green gh run URL on this tip yet.
mdBook docs site builds locally (A9 / this branch)./scripts/docs-build.sh on cursor/a9-os-app-slots-cf00Verified2026-09-12 this cloud VM: mdbook v0.5.4; mdbook-mermaid 0.17.1; docs-build: ok; book/CNAME is ctos.artof.link. Generator only. Pages GHA on this PR is a separate row.
mdBook docs site builds locally (ADR-002 Track A merge-hat / this branch)./scripts/docs-build.sh on cursor/adr-002-track-a-merge-hat-0e11Verified2026-09-12 this cloud VM: mdbook v0.5.4; mdbook-mermaid 0.17.1; Linux auto-install x86_64-unknown-linux-gnu; docs-build: ok. Generator only. Kernel qemu-smoke not run (no src/ edit). Distinct-identity merge stays Unknown.
Linux-compat goals documented (ADR-031 / Track B B1)Read ADR-031 and track-b.mdVerifiedDocument inspection. ABI subset research frame. Not claiming Linux userspace. Guest containers stay non-goal (ADR-029). B2–B6 stay Planned. File presence is not a Linux ABI. Kernel qemu-smoke not run (no src/ edit).
mdBook docs site builds locally (B1 / this branch)./scripts/docs-build.sh on cursor/b1-linux-compat-goals-9374Verified2026-09-12 this cloud VM after rebase onto 4459b93 (#61): mdbook v0.5.4; mdbook-mermaid 0.17.1; Linux auto-install x86_64-unknown-linux-gnu; docs-build: ok; book/CNAME is ctos.artof.link; generated ADR-031 + track-b contain not claiming Linux userspace; ADR-002 page still has the Track A historical exception. Generator only. Pages GHA on this PR is a separate row.
mdBook docs site builds locally (mobile theme / this branch)./scripts/docs-build.sh on cursor/docs-mobile-responsive-81c4Verified2026-09-12 this cloud VM: mdbook v0.5.4; mdbook-mermaid 0.17.1; Linux auto-install x86_64-unknown-linux-gnu; docs-build: ok; book/CNAME is ctos.artof.link; *theme.css present in book/. Generator only. Pages GHA on this PR is a separate row.
Docs site mobile-compliant (narrow viewport)./scripts/docs-mobile-probe.sh after ./scripts/docs-build.sh (viewport meta + theme.css + Chrome iframe scrollWidth vs clientWidth)Verified2026-09-12 this cloud VM (Google Chrome 148.0.7778.96): landing index.html overflow=no at 320 (client=320 scroll=320), 768, and 1280; framework/honesty-ledger.html @ 320 overflow=no; website.html @ 320 overflow=no. Wide tables/pre may scroll inside their widget. Mermaid CDN was blocked for this numeric probe (source text). Not a live https://ctos.artof.link mobile claim until a post-merge Pages fetch.
scripts/qemu-smoke.sh (Track A leftovers / this branch)Ran on this cloud VMVerified2026-09-12 800f52d (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305): host OS perf: elf-size bytes=4410056; app 8624 bytes; hello (timeout 124) with A1–A9 markers (svc: ok / libctos: ok / loader: ok / el0: task-ok / fs: ok / blk: ok / fat: ok / slot: ok / ident: data-stay / ident: heap-stay / ident: ok / pan: id=0 / pan: absent); host embed ratchet; same-app cross-update on ba6541c + this tip (sha256=142dedf1…); cargo test Running 83 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Re-probe on 17778ca and post-rebase 6abcb1b (onto e6255c0; same markers + qemu-smoke: ok). Re-probe on 7f52572 after the Docker-git / portable-sha256 ratchet: same A1–A9 + stay + sha256=142dedf1… + both cross-update … slot:ok + 83 tests + force-fail 1 + qemu-smoke: ok. Re-probe on 4a6b4f7 after moving /tmp/ctos-prior-os-* aside (stale .git/worktrees registration): git worktree add prior OS ba6541c then both cross-update … slot:ok + 83 tests + force-fail 1 + qemu-smoke: ok. Not Docker. GHA on 7b68d08 / 7f52572 are separate Verified rows. GHA on 4a6b4f7 / 70058db is Unknown until grepped.
CI on GitHub (Track A leftovers / 17778ca).github/workflows/smoke.yml on cursor/track-a-leftovers-d2ee @ 17778caVerifiedPush run 34701455001 success (ubuntu-24.04 job 103573837236 + ubuntu-24.04-arm job 103573837300). Both matrices grepped: kernel does not include_bytes! hello-libctos; hello slot: fat bytes=8624 / slot: ok / ident: data-stay / ident: heap-stay / svc: ok / libctos: ok / loader: ok / el0: task-ok / fs: ok / blk: ok / fat: ok / pan: absent; cross-update prior-os=ba6541c… slot:ok + cross-update this-os=17778ca… slot:ok; Running 83 tests; force-fail exited 1; qemu-smoke: ok. Prior ef9ff56 Failed closed (shallow clone missing ba6541c).
CI on GitHub (Track A leftovers / 7b68d08).github/workflows/smoke.yml on cursor/track-a-leftovers-d2ee @ 7b68d08 (rebase onto 522c543 / #63)VerifiedPush run 34702218168 success (ubuntu-24.04-arm job 103575880564 + ubuntu-24.04 job 103575880766). PR run 34702220491 success (merge-ref ebdcc8d). Both matrices grepped: kernel does not include_bytes! hello-libctos; hello slot: fat bytes=8624 / slot: ok / ident: data-stay / ident: heap-stay / ident: ok / svc: ok / libctos: ok / loader: ok / el0: task-ok / fs: ok / blk: ok / fat: ok / pan: absent; cross-update prior-os=ba6541c… slot:ok + cross-update this-os=7b68d08… slot:ok (PR event this-os=ebdcc8d); Running 83 tests; force-fail exited 1; qemu-smoke: ok. Later ledger-only SHA is not this probe.
mdBook docs site builds locally (leftovers / this branch)./scripts/docs-build.sh on cursor/track-a-leftovers-d2eeVerified2026-09-12 this cloud VM: mdbook v0.5.4; mdbook-mermaid 0.17.1; docs-build: ok. Generator only. Pages GHA on this PR is a separate row.
qemu-smoke requires A9 cross-update + stay + embed-offscripts/qemu-smoke.sh greps both cross-update … slot:ok lines, ident: data-stay / ident: heap-stay, and rejects src/ hello include_bytes!Verified2026-09-12 this cloud VM: qemu-smoke: A9 cross-update … strings present + identity-tear strings (includes stay) + kernel does not include_bytes! hello-libctos.
Leftover docker-smoke (git + prior SHA)Dockerfile installs git; .dockerignore keeps .git objects but drops .git/worktrees; smoke git worktree prunes before addUnknownSource ratchet after Bugbot Mediums on 7b68d08 / 7b296e2. This cloud VM: command -v docker empty. File presence is not docker-smoke.
Leftover app sha256 pin (NFR-03)qemu-smoke hashes via sha256sum / shasum -a 256 / openssl dgst -sha256UnknownSource ratchet after Bugbot Medium on 7b68d08. This Linux tip used sha256sum (sha256=142dedf1… on 7f52572). Darwin shasum path not probed here. Same-app proof remains the cp of the published ELF.
CI on GitHub (Track A leftovers / 7f52572).github/workflows/smoke.yml on cursor/track-a-leftovers-d2ee @ 7f52572VerifiedPush run 34702699720 success (ubuntu-24.04 job 103577150895 + ubuntu-24.04-arm job 103577151020). PR run 34702702359 success (merge-ref 709bafd). Both matrices grepped: embed-off, A9 slot: + A1–A8 + stay, both cross-update … slot:ok (push this-os=7f52572), 83 tests, force-fail 1, qemu-smoke: ok. Later ledger-only SHA is not this probe.
Linux AArch64 vs ctos SVC gap documented (Track B B2)Read linux-aarch64-syscall-gap.md, src/syscall.rs (SYS_* 16–23), and Linux v6.10 include/uapi/asm-generic/unistd.hVerifiedDocument inspection. Representative table + convention gap. Not claiming Linux userspace. No Linux numbers in src/. B3–B6 stay Planned. File presence is not a Linux ABI. Kernel qemu-smoke not run (no src/ edit).
mdBook docs site builds locally (B2 / this branch)./scripts/docs-build.sh on cursor/b2-linux-syscall-gap-c35bVerified2026-09-12 this cloud VM: mdbook v0.5.4; mdbook-mermaid 0.17.1; Linux auto-install x86_64-unknown-linux-gnu; docs-build: ok; book/CNAME is ctos.artof.link; generated gap page + track-b contain never-per-ADR-031 and not claiming Linux userspace. Generator only. Pages GHA on this PR is a separate row. Kernel qemu-smoke not run (no src/ edit).
CI on GitHub (B2 / this branch).github/workflows/smoke.yml on cursor/b2-linux-syscall-gap-c35b @ 60478acVerifiedPR run 34701938481 success (ubuntu-24.04 job 103575136780 + ubuntu-24.04-arm job 103575136921). Both matrices grepped: hello slot: fat bytes=8624 / slot: mapped / slot: ok / svc: ok / libctos: ok / loader: ok / el0: task-ok / fs: ok / blk: ok / fat: ok; Running 82 tests; force-fail exit 1; qemu-smoke: ok. Push run 34701935451 success. Docs-only; not a new kernel mile. Later ledger SHAs on this branch are not this probe.
Pages workflow builds this PR (B2 syscall gap).github/workflows/pages.yml on cursor/b2-linux-syscall-gap-c35b @ 60478acVerifiedPR run 34701938331 success (mdBook build job 103575136524). Deploy job skipped on pull_request (expected). Not a new live-site deploy.
Linux ELF / auxv / PT_INTERP vs freestanding loader documented (ADR-033 / Track B B4)Read ADR-033, src/loader.rs (PT_INTERP reject, ET_EXEC / PT_LOAD only), Linux v6.10 include/uapi/linux/elf.h + auxvec.hVerifiedDocument inspection. Gap ADR. Track A loader stays reusable. Does not accept PT_INTERP. Not claiming dynamic Linux ELF. No src/ change. B3 / B5 / B6 stay Planned. File presence is not a Linux ELF ABI. Kernel qemu-smoke not run (no src/ edit).
mdBook docs site builds locally (B4 / this branch)./scripts/docs-build.sh on cursor/b4-elf-auxv-pt-interp-51bcVerified2026-09-12 this cloud VM: mdbook v0.5.4; mdbook-mermaid 0.17.1; Linux auto-install x86_64-unknown-linux-gnu; docs-build: ok; book/CNAME is ctos.artof.link; generated ADR-033 + track-b contain stay-rejected and not claiming dynamic Linux ELF. Generator only. Pages GHA on this PR is a separate row. Kernel qemu-smoke not run (no src/ edit).
Process model vs standing EL0 documented (ADR-035 / Track B B3)Read ADR-035 and track-b.mdVerifiedDocument inspection. Standing EL0 stays the ctos model. Linux clone/execve/wait4 is a gap. Not claiming Linux userspace. Do not add those syscalls to src/. B4 is Documented on main (774c672 / #68). B5 / B6 stay Planned. File presence is not a Linux process table. Kernel qemu-smoke not run (no src/ edit).
mdBook docs site builds locally (B3 / this branch)./scripts/docs-build.sh on cursor/b3-process-model-3443Verified2026-09-12 this cloud VM after rebase onto 774c672 (#68 / ADR-033 ELF): mdbook v0.5.4; mdbook-mermaid 0.17.1; Linux auto-install x86_64-unknown-linux-gnu; docs-build: ok; book/CNAME is ctos.artof.link; generated ADR-035 + track-b contain Standing EL0 stays the ctos model and not claiming Linux userspace; generated ADR-033 page contains stay-rejected and not claiming dynamic Linux ELF. No ADR-033-process-model or ADR-034-process-model page. Generator only. Pages GHA on this PR is a separate row. Kernel qemu-smoke not run (no src/ edit).
Linux VFS vs thin ctos VFS documented (ADR-034 / Track B B5)Read ADR-034, src/vfs.rs (VfsOps + valid_path), src/fat.rs, and linux-aarch64-syscall-gap.md (openat / read / write / close)VerifiedDocument inspection. Concept compare only. Not claiming a Linux filesystem. No POSIX flags / dentries in src/. B6 stays Planned. B3 is Documented (ADR-035). B4 is Documented (ADR-033). File presence is not Linux VFS. Kernel qemu-smoke not run (no src/ edit).
mdBook docs site builds locally (B5 / this branch)./scripts/docs-build.sh on cursor/b5-linux-vfs-vs-thin-ctos-f294Verified2026-09-12 this cloud VM: mdbook v0.5.4; mdbook-mermaid 0.17.1; Linux auto-install x86_64-unknown-linux-gnu; docs-build: ok; book/CNAME is ctos.artof.link; generated ADR-034 + track-b contain Not claiming a Linux filesystem and No Linux VFS object is present. Generator only. Pages GHA on this PR is a separate row. Kernel qemu-smoke not run (no src/ edit).
CI on GitHub (B5 / prior SHA fc4bf89).github/workflows/smoke.yml on cursor/b5-linux-vfs-vs-thin-ctos-f294 @ fc4bf89 (then titled ADR-034)VerifiedMRC-grepped on #66: push 34713366059 (this-os=fc4bf89… slot:ok) + PR 34713368698 (merge-ref this-os=c0492c3… slot:ok). Both matrices: slot: fat / mapped / ok + stay + qemu-smoke: ok. Docs-only; not a new kernel mile. Later rename SHAs on this branch are not this probe.
CI on GitHub (B5 / prior SHA ea2469e).github/workflows/smoke.yml on cursor/b5-linux-vfs-vs-thin-ctos-f294 @ ea2469eVerifiedPre-rebase tip grepped: push 34713639443 (ubuntu-24.04 job 103606766099 + ubuntu-24.04-arm job 103606766239; this-os=ea2469e… slot:ok). PR 34713642019 success (merge-ref this-os=178d7fd… slot:ok). Both matrices: embed-off, slot: fat / mapped / ok, stay, prior-os=ba6541c… slot:ok, 83 tests, force-fail 1, qemu-smoke: ok. Docs-only; not a new kernel mile. Later rebase SHAs are not this probe.
Pages workflow builds this PR (B5 VFS compare / ea2469e).github/workflows/pages.yml on cursor/b5-linux-vfs-vs-thin-ctos-f294 @ ea2469eVerifiedPR run 34713642021 success (mdBook build job 103606772101). Deploy job skipped on pull_request (expected). Not a new live-site deploy. Later rebase SHAs are not this probe.
mdBook docs site builds locally (B5 / rebase onto 774c672)./scripts/docs-build.sh on cursor/b5-linux-vfs-vs-thin-ctos-f294 after rebase onto #68Verified2026-09-12 this cloud VM: mdbook v0.5.4; mdbook-mermaid 0.17.1; Linux auto-install x86_64-unknown-linux-gnu; docs-build: ok; book/CNAME is ctos.artof.link; generated ADR-033 ELF + ADR-034 VFS both present; track-b has B4 Documented and B5 Documented with Not claiming a Linux filesystem and No Linux VFS object is present. Generator only. Kernel qemu-smoke not run (no src/ edit).
CI on GitHub (B5 / rebase tip 0cd62c2).github/workflows/smoke.yml on cursor/b5-linux-vfs-vs-thin-ctos-f294 @ 0cd62c2 (onto 774c672)VerifiedThis session grepped: push 34714163309 (ubuntu-24.04 job 103608155056 + ubuntu-24.04-arm job 103608154963; this-os=0cd62c2… slot:ok). PR 34714165232 success (merge-ref this-os=20a544f6… slot:ok). Both matrices: embed-off, slot: fat / mapped / ok, stay, prior-os=ba6541c… slot:ok, 83 tests, force-fail 1, qemu-smoke: ok. Docs-only; not a new kernel mile. Later ledger SHAs are not this probe.
Pages workflow builds this PR (B5 VFS compare / 0cd62c2).github/workflows/pages.yml on cursor/b5-linux-vfs-vs-thin-ctos-f294 @ 0cd62c2VerifiedPR run 34714165116 success (mdBook build job 103608159994). Deploy job skipped on pull_request (expected). Not a new live-site deploy.
mdBook docs site builds locally (B5 / rebase onto d81a53a)./scripts/docs-build.sh on cursor/b5-linux-vfs-vs-thin-ctos-f294 after rebase onto #67Verified2026-09-12 this cloud VM: mdbook v0.5.4; mdbook-mermaid 0.17.1; Linux auto-install x86_64-unknown-linux-gnu; docs-build: ok; book/CNAME is ctos.artof.link; generated ADR-033 ELF + ADR-034 VFS + ADR-035 process-model all present; track-b has B3/B4/B5 Documented with Not claiming a Linux filesystem. Generator only. This-tip GHA Unknown until grepped. Kernel qemu-smoke not run (no src/ edit).
qemu-smoke requires identity heap tear stringsscripts/qemu-smoke.sh hello phase greps ident: heap-reloc / ident: heap lo= / ident: heap-fault / ident: heap-high; rejects ident: heap-stayVerified2026-09-12 agent-box: qemu-smoke: identity-tear strings present and heap-stay absent.
scripts/qemu-smoke.sh (ADR-038 identity heap tear / this branch)Ran on this agent boxVerified2026-09-12 ident-heap-tear after dde185f (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305): host perf: elf-size bytes=4432632; hello (timeout 124) with ident: heap-reloc n=0 / ident: heap lo=0x40258000 hi=0x40268000 pages=16 / ident: heap-fault / ident: heap-high plus A1–A9 + data tear markers; no ident: heap-stay; cargo test Running 86 tests all [ok]; force-fail exit 1; A9 cross-update prior-os=ba6541c… slot:ok + this-os=dde185f… slot:ok; qemu-smoke: ok. Not Docker. GHA on this PR is a separate row.
CI on GitHub (ADR-038 / this branch).github/workflows/smoke.yml on ident-heap-tearUnknownNo green GHA URL on the cloud-probe commit yet.
the arm64 workstation / amd64 workstation Docker smoke on dde185f (ADR-037 tip)(amd64 workstation) 2026-09-13 amd64 Docker: ident: data-reloc n=241, ident: data … pages=87, ident: data-fault / ident: data-high, ident: heap-stay, slot: ok, A9 cross-update this-os=dde185f, qemu-smoke: okVerifiedHost bash/WSL still missing — docker build/docker run directly. Heap stay is Historical after ADR-038. PAN enable / umbrella isolation still Planned.
Threat-model v1.19 slice (NFR-10)Read security.md (EL0 trampoline without VMALLE1; PAN enable Planned)VerifiedFile + this PR. v1.19 update, not “secure.”
EL0 entry without TLBI VMALLE1 (ADR-039)Hello-kernel serial el0: no-vmalle1; src/exception.rs has no tlbi vmalle1; #[test_case] el0_entry_without_vmalle1Verified2026-09-12 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on isolation/adr-039-el0-tlbi-without-vmalle1 after f2c0c5a: hello el0: no kernel read / el0: standing / el0: restored / el0: no-vmalle1 / el0: ok / el0: task-ok / asid: ok; qemu-smoke: exception.rs has no tlbi vmalle1; Running 87 tests all [ok] incl. el0_entry_without_vmalle1; force-fail exit 1; A9 cross-update both slot:ok; qemu-smoke: ok. What TLBI remains: paging::init VMALLE1 at MMU enable + TLBI VAAE1 on unmaps. Not “EL0 isolated.” Not PAN. (amd64 workstation) intent; CloudAgent HELD — probe on agent box clone.
qemu-smoke requires el0: no-vmalle1 + exception.rs ratchetscripts/qemu-smoke.sh greps el0: no-vmalle1 and rejects tlbi vmalle1 in src/exception.rsVerified2026-09-12 agent-box: qemu-smoke: exception.rs has no tlbi vmalle1 + EL0 first-mile + read-mile + standing + no-vmalle1 strings present.
scripts/qemu-smoke.sh (ADR-039 EL0 no-VMALLE1 / this branch)Ran on this agent boxVerified2026-09-12 after f2c0c5a (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305): host perf: elf-size bytes=4432656; hello (timeout 124) with el0: no-vmalle1 plus A1–A9 + identity data/heap tear markers; Running 87 tests all [ok]; force-fail exit 1; A9 cross-update prior-os=ba6541c… slot:ok + this-os=f2c0c5a… slot:ok; qemu-smoke: ok. Not Docker. GHA on this PR is a separate row.
CI on GitHub (ADR-039 / this branch).github/workflows/smoke.yml on this branchUnknownNo green GHA URL yet.
Threat-model v1.20 slice (NFR-10)Read security.md (lower-EL IRQ while standing; FIQ/SError still park; PAN enable Planned)VerifiedFile + this PR. v1.20 update, not “secure.”
Lower-EL IRQ while standing (ADR-040)Hello-kernel serial el0: irq; #[test_case] lower_el_irq_while_standing; vector irq_lower_elVerified2026-09-13 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on isolation/adr-040-lower-el-irq-standing after 28d3755: hello el0: standing / el0: restored / el0: standing / timer: tick / el0: irq / el0: restored / el0: no-vmalle1 / el0: ok; Running 88 tests all [ok] incl. lower_el_irq_while_standing; force-fail exit 1; A9 cross-update both slot:ok; qemu-smoke: ok. What still parks: lower-EL FIQ/SError. Default ERET I-clear is ADR-041. Not “EL0 isolated.” Not PAN. (amd64 workstation) intent; CloudAgent HELD — probe on agent box clone.
qemu-smoke requires el0: irqscripts/qemu-smoke.sh greps el0: irqVerified2026-09-13 agent-box: qemu-smoke: EL0 first-mile + read-mile + standing + irq + no-vmalle1 strings present.
scripts/qemu-smoke.sh (ADR-040 lower-EL IRQ / this branch)Ran on this agent boxVerified2026-09-13 after 28d3755 (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305): host perf: elf-size bytes=4438008; hello (timeout 124) with el0: irq plus A1–A9 + identity data/heap tear + no-vmalle1 markers; Running 88 tests all [ok]; force-fail exit 1; A9 cross-update prior-os=ba6541c… slot:ok + this-os=28d3755… slot:ok (pre-commit tip); qemu-smoke: ok. Not Docker. GHA on this PR is a separate row.
CI on GitHub (ADR-040 / this branch).github/workflows/smoke.yml on this branchUnknownNo green GHA URL yet.
Threat-model v1.21 slice (NFR-10)Read security.md (IRQ-unmasked default ERET; FIQ/SError still park; PAN enable Planned)VerifiedFile + this PR. v1.21 update, not “secure.”
IRQ-unmasked default standing/task ERET (ADR-041)Hello-kernel serial el0: irq-default (+ el0: irq); #[test_case] lower_el_irq_default_eret; default eret_to_el0 SPSR 0x340Verified2026-09-13 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on isolation/adr-041-irq-unmasked-standing after 473a73d: hello el0: standing / el0: restored / el0: standing / el0: irq / el0: restored / el0: irq-default / el0: no-vmalle1 / el0: ok; Running 89 tests all [ok] incl. lower_el_irq_default_eret + lower_el_irq_while_standing; force-fail exit 1; A9 cross-update both slot:ok; qemu-smoke: ok. Entry 0x40080000. Short non-standing probes use eret_to_el0_masked. What still parks: lower-EL FIQ/SError. Not “EL0 isolated.” Not PAN. (amd64 workstation) intent; CloudAgent HELD — probe on agent box clone.
qemu-smoke requires el0: irq-defaultscripts/qemu-smoke.sh greps el0: irq-defaultVerified2026-09-13 agent-box: qemu-smoke: EL0 first-mile + read-mile + standing + irq + irq-default + no-vmalle1 strings present.
scripts/qemu-smoke.sh (ADR-041 default IRQ-unmasked ERET / this branch)Ran on this agent boxVerified2026-09-13 after 473a73d (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305): host perf: elf-size bytes=4438016; hello (timeout 124) with el0: irq-default plus A1–A9 + identity data/heap tear + no-vmalle1 markers; Running 89 tests all [ok]; force-fail exit 1; A9 cross-update prior-os=ba6541c… slot:ok + this-os=473a73d… slot:ok (pre-commit tip); qemu-smoke: ok. Not Docker. GHA on this PR is a separate row.
CI on GitHub (ADR-041 / this branch).github/workflows/smoke.yml on this branchUnknownNo green GHA URL yet.
Threat-model v1.22 slice (NFR-10)Read security.md (start-stay / ram-stay honesty; FIQ/SError still park; PAN enable Planned; umbrella Planned)VerifiedFile + this PR. v1.22 update, not “secure.”
Identity _start / leftover-RAM stay honesty (ADR-042)Hello-kernel serial ident: start-stay + ident: ram-stay; #[test_case] identity_boot_stub_stays_while_live_tornVerified2026-09-13 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on isolation/adr-042-isolation-wrap after 75feef9: hello ident: start-stay lo=0x40080000 hi=0x40081000 / ident: ram-stay lo=0x40268000 hi=0x48000000 / ident: ok; Running 90 tests all [ok] incl. identity_boot_stub_stays_while_live_torn; force-fail exit 1; A9 cross-update both slot:ok; qemu-smoke: ok. Entry 0x40080000. Still Planned: PAN enable; lower-EL FIQ/SError; yank _start; umbrella “EL0 isolated.” Not “the kernel moved.” (amd64 workstation) intent; CloudAgent HELD — probe on agent box clone.

| Threat-model v1.23 slice (NFR-10) | Read security.md (lower-EL FIQ while standing; SError park honesty; PAN enable Planned; umbrella Planned) | Verified | File + this PR. v1.23 update, not “secure.” | | Lower-EL FIQ while standing + SError park (ADR-043) | Hello-kernel serial el0: fiq + el0: serror-park; #[test_case] lower_el_fiq_while_standing; vector fiq_lower_el | Verified | 2026-09-13 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on isolation/adr-043-lower-el-fiq-serror after 472d78e: hello el0: standing / el0: irq / el0: fiq / el0: irq-default / el0: serror-park / el0: no-vmalle1 / el0: ok; Running 91 tests all [ok] incl. lower_el_fiq_while_standing; force-fail exit 1; A9 cross-update both slot:ok; qemu-smoke: ok. FIQ via temporary GICC FIQEn + SPSR 0x380. Taken SError still Planned (no safe inject). Entry 0x40080000. Not “EL0 isolated.” Not PAN. (amd64 workstation) intent; CloudAgent HELD — probe on agent box clone. | | qemu-smoke requires ident: start-stay / ident: ram-stay | scripts/qemu-smoke.sh greps ident: start-stay and ident: ram-stay | Verified | 2026-09-13 agent-box: qemu-smoke: identity-tear strings present (includes start-stay / ram-stay). | | scripts/qemu-smoke.sh (ADR-042 isolation leftover wrap / this branch) | Ran on this agent box | Verified | 2026-09-13 after 75feef9 (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305): host perf: elf-size bytes=4440560; hello (timeout 124) with ident: start-stay lo=0x40080000 hi=0x40081000 / ident: ram-stay plus ADR-037…041 markers; Running 90 tests all [ok]; force-fail exit 1; A9 cross-update prior-os=ba6541c… slot:ok + this-os=75feef9… slot:ok (pre-commit tip); qemu-smoke: ok. Not Docker. GHA on this PR is a separate row. | | CI on GitHub (ADR-042 / this branch) | .github/workflows/smoke.yml on this branch | Unknown | No green GHA URL yet. |

How to update

  1. Run or cite the probe (command, file path + revision, or gh run URL).
  2. Change only the rows you probed.
  3. If you could not run QEMU, leave boot Unknown and say so in the PR. A prior Verified row is one environment and one boot path; do not copy the 2026-09-08 x86 VGA probe forward.

| Track B B6 decision documented (ADR-036 / never) | Read ADR-036 and track-b.md | Verified | Document inspection. Explicit never: no Linux-compat layer or reimplementation from B1–B5. Research ladder complete. Not claiming Linux userspace. Track A freestanding path stays. No src/ change. Kernel qemu-smoke not run. | | mdBook docs site builds locally (B6 / Grok Bot docs path) | ./scripts/docs-build.sh when available on a host with mdBook; else Pages GHA on the PR | Unknown | Generator probe deferred to GHA/docs-build on this PR. File presence of ADR-036 is document inspection only. | | the arm64 workstation / amd64 workstation Docker smoke on aeec16c (B6 tip) | (arm64 workstation) earlier on 00ebfc2: slot: ok + A9 cross-update; (amd64 workstation) 2026-09-13 amd64 Docker Desktop 29.7.2, 32 CPU / 31 GiB: docker build -t ctos-smoke . && docker run --rm ctos-smoke → slot: ok, cross-update prior ba6541c + this tip, qemu-smoke: ok | Verified | Host bash/WSL missing on the amd64 workstation — smoke via docker CLI directly (same as scripts/docker-smoke.sh). GHA also green on #69. | | the amd64 workstation Docker smoke on f2c0c5a (ADR-038 tip) | (amd64 workstation) 2026-09-13 amd64 Docker: ident: heap-reloc n=0, ident: heap … pages=16, ident: heap-fault / ident: heap-high, ident: data-*, slot: ok, qemu-smoke: ok | Verified | Host bash/WSL still missing — docker CLI directly. PAN enable / umbrella isolation still Planned. | | the amd64 workstation Docker smoke on 28d3755 (ADR-039 tip) | (amd64 workstation) 2026-09-13 amd64 Docker: el0: no-vmalle1, ident: heap-*, slot: ok, qemu-smoke: ok | Verified | Host bash/WSL missing — docker CLI. PAN enable / lower-EL IRQ while standing / umbrella still Planned. | | the amd64 workstation Docker smoke on 473a73d (ADR-040 tip) | (amd64 workstation) 2026-09-13 amd64 Docker: el0: irq, el0: no-vmalle1, slot: ok, qemu-smoke: ok | Verified | Host bash/WSL missing — docker CLI. PAN / FIQ-SError / default IRQ-unmasked standing / umbrella still Planned. | | the amd64 workstation Docker smoke on 75feef9 (ADR-041 tip) | (amd64 workstation) 2026-09-13 amd64 Docker: el0: irq / el0: irq-default, slot: ok, qemu-smoke: ok | Verified | Host bash/WSL missing — docker CLI. PAN / FIQ-SError / identity _start / umbrella still Planned. | | the amd64 workstation Docker smoke on 472d78e (ADR-042 tip) | (amd64 workstation) 2026-09-13 amd64 Docker: ident: start-stay, ident: ram-stay, el0: irq-default, slot: ok, qemu-smoke: ok | Verified | Host bash/WSL missing — docker CLI. Isolation ladder wrap complete; PAN / FIQ-SError / full _start tear / umbrella stay Planned (ADR-042). | | the amd64 workstation Docker smoke on 65cf515 (ADR-043 tip) | (amd64 workstation) 2026-09-13 amd64 Docker: el0: fiq, el0: serror-park, slot: ok, qemu-smoke: ok | Verified | Host bash/WSL missing — docker CLI. Taken SError / PAN enable / _start tear / umbrella still Planned. | | Taken lower-EL SError research (ADR-044) | Read ADR-044; park marker still ADR-043 | Verified (research) | Docs-only. Candidate inject failed on QEMU 10 virt+a57 (ADR-045). Taken path = deferred/non-goal on this smoke machine (ADR-047) — do not copy Verified onto a taken serial. | | the amd64 workstation Docker smoke on 3f7d3c5 (ADR-045 tip) | (amd64 workstation) 2026-09-13 amd64 Docker: el0: serror-arm, qmp inject-nmi => error:machine does not provide NMIs, el0: serror-park, qemu-smoke: ok | Verified | Taken el0: serror not printed — inject blocked on this QEMU virt. Taken SError / PAN / _start tear / umbrella still Planned. | | the amd64 workstation Docker smoke on 608f643 (ADR-046 tip) | (amd64 workstation) 2026-09-13 amd64 Docker: perf: app-load, perf: embed-load, perf: slot-delta app=… embed=…, slot: ok, qemu-smoke: ok | Verified | Measurement pair only (QEMU TCG lab). No percent / faster-slower claim. Isolation leftovers decided in ADR-047 (this PR). | | Taken lower-EL SError on stock smoke QEMU (ADR-047 / ADR-053 / ADR-081) | Stock distro QEMU: deferred/non-goal; park + A-clear prep | Deferred / non-goal (stock) | Stock path unchanged. Pin Verified: ADR-083; the CI job qemu-nmi-pin has been unconditional since ADR-090 (2026-09-26, D1 evidence class with B2-P). B2-P (real arm64 + KVM) Verified opt-in, one-off: ADR-085. Keep el0: serror-park on distro QEMU. Do not require taken without CTOS_REQUIRE_TAKEN_SERROR=1. B2 (different runner/accelerator) blocked on free hosted runners: ADR-084 (2026-09-25). | | PAN enable on default cortex-a57 (ADR-047 / ADR-054) | Decision: enable non-goal locked; ID-field stays Verified | Non-goal (default probe CPU) — locked | ADR-054 / ADR-047 / ADR-026. Explicit reopen gate in ADR-054. Do not silent -cpu switch. | | Never yank _start (ADR-047) | Decision while QEMU -kernel needs 0x4008_0000; ident: start-stay | Decided (honesty Verified) | ADR-047 / ADR-042. Leftover RAM after heap Verified optional (ADR-049). | | App hosting claim criteria documented (ADR-048) | Read ADR-048; product row still Planned | Verified (docs) | Checklist only. Do not claim “app hosting is done.” | | Threat-model v1.26 slice (NFR-10) | Read security.md (ADR-047 leftover decisions + ADR-048 claim gate) | Verified | File + this PR. v1.26 update, not “secure.” Not “EL0 isolated.” | | Leftover identity RAM tear (ADR-049) | Hello-kernel serial ident: ram lo= + ident: ram-fault + ident: ram-high; reject ident: ram-stay; ident: start-stay still required; #[test_case] identity_ram_torn_high_stays / identity_ram_unmapped_high_stays | Verified | 2026-09-13 agent-box smoke on this branch (see smoke row). High twin stays. Never yank _start. Not “EL0 isolated.” Not “kernel moved.” ADR-049. | | Threat-model v1.27 slice (NFR-10) | Read security.md (ADR-049 leftover identity RAM tear) | Verified | File + this PR. v1.27 update, not “secure.” Not “EL0 isolated.” | | scripts/qemu-smoke.sh (ADR-049 leftover identity RAM tear / this branch) | Ran on this agent box | Verified | 2026-09-13 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/ident-ram-unmap: hello ident: ram lo=0x40268000 hi=0x48000000 pages=32152 / ident: ram-fault / ident: ram-high / ident: start-stay lo=0x40080000 hi=0x40081000 / ident: ok (no ident: ram-stay); Running 94 tests all [ok] incl. identity_ram_torn_high_stays / identity_ram_unmapped_high_stays / identity_boot_stub_stays_while_live_torn; force-fail exit 1; A9 cross-update both slot:ok; qemu-smoke: ok. Not Docker. Not “EL0 isolated.” Not “kernel moved.” (amd64 workstation) not re-run — CloudAgent HELD; evidence tagged agent-box. | | ADR-048 progress checklist + stale embed/cross-update doc honesty (Future slice 1) | Read progress tables in this ledger + hosting-apps.md; stale “A2–A4 still embed” / “Cross-update stays Planned” / Planned delta wording aligned with Verified ADR-032/046 rows; product row still Planned | Verified (docs) | Tip b47bbfc audit on this PR. Docs-only; no src/ change; kernel smoke not re-run. Do not claim “app hosting is done.” | | (amd64 workstation) Alpine WSL bash + Docker Desktop PATH hygiene | Documented host notes for later smoke (docs-only OK) | Verified (docs) | Alpine WSL2 default; install bash: wsl -d Alpine -u root -- apk add bash. Use wsl -d Alpine -u <user> -- bash (not WindowsApps bash stub). Docker Desktop WSL integration not enabled for Alpine — call docker.exe via PATH /mnt/c/Users/<user>/AppData/Local/Programs/DockerDesktop/resources/bin or cmd docker build/run. scripts/docker-smoke.sh is already #!/bin/sh. (amd64 workstation) intent for any later smoke; CloudAgent HELD — this PR is docs-only. | | FAT16 write behind thin VFS (ADR-050 / Future slice 3) | Hello-kernel serial fat: write + fat: rewrite + fat: create + keep fat: ok; host mkfat16.py --check-write; #[test_case] fat16_vfs_write_and_restore + fat16_create_write_small_file | Verified | 2026-09-13 agent-box smoke on feat/fat16-write @ e539813 (see smoke row). Guest wrote FAT16 bytes through thin VFS; created /fwr=fat-nw; restored /probe=fat-hi. Not POSIX. Not FAT32. Not “supports FAT.” App hosting stays Planned. ADR-050. | | Threat-model v1.28 slice (NFR-10) | Read security.md (ADR-050 FAT16 write) | Verified | File + this PR. v1.28 update, not “secure.” | | scripts/qemu-smoke.sh (ADR-050 FAT16 write / this branch) | Ran on this agent box | Verified | 2026-09-13 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/fat16-write @ e539813: hello fat: mount / fat: read / fat: write / fat: rewrite / fat: create / fat: ok; host mkfat16: check-write ok file=/fwr bytes=fat-nw; slot: ok; A9 cross-update both slot:ok; Running 96 tests all [ok] incl. fat16_vfs_write_and_restore + fat16_create_write_small_file; force-fail exit 1; qemu-smoke: ok. Not Docker. Not POSIX. Not FAT32. (amd64 workstation) not re-run — CloudAgent HELD; evidence tagged agent-box. Tip e539813. |

| FAT16 vs memfs write CNTPCT pair (ADR-051 / Future slice 4 / P-PERF-6) | Hello serial perf: fat-write ticks=<n> + perf: memfs-write ticks=<n> + perf: fs-write-delta fat=<a> memfs=<b>; #[test_case] fat_memfs_write_cntpct_pair samples > 0; smoke rejects percent marketing | Verified | 2026-09-13 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/fat-write-cntpct @ 45dc9e1: hello fat: write / fat: rewrite / fat: create / perf: fat-write ticks=66638 / perf: memfs-write ticks=5303 / perf: fs-write-delta fat=66638 memfs=5303 / fat: ok / slot: ok; Running 97 tests all [ok] incl. fat_memfs_write_cntpct_pair; force-fail exit 1; qemu-smoke: ok. QEMU TCG lab — not a published bench, not a percent. Keep ADR-050 write mile. App hosting stays Planned. (amd64 workstation) intent; CloudAgent HELD — probe on agent box clone. ADR-051. | | scripts/qemu-smoke.sh (ADR-051 FAT-vs-memfs write CNTPCT / this branch) | Ran on this agent box | Verified | 2026-09-13 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/fat-write-cntpct: markers above; host --check-write still ok; A9 cross-update both slot:ok; Running 97 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not a latency SLA. Tip 45dc9e1. | | Companion / overview polish (Future slice 6) | Read overview + linked framework pages: no live “A2–A4 still embed,” “cross-update Planned,” ram-stay as the story, “FAT read-only” as the only FAT story; ADR-048 checklist pointer present; ADR-047…051 reflected; product hosting row still Planned | Verified (docs) | Tip ebc968e (includes ADR-051). Docs-only; ./scripts/docs-build.sh ok; no src/ change; kernel smoke not re-run. Do not claim “app hosting is done,” “EL0 isolated,” PAN enable, or taken SError Verified. |

| Sponsor accept product app hosting (ADR-048 #5 / ADR-052) | Sponsor signal + this flip PR; criteria 1–4 already Met on tip be789c5+; no self-merge | Verified (docs) | 2026-09-13: sponsor said “address 1. app hosting” after a still-Planned review that listed product app hosting as blocked only on criterion #5. ADR-052. Product row flipped Planned → Verified. Docs-only; ./scripts/docs-build.sh ok; no src/ change; kernel smoke not re-run (criteria 1–4 already Met). Not Linux userspace, POSIX, containers, “EL0 isolated,” PAN enable, taken SError Verified, immutable OS, or OTA. | | Threat-model v1.29 slice (NFR-10) | Read security.md (ADR-048/052 product app-hosting claim Verified; non-claims unchanged) | Verified | File + this PR. v1.29 update, not “secure.” Not “EL0 isolated.” Not Linux/POSIX/containers. | | Taken lower-EL SError hard-stop (ADR-053) | Dated QMP re-probe: inject-nmi / HMP nmi → machine does not provide NMIs; park stays Verified; taken locked deferred/non-goal | Deferred / non-goal (locked) | 2026-09-14 agent-box QEMU 10.0.13 -machine virt -cpu cortex-a57: QMP inject-nmi error machine does not provide NMIs; HMP nmi same. Prior smoke: amd64 workstation 3f7d3c5 / ADR-045. Reopen attempt ADR-081. Do not claim taken Verified. ADR-053. | | PAN enable lock (ADR-054) | Enable non-goal locked on default cortex-a57; reopen gate documented; ID-field stays Verified | Non-goal (locked) | Docs-only 2026-09-14. Evidence: ADR-026 pan: absent; scripts keep -cpu cortex-a57. Reopen = sponsor-approved CPU + FEAT_PAN + enable+fault ADR. Do not claim PAN enabled. ADR-054. | | Umbrella “EL0 isolated” checklist (ADR-055; audited 2026-09-26 in ADR-086) | Evidence table: Verified miles vs unmet (PAN enable, taken SError, yank _start, sponsor umbrella accept) | Verified (§B rows 1–3 Met; umbrella = the ADR-091 sentence only) | M6 (2026-09-28): row 3 Met (sponsor’s written accept, B3); see the P-SEC-3 row. Docs-only 2026-09-14: “Cannot become Verified under current constraints” (as of then). Do not round Verified miles into “EL0 isolated.” ADR-055. Sponsor one-pager ADR-060. | | Threat-model v1.30 slice (NFR-10) | Read security.md (ADR-053/054/055 isolation-gap locks; non-claims unchanged) | Verified | File + this PR. v1.30 update, not “secure.” Not “EL0 isolated.” Not PAN enabled. Not taken SError Verified. |

| FAT16 readdir behind thin VFS (ADR-056) | Hello-kernel serial fat: readdir + fat: entries n=<k> + keep fat: ok / write / slot: ok; #[test_case] fat16_readdir_lists_root + fat16_readdir_fail_closed_cap | Verified | 2026-09-14 agent-box smoke on feat/fat16-readdir: hello fat: readdir / fat: entries n=3 / fat: ok; Running 99 tests all [ok] incl. readdir cases; force-fail exit 1; qemu-smoke: ok. Guest listed FAT16 root paths through thin VFS (/probe, /hello, /fwr). Not POSIX getdents/opendir. Not FAT32. Not “supports FAT.” Not EL0 isolated / PAN / taken SError. CloudAgent HELD — evidence tagged agent-box. Tip abe72cf. ADR-056. | | Threat-model v1.31 slice (NFR-10) | Read security.md (ADR-056 FAT16 readdir) | Verified | File + this PR. v1.31 update, not “secure.” Not POSIX getdents. | | scripts/qemu-smoke.sh (ADR-056 FAT16 readdir / this branch) | Ran on this agent box | Verified | 2026-09-14 agent-box qemu-smoke (QEMU 10.0.13, -cpu cortex-a57) on feat/fat16-readdir: markers fat: mount / fat: read / fat: write / fat: rewrite / fat: create / fat: readdir / fat: entries n=3 / fat: ok / slot: ok; host --check-write ok; A9 cross-update both slot:ok; Running 99 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not POSIX. Tip abe72cf. |

| FAT16 delete behind thin VFS (ADR-057) | Hello-kernel serial fat: delete + keep fat: ok / write / readdir / slot: ok; #[test_case] fat16_unlink_removes_file + fat16_unlink_missing_is_err | Verified | 2026-09-14 agent-box smoke on feat/fat16-delete: hello fat: delete / fat: readdir / fat: entries n=3 / fat: ok; Running 101 tests all [ok] incl. unlink cases; force-fail exit 1; qemu-smoke: ok. Guest deleted FAT16 root dirent /fdel through thin VFS (vfs::unlink). Not POSIX unlink/remove. Not FAT32. Not “supports FAT.” Not EL0 isolated / PAN / taken SError. CloudAgent HELD — evidence tagged agent-box. Tip c945ec8. ADR-057. | | Threat-model v1.32 slice (NFR-10) | Read security.md (ADR-057 FAT16 delete) | Verified | File + this PR. v1.32 update, not “secure.” Not POSIX unlink. | | scripts/qemu-smoke.sh (ADR-057 FAT16 delete / this branch) | Ran on this agent box | Verified | 2026-09-14 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/fat16-delete: markers fat: mount / fat: read / fat: write / fat: rewrite / fat: create / fat: readdir / fat: entries n=3 / fat: delete / fat: ok / slot: ok; host --check-write ok; A9 cross-update both slot:ok; Running 101 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not POSIX. Tip c945ec8. |

| Thin VFS prefix mounts (ADR-058) | Hello-kernel serial vfs: mount + vfs: mounts + keep fs: ok / fat: ok / slot: ok; #[test_case] vfs_prefix_mounts_route_backends + vfs_memfs_mount_create_stays_off_fat | Verified | 2026-09-14 agent-box smoke on feat/vfs-virtual-mounts @ 4814735: hello vfs: mount /kprobe -> memfs … vfs: mount / -> fat / vfs: mounts / fs: ok / fat: ok / slot: ok; Running 103 tests all [ok] incl. mount cases; force-fail exit 1; qemu-smoke: ok. Guest routed path prefixes through a thin VFS mount table (/mem+A6 names → memfs; / → FAT16). Not POSIX mount(2). Not Linux VFS. Not a new disk FS. Not EL0 isolated / PAN / taken SError. CloudAgent HELD — evidence tagged agent-box. ADR-058. | | scripts/qemu-smoke.sh (ADR-058 VFS prefix mounts / this branch) | Ran on this agent box | Verified | 2026-09-14 agent-box qemu-smoke (QEMU 10.x, -cpu cortex-a57) on feat/vfs-virtual-mounts @ 4814735: markers vfs: mount / vfs: mounts / fs: ok / fat: mount / fat: read / fat: write / fat: rewrite / fat: create / fat: readdir / fat: entries n=3 / fat: delete / fat: ok / slot: ok; host --check-write ok; A9 cross-update both slot:ok; Running 103 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not POSIX. Tip 4814735. |

| Freestanding fs-libctos sample + catalog (ADR-059) | Hello-kernel serial libctos: fs-hi + libctos: fs-ok + fsdemo: ok + keep slot: ok / /hello; #[test_case] fsdemo_fat_is_elf + fsdemo_load_from_fat_erets + fsdemo_path_is_memdemo_mount | Verified | 2026-09-14 agent-box smoke on feat/fs-libctos-sample: hello libctos: fs-hi / libctos: fs-ok / fsdemo: fat / fsdemo: mapped / fsdemo: ok / slot: ok; Running 106 tests all [ok] incl. fsdemo cases; force-fail exit 1; qemu-smoke: ok. Guest loaded FAT /fsdemo and the payload exercised libctos VFS on /memdemo (memfs via ADR-058 /mem prefix; EL0 exec pages fetch-only — compare with immediates). Not POSIX. Not getdents. Not EL0 isolated / PAN / taken SError. Do not reopen ADR-048/052 app hosting. CloudAgent HELD — evidence tagged agent-box (amd64 workstation Computer tool unavailable this turn). ADR-059. | | Threat-model v1.34 slice (NFR-10) | Read security.md (ADR-059 fs-libctos catalog) | Verified | File + this PR. v1.34 update, not “secure.” Not POSIX. | | scripts/qemu-smoke.sh (ADR-059 fs-libctos / this branch) | Ran on this agent box | Verified | 2026-09-14 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/fs-libctos-sample: markers libctos: fs-hi / libctos: fs-ok / fsdemo: ok + preserved vfs: mounts / fs: ok / fat: ok / slot: ok / A9 cross-update both slot:ok; Running 106 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not POSIX. Tip ca707e7. | | Sponsor isolation leftovers closure checklist (ADR-060) | Single table: taken SError / PAN enable / umbrella / never-yank _start — locks + reopen gates only | Docs lock (non-Verified) | Docs-only 2026-09-15. Cites ADR-047 / ADR-053 / ADR-054 / ADR-055 / #98. Does not invent Verified for taken SError / PAN enable / “EL0 isolated.” M6 (2026-09-28): its umbrella row now reads Verified, only as the accepted ADR-091 sentence; the other locks are unchanged. Does not silent -cpu, yank _start, or reopen Guest Linux/containers/immutable marketing. ADR-060. | | Threat-model v1.35 slice (NFR-10) | Read security.md (ADR-060 sponsor leftovers closure; non-claims unchanged) | Verified | File + this PR. v1.35 update, not “secure.” Not “EL0 isolated.” Not PAN enabled. Not taken SError Verified. |

| Freestanding fat-libctos sample + catalog (ADR-061) | Hello-kernel serial libctos: fat-hi + libctos: fat-ok + fatdemo: ok + keep slot: ok / fsdemo: ok / /hello / /fsdemo; #[test_case] fatdemo_fat_is_elf + fatdemo_load_from_fat_erets + fatdemo_path_is_fat_probe | Verified | 2026-09-15 agent-box smoke on feat/fat-libctos-sample: hello libctos: fat-hi / libctos: fat-ok / fatdemo: fat / fatdemo: mapped / fatdemo: ok / slot: ok / fsdemo: ok; Running 109 tests all [ok] incl. fatdemo cases; force-fail exit 1; qemu-smoke: ok. Guest loaded FAT /fatdemo and the payload opened/read FAT /probe (fat-hi) via thin VFS (EL0 exec pages fetch-only — compare with immediates). Not POSIX. Not getdents. Not EL0 isolated / PAN / taken SError. Do not reopen ADR-048/052 app hosting. CloudAgent HELD — evidence tagged agent-box (amd64 workstation Computer/CloudAgent unavailable this turn). Tip 8bf25d2. ADR-061. | | Threat-model v1.36 slice (NFR-10) | Read security.md (ADR-061 fat-libctos catalog) | Verified | File + this PR. v1.36 update, not “secure.” Not POSIX. | | Freestanding yield-libctos sample + catalog (ADR-062) | Hello-kernel serial libctos: yld-hi + ≥3× libctos: beat + libctos: yld-ok + yldemo: ok + keep slot: ok / fsdemo: ok / fatdemo: ok / /hello / /fsdemo / /fatdemo; #[test_case] yldemo_fat_is_elf + yldemo_load_from_fat_erets + yldemo_path_is_fat | Verified | 2026-09-16 agent-box smoke on feat/yield-libctos-el0-sample: hello libctos: yld-hi / 3× libctos: beat / libctos: yld-ok / yldemo: fat / yldemo: mapped / yldemo: ok / slot: ok / fsdemo: ok / fatdemo: ok; Running 112 tests all [ok] incl. yldemo cases; force-fail exit 1; qemu-smoke: ok (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305). Guest loaded FAT /yldemo and the payload issued several cooperative yield_now() rounds (deeper than hello’s single yield). Not preemption. Not multi-task EL0. Not a process table. Not POSIX. Not EL0 isolated / PAN / taken SError. Do not reopen ADR-048/052 app hosting. CloudAgent HELD — evidence tagged agent-box. Tip 7a5a529. ADR-062. | | Threat-model v1.37 slice (NFR-10) | Read security.md (ADR-062 yield-libctos catalog) | Verified | File + this PR. v1.37 update, not “secure.” Not preemption. | | scripts/qemu-smoke.sh (ADR-061 fat-libctos / this branch) | Ran on this agent box | Verified | 2026-09-15 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/fat-libctos-sample: markers libctos: fat-hi / libctos: fat-ok / fatdemo: ok + preserved libctos: fs-ok / fsdemo: ok / vfs: mounts / fs: ok / fat: ok / slot: ok / A9 cross-update both slot:ok; Running 109 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not POSIX. Tip 8bf25d2. |

| Track N network foundation scope (ADR-063 / N0) | Read ADR-063 + track-n.md; SUMMARY / roadmap / limits wired; no virtio-net driver in this PR | Docs lock (non-Verified) | Docs-only 2026-09-16. Opens Track N. Runtime link/first-frame stays Planned. Do not invent Verified. Do not claim TCP/UDP/sockets/DHCP/DNS/Wi‑Fi/Linux net/“has networking.” ADR-063. | | Track N virtio-net first frame (N1 / ADR-066) | Hello-kernel serial net: virtio + net: mac + net: tx + net: rx + net: ok; QEMU -netdev user,id=net0 -device virtio-net-device,netdev=net0; keep blk: ok / fat: ok; #[test_case] virtio_net_discover_and_arp | Verified | 2026-09-16 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/n1-virtio-net-first-frame: hello net: virtio / net: mac 52:54:00:12:34:56 / net: tx / net: rx / net: ok + optional perf: net-tx / perf: net-rx + keep blk: ok / fat: ok / slot: ok; Running 116 tests all [ok] incl. virtio_net_discover_and_arp; force-fail exit 1; qemu-smoke: ok. Probe = ARP request for SLIRP gateway 10.0.2.2 + reply. Not TCP/UDP/sockets/DHCP/DNS/Wi‑Fi/“has networking.” CloudAgent HELD — evidence tagged agent-box. ADR-066. | | Threat-model v1.38 slice (NFR-10) | Read security.md (ADR-063 Track N Planned; I/O surface) | Verified | File + #109. v1.38 update, not “secure.” Not “has networking.” | | FAT16 multi-cluster grow (ADR-064) | Hello-kernel serial fat: grow + libctos: fat-grow + keep fat: ok / fat: write / fatdemo: ok; #[test_case] fat16_multi_cluster_grow + fat16_grow_refuses_over_file_max | Verified | 2026-09-16 agent-box smoke on feat/fat-multi-cluster-grow (rebased on main post-#109): hello fat: grow / libctos: fat-grow / libctos: fat-ok / fatdemo: ok / fat: ok; Running 114 tests all [ok] incl. grow cases; force-fail exit 1; qemu-smoke: ok (QEMU 10.x, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57). Guest allocated a FAT chain and wrote 600 bytes across a 512-byte cluster boundary; EL0 chunked SVC write/read-back. FILE_MAX raised to 1024. Not POSIX write API. Not exFAT/FAT32. Not Track N. CloudAgent HELD — evidence tagged agent-box. Tip cb3a946. ADR-064. | | Threat-model v1.39 slice (NFR-10) | Read security.md (ADR-064 FAT16 multi-cluster grow; keeps ADR-063 Track N Planned) | Verified | File + this PR. v1.39 update on tip post-#109 v1.38, not “secure.” Not POSIX. Not “has networking.” | | scripts/qemu-smoke.sh (ADR-064 multi-cluster grow / this branch) | Ran on this agent box | Verified | 2026-09-16 agent-box qemu-smoke on feat/fat-multi-cluster-grow (rebased on b4efb5f): markers fat: grow / libctos: fat-grow + preserved fat: write / fat: delete / fat: ok / fatdemo: ok / slot: ok; Running 114 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not POSIX. Tip cb3a946. | | FAT16 vs memfs read CNTPCT pair (ADR-065 / perf slice 3 / P-PERF-7) | Hello serial perf: fat-read ticks=<n> + perf: memfs-read ticks=<n> + perf: fs-read-delta fat=<a> memfs=<b>; #[test_case] fat_memfs_read_cntpct_pair samples > 0; smoke rejects percent marketing; /mrprobe memfs mount prefix | Verified | 2026-09-16 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/fat-memfs-read-cntpct @ 0abba31: hello perf: fat-read ticks=7752 / perf: memfs-read ticks=2210 / perf: fs-read-delta fat=7752 memfs=2210 + keep perf: fat-write / perf: fs-write-delta / fat: ok / slot: ok; Running 115 tests all [ok] incl. fat_memfs_read_cntpct_pair; force-fail exit 1; qemu-smoke: ok. QEMU TCG lab — not a published bench, not a percent. Keep ADR-051 write pair. App hosting stays Planned. CloudAgent HELD — probe on agent box. ADR-065. | | Threat-model v1.40 slice (NFR-10) | Read security.md (ADR-065 FAT vs memfs read CNTPCT; keeps ADR-063 Track N Planned) | Verified | File + this PR. v1.40 update on tip post-#110 v1.39, not “secure.” Not a latency SLA. Not “has networking.” | | scripts/qemu-smoke.sh (ADR-065 FAT-vs-memfs read CNTPCT / this branch) | Ran on this agent box | Verified | 2026-09-16 agent-box qemu-smoke on feat/fat-memfs-read-cntpct: markers above; ADR-051 write pair still present; Running 115 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not a latency SLA. Tip 0abba31. | | Threat-model v1.41 slice (NFR-10) | Read security.md (ADR-066 N1 virtio-net first frame; I/O surface names blk+net) | Verified | File + this PR. v1.41 update on tip post-#111 v1.40, not “secure.” Not TCP/UDP. Not “has networking.” | | scripts/qemu-smoke.sh (ADR-066 N1 virtio-net / this branch) | Ran on this agent box | Verified | 2026-09-16 agent-box qemu-smoke on feat/n1-virtio-net-first-frame: markers net: virtio / net: mac / net: tx / net: rx / net: ok + preserved blk: ok / fat: ok / slot: ok / ADR-065 read pair; qemu-smoke: virtio-net (N1) strings present; Running 116 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not a product stack. Tip b87ddba. |

| Track N virtio-net ICMP ping (N2 / ADR-067) | Hello-kernel serial net: icmp-tx + net: icmp-rx + net: ping-ok (+ keep N1 net: ok); QEMU -netdev user,id=net0 -device virtio-net-device,netdev=net0 unchanged; keep blk: ok / fat: ok; #[test_case] virtio_net_icmp_echo_ping | Verified | 2026-09-16 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/n2-icmp-ping: hello net: virtio / net: mac 52:54:00:12:34:56 / net: tx / net: rx / net: ok / net: icmp-tx / net: icmp-rx / net: ping-ok + optional perf: net-icmp-tx / perf: net-icmp-rx + keep blk: ok / fat: ok / slot: ok; Running 117 tests all [ok] incl. virtio_net_discover_and_arp + virtio_net_icmp_echo_ping; force-fail exit 1; qemu-smoke: ok. Probe = ICMP echo vs SLIRP gateway 10.0.2.2 after ARP. Not TCP/UDP/sockets/DHCP/DNS/Wi‑Fi/“has networking.” CloudAgent HELD — evidence tagged agent-box. ADR-067. | | Threat-model v1.42 slice (NFR-10) | Read security.md (ADR-067 N2 ICMP ping; I/O surface still blk+net) | Verified | File + this PR. v1.42 update on tip post-#112 v1.41, not “secure.” Not TCP/UDP. Not “has networking.” | | scripts/qemu-smoke.sh (ADR-067 N2 ICMP ping / this branch) | Ran on this agent box | Verified | 2026-09-16 agent-box qemu-smoke on feat/n2-icmp-ping: markers net: icmp-tx / net: icmp-rx / net: ping-ok + N1 net: ok + preserved blk: ok / fat: ok / slot: ok / ADR-065 read pair; qemu-smoke: virtio-net (N1+N2) strings present; Running 117 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not a product stack. Tip b71d7f4. |

| Track N EL0 net SVC + net-libctos (N4 / ADR-068) | Hello-kernel serial libctos: net-hi + libctos: net-mac + libctos: net-ok + netdemo: ok (+ keep N1 net: ok / N2 net: ping-ok / prior samples); #[test_case] netdemo_fat_is_elf + netdemo_load_from_fat_erets + netdemo_path_is_fat | Verified | 2026-09-16 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/n4-el0-net-svc: hello libctos: net-hi / libctos: net-mac / libctos: net-ok / netdemo: fat / netdemo: mapped / netdemo: ok + keep net: ok / net: ping-ok / blk: ok / fat: ok / slot: ok / yldemo: ok; Running 120 tests all [ok] incl. netdemo cases; force-fail exit 1; qemu-smoke: ok. SVC surface: net_mac=24, net_ping=25. Kernel owns virtio-net. Not TCP/UDP/sockets/DHCP/DNS/Wi‑Fi/“has networking.” CloudAgent HELD — evidence tagged agent-box. Tip f1942ab. ADR-068. | | Threat-model v1.43 slice (NFR-10) | Read security.md (ADR-068 N4 EL0 net SVC sample; I/O surface still blk+net) | Verified | File + this PR. v1.43 update on tip post-#113 v1.42, not “secure.” Not TCP/UDP. Not “has networking.” | | scripts/qemu-smoke.sh (ADR-068 N4 net-libctos / this branch) | Ran on this agent box | Verified | 2026-09-16 agent-box qemu-smoke on feat/n4-el0-net-svc: markers libctos: net-ok / netdemo: ok + preserved N1/N2/samples; qemu-smoke: ADR-068 net-libctos /netdemo strings present; Running 120 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not a product stack. Tip f1942ab. |

| Net-ping path CNTPCT (ADR-069 / P-PERF-8) | Hello serial perf: net-ping ticks=<n> (prefix grep; not exact ticks); #[test_case] el0_net_ping_cntpct_advances sample > 0; smoke rejects percent marketing; keep N1/N2/N4 | Verified | 2026-09-16 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/net-ping-cntpct: hello perf: net-ping ticks=20370 + keep net: ping-ok / libctos: net-ok / netdemo: ok / blk: ok / fat: ok / slot: ok; Running 121 tests all [ok] incl. el0_net_ping_cntpct_advances (perf: net-ping ticks=13052 in that case); force-fail exit 1; qemu-smoke: ok. QEMU TCG lab — not a published bench, not a percent, not a ping SLA. Window = quiet ARP+ICMP as SYS_NET_PING / el0_icmp_ping. CloudAgent HELD — evidence tagged agent-box. ADR-069. | | Threat-model v1.44 slice (NFR-10) | Read security.md (ADR-069 net-ping CNTPCT; keeps N1/N2/N4) | Verified | File + this PR. v1.44 update on tip post-#114 v1.43, not “secure.” Not a latency SLA. Not “has networking.” | | scripts/qemu-smoke.sh (ADR-069 net-ping CNTPCT / this branch) | Ran on this agent box | Verified | 2026-09-16 agent-box qemu-smoke on feat/net-ping-cntpct: marker perf: net-ping ticks=20370 + preserved N1/N2/N4/samples; qemu-smoke: ADR-069 net-ping CNTPCT marker present; Running 121 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not a product stack. Tip e20c852. |

| Track N UDP transport (N3 / ADR-070) | Hello-kernel serial net: udp-tx + net: udp-rx + net: udp-ok (+ keep N1 net: ok / N2 net: ping-ok / N4 samples / ADR-069); QEMU -netdev user,id=net0 -device virtio-net-device,netdev=net0 unchanged; #[test_case] virtio_net_udp_dns_probe | Verified | 2026-09-16 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/n3-udp-transport: hello net: udp-tx / net: udp-rx / net: udp-ok + keep net: ok / net: ping-ok / libctos: net-ok / netdemo: ok / perf: net-ping / blk: ok / fat: ok / slot: ok; Running 122 tests all [ok] incl. virtio_net_udp_dns_probe; force-fail exit 1; qemu-smoke: ok. Probe = UDP DNS query to SLIRP 10.0.2.3:53 + matching DNS response (QR=1). Not BSD sockets / TCP product / DHCP/DNS product / Wi‑Fi / “has networking.” CloudAgent HELD — evidence tagged agent-box. ADR-070. | | Threat-model v1.45 slice (NFR-10) | Read security.md (ADR-070 N3 minimal UDP; I/O surface still blk+net) | Verified | File + this PR. v1.45 update on tip post-#115 v1.44, not “secure.” Not BSD sockets. Not “has networking.” | | scripts/qemu-smoke.sh (ADR-070 N3 UDP / this branch) | Ran on this agent box | Verified | 2026-09-16 agent-box qemu-smoke on feat/n3-udp-transport: markers net: udp-tx / net: udp-rx / net: udp-ok + preserved N1/N2/N4/samples/ADR-069; qemu-smoke: virtio-net (N1+N2+N3) strings present; Running 122 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not a product stack. Tip 7e74752. | | Track N EL0 UDP DNS SVC + udp-libctos (N3.x / ADR-071) | Hello-kernel serial libctos: udp-hi + libctos: udp-ok + udpdemo: ok (+ keep N1 net: ok / N2 net: ping-ok / N3 net: udp-ok / N4 libctos: net-ok / prior samples / ADR-069); #[test_case] udpdemo_fat_is_elf + udpdemo_load_from_fat_erets + udpdemo_path_is_fat + el0_udp_dns_probe | Verified | 2026-09-16 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/n3x-el0-udp-svc: hello libctos: udp-hi / libctos: udp-ok / udpdemo: fat / udpdemo: mapped / udpdemo: ok + keep net: ok / net: ping-ok / net: udp-ok / libctos: net-ok / netdemo: ok / perf: net-ping / blk: ok / fat: ok / slot: ok; Running 126 tests all [ok] incl. el0_udp_dns_probe + udpdemo cases; force-fail exit 1; qemu-smoke: ok. SVC surface: net_udp_dns=26 (keep 24/25). FAT /udpdemo. Kernel owns virtio-net. DNS probe bait only. Not TCP/sockets/DHCP/DNS product/Wi‑Fi/“has networking.” CloudAgent HELD — evidence tagged agent-box. Tip 4c9e8a8. ADR-071. | | Threat-model v1.46 slice (NFR-10) | Read security.md (ADR-071 N3.x EL0 UDP DNS SVC; I/O surface still blk+net) | Verified | File + this PR. v1.46 update on tip post-#116 v1.45, not “secure.” Not BSD sockets. Not “has networking.” | | scripts/qemu-smoke.sh (ADR-071 N3.x / this branch) | Ran on this agent box | Verified | 2026-09-16 agent-box qemu-smoke on feat/n3x-el0-udp-svc: markers libctos: udp-hi / libctos: udp-ok / udpdemo: ok + preserved N1/N2/N3/N4/samples/ADR-069; qemu-smoke: ADR-071 udp-libctos /udpdemo strings present; Running 126 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not a product stack. Tip 4c9e8a8. |

| UDP DNS path CNTPCT (ADR-072 / P-PERF-9) | Hello serial perf: udp-dns ticks=<n> (prefix grep; not exact ticks); #[test_case] el0_udp_dns_cntpct_advances sample > 0; smoke rejects percent marketing; keep N1/N2/N3/N3.x/N4 + ADR-069 | Verified | 2026-09-16 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/udp-dns-cntpct: hello perf: udp-dns ticks=30943 + keep net: udp-ok / libctos: udp-ok / udpdemo: ok / perf: net-ping / blk: ok / fat: ok / slot: ok; Running 127 tests all [ok] incl. el0_udp_dns_cntpct_advances (perf: udp-dns ticks=65700 in that case); force-fail exit 1; qemu-smoke: ok. QEMU TCG lab — not a published bench, not a percent, not a DNS SLA. Window = quiet ARP+UDP DNS as SYS_NET_UDP_DNS / el0_udp_dns. CloudAgent HELD — evidence tagged agent-box. Tip 4989d68. ADR-072. | | Threat-model v1.47 slice (NFR-10) | Read security.md (ADR-072 UDP DNS CNTPCT; keeps N1–N4 + ADR-069) | Verified | File + this PR. v1.47 update on tip post-#117 v1.46, not “secure.” Not a latency SLA. Not “has networking.” | | scripts/qemu-smoke.sh (ADR-072 UDP DNS CNTPCT / this branch) | Ran on this agent box | Verified | 2026-09-16 agent-box qemu-smoke on feat/udp-dns-cntpct: marker perf: udp-dns ticks=30943 + preserved N1/N2/N3/N3.x/N4/samples/ADR-069; qemu-smoke: ADR-072 udp-dns CNTPCT marker present; Running 127 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not a product stack. Tip 4989d68. |

| FAT16 mkdir behind thin VFS (ADR-073 / sponsor mile 2) | Hello-kernel serial fat: mkdir + keep fat: ok / write / readdir / delete / grow / slot: ok; #[test_case] fat16_mkdir_root_dir + fat16_mkdir_exists_and_bad_path | Verified | 2026-09-16 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/fat-mkdir: hello fat: mkdir + keep fat: grow / fat: delete / fat: ok / slot: ok / net+sample markers; Running 129 tests all [ok] incl. mkdir cases; force-fail exit 1; qemu-smoke: ok. Guest created FAT16 root dir /fdir (dirent + ./.. cluster) via thin VFS. Not POSIX mkdir. Not a directory tree. Not exFAT/FAT32. No new SVC (EL0 follow-up). CloudAgent HELD — evidence tagged agent-box. Tip b0aa139. ADR-073. | | Threat-model v1.48 slice (NFR-10) | Read security.md (ADR-073 FAT16 mkdir; keeps N1–N4 + ADR-069/072) | Verified | File + this PR. v1.48 update on tip post-#118 v1.47, not “secure.” Not POSIX mkdir. Not TCP. | | scripts/qemu-smoke.sh (ADR-073 FAT16 mkdir / this branch) | Ran on this agent box | Verified | 2026-09-16 agent-box qemu-smoke on feat/fat-mkdir: marker fat: mkdir + preserved grow/delete/ok/net/samples; qemu-smoke: FAT16 mkdir (ADR-073) strings present; force-fail exit 1; qemu-smoke: ok. Not Docker. Not POSIX. Tip b0aa139. |

ADR-048 progress checklist

Living audit of ADR-048 against tip main @ be789c5 (or newer). Site mirror: hosting-apps.md. Product “app hosting is done” is Verified under criteria Met×5 (ADR-052). Still not Linux/POSIX/containers/“EL0 isolated.”

#CriterionStatusTip evidence
1Documented recipes (A8)MetRow “Documented sample apps (Track A A8)” Verified; what-can-run.md.
2Slot cross-updateMetRow “A9 cross-update (same app on OS n and n+1)” Verified (cross-update … slot:ok on this OS + ba6541c). Later tips re-probed (ADR-046 / amd64 workstation Docker rows).
3No production embedMetRow “A2–A4 hello without include_bytes! (ADR-032)” Verified; smoke rejects slot: embed. Probe-only include_bytes! in src/slot.rs only (ADR-046). Tip src/ audit: production A2–A4 FAT-only.
4Honesty: no Linux/POSIX/containersMetContainers non-goal Verified (ADR-029); docs refuse Linux ABI / POSIX / guest OCI.
5Explicit sponsor acceptMetSponsor: “address 1. app hosting” after still-Planned review blocked only on #5. Record: ADR-052. Flip PR cites ADR-048 checklist; no self-merge (ADR-002).

| Track N thin TCP (N5 / ADR-074) | Hello-kernel serial net: tcp-syn + net: tcp-est + net: tcp-tx + net: tcp-rx + net: tcp-ok (+ keep N1 net: ok / N2 net: ping-ok / N3 net: udp-ok / N3.x/N4 samples / FAT); QEMU -netdev user,id=net0,guestfwd=tcp:10.0.2.4:7-cmd:…/tcp-echo-stdio.sh -device virtio-net-device,netdev=net0; #[test_case] virtio_net_tcp_echo_probe | Verified | 2026-09-16 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/n5-thin-tcp: hello net: tcp-syn / net: tcp-est / net: tcp-tx / net: tcp-rx / net: tcp-ok + keep prior net/FAT/sample markers; Running 130 tests all [ok] incl. virtio_net_tcp_echo_probe; force-fail exit 1; qemu-smoke: ok. Probe = TCP active-open + one payload echo via QEMU guestfwd 10.0.2.4:7. Not BSD sockets / listen-accept / TLS/HTTP / DHCP/DNS product / Wi‑Fi / “has networking.” CloudAgent HELD — evidence tagged agent-box. Tip a780f67. ADR-074. | | Threat-model v1.49 slice (NFR-10) | Read security.md (ADR-074 N5 thin TCP; I/O surface still blk+net) | Verified | File + this PR. v1.49 update on tip post-#119 v1.48, not “secure.” Not BSD sockets. Not “has networking.” | | scripts/qemu-smoke.sh (ADR-074 N5 thin TCP / this branch) | Ran on this agent box | Verified | 2026-09-16 agent-box qemu-smoke on feat/n5-thin-tcp: markers net: tcp-syn / net: tcp-est / net: tcp-tx / net: tcp-rx / net: tcp-ok + preserved N1–N3/N3.x/N4/FAT/samples; qemu-smoke: virtio-net (N1+N2+N3+N5) strings present; force-fail exit 1; qemu-smoke: ok. Not Docker. Not a sockets product. Tip a780f67. |

| EL0 fs_mkdir + mkdir-libctos catalog (ADR-075) | Hello-kernel serial libctos: mkdir-hi + libctos: mkdir-ok + mkdemo: ok + keep fat: mkdir / prior six samples / net; #[test_case] mkdemo_fat_is_elf + mkdemo_load_from_fat_erets + mkdemo_path_is_fat | Verified | 2026-09-17 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/el0-fs-mkdir: hello libctos: mkdir-hi / libctos: mkdir-ok / mkdemo: fat / mkdemo: mapped / mkdemo: ok + keep fat: mkdir / sample+net markers; Running 133 tests all [ok] incl. mkdemo cases; force-fail exit 1; qemu-smoke: ok. Guest loaded FAT /mkdemo and EL0 issued fs_mkdir (27) creating /edir with Exists→1 / BadPath→2 mapping. Not POSIX mkdir. Not a nested tree. Not EL0 TCP. CloudAgent HELD — evidence tagged agent-box. Tip 1c9c4dd. ADR-075. | | Threat-model v1.50 slice (NFR-10) | Read security.md (ADR-075 EL0 fs_mkdir; keeps N1–N5 + ADR-073) | Verified | File + this PR. v1.50 update on tip post-#120 v1.49, not “secure.” Not POSIX mkdir. Not nested trees. | | scripts/qemu-smoke.sh (ADR-075 EL0 fs_mkdir / this branch) | Ran on this agent box | Verified | 2026-09-17 agent-box qemu-smoke on feat/el0-fs-mkdir: markers libctos: mkdir-ok / mkdemo: ok + preserved fat: mkdir / net/samples; qemu-smoke: ADR-075 mkdir-libctos /mkdemo strings present; force-fail exit 1; qemu-smoke: ok. Not Docker. Not POSIX. Tip 1c9c4dd. | | EL0 net_tcp_echo + tcp-libctos catalog (ADR-076) | Hello-kernel serial libctos: tcp-hi + libctos: tcp-ok + tcpdemo: ok + keep net: tcp-ok / prior seven samples / FAT; #[test_case] tcpdemo_fat_is_elf + tcpdemo_load_from_fat_erets + tcpdemo_path_is_fat + el0_tcp_echo_probe | Verified | 2026-09-17 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/el0-tcp-svc: hello libctos: tcp-hi / libctos: tcp-ok / tcpdemo: fat / tcpdemo: mapped / tcpdemo: ok + keep net: tcp-ok / sample+FAT markers; Running 137 tests all [ok] incl. tcpdemo + el0_tcp_echo_probe; force-fail exit 1; qemu-smoke: ok. Guest loaded FAT /tcpdemo and EL0 issued net_tcp_echo (28) quiet guestfwd echo. Not BSD sockets. Not listen/accept. Not TLS. Not TCP CNTPCT. CloudAgent HELD — evidence tagged agent-box. Tip 3704f2a. ADR-076. | | Threat-model v1.51 slice (NFR-10) | Read security.md (ADR-076 EL0 TCP echo SVC; keeps N1–N5 + ADR-075) | Verified | File + this PR. v1.51 update on tip post-#121 v1.50, not “secure.” Not BSD sockets. Not “has networking.” | | scripts/qemu-smoke.sh (ADR-076 EL0 TCP SVC / this branch) | Ran on this agent box | Verified | 2026-09-17 agent-box qemu-smoke on feat/el0-tcp-svc: markers libctos: tcp-ok / tcpdemo: ok + preserved net: tcp-ok / net/samples/FAT; qemu-smoke: ADR-076 tcp-libctos /tcpdemo strings present; force-fail exit 1; qemu-smoke: ok. Not Docker. Not a sockets product. Tip 3704f2a. |

| FAT16 nested mkdir + empty rmdir (ADR-077) | Hello-kernel serial fat: nested + fat: rmdir + keep fat: mkdir / grow / delete / ok / net / samples; #[test_case] fat16_nested_mkdir_and_empty_rmdir + fat16_dir_path_grammar | Verified | 2026-09-17 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/fat-nested-rmdir: hello fat: nested / fat: rmdir + keep fat: mkdir / sample+net markers; Running 139 tests all [ok] incl. nested+rmdir + path-grammar cases; force-fail exit 1 (timeout 180s); qemu-smoke: ok. Guest created /fdir/nest (dirent + ./.. with parent cluster) and empty-rmdir’d it via thin VFS. Not POSIX mkdir/rmdir. Not arbitrary-depth trees. No new SVC. Not TCP CNTPCT. CloudAgent HELD — evidence tagged agent-box. Tip 0025721. ADR-077. | | Threat-model v1.52 slice (NFR-10) | Read security.md (ADR-077 nested mkdir + empty rmdir; keeps N1–N5 + ADR-073/075/076) | Verified | File + this PR. v1.52 update on tip post-#122 v1.51, not “secure.” Not POSIX mkdir/rmdir. Not TCP CNTPCT. | | scripts/qemu-smoke.sh (ADR-077 nested+rmdir / this branch) | Ran on this agent box | Verified | 2026-09-17 agent-box qemu-smoke on feat/fat-nested-rmdir: markers fat: nested / fat: rmdir + preserved fat: mkdir / net/samples; qemu-smoke: FAT16 nested+rmdir (ADR-077) strings present; force-fail exit 1; qemu-smoke: ok. Not Docker. Not POSIX. Tip 0025721. |

| Thin TCP echo path CNTPCT (ADR-078 / P-PERF-10) | Hello serial perf: tcp-echo ticks=<n> (prefix grep; not exact ticks); #[test_case] el0_tcp_echo_cntpct_advances sample > 0; smoke rejects percent marketing; keep N1–N5 / N5.x + ADR-069/072/076/077 | Verified | 2026-09-17 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/tcp-echo-cntpct: hello perf: tcp-echo ticks=1124937 + keep net: tcp-ok / libctos: tcp-ok / tcpdemo: ok / perf: net-ping / perf: udp-dns / fat: nested / fat: rmdir / samples; Running 140 tests all [ok] incl. el0_tcp_echo_cntpct_advances (perf: tcp-echo ticks=1028684 in that case); force-fail exit 1 (timeout 180s); qemu-smoke: ok. QEMU TCG lab — not a published bench, not a percent, not a TCP SLA. Window = quiet ARP+thin TCP guestfwd as SYS_NET_TCP_ECHO / el0_tcp_echo (EL0 surface; kernel N5 untimed). CloudAgent HELD — evidence tagged agent-box. Tip 8f08f88. ADR-078. | | Threat-model v1.53 slice (NFR-10) | Read security.md (ADR-078 TCP echo CNTPCT; keeps N1–N5 + ADR-069/072/076/077) | Verified | File + this PR. v1.53 update on tip post-#123 v1.52, not “secure.” Not a latency SLA. Not “has networking.” | | scripts/qemu-smoke.sh (ADR-078 TCP echo CNTPCT / this branch) | Ran on this agent box | Verified | 2026-09-17 agent-box qemu-smoke on feat/tcp-echo-cntpct: marker perf: tcp-echo ticks=1124937 + preserved net/samples/FAT/ADR-069/072/076/077; qemu-smoke: ADR-078 tcp-echo CNTPCT marker present; Running 140 tests all [ok]; force-fail exit 1 (timeout 180s); qemu-smoke: ok. Not Docker. Not a sockets product. Tip 8f08f88. |

| PAN CPU reopen foundation (ADR-079 / #125) | Default -cpu cortex-a76; hello serial pan: id= + pan: present; smoke rejects pan: enabled / unexpected pan: absent; #[test_case] pan_present_on_probe_cpu | Verified (ID-field on new default) | 2026-09-19 agent-box research + smoke (QEMU 10.0.13): short probe table in ADR-079; hello pan: present on cortex-a76. Enable not Verified (ADR-080). inject-nmi still machine does not provide NMIs (ADR-081). CloudAgent HELD — agent-box. Tip 634ed8d. | | Threat-model v1.54 slice (NFR-10) | Read security.md (ADR-079 PAN CPU reopen; enable still locked) | Verified | File + this PR. v1.54 update on tip post-#124 v1.53, not “secure.” Not PAN enabled. Not “EL0 isolated.” | | scripts/qemu-smoke.sh (ADR-079 PAN CPU reopen / this branch) | Ran on this agent box | Verified | 2026-09-19 agent-box qemu-smoke on isolation/adr-079-pan-cpu-reopen: markers pan: id= / pan: present + preserved net/FAT/samples; qemu-smoke: PAN capability strings present; force-fail exit 1; qemu-smoke: ok. Not Docker. Tip 634ed8d. |

| PAN enable + EL1-vs-EL0 fault (ADR-080 / #125 mile 2) | Hello serial pan: enabled + pan: el1-fault (+ keep pan: id= / pan: present); smoke requires enable markers; #[test_case] pan_enable_el1_fault_on_probe_cpu; syscall uaccess clears PAN around copies | Verified | 2026-09-19 agent-box (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a76): hello pan: id=2 / pan: present / pan: enabled / pan: el1-fault; SPSR.PAN at fault; net/FAT/samples preserved. Umbrella still non-claim. CloudAgent HELD — agent-box. Tip 765a8d0. | | Threat-model v1.55 slice (NFR-10) | Read security.md (ADR-080 PAN enable + EL1-vs-EL0 fault; umbrella still non-claim) | Verified | File + this PR. v1.55 update on tip post-#126 v1.54, not “secure.” Not “EL0 isolated.” | | scripts/qemu-smoke.sh (ADR-080 PAN enable / this branch) | Ran on this agent box | Verified | 2026-09-19 agent-box qemu-smoke on isolation/adr-080-pan-enable-fault: markers pan: enabled / pan: el1-fault + preserved net/FAT/samples; qemu-smoke: PAN capability + enable + el1-fault strings present; force-fail exit 1; qemu-smoke: ok. Not Docker. Tip 765a8d0. |

| Taken lower-EL SError reopen (ADR-081 / #125 mile 3) | Dated QMP/HMP matrix on virt TCG + cortex-a76 (+ GICv3 / virtualization=on / secure=on / max): inject-nmi → machine does not provide NMIs; KVM absent; FEAT_NMI ≠ SError | Deferred / non-goal (locked; reopen unmet) | 2026-09-19 agent-box QEMU 10.0.13. Research brief research/daily-briefs/2026-09-19-adr-081-serror-inject-probe.md. Unlock = ADR-081 B1/B2/B3; B1 research ADR-082 (2026-09-19): no shippable pin yet. Park Verified; A-clear dormant prep. Do not invent Verified. ADR-081. | | Threat-model v1.56 slice (NFR-10) | Read security.md (ADR-081 taken-SError re-probe still blocked; umbrella still non-claim) | Verified | File + this PR. v1.56 update on tip post-#127 v1.55, not “secure.” Not taken SError Verified. Not “EL0 isolated.” | | scripts/qemu-smoke.sh (ADR-081 docs / this branch) | Ran on this agent box | Verified | 2026-09-19 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a76) on isolation/adr-081-taken-serror-reopen: hello el0: serror-arm / qmp inject-nmi => error:machine does not provide NMIs (ADR-045/081 dormant; park if not taken) / el0: serror-park (no bare el0: serror); pan: enabled / pan: el1-fault; net/FAT/samples green; force-fail exit 1; qemu-smoke: ok. Tip 903c7448d508ae0338c64868aac2319941f3c4cb. |

| Taken lower-EL SError B1 QEMU pin research (ADR-082 / #125) | Sponsor chose B1; dated upstream+agent-box research: virt through QEMU master still has no TYPE_NMI→async SError; apt/git pin alone does not unlock; draft local patch research-only | Deferred / non-goal (locked; B1 unmet this session) | 2026-09-19 agent-box QEMU 10.0.13. Probe scripts/qemu-nmi-probe.py + brief research/daily-briefs/2026-09-19-adr-082-b1-qemu-nmi.md. Blocker: wait upstream or sponsor-reviewed pinned QEMU build with proven el0: serror. Park Verified; do not require el0: serror. Not B2/B3. ADR-082. | | Taken lower-EL SError B1 pinned QEMU (ADR-083 / #125) | Pin QEMU v10.0.0 + TYPE_NMI→SError patch; CTOS_QEMU + CTOS_REQUIRE_TAKEN_SERROR=1 smoke shows bare el0: serror; stock QEMU still parks | Verified (pin; CI job unconditional since ADR-090, 2026-09-26) | 2026-09-19 agent-box: build 7c949c53 + patch; probe inject-nmi ok; smoke qemu-smoke: taken 'el0: serror' present + qemu-smoke: ok. Stock 10.0.13: inject error + el0: serror-park. (amd64 workstation) intent <machine-id>; CloudAgent HELD — proof on agent-box. Not umbrella isolated. ADR-083. | | Threat-model v1.58 slice (NFR-10) | Read security.md (ADR-083 pinned QEMU opt-in taken SError; umbrella still non-claim) | Verified | File + this PR. v1.58 on tip post-#129 v1.57. Not “secure.” Not “EL0 isolated.” | | scripts/qemu-smoke.sh (ADR-083 pin / this branch) | Ran on this agent box with pin | Verified | 2026-09-19 agent-box CTOS_QEMU=tools/qemu-nmi/bin/qemu-system-aarch64 CTOS_REQUIRE_TAKEN_SERROR=1: taken el0: serror; force-fail exit 1; qemu-smoke: ok. Default path (no env) still park. | | Threat-model v1.57 slice (NFR-10) | Read security.md (ADR-082 B1 research blocker; umbrella still non-claim) | Verified | File + this PR. v1.57 update on tip post-#128 v1.56, not “secure.” Not taken SError Verified. Not “EL0 isolated.” | | scripts/qemu-smoke.sh (ADR-082 docs / this branch) | Ran on this agent box | Verified | 2026-09-19 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a76) on isolation/adr-082-b1-qemu-nmi-research: expect el0: serror-arm / inject-nmi error / el0: serror-park (no bare el0: serror); pan: enabled / pan: el1-fault; net/FAT/samples green; force-fail exit 1; qemu-smoke: ok. Hello el0: serror-arm / el0: serror-park (no bare el0: serror); pan: enabled / pan: el1-fault; net/FAT green; force-fail exit 1; qemu-smoke: ok. Tip ac61efbdd97a84f20aef67568d772d7e6aaabf44. | | Taken lower-EL SError via B2 on free GitHub-hosted runners (ADR-084; not the B1 pin) | Temp workflow b2-serror-probe on probe/adr-084-b2-kvm: /dev/kvm + -accel kvm on ubuntu-24.04-arm / ubuntu-22.04-arm; -accel hvf on macos-15; VT/WHPX on windows-11-arm; x86 ubuntu-24.04 control; stock TCG inject-nmi matrix + all-machine sweep | Failed | 2026-09-25 CEST GHA runs 36193423604 + 36193613972 (all 5 jobs green on run 2; the probe lost). arm64 Linux: kvm [1]: HYP mode not available, /dev/kvm=absent, Could not access KVM kernel module. macOS M1 (Virtual): HV_UNSUPPORTED. Windows arm64: VMMonitorModeExtensions : False, WHPX Disabled. x86: /dev/kvm present but invalid accelerator kvm for AArch64. TCG everywhere: machine does not provide NMIs; sweep inject-nmi-ok=0 (118/101/117 machines). Free (public repo, standard runners). No self-hosted / paid / larger runner. Raw lines research/daily-briefs/2026-09-25-adr-084-b2-probe.txt. ADR-084. | | P-SEC-3r B2 path (non-B1: different runner/accelerator delivering async SError) | arm64 host exposing KVM + QEMU 0002 KVM_SET_VCPU_EVENTS serror_pending on el0: serror-arm + fail-closed smoke requiring bare el0: serror | Verified (opt-in, one-off paid run) | 2026-09-26: sponsor chose B2-P; ADR-085 run 4 on AWS Graviton3 c7g.metal (eu-north-1b, Spot) PASS. Not a CI gate: no ctos CI job runs KVM, and re-running needs paid metal or a self-hosted arm64 KVM runner (B2-S, not registered). Free-runner probe stays Failed (ADR-084). B1 opt-in pin is not B2. Not “EL0 isolated.” | | Threat-model v1.59 slice (NFR-10) | Read security.md (ADR-084 B2 free-runner spike blocked; umbrella still non-claim) | Verified | File + this PR. v1.59 on tip post-#130 v1.58. Not “secure.” Not taken SError Verified via B2. Not “EL0 isolated.” | | CI on GitHub (ADR-084 docs / this PR) | .github/workflows/smoke.yml + pages.yml on isolation/adr-084-b2-free-runner-serror @ 5c4c38b (PR #136) | Verified | 2026-09-26 CEST PR run 36193979804 success: ubuntu-24.04-arm job 108265450556 (native arm64 runner, TCG) inject-nmi=>error:machine does not provide NMIs + el0: serror-park + qemu-smoke: ok; ubuntu-24.04 job 108265450597 success; opt-in qemu-nmi-pin job 108265450350 (B1, ADR-083) inject-nmi=>ok + bare el0: serror + qemu-smoke: taken 'el0: serror' present. Pages run 36193979996 mdBook build success. The B1 pin job is not B2 evidence. | | Taken lower-EL SError via B2-P: real arm64 + KVM (ADR-085; not the B1 pin) | AWS c7g.metal (Graviton3 Neoverse-V1, CPU: All CPU(s) started at EL2, /dev/kvm=present); QEMU v10.0.0 + 0001 + 0002 -accel kvm -machine virt,gic-version=3 -cpu host; cargo build --features b2-serror; scripts/b2-kvm-smoke.py (fail-closed) via no-SSH user-data; serial console via ec2 get-console-output --latest | Verified (opt-in, one-off paid run) | 2026-09-26 13:26–13:33 CEST run 4 <instance-id-run4> (eu-north-1b Spot, head 159b178): qmp stop=>ok inject-nmi=>ok cont=>ok (ADR-085: KVM_SET_VCPU_EVENTS serror_pending) → el0: standing / el0: serror / el0: restored / b2: taken → b2-kvm-smoke: PASS taken lower-EL SError observed under accel=kvm. Excerpt: research/daily-briefs/2026-09-26-adr-085-b2p-console.txt. Limits: b2 profile only (no GIC/timer/virtio/FAT under KVM); no host ioctl trace; not CI. Not “EL0 isolated.” | | B2-P metal runs 0–3 (ADR-085 bring-up) | Same host class; earlier heads | Failed | Run 0 m6g.metal <instance-id-run0> and run 1 c7g.metal <instance-id-run1>: cloud-init HOME unset, so cargo: command not found (patched QEMU did build with kvm on Graviton2). Run 2 <instance-id-run2>: zero guest UART bytes (3×90 s). Run 3 <instance-id-run3>: reached b2: p1 mmu-on, then PC=ffffff8040080a00 (high VBAR+0x200) PSTATE=a00003c5 EL1h. A pre-MMU-spin-lock control behaved the same, so that hypothesis was refuted. Root cause: TCR_EL1.TG1 reserved. All fail-closed, none claimed. | | TCR_EL1.TG1 = 4 KiB explicit (ADR-085; real-hardware fix) | src/paging.rs TCR_TG1_4K = 0b10 << 30 (was reserved 0b00; TCG sanitised it to 4K) | Verified | Real Graviton3 run 3 stalled on the first TTBR1 fetch; run 4 with the fix printed ident: jump … ident: ram … pages=32150 under KVM. TCG unchanged: GHA run 36238549525 at 159b178 green (ubuntu-24.04-arm + ubuntu-24.04 el0: serror-park + qemu-smoke: ok; qemu-nmi-pin bare el0: serror). | | Threat-model v1.60 slice (NFR-10) | Read security.md (ADR-085 B2-P Verified opt-in on Graviton3 KVM; TG1 fix; umbrella still non-claim) | Verified | File + this PR. v1.60 on the ADR-084 v1.59 branch. Not “secure.” Not “EL0 isolated.” Not “runs on real hardware” beyond the b2 profile. | | AWS B2-P spike resources (ADR-085) | Tag audit project=ctos in eu-north-1 after runs: DescribeInstances (non-terminated) + resourcegroupstaggingapi GetResources + SG/key-pair/IAM check | Verified | See research/daily-briefs/2026-09-26-adr-085-b2p-graviton-kvm.md §Cleanup. Metal ≈43 min Spot, ≈ $0.19 total. | | CI on GitHub (ADR-085 / PR #137) | .github/workflows/smoke.yml on isolation/adr-085-b2p-graviton-kvm-serror @ 159b178 (code tip incl. TG1 fix) | Verified | 2026-09-26 CEST push run 36238549525 success: ubuntu-24.04-arm job 108394690860 and ubuntu-24.04 job 108394690937 inject-nmi=>error:machine does not provide NMIs + el0: serror-park + qemu-smoke: ok; opt-in qemu-nmi-pin job 108394690974 (B1) inject-nmi=>ok + qemu-smoke: taken 'el0: serror' present. The docs-only commit on top is covered by the PR checks. CI never runs KVM, so it is not B2 evidence. | | Umbrella checklist audit (ADR-086) | Every ADR-055 / ADR-060 row vs this-session probes: agent-box qemu-smoke on 915796f (2026-09-26 13:59–14:04 CEST, QEMU 10.0.13, cortex-a76 TCG, qemu-smoke: ok) + GHA PR run 36239619910 (13:41–13:53 CEST, all jobs success) + ADR-085 run 4 | Verified (audit) / umbrella Planned / non-claim | ADR-086: §A 13/13 miles + PAN enable Verified, CI-gated by qemu-smoke.sh. §B not Met: taken SError only on opt-in paths (B1 qemu-nmi-pin needs repo var CTOS_BUILD_QEMU_NMI=1; B2-P one-off, not CI), and the accepted smoke machine still parks. Full identity teardown Unmet and unmeetable as worded (yank _start vs ADR-047 never-yank). No umbrella accept ADR. Sponsor decisions D1–D5. Not “EL0 isolated.” | | Residual TTBR0 identity inventory (ADR-086 scratch probe) | 915796f + scratch walker patch (not committed to src/); walks kernel + user TTBR0 before Hello World! | Documented (scratch; ratchet Planned M1) | 2026-09-26 ~14:10 CEST box run (output): kernel TTBR0 identity = MMIO 1 GiB Device block 0x0–0x4000_0000; low RAM 0x4000_0000–0x4008_0000 RW; stub page 0x4008_0000 RO+X (start-stay); padding tail [.ident_tear end, 0x4020_1000) RO+X; __kernel_end page RW. User TTBR0: stub page + padding tail (EL1-only). Not a CI gate until M1 commits ident: inv. | | Threat-model v1.61 slice (NFR-10) | Read security.md (ADR-086 audit; residual identity inventory; umbrella still non-claim) | Verified | File + this PR. v1.61 on the ADR-085 v1.60 branch. Not “secure.” Not “EL0 isolated.” | | CI on GitHub (ADR-086 docs / PR #138) | .github/workflows/smoke.yml + pages.yml on isolation/adr-086-el0-isolated-audit @ 3f4de00 | Verified | 2026-09-26 14:13–14:26 CEST PR run 36241262508 + push run 36241255248 success: ubuntu-24.04-arm + ubuntu-24.04 default smoke (park + qemu-smoke: ok), opt-in qemu-nmi-pin (B1) pass, Pages mdBook build 36241262505 pass. Docs-only. | | Identity leftovers I2/I4/I5 tear + TTBR0 identity inventory ratchet (ADR-087, M1) | Hello serial ident: low/tail/kend lo=… + ident: low/tail/kend-fault + ident: inv-range … + ident: inv k=2 u=1 a=2 leaks=0 + ident: inv-neg k/u caught + ident: inv-neg clean + ident: inv-ok allow=mmio,stub; smoke rejects ident: inv-leak; inv-leak-probe kernel must be caught; #[test_case] identity_leftovers_torn_stub_stays + identity_inventory_allowlist_only_catches_plant | Verified | 2026-09-26 14:35–14:40 CEST agent-box qemu-smoke: ok (QEMU 10.0.13, cortex-a76 TCG, rustc 0fc141305): low 128 pages, tail 270 (+270 user), kend 1; kernel TTBR0 identity ranges 5 → 2 (MMIO block, _start stub), user 2 → 1 (stub). inv-leak-probe: ident: inv-leak k lo=0x40000000 + no ident: inv-ok → qemu-smoke: inv-leak-probe caught. MMIO + stub still identity-mapped: not full teardown. Not “EL0 isolated.” | | Threat-model v1.62 slice (NFR-10) | Read security.md (ADR-087 residual identity = MMIO block + stub; ratchet) | Verified | File + this PR. v1.62 on the ADR-086 v1.61 branch. Not “secure.” Not “EL0 isolated.” | | CI on GitHub (ADR-087 M1 / PR #139) | .github/workflows/smoke.yml on isolation/adr-087-identity-inventory-ratchet @ 6fcdc4e | Verified | 2026-09-26 14:41–14:52 CEST push run 36242707571 success: ubuntu-24.04-arm job 108406107386 + ubuntu-24.04 job 108406107365 ident: inv k=2 u=1 a=2 leaks=0 / ident: inv-ok allow=mmio,stub / qemu-smoke: inv-leak-probe caught (fail-closed ok) / qemu-smoke: ok; opt-in qemu-nmi-pin job 108406107228 inv-ok + taken el0: serror. PR run 36242713730 default jobs success. | | Device MMIO TTBR1 alias + identity MMIO block tear (ADR-088, M2) | Hello serial ident: mmio-high gic=0xffffff8008000000 uart=0xffffff8009000000 virtio=0xffffff800a000000 + ident: mmio lo=0x0 hi=0x40000000 l1=1 + ident: mmio-fault va=0x9000018 + ident: inv k=1 u=1 a=1 leaks=0 + ident: inv-ok allow=stub; smoke rejects any inv-range k|a lo=0x0; #[test_case] identity_mmio_torn_high_alias_xn; wx: ok via mmio_xn | Verified | 2026-09-26 ~14:43 CEST agent-box qemu-smoke: ok. Only the _start stub page stays identity-mapped (kernel, user, ASID-B; EL1 RO+X). b2 profile TCG-booted only; B2-P KVM evidence predates M1/M2 (re-run ≈ $0.2, not done). Not “EL0 isolated.” | | Threat-model v1.63 slice (NFR-10) | Read security.md (ADR-088 MMIO high alias; identity = stub only) | Verified | File + this PR. Not “secure.” Not “EL0 isolated.” | | ADR-055 §B row 2 reworded: full identity teardown except the _start stub page (ADR-089, M3, sponsor D2) | Default-smoke inventory ident: inv k=1 u=1 a=1 leaks=0 + ident: inv-ok allow=stub + ident: start-stay (ADR-087/088) | Met (this session; not the umbrella) | Sponsor D2 relayed 2026-09-26 ~14:35 CEST. Box smoke ~14:43 CEST on ADR-088; CI on #140 recorded separately. ADR-047 §3 amended in place. Umbrella still needs taken-SError evidence class (M4) + sponsor accept (M5 draft → sponsor). Not “EL0 isolated.” | | Threat-model v1.64 slice (NFR-10) | Read security.md (ADR-089 stub exception) | Verified | File + this PR. Not “secure.” Not “EL0 isolated.” | | CI on GitHub (ADR-088 M2 / PR #140) | .github/workflows/smoke.yml on isolation/adr-088-mmio-high-alias @ 09076d4 | Verified | 2026-09-26 14:56–15:10 CEST push run 36243493356 + PR run 36243501093 all success. ubuntu-24.04-arm job 108408303774: ident: mmio-high gic=0xffffff8008000000 uart=0xffffff8009000000 virtio=0xffffff800a000000 / ident: mmio-fault va=0x9000018 / ident: inv k=1 u=1 a=1 leaks=0 / ident: inv-ok allow=stub / qemu-smoke: inv-leak-probe caught (fail-closed ok) / qemu-smoke: ok. Pin job 108408283187: inject-nmi=>ok → bare el0: serror + inv-ok allow=stub. Not “EL0 isolated.” | | CI on GitHub (ADR-089 M3 docs / PR #141) | smoke.yml + pages.yml on docs/adr-089-stub-exception @ 8093721 | Verified | 2026-09-26 14:56–15:10 CEST push run 36243529890 all success (both default jobs + pin job 108408379969); mdBook build success. Not “EL0 isolated.” | | ADR-055 §B row 1 reworded: taken lower-EL SError on the D1 evidence class (ADR-090, M4, sponsor D1) | B1 qemu-nmi-pin (pinned QEMU v10.0.0 + patch 0001, CTOS_REQUIRE_TAKEN_SERROR=1, bare el0: serror fail-closed), now unconditional on every push/PR, + B2-P ADR-085 run 4 (Graviton3 KVM, el0: serror + b2: taken, one-off at 159b178) | Met (D1 evidence class; not the umbrella) | Sponsor D1 relayed 2026-09-26 ~14:35 CEST. Workflow if: vars.CTOS_BUILD_QEMU_NMI removed (a skipped job would pass a D5 required check); job renamed “QEMU NMI pin smoke (taken SError)”. Stock QEMU still parks (el0: serror-park). B2-P predates M1/M2 (KVM re-verify ≈ $0.2, not done). First unconditional run: CI on this PR. Umbrella still needs the sponsor’s written accept (B3). Not “EL0 isolated.” | | Threat-model v1.65 slice (NFR-10) | Read security.md (ADR-090 taken-SError evidence class) | Verified | File + this PR. Not “secure.” Not “EL0 isolated.” | | Umbrella accept draft (ADR-091, M5, sponsor D4) — ADR-055 §B row 3 | Proposed umbrella sentence + scope S1–S7 + non-scope + gaps G1–G3 + M6 prerequisites, quoting ADR-090 §1 verbatim | Documented (Draft); row 3 not Met; umbrella Planned / non-claim | Drafted 2026-09-26. B3 needs the sponsor’s written accept after reading this draft; merging the draft is not acceptance. Found while drafting: no exhaustive EL0-permission walk (G1) and no EL0 write probe (G2); B2-P re-verify optional (G3, ≈ $0.2). Not “EL0 isolated.” → Accepted 2026-09-28 (M6 row below). | | Threat-model v1.66 slice (NFR-10) | Read security.md (ADR-091 draft scope / non-scope) | Verified | File + this PR. Not “secure.” Not “EL0 isolated.” | | CI on GitHub (ADR-090 M4 / PR #142) | smoke.yml on isolation/adr-090-serror-accepted-machine @ be829fe | Verified | 2026-09-26 15:12–15:28 CEST push run 36244384336 + PR run 36244391658 all success. First unconditional pin job 108410713515 (“QEMU NMI pin smoke (taken SError)”, 15:12–15:26): inject-nmi=>ok → bare el0: serror + ident: inv-ok allow=stub + qemu-smoke: ok. Default jobs 108410713544 (arm) / 108410713462 (x86). Not “EL0 isolated.” | | CI on GitHub (ADR-091 M5 draft / PR #143) | smoke.yml + pages.yml on docs/adr-091-el0-isolated-accept-draft @ b6f324a | Verified | 2026-09-26 15:13–15:28 CEST PR run 36244433643 (arm 108410849837, x86 108410849682, pin 108410849707) + push run 36244425884 all success; mdBook build (run 36244433661) success. Not “EL0 isolated.” | | EL0-reachability walk, fail-closed (ADR-092 G1) | Every valid leaf of kernel / user / ASID-B TTBR0 + TTBR1; EL0-reachable = AP[1] or UXN clear; allowlist = five 4 KiB user slots; pool frame outside heap; not Device; not EL0 W+X; nothing in TTBR1. Live (hello-libctos standing) + steady + 4 plants + reach-leak-probe build | Verified | Agent box 2026-09-27 00:20–00:23 CEST qemu-smoke: ok: el0-reach: live k=3 u=3 a=0 h=0 pages=6 leaks=0, el0-reach: steady k=0 u=0 a=0 h=0 pages=0 leaks=0, el0-reach: neg kernel-pa/va/ttbr1/el0-wx caught, el0-reach: neg clean, el0-reach: ok allow=…; leak build el0-reach: leak k/u lo=0x80007000 … why=kernel-pa → caught. CI: row below. Not “EL0 isolated.” | | EL0 write → permission fault (ADR-092 G2) | EL0 STR to kernel .data via TTBR1 and to an EL0-RO user page: lower-EL data abort, EC 0x24, WnR 1, DFSC perm, FAR = target, ELR = STR, target unchanged; write-leak-probe build must report the completed store | Verified | Agent box 2026-09-27 00:20–00:23 CEST: el0: write-fault kernel va=0xffffff8040201008 esr=0x9200004f far=0xffffff8040201008 ec=0x24 wnr=1 dfsc=perm-l3, el0: write-fault user-ro va=0x80009000 esr=0x9200004f far=0x80009000 …, el0: write-ok kernel,user-ro; leak build el0: write-succeeded user-ro va=0x80009000 esr=0xf2000000 → caught. _start not targeted. Not “EL0 isolated.” | | Syscall copy helpers check EL0 permission (ADR-092 G4) | user_range_ok_max would need EL0 read/write permission on each user leaf | Documented gap (code reading; not probed, not fixed) → closed by ADR-094 (next row) | Today it accepts any valid user-TTBR0 leaf and copy_user clears PAN. So SYS_UART_WRITE could copy the EL1-only _start stub page or an EL1-only window page; copy_to_user could write the app’s execute-only text. ADR-091 revision 2 limits the sentence to direct EL0 access and lists G4 for the sponsor. | | Threat-model v1.67 slice (NFR-10) | Read security.md (ADR-092 EL0 reach walk + write fault; G4) | Verified | File + this PR. Not “secure.” Not “EL0 isolated.” | | CI on GitHub (ADR-092 / PR #144) | smoke.yml + pages.yml on isolation/adr-092-el0-reach-walk @ ab158cb | Verified | 2026-09-27 00:25–00:40 CEST push run 36276236937 + PR run 36276246462 all success; mdBook (run 36276246435) success. ubuntu-24.04-arm job 108499427750, ubuntu-24.04 job 108499427771 and pin job 108499427726 (“QEMU NMI pin smoke (taken SError)”, also bare el0: serror) each show el0: write-fault kernel va=0xffffff8040203008 esr=0x9200004f far=0xffffff8040203008 ec=0x24 wnr=1 dfsc=perm-l3, el0: write-fault user-ro va=0x80009000 …, el0: write-ok kernel,user-ro, el0-reach: live k=3 u=3 a=0 h=0 pages=6 leaks=0, el0-reach: steady k=0 u=0 a=0 h=0 pages=0 leaks=0, el0-reach: neg clean, el0-reach: ok allow=…, qemu-smoke: el0-leak-probe caught (fail-closed ok) (leak build: el0: write-succeeded user-ro …, el0-reach: steady k=1 u=1 a=0 h=0 pages=2 leaks=2), qemu-smoke: ok. Not “EL0 isolated.” | | Syscall pointers need EL0 permission on every page (ADR-094 G4) | user_range_ok_max requires EL0-readable (AP[1]) for copies from EL0 and EL0-writable (AP[1], !AP[2]) for copies to EL0, per page; loader maps app text+rodata EL0 RO+X (map_el0_text); probes: control + kernel-stub / kernel-window read + xo-text / RO write + straddle + 7-syscall sweep, each needing exactly one refusal by the check; sysptr-leak-probe build | Verified | Agent box 2026-09-27 (CEST) local qemu-smoke: ok: el0: sys-ptr control rw-data n=6, el0: sys-ptr denied kernel-stub, … kernel-window, … xo-text, … write-ro, … straddle, el0: sys-ptr denied sys=uart_write nr=17 read kernel-stub … sys=net_mac nr=24 write xo-text (7), el0: sys-ptr sweep syscalls=7 denied=7, el0: sys-ptr ok; leak build el0: sys-ptr leaked kernel-stub n=8, … kernel-window n=8, … xo-text n=6 intact=false checks=0, el0: sys-ptr skip write-ro → caught. G1 allowlist unchanged (5 slots); el0-reach: live k=3 u=3 a=0 h=0 pages=6 leaks=0 unchanged. App ELF sha256 unchanged (A9 cross-update ok). CI: row below. _start only a refused read target. Not “EL0 isolated.” | | Umbrella accept draft revision 3 (ADR-091, after ADR-094) | Sentence drops “direct EL0 memory access only”; adds syscall-pointer clause (S9); G3 the only remaining optional limit | Documented (Draft); row 3 not Met; umbrella Planned / non-claim | Standing order applied (sponsor skipped the G4 question). Merging is not acceptance. Not “EL0 isolated.” → Accepted 2026-09-28 (M6 row below). | | Threat-model v1.68 slice (NFR-10) | Read security.md (ADR-094 syscall pointer EL0 permission) | Verified | File + this PR. Not “secure.” Not “EL0 isolated.” | | CI on GitHub (ADR-094 / PR #146) | smoke.yml + pages.yml on isolation/adr-094-sys-ptr-perm @ 77cae2a | Verified | 2026-09-27 13:25–13:41 CEST PR run 36315704190 (ubuntu-24.04 job 108609879034 13:25–13:35, ubuntu-24.04-arm job 108609879082 13:25–13:36, pin job 108609879083 “QEMU NMI pin smoke (taken SError)” 13:25–13:40) + push run 36315678386 (ubuntu-24.04 108609807133 13:25–13:35, ubuntu-24.04-arm 108609807340 13:25–13:36, pin 108609807322 13:25–13:41) all success; mdBook (run 36315704200, 13:25–13:26) success. Arm job log shows all 15 el0: sys-ptr … lines (control, 5 denied, 7-syscall sweep, sweep syscalls=7 denied=7, ok), qemu-smoke: syscall pointer EL0-permission strings present (ADR-094), qemu-smoke: el0-leak-probe caught (fail-closed ok; reach + write + sys-ptr), qemu-smoke: ok. Not “EL0 isolated.” | | Public-identifier scrub (ADR-093) | git grep for the replaced values on this branch + scripts/check-public-ids.sh | Verified | ADR-093: account ID replaced in 11 files, plus hosted-zone, instance, Spot, SG, IAM-user and local-path identifiers (17 content files). Values remain in git history; cleanup is a separate sponsor decision. Not “secure,” not “compliant.” | | Public-identifier guard is fail-closed (ADR-093) | sh scripts/check-public-ids.sh --self-test (planted synthetic tokens, 7 classes) + scan of unmodified main must FAIL | Verified | Self-test ok; unmodified main FAIL hits=77 (every value line, no extras); this branch hits=0. Wired as first step of qemu-smoke.sh (CI + Docker). Bare 12-digit numbers without context are not flagged (see ADR limits). | | CI on GitHub (ADR-093 / PR #145) | .github/workflows/smoke.yml + pages.yml on security/scrub-public-ids @ d53eb29 | Verified | PR smoke 36310073042 + push smoke 36310067233: ubuntu-24.04 / ubuntu-24.04-arm / NMI pin all success, 2026-09-27 11:39–11:52 CEST; logs show public-ids: self-test ok classes=7 + public-ids: ok … hits=0 before qemu-smoke: ok. Pages build 36310073058 success (deploy skipped on PR). Docs-only follow-up commit adds this row. | | CI on GitHub (ADR-093 / PR #145, after merging main @ 8107578) | smoke.yml + pages.yml on security/scrub-public-ids @ 4d644d6 (merge of main into the branch; conflicts in SUMMARY / ledger / MOC resolved by keeping both sides) | Verified | 2026-09-28 13:30-13:45 CEST PR run 36416030136 (ubuntu-24.04 job 108907277988 13:30-13:40, ubuntu-24.04-arm job 108907278179 13:30-13:42, pin job 108907277537 “QEMU NMI pin smoke (taken SError)” 13:30-13:45) + push run 36416025183 (ubuntu-24.04 108907261648 13:30-13:38, ubuntu-24.04-arm 108907261918 13:30-13:40, pin 108907261888 13:30-13:44) all success; mdBook (run 36416030182, job 108907277345, 13:30) success, deploy skipped on PR. First run with the three D5-required checks on this PR. Pin job log: public-ids: self-test ok classes=7, public-ids: ok files=451 hits=0 (whole merged tree, including ADR-087..094 files from the stack; no new scrubs needed), inject-nmi=>ok, bare el0: serror, qemu-smoke: ok. Docs-only follow-up commit adds this row. | | Taken lower-EL SError via B2-P re-run on current main (ADR-095, G3; not the B1 pin) | Same as ADR-085 run 4: AWS c7g.metal (Graviton3 Neoverse-V1, CPU: All CPU(s) started at EL2, /dev/kvm=present, KVM API 12, host kernel 7.0.0-1013-aws); QEMU v10.0.0 + 0001 + 0002 -accel kvm -machine virt,gic-version=3 -cpu host; cargo build --features b2-serror at main 8107578; scripts/b2-kvm-smoke.py (fail-closed) via no-SSH user-data (in-tree script + small delta in ADR-095); serial console via ec2 get-console-output --latest | Verified (opt-in, one-off paid run) | 2026-09-28 13:50–13:58 CEST (eu-north-1b Spot, all resources tagged project=ctos / component=g3-kvm-recheck, deleted and verified): CTOSB2: head 8107578a6c715a89c9e5861ebb4102460ba20e80 → ident: mmio-high … / ident: low|tail|kend … / ident: mmio lo=0x0 hi=0x40000000 l1=1 / pan: id=2 / pan: enabled / pan: el1-fault → b2: boot → qmp stop=>ok inject-nmi=>ok cont=>ok (ADR-085: KVM_SET_VCPU_EVENTS serror_pending) → el0: standing / el0: serror / el0: restored / b2: taken → b2-kvm-smoke: PASS taken lower-EL SError observed under accel=kvm (attempt 1). Cost ≈ $0.04. Excerpt: research/daily-briefs/2026-09-28-adr-095-g3-console.txt. Closes ADR-091 G3. Limits: b2 profile only (no GIC/timer/virtio/FAT; no inventory / reach / write / sys-ptr probes under KVM); no host ioctl trace; not CI. ADR-091 still Draft, B3 not Met. Not “EL0 isolated.” | | Threat-model v1.69 slice (NFR-10) | Read security.md (ADR-095 B2-P KVM re-run on current main) | Verified | File + this PR. No posture change. Not “secure.” Not “EL0 isolated.” | | CI on GitHub (ADR-095 / PR #147) | smoke.yml + pages.yml on isolation/adr-095-g3-kvm-recheck @ 31cdb1d (ADR-095 commit 5af2fd3 + merge of main e1ee4a8, ADR-093) | Verified | 2026-09-28 14:19–14:37 CEST PR run 36421105682 (ubuntu-24.04 job 108923761266 14:19–14:29, ubuntu-24.04-arm job 108923761452 14:19–14:30, pin job 108923761116 “QEMU NMI pin smoke (taken SError)” 14:19–14:36) + push run 36421101861 (ubuntu-24.04 108923747720 14:19–14:28, ubuntu-24.04-arm 108923747783 14:19–14:30, pin 108923747527 14:19–14:33) all success; mdBook (run 36421105718, job 108923761567, 14:19) success. Arm job: public-ids: self-test ok classes=7 / public-ids: ok files=453 hits=0, el0: serror-park, qemu-smoke: ok. PR pin job: qmp stop=>ok inject-nmi=>ok cont=>ok → bare el0: serror → qemu-smoke: taken 'el0: serror' present (ADR-083 pinned QEMU) → qemu-smoke: ok. The first head 5af2fd3 also passed its push runs (36419823465: jobs 108919578949 / 108919579243 / 108919579461; 36419825002: jobs 108919583154 / 108919583418 / 108919583519; 14:07–14:25 CEST); its PR run did not start because main had moved (ADR-093 merge) and the PR conflicted in the append-only ledger, resolved by merging main. Docs only. Not “EL0 isolated.” | | Umbrella “EL0 isolated” (M6 flip; ADR-091 Accepted; ADR-055 §B row 3) | Sponsor’s written accept (B3) of the revision-3 sentence and scope + ADR-055 §A/§B all Met + D5 branch protection (three required checks) + #139–#147 merged (main 68d4c8f, push CI green) + the M6 PR CI run as the one-session re-probe | Verified (scoped: the accepted ADR-091 sentence only) | B3 verbatim: “I accept the ADR-091 revision-3 sentence and scope as written. After M6, have ctos propose a short costed list of next milestones.” 2026-09-28 16:41 CEST; relay path: sponsor wrote it in the DSO agent chat, DSO relayed it to ctos. The claim is only this sentence: “On the ctos default smoke machine (QEMU virt, -cpu cortex-a76, TCG, one core), EL0 is isolated from the kernel at the architectural page-table / exception-level boundary: each EL0 task runs on its own TTBR0 with its own ASID; EL0 faults when it reads, writes or executes kernel memory; in every translation table (kernel, user and ASID-B TTBR0, and TTBR1) the only EL0-reachable pages are five allowlisted user slots backed by non-kernel frames, checked fail-closed while a user app runs and at rest; PAN is enabled and EL1 faults on EL0 memory; lower-EL IRQ and FIQ are taken while EL0 stands; taken lower-EL SError is evidenced by the ADR-090 evidence class; the only identity (VA = PA) mapping left in any TTBR0 is the documented EL1-only _start stub page (ADR-089); and a system call copies from or to an EL0 pointer only if EL0 itself may read (for a copy from EL0) or write (for a copy to EL0) every page in the range. It is not a speculative-execution or side-channel claim, not a real-hardware claim, and not a certification.” Limits: one core, the TCG default smoke machine; not a speculative-execution or side-channel claim, not a real-hardware claim, not a certification; EL0 can read its own code bytes; no multi-core race story. ADR-091 non-scope stands. Earlier rows that say “Not ‘EL0 isolated’” record their own PR’s scope at the time. ADR-091. Note 2026-10-02: this row records revision 3 as accepted at M6. The current claim is the revision-4 sentence (revision-4 row below); only the PAN clause and the code-bytes limit differ. | | Threat-model v1.70 slice (NFR-10) | Read security.md (M6: umbrella “EL0 isolated” Verified only as the ADR-091 sentence) | Verified | File + this PR. Not “secure.” Not wider than the ADR-091 sentence. | | CI on GitHub (M6 / PR #148, one-session re-probe of the ADR-091 sentence) | smoke.yml + pages.yml on isolation/m6-el0-isolated-flip @ 6956d31 (the M6 flip commit on main 68d4c8f) | Verified | 2026-09-28 16:59–17:16 CEST PR run 36440169580 (ubuntu-24.04 job 108988297915 16:59–17:07, ubuntu-24.04-arm job 108988297602 16:59–17:11, pin job 108988298105 “QEMU NMI pin smoke (taken SError)” 16:59–17:16) + push run 36440144452 (ubuntu-24.04 108988211209 16:59–17:10, ubuntu-24.04-arm 108988211425 16:59–17:10, pin 108988211602 16:59–17:16) all success; mdBook (run 36440169616, job 108988298049, 16:59) success, deploy skipped on PR. Every smoke job: ident: inv k=1 u=1 a=1 leaks=0 / ident: inv-ok allow=stub, el0-reach: live k=3 u=3 a=0 h=0 pages=6 leaks=0 / el0-reach: ok allow=…, el0: write-fault kernel … / el0: write-fault user-ro … / el0: write-ok kernel,user-ro, pan: enabled / pan: el1-fault, el0: sys-ptr sweep syscalls=7 denied=7 / el0: sys-ptr ok, qemu-smoke: ok. Default jobs: el0-reach: neg clean, ident: inv-neg clean, el0: serror-park, public-ids: self-test ok classes=7 / public-ids: ok files=454 hits=0. Pin jobs: el0: serror-arm, inject-nmi=>ok, bare el0: serror. Docs only; this row is a follow-up docs-only commit. | | Task-to-task EL0 isolation probe (ADR-096) | Serial t2t: walk b-frame u-refs=0 u-el0=0 a-el0=1 / t2t: b-own ok asid=2 … / t2t: a-read fault asid=1 va=0x80007000 … got=0x0 after-b no-tlbi / t2t: a-write fault … b-intact=true / t2t: same-va … ×4 / t2t: ok asid-a=1 asid-b=2 va=0x80007000 read,write,same-va; #[test_case] el0_task_cannot_read_or_write_other_task; t2t-leak-probe build must report t2t: leak walk / t2t: leak read … got=0x7a5cb0b0 / t2t: leak write and withhold t2t: ok | Verified (scoped) | Task B (ASID-B TTBR0, ASID 2) writes sentinel 0x7a5cb0b0 to its private page at 0x80007000. Task A (user TTBR0, ASID 1) then loads and stores the same VA with no TLBI of it in between. Both take a lower-EL translation fault (FAR = the VA), A never sees the sentinel, and B’s frame is unchanged. A’s root has no leaf for B’s frame. With A’s own frame at the same VA, each task reads and writes only its own frame. Fail-closed in scripts/qemu-smoke.sh: every marker is required, t2t: leak / t2t: bad / t2t: probe missed are rejected, and the sentinel on any task-A line is rejected. A local negative test of the check block (missing t2t: ok, sentinel on an A line, got= changed on the read line, probe missed, the leak-build serial) failed each time. Limits: one core on the TCG default smoke machine. QEMU TCG flushes its whole software TLB when a TTBR write changes the ASID, so this is not a hardware stale-TLB/ASID-tagging test. It uses two kernel-built tasks, not two loaded apps under a scheduler. It is not a side-channel or speculative claim. Additional evidence only: not part of the ADR-091 sentence. Task-vs-task stays in ADR-091’s non-scope, and a revision 4 needs the sponsor in writing. | | Threat-model v1.71 slice (NFR-10) | Read security.md (ADR-096 task-to-task probe section + mitigation row) | Verified | File + this PR. Not “secure.” The ADR-091 sentence is unchanged. | | CI on GitHub (ADR-096 / PR #149, task-to-task EL0 isolation probe) | smoke.yml + pages.yml on isolation/t2t-probe @ 7439607 | Verified | PR run 36476725257: ubuntu-24.04 job 109112172166 (22:05–22:16 CEST), ubuntu-24.04-arm job 109112171989 (22:05–22:16), pin job 109112172193 “QEMU NMI pin smoke (taken SError)” (22:05–22:22). Push run 36476682111: ubuntu-24.04 109112035715 (22:05–22:15), ubuntu-24.04-arm 109112035659 (22:05–22:16), pin 109112035424 (22:05–22:21). All success. mdBook (run 36476725338, job 109112172352, 22:05) success; deploy skipped on PR. Every smoke job printed all nine t2t: lines at boot and again in cargo test (el0_task_cannot_read_or_write_other_task), then qemu-smoke: task-to-task EL0 isolation strings present (ADR-096). The leak build printed t2t: leak walk b-frame u-refs=1 u-el0=1, t2t: leak read asid=1 va=0x80007000 got=0x7a5cb0b0, t2t: leak write … and t2t: probe missed, then qemu-smoke: el0-leak-probe caught (fail-closed ok; reach + write + sys-ptr + t2t). Every job also printed public-ids: ok … hits=0 and qemu-smoke: ok. Local: the full smoke passed in QEMU TCG on a Linux box. A Docker Desktop smoke on the Windows workstation printed all nine t2t: lines, but stopped earlier at net probe missed (UDP step); main 9a1c2bb fails the same way there, so it is not caused by this change. Docs only; this row is a follow-up docs-only commit. | | CI on GitHub (ADR-096 / PR #149 refreshed onto main 3c47c0c, i.e. after ADR-098 / #151) | smoke.yml + pages.yml on isolation/t2t-probe @ e6e7574 (plain merge of main into the branch; no rebase, no force-push) | Verified | PR run 37000471274: ubuntu-24.04 job 110816770656 (13:20-13:30 CEST), ubuntu-24.04-arm job 110816771290 (13:20-13:31), pin job 110816770893 “QEMU NMI pin smoke (taken SError)” (13:20-13:37). Push run 37000465958: ubuntu-24.04 110816754112 (13:20-13:31), ubuntu-24.04-arm 110816754218 (13:20-13:31), pin 110816754276 (13:20-13:37). All success. mdBook (run 37000471286, job 110816771395) success; deploy skipped on PR. The merged smoke script requires both the nine t2t: lines and the ADR-098 pan: live-syscall / pan: after-el0 / pan: ok held lines. The combined el0-leak-probe build enables t2t-leak-probe and pan-keep-leak-probe together; the t2t: leak ... lines and the pan: lost ... lines are each required individually and t2t: ok / pan: ok held are rejected there. Local: full ./scripts/qemu-smoke.sh on a Linux box (QEMU 10.0.13 TCG) printed qemu-smoke: ok, public-ids: ok files=459 hits=0, all nine t2t: lines, the three pan: lines and both leak families caught. Tampered-log check: deleting the t2t: ok line, and separately the pan: ok held line, each made the checker fail. Docs only; this row is a follow-up docs-only commit. | | Correction: ADR-091 clause “PAN is enabled” was not true while the kernel handled EL0 exceptions (ADR-098) | Pre-fix behaviour reproduced by the pan-keep-leak-probe build: pan: enabled + pan: el1-fault (boot) and pan: lost live-syscall pstate-pan=0 sctlr-span=1 + pan: lost after-el0 trips=… mrs-pan=0 … fault=no spsr-pan=0 | Verified (defect found and fixed) | From the ADR-080 enable (2026-09-19), and so also from the M6 accept (2026-09-28 16:41 CEST) until this fix, the “PAN is enabled” clause held at boot but not while handling EL0 exceptions: SCTLR_EL1.SPAN was 1, so every exception from EL0 entered EL1 with PAN clear, and return_from_el0 resumed the kernel with SPSR 0x3C4 (PAN clear), so PAN stayed clear after EL0 trips until the next with_user_access. The boot probe missed this because it ran before any EL0 trip. The M6 umbrella row stays Verified only with this correction; the sentence is unchanged and true again with ADR-098. | | PAN held during EL0 exception handling + after EL0 trips (ADR-098) | Serial pan: live-syscall pstate-pan=1 sctlr-span=0 (in the loaded app’s SYS_YIELD handler) / pan: after-el0 trips=<n≥1> mrs-pan=1 el1-load va=0x80003000 fault=yes spsr-pan=1 / pan: ok held boot,live-syscall,after-el0; #[test_case] pan_span_cleared_after_enable; pan-keep-leak-probe build must report pan: lost … and withhold pan: ok held | Verified (scoped) | Fix: SCTLR_EL1.SPAN=0 after the ADR-080 enable; SPSR.PAN set on return_from_el0. MRS PAN calibrated both ways (1 with a real EL1 fault, 0 in the negative build without one). Tampered-log tests: 7/7 edited logs fail, unedited passes. One core, TCG; one exception type and one post-trip point sampled. | | Threat-model v1.72 slice (NFR-10) | Read security.md (ADR-098 PAN correction section + mitigation row) | Verified | File + this PR. v1.71 is ADR-096 (PR #149), numbered before this slice and merged after it. Not “secure.” ADR-091 sentence unchanged. | | EL0 execute-only app text (ADR-097; merge requires sponsor rev 4 written accept) | Serial xo: live app=hello text va=0x80002000 ap=2 uxn=0 pxn=1 el0=x reach=1 hdr … rodata=0x80000120 in-text=0 sys-read text=deny rodata=allow / xo: layout loader app-hdr … app-text va=0x80002000 ap=2 uxn=0 pxn=1 el0=x reach=uxn / xo: el0-exec ok … got=0x584f524441544131 / xo: el0-read-text fault va=0x80002000 esr=0x9200000f far=0x80002000 ec=0x24 wnr=0 dfsc=perm-l3 got=0x0 / xo: el0-uart rodata-str ok / xo: sys-read rodata ok … / xo: sys-read text denied sys=uart_write nr=17 va=0x80002000 n=0 checks=1 / xo: el1-pan … ×2 / xo: el1-write text … fault=yes intact=true / xo: reach xo-plant caught … ap-el0=0 uxn=0 / xo: ok text=el0-xo rodata=app-hdr el0-read=fault sys-read=deny slots=5; #[test_case] el0_app_text_is_execute_only; host tests loader_plans_hello_as_ro_hdr_and_xo_text / loader_rejects_ro_on_text_page / loader_rejects_pages_outside_app_slots; xo-leak-probe build must report xo: leak … and withhold xo: ok | Verified (scoped; branch only until merged; update 2026-10-02: accepted by the sponsor as ADR-091 rev 4 and merged as 8344797) | App .rodata moved onto the app-hdr page (EL0 RO + NX); .text alone on app-text, mapped EL1 RO / EL0 none / UXN=0. Still five slots; allowlist and el0-reach: lines byte-identical. EL0 loads of its text fault, text pointers are refused by the ADR-094 check. EPAN finding: ID_AA64MMFR1_EL1.PAN=2 (no FEAT_EPAN) on QEMU 10.0.13 cortex-a76 (box and CI); with PAN set an EL1 load of the XO page does not fault (fault=no match=true). So the ADR-091 clause “PAN is enabled and EL1 faults on EL0 memory” is not literally true for that page: merge needs a sponsor rev 4 written accept. ADR-091 not edited. Tampered-log tests: 10/10 edited logs fail, unedited passes (ADR-097 §Negative tests). One core, TCG; not real hardware. | | Threat-model v1.73 slice (NFR-10) | Read security.md (ADR-097 execute-only app text section + mitigation row) | Verified | File + this PR (was v1.72 before the ADR-098 PAN fix was split out and took v1.72). Not “secure.” ADR-091 sentence unchanged; merge requires sponsor rev 4 written accept. Update 2026-10-02: accepted and merged as 8344797; threat model is v1.74 (row below). | | CI on GitHub (ADR-097 / PR #150, EL0 execute-only app text; stacked on #149; merge requires sponsor rev 4 written accept) | smoke.yml + pages.yml on isolation/xo-text @ e1135b6 | Verified | PR run 36485127669: ubuntu-24.04 job 109140067947 (23:17–23:29 CEST), ubuntu-24.04-arm job 109140068167 (23:17–23:29), pin job 109140068292 “QEMU NMI pin smoke (taken SError)” (23:17–23:35). Push run 36485081546: ubuntu-24.04 109139912849 (23:17–23:27), ubuntu-24.04-arm 109139913176 (23:17–23:28), pin 109139913104 (23:17–23:32). All success. mdBook (run 36485127464, job 109140067363, 23:17) success; deploy skipped on PR. Every smoke job: public-ids: ok files=460 hits=0; all thirteen xo: lines at boot (incl. xo: el0-read-text fault va=0x80002000 esr=0x9200000f … dfsc=perm-l3 got=0x0, xo: el1-pan text … fault=no match=true mmfr1-pan=2 epan=no, xo: pan live-syscall pstate-pan=1 sctlr-span=0, xo: ok … slots=5) and again in cargo test (152 tests, el0_app_text_is_execute_only); qemu-smoke: EL0 execute-only text strings present (ADR-097); the leak build printed xo: leak live text ap=3 el0=rx …, xo: leak el0-read-text va=0x80002000 got=0xf9400001d2800001, xo: leak sys-read text n=8 checks=0, xo: bad pan live-syscall pstate-pan=0 sctlr-span=1, xo: probe missed and was caught (… + t2t + xo)); qemu-smoke: ok. | | CI on GitHub (ADR-097 / PR #150 refreshed onto main 3c47c0c via the refreshed #149 branch; merge requires sponsor rev 4 written accept) | smoke.yml + pages.yml on isolation/xo-text @ c63d712 (plain merge of isolation/t2t-probe @ e6e7574; no rebase, no force-push) | Verified | PR run 37001810303: ubuntu-24.04 job 110820981036 (13:35-13:45 CEST), ubuntu-24.04-arm job 110820980926 (13:35-13:45), pin job 110820980975 “QEMU NMI pin smoke (taken SError)” (13:35-13:48). Push run 37001811973: ubuntu-24.04 110820986072 (13:35-13:44), ubuntu-24.04-arm 110820985824 (13:35-13:45), pin 110820986091 (13:35-13:51). All success. mdBook (run 37001810402, job 110820981516) success; deploy skipped on PR. The merged smoke script requires the t2t:, xo: and ADR-098 pan: lines; the combined el0-leak-probe build enables t2t-leak-probe, xo-leak-probe and pan-keep-leak-probe together, and each family’s leak lines are required individually (t2t: ok, xo: ok and pan: ok held are rejected there). Local: full ./scripts/qemu-smoke.sh on a Linux box (QEMU 10.0.13 TCG, 153 tests) printed qemu-smoke: ok, public-ids: ok files=462 hits=0, all t2t:, xo: (incl. xo: pan live-syscall pstate-pan=1 sctlr-span=0) and pan: lines, and all three leak families caught (. + t2t + xo + pan)). Tampered-log check: deleting t2t: ok, xo: ok, xo: pan live-syscall . and pan: ok held (one at a time) each made the checker fail. Docs only; this row is a follow-up docs-only commit. | | Umbrella “EL0 isolated” revision 4 (ADR-091 rev 4 accepted; ADR-097 Accepted; docs only) | Sponsor’s written accept, verbatim “I accept ADR-091 rev 4” (2026-10-02; written in the DSO chat, recorded in the DSO vault, relayed by DSO; answer to the exact wording in ADR-097) + PR #150 (ADR-097, XO app text) merged as 8344797 + PR #151 (ADR-098, PAN held in EL0 exception handling) on main as 3c47c0c. No code change in this PR; the xo: and pan: probes were already CI-gated by those PRs | Verified (scoped: the revision-4 ADR-091 sentence only) | Only the PAN clause and the code-bytes limit changed from revision 3. The claim is only this sentence: “On the ctos default smoke machine (QEMU virt, -cpu cortex-a76, TCG, one core), EL0 is isolated from the kernel at the architectural page-table / exception-level boundary: each EL0 task runs on its own TTBR0 with its own ASID; EL0 faults when it reads, writes or executes kernel memory; in every translation table (kernel, user and ASID-B TTBR0, and TTBR1) the only EL0-reachable pages are five allowlisted user slots backed by non-kernel frames, checked fail-closed while a user app runs and at rest; PAN is enabled, including while the kernel handles exceptions from EL0, and EL1 faults on EL0-readable memory (the execute-only app text page is not covered: this CPU has no FEAT_EPAN); lower-EL IRQ and FIQ are taken while EL0 stands; taken lower-EL SError is evidenced by the ADR-090 evidence class; the only identity (VA = PA) mapping left in any TTBR0 is the documented EL1-only _start stub page (ADR-089); and a system call copies from or to an EL0 pointer only if EL0 itself may read (for a copy from EL0) or write (for a copy to EL0) every page in the range. It is not a speculative-execution or side-channel claim, not a real-hardware claim, and not a certification.” Limits: one core, the TCG default smoke machine; not a speculative-execution or side-channel claim, not a real-hardware claim, not a certification; app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN); no multi-core race story. ADR-091 non-scope stands. The status rests on the existing probes (pan: live-syscall …, pan: ok held …, xo: ok …, xo: el1-pan text … fault=no … epan=no), not on this PR’s text; this PR’s CI is the re-probe. ADR-091, ADR-097. | | Threat-model v1.74 slice (NFR-10) | Read security.md (ADR-091 revision 4: PAN clause + code-bytes limit; v1.73 section marked accepted) | Verified | File + this PR. Docs only. Not “secure.” Not wider than the revision-4 sentence. | | KVM re-check in CI via GitHub OIDC (ADR-099, milestone item 2) | ADR-099: trust and permission policy shape (placeholders), results-out decision (console output, no PassRole), teardown and cost cap, Spot capacity story, risk table; inert .github/workflows/kvm-recheck.yml + scripts/kvm-recheck-userdata.sh | Planned | Documented design sketch, not implemented. The workflow has never run: dispatch-only, not scheduled, not a required check, fails closed while KVM_ROLE_ARN is unset. No AWS call was made. CloudTrail, the OIDC provider, the Spot service-linked role and the exact immutable-subject sub format are not verified; the DSO vault decision record was not read. Nothing about EL0 isolation changes. | | ADR-099 files present (docs only) | ADR-099, .github/workflows/kvm-recheck.yml, scripts/kvm-recheck-userdata.sh in this PR; YAML parse, actionlint + shellcheck, bash -n, guard run with simulated variables (no AWS) | Verified | File presence and local lint only. Says nothing about AWS behaviour. Not “secure.” | | ADR-099 pilot spec (2026-10-13, sponsor decision 20:49 CEST via DSO) | ADR-099 “Pilot spec” | One supervised, manual-dispatch run only; environment kvm-pilot; component tag kvm-ci-pilot; region eu-north-1; actions SHA-pinned; exact immutable sub with numeric IDs supplied to DSO out of band. Spot price is not IAM-enforceable (no condition key); capped by MaxPrice guard and budget. No AWS call by the agent; environment and role not created. CloudTrail (P1) met: DSO verified a multi-region trail, home region eu-north-1, from the arm64 workstation on 2026-10-13 about 20:56 CEST. | | SMP race-story plan + global-state inventory (ADR-100) | rg of static items under src/ on main af5a9d1 + reading the owner modules | Documented (code reading; not a probe of race freedom) | 262 statics: 25 static mut, 10 spin-mutex, 208 atomics, 19 immutable. Riskiest: local-only TLBI, page-table static mut under a convention lock, fixed ASIDs, global EL0/expect flags, per-core PAN/SPAN/VBAR. Nothing here is “SMP-safe”. | | Secondary CPU bring-up smoke, opt-in (ADR-100 S1, --features smp2-probe, QEMU -smp 2) | Serial smp: cpu0 mpidr=0x0000000080000000 / smp: secondary up mpidr=0x0000000080000001 / smp: cpu_on target=0x1 rc=0 / smp: ok cpu1 mpidr=0x0000000080000001 parked; negative: same ELF under -smp 1 → rc=-2, smp: cpu1 missed seen=0x0, checker fails; default run: no smp: line | Verified (local Linux box, QEMU 10.0.13 TCG, 2026-10-13; CI row below) | Secondary runs MMU off, prints, stores MPIDR to one flag word, parks in wfe. No EL0, no page tables, no heap/scheduler/drivers on cpu1. Not SMP support. ADR-091 rev 4 stays one core. | | Threat-model v1.75 slice (NFR-10) | Read security.md (ADR-100 SMP plan; one-core non-goal; S1 mitigation row) | Verified | File + this PR. No posture change. Not “secure.” ADR-091 sentence unchanged. | | CI on GitHub (ADR-100 / PR #154, SMP race story S1) | smoke.yml + pages.yml on smp/adr-100-s1 @ 6d0c324 | Verified | 2026-10-13 ~21:07-21:25 CEST PR run 38452394317 + push run 38452386420: both QEMU aarch64 smoke jobs (ubuntu-24.04-arm, ubuntu-24.04) green with qemu-smoke: default one-core (no smp: markers, ADR-100); QEMU NMI pin smoke (taken SError) green; non-required QEMU -smp 2 bring-up green with smp: secondary up mpidr=0x0000000080000001 / smp: ok cpu1 ... parked and the -smp 1 negative caught (rc=-2, smp: cpu1 missed seen=0x0); mdBook build green. Not SMP support; ADR-091 rev 4 stays one core. | | Host / user name scrub + guard classes (ADR-101) | ADR-101; sh scripts/check-public-ids.sh --self-test → public-ids: self-test ok classes=15; scan → public-ids: ok … hits=0; same guard on unmodified main af5a9d1 → FAIL hits=151 | Verified (scoped: current tree only) | Forward-only, no history rewrite (sponsor 2026-10-13). Machine names, agent VM / EC2 private / GitHub runner host names and a machine ID replaced by role names or placeholders (counts by class in ADR-101, no values). Two scripts no longer print a host name. Not DLP; history, PR bodies and CI logs not scanned. | | Repo variable CTOS_BUILD_QEMU_NMI deleted (sponsor 2026-10-13) | gh variable list before: one variable; workflows on main and on #152’s branch grep clean (no vars.CTOS_BUILD_QEMU_NMI since ADR-090 made the pin job unconditional); gh variable delete; gh variable list after: empty (2026-10-13 20:51 CEST) | Verified | GitHub setting, $0. The Dockerfile --build-arg CTOS_BUILD_QEMU_NMI is a separate build argument and is unchanged. | | ADR-099 pilot setup recorded (DSO, 2026-10-13 22:12-22:16 CEST, on the sponsor’s 22:11 approval) | DSO relay to the agent; text only, no identifier values | Documented (relayed; not probed by this agent, no AWS call) | kvm-pilot environment + OIDC role created by DSO (role tagged project=ctos / component=kvm-ci-pilot, inline policy per ADR-099 incl. hop-limit fix); OIDC provider pre-existing and shared (another project’s Terraform); AMI Ubuntu 24.04 arm64 dated 20261004 (differs from ADR-095); SG no ingress; env: sole reviewer the sponsor, prevent_self_review off, main only, admin bypass off (sponsor 22:15); budget ctos-monthly-5 USD 5/month on project=ctos, alerts 80% actual / 100% forecast. Repo variables still unset; workflow inert. | | ADR-099 review fixes (Bugbot on PR #152 + pilot findings, 2026-10-13) | Code/text reading of .github/workflows/kvm-recheck.yml + ADR-099; teardown logic exercised with a mocked aws (clean / slow / leak) | Documented (not run on AWS by the agent) | (1) snapshot resource kept out: AWS docs list it for RunInstances, but the live pilot launched without it (public AMI); add only on a concrete denial; (2) sweeper/LEAK counts only this run’s instances, so REFUSED-active-instance is not overwritten; (3) new ABORTED-after-aws-call, NOT-RUN = no AWS call; (4) teardown polls up to 20 min, shutting-down at the end = TEARDOWN-SLOW (red, not LEAK); (5) KVM_SUBNET_IDS accepts spaces and/or commas. | | KVM pilot smoke under accel=kvm (ADR-099 pilot, relayed by DSO) | Workflow run 38468490030 at main 069fe83, c7g.metal; console markers el0: serror + b2: taken; nonce 685780b675853a95 | Verified (relayed by DSO; smoke PASS) | Run verdict red only from a false LEAK (5-min teardown wait timed out while the metal instance was shutting down); DSO verified the instance terminated. RunInstances worked without a snapshot resource. Reduced b2-serror profile only; not the default image. | | SMP slice S2 opt-in probe (ADR-102) | ADR-102; CTOS_SMP2_ONLY=1 ./scripts/qemu-smoke.sh on the Linux box: -smp 2 → smp: ok cpu1 s2 mmu-on pan=1 span=0 vbar=own exc=handled console=locked; -smp 1, smp2-neg-pan, smp2-neg-span, smp2-neg-vbar each fail the checker with their miss line | Verified (QEMU TCG, -cpu cortex-a76, -smp 2, opt-in build only) | Core 1 MMU on with shared kernel tables, own stack and vectors, PAN=1 / SPAN=0, one brk handled on core 1. No SMP safety claim; core 1 runs no EL0, scheduler, heap, page-table code, drivers or interrupts. ADR-091 rev 4 (one core) unchanged. Console lock: positive only (no deterministic interleave negative). | | SMP slice S3 (ADR-103): type-enforced TABLES, broadcast TLBI, cross-core unmap probe | ADR-103; page-table writers need &mut TablesGuard (compile-time); TLBI helpers use dsb ishst; tlbi …is; dsb ish; isb; CTOS_SMP2_SMOKE=1 ./scripts/qemu-smoke.sh on the Linux box: -smp 2 → smp: xtlb unmap cpu0 → cpu1 fault; local-tlbi-leak-probe → smp: xtlb leak (checker fails) | Verified (QEMU TCG, -cpu cortex-a76, -smp 2; leak 27/27 and fault 27/27 runs on the box) | One page, one direction (core 0 unmaps, core 1 faults). Reads of tables stay unlocked. No SMP safety claim; core 1 runs no EL0. ADR-091 rev 4 (one core) unchanged. The leak is a QEMU soft-TLB property, not a hardware claim. | | Threat-model v1.76 slice (NFR-10) | Read security.md (new section “KVM re-check via GitHub OIDC (ADR-099)”: ADR-099 risk table folded in, as on main after PR #157) | Verified | File + this PR. Docs only. No posture change. Not “secure.” ADR-091 sentence unchanged. | | SMP slice S4 (ADR-104): PerCpu via TPIDR_EL1, ASID allocator with generations, first EL0 on core 1 | ADR-104; CTOS_SMP2_SMOKE=1 ./scripts/qemu-smoke.sh on the Linux box: -smp 2 → 13 × smp: cpu1 probe <name> ok cpu=1, smp: asidgen alloc cpu0+cpu1 n=300 unique …, smp: asidgen reissue cpu0 rollover → cpu1 fault, smp: ok cpu1 s4 …; S4 leak build (all existing leak features + asid-rollover-local-probe) → every leak reported on cpu1, smp: asidgen leak, checker fails | Verified (QEMU 10.0.13 TCG, -cpu cortex-a76, -smp 2, opt-in build only; positive 20/20, leak 20/20, S3 xtlb leak 20/20) | Probes run on one core at a time (core 0 waits); result flags, user tables and the fatal stack stay shared; allocator not used by the default build. TCG flushes the TLB on any ASID change, so ASID tagging itself is not shown. No multi-core safety claim; ADR-091 rev 4 (one core) unchanged. | | SMP slice S5 (ADR-105): concurrent EL0, run-queue lock, reserved ASID | ADR-105; CTOS_SMP2_SMOKE=1 ./scripts/qemu-smoke.sh on the Linux box: both cores run EL0; mid-run rollover keeps core 1’s ASID; asid-reserve-drop-probe → smp: s5 asid reissued while running cpu=1; cpu1 fatal on own stack; guards unmapped | Verified (QEMU 10.0.13 TCG, -cpu cortex-a76, -smp 2, opt-in; positive 10/10, reserve-drop 10/10) | No preemption; allocator opt-in only; TCG ASID-tag limit as in ADR-104. No multi-core safety claim; ADR-091 rev 4 unchanged. Multi-core sentence is a separate Proposed ADR. | | Proposed multi-core claim sentence (ADR-106, docs only) | ADR-106; docs-only PR; no code; does not edit ADR-091 rev 4 | Proposed (not accepted) | Needs the sponsor’s written acceptance. Scope: QEMU TCG, cortex-a76, -smp 2, opt-in `smp2-probe`, with explicit TCG ASID-tag / no-preemption / no-KVM limits. | | FAT32 read path beyond FAT16 (ADR-107): BPB + FSInfo, FAT32 chains, root as cluster chain, on a second read-only virtio-blk | ADR-107; ./scripts/qemu-smoke.sh: mkfs.fat -F 32 image → fat32: mount clusters=N≥65525 … fsinfo=ok, fat32: read /probe32.txt bytes=9, fat32: chain /big.bin clusters=6 jumps≥1 bytes=3000, fat32: root-chain clusters≥2 last=found, fat32: ok; negative boot with FAT[root] freed → fat32: fail reason=bad-chain, no fat32: ok, FAT16 fat: ok still; cargo test in-memory negatives (free / bad / out-of-range link, cycle, size mismatch, bad FSInfo, FAT16 geometry) | Verified (QEMU virt TCG, this PR’s CI smoke jobs) | Read-only; root directory only; 8.3 names only (LFN skipped); not mounted in the VFS; no write / FSInfo update / FAT mirror check. FAT16 path unchanged. Not hardware. | | Taken-SError QEMU spike (ADR-108, research) | ADR-108; stand-alone image, research/serror-spike/serror-matrix.py 52/52 ok on stock 8.2.2 / 10.2.1 / master and pinned B1 | Shown: virtual SError (HCR_EL2.VSE, EL2-owned guest) taken from EL0 on cpu0/cpu1 on stock QEMU; pinned nmi targets cpu1. Not shown: any of this in the ctos kernel; physical SError on stock QEMU; bad-PA → SError (it is a sync abort). No claim change. | | Inbound TCP over QEMU hostfwd (ADR-109 / N6): one passive open on guest port 8080, line → reply → close, RST on closed port 8081 | ADR-109; ./scripts/qemu-smoke.sh: net: tcp-in-listen port=8080 closed=8081, net: tcp-in-rst port=8081, net: tcp-in-syn, net: tcp-in-est, net: tcp-in-rx bytes=11, net: tcp-in-tx bytes=19, net: tcp-in-fin closed, net: tcp-in-ok; host hostfwd-client: neg=reset pos=ok rx=19 eof=yes; cargo test parse / checksum / RST / fragment unit cases | Verified (QEMU virt TCG, SLIRP on host loopback, this PR’s CI smoke jobs) | Kernel path only; one connection; no sockets API / EL0 SVC / retransmit timer / options / IPv6. Not a real-network or security claim. N5 outbound unchanged. | | N1: taken SError on core 1 at EL0 (ADR-110, opt-in) | ADR-110; scripts/smp2-serror-pin.sh in the qemu-nmi-pin CI job | Shown: pinned QEMU, -smp 2, HMP nmi cpu-index=1 → core 1 takes it at EL0; nmi to cpu0 and A-masked are not taken (fail-closed). Not shown: stock QEMU, hardware, concurrency with S5. No claim change. | | N2: EL2 shim, taken virtual SError at EL0 on stock QEMU (ADR-111, opt-in) | ADR-111; scripts/el2-vserror-smoke.sh, opt-in CI job | Shown: full image, stock QEMU 8.2.2/10.2.1 virt,virtualization=on + shim → virtual SError (HCR_EL2.VSE, EC 0x2f) taken at EL0; not taken without ask, A-masked, or without EL2. Not shown: physical SError, cpu1, hardware. Default boot has no EL2. No claim change. |

Three pillars

After the cooperative scheduler (M9), ctos treats antifragility, security, and performance as first-class pillars (ADR-011). In everyday words: we turn repeated misses into sensors, we prove security slices instead of saying “secure OS,” and we measure before we tune. They share one rule: a status word needs a probe (honesty ledger).

Principles drive; pillars sit under them; tracks sit under both. Visitor-facing list: landing — Driving principles. In-repo list: principles.md. This page is the deep hub, not a second marketing copy. Extra stance: overview.md, apps-today.md, building-or-porting.md, immutability.md.

flowchart TD
  P["Core principles<br/>honesty · antifragility · security<br/>performance · document-first"]
  L["Three pillars — this page<br/>NFR-05 · NFR-10 · NFR-07"]
  T["Tracks A / B<br/>subordinate workstreams"]
  P --> L --> T

Tracks (loader/ABI, later slots/FS) do not outrank a principle.

PillarFrozen IDHomeWhat “done” looks like
AntifragilityNFR-05antifragility.mdRepeated failures become sensors/gates, not extra README advice
SecurityNFR-10security.mdThreat-model v1.29 + probes before any “secure OS” claim. EL0 miles: el0.md (umbrella “EL0 isolated” Verified only as the ADR-091 sentence; standing + TTBR1 first cut + EL1 high-VA fetch + identity .text range tear + live .text tear + identity .rodata/.data/heap tears + PAN ID-field + ASID + SVC ABI + libctos CRT + memfs + FAT16 + A9 slot first cut / cross-update / embed-off are separate rows). App hosting: syscall.md / ADR-048 / ADR-052 (product claim Verified under Met×5; not Linux/POSIX/containers)
PerformanceNFR-07performance.mdMeasurable CNTPCT + IRQ-delta + host ELF size + boot-delta; optimize only with a probe

Bring-up M0–M9 stays on the roadmap. Pillar work after M9 is listed there as a separate section so a docs PR does not pretend to close paging or a scheduler.

Immutability is the same rule: scoped RO+NX / text-tear probes are fine; absolute “immutable OS” is not. Overview: Advantages — Immutability.

Do not import florist / AEA role names. These pillars are ctos-native.

Antifragility SOP

First-class pillar (ADR-011, NFR-05). Hub: pillars.md.

When the same failure happens twice, strengthen the strongest layer (a sensor or a gate), not another paragraph of advice.

Fail closed

  • Unprobed boot → Unknown, not Verified.
  • Missing nightly / rust-src / QEMU → say the environment blocked the probe; do not invent success.
  • MRC / author conflict → do not merge. GitHub author == mergedBy is a process miss after the 2026-09-12 ADR-002 amendment (Track A A1–A9 is a named historical exception, not a going-forward allowance).

Ratchet

  1. Name the failure once in research/random-thoughts/ (what broke, command, error).
  2. If it repeats, add a sensor: a script, a #[test_case], a CI check, or a ledger row with a real probe command.
  3. If people keep skipping the sensor, add a gate: CI required, or MRC refuses the PR.
  4. Only then tighten a guide (AGENTS.md or a skill). Guides without sensors rot.

Recent ratchets (keep Failed history)

These are history rows, not a claim that the current tip is broken.

  • arm64 workstation Docker on b2bbb99 Failed after PR #20 (paging: probe missed / heap: probe missed / sched: probe missed / el0: probe missed while guard/RO still printed). Same class as the PR #20 pre-MMU .bss miss plus a single shared L3 / first-2-MiB user map. Sensor: layout L3 pool + post-MMU frame init (#21). Docker on 71ee15f then Verified. Keep the Failed row in the honesty ledger.
  • Live identity .text yank Failed (unhandled sync on first println! — rustc dyn Write vtables are identity fn pointers). Sensor / cut: ADR-019 kept live .text and tore a 16 KiB dedicated range. ADR-020 then rewrote those vtables to high aliases and tore live .text. The arm64 workstation Docker + GHA on e80dc93 Verified that cut (ident: reloc n=12, live pages=37, 50 tests). Keep the Failed first yank.
  • PR #27 ac3ad0b GHA Failed (ident: range then ident: probe missed — init still required a 4 KiB tear page). Sensor: 16 KiB range + relaxed init. Merge 24d94e6 GHA 34640667227 Verified. Keep the Failed SHA.
  • Track A same-login merge (A1–A9, 2026-09-11–2026-09-12). gh pr view --json author,mergedBy on #49, #52–#57, #59, #60 showed artofdream on both hats (commits typically cursoragent + Co-authored-by: artofdream). Sensor / docs ratchet: named historical exception in ADR-002; MRC must read the GitHub author field, not a brief’s “expected bot.” Gate going forward: owner-opened → cursor[bot] merges; bot-opened → artofdream merges. Same-login merge is a miss. Keep this history row; do not rewrite those PRs.

No self-approval / no self-merge

The author does not APPROVE or merge their own PR. Same GitHub login is not a second identity. Co-authored-by / cursoragent on commits does not change the GitHub author.

Going forward: GitHub shows artofdream as author → cursor[bot] merges (owner must not). GitHub shows cursor[bot] as author → artofdream merges (bot must not). MRC writes COMMENT and names author / reviewer / merger from gh, not from chat. See ADR-002 and .cursor/skills/ctos-mr-coordinator/SKILL.md. Do not enable GitHub author self-APPROVE to make the gate “count.”

Historical (Track A A1–A9 only): same-login merges listed in ADR-002. Named exception; not a license to repeat.

Security — threat model v1.76 (NFR-10 / ADR-011 / ADR-012 / ADR-013 / ADR-014 / ADR-015 / ADR-016 / ADR-017 / ADR-018 / ADR-019 / ADR-020 / ADR-021 / ADR-022 / ADR-023 / ADR-024 / ADR-025 / ADR-026 / ADR-027 / ADR-028 / ADR-030 / ADR-032 / ADR-037 / ADR-038 / ADR-039 / ADR-040 / ADR-041 / ADR-042 / ADR-043 / ADR-044 / ADR-045 / ADR-047 / ADR-048 / ADR-049 / ADR-050 / ADR-052 / ADR-053 / ADR-054 / ADR-055 / ADR-056 / ADR-057 / ADR-058 / ADR-059 / ADR-060 / ADR-061 / ADR-062 / ADR-063 / ADR-064 / ADR-065 / ADR-066 / ADR-067 / ADR-068 / ADR-069 / ADR-070 / ADR-071 / ADR-072 / ADR-073 / ADR-074 / ADR-075 / ADR-076 / ADR-077 / ADR-078 / ADR-079 / ADR-080 / ADR-081 / ADR-082 / ADR-083 / ADR-084 / ADR-085 / ADR-086 / ADR-087 / ADR-088 / ADR-089 / ADR-090 / ADR-091 / ADR-092 / ADR-094 / ADR-095 / ADR-096 / ADR-097 / ADR-098 / ADR-099 / ADR-100)

This is a written threat model for a QEMU virt learning kernel. It is not a certification, not an audit, and not a “secure OS” / “hardened” claim. File presence is not W^X. Image W^X is a separate ledger row that needs a QEMU probe.

Version: v1.76 (2026-10-19). Slice/update of v1.75. Docs only, no code change. Folds the ADR-099 risk and impact table (KVM re-check of the reduced b2-serror profile via GitHub OIDC on a Graviton metal Spot instance) into this threat model as section “KVM re-check via GitHub OIDC (ADR-099)”. Source: ADR-099 on main after PR #157 (merged 2026-10-14), so the table includes the #157 changes (run-scoped sweeper, TEARDOWN-SLOW, ABORTED-after-aws-call, snapshot resource kept out). No posture change for the kernel. Not “secure.” ADR-091 rev 4 sentence unchanged.

Prior v1.75 (2026-10-13). Slice/update of v1.74. ADR-100 (milestone item 5, the SMP race story) writes down the multi-core threat: an inventory of global mutable kernel state (262 statics: 25 static mut, 10 spin-mutex-wrapped, 208 atomics, 19 immutable) and the races each would have on a second core. The riskiest are local-only TLB invalidation (no …IS TLBI anywhere), page-table static mut storage guarded only by a convention lock, fixed ASIDs, global EL0-context / expected-fault flags, and per-core PAN/SPAN/VBAR setup. No posture change: the default kernel is one core and the ADR-091 rev 4 sentence (“-cpu cortex-a76, TCG, one core”) is unchanged. S1 adds an opt-in smp2-probe build that powers one secondary on via PSCI CPU_ON (MMU off, prints its MPIDR, parks; no EL0, no page tables, no shared kernel state beyond one flag word). Any multi-core claim needs a new written sponsor accept. Not “secure.”

Prior v1.74 (2026-10-02). Slice/update of v1.73. Docs only, no code change. The sponsor accepted ADR-091 revision 4 in writing (“I accept ADR-091 rev 4”, 2026-10-02; written in the DSO chat, recorded in the DSO vault, relayed by DSO). ADR-097 (PR #150) is Accepted and merged as 8344797; ADR-098 (PR #151) is on main as 3c47c0c. ADR-091 now carries revision 4: only the PAN clause (“including while the kernel handles exceptions from EL0, and EL1 faults on EL0-readable memory (the execute-only app text page is not covered: this CPU has no FEAT_EPAN)”) and the limit (“app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN)”, was “EL0 can read its own code bytes”) changed; every other clause is as accepted at revision 3. Threat-model effect: none on the mechanism; the written claim now matches it (no FEAT_EPAN, so EL1 can still read the execute-only app text page; PAN is held during EL0 exception handling). See below. Not “secure.”

Prior v1.73 (2026-09-29). Slice/update of v1.72 (ADR-098 PAN correction, PR #151), which builds on v1.71 (#149 ADR-096). ADR-097 (proposed at the time; merge required sponsor rev 4 written accept; accepted 2026-10-02, see v1.74): EL0 app text is execute-only. Every app’s .rodata moved onto the ELF-header page (app-hdr slot, EL0 RO + NX); .text alone on app-text is mapped EL1 RO / EL0 none / UXN=0. Still five slots. An EL0 load of its own text takes a permission fault, a syscall given a text pointer is refused, rodata still reads and prints, and the inventory still counts the UXN=0 page. Two findings: (1) the CPU has no FEAT_EPAN (ID_AA64MMFR1_EL1.PAN=2), so with PAN set an EL1 load of the XO text page does not fault; the ADR-091 clause “PAN is enabled and EL1 faults on EL0 memory” is therefore not literally true for that page, which is why merge waits for a sponsor rev 4 accept (ADR-091 is not edited). (2) The pre-existing PAN gap found here (PAN clear during EL0 exception handling and after EL0 trips) is split out and fixed by ADR-098 (v1.72); this branch merges it. Leak builds for both are caught.

Prior v1.72 (2026-09-29). Slice/update of v1.70 (#148 M6); v1.71 is ADR-096 (PR #149), numbered before this slice and merged after it. ADR-098 (correction): PAN was not held while the kernel handled EL0 exceptions (SCTLR_EL1.SPAN was 1) or after EL0 trips (return_from_el0 resumed with SPSR.PAN clear). The ADR-080 boot probe ran before any EL0 trip and missed it, so from the ADR-080 enable (and from the M6 accept) until this fix the ADR-091 clause “PAN is enabled” held at boot but not while handling EL0 exceptions. Fixed (SPAN=0 after enable; SPSR.PAN on return) with fail-closed live checks (pan: live-syscall pstate-pan=1 sctlr-span=0, pan: after-el0 …, pan: ok held …) and a negative build that undoes the fix (boot probe still passes; live checks report pan: lost …). Every clause of the ADR-091 sentence is literally true again; the sentence is unchanged.

Prior v1.71 (2026-09-28). Slice/update of v1.70 (#148 M6). ADR-096: a fail-closed task-to-task probe. An EL0 task on the user root (ASID 1) cannot read or write another EL0 task’s private page on the ASID-B root (ASID 2) at the same VA, right after that task ran (no TLBI of the VA in between). A’s root has no leaf for B’s frame. With its own page at that VA, each task sees and changes only its own frame. A leak build that maps B’s frame into A’s root is caught. Evidence next to the ADR-091 sentence, not part of it: the accepted sentence and its non-scope (task-vs-task) are unchanged. Limits: one core; QEMU TCG flushes its software TLB on an ASID change, so this is not a hardware stale-TLB test; two kernel-built tasks, not two loaded apps.

Prior v1.70 (2026-09-28). Slice/update of v1.69 (#147 ADR-095). M6: the sponsor accepted the ADR-091 revision-3 sentence and scope in writing (B3, 2026-09-28 16:41 CEST). ADR-091 is Accepted. Umbrella “EL0 isolated” is Verified, only as that sentence (below), with its limits: one core, the TCG default smoke machine; not a speculative-execution or side-channel claim, not a real-hardware claim, not a certification; EL0 can read its own code bytes; no multi-core race story. Docs only: no code change; the M6 PR CI run is the re-probe. (Revision 4, 2026-10-02: the limit and the PAN clause were reworded, see v1.74.)

Prior v1.69 (2026-09-28). Slice/update of v1.68 (#146 ADR-094). ADR-095 (G3): the ADR-085 B2-P procedure was re-run on main 8107578 on an AWS Graviton3 c7g.metal under KVM and passed (el0: serror / b2: taken, fail-closed smoke, attempt 1). The B2-P leg of the ADR-090 evidence class now also holds on the post-M1/M2 boot path. No posture change: reduced b2-serror profile, one-off, not CI. Umbrella still Planned / non-claim; ADR-091 is still Draft (B3 not Met).

Prior v1.68 (2026-09-27). Slice/update of v1.67 (#144 ADR-092). ADR-094 closes G4. Syscall user pointers now need EL0 read (copy from EL0) or write (copy to EL0) permission on every page. App text+rodata is mapped EL0 read-only + executable. Probes: refused kernel-stub / kernel-window reads, refused xo-text / RO writes, a straddle, a seven-syscall sweep, and a sysptr-leak-probe build that must fail. G1 allowlist unchanged. Umbrella still Planned / non-claim; ADR-091 revision 3 is still Draft (B3 not Met); G3 is optional.

Prior v1.67 (2026-09-26). Slice/update of v1.66 (#143 ADR-091). ADR-092 (sponsor option (b)): a fail-closed EL0-reachability walk over kernel / user / ASID-B TTBR0 and TTBR1, plus an EL0 store permission-fault probe, close ADR-091 gaps G1/G2. New gap G4: syscall copy helpers skip the EL0 permission check. Umbrella still Planned / non-claim.

Prior v1.66 (2026-09-26). Slice/update of v1.65 (#142 ADR-090). ADR-091 (M5, D4) is a draft umbrella sentence and scope, pending the sponsor’s written accept. No posture change. The umbrella stays Planned / non-claim.

Prior v1.65 (2026-09-26). Slice/update of v1.64 (#141 ADR-089). ADR-090 (M4, sponsor D1): the umbrella’s taken-SError evidence class is the B1 pinned-QEMU CI job (now unconditional) plus the B2-P one-off KVM run. Stock TCG still parks. Not the umbrella: the sponsor accept (B3) is still absent.

Prior v1.64 (2026-09-26). Slice/update of v1.63 (#140 ADR-088). ADR-089 (M3, sponsor D2): the _start stub page is the one documented identity exception, the checklist identity row is reworded accordingly and Met on this session’s evidence. Docs only. Still not “EL0 isolated.”

Prior v1.63 (2026-09-26). Slice/update of v1.62 (#139 ADR-087). ADR-088 (M2, sponsor D3): GIC / PL011 / virtio-mmio are reached only through three EL1-only, XN TTBR1 Device blocks (6 MiB); the identity 1 GiB MMIO block is cleared from kernel and ASID-B TTBR0. The fail-closed inventory now allows only the _start stub page (ident: inv k=1 u=1 a=1 leaks=0). Still not “EL0 isolated.”

Prior v1.62 (2026-09-26). Slice/update of v1.61 (#138 ADR-086). ADR-087 (M1): the RAM-side identity leftovers (low RAM below the image, the RO+X image padding tail, the pre-heap frame) are unmapped from kernel and user TTBR0. A committed fail-closed walk over kernel / user / ASID-B TTBR0 allows only the MMIO 1 GiB Device block and the _start stub page, and a planted identity page is caught (in-boot + inv-leak-probe kernel). The EL1-executable identity surface while EL0 runs shrinks from ≈ 1 MiB + 1 page to the single stub page. Still not “EL0 isolated.”

Prior v1.61 (2026-09-26). Slice/update of v1.60 (#137 ADR-085). ADR-086: sponsor unlocked the umbrella item. The ADR-055/060 checklist audit found every §A mile and PAN enable Verified this session (CI-gated by qemu-smoke.sh). §B is not Met: taken SError is Verified only on opt-in paths (B1 pin, B2-P one-off), while the accepted smoke machine parks. A scratch TTBR0 walker shows five identity ranges left in kernel TTBR0 (MMIO 1 GiB Device block, low RAM below the image, _start stub page, image padding tail RO+X, __kernel_end page) and two in user TTBR0 (stub + tail, EL1-only). No umbrella accept exists. Still not “EL0 isolated.”

Prior v1.60 (2026-09-26). Slice/update of v1.59 (#136 ADR-084). ADR-085: sponsor B2-P. On an AWS Graviton3 c7g.metal (eu-north-1, Spot), QEMU -accel kvm with patch 0002 set serror_pending via KVM_SET_VCPU_EVENTS, and the opt-in b2-serror profile took the lower-EL SError while standing at EL0 (el0: serror / b2: taken; fail-closed smoke PASS). That makes B2 Verified (opt-in, one-off paid run, not CI). Real hardware also exposed a latent TCR_EL1.TG1 reserved-encoding bug that TCG masked; it is fixed (TG1=4K). Stock distro QEMU still parks. B1 pin unchanged. Still not “EL0 isolated”, not “runs on real hardware” beyond the b2 profile, not “secure.”

Prior v1.59 (2026-09-25): Slice/update of v1.58 (#130 ADR-083). ADR-084: sponsor B2 spike (different runner/accelerator) on free GitHub-hosted runners is blocked — no arm64 KVM/HVF/WHPX; stock TCG still machine does not provide NMIs. B2 stays Planned / blocked on a sponsor decision. B1 opt-in pin (ADR-083) unchanged. Stock distro QEMU still parks. Still not “EL0 isolated” / BSD sockets / Linux/containers. Not a v2 model and not “secure.”

Prior v1.58 (2026-09-19): Slice/update of v1.57 (#129 ADR-082). ADR-083 ships opt-in pinned QEMU v10.0.0 + TYPE_NMI→async SError; CTOS_QEMU + CTOS_REQUIRE_TAKEN_SERROR=1 Verifies taken el0: serror. Stock distro QEMU still parks. Default CI/Docker stay on distro QEMU. Keep PAN enable + N1–N5 + samples. Still not “EL0 isolated” / BSD sockets / Linux/containers. Not a v2 model and not “secure.”

Scope

In: the ctos guest as built for qemu-system-aarch64 -machine virt (EL1, identity map + TTBR1 private page + TTBR1 RAM alias for EL1 fetch, PL011, GICv2, CNTP, first-fit heap, cooperative EL1 tasks, a deliberate EL0 first mile and a bounded standing context). The git repo (no secrets).

Out: Raspberry Pi or other boards, a second ISA, multi-tenant hosting, secure / measured boot, physical side channels, a hostile hypervisor. Networking: N1 ARP + N2 ICMP ping + N3 minimal UDP + N3.x EL0 UDP DNS SVC + N4 EL0 net SVCs + N5 thin TCP + N5.x EL0 TCP echo SVC (ADR-066 / ADR-067 / ADR-070 / ADR-071 / ADR-068 / ADR-074 / ADR-076); not a BSD sockets / TCP product stack.

QEMU and the host are the TCB we do not defend against. If the emulator or the CI runner is hostile, the guest cannot recover.

Assets

AssetWhy it mattersWhere it lives
Kernel integrityText, VBAR_EL1 table, page tables. Corruption ends the learning probe.EL1 identity map (0x4008_0000…__kernel_end) plus TTBR1 RAM alias / high VBAR (ADR-017)
Secrets-noneCredentials in git would be a host/CI incident, not a guest exploit.Repo policy (NFR-10). The guest stores no keys.
Availability of the virt guestSmoke / cargo test must still print markers and exit. A silent lockup hides bugs.UART, GIC, timer, fail-closed scripts/qemu-smoke.sh
Heap + cooperative stacksWritable RAM used by Box/Vec and M9 workers. Execute-from-here is the W^X question.Frame pool after __kernel_end (ADR-012)
Linker stacks (SP_EL0 / SP_EL1 / fatal)Thread, first-level exception, and nested-fatal stacks. Live pages are RW+NX with .data (ADR-015).Linker holes after .bss (ADR-005)
Guard pages (when mapped invalid)4 KiB unmapped holes below each linker stack. Downward overflow should become a translation fault, not a silent smash of the previous image bytes.__stack_guard / __exc_stack_guard / __fatal_stack_guard (ADR-014)
EL0 trampoline + bait + standing contextOne UXN-clear map-window page and a kernel .data bait. Proves enter/return, “cannot execute kernel data,” “cannot read kernel .data,” and a bounded standing dual-SVC. Not POSIX, not isolation.paging::EL0_PAGE + KERNEL_DATA_BAIT + L1_USER (ADR-013)
TTBR1 private pageOne EL1-only high page. Proves EL0 cannot load it. Not a relocated kernel.paging::TTBR1_PRIV (ADR-016)
TTBR1 RAM alias / high VBAREL1 can fetch .text at identity + TTBR1_BASE. VBAR_EL1 is that alias. Identity -kernel stub stays.L1_HIGH[1] → cloned L2_HIGH_RAM (ADR-017, ADR-018)
Identity-tear pageOne dedicated identity text page unmapped from TTBR0. High twin stays. Not a relocated kernel.__ident_tear_* (ADR-018)
Identity text rangeDedicated 16 KiB identity text range unmapped. High twins stay. Not a relocated kernel.__ident_tear_* 16 KiB (ADR-019)
Live identity .textrustc vtables rewritten to high aliases; live identity .text after _start unmapped. High twins stay. Not a relocated kernel.[0x4008_1000, __text_end) (ADR-020)
Identity .rodataIdentity .rodata unmapped after a pointer rewrite. High twin stays. Not a relocated kernel.[__rodata_start, __ident_tear_start) (ADR-025)
Identity .data / stacksIdentity .data/.bss/linker stacks unmapped after SP relocate + pointer rewrite. High twin stays. Not a relocated kernel.[__data_start, __kernel_end) (ADR-037)
Identity heapIdentity heap unmapped after high GlobalAlloc VAs. High twin stays. Remaining frames after the heap stay identity-mapped. Not a relocated kernel.64 KiB first-fit pool (ADR-038)
PAN capability + enableID_AA64MMFR1_EL1.PAN printed; PSTATE.PAN enabled on cortex-a76; EL1-vs-EL0 fault. Historical a57 enable non-goal (ADR-047).src/pan.rs (ADR-026 / ADR-079 / ADR-080)
Remaining isolation gapsShared boot-stub text stays by decision (ident: start-stay; never yank _start). PAN enable Verified on cortex-a76 (ADR-080). Leftover identity RAM torn (ident: ram*, ADR-049). Taken lower-EL SError hard-stopped on stock QEMU after ADR-081 re-probe (el0: serror-park; A-clear dormant prep; ADR-053 / ADR-081 / ADR-082); Verified only under the B1 opt-in pin (ADR-083); B2 free-runner spike blocked (ADR-084).User TTBR0 + ASID + TTBR1 + torn live .text / .rodata / .data/stacks / heap / leftover frame RAM + EL0 entry without VMALLE1 + lower-EL IRQ while standing + default IRQ-unmasked ERET + lower-EL FIQ while standing + SError park honesty + A-clear dormant prep + start-stay honesty + PAN enable + SVC ABI; umbrella “EL0 isolated” Verified only as the accepted ADR-091 sentence (M6; checklist ADR-055 all Met); sponsor closure table ADR-060
SVC ABI (exit / uart_write / yield)Documented numbers 16–18. uart_write rejects a kernel .data pointer and a TTBR1 / non-canonical alias. Not Linux. Not app hosting.src/syscall.rs (ADR-021)
libctos CRTWrappers + _start CRT. Hello payload copied onto the standing EL0 page. Must not issue SVC #0–#2.libctos/ + src/libctos.rs (ADR-022)
Guest ELF PT_LOAD loaderGuest parses embedded ELF64 ET_EXEC, maps PT_LOAD into the user map-window, ERETs to e_entry. Rejects PT_INTERP and W+X. Not a Linux ABI.src/loader.rs (ADR-023)
Standing EL0 as normal modeLoaded image stands until SYS_EXIT. is_active() is a task flag. Unexpected lower-EL sync restores fail-closed. Not a process table.src/el0.rs (ADR-024)
Thin VFS + memfsNamed heap buffers. Create / open / read / write / close. User path/I/O pointers must be user-mapped and kernel-mapped. Not POSIX.src/vfs.rs (ADR-027)
virtio-net (N1 ARP + N2 ICMP + N3 UDP + N5 thin TCP + N3.x/N4/N5.x EL0 SVC)Guest programs a virtio-mmio net DMA master for ARP, ICMP echo, one UDP datagram, then one thin TCP active-open + payload vs QEMU SLIRP/guestfwd. EL0 uses tiny SVCs only (kernel owns NIC), including quiet TCP echo. Not a BSD sockets / TCP product.src/virtio.rs + src/syscall.rs (ADR-066, ADR-067, ADR-070, ADR-071, ADR-068, ADR-074, ADR-076)
virtio-blk + FAT16Guest programs a virtio-mmio DMA master. Image is host-built FAT16. Guest may write/create small root files (ADR-050), grow a file across a cluster boundary (ADR-064), list root names (ADR-056), delete a root file (ADR-057), create a root directory (ADR-073), and create one nested subdirectory + empty-rmdir (ADR-077). Same VFS open/write/readdir/unlink/mkdir/rmdir behind prefix mounts (ADR-058). Not a trusted disk.src/virtio.rs + src/fat.rs + src/vfs.rs (ADR-028, ADR-050, ADR-056, ADR-057, ADR-058, ADR-064, ADR-073, ADR-077)
OS/app slotsHost kernel ELF + published app ELF. Guest reads FAT /hello and maps PT_LOAD. A2–A4 use the same file (no embed). Cross-update is a host two-boot probe. Not a trusted disk.src/slot.rs (ADR-030, ADR-032)
Console / sensorsPL011 is how we see whether a probe ran.Device MMIO 0x0900_0000

Adversaries

AdversaryIn scope?Notes
Buggy kernel code (wrong store, bad unsafe, execute-from-heap, stack smash)YesPrimary adversary today. Mitigate with PXN on heap/frames, unmapped linker-stack guards, minimize unsafe (NFR-01), fail-closed smoke.
Malicious EL0 (standing or trampoline task executing kernel data or escalating via a bad map)Named; standing context is boundedFirst mile + user-TTBR0 read mile + ASID TLB mile + standing dual-SVC + standing task until exit + TTBR1 private page + EL1 high-VA fetch + a documented SVC ABI exist. Since M6 the umbrella “EL0 isolated” is Verified only as the accepted ADR-091 sentence (one core, TCG default smoke machine; not speculative-execution / side-channel, not real hardware, not a certification). The boot-stub identity page stays by decision (ident: start-stay, ADR-089); stock-QEMU taken SError still parks (el0: serror-park). Product freestanding hosting Verified under ADR-048/052; not Linux/POSIX/containers (ADR-048 / ADR-052).
Compromised device treeMostly outM7 does not walk FDT. DTB sits at RAM base below the image. A hostile DTB is a QEMU/host problem until a walker exists; then it becomes an input-validation ADR.
DMA / virtio devicesNamed; boundedA7 programs virtio-mmio blk (one queue, poll used.idx, identity-PA DMA). N1–N3/N5 program virtio-mmio net (RX+TX queues, same poll/DMA pattern) for ARP, ICMP echo, one UDP datagram, then thin TCP (ADR-066 / ADR-067 / ADR-070 / ADR-074). QEMU is still TCB. A malicious virtio device is out (hostile hypervisor). No BSD sockets / TCP product.
Hostile QEMU or CI hostOutHypervisor / runner is trusted. Repo-secret leak is a host control (.gitignore, review), not a guest mitigation.
Network attackerNamed; bounded (link only)N1–N3/N5 attach QEMU user netdev (+ N5 guestfwd echo) + guest virtio-net for ARP + ICMP + one UDP datagram + thin TCP (ADR-066 / ADR-067 / ADR-070 / ADR-074). No BSD sockets / TCP product. QEMU SLIRP/guestfwd is still TCB. Hostile on-wire peers beyond this probe are out of product scope.

Trust boundaries

[ host / QEMU / CI ]  --trusted config--  [ virt guest ]
                                              |
                                         EL1 kernel  ← TCB today
                                              |
                                         EL0 first mile + standing  ← entered / stood / left
                                              |                       (not a POSIX user)
                                         EL0 isolation    ← Verified only as the ADR-091 sentence (M6)
  • EL1 now. Page tables distinguish execute (RO+X text vs RW+NX data/stacks/heap vs Device XN), write (AP[2] on text), and presence (guard holes).
  • EL0 first mile + user TTBR0 + ASID TLB + standing + standing-as-normal + TTBR1 first cut + EL1 high-VA fetch + identity-tear first cut + identity .text range tear + live .text tear + identity .rodata tear + EL0 entry without VMALLE1 + lower-EL IRQ while standing + IRQ-unmasked default ERET + lower-EL FIQ while standing + SError park honesty + PAN ID-field + SVC ABI + libctos CRT + guest PT_LOAD loader; umbrella “EL0 isolated” = the ADR-091 sentence only (M6). Lower-EL AArch64 sync is taken (SVC / IABORT / DABORT). Lower-EL IRQ while standing is taken and returns to EL0 (el0: irq); default standing/task ERET clears I (el0: irq-default); lower-EL FIQ while standing is taken when FIQEn is armed (el0: fiq); SError park honesty (el0: serror-park); ADR-045 A-clear + QMP attempt (inject failed). Short non-standing trampoline probes stay IRQ-masked. Dual ASID without TLBI VMALLE1 is a probed mile (asid: ok). Standing/EL0 trampoline also switches without TLBI VMALLE1 after identity .data/heap tear (el0: no-vmalle1). Standing dual-SVC flips is_active(). A standing task runs a loaded image until exit (el0: task-ok); unexpected sync restores fail-closed (el0: restore-fail). TTBR1 private page is EL1-only (ttbr1: ok). EL1 can fetch a real path from the high RAM alias (ttbr1: el1 exec); VBAR_EL1 is that alias (ttbr1: vbar). A 16 KiB dedicated identity text range is unmapped (ident: range); live identity .text after _start is unmapped after a vtable rewrite (ident: reloc / ident: live); identity .rodata is unmapped (ident: rodata); identity .data/stacks are unmapped (ident: data); identity heap is unmapped (ident: heap); _start stays (ident: start-stay); leftover identity RAM after the heap is torn (ident: ram). ID_AA64MMFR1_EL1.PAN is 0 on -cpu cortex-a57 (pan: absent). A documented SVC ABI (svc: ok) is an ABI mile. A linked libctos hello (libctos: ok) is a CRT mile. A guest PT_LOAD (loader: ok) is a loader mile. Standing-as-normal (el0: task-ok) is a mile; product freestanding hosting is Verified separately under ADR-048/052. Umbrella “EL0 isolated” is Verified only as the accepted ADR-091 sentence (M6; PAN enabled on default cortex-a76 per ADR-080; taken SError on the ADR-090 evidence class while stock QEMU still parks; never yank _start; leftover RAM torn via ADR-049). A caught load of .data or of TTBR1_PRIV is that mile, not the umbrella by itself.
  • Repo vs guest. “No secrets in repo” is a host boundary. It does not harden the UART.

PAN enable + EL1-vs-EL0 fault (v1.55)

ADR-080 / #125 mile 2: on default -cpu cortex-a76, MSR PAN + EL1 load of an EL0-accessible page faults (pan: enabled / pan: el1-fault; SPSR.PAN at the fault). Syscall uaccess clears PAN around copies. Umbrella “EL0 isolated” stays non-claim (ADR-055). Taken SError still hard-stopped (ADR-053).

Taken SError reopen re-probe (v1.56)

ADR-081 / #125 mile 3: dated agent-box re-probe on default -cpu cortex-a76 plus GICv3 / virtualization=on / secure=on / max — QMP inject-nmi / HMP nmi still machine does not provide NMIs. FEAT_NMI ≠ SError. KVM absent on this smoke host. Unlock = sponsor B1/B2/B3. Park Verified; A-clear dormant prep. Still not taken Verified / “EL0 isolated.”

Taken SError B2-P on real arm64 + KVM (v1.60)

ADR-085 (2026-09-26): a paid one-off run on AWS c7g.metal (Graviton3, CPU: All CPU(s) started at EL2, /dev/kvm=present). The SError source is KVM’s virtual SError (HCR_EL2.VSE), which the host requests through KVM_SET_VCPU_EVENTS{serror_pending=1} (QEMU v10.0.0 + 0001 + 0002, inject-nmi on the el0: serror-arm cue). The guest runs natively. Run 4 console: el0: standing / el0: serror / el0: restored / b2: taken / b2-kvm-smoke: PASS ... accel=kvm. Threat-model relevance: the lower-EL SError vector, standing-EL0 A-clear window and restore path now have evidence on real Neoverse-V1 hardware, not only TCG. Real hardware also found TCR_EL1.TG1=0b00 (reserved), which broke every TTBR1 high-VA walk on Graviton3; it is fixed to 4K. Limits: opt-in profile only (no GIC/timer/virtio/FAT under KVM), no CI gate, no host-side ioctl trace. Umbrella “EL0 isolated” still a non-claim.

Taken SError B2 free-runner spike (v1.59)

ADR-084 (2026-09-25): sponsor chose ADR-081 B2 — a genuinely different runner/accelerator, not the B1 pin. Free GitHub-hosted runners were probed (runs 36193423604 / 36193613972): arm64 Linux kvm [1]: HYP mode not available / /dev/kvm absent; macOS arm64 HVF HV_UNSUPPORTED; Windows arm64 no VT / WHPX disabled; x86 /dev/kvm cannot run an AArch64 guest. Stock TCG on those runners still machine does not provide NMIs. The honest B2 source would be a KVM virtual SError (serror_pending) injected by a small VMM on real arm64 hardware — Planned / blocked on a sponsor choice (self-hosted arm64 KVM runner, paid bare metal, or wait). No new Verified. Threat surface unchanged; still not “EL0 isolated.”

Residual identity surface + umbrella audit (v1.61)

ADR-086 (2026-09-26). Threat-model relevance of the residual identity map: EL0 cannot touch any of it (every leaf is EL1-only, el0=false). The EL1-side exposure is: (1) an identity RO+X padding tail (≈ 1 MiB, no image content) and the stub page stay executable at EL1 through TTBR0, including while EL0 runs (user TTBR0 keeps both), which widens the EL1 gadget/ret2 surface through low VAs; (2) the MMIO block and two small RW RAM ranges stay reachable at EL1 through identity VAs, so a stray EL1 pointer into low VAs still hits memory instead of faulting. Mitigation plan: M1 ratchet + RAM-side tears, M2 MMIO high alias, M3 stub policy (sponsor D2). Taken SError evidence class is a sponsor decision (D1). Umbrella stays non-claim.

Identity inventory ratchet (v1.62)

ADR-087 (2026-09-26). Residual identity after M1: I1 MMIO block (EL1 RW, XN, Device; kernel + ASID-B only) and I3 _start stub page (EL1 RO+X; kernel + user + ASID-B). The ratchet turns “what we tore” into “what is still mapped”, so a regression that re-maps any identity page fails CI. Found while building it: the ADR-020/025 .rodata pointer rewrite silently relocated the allowlist constants themselves (compare physically). M2 removes I1; I3 stays as the one documented exception (sponsor D2).

MMIO high alias (v1.63)

ADR-088 (2026-09-26). Threat-model effect: a stray EL1 pointer into low VAs < 0x4000_0000 now faults instead of touching a device, and the device surface reachable at EL1 shrinks from a 1 GiB identity hole to three 2 MiB TTBR1 blocks. Residual identity: the _start stub page (EL1 RO+X) in kernel / user / ASID-B TTBR0, the documented exception (sponsor D2). B2-P KVM evidence predates this boot-path change.

_start stub identity exception (v1.64)

ADR-089 (2026-09-26). The residual identity page is EL1 RO+X (never EL0-accessible) in kernel / user / ASID-B TTBR0; it holds _start and the identity copy of the vectors. Any additional identity leaf fails CI. A possible later narrowing (drop it from user TTBR0 only) is noted, not decided.

Taken-SError evidence class (v1.65)

ADR-090 (2026-09-26). Threat-model effect: the lower-EL SError vector path (standing EL0, PSTATE.A clear) is exercised on every push/PR by a patched-QEMU TCG job; it is no longer opt-in. What this evidence does not show: RAS/hardware error semantics, SError on stock QEMU, or a CI-gated real-hardware run (B2-P was one paid run on a reduced profile at 159b178).

Umbrella accept draft: scope and non-scope (v1.66)

ADR-091 (2026-09-26, Draft). If accepted, the umbrella would cover only the architectural EL0→kernel page-table / exception-level boundary on the default QEMU TCG machine (one core). It explicitly excludes speculative execution / side channels (no KPTI: TTBR1 stays mapped EL1-only while EL0 runs), real hardware, multi-core, task-vs-task isolation beyond the ASID mile, DMA/IOMMU and EL2. Gaps recorded for the sponsor: no exhaustive EL0-permission walk (G1), no EL0 write probe (G2).

EL0 reachability + EL0 write fault (v1.67)

ADR-092 (2026-09-27). Threat-model effect: the EL0 attack surface is now enumerated rather than assumed. While an app stands, the only EL0-reachable pages are its header, text and stack slots (3 pages per shared window root); at rest there are none; nothing in TTBR1 is EL0-reachable. Any new EL0 leaf outside five allowlisted slots, or backed by kernel frames, fails CI. EL0 stores to kernel data fault. Open (G4): the kernel’s syscall copy path trusts “mapped in user TTBR0” instead of “EL0 may access”, a confused-deputy route to EL1-only pages mapped in user TTBR0 (the _start stub page; transient EL1-only window pages).

Syscall pointers need EL0 permission (v1.68)

ADR-094 (2026-09-27). Threat-model effect: the confused-deputy route recorded as G4 in v1.67 is closed on the default smoke machine. user_range_ok_max now checks the user leaf’s AP bits on every page in the range: AP[1] for a copy from EL0, AP[1] with !AP[2] for a copy to EL0. So no syscall can make the kernel read a page EL0 cannot read, or write a page EL0 cannot write. That covers the _start stub, kernel-only window pages, EL0 execute-only text and EL0-RO pages. CI gates every pointer-taking syscall (17, 19, 20, 21, 22, 24, 27), and a build without the check must fail. App text is now EL0-readable, the normal model for a shared text+rodata page, and still not writable. Residual: a check-then-copy software rule on one core. No AT S1E0R/W or LDTR/STTR copy, and no SMP TOCTOU story.

Taken SError B2-P re-run on current main (v1.69)

ADR-095 (2026-09-28). Threat-model effect: none on the posture; it removes a staleness caveat on the evidence. ADR-090’s B2-P leg was taken at 159b178, before the RAM-side identity tears (ADR-087), the TTBR1 MMIO alias and identity MMIO tear (ADR-088), ADR-092 and ADR-094. On main 8107578 the b2-serror profile booted natively on Graviton3 under KVM through ident: mmio-high, ident: low/tail/kend, ident: mmio, PAN enable and EL1 fault (Graviton3 reports pan: id=2), and took the KVM virtual SError (KVM_SET_VCPU_EVENTS{serror_pending=1} → HCR_EL2.VSE) at the lower-EL SError vector while standing at EL0. Residual: reduced profile (no GIC / timer / virtio / FAT, no inventory / reach / write / sys-ptr probes under KVM); one-off paid run, not CI; no host ioctl trace; not a real-hardware claim for the default image.

Umbrella “EL0 isolated” accepted (v1.70)

ADR-091 (2026-09-28, Accepted, M6). The sponsor accepted the revision-3 sentence and scope in writing (B3, 2026-09-28 16:41 CEST; sponsor wrote it in the DSO agent chat, DSO relayed it to ctos). Threat-model effect: the umbrella row moves from Planned / non-claim to Verified, and only as the sentence below. Revision 4 (2026-10-02, v1.74): the sponsor accepted ADR-091 rev 4 in writing (“I accept ADR-091 rev 4”); the sentence below is revision 4. Revision 3 differed only in its PAN clause (“PAN is enabled and EL1 faults on EL0 memory”) and in the limit “EL0 can read its own code bytes”:

“On the ctos default smoke machine (QEMU virt, -cpu cortex-a76, TCG, one core), EL0 is isolated from the kernel at the architectural page-table / exception-level boundary: each EL0 task runs on its own TTBR0 with its own ASID; EL0 faults when it reads, writes or executes kernel memory; in every translation table (kernel, user and ASID-B TTBR0, and TTBR1) the only EL0-reachable pages are five allowlisted user slots backed by non-kernel frames, checked fail-closed while a user app runs and at rest; PAN is enabled, including while the kernel handles exceptions from EL0, and EL1 faults on EL0-readable memory (the execute-only app text page is not covered: this CPU has no FEAT_EPAN); lower-EL IRQ and FIQ are taken while EL0 stands; taken lower-EL SError is evidenced by the ADR-090 evidence class; the only identity (VA = PA) mapping left in any TTBR0 is the documented EL1-only _start stub page (ADR-089); and a system call copies from or to an EL0 pointer only if EL0 itself may read (for a copy from EL0) or write (for a copy to EL0) every page in the range. It is not a speculative-execution or side-channel claim, not a real-hardware claim, and not a certification.”

Limits that stay with it: one core; the TCG default smoke machine; not speculative-execution / side-channel; not real hardware; not a certification; app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN) (ADR-097); no multi-core race story. The ADR-091 non-scope list stands (no task-vs-task isolation beyond ASIDs, no DMA / IOMMU / EL2, stock-QEMU taken SError still parks). No code change: the M6 PR CI run on main plus this change is the one-session re-probe. Older version notes and sections in this file that say “not ‘EL0 isolated’” record the posture at their version.

Task-to-task EL0 isolation probe (v1.71)

ADR-096 (2026-09-28). Threat-model effect: the “malicious EL0 task reads or corrupts another EL0 task” case now has a probe on the page-table/ASID mechanism, not only the EL1-side asid: ok mile. Task B (ASID-B root, ASID 2) stores a sentinel at 0x80007000; task A (user root, ASID 1) then loads and stores the same VA and takes a lower-EL translation fault (FAR = the VA) without seeing B’s value; B’s frame is unchanged; a walk of A’s root finds no leaf for B’s frame. With A’s own frame at the VA, each task reads and writes only its own frame. Markers t2t: … / t2t: ok, fail-closed in scripts/qemu-smoke.sh, plus a t2t-leak-probe build that must be caught. Not part of the ADR-091 sentence (task-vs-task stays in its non-scope; a revision 4 needs the sponsor in writing). Limits: one core; TCG flushes its own TLB on an ASID change, so no hardware stale-TLB claim; kernel-built tasks, not two loaded apps under a scheduler; not side-channel.

PAN held during EL0 exception handling (v1.72, correction)

ADR-098 (2026-09-29). Threat-model effect: PAN is the net for kernel bugs that dereference a user pointer outside the copy helpers. That net was down at the start of every EL0 exception handler and after EL0 trips (until the next copy helper restored PAN), i.e. exactly while the kernel serves EL0. The boot probe (pan: el1-fault) could not see it. Now SCTLR_EL1.SPAN=0 and return_from_el0 sets SPSR.PAN; the smoke checks PAN live inside the loaded app’s SYS_YIELD handler (MRS PAN, calibrated) and after the app exits (an EL1 load of an EL0-readable page faults with SPSR.PAN set). A pan-keep-leak-probe build that undoes the fix is caught. Limits: one core, TCG; one exception type and one post-trip point sampled.

EL0 execute-only app text (v1.73; accepted 2026-10-02)

ADR-097 (2026-09-28; accepted 2026-10-02 as ADR-091 revision 4, “I accept ADR-091 rev 4”; merged as 8344797; at v1.73 it was proposed and merge required that accept). Threat-model effect: a compromised or buggy EL0 app can no longer read its own code bytes (e.g. to harvest gadgets or leak code through a syscall): EL0 loads of 0x80002000 take a lower-EL permission fault (xo: el0-read-text fault … dfsc=perm-l3 got=0x0) and uart_write of a text pointer is refused (xo: sys-read text denied … checks=1). Rodata lives on the app-hdr page (EL0 RO + NX). Still five EL0-reachable slots; the fail-closed walk counts UXN=0 pages as reachable (reach=uxn) and catches an unallowlisted XO plant. What gets weaker in the wording, not in the mechanism: without FEAT_EPAN, PAN does not stop EL1 from reading the XO page (xo: el1-pan text … fault=no … epan=no), so “EL1 faults on EL0 memory” no longer covered every EL0 page; revision 4 words this exactly (“EL1 faults on EL0-readable memory (the execute-only app text page is not covered: this CPU has no FEAT_EPAN)”). PAN fix split out: PAN being clear during EL0 exception handling and after EL0 trips is fixed by ADR-098 (v1.72, merged into this branch); the xo live line echoes its reading (xo: pan live-syscall pstate-pan=1 sctlr-span=0). Leak builds xo-leak-probe / pan-keep-leak-probe are caught. Limits: one core, TCG, not real hardware; live check on hello-libctos only (other apps by build/loader checks).

KVM re-check via GitHub OIDC (ADR-099)

Folded from ADR-099 §7 (risk and impact evaluation, as on main after PR #157) in v1.76. ADR-099 stays the source of truth for the trust and permission policies, preconditions P1–P15 and the workflow; if the two disagree, ADR-099 wins and this section must be re-synced.

Asset and boundary. A GitHub Actions job (.github/workflows/kvm-recheck.yml, manual dispatch only, from main, behind the kvm-pilot environment) assumes one AWS role through GitHub OIDC (no stored keys) and starts one Graviton metal Spot instance that runs the reduced b2-serror profile under KVM, then terminates it. New trust boundary: GitHub’s OIDC token → the AWS role. New assets: the role, the cloud account’s spend, and the public CI log.

Status (2026-10-13 pilot, relayed by DSO; see the honesty ledger). One supervised run: smoke PASS under accel=kvm; the run verdict was red only from a false LEAK (teardown wait too short for metal), fixed by PR #157. Not a recurring check, not a required check.

Impact is “what an attacker or a mistake could cost”. Residual is after the listed mitigations, assuming ADR-099 P1–P15 are met.

#ThreatLikelihood × impact (before)MitigationsResidual risk
1Another repo, branch, fork or pull request assumes the roleLow × HighExact aud and sub (no wildcard); environment with required reviewers and main-only branch policy; fork PRs get a different sub and no id-token for secretsLow. Depends on the exact sub being right (P14) and the environment existing first (P5).
2A PR edits the workflow to abuse the roleMedium × HighDispatch runs only the default-branch file; guard refuses any ref but main; environment gate; enforce_admins is onLow–Medium. main requires no approving review (observed), so the control is the sponsor merging by convention. DSO may ask the sponsor to require one review on main or a code-owner rule for .github/workflows/.
3Role is over-permissioned (miner, wrong instance type, other region)Medium × HighConditions on instance type, Spot market, IMDSv2, tags and region; pinned image/subnets/SG; 1 h session; no IAM, no PassRole; Spot quota; budget alarmLow. IAM conditions are untested until the pilot (P13).
4Tag spoofing (a resource tagged project=ctos to dodge the sweeper, or the terminate scope abused)Low × LowTags only at creation (CreateAction); terminate only matches tags the role itself applied; the sweeper counts the tag broadlyLow. A person with other credentials could still tag things; out of scope for this role.
5Privilege escalation through the instanceLow × HighNo instance profile, no iam:PassRole, explicit Deny on iam:*; IMDSv2 required, hop limit 1; no credentials on the boxLow.
6Secret in user-dataLow × MediumUser-data holds only a commit, a nonce and a timer; no keys, tokens or URLs with secretsVery low.
7Identifier leakage in public logs (instance ID, account ID, host name, IP)Medium × Low–MediumAccount-ID masking; instance IDs masked at creation; only ^CTOSB2: lines and error codes are printed; no raw console or stderr; identifier guard on repo textLow. Confirm on the pilot (P15).
8Leaked running instance (cost, exposure)Medium × LowPre-flight refusal; in-instance shutdown -h +45 and terminate-on-shutdown; always-run terminate with fresh credentials, by tag; wait for terminated; sweeper gives LEAK; budget alarmLow. Worst case about $0.40/h until noticed if the runner and the user-data both fail.
9Console text is forged or replayedLow × MediumPer-run nonce and commit in the result line; anchored PASS line; KVM and arch markers; terminate-by-tag so one run’s console is read onceLow. The code under test can print its own PASS (inherent; the commit is recorded and the tested commit must be on main).
10Silent green on a skip or timeoutMedium × High (trust)Only PASS exits 0; SKIPPED, INCONCLUSIVE, REFUSED, LEAK, TEARDOWN-SLOW, ABORTED-after-aws-call, NOT-RUN are red; guard fails (not skips) on an unset variableLow.
11Spot capacity or interruptionHigh × LowMulti-AZ retry; honest SKIPPED / INCONCLUSIVE; no on-demand fallbackAccepted: the check is sometimes unavailable.
12Third-party action supply chain (configure-aws-credentials, checkout)Low × HighPin by full commit SHA before enabling (P12); permissions: {} at top level; the job only gets id-token: write and contents: readLow–Medium until pinned.
13Supply chain on the instance (apt, the rustup install script, git clone, QEMU source)Medium × LowQEMU is built from the pinned v10.0.0 source plus the repo’s patches; the repo is cloned at a commit already on main; the instance holds no credentials, has no ingress, IMDSv2 hop limit 1, and lives under an hourLow. Open egress remains (needed for apt/rustup/git).
14Repo rename or transfer (subject reuse by a new owner)Low × HighThe immutable subject (owner and repo IDs) is what the trust policy pinsLow once P14 is confirmed.
15Cost runawayLow × LowCost-cap guard; MaxPrice; lifetime; concurrency 1; budget alarmLow; about $0.33 worst case per run [estimate].
16CloudTrail off, so no audit trailUnknown × HighCondition (a) is a hard precondition before the role is created or usedP1 met 2026-10-13 (multi-region trail, DSO-verified).
17Unprotected environment auto-created by the workflow’s first referenceMedium × HighP5: create the environment with protections first; order mattersLow if the order is followed.
18Unknown account facts (Spot service-linked role, quota, EBS default key)Medium × LowP8; pilot; any new permission goes to the sponsor, not added silentlyLow.
19Credential rule 4 is only relaxed for this roleLow × MediumOne role, one repository, one environment, no stored key, 1 h tokens; the exception record is referenced in the enabling PR (P10)Low. Any second role or a schedule needs its own approval.

Not claimed here: the default image, real-hardware boot of the default image, or anything beyond the reduced b2-serror profile on one KVM host per run.

Non-goals

  • Not a certified secure OS (no Common Criteria, no PSA, no “hardened”).

  • Not side-channel complete (no cache/timing/Spectre story; QEMU TCG is the wrong lab).

  • Not a product “the kernel is W^X” sentence. The identity image on this virt guest is RO+X / RW+NX with WXN (ADR-015). Future mappings are not automatically covered. Guard pages remain holes.

  • Not ASAN / canaries / heap-stack guards. Coop worker stacks have no unmapped holes. Overflow there is still image-adjacent PXN RAM.

  • Not secure boot, PAN enable on historical cortex-a57 (non-goal on that CPU, ADR-047; enable Verified on cortex-a76 / ADR-080), or a fully torn-down identity map (ADR-020/025/037/038/049 unmap live .text / .rodata / .data+stacks / heap / leftover frame RAM; _start stays at 0x4008_0000 by decision).

  • Not multi-core. Every isolation claim is one core (ADR-091 rev 4). The ADR-100 smp2-probe secondary bring-up is a smoke, not SMP support.

Mitigations mapped to probes

MitigationProbeLedger
Secondary CPU powered on, parked, never runs kernel state or EL0 (ADR-100 S1, opt-in)smp2-probe under -smp 2: smp: secondary up mpidr=0x0000000080000001 + smp: ok cpu1 … parked; -smp 1 negative must print smp: cpu1 missed; default run must show no smp: lineVerified locally 2026-10-13 + this PR’s non-required CI job. Not SMP support.
Threat-model v1.23 writtenRead this fileVerified (file + review). Still no “secure OS”.
Device MMIO XN (L1 block 0)pxn_for(0x0900_0000) == Some(true)Covered by the W^X tests when they run.
Heap + coop stacks PXNSerial wx: ok; #[test_case] flags + execute-from-heap IABORTVerified: 2026-09-10 cloud qemu-smoke (honesty ledger).
Kernel text still executableis_executable(0x4008_0000)Same W^X probe.
Execute-from-writable heap forbiddenArmed permission IABORT → wx: nx heap then wx: okFail-closed in scripts/qemu-smoke.sh.
Linker-stack guard holesSerial guard: fault / guard: ok; store to __stack_guardVerified: 2026-09-10 cloud qemu-smoke (honesty ledger).
Minimize unsafeReview (NFR-01). Count is not a proof.Policy.
Fail-closed smokescripts/qemu-smoke.sh greps + force-fail exit ≠ 0NFR-04 / NFR-05.
No secrets in repoPolicy + .gitignore (including .obsidian/)Host control.
IRQ least privilegeIRQ path does not allocate or yield_nowConvention. Ratchet if it fails twice.
EL0 entered and returnedSerial el0: svc / el0: ok; #[test_case]Verified first mile — not isolation.
EL0 cannot execute kernel dataSerial el0: nx kernel; lower-EL IABORTVerified first mile.
EL0 cannot read kernel .dataSerial el0: no kernel read; user TTBR0 omits .dataVerified read mile. One mile; alone it is not the ADR-091 umbrella claim.
ASID isolation (no VMALLE1)Serial asid: dual / asid: conflict / asid: okSpecific mile. One mile; alone it is not the ADR-091 umbrella claim.
Task-to-task EL0 isolation (ADR-096)Serial t2t: walk b-frame … / t2t: a-read fault … / t2t: a-write fault … / t2t: same-va … / t2t: ok …; t2t-leak-probe build caught; #[test_case] el0_task_cannot_read_or_write_other_taskVerified (one core, TCG, kernel-built tasks). Not part of the ADR-091 sentence.
EL0 execute-only app text (ADR-097, accepted 2026-10-02)Serial xo: live … / xo: layout loader … / xo: el0-exec ok … / xo: el0-read-text fault … / xo: sys-read text denied … / xo: el1-pan … / xo: pan live-syscall pstate-pan=1 sctlr-span=0 / xo: ok …; xo-leak-probe + pan-keep-leak-probe build caught; #[test_case] el0_app_text_is_execute_onlyVerified (one core, TCG); merged as 8344797 after the sponsor accepted ADR-091 rev 4 (2026-10-02).
EL0 entry without VMALLE1Serial el0: no-vmalle1; src/exception.rs has no tlbi vmalle1Trampoline cut (ADR-039). Not isolation.
Lower-EL IRQ while standingSerial el0: irqTaken IRQ from EL0 returns to standing (ADR-040). Not isolation.
IRQ-unmasked default ERETSerial el0: irq-defaultDefault standing/task ERET clears I (ADR-041). Short probes stay masked. Not isolation.
Lower-EL FIQ while standingSerial el0: fiqTaken FIQ from EL0 returns to standing (ADR-043). Not isolation.
SError park honestySerial el0: serror-parkNo working SError inject on virt TCG (ADR-043 / ADR-045 / ADR-081). Taken path deferred/non-goal (ADR-047).
Taken SError QMP attemptSerial el0: serror-arm + QMP logGuest A-clear path (ADR-045); inject-nmi fails on QEMU 10 virt+a76 (ADR-081).
Identity _start staySerial ident: start-stayHonesty (ADR-042 / ADR-047). Not isolation.
Leftover identity RAM tearSerial ident: ram / ident: ram-fault / ident: ram-highOptional mile (ADR-049). High twin stays. Not isolation.
Standing EL0 contextSerial el0: standing / el0: restored; is_active() flipsSpecific mile. Not POSIX. Not isolation.
Standing EL0 as normal modeSerial el0: task-enter / el0: task-active / el0: task-exit / el0: task-restored / el0: restore-fail / el0: task-okLoaded image until exit. Fail-closed restore. Not isolation.
TTBR1 private pageSerial ttbr1: el1 / ttbr1: no el0 / ttbr1: okFirst cut (ADR-016).
EL1 fetch from TTBR1 high VASerial ttbr1: el1 exec / ttbr1: vbarExec mile (ADR-017).
Identity-tear first cutSerial ident: split / ident: fault / ident: high / ident: no el0 / ident: okFirst cut (ADR-018). Yank _start decided never; leftover RAM torn (ADR-049).
Identity text range tearSerial ident: jump / ident: range / ident: textRange cut (ADR-019).
High-VA vtable rewrite + live .text tearSerial ident: reloc / ident: liveLive .text cut (ADR-020).
Identity .rodata tearSerial ident: rodata / ident: rodata-fault / ident: rodata-high.rodata cut (ADR-025).
Identity .data / stack tearSerial ident: data / ident: data-fault / ident: data-high.data/stacks cut (ADR-037).
Identity heap tearSerial ident: heap-reloc / ident: heap / ident: heap-fault / ident: heap-highHeap cut (ADR-038).
PAN ID fieldSerial pan: id= / pan: presentCapability probe (ADR-026 / ADR-079).
PAN enable + EL1-vs-EL0 faultSerial pan: enabled / pan: el1-faultADR-080. One mile; alone it is not the ADR-091 umbrella claim.
PAN held during EL0 exceptions + after EL0 trips (ADR-098)Serial pan: live-syscall pstate-pan=1 sctlr-span=0 / pan: after-el0 trips=… mrs-pan=1 el1-load … fault=yes spsr-pan=1 / pan: ok held boot,live-syscall,after-el0; pan-keep-leak-probe build caught; #[test_case] pan_span_cleared_after_enableVerified (one core, TCG). Correction of the ADR-091 PAN clause’s evidence; sentence unchanged.
SVC ABI (exit / uart_write / yield)Serial svc: ok; user buffer + kernel-.data / TTBR1-alias rejectABI mile (ADR-021). Not app hosting.
libctos CRTSerial libctos: hi / libctos: ok / libctos: linkedCRT mile (ADR-022). Not app hosting.
Guest ELF PT_LOAD loaderSerial loader: mapped / loader: okLoader mile (ADR-023). Not a Linux ABI. Not app hosting.
Thin VFS + memfsSerial fs: create / fs: write / fs: read / fs: el0 / fs: okmemfs mile (ADR-027). Not POSIX. Not app hosting.
virtio-net ARP (N1) + ICMP ping (N2) + UDP (N3) + thin TCP (N5)Serial net: virtio / net: mac / net: tx / net: rx / net: ok + net: icmp-tx / net: icmp-rx / net: ping-ok + net: udp-tx / net: udp-rx / net: udp-ok + net: tcp-syn / net: tcp-est / net: tcp-tx / net: tcp-rx / net: tcp-okADR-066 / ADR-067 / ADR-070 / ADR-074. ARP + ICMP + one UDP + thin TCP vs QEMU SLIRP/guestfwd only. Not BSD sockets / TCP product. Host -netdev alone is not the probe.
EL0 net SVC sample (N4)Serial libctos: net-hi / libctos: net-mac / libctos: net-ok / netdemo: okADR-068. SVCs only; kernel owns NIC. Not sockets.
EL0 UDP DNS SVC sample (N3.x)Serial libctos: udp-hi / libctos: udp-ok / udpdemo: okADR-071. net_udp_dns only; DNS is probe bait. Not sockets.
virtio-blk + FAT16Serial blk: ok / fat: ok / fat: write / fat: create / fat: grow / fat: mkdir / fat: nested / fat: rmdir / fat: readdir / fat: entries / fat: delete; VFS /probeblock + FAT mile (ADR-028) + write depth (ADR-050) + multi-cluster grow (ADR-064) + mkdir (ADR-073) + nested+rmdir (ADR-077) + readdir (ADR-056) + delete (ADR-057). Host -drive alone is not the probe. Not POSIX getdents/unlink/mkdir/rmdir/write API.
OS/app slot first cutSerial slot: fat / slot: mapped / slot: okSlot mile (ADR-030). A2–A4 load FAT (no embed).
A9 cross-updateHost smoke: same app sha256 on this OS and ba6541cLeftover (ADR-032). Not “apps update independently.”
RO+NX text/dataSerial ro: ok; execute-from-.data + write-to-RO-textVerified: 2026-09-11 cloud qemu-smoke (honesty ledger).

Claim gate

A PR may say “threat-model v1.22 exists” after a file read. It may say “heap NX” / “identity image is W^X on this virt guest” only when the honesty ledger has a matching Verified QEMU probe. It may say “linker-stack guard faults” only with a matching translation-abort probe. It may say “EL0 entered and returned,” “EL0 cannot execute kernel data,” “EL0 cannot read kernel .data,” “standing EL0 context,” “ASID isolation,” “TTBR1 private page,” “EL1 fetched from a TTBR1 high VA,” “one identity text page was unmapped,” “a 16 KiB dedicated identity text range was unmapped,” “rustc vtables were rewritten to high aliases,” “live identity .text after the boot stub was unmapped,” “identity .rodata was unmapped,” “identity .data/stacks were unmapped,” “ID_AA64MMFR1_EL1.PAN is 0 on -cpu cortex-a57,” “EL0 issued the documented SVC ABI,” “a program linked against libctos issued that ABI,” “the guest mapped PT_LOAD and ERETed to e_entry,” or “a loaded image stood at EL0 until exit with fail-closed restore,” or “the guest created, wrote, and read a named in-RAM file,” or “the guest programmed virtio-blk and read a known FAT16 file through the thin VFS,” or “the guest wrote FAT16 bytes through the thin VFS,” or “the guest listed FAT16 root directory entries through the thin VFS,” or “the guest loaded a separate app ELF from FAT /hello into user TTBR0,” or “the same published app ELF loaded on this OS and on ba6541c,” or “identity heap was unmapped,” or “standing/EL0 trampoline switches without TLBI VMALLE1,” or “taken lower-EL IRQ while standing returned to EL0,” or “default standing/task ERET enters with IRQ unmasked,” or “_start stays mapped while live image ranges are torn” only for the mile that actually passed.

It may not say “secure OS,” “hardened,” “EL0 works,” “the kernel moved,” “immutable OS,” Linux app host, POSIX host, or container host. It may say the freestanding product app-hosting claim is Verified under ADR-048 when the ledger product row is Verified (ADR-052). It may say “EL0 isolated” only as the accepted ADR-091 sentence, quoted verbatim or pointed to, with its limits (M6). Scoped RO is immutability.md. Unprobed stays Unknown. https://ctos.artof.link HTTPS is Verified (2026-09-11 after #30; see the honesty ledger).

Taken SError research + QMP attempt (v1.25)

Research (ADR-044); implementation attempt (ADR-045): guest A-clear standing path + host QMP inject-nmi. On QEMU 10 virt+-cpu cortex-a57, QMP returns machine does not provide NMIs (no ARM_CPU_SERROR via nmi; FEAT_NMI needs a different CPU/GIC). Park marker remains Verified.

Isolation leftovers decisions + app-hosting claim gate (v1.26)

ADR-047: taken SError = deferred/non-goal on this smoke machine (A-clear dormant prep); PAN enable = non-goal on default cortex-a57; never yank _start; leftover identity RAM torn via ADR-049 (ident: ram*); umbrella “EL0 isolated” stays Planned/non-claim. ADR-048 (at v1.26): A1–A9 first cuts ≠ product “app hosting done”; claim stayed Planned until criteria Met + sponsor accept.

Product app-hosting claim Verified (v1.29)

ADR-048 criteria 1–5 Met; sponsor accept ADR-052. Product freestanding “apps run independently / hosting done” is Verified. Still not Linux userspace, POSIX, glibc/musl, guest OCI/Docker/k8s, “EL0 isolated,” PAN enable, taken SError Verified, immutable-OS marketing, or OTA

Isolation-gap locks (v1.30)

ADR-053: taken lower-EL SError hard-stopped on QEMU 10 virt+cortex-a57 (dated 2026-09-14 re-probe: inject-nmi / HMP nmi → machine does not provide NMIs); park Verified; A-clear dormant prep. ADR-054: PAN enable locked non-goal on default cortex-a57; explicit reopen gate (sponsor CPU + FEAT_PAN + enable+fault ADR). ADR-055: umbrella “EL0 isolated” non-claim until checklist — Verified miles listed; unmet blockers include PAN enable, taken SError, never-yank _start. Still not Guest Linux / containers / immutable-OS marketing..

FAT16 readdir (v1.31)

ADR-056: guest lists FAT16 root dirents as thin-VFS paths (fat: readdir / fat: entries). Not POSIX getdents / opendir. Keep write / /hello / slot: ok. Still not Linux/containers/“EL0 isolated.”

FAT16 delete (v1.32)

ADR-057: guest deletes a FAT16 root file via thin-VFS unlink (fat: delete). Not POSIX unlink / remove. Keep write / readdir / /hello / slot: ok. Still not Linux/containers/“EL0 isolated.”

VFS prefix mounts (v1.33)

ADR-058: guest routes path prefixes through a thin VFS mount table (vfs: mount / vfs: mounts). Not POSIX mount(2) / Linux vfsmount. Keep memfs / FAT write / readdir / delete / /hello / slot: ok. Still not Linux/containers/“EL0 isolated.”

Freestanding sample catalog (v1.34)

Two freestanding EL0 samples on FAT: /hello (hello-libctos, UART+yield) and /fsdemo (fs-libctos, libctos VFS create/open/read/write/close on /memdemo). Catalog docs list both. Not POSIX. Not getdents. Not a reopen of product app hosting.

ADR-060: single sponsor-facing table for taken SError / PAN enable / umbrella “EL0 isolated” / never-yank _start — locks + reopen gates only. Cites ADR-047 / ADR-053 / ADR-054 / ADR-055 / #98. Does not invent Verified. Does not reopen Guest Linux / containers / immutable-OS marketing.

Freestanding FAT-via-VFS sample (v1.36)

Three freestanding EL0 samples on FAT: /hello, /fsdemo (memfs /memdemo), and /fatdemo (fat-libctos, open/read FAT /probe via thin VFS / → FAT16). Catalog docs list all three (ADR-061). Not POSIX. Not getdents. Not a reopen of product app hosting.

FAT16 multi-cluster grow (v1.39)

ADR-064: guest grows a FAT16 root file across a cluster boundary (fat: grow); EL0 fat-libctos chunked SVC write/read-back (libctos: fat-grow). Not POSIX write API. Not exFAT. Keep write / readdir / delete / /hello / slot: ok. Still not Linux/containers/“EL0 isolated.” Track N untouched.

FAT vs memfs read CNTPCT (v1.40)

ADR-065: same-boot CNTPCT pair around FAT /probe read vs memfs /mrprobe read (perf: fat-read / perf: memfs-read / perf: fs-read-delta). Measure-first lab ticks only — not a latency SLA, percent, or product KPI. Keep ADR-051 write pair / fat: ok / Track N untouched. Still not “secure.”

Freestanding cooperative-yield sample (v1.37)

ADR-062: fourth freestanding EL0 sample yield-libctos on FAT /yldemo. Several cooperative yield_now() rounds (libctos: yld-hi / libctos: beat / libctos: yld-ok). Not preemption. Not multi-task EL0. Not a process table. Does not reopen ADR-048/052.

Track N network foundation scope (v1.38)

ADR-063: new Track N (network), separate from A/B. N0 = docs scope. Non-goals locked: TCP/UDP/sockets/DHCP/DNS product, Wi‑Fi, virtio-pci-only foundation, Linux net stack, “has networking” marketing, EL0 net ABI before deeper net miles.

Track N virtio-net first frame (v1.41)

ADR-066: N1 discovers virtio-net, programs RX+TX, TX ARP for 10.0.2.2, RX SLIRP reply (net: virtio / net: mac / net: tx / net: rx / net: ok). Host -netdev user,id=net0 -device virtio-net-device,netdev=net0. I/O surface: virtio-blk and virtio-net are programmed DMA masters. Still not TCP/UDP/sockets/DHCP/DNS/Wi‑Fi/“has networking.”

Track N virtio-net ICMP ping (v1.42)

ADR-067: after N1 ARP, guest TX ICMP echo request to 10.0.2.2 and RX echo reply (net: icmp-tx / net: icmp-rx / net: ping-ok). QEMU args unchanged. Kernel-path only. Still not TCP/UDP/sockets/DHCP/DNS/Wi‑Fi/“has networking.”

Track N EL0 net SVC sample (v1.43)

ADR-068: SYS_NET_MAC (24) + SYS_NET_PING (25); freestanding net-libctos on FAT /netdemo (libctos: net-hi / libctos: net-mac / libctos: net-ok / netdemo: ok). Kernel still owns virtio-net. Still not TCP product / sockets / DHCP/DNS product / Wi‑Fi / “has networking.”

Net-ping CNTPCT (v1.44)

ADR-069: CNTPCT around EL0 net_ping quiet ARP+ICMP (perf: net-ping ticks=<n>). Measure-first lab ticks only — not a latency SLA, percent, or product KPI. Smoke greps the marker prefix (tick values vary). Keep N1/N2/N4 / FAT / slot. Still not TCP/UDP/sockets/“has networking.” Still not “secure.”

Track N UDP transport (v1.45)

ADR-070: after N2 ICMP, guest TX one UDP DNS query to SLIRP 10.0.2.3:53 and RX a matching DNS response (net: udp-tx / net: udp-rx / net: udp-ok). QEMU args unchanged. Kernel-path only this mile. DNS is probe bait, not a guest DNS product. Still not BSD sockets / TCP product / DHCP/DNS product / Wi‑Fi / “has networking / sockets OS.” Still not “secure.”

Track N EL0 UDP DNS SVC (v1.46)

ADR-071: SYS_NET_UDP_DNS (26); freestanding udp-libctos on FAT /udpdemo (libctos: udp-hi / libctos: udp-ok / udpdemo: ok). Quiet kernel-path ARP + UDP DNS to SLIRP 10.0.2.3:53 (same bait as ADR-070). Kernel still owns virtio-net. Still not BSD sockets / TCP product / DHCP/DNS product / Wi‑Fi / “has networking.” Still not “secure.”

UDP DNS path CNTPCT (v1.47)

ADR-072: CNTPCT around EL0 net_udp_dns quiet ARP+UDP DNS (perf: udp-dns ticks=<n>). Measure-first lab ticks only — not a latency SLA, percent, or product KPI. Smoke greps the marker prefix (tick values vary). Keep N1/N2/N3/N3.x/N4 / ADR-069 / FAT / slot. Still not TCP/sockets/DNS product/“has networking.” Still not “secure.”

FAT16 mkdir (v1.48)

ADR-073: guest creates a FAT16 root directory through the thin VFS (fat: mkdir); dirent + empty cluster with . / ... Not POSIX mkdir. Not a directory tree. Not exFAT. No new SVC this mile (EL0 follow-up). Keep write / readdir / delete / grow / /hello / slot: ok / Track N markers. Still not Linux/containers/“EL0 isolated.” Track N / TCP untouched.

Thin TCP N5 (v1.49)

ADR-074: after N3 UDP, guest active-opens TCP to QEMU guestfwd 10.0.2.4:7, sends one payload ctos-tcp\n, requires matching echo (net: tcp-syn / net: tcp-est / net: tcp-tx / net: tcp-rx / net: tcp-ok). QEMU -netdev gains guestfwd=tcp:10.0.2.4:7-cmd:…/tcp-echo-stdio.sh. Kernel-path N5. One connection; no listen/accept. ADR-076 adds EL0 net_tcp_echo (28) + FAT /tcpdemo quiet re-run of the same probe. Still not BSD sockets / TCP product / TLS/HTTP / DHCP/DNS product / Wi‑Fi / “has networking / sockets OS.” Still not “secure.”

FAT16 nested mkdir + empty rmdir (v1.52)

ADR-077: guest creates one nested FAT16 subdirectory (/fdir/nest) and removes an empty directory through the thin VFS (fat: nested / fat: rmdir). Path grammar: flat /name or one /parent/child; deeper → BadPath. Not POSIX mkdir/rmdir. Not arbitrary-depth trees. Not recursive rm. No new SVC this mile. Keep fat: mkdir / write / readdir / delete / grow / /hello / slot: ok / Track N markers. Still not Linux/containers/“EL0 isolated.” TCP CNTPCT untouched.

Thin TCP echo CNTPCT (v1.53)

ADR-078: CNTPCT around the quiet EL0 net_tcp_echo path (perf: tcp-echo ticks=<n>). Measure-first lab probe on QEMU TCG — not a latency SLA, not SPEC, not “has networking.” Chosen surface is EL0 SVC (same as ADR-069/072); kernel N5 net: tcp-ok stays transport proof only. Keep N1–N5 / N5.x / ADR-069/072 / FAT nested+rmdir / samples. Still not BSD sockets / listen-accept / TLS/HTTP. Still not “secure.”

PAN CPU reopen foundation (v1.54)

ADR-079 / #125: sponsor unlock + default smoke CPU -cpu cortex-a76 + Verified ID-field pan: present (ADR-054 gate items 1–2). Historical cortex-a57 pan: absent retained. Enable+fault is ADR-080 (v1.55). Does not unlock taken SError — confirmed blocked in ADR-081. Umbrella “EL0 isolated” stays non-claim. Never yank _start. Still not “secure.”

EL0 isolation (P-SEC-3 / ADR-013)

Isolation: “EL0 isolated” is Verified only as the accepted ADR-091 sentence (M6, 2026-09-28; quoted with its limits below). History (2026-09-26 audit + gap list: ADR-086; M1 identity inventory ratchet: ADR-087 ; M2 MMIO high alias: ADR-088 — only the _start stub page remains identity-mapped, the one documented exception per ADR-089; M4 taken-SError evidence class B1 pin + B2-P per ADR-090; M5 umbrella sentence drafted in ADR-091, accepted in writing by the sponsor 2026-09-28 (B3, M6); G1 EL0-reachability walk + G2 EL0 write fault closed: ADR-092; G4 syscall pointers need EL0 permission on every page, closed: ADR-094; ADR-091 revision 3 drops the direct-access-only carve-out, G3 optional only) (ADR-047 / ADR-055; sponsor closure ADR-060). A first mile, a user-TTBR0 read mile, an ASID TLB mile, a standing EL0 context, a TTBR1 private-page first cut, an EL1 high-VA fetch mile, an identity-tear first cut, an identity .text range tear, a live identity .text tear after a high-VA vtable rewrite, an identity .rodata tear, a PAN ID-field probe (pan: present on cortex-a76 / ADR-079) and PAN enable + EL1-vs-EL0 fault (pan: enabled / pan: el1-fault / ADR-080), a documented SVC ABI mile, a libctos CRT mile, a guest ELF PT_LOAD loader mile, a standing-task (normal mode) mile, a thin VFS + memfs mile, a virtio-blk + FAT16 mile, and an OS/app slot first cut exist. Those miles alone are not the product claim. A8 is documented rebuild recipes (what-can-run.md). A9 is two artifacts + FAT /hello (ADR-030); leftover cross-update / embed-off landed (ADR-032). Product “app hosting is done” is Verified under ADR-048 / ADR-052 (freestanding slots). Do not claim Linux userspace, POSIX or containers, and do not widen “EL0 isolated” past the ADR-091 sentence. Standing enter/leave as a guest sample: apps-today.md. CRT + loader + standing-as-normal: building-or-porting.md.

Direction: ADR-013. TTBR1 first cut: ADR-016. EL1 fetch mile: ADR-017. Identity-tear first cut: ADR-018. Identity .text range tear: ADR-019. Live .text tear: ADR-020. Identity .rodata tear: ADR-025. Identity .data tear: ADR-037. Identity heap tear: ADR-038. Leftover identity RAM tear: ADR-049. EL0 entry without VMALLE1: ADR-039. Lower-EL IRQ while standing: ADR-040. IRQ-unmasked default ERET: ADR-041. Isolation leftover wrap: ADR-042. Lower-EL FIQ + SError park: ADR-043. Taken SError QMP attempt: ADR-045. Isolation leftovers decisions: ADR-047. Taken SError hard-stop: ADR-053. Taken SError reopen (still blocked): ADR-081 / ADR-082. PAN enable lock: ADR-054. Umbrella checklist: ADR-055. Sponsor leftovers closure: ADR-060. App hosting claim criteria: ADR-048. PAN capability: ADR-026. SVC ABI: ADR-021 / syscall.md. CRT / libctos: ADR-022. Guest loader: ADR-023. Standing-as-normal: ADR-024. Thin VFS + memfs: ADR-027. Threat model: security.md. Code: src/el0.rs, src/syscall.rs, src/libctos.rs, src/loader.rs, src/vfs.rs, libctos/, src/asid.rs, src/ttbr1.rs, src/teardown.rs, src/pan.rs.

flowchart TD
  EL1["EL1 kernel<br/>TTBR · SVC · VFS"]
  EL0["Standing EL0 task<br/>freestanding image"]
  SVC["SVC ABI<br/>exit · uart · yield · fs_*"]
  EL0 -->|"asks"| SVC --> EL1
  EL1 -->|"ERET"| EL0

Standing EL0 + SVC is a Verified mile class. It is not a Linux process; the umbrella claim is only the ADR-091 sentence.

flowchart LR
  M["Verified miles<br/>standing · tears · IRQ/FIQ<br/>PAN ID-field · SError park"]
  L["Limits kept<br/>stock-TCG SError parks · never-yank _start<br/>one core · not side-channel"]
  U["Umbrella “EL0 isolated”<br/>Verified: ADR-091 sentence only<br/>ADR-055 checklist Met"]
  M -.-> U
  L -.-> U

Miles alone are not the umbrella. “EL0 isolated” means only the accepted ADR-091 sentence (M6), never wider.

What exists today

  • Kernel runs at EL1 (SPSel = 0).
  • One map-window page (paging::EL0_PAGE) can be UXN-clear / PXN for a trampoline or standing payload.
  • ERET to EL0 switches to a user TTBR0 (L1_USER, ASID=1) that maps kernel text/rodata + the exception stack + the trampoline window (every 2 MiB those ranges occupy), and omits .data / .bss / heap. The lower-EL handler restores kernel TTBR0 from TPIDR_EL1 before touching kernel data. After identity .data/heap tear, that path is MSR TTBR0 + ISB only — no TLBI VMALLE1 (ADR-039; serial el0: no-vmalle1).
  • Dual EL1 ASIDs (1 vs 2) with nG probe pages switch without TLBI VMALLE1 (src/asid.rs).
  • A bounded standing user context on that TTBR0: SVC #1 stays at EL0 (el0: standing), user MOVZ runs, SVC #2 restores EL1 (el0: restored). is_active() is true only for that lifetime. That dual-SVC is the ADR-013 probe.
  • A standing task (A4 / ADR-024) is the supported path for a loaded freestanding image: the A3 loader maps PT_LOAD, is_active() is true while it runs, yield / uart_write stay at EL0 (el0: task-active), exit restores (el0: task-exit / el0: task-restored). An unexpected lower-EL sync restores fail-closed (el0: restore-fail) instead of parking. That is the ctos process model (ADR-035) — not Linux fork / execve / wait.
  • TTBR1 walks are enabled. One kernel-private high page (TTBR1_PRIV) is EL1-only; EL0 load faults. Identity RAM is aliased at va + TTBR1_BASE via cloned RAM tables; EL1 can fetch a real path there and VBAR_EL1 is the high alias. After a high-VA jump, rustc vtables are rewritten to high aliases (ident: reloc) and live identity .text after the boot stub is unmapped (ident: live), plus the dedicated 16 KiB range (ident: range / ident: ok). Identity .rodata is unmapped after a pointer rewrite (ident: rodata / ident: rodata-fault / ident: rodata-high). Identity .data/stacks are unmapped after SP relocate (ident: data). Identity heap is unmapped after high GlobalAlloc VAs (ident: heap). _start / QEMU -kernel stay at 0x4008_0000 (ident: start-stay, ADR-042). Leftover identity frames after the heap are unmapped (ident: ram / ident: ram-fault / ident: ram-high, ADR-049); high twins stay. Never yank _start while -kernel needs that address (ADR-047). PAN ID field is printed (pan: present on -cpu cortex-a76, ADR-079; historical a57 pan: absent); PAN enable + EL1-vs-EL0 fault is Verified (ADR-080).
  • Lower-EL AArch64 sync handles SVC (ADR-013 probes #0/#1/#2 plus public ABI #16–#23), a kernel-data IABORT, a kernel-data DABORT, and the TTBR1 private-page DABORT, then returns to EL1t (or stays at EL0 on standing SVC #1 / SYS_YIELD / SYS_UART_WRITE / memfs SVCs). Lower-EL IRQ while standing handles the timer (or other) IRQ, then returns to EL0 (el0: irq, ADR-040); default standing/task ERET clears IRQ mask (el0: irq-default, ADR-041). Lower-EL FIQ while standing is taken when GICC FIQEn is armed (el0: fiq, ADR-043). ADR-045 wires an A-clear standing SError probe (el0: serror-arm / eret_to_el0_serror); QEMU 10 virt TCG (a57 and a76) has no working host inject (ADR-081 / ADR-082), so hello keeps el0: serror-park. Short non-standing trampoline probes stay IRQ-masked. Default standing SPSR still masks FIQ and SError except the dedicated probes.
  • Public SVC ABI (syscall.md): exit / uart_write / yield plus memfs fs_* (19–23). A libctos CRT wraps those numbers; a hello payload is copied onto the standing EL0 page (A2) and also parsed as ELF64 PT_LOAD on the guest (A3). A4 runs that loaded image as a standing task until exit. A6 is in-RAM memfs. ABI + CRT + loader + standing-task + memfs miles. App hosting Planned.
  • Syscall pointers (ADR-094, G4 closed). Every pointer the kernel copies from EL0 must be EL0-readable (AP[1]=1) on every page. Every pointer it copies to EL0 must be EL0-writable (AP[1]=1, AP[2]=0). Execute-only, kernel-only and unmapped pages return the syscall’s error value, and no byte moves. The loader maps app text+rodata (one R+X page) EL0 read-only + executable (map_el0_text: AP=11, UXN clear, PXN set), so apps can pass their own string literals. Kernel-built probe payloads keep execute-only code and put their data on an EL0-RW NX page. Smoke: el0: sys-ptr denied {kernel-stub,kernel-window,xo-text,write-ro,straddle}, a seven-syscall sweep, el0: sys-ptr ok, and a sysptr-leak-probe build that must print el0: sys-ptr leaked …. ADR-097 (proposed): app text is EL0 execute-only, so a pointer into it now fails the copy-from-EL0 check (xo: sys-read text denied).

Probed miles

ProbeWhat closes itHonesty
EL0 entered and returnedSerial el0: svc + el0: ok; #[test_case] el0_svc_roundtripEntered and left. Not a user process.
EL0 cannot execute kernel dataSerial el0: nx kernel; #[test_case] el0_cannot_execute_kernel_dataIABORT on .data (UXN and/or unmapped). Not isolation.
User TTBR0 omits kernel .dataWalk L1_USER; #[test_case] user_ttbr0_omits_kernel_dataDistinct user table. Text is still mapped so the handler can run.
EL0 cannot read kernel .dataSerial el0: no kernel read; #[test_case] el0_cannot_read_kernel_dataTranslation/permission DABORT. Not PAN.
Standing EL0 contextSerial el0: standing / el0: restored; #[test_case] standing_el0_enter_leaveReal enter/leave on user TTBR0. is_active() true only while standing. Not POSIX. Default ERET clears IRQ mask (ADR-041).
Standing EL0 as normal modeSerial el0: task-enter / el0: task-active / el0: task-exit / el0: task-restored / el0: restore-fail / el0: task-ok; #[test_case] standing_task_runs_until_exit + standing_task_fault_restoresLoaded A3 image until exit. Fail-closed restore. Not POSIX. Not isolation.
ASID field programmeduser_ttbr0() >> 48 == 1Programming fact on the EL0 trampoline. Not the isolation mile.
ASID isolationSerial asid: dual / asid: conflict / asid: ok; #[test_case] asid_isolation_without_vmalle1Dual TTBR0 without TLBI VMALLE1. Stale ASID-1 data under ASID 2 is Failed. One mile; alone it is not the ADR-091 umbrella claim.
TTBR1 private pageSerial ttbr1: el1 / ttbr1: no el0 / ttbr1: ok; #[test_case] ttbr1_el1_sees_priv_el0_does_notEL1-only high page. Not a relocated kernel.
EL1 fetch from TTBR1 high VASerial ttbr1: el1 exec / ttbr1: vbar; #[test_case] el1_executes_from_ttbr1_high_vaReal EL1 path + high VBAR. Identity boot stub stays.
Identity-tear first cutSerial ident: split / ident: fault / ident: high / ident: no el0 / ident: ok; #[test_case] identity_tear_el1_faults_high_staysOne identity text page unmapped; high twin still fetches. Not a relocated kernel. Full teardown Planned.
Identity text range tearSerial ident: jump / ident: range / ident: text; #[test_case] identity_text_range_unmapped_boot_stub_staysHigh-VA continuation + 16 KiB dedicated range unmapped; high twins still fetch. Not a relocated kernel.
High-VA vtable rewrite + live .text tearSerial ident: reloc / ident: live; #[test_case] identity_fn_ptrs_rewritten_high + live_identity_text_unmapped_boot_stub_staysrustc dyn Write / fmt tables patched to high aliases; live identity .text after _start unmapped; println! still runs. Not a relocated kernel.
Identity .rodata tearSerial ident: ro-reloc / ident: rodata / ident: rodata-fault / ident: rodata-high; #[test_case] identity_rodata_unmapped_high_staysIdentity .rodata unmapped; high twin still loads. Not a relocated kernel.
Identity .data / stack tearSerial ident: data-reloc / ident: data / ident: data-fault / ident: data-high; #[test_case] identity_data_torn_high_staysIdentity .data/stacks unmapped; SP high-only (ADR-037). Not a relocated kernel.
Identity heap tearSerial ident: heap-reloc / ident: heap / ident: heap-fault / ident: heap-high; #[test_case] identity_heap_torn_high_staysIdentity heap unmapped; GlobalAlloc returns high VAs (ADR-038). Not a relocated kernel.
EL0 entry without full TLBISerial el0: no-vmalle1; #[test_case] el0_entry_without_vmalle1; src/exception.rs has no tlbi vmalle1Standing/EL0 trampoline is MSR TTBR0 + ISB only after ADR-037/038 (ADR-039). One mile; alone it is not the ADR-091 umbrella claim.
Lower-EL IRQ while standingSerial el0: irq; #[test_case] lower_el_irq_while_standingTimer IRQ taken from standing EL0 and returned (ADR-040). FIQ/SError still park. One mile; alone it is not the ADR-091 umbrella claim.
IRQ-unmasked default ERETSerial el0: irq-default; #[test_case] lower_el_irq_default_eretStanding/task default ERET clears I (ADR-041). Short non-standing probes stay masked. One mile; alone it is not the ADR-091 umbrella claim.
Lower-EL FIQ while standingSerial el0: fiq; #[test_case] lower_el_fiq_while_standingTimer as FIQ via temporary GICC FIQEn (ADR-043). Default SPSR still masks F. One mile; alone it is not the ADR-091 umbrella claim.
SError park honestySerial el0: serror-parkNo working SError inject on virt TCG (ADR-043 / ADR-045 / ADR-081 / ADR-082). Taken path deferred/non-goal (ADR-047). One mile; alone it is not the ADR-091 umbrella claim.
Taken SError QMP attemptSerial el0: serror-arm + host QMP log; taken el0: serror only if inject worksGuest A-clear path wired (ADR-045). QMP inject-nmi fails on QEMU 10 virt+a76 (ADR-081). No #[test_case]. One mile; alone it is not the ADR-091 umbrella claim.
Identity _start staySerial ident: start-stay; #[test_case] identity_boot_stub_stays_while_live_tornHonesty (ADR-042 / ADR-047): stub stays while live image + heap + leftover RAM are torn. One mile; alone it is not the ADR-091 umbrella claim.
Leftover identity RAM tearSerial ident: ram / ident: ram-fault / ident: ram-high; #[test_case] identity_ram_torn_high_staysOptional mile taken (ADR-049). High twin stays. Not a relocated kernel. One mile; alone it is not the ADR-091 umbrella claim.
PAN ID fieldSerial pan: id= / pan: present; #[test_case] pan_present_on_probe_cpuID_AA64MMFR1_EL1.PAN != 0 on -cpu cortex-a76 (ADR-079). Historical a57 pan: absent.
PAN enable + EL1-vs-EL0 faultSerial pan: enabled / pan: el1-fault; #[test_case] pan_enable_el1_fault_on_probe_cpuPSTATE.PAN set; EL1 load of EL0-accessible page faults (ADR-080). One mile; alone it is not the ADR-091 umbrella claim.
SVC ABI (exit / uart_write / yield)Serial svc: yield / svc: user-hi / svc: uart / svc: exit / svc: ok; #[test_case] el0_svc_abi_yield_uart_exit + kernel-.data / TTBR1-alias rejectDocumented numbers 16–18. Not Linux. Not app hosting.
libctos CRTSerial libctos: hi / libctos: ok / libctos: linked; #[test_case] libctos_hello_yield_uart_exitLinked wrappers, not a hand-encoded trampoline. Not app hosting.
Guest ELF PT_LOAD loaderSerial loader: mapped / loader: ok; #[test_case] loader_maps_hello_and_eretsGuest parse + map into user TTBR0 + ERET to e_entry. Not a Linux ABI. Not app hosting.
Thin VFS + memfsSerial fs: create / fs: write / fs: read / fs: el0 / fs: ok; #[test_case] vfs_memfs_create_write_read_close + el0_fs_svc_roundtripIn-RAM named buffers. Not POSIX. Not FAT. Not app hosting.
OS/app slot first cutSerial slot: fat / slot: mapped / slot: ok; #[test_case] slot_load_from_fat_eretsA9 (ADR-030). FAT /hello + A3 map. A2–A4 load FAT (no production embed, ADR-032). Product freestanding hosting Verified under ADR-048/052 (this mile alone ≠ Linux hosting).

Still Planned / decided leftovers (isolation) — ADR-047 / locks ADR-053–ADR-055 / sponsor closure ADR-060

ItemStatusNotes
PAN enableVerified (ADR-080)One clause of the ADR-091 umbrella sentence, not the umbrella by itself (ADR-055).
Yank _startDecided: never while QEMU -kernel needs 0x4008_0000ident: start-stay Verified honesty (ADR-042).
Leftover identity RAM after heapVerified (optional mile)ident: ram* (ADR-049). Not required for the umbrella row.
Taken lower-EL SError on this smoke machineDeferred / non-goal (hard-stopped)Park Verified; A-clear dormant prep (ADR-045). Hard-stop ADR-053; reopen re-probe ADR-081 / ADR-082. Unlock = B1/B2/B3; B1 research ADR-082 (no shippable pin 2026-09-19). Do not claim taken on stock QEMU; the umbrella clause rests on the ADR-090 evidence class (B1 pin + B2-P).
Umbrella “EL0 isolated”Verified (scoped: the accepted ADR-091 sentence only; M6, 2026-09-28; revision 4 accepted 2026-10-02)ADR-091 Accepted (B3 met 2026-09-28 16:41 CEST); checklist ADR-055 §A/§B all Met; sponsor closure ADR-060. Limits: one core, the TCG default smoke machine; not a speculative-execution or side-channel claim, not a real-hardware claim, not a certification; app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN); no multi-core race story. Specific miles alone are not this row.
Product app hosting “done”VerifiedADR-048 Met×5 / ADR-052. Freestanding slots only — not Linux/POSIX/containers.

Unprobed stays Unknown. Do not say “EL0 works,” “Linux app host,” or “the kernel moved.” Say “EL0 isolated” only as the accepted ADR-091 sentence (below). The freestanding ADR-048 product claim is Verified separately.

Taken SError research + QMP attempt + hard-stop + reopen (ADR-044 / ADR-045 / ADR-047 / ADR-053 / ADR-081)

Taken lower-EL SError on this smoke machine is deferred / non-goal (hard-stopped) (ADR-053 / ADR-081 / ADR-082 / ADR-047). Research (ADR-044) recommended QMP/nmi; ADR-045 wired the guest A-clear path and attempted QMP — QEMU 10 virt reports machine does not provide NMIs. Re-probed 2026-09-14 (a57) and 2026-09-19 (default a76 + GICv3/virtualization=on/max): same error. Unlock = ADR-081 B1/B2/B3; B1 research ADR-082 (no shippable pin 2026-09-19). el0: serror-park stays Verified; A-clear stays dormant prep. Do not claim taken SError Verified on stock QEMU. 2026-09-26 (sponsor D1, ADR-090): the umbrella’s taken-SError evidence class is the B1 pinned-QEMU CI job qemu-nmi-pin (unconditional, fail-closed bare el0: serror) plus the B2-P one-off Graviton3 KVM run (ADR-085 run 4). ADR-055 §B row 1 is Met on that class; stock TCG still parks.

Umbrella “EL0 isolated” accepted (ADR-091, M6)

Since M6 (2026-09-28) umbrella “EL0 isolated” is Verified, and only as the sentence the sponsor accepted in writing in ADR-091 (B3, 2026-09-28 16:41 CEST). Revision 4 (2026-10-02): the sponsor accepted ADR-091 rev 4 in writing (“I accept ADR-091 rev 4”); only the PAN clause and the code-bytes limit changed. Verbatim, revision 4:

“On the ctos default smoke machine (QEMU virt, -cpu cortex-a76, TCG, one core), EL0 is isolated from the kernel at the architectural page-table / exception-level boundary: each EL0 task runs on its own TTBR0 with its own ASID; EL0 faults when it reads, writes or executes kernel memory; in every translation table (kernel, user and ASID-B TTBR0, and TTBR1) the only EL0-reachable pages are five allowlisted user slots backed by non-kernel frames, checked fail-closed while a user app runs and at rest; PAN is enabled, including while the kernel handles exceptions from EL0, and EL1 faults on EL0-readable memory (the execute-only app text page is not covered: this CPU has no FEAT_EPAN); lower-EL IRQ and FIQ are taken while EL0 stands; taken lower-EL SError is evidenced by the ADR-090 evidence class; the only identity (VA = PA) mapping left in any TTBR0 is the documented EL1-only _start stub page (ADR-089); and a system call copies from or to an EL0 pointer only if EL0 itself may read (for a copy from EL0) or write (for a copy to EL0) every page in the range. It is not a speculative-execution or side-channel claim, not a real-hardware claim, and not a certification.”

Limits that stay with it:

  • One core. No multi-core / SMP race story (no SMP TLB-shootdown story; syscall pointer checks are a one-core check-then-copy, ADR-094).
  • The TCG default smoke machine only. Not a real-hardware claim (the B2-P KVM runs are a reduced one-off profile, ADR-085 / ADR-095).
  • Not a speculative-execution or side-channel claim. Not a certification.
  • app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN). ADR-097 moves rodata to the app-hdr page and maps app text EL0 execute-only (still five slots); accepted as ADR-091 revision 4 on 2026-10-02. Because this CPU has no FEAT_EPAN, PAN does not stop EL1 from reading that page, which is why the revision-4 PAN clause says the execute-only app text page is not covered. History: before 2026-10-02 the limit read “EL0 can read its own code bytes” (app text+rodata EL0 read-only + executable, ADR-094). The PAN-held fix is ADR-098.
  • Also outside it (ADR-091 non-scope): task-vs-task isolation beyond ASIDs, DMA / IOMMU / EL2, and taken SError on stock QEMU (still parks). Separate evidence (not part of the sentence): ADR-096 probes that an EL0 task on ASID 1 cannot read or write another EL0 task’s page on ASID 2 at the same VA (t2t: ok; one core, TCG, kernel-built tasks).

Correction (2026-09-29): until ADR-098 the “PAN is enabled” clause held at boot but not while the kernel handled EL0 exceptions or after EL0 trips (the boot probe ran before any EL0 trip). ADR-098 fixes it and checks it live; the sentence was unchanged and true again. Update 2026-10-02: revision 4 now says “including while the kernel handles exceptions from EL0” in the PAN clause itself.

Checklist: ADR-055 (§A and §B all Met). Sponsor closure: ADR-060. Do not round single miles into the umbrella, and do not widen the sentence.

Before M6 (kept for history): Umbrella isolation stays Planned / non-claim until checklist. Verified miles vs unmet requirements: ADR-055. Sponsor-facing closure table (locks + reopen gates only): ADR-060. Unmet blockers include taken SError (ADR-053 / ADR-081 / ADR-082), never-yank _start, and umbrella sponsor accept (PAN enable is Met separately via ADR-080). Do not round Verified miles into “EL0 isolated.”

SVC syscall ABI (Track A / A1 / ADR-021)

ABI + CRT + loader + standing-task + memfs + FAT16 + slot first-cut miles. Sample rebuild recipes are A8 (docs). A9 is two artifacts + FAT /hello (ADR-030) plus leftover cross-update / FAT-only hello (ADR-032). Product “app hosting is done” is Verified under ADR-048 / ADR-052 (freestanding slots). Not Linux. Not POSIX.

Kernel contract: ADR-021. CRT / libctos: ADR-022. Guest loader: ADR-023. Standing-as-normal: ADR-024. Thin VFS + memfs: ADR-027. virtio-blk + FAT16: ADR-028. Parent plan: issue #31. A1: issue #32. A2: issue #33. A3: issue #34. A4: issue #35. A6: issue #37. Code: src/syscall.rs, src/vfs.rs, libctos/, user/hello-libctos/, src/loader.rs, src/el0.rs.

Calling convention

AArch64 SVC #<n> where n is the syscall number. Arguments in x0, x1, x2. Return in x0.

Reserved 0–2 are ADR-013 probes (#0 first-mile return, #1 standing, #2 restore). libctos must not issue them. Public numbers below are the freestanding contract; Linux ABI is out of scope.

Public numbers

NumberNameArgsEffect
16exitx0 = statusHalt the EL0 trip; return to the EL1 caller. Status is recorded. Not a process table.
17uart_writex0 = pointer, x1 = lengthWrite up to 64 user-mapped and kernel-mapped bytes to the virt PL011. \n → \r\n. Reject (including non-canonical / TTBR1 aliases) → x0 = 0.
18yieldnoneDispatch-only hint; return to EL0. Does not switch EL1 tasks (ADR-010).
19fs_createx0 = path, x1 = lengthCreate an empty memfs name and a handle. Path / + [a-z0-9_-].
20fs_openx0 = path, x1 = lengthOpen an existing memfs name.
21fs_readx0 = fd, x1 = buf, x2 = lengthCopy from the handle offset into a user-mapped and kernel-mapped range (cap 64).
22fs_writex0 = fd, x1 = buf, x2 = lengthCopy into the file at the handle offset (cap 64, file cap 256).
23fs_closex0 = fdDrop the handle. The file stays.
24net_macx0 = buf, x1 = length (≥6)Copy guest virtio-net MAC (6 bytes). Kernel owns the NIC (ADR-068).
25net_pingnoneKernel-path ARP + ICMP echo to SLIRP 10.0.2.2. Not sockets.
26net_udp_dnsnoneKernel-path ARP + UDP DNS probe vs SLIRP 10.0.2.3:53 (ADR-071). DNS is probe bait only. Not sockets. Not a DNS product.
27fs_mkdirx0 = path, x1 = lengthCreate a FAT16 root directory via thin VFS (ADR-075). Memfs / nested paths fail closed. Not POSIX mkdir.
28net_tcp_echononeKernel-path thin TCP echo vs QEMU guestfwd 10.0.2.4:7 (ADR-076). Same probe as ADR-074, quiet. Not sockets. Not listen/accept.

Unknown SVC immediates park (fail-closed). Not POSIX. Not Linux VFS.

Track B B2 maps these numbers against Linux AArch64 (svc #0, x8): linux-aarch64-syscall-gap.md. Track B B4 maps ELF / auxv / PT_INTERP against the freestanding loader: ADR-033. Inspection only. Not claiming Linux userspace. Not claiming dynamic Linux ELF. Do not retarget 16–28 to Linux x8.

Probe

Serial svc: yield / svc: user-hi / svc: uart / svc: exit / svc: ok. #[test_case] el0_svc_abi_yield_uart_exit + el0_uart_write_rejects_kernel_data + el0_uart_write_rejects_high_alias. File presence is not that probe.

libctos (A2)

A no_std crate wraps the three public numbers. A hello payload linked against it is copied onto the standing EL0 page (A2) and parsed as ELF64 PT_LOAD on the guest (A3). Serial libctos: hi / libctos: ok / libctos: linked. File presence is not that probe.

Guest loader (A3)

The kernel parses the hello ELF, maps each PT_LOAD into the user map-window, and ERETs to e_entry (ADR-023). Serial loader: mapped / loader: ok. Not a Linux ELF ABI. Not PT_INTERP. A2–A4 and A9 read FAT /hello. Linux ELF/auxv gaps: ADR-033. File presence is not that probe.

Standing task (A4)

The A3 loader is how a payload appears. A4 makes standing EL0 the supported path: is_active() is true for the loaded task until exit; an unexpected fault restores fail-closed (ADR-024). Serial el0: task-enter / el0: task-active / el0: task-exit / el0: task-restored / el0: restore-fail / el0: task-ok. Not isolation. Not a process table. Linux fork/exec/wait is a documented gap (ADR-035). File presence is not that probe.

memfs (A6)

In-RAM named buffers behind a thin VFS (ADR-027, filesystem.md). Serial fs: create / fs: write / fs: read / fs: el0 / fs: ok. File presence is not that probe.

FAT16 (A7)

Same VFS open on virtio-blk (ADR-028). Serial blk: ok / fat: ok. /probe is the known FAT16 file. Write depth: ADR-050 (fat: write / fat: create). Root listing: ADR-056 (fat: readdir / fat: entries) — kernel/VFS probe, no new SVC. Root delete: ADR-057 (fat: delete) — kernel/VFS probe, no new SVC. Root mkdir kernel probe: ADR-073 (fat: mkdir). EL0 fs_mkdir (27): ADR-075. EL0 net_tcp_echo (28): ADR-076. Not POSIX getdents / unlink / mkdir. File presence is not that probe.

Sample recipes (A8)

Rebuild recipes for the probed classes: what-can-run.md, in-tree user/README.md. Coop UART, RX echo, standing EL0 / libctos hello; optional memfs + FAT16. N4 adds 24–25 (ADR-068). N3.x adds 26 (ADR-071). ADR-075 adds 27 (fs_mkdir). ADR-076 adds 28 (net_tcp_echo). File presence is not a new runtime.

OS/app slots (A9)

Host kernel ELF + published hello-libctos.elf. Guest reads FAT /hello through the same VFS and maps it with the A3 loader (ADR-030). Serial slot: fat / slot: mapped / slot: ok / perf: app-load. No production embed on this path (ADR-032): A2–A4 load the same FAT file. Cross-update (same published ELF on this OS and ba6541c) is ledger-Verified. Probe-only include_bytes! in src/slot.rs is ADR-046 measurement only — never print slot: embed. File presence is not that probe.

Product claim + remaining non-goals (Track A)

Product “app hosting is done” is Verified under ADR-048 / ADR-052 for freestanding OS/app slots. Isolation enable (PAN) is non-goal on default cortex-a57 (ADR-047). Identity .data/heap tears are Verified (ADR-037/038). Still not Linux/POSIX/containers; “EL0 isolated” only as the ADR-091 sentence.

Performance (NFR-07 / ADR-011)

Performance is a first-class pillar, not a Later learning-only note. That does not authorize fake benches.

Rules

  1. A latency, cycle, or “faster than” sentence is a claim. It needs a probe in the honesty ledger.
  2. Optimize only after a probe shows a cost. Do not rewrite the scheduler or heap “for speed” on a hunch.
  3. QEMU virt numbers are one environment. They are not a Raspberry Pi probe and not a published SPEC run.

Baseline probe (this tree)

CNTPCT_EL0 is already used to timeout the M5 timer observe window. The pillars ratchet (when landed) measures a fixed trivial loop:

  • Serial marker perf: cntpct delta=<n> (fail closed on perf: probe missed).
  • #[test_case] asserts the counter advanced.

That probe proves the physical counter is readable and moves. It does not claim a microsecond budget, interrupt latency, or a comparison to other kernels.

IRQ-to-handler probe (this tree)

When CNTP fires, the handler records CNTPCT − CNTP_CVAL before rearm. After several ticks the hello kernel prints:

  • Serial marker perf: irq-delta min=<a> max=<b> spread=<b-a> n=<n> (fail closed on perf: irq-delta missed).
  • #[test_case] asserts samples exist and max >= min.

That is a spread of IRQ-to-handler counter deltas on this QEMU virt guest. It is not a latency budget, not “faster than X,” and not a published bench. QEMU TCG jitter is one environment.

Host debug ELF size (NFR-08, this tree)

scripts/qemu-smoke.sh prints the host byte size of target/aarch64-ctos/debug/ctos after cargo build:

  • Host marker perf: elf-size bytes=<n> (fail closed if missing or < 4096).
  • This is a measurement, not a size budget and not a “smaller is better” claim.

It does not time QEMU boot.

Boot-to-ready CNTPCT (NFR-08, this tree)

kernel_main samples CNTPCT_EL0 after paging::init (MMU + D-cache on) and again after Hello World! (init complete):

  • Serial marker perf: boot-delta ticks=<n> (fail closed on perf: boot-delta missed).
  • #[test_case] asserts a sample exists and the counter advanced.

That is kernel_main-entry to after-init on this QEMU virt guest. It is not a latency budget, not QEMU startup time, not a published bench, and not criterion.

OS/app slot disconnect (A9) — compared pair (QEMU TCG lab)

A9 #48 / ADR-030 loads a separate app payload from FAT /hello after the OS image (immutability.md, site measure.md). The load path prints perf: app-load ticks=<n>. ADR-046 adds a probe-only linked-in trip of the same hello-libctos.elf bytes (include_bytes! in src/slot.rs only) that prints perf: embed-load ticks=<n>, then an honest pair perf: slot-delta app=<a> embed=<b>. Production A9 stays FAT-only (slot: embed must never print). That is a measurement pair, not a bench and not a percent.

ClassWhat we measureHonesty
FAT pathVFS/FAT read + ELF parse + user map + ERETperf: app-load ticks=<n>
Probe-only embedSame bytes already in the kernel image; parse + map + ERET (no FAT)perf: embed-load ticks=<n> — not slot: embed
PairRaw tick counts on one bootperf: slot-delta app=<a> embed=<b>. No “faster/slower.” No invented percent. QEMU TCG jitter.
OperationalSmaller OS updates without rebuilding appsOperational ≠ measured latency. A2–A4 stay FAT-only (ADR-032).
GateKeep perf: boot-delta. Fail closed on missing perf: app-load / perf: embed-load / perf: slot-deltaVerified only when the ledger has serial evidence on a named tip.

QEMU TCG jitter is still one lab. No criterion crate. No “slot disconnect is free.”

FAT16 vs memfs write (ADR-051) — compared pair (QEMU TCG lab)

ADR-050 writes a small payload through the thin VFS onto the FAT16 volume. ADR-051 samples CNTPCT around that single vfs::write (perf: fat-write ticks=<n>), then on the same boot times a memfs vfs::write of the same bytes (perf: memfs-write ticks=<n>), and prints perf: fs-write-delta fat=<a> memfs=<b>. That is a measurement pair, not a bench and not a percent.

ClassWhat we measureHonesty
FAT writeOne VFS write of the /probe rewrite payload (fat-wr)perf: fat-write ticks=<n>
Memfs writeOne VFS write of the same bytes on /mwprobeperf: memfs-write ticks=<n>
PairRaw tick counts on one bootperf: fs-write-delta fat=<a> memfs=<b>. No “faster/slower.” No invented percent. QEMU TCG jitter.
GateKeep fat: write / fat: ok / A9 markers. Fail closed on missing pair markersVerified only when the ledger has serial evidence on a named tip.

QEMU TCG jitter is still one lab. No criterion crate. No latency SLA for FAT write.

FAT16 vs memfs read (ADR-065) — compared pair (QEMU TCG lab)

The guest already reads known FAT16 /probe bytes through the thin VFS. ADR-065 samples CNTPCT around that single vfs::read (perf: fat-read ticks=<n>), then on the same boot seeds memfs /mrprobe with the same bytes and times vfs::read (perf: memfs-read ticks=<n>), and prints perf: fs-read-delta fat=<a> memfs=<b>. That is a measurement pair, not a bench and not a percent. Symmetric to ADR-051 (write).

ClassWhat we measureHonesty
FAT readOne VFS read of /probe (fat-hi)perf: fat-read ticks=<n>
Memfs readOne VFS read of the same bytes on /mrprobeperf: memfs-read ticks=<n>
PairRaw tick counts on one bootperf: fs-read-delta fat=<a> memfs=<b>. No “faster/slower.” No invented percent. QEMU TCG jitter.
GateKeep fat: read / fat: ok / ADR-051 write pair / A9 markers. Fail closed on missing pair markersVerified only when the ledger has serial evidence on a named tip.

QEMU TCG jitter is still one lab. No criterion crate. No latency SLA for FAT read.

Net-ping path CNTPCT (ADR-069) — single path (QEMU TCG lab)

ADR-068 EL0 net_ping (SVC 25) already runs a quiet kernel-owned ARP + ICMP echo vs SLIRP 10.0.2.2. ADR-069 samples CNTPCT around that full quiet path and prints perf: net-ping ticks=<n>. That is a measurement, not a bench and not a latency SLA. Related optional N2 crumbs (perf: net-icmp-tx / perf: net-icmp-rx) stay as ADR-067 lab notes.

ClassWhat we measureHonesty
EL0 net_pingQuiet ARP + ICMP echo RTT as SYS_NET_PING / el0_icmp_pingperf: net-ping ticks=<n>
GateSmoke greps the prefix perf: net-ping (not an exact tick count). Reject faster/slower/percent. Keep N1/N2/N4 / FAT / slot markers.Verified only when the ledger has serial evidence on a named tip.

QEMU TCG jitter is still one lab. Tick values vary — do not invent a budget. No criterion crate. No sockets / TCP/UDP this mile.

UDP DNS path CNTPCT (ADR-072) — single path (QEMU TCG lab)

ADR-071 EL0 net_udp_dns (SVC 26) already runs a quiet kernel-owned ARP + UDP DNS query vs SLIRP 10.0.2.3:53. ADR-072 samples CNTPCT around that full quiet path and prints perf: udp-dns ticks=<n>. That is a measurement, not a bench and not a latency SLA. Same pattern as ADR-069 on net_ping — EL0 SVC surface, not a separate kernel-only N3 timer.

ClassWhat we measureHonesty
EL0 net_udp_dnsQuiet ARP + UDP DNS RTT as SYS_NET_UDP_DNS / el0_udp_dnsperf: udp-dns ticks=<n>
GateSmoke greps the prefix perf: udp-dns (not an exact tick count). Reject faster/slower/percent. Keep N1/N2/N3/N3.x/N4 / ADR-069 / FAT / slot markers.Verified only when the ledger has serial evidence on a named tip.

QEMU TCG jitter is still one lab. Tick values vary — do not invent a budget. No criterion crate. No TCP / FAT mkdir this mile.

Thin TCP echo path CNTPCT (ADR-078) — single path (QEMU TCG lab)

ADR-076 EL0 net_tcp_echo (SVC 28) already runs a quiet kernel-owned ARP + thin TCP active-open + one payload vs QEMU guestfwd 10.0.2.4:7. ADR-078 samples CNTPCT around that full quiet path and prints perf: tcp-echo ticks=<n>. That is a measurement, not a bench and not a latency SLA. Same pattern as ADR-069 / ADR-072 — EL0 SVC surface, not a separate kernel-only N5 timer. Kernel net: tcp-ok markers stay as ADR-074 transport proof.

ClassWhat we measureHonesty
EL0 net_tcp_echoQuiet ARP + thin TCP guestfwd RTT as SYS_NET_TCP_ECHO / el0_tcp_echoperf: tcp-echo ticks=<n>
GateSmoke greps the prefix perf: tcp-echo (not an exact tick count). Reject faster/slower/percent. Keep N1–N5 / N5.x / ADR-069/072 / FAT / slot markers.Verified only when the ledger has serial evidence on a named tip.

QEMU TCG jitter is still one lab. Tick values vary — do not invent a budget. No criterion crate. No listen/accept / sockets / TLS this mile.

Later probes (Planned)

  • A tighter “first instruction of _start” sample if someone maps a .data slot that BSS-clear will not wipe.

Do not add a host criterion crate or a “bench.yml” that prints invented numbers.

Overview (plain English)

Core principles drive ctos. Track A / Track B / A9 are subordinate and never override this list for speed or marketing. Full list: principles.md.

  • Honesty ledger — Verified only with a probe
  • Antifragility — fail-closed sensors + ratchets
  • Security — threat model; probed mitigations only
  • Performance — measure first; no invented benches
  • Document-first / one milestone → one PR

ctos is a learning AArch64 kernel for QEMU virt. It is not a desktop, not POSIX, and not a “secure OS.” Status words need a probe in the honesty ledger. This note does not invent latency, size, or “faster than” numbers. Measured markers live in the ledger; they are one environment each.

Vision: product-vision.md. Pillars: pillars.md. Frozen IDs: fr-nfr.md. Samples: apps-today.md. Porting: building-or-porting.md. Immutability: immutability.md (scoped only). Tracks: A / B / N (subordinate; N1 ADR-066, N2 ADR-067, N3 ADR-070, N3.x ADR-071, N4 ADR-068, N5 ADR-074).

Site chapters (source of truth for the published book): Overview. HTTPS at https://ctos.artof.link is Verified (2026-09-11 after #30). See website.md.

KPIs (what we actually measure)

These are sensors, not product SLOs. A missing marker is a fail. A printed number is not a budget.

Performance (NFR-07 / performance.md)

What we watchProbe (do not invent a target)
Physical counter movesSerial perf: cntpct delta=<n> + #[test_case]
IRQ-to-handler spread on this virt guestSerial perf: irq-delta min=… max=… spread=… n=…
Debug ELF byte sizeHost perf: elf-size bytes=<n> (measurement, not “smaller is better”)
kernel_main after-MMU → after-initSerial perf: boot-delta ticks=<n>
App-load after OS/app split (A9)Verified when ledger has perf: app-load (+ ADR-046 / ADR-051 pairs). Not a published bench.

QEMU TCG jitter is one lab. These are not Raspberry Pi numbers and not a published bench. Read the ledger row for the SHA you care about. A9 expected costs (boot/load, SVC, ASID/TTBR, later COW) vs steady EL0 compute: performance.md. No Verified delta — still one ELF.

Antifragility (NFR-05 / antifragility.md)

What we watchProbe
Hello + milestone strings still printscripts/qemu-smoke.sh greps; missing string → fail
Tests stay fail-closedcargo test exit 0; --features force-fail must be non-zero
Same failure twiceBecomes a sensor/gate, not another README paragraph
Failed history keptExample: Docker b2bbb99 Failed, then layout fix Verified — the Failed row stays

Unprobed boot stays Unknown. File presence is not QEMU boot.

Application-support scope

Do not say “applications run on ctos.” First-class samples and the cannot-run list live in apps-today.md. Porting stance: building-or-porting.md.

  • Can run (probed): coop EL1 UART workers (sched: task a/b/ok); one-byte UART RX (input: rx 0x41); standing EL0 stub (el0: standing / el0: restored); a loaded libctos hello as a standing task until exit (el0: task-ok). A heartbeat/counter variant is the same shape — not in tree until a probe greps it.
  • Cannot run: Linux ELF, shell, Python, network servers (Track N is ARP+ICMP+UDP + tiny EL0 net/UDP SVC samples — not sockets), POSIX disk apps, SMP, a POSIX-style userspace. “EL0 isolated” is Verified only as the accepted ADR-091 sentence (one core, TCG default smoke machine; limits there); PAN is enabled on default cortex-a76 (ADR-080). Identity .data/heap/leftover-RAM tears are ADR-037/038/049. Filesystem stance: filesystem.md (memfs + FAT16 read/write). Gaps to host apps + containers: non-goal: host-apps.md.

Building or porting

First-class page: building-or-porting.md. Short honesty:

  • Easiest = in-tree no_std coop EL1 on aarch64-ctos.json, proven with cargo / qemu-smoke / docker-smoke.
  • POSIX / glibc = not easy, not started.
  • SVC ABI + libctos + guest PT_LOAD + standing-as-normal for freestanding EL0 = A1 ABI + A2 CRT + A3 loader + A4 standing-task miles (dual-SVC stub still exists; not a syscall table; not isolation).

OS image vs app payloads (A9)

The sponsor goal for “immutability” here is not a frozen kernel. It is to disconnect OS updates from apps: one OS image artifact, separate app payloads, so you can replace the kernel without rebuilding apps and replace apps without rebuilding the kernel.

  • First cut + leftover: two host artifacts (target/aarch64-ctos/debug/ctos + target/hello-libctos.elf) and a FAT16 /hello load path (ADR-030, ADR-032, A9 #48). A2–A4 load that file. Stance: immutability.md.
  • Cross-update: same app ELF on this OS and documented prior OS ba6541c. Product freestanding app hosting is Verified under ADR-048/052. Do not say OTA / “immutable OS” marketing / Linux hosting.
  • Do not say “immutable OS.”
  • Performance: perf: app-load measures the FAT load path; ADR-046 adds probe-only perf: embed-load + perf: slot-delta (QEMU TCG lab pair — not a percent). ADR-051 adds perf: fat-write / perf: memfs-write / perf: fs-write-delta (same-boot write pair — not a percent). Expected costs: boot/load, SVC, ASID/TTBR, optional later COW. Details: performance.md.

Prerequisites

  • Nightly Rust (rust-toolchain.toml), rust-src, llvm-tools-preview
  • qemu-system-aarch64 (Debian package is often qemu-system-arm)
  • Optional: Docker linux/arm64 (./scripts/docker-smoke.sh) — do not pin linux/amd64
  • To change the kernel: one milestone → one branch → one GitHub PR; author ≠ merger (ADR-002)

A machine that has not run scripts/qemu-smoke.sh (or Docker/GHA equivalent) has Unknown boot.

Advantages

  • Small, readable no_std tree on one ISA (AArch64 / QEMU virt / PL011 UART)
  • Fail-closed smoke: serial markers, cargo test, force-fail
  • Honesty ledger: Verified / Unknown / Planned / Failed, with a probe each
  • Failures become sensors (layout miss, live-.text vtable yank, CRLF shebang)
  • Three pillars named up front — security and speed still need probes

Drawbacks

  • QEMU virt only. No Raspberry Pi or board claim
  • Not POSIX, not multi-tenant, not a product runtime. memfs + FAT16 read/write (filesystem.md; ADR-050). Not a container host (non-goal, ADR-029); host docker-smoke ≠ guest Docker.
  • Umbrella “EL0 isolated” is Verified only as the accepted ADR-091 sentence (M6; history ADR-047). Live identity .text / .rodata / .data / heap / leftover RAM are torn (ADR-020/025/037/038/049); _start stays; PAN is enabled on the default -cpu cortex-a76 (ADR-080; historical a57 pan: absent)
  • Scoped immutability only (immutability.md): RO+NX / WXN / live .text tear are probed. Absolute “immutable OS” is incompatible (heap/PTEs/devices must mutate). OS/app slot first cut (A9 / ADR-030) is two artifacts + FAT /hello. Leftover (ADR-032): A2–A4 load FAT; same ELF on this OS and ba6541c.
  • Performance numbers are guest counter deltas, not a latency budget
  • Docs website / custom domain: HTTPS serving the book is Verified (website.md)
  • Same-login cannot self-merge; a second identity has to land the PR

Honesty

Do not say “secure OS,” “the kernel moved,” “immutable OS,” or “apps update independently of the OS.” Point at the ledger for any number you quote. Say “EL0 isolated” only as the accepted ADR-091 sentence, never wider. The live docs URL is a separate Verified row — do not invent other site claims.

What can run today

Do not say “applications run on ctos.” These are the probed guest examples on QEMU virt. Each needs a matching honesty ledger row. POSIX and a userspace ABI stay Planned. “EL0 isolated” is Verified only as the accepted ADR-091 sentence (one core, TCG default smoke machine; limits there).

Hub: overview.md. How to add something: building-or-porting.md. Frozen IDs: FR-08 (UART RX), FR-11 (coop yield).

Site source of truth: What can run today (A8 rebuild recipes). In-tree index: user/README.md. This page is extra stance (walkthroughs). HTTPS at https://ctos.artof.link is Verified (2026-09-11 after #30).

One rebuild for every sample: cargo build then ./scripts/qemu-smoke.sh. There is no separate app runner. A8 documents that path. A9 first cut adds slot:* when FAT /hello loads. Cross-update (same ELF on this OS and ba6541c) is ledger-Verified (ADR-032). Product freestanding app hosting is Verified (ADR-048 / ADR-052); not Linux/POSIX/containers.

Sample: cooperative EL1 UART workers (M9)

What it is. Two heap-backed EL1 workers take turns. Each prints one UART line, then the idle thread sees both flags and prints sched: ok. Cooperative only — a task runs until it returns or calls yield_now(). Not a timer slice. Not SMP. Not EL0.

Where. src/sched.rs (task_a / task_b / run_two_tasks). Decision: ADR-010.

Rebuild. Edit those functions. cargo build then ./scripts/qemu-smoke.sh. Recipe hub: what-can-run.md.

How it works.

  1. Heap must already be up (src/heap.rs). Worker stacks are 8 KiB Vec<u8> boxes, not the linker SP_EL0 stack.
  2. spawn(task_a) and spawn(task_b) install AAPCS64 callee-saved frames. After ADR-020 the first ret lands on the high-VA alias of the trampoline.
  3. Idle (kernel_main / the test runner) calls yield_now() until both atomics are set, or a bound is hit.
  4. Each worker stores its SP, writes a raw UART line, sets its flag, and returns (Done).

Probe (do not invent extra markers).

SerialMeaning
sched: task aWorker A ran on a heap stack
sched: task bWorker B ran on a heap stack
sched: okBoth ran; smoke greps all three

#[test_case] two_tasks_run_on_distinct_heap_stacks and yield_round_robin_resumes_both close the same claim. Idle tip Docker on main e80dc93: those strings still print on the hello path (50 tests). File presence is not that probe.

Heartbeat / counter variant (not in tree)

Same shape as task_a / task_b, not a new subsystem. A worker that loops, increments an AtomicU64, prints sched: beat n=<n>, and yields would be a counter / heartbeat gadget.

That variant is not in src/sched.rs today. There is no sched: beat smoke string. Adding it is the easiest in-tree exercise (building-or-porting.md): one extra fn, one spawn, one milestone → one PR. Until a probe greps the new marker, treat it as Planned, not Verified.

Sample: UART RX echo gadget (M6)

What it is. The guest polls the virt PL011 RX FIFO and prints the first byte the host injects. On the smoke path that byte is 0x41 ('A'). That is one-shot observe, not a line editor and not a shell.

Where. src/uart.rs (observe_rx / observe_probe_byte). Host inject: scripts/qemu-serial-inject.py (default CTOS_INPUT_BYTE=0x41). Decision: ADR-007.

Rebuild. Same cargo build + ./scripts/qemu-smoke.sh (the script injects 0x41). Recipe: what-can-run.md.

How it works.

  1. After the hello path has printed earlier markers, observe_probe_byte spins on UARTFR.RXFE until a byte arrives or CNTPCT times out.
  2. IRQs stay masked on this path. There is no UART IRQ in M6. Do not wfi waiting for RX.
  3. The expected byte prints input: rx 0x41. Any other byte prints input: rx 0xNN (unexpected) and fails the probe. Timeout prints input: rx missed.

Probe.

SerialMeaning
input: rx 0x41Host-injected 0x41 landed in the FIFO
input: rx missedFail-closed (smoke rejects this)

#[test_case] uart_rx_fifo_empty_without_host_byte only proves the FIFO is empty under cargo test (no inject). Character proof is the hello serial row.

This is not interactive echo, virtio-keyboard, or a TTY. A later line-oriented gadget would be a new milestone with its own marker.

Sample: standing EL0 stub (SVC enter / leave)

What it is. A bounded user context on the user TTBR0. The kernel ERETs to EL0, the payload announces with SVC #1, runs one user MOVZ, then SVC #2 restores EL1. el0::is_active() is true only for that lifetime.

Where. src/el0.rs (standing install) and the lower-EL sync path in src/exception.rs. Direction: ADR-013. Narrative: el0.md.

How it works.

  1. First miles already proved enter/return (el0: svc), UXN fetch (el0: nx kernel), and no kernel .data read (el0: no kernel read).
  2. Standing payload on the EL0 window: SVC #1 → serial el0: standing (stay at EL0) → MOVZ X1, #0x51A4 → SVC #2 → serial el0: restored (back to EL1).
  3. Saved user PC / SP / TTBR0 live only while is_active() is true. The handler clears that flag on restore.

Probe.

SerialMeaning
el0: standingSVC #1 taken; still at EL0
el0: restoredSVC #2 taken; back to EL1
el0: okFirst-mile bundle including standing

#[test_case] standing_el0_enter_leave closes enter/leave. After A1 the standing stub also runs the documented ABI trip (svc: yield / svc: user-hi / svc: uart / svc: exit / svc: ok) — syscall.md. After A2 a hello linked against libctos prints libctos: hi / libctos: ok (ADR-022). After A3 the same ELF is guest-parsed (loader: ok, ADR-023). After A4 that loaded image is a standing task until exit (el0: task-ok, ADR-024); an unexpected fault restores fail-closed (el0: restore-fail). Lower-EL IRQ still parks. PAN is enabled on the default -cpu cortex-a76 (ADR-080; PAN enable was a non-goal on the historical -cpu cortex-a57, ADR-047; a57 ID-field pan: absent Verified). This is not a user process and not POSIX; “EL0 isolated” means only the accepted ADR-091 sentence. The ABI + CRT + loader + standing-task miles are not app hosting (ADR-048).

Rebuild the hello. Source is user/hello-libctos/ (user/hello-libctos/README.md). A kernel cargo build publishes it via build.rs onto FAT /hello. Standalone:

cargo build --release \
  --manifest-path user/hello-libctos/Cargo.toml \
  --target user/hello-libctos/aarch64-ctos-user.json

That ELF still has to land on FAT /hello to run. The hello does not call fs_open. A second freestanding sample (user/fs-libctos, FAT /fsdemo) does exercise create/open/read/write/close on /memdemo (ADR-059; markers libctos: fs-hi / libctos: fs-ok / fsdemo: ok). A third (user/fat-libctos, FAT /fatdemo) opens/reads FAT /probe (fat-hi) via thin VFS (ADR-061; markers libctos: fat-hi / libctos: fat-ok / fatdemo: ok). A fourth (user/yield-libctos, FAT /yldemo) exercises several cooperative yield_now() rounds (ADR-062; markers libctos: yld-hi / libctos: beat / libctos: yld-ok / yldemo: ok). A fifth (user/net-libctos, FAT /netdemo) exercises EL0 net SVCs (ADR-068; markers libctos: net-hi / libctos: net-mac / libctos: net-ok / netdemo: ok). A sixth (user/udp-libctos, FAT /udpdemo) exercises EL0 net_udp_dns (ADR-071; markers libctos: udp-hi / libctos: udp-ok / udpdemo: ok). A seventh (user/mkdir-libctos, FAT /mkdemo) exercises EL0 fs_mkdir (ADR-075; markers libctos: mkdir-hi / libctos: mkdir-ok / mkdemo: ok). An eighth (user/tcp-libctos, FAT /tcpdemo) exercises EL0 net_tcp_echo (ADR-076; markers libctos: tcp-hi / libctos: tcp-ok / tcpdemo: ok). The kernel /eprobe trampoline (fs: el0) remains a separate Verified trip. Not POSIX. Not getdents. Not preemption. Not sockets.

Sample: freestanding libctos VFS (fs-libctos, ADR-059)

What it is. Second freestanding EL0 ELF linked against libctos. Creates /memdemo on memfs (ADR-058 /mem prefix), writes fs-hi, reads it back, prints libctos: fs-ok. Loaded from FAT /fsdemo.

Where. user/fs-libctos/, src/fsdemo.rs, scripts/mkfat16.py --app2. Decision: ADR-059.

Rebuild. Same cargo build + ./scripts/qemu-smoke.sh.

Probe. libctos: fs-hi / libctos: fs-ok / fsdemo: fat / fsdemo: mapped / fsdemo: ok. Keep slot: ok / /hello. Not POSIX. Not getdents.

Sample: freestanding libctos FAT (fat-libctos, ADR-061)

What it is. Third freestanding EL0 ELF linked against libctos. Opens FAT /probe, reads fat-hi through thin VFS (/ → FAT16), prints libctos: fat-ok. Loaded from FAT /fatdemo. Deeper than fs-libctos (memfs only).

Where. user/fat-libctos/, src/fatdemo.rs, scripts/mkfat16.py --app3. Decision: ADR-061.

Rebuild. Same cargo build + ./scripts/qemu-smoke.sh.

Probe. libctos: fat-hi / libctos: fat-ok / fatdemo: fat / fatdemo: mapped / fatdemo: ok. Keep slot: ok / fsdemo: ok / /hello / /fsdemo. Not POSIX. Not getdents. Fourth sample: Recipe yield below.

Sample: freestanding libctos yield rounds (yield-libctos, ADR-062)

What it is. Fourth freestanding EL0 ELF linked against libctos. Prints libctos: yld-hi, then several libctos: beat lines across yield_now() rounds (deeper than hello’s single yield), then libctos: yld-ok. Loaded from FAT /yldemo. Cooperative only — not preemption, not multi-task EL0, not a process table.

Where. user/yield-libctos/, src/yldemo.rs, scripts/mkfat16.py --app4. Decision: ADR-062.

Rebuild. Same cargo build + ./scripts/qemu-smoke.sh.

Probe. libctos: yld-hi / ≥3× libctos: beat / libctos: yld-ok / yldemo: fat / yldemo: mapped / yldemo: ok. Keep slot: ok / fsdemo: ok / fatdemo: ok / /hello / /fsdemo / /fatdemo.

Sample: freestanding libctos net SVCs (net-libctos, ADR-068)

What it is. Fifth freestanding EL0 ELF linked against libctos. Prints libctos: net-hi, reads guest MAC (libctos: net-mac), asks the kernel for an ICMP echo to SLIRP (net_ping), then libctos: net-ok. Loaded from FAT /netdemo. Kernel owns virtio-net — no guest driver in EL0.

Where. user/net-libctos/, src/netdemo.rs, scripts/mkfat16.py --app5. Decision: ADR-068.

Probe. libctos: net-hi / libctos: net-mac / libctos: net-ok / netdemo: fat / netdemo: mapped / netdemo: ok. Keep slot: ok / fsdemo: ok / fatdemo: ok / yldemo: ok / N1 net: ok / N2 net: ping-ok / N3 net: udp-ok. Not TCP product. Not sockets. Not “has networking.”

Sample: freestanding libctos UDP DNS SVC (udp-libctos, ADR-071)

What it is. Sixth freestanding EL0 ELF linked against libctos. Prints libctos: udp-hi, asks the kernel for a quiet ARP + UDP DNS probe vs SLIRP (net_udp_dns), then libctos: udp-ok. Loaded from FAT /udpdemo. Kernel owns virtio-net — no guest driver in EL0. DNS is probe bait only.

Where. user/udp-libctos/, src/udpdemo.rs, scripts/mkfat16.py --app6. Decision: ADR-071.

Probe. libctos: udp-hi / libctos: udp-ok / udpdemo: fat / udpdemo: mapped / udpdemo: ok. Keep slot: ok / fsdemo: ok / fatdemo: ok / yldemo: ok / netdemo: ok / N1 net: ok / N2 net: ping-ok / N3 net: udp-ok. Not TCP product. Not sockets. Not a DNS product. Not “has networking.”

Freestanding mkdir-libctos (EL0 fs_mkdir)

What it is. Seventh freestanding EL0 ELF linked against libctos. Prints libctos: mkdir-hi, creates FAT root /edir via fs_mkdir (SVC 27), proves Exists/BadPath mapping, then libctos: mkdir-ok. Loaded from FAT /mkdemo (8.3 — not /mkdirdemo).

Where. user/mkdir-libctos/, src/mkdemo.rs, scripts/mkfat16.py --app7. Decision: ADR-075.

Probe. libctos: mkdir-hi / libctos: mkdir-ok / mkdemo: fat / mkdemo: mapped / mkdemo: ok. Keep fat: mkdir / prior six samples / net. Not POSIX mkdir. Not a nested tree.

Freestanding tcp-libctos (EL0 net_tcp_echo)

What it is. Eighth freestanding EL0 ELF linked against libctos. Prints libctos: tcp-hi, runs quiet thin TCP guestfwd echo via net_tcp_echo (SVC 28), then libctos: tcp-ok. Loaded from FAT /tcpdemo.

Where. user/tcp-libctos/, src/tcpdemo.rs, scripts/mkfat16.py --app8. Decision: ADR-076.

Probe. libctos: tcp-hi / libctos: tcp-ok / tcpdemo: fat / tcpdemo: mapped / tcpdemo: ok. Keep net: tcp-ok / prior seven samples. Not BSD sockets. Not listen/accept. Not TLS.

Sample: memfs named-buffer probe (A6)

What it is. Thin VFS + in-RAM buffers. Kernel creates /kprobe, writes, reads. An EL0 trampoline does the same for /eprobe (memfs-el0). Not POSIX. Not a directory tree.

Where. src/vfs.rs, src/syscall.rs (el0_fs_svc_roundtrip). Decision: ADR-027.

Rebuild. Same cargo build + ./scripts/qemu-smoke.sh.

Probe. fs: create / fs: write / fs: read / fs: el0 / fs: ok. File presence is not that probe.

Sample: FAT16 /probe on virtio-blk (A7)

What it is. Guest programs virtio-mmio block and vfs::open("/probe") reads fat-hi from a host-built FAT16 image. Same open / write as memfs. Write depth + small create: ADR-050. Same-boot CNTPCT vs memfs write: ADR-051.

Where. src/virtio.rs, src/fat.rs, scripts/mkfat16.py. Decision: ADR-028.

Rebuild. Same smoke. The script writes target/fat16.img and attaches -drive if=none,file=…,id=hd0 -device virtio-blk-device,drive=hd0. Host -drive without fat: ok is not the probe.

Probe. blk: virtio / blk: cap / blk: rw / blk: ok and fat: mount / fat: read / fat: write / fat: create / fat: ok, plus perf: fat-write / perf: memfs-write / perf: fs-write-delta when the ledger has ADR-051. Do not say “supports FAT.”

Also on the same hello path

These are kernel sensors, not apps. Cite the ledger SHA if you quote a number. Recent A7 smoke still printed:

  • UART Hello World!
  • paging: ok, heap: ok (Box / Vec on the first-fit heap)
  • timer: tick (GICv2 + CNTP)
  • wx: ok, guard: ok, ro: ok
  • ident: reloc / ident: live / ident: rodata / ident: ok
  • svc: ok / libctos: ok / loader: ok / el0: task-ok
  • fs: ok / blk: ok / fat: ok
  • exception: sync BRK then exception: fatal nested

What cannot run today

Explicit no. Do not paper over these with a “porting guide.”

WantWhy not
A Linux ELF / glibc / musl binaryNo exec, no ELF loader, no syscall table. Custom target is os: none.
A shell (sh, bash) or line-oriented TTYRX probe is one injected byte. No line discipline.
Python, Node, or any hosted interpreterNeeds a process ABI, heap policy, and usually a filesystem.
Network / sockets / HTTPNo virtio-net, no stack, no sockets.
POSIX / Linux disk appsmemfs (A6) + FAT16 on virtio-blk (A7 / ADR-028 + write ADR-050). Not POSIX. Not FAT32. Stance: filesystem.md.
SMP / a second CPU / preemptive threadsM9 is cooperative EL1 on one vCPU.
Isolated userspace / “an app you compile and exec”Standing EL0 is a stub. “EL0 isolated” is Verified only as the accepted ADR-091 sentence (one core, TCG; limits there): PAN enabled on a76 (ADR-080); _start stays; leftover identity RAM torn (ADR-049). Identity .rodata/.data/heap are torn (ADR-025/037/038). Gaps: host-apps.md.
OCI / Docker / k8s in the guestNo — non-goal (ADR-029). Host docker-smoke.sh only builds the kernel.
Raspberry Pi or any board other than QEMU virtUnprobed. Do not copy virt Verified onto hardware.
GPU / desktop / windowing / virtio devicesOut of scope on this horizon.

Umbrella “EL0 isolated” is Verified only as the accepted ADR-091 sentence (M6). PAN is enabled on default cortex-a76 (ADR-080). Identity .rodata / .data / heap / leftover RAM tears are probed (ADR-025/037/038/049); _start stays. See el0.md.

Building or porting

Honesty first: there is a tiny libctos CRT for the A1 SVC ABI, a guest PT_LOAD loader for that in-tree ELF, and no libc. ctos is a freestanding no_std kernel on a custom target. Status words need a probe in the honesty ledger.

What already runs: apps-today.md. Recipes: What can run today, in-tree user/README.md. KPIs and trade-offs: overview.md.

Site source of truth: Building or porting. This page is extra stance. HTTPS at https://ctos.artof.link is Verified (2026-09-11 after #30).

Easiest path — in-tree no_std cooperative EL1

The only path that matches today’s probes is another kernel task in this repo, built for aarch64-ctos.json, proven with the same smoke you already have.

Target and build

PieceWhat it is
aarch64-ctos.jsonCustom rustc target: os: none, abort, soft-float, static, rust-lld. Not aarch64-unknown-linux-gnu.
.cargo/config.tomlPins that JSON, build-std (core / alloc / compiler_builtins), -Tlinker.ld, QEMU runner.
rust-toolchain.tomlNightly + rust-src / llvm-tools-preview.
cargo build                 # ELF at target/aarch64-ctos/debug/ctos
cargo run                   # qemu-system-aarch64 -machine virt
./scripts/qemu-smoke.sh     # hello greps + cargo test + force-fail
./scripts/docker-smoke.sh   # same smoke in linux/arm64 (do not pin amd64)

A machine that has not run qemu-smoke.sh (or Docker / GHA equivalent) has Unknown boot. Idle tip: sponsor Docker is Verified on main e80dc93 (50 tests, ident: reloc n=12, live pages=37, force-fail). That is one host, not this VM and not a Pi.

Add a cooperative EL1 task

Same shape as apps-today.md (src/sched.rs task_a / task_b):

  1. Write a fn my_task() that uses alloc / UART / sched::yield_now() only. No std, no files, no sockets.
  2. spawn(my_task) next to the existing workers (heap must be up; stacks are heap Vecs).
  3. Print a new serial marker (sched: beat n=… or similar). Teach scripts/qemu-smoke.sh to grep it. Fail closed on miss.
  4. One milestone → one branch → one GitHub PR. Author ≠ merger (ADR-002).

A heartbeat / counter loop is the natural variant. It is not in the tree until that PR lands. Do not claim it is Verified from this paragraph.

Stay on AArch64 QEMU virt / PL011 (ADR-003). Do not restore bootloader 0.9 or VGA as primary.

POSIX / glibc port — not easy, not started

A Linux, musl, or glibc binary will not run. Missing, among other things:

  • exec / ELF loader / dynamic linker
  • syscall table (read / write / open / mmap / clone / …)
  • filesystem as Linux defines it (thin VFS + memfs + FAT16 is not POSIX open — filesystem.md), signals, sockets, environ, TLS as Linux defines them
  • a C runtime (crt0, libgcc helpers as a POSIX process)

Do not publish a “port busybox / musl to ctos” guide that skips those gaps. That work would be many ADRs, not a weekend #ifdef. Frozen Out list: fr-nfr.md (userspace processes, POSIX, networking).

libctos (A2) — freestanding EL0 CRT

Standing EL0 is still a dual-SVC stub (SVC #1 stay / SVC #2 restore) plus first-mile SVC #0, plus the A1 public ABI trip. A2 adds a no_std crate you can link:

PieceWhat it is
libctos/Wrappers exit / uart_write / yield_now over SVC #16 / #17 / #18. Must not issue #0–#2.
libctos/src/crt0.S_start → main → ctos_exit. No argv / environ.
user/hello-libctos/Hello that prints libctos: hi / libctos: ok via uart_write. Recipe: user/hello-libctos/README.md.

That is not glibc. Not exec of a Linux ELF. The hello image is host-built. A2 copies the flattened PT_LOAD onto the standing EL0 page (ADR-022). A3 parses the same ELF on the guest and maps PT_LOAD into user TTBR0 (ADR-023). File presence is not the probe — see the ledger for libctos: ok and loader: ok.

Guest loader (A3)

src/loader.rs walks ELF64 LE AArch64 ET_EXEC program headers, maps PT_LOAD pages in the user map-window, and ERETs to e_entry. Rejects PT_INTERP and W+X. A2–A4 and A9 read FAT /hello (ADR-030, ADR-032). Not a Linux ABI. Linux ELF / auxv / PT_INTERP gaps: ADR-033. Do not claim dynamic Linux ELF.

Standing EL0 as normal mode (A4)

A4 keeps the A3 loader as the way a payload appears. install_task + ERET is the supported path: the loaded image runs until exit; is_active() is true only for that lifetime; an unexpected EL0 fault restores fail-closed (ADR-024). Serial el0: task-ok. File presence is not that probe.

That is not a process and not POSIX.

Still Planned / decided:

  1. Umbrella “EL0 isolated” is Verified only as the accepted ADR-091 sentence (M6; one core, TCG default smoke machine; not speculative-execution / side-channel, not real hardware, not a certification). PAN is enabled on default cortex-a76 (ADR-080); leftover identity RAM after the heap is torn (ADR-049); never yank _start. A5 took .rodata (ADR-025), .data (ADR-037), heap (ADR-038), and the PAN ID-field cut.
  2. Product freestanding app hosting is Verified (ADR-048 / ADR-052). Leftover cross-update is Verified host smoke on this OS and ba6541c (ADR-032). A2–A4 load FAT /hello (no production embed). Not Linux/POSIX/containers.

“Write a user program for ctos” still means: link libctos in-tree, publish target/hello-libctos.elf, and put it on FAT /hello — or add an EL1 task. The easiest thing you can do today remains an in-tree EL1 task.

The OS image vs app payload split (A9 #48) has a first cut plus leftover two-boot smoke. Product freestanding app hosting is Verified under ADR-048/052 (not Linux/POSIX). See overview.md and immutability.md.

Do not invent

  • A porting guide that assumes POSIX, a shell, Python, or containers (host-apps.md: containers are a non-goal)
  • A claim about the docs URL that skips the ledger (HTTPS is Verified as of 2026-09-11; do not invent extra site KPIs)
  • “Secure OS,” “the kernel moved,” or “EL0 isolated” wider than the ADR-091 sentence

Cite the ledger for any Verified SHA you quote.

Filesystem stance

Thin VFS with two backends and prefix mounts. memfs (A6 / ADR-027) is in-RAM named buffers. FAT16 on virtio-blk (A7 / ADR-028) is an on-disk volume; write depth is ADR-050; multi-cluster grow is ADR-064; root listing is ADR-056; root delete is ADR-057; root mkdir is ADR-073; nested mkdir + empty rmdir is ADR-077; EL0 fs_mkdir is ADR-075. Path prefixes route through a mount table (ADR-058). Same vfs::open / read / write / close (+ readdir / unlink / mkdir / rmdir for FAT). No POSIX mount / getdents / unlink / rmdir, no FAT32, no xv6-like inode FS. Linux VFS objects vs this thin surface: ADR-034 (Track B B5). Not claiming a Linux filesystem.

Do not say “ctos has files” as if it were a Linux volume. Do not mint a new FR/NFR ID in chat. Frozen Out list already names POSIX / userspace as later (fr-nfr.md). Hub: overview.md. Samples that do run: apps-today.md.

Site source of truth: Filesystem. This page is extra stance. HTTPS at https://ctos.artof.link is Verified (2026-09-11 after #30).

Order

Each step is one milestone → one branch → one PR.

StepWhat it isWhy this orderStatus
1. memfsIn-RAM named buffers on the first-fit heap (src/heap.rs). Create / lookup / read / write of a path without DMA.Same “easiest in-tree” shape as a coop EL1 task.Verified (A6 / ADR-027). Serial fs: ok.
2. virtio-blkQEMU virt virtio-mmio block: virtqueues, DMA, sector R/W.Host -drive is not a probe.Verified when the ledger has blk: ok (A7 / ADR-028).
3. FAT16Host-visible 4 MiB raw image; guest opens /probe (PROBE) and reads fat-hi. Same volume also stores freestanding samples /hello (A9), /fsdemo (ADR-059), /fatdemo (ADR-061), /yldemo (ADR-062), /netdemo (ADR-068), /udpdemo (ADR-071), /mkdemo (ADR-075).Chosen over xv6-like so the host can inspect the image (scripts/mkfat16.py --check). FAT32 / xv6 rejected this mile.Verified when the ledger has fat: ok. Write depth: ADR-050. Multi-cluster grow: ADR-064. Root listing: ADR-056. Root delete: ADR-057. Sample paths are catalog slots, not a second FS.
4. Prefix mountsMount table routes /mem (+ A6 names) → memfs and / → FAT16.Product-depth virtual mounts without a new disk format.Verified when the ledger has vfs: mounts (ADR-058). Not mount(2).

fs: create / fs: write / fs: read / fs: el0 / fs: ok stay fail-closed. A7 adds blk: virtio / blk: cap / blk: rw / blk: ok and fat: mount / fat: read / fat: ok. ADR-050 adds fat: write / fat: rewrite / fat: create. ADR-064 adds fat: grow. ADR-056 adds fat: readdir / fat: entries. ADR-057 adds fat: delete. ADR-073 adds fat: mkdir. ADR-077 adds fat: nested / fat: rmdir. ADR-075 adds EL0 fs_mkdir / libctos: mkdir-ok / mkdemo: ok. ADR-058 adds vfs: mount / vfs: mounts.

QEMU flags (smoke + cargo runner):

-drive if=none,file=target/fat16.img,format=raw,id=hd0 -device virtio-blk-device,drive=hd0

What this is not

  • Not POSIX open / stat / mount. Handles are a thin VFS fd, not a Linux fd table (building-or-porting.md).
  • Not virtio-net, 9p, or a Linux rootfs.
  • Not “we have a disk because QEMU can attach one.” Host -drive without guest virtio + a VFS read is not a probe.
  • Not POSIX writeable volumes. Guest FAT write is a thin-VFS depth mile (ADR-050); multi-cluster grow is ADR-064; say “wrote / grew FAT16 bytes” when markers pass. Not a POSIX write API.
  • Not POSIX getdents / opendir. Guest FAT root listing is a thin-VFS depth mile (ADR-056); say “listed FAT16 root entries” when markers pass.
  • Not POSIX unlink / remove. Guest FAT root delete is a thin-VFS depth mile (ADR-057); say “deleted a FAT16 directory entry” when markers pass.
  • Not POSIX mount(2) / Linux vfsmount. Prefix mounts are a thin-VFS depth mile (ADR-058); say “routed path prefixes through a mount table” when markers pass.
  • Not app hosting. A8 is documented sample rebuild recipes (what-can-run.md).

Honesty

ClaimProbeStatus
memfs create/write/read/closeSerial fs: ok + #[test_case]Verified (A6; honesty ledger)
virtio-blk worksSerial blk: ok + sector R/W #[test_case]Verified on this tip when the ledger has the probe
FAT16 on that deviceSerial fat: ok; VFS /probe = fat-hiVerified on this tip when the ledger has the probe
FAT16 write + small createSerial fat: write / fat: create; host --check-writeVerified on this tip when the ledger has the probe (ADR-050)
FAT16 multi-cluster growSerial fat: grow / libctos: fat-growVerified on this tip when the ledger has the probe (ADR-064)
FAT16 root mkdirSerial fat: mkdirVerified on this tip when the ledger has the probe (ADR-073)
EL0 fs_mkdirSerial libctos: mkdir-ok / mkdemo: okVerified on this tip when the ledger has the probe (ADR-075)
FAT16 nested mkdir + empty rmdirSerial fat: nested / fat: rmdirVerified on this tip when the ledger has the probe (ADR-077)
FAT16 root readdirSerial fat: readdir / fat: entriesVerified on this tip when the ledger has the probe (ADR-056)
FAT16 root deleteSerial fat: deleteVerified on this tip when the ledger has the probe (ADR-057)
Thin VFS prefix mountsSerial vfs: mount / vfs: mountsVerified on this tip when the ledger has the probe (ADR-058)

Unprobed stays Unknown. File presence is not that probe. See the honesty ledger.

Gaps to host apps — and no containers

Do not say “you can run host apps on ctos.” A host app here means a program you already run on Linux or macOS: a shell, Python, a browser, a Docker/OCI container, a glibc ELF you exec. Today’s guest can run the probed samples only. Closing the gap is Planned in pieces, not a product claim.

./scripts/docker-smoke.sh is a host harness (build the kernel inside a Linux container, then QEMU). It is not a guest container runtime. ctos is not a container host.

Hub: overview.md. Porting: building-or-porting.md. Filesystem: filesystem.md.

Site source of truth: Hosting apps / containers. Product “hosting done” claim: Verified under ADR-048 / ADR-052 (freestanding slots; containers remain non-goal). This page is extra stance. HTTPS at https://ctos.artof.link is Verified (2026-09-11 after #30).

What a host app needs vs what exists

Need (typical host app)On ctos todayStatus
A process you execStanding EL0 + A1 ABI + A2 libctos + A3 PT_LOAD + A4 standing task + A9 FAT /hello first cutPlanned as Linux exec. Stance: ADR-035. A9 is two artifacts + a FAT load path, not a process table.
POSIX / glibc / muslCustom aarch64-ctos.json, os: none, no libcNot easy, not started.
Files (open / a disk)Thin VFS: memfs + FAT16 (read + write depth, ADR-050)memfs + FAT miles (A6/A7 + ADR-050). Not POSIX.
Sockets / HTTPNo virtio-net, no stackPlanned at best; not a Now mile
Shell / TTY / PythonOne injected UART byte; no interpreterNo until ABI + FS + line discipline
Isolated userspaceStanding miles + identity tears incl. leftover RAM (ADR-049); PAN enabled on a76 (ADR-080); _start staysUmbrella “EL0 isolated” Verified only as the accepted ADR-091 sentence (one core, TCG; limits there)
Preemption / SMPCooperative EL1, one vCPUNo on this horizon
Containers (OCI / Docker / k8s as the guest)NothingNon-goal. See below.

The easiest thing you can add today is still an in-tree no_std coop EL1 task — not a host binary.

Containers: non-goal

ctos will not run containers as a guest feature. This is a non-goal, not “later on this horizon.” ADR-029. Site SoT: hosting-apps.md.

  • No OCI image pull, no runc, no cgroups, no Linux namespaces, no overlay FS, no containerd/CRI.
  • A Linux Docker/Podman host that builds this kernel is unrelated. That smoke does not make QEMU virt a container host.
  • “Run Alpine on ctos” / “k8s node” is the same class of claim as “POSIX port is easy.” It is not.

Do not write a container roadmap. Track A gaps (ABI, FS, isolation) are not a path to OCI. Do not mint a new FR ID for containers. Reopen only with a new GitHub epic plus a new ADR.

Honesty

ClaimProbeStatus
Guest is not a container hostSource: no OCI/runc/cgroup/namespace code in src/Verified (absence)
Host docker-smoke.sh builds the kernelLedger Docker rows (sponsor / GHA)Separate claim — host harness only
A host app (Linux ELF, shell, Python) runs in the guestNo such serial markerPlanned
Guest is a container hostNo OCI/runc/cgroup codeVerified absence; non-goal (ADR-029; not a later Planned feature)
Linux-compat goals documented (Track B B1)Read ADR-031Documented frame. Not Linux userspace. B6 stays Planned research.
Linux AArch64 vs ctos SVC gap (Track B B2)Read linux-aarch64-syscall-gap.mdDocumented inspection map. No Linux numbers in src/. Not a Linux ABI.
Process model vs standing EL0 (Track B B3)Read ADR-035Documented stance. Standing EL0 stays the ctos model. Not fork/execve/wait.
Linux ELF / auxv / PT_INTERP vs freestanding loader (Track B B4)Read ADR-033Documented gap ADR. Track A loader stays reusable. Does not accept PT_INTERP. Not dynamic Linux ELF.
Linux VFS vs thin ctos VFS (Track B B5)Read ADR-034Documented concept compare. No POSIX flags / dentries in src/. Not a Linux filesystem.

File presence of this note is not an app runtime. See the honesty ledger.

Immutability (scoped, not absolute)

Compatible with ctos principles only as scoped immutability: code and other RO regions stay non-writable after a probed lock-down; later, loaded app images can be RO too. That lines up with security (NFR-10), honesty (NFR-06), and antifragility (NFR-05) — a region is RO when a probe says so, and a Failed write-to-RO stays in the ledger.

Incompatible if absolute. A kernel must mutate heap, page tables, device MMIO, and task state. “Immutable OS” as marketing is a status inflation. Do not write it.

Hub: overview.md. Site SoT: Advantages — Immutability. Pillars: pillars.md. Threat model: security.md.

Plan issues (open on 2026-09-11): Track A #31 / in-repo track-a.md, Track B #40 / track-b.md. A9: #48. Tracks are subordinate to principles.md.

Already practiced (probed)

These are scoped cuts on QEMU virt. They are not an immutable kernel.

ScopeProbeADR
Identity .text/.rodata RO+X; .data/heap RW+NXro: nx data / ro: write fault / ro: okADR-015
SCTLR.WXN onsame RO+NX smokeADR-015
Live identity .text torn after high-VA jump + vtable rewriteident: reloc / ident: live / ident: okADR-020

Heap, PTEs, UART/GIC, and coop stacks stay writable on purpose. Identity .rodata / .data / heap tears are ledger-Verified (ADR-025 / ADR-037 / ADR-038). Leftover identity RAM after the heap is torn (ident: ram*, ADR-049); smoke rejects ident: ram-stay. _start stays mapped (ident: start-stay, ADR-047).

Track A — RO app payloads, then A9 slot disconnect

The goal of this stance (sponsor clarification) is to disconnect OS updates from apps: a separate OS image vs app payloads. Update/replace the kernel without rebuilding apps, and the reverse. That is A9 #48. It is not an “immutable OS” product sentence.

Track A #31 landed A1–A8 first. A1 is a kernel SVC ABI mile (syscall.md). A2 is a libctos CRT mile (ADR-022). A3 is a guest ELF64 PT_LOAD loader (ADR-023). A4 is standing EL0 as normal mode for that loaded image (ADR-024). RO app payloads are that loader mapping an image RO+X (R-only PT_LOAD already uses map_el0_ro; RX text stays Exec). A9 is the slot first cut (ADR-030), not an absolute immutable OS.

Today (Verified freestanding hosting under ADR-048/052): two host artifacts (kernel ELF + target/hello-libctos.elf) and a FAT16 /hello load path. A2–A4 load that file (no production include_bytes!). Cross-update is the host two-boot probe on this OS and ba6541c (ADR-032). Product claim: ADR-048 / ADR-052. Do not say OTA / “immutable OS” marketing / Linux hosting. Performance: perf: app-load plus probe-only perf: embed-load / perf: slot-delta are a QEMU TCG measurement pair (ADR-046, performance.md) — not a percent and not “slots are free.”

Track B #40 must not use Linux-compat research to claim an immutable or container host. Frame: ADR-031 (not claiming Linux userspace yet). Containers stay a non-goal (ADR-029).

Claim gate (ADR-style, no new ADR here)

Same rule as ADR-001 / ADR-011:

  1. Name the scope (which pages, which image).
  2. Cite a probe (serial + #[test_case], or source absence).
  3. Only then Verified. Unprobed stays Unknown. Future work stays Planned.
  4. Do not say “immutable OS,” “W^X kernel” as a product sentence, or “secure because RO.”

File presence of this note is not that probe. See the honesty ledger.

Docs website + DNS

The published site is this mdBook (book.toml, src = "docs"). The website and the repo are the same markdown. Visitor-facing source of truth is the landing plus docs/overview/*. docs/framework/* is for deep links (ledger, pillars, threat model). Do not keep a second marketing copy.

Required site chapters (sidebar + landing). A missing file fails mdbook build (create-missing = false) and scripts/docs-build.sh:

Website pageSource file
What can run todaydocs/overview/what-can-run.md
Building or portingdocs/overview/porting.md
Filesystem (memfs + FAT16)docs/overview/filesystem.md
Hosting apps / containersdocs/overview/hosting-apps.md (guest OCI/Docker: non-goal, ADR-029)
KPIs / how we measuredocs/overview/measure.md
Prerequisitesdocs/overview/prerequisites.md
Advantagesdocs/overview/advantages.md
Drawbacks / limitsdocs/overview/limits.md

Frozen IDs touched by the publish path: document-first (FR-14), honesty (NFR-06), written acceptance matches what is proven (NFR-13). No new FR/NFR IDs.

Diagrams use mermaid fences. mdbook-mermaid 0.17.1 wraps them; docs/mermaid-init.js loads mermaid 11.6.0 from jsDelivr in the browser. A local mdbook build without mdbook-mermaid on PATH fails (book.toml lists the preprocessor). Use ./scripts/docs-build.sh.

Local build

You need mdBook 0.5.4 and mdbook-mermaid 0.17.1 (pinned in .github/workflows/pages.yml and scripts/docs-build.sh).

./scripts/docs-build.sh          # downloads both binaries if missing, then `mdbook build`
# or, if both are already on PATH:
mdbook build                     # writes ./book/
mdbook serve                     # http://localhost:3000

Auto-install keys on uname -s:uname -m. Linux stays x86_64-unknown-linux-gnu / aarch64-unknown-linux-musl. macOS uses the published x86_64-apple-darwin / aarch64-apple-darwin tarballs for mdBook and mdbook-mermaid. Other hosts: install both pins yourself. mdbook --version prints mdbook v0.5.4; the script strips the leading v before comparing to pin 0.5.4.

create-missing is off: a SUMMARY.md link to a missing file fails the build. Output directory book/ is gitignored.

A successful local mdbook build is a generator probe. It is not a new “the website is published” claim. The live URL is a separate ledger row.

Mobile / narrow viewport

Sponsor bar (same as architecture.artof.link): viewport, readable type, usable nav on a phone, no page-wide horizontal overflow. Tables, pre, and Mermaid may scroll inside their widget.

Stock mdBook 0.5.4 already has (audited on the live HTML 2026-09-12):

  • <meta name="viewport" content="width=device-width, initial-scale=1">
  • Sidebar toggle (#mdbook-sidebar-toggle) plus --sidebar-width: min(300px, 80vw)
  • #mdbook-menu-bar { flex-wrap: wrap }
  • .table-wrapper { overflow-x: auto } around markdown tables
  • .content img { max-width: 100% }

Gaps closed in docs/theme.css + mermaid extras (this tree):

  • pre / code: overflow-x: auto so long lines do not widen the page
  • Table cells: overflow-wrap; wide tables keep a min-width and scroll
  • Touch: 44px minimum on sidebar links, menu icons, and mobile chapter arrows (≤768px)
  • Body type stays 16px; heading strings may wrap
  • Mermaid: on ≤768px, useMaxWidth is off so diagrams stay readable and the .mermaid box scrolls (docs/mermaid.css, docs/mermaid-init.js)

Desktop content column stays --content-max-width: 750px. Do not treat this section as a live-site deploy probe.

Narrow-viewport probe

After ./scripts/docs-build.sh, ./scripts/docs-mobile-probe.sh serves book/ and uses headless Chrome. An iframe forces the chapter width so --dump-dom cannot fake a wider layout. It records whether documentElement.scrollWidth exceeds clientWidth on the landing, the honesty ledger, and this page.

2026-09-12 this cloud VM (Google Chrome 148.0.7778.96): landing overflow=no at 320 / 768 / 1280 (client = scroll); ledger and this page overflow=no at 320. Internal .table-wrapper / pre / .mermaid scroll is allowed. The numeric probe maps cdn.jsdelivr.net to 127.0.0.1 so mermaid.js cannot hang Chrome (diagrams stay as source text for that run). A green local probe is not “https://ctos.artof.link is mobile-verified” until a post-merge Pages fetch repeats it.

Production URLs

URLRoleStatus word
https://ctos.artof.linkCustom domainVerified. HTTPS 200, cert for this name, landing includes Driving principles. Main deploy 34653046584 after #30. Pages API: cname=ctos.artof.link, https_enforced=true, cert approved (expires 2026-12-10).
https://artofdream.github.io/ctosProject-site path (no trailing slash)Redirect (301) to the custom domain on the 2026-09-11 probe. A trailing-slash .../ctos/ 404’d. Do not treat github.io as a second live tree.

Do not write Verified on a new deploy from file presence alone. Re-fetch after the next main Pages run if you change the claim.

GitHub Pages workflow

.github/workflows/pages.yml:

  • Pull requests: mdbook build + check that the output CNAME is ctos.artof.link. Does not deploy.
  • Push to main: build, upload actions/upload-pages-artifact, deploy with actions/deploy-pages.

Settings already in use after #30: Source = GitHub Actions, custom domain ctos.artof.link, Enforce HTTPS on. Do not flip Source away from Actions. A CNAME file in the artifact (from book.toml cname = "ctos.artof.link", also the repo-root CNAME) should stay in lockstep.

See GitHub: publishing with Actions.

DNS — Amazon Route 53 (CNAME in place)

artof.link is hosted on Amazon Route 53. The sponsor states the ctos CNAME is already created. Do not run CREATE again.

FieldValue
AWS account<aws-account-id>
Region for CLIus-east-1 (Route 53 API is global)
Hosted zone nameartof.link
Hosted zone ID<route53-zone-id> (sponsor-stated)
RecordCNAME ctos.artof.link. → artofdream.github.io. (trailing dot)

This is a project site on a subdomain. Do not CNAME to artofdream.github.io/ctos. Do not touch the artof.link apex.

Public dig CNAME ctos.artof.link returns artofdream.github.io. DNS is in place at the resolver. That is not a Route 53 API list-resource-record-sets from this environment (no AWS CLI / credentials here). Next session may LIST zone <route53-zone-id> to confirm; skip change-resource-record-sets unless the row is missing.

export AWS_REGION=us-east-1
aws sts get-caller-identity --query Account --output text   # expect "$AWS_ACCOUNT_ID" (set locally, not in repo)
aws route53 list-resource-record-sets --hosted-zone-id "${ROUTE53_ZONE_ID:?set ROUTE53_ZONE_ID locally}" \
  --query "ResourceRecordSets[?Name=='ctos.artof.link.']"

scripts/route53-ctos-cname.json is a leftover CREATE batch for recovery only.

Probes

dig CNAME ctos.artof.link +short    # expect artofdream.github.io.
curl -sSI https://ctos.artof.link   # expect HTTP 200 (Verified 2026-09-11)

Honesty

ClaimProbeUntil then
mdBook builds this tree./scripts/docs-build.sh (or mdbook build with mermaid preprocessor) exit 0—
Narrow viewport / mobile layout./scripts/docs-mobile-probe.sh after a local build (320 / 768 / 1280; landing + ledger + this page)Verified locally 2026-09-12 (iframe overflow=no). Not a live-site mobile claim until a post-merge Pages fetch.
Pages workflow existsRead .github/workflows/pages.ymlFile presence only
Pages workflow builds a PRGreen pages run on this branch (build job; deploy skipped)See honesty ledger
Docs website publishedGreen pages workflow on main and HTTPS fetch of https://ctos.artof.linkVerified — deploy 34653046584 + HTTPS 200 + Driving principles
Public CNAME ctos.artof.linkdig CNAME ctos.artof.link +shortVerified — DNS in place (artofdream.github.io.)
Route 53 API row in <aws-account-id> / <route53-zone-id>list-resource-record-sets as that accountUnknown here (no AWS CLI). Sponsor states CREATE already done. Do not CREATE again.
Custom domain reachabilityPages lists the hostname and HTTPS 200 with a matching certVerified — Pages API cname + https_enforced + cert approved; curl -sSI https://ctos.artof.link HTTP 200

Do not say “secure OS,” or that QEMU boot was proven by this docs PR. Say “EL0 isolated” only as the accepted ADR-091 sentence, with its limits.

Second brain

Session memory lives in git under research/, not on a wiki. This page is a curated landing for the published book. Daily briefs and scratch notes stay on GitHub so the site does not pretend every session log is product documentation.

Published Track B research in the book (not session scratch): Linux AArch64 syscall gap (B2) (inspection only), ADR-035 (B3 process-model stance), ADR-033 (B4 ELF / auxv / PT_INTERP gap), and ADR-034 (B5 VFS concept compare). Not claiming Linux userspace. Not claiming dynamic Linux ELF. Not claiming a Linux filesystem.

VaultWhereJob
Procedure.cursor/skills/ctos-*/How to do a repeatable job
Correction.cursor/rules/ + hard constraints in AGENTS.mdMistakes we must not repeat
RelationshipDocs cross-links (vision → architecture → ADR → roadmap → ledger)How pieces connect
Daily Briefresearch/daily-briefs/Handoff: where we stopped

Session scratch: research/random-thoughts/. Do not treat scratch as the honesty ledger.

Tracker is GitHub. Optional Obsidian UI is local-only (.obsidian/ is gitignored). See research/README.md on GitHub.

Route 53: CNAME already created (zone <route53-zone-id>). Playbook / LIST-only: research/dns-route53-ctos.md. https://ctos.artof.link HTTPS is Verified after #30 (website.md).