ctos (ctsOS)
ctos is a small operating-system kernel you can study. It is written in Rust for 64-bit ARM (AArch64). You run it in the QEMU emulator, not as a desktop or phone OS. The nickname ctsOS is only for display; the repo and crate stay ctos. Why Rust (and what it does not buy): Advantages — Why Rust.
This book is the website. The same markdown lives under docs/ in the GitHub repo. There is no second marketing copy.
It is not a product site. Do not say “secure OS” or “production ready.” Say “EL0 isolated” only as the accepted ADR-091 sentence, with its limits (one core, TCG default smoke machine; not speculative-execution / side-channel, not real hardware, not a certification). Status words need a probe.
What it is
ctos boots under QEMU’s virt machine and prints on a serial port (the PL011 UART). After that it grows one honest mile at a time: exceptions, paging, a heap, a tiny scheduler, then measured security and performance cuts.
Core principles drive. Roadmap tracks (A = freestanding hosting, B = Linux-compat research, N = network foundation — N1 first-frame ADR-066, N2 ICMP ping ADR-067, N3 UDP ADR-070, N3.x EL0 UDP SVC ADR-071, N4 EL0 net sample ADR-068, N5 thin TCP ADR-074, scope ADR-063) are subordinate. A track does not outrank a principle.
Primary path: QEMU virt + UART (ADR-003). Frozen requirement IDs: FR-01–FR-15 and NFR-01–NFR-14.
Driving principles
Everyday meaning first; frozen IDs second. Same list as the pillars hub. This landing is the visitor-facing source. docs/framework/ stays for deep links — do not keep a second marketing copy.
| Principle | In everyday words | Frozen ID |
|---|---|---|
| Honesty | If we did not run a check, we do not say it works. Unprobed stays Unknown. | NFR-06 · ledger |
| Antifragility | The same miss twice becomes an automated sensor, not another README paragraph. | NFR-05 · Antifragility |
| Security | Write down what we fear, then prove a slice. Not a “secure OS” slogan. | NFR-10 · Security |
| Performance | Measure a known path first. No invented benches. | NFR-07 · Performance |
| Document-first | Write the decision, then the code. One milestone → one branch → one PR. | FR-14 / NFR-13 |
flowchart TD P["Core principles<br/>honesty · antifragility · security<br/>performance · document-first"] L["Three pillars<br/>antifragility · security · performance"] T["Tracks A / B / N<br/>hosting · Linux research · net first-frame<br/>subordinate — not the driver"] P --> L --> T
Principles sit above pillars. Tracks sit below both. A1–A4 are ABI + CRT + loader + standing-task miles only. Not a claim that Track A hosting is built.
What runs today
Rebuild recipes for samples that already have probes. Details: What can run today. In-tree index: user/README.md.
- Two kernel tasks that take turns — they print on the serial port and yield. Not preemptive. Not two CPUs.
- A serial echo gadget — one byte in, a line out. No terminal, no line editor.
- A short lower-privilege stub — a few instructions in the CPU’s user mode, including the A1 SVC ABI trip, an A2
libctoshello, an A3 guestPT_LOADof that same hello, and an A4 standing task untilexit(exit/uart_write/yield), then a call back into the kernel. Not a process. No libc. Freestanding catalog on FAT:/hello,/fsdemo(memfs VFS),/fatdemo(FAT/probe),/yldemo(yield rounds). Optional recipes: memfs named buffers and FAT16 on virtio-blk (read + write depth).
Cannot run: Linux programs, a shell, Python, network servers (Track N has virtio-net ARP + ICMP + minimal UDP + thin TCP plus tiny EL0 net/UDP SVC samples — ADR-066 / ADR-067 / ADR-070 / ADR-071 / ADR-068 / ADR-074; not BSD sockets / TCP product), POSIX filesystem apps, extra CPUs, or containers (guest OCI/Docker is a non-goal). An OS slot vs a separate app slot has an A9 first cut (FAT /hello) plus leftover cross-update on ba6541c (Verified miles); catalog also has /fsdemo / /fatdemo / /yldemo / /netdemo / /udpdemo. Product freestanding app hosting is Verified under ADR-048 / ADR-052 (not Linux/POSIX). Umbrella “EL0 isolated” is Verified only as the accepted ADR-091 sentence (M6, 2026-09-28; closure table ADR-060).
flowchart LR
subgraph today ["Verified today"]
W["Two UART workers"]
E["Serial echo gadget"]
S["Short user-mode stub"]
end
subgraph no ["Cannot run"]
L["Linux binaries / shell / Python"]
N["Network / files / extra CPUs"]
C["Containers"]
end
today -.-> no
Left side matches existing smoke markers. Right side is out of scope. Do not say “apps run.”
How to build
You need nightly Rust and QEMU. Pages is not required for kernel work. Full list: Prerequisites.
rustup toolchain install nightly
rustup component add rust-src llvm-tools-preview
cargo build # ELF at target/aarch64-ctos/debug/ctos
./scripts/qemu-smoke.sh # fail-closed serial + tests
flowchart LR R["rustup nightly"] --> B["cargo build"] --> Q["qemu-smoke"]
That rebuilds the kernel, including any in-tree code you add. It is not “port an app.”
Read next
| If you want… | Go here |
|---|---|
| What is in vs out (rebuild recipes) | What can run today · Drawbacks / limits |
| Why POSIX does not port | Building or porting |
| Files (memfs + FAT16) | Filesystem |
| App host / containers | Hosting apps — freestanding product claim Verified (ADR-048 / ADR-052); containers: non-goal (ADR-029). Linux-compat frame: ADR-031 (not userspace). |
| How we measure | KPIs |
| Why the repo is run this way · Why Rust | Advantages · Why Rust |
| Deep dives | Vision · FR / NFR · Architecture · Roadmap · Ledger |
Kernel build notes also live in the GitHub README.
URLs
| URL | Honesty |
|---|---|
https://ctos.artof.link | Verified. HTTPS 200, cert for this name, landing shows Driving principles. Main deploy 34653046584 after #30. |
https://artofdream.github.io/ctos | Redirect. This path (no trailing slash) 301s to the custom domain. A trailing slash 404’d on the 2026-09-11 probe — do not treat github.io as a second live tree. |
Publish mechanics: Docs website + DNS.
What can run today
Plain English. These are rebuild recipes for in-tree samples that already have probes. They are not third-party applications and not a product runtime.
Track A / A8 (issue #39) is this page: document the sample classes and how to rebuild them. A9 (ADR-030) adds two host artifacts + FAT /hello. Leftover (ADR-032): A2–A4 load that file (no production embed); same ELF on this OS and ba6541c (cross-update Verified). Product freestanding app hosting is Verified under ADR-048 / ADR-052 (checklist: hosting-apps.md). Not Linux/POSIX/containers.
Status of each probe: honesty ledger. Walkthroughs: apps-today.md. In-tree index: user/README.md. Umbrella “EL0 isolated” is Verified only as the accepted ADR-091 sentence (M6; one core, TCG default smoke machine; not speculative-execution / side-channel, not real hardware, not a certification). Do not say “apps,” “userspace,” or “secure OS” as if a general-purpose OS existed.
Freestanding sample catalog (glance)
Six published EL0 ELFs on one FAT16 volume. Same rebuild path. Not Linux/POSIX.
flowchart LR FAT["FAT16 volume"] --> H["/hello<br/>UART + one yield"] FAT --> F["/fsdemo<br/>memfs VFS"] FAT --> D["/fatdemo<br/>FAT /probe"] FAT --> Y["/yldemo<br/>yield rounds"] FAT --> N["/netdemo<br/>net SVCs"] FAT --> U["/udpdemo<br/>UDP DNS SVC"] FAT --> M["/mkdemo<br/>fs_mkdir SVC"] FAT --> T["/tcpdemo<br/>net_tcp_echo SVC"] H --> EL0["Standing EL0<br/>libctos"] F --> EL0 D --> EL0 Y --> EL0 N --> EL0 U --> EL0
/hello is the A9 product-slot sample. /fsdemo (ADR-059), /fatdemo (ADR-061), /yldemo (ADR-062), /netdemo (ADR-068), /udpdemo (ADR-071), /mkdemo (ADR-075), and /tcpdemo (ADR-076) deepen the catalog. Umbrella “EL0 isolated”: only the accepted ADR-091 sentence.
Privilege — where code runs
The CPU has privilege levels. EL1 is kernel privilege (where ctos runs). EL0 is lower privilege (user mode). A process would be a loaded program with its own address space, files, and a public ABI. That process does not exist yet.
flowchart TD EL1["EL1 — kernel privilege<br/>Verified: this is where ctos runs"] EL0["EL0 — lower privilege<br/>Verified: a short standing stub only"] PROC["A process / application<br/>Planned: not built"] EL1 --> EL0 EL0 -.-> PROC
The stub is not a process. “EL0 isolated” means only the ADR-091 sentence.
A supervisor call (SVC) is the instruction the stub uses to ask the kernel for something. Reserved SVC #0–#2 stay test miles. A1 documented exit / uart_write / yield (syscall.md). A2 wraps those in libctos. That is not a process ABI.
One rebuild for every recipe
Every sample below rides the same hello path. There is no separate “run this app” command. You rebuild the kernel (and, for freestanding EL0 samples, build.rs publishes target/hello-libctos.elf + target/fs-libctos.elf + target/fat-libctos.elf + target/yield-libctos.elf + target/net-libctos.elf + target/udp-libctos.elf + target/mkdir-libctos.elf + target/tcp-libctos.elf; A2–A4 and A9 read FAT /hello; ADR-059 loads FAT /fsdemo; ADR-061 loads FAT /fatdemo; ADR-062 loads FAT /yldemo; ADR-068 loads FAT /netdemo; ADR-071 loads FAT /udpdemo; ADR-075 loads FAT /mkdemo; ADR-076 loads FAT /tcpdemo).
cargo build # aarch64-ctos.json; also builds user/hello-libctos + user/fs-libctos + user/fat-libctos + user/yield-libctos + user/net-libctos + user/udp-libctos
./scripts/qemu-smoke.sh # fail-closed serial greps + cargo test + force-fail
qemu-smoke attaches the A7 FAT16 image (target/fat16.img) and injects UART 0x41. A machine that has not run that script (or Docker / GHA equivalent) stays Unknown for boot. File presence is not the probe.
flowchart LR R["edit in-tree source"] --> C["cargo build"] --> S["qemu-smoke"] S --> M["existing serial markers"]
Rebuild the guest. Do not drop in a foreign ELF.
Recipe 1 — Cooperative UART workers (EL1)
Two tasks on heap stacks that print a line and yield to each other. Same class as sched: task a / sched: task b / sched: ok (ADR-010, FR-11).
| Piece | Path |
|---|---|
| Workers | src/sched.rs (task_a / task_b / run_two_tasks) |
| Probe | scripts/qemu-smoke.sh greps sched: ok and both task lines |
Rebuild: edit those functions, then the one-rebuild commands above. A heartbeat / counter variant (print, yield, loop) is the natural next in-tree exercise. It is not in the tree and has no sched: beat marker — Planned until a probe greps one.
Not preemptive. Not two CPUs. Not EL0.
Recipe 2 — UART RX echo gadget
Read a byte from PL011 RX and print it. Same class as input: rx 0x41 (ADR-007, FR-08).
| Piece | Path |
|---|---|
| Guest poll | src/uart.rs (observe_rx / observe_probe_byte) |
| Host inject | scripts/qemu-serial-inject.py (default CTOS_INPUT_BYTE=0x41) |
Rebuild: same cargo build + ./scripts/qemu-smoke.sh. The smoke host injects one byte. IRQs stay masked on this path. Do not wfi waiting for RX.
That is a byte in, a line out. No TTY, no line editor, no canonical mode, no virtio-keyboard.
Recipe 3 — Standing EL0 / libctos-loaded hello
A short payload in user mode (EL0) that uses the documented ABI, then returns. Miles stacked on FAT /hello:
| Mile | Serial (do not invent extras) | Decision |
|---|---|---|
| Standing stub | el0: standing / el0: restored | ADR-013 |
| SVC ABI | svc: yield / svc: user-hi / svc: uart / svc: exit / svc: ok | ADR-021 |
libctos hello | libctos: hi / libctos: ok / libctos: linked | ADR-022 |
Guest PT_LOAD | loader: mapped / loader: ok | ADR-023 |
| Standing task | el0: task-enter / el0: task-active / el0: task-exit / el0: task-ok | ADR-024 |
| Piece | Path |
|---|---|
| Hello source | user/hello-libctos/ (recipe README) |
| CRT / wrappers | libctos/ |
| Host publish | build.rs writes target/hello-libctos.elf; guest reads FAT /hello |
| Guest map | src/loader.rs |
Rebuild the hello (optional, standalone):
cargo build --release \
--manifest-path user/hello-libctos/Cargo.toml \
--target user/hello-libctos/aarch64-ctos-user.json
A kernel cargo build already does that via build.rs and publishes the result. Then ./scripts/qemu-smoke.sh. There is no exec of a file on disk. The kernel ELF no longer embeds the hello bytes.
This is not a process. No libc, no argv, no loader for a foreign ELF. “EL0 isolated” means only the accepted ADR-091 sentence.
libctos wraps fs_create / fs_open / fs_read / fs_write / fs_close (SVC 19–23). The hello payload does not call them (UART + one yield). A second freestanding sample does: see Recipe 3b (fs-libctos, ADR-059). A third reads FAT via the same wrappers: see Recipe 3c (fat-libctos, ADR-061). A fourth exercises several cooperative yields: see Recipe 3d (yield-libctos, ADR-062). A fifth exercises EL0 net SVCs: see Recipe 3e (net-libctos, ADR-068). A sixth exercises EL0 UDP DNS SVC: see Recipe 3f (udp-libctos, ADR-071). The kernel /eprobe trampoline (fs: el0) remains a separate Verified EL0 VFS trip.
Recipe 3b — Standing EL0 / libctos VFS sample (fs-libctos)
Second freestanding EL0 payload that calls the VFS wrappers (ADR-059). Same class as libctos: fs-hi / libctos: fs-ok / fsdemo: ok.
| Piece | Path |
|---|---|
| Sample | user/fs-libctos/ |
| In-app path | /memdemo (ADR-058: /mem prefix → memfs; flat grammar — no /mem/...) |
| FAT slot | /fsdemo (target/fs-libctos.elf via mkfat16.py --app2) |
| Kernel probe | src/fsdemo.rs |
Rebuild: same one-rebuild commands. Keep /hello + slot: ok. Not POSIX. Not getdents. Not a reopen of product app hosting (ADR-048/052).
Recipe 3c — Standing EL0 / libctos FAT sample (fat-libctos)
Third freestanding EL0 payload that opens/reads a FAT path through thin VFS (ADR-061). Same class as libctos: fat-hi / libctos: fat-ok / fatdemo: ok. Deeper than Recipe 3b (memfs only).
| Piece | Path |
|---|---|
| Sample | user/fat-libctos/ |
| In-app path | /probe (ADR-058: / → FAT16; existing A7 payload fat-hi) |
| FAT slot | /fatdemo (target/fat-libctos.elf via mkfat16.py --app3) |
| Kernel probe | src/fatdemo.rs |
Rebuild: same one-rebuild commands. Keep /hello + /fsdemo + slot: ok + fsdemo: ok. Not POSIX. Not getdents. Not a reopen of product app hosting (ADR-048/052).
Recipe 3d — Standing EL0 / libctos yield rounds (yield-libctos)
Fourth freestanding EL0 payload that issues several cooperative yield_now() rounds (ADR-062). Same class as libctos: yld-hi / libctos: beat / libctos: yld-ok / yldemo: ok. Deeper than hello’s single yield. Not preemption. Not multi-task EL0. Not a process table.
| Piece | Path |
|---|---|
| Sample | user/yield-libctos/ |
| FAT slot | /yldemo (target/yield-libctos.elf via mkfat16.py --app4) |
| Kernel probe | src/yldemo.rs |
Rebuild: same one-rebuild commands. Keep /hello + /fsdemo + /fatdemo + slot: ok + fsdemo: ok + fatdemo: ok. Not POSIX. Not preemption. Not a reopen of product app hosting (ADR-048/052).
Recipe 3e — Standing EL0 / libctos net SVCs (net-libctos)
Fifth freestanding EL0 payload that exercises EL0 net SVCs (ADR-068). Same class as libctos: net-hi / libctos: net-mac / libctos: net-ok / netdemo: ok. Kernel owns virtio-net.
| Piece | Path |
|---|---|
| Sample | user/net-libctos/ |
| FAT slot | /netdemo (target/net-libctos.elf via mkfat16.py --app5) |
| Kernel probe | src/netdemo.rs |
Rebuild: same one-rebuild commands. Keep prior samples + N1/N2 markers. Not TCP product. Not sockets. Not “has networking.” Not a reopen of product app hosting (ADR-048/052).
Recipe 3f — Standing EL0 / libctos UDP DNS SVC (udp-libctos)
Sixth freestanding EL0 payload that exercises EL0 net_udp_dns (ADR-071). Same class as libctos: udp-hi / libctos: udp-ok / udpdemo: ok. DNS is probe bait only.
| Piece | Path |
|---|---|
| Sample | user/udp-libctos/ |
| FAT slot | /udpdemo (target/udp-libctos.elf via mkfat16.py --app6) |
| Kernel probe | src/udpdemo.rs |
Rebuild: same one-rebuild commands. Keep /hello + /fsdemo + /fatdemo + /yldemo + /netdemo + N1/N2/N3 markers. Not TCP product. Not sockets. Not a DNS product. Not “has networking.” Not a reopen of product app hosting (ADR-048/052).
Standing EL0 / libctos fs_mkdir sample
Seventh freestanding EL0 payload that exercises EL0 fs_mkdir (ADR-075). Same class as libctos: mkdir-hi / libctos: mkdir-ok / mkdemo: ok. Creates FAT root /edir only — not POSIX mkdir. One-level nested mkdir is a kernel/VFS mile (ADR-077), not this sample.
| Piece | Where |
|---|---|
| FAT slot | /mkdemo (target/mkdir-libctos.elf via mkfat16.py --app7) |
| Kernel probe | src/mkdemo.rs |
Standing EL0 / libctos TCP echo sample
Eighth freestanding EL0 payload that exercises EL0 net_tcp_echo (ADR-076). Same class as libctos: tcp-hi / libctos: tcp-ok / tcpdemo: ok. Quiet thin TCP guestfwd echo — not BSD sockets, not listen/accept.
| Piece | Detail |
|---|---|
| FAT slot | /tcpdemo (target/tcp-libctos.elf via mkfat16.py --app8) |
| SVC | net_tcp_echo (28) |
| Markers | libctos: tcp-hi / libctos: tcp-ok / tcpdemo: ok (keep net: tcp-ok) |
Recipe 4 — memfs named-buffer probe (optional)
In-RAM named buffers behind the thin VFS (ADR-027). Same class as fs: create / fs: write / fs: read / fs: el0 / fs: ok.
| Piece | Path |
|---|---|
| VFS + memfs | src/vfs.rs |
| EL0 trampoline | src/syscall.rs (/eprobe writes memfs-el0) |
| Wrappers | libctos fs_create / fs_open / fs_read / fs_write / fs_close |
Rebuild: same one-rebuild commands. Not POSIX. Not a volume. Not app hosting.
Recipe 5 — FAT16 /probe on virtio-blk (optional)
FAT16 on QEMU virtio-mmio block (ADR-028, write depth ADR-050, multi-cluster grow ADR-064). Same VFS open / write as memfs. Same class as blk: ok / fat: mount / fat: read / fat: write / fat: ok. Guest /probe must read fat-hi; write probe restores it; /fwr holds fat-nw.
| Piece | Path |
|---|---|
| Host image | scripts/mkfat16.py → target/fat16.img |
| Guest virtio | src/virtio.rs |
| FAT + VFS | src/fat.rs, src/vfs.rs |
qemu-smoke builds the image and attaches:
-drive if=none,file=target/fat16.img,format=raw,id=hd0 -device virtio-blk-device,drive=hd0
Host -drive without the guest serial is not the probe. Not POSIX. Not FAT32. Do not say “supports FAT” as a product; say the guest wrote FAT16 bytes when fat: write / fat: create pass, and grew across a cluster boundary when fat: grow / libctos: fat-grow pass. Same-boot CNTPCT pairs vs memfs: write ADR-051 (perf: fat-write / perf: memfs-write / perf: fs-write-delta) and read ADR-065 (perf: fat-read / perf: memfs-read / perf: fs-read-delta) — KPIs; measurement only, not a latency SLA.
Recipe 6 — OS image vs app slot (A9 first cut)
Two host artifacts and a FAT load path (ADR-030). Same class as slot: fat / slot: mapped / slot: ok / perf: app-load.
| Piece | Path |
|---|---|
| OS image | target/aarch64-ctos/debug/ctos (QEMU -kernel) |
| App payload | target/hello-libctos.elf (published by build.rs) |
| FAT slot | scripts/mkfat16.py --app → /hello |
| Guest load | src/slot.rs + A3 src/loader.rs |
A2–A4 load the same FAT file (no embed). Cross-update (same ELF on this OS and ba6541c) is the leftover host smoke, not this recipe. Not OTA. Product freestanding claim is Verified under ADR-048/052 (this recipe page is not that claim by itself).
What cannot run
flowchart TD
Q{"Want to run it on ctos today?"}
Q -->|UART worker / echo / stub / memfs / FAT probe| Y["Yes — extend the kernel in-tree"]
Q -->|Linux binary, shell, Python| N1["No"]
Q -->|Network server or Linux disk apps| N2["No — ARP/ICMP/UDP probe only, no sockets / no POSIX FS"]
Q -->|Docker / OCI container| N3["No — not a goal"]
“Yes” means rebuild the kernel. It does not mean drop in an app.
Do not imply these work:
- Linux binaries (no Linux ABI, no ELF loader for third-party programs)
- A shell
- Python (or any hosted language runtime)
- Network servers (virtio-net ARP + ICMP + minimal UDP + thin TCP probe + EL0 UDP DNS SVC + EL0 TCP echo SVC samples only — ADR-066 / ADR-067 / ADR-070 / ADR-071 / ADR-074 / ADR-076; no BSD sockets / TCP product)
- POSIX / Linux filesystem apps (memfs + FAT16 read/write miles are not that — Filesystem)
- Extra-CPU workloads (one CPU, cooperative yield only)
- Product freestanding app hosting (Verified under ADR-048 / ADR-052; not Linux/POSIX/containers/OTA — issue #48)
Also not claimed: POSIX, GPU, Raspberry Pi, certified security, “production ready,” or containers (non-goal, ADR-029; Hosting apps / containers).
How you would add something in-tree (and why Linux apps do not port): Building or porting.
Prerequisites
What you need to build and run the kernel. Publishing this docs site is optional. GitHub Pages is not required for kernel work.
Kernel (required)
| Need | Why |
|---|---|
Nightly Rust + rust-src + llvm-tools-preview | Freestanding build-std for the custom aarch64-ctos.json target. Pin is rust-toolchain.toml. |
qemu-system-aarch64 | Primary guest is QEMU -machine virt (ADR-003). Debian/Ubuntu package is often qemu-system-arm. |
| A host that can run those tools | Windows, macOS, or Linux. The kernel target stays AArch64. |
rustup toolchain install nightly
rustup component add rust-src llvm-tools-preview
# Debian/Ubuntu: sudo apt-get install qemu-system-arm
cargo build # ELF at target/aarch64-ctos/debug/ctos
cargo run # qemu-system-aarch64 -machine virt
./scripts/qemu-smoke.sh
flowchart LR R["rustup nightly + rust-src"] --> B["cargo build"] B --> Q["qemu-smoke / cargo run"]
That rebuilds the kernel, including any in-tree no_std code you add. It is not a port of a Linux app. See Building or porting.
Commands and honesty notes: GitHub README. A successful cargo build on your machine is not a copied Verified boot from another host.
Optional
| Extra | When |
|---|---|
Docker linux/arm64 | the arm64 workstation (Windows ARM64) path: ./scripts/docker-smoke.sh. Do not pass --platform linux/amd64. |
| Docker on the amd64 workstation (Alpine WSL2) | Install bash (apk add bash); use wsl -d Alpine -u <user> -- bash (not WindowsApps bash). Docker Desktop WSL integration not enabled for Alpine — use docker.exe on PATH (…/DockerDesktop/resources/bin) or cmd docker build/run. scripts/docker-smoke.sh is #!/bin/sh. |
| mdBook 0.5.4 + mdbook-mermaid 0.17.1 | Local docs site only: ./scripts/docs-build.sh. See Docs website + DNS. |
Not required
- GitHub Pages, a custom domain, or
ctos.artof.link(the site is live; you still do not need it to hack the kernel) - Raspberry Pi hardware, an x86_64 boot path, or VGA
- Obsidian (
.obsidian/is gitignored) - AWS / Route 53 credentials (DNS is a publish concern, not a kernel build concern)
Building or porting something to ctos
Plain English. There is no easy POSIX port today. ctos is a kernel you extend in-tree, not a host you drop apps onto.
What already runs (rebuild recipes): What can run today. In-tree index: user/README.md. How to build the kernel: prerequisites. Probes: honesty ledger.
Do not claim an “easy port” path that does not exist.
Today (honest)
Nothing POSIX ports easily.
There is no C library (libc), no dynamic linker, and no POSIX filesystem. Thin VFS + memfs + FAT16 are not Linux open. A1 documented a kernel SVC ABI (exit / uart_write / yield — syscall.md, ADR-021). A2 adds a freestanding libctos CRT you can link in-tree (ADR-022). A3 parses that in-tree ELF on the guest and maps PT_LOAD (ADR-023). That is not an application porting target for Linux binaries. There is no PT_INTERP / glibc path that would run a foreign user-mode binary if you built one elsewhere.
A Linux, musl, or glibc program is a different contract. Recompiling it “for AArch64” does not make it a ctos program.
Easiest today
Write in-tree no_std Rust and ship it as part of the kernel image. Language discipline notes: Why Rust.
Typical shape (A8 recipes — what-can-run.md):
- A cooperative kernel task (same class as
sched: task a/b) or a small kernel module - Use UART /
println!for output;yieldto other tasks - Optional: serial receive for a byte-in gadget
- Optional: link
libctoslikeuser/hello-libctos(published + FAT/hello; not a second-kernel)
Rebuild the whole guest with the existing target:
cargo build # aarch64-ctos.json
./scripts/qemu-smoke.sh # or ./scripts/docker-smoke.sh on linux/arm64
That is rebuild the kernel, not “port an app.” The new code lives in src/ and links into target/aarch64-ctos/debug/ctos.
flowchart LR R["rustup nightly"] --> C["cargo build"] --> S["qemu-smoke"] S --> E["OS ELF + published app ELF<br/>A9 first cut"]
Two host artifacts. QEMU still -kernels the OS ELF. FAT /hello is the app slot. A2–A4 load that file. Cross-update is the leftover two-boot smoke.
Not easy
- Recompile Linux / C / Rust apps against glibc or musl
- Drop in a userspace ELF (
ET_DYNor a LinuxET_EXEC) - Expect
std, files, sockets, threads, or a process table
Those need a loader and a userspace that ctos does not have. The standing user-mode stub is a test mile plus an A1 ABI trip and an A2 libctos hello, not that runtime (el0.md, syscall.md).
Track A miles that already have recipes
A1–A7 are probed. A8 is docs: the recipes on what-can-run.md. Still not “applications port to ctos.” You extend the kernel or link libctos in-tree.
- A stable SVC ABI — A1 (
exit/uart_write/yield). - A freestanding CRT /
libctos— A2. - Guest
PT_LOADof that in-tree ELF — A3. Image is FAT/hello. - Standing EL0 as normal until
exit— A4 / ADR-024. - Isolation cut (identity
.rodata+ PAN ID-field) — A5. PAN is enabled on the default-cpu cortex-a76(ADR-080; PAN enable was a non-goal on the historical-cpu cortex-a57, ADR-047). Leftover identity RAM after the heap is torn (ADR-049);_startstays. - Thin VFS + memfs — A6. Recipe 4.
- virtio-blk + FAT16 read — A7. Recipe 5. FAT16 write depth — ADR-050.
- OS/app slot first cut — A9 / ADR-030. Recipe 6. Leftover cross-update — ADR-032 (Verified host smoke).
A real userspace stays Planned. “EL0 isolated” is Verified only as the accepted ADR-091 sentence (one core, TCG default smoke machine; limits there). Gaps before hosting, and why containers are a non-goal: Hosting apps / containers.
A POSIX filesystem is the same story: not present. Thin VFS + memfs + FAT16 are not Linux open. Direction: Filesystem: new vs extend.
Product hosting still Planned (ADR-048)
An OS image vs app payload first cut and leftover cross-update already exist (A9 #48, ADR-030, ADR-032): same hello-libctos.elf on this OS and documented prior OS ba6541c. Product freestanding “hosting done” is Verified under ADR-048 / ADR-052 (not Linux/POSIX). See hosting-apps.md, overview.md, and immutability.md.
Filesystem: new vs extend
Today: thin VFS with two backends and a prefix mount table. In-RAM memfs (A6 / ADR-027) and FAT16 on virtio-blk (A7 / ADR-028). Same open / read / write / close. Path prefixes route to a backend (ADR-058): /mem (+ A6 probe names) → memfs; / → FAT16 (/probe, plus freestanding samples /hello / /fsdemo / /fatdemo / /yldemo). FAT16 write is a depth mile (ADR-050). Multi-cluster grow is a depth mile (ADR-064). FAT16 root listing is a depth mile (ADR-056). FAT16 root delete is a depth mile (ADR-057). Not POSIX open / mount / getdents / unlink, not Linux VFS, not FAT32, not xv6. Track B B5 compares the concepts: ADR-034. Not claiming a Linux filesystem.
Do not say “supports FAT” as a product. Say the guest read a known FAT16 file when the ledger has fat: ok, wrote FAT16 bytes when it has fat: write / fat: create, grew a file across a cluster boundary when it has fat: grow, listed root entries when it has fat: readdir / fat: entries, deleted a root file when it has fat: delete, created a root directory when it has fat: mkdir, created a one-level nested directory when it has fat: nested, and removed an empty directory when it has fat: rmdir. Hub: honesty ledger, What can run today. Extra stance: filesystem.md.
flowchart TD CALL["EL0 / EL1 callers"] --> VFS["Thin VFS<br/>open · read · write · close · readdir · unlink"] VFS --> MT["Prefix mount table<br/>ADR-058"] MT -->|"/mem · A6 names"| MEM["memfs<br/>in-RAM buffers"] MT -->|"/"| FAT["FAT16<br/>read · write · root readdir · unlink"] FAT --> BLK["virtio-blk<br/>host fat16.img"]
Same thin API, two backends, prefix mounts. Not POSIX open / mount / getdents / unlink. Do not say “supports FAT” as a product.
New vs extend
Prefer implementing a known small filesystem behind this thin VFS over inventing a novel on-disk format.
| Fit | What | Why |
|---|---|---|
| Landed | Ramdisk / memfs | Named heap buffers. Create / lookup / read / write without DMA. |
| Landed | virtio-blk + FAT16 read | Host-visible raw image (scripts/mkfat16.py). xv6-like was rejected so the host can inspect the volume. |
| Landed | FAT16 write | Guest vfs::write on an open FAT handle + backend create of a small file (ADR-050). |
| Landed | FAT16 multi-cluster grow | Guest write allocates a FAT chain and crosses a cluster boundary (ADR-064). |
| Landed | FAT16 readdir | Guest vfs::readdir lists FAT16 root paths (ADR-056). Not POSIX getdents. |
| Landed | FAT16 delete | Guest vfs::unlink deletes a FAT16 root file (ADR-057). Not POSIX unlink. |
| Landed | FAT16 mkdir | Guest vfs::mkdir creates a FAT16 root directory with . / .. (ADR-073). Not POSIX mkdir. |
| Landed | EL0 fs_mkdir | SVC 27 + freestanding /mkdemo (ADR-075). Exists/BadPath fail-closed. Root create from EL0. |
| Landed | FAT16 nested mkdir + empty rmdir | One-level /parent/child + empty vfs::rmdir (ADR-077). Not POSIX. Not arbitrary depth. |
| Landed | ctos-specific virtual mounts | Prefix mounts behind the thin VFS (ADR-058). Not a new magic format. Not mount(2). Tree mounts (extra /) stay later. |
Avoid early
Do not start with ext4, btrfs, ZFS, or NTFS. Those are large, journaled or feature-heavy, and hide the block mile. They are not a first cut.
Host QEMU -drive without guest virtio + a VFS read is not a filesystem.
Roadmap order
One milestone → one branch → one PR.
flowchart LR V["1. VFS ADR<br/>ADR-027"] --> M["2. memfs<br/>A6"] M --> B["3. virtio-blk<br/>A7"] B --> F["4. FAT16 read<br/>A7"] F --> H["5. host image + guest read<br/>A7/A9"] H --> W["6. FAT16 write<br/>ADR-050"] W --> P["7. FAT vs memfs write CNTPCT<br/>ADR-051"] P --> PR["7b. FAT vs memfs read CNTPCT<br/>ADR-065"] PR --> R["8. FAT16 readdir<br/>ADR-056"] R --> D["9. FAT16 delete<br/>ADR-057"] D --> V2["10. Prefix mounts<br/>ADR-058"]
A7 is virtio-blk + FAT16 read. ADR-050 is FAT16 write depth. ADR-051 / ADR-065 are same-boot measurement pairs (not benches). ADR-056 is FAT16 root listing. ADR-057 is FAT16 root delete. ADR-058 is prefix mounts. A8 is documented recipes. A9 / ADR-059 / ADR-061 / ADR-062 use the same volume for /hello / /fsdemo / /fatdemo / /yldemo. Do not say “supports FAT” as a product.
- VFS ADR — thin interface (create / open / read / write / close of a path).
- memfs — in-RAM named buffers; serial
fs: ok. - virtio-blk — virtqueues + sector I/O on QEMU
virt. Serialblk: ok. - On-disk FS — FAT16 (ADR-028). Serial
fat: ok. - Host-checkable image —
scripts/mkfat16.pywritestarget/fat16.img; smoke attaches-drive if=none,file=…,id=hd0 -device virtio-blk-device,drive=hd0. Guestvfs::open("/probe")must readfat-hi. A9 adds--appso/hellois the published app ELF; ADR-059/061/062 add--app2/--app3/--app4for/fsdemo//fatdemo//yldemo. - FAT16 write — guest
vfs::writeon an open FAT handle + backend create of/fwr(ADR-050). Serialfat: write/fat: create; host--check-write. - FAT vs memfs write CNTPCT — same-boot pair
perf: fat-write/perf: memfs-write/perf: fs-write-delta(ADR-051). QEMU TCG lab — not a percent or latency SLA. 7b. FAT vs memfs read CNTPCT — same-boot pairperf: fat-read/perf: memfs-read/perf: fs-read-delta(ADR-065). QEMU TCG lab — not a percent or latency SLA. - FAT16 readdir — guest
vfs::readdirlists root thin-VFS paths (ADR-056). Serialfat: readdir/fat: entries. Not POSIXgetdents. - FAT16 delete — guest
vfs::unlinkdeletes a root file (ADR-057). Serialfat: delete. Not POSIXunlink. - Prefix mounts — mount table routes path prefixes to memfs or FAT16 (ADR-058). Serial
vfs: mount/vfs: mounts. Not POSIXmount(2).
Honesty
| Claim | Probe | Status |
|---|---|---|
| Thin VFS + memfs create/write/read/close | Serial fs: ok + #[test_case] | Verified (A6; honesty ledger) |
| virtio-blk sector R/W | Serial blk: ok + #[test_case] | Verified on this tip when the ledger has the probe |
FAT16 /probe via the same open | Serial fat: ok + #[test_case] | Verified on this tip when the ledger has the probe |
| FAT16 write + small create | Serial fat: write / fat: rewrite / fat: create + host --check-write | Verified on this tip when the ledger has the probe (ADR-050) |
| FAT vs memfs write CNTPCT pair | Serial perf: fat-write / perf: memfs-write / perf: fs-write-delta | Verified on this tip when the ledger has the probe (ADR-051). Not a bench. |
| FAT vs memfs read CNTPCT pair | Serial perf: fat-read / perf: memfs-read / perf: fs-read-delta | Verified on this tip when the ledger has the probe (ADR-065). Not a bench. |
| FAT16 root readdir | Serial fat: readdir / fat: entries + #[test_case] | Verified on this tip when the ledger has the probe (ADR-056). Not POSIX. |
| FAT16 root delete | Serial fat: delete + #[test_case] | Verified on this tip when the ledger has the probe (ADR-057). Not POSIX. |
| FAT16 root mkdir | Serial fat: mkdir + #[test_case] | Verified on this tip when the ledger has the probe (ADR-073). Not POSIX. |
EL0 fs_mkdir sample | Serial libctos: mkdir-ok / mkdemo: ok | Verified on this tip when the ledger has the probe (ADR-075). Not POSIX. |
| FAT16 nested mkdir + empty rmdir | Serial fat: nested / fat: rmdir + #[test_case] | Verified on this tip when the ledger has the probe (ADR-077). Not POSIX. One nest level only. |
| Thin VFS prefix mounts | Serial vfs: mount / vfs: mounts + #[test_case] | Verified on this tip when the ledger has the probe (ADR-058). Not mount(2). |
FAT16 /hello app slot (A9) | Serial slot: ok + #[test_case] | First cut when the ledger has the probe |
Do not claim compatibility with anyone’s existing disk.
Hosting applications — gaps, and containers
What ctos can host today under the scoped Track A product claim (a loaded freestanding user-mode binary with a stable ABI on an OS/app slot), what is still missing for stronger isolation, and whether it can host containers.
Today you also extend the kernel (Building or porting). Samples that exist: What can run today. Isolation notes: el0.md. Kernel SVC numbers: syscall.md.
Do not say ctos hosts Linux programs, POSIX apps, or containers. The scoped freestanding slot-hosting claim is Verified under ADR-048 / ADR-052.
Today vs the slot split (A9 + product claim)
Today (Verified miles + product claim): QEMU -kernel still loads one OS ELF. Published freestanding samples live on FAT /hello (A9), /fsdemo (ADR-059), /fatdemo (ADR-061), and /yldemo (ADR-062). A2–A4 load /hello (no production embed). The same hello ELF boots on this OS and on ba6541c (ADR-032). Under ADR-048 criteria Met×5 (ADR-052), the product sentence “apps run independently of the OS” / “app hosting is done” is Verified for that freestanding slot path. It is not containers, not OTA, and not Linux/POSIX.
Product claim (Verified): “apps run independently / hosting done” is Verified under ADR-048 (checklist below). Scope stays freestanding slots — not Linux userspace.
flowchart LR OS["OS ELF<br/>QEMU -kernel"] --> FAT["FAT16 volume"] H["/hello"] --> FAT F["/fsdemo"] --> FAT D["/fatdemo"] --> FAT Y["/yldemo"] --> FAT FAT --> LOAD["Guest PT_LOAD<br/>standing EL0"] LOAD --> CLAIM["Product hosting<br/>Verified — ADR-048/052"]
Freestanding slot-hosting is Verified (Met×5). Catalog samples deepen recipes; they do not reopen the claim. Do not say Linux hosting, containers or OTA. Say “EL0 isolated” only as the accepted ADR-091 sentence. Runtime cost vs embed is a lab measurement pair (ADR-046), not a marketing delta.
Gaps before hosting applications
These are missing pieces for richer hosting / isolation, not a schedule. Rows without a probe stay Planned or unbuilt. “Later” is not a promise.
A supervisor call (SVC) is how user-mode code asks the kernel for help. Reserved SVC #0–#2 stay test miles. A1 documented exit / uart_write / yield (syscall.md). That is not a CRT or loader.
| Gap | Why it matters | Status |
|---|---|---|
| Stable SVC ABI | A1 documented exit / uart_write / yield (ADR-021). That is a kernel contract, not a loader. | ABI mile — Verified only when the ledger has svc: ok on this tip. Mile ≠ whole product claim. |
| libctos / CRT | A2: libctos wrappers + _start CRT. Hello is host-built and published for the slot path (ADR-022). | CRT mile — Verified only when the ledger has libctos: ok on this tip. Not a guest loader. |
| ELF / user loader | A3: guest ELF64 PT_LOAD into user TTBR0, then ERET to e_entry (ADR-023). A2–A4 load FAT /hello (no production embed, ADR-032). Not a Linux ELF ABI. | Loader mile — Verified only when the ledger has loader: ok on this tip. Not a Linux ABI. |
| OS vs app slots | A9: host kernel ELF + hello-libctos.elf; guest FAT /hello (ADR-030). Same ELF on this OS and ba6541c (ADR-032). | Slot first cut + leftover — Verified only when the ledger has slot: ok and the cross-update host lines on this tip. Product claim uses this mile under ADR-048. |
| Standing user mode as normal | A4: loaded image stands until exit; fail-closed restore (ADR-024). Not a process table. | Standing-task mile — Verified only when the ledger has el0: task-ok on this tip. Not isolation. |
| Stronger isolation | Umbrella “EL0 isolated” accepted by the sponsor 2026-09-28; revision 4 accepted 2026-10-02 (ADR-091; checklist ADR-055 all Met; sponsor closure ADR-060); PAN enabled (ADR-080); _start stub page stays the one identity exception (ADR-089); stock-QEMU taken SError still parks (ADR-053) | Verified only as the ADR-091 sentence: one core, the TCG default smoke machine; not a speculative-execution or side-channel claim, not a real-hardware claim, not a certification; app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN); no multi-core race story |
| VFS + memfs | Thin VFS + in-RAM named buffers (ADR-027); see Filesystem | memfs mile — Verified only when the ledger has fs: ok on this tip. Not POSIX. Not FAT. |
| On-disk FS | virtio-blk + FAT16 behind the same VFS (ADR-028); write depth (ADR-050) | block + FAT mile — Verified only when the ledger has blk: ok / fat: ok (and fat: write / fat: create for write). Not POSIX. Not “supports FAT” as a product. |
| Richer I/O | UART byte in/out only; no TTY, disk beyond FAT, or sockets | UART probed; the rest unbuilt |
| Preemption / extra CPUs / net | Cooperative one-CPU yield; no NIC | Later — not required for the ADR-048 claim |
“Host an application independently of the OS” / “app hosting is done” is Verified under ADR-048 (documented recipes + slot cross-update + no production embed + honesty about missing Linux/POSIX/containers + sponsor accept / ADR-052). That block is Track A. A1–A9 first cuts alone were never enough; the checklist Met×5 is. A5: PAN ID-field Verified; enable is non-goal on default cortex-a57 (ADR-047). A8 documented rebuild recipes (what-can-run.md). A9 + leftover: FAT /hello, no production embed, cross-update on ba6541c. See Immutability.
Containers
Non-goal. ctos cannot host OCI / Docker / Kubernetes workloads, and it is not aiming to.
Those need Linux kernel features (namespaces, cgroups, a Linux ABI, usually overlay or equivalent, a rich syscall surface) that this learning kernel does not have. Decision: ADR-029. Track B B7 (#47).
Today the arrow is the other way: Docker on a host (arm64 workstation linux/arm64) runs the ctos smoke image. That is “Docker hosts ctos,” not “ctos hosts containers.” See the README Docker notes and the Docker rows in the ledger.
flowchart LR HOST["Linux host + Docker"] -->|"builds and QEMU-smokes"| CTOS["ctos guest"] CTOS -.->|"does not host"| OCI["OCI / Docker / k8s"]
Host harness ≠ guest runtime. File presence of this page is not a container probe.
Container support is not Planned on this page. Do not add a Planned row. Reopen only with a new GitHub epic plus a new ADR — not a Track B win, and not a new FR/NFR ID.
Claim criteria (ADR-048)
Product “apps run independently / hosting done” is Verified when all of the following are ledger-Verified and sponsor-accepted (ADR-048 / ADR-052):
- Documented freestanding-app recipes (existing markers only).
- Slot cross-update (same published ELF on this OS and a documented prior OS).
- No production hello embed (
slot: embednever appears on the production path). - Docs still refuse Linux ABI / POSIX / guest containers (ADR-029).
- Explicit sponsor accept on the flip PR (no self-merge).
Progress vs tip main (honesty checklist)
Tip audited: 190ef5a (Merge #106 / docs polish; or newer main with ADR-062). Product row Verified — criteria 1–5 Met. Catalog samples /fsdemo / /fatdemo / /yldemo deepen recipes; they do not reopen ADR-048/052.
| # | Criterion | Status | Tip evidence (cite ledger / tip) |
|---|---|---|---|
| 1 | Documented recipes (A8) | Met | Ledger “Documented sample apps (Track A A8)” Verified; hub what-can-run.md / apps-today.md / user/README.md. |
| 2 | Slot cross-update | Met | Ledger “A9 cross-update …” Verified: cross-update prior-os=ba6541c… slot:ok + cross-update this-os=… slot:ok (ADR-032). Re-probed on later tips (incl. ADR-046 / amd64 workstation Docker rows). |
| 3 | No production embed | Met | Ledger “A2–A4 hello without include_bytes! (ADR-032)” Verified; smoke rejects slot: embed. Probe-only include_bytes! in src/slot.rs only (ADR-046). |
| 4 | Honesty: no Linux/POSIX/containers | Met | Containers non-goal (ADR-029); this page + Track B frame refuse Linux userspace / POSIX / guest OCI. |
| 5 | Explicit sponsor accept | Met | Sponsor: “address 1. app hosting” after still-Planned #5-only review. ADR-052. Flip PR; no self-merge. |
A1–A9 first cuts alone remain insufficient without the checklist. Matching ledger section: honesty-ledger.md — ADR-048 progress.
Honesty
| Claim | Status |
|---|---|
| ctos hosts freestanding slot apps / “hosting done” (ADR-048 scope) | Yes — Verified (ADR-048 / ADR-052) |
| ctos hosts Linux programs / POSIX / glibc/musl | No |
| ctos hosts OCI/Docker containers | No — non-goal (ADR-029) |
| Linux-compat research (Track B) | Frame (ADR-031) + B2 gap map (linux-aarch64-syscall-gap.md) + B3 process-model stance (ADR-035) + B4 ELF/auxv gap (ADR-033) + B5 VFS compare (ADR-034). Not claiming Linux userspace yet. Not claiming dynamic Linux ELF. Not claiming a Linux filesystem. |
Host Docker runs ctos-smoke | Separate ledger row (host tool, not a guest runtime) |
KPIs / how we measure
Plain English. These are the probes this repo actually runs. They are not a product dashboard, a latency promise, or invented KPIs. Numbers live in the honesty ledger as one environment and one revision.
The three first-class pillars are antifragility (NFR-05), security (NFR-10), and performance (NFR-07). “Application support” below is an honest scope statement, not a new requirement ID.
Performance
What we measure today on QEMU virt (serial markers + tests, fail-closed in scripts/qemu-smoke.sh).
CNTPCT is the CPU’s physical cycle counter — a hardware clock we read to see that time moved, not a published bench.
| Probe | What it is | What it is not |
|---|---|---|
Cycle-counter loop (perf: cntpct) | The counter is readable and moves over a fixed trivial loop | A microsecond budget, interrupt latency, or “faster than X” |
IRQ-to-handler delta (perf: irq-delta) | Counter minus the timer’s compare value; min / max / spread on several ticks | A latency SLA, SPEC, or a comparison to other kernels |
Boot-delta (perf: boot-delta) | Cycle count from after paging init to after Hello World! | QEMU process start time, a boot budget, or a published bench |
Host ELF size (perf: elf-size) | Byte size of the debug ctos ELF after cargo build | A size budget or “smaller is better” |
App-load (perf: app-load) | Cycle count around FAT /hello read + A3 map + ERET (A9) | A budget, or “slots are free” |
Embed-load (perf: embed-load) | Probe-only CNTPCT around map + ERET of the same linked-in hello ELF (ADR-046) | Production slot path, slot: embed, or a bench |
Slot delta (perf: slot-delta) | Raw app=<a> embed=<b> tick pair on one boot | A percent, “faster/slower,” or SPEC |
FAT write (perf: fat-write) | CNTPCT around one FAT VFS write of the small /probe rewrite payload (ADR-051) | A write latency SLA or “disk cost” |
Memfs write (perf: memfs-write) | CNTPCT around one memfs VFS write of the same bytes on the same boot | A memfs SLA or published bench |
FS write delta (perf: fs-write-delta) | Raw fat=<a> memfs=<b> tick pair on one boot | A percent, “faster/slower,” or SPEC |
FAT read (perf: fat-read) | CNTPCT around one FAT VFS read of /probe (fat-hi) (ADR-065) | A read latency SLA or “disk cost” |
Memfs read (perf: memfs-read) | CNTPCT around one memfs VFS read of the same bytes on the same boot | A memfs SLA or published bench |
FS read delta (perf: fs-read-delta) | Raw fat=<a> memfs=<b> tick pair on one boot | A percent, “faster/slower,” or SPEC |
Net-ping (perf: net-ping) | CNTPCT around EL0 net_ping quiet ARP+ICMP vs SLIRP (el0_icmp_ping) (ADR-069) | A ping latency SLA, “has networking,” or SPEC |
UDP DNS (perf: udp-dns) | CNTPCT around EL0 net_udp_dns quiet ARP+UDP DNS vs SLIRP (el0_udp_dns) (ADR-072) | A DNS latency SLA, “has networking,” or SPEC |
TCP echo (perf: tcp-echo) | CNTPCT around EL0 net_tcp_echo quiet ARP+thin TCP vs guestfwd (el0_tcp_echo) (ADR-078) | A TCP latency SLA, sockets, “has networking,” or SPEC |
QEMU virt is one guest. It is not Raspberry Pi, not real silicon, and not SPEC. Optimize only after a probe shows a cost. Details: performance.md.
OS slot vs app slot (performance)
ADR-030 first cut exists: FAT /hello + perf: app-load. ADR-032 removes the A2–A4 production embed. ADR-046 adds a probe-only linked-in trip and perf: slot-delta app=<a> embed=<b> on the same boot. Immutability means disconnect OS update from apps. That can cost at runtime or be neutral. We do not invent a percentage, a budget, or “faster than linking the app into the kernel.” The pair is a QEMU TCG lab measurement.
| Kind | Honest claim shape | What it is not |
|---|---|---|
| Compared pair | perf: app-load and perf: embed-load ticks plus perf: slot-delta app=<a> embed=<b> on one guest | A percent, SLA, or “slots are free” |
| Possible cost | Extra VFS/FAT read before the shared parse/map/ERET work | Marketing “slower than embed” without naming the tip + ticks |
| Possible neutral | Steady-state UART print / yield after the app is mapped | A win claimed without a probe |
| Build-time, not a bench | Two host artifacts; FAT slot can change without a kernel rebuild | perf: elf-size is still one image’s byte count. Kernel may include_bytes! only in src/slot.rs for the probe. |
FAT write vs memfs write (performance)
ADR-050 is the write mile. ADR-051 adds a same-boot CNTPCT pair: perf: fat-write vs perf: memfs-write plus perf: fs-write-delta fat=<a> memfs=<b>. That can show a cost shape for going through virtio-blk/FAT versus in-RAM memfs — or be noisy under TCG. We do not invent a percentage, a budget, or “FAT write is X× slower.”
| Kind | Honest claim shape | What it is not |
|---|---|---|
| Compared pair | Raw tick counts for the same small payload on one guest | A percent, SLA, or product KPI |
| Possible cost | Extra block/FAT work vs memfs memcpy | Marketing “slower” without naming the tip + ticks |
| Gate | Keep fat: write / fat: ok; fail closed on missing pair markers | Criterion / invented benches |
Measure first (NFR-07). Do not tune a loader “for speed” on a hunch. Do not copy QEMU ticks into a product slide.
FAT read vs memfs read (performance)
ADR-065 adds the symmetric same-boot CNTPCT pair for read: perf: fat-read vs perf: memfs-read plus perf: fs-read-delta fat=<a> memfs=<b>. Same honesty rules as the write pair — measurement only, not a latency SLA.
| Kind | Honest claim shape | What it is not |
|---|---|---|
| Compared pair | Raw tick counts for the same small /probe payload on one guest | A percent, SLA, or product KPI |
| Possible cost | Extra block/FAT work vs memfs memcpy | Marketing “slower” without naming the tip + ticks |
| Gate | Keep fat: read / fat: ok / ADR-051 write markers; fail closed on missing pair markers | Criterion / invented benches |
Net-ping path (performance)
ADR-069 times the EL0 net_ping quiet path: perf: net-ping ticks=<n>. Smoke checks the prefix only — tick values vary under QEMU TCG. Optional N2 crumbs perf: net-icmp-tx / perf: net-icmp-rx remain lab-only (ADR-067).
| Kind | Honest claim shape | What it is not |
|---|---|---|
| Single-path sample | Raw tick count for quiet ARP+ICMP as SYS_NET_PING does it | A percent, SLA, or product KPI |
| Gate | Prefix perf: net-ping present; reject faster/slower/percent | Exact tick count; criterion / invented benches |
| Non-claim | — | TCP/UDP, sockets, “has networking,” Wi‑Fi |
UDP DNS path (performance)
ADR-072 times the EL0 net_udp_dns quiet path: perf: udp-dns ticks=<n>. Smoke checks the prefix only — tick values vary under QEMU TCG. Window = quiet ARP + UDP DNS as SYS_NET_UDP_DNS / el0_udp_dns (same pattern as ADR-069).
| Kind | Honest claim shape | What it is not |
|---|---|---|
| Single-path sample | Raw tick count for quiet ARP+UDP DNS as SYS_NET_UDP_DNS does it | A percent, SLA, or product KPI |
| Gate | Prefix perf: udp-dns present; reject faster/slower/percent | Exact tick count; criterion / invented benches |
| Non-claim | — | TCP, sockets, DNS product, “has networking,” Wi‑Fi |
Thin TCP echo path (performance)
ADR-078 times the EL0 net_tcp_echo quiet path: perf: tcp-echo ticks=<n>. Smoke checks the prefix only — tick values vary under QEMU TCG. Window = quiet ARP + thin TCP guestfwd echo as SYS_NET_TCP_ECHO / el0_tcp_echo (same pattern as ADR-069/072). Kernel N5 net: tcp-ok stays untimed this mile.
| Kind | Honest claim shape | What it is not |
|---|---|---|
| Single-path sample | Raw tick count for quiet ARP+TCP as SYS_NET_TCP_ECHO does it | A percent, SLA, or product KPI |
| Gate | Prefix perf: tcp-echo present; reject faster/slower/percent | Exact tick count; criterion / invented benches |
| Non-claim | — | BSD sockets, listen/accept, TLS/HTTP, “has networking,” Wi‑Fi |
Stability / antifragility
We do not publish an uptime KPI. We measure whether sensors stay fail-closed and whether a repeated miss becomes a ratchet (NFR-05, antifragility.md).
| Mechanism | What it proves |
|---|---|
scripts/qemu-smoke.sh | Build + required serial strings + tests exit 0. Missing a marker is a fail, not a skip. |
cargo test --features force-fail | The panic path is fail-closed (host/QEMU exit 1). A green suite that cannot fail is not a sensor. |
| Honesty ledger | Every status word is a claim. Verified needs a probe. Unknown = unprobed. Planned = not built yet. Failed = the probe ran and lost. File presence is not QEMU boot. |
| Host ratchets | When a machine finds what CI missed, we keep the Failed row and add a sensor. Example: arm64 workstation Docker on main b2bbb99 missed paging: ok after a larger nightly layout; GHA had been green. |
A green GitHub Actions run is one pair of runners. It is not “every host.” Docker on the arm64 workstation is a separate row.
Application support
Scope only — not a new requirement ID. Concrete examples and the cannot-run list: What can run today.
Security work is probed (heap not-executable, stack guards, read-only code / not-executable data, user-mode miles) — that is not a product security KPI. See security.md. Do not say “production ready.”
Advantages
These are properties of how the repo is run, not a product pitch. They are useful if you want a small AArch64 kernel you can study without inflated status.
The landing Driving principles block (honesty, antifragility, security, performance, document-first) is the force; Track A/B stay subordinate. Visitor-facing source: landing. Deep pillar notes: pillars.md.
Document-first
Vision → architecture → ADR → roadmap stay ahead of code for each loop (document-first — FR-14, NFR-13). Frozen IDs are FR-01–FR-15 and NFR-01–NFR-14. New IDs are not invented in chat.
Probed claims
Status words need a command, a serial capture, a gh run URL, or a file read that matches the claim (honesty — NFR-06, honesty ledger). Unprobed stays Unknown. That is slower to write and harder to game than a green README badge.
QEMU virt learning scope
Primary ISA is AArch64 on QEMU virt + serial UART (ADR-003). The scope is small on purpose: UART, exceptions, paging, heap, a cooperative scheduler, then pillar cuts. It is not a distro and not a board bring-up until a board probe exists.
Why Rust (for this kernel)
Rust is the language of this kernel on purpose. It is not a slogan. It is a tools-and-rules choice for a small AArch64 guest you can study.
- Memory safety by default. Ownership and borrowing catch many C-class bugs (use-after-free, double-free, many buffer overruns) at compile time, before the guest boots.
- No garbage collector, still systems-friendly. You keep control of layout and when things free. There is no stop-the-world collector in the way of a freestanding kernel.
- Safer concurrency relative to unchecked shared mutable state. In safe Rust,
Send/Syncmake “who can share this across threads” a type-system question. That does not invent a scheduler; it reduces a class of data races in safe code. - Zero-cost abstractions and fail-closed paths. You can keep thin wrappers without paying for them at runtime.
Result/Optionpush “did this fail?” into the type, which fits probed, fail-closed sensors better than silentnull. - Modern freestanding tooling.
cargo,no_std, and crates like in-treelibctosare the everyday build path (prerequisites, porting).
flowchart LR S["Safe Rust"] --> M["Ownership / borrowing"] S --> C["Send / Sync"] S --> F["Result / Option"] U["unsafe still required"] --> H["page tables · MMIO · asm"]
Safe Rust helps discipline. unsafe is still how you talk to the machine.
Honesty caveats
unsafestill exists for page tables, MMIO, and assembly. The type system does not cover those edges.- Rust does not invent a correct architecture by itself. Wrong maps, wrong EL, and wrong ABI stay wrong.
- This does not mean “secure OS.” “EL0 isolated” is Verified only as the accepted ADR-091 sentence: one core, TCG default smoke machine; not speculative-execution / side-channel, not real hardware, not a certification. Its clauses (PAN, taken SError, …) each have their own probe rows (limits, honesty ledger).
- Status words still need probes / the honesty ledger. Language choice is not a substitute for a serial capture or a
gh runURL.
Pair with Drawbacks / limits: Rust helps discipline; it does not remove those limits.
Three pillars as requirements
After the cooperative scheduler (M9), antifragility, security, and performance are first-class (ADR-011, pillars.md). They still need probes. “Secure” and “fast” are not default adjectives.
Author ≠ merger
ADR-002: the author does not merge their own PR. artofdream vs cursor[bot]. Same-login Approve is still self-review.
Fail-closed sensors
scripts/qemu-smoke.sh and force-fail are meant to break when a marker disappears. Repeated host misses become ratchets, not extra paragraphs of advice (antifragility.md).
Immutability
Scoped yes. Absolute no. Two different sentences:
Page-table scope (probed today). Some kernel memory is read-only and executable, other memory is writable and not executable (W^X on that image slice — write or execute, not both). Smoke markers: ro: ok (ADR-015), identity .text / .rodata / .data / heap tears, and leftover identity frame RAM after the heap (ident: ram*, ADR-049) on this QEMU virt guest. _start stays (ident: start-stay). Not “the kernel is immutable,” “W^X everywhere,” or full identity teardown. Claim only with a ledger probe.
OS slot vs app slot (first cut + leftover cross-update). The useful product meaning is: apps run independently of the OS on a separate OS slot (the kernel image you -kernel) and app slot (a loaded freestanding user-mode binary). ADR-030 is the first cut: host ctos ELF + hello-libctos.elf, guest FAT /hello, A3 PT_LOAD map. ADR-032 loads A2–A4 from that FAT file (no embed) and boots the same published ELF on this OS and on ba6541c. Product freestanding hosting is Verified under ADR-048 / ADR-052. Do not say OTA / “immutable OS” marketing / Linux hosting.
This is not containers (ADR-029 — non-goal, not later), and not OTA / A-B firmware updates (later and unclaimed). Do not say “immutable OS updates.”
Runtime cost vs neutral is unmeasured. Measure first; no invented numbers. KPIs — OS slot vs app slot.
Limits of this shape: Drawbacks / limits.
Drawbacks / limits
What this project is not, and what is still unfinished. Pair with KPIs / how we measure and product vision.
Learning kernel, not a product
ctos is a research / teaching AArch64 kernel. It is not production-ready, not a desktop, not POSIX, not a container host (non-goal, ADR-029), and not a “secure OS.” Do not treat a green QEMU smoke as certification. Rust helps discipline; it does not remove these limits (Why Rust).
Boot contract still starts at 0x4008_0000
QEMU -kernel and _start stay at 0x4008_0000. High-address work (kernel page-table alias, live code / .rodata / .data / heap / leftover frame-RAM tears) does not mean the kernel moved. Leftover identity RAM after the heap is unmapped (ident: ram*, ADR-049); smoke rejects ident: ram-stay. _start stays mapped by decision (ident: start-stay, ADR-047). Do not claim full identity teardown. See el0.md.
Privileged Access Never — enable Verified on cortex-a76 (ADR-080)
PAN stops the kernel from casually reading user-accessible pages when PSTATE.PAN is set. Default smoke CPU is -cpu cortex-a76 (ADR-079 / #125): ID-field Verified (pan: present) and enable + EL1-vs-EL0 fault Verified (pan: enabled / pan: el1-fault, ADR-080). Historical -cpu cortex-a57 remains pan: absent (ADR-026). Say “EL0 isolated” only as the accepted ADR-091 sentence (below). Do not silent -cpu.
Network is virtio-net ARP + ICMP + minimal UDP + thin TCP (+ tiny EL0 net/UDP SVCs); disk is virtio-blk + FAT16
N1 programs virtio-mmio virtio-net enough to TX/RX one raw Ethernet frame (ARP vs QEMU SLIRP gateway) — ADR-066. N2 adds a kernel-path ICMP echo to 10.0.2.2 (net: ping-ok) — ADR-067. N3 adds a kernel-path UDP datagram to SLIRP DNS 10.0.2.3:53 (net: udp-ok) — ADR-070 (DNS is probe bait, not a DNS product). N5 adds a kernel-path thin TCP active-open + one payload echo via QEMU guestfwd 10.0.2.4:7 (net: tcp-ok) — ADR-074 (not a sockets product). N4 adds a tiny EL0 net SVC (net_mac / net_ping) and freestanding /netdemo — ADR-068. N3.x adds EL0 net_udp_dns and /udpdemo — ADR-071 / track-n.md. That is not BSD sockets, TCP product, DHCP/DNS product, Wi‑Fi, or “has networking / sockets OS.” A7 programs virtio-mmio block and reads a host-built FAT16 image through the thin VFS; ADR-050 adds a guest FAT16 write / small create mile; ADR-051 / ADR-065 add same-boot FAT vs memfs write and read CNTPCT pairs (lab measurements, not benches) (Filesystem, KPIs). That is not a general DMA API, not virtio-pci, not POSIX, and not a Linux rootfs. Input on virt is serial receive. Timer is the virt interrupt controller plus the generic timer.
No real userspace apps
The standing user-mode stub is a mile, not an application runtime. No ELF loader for third-party programs, no libc, no extra CPUs, no GPU. Rebuild recipes: What can run today — four freestanding samples on FAT (/hello, /fsdemo, /fatdemo, /yldemo). How to extend the kernel (not port POSIX): Building or porting. OS vs app slot has an A9 first cut (FAT /hello) plus leftover cross-update on ba6541c (Verified miles). Product freestanding app hosting is Verified under ADR-048 / ADR-052 (not Linux/POSIX). Umbrella “EL0 isolated” is Verified only as the accepted ADR-091 sentence (M6); the other isolation locks stay (ADR-060).
Docs URL
https://ctos.artof.link serves this book (HTTPS Verified after #30 — website.md). A trailing-slash github.io/ctos/ path 404’d on the 2026-09-11 probe; use the custom domain. Publishing mechanics can still lag a new commit until the next main Pages deploy.
Immutability is not absolute
Read-only code / not-executable data, and torn identity ranges (including leftover frame RAM), are scoped probes. Do not upgrade them to “immutable kernel,” “W^X everywhere,” or “OS updates without touching apps.” The A9 OS vs app slot first cut exists; the freestanding product hosting claim is Verified (ADR-048 / ADR-052). Not OTA or containers. Details: Advantages — Immutability.
Other honest gaps
- Umbrella user-mode isolation: Verified only as the accepted ADR-091 sentence (M6, 2026-09-28; checklist ADR-055 all Met; sponsor closure ADR-060). The sentence, verbatim (revision 4, accepted 2026-10-02):
“On the ctos default smoke machine (QEMU
virt,-cpu cortex-a76, TCG, one core), EL0 is isolated from the kernel at the architectural page-table / exception-level boundary: each EL0 task runs on its own TTBR0 with its own ASID; EL0 faults when it reads, writes or executes kernel memory; in every translation table (kernel, user and ASID-B TTBR0, and TTBR1) the only EL0-reachable pages are five allowlisted user slots backed by non-kernel frames, checked fail-closed while a user app runs and at rest; PAN is enabled, including while the kernel handles exceptions from EL0, and EL1 faults on EL0-readable memory (the execute-only app text page is not covered: this CPU has no FEAT_EPAN); lower-EL IRQ and FIQ are taken while EL0 stands; taken lower-EL SError is evidenced by the ADR-090 evidence class; the only identity (VA = PA) mapping left in any TTBR0 is the documented EL1-only_startstub page (ADR-089); and a system call copies from or to an EL0 pointer only if EL0 itself may read (for a copy from EL0) or write (for a copy to EL0) every page in the range. It is not a speculative-execution or side-channel claim, not a real-hardware claim, and not a certification.” Limits that stay with it: one core; the TCG default smoke machine; not speculative-execution / side-channel; not real hardware; not a certification; app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN) (ADR-097, accepted as ADR-091 revision 4 on 2026-10-02; before that EL0 could read its own code bytes, ADR-094); no multi-core race story. Also outside it: task-vs-task isolation beyond ASIDs, DMA / IOMMU / EL2, and taken SError on stock QEMU (still parks, ADR-053)._startstays (ADR-089). - x86_64 is not a primary path.
- Raspberry Pi / other boards: unprobed.
- CI on GitHub is a separate ledger row from a cloud-VM
qemu-smoke.
Archify diagrams
Same-origin views on this site (Documented aid — not Live product status; snapshot from before M6; for “EL0 isolated” see the accepted ADR-091 sentence).
| Diagram | Open |
|---|---|
| QEMU NMI pin vs stock park (architecture) | HTML · JSON |
| Taken SError inject sequence | HTML · JSON |
Honesty
- Taken
el0: serroris Verified opt-in (CTOS_QEMU+CTOS_REQUIRE_TAKEN_SERROR=1); see ADR-083. - Stock distro QEMU still parks (
el0: serror-park). - Not FEAT_NMI / FIQ / BRK as SError. The pin alone is not the umbrella; since M6, “EL0 isolated” is only the accepted ADR-091 sentence.
Source folder: docs/archify/ (copied into the published book/archify/ on Pages). Generated with Archify v3.2.0; the former guided chapters are now cards under each drawing, and the viewer’s Route and Lens controls replace guided views. Regenerate steps (pinned to v3.2.0, ARCHIFY_UPDATE_CHECK_DISABLED=1) are in the folder README.
Related
- Chronify evolution rail — dated milestones (Documented until Pages probe).
Chronify — evolution rail
Dated milestones for ctos (ctsOS). Sister to Archify diagrams.
| Artifact | Open |
|---|---|
| Pages rail (SVG) | |
| Interactive HTML | ctos-evolution.timeline.html |
| IR (source of truth) | ctos-evolution.timeline.json |
Honesty
Documented aid until ctos.artof.link Pages probe this session. Not Live product status. Snapshot from before M6; for “EL0 isolated” see the accepted ADR-091 sentence.
Regenerate with chronify v0.1.0:
node bin/chronify.mjs deliver timeline docs/chronify/ctos-evolution.timeline.json \
docs/chronify/ctos-evolution-rail.svg --surface pages-rail
Product vision
Core principles are the driving force for ctos. Tracks and nicknames are subordinate. Non-negotiable: honesty ledger, antifragility (fail-closed + ratchets), security (threat model, probed mitigations only), performance (measure first), document-first / one-PR loops. principles.md.
What ctos is
ctos is a learning and research bare-metal OS kernel written in Rust for AArch64 (arm64 primary; ADR-003). The near-term product is a QEMU virt guest that owns the machine after -kernel load: UART text, then exceptions, paging, a heap, and a tiny scheduler — in that order.
It exists so we can study kernel mechanics with a document-first harness: claims stay honest, failures become sensors, and agents do not merge their own work. After M9 that harness sits on three first-class pillars — antifragility, security, and performance (ADR-011). Security and speed are not afterthoughts; they still need probes before anyone writes “secure” or “fast.”
What ctos is not
- Not a production operating system, desktop, or app runtime. Probed guest samples: apps-today.md.
- Not a Linux distro, not POSIX, not a container host (non-goal, ADR-029). Linux-compat is a research frame (ADR-031); not claiming Linux userspace yet. Porting stance: building-or-porting.md. No filesystem today: filesystem.md. Gaps to host apps: host-apps.md.
- Not a florist / commerce platform and not a clone of any shop case study.
- Not a Raspberry Pi (or other board) port until a board probe exists.
- Not a claim that QEMU boot, CI, hardware bring-up, a “secure OS,” an “immutable OS,” or a published bench is finished until a probe says so. Scoped immutability: immutability.md.
- Not an x86_64-primary kernel. x86_64 may become a secondary target later; it is not implemented now.
Success (current horizon)
A new session can read the vision, principles, overview, what can run today, building or porting, host-app gaps, architecture, honesty ledger, and latest daily brief, then take one roadmap milestone to a GitHub PR without inventing status.
ctos — Functional & Non-Functional Requirements
Learning/research AArch64 (arm64) Rust bare-metal kernel (not a general-purpose desktop OS).
ISA revision: ADR-003 (2026-09-08) replaced x86_64 / VGA / bootimage wording with AArch64 / UART / QEMU virt. IDs are unchanged. Sponsor authorized the text revision; do not mint new FR/NFR IDs here. The sponsor brief called this “ADR-002”; that number was already the identity-split ADR on the harness tip.
FR-06 stage note (2026-09-09): Stage moved Next → Now when M3 landed VBAR_EL1 + resumable BRK (ADR-004). ID unchanged.
FR-07 stage note (2026-09-09): Stage moved Next → Now when M4 landed the dedicated exception / fatal stacks (ADR-005). ID unchanged.
FR-08 stage note (2026-09-09): Stage moved Next → Now when M5 landed GICv2 + the EL1 physical timer (ADR-006). ID unchanged. FR-08 “later input” is M6 / ADR-007 (PL011 RX on virt). No new FR ID.
FR-09 stage note (2026-09-09): Stage moved Next → Now when M7 landed the EL1 identity map + bump frame allocator (ADR-008). ID unchanged. DTB parse remains staged (Unknown). Heap is M8 / FR-10.
FR-10 stage note (2026-09-09): Stage moved Next → Now when M8 landed GlobalAlloc on a first-fit heap backed by identity-mapped frames (ADR-009). ID unchanged. Growing / slab heaps remain later.
FR-11 stage note (2026-09-09): Stage moved Later → Now when M9 landed cooperative round-robin yield on EL1 (ADR-010). ID unchanged. Preemption / SMP remain later. A documented EL0 SVC ABI is ADR-021 / Track A A1 — not a new FR and not app hosting.
NFR-10 note (2026-09-17, ADR-078): EL0 net_tcp_echo quiet-path CNTPCT (perf: tcp-echo ticks=<n>, ADR-078). Threat-model v1.53. Measure-first only — not a latency SLA. Not BSD sockets. Not listen/accept. Keep net: tcp-ok / libctos: tcp-ok. IDs unchanged.
NFR-10 note (2026-09-17, ADR-077): FAT16 one-level nested mkdir + empty rmdir behind the thin VFS (fat: nested / fat: rmdir, ADR-077). Threat-model v1.52. Not POSIX mkdir/rmdir. Not arbitrary-depth trees. No new SVC. Not TCP CNTPCT. Keep fat: mkdir. IDs unchanged.
NFR-10 note (2026-09-17, ADR-075): EL0 fs_mkdir (SVC 27) + freestanding /mkdemo (libctos: mkdir-ok / mkdemo: ok, ADR-075). Threat-model v1.50. Not POSIX mkdir. Not a directory tree. Not EL0 TCP. Keep fat: mkdir. IDs unchanged.
NFR-10 note (2026-09-16, ADR-073): FAT16 root mkdir behind the thin VFS (fat: mkdir, ADR-073). Threat-model v1.48. Not POSIX mkdir. Not a directory tree. No new SVC this mile. Track N / TCP untouched. IDs unchanged.
NFR-10 note (2026-09-16, ADR-064): FAT16 multi-cluster grow behind the thin VFS (fat: grow / libctos: fat-grow, ADR-064). Threat-model v1.39. Not POSIX write API. Not exFAT. Track N untouched. IDs unchanged.
NFR pillars note (2026-09-09): ADR-011 revises NFR-05, NFR-07, and NFR-10 text in place (antifragility, performance, security as first-class pillars). IDs unchanged. Do not mint NFR-15+.
NFR-10 W^X note (2026-09-10): ADR-012 revises NFR-10 text in place for the heap / coop-stack NX cut. ADR-013 records EL0 direction; the first mile is enter/return + UXN fetch, not isolation. ADR-014 adds unmapped linker-stack guards. IDs unchanged.
NFR-10 / NFR-08 note (2026-09-11): ADR-015 revises NFR-10 text in place for the RO+NX text/data cut (SCTLR.WXN on). ADR-013 adds a user-TTBR0 read mile, an ASID-tagged TLB mile, and a standing EL0 context; ADR-016 adds a TTBR1 private-page first cut; ADR-017 adds EL1 fetch from the TTBR1 RAM alias; ADR-018 adds the first identity-tear cut (split tables + one torn text page); ADR-019 jumps the post-MMU continuation to high VA and unmaps a 16 KiB dedicated identity text range; ADR-020 rewrites rustc vtables to high aliases and unmaps live identity .text after the boot stub (.rodata/.data/heap stay). ADR-021 adds a documented EL0 SVC ABI (exit / uart_write / yield). ADR-022 adds a freestanding libctos CRT that wraps those numbers. ADR-023 adds a guest ELF64 PT_LOAD loader into user TTBR0. ADR-024 promotes standing EL0 so a loaded image runs until exit with fail-closed restore. Those are ABI + CRT + loader + standing-task miles, not app hosting. ADR-025 unmaps identity .rodata after a pointer rewrite (.data/heap stay). ADR-026 reads ID_AA64MMFR1_EL1.PAN on -cpu cortex-a57 (enable stays Planned). ADR-027 adds a thin VFS + in-RAM memfs (create / open / read / write / close; public SVC 19–23). ADR-028 adds virtio-mmio block + FAT16 behind that same VFS (/probe read). ADR-050 deepens FAT16 with guest write + small create. Those are memfs / block-FS miles, not POSIX and not app hosting. The umbrella isolation row stays Planned/non-claim (ADR-047). NFR-08 boot-to-ready CNTPCT is a measurement, not a budget. IDs unchanged.
Status legend: Now = hello-UART / QEMU virt / smoke sensors · Next = near roadmap · Later = aspirational.
Tracker is GitHub (gh). New IDs go through a GitHub issue plus an ADR/docs change — not chat.
Scope
In: freestanding no_std Rust, QEMU -kernel → _start, UART/earlycon output, exceptions/interrupts, paging, heap, basic multitasking, reproducible build/QEMU probes, docs-first harness (honesty, three pillars — antifragility / security / performance — second brain, thin multi-agent).
Out (for now): userspace processes, POSIX, networking stack, GPU, SMP, real-hardware certification, secure-boot productization, Raspberry Pi or other board bring-up (unprobed), x86_64 as a primary path.
IDs below are frozen. Do not invent new FR/NFR IDs in chat; add via issue + ADR/docs change.
Functional requirements
| ID | Requirement | Priority | Stage |
|---|---|---|---|
| FR-01 | Kernel builds as a freestanding no_std / no_main binary for a custom aarch64-*-none target. | Must | Now |
| FR-02 | QEMU (-kernel on virt) loads the kernel ELF and transfers control to a stable _start entry. | Must | Now |
| FR-03 | Kernel can write text to the virt PL011 UART (MMIO, or later earlycon) via print! / println!. Not VGA 0xb8000. | Must | Now |
| FR-04 | Panic handler prints a message (best-effort) and halts without unwinding. | Must | Now |
| FR-05 | Developer can produce a runnable AArch64 kernel ELF and boot it under qemu-system-aarch64 (-machine virt). | Must | Now |
| FR-06 | Synchronous exceptions are handled via VBAR_EL1 vectors (at least a breakpoint / fault path). | Must | Now |
| FR-07 | A fatal exception uses a dedicated stack so overflow does not silently lock the VM. | Must | Now |
| FR-08 | Hardware interrupts: timer via the virt GIC (M5). Input on QEMU virt is PL011 UART RX (M6). The x86-era keyboard wording is this UART path; virtio-keyboard remains later. | Should | Now |
| FR-09 | Kernel reads the firmware/QEMU memory map (DTB when probed) and establishes paging / virtual memory. M7: virt RAM convention + linker __kernel_end + EL1 identity map (ADR-008). DTB walk is staged. | Must | Now |
| FR-10 | GlobalAlloc heap so alloc types (Box, Vec) work in kernel. M8: first-fit list on a 64 KiB identity-mapped frame run (ADR-009). | Should | Now |
| FR-11 | Cooperative or simple round-robin task switching (threads or async tasks). M9: EL1 yield of AAPCS64 callee-saved GPRs; two heap-backed workers (ADR-010). | Should | Now |
| FR-12 | Serial remains usable for headless/CI logs (PL011 or extra earlycon), including later test-exit telemetry. | Should | Next |
| FR-13 | Integration tests that boot in QEMU virt and exit with a deterministic success/fail code (ARM semihosting SYS_EXIT, not x86 isa-debug-exit). | Must | Now |
| FR-14 | Documented milestone path (vision → architecture → ADR → roadmap) stays ahead of code for each stage. | Must | Now |
| FR-15 | Honesty ledger records each major capability as claim + probe + status (Verified / Unknown / Planned). | Must | Now |
NFR-10 note (2026-09-12): ADR-039 revises NFR-10 text in place for the EL0 trampoline without TLBI VMALLE1 cut (after ADR-037/038). Threat-model v1.19. IDs unchanged.
NFR-10 note (2026-09-13): ADR-040 revises NFR-10 text in place for taken lower-EL IRQ while standing. Threat-model v1.20. FIQ/SError lower-EL still park. IDs unchanged.
NFR-10 note (2026-09-13, ADR-041): ADR-041 revises NFR-10 text in place for IRQ-unmasked default standing/task ERET. Threat-model v1.21. Short non-standing trampoline probes stay masked. FIQ/SError lower-EL still park. IDs unchanged.
NFR-10 note (2026-09-13, ADR-049): ADR-049 revises NFR-10 text in place for leftover identity RAM tear (ident: ram / ident: ram-fault / ident: ram-high; ident: ram-stay superseded). Threat-model v1.27. Umbrella isolation / PAN enable / taken SError / yank _start stay decided as in ADR-047. IDs unchanged.
NFR-10 note (2026-09-13, ADR-042): ADR-042 revises NFR-10 text in place for the isolation leftover honesty wrap (ident: start-stay / ident: ram-stay). Threat-model v1.22. Umbrella isolation / PAN enable / FIQ-SError / full _start teardown stay Planned. IDs unchanged.
NFR-10 note (2026-09-13, ADR-043): ADR-043 revises NFR-10 text in place for taken lower-EL FIQ while standing (el0: fiq) and SError park honesty (el0: serror-park). Threat-model v1.23. Taken SError / PAN enable / umbrella isolation stay Planned. IDs unchanged.
NFR-10 note (2026-09-13, ADR-047 / ADR-048): ADR-047 decides isolation leftovers (taken SError deferred/non-goal on this smoke machine; PAN enable non-goal on default cortex-a57; never yank _start; leftover RAM optional Planned; umbrella Planned/non-claim). ADR-048 writes product app-hosting claim criteria (still Planned at that note). Threat-model v1.26. IDs unchanged.
NFR-10 note (2026-09-13, ADR-048 / ADR-052): Product app-hosting claim is Verified under ADR-048 criteria Met×5; sponsor accept recorded in ADR-052. Threat-model v1.29. Still not Linux/POSIX/containers/“EL0 isolated.” IDs unchanged.
NFR-10 note (2026-09-14, ADR-053 / ADR-054 / ADR-055): Isolation-gap locks — taken SError hard-stopped (ADR-053); PAN enable locked non-goal on default cortex-a57 (ADR-054); umbrella “EL0 isolated” non-claim until checklist (ADR-055). Threat-model v1.30. IDs unchanged.
NFR-10 note (2026-09-14, ADR-056): FAT16 root listing behind the thin VFS (fat: readdir / fat: entries, ADR-056). Threat-model v1.31. Not POSIX getdents. IDs unchanged.
NFR-10 note (2026-09-14, ADR-057): FAT16 root delete behind the thin VFS (fat: delete, ADR-057). Threat-model v1.32. Not POSIX unlink. IDs unchanged.
NFR-10 note (2026-09-14, ADR-058): Thin VFS prefix mounts (vfs: mount / vfs: mounts, ADR-058). Threat-model v1.33. Not POSIX mount(2). IDs unchanged.
NFR-10 note (2026-09-28, ADR-091 / M6): the sponsor accepted the ADR-091 revision-3 sentence and scope in writing (B3, 2026-09-28 16:41 CEST). Umbrella “EL0 isolated” (P-SEC-3l) is Verified, only as that sentence: one core, the TCG default smoke machine; not a speculative-execution or side-channel claim, not a real-hardware claim, not a certification; EL0 can read its own code bytes; no multi-core race story. Checklist ADR-055 all Met. Threat-model v1.70. Earlier notes that say the umbrella is Planned / non-claim are as of their dates. IDs unchanged.
NFR-10 note (2026-10-02, ADR-091 revision 4): the sponsor accepted ADR-091 revision 4 in writing (“I accept ADR-091 rev 4”). Only the PAN clause and the code-bytes limit changed: the limit is now “app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN)” (was “EL0 can read its own code bytes”). The umbrella “EL0 isolated” (P-SEC-3l) stays Verified, only as the revision-4 sentence in ADR-091. Threat-model v1.74. The 2026-09-28 note above records revision 3 as of its date. IDs unchanged.
Non-functional requirements
| ID | Requirement | Priority | Stage |
|---|---|---|---|
| NFR-01 Safety | Prefer safe Rust; unsafe only at hardware/FFI boundaries, minimized and justified. | Must | Now |
| NFR-02 Reproducibility | Toolchain pinned (rust-toolchain.toml); target JSON + .cargo/config.toml checked in. | Must | Now |
| NFR-03 Portability (dev) | Build+QEMU path works on Windows/macOS/Linux (host tools); kernel target remains freestanding AArch64. | Must | Now |
| NFR-04 CI / sensors | PR gate runs scripts/qemu-smoke.sh (build + UART hello string + cargo test exit codes). GitHub Actions + optional Docker. | Should | Now |
| NFR-05 Antifragility | First-class pillar (ADR-011). Repeated failures become ratchets (#[test_case], scripts/qemu-smoke.sh grep, Dockerfile pin, GHA) — not README-only fixes. Fail-closed sensors stay on the strongest layer. | Must | Now |
| NFR-06 Honesty | Status words in docs/PRs need a probe; unprobed stays Unknown; no self-merge as “verified.” | Must | Now |
| NFR-07 Performance | First-class pillar (ADR-011). Performance claims need a measurable probe (CNTPCT delta around a known path; IRQ-to-handler CNTPCT−CVAL min/max/spread). No fake benches and no “faster than X.” Optimize only after a probe shows a cost. Not a hard latency budget. | Should | Now |
| NFR-08 Footprint | Debug image size and boot time tracked once measurable; no premature optimization. | Could | Later |
| NFR-09 Maintainability | Clear module layout; ADRs for boot-path, console, or allocator choices. | Must | Now |
| NFR-10 Security | First-class pillar (ADR-011). No secrets in repo. Do not claim “secure OS” / “hardened” / “the kernel is W^X” without a written threat model and a probe. Prefer minimize unsafe (NFR-01). Heap and heap-backed cooperative stacks are PXN (ADR-012). Linker-stack downward overflow hits an unmapped 4 KiB guard (ADR-014). .text/.rodata are RO+X and .data/.bss/live linker stacks are RW+NX; SCTLR.WXN is on (ADR-015). That scoped image cut is not a “secure OS” sentence. Device MMIO XN. Least privilege on IRQ paths. EL0 first mile (enter/return + cannot execute kernel data), the user-TTBR0 read mile, the ASID-tagged TLB mile, and the standing EL0 context are probes (ADR-013). TTBR1 private-page first cut is a probe (ADR-016). EL1 fetch from the TTBR1 RAM alias (plus high VBAR_EL1) is a probe (ADR-017). The first identity-tear cut (split TTBR1 RAM tables + one unmapped identity text page) is a probe (ADR-018). A 16 KiB dedicated identity text range plus a high-VA continuation is a probe (ADR-019). A high-VA vtable / fn-pointer rewrite plus live identity .text tear after the boot stub is a probe (ADR-020); identity -kernel stub stays. An identity .rodata tear after a pointer rewrite is a probe (ADR-025); .data/heap stay; full identity teardown stays Planned. A PAN ID-field read on -cpu cortex-a57 is a probe (ADR-026); PAN enable stays Planned. A documented EL0 SVC ABI (exit / uart_write / yield) is a probe (ADR-021). A freestanding libctos CRT wrapping those numbers is a probe (ADR-022). A guest ELF64 PT_LOAD loader into user TTBR0 is a probe (ADR-023); not a Linux ABI. Standing EL0 as normal mode for a loaded image until exit (fail-closed restore) is a probe (ADR-024). A thin VFS + in-RAM memfs (create / open / read / write / close; SVC 19–23) is a probe (ADR-027); not POSIX (product freestanding hosting is Verified under ADR-048/052 separately). virtio-mmio block + FAT16 behind that same VFS (/probe read) is a probe (ADR-028); FAT16 write + small create is a probe (ADR-050); FAT16 root listing is a probe (ADR-056); FAT16 root delete is a probe (ADR-057); thin VFS prefix mounts are a probe (ADR-058); host -drive without guest code is not that probe. An OS-image vs app-payload slot first cut (published hello-libctos.elf + FAT /hello + A3 map) is a probe (ADR-030). A2–A4 load that FAT file (no hello include_bytes!); the same published ELF on this OS and prior ba6541c is a probe (ADR-032). Identity .data/stacks and heap are torn (ident: data / ident: heap, ADR-037 / ADR-038); standing/EL0 trampoline switches without TLBI VMALLE1 (el0: no-vmalle1, ADR-039); taken lower-EL IRQ while standing returns to EL0 (el0: irq, ADR-040); default standing/task ERET clears IRQ mask (el0: irq-default, ADR-041); taken lower-EL FIQ while standing (el0: fiq, ADR-043); SError park honesty (el0: serror-park, ADR-045 A-clear + QMP attempt); short non-standing trampoline probes stay masked; _start stays (ident: start-stay); leftover identity frames after the heap are torn (ident: ram*, ADR-049); taken SError is hard-stopped/deferred (ADR-053); PAN enable is locked non-goal on default cortex-a57 (ADR-054); never yank _start; umbrella “EL0 isolated” is Verified only as the accepted ADR-091 sentence since M6 (2026-09-28; checklist ADR-055 / ADR-047); product “app hosting is done” is Verified under ADR-048 / ADR-052 (freestanding slots; not Linux/POSIX/containers). Do not claim PAN on -cpu cortex-a57. | Must | Now |
| NFR-11 Observability | QEMU UART + explicit test exit codes are first telemetry; host Grafana out of scope. | Should | Next |
| NFR-12 Multi-agent | Thin roles (engineer / knowledge / coherence / MRC); implementer ≠ merge approver. | Should | Now |
| NFR-13 Document-first | Written milestone acceptance criteria match what code/CI actually prove. | Must | Now |
| NFR-14 Second brain | Session memory + daily briefs in git so agents inherit state without chat archaeology. | Should | Now |
Acceptance sketch (Now)
cargo buildsucceeds with pinned nightly foraarch64-ctos.json.qemu-system-aarch64 -machine virtshows the hello line on serial or honesty ledger says Unknown until probed.scripts/qemu-smoke.shfails closed if the hello string is missing orcargo testdoes not exit 0.cargo testuses ARM semihosting so QEMU exits 0 on pass and 1 on panic (force-fail).- Current-EL
BRKis handled viaVBAR_EL1(serial string and/or#[test_case]) or the honesty ledger says Unknown until probed. - Nested / fatal exception prints a serial marker from a dedicated stack (FR-07) or the honesty ledger says Unknown until probed.
- A timer tick is observable on serial and/or
#[test_case]via the virt GIC (FR-08 / M5) or the honesty ledger says Unknown until probed. - A received PL011 byte is observable on serial (FR-08 input / M6) or the honesty ledger says Unknown until probed.
- MMU-on + allocate-frame / map-unmap is observable on serial and/or
#[test_case](FR-09 / M7) or the honesty ledger says Unknown until probed. Box/Vecon the kernel heap is observable on serial and/or#[test_case](FR-10 / M8) or the honesty ledger says Unknown until probed.- Two cooperative tasks are observable on serial and/or
#[test_case](FR-11 / M9) or the honesty ledger says Unknown until probed. - A CNTPCT baseline delta is observable on serial and/or
#[test_case](NFR-07) or the honesty ledger says Unknown / Planned until probed. - An IRQ-to-handler CNTPCT delta (min/max/spread) is observable on serial and/or
#[test_case](NFR-07) or the honesty ledger says Unknown until probed. - Threat-model v1.1 exists under
docs/framework/security.md; “secure OS” stays unclaimed (NFR-10). - Heap NX / execute-from-heap caught is observable on serial and/or
#[test_case](NFR-10 / ADR-012) or the honesty ledger says Unknown until probed. - Linker-stack guard store faults observably (NFR-10 / ADR-014) or the honesty ledger says Unknown until probed.
- EL0 first mile (SVC return and/or UXN IABORT) is observable on serial and/or
#[test_case](NFR-10 / ADR-013) or the honesty ledger says Unknown / Planned until probed. - Host debug ELF size is printed by
scripts/qemu-smoke.sh(NFR-08) or the honesty ledger says Unknown until probed. - Boot-to-ready CNTPCT (
perf: boot-delta) is observable on serial and/or#[test_case](NFR-08) or the honesty ledger says Unknown until probed. - RO+NX text/data (execute-from-
.dataand write-to-RO-text) is observable on serial and/or#[test_case](NFR-10 / ADR-015) or the honesty ledger says Unknown until probed. - EL0 cannot-read-kernel-data (user TTBR0) is observable on serial and/or
#[test_case](NFR-10 / ADR-013) or the honesty ledger says Unknown / Planned until probed. - ASID isolation (dual ASID without
TLBI VMALLE1+ conflict/stale-entry probe) is observable on serial and/or#[test_case](NFR-10 / ADR-013) or the honesty ledger says Unknown / Planned until probed. - Standing EL0 (
el0: standing/el0: restored,is_active()flips) is observable on serial and/or#[test_case](NFR-10 / ADR-013) or the honesty ledger says Unknown / Planned until probed. - TTBR1 private page (
ttbr1: el1/ttbr1: no el0/ttbr1: ok) is observable on serial and/or#[test_case](NFR-10 / ADR-016) or the honesty ledger says Unknown / Planned until probed. - EL1 fetch from the TTBR1 RAM alias (
ttbr1: el1 exec/ttbr1: vbar) is observable on serial and/or#[test_case](NFR-10 / ADR-017) or the honesty ledger says Unknown / Planned until probed. Identity-kernelstub at0x40080000stays. - Identity-tear first cut (
ident: split/ident: fault/ident: high/ident: no el0/ident: ok) is observable on serial and/or#[test_case](NFR-10 / ADR-018) or the honesty ledger says Unknown / Planned until probed. High-VA continuation plus a 16 KiB dedicated identity text range (ident: jump/ident: range/ident: text) is observable on serial and/or#[test_case](NFR-10 / ADR-019) or the honesty ledger says Unknown / Planned until probed. High-VA vtable rewrite plus live identity.texttear (ident: reloc/ident: live) is observable on serial and/or#[test_case](NFR-10 / ADR-020) or the honesty ledger says Unknown / Planned until probed. Identity.rodatatear (ident: rodata/ident: rodata-fault/ident: rodata-high) is observable on serial and/or#[test_case](NFR-10 / ADR-025) or the honesty ledger says Unknown / Planned until probed. PAN ID field (pan: id=/pan: absent) is observable on serial and/or#[test_case](NFR-10 / ADR-026) or the honesty ledger says Unknown / Planned until probed. Identity.data/stacks tear (ident: data/ident: data-fault/ident: data-high) is observable (NFR-10 / ADR-037) or the honesty ledger says Unknown / Planned until probed. Identity heap tear (ident: heap-reloc/ident: heap/ident: heap-fault/ident: heap-high) is observable (NFR-10 / ADR-038) or the honesty ledger says Unknown / Planned until probed. EL0 entry withoutTLBI VMALLE1(el0: no-vmalle1) is observable (NFR-10 / ADR-039) or the honesty ledger says Unknown / Planned until probed. Lower-EL IRQ while standing (el0: irq) is observable (NFR-10 / ADR-040) or the honesty ledger says Unknown / Planned until probed. Lower-EL FIQ while standing (el0: fiq) and SError park honesty (el0: serror-park) are observable (NFR-10 / ADR-043 / ADR-045) or the honesty ledger says Unknown / Planned until probed. Yank_startdecided never; leftover identity RAM after the heap is torn (ADR-049). Taken lower-EL SError is deferred/non-goal on this smoke machine. PAN is enabled on default cortex-a76 (ADR-080; was a non-goal on the historical cortex-a57). Umbrella “EL0 isolated”: see the M6 note above (ADR-091 sentence only). Do not claim PAN. - EL0 SVC ABI (
svc: yield/svc: user-hi/svc: uart/svc: exit/svc: ok) is observable on serial and/or#[test_case](NFR-10 / ADR-021 / Track A A1) or the honesty ledger says Unknown / Planned until probed. This mile alone ≠ the ADR-048 product claim (Verified separately under ADR-048/052). Not a Linux ABI. libctosCRT (libctos: hi/libctos: ok/libctos: linked) is observable on serial and/or#[test_case](NFR-10 / ADR-022 / Track A A2) or the honesty ledger says Unknown / Planned until probed. This mile alone ≠ the ADR-048 product claim (Verified separately under ADR-048/052).- Guest ELF PT_LOAD loader (
loader: mapped/loader: ok) is observable on serial and/or#[test_case](NFR-10 / ADR-023 / Track A A3) or the honesty ledger says Unknown / Planned until probed. Not a Linux ABI. This mile alone ≠ the ADR-048 product claim (Verified separately under ADR-048/052). - Standing EL0 as normal mode (
el0: task-enter/el0: task-active/el0: task-exit/el0: task-restored/el0: restore-fail/el0: task-ok) is observable on serial and/or#[test_case](NFR-10 / ADR-024 / Track A A4) or the honesty ledger says Unknown / Planned until probed. Not isolation. This mile alone ≠ the ADR-048 product claim (Verified separately under ADR-048/052). - Thin VFS + memfs (
fs: create/fs: write/fs: read/fs: el0/fs: ok) is observable on serial and/or#[test_case](NFR-10 / ADR-027 / Track A A6) or the honesty ledger says Unknown / Planned until probed. Not POSIX. This mile alone ≠ the ADR-048 product claim (Verified separately under ADR-048/052). - virtio-blk + FAT16 (
blk: virtio/blk: cap/blk: rw/blk: ok/fat: mount/fat: read/fat: ok) is observable on serial and/or#[test_case](NFR-10 / ADR-028 / Track A A7) or the honesty ledger says Unknown / Planned until probed. Same VFSopen. Not POSIX. Not FAT32. This mile alone ≠ the ADR-048 product claim (Verified separately under ADR-048/052). - FAT16 write depth (
fat: write/fat: rewrite/fat: create/ host--check-write) is observable on serial and/or#[test_case](NFR-10 / ADR-050) or the honesty ledger says Unknown / Planned until probed. Same VFSwrite. Not POSIX. Not FAT32. This mile alone ≠ the ADR-048 product claim (Verified separately under ADR-048/052). - FAT16 root readdir (
fat: readdir/fat: entries) is observable on serial and/or#[test_case](NFR-10 / ADR-056) or the honesty ledger says Unknown / Planned until probed. Same thin VFS. Not POSIXgetdents. Not FAT32. This mile alone ≠ the ADR-048 product claim (Verified separately under ADR-048/052). - FAT16 root delete (
fat: delete) is observable on serial and/or#[test_case](NFR-10 / ADR-057) or the honesty ledger says Unknown / Planned until probed. Same thin VFS. Not POSIXunlink. Not FAT32. This mile alone ≠ the ADR-048 product claim (Verified separately under ADR-048/052). - Thin VFS prefix mounts (
vfs: mount/vfs: mounts) are observable on serial and/or#[test_case](NFR-10 / ADR-058) or the honesty ledger says Unknown / Planned until probed. Same thin VFS. Not POSIXmount(2). Not a new disk FS. This mile alone ≠ the ADR-048 product claim (Verified separately under ADR-048/052). - Panic path compiles and is reachable in principle.
- This FR/NFR file + honesty ledger live under
docs/.
Traceability
Map roadmap milestones and PRs to FR/NFR IDs in PR descriptions when touching behavior.
Taken lower-EL SError on this smoke machine is deferred / non-goal (hard-stopped) (ADR-053 / ADR-047); research ADR-044; attempt ADR-045. Park honesty stays ADR-043. Product app hosting claim gate: ADR-048.
Technical architecture
ctos is a #![no_std] #![no_main] binary. There is no Rust standard library and no OS underneath. The crate builds for a custom target (aarch64-ctos.json): os: none, panic abort, red zone off, static relocation, soft-float (no early SIMD/FP). See ADR-003. Guest samples (not “apps”): apps-today.md. Porting stance: building-or-porting.md.
Boot path
qemu-system-aarch64 -machine virt -cpu cortex-a57(ormax;gic-version=3is allowed).-kernelloads the kernel ELF into virt RAM (linker base0x40080000).- Entry is
_start(assembly insrc/main.rs): set SP, zero BSS, callkernel_main. - Early output is the virt PL011 UART at
0x0900_0000(src/uart.rs). kernel_maininstallsVBAR_EL1(src/exception.rs, ADR-004) after UART init, then the EL1 identity map plus a TTBR1 private page and a cloned TTBR1 RAM alias (ADR-008, W^X split ADR-012, stack guards ADR-014, RO+NX ADR-015, TTBR1 first cut ADR-016, EL1 fetch mile ADR-017, identity-tear first cut ADR-018, identity.textrange tear ADR-019, live.texttear ADR-020, identity.rodatatear ADR-025), then the frame pool after MMU +SCTLR.C(pre-MMU.bssstores can vanish), then the first-fit heap (ADR-009), then the cooperative scheduler (ADR-010), then GICv2 + CNTP (ADR-006). AfterHello World!it proves map/unmap,Box/Vec, two tasks, heap NX, linker-stack guards, the EL0 first mile + standing context, ASID isolation, the TTBR1 private page and high-VA EL1 fetch, torn identity.textand.rodata, the PAN ID field, CNTPCT + IRQ-delta, several timer ticks, and polls PL011 RX (ADR-007).
flowchart LR Q["QEMU virt<br/>cortex-a57"] --> K["-kernel ELF<br/>@ 0x40080000"] K --> S["_start<br/>SP + BSS"] S --> U["PL011 UART"] U --> M["kernel_main"]
QEMU loads one OS ELF. Early print is UART; everything else starts in kernel_main. No bootloader crate, no VGA.
There is no bootloader 0.9 crate and no VGA buffer. The x86_64 phil-opp path was deleted when this ADR landed.
.cargo/config.toml still needs json-target-spec = true on rustc 1.100 nightly. The AArch64 JSON is taken from aarch64-unknown-none-softfloat plus os: none / numeric widths. That nightly rejected the file until both "abi": "softfloat" and "rustc-abi": "softfloat" were set. Behavior is bare-metal AArch64, abort, rust-lld.
This architecture does not claim Raspberry Pi or other SoC support.
Privilege: EL1 kernel vs standing EL0
The kernel runs at EL1. A freestanding image can stand at EL0 and ask the kernel for help with SVC (exit / uart_write / yield / thin-VFS numbers). That is the ctos process model (ADR-035) — not Linux fork / execve. Many isolation miles are Verified; the umbrella “EL0 isolated” is Verified only as the accepted ADR-091 sentence (M6, 2026-09-28; one core, TCG default smoke machine; not speculative-execution / side-channel, not real hardware, not a certification) (ADR-055; sponsor leftovers closure ADR-060). Details: el0.md.
flowchart TD EL1["EL1 — kernel<br/>page tables · SVC handler · VFS"] EL0["EL0 — standing task<br/>loaded freestanding image"] SVC["SVC<br/>exit · uart_write · yield · fs_*"] UMB["Umbrella “EL0 isolated”<br/>Verified: ADR-091 sentence only — M6"] EL0 -->|"asks"| SVC --> EL1 EL1 -->|"ERET back"| EL0 EL0 -.-> UMB
Standing EL0 + SVC is real. “EL0 isolated” is claimed only as the accepted ADR-091 sentence (one core, TCG; not side-channel, not real hardware, not a certification). Stock-QEMU taken SError still parks, and _start is never yanked (ADR-089 exception).
Memory sketch (identity / TTBR0 / TTBR1)
Simplified view of what the guest maps after bring-up. Live identity .text / .rodata / .data / heap / leftover RAM tears are Verified miles; the QEMU -kernel stub at _start stays mapped (ident: start-stay). High twins live under TTBR1. User TTBR0 is what standing EL0 walks.
flowchart TD
subgraph ttbr0k ["TTBR0 kernel (EL1)"]
ID["Identity map<br/>torn live ranges"]
ST["_start stub stays<br/>0x40080000"]
end
subgraph ttbr0u ["TTBR0 user (EL0)"]
UW["User window<br/>text/rodata + trampoline<br/>omits kernel .data/heap"]
end
subgraph ttbr1 ["TTBR1 high"]
HI["High RAM alias + VBAR<br/>EL1 fetch twin"]
PRIV["TTBR1_PRIV<br/>EL1-only page"]
end
ID --> ST
ID -.->|"high twin"| HI
UW -->|"SVC / ERET"| ID
_start stays while live identity ranges are torn. User TTBR0 is not a full process mm. Do not say “the kernel moved.”
OS vs app slots
QEMU still -kernels one OS ELF. Published freestanding EL0 samples live on the same FAT16 volume: /hello (A9 / UART+one yield), /fsdemo (ADR-059 / memfs via libctos), /fatdemo (ADR-061 / FAT /probe via libctos), /yldemo (ADR-062 / cooperative yield rounds). A2–A4 load /hello (no production embed). Cross-update is the same hello ELF on this OS and prior OS ba6541c. Product freestanding “app hosting done” is Verified under ADR-048 / ADR-052 — not Linux/POSIX/containers/OTA. Recipes: what-can-run.md. Companion: hosting-apps.md.
flowchart LR OS["OS ELF<br/>QEMU -kernel"] --> FAT["FAT16 volume<br/>virtio-blk"] H["/hello<br/>UART + yield"] --> FAT F["/fsdemo<br/>memfs VFS"] --> FAT D["/fatdemo<br/>FAT /probe"] --> FAT Y["/yldemo<br/>yield rounds"] --> FAT FAT --> LOAD["Guest PT_LOAD<br/>→ standing EL0"]
One guest volume, three freestanding samples. Slots disconnect OS rebuilds from the published ELFs. Not a container runtime. Not POSIX.
Thin VFS stack
One thin VFS surface (open / read / write / close / readdir / unlink) with two backends and a prefix mount table: in-RAM memfs and FAT16 on virtio-blk. /mem (+ A6 names) → memfs; / → FAT16. Read, write, root listing, delete, and mounts are depth miles — not POSIX. Stance: filesystem.md. ADR-058.
flowchart TD APP2["Standing EL0 / EL1 callers"] --> VFS["Thin VFS<br/>open · read · write · close · readdir · unlink"] VFS --> MT["Prefix mount table<br/>ADR-058"] MT -->|"/mem · A6 names"| MEM["memfs<br/>in-RAM named buffers"] MT -->|"/"| FAT2["FAT16<br/>read · write · root readdir · unlink"] FAT2 --> BLK["virtio-blk<br/>host fat16.img"]
Same API, two backends, prefix mounts. Say “routed path prefixes through a mount table” when vfs: mounts passes — not mount(2) or “supports FAT” as a product.
Track N — virtio-net path (ARP + ICMP + UDP + EL0 net/UDP SVC)
QEMU user netdev (SLIRP) plus guest virtio-net-mmio. N1 exchanges one ARP request/reply with gateway 10.0.2.2 (ADR-066). N2 sends an ICMP echo and expects a reply (ADR-067). N3 sends one UDP datagram to SLIRP DNS 10.0.2.3:53 and expects a UDP reply (ADR-070) — DNS is probe bait, not a DNS product. N4 exposes tiny EL0 SVCs (net_mac / net_ping) and FAT /netdemo (ADR-068). N3.x adds net_udp_dns and FAT /udpdemo (ADR-071) — kernel still owns virtio-net. No BSD sockets / TCP product. Spec/ADR wins on conflict with this sketch.
flowchart LR QN["QEMU user netdev<br/>SLIRP 10.0.2.2 / 10.0.2.3"] --> VN["virtio-net-device<br/>mmio"] VN --> GTX["Guest TX<br/>ARP / ICMP / UDP"] VN --> GRX["Guest RX<br/>ARP / ICMP / UDP reply"] GTX --> VN GRX --> OK["Markers<br/>net: ok · net: ping-ok · net: udp-ok"] OK --> EL0["EL0 SVCs<br/>net_mac / net_ping<br/>/netdemo"] OK --> EL0U["EL0 SVC<br/>net_udp_dns<br/>/udpdemo"]
Host flags alone are not a probe. Do not say “has networking” or “has sockets.” Cite ADR-063 / ADR-066 / ADR-067 / ADR-070 / ADR-071 / ADR-068 / ADR-071.
Current stage (UART hello + M2–M9 + ADR-011 pillars)
Pl011writer with TX-full wait and\n→\r\n;try_recvonUARTFR.RXFE/UARTDRprint!/println!viaspin::Mutex; fatal / unhandled / IRQ paths write the PL011 without the mutex- After EL1 + VBAR, a bump frame allocator (
src/frame.rs) and identity map (src/paging.rs) turn the MMU on (ADR-008); RAM after__kernel_endis PXN (ADR-012); 4 KiB holes sit under the linker stacks (ADR-014) - First-fit
GlobalAlloc(src/heap.rs) on a 64 KiB identity-mapped frame run (ADR-009);build-stdincludesalloc - Cooperative round-robin (
src/sched.rs): idle on the linker thread stack, two heap-backed workers, AAPCS64 callee-saved yield (ADR-010) kernel_mainprintsHello World!, proves map/unmap (serialpaging: ok), provesBox/Vec(serialheap: ok), proves two tasks (serialsched: task a/sched: task b/sched: ok), proves heap NX (serialwx: ok), proves linker-stack guards (serialguard: ok), proves the EL0 first mile + standing context (serialel0: ok/el0: standing/el0: restored), proves the SVC ABI (serialsvc: yield/svc: user-hi/svc: uart/svc: exit/svc: ok), proves thelibctosCRT (seriallibctos: hi/libctos: ok/libctos: linked), proves the guest ELF PT_LOAD loader (serialloader: mapped/loader: ok), proves standing EL0 as a loaded task untilexit(serialel0: task-enter/el0: task-active/el0: task-exit/el0: task-restored/el0: restore-fail/el0: task-ok), proves ASID isolation (serialasid: ok), proves the TTBR1 private page and high-VA EL1 fetch (serialttbr1: ok/ttbr1: el1 exec/ttbr1: vbar), proves the identity-tear first cut,.textrange tear, vtable rewrite, live.texttear, and identity.rodatatear (serialident: ok/ident: jump/ident: reloc/ident: range/ident: live/ident: rodata/ident: rodata-fault/ident: rodata-high/ident: text/ident: split/ident: fault/ident: high/ident: no el0), proves the PAN ID field (serialpan: id=/pan: absent), proves CNTPCT advances (serialperf: cntpct delta=…), observes CNTP ticks (serialtimer: tick) plus IRQ-to-handler deltas (serialperf: irq-delta), polls one host-injected RX byte (serialinput: rx 0x41), fires one healthy-stackBRK #0(serialexception: sync BRK), then the FR-07 nest probe (serialexception: fatal nested)VBAR_EL1vector table (high alias after MMU, ADR-017); kernel runs onSP_EL0; first-level current-EL sync (SP_EL0 bank) handles AArch64BRK, heap NX, guard-page data aborts, and the identity-tear IABORT; first-level IRQ handles GICv2 PPI 30; lower-EL AArch64 sync handles the EL0 SVC / UXN IABORT / standing dual-SVC / public SVC ABI / TTBR1 DABORT / torn-page DABORT; nested current-EL (SP_ELx bank) switches to the fatal stack (ADR-004, ADR-005, ADR-006, ADR-007, ADR-013, ADR-016, ADR-017, ADR-018, ADR-019, ADR-020, ADR-021)cargo testuses#![feature(custom_test_frameworks)]and#[test_case](including VBAR, BRK, SPSel, stack ranges, guards, GIC TYPER, CNTFRQ, timer tick, IRQ-delta samples, empty UART RX FIFO, MMU on, frames, map/unmap, heapBox/Vec, two-task yield, CNTPCT loop, W^X flags + execute-from-heap, EL0 first mile + standing enter/leave, SVC ABI yield/uart/exit + kernel-.data/ TTBR1-alias reject, libctos hello trip, guest ELF loader trip, standing-task until exit + fail-closed fault restore, ASID isolation, TTBR1 private page + high-VA EL1 fetch, identity-tear first cut +.textrange tear + vtable rewrite + live.texttear;is_active()is false at rest)- QEMU exit is ARM semihosting
SYS_EXIT/hlt #0xf000(src/qemu.rs), notisa-debug-exit. Needs-semihostingon the QEMU line (scripts/qemu-aarch64.sh). - Host smoke:
scripts/qemu-smoke.sh(hello + paging + heap + two-task sched + W^X + guards + EL0 first mile + standing + SVC ABI + libctos CRT + guest ELF loader + standing-task (A4) + ASID isolation + TTBR1 private page + high-VA EL1 fetch + identity-tear first cut + identity.textrange tear + vtable rewrite + live.texttear + CNTPCT baseline + IRQ-delta + host ELF size + timer tick + injected UART RX + BRK + fatal nested strings + tests +force-failmust be non-zero) - Docker:
Dockerfile/scripts/docker-smoke.sh(linux/arm64-friendly; do not pin amd64) - GHA:
.github/workflows/smoke.yml(ubuntu-24.04-armandubuntu-24.04)
Source + local smoke were first probed on 2026-09-08 (see the honesty ledger). Current idle tip is main ≈ 190ef5a (Merge PR #106 / docs polish after ADR-061). Recent docs/miles: ADR-058–061 plus ADR-062 yield-libctos (this PR when merged). Earlier Docker Verified on tip e80dc93 (Merge PR #28 / ADR-020; 50 tests, ident: reloc n=12, live pages=37, force-fail ok) and prior SHAs 24d94e6 (ADR-019), b0f0ee5 (#24–#26), 71ee15f (layout L3). Keep the b2bbb99 Failed row. GHA merge-commit 34651404108 on e80dc93 grepped that ADR-020 class of markers. A Route 53 CNAME ctos.artof.link → artofdream.github.io. exists; https://ctos.artof.link HTTPS is Verified (2026-09-11 after #30). Do not claim “secure OS” or “the kernel moved”; say “EL0 isolated” only as the ADR-091 sentence. Tip smoke / Docker / GHA on 97a8559 itself: see the honesty ledger (do not invent Verified from this docs edit).
Planned stages
| Stage | Domain work |
|---|---|
| Custom test framework | Landed (M2): #[test_case], semihosting exit, UART |
| CPU exceptions | M3: VBAR_EL1, resumable BRK. M4: dedicated exception + fatal stacks (FR-07) — cloud qemu-smoke Verified (honesty ledger); GHA Unknown until a run URL |
| Hardware interrupts | M5: GICv2 + CNTP tick (FR-08) — cloud qemu-smoke + GHA Verified (honesty ledger). M6: PL011 UART RX (FR-08 input / ADR-007) |
| Paging | M7: EL1 identity map + bump frames (FR-09 / ADR-008) — probe status in the honesty ledger. DTB walk staged. |
| Heap | M8: first-fit GlobalAlloc on identity-mapped frames (FR-10 / ADR-009) — probe status in the honesty ledger. |
| Scheduler | M9: cooperative EL1 yield (FR-11 / ADR-010) — probe status in the honesty ledger. Not preemptive. |
| Pillars | ADR-011: antifragility / security / performance. Threat-model v1.37 (security.md). Heap NX (ADR-012). Linker-stack guards (ADR-014). EL0 first mile + standing + ASID TLB mile; TTBR1 first cut (ADR-016); EL1 high-VA fetch (ADR-017); identity-tear first cut (ADR-018); identity .text range tear (ADR-019); live .text tear (ADR-020); SVC ABI (ADR-021); libctos CRT (ADR-022); guest ELF PT_LOAD loader (ADR-023); standing EL0 as normal mode (ADR-024); identity .rodata tear (ADR-025); PAN capability (ADR-026, enable locked non-goal on default a57 per ADR-054); thin VFS + memfs (ADR-027); virtio-blk + FAT16 (ADR-028); FAT16 write (ADR-050); FAT16 readdir (ADR-056); OS/app slot first cut (ADR-030); leftover cross-update + FAT-only hello (ADR-032); freestanding sample catalog /hello + /fsdemo + /fatdemo + /yldemo (ADR-059 / ADR-061 / ADR-062); umbrella “EL0 isolated” Verified only as the ADR-091 sentence since M6 (ADR-013 / ADR-047 / ADR-055; sponsor leftovers closure ADR-060). Product freestanding app hosting Verified under ADR-048 / ADR-052 (Track A #31; not Linux/POSIX/containers). |
| Filesystem | Thin VFS + memfs (ADR-027) + virtio-blk / FAT16 read (ADR-028) + write (ADR-050) + root readdir (ADR-056) + root delete (ADR-057) + prefix mounts (ADR-058). FAT also carries freestanding samples /hello / /fsdemo / /fatdemo / /yldemo (catalog, not a second FS). Stance: filesystem.md. |
| Host apps / containers | Gaps to Linux/shell/Python: host-apps.md. Linux-compat is a research frame (ADR-031); not claiming Linux userspace yet. Guest container runtime is a non-goal (ADR-029). Host docker-smoke is unrelated. A1 is a kernel SVC ABI mile. A2 is a libctos CRT mile. A3 is a guest ELF PT_LOAD loader mile. A4 is standing EL0 as normal mode for a loaded image. A6 is thin VFS + memfs. A7 is virtio-blk + FAT16. A8/A9 recipes + slot; freestanding catalog /hello + /fsdemo + /fatdemo + /yldemo (ADR-059 / ADR-061 / ADR-062). |
| Immutability | Scoped RO only (immutability.md). Absolute “immutable OS” is incompatible. Track A #31 / Track B #40. |
Each stage is one loop unit on the roadmap.
x86_64 remains a possible future secondary ISA. It is not a current tree.
Harness mapping (short)
Hardware and QEMU are the domain. Docs, ADRs, and this architecture note are shared understanding. Guides, sensors, the one-PR loop, second-brain vaults, merge permissions, and the honesty ledger are the outer harness. Details: formula.md.
ADR-001 — Apply a honesty/harness practice to ctos
- Status: Accepted
- Date: 2026-09-08
Context
ctos is a bare-metal kernel. Agents and humans will extend it across sessions. Status words drift: “it boots” gets written because source exists, or because a README says cargo run. Without a probe, that is fiction.
Harness-engineering practice (guides, sensors, a tight loop, persistent memory, no self-merge, observability of claims) is useful here. Prior art: architecture.artof.link. This ADR does not adopt that site’s product, florist domain, or stakeholder roster.
Decision
Apply a ctos-native honesty/harness practice:
- Honesty — every status word is a claim; the honesty ledger records the probe. Unprobed = Unknown. Never round Unknown up to Verified.
- Document-first — vision, architecture, ADRs, and roadmap land before more kernel features.
- Second brain — four vaults under
research/(procedure via skills, correction via constraints/rules, relationship via doc links, daily brief / session memory). - Thin roles — Knowledge Guardian, Coherence Guardian, Kernel Engineer, MR Coordinator (
ctos-*skills only). - Loop — one milestone → one branch → one GitHub PR. The author does not self-approve or merge (see ADR-002).
- Sensors over vibes — when a failure repeats, strengthen a sensor or gate (see antifragility.md).
Tracker and reviews stay on GitHub (gh). No GitLab workflow is part of this repo.
Consequences
- README and docs may teach
cargo runwithout claiming QEMU boot is Verified. - New kernel work cites a roadmap milestone and updates the ledger when something is actually probed.
- Role names stay
ctos-*. Do not introduceaea-*hats or shop case-study content.
ADR-002 — PR author ≠ merger (two GitHub identities)
- Status: Accepted
- Date: 2026-09-08
- Amended: 2026-09-12 (Track A historical exception + going-forward restatement)
Context
NFR-12 says implementer ≠ merge approver. On a solo GitHub login, a second Cursor agent cannot APPROVE a PR that login opened (GitHub self-APPROVE rule). Enabling “authors can approve their own PRs” would count same-login review as a gate. That is theater.
Café Fausse (artofdream/aea-interactive-design, .cursor/skills/pr-coordinator/SKILL.md) already probed this: cursor[bot] is the distinct second identity. Same principle applies here. This ADR does not import that repo’s restaurant/SRS content.
Cloud / Cursor agents on this repo often open PRs as artofdream. Commits commonly list cursoragent plus Co-authored-by: artofdream. That is still an owner-opened PR. A trailer does not change the GitHub author login and does not authorize a same-login merge.
Decision
Reuse that split on ctos:
- Do not enable GitHub author self-APPROVE. Do not add a required-approval ruleset until a second human exists.
- Author does not merge their own PR.
- Identities:
artofdream(owner) andcursor[bot](Cursor GitHub App, already used for cloud PRs). Do not install another App or mint a second personal account for this. - Written MRC (
ctos-mr-coordinator) is the review-in-the-room. PreferCOMMENT/REQUEST_CHANGES. Record who authored / who reviewed / who merges on the PR. Name those logins fromgh pr view --json author,mergedBy(or the GitHub PR header), not from a daily-brief guess. - If
cursor[bot]RESTAPPROVEreturns 403 (probed on Café Fausse #27; unprobed on ctos), do not block a validcursor[bot]merge of anartofdream-authored PR after this-run green checks (when CI exists) and Bugbot resolved-or-declined. Missing Approve is not a reason to treat that merge as invalid. - Owner PAT /
artofdreamstill must notAPPROVEor merge anartofdream-authored PR.
Going-forward (2026-09-12 restatement)
The GitHub PR author login is the identity that matters.
| Who GitHub shows as author | Who writes the review | Who merges |
|---|---|---|
artofdream | New MRC session (COMMENT) | cursor[bot] after this-run green checks (when CI exists) and Bugbot resolved-or-declined. Owner must not Approve or merge. |
cursor[bot] (cloud / this agent when GitHub actually attributes the PR to the App) | Owner, optionally plus MRC COMMENT | artofdream. The bot must not merge. |
No self-merge. Owner-opened → bot merges. Bot-opened → owner merges.
Historical exception — Track A (A1–A9)
Named exception. Do not rewrite those PR pages. Sponsor directed this cleanup after Track A closed its first cuts.
- When: 2026-09-11–2026-09-12
- Scope: Track A first-cut PRs A1–A9: #49 (A1), #52 (A2), #53 (A3), #54 (A4), #55 (A5), #56 (A6), #57 (A7), #59 (A8), #60 (A9)
- What
ghshowed (2026-09-12): each of those PRs hadauthor=artofdreamandmergedBy=artofdream. Commits were typicallycursoragentwithCo-authored-by: artofdream. That is same-login merge. It does not satisfy this ADR. - What this is not: not a claim those kernel or docs miles are invalid (their probes stay on their own ledger rows); not a rewrite of git history; not permission to repeat the pattern.
- Earlier miles: many pre-Track-A merged PRs show the same GitHub
author/mergedBypair. This note names Track A as the sponsor-directed exception. It does not relitigate those older PR pages.
After this amendment, a same-login merge is a process miss, not an undocumented habit.
Consequences
- Probe the GitHub
authorfield before naming the merge hat. Chat “expectedcursor[bot]” is not a probe. - #2:
ghon 2026-09-12 showsauthor=artofdream,mergedBy=artofdream. The 2026-09-08 sentence that called itcursor[bot]-authored is withdrawn. - Smoke CI exists and has been probed on this repo (honesty ledger CI rows). “Green checks” on a new PR still need that PR’s run. Missing this-run green is not green.
cursor[bot]merge and AppAPPROVEon this repo stay Unknown until a distinct-identity merge is probed here. Do not copy Café Fausse #16/#27 as a ctos probe. Track A same-login merges are not that probe.
ADR-003 — Primary ISA is AArch64
- Status: Accepted
- Date: 2026-09-08
The sponsor brief named this decision “ADR-002.” On the harness tip, ADR-002 is already the author ≠ merger identity split. This file is ADR-003 so that accepted decision stays intact. Frozen FR/NFR IDs are unchanged; ISA-specific text is revised under this ADR.
Context
The sponsor host (arm64 workstation) is Windows ARM64 with Docker linux/arm64. Native AArch64 QEMU is the right guest. Keeping x86_64 as primary would mean a translated or foreign ISA on that machine.
The previous primary path (phil-opp VGA text at 0xb8000, bootloader 0.9, bootimage, qemu-system-x86_64) is superseded. It was a tutorial-era lift, not a host-ISA decision.
x86_64 may be noted later as a secondary target. It is not implemented in the change that lands this ADR.
Decision
- Primary ISA is AArch64 (arm64). The checked-in custom target is
aarch64-ctos.json(aarch64-unknown-nonestyle:os: none, panic abort, static relocation, soft-float so early boot does not require enabling SIMD/FP inCPACR_EL1). - Boot path is QEMU
-kernelof the kernel ELF on-machine virt(orvirt,gic-version=3) with-cpu cortex-a57ormax. Entry is_startat a linker-script address in virt RAM. Nobootloader0.9 crate and nobootimagedisk as the primary path. - Console is the virt PL011 UART (MMIO
0x0900_0000) viaprint!/println!. Not VGA0xb8000.earlyconremains an allowed later probe; this ADR does not claim it. - Do not claim Raspberry Pi or other board support until a board-specific probe exists. virt is the supported guest.
- Delete the x86-only primary tree (
x86_64-ctos.json,src/vga_buffer.rs) rather than leave a broken dual tree.
Consequences
- New target JSON, linker script, and UART writer replace the VGA / bootloader 0.9 stack.
.cargo/config.tomldefault target isaarch64-ctos.json.json-target-specstays for current nightly.- Roadmap M0/M1 probes are UART +
qemu-system-aarch64, not a VGA dump at0xb8e60. - FR-01, FR-02, FR-03, FR-05 (and other x86-specific wording) are revised in place. No new FR/NFR IDs.
- Later exception/interrupt work is VBAR / GIC, not IDT / PIC / TSS. M3 / ADR-004 is the
VBAR_EL1+BRKpath. - Honesty ledger rows for the x86 VGA path are historical. They do not verify the AArch64 path.
ADR-004 — EL1 VBAR and resumable BRK
- Status: Accepted
- Date: 2026-09-09
Context
FR-06 requires synchronous exceptions via VBAR_EL1 (at least a breakpoint / fault path). Roadmap M3 is that path. M4 (FR-07) is a dedicated fatal stack. M5 (FR-08) is GIC IRQs.
QEMU virt -kernel usually starts the guest at EL1. -machine virt,virtualization=on starts at EL2. Exceptions taken to the current EL use that EL’s VBAR_*. Staying at EL2 and programming only VBAR_EL2 would not satisfy FR-06.
Decision
- Live at EL1.
exception::initcallsensure_el1: ifCurrentELis EL2, setHCR_EL2.RW, copySPtoSP_EL1, andERETto EL1h (DAIF masked). If the EL is not 1 after that, print and park. Do not treatVBAR_EL2as the primary table. - One 2 KiB-aligned table at
exception_vectors, written toVBAR_EL1. All sixteen AArch64 slots exist. Current EL / SP_EL0 / synchronous saves a frame and mayERET. Current EL / SP_EL0 / IRQ is live as of ADR-006. Every other slot parks (UART line +wfe, or semihosting fail undercargo test/force-fail). - Context format (current-EL sync only):
x0–x29,x30,ELR_EL1,SPSR_EL1,ESR_EL1. No SIMD/FP save (soft-float, ADR-003). BRKis resumable. ESR exception class0x3C(AArch64BRK) increments a counter, printsexception: sync BRK, adds 4 toELR_EL1, and returns. Other synchronous exceptions are fatal for this milestone (print + park). That is not FR-07’s dedicated overflow stack.
Consequences
#[test_case]can executebrk #0and continue. The hello kernel fires oneBRKsoscripts/qemu-smoke.shcan require the handler string on serial.- Lower-EL and FIQ/SError stubs are parks. Current-EL IRQ is M5 / ADR-006, not a syscall ABI.
- A nested fault while
println!holds the UART mutex can deadlock. ADR-005 (M4 / FR-07) splitsSP_EL0/SP_EL1, adds a fatal stack, and uses a raw UART write on that path. - This ADR does not claim Raspberry Pi, EL0, or a taken lower-EL exception.
ADR-005 — Dedicated exception and fatal stacks
- Status: Accepted
- Date: 2026-09-09
Context
FR-07 requires a fatal exception to use a dedicated stack so overflow does not silently lock the VM. Roadmap M4 is that path. ADR-004 already notes that a nested fault while println! holds the UART mutex can deadlock.
M3 ran the kernel and first-level current-EL exceptions on the same SP_EL1 (SPSel = 1). A nested sync exception would store another 272-byte frame on that stack. Without paging a downward overflow is not a hardware fault on QEMU virt — it smashes .bss / code with no serial evidence. M7 identity-maps RAM as one Normal block, so this is still true (no guard pages).
A data abort to an unused physical hole is QEMU-map-dependent. A nested BRK from the first-level handler is a real current-EL exception we already know how to take (FR-06).
Decision
- Thread stack on
SP_EL0. AfterVBAR_EL1is installed, copy the_startstack intoSP_EL0(MSR SP_EL0is legal at EL1), write__exc_stack_topintoSPwhileSPSelis still 1 (that isSP_EL1—MSR SP_EL1at EL1 is UNDEF), thenmsr spsel, #0. Normal kernel code uses the 64 KiB thread stack. - Exception stack on
SP_EL1. First-level current-EL exceptions (vector bank “Current EL, SP_EL0”) use the 16 KiB__exc_stack_*region automatically. The live sync slot moves from offset0x200(M3 / SP_ELx) to0x000. - Fatal stack before any nested store. Current-EL / SP_ELx slots (
0x200–0x380) loadSPfrom__fatal_stack_top(8 KiB) in asm, then callhandle_fatal_exception. Do not push a frame on the exception stack that may already be exhausted. - Raw UART on fatal / unhandled paths. Write the PL011 without
spin::Mutexso a nest duringprintln!still produces serial. - Probe is nested
BRKafter a near-empty thread SP. The hello kernel fires a healthy-stackBRK(M3 string), then sets a flag, movesSP_EL0to__stack_bottom + 64(less than the 272-byte frame), andBRKs again. The first-level handler printsexception: sync BRKfromSP_EL1, thenBRKs while SPSel = 1. That nest must printexception: fatal nestedand park. This is not an MMU stack-overflow fault and not a GIC path (M5 / FR-08).
Consequences
#[test_case]can stillbrk #0and return. Tests do not set the nest flag.scripts/qemu-smoke.shrequires the fatal marker in addition to hello + BRK. Afatal probe missedline is fail-closed.- Lower-EL and first-level FIQ/SError stubs still park. First-level IRQ is M5 / ADR-006. Nested IRQ/FIQ/SError still use the fatal stack. Taking a nested IRQ remains unprobed.
- This ADR does not claim Raspberry Pi or x86. Unmapped linker-stack guard pages are ADR-014 (a later cut; the nested-
BRKprobe here still stands).
ADR-006 — GICv2 and the EL1 physical timer
- Status: Accepted
- Date: 2026-09-09
Context
FR-08 requires hardware interrupts via the virt GIC, with a timer tick observable. Roadmap M5 is that path. ADR-004 installed VBAR_EL1; ADR-005 split stacks. First-level IRQ was still a park.
QEMU -machine virt can expose GICv2 or GICv3 (gic-version=2|3|4|host|max). The current smoke line is -machine virt with no gic-version override. QEMU 8.2’s virt default is GICv2 (distributor at 0x0800_0000, CPU interface at 0x0801_0000). GICv3 uses a redistributor at 0x080A_0000 instead of that CPU interface.
The ARM generic timer is already in the core (CNTP_*_EL0 / CNTV_*_EL0). No extra virtio device is required.
Decision
- Program GICv2, matching default
-machine virt. Do not add a GICv3 redistributor driver in this milestone. If a later QEMU default flips to v3, pingic-version=2on the smoke line or add a v3 driver in a new ADR. - Use the non-secure EL1 physical timer (
CNTP_CTL_EL0/CNTP_TVAL_EL0/CNTPCT_EL0) and PPI 30. Ifensure_el1drops from EL2, setCNTHCTL_EL2.EL1PCTEN|EL1PCENand zeroCNTVOFF_EL2so EL1 can use the counter and physical timer. - Take IRQs on the current-EL / SP_EL0 bank (vector offset
0x080). The kernel runs withSPSel = 0, so first-level IRQs useSP_EL1(the M4 exception stack). Save the same GPR + ELR/SPSR/ESR frame as sync, then EOI. - Keep DAIF.I masked except for an observe window. Hello and
#[test_case]unmask, wait for one tick (or aCNTPCTtimeout), then remask and stop the timer so M3BRK/ M4 nested fatal are not interrupted. - Raw UART on the IRQ path. The handler must not take
spin::Mutex(same reason as ADR-005). First tick printstimer: tick.
Consequences
scripts/qemu-smoke.shrequirestimer: tickand rejectstimer: tick missed, then still requires the M3/M4 strings.#[test_case]can wait for a tick with IRQs unmasked and continue.- FIQ, SError, lower-EL, and unexpected IRQ IDs stay parks / raw markers. UART input is M6 / ADR-007.
- This ADR does not claim Raspberry Pi, GICv3, virtualization=on as the primary path, or a taken FIQ.
ADR-007 — PL011 UART RX as virt input
- Status: Accepted
- Date: 2026-09-09
Context
FR-08 already covers hardware interrupts (timer via the virt GIC, M5 / ADR-006) and named “later input” as the rest of that ID. Roadmap M6 is that input path. The x86-era reading of FR-08 was a keyboard. Frozen IDs stay; the arm64 text is revised here.
Options on QEMU virt:
- PL011 UART RX at
0x0900_0000(already the console). - virtio-input / virtio-keyboard (virtio-mmio, virtqueues, DMA). No paging or heap yet (M7/M8). Too much device stack for one milestone.
- PL011 UARTCR.LBE loopback as a self-test. QEMU 8.2 (Ubuntu 24.04 / this repo’s GHA and cloud probe) does not implement LBE —
hw/char/pl011.cstill says the loopback bit is unimplemented. LBE landed in later QEMU. A loopback-only probe would be a false pass on newer hosts and a false fail on 8.2.
Host stdin on -serial stdio does reach pl011_receive → the RX FIFO. UARTLCR_H.FEN toggles reset that FIFO, so a byte sent before uart::init is lost. Inject after Hello World! (init already ran).
Decision
- M6 input is PL011 RX poll, not virtio-keyboard and not a UART RX IRQ. FR-08’s “via the virt GIC” remains the M5 timer. Input does not take a new FR ID.
- Prove a received byte with a host inject:
scripts/qemu-serial-inject.pywrites0x41('A') to QEMU stdin after it seesHello World!. The hello kernel pollsUARTFR.RXFE/UARTDR(DAIF.I still masked after the timer window) and printsinput: rx 0x41. - Fail closed. Missing marker or
input: rx missedfailsscripts/qemu-smoke.sh.#[test_case]only asserts the FIFO is empty when cargo test does not inject — the character proof is the serial smoke. - Do not claim Raspberry Pi UART, virtio-input, GICv3 UART SPI, or QEMU LBE.
Consequences
scripts/qemu-smoke.shneedspython3to drive the inject (Dockerfile installs it).cargo test/scripts/qemu-aarch64.shstay inject-free so M2–M5 cases do not wait on stdin.- A later virtio-keyboard or UART-RX-via-GIC path needs a new ADR; it is not this milestone.
ADR-008 — Identity map and bump frame allocator
- Status: Accepted
- Date: 2026-09-09
Context
FR-09 requires the kernel to read a firmware/QEMU memory map (DTB when probed) and establish paging / virtual memory. Roadmap M7 is that path. Heap GlobalAlloc is M8; the scheduler is M9.
QEMU -machine virt memory (see hw/arm/virt.c):
| Region | Physical |
|---|---|
| flash / boot ROM | 0x0000_0000 |
| GICv2 distributor / CPU interface | 0x0800_0000 / 0x0801_0000 |
| PL011 UART | 0x0900_0000 |
RAM (VIRT_MEM) | 0x4000_0000, default 128 MiB |
Kernel (-kernel TEXT_OFFSET 0x80000) | 0x4008_0000 |
DTB (typical -kernel placement) | RAM base 0x4000_0000, below the kernel |
_start currently overwrites x0 (the firmware DTB pointer) to set SP. A full FDT memory-node walk is extra surface for one milestone and is not required to prove MMU-on + allocate-frame on this machine.
Options:
- 1 GiB L1 identity blocks (39-bit VA,
T0SZ=25, 4 KiB granule) plus a dedicated 4 KiB map window. - Fine-grained 4 KiB identity of every used page (more tables, same probe).
- Higher-half kernel (breaks the current linker / VBAR / UART absolute addresses).
- Parse the DTB before enabling the MMU (correct long-term; not the M7 fail-closed proof).
Decision
- Enable the MMU at EL1 after
exception::init(ensure_el1). ProgramMAIR_EL1(Attr0 Device-nGnRnE, Attr1 Normal WB),TCR_EL1(TTBR0 only, 40-bit IPS, inner-shareable WB),TTBR0_EL1, then setSCTLR_EL1.M|C|I|SA. Identity: VA == PA for everything the kernel already touches. - L1 blocks, not a full 4 KiB identity. Entry 0 (
0x0–1 GiB) is Device-nGnRnE + XN (UART, GIC, flash). Entry 1 (0x4000_0000–0x7FFF_FFFF) was a Normal WB executable 1 GiB RAM block in M7. ADR-012 supersedes that RAM block: L1 slot 1 is now an L2 table, only 128 MiB is mapped, and__kernel_end…RAM-end is PXN. The allocator still must not hand out frames pastRAM_BASE + 128 MiB. - Frame pool from linker + virt convention.
__kernel_end(after image + stacks, 4 KiB-aligned) through0x4000_0000 + 128 MiB. Bump cursor plus a 16-entry free stack. Do not allocate0x4000_0000–0x4008_0000(DTB / TEXT_OFFSET hole). Smoke pins-m 128Mso the pool matches the VM. - Map/unmap probe uses a third GiB window (
0x8000_0000) via one L2 + L3 table, not by splitting the RAM block. Serial markerpaging: ok. Do not load an unmapped VA (that would take an unhandled data abort). - DTB parse is staged. M7 does not walk FDT. FR-09’s “DTB when probed” stays Unknown until a later ADR. The probed map is the virt table above plus
__kernel_end. - No
GlobalAlloc. Frames are physical pages only.
Consequences
scripts/qemu-smoke.shrequirespaging: okand rejectspaging: probe missed, then still requires M2–M6 strings.#[test_case]can assertSCTLR_EL1.M, distinct aligned frames, and a map/unmap write-through.- Stack overflow was not a translation fault in M7 (RAM was one Normal block). Guard pages are ADR-014, not this milestone.
- A later DTB walker or higher-half map needs a new ADR; it is not M8’s heap.
- This ADR does not claim Raspberry Pi, GICv3, EL0 user maps, or ASID isolation. Heap NX is ADR-012, not M7.
ADR-009 — First-fit heap on identity-mapped frames
- Status: Accepted
- Date: 2026-09-09
Context
FR-10 requires a GlobalAlloc heap so alloc types (Box, Vec) work in the kernel. Roadmap M8 is that path. The scheduler is M9.
M7 already identity-maps virt RAM and hands out 4 KiB frames from __kernel_end through 128 MiB (ADR-008). Frames are physical pages, not a byte allocator.
Options:
- Bump-only heap (no
dealloc).Box/Vecconstruct, but drop leaks. Reuse is unproved. linked_list_allocatorcrate. Common rust-osdev choice; extra dependency for one milestone.- First-fit free list on a fixed contiguous frame run taken from the bump pool after the MMU is on.
- Grow-on-demand via the M7 map window (
0x8000_0000). Extra paging surface; not needed to prove FR-10. - Buddy / slab. Overkill for a Box/Vec smoke.
Decision
- Fixed 64 KiB heap (16 frames) via
frame::alloc_contiguousafterpaging::init. Identity VA == PA. Do not map through the M7 probe window. - First-fit + address-sorted coalesce in
src/heap.rs.deallocis real; the hello probe requires a freedBoxpointer to be reused. #[global_allocator]+extern crate alloc..cargo/config.tomlbuild-stdincludesalloc. OOM returns a null pointer;#[alloc_error_handler]printsheap: oomon the raw UART and parks (orSYS_EXIT1 under test). The handler must not format — formatting an OOM can allocate again.- Serial marker
heap: ok. Fail closed onheap: probe missed.#[test_case]covers pool bounds,Boxreuse, andVecgrowth. - Not a growing heap, not userspace, not the scheduler. A later slab / grow-on-demand / higher-half heap needs a new ADR.
Consequences
scripts/qemu-smoke.shrequiresheap: okafterpaging: ok, then still requires M2–M7 strings.- The frame pool shrinks by 64 KiB at boot. M7 frame/map tests still have the rest of the 128 MiB guest.
- This ADR does not claim Raspberry Pi or a production allocator. M9 task stacks on this heap are ADR-010.
- ADR-038 keeps the first-fit algorithm and moves the pool VAs to the TTBR1 alias, then unmaps identity heap. The 64 KiB size is unchanged.
ADR-010 — Cooperative round-robin on EL1
- Status: Accepted
- Date: 2026-09-09
Context
FR-11 requires cooperative or simple round-robin task switching. Roadmap M9 is that path. The heap already exists (ADR-009); paging is an identity map (ADR-008). The kernel runs at EL1 with SPSel = 0 so SP is SP_EL0 (ADR-005).
Options:
- Async/await executor. Extra
Futureinfrastructure for two markers. - Timer preemption. Would steal the M5 CNTP path and needs IRQ-safe yield. Not required to prove FR-11.
- Cooperative yield that saves AAPCS64 callee-saved GPRs on a per-task stack and switches
SP. - SMP / EL0 processes. Out of scope (vision Out list).
The callee-saved layout is not obvious next to the exception frame in ADR-004 (that frame is x0–x30 + ELR/SPSR/ESR on SP_EL1). A yield must not touch the exception stacks.
Decision
- Cooperative only. Tasks call
sched::yield_now(). No timer slice. DAIF.I stays masked during the hello/test probe (M5 still remasks after its own tick). - AAPCS64 callee-saved switch in
context_switch: save/restorex19–x28,x29,x30on the task stack; store SP in the task slot; load the next SP. No SIMD/FP (ADR-003 soft-float). Not the exception context. - Idle slot 0 is
kernel_main/ the test runner on the linker thread stack. Two workers get 8 KiB stacks from the M8 heap (Vec<u8>, not a stack-allocated[u8; N]that would overflowSP_EL0duringBox::new). Identity VA == PA. - Round-robin among
Readyslots. A worker that returns isDone(trampoline). Idle staysReadyso control returns to the caller ofyield_now. Unlock the scheduler mutex beforecontext_switch. - Serial markers
sched: task a,sched: task b, thensched: okafter both SPs land on distinct heap stacks. Fail closed onsched: probe missed. - Not preemptive, not SMP, not EL0, not async. A later preemptive or process ADR is a new file.
Consequences
scripts/qemu-smoke.shrequires the threesched:strings afterheap: ok, then still requires M2–M8 strings.- Two 8 KiB worker stacks come out of the 64 KiB heap. M8
Box/Vecprobes drop before spawn. - Exception / fatal stacks are unchanged. A yield never runs from an IRQ handler.
- This ADR does not claim Raspberry Pi, preemption, or userspace.
ADR-011 — Three pillars: antifragility, security, performance
- Status: Accepted
- Date: 2026-09-09
Frozen FR/NFR IDs are unchanged. This ADR revises NFR-05, NFR-07, and NFR-10 text in place. Do not mint FR-16+ or NFR-15+.
Context
Bring-up M0–M9 is on main (UART through a cooperative EL1 scheduler). The first-class practice so far was the honesty/harness loop (ADR-001): probes, ratchets, fail-closed CI, no self-merge.
Sponsor direction after M9: antifragility is not enough by itself. Security and performance are first-class pillars too — not afterthoughts bolted on when the kernel “feels done.”
Risks if we leave the NFR text as-is:
- NFR-07 is still “Could / Later / learning-only.” That invites fake benches or silent “it’s fast enough” claims.
- NFR-10 is mostly “no secrets in repo.” That is necessary and not a threat model.
- NFR-05 already names ratchets, but it reads like a Next-stage SOP rather than a standing pillar.
This ADR does not clone florist / AEA vocabulary. The three pillars are ctos-native names for existing IDs.
Decision
-
Three first-class pillars, same weight, different sensors:
- Antifragility — keep the existing harness: honesty ledger, fail-closed
scripts/qemu-smoke.sh, GHAsmoke.yml, Docker smoke,#[test_case]. Repeated failures become ratchets, not README-only advice (antifragility.md). - Security — no “secure OS” / “hardened” / “W^X done” claim without a written threat model and a probe. Prefer minimize
unsafe(already NFR-01). Prefer W^X / NX stacks and heap when paging can express it. No execute-from-writable heap by default once maps can mark NX. Least privilege on IRQ paths (no heap alloc, nosched::yield_nowfrom an IRQ). Future EL0 isolation is Planned. - Performance — no fake benches and no invented latency numbers. Add honest measurable probes (CNTPCT delta around a known path; later timer-tick jitter). Optimize only after a probe shows a cost.
- Antifragility — keep the existing harness: honesty ledger, fail-closed
-
Revise frozen NFR text in place (IDs stay NFR-05 / NFR-07 / NFR-10):
- NFR-05: Must / Now. Ratchets stay the mechanism.
- NFR-07: Should / Now. Probe language; not a hard latency budget.
- NFR-10: Must / Now. Threat-model stub + claim gate, not only “no secrets.”
-
W^X is Planned on this identity map. ADR-008 maps virt RAM as one executable 1 GiB L1 Normal block. The heap (ADR-009) and cooperative stacks (ADR-010) live in that block, so they are W+X today. Forbidding execute-from-heap needs an L2/L3 split (or a later higher-half map). Device MMIO is already XN. Do not claim heap NX because the L3 map window can set PXN — that window is not the heap.
-
One small code ratchet may land with this docs PR. This change lands a baseline
CNTPCTloop delta (serialperf: cntpct+#[test_case]). It does not split the L1 RAM block to fake W^X. Honesty over heroics. -
Post-M9 work is a pillars section, not a second bring-up stack on an open PR. Threat-model v1, W^X / NX heap+stacks, and further perf probes were listed as later loop units. The sponsor later asked for one coherent follow-up (ADR-012, ADR-013, irq-delta) rather than conflicting parallel branches.
Consequences
- README,
AGENTS.md, the roadmap, and the honesty ledger point at pillars.md. - A PR that says “secure” or “faster” without a ledger row is a coherence fail (NFR-06 / NFR-13).
- The CNTPCT probe (when landed) is a baseline that the counter advances. It is not a published benchmark and not a comparison to other kernels.
- PAN, full higher-half identity teardown, and preemption remain later. Standing EL0 and the ASID-tagged TLB mile are ADR-013. TTBR1 private page is ADR-016. EL1 high-VA fetch is ADR-017. The first identity-tear cut is ADR-018. The identity
.textrange tear is ADR-019. The live.texttear after a high-VA vtable rewrite is ADR-020. The SVC ABI mile is ADR-021. ThelibctosCRT mile is ADR-022. The guest ELF loader mile is ADR-023. Standing EL0 as normal mode is ADR-024 (app hosting Planned). Umbrella isolation still Planned. RO+NX text/data is ADR-015. Stack guard pages are ADR-014. EL0 first mile + user-TTBR0 read mile are also ADR-013.
ADR-012 — W^X: NX heap and cooperative stacks
- Status: Accepted
- Date: 2026-09-10
Context
NFR-10 and ADR-011 require a probe before any W^X claim. ADR-008 mapped virt RAM as one executable 1 GiB L1 Normal block. The heap (ADR-009) and cooperative stacks (ADR-010) live in that block, so they were W+X. Setting PXN only on the M7 map window (0x8000_0000) would have been a fake “NX heap” claim.
Device MMIO (L1 block 0) was already XN. The missing cut is: heap and heap-backed stacks NX, kernel text still executable.
SCTLR_EL1.WXN is not usable while kernel text pages are writable — it would make text NX too.
Decision
- Replace the RAM L1 block with an L2 table (still 39-bit / 4 KiB / TTBR0). Map only the 128 MiB guest (
0x4000_0000…+128 MiB). The rest of that GiB stays invalid. - 2 MiB L2 blocks where a block is entirely kernel-image or entirely frame-pool. One L3 table per mixed 2 MiB (originally the single block that straddled
__kernel_end; ADR-015 splits on__data_startandKERNEL_TEXT, which can be two blocks). - Executable: pages in
[0x4008_0000, align_4k(__kernel_end))(text, rodata, data, BSS, linker stacks). PXN+UXN:[__kernel_end, RAM end)(frame pool, heap, cooperative stacks) and the DTB hole below0x4008_0000when that hole is in the straddling L3. Device L1 stays XN. The M7 map window L3 pages are PXN (data-only). - Do not claim the linker stacks NX. SP_EL0 / SP_EL1 / fatal stacks sit in the executable image. Overflow there is still not a translation fault (ADR-005).
- Fail-closed probe: walk PXN on kernel text (clear) and heap (set), then write
RETon the heap,blrto it, and catch a current-EL permission instruction abort. Serialwx: nx heap+wx: ok.scripts/qemu-smoke.shgrepswx: okand rejectswx: probe missed. - NFR-10 text is revised in place (ID unchanged): heap + coop-stack NX is the scoped W^X claim, not “the kernel is W^X.”
Consequences
src/paging.rsgrowsL2_RAMand a kernel L3 pool (one table per mixed 2 MiB). Identity VA == PA is unchanged.- Linker-stack guard pages are ADR-014. The RO+NX text/data split and
SCTLR.WXNare ADR-015. - EL0 isolation is ADR-013 (Planned). UXN is already set on RAM so a future EL0 cannot fetch kernel or heap by accident; that is not an EL0 probe.
- This ADR does not claim a secure OS, Raspberry Pi, or side-channel resistance.
ADR-013 — EL0 isolation direction (first mile)
- Status: Accepted (direction + first mile + user-TTBR0 read mile + ASID isolation mile + standing EL0; umbrella isolation still Planned)
- Follow-on (2026-09-28, M6): B3 met: the sponsor accepted the ADR-091 revision-3 sentence and scope in writing (2026-09-28 16:41 CEST). ADR-091 is Accepted. “EL0 isolated” is Verified only as that sentence, with its limits. Text in this ADR that says the umbrella is Planned / non-claim is as of this ADR’s date.
- Date: 2026-09-10
- Updated: 2026-09-11 (standing EL0 dual-SVC; TTBR1 first cut is ADR-016; isolation still Planned)
Context
Vision and FR-11 leave userspace / EL0 as later work. NFR-10 names EL0 isolation as Planned. ADR-011 and the threat model (security.md) treat “malicious EL0” as a named adversary, not a standing userspace.
The first revision of this ADR recorded direction only (is_active() == false, lower-EL slots parked). The post-#18 deepen implements the smallest honest mile toward the closing probe. It does not invent a POSIX process model.
Decision
- First mile (this tree). Deliberate
ERETto EL0t (DAIF masked) on one map-window page that is UXN-clear and PXN (EL1 must not fetch it). Two paths:SVC #0taken on the lower-EL AArch64 sync slot; handler printsel0: svcandERETs back to EL1t.BR X0to a kernel.databait; UXN on the identity image must take a lower-EL permission IABORT; handler printsel0: nx kerneland returns to EL1t. Serialel0: okafter the first-mile pair and the later miles on this path. The first mile itself does not install a standing context.
- Closing probe for “cannot execute kernel data.” The UXN / translation IABORT is that instruction-fetch probe. Mark that mile Verified when the serial /
#[test_case]pass. - User TTBR0 window (this tree, post-#19). A second L1 (
L1_USER) is installed onERETto EL0 (ASID=1 in TTBR0[63:48]). It maps kernel.text/.rodataand the exception stack so the lower-EL handler can restore kernel TTBR0 (kept inTPIDR_EL1), plus the shared map-window L2 for the trampoline. Coverage is every 2 MiB that holds those ranges, not only the first RAM 2 MiB. It omits.data/.bss/ heap. An EL0LDRfrom kernel.datais a lower-EL translation (or permission) DABORT (el0: no kernel read). Historically the EL0 trampoline stillTLBI VMALLE1because kernel.dataleaves were global (nG=0); dropping that flush while identity.datastayed mapped would leak a cached.datatranslation into the user ASID. After ADR-037 / ADR-038, ADR-039 drops trampolineVMALLE1(el0: no-vmalle1). Programming ASID=1 on that path is still not the isolation mile. - ASID isolation mile (this tree). Two EL1 TTBR0 values (kernel L1 + ASID 1 vs a clone L1 + ASID 2) map one window VA to different PAs with
nG=1, and a second VA only under ASID 1. The switch isMSR TTBR0+ISB— noTLBI VMALLE1. Dual read must see the active ASID’s magic (asid: dual). The ASID-1-only VA must take a current-EL translation DABORT under ASID 2 (asid: conflict). Seeing ASID 1’s magic under ASID 2 is Failed (asid: stale). This is TLB ASID isolation on this virt guest, not “EL0 isolated.” - Standing EL0 (this tree). A bounded user context on the user TTBR0:
SVC #1printsel0: standingand ERETs back to EL0; the userMOVZmust run;SVC #2printsel0: restoredand returns to EL1t.is_active()is true only between install and teardown (saved user PC / SP / TTBR0). A trampoline-only flag is not this mile. Lower-EL IRQ / FIQ / SError still park (not exercised). Not a POSIX process. Not preemption. - Isolation remains Planned. Missing: PAN enable (typically unimplemented on
-cpu cortex-a57/ ARMv8.0 — ADR-026 readsID_AA64MMFR1_EL1.PANand records absent; do not silent-cpuswitch), full TTBR1 / higher-half identity teardown (ADR-016 is the private-page cut; ADR-017 is the EL1-fetch cut; ADR-018 is the first identity-tear cut; ADR-019 jumps the post-MMU continuation high and unmaps a 16 KiB dedicated text range; ADR-020 rewrites rustc vtables and unmaps live identity.text; ADR-025 unmaps identity.rodata; ADR-037 / ADR-038 unmap.data/stacks/heap — leftover identity frames /_startstay), EL0 entry without a full TLBI (superseded by ADR-039). A documented SVC ABI is ADR-021 (not isolation, not app hosting). ThelibctosCRT is ADR-022 (not isolation, not app hosting). Do not move FR IDs. Do not mint FR-16+. Do not claim PAN. - Lower-EL slots. AArch64 sync is live for SVC / IABORT / DABORT (including standing dual-SVC, the public ABI numbers 16–18, and the TTBR1 private-page load). Lower-EL IRQ / FIQ / SError and all AArch32 slots stay parks (ADR-004).
- Honesty. Docs and PRs may say “EL0 entered and returned,” “EL0 cannot execute kernel data,” “EL0 cannot read kernel
.data,” “user TTBR0 omits kernel data,” “standing EL0 context,” or “ASID isolation” when those probes pass. The TTBR1 private-page mile is ADR-016. The EL1 high-VA fetch mile is ADR-017. The identity-tear first cut is ADR-018. The identity.textrange tear is ADR-019. The live.texttear after a high-VA vtable rewrite is ADR-020. The SVC ABI mile is ADR-021. ThelibctosCRT mile is ADR-022. The guest ELF loader mile is ADR-023. Standing EL0 as normal mode (loaded task untilexit, fail-closed restore) is ADR-024. The identity.rodatatear is ADR-025. The PAN ID-field probe is ADR-026. They may not say “EL0 works,” “userspace,” “EL0 isolated,” “app hosting,” or “the kernel moved.”
Consequences
- el0.md splits first-mile vs isolation rows.
- Preemption and SMP remain separate later ADRs (ADR-010).
- This ADR does not claim Raspberry Pi or a POSIX process model.
ADR-014 — Unmapped guard pages under linker stacks
- Status: Accepted
- Date: 2026-09-10
Context
ADR-012 made the heap and heap-backed cooperative stacks PXN. It explicitly left SP_EL0 / SP_EL1 / fatal stacks in the executable kernel image and named stack guard pages as a later ADR. ADR-005 still mitigates overflow with a dedicated fatal stack and a nested-BRK probe — that is not a translation fault.
A downward overflow of a linker stack today smashes the previous image bytes (.bss or the stack above). QEMU virt will not notice.
Options:
- Unmapped 4 KiB guard page(s) below each linker stack. Overflow becomes a current-EL data abort (translation). Observable serial +
#[test_case]. - PXN on stack pages after splitting them from text. Stops execute-from-stack; overflow still smashes mapped data unless we also unmap something.
SCTLR_EL1.WXN— still unusable while kernel text pages are writable.
Decision
- Option 1. Insert a 4 KiB linker hole below the thread, exception, and fatal stacks (
__stack_guard,__exc_stack_guard,__fatal_stack_guard). Afterfill_ram_wx, split any 2 MiB L2 block that contains a guard into L3 and clear that L3 slot (invalid). Identity VA == PA is unchanged. - Fail-closed probe: arm a current-EL translation data-abort catch, store to the thread-stack guard, resume at LR. Serial
guard: fault+guard: ok.scripts/qemu-smoke.shgrepsguard: okand rejectsguard: probe missed. - Guard pages are holes, not NX stacks. This ADR does not by itself make live stack pages NX. ADR-015 maps those live pages RW+NX with
.data. Do not claim “the kernel is W^X” from guard holes alone. - NFR-10 text is revised in place (ID unchanged): mention the guard-page overflow cut. Do not mint NFR-15+.
- The ADR-005 nested-
BRKfatal probe stays. It does not write below__stack_bottom.
Consequences
src/paging.rsmay add one or two extra L3 tables to split an executable L2 block that is not the__kernel_endstraddle.- Overflow of a heap-backed cooperative stack is still not this probe (those stacks live in PXN frame-pool RAM with no guard holes).
- This ADR does not claim a secure OS, Raspberry Pi, or ASAN-quality stack checking.
ADR-015 — RO+NX text / data split
- Status: Accepted
- Date: 2026-09-11
Context
ADR-012 made the heap and heap-backed cooperative stacks PXN. ADR-014 punched unmapped holes under the linker stacks. Both left .text / .rodata / .data / .bss / live linker-stack pages in one executable, writable image mapping. SCTLR_EL1.WXN was unusable while text was writable.
NFR-10 still forbids saying “the kernel is W^X” without a scoped probe. This ADR is the next cut: split RO+X from RW+NX.
Decision
- Page-align
__data_startinlinker.ldso.text/.rodatanever share a 4 KiB page with.data. - Identity flags (still 39-bit / 4 KiB / TTBR0):
[KERNEL_TEXT, __data_start)— RO+X (AP[2]=1, PXN clear, UXN set)[__data_start, RAM end)— RW+NX (including.data,.bss, live linker stacks, frame pool / heap)- Device L1 stays XN. Guard holes stay invalid (ADR-014).
SCTLR_EL1.WXNon once text is RO. Writable pages are treated as XN even if a descriptor forgets PXN.- Fail-closed probe: execute-from-
.data(blrto aRETbait) is a current-EL permission IABORT (ro: nx data). Store to RO text is a current-EL permission DABORT (ro: write fault). Serialro: ok.scripts/qemu-smoke.shgreps those strings and rejectsro: probe missed.#[test_case]covers flags + both faults. - NFR-10 text is revised in place (ID unchanged): mention the RO+NX image cut. Do not mint NFR-15+.
- One L3 per mixed 2 MiB (2026-09-11 follow-up). ADR-012 assumed a single L3 for the 2 MiB that straddles
__kernel_end. After this ADR the mixed block is the one that contains__data_startand the first RAM 2 MiB that containsKERNEL_TEXT(0x4008_0000). Those are different blocks once.text/.rodata(or a linker ratchet) crosses0x4020_0000. Reusing oneL3_RAMoverwrites the kernel-text walk. The kernel L3 pool hands out a fresh table per straddle / guard split. The user TTBR0 maps every 2 MiB that holds text or the exception stack, not only the first RAM block. - Post-MMU
.bsspublishes (same follow-up).USER_MAP_OKandframe::initrun afterSCTLR.C. A pre-MMU store to.bsscan be invisible to a later cached read (boot-delta on the PR #20 test image; arm64 workstation Docker hello onb2bbb99lost the frame pool / user-map ready flag while guard/ro still passed). Do not pin an old nightly for this.
Honesty — is this “the kernel is W^X”?
On this QEMU virt guest, the identity image plus heap is W^X: text/rodata are RO+X, data/bss/live linker stacks/heap are RW+NX, WXN is on. That is not:
- a “secure OS” / “hardened” claim
- a promise that every future mapping (DMA, new windows) stays W^X
- Raspberry Pi or a second ISA
- a reason to drop the heap / guard probes
Say “identity image is W^X on this virt guest (ADR-015 probe)” only when the serial / tests pass. Do not say “the kernel is W^X” as a product sentence.
Consequences
src/paging.rssets AP[2] on text and PXN on data/stacks.src/ro.rsowns the two faults.- Live linker stacks are NX here. If a later change must execute from a linker stack, this ADR is the one to revisit.
- EL0 user TTBR0 (ADR-013 mile) still maps kernel text so the lower-EL handler can restore kernel TTBR0; it omits
.data. Coverage follows the image across L2 blocks. That is isolation-adjacent, not this ADR. linker.ldparks__data_startat0x4020_1000so GHA / Docker / this cloud always exercise two kernel L3s. File presence of that address is not a boot probe.- This ADR does not claim PAN, ASID isolation, or a higher-half kernel.
ADR-016 — TTBR1 kernel-private page (first cut)
- Status: Accepted (first cut; identity teardown Planned)
- Date: 2026-09-11
Context
ADR-013 left TTBR1 / higher-half as a missing isolation mile. The kernel still runs from the QEMU virt identity map at 0x4008_0000 (TTBR0, T0SZ=25). TCR.EPD1 was set, so TTBR1 walks were disabled. EL0 and EL1 share the same translation regime: a high mapping is not automatically invisible to EL0 — access is an AP / UXN question.
A full higher-half relocate (kernel fetch, VBAR_EL1, stacks, UART MMIO, then tearing down the identity image) is too large for one honest PR. This ADR is the smallest Verified cut: enable TTBR1 and prove one kernel-private high page.
Decision
- Enable TTBR1 walks. Clear
TCR.EPD1. Set T1SZ=25 (same 39-bit window as T0SZ) with WBWA / inner-shareable attrs on TTBR1. ASID still comes from TTBR0 (TCR.A1 = 0). - High-half base. The TTBR1 region is
0xFFFF_FF80_0000_0000…0xFFFF_FFFF_FFFF_FFFF. One private page lives atTTBR1_PRIV = 0xFFFF_FF80_0000_0000(src/paging.rs). - EL1-only leaf. The page is Normal, RW, PXN, UXN, AP[2:1]=00 (EL1 RW, EL0 no data access). Backing store is a frame-pool page, also identity-mapped for EL1. User TTBR0 does not describe this VA.
- Fail-closed probe. EL1 store/load via the high VA prints
ttbr1: el1. An EL0LDRofTTBR1_PRIVis a lower-EL permission (or translation) DABORT (ttbr1: no el0). Serialttbr1: ok.scripts/qemu-smoke.shgreps those strings and rejectsttbr1: probe missed/ttbr1: leaked.#[test_case]covers the same path. - Identity map stays.
_start, vectors, linker stacks, PL0110x0900_0000, and the multi-L3 / layout ratchets from #21 keep using TTBR0 identity VAs. Full higher-half + identity teardown is Planned. - NFR-10 text is revised in place (ID unchanged). Do not mint NFR-15+. Do not claim “the kernel runs in the high half” or “EL0 isolated.”
Honesty
Say “TTBR1 maps a kernel-private page EL0 cannot access” only when the serial / tests pass. Do not say:
- the kernel has moved to a high VA
- identity mappings were torn down
- “secure OS” / “hardened” / “EL0 isolated”
- PAN (still unclaimed on
-cpu cortex-a57)
Consequences
paging::initprogramsTTBR1_EL1and leavesEPD1clear.src/ttbr1.rsowns the probe.- Standing EL0 (ADR-013) is a separate mile. Umbrella isolation stays Planned while PAN is unclaimed.
- ADR-017 aliases identity RAM in TTBR1 and fetches a real EL1 path (plus high
VBAR_EL1). ADR-018 splits those tables and unmaps one identity text page. ADR-019 unmaps a 16 KiB dedicated text range after a high-VA jump. ADR-020 unmaps live identity.textafter a vtable rewrite. Full identity teardown (.rodata/.data/heap) is still Planned.
ADR-017 — EL1 fetch from the TTBR1 RAM alias
- Status: Accepted (exec mile; first identity-tear cut is ADR-018)
- Date: 2026-09-11
Context
ADR-016 enabled TTBR1 walks and proved one EL1-only data page at TTBR1_PRIV. The kernel still fetched instructions from the QEMU virt identity map at 0x4008_0000. Full higher-half relocate (every pointer, every fn item, then tearing down identity) is still too large for one honest PR: rustc emits link-time identity addresses (relocation-model: static), and -kernel loads _start at TEXT_OFFSET.
This ADR is the largest Verified cut that still keeps the boot stub: alias identity RAM in TTBR1 and run a real EL1 code path (instruction fetch, not a private store) at high VA.
Decision
- RAM alias.
L1_HIGH[1]points at the sameL2_RAMused by identity TTBR0. High VA isidentity + TTBR1_BASE(0x4008_0000→0xFFFF_FF80_4008_0000). Permissions match the identity image (text RO+X, data/stacks/heap RW+NX, guards unmapped). - Shared tables (superseded for the tear mile). This ADR reused
L2_RAM. ADR-018 clones RAM tables so one identity text page can be unmapped without dropping the high twin. Full identity teardown is still Planned. - EL1 exec probe.
ttbr1_high_el1_pathis invoked through its high VA (BLR/ fn pointer). It captures PC withADR, printsttbr1: el1 execfrom that path, and returns the PC. The caller checksPCis in the TTBR1 window and near the high entry. UART MMIO stays the absolute identity0x0900_0000. VBAR_EL1high. After the MMU is on,VBAR_EL1is programmed to the high alias ofexception_vectors(ttbr1: vbar). Later BRK / IRQ / EL0 sync fetch the table via TTBR1. Identityexception::initstill installs the link address first (pre-MMU).- Identity boot stub stays.
_start, QEMU-kernelload, and link-timefnitems remain at0x4008_0000. Most EL1 data access still uses identity VAs. Do not say the kernel moved. - Still Planned. Full identity teardown (unmap all low
.text/.data/heap after a complete high-VA jump); PAN on-cpu cortex-a57; umbrella EL0 isolation. ADR-018 is the first cut (one torn text page). User TTBR0 still maps most kernel text so a missed high-VBAR path can fetch. Do not change default-cpu. - NFR-10 text is revised in place (ID unchanged). Do not mint NFR-15+.
- ADRP is not a PA. After VBAR is high, handler code that does
addr_of!(table)/user_ttbr0()must mask to the 39-bit identity VA before programmingTTBR0or comparingFAR_EL1. First attempts Failed: guardFARcompared to a high__stack_guard, then standingstay_at_el0programmed a highL1_USERas TTBR0.paging::identity_pa/linker_symare the ratchet.
Honesty
Say “EL1 fetched a real path from a TTBR1 high VA” or “VBAR lives at the high alias” only when the serial / tests pass. Do not say:
- the kernel has moved to the high half
- identity mappings were torn down
- “secure OS” / “hardened” / “EL0 isolated”
- PAN (still unclaimed on
-cpu cortex-a57)
Consequences
paging::initinstalls the RAM alias and relocates VBAR.src/ttbr1.rsowns both the private-page probe and the exec probe.- ADR-016 remains the private-page first cut. This ADR is the exec mile.
- ADR-018 splits TTBR1 RAM tables and unmaps one identity text page. ADR-019 jumps the post-MMU continuation to high VA and unmaps a 16 KiB dedicated text range. ADR-020 unmaps live identity
.textafter a vtable rewrite. Unmapping.rodata/.data/heap is still later.
ADR-018 — Identity teardown first cut (split tables + one torn text page)
- Status: Accepted (partial tear; full identity teardown still Planned)
- Date: 2026-09-11
Context
ADR-017 proved EL1 can fetch a real path from the TTBR1 RAM alias and moved VBAR_EL1 there. The alias reused identity L2_RAM, so unmapping a low page also dropped the high twin. _start / QEMU -kernel still load at 0x4008_0000. rustc emits link-time identity addresses (relocation-model: static), so a complete high-VA jump of kernel_main plus unmap of all identity .text/.data/heap is still too large for one honest PR.
This ADR is the largest Verified cut that still boots on virt: split the RAM tables, then unmap one defined identity text page that EL1 no longer needs for fetch.
Decision
- Split RAM tables. After identity W^X + stack guards, clone
L2_RAM(and every L3 it points at) intoL2_HIGH_RAM/L3_HIGH_RAM.L1_HIGH[1]points at the clone. L2 block descriptors are copied; L3 tables are duplicated. Unmapping an identity page no longer unmaps the high twin. - Dedicated tear page. The linker keeps a 4 KiB
ident_tearsection (__ident_tear_start…__ident_tear_end) in the RO+X image, after.rodataand before__data_start. It is not the0x4008_0000_startpage. - Unmap after MMU + high VBAR.
paging::initturns the MMU on, programs the high VBAR, then clears the identity and user-TTBR0 L3 slots for that page andTLBI VAAE1s the low VA. The high twin stays PXN-clear. - Fail-closed probe. Serial
ident: split(clone + torn low / live high walks). EL1BLRof the low VA is a current-EL translation IABORT (ident: fault). EL1BLRof the high twin still runsident_tear_el1_path(ident: high). EL0LDRof the low VA is a lower-EL translation DABORT (ident: no el0). Serialident: ok.scripts/qemu-smoke.shgreps those strings and rejectsident: probe missed/ident: leaked. - Identity boot stub stays.
_start, QEMU-kernelload, remaining identity.text/.data/heap, PL0110x0900_0000, and later probes keep using TTBR0 identity VAs. Do not say the kernel moved. - Still Planned. Full identity teardown (unmap all low
.text/.data/heap after a complete high-VA jump); PAN on-cpu cortex-a57; umbrella EL0 isolation. Do not change default-cpu. - NFR-10 text is revised in place (ID unchanged). Do not mint NFR-15+.
Honesty
Say “TTBR1 RAM tables are independent of identity” or “one identity text page was unmapped while EL1 still fetched the high twin” only when the serial / tests pass. Do not say:
- the kernel has moved to the high half
- identity mappings were fully torn down
- “secure OS” / “hardened” / “EL0 isolated”
- PAN (still unclaimed on
-cpu cortex-a57)
Consequences
paging::initclones RAM tables and unmaps__ident_tear_*.src/teardown.rsowns the serial probe.- ADR-017 remains the exec mile. This ADR is the first identity-teardown cut.
- ADR-019 jumps the post-MMU continuation to high VA and expands the dedicated tear to 16 KiB. ADR-020 rewrites rustc vtables and unmaps live identity
.text..rodata/.data/heap stay.
ADR-019 — High-VA continuation + dedicated identity text range
- Status: Accepted (partial tear; live
.text/.rodata/.data/ heap still identity-mapped) - Date: 2026-09-11
Context
ADR-018 cloned TTBR1 RAM tables and unmapped one dedicated identity text page. rustc still emits link-time identity addresses (relocation-model: static). _start / QEMU -kernel still load at 0x4008_0000.
The preferred next cut was to unmap contiguous live identity .text after the boot stub once EL1 fetched from the high alias. A first attempt on this cloud VM Failed: after unmapping [0x4008_1000, __text_end), the first writeln! / println! took an unhandled current-EL sync. core::fmt::write takes &mut dyn Write; those vtable methods are identity fn pointers. Unmapping live .text is therefore blocked until fmt / dyn dispatch is proven high-only.
This ADR is the largest Verified cut that still boots: jump the post-MMU continuation to its TTBR1 alias, then unmap a 16 KiB dedicated identity text range (four pages), not one probe page.
Decision
- High-VA continuation. After
paging::init(MMU on, highVBAR_EL1, ADR-018 first tear page),kernel_mainBRs to the TTBR1 alias ofkernel_main_high. Serialident: jump. DirectBLstays in the high window (PC-relative). UART MMIO stays the absolute identity0x0900_0000. Stacks / heap /.datastay identity VAs. - Boot stub stays mapped. The
0x4008_0000page (_start+exception_vectors) stays identity-mapped. Live.textafter that page also stays — rustc fmt /dynstill needs those identity fn pointers. - Dedicated 16 KiB tear range. Linker
__ident_tear_start…__ident_tear_endis four pages: page 0 holdsident_tear_el1_path(ADR-018), page 1 holdsident_range_el1_path, pages 2–3 are aligned pad. After the high jump, unmap the whole range from kernel and user TTBR0 andTLBI VAAE1each low VA. High twins stay PXN-clear. - Keep live
.text/.rodata/.dataidentity-mapped. Do not yank them here. A botched full.textyank is worse than a smaller Verified slice. - Identity
fnpointers used after the jump go high where we control them (scheduler trampoline / task entries; custom test runnerdyn Testablemethod). That is preparation, not a claim that rustc fmt is high-only. - Fail-closed probe. Serial
ident: jump.ident: range lo=… hi=… pages=NwithN >= 4. Existingident: split/fault/high/no el0/okstay.ident: textis EL1 fetch of the second torn page via the high twin.scripts/qemu-smoke.shgreps those strings and rejectsident: probe missed/ident: leaked/ident: range missed. - Identity boot stub stays.
_start, QEMU-kernelload, live identity.text/.rodata/.data/heap, PL0110x0900_0000. Do not say the kernel moved. - Still Planned. Unmap live identity
.textafter fmt /dyncalls are proven high-only; then.rodata/.data/heap; PAN on-cpu cortex-a57; umbrella EL0 isolation. Do not change default-cpu. - NFR-10 text is revised in place (ID unchanged). Do not mint NFR-15+.
Honesty
Say “the post-MMU continuation ran at a high VA” or “a 16 KiB dedicated identity text range was unmapped while EL1 still fetched the high twins” only when the serial / tests pass. Do not say:
- the kernel has moved to the high half
- identity
.textafter_startwas fully torn down - identity mappings were fully torn down
- “secure OS” / “hardened” / “EL0 isolated”
- PAN (still unclaimed on
-cpu cortex-a57)
Consequences
paging::jump_high/tear_identity_text_rangeown the cut.src/teardown.rsextends the serial probe.- ADR-018 remains the split-tables + first dedicated page cut.
- ADR-020 rewrites rustc vtables to high aliases and unmaps live identity
.textafter the boot stub..rodata/.data/ heap stay.
ADR-020 — High-VA fn-pointer rewrite + live identity .text tear
- Status: Accepted (live
.textafter the boot stub;.rodata/.data/ heap still identity-mapped) - Date: 2026-09-11
Context
ADR-019 jumps the post-MMU continuation to its TTBR1 alias and unmaps a 16 KiB dedicated identity text range. A first attempt to unmap live .text after _start Failed: the first writeln! / println! took an unhandled current-EL sync.
This ADR investigates that failure and takes the largest honest Verified cut toward tearing live identity .text.
Probed root cause (this cloud VM, main 24d94e6 hello ELF, then this branch):
- rustc
dyn Writevtables live in.rodata. ThePl011table isdrop=0 / size=8 / align=8 / write_str / write_char / write_fmtwith identity method addresses (write_charat0x4008_1084, immediately after the vectors page). - The hello
.rodataheld a handful of other identity.textwords (Debug::fmt,PadAdapterasWrite)..datahad none..texthad no 8-byte identity literal pools. - AArch64 codegen used
ADRP(PC-relative), notMOVZ/MOVKof0x4008. After the high-VA jump, newfn()values are already high; tables linked asR_AARCH64_ABS64stay identity until rewritten. The ELF isET_EXEC/relocation-model: static— no leftover reloc records to apply. .textand.rodatashared page0x400a4000on the ADR-019 layout (vtable at0x400a4338). Unmapping “all of.text” without a page split would also drop the vtable.
relocation-model: pic was considered and not taken: static + a post-jump patcher matches the existing -kernel / 0x4008_0000 contract and does not need a dynamic linker.
Decision
- Page-split
.text/.rodata. Linker__text_end/__rodata_startare 4 KiB aligned so live.textand fmt tables do not share a page. - High-VA fn-pointer rewrite. After
ident: jump, walk.rodata(through__ident_tear_start) and.data. Every 8-byte word that is an identity address in.textor__ident_tear_*is rewritten to its TTBR1 alias. RO pages are made writable only for that store, then restored (SCTLR.WXNmakes a writable page XN — they are not fetched). Serialident: reloc n=NwithN >= 1. APl011 as dyn Writefat-pointer read must see a highwrite_str. - Live identity
.texttear. Unmap[0x4008_1000, __text_end)from kernel and user TTBR0 (TLBI VAAE1each low VA). High twins stay PXN-clear. The0x4008_0000page (_start+exception_vectors) stays identity-mapped. Serialident: live lo=… hi=… pages=NwithN >= 8. - Keep
.rodata/.data/ heap identity-mapped. String literals and rewritten vtables still live there. Do not yank them here. - Keep
_start/ QEMU-kernelat0x4008_0000. Do not change default-cpu. Do not claim PAN. - Fail-closed probe. Existing
ident: jump/range/split/fault/high/text/no el0/okstay. Newident: reloc/ident: live.scripts/qemu-smoke.shgreps those strings and rejectsident: reloc missed/ident: live missed.#[test_case]covers the rewrite and the live unmap. - Still Planned (this ADR). Unmap identity
.rodata/.data/ heap after those accesses are proven high-only; PAN on-cpu cortex-a57; umbrella EL0 isolation. ADR-025 takes the.rodatacut. ADR-026 probes the PAN ID field (enable stays Planned). - NFR-10 text is revised in place (ID unchanged). Do not mint NFR-15+.
Honesty
Say “rustc vtable / fn-pointer words were rewritten to high aliases” or “live identity .text after the boot stub was unmapped while println! still ran” only when the serial / tests pass. Do not say:
- the kernel has moved to the high half
- identity mappings were fully torn down (
.rodata/.data/ heap stay) - “secure OS” / “hardened” / “EL0 isolated”
- PAN (still unclaimed on
-cpu cortex-a57)
Consequences
paging::rewrite_identity_fn_ptrs/paging::tear_live_identity_textown the cut. ADR-019 remains the high-VA jump + dedicated 16 KiB range.- A later ADR may unmap
.rodataonce string / table accesses are high-only, then.data/ heap. That work is not this cut.
ADR-021 — Minimal EL0 SVC syscall ABI
- Status: Accepted (ABI mile Verified when the serial / tests pass; Track A / app hosting still Planned)
- Date: 2026-09-11
Context
Track A (issue #31) wants freestanding apps to call the kernel. Issue #32 is the first mile: a documented, probed SVC contract.
ADR-013 already takes lower-EL AArch64 SVC for probe immediates #0 (first-mile return), #1 (standing announce), and #2 (standing restore). Those numbers stay reserved. This ADR adds a tiny public set so a later CRT / libctos (A2) has numbers to call. It does not load ELF, host apps, or claim Linux/POSIX.
Options:
- Linux AArch64 (
svc #0,x8= number). Collides with the first-mileSVC #0probe and invites a Linux-ABI claim. - SVC immediate is the number, public set starts at 16. Leaves 0–2 for ADR-013 probes. Arguments in
x0–x2, return inx0. - Wait for a loader. Would block Track A on A3.
Decision
-
Calling convention. AArch64
SVC #<n>wherenis the syscall number. Arguments inx0,x1,x2. Return value inx0. DAIF stays masked on the existing EL0 path. Not Linux. Not POSIX. -
Reserved 0–2. Probe-only (ADR-013). Not part of the public ABI. A later CRT must not issue them.
-
Public numbers (this tree).
Number Name Args Effect Return 16 exitx0= status (recorded; not a process model)End the EL0 trip; ERETto the EL1 caller(does not return to EL0) 17 uart_writex0= user pointer,x1= lengthCopy up to 64 bytes from a user-mapped and kernel-mapped range; write to PL011 ( \n→\r\n, same as the kernel console)bytes written, or 0if rejected18 yieldnone Record the call; ERETback to EL0 (next insn)0 -
yieldis dispatch-only. It does not callsched::yield_now(ADR-010: that switch is EL1-only and must not run from an exception frame). A later ADR may connect the two. -
uart_writeis fail-closed. Length0or> 64, overflow, a non-canonical or TTBR1 alias, or a range that is not user-mapped or not kernel-mapped returns0and writes nothing. Kernel.datais rejected. Table walks mask to 39 bits — the pointer itself must already be a TTBR0 identity VA, orcopy_userwould load a tagged address. This is not a VFS. -
Unknown immediates stay unhandled (park). Do not silently succeed.
-
Fail-closed probe. Hello serial
svc: yield+ user buffersvc: user-hi+svc: uart+svc: exit+svc: ok.scripts/qemu-smoke.shgreps those and rejectssvc: probe missed.#[test_case]covers the success trip, a kernel-.datareject, and a TTBR1-alias reject. Existingel0:markers stay. -
Honesty. Say “EL0 issued the documented SVC ABI and the kernel performed the documented effect” only when the serial / tests pass. Do not say: app hosting is done, Linux ABI, POSIX, userspace, “EL0 isolated,” or “secure OS.” Standing-as-normal is ADR-024. Thin VFS numbers 19–23 are ADR-027. Track A A7–A9 stay Planned.
-
NFR-10 text is revised in place (ID unchanged) to name this ABI mile. Do not mint FR-16+ or NFR-15+.
Consequences
- Code:
src/syscall.rs. Dispatch is hooked fromhandle_sync_lower_elafter the ADR-013 probe immediates. - Docs: syscall.md, el0.md. Roadmap cites #32 / Track A #31.
- A2 (
libctos) wraps these three numbers (ADR-022). A3 (loader) is not that PR.
ADR-022 — Freestanding CRT / libctos
- Status: Accepted (CRT mile Verified when the serial / tests pass; Track A / app hosting still Planned)
- Date: 2026-09-12
Context
Track A (issue #31) wants a freestanding program to call the A1 ABI without glibc or POSIX. Issue #33 is this mile: a documented, probed CRT + libctos that wraps ADR-021 numbers 16–18.
A1 already proved the kernel performs exit / uart_write / yield for a hand-encoded trampoline. That is not a library a program can link. A3 (ELF loader) must not start until something exists to link against.
Options:
- Wait for a loader. Would block A2 on A3.
- C/asm static lib only. Fine later; this tree is Rust-first and already uses
aarch64-ctos. no_stdcrate + asm stubs + a hello payload copied onto today’s standing EL0 page. Proves link + run without a guest ELF loader.
Decision
libctoscrate (libctos/).no_stdwrappersexit/uart_write/yield_nowover asm stubs (sys.S:ctos_exit/ctos_uart_write/ctos_yield). SVC immediates are 16 / 17 / 18. Reserved 0–2 stay ADR-013 probes — the CRT must not issue them.- CRT is
_startinlibctos/src/crt0.S: kernelERETalready setSP_EL0;_startcallsmain, thenctos_exit. No argv, noenviron, no libc init. Not glibc/muslcrt0. - Hello payload (
user/hello-libctos). Ano_std/no_mainprogram linkslibctos, printslibctos: hiviauart_write,yields, printslibctos: ok, returns 0. Linked atpaging::EL0_PAGE(0x80002000) withuser/hello-libctos/linker.ld. - Not a guest loader.
build.rscompiles that program on the host and extractsPT_LOADbytes at/afterEL0_PAGE(size metadata + publish). The kernel copies a flatten of FAT/helloonto the standing EL0 page andERETs — same style as A1. Host flatten inbuild.rsis not A3. Guest flatten of the FAT ELF (ADR-032) is still A2 memcpy, not the A3 map. - Code page + stack page. The image must fit in 3584 bytes on
EL0_PAGE(EL0-exec, no EL0 data — same AP as A1). Rustmainsavesx30on SP; that store cannot land on the code page (SCTLR.WXNforbids making it W+X). A second map-window page is EL0-RW NX. rust-lld’s extra ELF-headerPT_LOADat0x80000000is discarded at extract time. - Fail-closed probe. Hello serial
libctos: hi+libctos: ok(from the payload viauart_write) + kernellibctos: linked.scripts/qemu-smoke.shgreps those and rejectslibctos: probe missed.#[test_case]covers the trip and that the blob encodes SVC #16/#17/#18 and not #0/#1/#2. Existingsvc:/el0:markers stay. - Honesty. Say “a program linked against libctos issued the A1 ABI and the kernel performed the documented effect” only when the serial / tests pass. Do not say: app hosting is done, Linux ABI, POSIX, userspace, “EL0 isolated,” or “secure OS.” Track A A3–A9 stay Planned.
- NFR-10 text is revised in place (ID unchanged) to name this CRT mile. Do not mint FR-16+ or NFR-15+. Guest ELF load is ADR-023, not this ADR.
Consequences
- Code:
libctos/,user/hello-libctos/,build.rs,src/libctos.rs. - Docs: syscall.md, el0.md, building-or-porting.md. Roadmap cites #33 / Track A #31.
- A3 (ELF / raw loader into user TTBR0) may load a separate image. It is not this PR.
ADR-023 — Guest ELF64 PT_LOAD loader into user TTBR0
- Status: Accepted (loader mile Verified when the serial / tests pass; Track A / app hosting still Planned)
- Date: 2026-09-12
Context
Track A (issue #31) wants a freestanding image loaded into user maps and entered with ERET. Issue #34 is this mile.
ADR-022 already links user/hello-libctos against libctos. That hello is host-built. build.rs extracts PT_LOAD and the kernel memcpys the blob onto paging::EL0_PAGE. That is not a guest loader.
A3 must parse and map on the guest. The bytes may still be embedded (no VFS yet — A6/A7). The point is map + load + ERET, not host flatten-and-copy onto a fixed probe page alone.
Existing A1 svc:* and A2 libctos:* markers stay. A4–A9 stay out of this PR.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Minimal ELF64 LE AArch64 ET_EXEC, PT_LOAD only | The in-tree payload is already that ELF. A2 already walks PT_LOAD on the host. Guest parse is the missing mile. |
| H2 (rejected) | Custom raw image (base VA + entry + blob) | Smaller parser, but the host would flatten it — same shape as A2. Would not prove guest ELF walk. |
| H3 (rejected) | Linux ELF ABI / PT_INTERP / ET_DYN / TLS / glibc | Inflates the claim. Fail-closed: reject PT_INTERP. Not Track B. |
Decision
- Guest parser in
src/loader.rs. Accept ELF64 little-endian AArch64ET_EXEC. Walk program headers. LoadPT_LOADonly. IgnorePT_PHDR/PT_GNU_STACK/ notes. RejectPT_INTERP. Reject aPT_LOADthat is W+X (segment flags or two segments that union to W+X on one page).SCTLR.WXNis already on (ADR-015); do not map a writable executable leaf. - User TTBR0 maps. Each page of a
PT_LOADis a freshly allocated frame mapped in the shared map-window L3 (paging::MAP_WINDOW…+ MAP_WINDOW_SIZE). That L3 is already inL1_USER, so EL0 and EL1 both see it. Permissions:PF_X→map_el0_exec(EL0 fetch, no EL0 data; EL1 may write the identity PA before / while mapping);PF_W→map_el0_rw; R-only →map_el0_ro. Page-granular union:.textRX and.rodataR on one 4 KiB page become Exec. Not a new user address space. - Copy + BSS. File bytes are written to the frame’s identity PA.
p_memsz > p_fileszis zeroed. I-cache is synced on executable pages.e_entrymust land in an Exec page. - Stack. A separate EL0-RW NX page at
paging::LOADER_STACK_VA(0x80007000), not the A2 code page. Rustmainstill savesx30onSP_EL0. - Image source. First cut embedded the ELF (
include_bytes!). ADR-032 reads the same bytes from FAT/hello.build.rsstill compiles and publishestarget/hello-libctos.elf. Not a second QEMU-kernel. - ERET.
el0::install_standing(e_entry, stack_top, user_ttbr0)thenexception::eret_to_el0.SYS_EXITreturns to EL1 as today. A2’s memcpy path stays solibctos:*markers remain. - Fail-closed probe. Hello serial
loader: mapped+loader: okafter a successful trip. The loaded payload still printslibctos: hi/libctos: okviauart_write.scripts/qemu-smoke.shgreps those and rejectsloader: probe missed.#[test_case]covers parse,PT_INTERPreject, W+X reject (flags and overlapping pages), RX+R union on the hello page, and the full map +ERETtrip. Existingsvc:/libctos:/el0:markers stay. - Honesty. Say “the guest parsed a freestanding ELF64, mapped
PT_LOADinto user TTBR0, andERETed toe_entry” only when the serial / tests pass. Do not say: Linux ELF ABI,PT_INTERP, glibc, app hosting is done, POSIX, userspace, “EL0 isolated,” or “secure OS.” Standing-as-normal is ADR-024. Track A A5–A9 stay Planned. - NFR-10 text is revised in place (ID unchanged) to name this loader mile. Do not mint FR-16+ or NFR-15+.
Consequences
- Code:
src/loader.rs,paging::map_el0_ro/window_range_ok/LOADER_STACK_VA,build.rsemits the ELF,scripts/qemu-smoke.sh. - Docs: el0.md, syscall.md, building-or-porting.md. Roadmap cites #34 / Track A #31.
- A4 (standing EL0 as normal mode) keeps this loader as the way a payload appears (ADR-024).
ADR-024 — Standing EL0 as normal mode
- Status: Accepted (standing-task mile Verified when the serial / tests pass; Track A / app hosting still Planned)
- Date: 2026-09-12
Context
Track A (issue #31) wants a loaded freestanding image to run at EL0, not only enter and leave. Issue #35 is this mile.
ADR-013 already has a bounded standing context: SVC #1 stays at EL0, a user MOVZ runs, SVC #2 restores EL1. is_active() is true only for that lifetime. That is a smoke probe, not the supported path for a loaded app.
ADR-021 / ADR-022 / ADR-023 already ERET to a payload and come back on SYS_EXIT. They reuse install_standing as a flag. A leftover is_active() after ERET fails the probe. That is still a one-shot trip, not “standing EL0 is how a loaded image runs.”
A4 keeps the A3 loader as the way a payload appears. It does not add VFS, slots, PAN, or a Linux exec. Existing svc:* / libctos:* / loader:* / el0: standing / el0: restored markers stay.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Distinguish probe vs task standing. A task is the supported path: load (A3) → is_active() true → stay at EL0 for yield / uart_write → exit restores. Unexpected lower-EL sync while a task is standing restores fail-closed instead of parking. | Proves real-task is_active and a restore that cannot leave a dangling standing context. |
| H2 (rejected) | Treat the ADR-013 dual-SVC as “normal mode.” | Still a one-shot SVC #1 / #2 trampoline. Does not run a loaded app until exit. |
| H3 (rejected) | Claim isolation / PAN / process table while promoting standing. | Inflates the mile. Isolation is A5 (issue #36). |
Decision
- Two standing kinds.
install_standingremains the ADR-013 / A1–A3 probe (dual-SVC or ABI trampoline).install_taskis the A4 task.is_active()is true for both.is_task()is true only for a task. Saved user PC / SP / TTBR0 live only whileis_active(). - Supported path. The A3 guest ELF loader maps
PT_LOADfrom FAT/hello(ADR-032), theninstall_task(e_entry, stack_top, user_ttbr0)andERET. The loaded image runs at EL0 untilSYS_EXIT.SYS_YIELD/SYS_UART_WRITEstay at EL0 and must observeis_active()(el0: task-active). A2’s memcpy path stays solibctos:*markers remain (same FAT file; no embed). - Fail-closed restore.
SYS_EXITon a task printsel0: task-exit/el0: task-restoredand clearsis_active(). An unexpected lower-EL sync (unhandled SVC, translation/permission abort, and anything that would otherwise park) while a task is standing printsel0: restore-fail, clearsis_active(), and returns to the EL1 caller. It does not park. Probe standing still parks on unexpected sync (A1–A3 unchanged). AfterERETreturns, leftoveris_active()is Failed (caller force-clears and fails the probe). - Fail-closed probe. Hello serial
el0: task-enter+el0: task-active+el0: task-exit+el0: task-restored+el0: restore-fail+el0: task-ok.scripts/qemu-smoke.shgreps those and rejectsel0: task missed.#[test_case]covers the loaded-hello task trip and the fault-restore trip. Existingsvc:/libctos:/loader:/el0: standing/el0: restoredmarkers stay. - Honesty. Say “a loaded freestanding image stood at EL0 until
exit, and an unexpected fault restored fail-closed” only when the serial / tests pass. Do not say: app hosting is done, POSIX, userspace, “EL0 isolated,” Linuxexec, or “secure OS.” Track A A5–A9 stay Planned. - NFR-10 text is revised in place (ID unchanged) to name this standing-task mile. Do not mint FR-16+ or NFR-15+.
Consequences
- Code:
src/el0.rs(install_task,restore_exit,restore_fault),src/loader.rs(run_hello_as_task),src/exception.rs(task restore instead of park),src/syscall.rs(note_active_while_standing),scripts/qemu-smoke.sh. - Docs: el0.md, syscall.md, track-a.md. Roadmap cites #35 / Track A #31.
- A5 (isolation completion — remaining identity
.rodata/.data/ heap tear; PAN only if the probe CPU implements it and an ADR says so) is not this PR.
ADR-025 — Identity .rodata tear after pointer rewrite
- Status: Accepted (identity
.rodataunmapped;.data/ heap still identity-mapped) - Date: 2026-09-12
Context
Track A / A5 (issue #36, parent #31) continues identity teardown where safe. ADR-020 rewrote rustc vtables and unmapped live identity .text after the boot stub. .rodata / .data / heap stayed mapped: string literals and rewritten tables still lived there.
After the ADR-019 high-VA jump, new PC-relative accesses (ADRP) already reach the TTBR1 alias. R_AARCH64_ABS64 words that point at .rodata stay identity until rewritten — the same class as the ADR-020 fn-pointer tables.
.data / .bss / linker stacks / heap are not safe to unmap here. SP is still the identity __stack_top. GlobalAlloc still returns identity frame VAs. Those stay Planned.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Rewrite identity pointers to .rodata, then unmap identity .rodata from kernel and user TTBR0. High twins stay. .data / heap stay. | Largest honest cut that does not relocate SP or change the allocator. |
| H2 (rejected) | Also unmap identity .data / heap. | SP and Box/Vec pointers are still identity VAs. A silent yank would fault. |
| H3 (rejected) | Claim PAN / “EL0 isolated” while tearing .rodata. | Isolation is not this mile. PAN is ADR-026. |
Decision
- Rewrite identity
.rodatapointers. After the live.texttear, walk.rodataand.data. Every 8-byte word that is an identity address in[__rodata_start, __ident_tear_start)is rewritten to its TTBR1 alias. RO pages are writable only for that store, then restored. Serialident: ro-reloc n=N.N == 0is allowed (no ABS64 hits); the tear still has to prove high-only access. - Unmap identity
.rodata. Unmap[__rodata_start, __ident_tear_start)from kernel and user TTBR0 (TLBI VAAE1each low VA). High twins stay. Serialident: rodata lo=… hi=… pages=NwithN >= 1. - Keep
.data/ heap / linker stacks identity-mapped. SP and the first-fit heap still use those VAs. Do not yank them here. - Keep
_start/ QEMU-kernelat0x4008_0000. Do not change default-cpu. Do not claim PAN. - Fail-closed probe. Existing
ident:*markers stay. Newident: ro-reloc/ident: rodata/ident: rodata-fault/ident: rodata-high. EL1LDRof a torn identity.rodataVA is a current-EL translation DABORT. EL1LDRof the high twin still returns a known.rodatamagic.scripts/qemu-smoke.shgreps those strings and rejectsident: rodata missed/ident: ro-reloc missed.#[test_case]covers the unmap. - Still Planned (at accept time). Identity
.data/ heap tear. Superseded in part by ADR-037 (.data/stacks) and ADR-038 (heap). PAN enable on-cpu cortex-a57; EL0 entry withoutTLBI VMALLE1; lower-EL IRQ while standing; umbrella EL0 isolation. - NFR-10 text is revised in place (ID unchanged). Do not mint NFR-15+.
Honesty
Say “identity .rodata was unmapped while println! still ran from the high twin” only when the serial / tests pass. Do not say:
- the kernel has moved to the high half
- identity mappings were fully torn down (
.data/ heap stay) - “secure OS” / “hardened” / “EL0 isolated”
- PAN (see ADR-026; unimplemented on
-cpu cortex-a57)
Consequences
paging::rewrite_identity_rodata_ptrs/paging::tear_identity_rodataown the cut. ADR-020 remains the live.texttear.- A later ADR may relocate SP and return high heap VAs, then unmap identity
.data/ heap. That work is not this cut.
ADR-026 — PAN capability: probe the ID field, do not invent the CPU
- Status: Accepted (capability probe). Historical a57:
pan: absent. Default smoke CPU since ADR-079:-cpu cortex-a76→pan: present. PAN enable + EL1-vs-EL0 fault: ADR-080 (ADR-054 gate items 3–4). - Date: 2026-09-12
Context
Track A / A5 (issue #36) may take Privileged Access Never only if ID_AA64MMFR1_EL1.PAN != 0 on the probe CPU and an ADR says so. The documented QEMU line is -cpu cortex-a57 (ARMv8.0). PAN is typically unimplemented there. Silently switching -cpu (for example to max) to make the field non-zero is not an honest probe.
ADR-013 already named PAN as a closing isolation probe: an EL1 access to an EL0-accessible page must fault when PSTATE.PAN is set. That enable + fault mile is not this ADR.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Read ID_AA64MMFR1_EL1.PAN on the default -cpu. Print the field. Enable PSTATE.PAN only if the field is non-zero and a later cut proves the EL1-vs-EL0 fault. On cortex-a57 record pan: absent. | CPU evidence without inventing hardware. |
| H2 (rejected) | Change default -cpu to max (or another v8.1+ model) so PAN appears. | Silent CPU switch. Forbidden by A5 and by this ADR. |
| H3 (rejected) | Claim PAN / “EL0 isolated” from the ID-field print alone. | File presence / a register read is not an access-fault probe. |
Decision
- Probe the ID field.
src/pan.rsreadsID_AA64MMFR1_EL1bits [23:20] and printspan: id=<n>. It does not executeMSR PAN. An unimplementedMSR PANis UNDEF on ARMv8.0. - Default CPU at accept time was
cortex-a57. Superseded for the default probe by ADR-079 (-cpu cortex-a76). Do not silent-edit without an ADR. - When
n == 0(expected). Serialpan: absent. Ledger: “PAN enable on virt cortex-a57” stays Planned, with this ID-field print as the CPU evidence.scripts/qemu-smoke.shgrepspan: id=/pan: absentand rejectspan: enabled/pan: probe missed. - When
n != 0. Serialpan: present. ADR-080 enables PSTATE.PAN and proves EL1-vs-EL0 fault. Smoke expectspan: present+pan: enabled+pan: el1-faulton-cpu cortex-a76. - Still Planned. PAN enable + EL1-vs-EL0 access fault; lower-EL IRQ while standing; EL0 entry without
TLBI VMALLE1; umbrella EL0 isolation. ADR-032 re-states that leftover: do notMSR PANwhile the ID field is 0. ADR-025 is the identity.rodatacut, not this. - NFR-10 text is revised in place (ID unchanged). Do not mint NFR-15+.
Honesty
Say “ID_AA64MMFR1_EL1.PAN is 0 on -cpu cortex-a57” only when the serial / tests pass. Do not say:
- PAN is enabled
- EL1 cannot access EL0 pages
- “secure OS” / “hardened” / “EL0 isolated”
- the probe used a different
-cputhan the scripts
Consequences
src/pan.rsowns the ID-field probe. Isolation docs cite this ADR as Planned for the enable mile.- ADR-047 records: PAN enable is a non-goal on the default probe CPU; ID-field probe stays Verified.
- A future CPU change needs its own ADR + sponsor and must not be a silent script edit.
- Enable lock: ADR-054 (2026-09-14) locks PAN enable as non-goal on default cortex-a57 and documents the reopen gate (sponsor CPU + FEAT_PAN + enable+fault ADR).
ADR-027 — Thin VFS + in-RAM memfs
- Status: Accepted (memfs mile Verified when the serial / tests pass; Track A / app hosting still Planned)
- Date: 2026-09-12
Context
Track A (issue #31) wants a freestanding app to read and write named buffers. Issue #37 is this mile: a thin VFS with one backend, an in-RAM memfs, plus a fail-closed create / write / read / close probe.
ADR-021 already has exit / uart_write / yield (16–18). There is no path, no handle, and no store. A later virtio-blk + on-disk FS (A7 / issue #38) must not invent a second “open” story. The VFS ADR lands first.
This is not Linux VFS. Not POSIX open / stat / mount. Not a disk. Not app hosting.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Tiny VfsOps (create / open / read / write / close) + one MemFs of heap Vec<u8> named files. New public SVC 19–23. Kernel probe + EL0 trampoline. | Same “easiest in-tree” shape as a coop task. No DMA. One backend this mile. |
| H2 (rejected) | Skip the trait and call memfs from SVC only. | A7 would then invent a second interface. The VFS is the mile. |
| H3 (rejected) | Ship virtio-blk or FAT in the same PR. | A7. Host -drive without guest code is not a probe. Do not say “supports FAT.” |
| H4 (rejected) | Linux/POSIX open flags, directories, cwd, O_CREAT. | Inflates the mile and invites a POSIX claim. |
Decision
-
Thin VFS.
src/vfs.rsdefinesVfsOps:create/open/read/write/closeon a path + handle. One global backend this mile:MemFs. Later A7 may add a block FS behind the same trait. Not a superblock. Not a dentry cache. -
memfs. Flat namespace. Paths are
/plus 1–15 bytes of[a-z0-9_-](no extra/, no.). At most 8 files, 256 bytes each, 4 handles. File bytes live on the first-fit heap (Vec<u8>, ADR-009). Handles are1..=4. Offset is per handle.createfails if the name exists.openfails if missing.writestores at the handle offset and grows up toFILE_MAX.readcopies from the offset.closedrops the handle; the file stays. Not a directory tree. -
Public SVC numbers (extends ADR-021). Same calling convention (
SVC #<n>,x0–x2, returnx0). Reserved 0–2 stay ADR-013 probes.Number Name Args Effect Return 19 fs_createx0= user path,x1= lengthCreate an empty named file and a handle fd >= 1, or020 fs_openx0= user path,x1= lengthOpen an existing name fd >= 1, or021 fs_readx0= fd,x1= user buf,x2= lengthCopy from the handle offset into a user-mapped and kernel-mapped range bytes, or u64::MAXon reject (0is a valid empty read)22 fs_writex0= fd,x1= user buf,x2= lengthCopy into the file at the handle offset bytes, or 023 fs_closex0= fdDrop the handle 0on success,u64::MAXon rejectPath / I/O length is capped (path 16, I/O 64). Fail-closed: empty length, overflow, non-canonical / TTBR1 alias, or a range that is not user-mapped or not kernel-mapped returns the reject value and does not mutate the file. Kernel
.datais rejected. Unknown immediates still park. -
libctoswrappers.fs_create/fs_open/fs_read/fs_write/fs_closeoversvc #19–#23. The A2 hello payload staysexit/uart_write/yieldonly so A2–A4 markers do not change. -
Fail-closed probe. Kernel create / write / read / close of
/kprobeprintsfs: create/fs: write/fs: read. An EL0 trampoline does the same for/eprobeand printsfs: el0. Thenfs: ok.scripts/qemu-smoke.shgreps those and rejectsfs: probe missed.#[test_case]covers the kernel round-trip, a missingopen, the EL0 trip, and a kernel-.datawrite reject. Existingsvc:/libctos:/loader:/el0: task-*/ A5ident: rodata*/pan:*markers stay. -
Honesty. Say “the guest created, wrote, and read a named in-RAM file behind a thin VFS” only when the serial / tests pass. Do not say: app hosting is done, POSIX, Linux VFS, FAT, virtio-blk, on-disk FS, userspace, “EL0 isolated,” or “secure OS.” Track A A7–A9 stay Planned.
-
NFR-10 text is revised in place (ID unchanged) to name this memfs mile. Do not mint FR-16+ or NFR-15+.
Consequences
- Code:
src/vfs.rs,src/syscall.rs(19–23),libctos/wrappers,scripts/qemu-smoke.sh. - Docs: filesystem.md, overview/filesystem.md, syscall.md, track-a.md. Roadmap cites #37 / Track A #31.
- A7 (virtio-blk + FAT or xv6-like on that block device) is not this PR. Choose the on-disk format in that ADR.
ADR-028 — virtio-blk + FAT16 behind the thin VFS
- Status: Accepted (block + FAT read mile Verified when the serial / tests pass; Track A / app hosting still Planned). Write depth: ADR-050.
- Date: 2026-09-12
Context
Track A (issue #31) wants a freestanding app to read a named file on a disk, not only an in-RAM buffer. Issue #38 is this mile: QEMU virt virtio-mmio block (virtqueues, DMA, sector R/W) plus one on-disk FS, mounted through the A6 thin VFS (ADR-027).
Host QEMU -drive without guest code is not a probe. memfs stays one backend. This mile must not invent a second open story. Not POSIX. Not ext4. Not app hosting. A8–A9 stay Planned.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | virtio-mmio blk (poll used ring) + FAT16 on a host-built raw image. Same VfsOps / vfs::open as memfs. Known file /probe holds fat-hi. | Host can inspect the image (scripts/mkfat16.py --check). Guest DMA + VFS read is the probe. |
| H2 (rejected) | Tiny xv6-like inode FS. | Fine as a teaching layout, but the host cannot file / mount / check the image without a ctos-specific tool. Issue #38 prefers a host-visible image. |
| H3 (rejected) | FAT32. | Root is a cluster chain; more code for the same probe. A 4 MiB image is naturally FAT16 once cluster count ≥ 4085. |
| H4 (rejected) | A second open / mount SVC, or a /d/ prefix that breaks the A6 path grammar. | ADR-027 forbids a second open story. Paths stay / + [a-z0-9_-]. |
| H5 (rejected this mile) | virtio-blk-pci, virtio IRQ via GIC, or write-back FAT this mile. | PCI and IRQs are extra surface. FAT stayed read-only here (guest write → FsError::ReadOnly). Sector R/W is proven on an unused last sector. Write-back depth is ADR-050. |
Decision
- On-disk format is FAT16. Chosen because the smoke host can build and inspect a raw image without a ctos-specific decoder. xv6-like and FAT32 stay unchosen. Do not say “supports FAT” as a product; say “the guest read a known FAT16 file from a virtio-blk image.”
- virtio-mmio on QEMU
virt. Scan transports at0x0a000000 + n×0x200(32 slots) for Device ID 2. QEMU 8.2virtships legacy version 1 on those transports (page-alignedQueuePFN); modern version 2 is also accepted. One split virtqueue. DMA addresses are identity PAs of a 4 KiB-aligned.bssbuffer (.data/heap stay identity-mapped; ADR-025). Pollused.idxafterQueueNotify. Cache maintenance (dc civac) covers the whole DMA object (desc/avail and used/req/status/data) — a page-0-only sync leaves a stale request on write-back D-cache. No virtio IRQ this mile. Device MMIO already sits in the Device-nGnRnE L1 block. - Thin VFS, two backends.
vfs::open/read/write/closestay the only story. memfs is tried first. If the name is missing there, FAT16 root (8.3, no extension →/probeforPROBE) is tried.createstays memfs and fails withExistsif FAT already has the name. FATwrite/createreturnReadOnly. Same path grammar as A6. No extra/. No new SVC numbers. - Host image.
scripts/mkfat16.pywrites a 4 MiB raw FAT16 volume (cluster count ≥ 4085 so the type is FAT16, not FAT12) with one filePROBE/ contentsfat-hi.scripts/qemu-smoke.sh,scripts/qemu-aarch64.sh, andscripts/qemu-serial-inject.pyattach it as:-drive if=none,file=<img>,format=raw,id=hd0 -device virtio-blk-device,drive=hd0 - Fail-closed probe. virtio capacity + last-sector write/read prints
blk: virtio/blk: cap/blk: rw/blk: ok. VFS open/read of/probeprintsfat: mount/fat: read/fat: ok.scripts/qemu-smoke.shgreps those and rejectsblk: probe missed/fat: probe missed. Host--checkof the image is extra, not a substitute for the guest.#[test_case]covers sector R/W, VFS/probe, and a missing FAT name. Existingsvc:*/libctos:*/loader:*/el0: task-*/ident:*/pan:*/fs:*markers stay. - Honesty. Say “the guest programmed virtio-blk and read a known FAT16 file through the thin VFS” only when the serial / tests pass. First hello on a modern-only (
Version==2) scan wasblk: probe missed— QEMU 8.2virtexposes legacy version 1. After accepting v1 +QueuePFN, this cloud VM printedblk: okandfat: ok. Do not say: app hosting is done, POSIX, Linux VFS, FAT32, xv6, writeable volumes, virtio-net, “EL0 isolated,” or “secure OS.” Track A A8–A9 stay Planned. Host-drivewithout guest virtio + VFS read is not a probe. - NFR-10 text is revised in place (ID unchanged) to name this virtio-blk + FAT16 mile. Do not mint FR-16+ or NFR-15+.
Consequences
- Code:
src/virtio.rs,src/fat.rs,src/vfs.rs(dispatch +ReadOnly),scripts/mkfat16.py, QEMU-drivein the smoke / runner scripts. - Docs: filesystem.md, overview/filesystem.md, track-a.md, threat-model v1.15. Roadmap cites #38 / Track A #31.
- A8 (documented sample apps, issue #39) is not this PR. Follow-up A8 documented rebuild recipes (what-can-run.md); it did not add a new
libctoshello that callsfs_open. The Verified EL0 VFS trip stays/eprobe. Not a new ABI and not a second FS.
ADR-029 — Guest OCI/Docker containers are a non-goal
- Status: Accepted (docs decision; no guest runtime)
- Date: 2026-09-12
Context
Track B (issue #40) is Linux-compat research. Child B7 / #47 asks for an explicit claim: ctos does not host Docker/OCI; today Docker on a Linux host runs the ctos smoke image.
Site pages already said no (hosting-apps.md, extra host-apps.md). Track B still listed B7 as Planned and said containers “stay out or far-later.” That is an honesty gap: far-later reads like a later Planned feature.
OCI / Docker / Kubernetes guests need a Linux host: namespaces, cgroups, a Linux ABI, usually overlay or equivalent, and a rich syscall surface. This learning kernel does not have those. Host docker-smoke.sh is a harness (NFR-04 / NFR-05), not a guest runtime.
This ADR does not implement a container runtime. It does not mint FR-16+ or NFR-15+. B1 (Linux-compat goals) may cite it and must not reopen containers as Planned.
Decision
- Non-goal. The ctos guest is not a container host. OCI / Docker / Kubernetes workloads on the guest are not Planned. Do not write a container roadmap on Track B.
- Arrow today. Docker (or Podman) on a Linux host may build and QEMU-smoke this kernel. That is “Docker hosts ctos,” not “ctos hosts containers.” Ledger Docker rows stay host-harness probes.
- Reopen gate. Far-later only with a new GitHub epic plus a new ADR that names the missing Linux-host features and a probe. Track B B1–B6 must not treat containers as a win or a silent Planned row. Do not mint a new FR/NFR ID for this.
- Principles drive. Honesty (NFR-06): absence of OCI/runc/cgroup/namespace code in
src/is the probe. Antifragility keeps the host Docker ratchet. Security / performance claims about “container isolation” stay unwritten.
Consequences
- Docs: track-b.md B7 becomes Documented. Site SoT: hosting-apps.md. Extra: host-apps.md. Ledger row stays Verified absence + non-goal.
- Track A A8 (sample apps) is unrelated. Do not add an in-tree “container” sample.
- “Run Alpine on ctos” / “k8s node” stays the same class of claim as “POSIX port is easy.” It is not.
ADR-030 — OS image vs app payload slots
- Status: Accepted (first-cut two-artifact + FAT load path). Cross-update is ADR-032 (same published ELF on this OS and
ba6541c). - Date: 2026-09-12
Context
Track A (issue #31) wants a freestanding app that is not rebuilt every time the kernel is. Issue #48 is this mile: disconnect OS image updates from app payloads.
The sponsor goal is scoped immutability, not an “immutable OS” product sentence (immutability.md). A1–A8 already landed ABI, libctos, a guest PT_LOAD loader, standing EL0, memfs, virtio-blk + FAT16, and sample recipes. Today the hello ELF is still include_bytes! into the kernel. That is one linked image.
This mile must not invent OTA, A-B flash, or OCI containers (ADR-029). No new FR/NFR IDs.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Two host artifacts (kernel ELF + hello-libctos.elf) plus a FAT16 app slot (/hello). Guest vfs::open + A3 PT_LOAD map. A2–A4 kept the embed on the first cut. | Honest first cut. Storage is the existing memfs/raw-image story (A6/A7). Cross-update and embed removal are ADR-032. |
| H2 (first-cut reject; leftover accept) | Remove include_bytes! and load A2–A4 from FAT only. | Rejected on the first cut (PROBE-only image risk). Leftover mile takes this after smoke already requires --app. |
| H3 (rejected) | Second QEMU -kernel, fw_cfg, or a custom raw slot format. | Extra boot contract. FAT16 + VFS is already probed. |
| H4 (rejected) | Claim cross-update Verified because the SVC numbers did not change. | ABI stability is not a two-boot probe. File presence of this ADR is not that probe. |
| H5 (rejected) | OTA / A-B flash / OCI layers. | Out of scope. Not Android. Not a container host. |
Decision
- Slots. The OS image is the kernel ELF QEMU
-kernels (target/aarch64-ctos/debug/ctos). The app payload is a separate host ELF (target/hello-libctos.elf) built fromuser/hello-libctos. The guest app slot is FAT16/hello(8.3HELLO) on the existing virtio-blk volume./probestays the A7 known file. Same VFSopen/read/close. No new SVC numbers. - Build.
build.rscompiles the hello and publishes the ELF totarget/hello-libctos.elf(and still writes host flatten metadata). After ADR-032 the kernel does notinclude_bytes!that file.scripts/mkfat16.py --appwrites that ELF as/hello(cluster chain; cap 64 KiB). Smoke and the QEMU runner fail closed if the app artifact is missing. - Boot/load. Hello path: OS boots, A1–A8 probes run. After ADR-032, A2–A4 also read FAT
/hello(noinclude_bytes!). Then A9 reads/hellothrough the thin VFS, parses ELF64PT_LOADwith the A3 loader, andERETs. No embed fallback on this path. Missing/helloprintsslot: probe missed. - Fail-closed probe. Serial
slot: fat+slot: mapped+slot: okafter a successful FAT load +ERET.perf: app-load ticks=<n>is a CNTPCT measurement around that path (NFR-07) — not a bench and not a delta vs the embed.scripts/qemu-smoke.shgreps those and rejectsslot: probe missed/slot: embed. Host smoke requires both artifacts andmkfat16.py --check --require-app.#[test_case]covers FAT/helloELF magic + parse + the FAT load trip. - Honesty. Say “the guest loaded a separate app ELF from the FAT slot into user TTBR0” only when the serial / tests pass. Do not say: apps update independently of the OS, immutable OS, OTA, A-B, containers, “EL0 isolated,” or “secure OS.” Cross-update is a host two-boot probe (ADR-032): same published
hello-libctos.elfon this OS and onba6541c. File presence of this ADR is not that probe. A kernel rebuild still compiles the payload (build.rs); it no longer embeds the bytes. - Performance. Expected costs stay the documented shape (boot/load, SVC, ASID/TTBR). The new marker proves the path is measurable. A compared CNTPCT pair vs a probe-only linked-in trip is ADR-046. Do not invent a bench.
- NFR-10 text is revised in place (ID unchanged) to name this slot first cut. Do not mint FR-16+ or NFR-15+.
Consequences
- Code:
src/slot.rs,src/loader.rs(run_image),build.rspublish path,scripts/mkfat16.py --app, QEMU runner / smoke. - Docs: immutability.md, advantages.md, track-a.md, threat-model v1.16. Roadmap cites #48 / Track A #31.
- Track A children A1–A9 now have a first cut. ADR-032 takes cross-update + embed-off + isolation honesty. Slot-load CNTPCT pair is ADR-046 (Verified measurement, not a percent). Remaining Planned / decided: PAN enable non-goal on default a57, never yank
_start, leftover identity RAM optional, umbrella isolation Planned/non-claim (ADR-047); product “app hosting is done” (ADR-048).
ADR-031 — Linux-compat goals, non-goals, keep ctos specificity
- Status: Accepted (docs frame; no Linux userspace claim)
- Date: 2026-09-12
Context
Track B (issue #40) is Linux-compat research. Child B1 / #41 asks for the frame: what “Linux-compat” means here (ABI subset vs a full Linux kernel), what stays out, and which ctos invariants must not move.
B7 / ADR-029 already records that guest OCI / Docker / Kubernetes is a non-goal. This ADR cites that decision. It does not reopen containers as Planned or “far-later on this track.”
Track A (issue #31) has first cuts A1–A9: a tiny SVC ABI (ADR-021), libctos (ADR-022), a freestanding ELF64 PT_LOAD loader that rejects PT_INTERP (ADR-023), standing EL0 (ADR-024), thin VFS + memfs (ADR-027), virtio-blk + FAT16 (ADR-028), sample recipes, and an OS/app slot first cut (ADR-030). Those miles are freestanding. They are not a Linux ABI and not Linux userspace.
Issue #40’s reality check stands: a full Linux ABI (and containers) is a multi-year kernel project. Track B is an ADR / research ladder, not a promise to become Linux.
This ADR does not implement B2–B6. It does not mint FR-16+ or NFR-15+. File presence of this page is not a Linux userspace probe.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Frame Linux-compat as research into a possible ABI subset. Keep the freestanding Track A path. Containers stay ADR-029. Say “not claiming Linux userspace yet.” | Matches #40 / #41. Leaves B2–B6 room to map gaps and then decide. |
| H2 (rejected) | Promise a full Linux ABI, glibc/musl, or “runs Alpine / busybox.” | Honesty fail (NFR-06). No probe exists. |
| H3 (rejected) | Reopen guest containers as Planned or “far-later” on Track B. | Conflicts with ADR-029. Reopen only with a new epic + ADR. |
| H4 (rejected) | Replace Track A SVC / libctos with Linux AArch64 svc #0 / x8 numbers in this mile. | Would break probed A1–A9 markers for a research frame. B2 maps the gap; it does not rewrite src/. |
| H5 (rejected) | Treat A3 PT_LOAD + A9 FAT /hello as Linux userspace. | The loader rejects PT_INTERP. Standing EL0 is not execve. File presence is not that claim. |
Decision
-
What Linux-compat means. On ctos it means: research whether a subset of the Linux AArch64 ABI could ever be useful for a named, probed workload, without dropping ctos specificity. It is not a promise to run a Linux distro, glibc, musl, a shell, Python, or “Linux userspace.” Full Linux ABI is out of scope for this track’s promise. B6 may later choose “never.”
-
Goals (research ladder only). Later children stay document-first. This PR does not write them.
Later Job Must not do in that child B2 (#42) Map Linux aarch64 syscalls vs today’s ctos SVC (16–23 + reserved 0–2) Do not add Linux syscall numbers to src/B3 (#43, ADR-035) Process model vs Linux fork/exec/waitDo not add those syscalls. Standing EL0 stays the ctos model. B4 (#44) Linux ELF / auxv / PT_INTERPvs the freestanding loaderDo not accept PT_INTERP. Gap ADR: ADR-033.B5 (#45) Linux VFS concepts vs the thin ctos VFS Do not grow POSIX openflags / dentriesB6 (#46) Decision: compat layer vs reimplement vs never Do not treat “compat layer” as already chosen Implementation, if any, comes after B6 names a path, in a later epic, with a probe. “Never” is a valid B6 outcome.
-
Non-goals.
- Full Linux ABI / POSIX product / glibc or musl ports.
- Guest OCI / Docker / Kubernetes — already ADR-029. Not Planned on Track B.
- “Run Alpine on ctos,” “k8s node,” “Linux userspace works.”
- Replacing the freestanding Track A path (A1–A9 first cuts stay).
- Widening off QEMU
virt/ AArch64 without a new ISA ADR (ADR-003). - New FR/NFR IDs. Frozen set stays fr-nfr.md.
- Author self-merge (ADR-002).
-
Keep (ctos specificity). Track B is subordinate to principles.md. Linux-compat research must not override:
Invariant Why it stays Honesty (NFR-06 / ADR-001) Status words need a probe. Unprobed stays Unknown. This ADR is not Verified Linux. Antifragility (NFR-05) Fail-closed qemu-smokeand host Docker/arm64 workstation ratchets stay. Hostdocker-smoke.shis still “Docker hosts ctos,” not a guest Linux ABI.Security (NFR-10) No “secure OS,” “Linux-compat isolation,” or “the kernel is W^X” from a research frame. Performance (NFR-07) No invented benches for a compat layer that does not exist. Measure first if a later mile adds a path. Document-first / one-PR loops One child → one branch → one GitHub PR. Freestanding Track A path In-tree apps keep SVC #n,libctos,PT_LOADwithout interpreter, thin VFS. A research map must not silently retarget those numbers to Linuxx8.Virt learning scope QEMU virt+ PL011 until an ADR widens it.ADR-gated memory / EL0 Isolation, PAN enable, remaining identity RAM / _startstay their own rows. -
Relationship to ADR-029. Guest containers remain a non-goal. B1–B6 must not treat containers as a win, a silent Planned row, or a reason to grow namespaces / cgroups / overlay. Reopen only with a new GitHub epic plus a new ADR that names the missing Linux-host features and a probe. Do not mint an FR/NFR ID for that.
-
Not claiming Linux userspace yet. Say that sentence in the roadmap and on the Track B page. Today:
- A3 maps freestanding ELF64
ET_EXECPT_LOADand rejectsPT_INTERP. - Standing EL0 is not a Linux process table.
- Thin VFS + memfs / read-only FAT16 is not Linux VFS.
- A9 FAT
/hellois notexecve. - The ledger row “Guest runs host apps (Linux ELF / shell / Python)” stays Planned. This ADR does not change that row to Verified.
- A3 maps freestanding ELF64
-
Principles drive. Do not override honesty, fail-closed sensors, or the three pillars for a “runs Linux” headline.
Consequences
- Docs: track-b.md B1 becomes Documented. SUMMARY.md and roadmap.md link here. Site extra: hosting-apps.md / host-apps.md point at the frame without a userspace claim.
- B2 is Documented (syscall gap note). B3 is Documented (ADR-035). B4 is Documented (ADR-033). B5 / B6 stay Planned research. No
src/change in this PR. - Track A remains the product path for in-tree freestanding apps. Cross-update and “app hosting is done” stay unclaimed.
- Honesty: document inspection only. Do not add a Verified Linux-userspace ledger row because this file exists.
ADR-032 — Track A leftovers: cross-update, FAT-only hello, isolation honesty
- Status: Accepted (leftover mile when the serial / host smoke / tests pass)
- Date: 2026-09-12
Context
Track A children A1–A9 have first cuts. After ADR-030 merged on main ba6541c, three honesty gaps remained:
- A9 cross-update. Same published
hello-libctos.elfon this OS and a documented prior OS. ABI stability is not that probe. - Embed honesty. A2–A4 still
include_bytes!a hello copy while A9 loaded FAT/hello. Dual path or remove the embed. - Isolation leftovers. After A5: identity
.data/ heap tear if safe; PAN enable only ifID_AA64MMFR1_EL1.PAN != 0on default-cpu cortex-a57. Do not claim “EL0 isolated.”
No new FR/NFR IDs. Author does not merge (ADR-002). The number is 032 because open #62 already claimed ADR-031 for Linux-compat goals.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Cross-update: build documented prior OS ba6541c (A9 merge — earliest main tip with the slot/FAT path) and boot the same published app ELF on that kernel and on this tip. Fail closed. | Real SHA. No stored kernel blob. |
| H2 (rejected) | Claim cross-update from unchanged SVC numbers, or invent a prior kernel artifact that was never built here. | Invents evidence. |
| H3 (chosen) | Remove include_bytes! of hello. A2 flattens FAT /hello PT_LOAD; A3/A4 parse the same file. build.rs still compiles and publishes the payload. Host smoke greps that src/ has no hello include_bytes!. | Prefer (a). Markers stay if FAT /hello is present (smoke already requires it). |
| H4 (rejected) | Keep the embed as a dual path and only document the gap. | Allowed if (a) broke smoke. Not needed if FAT load stays green. |
| H5 (chosen) | Isolation: do not enable PAN (pan: absent on cortex-a57). Do not unmap identity .data / heap (SP and GlobalAlloc are still identity VAs). Print ident: data-stay / ident: heap-stay. Umbrella isolation stays Planned. | Advance the honest mile. A silent tear would fault. |
| H6 (rejected) | Switch -cpu so PAN appears, or yank .data/heap without relocating SP/allocator. | Forbidden by ADR-026 / ADR-025. |
Decision
- Prior OS is
ba6541c. That is the A9 merge commit (A9: Disconnect OS image from app payloads (ADR-030) (#60)). No earliermainSHA has the slot/FAT path. Do not store a prior kernel ELF in git.scripts/qemu-smoke.shbuilds that commit in a git worktree and QEMU-kernels it with the same app bytes this tip published (sha256 pin viasha256sum,shasum -a 256, oropenssl dgst -sha256). Docker smoke needsgitin the image and.gitobjects in the build context (not host.git/worktreesregistrations). Smoke prunes stale worktrees before add. Do not invent a stored kernel blob. Both boots must printslot: fat/slot: mapped/slot: okand must not printslot: embed/slot: probe missed. This OS SHA must differ fromba6541c. - No hello embed.
src/libctos.rsandsrc/loader.rsread FAT/hellothroughfat::read_file(same VFSopenas A7). A2 still memcpy-flattensPT_LOADat/afterEL0_PAGE(not a guest ELF map). A3/A4 still parse andERET. Size must matchHELLO_ELF_LEN/HELLO_LENfrom thisbuild.rsso a swapped slot cannot silently satisfy those miles. Host smoke rejectsinclude_bytes!.*hello-libctosundersrc/. - Isolation honesty. Serial
ident: data-stay/ident: heap-stayafter walking those identity pages.#[test_case]identity_data_and_heap_still_mapped. PAN enable stays Planned (pan: id=0/pan: absent; smoke still rejectspan: enabled). Do not claim “EL0 isolated.” - Still Planned (at accept time). Identity
.data/ heap tear (needs SP relocate + high allocator VAs). Superseded in part by ADR-037 and ADR-038. PAN enable on-cpu cortex-a57; lower-EL IRQ while standing; EL0 entry withoutTLBI VMALLE1; umbrella isolation; A9 slot-disconnect delta vs the old embed (later ADR-046); product “app hosting is done.” - NFR-10 text is revised in place (ID unchanged). Threat-model v1.17. Do not mint FR-16+ or NFR-15+.
Honesty
Say “the same published hello-libctos.elf loaded on this OS and on ba6541c” only when host smoke printed both cross-update … slot:ok lines and the app sha256 pin. Say “A2–A4 no longer embed the hello” only when src/ has no hello include_bytes! and the FAT markers still pass. Say “identity .data / heap are still mapped” when ident: data-stay / ident: heap-stay print. Do not say:
- apps update independently / app hosting is done / OTA / A-B / containers
- EL0 isolated / PAN enabled / the kernel moved / identity mappings fully torn down
- a prior OS that does not exist in git
- Verified without this cloud (or GHA) qemu-smoke
Consequences
-
ADR-037 supersedes the
.datastay marker. ADR-038 supersedesident: heap-staywith an honest heap tear (ident: heap-reloc/ident: heap). -
Code:
fat::read_file, loader/libctos FAT path, teardown stay markers,scripts/qemu-smoke.shembed + cross-update + stay greps. -
Docs: this ADR, ADR-030, A2–A4 ADRs (payload source), ADR-025 / ADR-026 (still Planned), ledger, Track A, threat-model v1.17.
ADR-033 — Linux ELF / auxv / PT_INTERP vs the freestanding loader
- Status: Accepted (gap ADR; no Linux ELF / dynamic-linker claim)
- Date: 2026-09-12
Context
Track B child B4 / #44 (parent #40) asks what would be needed to load a musl-linked binary versus today’s freestanding image, and to keep the Track A loader reusable.
Frame: ADR-031. That frame says B4 must not accept PT_INTERP, must not claim Linux userspace, and must not replace the freestanding Track A path. Syscall convention is already a gap (linux-aarch64-syscall-gap.md / B2). Process execve is B3 / #43. This ADR does not write B3, B5, or B6.
Track A A3 (ADR-023) already parses ELF64 little-endian AArch64 ET_EXEC, maps PT_LOAD only, rejects PT_INTERP, rejects W+X, and ERETs to e_entry. A2–A4 and A9 reuse that parser on FAT /hello (ADR-030, ADR-032). The in-tree payload is libctos, not musl or glibc.
This mile is docs only. It does not change src/loader.rs. File presence of this page is not a Linux ELF probe and not dynamic-linker support.
Probe (what Verified means here)
| Claim | Probe | Status |
|---|---|---|
Linux ELF types / PT_* | Read Linux v6.10 include/uapi/linux/elf.h (ET_EXEC 2, ET_DYN 3, PT_LOAD 1, PT_DYNAMIC 2, PT_INTERP 3, PT_PHDR 6, PT_TLS 7) | Document inspection |
| Linux auxv keys | Read Linux v6.10 include/uapi/linux/auxvec.h (AT_NULL … AT_EXECFN, AT_MINSIGSTKSZ) | Document inspection |
| ctos freestanding loader | Read src/loader.rs (parse_elf64, PT_INTERP reject, LOADER_STACK_VA) + ADR-023 on this tree | Document inspection |
| Dynamic Linux ELF / musl / glibc on the guest | No such serial marker | Not claimed. Ledger “Guest runs host apps” stays Planned. |
A later kernel qemu-smoke is a different row. This mile did not edit src/.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Document the Linux vs ctos gap. Keep the ADR-023 loader reusable as the freestanding contract. Do not accept PT_INTERP. Do not claim musl/glibc/dynamic ELF. | Matches #44 / ADR-031. Leaves B6 room to choose never. |
| H2 (rejected) | Accept PT_INTERP / ET_DYN / auxv in this mile, or rewrite parse_elf64 as a Linux binfmt_elf. | ADR-031 forbids accepting PT_INTERP here. Would break A3 Interp tests and A2–A9 markers. |
| H3 (rejected) | Treat A3 PT_LOAD + A9 FAT /hello as Linux execve / dynamic linking. | The loader rejects PT_INTERP. Standing EL0 is not execve. Honesty fail (NFR-06). |
| H4 (rejected) | Promise a musl or glibc port, “runs Alpine,” or a guest ld-linux / ld-musl. | ADR-031 H2. No probe exists. |
| H5 (rejected) | Fork a second incompatible parser this mile so Track A and “Linux ELF” diverge silently. | #44 asked to keep the Track A loader reusable. A later Linux path, if B6 ever names one, is a new layer that may call the PT_LOAD helpers — not a silent retarget. |
Decision
-
What Linux
execof an ELF does (inspection). Linuxfs/binfmt_elf.c(not copied here) loadsET_EXECorET_DYN(PIE). If aPT_INTERPstring is present, it also maps that interpreter (typically glibcld-linux-aarch64.so.1or muslld-musl-aarch64.so.1) and transfers control to the interpreter entry, not the binarye_entry. The kernel builds a user stack:argc→argv[]→NULL→envp[]→NULL→auxv[](AT_*key/value pairs) →AT_NULL.The auxiliary vector (Linux v6.10
auxvec.h) is how the dynamic linker and libc CRT find program headers, page size, interpreter base, the real entry, and entropy. Typical keys a musl/glibc_startreads:AT_PHDR/AT_PHENT/AT_PHNUM,AT_PAGESZ,AT_BASE,AT_ENTRY,AT_UID/AT_EUID/AT_GID/AT_EGID,AT_SECURE,AT_RANDOM(16 bytes),AT_HWCAP/AT_HWCAP2,AT_CLKTCK,AT_PLATFORM,AT_EXECFN, oftenAT_SYSINFO_EHDR(vDSO) andAT_MINSIGSTKSZ.After that, the interpreter walks
PT_DYNAMIC(DT_NEEDED,DT_SYMTAB,DT_RELA/DT_REL,DT_INIT_ARRAY), maps shared objects, applies relocations, sets upPT_TLS, then jumps toAT_ENTRY. That is dynamic linking. It is notPT_LOADcopy. -
What the ctos loader does (inspection).
src/loader.rs/ ADR-023:Item ctos today File type ELF64 LE AArch64 ET_EXEConly (NotExecon anything else, includingET_DYN)Segments PT_LOADonly.PT_PHDR/PT_GNU_STACK/ notes ignored.PT_INTERPParseError::Interp. Fail-closed. Smoke/tests require the reject.W+X Reject (segment flags or page-union). SCTLR.WXNis on (ADR-015).VA window paging::MAP_WINDOW0x8000_0000…+ MAP_WINDOW_SIZE(512 × 4 KiB). Caps: 8PT_LOADs, 8 pages.Stack One zeroed EL0-RW NX page at LOADER_STACK_VA(0x80007000). Noargc/argv/envp/ auxv.Entry ERETto the binarye_entry. No interpreter.CRT libctoswrappers +_start. Does not read auxv. CallsSVC #<n>(ADR-021).Image source FAT /hello(A9 / ADR-032). Notexecve.No Linux ELF feature in the gap table below is
present.PT_LOADof a freestandingET_EXECis partial (related map, different contract). -
Status words (same idea as B2).
Status Means present Same Linux ELF/auxv/ PT_INTERPmeaning, so a musl/glibc AArch64 binary could start with no translation.partial Track A has a related PT_LOAD/ stack / entry behavior. Type, stack, or control flow still differ.absent No analog. B6 may still discuss a named subset. Not a promise to implement. stay-rejected A3 fail-closed reject. This ADR does not flip it. Reopen only after B6 names a path, in a later epic. never-per-ADR-031 Would imply a Linux distro / glibc-musl product / guest interpreter that ADR-031 already ruled out as a Track B promise. -
Gap table.
Image type and program headers
Linux / ELF item ctos status Notes ELF64 LE AArch64 magic partial Same header walk. Not a Linux ABI. ET_EXECPT_LOADmap + BSS zeropartial A3 does this inside the 2 MiB window, 8 segments / 8 pages. ET_DYN/ PIE / loader-chosen baseabsent parse_elf64returnsNotExec. Linux PIE isET_DYN.PT_INTERPstay-rejected A3 tests require Interp. ADR-031: do not accept it in this child.Load the interpreter ELF absent No second image, no AT_BASE.Transfer to interpreter entry absent ERETis always binarye_entry.PT_DYNAMIC/DT_NEEDEDabsent Ignored (not PT_LOAD). No shared-object search.Relocations ( DT_RELA/REL/RELR/ IFUNC)absent Freestanding image is already linked. PT_TLS/ TPIDR_EL0absent PT_GNU_STACKexec-stackabsent Ignored. Stack is NX. A Linux exec-stack request would fight W^X. PT_GNU_RELROabsent PT_PHDRusable at runtimeabsent No AT_PHDR. Headers are not guaranteed mapped at a libc-known VA.Stack and auxv (Linux v6.10
auxvec.h)Item ctos status Notes argc/argv[]/envp[]absent Stack page is zeros. libctosdoes not readSP.AT_NULLterminator + key/value pairsabsent AT_PHDR/AT_PHENT/AT_PHNUMabsent musl/glibc ldso needs these. AT_PAGESZabsent Guest pages are 4 KiB; nothing publishes that to EL0. AT_BASEabsent Interpreter load base. AT_ENTRYabsent Real program entry after ldso init. AT_RANDOM(16 bytes)absent Typical stack canary / SSP. No getrandomeither (B2).AT_UID/AT_EUID/AT_GID/AT_EGID/AT_SECUREabsent No uid model. AT_HWCAP/AT_HWCAP2/AT_PLATFORMabsent AT_EXECFNabsent A9 path is FAT /hello, not a Linux pathname.AT_SYSINFO_EHDR(vDSO)absent No vDSO mapping. AT_MINSIGSTKSZabsent No POSIX signals (B2). musl-linked vs freestanding (the #44 question)
Workload What it still needs on ctos Honesty In-tree libctosET_EXECAlready the A3/A9 path. Keep it. Not Linux. Static musl ( -static, often noPT_INTERP)Linux stack + auxv; Linux svc #0/x8(exit_group,set_tid_address,brk/mmap,writeon fd 1 — B2); oftenET_DYNstatic-PIE; VAs usually not in0x8000_0000…+2 MiB; image larger than 8 pages.Not a small subset. ADR-031 rejected “runs musl” as a promise. Dynamic musl ( PT_INTERP→ld-musl-aarch64.so.1)Everything static needs, plus accept PT_INTERP, load the interpreter,AT_BASE/AT_PHDR/AT_ENTRY,PT_DYNAMIC+DT_NEEDEDfiles on a real FS, relocations, TLS.stay-rejected + absent. Do not claim dynamic Linux ELF. glibc dynamic Same class as dynamic musl, usually heavier ( ld-linux, vDSO, more CRT syscalls).Same non-claim. -
Keep the Track A loader reusable. A2 flatten, A3 map+
ERET, A4 standing task, and A9 FAT/helloall callparse_elf64/run_image. This ADR does not widen that function. A later Linux ELF path — only if B6 / #46 chooses a compat layer, and only with a probe — would be a new module that may reuse page-map helpers. It must not silently retargetparse_elf64to acceptPT_INTERPorET_DYN. Track A tests that rejectInterpstay. -
Not claiming dynamic Linux ELF support. Say that sentence on the Track B page. Today:
- A3 maps freestanding ELF64
ET_EXECPT_LOADand rejectsPT_INTERP. - The stack has no auxv.
ERETis not interpreter start.- A9 FAT
/hellois notexecve. - The ledger row “Guest runs host apps (Linux ELF / shell / Python)” stays Planned.
- A3 maps freestanding ELF64
-
If a subset is ever useful (not a decision). Research hint for B6, not a backlog:
- In-tree
libctosapps need no Linux ELF features. Keep ADR-023. - Static musl is already a libc-shaped surface (B2) plus auxv/stack/
ET_DYN/VA-window work. That is not a B4 implementation item. - Dynamic musl/glibc starts with accepting
PT_INTERP. This track’s frame forbids that in B4. B6 may choose never. - Do not start with relocations, TLS, or a guest
ld.so.
- In-tree
-
Non-goals. Full Linux ELF ABI; glibc/musl ports; accepting
PT_INTERPinsrc/; rewriting Track A probes; guest containers (ADR-029); new FR/NFR IDs; author self-merge (ADR-002).
Consequences
- Docs: this ADR. track-b.md B4 becomes Documented. SUMMARY.md and roadmap.md link here. Site extras point at the gap without a userspace claim.
- B3 / B5 / B6 stay Planned research. No
src/change in this PR. - Track A A3 remains the product loader for in-tree freestanding apps.
PT_INTERPstays rejected. - Honesty: document inspection only. Do not add a Verified Linux-ELF or dynamic-linker ledger row because this file exists.
ADR-034 — Linux VFS concepts vs thin ctos VFS
- Status: Accepted (docs compare; no Linux FS claim)
- Date: 2026-09-12
- Number: ADR-034 is frozen for this mile. Main holds ADR-033 (B4 ELF / auxv /
PT_INTERP, #68 /774c672) and ADR-035 (B3 process model, #67 /d81a53a). Do not yield this number again.
Context
Track B (issue #40) is Linux-compat research. Child B5 / #45 asks how ctos VFS / memfs / FAT stays thin while optionally mapping to Linux-like semantics later.
ADR-031 already framed Linux-compat as ABI-subset research and told this child: compare Linux VFS concepts vs the thin ctos VFS; do not grow POSIX open flags / dentries. B2 already mapped the file-descriptor slice: Linux AArch64 openat / read / write / close are partial against ctos fs_create / fs_open / fs_read / fs_write / fs_close (SVC 19–23). Linux mount / umount2 / pivot_root stay never-per-ADR-031.
Track A first cuts stay freestanding:
- Thin VFS + in-RAM memfs (ADR-027):
VfsOpsis create / open / read / write / close. Flat path/+ 1–15 of[a-z0-9_-]. At most 8 files, 256 bytes, 4 handles. - virtio-blk + FAT16 behind the same
vfs::open(ADR-028); write depth ADR-050. Known files/probeand A9/hello; guest may create/fwr. Serialfat: mountis a probe marker, notmount(2).
This ADR does not implement B3, B4, or B6. It does not mint FR-16+ or NFR-15+. It does not add POSIX flags, dentries, cwd, or a second open story to src/. File presence of this page is not a Linux filesystem.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Compare Linux VFS objects (dentry / inode / path / mount) to today’s thin VFS. Stay thin. Document gaps. Optional later mapping is translation after B6, not a dentry cache this mile. | Matches #45 / ADR-031. Leaves B6 room to choose never. |
| H2 (rejected) | Grow POSIX open flags, dentries, cwd, or O_CREAT in this PR. | ADR-031 forbids it. Inflates Track A probes and invites a POSIX claim. |
| H3 (rejected) | Say “ctos has Linux VFS” because memfs + FAT16 exist. | Honesty fail (NFR-06). Serial fs: ok / fat: ok are thin-VFS probes, not Linux VFS. |
| H4 (rejected) | Add mount(2), overlay, or a mount namespace as Planned on Track B. | Conflicts with ADR-031 / ADR-029. A7 FAT attach is not a Linux mount tree. |
| H5 (rejected) | Decide compat layer vs reimplement vs never here. | That is B6 / #46. |
Decision
-
Stay thin.
VfsOpsremains five calls on a path + handle. Oneopenstory. memfs first; if the name is missing, FAT16 root (8.3) is tried. FAT fds are tagged so the router does not invent a second interface. Not a superblock. Not a dentry cache. Not POSIXopen. -
What Linux VFS is (concepts, not a port). Linux path lookup is a walk of cached dentries that bind a name to an inode, under a vfsmount, on a super_block. An open file is a separate file object (
f_pos,f_op). A path is the pair(mnt, dentry). Hard links share an inode across dentries../../ extra// relative names /AT_FDCWD/ symlinks are part of that walk. This paragraph is a teaching sketch of Linux, not a claim that ctos implements any of it. -
Concept map. Status words match B2: present means the Linux object exists here with comparable semantics; partial means a related ctos object exists but identity or walk still differ; absent means no analog; never-per-ADR-031 means a Linux mount / overlay / namespace class already ruled out.
No Linux VFS object is
present.Linux concept ctos today Status Why not presentsuper_block(FS instance,s_op, root)Two backends behind one router ( MemFs+Fat16)absent No superblock, no FS type registration, no s_root.inode(identity,i_ino,i_mode,i_op)memfs File(name +Vec<u8>) or FAT 8.3 dirent + clusterabsent No inode number, mode, nlink, or inode ops. Close does not destroy the memfs bytes; that is not an inode lifetime. dentry(cached name → inode,d_parent)Exact string match on a flat table absent No dentry cache, no parent, no hash walk, no RCU-walk. ADR-027 rejected a dentry cache. path(vfsmount+dentry)Canonical byte string /+[a-z0-9_-]partial A path exists as a name. It is not a (mnt, dentry)pair.file(open instance,f_pos,f_op)Handle 1..=4with a per-handle offsetpartial Related effect to an fd + offset. Four handles. No f_op, no fd 0/1/2, nodup. FAT handles use a tag bit, not a Linux fd table.vfsmount/ mount treeSerial fat: mountafter virtio + FAT attachnever-per-ADR-031 Not mount(2). No mount namespace, no overlay, noumount2/pivot_root. B2 already marked those never-per-ADR-031.Directory / getdents64Flat namespace; FAT root is a lookup table, not a listed dir absent No mkdirat, no cwd.stat/statx/ inode metadataNone absent B2: newfstatat/fstat/statxabsent.Symlink / hard link None absent One name per memfs slot; FAT 8.3 one dirent. POSIX path walk ( .,..,//, relative,AT_FDCWD)valid_pathrejects extra/,., and anything outside[a-z0-9_-]absent See path grammar below. -
POSIX path grammar vs ctos names.
Item Linux / POSIX ctos ( valid_path)Shape Absolute or relative; /separators; empty components collapseMust start with /; no extra/./..Current / parent Rejected Length PATH_MAX(typically 4096)2–16 bytes total ( /+ 1–15)Alphabet Almost any byte except NUL and /[a-z0-9_-]after the first/cwd / chdir/getcwdYes No. B2: chdir/getcwdabsentdirfd/AT_FDCWDopenatfirst argumentNo. fs_opentakes a user path pointer + lengthO_*/ modeopenatflags + modeNo. Create is a separate SVC ( fs_create19). Existing name isfs_open20.FAT 8.3 vfatlong names + short namesGuest path /probemaps toPROBE;/hellotoHELLO. Not VFAT. -
Backends are not Linux FS types. memfs is named heap buffers, not
tmpfs. FAT16 on virtio-blk is a host-built 4 MiB image with two known files, not Linuxvfat/msdos. Do not say “supports FAT” as a product. Say “the guest read a known FAT16 file through the thin VFS” only when the ledger hasfat: ok. -
Syscall slice stays on B2. This ADR does not retarget SVC 19–23 to Linux
x8. The B2 rows still stand:openatpartial,readpartial,writepartial (uart_writeorfs_write, neither iswrite(1, …)),closepartial,lseek/pread64/getdents64/ioctlabsent,mountnever-per-ADR-031. -
Optional later mapping (not a decision). If B6 ever names a compat path, the smallest file slice would still be convention translation of
openat/read/write/closeonto the existing thin VFS — not a Linux dentry cache, not POSIX flags, not cwd, notmount(2). Growing those objects needs a later epic + ADR after B6. Never remains valid. -
Not claiming a Linux filesystem. Say that sentence on the Track B page. Today:
- Thin VFS + memfs / read-only FAT16 is not Linux VFS.
fat: mountis notmount(2).- A9 FAT
/hellois not a Linux rootfs. - The ledger row “Guest runs host apps (Linux ELF / shell / Python)” stays Planned. This ADR does not change that row to Verified.
-
Keep (ctos specificity). Honesty (NFR-06): this file is not Verified Linux VFS. Antifragility keeps
fs:/blk:/fat:smoke strings. Security: no “Linux-compat isolation” from a compare. Performance: no invented VFS bench. Freestanding Track A path stays. Guest containers stay ADR-029.
Consequences
- Docs: track-b.md B5 becomes Documented. SUMMARY.md and roadmap.md link here. Site extra: filesystem.md / filesystem.md point at the compare without a Linux FS claim.
- B6 stays Planned research. B3 is Documented (ADR-035). B4 is Documented (ADR-033). No
src/change in this PR. - Track A A6/A7 probes stay the product path for named files. Do not grow
VfsOpsto look like Linux so a research ADR can saypresent. - Honesty: document inspection of this ADR +
src/vfs.rs/src/fat.rs+ the B2 gap table. Do not add a Verified Linux-filesystem ledger row because this file exists.
ADR-035 — Process model: Linux fork/exec/wait vs ctos standing EL0
- Status: Accepted (docs frame; no Linux process-model claim)
- Date: 2026-09-12
Context
Track B (issue #40) is Linux-compat research. Child B3 / #43 asks to compare the Linux process model (fork / exec / wait) to ctos standing EL0 and the EL1 cooperative scheduler, and to decide an approach without abandoning probes.
ADR-031 already frames Linux-compat as ABI-subset research. It is not claiming Linux userspace. Guest OCI / Docker / Kubernetes stays a non-goal (ADR-029). This ADR cites those decisions. It does not reopen containers, mint a process table, or add syscalls.
B2 mapped numbers only. Process and lifetime rows there are absent or partial:
| Linux AArch64 (v6.10) | ctos today |
|---|---|
No fork syscall — userspace fork() is clone (220) / clone3 (435) | absent |
execve (221) / execveat (281) | absent — A3 maps freestanding PT_LOAD and rejects PT_INTERP. A9 FAT /hello is an EL1 load path. |
wait4 (260) / waitid (95) | absent — no child to wait for |
exit (93) / exit_group (94) | partial — SYS_EXIT 16 ends the standing EL0 trip |
getpid / gettid / set_tid_address / kill | absent — no pid, tid, or signals |
clone + CLONE_NEW* | never-per-ADR-031 (namespaces; see ADR-029) |
Track A first cuts that this mile must not rewrite:
- Standing EL0 as normal mode for one loaded image (ADR-024).
- Freestanding ELF64
PT_LOADloader (ADR-023). - EL1 cooperative yield of two heap-backed workers (ADR-010).
- Tiny SVC ABI (
exit/uart_write/yield, then memfs 19–23) (ADR-021).
This ADR does not implement B4–B6. It does not add fork / clone / execve / wait* to src/. It does not mint FR-16+ or NFR-15+. File presence of this page is not a Linux process table.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Keep standing EL0 (one loaded trip) plus EL1 cooperative yield as the ctos process model. Document Linux fork / exec / wait as a gap. Do not add those syscalls. Do not rewrite A1–A9 probes. Any Linux-shaped process table is a later epic after B6 names a path, or never. | Matches #43 (“decide approach without abandoning probes”) and ADR-031. Leaves B4–B6 room. |
| H2 (rejected) | Promise Linux fork / execve / wait, a pid table, or “runs a shell.” | Honesty fail (NFR-06). No probe exists. |
| H3 (rejected) | Treat A4 standing EL0 or A9 FAT /hello as execve. | The loader is EL1-driven. It rejects PT_INTERP. SYS_EXIT returns to the EL1 caller, not a parent pid. |
| H4 (rejected) | Treat M9 EL1 coop workers as Linux threads / clone(CLONE_THREAD). | Those tasks are kernel EL1. SYS_YIELD does not call sched::yield_now. |
| H5 (rejected) | Add clone / wait4 as Planned implementation on this track, or reopen CLONE_NEW*. | Would turn a research compare into a backlog. Namespace flags stay never-per-ADR-031. B6 may still choose never. |
Decision
-
ctos process model stays standing EL0. One freestanding image is loaded by EL1 (A3 / A9), installed as a standing task (
install_task), andERETed. It runs at EL0 untilSYS_EXITor an unexpected lower-EL sync. Restore is fail-closed (el0: task-restored/el0: restore-fail).is_active()/is_task()are true only for that lifetime. There is no process table, pid, ppid, tid, or parent/child wait queue. -
EL1 cooperative yield stays a different machine. ADR-010 switches two heap-backed EL1 workers. It is not preemptive, not SMP, and not EL0.
SYS_YIELD18 is a dispatch-only hint that returns to EL0. Do not describe those workers as Linux threads. -
Linux
fork/exec/waitis a documented gap, not a present analog.Linux idea What it needs What ctos has fork/cloneChild pid, optional new mm, COW or share, parent continuesOne standing context. Shared user TTBR0 window. No child. execveReplace the current image; keep pid / fds / cwd; load interpreter via PT_INTERP/ auxvEL1 maps a freestanding ET_EXECandERETs. RejectsPT_INTERP. No fd table to keep. B4 is the ELF/auxv compare.wait/waitidParent blocks or polls; reaps a child status SYS_EXITreturns to the EL1 hello path that launched the trip. No waiter.exitvsexit_groupThread vs process-group teardown One trip ends. No thread group. musl/glibc CRT usually hits exit_group.Signals / killDelivery + waitidLower-EL IRQ still parks. No POSIX signals. On Linux AArch64, say
clone+execve+wait4, not aforksyscall. Userspacefork()is a libc wrapper. -
Goals (research stance only).
Goal Why Name the two models so later children cannot blur them Honesty (NFR-06). File presence is not execve.Keep A1–A9 serial markers and fail-closed smoke Antifragility (NFR-05). This mile must not retarget SVC 16–23 or drop el0: task-ok.Leave B6 free to choose compat layer vs reimplement vs never ADR-031. A process table is not pre-chosen. Keep CLONE_NEW*/ mount / guest OCI outADR-029 / ADR-031 never. -
Non-goals. Same family as ADR-031:
- Implementing
clone/clone3/execve/execveat/wait4/waitid/getpid/killinsrc/. - A pid / tid / process-group table.
- COW address spaces,
CLONE_VM/CLONE_THREAD/CLONE_FILES. - Treating A3 / A4 / A9 as Linux
exec. - Treating
SYS_EXITasexit_group. - Full Linux ABI / POSIX product / glibc or musl ports / “runs a shell.”
- Guest OCI / Docker / Kubernetes (ADR-029).
- Replacing the freestanding Track A path.
- B4 (ELF / auxv /
PT_INTERP) and B5 (Linux VFS concepts) — those stay their own children. - New FR/NFR IDs. Frozen set stays fr-nfr.md.
- Author self-merge (ADR-002).
- Implementing
-
Approach if B6 later names a path (not a decision). A hint only:
- In-tree
libctosapps need no Linux process model. Sequential standing trips (EL1 loads the next FAT image afterexit) would still be ctos-native, notfork/wait. - A static musl
_startis not a small subset. Typical CRT wantsset_tid_address,exit_group, and often a pid.clone/execve/wait4are larger than convention translation +writeon a console fd (B2’s smallest named slice). - If B6 chooses a compat path, a later epic would still need: a process table, a child
mm(copy or COW), a wait queue, and an EL0-driven replace-image path. That epic needs its own probes. It must not silently retarget today’sSYS_EXITor drop standing-task markers. - Never remains valid. Standing EL0 can stay the model indefinitely.
- In-tree
-
Keep (ctos specificity). Same invariants as ADR-031: honesty, antifragility, security, performance, document-first / one-PR loops, freestanding Track A path, QEMU
virt/ AArch64 (ADR-003), ADR-gated EL0 / identity teardown. Do not claim “EL0 isolated,” “secure OS,” or a bench for a process table that does not exist. -
Not claiming Linux userspace yet. Say that sentence on the Track B page. The ledger row “Guest runs host apps (Linux ELF / shell / Python)” stays Planned. This ADR does not change that row to Verified.
flowchart TD LNX["Linux: clone then execve then wait4"] CTOS["ctos: EL1 load + standing EL0 until exit"] LNX -->|"needs pid, child mm, wait queue"| GAP["Documented gap — not present"] CTOS -->|"A3 / A4 / A9 probes stay"| KEEP["Keep — do not rewrite"] GAP -.->|"B6 may choose never"| KEEP
Consequences
- Docs: this ADR. track-b.md B3 becomes Documented. SUMMARY.md and roadmap.md link here. linux-aarch64-syscall-gap.md points process rows at this file.
- B4 is Documented on
main(ADR-033). B5 / B6 stay Planned research. Nosrc/change in this PR. - Track A remains the product path for in-tree freestanding apps. Standing EL0 is not a Linux process. “App hosting is done” stays unclaimed.
- Honesty: document inspection only. Do not add a Verified Linux-process or
execveledger row because this file exists.
ADR-036 — Track B decision: compat layer vs reimplement vs never
- Status: Accepted (never — research ladder complete; no Linux-compat implementation track)
- Date: 2026-09-12
Context
Track B child B6 / #46 (parent #40) asks for an explicit decision after the research ladder:
- thin Linux-compat layer on top of ctos,
- reimplement Linux-like surfaces in ctos identity, or
- never (stop at research docs).
Frame and evidence already on main:
| Mile | Artifact | Honest readout |
|---|---|---|
| B1 | ADR-031 | ABI subset research only; not claiming Linux userspace; containers stay ADR-029 |
| B2 | linux-aarch64-syscall-gap.md | No Linux syscall is present (convention + number space miss); Track A SVCs are partial |
| B3 | ADR-035 | Standing EL0 ≠ clone/execve/wait4; do not add those syscalls |
| B4 | ADR-033 | Freestanding loader stays; PT_INTERP stay-rejected; dynamic musl/glibc not a small subset |
| B5 | ADR-034 | Thin VfsOps ≠ Linux VFS; mount/overlay never-per-ADR-031 |
| B7 | ADR-029 | Guest OCI/Docker already never |
Track A A1–A9 (+ leftovers ADR-032) is the probed freestanding product path. Core principles (principles.md) outrank Track B.
This mile is docs only. It does not change src/. File presence of this ADR is not a Linux userspace probe.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (rejected) | Thin Linux-compat layer (translate Linux AArch64 svc #0 / x8 onto ctos, grow auxv/PT_INTERP/process table behind a flag) | B2–B5 show the gap is wide (syscalls, process model, dynamic ELF, VFS). A layer would become a second ABI to honesty-ledger and fail-closed smoke. No named probed workload justified the cost. |
| H2 (rejected) | Reimplement Linux-like surfaces as native ctos (Linux numbers/names in src/, still “ctos”) | Same gap surface as H1, plus a silent retarget of Track A SVC #n / libctos / reject-PT_INTERP markers. Conflicts with ADR-031 “keep freestanding Track A.” |
| H3 (chosen) | Never — Track B stops as a completed research ladder. No Linux-compat implementation epic from these ADRs. | Matches honesty (NFR-06), antifragility (do not grow an unprobed ABI), and “principles drive.” Leaves the door open only via a new epic + ADR with a named workload and probe — not by treating B2–B5 as a backlog. |
Decision
-
Choose never. ctos will not start a Linux-compat layer or a Linux-like reimplementation from Track B B1–B5. Track B’s job was to map the gap honestly. That job is done.
-
What “never” means here.
- Do not treat the B2 syscall table, B3 process gap, B4 ELF/auxv gap, or B5 VFS compare as an implementation backlog.
- Do not accept
PT_INTERP, Linuxx8numbers,clone/execve/wait*, POSIX VFS, or guest containers “later on Track B.” - Do not change the ledger row “Guest runs host apps (Linux ELF / shell / Python)” from Planned to a softer promise. It stays a distant possible future only if a new epic says so with a probe.
- Track A freestanding apps (SVC /
libctos/PT_LOAD/ standing EL0 / thin VFS / FAT slots) remain the supported path.
-
What remains allowed.
- Keep and cite B1–B5 / B7 docs as the honesty record of why Linux userspace is out.
- Learn from Linux ideas in future freestanding miles only when they fit ctos pillars and get their own ADR + probe (not “compat”).
- Reopen Linux-compat only with a new GitHub epic, a new ADR that names a concrete workload + probe, and an explicit override of this decision. Do not mint FR/NFR IDs here.
-
Relationship to ADR-029 / ADR-031. Containers stay a non-goal. ADR-031’s frame (“B6 may choose never”) is exercised by this ADR. Principles still drive over tracks.
Consequences
- track-b.md marks B6 Decided (never) and the research ladder complete.
- Plan issue #40 may close once this ADR merges (children B1–B7 documented/decided).
- No
src/change. No new FR/NFR IDs. No self-merge (ADR-002). - Future “runs musl/Alpine/Linux ELF” headlines without a new epic + probe are honesty failures.
Probe (what Verified means here)
| Claim | Probe | Status |
|---|---|---|
| B1–B5 / B7 evidence cited above exists on this tip | Read ADR-029/031/033/034/035 + B2 gap note + track-b on main | Document inspection |
| This ADR records an explicit never decision | Read this file’s Decision section | Document inspection |
| Linux userspace / compat layer on the guest | No serial marker; not claimed | Not claimed |
ADR-037 — Identity .data / .bss / linker-stack tear after SP relocate
- Status: Accepted (identity
.data/.bss/linker stacks unmapped; heap still identity-mapped) - Date: 2026-09-12
Context
Track A / A5 continues identity teardown where safe. ADR-025 unmapped identity .rodata after a pointer rewrite. .data / .bss / linker stacks / heap stayed mapped: SP was still the identity __stack_top, page-table software walked tables via identity PAs that live in .bss, and GlobalAlloc still returned identity frame VAs.
After the ADR-019 high-VA jump, PC-relative accesses (ADRP) already reach the TTBR1 alias of statics. Absolute words that point into .data/stacks stay identity until rewritten. SP and SP_EL1 must move to the high twin of the same physical pages before those identity pages can be unmapped. Heap tear needs a high allocator and is not this cut.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Rewrite identity pointers into .data/stacks, relocate SP/SP_EL1 to high twins, switch page-table CPU accessors to the high twin, then unmap identity [__data_start, __kernel_end) from kernel+user TTBR0. High twins stay. Heap stays. | Largest honest cut that does not change GlobalAlloc. |
| H2 (rejected) | Also unmap identity heap. | Allocator still returns identity VAs. A silent yank would fault. |
| H3 (rejected) | Claim PAN / “EL0 isolated” / “kernel moved” while tearing .data. | Isolation is not this mile. PAN is ADR-026. _start stays at 0x4008_0000. |
| H4 (rejected) | Unmap .data without relocating SP or fixing PA-as-pointer table walks. | Would fault on the next store or l3_write. |
Decision
- Rewrite identity
.datapointers first. After the live.texttear and before the.rodatatear (so identity.rodatais still walkable), walk.rodataand[__data_start, __kernel_end). Every 8-byte-aligned word that is an identity address in that data/stack range is rewritten to its TTBR1 alias. Valid page-table descriptors are excluded (low bits set). Serialident: data-reloc n=N.N == 0is allowed. - Keep the ADR-025
.rodatacut.ident: ro-reloc/ident: rodataunchanged. - Relocate SP. Move the current SP (SP_EL0) and SP_EL1 to
to_high_vaof the same physical stack pages. High guard holes already exist inL2_HIGH_RAM(cloned after identity guards). Serial path continues only if SP is high. - Page-table CPU VA. After the high split,
desc_at/l3_write/ flag publish-load use the TTBR1 twin of table/flag PAs so identity.bsscan be torn while the hardware walk still uses PAs. - Unmap identity
.data/.bss/linker stacks. Unmap[__data_start, page_align_up(__kernel_end))from kernel and user TTBR0 (TLBI VAAE1each low VA). High twins stay. Serialident: data lo=… hi=… pages=NwithN >= 1. - Keep heap identity-mapped. Frames at/after
__kernel_endstay. Printident: heap-stay. Do not claim a high allocator. - Keep
_start/ QEMU-kernelat0x4008_0000. Do not change default-cpu. Do not claim PAN. - Fail-closed probe. New
ident: data-reloc/ident: data/ident: data-fault/ident: data-high. EL1LDRof a torn identity.dataVA is a current-EL translation DABORT. EL1LDRof the high twin still returns a known.datamagic.scripts/qemu-smoke.shgreps those strings and rejects missed markers. Replaceident: data-stay. Split#[test_case]into heap-still-mapped vs data-torn. - Still Planned (at accept time). Identity heap tear (high allocator VAs). Superseded by ADR-038. PAN enable on
-cpu cortex-a57; EL0 entry withoutTLBI VMALLE1; lower-EL IRQ while standing; umbrella EL0 isolation. - NFR-10 text is revised in place (ID unchanged). Do not mint NFR-15+.
Honesty
Say “identity .data/.bss/linker stacks were unmapped while the high twin and heap stayed” only when the serial / tests pass. Do not say:
- the kernel has moved to the high half
- identity mappings were fully torn down (heap stays;
_startstays) - “secure OS” / “hardened” / “EL0 isolated”
- PAN (see ADR-026; unimplemented on
-cpu cortex-a57)
Consequences
paging::rewrite_identity_data_ptrs/paging::relocate_stacks_high/paging::tear_identity_dataown the cut. ADR-025 remains the.rodatatear. ADR-032 stay markers for.dataare superseded by this tear; heap stay markers remain.- ADR-038 returns high heap VAs and unmaps identity heap. That work is not this cut.
ADR-038 — Identity heap tear after high GlobalAlloc VAs
- Status: Accepted (identity heap unmapped;
GlobalAllocreturns TTBR1 VAs) when the serial / tests pass - Date: 2026-09-12
Context
Track A / A5 continues identity teardown where safe. ADR-037 unmapped identity .data/.bss/linker stacks after SP relocate. Heap stayed identity-mapped: frame::alloc_contiguous handed out PAs and GlobalAlloc returned those identity VAs (ident: heap-stay).
Heap init already runs after the high-VA jump and the .data tear. High twins of the frame pool exist (L2_HIGH_RAM clone). The next honest cut is to install the first-fit pool at the TTBR1 alias, rewrite leftover identity heap pointers, then unmap the identity heap range. PAN enable and umbrella EL0 isolation are not this cut.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Install the heap free list at to_high_va(pa), rewrite leftover identity heap words, then unmap identity [heap_pa, heap_pa+HEAP_SIZE) from kernel+user TTBR0. High twins stay. Remaining frames after the heap stay identity-mapped. | Smallest honest tear that does not yank live identity VAs. |
| H2 (rejected) | Unmap identity heap while GlobalAlloc still returns identity VAs. | Silent yank would fault. |
| H3 (rejected) | Claim PAN / “EL0 isolated” / “kernel moved” while tearing the heap. | Isolation is not this mile. PAN is ADR-026. _start stays at 0x4008_0000. |
| H4 (rejected) | Remap only new allocations high and leave the existing identity pool mapped. | Would print a stay marker forever. |
Decision
- High allocator first. After
frame::init,heap::inittakes the same 16-frame run and installs the free list at the TTBR1 alias.GlobalAllocreturns high VAs. Serial later printsident: heap-reloc n=N.N == 0is allowed (init already high). - Rewrite leftover identity heap pointers. Walk the heap pool via the high twin (do not walk live stacks or page-table
.bss). Every 8-byte-aligned word that is an identity address in the heap PA range is rewritten to its TTBR1 alias.heap::initalready installed high VAs, son == 0is the expected path. A leakedBoxplantsidentmagic for the high-twin load. - Unmap identity heap. Unmap
[heap_pa, heap_pa+HEAP_SIZE)from kernel TTBR0 (TLBI VAAE1each low VA). User TTBR0 already omits heap; clearing a present slot is fine. High twins stay. Frames after the heap stay identity-mapped. Serialident: heap lo=… hi=… pages=NwithN >= 1. - Keep
_start/ QEMU-kernelat0x4008_0000. Do not change default-cpu. Do not claim PAN. - Fail-closed probe. New
ident: heap-reloc/ident: heap/ident: heap-fault/ident: heap-high. EL1LDRof a torn identity heap VA is a current-EL translation DABORT. EL1LDRof the high twin still returns the planted free-list header size.scripts/qemu-smoke.shgreps those strings and rejectsident: heap-stay. Replace the ADR-032 / ADR-037 stay marker. - Still Planned (at accept time). PAN enable on
-cpu cortex-a57; EL0 entry withoutTLBI VMALLE1; lower-EL IRQ while standing; umbrella EL0 isolation. Remaining identity RAM after the heap (frame bump pool) is not this cut. EL0 entry withoutVMALLE1is superseded by ADR-039. - NFR-10 text is revised in place (ID unchanged). Threat-model v1.18 at accept (v1.19 after ADR-039). Do not mint NFR-15+.
Honesty
Say “identity heap was unmapped while the high twin stayed and GlobalAlloc returned TTBR1 VAs” only when the serial / tests pass. Do not say:
- the kernel has moved to the high half
- identity mappings were fully torn down (
_startstays; frame pool after the heap stays identity-mapped) - “secure OS” / “hardened” / “EL0 isolated”
- PAN (see ADR-026; unimplemented on
-cpu cortex-a57)
Consequences
heap::init/paging::rewrite_identity_heap_ptrs/paging::tear_identity_heapown the cut. ADR-009 stays the first-fit algorithm; VAs are now high. ADR-037 stay marker for the heap is superseded.- A later ADR may tear remaining identity frames or enable PAN. That work is not this cut.
ADR-039 — EL0 entry / ASID switch without TLBI VMALLE1
- Status: Accepted (standing/EL0 trampoline path is
MSR TTBR0+ISBonly) - Date: 2026-09-12
Context
Track A / A5 isolation leftovers. Dual-ASID EL1 switches already avoided TLBI VMALLE1 (ADR-013 P-SEC-3c). The standing/EL0 trampoline still flushed the whole EL1 TLB on every ERET to EL0, every stay-at-EL0 after a standing SVC, and every lower-EL sync restore of kernel TTBR0 — because kernel .data leaves were global (nG=0). Dropping that flush while identity .data stayed mapped would leak a cached translation into the user ASID and break el0: no kernel read.
ADR-037 and ADR-038 unmap identity .data/.bss/stacks and the identity heap (TLBI VAAE1 each low VA). EL0 probes run after those tears. The historical global-.data reason for trampoline VMALLE1 is gone for those ranges. PAN enable and umbrella EL0 isolation are not this cut.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Remove TLBI VMALLE1 from eret_to_el0, stay_at_el0, and sync_lower_el TTBR0 restore. Keep MSR TTBR0 + ISB. Keep TLBI VMALLE1 only at MMU enable (paging::init) and keep TLBI VAAE1 on unmap/setup. | Smallest honest mile after ADR-037/038. |
| H2 (rejected) | Mark all kernel leaves nG and keep a full flush “just in case.” | Larger surface; not required once identity .data/heap are torn + VAAE1’d. |
| H3 (rejected) | Claim “EL0 isolated” / PAN / “kernel moved” because the trampoline no longer full-flushes. | Isolation umbrella stays Planned. PAN is ADR-026. |
| H4 (fallback) | If H1 regresses el0: no kernel read / standing / task smoke, ship a probe documenting where VMALLE1 still fires and why — do not regress smoke. | Honesty over the mile. |
Decision
- Trampoline path without
VMALLE1.exception::eret_to_el0,stay_at_el0, andsync_lower_el(kernel TTBR0 restore fromTPIDR_EL1) useMSR TTBR0_EL1+ISBonly. - What TLBI remains.
paging::initstillTLBI VMALLE1once at MMU enable. Unmaps / map-window / identity tears stillTLBI VAAE1(or equivalent per-VA invalidate). ASID dual probe stays no-full-flush (ADR-013). - Fail-closed probe. Serial
el0: no-vmalle1after the existing EL0 first-mile + read + standing markers, only when identity.dataand heap tears are ready.scripts/qemu-smoke.shgreps that string and rejectstlbi vmalle1insrc/exception.rs.#[test_case]el0_entry_without_vmalle1. - Keep
_start/ QEMU-kernelat0x4008_0000. Do not change default-cpu. Do not claim PAN. - Still Planned. PAN enable on
-cpu cortex-a57; remaining identity RAM /_startteardown; umbrella EL0 isolation. Lower-EL IRQ while standing is ADR-040. - NFR-10 text is revised in place (ID unchanged). Threat-model v1.19. Do not mint NFR-15+.
Honesty
Say “standing/EL0 trampoline switches TTBR0 without TLBI VMALLE1 after identity .data/heap tear” only when the serial / tests pass. Do not say:
- “EL0 isolated”
- PAN (see ADR-026; unimplemented on
-cpu cortex-a57) - the kernel has moved / identity mappings were fully torn down (
_startand leftover identity frames stay) - “secure OS” / “hardened”
Consequences
src/exception.rsowns the trampoline TLBI drop. ADR-013 remains the isolation direction; this ADR is the EL0-entry flush cut.- A later ADR may enable PAN or tear remaining identity RAM. That work is not this cut.
ADR-040 — Lower-EL IRQ while standing EL0
- Status: Accepted (taken lower-EL IRQ while standing is a supported/probed path when the serial / tests pass; FIQ/SError lower-EL still park; umbrella isolation Planned)
- Date: 2026-09-13
Context
Track A / A5 isolation leftovers. Lower-EL AArch64 sync has long been live for SVC / IABORT / DABORT. The lower-EL IRQ / FIQ / SError vector slots historically parked (ADR-004). Standing EL0 (ADR-013 / ADR-024) entered with DAIF IRQ masked in SPSR, so a timer IRQ from EL0 was never taken.
After ADR-039, the standing/EL0 trampoline restores kernel TTBR0 with MSR TTBR0 + ISB only. The same restore is required on a lower-EL IRQ path before GIC / .data access (user TTBR0 omits MMIO and kernel data).
PAN enable and umbrella EL0 isolation are not this cut. Do not switch default -cpu cortex-a57. Keep _start at 0x4008_0000.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Wire lower-EL IRQ (irq_lower_el): restore kernel TTBR0, gic::handle_irq, if el0::is_active() put user TTBR0 back and ERET to EL0. Probe: standing dual-SVC with WFI and SPSR IRQ-unmasked; serial el0: irq. Default ERET to EL0 stays IRQ-masked. FIQ/SError stay parked. | Largest honest first cut that does not race existing EL0 probes. |
| H2 (rejected) | Unmask IRQ on every standing/task ERET and claim “IRQ while standing is normal for all tasks.” | Larger surface; timer mid-task may interact with A4/A1–A3 probes before this mile is proven. |
| H3 (rejected) | Claim “EL0 isolated” / PAN / “secure OS” because IRQ returns to EL0. | Isolation umbrella stays Planned. |
| H4 (fallback) | If H1 regresses smoke, ship a scaffold documenting the park and why — do not regress smoke. | Honesty over the mile. |
Decision
- Lower-EL IRQ live while standing. Vector slot
0x480callsirq_lower_el(same frame + TTBR0 restore assync_lower_el).handle_irq_lower_elrunsgic::handle_irq, thenstay_at_el0whenis_active(). Non-standing lower-EL IRQ still fails closed (park). - Probe. Hello path: arm standing + expect IRQ,
timer::arm_soon,ERETwith SPSR0x340(EL0t, I clear), payloadSVC #1/WFI/MOVZ/SVC #2. Serialel0: irqwhen the IRQ was taken while standing. Existingel0: standing/el0: restoredstill print on that trip. - Default ERET stays masked.
eret_to_el0keeps SPSR0x3c0. Only the IRQ probe useseret_to_el0_irq_enabled. Enabling IRQ for all standing tasks is a later cut. - Fail-closed sensors.
scripts/qemu-smoke.shgrepsel0: irq.#[test_case]lower_el_irq_while_standing. - Still Planned. Lower-EL FIQ/SError; IRQ-unmasked default standing/task
ERET; PAN enable on-cpu cortex-a57; remaining identity RAM /_startteardown; umbrella EL0 isolation. - NFR-10 text is revised in place (ID unchanged). Threat-model v1.20. Do not mint NFR-15+.
Honesty
Say “taken lower-EL IRQ while standing returned to EL0” only when the serial / tests pass. Do not say:
- “EL0 isolated”
- PAN (see ADR-026)
- lower-EL FIQ/SError are handled
- every standing task takes IRQs by default (default
ERETstill masks I) - “secure OS” / “hardened”
Consequences
src/exception.rsownsirq_lower_el/handle_irq_lower_el.src/el0.rsowns theWFIstanding probe.src/timer.rsownsarm_soon.- ADR-013 remains the isolation direction; this ADR is the lower-EL IRQ-while-standing cut.
- A later ADR may unmask IRQ for standing tasks by default, or take FIQ/SError. That work is not this cut.
ADR-041 — IRQ-unmasked default standing/task ERET
- Status: Accepted (default
ERETto EL0 clears SPSR.I when the serial / tests pass; short non-standing trampoline probes stay masked; FIQ/SError lower-EL still park; umbrella isolation Planned) - Date: 2026-09-13
Context
ADR-040 made taken lower-EL IRQ while standing a live path, but only via a dedicated probe SPSR (eret_to_el0_irq_enabled, I clear). Default eret_to_el0 still used SPSR 0x3c0 (I set), so standing/task entry remained IRQ-masked unless that special trampoline was used.
This cut makes standing EL0 and the loaded-task / standing-payload paths enter with IRQ unmasked by default, and proves a timer tick can arrive without the special probe helper. PAN enable and umbrella EL0 isolation are not this cut. Do not switch default -cpu cortex-a57. Keep _start at 0x4008_0000.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Default eret_to_el0 uses SPSR 0x340 (EL0t, I clear). Short non-standing trampoline probes (SVC / IABORT / DABORT without is_active()) use eret_to_el0_masked. Standing IRQ probe uses default eret_to_el0 + WFI + arm_soon (no special SPSR helper). Serial el0: irq + el0: irq-default. Task/loader/syscall/libctos standing paths inherit the default. FIQ/SError stay parked. | Largest honest cut that matches the deferred ADR-040 item. |
| H2 (rejected) | Unmask every ERET including short non-standing probes and claim “IRQ mid-SVC is always fine.” | Non-standing lower-EL IRQ still parks (handle_irq_lower_el); a mid-probe tick would fail-closed. |
| H3 (rejected) | Claim “EL0 isolated” / PAN / “secure OS” because default SPSR clears I. | Isolation umbrella stays Planned. |
| H4 (fallback) | If H1 regresses smoke, keep default masked and document the park — do not regress smoke. | Honesty over the mile. |
Decision
- Default
ERETIRQ-unmasked.eret_to_el0writes SPSR0x340.eret_to_el0_irq_enabledbecomes an alias (or is removed).eret_to_el0_maskedkeeps SPSR0x3c0for short non-standing probes. - Probe. Hello path: same standing dual-SVC +
WFIas ADR-040, but enter via defaulteret_to_el0. Serial stillel0: irq; also printel0: irq-defaultafter that trip succeeds. Existingel0: standing/el0: restoredstill print. - Task path. Loaded standing task / A1–A3 standing payloads use
eret_to_el0and therefore inherit I-clear. No separate task WFI probe required for this cut; IRQ whileis_active()already returns viastay_at_el0. - Fail-closed sensors.
scripts/qemu-smoke.shgrepsel0: irq-default.#[test_case]lower_el_irq_default_eret(or extendlower_el_irq_while_standingto assert default path). - Still Planned. Lower-EL FIQ/SError; PAN enable on
-cpu cortex-a57; remaining identity RAM /_startteardown; umbrella EL0 isolation. - NFR-10 text is revised in place (ID unchanged). Threat-model v1.21. Do not mint NFR-15+.
Honesty
Say “default standing/task ERET enters with IRQ unmasked” only when the serial / tests pass. Do not say:
- “EL0 isolated”
- PAN (see ADR-026)
- lower-EL FIQ/SError are handled
- short non-standing trampoline probes take IRQs (those stay masked)
- “secure OS” / “hardened”
Consequences
src/exception.rsowns SPSR defaults.src/el0.rsowns the default-path IRQ probe. Short probes inel0/ttbr1/teardowncall the masked helper.- ADR-013 remains the isolation direction; this ADR closes the ADR-040 “default ERET still masks I” park.
- A later ADR may take FIQ/SError or enable PAN. That work is not this cut.
ADR-042 — Isolation leftover honesty wrap (start-stay + status)
- Status: Accepted (serial / tests prove
_startstays while live image + heap are torn; umbrella isolation stays Planned) - Date: 2026-09-13
- Follow-on (2026-09-26):
ident: ram-staywas superseded by ADR-049. The remaining identity (low RAM, padding tail, pre-heap frame, MMIO) was torn by ADR-087 / ADR-088.ident: start-stayremains, and the stub page is the one documented identity exception (ADR-089, sponsor D2).
Context
Track A / A5 closed a deep isolation ladder: identity .data (ADR-037), heap (ADR-038), EL0 trampoline without TLBI VMALLE1 (ADR-039), lower-EL IRQ while standing (ADR-040), and IRQ-unmasked default standing/task ERET (ADR-041). Former stay markers (ident: data-stay / ident: heap-stay) were superseded by those tears.
What remains Planned must stay fail-closed in the ledger and threat model: PAN enable on -cpu cortex-a57, lower-EL FIQ/SError, full identity teardown including _start, and the umbrella “EL0 isolated” row. QEMU -kernel still needs _start at 0x4008_0000 — do not yank it. This ADR is an honesty wrap of those leftovers, not a claim that isolation closed.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Print ident: start-stay (and ident: ram-stay for leftover identity frames after the heap) + #[test_case] that 0x4008_0000 stays mapped while live .text / .rodata / .data / heap are torn. Document Verified ladder ADR-037…041 vs still-Planned leftovers in this ADR; bump threat-model; keep P-SEC-3l Planned. | Honest wrap. Do not switch -cpu. Do not claim “EL0 isolated.” |
| H2 (rejected) | Unmap _start / claim “kernel moved” / claim umbrella isolation because the ladder exists. | Would break -kernel and round up. |
| H3 (rejected) | Switch default -cpu to enable PAN and call isolation done. | Forbidden by ADR-026 / roadmap. |
| H4 (deferred) | Take lower-EL FIQ/SError while standing. | Not cheap on this wrap; leave Planned. |
Decision
- Start-stay marker. After the ADR-038 heap tear proves live image ranges are unmapped, assert identity
_start(KERNEL_TEXT/0x4008_0000) is still mapped+executable and printident: start-stay lo=… hi=…. - Remaining-identity-RAM note. Print
ident: ram-stay lo=… hi=…for the still-mapped identity frame bump after the torn heap (heap_pa_end…pool_end). - Fail-closed probe.
scripts/qemu-smoke.shgrepsident: start-stayandident: ram-stay.#[test_case]identity_boot_stub_stays_while_live_torn. - Status table (this wrap).
| Item | Status | Notes |
|---|---|---|
ADR-037 identity .data/stacks tear | Verified | ident: data* |
| ADR-038 identity heap tear | Verified | ident: heap* |
ADR-039 EL0 entry without VMALLE1 | Verified | el0: no-vmalle1 |
| ADR-040 lower-EL IRQ while standing | Verified | el0: irq |
| ADR-041 IRQ-unmasked default ERET | Verified | el0: irq-default |
_start stay at 0x4008_0000 | Verified (honesty) | ident: start-stay; not a teardown |
| Remaining identity RAM after heap | Verified (honesty) | ident: ram-stay; tear still Planned |
PAN enable on -cpu cortex-a57 | Planned | pan: absent (ADR-026). Do not switch -cpu. |
| Lower-EL FIQ/SError while standing | Superseded by ADR-043 | FIQ Verified / SError park honesty on ADR-043. |
Full identity teardown including _start | Planned | Do not yank _start for QEMU -kernel. |
| Umbrella “EL0 isolated” (P-SEC-3l) | Planned | Ladder miles are not this row. |
- Docs. Update el0.md, security.md (threat-model v1.22), honesty ledger, roadmap P-SEC-3l, NFR-10 text in place. Do not mint NFR-15+.
- Keep
_startat0x4008_0000. Do not change default-cpu. Do not claim PAN or “EL0 isolated.”
Honesty
Say “_start / boot-stub page stays mapped while live .text/.rodata/.data/heap are torn” only when the serial / tests pass. Do not say:
- “EL0 isolated”
- the kernel has moved to the high half
- identity mappings were fully torn down
- PAN enable / lower-EL FIQ/SError are handled
- “secure OS” / “hardened”
Consequences
src/teardown.rsprints the stay markers; smoke and the new#[test_case]ratchet them.- ADR-013 remains the isolation direction; P-SEC-3l stays Planned.
- ADR-043 takes FIQ and documents SError park honesty. ADR-047 decides: never yank
_startwhile-kernelneeds it; PAN enable non-goal on default a57; taken SError deferred/non-goal on this smoke machine; leftover RAM optional; umbrella stays Planned/non-claim. A later ADR may reopen with sponsor scope (honest inject, new CPU, boot redesign).
ADR-043 — Lower-EL FIQ while standing (+ SError park honesty)
- Status: Accepted (taken lower-EL FIQ while standing is a supported/probed path when the serial / tests pass; SError has no safe trigger on this virt/cortex-a57 cut — park honesty; umbrella isolation Planned)
- Date: 2026-09-13
Context
ADR-040 / ADR-041 made taken lower-EL IRQ while standing a live path. Lower-EL FIQ and SError still parked (ADR-004 / ADR-042 H4).
On QEMU virt + GICv2 + -cpu cortex-a57, Group 0 interrupts signal as IRQ by default (GICC_CTLR.FIQEn clear). Setting FIQEn routes Group 0 (including the CNTP PPI) as FIQ without changing the default -cpu. True asynchronous SError has no safe, deterministic inject on this guest without exotic QEMU/monitor setup — do not fake a taken path.
PAN enable and umbrella EL0 isolation are not this cut. Do not switch default -cpu cortex-a57. Keep _start at 0x4008_0000.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Wire fiq_lower_el (same TTBR0 restore as IRQ). Probe: temporarily GICC_CTLR.FIQEn, standing dual-SVC + WFI, SPSR 0x380 (F clear, I set), serial el0: fiq; restore Group-0 → IRQ after the trip. SError: wire a defensive standing-aware stub + print el0: serror-park (Planned / no safe trigger). | Largest honest first cut mirroring IRQ. |
| H2 (rejected) | Claim both FIQ and SError are Verified taken paths without a safe SError inject. | Would round up. |
| H3 (rejected) | Switch -cpu / enable PAN / claim “EL0 isolated” because FIQ returns to EL0. | Forbidden. |
| H4 (fallback) | If FIQEn does not deliver a lower-EL FIQ on this virt guest, ship scaffold + park markers and document why — do not fake Verified. | Honesty over the mile. |
Decision
- Lower-EL FIQ live while standing. Vector slot
0x500callsfiq_lower_el.handle_fiq_lower_elrunsgic::handle_fiq(same IAR/EOI as IRQ for Group 0), thenstay_at_el0whenis_active(). Non-standing lower-EL FIQ still fails closed (park). - Probe. Hello path:
gic::route_group0_as_fiq,timer::arm_soon,eret_to_el0_fiq(SPSR0x380), payloadSVC #1/WFI/MOVZ/SVC #2. Serialel0: fiq. Alwaysgic::route_group0_as_irqafter the trip so the IRQ path stays default. - SError honesty. Vector slot
0x580callsserror_lower_el(defensive: if standing, stay at EL0 and printel0: serror). No taken-path probe on this cut — hello printsel0: serror-park. Taken SError stays Planned. - Fail-closed sensors.
scripts/qemu-smoke.shgrepsel0: fiqandel0: serror-park.#[test_case]lower_el_fiq_while_standing. - Still Planned. Taken lower-EL SError; PAN enable on
-cpu cortex-a57; remaining identity RAM / yank_start; umbrella EL0 isolation. - NFR-10 text is revised in place (ID unchanged). Threat-model v1.23. Do not mint NFR-15+.
Honesty
Say “taken lower-EL FIQ while standing returned to EL0” only when the serial / tests pass. Do not say:
- “EL0 isolated”
- PAN (see ADR-026)
- lower-EL SError is taken / Verified on this guest
- default standing/task
ERETunmasks FIQ (default SPSR still sets F; only the FIQ probe clears F and arms FIQEn) - “secure OS” / “hardened”
Consequences
src/gic.rsowns FIQEn route helpers.src/exception.rsownsfiq_lower_el/serror_lower_el.src/el0.rsowns the standing FIQ probe and the SError park marker.- ADR-013 remains the isolation direction; this ADR closes the FIQ half of ADR-042 H4.
- A later ADR may take SError safely or enable PAN on a different CPU story. That work is not this cut.
ADR-044 — Taken lower-EL SError research (no Verified taken path yet)
- Status: Accepted as research — taken lower-EL SError while standing stays Planned;
el0: serror-parkremains the Verified honesty marker (ADR-043) - Date: 2026-09-13
Context
ADR-043 wired a defensive standing-aware serror_lower_el stub and prints el0: serror-park because this cut had no safe, deterministic in-guest inject on QEMU virt + TCG + -cpu cortex-a57. Sponsor asked for a research mile (Grok/docs) before claiming a taken path.
Default standing/task SPSR still masks SError (PSTATE.A set; ADR-041 SPSR 0x340 is D+A+F with I clear). A taken probe must clear A (and keep the guest standing) the same way the FIQ probe clears F.
Do not switch default -cpu. Do not claim “EL0 isolated.” Keep _start at 0x4008_0000.
Options surveyed
| ID | Option | Feasibility on ctos smoke | Honesty |
|---|---|---|---|
| H1 (recommended next mile) | Host-driven inject via QEMU HMP/QMP nmi, which on virt raises the CPU SError line (ARM_CPU_SERROR / TCG CPU_INTERRUPT_SERROR). Run smoke with a QMP socket; while standing EL0 (A clear + WFI), issue {"execute":"inject-nmi"} / HMP nmi. Serial el0: serror (taken) + existing park marker only when not probing. | Feasible with scripted QMP; needs smoke harness change (-qmp / unix socket), timing race around standing window, and fail-closed if QMP unavailable. Not an in-guest SVC. | Honest: host inject is how virt simulates NMI→SError. |
| H2 (rejected for Verified) | Guest-only RAS / imprecise abort / poisoned load to force async SError. | Unreliable on -cpu cortex-a57 TCG virt without RAS story; easy to confuse with sync aborts. | Would round up. |
| H3 (rejected) | Treat a sync abort / BRK / FIQ as “SError.” | Easy, wrong EC. | Forbidden. |
| H4 (status quo) | Keep el0: serror-park Verified; taken path Planned until H1 ships. | Current main (65cf515). | Correct until H1 probes. |
| H5 (out of scope) | KVM KVM_SET_VCPU_EVENTS serror_pending or plugin arm_cpu_inject_exception. | ctos smoke is TCG user-mode style -kernel on virt, not KVM guest tooling in CI. | Useful notes only. |
Vector / mask notes
- Lower-EL AArch64 SError vector is
VBAR_EL1 + 0x580(already wired). - Taken SError requires
PSTATE.Aclear at EL0. Adderet_to_el0_serror(e.g. SPSR with A clear, I/F set as needed) for the probe only — do not unmask A on default standingERETin the research mile. - After a taken trip, restore A-masked default so normal smoke stays stable.
Smoke harness sketch (not implemented here)
qemu-system-aarch64 … -qmp unix:./qmp.sock,server,nowait(or stdio QMP in a side channel).- Hello path: arm
EXPECT_EL0_SERROR,eret_to_el0_serror, payloadSVC #1/ short spin orWFI. - Host: wait for a UART cue (or fixed delay after
el0: standing), then QMPinject-nmi. - Grep
el0: serror(taken); reject claiming Verified if only park printed. - GHA matrix must have a QMP-capable
qemu-system-aarch64(already true on ubuntu runners once flags are added).
Decision
- This ADR is research only. No taken-path Verified claim. No
src/change required to accept it. - Recommended follow-up implementation is H1 (QMP/
nmiwhile standing with A clear), as a separate ADR (e.g. ADR-045) with probes + qemu-smoke greps. - Keep H4 until that follow-up lands:
el0: serror-parkstays Verified; taken lower-EL SError stays Planned. - Still Planned (unchanged honesty): PAN enable on
-cpu cortex-a57(ADR-026); yank_start/ leftover identity RAM (ADR-042); umbrella “EL0 isolated.” - Threat-model text may note “taken SError research recorded; inject path identified as QMP nmi” without minting NFR-15+.
Honesty
Say “we know how we would take lower-EL SError on virt” (QMP nmi + A clear). Do not say:
- taken SError is Verified on this guest
- “EL0 isolated” / PAN / “secure OS”
- default standing
ERETunmasks SError - guest-only RAS inject works on cortex-a57 TCG
Consequences
- ADR-043 remains the FIQ Verified + SError park cut.
- Implementers should open a new ADR for H1, not widen this file into a silent Verified.
- Follow-up: ADR-045 attempted H1; QEMU 10 virt+cortex-a57 has no working
inject-nmi→SError. Taken stays Planned. - Docker/(amd64 workstation) smoke does not need re-probe for this docs-only research PR.
ADR-045 — Taken lower-EL SError via QMP/nmi (honest partial)
- Status: Accepted as implementation attempt — guest A-clear standing path is wired; host QMP
inject-nmion QEMU 10virt+-cpu cortex-a57does not deliver SError (machine does not provide NMIs). Taken lower-EL SError while standing stays Planned.el0: serror-parkremains the Verified honesty marker (ADR-043 / ADR-044). - Date: 2026-09-13
Context
ADR-044 recommended H1: host QMP/inject-nmi while standing with PSTATE.A clear, expecting virt → ARM_CPU_SERROR. That recommendation tracked a 2020 qemu-devel series; upstream QEMU 10 does not implement that path on this smoke machine:
inject-nmi/ HMPnmi→machine does not provide NMIs(noTYPE_NMIon virt for cortex-a57 + default GICv2).- Modern virt “NMI” support is FEAT_NMI (GICv3 + a CPU with
aa64_nmi), not async SError, and requires switching-cpu/ GIC — forbidden here. - TCG has virtual SError (
HCR_EL2.VSE/EXCP_VSERR) for EL2→EL1, not a host inject into our EL1/EL0 standing guest. - Guest-only RAS inject remains rejected for Verified (ADR-044 H2).
Do not switch default -cpu cortex-a57. Do not claim “EL0 isolated.” Keep _start at 0x4008_0000. Do not unmask A on default standing/task ERET (SPSR 0x340 keeps A set).
Decision
- Guest plumbing (shipped).
eret_to_el0_serroruses SPSR0x2c0(D+I+F set, A clear). Hello armsEXPECT_EL0_SERROR, prints UART cueel0: serror-arm, enters standing dual-SVC with a bounded spin (notWFI— I/F stay masked, so a failed inject must not hang). Handlerserror_lower_elprintsel0: serroronly when EXPECT is armed, thenstay_at_el0. - Host smoke (attempted).
scripts/qemu-serial-inject.pyopens-qmp unix:…,server,nowaitand, afterel0: serror-arm, issues{"execute":"inject-nmi"}, logging the result. On this QEMU it errors; guest printsel0: serror-park.qemu-smoke.shstill requiresel0: serror-parkand must not requireel0: serror. - No
#[test_case]for the taken path — it needs a host inject. Smoke-only honesty. - Taken path stays Planned. Do not mark Verified. Park marker stays Verified.
- Still Planned (unchanged honesty): PAN enable on
-cpu cortex-a57(ADR-026); yank_start/ leftover identity RAM (ADR-042); umbrella “EL0 isolated.” - NFR-10 text revised in place (ID unchanged). Threat-model v1.25. Do not mint NFR-15+.
Honesty
Say “guest A-clear standing SError path is wired; QMP inject-nmi does not take SError on virt/cortex-a57.” Do not say:
- taken lower-EL SError is Verified on this guest
- “EL0 isolated” / PAN / “secure OS”
- default standing
ERETunmasks SError - ADR-044 H1 landed as a working inject on QEMU 10
- guest-only RAS inject works on cortex-a57 TCG
Follow-on
ADR-083 (2026-09-19): pinned QEMU opt-in Verifies taken el0: serror; stock park path unchanged.
Consequences
src/exception.rs/src/el0.rsown the A-clear probe + EXPECT.scripts/qemu-serial-inject.pyowns the QMP attempt.el0: serror-parkstays the fail-closed serial when inject does not land.- ADR-047 records the sponsor decision: taken path is deferred / non-goal on this smoke machine until an honest inject exists; park stays Verified; A-clear plumbing stays as dormant prep.
- A later ADR may find a different honest inject (new QEMU feature, or a constrained CPU story with sponsor approval). That work is not this cut.
- Hard-stop lock: ADR-053 (2026-09-14) re-probed
inject-nmi/ HMPnmion QEMU 10.0.13 virt+cortex-a57 — samemachine does not provide NMIs. Taken path is locked deferred/non-goal, not a fuzzy Planned mile. - Reopen re-probe: ADR-081 / ADR-082 (2026-09-19) on default
-cpu cortex-a76+ GICv3/virtualization=on/max— stillmachine does not provide NMIs. Dormant prep unchanged. - Docker/(amd64 workstation) re-probe is optional; agent-box smoke must show park + QMP error log, not a fake
el0: serror.
ADR-046 — A9 slot-disconnect performance delta (probe-only embed)
- Status: Accepted (compared CNTPCT pair on one boot — QEMU TCG lab measurement, not a published bench).
- Date: 2026-09-13
Context
A9 / ADR-030 prints perf: app-load ticks=<n> around the FAT /hello read + A3 map + ERET. ADR-032 removed the production A2–A4 / A9 include_bytes! path (FAT-only; smoke rejects slot: embed). The honesty ledger still had a Planned row: compare that FAT trip to the old embedded A3 trip on the same guest.
Do not restore production embed. Do not invent a percent or “faster/slower” claim. Do not switch -cpu. Keep _start at 0x4008_0000. QEMU TCG jitter is one lab — not criterion, not SPEC.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Probe-only include_bytes! of the same OUT_DIR/hello-libctos.elf in src/slot.rs. Time map+ERET with CNTPCT → perf: embed-load ticks=<n>. Keep FAT perf: app-load. Print perf: slot-delta app=<a> embed=<b> (raw ticks). Never print slot: embed. | Honest compared pair. Production slot stays FAT-only. |
| H2 (rejected) | Restore A3 embed as a production fallback when FAT is missing. | Violates ADR-032. Smoke must keep rejecting slot: embed. |
| H3 (rejected) | Close the Planned row as non-comparable (embed removed) without measuring. | Prefer H1 when feasible. Docs-only fallback only if dual-path is unsafe. |
| H4 (rejected) | Publish a percent / “slots are free” / criterion bench. | NFR-07 forbids invented benches. |
Decision
- Production path unchanged. A9 still loads FAT
/helloonly. Serial still printsslot: fat/slot: mapped/slot: ok/perf: app-load ticks=<n>. Missing/hellois stillslot: probe missed.slot: embedmust never appear. - Probe-only embed.
src/slot.rsmayinclude_bytes!(concat!(env!("OUT_DIR"), "/hello-libctos.elf"))for measurement. Bytes must match FAT/hello/HELLO_ELF_LEN. A2–A4 (src/libctos.rs,src/loader.rs) must not embed. - Markers. After a successful FAT trip on the same boot:
perf: embed-load ticks=<n>thenperf: slot-delta app=<a> embed=<b>. Fail closed onperf: embed-load missed. No ratio percent; no “faster/slower” words on the serial line. - Smoke.
scripts/qemu-smoke.shgreps the new markers, still rejectsslot: embed, rejects A2/A3 helloinclude_bytes!, requires the slot.rs probe include, and rejectsslot-deltalines that claim percent/faster/slower. - Honesty. This is a measurement pair on QEMU
virtTCG. It is not a latency budget, not a product KPI, and not “slots are free.” Record Verified only with serial evidence on a named tip.
Consequences
- Code:
src/slot.rs(probe include + markers); smoke greps;#[test_case]that embed CNTPCT advanced and bytes match FAT. - Docs: performance.md, measure.md, honesty ledger, track-a / roadmap as needed.
- Kernel debug ELF grows by the hello ELF size (~8 KiB). That is expected for the probe.
- Isolation leftovers (PAN enable, taken SError, yank
_start, umbrella) stay Planned — unchanged by this mile.
ADR-047 — Isolation leftovers decisions (SError / PAN / _start / umbrella)
- Status: Accepted (docs decision). Does not mint Verified for taken SError, PAN enable, yank
_start, or “EL0 isolated.” - Date: 2026-09-13
Context
Track A / A5 closed a deep isolation ladder through ADR-042 (start-stay honesty), ADR-043 (FIQ + SError park), ADR-044 / ADR-045 (taken SError research + QMP attempt), and ADR-026 (PAN ID-field). Four leftovers still read as open Planned work in the ledger and roadmap even though sponsor intent is to decide them honestly rather than leave them as pending miles that invite a fake Verified.
This ADR records those decisions. It does not change guest code. CloudAgent HELD; (amd64 workstation) Docker re-smoke is optional for this docs-only PR.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Decide each leftover: taken SError = non-goal/deferred on this smoke machine; PAN enable = non-goal on default -cpu cortex-a57; never yank _start while -kernel needs 0x4008_0000; leftover frame RAM optional Planned; umbrella “EL0 isolated” stays Planned/non-claim. | Honest close of the leftover decision without inventing probes. |
| H2 (rejected) | Mark taken SError / PAN enable / “EL0 isolated” Verified from existing miles. | Round-up. Forbidden. |
| H3 (rejected) | Silent -cpu switch or yank _start to “finish” isolation. | Breaks honesty / QEMU -kernel. |
Decision
1. Taken lower-EL SError — non-goal / deferred on this smoke machine
- QMP
inject-nmifails on QEMU 10virt+-cpu cortex-a57(machine does not provide NMIs). There is no honest host inject on this probe today (ADR-045). - Decision: taken lower-EL SError while standing is a non-goal / deferred on this smoke machine until an honest inject exists (new QEMU feature, accepted different machine story, or sponsor-approved CPU/GIC change with its own ADR).
- Keep
el0: serror-parkVerified (park honesty). - Keep guest A-clear standing plumbing (
el0: serror-arm/eret_to_el0_serror/ EXPECT) as dormant prep — do not delete; do not requireel0: serrorin smoke. - Do not claim taken SError Verified.
2. PAN enable on default -cpu cortex-a57 — non-goal on the default probe CPU
- ID field is 0 (
pan: absent). Enable + EL1-vs-EL0 fault is impossible without inventing hardware (ADR-026). - Decision: PAN enable is a non-goal on the default probe CPU. Do not silent
-cpuswitch. - ID-field probe (
pan: id=/pan: absent) stays Verified. - An optional future CPU story (different default
-cpu, FEAT_PAN present, enable + fault probe) needs its own ADR + sponsor approval. Until then, do not claim PAN.
3. Yank _start / leftover identity RAM
Amended 2026-09-26 by ADR-089 (sponsor D2): “never yank
_start” stands. Leftover identity RAM was torn (ADR-049, ADR-087), and the MMIO identity block was torn too (ADR-088). The_startstub page[0x4008_0000, 0x4008_1000)is the one documented identity exception, and a fail-closed TTBR0 inventory enforces it. The original text below is kept for history.
- QEMU
-kernelstill needs_startat0x4008_0000. - Decision: never yank
_startwhile that boot contract holds.ident: start-stay/ident: ram-staystay Verified honesty (ADR-042). - Leftover identity frame RAM after the heap may stay Planned as an optional later unmap. It is not required to close the umbrella isolation row under current constraints.
- Do not claim “the kernel moved” or full identity teardown.
4. Umbrella “EL0 isolated” (P-SEC-3l) — Planned / non-claim
- Decision: remains Planned / non-claim under current constraints (no PAN enable on a57,
_startstays, taken SError deferred). - Specific Verified miles (IRQ/FIQ/tears/start-stay/PAN ID-field/…) are not this row. Do not round them up to “EL0 isolated.”
- Follow-on (2026-09-28, M6): every ADR-055 §B row is Met and the sponsor accepted the umbrella sentence in writing (B3). This row is Verified, only as the accepted ADR-091 sentence. The decision above is as of 2026-09-14.
Status table (after this ADR)
| Item | Status | Notes |
|---|---|---|
el0: serror-park | Verified | Park honesty |
| Guest A-clear SError prep (ADR-045) | Dormant prep | Not a taken probe |
| Taken lower-EL SError on this smoke machine | Deferred / non-goal (locked) | ADR-053 |
| PAN ID-field on cortex-a57 | Verified | pan: absent |
| PAN enable on default cortex-a57 | Non-goal (locked) | ADR-054 reopen gate |
ident: start-stay / never yank _start | Verified (honesty) / decided | Keep 0x4008_0000 |
| Leftover identity RAM after heap | Verified (optional) via ADR-049 | Not umbrella-required |
| Umbrella “EL0 isolated” | Planned / non-claim until checklist | ADR-055; constraints above |
Honesty
Say the decisions above only as documentation. Do not say:
- taken lower-EL SError is Verified
- PAN is enabled / “EL0 isolated”
_startwas yanked / the kernel moved- app hosting is done (see ADR-048)
Consequences
- Docs: el0.md, security.md (threat-model v1.26), honesty ledger (Planned→decided wording), roadmap P-SEC-3k/3l/3r + leftover blurb, NFR-10 text in place, SUMMARY.
- Code: unchanged. Smoke still requires park / start-stay /
pan: absent; still rejectsel0: serroras a required marker and rejectspan: enabled. - A later ADR may reopen taken SError (honest inject), PAN (new CPU story), or boot redesign so
_startcan leave identity — each needs sponsor scope. - 2026-09-14 locks: ADR-053 hard-stops taken SError with dated QMP evidence; ADR-054 locks PAN enable + reopen gate; ADR-055 publishes the umbrella non-claim checklist. Decisions in this ADR stand; those ADRs remove fuzzy Planned wording.
- 2026-09-15 sponsor closure: ADR-060 is the sponsor-facing one-pager (locks + reopen gates). Does not rewrite decisions here.
ADR-048 — App hosting “done” claim criteria
- Status: Accepted (claim gate). Product “apps run independently / hosting done” is Verified when criteria 1–5 are Met (ADR-052 records sponsor accept for #5). Do not claim Linux userspace, POSIX, containers, or “EL0 isolated.”
- Date: 2026-09-13
Context
Track A children A1–A9 have first cuts (ABI, CRT, loader, standing-as-normal, isolation miles, memfs, FAT, recipes, OS/app slots + cross-update / embed-off). The honesty ledger and roadmap correctly refuse to round those miles up to “app hosting is done.” Sponsors still need explicit criteria so a Verified claim is fail-closed rather than mood.
Related: ADR-030, ADR-032, ADR-029, ADR-052 (sponsor accept), hosting-apps.md, ADR-047 (isolation leftovers are separate — closing them is not required to meet these criteria, and meeting these criteria does not make “EL0 isolated” true).
Decision
A1–A9 first cuts
Treat A1–A9 first cuts as Verified when the honesty ledger has matching rows for this tip (or cited merge tips). That is not product app hosting by itself.
Product claim Verified only when all criteria below are met
The sentence “apps run independently of the OS” / “app hosting is done” may move from Planned to Verified only when every criterion holds, with named probes in the honesty ledger:
- Documented recipes — Rebuild / run recipes for freestanding apps cite only existing serial markers and stay buildable (
./scripts/docs-build.sh+ A8 hub). No invented runtime. - Slot cross-update — Same published app ELF loads on this OS and a documented prior OS (
ba6541cor a later sponsor-pinned prior) with fail-closed host smoke (cross-update … slot:okboth boots). ADR-032. - No production embed — A2–A4 / production slot path must not
include_bytes!the hello app. FAT/hello(or an equivalent published slot file) is the production payload. Probe-only embeds (e.g. ADR-046 measurement) must never printslot: embedand must stay out of the production path. - Honesty about missing Linux / POSIX / containers — Docs and PRs must still say: not a Linux ABI, not POSIX, not glibc/musl, guest OCI/Docker/k8s remains non-goal (ADR-029). Meeting (1)–(3) does not authorize those claims.
- Explicit sponsor accept — A PR that flips the product row to Verified must cite this ADR’s checklist and must not self-merge (ADR-002). Sponsor accept for the 2026-09-13 flip is recorded in ADR-052.
What does not satisfy the claim
- A1–A9 first cuts alone
- Isolation ladder miles / ADR-047 decisions
- Host
docker-smoke.sh(Docker hosts ctos, not the reverse) - Slot CNTPCT pairs / “slots are free” marketing (ADR-046)
- Linux-compat research (Track B) or any “runs Alpine” sentence
Accepted claim (amendment)
On tip main @ be789c5+ with ADR-052: criteria 1–5 Met. Product row Verified for the scoped freestanding slot-hosting sentence. Still refuse Linux/POSIX/containers/“EL0 isolated”/OTA/immutable-OS marketing.
Honesty
Say “A1–A9 first cuts exist” when the ledger rows say so. Say the product claim is Verified under ADR-048 only with the checklist Met×5 and sponsor accept cited. Do not say “apps update independently” as OTA, or “container host” — containers stay non-goal regardless.
Consequences
- Ledger row “Track A app hosting (slots)” is Verified and cites this ADR + ADR-052.
- hosting-apps.md, Track A / roadmap, NFR-10 text point here.
- No new FR/NFR IDs. Do not mint NFR-15+.
Progress checklist
Living Met/Partial/Blocked table (tip evidence): hosting-apps.md — Claim criteria and honesty-ledger.md — ADR-048 progress. Product claim is Verified when criterion 5 is Met (ADR-052).
ADR-049 — Leftover identity RAM tear after the heap
- Status: Accepted (leftover identity frames after the heap unmapped; high twins stay;
_startstays) when the serial / tests pass - Date: 2026-09-13
Context
ADR-038 unmapped identity heap while GlobalAlloc returned TTBR1 VAs. ADR-042 printed honesty stay markers: ident: start-stay for _start at 0x4008_0000 and ident: ram-stay for leftover identity frames in [heap_pa_end, pool_end). ADR-047 decided never yank _start while QEMU -kernel needs that address, and left leftover frame RAM as an optional Planned mile (not umbrella-required).
This ADR takes that optional mile: unmap leftover identity frame RAM after the torn heap, keep high twins, keep _start mapped+executable, and fail-closed on serial / tests / smoke. It does not claim “EL0 isolated,” “kernel moved,” PAN enable, or taken SError Verified. Umbrella P-SEC-3l stays Planned/non-claim.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Unmap identity frames in [heap_pa_end, pool_end). Keep boot-stub pages needed for -kernel. Print ident: ram / ident: ram-fault / ident: ram-high. Access newly allocated frames via the TTBR1 twin (frame_cpu_va). | Smallest honest optional tear. |
| H2 (rejected) | Yank _start / claim full higher-half / claim “EL0 isolated.” | Forbidden by ADR-047 / roadmap. |
| H3 (rejected) | Leave ident: ram-stay forever and mark the optional mile Verified without a tear. | Would invent closure. |
Decision
- Unmap leftover identity RAM. After
tear_identity_heap,tear_identity_ramunmaps[heap_pa_end, pool_end)from kernel TTBR0. Full 2 MiB L2 blocks are cleared as one slot; partial edge blocks are punched page-by-page. User TTBR0 already omits that range. High twins stay (L2_HIGH_RAM). One boot-timeTLBI VMALLE1after the bulk unmap. - Plant + probe. Before unmap, plant a magic word via the high twin. Serial
ident: ram lo=… hi=… pages=NwithN >= 1. EL1LDRof a torn identity leftover VA is a current-EL translation DABORT (ident: ram-fault). EL1LDRof the high twin still returns the planted magic (ident: ram-high). - Keep
_start.ident: start-staymust still pass. Do not change default-cpu. Do not claim PAN / “EL0 isolated” / “kernel moved.” - Frame CPU VA. Call sites that used a frame PA as an identity VA (
paging::observe_probe,map_unmap_roundtrip,loader,asid,ttbr1private-page plant) switch topaging::frame_cpu_va(TTBR1 twin after the high split). - Fail-closed smoke.
scripts/qemu-smoke.shrequiresident: ram lo=/ident: ram-fault/ident: ram-highand rejectsident: ram-stay. Keep requiringident: start-stay. - Still Planned / decided. Umbrella “EL0 isolated” (P-SEC-3l) stays Planned/non-claim. PAN enable non-goal on default a57. Taken SError deferred/non-goal on this smoke machine. Never yank
_start. - NFR-10 text is revised in place (ID unchanged). Threat-model v1.27. Do not mint NFR-15+.
Honesty
Say “leftover identity frame RAM after the heap was unmapped while the high twin stayed and _start stayed mapped” only when the serial / tests pass. Do not say:
- the kernel has moved to the high half
- identity mappings were fully torn down (
_startstays) - “secure OS” / “hardened” / “EL0 isolated”
- PAN enable / taken SError Verified
Consequences
paging::tear_identity_ram/frame_cpu_va/ teardown probes own the cut. ADR-042ident: ram-stayis superseded as a stay marker;ident: start-stayremains.- ADR-047 leftover-RAM row becomes Verified (optional mile taken); umbrella stays Planned/non-claim.
ADR-050 — FAT16 write behind the thin VFS
- Status: Accepted (FAT16 write mile Verified when the serial / tests / host-check pass; Track A / app hosting still Planned)
- Date: 2026-09-13
Context
ADR-028 landed virtio-blk + read-only FAT16 behind the thin VFS (ADR-027). Guest write on a FAT handle returned ReadOnly. Sector R/W was proven on an unused last sector, not by mutating the volume.
Docs “new vs extend” prefer deepening the known small FS (FAT16) over inventing a format or jumping to ext4 / FAT32 / xv6. This mile is Future slice 3: guest FAT16 write (and a small create) on the virtio-blk volume, same thin VFS, fail-closed serial + tests + optional host check.
Not POSIX open/write. Not FAT32. Not “supports FAT” as a product. Not app hosting. A9 /hello load path must keep working.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Guest FAT16 write on an open handle via the same vfs::write, plus FAT backend create of a small single-cluster file. Serial fat: write / fat: rewrite / fat: create / keep fat: ok. Host --check-write sees /fwr. | Deepens ADR-028. Same path grammar. Caps: one cluster, FILE_MAX (256). |
| H2 (alt, rejected this PR) | Directory listing / readdir-style thin VFS mile only. | Write is in scope and fits one PR. Listing is a later depth mile. |
| H3 (rejected) | FAT32, ext4, xv6 FS, or a Linux VFS clone. | Forbidden by overview + ADR-028. |
| H4 (rejected) | Route vfs::create to FAT when mounted. | Would put A6 /kprobe / /eprobe on the volume; vfs::reset cannot clear them; second create → Exists. vfs::create stays memfs-first. FAT create is fat::create (backend), then confirmed with vfs::open / read. |
| H5 (rejected) | Claim POSIX write or “writeable FAT volumes” as a product. | Honesty: say the guest wrote FAT16 bytes when markers pass. |
Decision
- FAT
writeon an open handle. Updates the file’s first (only) cluster and the root dirent size. Offset starts at 0 onopen. Empty write →TooBig(same shape as memfs). Refuse multi-cluster grow and sizes aboveFILE_MAX/ one cluster this mile. - Probe rewrite of
/probe.vfs::open("/probe")→ writefat-wr→ read back → writefat-hirestore →fat: write/fat: rewrite. Keeps later/probereaders and tests stable. - FAT backend create of
/fwr.fat::create("/fwr")allocates one free cluster + root dirent, writesfat-nw, thenvfs::open/readconfirms. Serialfat: create. Left on the image for host--check-write. vfs::createstays memfs-first (A6). Do not invent a second open story; do not put memfs probe names on the volume.- Fail-closed smoke. Grep
fat: write/fat: rewrite/fat: create/fat: ok. Rejectfat: probe missed. Keep A9slot: ok/ cross-update. Hostscripts/mkfat16.py --check-writerequires/fwr=fat-nwand restored/probe=fat-hi. - Honesty. Say “the guest wrote FAT16 bytes through the thin VFS (and created a small FAT16 file)” only when serial / tests / host-check pass. Do not say: POSIX, FAT32, “supports FAT,” app hosting done, EL0 isolated, PAN, taken SError.
- NFR-10 text is revised in place (ID unchanged). Threat-model v1.28. Do not mint NFR-15+.
Consequences
- Code:
src/fat.rs(write / create / restore probe),scripts/mkfat16.py --check-write,scripts/qemu-smoke.shgreps. - Docs: this ADR, filesystem.md, framework/filesystem.md, honesty ledger, roadmap note, threat-model v1.28.
- Follow-ups (not this PR): multi-cluster grow, dirent shrink helpers,
vfs::create→ FAT without breaking A6. Rootreaddir: ADR-056. Delete: ADR-057.
ADR-051 — FAT16 vs memfs write CNTPCT pair (measure-first)
- Status: Accepted (compared CNTPCT pair on one boot — QEMU TCG lab measurement, not a published bench).
- Date: 2026-09-13
Context
ADR-050 landed guest FAT16 write behind the thin VFS (fat: write / fat: rewrite / fat: create / fat: ok). Existing performance probes cover loop CNTPCT, IRQ delta, boot-delta, ELF size, and the A9 slot pair (ADR-046). Future slice 4 asks for a measure-first CNTPCT sample around the new FAT write path, compared on the same boot to a memfs write of the same small payload.
Do not invent a latency SLA, percent, or “faster/slower” claim. Do not add a criterion crate. Keep fat: write / fat: ok / A9 slot markers. QEMU TCG jitter is one lab — not SPEC.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Time the FAT VFS write of the /probe rewrite payload (fat-wr) with CNTPCT → perf: fat-write ticks=<n>. On the same boot, time a memfs write of the same bytes → perf: memfs-write ticks=<n>. Print perf: fs-write-delta fat=<a> memfs=<b> (raw ticks). | Honest compared pair. Not a bench. |
| H2 (rejected) | Publish a percent / “FAT is slower than memfs” / criterion bench. | NFR-07 forbids invented benches. |
| H3 (rejected) | Time the whole open/read/restore/create path as one number and call it “write latency.” | Too wide; pair the write of the same payload. |
| H4 (rejected) | Skip the memfs compare and only print FAT ticks. | Prefer a same-boot pair when feasible (same shape as ADR-046). |
Decision
- FAT write sample. During the ADR-050
/proberewrite, sampleCNTPCTaround the singlevfs::writeofWRITE_BYTES(fat-wr). Store ticks; fail closed if the counter does not advance. - Memfs compare. After the FAT write/create probes succeed,
vfs::create/opena memfs path (/mwprobe) and sampleCNTPCTaroundvfs::writeof the same bytes. Fail closed if ticks are zero. - Markers. Serial:
perf: fat-write ticks=<n>,perf: memfs-write ticks=<n>,perf: fs-write-delta fat=<a> memfs=<b>. No ratio percent; no “faster/slower” words on the serial line. Missed samples printperf: fat-write missed/perf: memfs-write missed. - Existing markers unchanged. Keep
fat: write/fat: rewrite/fat: create/fat: okand A9slot:*/ ADR-046 pair. - Smoke.
scripts/qemu-smoke.shgreps the new markers and rejectsfs-write-deltalines that claim percent/faster/slower. - Honesty. This is a measurement pair on QEMU
virtTCG. It is not a latency budget, not a product KPI, and not “FAT write cost is X.” Record Verified only with serial evidence on a named tip. Do not claim app hosting done, EL0 isolated, POSIX, or FAT32.
Consequences
- Code:
src/fat.rs(CNTPCT around FAT write + memfs compare + markers); smoke greps;#[test_case]that both samples are> 0. - Docs: this ADR, performance.md, measure.md, honesty ledger, roadmap
P-PERF-6. - Follow-ups (not this PR): multi-cluster write cost, create-path timing, host-side FS benches — only after a probe shows a need.
ADR-052 — Sponsor accept: product app-hosting claim (ADR-048 #5)
- Status: Accepted (docs). Product “apps run independently of the OS” / “app hosting is done” is Verified under ADR-048 criteria 1–5. Not Linux userspace. Not POSIX. Not containers. Not “EL0 isolated.”
- Date: 2026-09-13
Context
ADR-048 gated the product Track A app-hosting sentence on five criteria. On tip main @ be789c5 (and newer), criteria 1–4 were already Met in the honesty ledger (documented A8 recipes; A9 cross-update on this OS + ba6541c; no production hello embed — probe-only include_bytes! in src/slot.rs for ADR-046; honesty that Linux/POSIX/guest OCI remain refused / non-goal per ADR-029). Criterion 5 (explicit sponsor accept on the flip PR, no self-merge) was the remaining Blocked gate.
Sponsor signal (user): after a still-Planned review that listed product app hosting as blocked only on criterion #5, the sponsor said “address 1. app hosting.” Treat that as explicit sponsor accept for ADR-048.
Related: ADR-048, ADR-030, ADR-032, ADR-029, ADR-002, hosting-apps.md, honesty-ledger.md.
Decision
- Accept the product claim. Flip the honesty-ledger row “Track A app hosting (slots)” from Planned → Verified for the scoped sentence: freestanding apps load from a separate OS/app slot (FAT
/hello), rebuild recipes exist, the same published app ELF cross-updates on this OS and documented priorba6541c, and production does not embed the hello payload. - Cite this ADR + ADR-048 checklist on the flip PR. Do not self-merge (ADR-002).
- Keep the non-claims. Meeting ADR-048 does not authorize: Linux userspace, POSIX, glibc/musl, guest OCI/Docker/k8s, “EL0 isolated,” PAN enabled, taken SError Verified, “immutable OS” marketing, or OTA.
- Threat-model patch. security.md may record the Verified product claim under ADR-048/052 and bump the patch version; it still must refuse the non-claims above.
Honesty
Say the product Track A app-hosting claim is Verified under ADR-048 (criteria Met×5, sponsor accept cited). Do not say ctos hosts Linux programs, POSIX apps, or containers. Do not round A1–A9 miles alone into the product claim without this accept record.
Consequences
- Ledger product row + ADR-048 progress checklist criterion #5 → Met / product Verified.
- Present-tense docs (hosting-apps.md, Track A / roadmap, syscall.md, NFR-10 note, companions) align without Linux/container/isolation round-ups.
- ADR-048 status amended: claim gate Accepted; product claim Verified via this ADR.
- No
src/change required for this accept. No new FR/NFR IDs.
ADR-053 — Taken lower-EL SError hard-stop (no honest inject on smoke machine)
- Status: Accepted (docs / evidence lock). Taken lower-EL SError while standing stays deferred / non-goal and is locked. Reopen re-probe on default
-cpu cortex-a76: ADR-081 / ADR-082 (2026-09-19) — still blocked. (not a fuzzy Planned mile).el0: serror-parkremains Verified. Guest A-clear plumbing remains dormant prep. Do not mark taken SError Verified. - Date: 2026-09-14
- Supersedes fuzzy Planned wording for this row; does not reopen ADR-047 decision #1 — it hard-stops it with dated re-probe evidence.
Context
ADR-044 recommended host QMP/inject-nmi (H1). ADR-045 wired the guest A-clear standing path and attempted H1; QEMU 10 virt + -cpu cortex-a57 returned machine does not provide NMIs. ADR-047 decided deferred/non-goal. Sponsor / DSO asked to secure this gap before Guest Linux / containers / immutable-OS marketing: re-check whether any honest inject exists; if not, ADR hard-stop with dated evidence so the taken path is not a fuzzy Planned mile.
Do not silent -cpu switch. Do not claim “EL0 isolated.” Keep _start at 0x4008_0000. Do not unmask A on default standing/task ERET. CloudAgent HELD; docs-only (no new guest inject code — no honest host path exists).
Dated re-probe (2026-09-14, agent box)
Smoke machine (same class as ADR-045 / ledger):
- Host: agent box
qemu-system-aarch6410.0.13 (Debian1:10.0.13+ds-0+deb13u1)-machine virt+-cpu cortex-a57- QMP unix socket; guest paused with
-S(machine/CPU identity only — inject availability does not need the ctos kernel)
Commands tried (documented H1 path only; no invented inject):
| Attempt | Result |
|---|---|
QMP {"execute":"inject-nmi"} | error class=GenericError desc=machine does not provide NMIs |
QMP human-monitor-command / HMP nmi | Error: machine does not provide NMIs |
query-commands on this QEMU shows inject-nmi present, but the virt machine does not implement TYPE_NMI for this CPU/GIC story. Other *-inject-* commands are CXL-only and are not an ARM SError path for ctos. ADR-044 alternatives remain rejected or out of scope:
| ID | Option | 2026-09-14 status |
|---|---|---|
| H1 | QMP/HMP nmi → ARM_CPU_SERROR | Impossible on this smoke machine (dated error text above). Matches ADR-045 / amd64 workstation 3f7d3c5 smoke log. |
| H2 | Guest-only RAS / poisoned load | Still rejected for Verified (ADR-044). |
| H3 | Treat sync/BRK/FIQ as SError | Still forbidden. |
| H5 | KVM serror_pending / plugin inject | Still out of scope for TCG virt -kernel smoke. |
Modern virt “NMI” / FEAT_NMI (GICv3 + aa64_nmi) is a different CPU/GIC story and would require a silent or sponsor-approved -cpu / GIC change — not an honest inject on the default probe.
Decision
- Hard-stop. Taken lower-EL SError while standing on QEMU 10
virt+-cpu cortex-a57is deferred / non-goal and locked. It is not a Planned implementation mile awaiting a “try harder” cut on this machine. - Park stays Verified. Smoke must keep requiring
el0: serror-park. Do not requireel0: serror. - A-clear plumbing stays dormant prep. Keep
el0: serror-arm/eret_to_el0_serror/ EXPECT from ADR-045. Do not delete; do not claim taken. - Reopen gate (only). A later cut may reopen taken SError only with: (a) an honest host inject that actually delivers SError on the accepted smoke machine, or a sponsor-approved machine/CPU/GIC change with its own ADR; (b) serial
el0: serrorunder EXPECT; (c) a new ADR that cites working evidence. Until then, do not mint Verified. 2026-09-19 re-probe: ADR-081 / ADR-082 — gate unmet on virt+cortex-a76 (and GICv3 / virtualization / max variants); explicit unlock = ADR-081 Decision §4 B1/B2/B3 (B1 research: ADR-082). - Still non-claims: PAN enable on default a57 (ADR-054); umbrella “EL0 isolated” (ADR-055); never yank
_start.
Honesty
Say “taken lower-EL SError is hard-stopped / deferred on this smoke machine; QMP inject-nmi still returns machine does not provide NMIs (re-probed 2026-09-14).” Do not say:
- taken lower-EL SError is Verified
- “EL0 isolated” / PAN enabled / “secure OS”
- a different
-cpuwas used to make inject work - guest-only RAS inject is Verified
- the taken path is still a near-term Planned mile on virt+a57
Follow-on
ADR-083 (2026-09-19): pinned QEMU opt-in Verifies taken el0: serror; stock park path unchanged.
Consequences
- Ledger / roadmap P-SEC-3r: cite this ADR as the hard-stop lock (park Verified; taken locked deferred/non-goal).
- ADR-045 remains the implementation attempt; this ADR is the dated impossibility lock.
- ADR-047 decision #1 stands; wording tightens from “until an honest inject exists” (open-ended) to “locked until reopen gate in Decision §4.”
- Threat-model patch version bump (security.md). No
src/change. No new FR/NFR IDs. - Reopen attempt: ADR-081 / ADR-082 (2026-09-19) — still blocked; park Verified.
- B2 spike: ADR-084 (2026-09-25) — free hosted runners have no arm64 KVM/HVF/WHPX; B2 Planned / blocked.
- FAT readdir and other product cuts stay out of this PR.
ADR-054 — PAN enable lock on default -cpu cortex-a57 (reopen gate)
- Status: Accepted (docs / evidence lock). Reopen gate items 1–4 are met: ADR-079 (CPU +
pan: present) and ADR-080 (enable + EL1-vs-EL0 fault). Historical a57 ID-field stays Verified (pan: absent, ADR-026). Do not claim “EL0 isolated.” - Date: 2026-09-14
- Locks ADR-047 decision #2 with an explicit reopen gate. Does not change guest code or default
-cpu.
Context
ADR-026 Verified that ID_AA64MMFR1_EL1.PAN == 0 on -cpu cortex-a57 (serial pan: id=0 / pan: absent). Enable + EL1-vs-EL0 access fault is impossible without inventing hardware or silently switching -cpu. ADR-047 already called enable a non-goal on the default probe CPU. Sponsor / DSO asked to secure this gap before Guest Linux / containers / immutable-OS marketing: ADR+evidence lock so enable is not a fuzzy Planned mile, with an explicit reopen gate.
Do not silent -cpu switch. Do not execute MSR PAN while the ID field is 0. Do not claim “EL0 isolated.” CloudAgent HELD; docs-only.
Evidence (locked)
| Evidence | Status | Cite |
|---|---|---|
Default smoke CPU is -cpu cortex-a57 | Scripts contract | scripts/qemu-aarch64.sh, scripts/qemu-serial-inject.py |
ID_AA64MMFR1_EL1.PAN == 0 → pan: absent | Verified | ADR-026; honesty-ledger PAN ID-field rows (e.g. 2026-09-12 pan: id=0 / pan: absent) |
Smoke rejects pan: enabled | Verified harness | scripts/qemu-smoke.sh |
| QEMU CPU model props do not expose a free “turn on PAN” on cortex-a57 without a model change | Agent-box QMP 2026-09-14 | query-cpu-model-expansion type=full model.name=cortex-a57 — no pan prop to flip on this model |
Decision
- Lock. PAN enable (PSTATE.PAN /
MSR PAN+ EL1 load of an EL0-accessible page must fault) is a non-goal on the default probe CPU-cpu cortex-a57. Roadmap P-SEC-3k stays Non-goal (default probe CPU), not Planned. - ID-field stays Verified. Keep printing
pan: id=/pan: absent. Keep#[test_case]pan_unimplemented_on_probe_cpu. Smoke must still rejectpan: enabled/pan: probe missed. - Explicit reopen gate. PAN enable may be reopened only when all of the following hold:
- Sponsor-approved new default probe CPU (or documented FEAT_PAN-present CPU story) — recorded in a new ADR, not a silent script edit.
- Serial (or equivalent) evidence that
ID_AA64MMFR1_EL1.PAN != 0on that accepted CPU (pan: present). - A follow-up ADR that enables PSTATE.PAN and proves an EL1-vs-EL0 access fault (fail-closed if the fault does not land).
- Honesty ledger + smoke greps updated for the new CPU; old a57
pan: absentrows remain historical truth for a57.
- Until the reopen gate is met, do not claim PAN, “privileged access never,” or “EL0 isolated.”
Reopen progress (2026-09-19)
ADR-079 / #125: sponsor unlock + default probe CPU -cpu cortex-a76 + serial pan: present (gate items 1–2). ADR-080: enable + EL1-vs-EL0 fault (gate items 3–4). a57 pan: absent remains historical truth for that CPU. Umbrella “EL0 isolated” still non-claim (ADR-055).
Honesty
Say “PAN enable is locked non-goal on default -cpu cortex-a57; ID-field Verified pan: absent (ADR-026).” Do not say:
- PAN is enabled
- EL1 cannot access EL0 pages (PAN mile)
- the probe used
-cpu max/ another model without a sponsor ADR - “EL0 isolated” / “secure OS”
Consequences
- Ledger / roadmap P-SEC-3k cite this ADR as the enable lock + reopen gate.
- ADR-026 remains the capability probe; this ADR is the enable hard-lock.
- ADR-055 lists PAN enable as an unmet umbrella requirement.
- Threat-model patch bump. No
src/change. No silent-cpuedit.
ADR-055 — Umbrella “EL0 isolated” non-claim checklist
- Status: Accepted (docs). Since M6 (2026-09-28) every §B row is Met and the umbrella is Verified only as the accepted ADR-091 sentence. Original status text (2026-09-14): Umbrella “EL0 isolated” stays Planned / non-claim and cannot become Verified under current constraints. This ADR tightens the non-claim with an evidence checklist; it does not invent Verified.
- Date: 2026-09-14
- Follow-on (2026-09-28, M6): the sponsor accepted the ADR-091 revision-3 sentence and scope in writing (B3, 2026-09-28 16:41 CEST; sponsor wrote it in the DSO agent chat, DSO relayed it to ctos). Row 3 is Met. ADR-091 is Accepted. The umbrella row flips to Verified, scoped to that sentence and its limits.
- Follow-on (2026-09-27, ADR-092): sponsor option (b) on ADR-091. G1 (EL0-reachability walk) and G2 (EL0 write fault) closed as §A miles below; new gap G4 (syscall copy helpers skip the EL0 permission check) recorded. Row 3 still not Met (ADR-091 revision 2 is Draft).
- Follow-on (2026-09-26, M5): the umbrella sentence and scope are drafted in ADR-091 (Draft / pending sponsor accept, D4). Row 3 stays not Met until the sponsor has read the draft and accepted it in writing. Nothing is flipped.
- Follow-on (2026-09-26, M4): sponsor D1 accepted. The taken-SError evidence class for the umbrella is the B1 pin job (now unconditional on every push/PR) plus the B2-P one-off KVM run (ADR-090). Row 1 is reworded and Met on that class. Stock QEMU still parks. The umbrella still needs the sponsor’s written accept (M5 draft → ADR-091).
- Audit (2026-09-26): ADR-086 re-probed every row this session. All §A miles are Verified. All §B rows are still not Met: taken SError is Verified only on opt-in paths (B1 pin / B2-P one-off). The identity inventory still leaves five identity ranges (
_startis one of them, by decision). No umbrella accept exists. §A markerel0: no datais stale wording: the code usesel0: no kernel read. - Locks ADR-047 decision #4. Optional ledger clarity: “non-claim until checklist.”
Context
Many Track A / A5 miles are Verified (standing EL0, IRQ/FIQ while standing, identity tears, PAN ID-field, SError park, start-stay, leftover RAM tear, …). Rounding those miles up to “EL0 isolated” is forbidden. Sponsor / DSO asked to secure the umbrella gap before Guest Linux / containers / immutable-OS marketing: publish a checklist of Verified miles vs umbrella requirements still unmet, given:
- PAN enable was locked on default a57 (ADR-054); reopen + enable on cortex-a76 (ADR-079 / ADR-080) — still not the umbrella
_startstays / never yank (ADR-042 / ADR-047)- Taken lower-EL SError hard-stopped / deferred (ADR-053; reopen re-probe ADR-081 / ADR-082)
CloudAgent HELD; docs-only.
Decision
- (Superseded by M6, 2026-09-28: all §B rows Met; the umbrella is Verified as the ADR-091 sentence only.) Umbrella stays Planned / non-claim. Ledger wording may say “non-claim until checklist” (this ADR). Do not flip to Verified.
- Checklist is the gate. “EL0 isolated” may be reconsidered only when every Unmet row below is Met under its own ADR + probes — not by summing existing miles.
- Specific Verified miles are not the umbrella. Cite them as themselves.
Evidence checklist
A. Verified miles (not sufficient for the umbrella)
| Mile | Evidence (serial / test) | ADR |
|---|---|---|
| EL0 first mile + NX kernel data | el0: ok / el0: nx kernel | ADR-013 |
| User TTBR0 read mile | el0: no kernel read (wording fixed 2026-09-26, ADR-087; earlier text said el0: no data, which no code prints) | ADR-013 |
| Standing EL0 + standing task | el0: standing / el0: task-ok | ADR-013 / ADR-024 |
| ASID isolation | asid: ok | ADR-013 |
| TTBR1 private page + high EL1 fetch | ttbr1: ok / ttbr1: el1 exec | ADR-016 / ADR-017 |
| Identity tears (text/rodata/data/heap/ram) | ident: * / ident: ok | ADR-018…020 / 025 / 037 / 038 / 049 |
EL0 entry without VMALLE1 | el0: no-vmalle1 | ADR-039 |
| Lower-EL IRQ while standing + default I clear | el0: irq / el0: irq-default | ADR-040 / ADR-041 |
| Lower-EL FIQ while standing | el0: fiq | ADR-043 |
| SError park honesty | el0: serror-park | ADR-043 / ADR-045 |
| PAN ID-field on default probe CPU | pan: present on cortex-a76 (ADR-079); historical a57 pan: absent | ADR-026 / ADR-079 |
| PAN enable + EL1-vs-EL0 fault | pan: enabled / pan: el1-fault | ADR-080 |
| EL0 store to kernel data (TTBR1) / EL0-RO page → permission fault (2026-09-27) | el0: write-fault kernel … ec=0x24 wnr=1 dfsc=perm-l3 / el0: write-fault user-ro … / el0: write-ok kernel,user-ro | ADR-092 |
| Fail-closed EL0-reachability walk (k/u/ASID-B TTBR0 + TTBR1; allowlist = five user slots) (2026-09-27) | el0-reach: live k=3 u=3 a=0 h=0 pages=6 leaks=0 / el0-reach: steady … pages=0 leaks=0 / el0-reach: neg … caught / el0-reach: ok allow=… | ADR-092 |
_start stay honesty | ident: start-stay | ADR-042 / ADR-047 |
B. Umbrella requirements (were Unmet; all Met since M6, 2026-09-28)
| Requirement | Current status | Why unmet | Reopen / lock ADR |
|---|---|---|---|
Taken lower-EL SError while standing, on the D1 evidence class (reworded 2026-09-26 per sponsor D1, ADR-090): B1 qemu-nmi-pin green, unconditional on every push/PR, plus B2-P ADR-085 run 4 | Met (D1 evidence class; not the umbrella). B1 bare el0: serror fail-closed in CI; B2-P el0: serror + b2: taken under Graviton3 KVM (one-off, at 159b178). Stock virt TCG itself: still deferred / non-goal (hard-stopped), el0: serror-park | Stock QEMU has no honest inject on virt TCG (machine does not provide NMIs) — re-probed a76 + GICv3/virt-on/max 2026-09-19. B1 opt-in pin Verified (ADR-083); B2 free-runner spike blocked 2026-09-25 (ADR-084) | ADR-053 / ADR-081 / ADR-082 / ADR-083 / ADR-084 |
Full identity teardown except the documented _start stub page (reworded 2026-09-26 per sponsor D2, ADR-089; was “including yank _start”) | Met (this session; not the umbrella). The fail-closed inventory reports only the stub page: ident: inv k=1 u=1 a=1 leaks=0 / ident: inv-ok allow=stub | M1 ADR-087 + M2 ADR-088. _start never yanked (ident: start-stay) | ADR-042 / ADR-047 (amended) / ADR-089 |
| Written sponsor accept that the umbrella sentence is in scope | Met (2026-09-28 16:41 CEST, B3). Verbatim: “I accept the ADR-091 revision-3 sentence and scope as written. After M6, have ctos propose a short costed list of next milestones.” Relay path: sponsor wrote it in the DSO agent chat, DSO relayed it to ctos. | Was absent until 2026-09-28 (draft since 2026-09-26, D4; revisions 2 and 3 after G1/G2/G4). | ADR-091 (Accepted) → M6 flip |
§B is cleared (M6, 2026-09-28): the umbrella row is Verified, scoped to the accepted ADR-091 sentence and its limits. Original text: Until §B is cleared under honest probes (and _start policy is redesigned with sponsor scope if ever), the umbrella row stays non-claim.
Honesty
Since M6, say “EL0 isolated” only as the accepted ADR-091 sentence, with its limits. Many isolation miles are Verified; one mile alone is not the umbrella. (Before M6 this read: “umbrella EL0 isolation is Planned / non-claim until the ADR-055 checklist; many isolation miles are Verified but that is not ‘EL0 isolated.’”) Do not say:
- “EL0 isolated” wider than the ADR-091 sentence / “secure OS” / “hardened isolation complete”
- “EL0 isolated” from PAN alone / taken SError Verified /
_startyanked - Guest Linux / containers / immutable-OS marketing as if isolation were closed
Consequences
- Honesty ledger umbrella row cites this checklist; optional label “non-claim until checklist.”
- el0.md, roadmap P-SEC-3l, security.md, limits.md point here.
- Sponsor-facing one-pager (locks + reopen gates table): ADR-060.
- No
src/change. No Verified invent. Threat-model patch bump only.
ADR-056 — FAT16 readdir behind the thin VFS
- Status: Accepted (FAT16 root listing mile Verified when the serial / tests pass; Track A / app hosting still Planned for Linux/POSIX claims)
- Date: 2026-09-14
Context
ADR-028 landed virtio-blk + FAT16 read. ADR-050 deepened write / small create and deferred directory listing. Docs “new vs extend” prefer deepening the known small FS (FAT16) behind the thin VFS (ADR-027).
This mile is guest root directory listing: walk FAT16 root dirents and return thin-VFS paths (/probe, /hello, /fwr). Same path grammar. Fail-closed serial + #[test_case].
Not POSIX getdents / opendir / readdir(3). Not a directory tree. Not delete/unlink. Not virtual mounts. Keep write, /hello, cross-update, slot:ok.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Kernel + thin VFS vfs::readdir lists FAT16 root file names as /… paths. Serial fat: readdir / fat: entries n=<k>. Expect /probe, /hello, /fwr after the write-mile create. No new SVC this mile. | Deepens ADR-028/050. Same path grammar. Caps: READDIR_MAX (16). |
| H2 (rejected) | Claim POSIX getdents / opendir or add Linux dirent ABI. | Forbidden honesty. |
| H3 (rejected) | New SVC / libctos wrapper this PR. | Write mile used existing vfs::write. Listing is a kernel/VFS depth probe; EL0 SVC can follow later. |
| H4 (rejected) | Subdirectory trees, LFN, delete/unlink, or virtual mounts. | Out of scope; follow-ups. |
| H5 (rejected) | Route memfs through the same listing as a union namespace. | memfs stays flat named buffers; this mile is FAT16 root only. |
Decision
fat::readdir/vfs::readdir. Walk the FAT16 root, skip deleted / volume / LFN / directory entries, map 8.3 (blank extension) to thin-VFS paths via the inverse ofpath_to_83. CapREADDIR_MAX. Overflow →FsError::Full(fail-closed).- Probe after create.
observe_probelists the root after/fwrexists so the write-mile name is visible. Serialfat: readdirthenfat: entries n=<k>. Require/probe,/hello,/fwr. Keepfat: ok/ write / A9 slot markers. - No new SVC numbers. Public ABI stays 16–23. Not a POSIX product claim.
- Fail-closed smoke. Grep
fat: readdir/fat: entries/ keepfat: ok.#[test_case]covers listing + cap Full. - Honesty. Say “the guest listed FAT16 root directory entries through the thin VFS” only when serial / tests pass. Do not say: POSIX
getdents/opendir, FAT32, “supports FAT,” EL0 isolated, PAN, taken SError, Linux, containers. - NFR-10 text is revised in place (ID unchanged). Threat-model v1.31. Do not mint NFR-15+.
Consequences
- Code:
src/fat.rs(list_root / readdir / probe),src/vfs.rs(vfs::readdir),scripts/qemu-smoke.shgreps. - Docs: this ADR, filesystem.md, framework/filesystem.md, honesty ledger, roadmap note, threat-model v1.31, SUMMARY.
- Follow-ups (not this PR): EL0
fs_readdirSVC, LFN, subdirectories, virtual mounts, memfs listing. Delete/unlink: ADR-057.
ADR-057 — FAT16 delete behind the thin VFS
- Status: Accepted (FAT16 root delete mile Verified when the serial / tests pass; Track A / app hosting still Planned for Linux/POSIX claims)
- Date: 2026-09-14
Context
ADR-028 landed virtio-blk + FAT16 read. ADR-050 deepened write / small create. ADR-056 deepened root listing. Docs “new vs extend” prefer deepening the known small FS (FAT16) behind the thin VFS (ADR-027).
This mile is guest root file delete: mark a FAT16 root dirent deleted (0xE5), free its cluster chain, and expose the same path grammar through vfs::unlink. Fail-closed serial + #[test_case].
Not POSIX unlink / remove / rm. Not a directory tree. Not virtual mounts. Keep write, readdir, /hello, cross-update, slot: ok.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Kernel + thin VFS vfs::unlink deletes a FAT16 root file. Serial fat: delete / keep fat: ok. Probe creates /fdel, deletes it, confirms Missing + absent from readdir. No new SVC this mile. | Deepens ADR-028/050/056. Same path grammar. |
| H2 (rejected) | Claim POSIX unlink / remove or add a Linux unlink ABI. | Forbidden honesty. |
| H3 (rejected) | New SVC / libctos wrapper this PR. | Write/readdir miles used existing VFS; delete is a kernel/VFS depth probe; EL0 SVC can follow later. |
| H4 (rejected) | Subdirectory trees, LFN, multi-cluster shrink helpers, or virtual mounts. | Out of scope; follow-ups. |
| H5 (rejected) | Route only memfs through vfs::unlink this mile. | Product-depth train is FAT16; memfs unlink is incidental for the same entry point. |
Decision
fat::unlink/vfs::unlink. Look up the 8.3 root dirent, mark first byte0xE5, free the cluster chain (walk to EOC; single-cluster files this mile), drop open handles to that name. Missing →FsError::Missing(fail-closed).- Probe after readdir. Create
/fdel, writefat-dl,vfs::unlink("/fdel"), confirm open/has_nameMissing and root listing still has/probe,/hello,/fwrwithout/fdel. Serialfat: delete. Keepfat: ok/ write / readdir / A9 slot markers. - No new SVC numbers. Public ABI stays 16–23. Not a POSIX product claim.
- Fail-closed smoke. Grep
fat: delete/ keepfat: ok/ readdir / write.#[test_case]covers delete + missing. - Honesty. Say “the guest deleted a FAT16 directory entry through the thin VFS” only when serial / tests pass. Do not say: POSIX
unlink/remove, FAT32, “supports FAT,” EL0 isolated, PAN, taken SError, Linux, containers. - NFR-10 text is revised in place (ID unchanged). Threat-model v1.32. Do not mint NFR-15+.
Consequences
- Code:
src/fat.rs(mark deleted / free chain / probe),src/vfs.rs(vfs::unlink),scripts/qemu-smoke.shgreps. - Docs: this ADR, filesystem.md, framework/filesystem.md, honesty ledger, roadmap note, threat-model v1.32, SUMMARY.
- Follow-ups (not this PR): EL0
fs_unlinkSVC, LFN, subdirectories, virtual mounts, richer samples.
ADR-058 — Thin VFS prefix mounts (virtual mounts)
- Status: Accepted (prefix-mount mile Verified when the serial / tests pass; Track A / app hosting still Planned for Linux/POSIX claims)
- Date: 2026-09-14
Context
ADR-027 landed a thin VFS + memfs. ADR-028 added FAT16 behind the same open, with memfs-first then FAT fallback. Write / readdir / delete deepened that FAT backend (ADR-050 / ADR-056 / ADR-057).
Docs “new vs extend” list ctos-specific virtual mounts (prefix / tree mounts; not a new on-disk format) as a later optional depth mile (overview/filesystem.md). Routing today is an implicit overlay, not a mount table.
This mile is a small mount table that routes path prefixes to a backend. Keep A9 /hello, FAT write/readdir/delete, memfs create/write/read. Flat path grammar unchanged (/ + [a-z0-9_-], no extra /).
Not Linux VFS. Not POSIX mount(2) / mount namespaces / overlays. Not a new disk FS format.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Static prefix mount table; longest prefix wins. /mem (+ pinned A6 probe names) → memfs; / → FAT16. Serial vfs: mount / vfs: mounts. #[test_case] + fail-closed smoke. | Matches /hello / /probe on FAT and A6 memfs paths. No new SVC. |
| H2 (rejected) | Remap everything under /fat/… and /mem/… (tree mounts with an extra /). | Breaks flat path grammar and A9 /hello. |
| H3 (rejected) | Claim Linux mount(2) / vfsmount / mount namespaces. | Forbidden honesty (ADR-034 / ADR-031). |
| H4 (rejected) | Invent a new on-disk FS format for “virtual mounts.” | Overview forbids novel magic formats this mile. |
| H5 (rejected) | Keep memfs-first overlay only; document it as mounts without a table. | Not a real first cut; product-depth train wants an honest router. |
Decision
- Mount table.
vfs::MOUNTS:(prefix, backend)entries.vfs::resolve(path)picks the longest matching prefix. First cut:/kprobe,/eprobe,/mwprobe,/mem→ memfs (A6 habits + documented/memhome)/→ FAT16 (/probe,/hello,/fwr,/fdel, …)
- Route
create/open/unlinkby mount. No memfs→FAT fallback overlay.read/write/closestill key off the FAT fd tag.readdirstays FAT-root only (not a union listing). - Probe. Hello prints each
vfs: mount <prefix> -> <memfs|fat>thenvfs: mounts, then the existingfs:*memfs trip. Keepfat:*/slot: ok. - No new SVC numbers. Public ABI stays 16–23. Not POSIX
mount. - Fail-closed smoke. Grep
vfs: mount/vfs: mounts/ keepfs: ok/fat: ok/slot: ok.#[test_case]covers resolve + memfs-mount create off the FAT volume. - Honesty. Say “the guest routed path prefixes through a thin VFS mount table” only when serial / tests pass. Do not say: Linux VFS,
mount(2), containers, POSIX, “supports FAT,” EL0 isolated, PAN, taken SError. - NFR-10 text is revised in place (ID unchanged). Threat-model v1.33. Do not mint NFR-15+.
Consequences
- Code:
src/vfs.rs(mount table,resolve, routed create/open/unlink, probe),scripts/qemu-smoke.shgreps. - Docs: this ADR, filesystem.md, framework/filesystem.md, honesty ledger, roadmap note, threat-model v1.33, SUMMARY, architecture glance.
- Follow-ups (not this PR): tree mounts with an extra
/, runtime mount/unmount SVC, memfs readdir, stripping the mount prefix inside backends.
ADR-059 — Second freestanding sample (fs-libctos) + catalog expansion
- Status: Accepted (sample + smoke Verified when serial / tests pass; product freestanding app hosting remains Verified under ADR-048/052 — do not reopen)
- Date: 2026-09-14
Context
Track A / A8 documented rebuild recipes for in-tree samples (what-can-run.md, user/README.md). Today only one freestanding EL0 sample exists: user/hello-libctos (UART + yield). Docs called out the gap: a libctos hello that calls fs_open was not in the tree. Wrappers exist (SVC 16–23); the Verified EL0 VFS trip was the kernel /eprobe trampoline.
After ADR-058, longest-prefix mounts route /mem (+ A6 names) → memfs and / → FAT16. Flat path grammar unchanged (/ + [a-z0-9_-], no extra /).
This mile adds a second freestanding EL0 sample that exercises libctos VFS, publishes it on FAT beside /hello, expands the catalog, and fail-closes smoke on new markers. Honesty: not POSIX, not Linux, not getdents from EL0.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | New crate user/fs-libctos mirroring hello target/linker/build-std. Create/write/read/close /memdemo (memfs via /mem prefix). FAT slot /fsdemo (8.3 FSDEMO). Kernel loads after A9 hello. Serial libctos: fs-hi / libctos: fs-ok + fsdemo: ok. ADR + catalog docs. | Cleanest fit; keeps /hello intact. |
| H2 (rejected) | Teach hello-libctos to call fs_open in-place. | Collapses two recipes into one; muddies UART-only vs VFS samples. |
| H3 (rejected) | Use /mem/... tree path or claim POSIX open. | Breaks flat grammar / honesty. |
| H4 (rejected) | Replace /hello with the VFS sample. | Breaks A9 cross-update and existing markers. |
| H5 (rejected) | Add EL0 getdents this mile. | Not wired; do not invent. |
Decision
- Sample crate.
user/fs-libctos: sameaarch64-ctos-user/linker.ld/build-stdas hello.mainprintslibctos: fs-hi, yields, thenfs_create/fs_write/fs_close/fs_open/fs_read/fs_closeon/memdemo, thenlibctos: fs-ok. Fail path printslibctos: fs-failand exits non-zero. - Path choice (hunch → verify).
/memdemolongest-prefix-matches/mem→ memfs under ADR-058. Flat grammar forbids/mem/foo. The ELF lives on FAT/fsdemo; the in-app file is memfs. - Publish.
build.rsbuilds and publishestarget/fs-libctos.elf.scripts/mkfat16.py --app2writes 8.3FSDEMO. Keep/hello+slot: ok. - Kernel probe.
src/fsdemo.rsloads FAT/fsdemo,ERETs vialoader::run_image_expecting(last uart len 15). Serialfsdemo: fat/fsdemo: mapped/fsdemo: ok. Runs after the A9 slot probe. - Smoke. Fail-closed greps for the new markers; preserve all existing greps (hello, FAT write/readdir/delete, mounts, A9 cross-update, …).
- Docs. This ADR; honesty ledger evidence-only;
user/README.md; what-can-run.md; apps-today.md; SUMMARY. Threat-model v1.34 (NFR-10 text in place). - EL0 exec pages. RX user pages are fetch-only for EL0 (existing
l3_page_el0_exec). The sample may pass.rodatapointers to SVCs (EL1 copy) but must notLDRthem at EL0; read-back compare uses immediates. - Honesty. Do not claim: EL0 isolated, PAN enable, taken SError, Linux/POSIX/containers, immutable-OS marketing, or reopen “app hosting is done.”
Consequences
- Code:
user/fs-libctos/,build.rs,scripts/mkfat16.py,scripts/qemu-aarch64.sh,scripts/qemu-smoke.sh,src/fsdemo.rs,src/loader.rs(run_image_expecting),src/main.rs. - Docs: this ADR + catalog / ledger / threat-model v1.34.
- Follow-ups (not this PR): richer samples, EL0 readdir when wired, tree mounts with an extra
/.
ADR-060 — Isolation leftovers closure checklist for sponsor
- Status: Accepted (docs). Updated 2026-09-19 for ADR-079 / ADR-080 PAN CPU reopen + enable, and ADR-081 / ADR-082 taken-SError re-probe (still blocked); 2026-09-25 ADR-084 B2 free-runner spike (blocked). Sponsor-facing closure checklist for isolation leftovers locked by ADR-047, ADR-053, ADR-054, ADR-055, and #98. Does not invent Verified for taken SError or “EL0 isolated.” Does not reopen Guest Linux / containers / immutable-OS marketing.
- Date: 2026-09-15
- Follow-on (2026-09-28, M6): B3 met: the sponsor accepted the ADR-091 revision-3 sentence and scope in writing (2026-09-28 16:41 CEST). ADR-091 is Accepted. The umbrella row below is Verified, only as that sentence. The other rows and locks are unchanged: stock TCG still parks SError, and
_startis never yanked. - Follow-on (2026-09-27, ADR-092): sponsor chose option (b) on ADR-091. G1 (fail-closed EL0-reachability walk) and G2 (EL0 write permission fault) are closed; new gap G4 (syscall copy helpers skip the EL0 permission check) is recorded. ADR-091 revision 2 goes back to the sponsor. The umbrella row below is unchanged: Planned / non-claim.
- Follow-on (2026-09-26, M5): draft accept ADR-091 (Draft / pending sponsor accept) proposes the exact umbrella sentence and scope, plus gaps G1–G3 for the sponsor to weigh. The umbrella row below is unchanged: Planned / non-claim.
- Follow-on (2026-09-26, M4): sponsor D1 accepted. The taken-SError evidence class for the umbrella is the B1 pin job (now unconditional on every push/PR) plus the B2-P one-off KVM run (ADR-090). The stock-TCG hard-stop in the row below still holds for stock QEMU.
- Follow-on (2026-09-26, M3): sponsor D2 accepted. The umbrella’s identity row now reads “full identity teardown except the documented
_startstub page” (ADR-089) and is Met on this session’s evidence (ADR-087/088). Taken SError (D1 → ADR-090) and the sponsor accept (D4 → ADR-091 draft) are still open. - Follow-on (2026-09-26): B2-P taken SError Verified opt-in on Graviton3 KVM (ADR-085). Checklist audit + gap list + milestone plan + sponsor decisions D1–D5: ADR-086. The umbrella stays non-claim, and the locks below are unchanged.
- Numbering note: next free after ADR-059 on
main@404d74c. FAT/fs-libctos sample already took ADR-059; this ADR is 060 (no collision with a parallel FAT-sample ADR-060).
Context
Sponsor / DSO asked for a single clear table that closes the leftover decision surface after the isolation-gap locks (#98 / ADR-053–055): what is locked, what would be required for an honest Verified, what is blocked on the default smoke machine (virt + -cpu cortex-a57), and the reopen gates. This ADR cites those decisions; it does not rewrite them.
CloudAgent HELD. Docs-only. Prefer the amd64 workstation / agent-box for docs build. Do not self-merge (ADR-002).
Decision (cite locks; do not invent Verified)
The sponsor closure checklist is the table below. Status labels are locks and reopen gates only.
Sponsor isolation leftovers closure checklist
| Leftover | Locked status | What’s required for honest Verified | What’s blocked on a57+virt today | Reopen gate |
|---|---|---|---|---|
| Taken lower-EL SError while standing | Deferred / non-goal (hard-stopped) (ADR-053 / ADR-081 / ADR-082 / ADR-047 §1). el0: serror-park stays Verified. A-clear path = dormant prep. | Honest host inject that delivers async SError on the accepted smoke machine; serial el0: serror under EXPECT; new ADR citing working evidence. | QMP inject-nmi / HMP nmi → machine does not provide NMIs on QEMU 10 virt + -cpu cortex-a76 (re-probed 2026-09-19; also a57 / GICv3 / virtualization=on / max). No silent invent. FEAT_NMI ≠ SError. KVM out of scope for TCG smoke. | ADR-081 Decision §4 B1/B2/B3: (B1) upstream/pinned QEMU virt TYPE_NMI→ARM_CPU_SERROR — research ADR-082: no shippable pin 2026-09-19; wait upstream or reviewed local pin; (B2) sponsor smoke-machine ADR with CI-runnable honest inject — spike ADR-084 2026-09-25: blocked on free hosted runners (no arm64 KVM/HVF/WHPX); needs sponsor choice (self-hosted arm64 KVM / paid bare metal / wait); (B3) permanent non-goal. B1 opt-in pin Verified (ADR-083). |
| PAN enable (PSTATE.PAN + EL1-vs-EL0 fault) | Verified (ADR-080 / #125 mile 2). Default -cpu cortex-a76 → pan: present + pan: enabled + pan: el1-fault. Historical a57 pan: absent (ADR-026 / ADR-054 / ADR-079). | Met: EL1 load of EL0-accessible page faults with SPSR.PAN. | a57 alone cannot enable (historical). | ADR-054 gate 1–4 met (079+080). |
| Umbrella “EL0 isolated” (P-SEC-3l) | Verified only as the accepted ADR-091 sentence (M6, 2026-09-28; B3 met 16:41 CEST; limits: one core, TCG default smoke machine, not side-channel, not real hardware, not a certification). Was: Planned / non-claim until checklist (ADR-055 / ADR-047 §4 / #98). Specific Verified miles ≠ this row. | Every ADR-055 §B Unmet row Met under its own ADR + probes, plus written sponsor accept that the umbrella sentence is in scope (ADR-048-style — not this file). | Under current constraints: no PAN enable on a57; _start stays; taken SError hard-stopped; umbrella sponsor accept absent until 2026-09-28 (now met, B3). Remaining miles after #98 = those §B blockers (not more Track A ladder work on virt+a57). | Path to Verified: clear §B under honest probes (and redesign boot if _start ever leaves identity — sponsor scope). Path to permanent non-claim: leave locks as-is; do not round Verified miles into the umbrella. |
Yank _start (2026-09-26: the stub page is the one documented identity exception, ADR-089; all other identity is torn, see ADR-087 / ADR-088) | Decided: never while QEMU -kernel needs 0x4008_0000 (ADR-042 / ADR-047 §3). ident: start-stay Verified honesty. | N/A under current boot contract — not a Verified target. | Boot stub must stay mapped at entry. | Only a future sponsor-scoped boot redesign ADR. Never yank _start in the meantime. |
| Guest Linux / containers / immutable-OS marketing | Still out (ADR-029 / ADR-036 / ADR-055 honesty). | N/A — out of scope for this checklist. | Do not reopen via isolation docs. | Separate product ADR + sponsor scope if ever. |
Explicit non-actions (this ADR)
- Do not invent Verified for taken SError / “EL0 isolated.”
- Do not silent
-cpuswitch (ADR-079 documented the cortex-a76 change). - Do not yank
_start. - Do not reopen Guest Linux / containers / immutable-OS marketing.
- Do not require
el0: serrorin smoke. Smoke does requirepan: enabled/pan: el1-fault(ADR-080).
Verified ladder vs unmet §B (glance)
flowchart TD V["Verified miles<br/>standing · tears · IRQ/FIQ<br/>PAN ID · SError park · start-stay"] B1["§A Met: PAN enable<br/>ADR-079 + ADR-080"] B2["§B Met: taken SError<br/>ADR-090 evidence class"] B3["§B Met: identity except _start stub<br/>ADR-089"] B4["§B Met: sponsor umbrella accept<br/>B3, 2026-09-28"] U["Umbrella EL0 isolated<br/>Verified: ADR-091 sentence only"] V -.-> U B1 --> U B2 --> U B3 --> U B4 --> U
≤12 nodes. Miles alone are not the umbrella. All §B rows are Met (M6, 2026-09-28); the claim is only the ADR-091 sentence.
Honesty
Say: “isolation leftovers are closed as locks + reopen gates (ADR-060); PAN enable is Verified (ADR-080); taken SError stays blocked after ADR-081 re-probe on stock QEMU; “EL0 isolated” is Verified only as the accepted ADR-091 sentence (M6).” Do not say:
- taken lower-EL SError is Verified
- “EL0 isolated” from PAN alone
- “EL0 isolated” wider than the ADR-091 sentence / “secure OS” / “hardened isolation complete”
_startwas yanked / the kernel moved- Guest Linux / containers / immutable-OS marketing as if isolation closed
Consequences
- Docs: this ADR is the sponsor one-pager; ADR-055 remains the detailed §A/§B evidence table; ADR-053 / ADR-054 remain the hard locks.
- Cross-links: honesty-ledger.md, el0.md, security.md (threat-model v1.35), roadmap.md, SUMMARY.md, limits.md.
- Code: unchanged. No
src/edits. CloudAgent HELD. Do not self-merge.
ADR-061 — Third freestanding sample (fat-libctos) — FAT via thin VFS
- Status: Accepted (sample + smoke Verified when serial / tests pass; product freestanding app hosting remains Verified under ADR-048/052 — do not reopen)
- Date: 2026-09-15
Context
ADR-059 added a second freestanding EL0 sample that exercises libctos VFS on memfs (/memdemo via ADR-058 /mem prefix). Product gap remains: no freestanding sample that opens/reads a FAT path from user mode through the same thin VFS (/ → FAT16). Kernel /probe (fat-hi) write/readdir/delete are already Verified at the EL1 path.
This mile adds a third freestanding EL0 sample that reads FAT /probe via libctos, publishes it on FAT beside /hello and /fsdemo, expands the catalog, and fail-closes smoke on new markers. Honesty: not POSIX, not Linux, not “EL0 isolated.”
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | New crate user/fat-libctos. Open/read/close FAT /probe expecting fat-hi. FAT slot /fatdemo (8.3 FATDEMO). Kernel loads after ADR-059 fsdemo. Serial libctos: fat-hi / libctos: fat-ok + fatdemo: ok. ADR + catalog docs. | Cleanest deeper product; keeps /hello + /fsdemo intact. |
| H2 (rejected) | Teach fs-libctos to also open /probe. | Collapses memfs vs FAT recipes; muddies ADR-059 markers. |
| H3 (rejected) | EL0 create/write/read a new FAT root name this mile. | Prefer existing /probe (already restored to fat-hi before this probe). Write-to-FAT from EL0 is already routed but not required for the product claim. |
| H4 (rejected) | Replace /fsdemo or /hello. | Breaks A9 / ADR-059 cross-update and markers. |
| H5 (rejected) | Claim POSIX open / Linux VFS / “EL0 isolated.” | Honesty forbid. |
Decision
- Sample crate.
user/fat-libctos: sameaarch64-ctos-user/linker.ld/build-stdas hello/fs.mainprintslibctos: fat-hi, yields, thenfs_open/fs_read/fs_closeon/probe, thenlibctos: fat-ok. Fail path printslibctos: fat-failand exits non-zero. - Path choice.
/proberesolves to FAT16 under ADR-058 (/→ FAT). Payload bytes are the existing A7fat-hi(restored byfat::observe_probebefore this slot runs). - Publish.
build.rsbuilds and publishestarget/fat-libctos.elf.scripts/mkfat16.py --app3writes 8.3FATDEMO. Keep/hello+/fsdemo+slot: ok+fsdemo: ok. - Kernel probe.
src/fatdemo.rsloads FAT/fatdemo,ERETs vialoader::run_image_expecting(last uart len 16). Serialfatdemo: fat/fatdemo: mapped/fatdemo: ok. Runs after the ADR-059 fsdemo probe. - Smoke. Fail-closed greps for the new markers; reject
libctos: fat-fail; preserve all existing greps. - Docs. This ADR; honesty ledger evidence-only;
user/README.md; what-can-run.md; apps-today.md; SUMMARY. Threat-model v1.36 (NFR-10 text in place). - EL0 exec pages. RX user pages are fetch-only for EL0. The sample may pass
.rodatapointers to SVCs (EL1 copy) but must notLDRthem at EL0; read-back compare uses immediates. - Honesty. Do not claim: EL0 isolated, PAN enable, taken SError, Linux/POSIX/containers, immutable-OS marketing, or reopen “app hosting is done.” Never yank
_start. No silent-cpu.
Consequences
- Code:
user/fat-libctos/,build.rs,scripts/mkfat16.py,scripts/qemu-aarch64.sh,scripts/qemu-smoke.sh,src/fatdemo.rs,src/main.rs. - Docs: this ADR + catalog / ledger / threat-model v1.36.
- Follow-ups (not this PR): EL0 FAT create/write sample, EL0 readdir when wired, tree mounts with an extra
/.
ADR-062 — Fourth freestanding sample (yield-libctos) — cooperative yield rounds
- Status: Accepted (sample + smoke Verified when serial / tests pass; product freestanding app hosting remains Verified under ADR-048/052 — do not reopen)
- Date: 2026-09-16
Context
ADR-022 / hello prints libctos: hi, yields once, then libctos: ok. ADR-059 and ADR-061 deepened the catalog on VFS/FAT. Product gap remains: no freestanding sample that exercises several cooperative yield_now() rounds from standing EL0 (still single-task; not preemption).
This mile adds a fourth freestanding EL0 sample that yields across multiple uart beats, publishes it on FAT beside /hello / /fsdemo / /fatdemo, expands the catalog, and fail-closes smoke on new markers. Honesty: not POSIX, not Linux, not “EL0 isolated,” not preemption, not a process table.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | New crate user/yield-libctos. Print libctos: yld-hi, then several libctos: beat across yield_now() rounds, then libctos: yld-ok. FAT slot /yldemo (8.3 YLDEMO). Kernel loads after ADR-061 fatdemo. Serial greps + yldemo: ok. ADR + catalog docs. | Cleanest distinct fourth product; keeps prior three intact. |
| H2 (rejected) | Teach hello-libctos to loop yields. | Collapses A2 markers; muddies hello smoke greps. |
| H3 (rejected) | Preemptive timer slice / multi-task EL0 / process table this mile. | Out of honesty scope; cooperative only. |
| H4 (rejected) | Replace /hello or another slot. | Breaks A9 / ADR-059 / ADR-061 cross-update and markers. |
| H5 (rejected) | Claim POSIX / Linux / “EL0 isolated” / preemption. | Honesty forbid. |
Decision
- Sample crate.
user/yield-libctos: sameaarch64-ctos-user/linker.ld/build-stdas hello/fs/fat.mainprintslibctos: yld-hi, then ≥3libctos: beatlines each afteryield_now(), then one more yield andlibctos: yld-ok. - Publish.
build.rsbuilds and publishestarget/yield-libctos.elf.scripts/mkfat16.py --app4writes 8.3YLDEMO. Keep/hello+/fsdemo+/fatdemo+ their*: okmarkers. - Kernel probe.
src/yldemo.rsloads FAT/yldemo,ERETs vialoader::run_image_expecting(last uart len 16 forlibctos: yld-ok\n), and requiressyscall::yield_count() >= 3. Serialyldemo: fat/yldemo: mapped/yldemo: ok. Runs after the ADR-061 fatdemo probe. - Smoke. Fail-closed greps for the new markers (≥3
libctos: beat); preserve all existing greps. - Docs. This ADR; honesty ledger evidence-only;
user/README.md; what-can-run.md; apps-today.md; SUMMARY. Threat-model v1.37 (NFR-10 text in place). - Honesty. Do not claim: EL0 isolated, PAN enable, taken SError, Linux/POSIX/containers, preemption, multi-task EL0, process table, immutable-OS marketing, or reopen “app hosting is done.” Never yank
_start. No silent-cpu.
Consequences
- Code:
user/yield-libctos/,build.rs,scripts/mkfat16.py,scripts/qemu-aarch64.sh,scripts/qemu-smoke.sh,src/yldemo.rs,src/main.rs. - Docs: this ADR + catalog / ledger / threat-model v1.37.
- Follow-ups (not this PR): preemptive scheduling (separate ADR), multi-task EL0, EL0 FAT create/write sample.
ADR-063 — Track N — network foundation scope (virtio-net first frame)
- Status: Accepted (docs scope; Planned until N1 probes — do not invent Verified)
- Date: 2026-09-16
Context
Sponsor treats networking as a new track (Track N), not a side edit on filesystem / freestanding-sample PRs. Today the guest has virtio-mmio + virtio-blk (ADR-028, src/virtio.rs) and no NIC / no virtio-net (limits.md). Track A freestanding hosting is Verified under ADR-048 / ADR-052. Track B Linux-compat research ended never (ADR-036).
This mile (N0) is document-first: name the track, lock foundation scope and non-goals, publish a mile ladder, and wire SUMMARY / roadmap / honesty ledger / limits. No virtio-net driver code in this PR unless trivial scaffolding is needed for honesty — prefer docs-only. File presence of this ADR is not a network probe.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Open Track N (network) as a separate ladder from Track A/B. N0 = this ADR (scope). Foundation target: QEMU virt + virtio-net (mmio); discover device; TX/RX one raw Ethernet frame; serial markers + smoke + QEMU -netdev / -device virtio-net-device; measure-first; fail-closed. | Matches sponsor “new track” + honesty. Reuses virtio-mmio patterns from blk. |
| H2 (rejected) | Fold networking into Track A FS / sample PRs as a side edit. | Sponsor forbids; muddies A7/FAT/smoke ownership. |
| H3 (rejected) | Ship a TCP/UDP stack, BSD sockets, DHCP/DNS product, or “has networking” marketing this mile. | Honesty fail. Locked non-goals until a new ADR + sponsor. |
| H4 (rejected) | Claim Verified for link bring-up or first frame from this docs PR. | No guest probe yet. Status stays Planned. |
| H5 (rejected) | virtio-pci-only or Wi‑Fi stories as the foundation path. | Keep learning scope on QEMU virt + virtio-mmio (same as blk). |
Decision
-
Track name. Track N (network). Separate from Track A (freestanding app hosting) and Track B (Linux-compat research). Subordinate to principles.md. Hub: track-n.md.
-
In scope for foundation (N1 and immediate follow-ons under this ADR’s intent).
- QEMU
virt+ virtio-net over virtio-mmio (scan transports like blk; Device ID for net). - Discover the device; program queues enough to TX and RX one raw Ethernet frame.
- Serial markers + fail-closed
scripts/qemu-smoke.shgreps. - Host QEMU flags:
-netdev+-device virtio-net-device(exact args named in the N1 PR). - Measure-first (NFR-07): CNTPCT or similar only when a path exists; no invented benches.
- Fail-closed: missing markers /
probe missedfail smoke. Do not break existing FAT / blk / slot / sample greps.
- QEMU
-
Explicit non-goals (locked until a new ADR + sponsor).
- TCP/UDP stack as product.
- BSD sockets API as product.
- DHCP / DNS as product features.
- Wi‑Fi.
- virtio-pci-only stories as the foundation path.
- Linux network stack / “runs Linux networking.”
- Marketing “has networking” / “network-ready OS.”
- EL0 net SVC / ABI until after link bring-up is Verified (N1+) — unlocked by ADR-068.
-
Mile ladder.
ID Work Probe that closes it Status N0 This ADR — scope, non-goals, wiring Docs-build; ledger Planned row; no Verified invent Accepted (docs) — this PR N1 Link bring-up / first frame Serial markers for discover + TX and/or RX one raw frame; smoke greps; QEMU -netdev/-device; keep blk/FAT smoke greenVerified via ADR-066 when tip prints net: okN2 ARP + ICMP ping Named markers + fail-closed smoke; still no sockets product Verified via ADR-067 when tip prints net: ping-okN3 Transport / sockets Only with new sponsor scope + new ADR Locked out until that ADR N4 Freestanding sample using net SVCs After N1+ Verified; catalog honesty like ADR-059/061/062 Verified via ADR-068 when tip prints libctos: net-ok/netdemo: ok -
Reuse. Reuse virtio-mmio patterns from blk (ADR-028,
src/virtio.rs): transport scan, split virtqueue, poll used ring, identity-PA DMA + cache maintenance. Do not break FAT/blk smoke or A9/slot/sample markers. Prefer extending virtio carefully over a second transport story. -
Honesty.
- Do not invent Verified for networking from this PR.
- Ledger status for Track N foundation / first frame stays Planned until N1 probes pass.
- Do not claim: TCP/UDP, sockets, DHCP/DNS product, Wi‑Fi, Linux net stack, “has networking,” EL0 net ABI, “secure OS,” or reopen Track B Linux-compat.
- No silent
-cpu. Never yank_start. Author does not self-merge (ADR-002). - CloudAgent remains HELD; N1 evidence tags agent-box or the amd64 workstation as available.
-
Docs wiring (this PR). SUMMARY + ADR list; roadmap Track N section; track-n.md; honesty-ledger Planned row; limits.md (“Track N Planned — see ADR”); index one-liner; threat-model light bump (security.md v1.38 — I/O surface names Track N Planned, still no stack).
-
No new FR/NFR IDs. Frozen set stays fr-nfr.md. Cite existing NFR-06 / NFR-07 / NFR-10 / FR-14 as needed.
Consequences
- Docs: this ADR; track-n.md; SUMMARY; roadmap; honesty ledger; limits; index; threat-model v1.38.
- Code: none in N0 (docs-only). N1 may add virtio-net guest code + QEMU netdev in a later PR.
- Follow-ups: N1 link bring-up / first frame PR; optional N2 ARP+ICMP; N3+ only with new sponsor ADR.
ADR-064 — FAT16 multi-cluster grow (write depth)
- Status: Accepted (multi-cluster grow mile Verified when the serial / tests / EL0 markers pass; Track A product claims unchanged; Track N untouched)
- Date: 2026-09-16
Context
ADR-050 landed guest FAT16 write + small create, explicitly refusing multi-cluster grow (cap: one cluster / FILE_MAX 256). ADR-057 frees a cluster chain (already walked to EOC). ADR-061 added an EL0 FAT read sample on /probe. Reads of multi-cluster host ELFs (/hello, …) already walked the FAT chain; guest write still stopped at one cluster.
Docs “new vs extend” prefer deepening the known small FS (FAT16) behind the thin VFS. This mile is honest multi-cluster file grow: allocate a FAT chain, update dirent size, write across a cluster boundary, prove with kernel serial + unit tests + EL0 chunked SVC write/read-back.
Not exFAT. Not FAT32. Not a POSIX full write API. Not Track N / virtio-net (ADR-063 landed on main as docs-only N0 — do not collide; this mile is 064 only).
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Extend fat::write_at to allocate additional clusters when end crosses a cluster boundary; raise FILE_MAX to 1024 (enough for two 512-byte clusters). Kernel probe /fgrow (600 bytes). EL0 fat-libctos creates /egrow and chunked-writes via existing SVC 19–23 (FS_IO_MAX 64). Serial fat: grow / libctos: fat-grow. | Deepens ADR-050. Same path grammar. Fail-closed. |
| H2 (rejected) | Claim POSIX write / pwrite / sparse files / exFAT. | Honesty forbid. |
| H3 (rejected) | New SVC numbers or a second write ABI. | Existing fs_create/fs_write/fs_read suffice. |
| H4 (rejected) | Only kernel probe; no EL0 proof. | Product-depth train wants EL0 via existing SVCs. |
| H5 (rejected) | Touch Track N / virtio-net. | Separate ladder; out of scope. |
Decision
FILE_MAX= 1024. Shared memfs/FAT cap. Was 256 (below one 512-byte cluster); multi-cluster grow is impossible under that ceiling. Still small; not a product volume claim.extend_chain/ multi-clusterwrite_at. Walk the existing chain; allocate free clusters, link prior→new→EOC, zero new clusters; write payload across cluster boundaries. Refuseend > FILE_MAXand empty writes (TooBig). No sparse holes (off > size→BadHandle). Still assumesspc=1(one sector per cluster) this mile.- Kernel probe
/fgrow. After delete probe: create, write 600 patterned bytes, confirm FATchain_len ≥ 2, VFS read-back, boundary bytes differ. Serialfat: grow. Keepfat: ok/ write / readdir / delete / A9. - EL0 via existing SVCs. Extend
user/fat-libctos: after/proberead, create-or-open/egrow, write 600 bytes in 64-byte SVC chunks, read-back boundary bytes with immediates. Seriallibctos: fat-growthenlibctos: fat-ok. Fail →libctos: fat-fail. - Fail-closed smoke + tests. Grep
fat: grow/libctos: fat-grow; reject missing.#[test_case]fat16_multi_cluster_grow+ over-FILE_MAXrefuse. - Honesty. Say “the guest grew a FAT16 file across a cluster boundary through the thin VFS” only when markers pass. Do not say: POSIX write API, exFAT/FAT32, “supports FAT,” app hosting reopen, EL0 isolated, PAN, taken SError, networking.
- NFR-10 text revised in place (ID unchanged). Threat-model v1.39 (slice after #109 / ADR-063 threat-model v1.38). Do not mint NFR-15+.
Consequences
- Code:
src/fat.rs(extend_chain / write_at / grow probe),src/vfs.rs(FILE_MAX),user/fat-libctos/,scripts/qemu-smoke.sh. - Docs: this ADR, filesystem overview/stance, honesty ledger, SUMMARY, threat-model v1.39, light fr-nfr NFR-10 note.
- Follow-ups (not this PR):
spc>1, shrink-on-truncate, exFAT, POSIX write API claim, Track N N1.
ADR-065 — FAT16 vs memfs read CNTPCT pair (measure-first)
- Status: Accepted (compared CNTPCT pair on one boot — QEMU TCG lab measurement, not a published bench).
- Date: 2026-09-16
Context
ADR-051 landed a same-boot FAT vs memfs write CNTPCT pair (perf: fat-write / perf: memfs-write / perf: fs-write-delta). Existing performance probes also cover loop CNTPCT, IRQ delta, boot-delta, ELF size, and the A9 slot pair (ADR-046). Perf slice 3 asks for another measure-first compared pair around a known path. The natural symmetric cut is the small FAT /probe read versus a memfs read of the same bytes on the same boot.
Do not invent a latency SLA, percent, or “faster/slower” claim. Do not add a criterion crate. Keep fat: read / fat: write / fat: ok / A9 slot markers and the ADR-051 write pair. QEMU TCG jitter is one lab — not SPEC. Do not touch Track N / virtio-net.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Time the FAT VFS read of /probe (fat-hi) with CNTPCT → perf: fat-read ticks=<n>. On the same boot, seed memfs /mrprobe with the same bytes and time vfs::read → perf: memfs-read ticks=<n>. Print perf: fs-read-delta fat=<a> memfs=<b> (raw ticks). | Honest compared pair. Not a bench. |
| H2 (rejected) | Publish a percent / “FAT is slower than memfs” / criterion bench. | NFR-07 forbids invented benches. |
| H3 (rejected) | Time open+read+close as one number and call it “read latency.” | Too wide; pair the read of the same payload (open outside the window). |
| H4 (rejected) | Only print FAT ticks without a memfs compare. | Prefer a same-boot pair when feasible (same shape as ADR-046 / ADR-051). |
| H5 (rejected) | Time SVC yield / memcpy instead of FAT read this mile. | Useful later; FAT read is the direct follow-on to ADR-051. |
Decision
- FAT read sample. During the existing
/probeVFS read proof, sampleCNTPCTaround the singlevfs::readthat expectsPROBE_BYTES(fat-hi). Store ticks; fail closed if the counter does not advance. - Memfs compare. After FAT probes that store the FAT sample,
vfs::create/opena memfs path (/mrprobe— added to the ADR-058 prefix table so it does not fall through to FAT/), write the same bytes outside the timed window, reopen, and sampleCNTPCTaroundvfs::read. Fail closed if ticks are zero or bytes mismatch. - Markers. Serial:
perf: fat-read ticks=<n>,perf: memfs-read ticks=<n>,perf: fs-read-delta fat=<a> memfs=<b>. No ratio percent; no “faster/slower” words on the serial line. Missed samples printperf: fat-read missed/perf: memfs-read missed. - Existing markers unchanged. Keep
fat: read/fat: write/ ADR-051 write pair / A9slot:*/ ADR-046 pair. Grow / readdir / delete / Track N untouched. - Smoke.
scripts/qemu-smoke.shgreps the new markers and rejectsfs-read-deltalines that claim percent/faster/slower. - Honesty. This is a measurement pair on QEMU
virtTCG. It is not a latency budget, not a product KPI, and not “FAT read cost is X.” Record Verified only with serial evidence on a named tip. Do not claim app hosting done, EL0 isolated, POSIX, or FAT32.
Consequences
- Code:
src/fat.rs(CNTPCT around FAT read + memfs compare + markers); smoke greps;#[test_case]that both samples are> 0. - Docs: this ADR, performance.md, measure.md, honesty ledger, roadmap
P-PERF-7, threat-model light bump. - Follow-ups (not this PR): yield/SVC CNTPCT pair, memcpy baseline, multi-cluster read cost — only after a probe shows a need.
ADR-066 — Track N / N1 — virtio-net first Ethernet frame
- Status: Accepted (N1 runtime — Verified only when this tip’s
qemu-smokeprintsnet: ok; else Documented/partial) - Date: 2026-09-16
Context
ADR-063 opened Track N (docs N0). This mile is N1: discover virtio-net on virtio-mmio, program RX+TX queues enough to send and receive one raw Ethernet frame, with serial markers, fail-closed smoke, and host QEMU -netdev / -device virtio-net-device. Reuse virtio-mmio patterns from blk (ADR-028, src/virtio.rs). Do not break FAT/blk/slot/sample/perf greps.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Discover Device ID 1 on the same mmio scan as blk. Setup queue 0 (RX) + queue 1 (TX). Post one RX buffer. TX a broadcast ARP request for QEMU SLIRP gateway 10.0.2.2 from guest 10.0.2.15. Wait for TX used.idx, then RX ARP reply from SLIRP. Markers + smoke. Host: -netdev user,id=net0 -device virtio-net-device,netdev=net0. | Deterministic on GHA ubuntu (no external network). Proves both directions. |
| H2 (rejected as primary) | TX-only: prove used.idx after TX without RX. | Allowed as a fallback minimum by ADR-063, but SLIRP ARP reply is available and stronger. Prefer both. |
| H3 (rejected) | Depend on DHCP/DNS product, outbound Internet, or a second peer VM. | Flaky / product creep. Fail-closed CI must not need external net. |
| H4 (rejected) | Inject a crafted RX via QMP / filter without guest TX. | Extra host machinery; ARP round-trip is enough and exercises TX+RX. |
| H5 (rejected) | virtio-pci-only, Wi‑Fi, TCP/UDP stack, BSD sockets, EL0 net SVC this mile. | Locked non-goals (ADR-063). |
| H6 (rejected) | Invent Verified without net: ok on a named tip. | Honesty. |
Decision
-
Transport. Same virtio-mmio scan (
0x0a000000 + n×0x200, 32 slots). Device ID 1 = net. Accept legacy v1 (QueuePFN) and modern v2 (desc/avail/used +QueueReady), matching blk. Identity-PA DMA + whole-objectdc civaccache maintenance. Pollused.idx. No virtio IRQ this mile. -
Queues. Queue 0 = receiveq, queue 1 = transmitq. Separate 4 KiB-aligned
.bssDMA objects (legacy page layout). 10-bytevirtio_net_hdr(noVIRTIO_NET_F_MRG_RXBUF). NegotiateVIRTIO_NET_F_MACwhen offered; read MAC from config bytes 0–5. -
Honest probe (CI-deterministic).
- Post one RX WRITE buffer (hdr+frame).
- TX Ethernet+ARP request: who-has 10.0.2.2 tell 10.0.2.15, src = guest MAC, dst = broadcast.
- QEMU user netdev SLIRP answers ARP for its gateway locally — no Internet, no DHCP client, no second VM.
- Require TX completion and an ARP reply whose spa is
10.0.2.2.
-
Host QEMU args (exact). Wired into
scripts/qemu-aarch64.shandscripts/qemu-serial-inject.py(hello smoke path):-netdev user,id=net0 -device virtio-net-device,netdev=net0Kept alongside existing
-drive … -device virtio-blk-device,drive=hd0. Host flags without guest markers are not a probe. -
Serial markers.
net: virtio— discovered + queues programmednet: mac xx:xx:xx:xx:xx:xx— config MACnet: tx— TX used ring advanced for the ARP requestnet: rx— RX used ring delivered an ARP reply from10.0.2.2net: ok— both directions proven- Miss →
net: probe missed(fail-closed) - Optional cheap CNTPCT:
perf: net-tx ticks=<n>/perf: net-rx ticks=<n>when the counter advances (lab only, not a bench)
-
Smoke.
scripts/qemu-smoke.shgreps the markers above and rejectsnet: probe missed. Must keep existingblk:*/fat:*/ slot / sample / perf greps green. -
Non-goals restated (locked). TCP/UDP product; BSD sockets; DHCP/DNS product; Wi‑Fi; virtio-pci-only foundation; Linux net stack; “has networking” marketing; EL0 net SVC/ABI. N2 ICMP is ADR-067; N3+ needs new sponsor ADR.
-
Honesty. Say “the guest programmed virtio-net and exchanged one ARP request/reply with QEMU SLIRP” only when serial/
qemu-smokeprintnet: okon a named tip. Do not say the OS “has networking.” CloudAgent HELD; tag agent-box / amd64 workstation. No self-merge (ADR-002).
Consequences
- Code: extend
src/virtio.rs(net discover/setup/TX/RX +observe_net_probe);src/main.rscallsinit_net+ probe; QEMU runners + smoke greps;#[test_case] virtio_net_discover_and_arp. - Docs: this ADR; track-n.md; SUMMARY; roadmap; honesty ledger; limits.md; threat-model bump; index one-liner.
- Follow-ups: N2 ICMP ping — ADR-067; N3/N4 locked out until new ADR + sponsor.
ADR-067 — Track N / N2 — ICMP ping after ARP (kernel-path)
- Status: Accepted (N2 runtime — Verified only when this tip’s
qemu-smokeprintsnet: ping-ok; else Documented/partial) - Date: 2026-09-16
Context
ADR-063 opened Track N. ADR-066 closed N1: virtio-net discover + ARP request/reply vs QEMU SLIRP gateway 10.0.2.2 (net: ok). This mile is N2: after (or with) that ARP exchange, send an ICMP echo request from guest 10.0.2.15 to gateway 10.0.2.2 and receive the echo reply — still on the kernel path only. Reuse N1 queues/DMA. Do not break blk/FAT/slot/net: ok greps.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Keep N1 ARP. Capture gateway MAC from ARP sha. Post a new RX buffer. TX Ethernet+IPv4+ICMP echo request (type 8) to 10.0.2.2. Wait for ICMP echo reply (type 0) with matching id/seq. Markers + fail-closed smoke. QEMU args unchanged: -netdev user,id=net0 -device virtio-net-device,netdev=net0. | Deterministic on GHA/agent-box (SLIRP answers ICMP locally). Proves L3 echo without a TCP/UDP stack. |
| H2 (rejected) | Add an EL0 net SVC / freestanding ping sample this mile. | N4 territory; prefer kernel-path ping only (sponsor). |
| H3 (rejected) | Depend on DHCP/DNS product, outbound Internet, TCP/UDP, or sockets. | Locked non-goals (ADR-063). |
| H4 (rejected) | Invent Verified without net: ping-ok on a named tip. | Honesty. |
| H5 (rejected) | Drop or weaken N1 ARP markers. | Keep net: virtio / net: mac / net: tx / net: rx / net: ok. |
Decision
-
Order. Run N1 ARP first (learn gateway MAC from ARP sha). Then ICMP echo TX/RX on the same virtio-net device. Same poll-
used.idx/ identity-PA DMA pattern as N1. -
Packet. Ethernet (dst = gateway MAC, src = guest MAC, ethertype IPv4) + IPv4 (src
10.0.2.15, dst10.0.2.2, proto ICMP, TTL 64) + ICMP echo request (type 8, code 0, fixed id/seq, small payloadctos-n2). Ones-complement checksums for IP header and ICMP message. No IP options. No TCP/UDP. -
Honest probe. Require ICMP echo reply (type 0) from
10.0.2.2to10.0.2.15with matching id/seq. Host-netdevwithout guest markers is not a probe. QEMU user-netdev args stay as in ADR-066 unless a future ADR names a change (none needed here). -
Serial markers (N2).
net: icmp-tx— TX used ring advanced for the ICMP echo requestnet: icmp-rx— RX delivered a matching ICMP echo replynet: ping-ok— N2 closed- Keep N1 markers including
net: ok - Miss →
net: probe missed(fail-closed) - Optional cheap CNTPCT:
perf: net-icmp-tx ticks=<n>/perf: net-icmp-rx ticks=<n>(lab only)
-
Smoke.
scripts/qemu-smoke.shgreps N1 markers andnet: icmp-tx/net: icmp-rx/net: ping-ok. Must keepblk:*/fat:*/ slot / sample / perf greps green. -
Non-goals restated (locked). TCP/UDP product; BSD sockets; DHCP/DNS product; Wi‑Fi; virtio-pci-only foundation; Linux net stack; “has networking” marketing; EL0 net SVC/ABI this PR (N4 later). N3+ needs new sponsor ADR.
-
Honesty. Say “the guest ARP’d the SLIRP gateway and completed one ICMP echo request/reply” only when serial/
qemu-smokeprintnet: ping-okon a named tip. Do not say the OS “has networking.” CloudAgent HELD; tag agent-box / amd64 workstation. No self-merge (ADR-002).
Consequences
- Code: extend
src/virtio.rs(fill_icmp_echo_request, ICMP RX match,observe_net_probeafter ARP); smoke greps;#[test_case] virtio_net_icmp_echo_ping(+ keep ARP test). - Docs: this ADR; track-n.md; SUMMARY; roadmap; honesty ledger; limits.md; threat-model bump; plain overview/architecture mermaid; index one-liner.
- Follow-ups: N3 sockets / N4 freestanding net sample locked out until new ADR + sponsor.
ADR-068 — Track N / N4 — EL0 net SVC + freestanding net-libctos sample
- Status: Accepted (sample + smoke Verified when serial / tests pass; product freestanding app hosting remains Verified under ADR-048/052 — do not reopen)
- Date: 2026-09-16
Context
ADR-063 locked EL0 net ABI until after link bring-up Verified. ADR-066 closed N1 (net: ok). ADR-067 closed N2 (net: ping-ok). Catalog freestanding samples exist for hello / VFS / FAT / yield (ADR-059 / ADR-061 / ADR-062).
This mile (N4) unlocks a tiny EL0 net SVC surface and a fifth freestanding sample that proves net via SVCs only. Kernel still owns virtio-net. Do not break blk/FAT/slot/N1/N2/sample greps.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Add SYS_NET_MAC (24) + SYS_NET_PING (25). New crate user/net-libctos on FAT /netdemo. Print libctos: net-hi / libctos: net-mac / libctos: net-ok. Kernel netdemo: ok. ADR + catalog + smoke. | Minimal ABI; proves MAC read + kernel-path ICMP from EL0 without a guest driver. |
| H2 (rejected) | Expose raw Ethernet frame TX/RX buffers to EL0 this mile. | Larger surface; prefer ping-as-proof. |
| H3 (rejected) | Ship BSD sockets / TCP/UDP / DHCP/DNS product. | Locked non-goals (ADR-063). |
| H4 (rejected) | Teach EL0 to program virtio-net MMIO. | Honesty: kernel owns the NIC; SVCs only. |
| H5 (rejected) | Replace /hello or another slot. | Breaks A9 / catalog greps. |
Decision
-
SVC surface (document in syscall.md).
24net_mac—x0= user buf,x1= length (≥6). Copy 6-byte guest MAC. Return6, or0if rejected / net not ready.25net_ping— no args. Kernel does quiet ARP + ICMP echo vs SLIRP10.0.2.2(reuse N1/N2 helpers). Return0on success,u64::MAXon fail.- Unknown SVCs still park. Not sockets. Not a process ABI.
-
Sample crate.
user/net-libctos: sameaarch64-ctos-user/linker.ld/build-stdas prior samples.mainprintslibctos: net-hi, callsnet_macthenlibctos: net-mac, callsnet_ping, thenlibctos: net-ok. -
Publish.
build.rsbuilds and publishestarget/net-libctos.elf.scripts/mkfat16.py --app5writes 8.3NETDEMO. Keep/hello+/fsdemo+/fatdemo+/yldemo+ their*: okmarkers. -
Kernel probe.
src/netdemo.rsloads FAT/netdemo,ERETs vialoader::run_image_expecting(last uart len 16 forlibctos: net-ok\n). Serialnetdemo: fat/netdemo: mapped/netdemo: ok. Runs after the ADR-062 yldemo probe. Requires N1/N2 virtio-net already brought up (init_net). -
Smoke. Fail-closed greps for the new markers; preserve all existing greps (blk/fat/slot/net/ping/samples).
-
Docs. This ADR; honesty ledger evidence-only;
user/README.md; what-can-run.md; apps-today.md; track-n.md; SUMMARY; threat-model v1.43. -
Honesty. Do not claim: TCP/UDP product, BSD sockets, DHCP/DNS, Wi‑Fi, “has networking,” EL0 virtio driver, Linux/POSIX, EL0 isolated, or reopen ADR-048/052. CloudAgent HELD; tag agent-box / amd64 workstation. No self-merge (ADR-002).
Consequences
- Code:
libctos+sys.S;src/syscall.rs;src/virtio.rs(guest_mac/el0_icmp_ping);user/net-libctos/;src/netdemo.rs;build.rs;scripts/mkfat16.py;scripts/qemu-aarch64.sh;scripts/qemu-smoke.sh;src/main.rs. - Docs: this ADR + catalog / ledger / threat-model v1.43 / track-n N4.
- Follow-ups (not this PR): N3 sockets (locked); richer EL0 frame I/O only with new ADR + sponsor.
ADR-069 — Net-ping path CNTPCT (measure-first)
- Status: Accepted (lab CNTPCT on QEMU TCG — not a published bench).
- Date: 2026-09-16
Context
ADR-065 landed a same-boot FAT vs memfs read CNTPCT pair. ADR-067 closed N2 kernel-path ICMP (net: ping-ok) with optional crumbs perf: net-icmp-tx / perf: net-icmp-rx. ADR-068 closed N4 with EL0 SYS_NET_PING (25) and freestanding /netdemo.
Perf slice next asks for another measure-first probe around a known path. The natural cut is the EL0 net_ping quiet path (kernel-owned ARP + ICMP echo vs SLIRP 10.0.2.2) that ADR-068 already exercises.
Do not invent a latency SLA, percent, or “faster/slower” claim. Do not add a criterion crate. Do not unlock N3 sockets / TCP/UDP. Keep N1/N2/N4 markers and all prior smoke greps.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Sample CNTPCT around the full quiet el0_icmp_ping body (ARP + ICMP echo RTT as SYS_NET_PING does it). Print perf: net-ping ticks=<n>. Fail closed if the counter does not advance after a successful ping. Smoke greps the prefix perf: net-ping (not an exact tick count). | Honest single-path measurement. Values vary under TCG. |
| H2 (rejected) | Publish a percent / “ping is X µs” / criterion bench / SLA. | NFR-07 forbids invented benches. |
| H3 (rejected) | Only time the kernel N2 icmp-tx or icmp-rx crumb and rename it net-ping. | Those optional crumbs stay as ADR-067 lab notes; ADR-069 names the EL0 SVC round-trip. |
| H4 (rejected) | Require an exact tick count in smoke. | Flaky under QEMU TCG jitter. |
| H5 (rejected) | Add TCP/UDP/sockets or widen the EL0 net ABI this mile. | N3 locked (ADR-063). |
Decision
- Window. In
virtio::el0_icmp_ping(called fromSYS_NET_PING), sampleCNTPCTfrom before the quiet ARP post through a matching ICMP echo reply. That is the EL0net_pinground-trip the sample already runs — not “network latency” as a product KPI. - Marker (exact form). On success with a non-zero delta:
perf: net-ping ticks=<n>(decimal ticks, no units, no percent). If the ping succeeds but the counter did not advance:perf: net-ping missedand treat the probe as failed. Related existing optional crumbs (unchanged):perf: net-icmp-tx ticks=<n>/perf: net-icmp-rx ticks=<n>on the kernel N2 probe path. - Smoke.
scripts/qemu-smoke.shgreps for the prefixperf: net-ping(presence). Rejects lines that claim faster/slower/percent. Does not assert a numeric tick value. Keep all prior greps (blk/fat/slot/net/ping/samples/ADR-065 pair). - Test.
#[test_case] el0_net_ping_cntpct_advancesasserts a successful quiet ping andnet_ping_ticks() > 0. - Honesty. QEMU
virtTCG lab only. Not a latency budget, not SPEC, not “has networking,” not sockets. Record Verified only with serial evidence on a named tip. CloudAgent HELD; tag agent-box / amd64 workstation. No self-merge (ADR-002).
Consequences
- Code:
src/virtio.rs(el0_icmp_pingCNTPCT +perf: net-ping+net_ping_ticks); smoke greps;#[test_case]. - Docs: this ADR, performance.md, measure.md, honesty ledger, roadmap
P-PERF-8, SUMMARY, threat-model light bump (v1.44). - Follow-ups (not this PR): yield/SVC CNTPCT pair, memcpy baseline — only after a probe shows a need. N3 sockets stay locked.
ADR-070 — Track N / N3 — UDP transport unlock (minimal IPv4 datagram)
- Status: Accepted (N3 runtime — Verified only when this tip’s
qemu-smokeprintsnet: udp-ok; else Documented/partial) - Date: 2026-09-16
Context
ADR-063 opened Track N and left N3 (transport / sockets) locked until a new sponsor scope ADR. N1 (ADR-066) Verified ARP first-frame (net: ok). N2 (ADR-067) Verified ICMP echo (net: ping-ok). N4 (ADR-068) Verified tiny EL0 net_mac / net_ping + /netdemo. Perf slice ADR-069 measures EL0 net_ping CNTPCT.
Sponsor unlocked N3: proceed with a real transport mile, not docs-only forever. Prefer UDP datagram first (measurable, fail-closed on SLIRP) over a fake POSIX sockets layer. Do not ship a full BSD sockets / TCP product this PR.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | After N1 ARP + N2 ICMP on the same virtio-net path: TX one IPv4 UDP datagram to QEMU SLIRP DNS 10.0.2.3:53 (minimal DNS query as payload), RX any matching DNS response (QR=1, same TXID). Markers net: udp-tx / net: udp-rx / net: udp-ok. QEMU args unchanged. | Deterministic on GHA/agent-box (SLIRP answers UDP DNS locally). Proves L4 datagram without sockets/TCP. DNS here is probe bait only, not a DNS product. |
| H2 (rejected) | QEMU guestfwd UDP echo this mile. | Extra host plumbing; H1 is enough if green. Revisit only if SLIRP DNS proves flaky. |
| H3 (rejected) | Ship a BSD sockets ABI / TCP stack / “has networking / sockets OS” marketing. | Honesty fail; locked product non-goals. |
| H4 (rejected) | Docs-only unlock with no guest probe. | Sponsor asked for a real transport mile. |
| H5 (rejected) | Require EL0 UDP SVCs this mile. | Optional later (N3.x); kernel-path probe matches N2 style. Keep existing net_mac/net_ping. |
Decision
-
Unlock N3 under this ADR. Ladder status moves from Locked → Verified when this tip’s
qemu-smokeprintsnet: udp-ok(with N1/N2 markers preserved). -
In scope this mile.
- Minimal UDP TX/RX (IPv4) on existing virtio-net (reuse ARP/IP helpers from the ICMP path).
- Honest CI probe: UDP DNS query from guest
10.0.2.15to SLIRP DNS10.0.2.3:53; accept any DNS response with matching transaction id and QR=1 (any RCODE). - Serial markers + fail-closed smoke greps.
- Optional thin EL0 UDP SVC(s): not this PR (natural follow-up only with another ADR / N3.x).
-
Still not product (locked).
- Full BSD sockets API.
- TCP stack as product.
- DHCP / DNS as product features (the probe may use SLIRP DNS as a peer; that is not a guest DNS resolver product).
- Wi‑Fi; Linux net stack; marketing “has networking / sockets OS.”
-
Packet. Ethernet (dst = gateway MAC from N1 ARP sha) + IPv4 (src
10.0.2.15, dst10.0.2.3, proto UDP, TTL 64) + UDP (src ephemeral0x6333, dst 53, checksum 0 allowed for IPv4) + minimal DNS A query for labelctos(TXID0x6333). No IP options. No TCP. No sockets ABI. -
Serial markers (N3).
net: udp-tx— TX used ring advanced for the UDP datagramnet: udp-rx— RX delivered a matching UDP DNS responsenet: udp-ok— N3 closed- Keep N1 (
net: ok) and N2 (net: ping-ok) markers - Miss →
net: probe missed(fail-closed)
-
Smoke.
scripts/qemu-smoke.shgreps N1 + N2 andnet: udp-tx/net: udp-rx/net: udp-ok. Must keepblk:*/fat:*/ slot / sample / perf greps green. -
Future N3.x. TCP and/or richer socket ABI only with another ADR + sponsor. Do not silently grow into a sockets product.
-
Honesty. Say “the guest sent one UDP datagram and received a UDP reply via SLIRP” only when serial/
qemu-smokeprintnet: udp-okon a named tip. Do not say the OS “has networking,” “has sockets,” or “has DNS.” CloudAgent HELD; tag agent-box / amd64 workstation. No self-merge (ADR-002).
Consequences
- Code: extend
src/virtio.rs(fill_udp_dns_query, UDP RX match,observe_net_probeafter ICMP); smoke greps;#[test_case] virtio_net_udp_dns_probe. - Docs: this ADR; track-n.md; SUMMARY; roadmap; honesty ledger; limits.md; threat-model bump (v1.45); plain overview/architecture; index one-liner.
- Follow-ups: EL0 UDP SVC / TCP / sockets ABI — new ADR only.
ADR-071 — Track N / N3.x — EL0 UDP SVC + freestanding udp-libctos sample
- Status: Accepted (sample + smoke Verified when serial / tests pass; product freestanding app hosting remains Verified under ADR-048/052 — do not reopen)
- Date: 2026-09-16
Context
ADR-070 closed N3 kernel-path UDP (net: udp-ok) and deferred EL0 UDP SVCs to N3.x. N4 (ADR-068) already Verified tiny EL0 net_mac (24) / net_ping (25) + FAT /netdemo. Catalog samples exist for hello / VFS / FAT / yield / net.
Sponsor: proceed as proposed proactively (N3.x). Keep the surface tiny and honest. Do not ship BSD sockets, TCP product, or DHCP/DNS as product. Kernel still owns virtio-net.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | One combined SYS_NET_UDP_DNS (26) that mirrors the kernel N3 UDP DNS probe (quiet ARP + UDP query to SLIRP 10.0.2.3:53 + matching QR=1 reply). New crate user/udp-libctos on FAT /udpdemo. Print libctos: udp-hi / libctos: udp-ok. Kernel udpdemo: ok. | Tiny ABI; same probe bait as ADR-070; no send/recv buffers this mile. |
| H2 (rejected) | Pair net_udp_send / net_udp_recv with user buffers this mile. | Larger surface; H1 is enough to prove EL0 can trigger the existing kernel UDP path. |
| H3 (rejected) | Ship BSD sockets / TCP / DHCP/DNS product. | Locked non-goals (ADR-063 / ADR-070). |
| H4 (rejected) | Teach EL0 to program virtio-net MMIO or copy raw frames. | Honesty: kernel owns the NIC; SVCs only. |
| H5 (rejected) | Replace /netdemo or another slot. | Breaks N4 / catalog greps. |
Decision
-
SVC surface (document in syscall.md).
- Keep
24net_macand25net_ping(ADR-068). 26net_udp_dns— no args. Kernel does quiet ARP + one UDP DNS query vs SLIRP10.0.2.3:53(reuse N3 helpers). Return0on success,u64::MAXon fail.- Unknown SVCs still park. Not sockets. Not a process ABI. DNS remains probe bait, not a guest resolver.
- Keep
-
Sample crate.
user/udp-libctos: sameaarch64-ctos-user/linker.ld/build-stdas prior samples.mainprintslibctos: udp-hi, callsnet_udp_dns, thenlibctos: udp-ok. -
Publish.
build.rsbuilds and publishestarget/udp-libctos.elf.scripts/mkfat16.py --app6writes 8.3UDPDEMO. Keep/hello+/fsdemo+/fatdemo+/yldemo+/netdemo+ their*: okmarkers. -
Kernel probe.
src/udpdemo.rsloads FAT/udpdemo,ERETs vialoader::run_image_expecting(last uart len 16 forlibctos: udp-ok\n). Serialudpdemo: fat/udpdemo: mapped/udpdemo: ok. Runs after the ADR-068 netdemo probe. Requires virtio-net already brought up (init_net). -
Smoke. Fail-closed greps for the new markers; preserve all existing greps (N1–N4, udp-ok, blk/FAT/samples/perf).
-
Docs. This ADR; honesty ledger evidence-only;
user/README.md; what-can-run.md; apps-today.md; track-n.md; SUMMARY; threat-model v1.46. -
Honesty. Do not claim: BSD sockets, TCP product, DHCP/DNS as product, Wi‑Fi, “has networking / sockets OS,” EL0 virtio driver, Linux/POSIX, EL0 isolated, or reopen ADR-048/052. CloudAgent HELD; tag agent-box / amd64 workstation. No self-merge (ADR-002).
Consequences
- Code:
libctos+sys.S;src/syscall.rs;src/virtio.rs(el0_udp_dns);user/udp-libctos/;src/udpdemo.rs;build.rs;scripts/mkfat16.py;scripts/qemu-aarch64.sh;scripts/qemu-smoke.sh;src/main.rs. - Docs: this ADR + catalog / ledger / threat-model v1.46 / track-n N3.x.
- Follow-ups (not this PR): TCP / richer socket ABI / generic UDP send/recv — new ADR + sponsor only.
ADR-072 — UDP DNS path CNTPCT (measure-first)
- Status: Accepted (lab CNTPCT on QEMU TCG — not a published bench).
- Date: 2026-09-16
Context
ADR-069 landed CNTPCT around the EL0 net_ping quiet path. ADR-070 closed N3 kernel UDP (net: udp-ok). ADR-071 closed N3.x with EL0 SYS_NET_UDP_DNS (26) and freestanding /udpdemo.
Perf slice next asks for another measure-first probe around a known path. The natural cut is the EL0 net_udp_dns quiet path (kernel-owned ARP + UDP DNS query vs SLIRP 10.0.2.3:53) that ADR-071 already exercises — same pattern as ADR-069 on net_ping.
Do not invent a latency SLA, percent, or “faster/slower” claim. Do not add a criterion crate. Do not implement FAT mkdir or TCP. Keep N1/N2/N3/N3.x/N4 markers and all prior smoke greps (including ADR-069 perf: net-ping).
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Sample CNTPCT around the full quiet el0_udp_dns body (ARP + UDP DNS RTT as SYS_NET_UDP_DNS / /udpdemo does it). Print perf: udp-dns ticks=<n>. Fail closed if the counter does not advance after a successful probe. Smoke greps the prefix perf: udp-dns (not an exact tick count). | Honest single-path measurement. Values vary under TCG. Matches ADR-069. |
| H2 (rejected) | Publish a percent / “DNS is X µs” / criterion bench / SLA. | NFR-07 forbids invented benches. |
| H3 (rejected) | Only time the kernel N3 net: udp-tx/udp-rx path and rename it udp-dns. | Kernel N3 markers stay as ADR-070 transport proof; ADR-072 names the EL0 SVC round-trip (same choice as ADR-069 H3). |
| H4 (rejected) | Require an exact tick count in smoke. | Flaky under QEMU TCG jitter. |
| H5 (rejected) | Add TCP / FAT mkdir / sockets or widen the EL0 net ABI this mile. | Out of scope; sponsor mile (2) is FAT mkdir — not this PR. |
Decision
- Window. In
virtio::el0_udp_dns(called fromSYS_NET_UDP_DNS/ FAT/udpdemo), sampleCNTPCTfrom before the quiet ARP post through a matching UDP DNS reply. That is the EL0net_udp_dnsround-trip the sample already runs — not “DNS latency” as a product KPI. Chosen surface: EL0net_udp_dns(not a separate kernel-only probe). - Marker (exact form). On success with a non-zero delta:
perf: udp-dns ticks=<n>(decimal ticks, no units, no percent). If the probe succeeds but the counter did not advance:perf: udp-dns missedand treat the probe as failed. - Smoke.
scripts/qemu-smoke.shgreps for the prefixperf: udp-dns(presence). Rejects lines that claim faster/slower/percent. Does not assert a numeric tick value. Keep all prior greps (blk/fat/slot/net/ping/udp/samples/ADR-069/ADR-071). - Test.
#[test_case] el0_udp_dns_cntpct_advancesasserts a successful quiet UDP DNS probe andudp_dns_ticks() > 0. - Honesty. QEMU
virtTCG lab only. Not a latency budget, not SPEC, not “has networking,” not sockets, not a DNS product. Record Verified only with serial evidence on a named tip. CloudAgent HELD; tag agent-box / amd64 workstation. No self-merge (ADR-002).
Consequences
- Code:
src/virtio.rs(el0_udp_dnsCNTPCT +perf: udp-dns+udp_dns_ticks); smoke greps;#[test_case]. - Docs: this ADR, performance.md, measure.md, honesty ledger, roadmap
P-PERF-9, SUMMARY, threat-model light bump (v1.47). - Follow-ups (not this PR): FAT mkdir (sponsor mile 2), TCP — new ADR + sponsor only.
ADR-073 — FAT16 mkdir (directory create depth)
- Status: Accepted (FAT16 root mkdir mile Verified when the serial / tests pass; Track A / app hosting still Planned for Linux/POSIX claims; Track N untouched)
- Date: 2026-09-16
Context
ADR-028 landed virtio-blk + FAT16 read. ADR-050 deepened write / small create. ADR-056 deepened root listing. ADR-057 deepened root file delete. ADR-064 deepened multi-cluster file grow. Docs “new vs extend” prefer deepening the known small FS (FAT16) behind the thin VFS (ADR-027).
Sponsor train mile (2) is honest FAT16 directory create (mkdir): allocate a root dirent with ATTR_DIR, allocate an empty directory cluster, write . / .., update the parent (root). Fail-closed serial + #[test_case].
Not POSIX mkdir / mkdirat. Not a nested path tree (thin-VFS path grammar stays flat /name). Not exFAT / FAT32. Not recursive rm. Do not implement thin TCP (sponsor mile 3) in this PR.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Kernel + thin VFS vfs::mkdir creates a FAT16 root directory. Serial fat: mkdir / keep fat: ok. Probe creates /fdir, confirms dirent + ./.. cluster, refuses second create (Exists). No new SVC this mile. | Deepens ADR-028/050/056/057/064. Same flat path grammar. |
| H2 (rejected) | Claim POSIX mkdir / mkdirat or a Linux mkdir ABI. | Forbidden honesty. |
| H3 (rejected) | New SVC / libctos wrapper this PR. | Delete/readdir/grow miles used existing VFS; mkdir is a kernel/VFS depth probe; EL0 SVC can follow later. |
| H4 (rejected) | Nested subdirectory trees, LFN, recursive rm, or exFAT. | Out of scope; follow-ups. |
| H5 (rejected) | Thin TCP / Track N mile 3 in this PR. | Separate sponsor mile; out of scope. |
Decision
fat::mkdir/vfs::mkdir. Allocate a free cluster (EOC), zero it, write.(self cluster) and..(parent = FAT16 root → cluster 0), allocate a free root dirent withATTR_DIR+ size 0. Name collision with a file or directory →FsError::Exists(fail-closed). Missing mount / bad path → fail-closed.- Probe after grow. Create
/fdir(leftover empty dir:rmdir_emptythen recreate). Confirmhas_dir, refuseopenas a file (Missing), confirm./..bytes, secondmkdir→Exists. Serialfat: mkdir. Keepfat: ok/ write / readdir / delete / grow / A9 / net markers. - No new SVC numbers. Public ABI stays as today (FS 19–23, net 24–26). Not a POSIX product claim. EL0/
libctosmkdir is an explicit follow-up. - Fail-closed smoke. Grep
fat: mkdir/ keepfat: ok/ grow / delete / net / samples.#[test_case]covers mkdir + exists/bad-path + create-on-dir-name refuse. - Honesty. Say “the guest created a FAT16 root directory through the thin VFS” only when serial / tests pass. Do not say: POSIX
mkdir, directory trees, exFAT/FAT32, “supports FAT,” EL0 isolated, PAN, taken SError, Linux, containers, TCP. - NFR-10 text revised in place (ID unchanged). Threat-model v1.48. Do not mint NFR-15+.
Consequences
- Code:
src/fat.rs(mkdir / dot entries / probe),src/vfs.rs(vfs::mkdir),scripts/qemu-smoke.shgreps. - Docs: this ADR, filesystem overview/stance, honesty ledger, roadmap note, threat-model v1.48, SUMMARY, light fr-nfr NFR-10 note.
- Follow-ups: EL0
fs_mkdirSVC + sample → ADR-075. Nested dirs + emptyrmdir→ ADR-077. Recursive rm (non-goal unless sponsored) remains open. Thin TCP closed in ADR-074.
ADR-074 — Track N / N5 — thin TCP unlock (active open + one payload)
- Status: Accepted (N5 runtime — Verified only when this tip’s
qemu-smokeprintsnet: tcp-ok; else Documented/partial) - Date: 2026-09-16
Context
ADR-063 opened Track N. N1–N4 + N3.x are Verified (ADR-066 / ADR-067 / ADR-070 / ADR-071 / ADR-068). FAT mkdir depth is ADR-073. Sponsor train mile (3) unlocks a thin / honest TCP probe — not a BSD sockets product.
Prior miles left TCP as a locked non-goal. This ADR opens a single-connection kernel-path mile only.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | After N1 ARP + N2 ICMP + N3 UDP on the same virtio-net path: active-open TCP to a QEMU guestfwd echo at 10.0.2.4:7 (cmd: → scripts/tcp-echo-stdio.sh), send one payload ctos-tcp\n, require matching echo. Markers net: tcp-syn / net: tcp-est / net: tcp-tx / net: tcp-rx / net: tcp-ok. | CI-deterministic (no external net). More honest than hoping SLIRP HTTP exists. Documents exact -netdev change. |
| H2 (rejected) | Rely on SLIRP built-in HTTP / external TCP peer. | Not fail-closed on GHA/agent-box; external net forbidden for smoke. |
| H3 (rejected) | Ship a BSD sockets ABI, listen/accept product, TLS, HTTP stack, or “has TCP sockets OS” marketing. | Honesty fail; locked product non-goals. |
| H4 (rejected) | Docs-only unlock with no guest probe. | Sponsor asked for a real thin TCP mile. |
| H5 (rejected) | Require EL0 TCP SVC this mile. | Optional follow-up ADR; kernel-path proof matches N2/N3 style. |
Decision
-
Unlock N5 under this ADR. Ladder status moves from Locked → Verified when this tip’s
qemu-smokeprintsnet: tcp-ok(with N1–N3 markers preserved). -
In scope this mile.
- Minimal TCP active open (SYN → SYN-ACK → ACK) + one payload send/recv on existing virtio-net (reuse ARP/IP helpers).
- One connection; no socket table; no listen/accept.
- Honest CI probe via QEMU user-netdev
guestfwd:-netdev user,id=net0,guestfwd=tcp:10.0.2.4:7-cmd:<repo>/scripts/tcp-echo-stdio.sh-device virtio-net-device,netdev=net0(unchanged device line)
- Host peer script echoes exactly nine bytes
ctos-tcp\nthen exits (TCP FIN). No commas in the-netdevpath. - Serial markers + fail-closed smoke greps.
- EL0 TCP SVC: not this PR (follow-up ADR only).
-
Still not product (locked).
- Full BSD sockets API;
listen/acceptproduct; multi-connection socket table. - TLS; HTTP stack; DHCP / DNS as product features.
- Wi‑Fi; Linux net stack; marketing “has TCP / sockets OS.”
- Full BSD sockets API;
-
Packet / state (tiny).
- Ethernet dst = gateway MAC from N1 ARP (SLIRP L2 for
10.0.2.4). - IPv4: src
10.0.2.15, dst10.0.2.4, proto TCP, TTL 64. - TCP: src ephemeral
0x6334, dst7, fixed ISN0x10000000, no options, window8192, real TCP checksum (pseudo-header). - Payload
ctos-tcp\n(9 bytes) with PSH+ACK after established. - Accept any matching peer segment that carries the echoed payload.
- Ethernet dst = gateway MAC from N1 ARP (SLIRP L2 for
-
Serial markers (N5).
net: tcp-syn— SYN TX used-ring advancednet: tcp-est— SYN-ACK seen + ACK TX’dnet: tcp-tx— payload PSH+ACK TX’dnet: tcp-rx— matching echo payload RX’dnet: tcp-ok— N5 closed- Keep N1 (
net: ok) / N2 (net: ping-ok) / N3 (net: udp-ok) - Miss →
net: probe missed(fail-closed)
-
Smoke.
scripts/qemu-smoke.shgreps N1–N3 and the N5 markers. Must keepblk:*/fat:*/ slot / sample / perf greps green. Runners that attach netdev (qemu-aarch64.sh,qemu-serial-inject.py) must include theguestfwdclause. DefaultCTOS_QEMU_TIMEOUTraised 12 → 20 so the longer N1–N5 + sample path still finishes under the inject capture window. -
Follow-ups. EL0 TCP SVC / richer socket ABI / listen-accept only with another ADR + sponsor.
-
Honesty. Say “the guest completed one TCP active open and echoed one payload via QEMU guestfwd” only when serial/
qemu-smokeprintnet: tcp-okon a named tip. Do not say the OS “has networking,” “has sockets,” or “has TCP.” CloudAgent HELD; tag agent-box / amd64 workstation. No self-merge (ADR-002).
Consequences
- Code: extend
src/virtio.rs(thin TCP TX/RX +observe_net_probeafter UDP);scripts/tcp-echo-stdio.sh+.py; QEMU netdevguestfwd; smoke greps;#[test_case] virtio_net_tcp_echo_probe. - Docs: this ADR; track-n.md; SUMMARY; roadmap; honesty ledger; limits.md; threat-model bump (v1.49); index / overview one-liners.
- Follow-ups: EL0 TCP SVC — new ADR only.
ADR-075 — EL0 fs_mkdir SVC + freestanding mkdir-libctos sample
- Status: Accepted (EL0 mkdir mile Verified when the serial / tests pass; POSIX
mkdir/ nested trees / Track N EL0 TCP still non-goals) - Date: 2026-09-17
Context
ADR-073 landed kernel fat::mkdir / vfs::mkdir with serial fat: mkdir and deferred the public EL0 SVC + sample. Public ABI today: FS SVCs 19–23, net 24–26 (syscall.md). Catalog samples exist for hello / VFS / FAT / yield / net / udp.
Sponsor follow-up unlocks EL0 directory create through the thin VFS — same fail-closed Exists / BadPath honesty as ADR-073. Do not implement EL0 TCP or nested directory trees in this PR.
FAT 8.3 path grammar caps the sample slot at eight characters. /mkdirdemo (9) is rejected by path_to_83; the slot is /mkdemo.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Add SYS_FS_MKDIR (27). Path ABI matches create/open (x0 ptr, x1 len). Map Ok→0, Exists→1, BadPath→2, other→u64::MAX. New crate user/mkdir-libctos on FAT /mkdemo. Print libctos: mkdir-hi / libctos: mkdir-ok. Kernel mkdemo: ok. | Minimal ABI; proves EL0 mkdir + Exists/BadPath without nested trees. |
| H2 (rejected) | Claim POSIX mkdir / mkdirat or Linux mkdir ABI. | Forbidden honesty. |
| H3 (rejected) | Extend fat-libctos instead of a seventh catalog slot. | Prior miles used a dedicated sample per capability; keep that style. |
| H4 (rejected) | Nested subdirectory trees / richer path grammar / recursive create. | Out of scope; no nested claim unless proven (not this mile). |
| H5 (rejected) | EL0 TCP SVC in this PR. | Separate follow-up; out of scope. |
Decision
SYS_FS_MKDIR= 27. Document in syscall.md. Args:x0= user path pointer,x1= length. Kernel copies the path and callsvfs::mkdir. Return:0(FS_MKDIR_OK) on success1(FS_MKDIR_EXISTS) onFsError::Exists2(FS_MKDIR_BAD_PATH) onFsError::BadPath(including failed user-path copy / invalid UTF-8 path)u64::MAX(FS_ERR) for other failures (Missing mount, Full, …) Unknown SVCs still park. Not POSIX. Not a process ABI.
libctos.ctos_fs_mkdir/fs_mkdir+SYS_FS_MKDIR/FS_MKDIR_*constants. CRT must not issue reserved 0–2.- Freestanding sample.
user/mkdir-libctoson FAT/mkdemo(8.3MKDEMO). Creates/edir(separate from kernel/fdir), requires second create → Exists, nested/bad/nest→ BadPath. Markers:libctos: mkdir-hi/libctos: mkdir-ok/mkdemo: ok. Keep prior six samples +fat: mkdir+ all net greps. - Fail-closed smoke. Grep the new markers; keep
fat: mkdir/fat: ok/ grow / delete / net / samples.#[test_case]on the slot module. - Honesty. Say “EL0 created a FAT16 root directory through
fs_mkdir” only when serial / tests pass. Do not say: POSIXmkdir, directory trees, exFAT/FAT32, “supports FAT,” EL0 isolated, PAN, taken SError, Linux, containers, EL0 TCP. - NFR-10 text revised in place (ID unchanged). Threat-model v1.50. Do not mint NFR-15+.
Consequences
- Code:
src/syscall.rs,libctos/,user/mkdir-libctos/,src/mkdemo.rs,build.rs,scripts/mkfat16.py,scripts/qemu-smoke.sh. - Docs: this ADR, syscall.md, honesty ledger, filesystem overview/stance, what-can-run, SUMMARY, roadmap note, threat-model v1.50, light fr-nfr NFR-10 note,
user/README.md. - Follow-ups: nested dirs + empty
rmdir→ ADR-077. EL0 TCP SVC → ADR-076.
ADR-076 — EL0 thin TCP SVC + freestanding tcp-libctos sample
- Status: Accepted (EL0 TCP mile Verified when the serial / tests pass; BSD sockets / listen-accept / TLS / multi-connection table still non-goals)
- Date: 2026-09-17
Context
ADR-074 closed N5 kernel-path thin TCP (net: tcp-ok via QEMU guestfwd 10.0.2.4:7) and deferred the EL0 TCP SVC + sample. Public ABI today: FS SVCs 19–23 + 27, net 24–26 (syscall.md). Catalog samples exist for hello / VFS / FAT / yield / net / udp / mkdir.
Sponsor follow-up unlocks EL0 through the same quiet echo probe — one SVC, return 0 / fail. Do not implement BSD sockets, listen/accept, TLS, multi-connection tables, nested FAT/rmdir, or TCP CNTPCT in this PR.
FAT 8.3 path grammar: /tcpdemo fits (7 chars under the 8.3 name).
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Add SYS_NET_TCP_ECHO (28). No args. Kernel runs quiet ARP + thin TCP active-open + one payload vs guestfwd 10.0.2.4:7 (reuse ADR-074 helpers; distinct ephemeral port from N5). Return 0 on success, u64::MAX on fail. New crate user/tcp-libctos on FAT /tcpdemo. Print libctos: tcp-hi / libctos: tcp-ok. Kernel tcpdemo: ok. Keep kernel net: tcp-ok. | Tiny ABI; proves EL0 can trigger the existing thin TCP path. |
| H2 (rejected) | Ship BSD sockets / connect/send/recv / listen-accept this mile. | Honesty fail; locked product non-goals. |
| H3 (rejected) | Teach EL0 to program virtio-net MMIO or copy raw frames. | Kernel owns the NIC; SVCs only. |
| H4 (rejected) | Replace /netdemo / /udpdemo or another slot. | Breaks prior catalog greps. |
| H5 (rejected) | TCP CNTPCT / perf: tcp-* this PR. | Separate follow-up; out of scope. |
Decision
SYS_NET_TCP_ECHO= 28. Document in syscall.md. No args. Kernelvirtio::el0_tcp_echodoes quiet ARP + SYN/est/tx/rx vs guestfwd (no serialnet: tcp-*markers on this quiet path). Return0on success,u64::MAX(NET_ERR) on fail. Unknown SVCs still park. Not sockets. Not a process ABI.libctos.ctos_net_tcp_echo/net_tcp_echo+SYS_NET_TCP_ECHO. CRT must not issue reserved 0–2.- Freestanding sample.
user/tcp-libctoson FAT/tcpdemo(8.3TCPDEMO). Markers:libctos: tcp-hi/libctos: tcp-ok/tcpdemo: ok. Keep prior seven samples + kernelnet: tcp-ok+ all other smoke. - Fail-closed smoke. Grep the new markers; keep N1–N5 / N3.x / N4 / FAT / mkdir / samples.
#[test_case]on the slot module +el0_tcp_echo_probe. DefaultCTOS_FORCE_FAIL_TIMEOUTis 180s (was 120): ubuntu-24.04-arm force-fail rebuild+boot with eight FAT samples exceeded 120s; force-fail must still exit non-zero. - Honesty. Say “EL0 triggered the thin TCP guestfwd echo through
net_tcp_echo” only when serial / tests pass. Do not say: BSD sockets, listen/accept, TLS, HTTP, “has networking / sockets OS,” EL0 virtio driver, Linux/POSIX, EL0 isolated, PAN, taken SError. - NFR-10 text revised in place (ID unchanged). Threat-model v1.51. Do not mint NFR-15+.
Consequences
- Code:
src/syscall.rs,src/virtio.rs(el0_tcp_echo),libctos/,user/tcp-libctos/,src/tcpdemo.rs,build.rs,scripts/mkfat16.py,scripts/qemu-smoke.sh. - Docs: this ADR, syscall.md, honesty ledger, track-n, SUMMARY, what-can-run, apps-today, roadmap note, threat-model v1.51,
user/README.md. - Follow-ups (not this PR): TCP CNTPCT, richer socket ABI, listen-accept — new ADR + sponsor only.
ADR-077 — FAT16 nested mkdir + empty rmdir
- Status: Accepted (nested mkdir + empty rmdir mile Verified when the serial / tests pass; Track A / app hosting still Planned for Linux/POSIX claims; Track N / TCP CNTPCT untouched)
- Date: 2026-09-17
Context
ADR-073 landed FAT16 root mkdir (fat: mkdir). ADR-075 added EL0 fs_mkdir (27) for root /edir. Nested subdirectory trees and empty rmdir were explicit follow-ups. ADR-057 deletes files only.
Sponsor follow-up: one-level nested directory create under an existing FAT dir, plus empty rmdir fail-closed. Do not implement TCP CNTPCT or a new EL0 SVC in this PR (kernel/VFS proof is enough this mile).
Path grammar (honest)
| Form | Example | Notes |
|---|---|---|
| Flat | /fdir | Same as ADR-073: / + 1..=8 of [a-z0-9_-] |
| One nested | /fdir/nest | / + parent + / + child; each component 1..=8 of [a-z0-9_-]; whole path ≤ PATH_MAX (16) |
| Deeper | /a/b/c | BadPath (fail-closed) |
| File ops | /probe | create/open/unlink/readdir stay flat (vfs::valid_path); nested file paths → BadPath |
Missing parent on a nested mkdir → Missing. Parent name occupied by a file → BadPath. Name collision → Exists.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Extend fat::mkdir / vfs::mkdir for one nested level; expose fat::rmdir / vfs::rmdir for empty dirs. Serial fat: nested / fat: rmdir. Keep fat: mkdir. No new SVC. | Deepens ADR-073; documents grammar. |
| H2 (rejected) | Claim POSIX mkdir / rmdir / mkdirat or a Linux directory ABI. | Forbidden honesty. |
| H3 (rejected) | New SYS_FS_RMDIR / EL0 sample this PR. | Optional; kernel/VFS proof is enough; EL0 can follow. |
| H4 (rejected) | Arbitrary-depth trees, LFN, recursive rm, multi-cluster directories, exFAT. | Out of scope. |
| H5 (rejected) | TCP CNTPCT in this PR. | Separate follow-up; out of scope. |
Decision
fat::mkdir. Flat root unchanged. Nested/parent/child: look up parent dirent, allocate child cluster with./..(..= parent cluster id), installATTR_DIRdirent in the parent cluster (spc=1). Fail-closed Exists / Missing / BadPath as above.fat::rmdir/vfs::rmdir. Empty only (.`` /..+ rest zero). Mark dirent0xE5, free cluster chain. Non-empty →BadHandle. Missing →Missing. File path →BadPath`. Not recursive rm.- Probe after root mkdir. Create
/fdir/nest, confirm dirent + dots, Exists on second create, BadPath on/fdir/nest/x, Missing on/nope/x. Empty-rmdir nest; refuse rmdir/fdirwhile nest exists; refuse rmdir/probe. Serialfat: nestedthenfat: rmdir. Keepfat: mkdir/ grow / delete / net / samples. - No new SVC numbers. Public ABI stays as today (FS 19–23 + 27, net 24–26 + 28). EL0
mkdir-libctosBadPath bait becomes/a/b/c(three-level) so Exists/BadPath proof still holds under the new grammar. - Fail-closed smoke. Grep
fat: nested/fat: rmdir; keepfat: mkdir/fat: ok/ net / samples.#[test_case]covers nested+rmdir + path grammar. - Honesty. Say “the guest created a one-level nested FAT16 directory and removed an empty one through the thin VFS” only when serial / tests pass. Do not say: POSIX
mkdir/rmdir, arbitrary directory trees, recursive rm, exFAT/FAT32, “supports FAT,” EL0 isolated, PAN, taken SError, Linux, containers, TCP CNTPCT. - NFR-10 text revised in place (ID unchanged). Threat-model v1.52. Do not mint NFR-15+.
Consequences
- Code:
src/fat.rs(parse_dir_path / nested mkdir / rmdir / probe),src/vfs.rs(vfs::rmdir+ mkdir routing),scripts/qemu-smoke.shgreps, lightuser/mkdir-libctosBadPath bait update. - Docs: this ADR, filesystem overview/stance, honesty ledger, roadmap note, threat-model v1.52, SUMMARY, light fr-nfr NFR-10 note.
- Follow-ups (not this PR): EL0
fs_rmdirSVC + sample, deeper path trees, multi-cluster directories, recursive rm (non-goal unless sponsored), TCP CNTPCT.
ADR-078 — Thin TCP echo path CNTPCT (measure-first)
- Status: Accepted (lab CNTPCT on QEMU TCG — not a published bench).
- Date: 2026-09-17
Context
ADR-069 timed EL0 net_ping. ADR-072 timed EL0 net_udp_dns. ADR-074 closed N5 kernel thin TCP (net: tcp-ok). ADR-076 closed N5.x with EL0 SYS_NET_TCP_ECHO (28) and freestanding /tcpdemo, and deferred TCP CNTPCT.
Perf slice next asks for another measure-first probe around a known path. The natural cut is the EL0 net_tcp_echo quiet path (kernel-owned ARP + thin TCP active-open + one payload vs QEMU guestfwd 10.0.2.4:7) that ADR-076 already exercises — same pattern as ADR-069 / ADR-072.
Do not invent a latency SLA, percent, or “faster/slower” claim. Do not add a criterion crate. Do not implement BSD sockets, listen/accept, TLS, or widen the EL0 net ABI. Keep N1–N5 / N5.x / N3.x / N4 markers, ADR-069/072 prefixes, FAT nested/rmdir, and all prior smoke greps. Force-fail timeout stays 180s.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Sample CNTPCT around the full quiet el0_tcp_echo body (ARP + TCP SYN/est/tx/rx as SYS_NET_TCP_ECHO / /tcpdemo does it). Print perf: tcp-echo ticks=<n>. Fail closed if the counter does not advance after a successful probe. Smoke greps the prefix perf: tcp-echo (not an exact tick count). | Honest single-path measurement. Values vary under TCG. Matches ADR-069/072. Chosen surface: EL0 net_tcp_echo (not a separate kernel-only N5 timer). |
| H2 (rejected) | Publish a percent / “TCP is X µs” / criterion bench / SLA. | NFR-07 forbids invented benches. |
| H3 (rejected) | Only time the kernel N5 net: tcp-* path and rename it tcp-echo. | Kernel N5 markers stay as ADR-074 transport proof; ADR-078 names the EL0 SVC round-trip (same choice as ADR-069/072 H3). |
| H4 (rejected) | Require an exact tick count in smoke. | Flaky under QEMU TCG jitter. |
| H5 (rejected) | Add listen/accept / sockets / TLS or widen the EL0 net ABI this mile. | Out of scope; honesty fail. |
Decision
- Window. In
virtio::el0_tcp_echo(called fromSYS_NET_TCP_ECHO/ FAT/tcpdemo), sampleCNTPCTfrom before the quiet ARP post through a matching TCP echo payload. That is the EL0net_tcp_echoround-trip the sample already runs — not “TCP latency” as a product KPI. Chosen surface: EL0net_tcp_echo(document explicitly; kernelnet: tcp-okpath unchanged / untimed this mile). - Marker (exact form). On success with a non-zero delta:
perf: tcp-echo ticks=<n>(decimal ticks, no units, no percent). If the probe succeeds but the counter did not advance:perf: tcp-echo missedand treat the probe as failed. - Smoke.
scripts/qemu-smoke.shgreps for the prefixperf: tcp-echo(presence). Rejects lines that claim faster/slower/percent. Does not assert a numeric tick value. Keep all prior greps (blk/fat/slot/net/ping/udp/tcp/samples/ADR-069/072/076/077). DefaultCTOS_FORCE_FAIL_TIMEOUTremains 180s. - Test.
#[test_case] el0_tcp_echo_cntpct_advancesasserts a successful quiet TCP echo andtcp_echo_ticks() > 0. - Honesty. QEMU
virtTCG lab only. Not a latency budget, not SPEC, not “has networking,” not sockets. Record Verified only with serial evidence on a named tip. CloudAgent HELD; tag agent-box / amd64 workstation. No self-merge (ADR-002).
Consequences
- Code:
src/virtio.rs(el0_tcp_echoCNTPCT +perf: tcp-echo+tcp_echo_ticks); smoke greps;#[test_case]. - Docs: this ADR, performance.md, measure.md, honesty ledger, roadmap
P-PERF-10, SUMMARY, threat-model light bump (v1.53), NFR-10 note. - Follow-ups (not this PR): richer socket ABI, listen-accept, TLS — new ADR + sponsor only.
ADR-079 — PAN CPU reopen foundation (sponsor unlock; default -cpu cortex-a76)
- Status: Accepted (CPU reopen foundation). Meets ADR-054 reopen gate items 1–2: sponsor-approved FEAT_PAN default probe CPU + serial
pan: present. Enable+fault is ADR-080 (Accepted). Taken SError reopen is ADR-081 (follow-up). Does not claim “EL0 isolated.” - Date: 2026-09-19
- Tracks #125. Sponsor unlock Sat 19 Sep 2026 via DSO. CloudAgent HELD — agent-box / amd64 workstation / GHA only.
Context
ADR-054 locked PAN enable as non-goal on default -cpu cortex-a57 (pan: absent, ADR-026). Reopen required: (1) sponsor-approved FEAT_PAN CPU + new ADR; (2) ID_AA64MMFR1_EL1.PAN != 0 / pan: present; (3) later enable+fault ADR; (4) ledger/smoke update; keep a57 pan: absent historical.
Sponsor / DSO unlocked mile (1): research honest QEMU virt CPUs, pick one CI-viable default, switch smoke scripts with this ADR cite, print pan: present. Do not implement enable or taken SError Verified in this PR. ADR-053 stays hard-stopped until ADR-081. Umbrella “EL0 isolated” stays non-claim (ADR-055 / ADR-060). Never yank _start. No silent -cpu without this ADR.
Research (agent-box, 2026-09-19 CEST)
Host: agent-box. qemu-system-aarch64 10.0.13 (Debian 1:10.0.13+ds-0+deb13u1). Machine: -machine virt. Short freestanding guest probe (MRS ID_AA64MMFR1_EL1, PL011 serial) — not a silent host invent.
-cpu | ID_AA64MMFR1_EL1.PAN [23:20] | Serial |
|---|---|---|
cortex-a57 | 0 | pan: id=0 / pan: absent (historical) |
cortex-a72 | 0 | pan: id=0 / pan: absent |
cortex-a55 | 2 | pan: id=2 / pan: present |
cortex-a76 | 2 | pan: id=2 / pan: present (chosen default) |
cortex-a710 | 2 | pan: id=2 / pan: present |
neoverse-n1 | 2 | pan: id=2 / pan: present (UDP not CI-viable here) |
neoverse-n2 | 2 | pan: id=2 / pan: present |
neoverse-v1 | 2 | pan: id=2 / pan: present |
max | 3 | pan: id=3 / pan: present (moving target — rejected as default) |
CI / smoke viability (agent-box, same tip ELF, inject runner)
-cpu | pan: | N1–N5 net | FAT / samples |
|---|---|---|---|
cortex-a57 | absent | ok | ok (historical) |
neoverse-n1 | present | UDP miss (net: udp-tx then net: probe missed) | ok after miss |
cortex-a76 | present | ok (udp+tcp) | ok |
cortex-a55 | present | ok | ok |
max | present (id=3) | ok | ok (rejected as default — moving target) |
Chosen default: cortex-a76. FEAT_PAN id=2, full existing smoke (net/FAT/samples) green on agent-box QEMU 10.0.13. neoverse-n1 is FEAT_PAN-present but not CI-viable for this tree’s UDP path without a separate timing/net ADR.
inject-nmi note (SError mile later)
QMP inject-nmi on the same host still returns machine does not provide NMIs for cortex-a57, cortex-a76, neoverse-n1, and max (re-probed 2026-09-19). CPU switch alone does not unlock taken SError — ADR-081 documents the blocker.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Default probe CPU → -cpu cortex-a76; smoke expects pan: present; cite this ADR; keep a57 rows historical; no MSR PAN. | FEAT_PAN id=2; full net/FAT/samples smoke green on agent-box; not max; neoverse-n1 UDP-fails on this tree. |
| H2 (rejected) | Stay on cortex-a57 and only document research. | Sponsor asked for concrete CPU switch evidence when a FEAT_PAN CPU works honestly. |
| H3 (rejected) | Default to -cpu max. | Field value drifts with QEMU; ADR-026 already rejected silent max. |
| H4 (rejected) | Enable PSTATE.PAN / claim Verified enable this PR. | Mile 2 = ADR-080. |
| H5 (rejected) | Fake pan: present on a57 or invent Verified SError. | Honesty fail. |
Decision
-
Sponsor unlock recorded. Isolation leftovers reopen mile (1) is authorized (#125). ADR-054 gate items 1–2 are met by this ADR + evidence below. Items 3–4 landed in ADR-080.
-
Default probe CPU.
scripts/qemu-aarch64.shandscripts/qemu-serial-inject.pyuse-cpu cortex-a76with this ADR cite. Do not silent-edit without an ADR. -
Serial contract (this mile). Guest reads
ID_AA64MMFR1_EL1.PAN(ADR-026 probe). On the new default:pan: id=<n>withn != 0andpan: present. ADR-080 addspan: enabled/pan: el1-fault. Historical a57pan: absentrows remain truth for a57. -
Probe success semantics.
pan::observe_probereturns success for bothabsentandpresent(ID field printed). Fail-closed only if the probe did not publish. NoMSR PANin this mile. -
Still non-claim. Taken SError until ADR-081. Umbrella “EL0 isolated” (ADR-055 / ADR-060). Never yank
_start. No Guest Linux / containers marketing. -
Toolchain pin.
rust-toolchain.tomlpinsnightly-2026-09-12(0fc141305) — same tip as ADR-078 ledger evidence. Floatingnightly@ 2026-09-18 failed freestandingcargo testwith E0463 (can't find crate for test). Smoke usescargo(pin) notcargo +nightly. -
Follow-ups. ADR-080 (Accepted): enable + EL1-vs-EL0 fault. ADR-081 (Accepted): taken SError reopen re-probe — still blocked; sponsor unlock B1/B2/B3.
Honesty
Say: “default smoke CPU is -cpu cortex-a76 (ADR-079); ID_AA64MMFR1_EL1.PAN != 0 → pan: present; enable still locked pending ADR-080; a57 pan: absent remains historical.” Do not say:
- PAN is enabled / “privileged access never”
- “EL0 isolated” / “secure OS”
- taken lower-EL SError is Verified
- the probe silently used a different
-cputhan the scripts - Guest Linux / containers / immutable-OS
Consequences
- Scripts + smoke +
src/pan.rs/#[test_case]updated forcortex-a76/pan: present. - Docs: this ADR, ADR-054 reopen note, ADR-060, honesty ledger, roadmap P-SEC-3k, threat-model light bump, SUMMARY, limits / el0 light touch.
- Preserve all existing smoke greps (net/FAT/samples). CloudAgent HELD. Do not self-merge (ADR-002).
ADR-080 — PAN enable + EL1-vs-EL0 access fault (ADR-054 reopen gate 3–4)
- Status: Accepted (enable + fault Verified on default
-cpu cortex-a76). Meets ADR-054 reopen gate items 3–4: enable PSTATE.PAN /MSR PANand prove an EL1 load of an EL0-accessible page faults (fail-closed); honesty ledger + smoke updated. Does not claim “EL0 isolated.” Taken SError reopen is ADR-081 (Accepted — still blocked). - Date: 2026-09-19
- Tracks #125 mile (2). Builds on ADR-079 (gate items 1–2). CloudAgent HELD — agent-box / amd64 workstation / GHA only.
Context
ADR-054 locked PAN enable until a follow-up ADR after FEAT_PAN CPU reopen. ADR-079 met gate items 1–2 (-cpu cortex-a76, pan: present). Gate item 3 required this ADR: enable PSTATE.PAN and prove EL1-vs-EL0 fault. Gate item 4: ledger + smoke for enable; keep a57 pan: absent historical.
ADR-013 already named the enable mile: with PSTATE.PAN set, an EL1 data access to a page that is unprivileged-accessible must take a permission fault. Umbrella “EL0 isolated” stays non-claim (ADR-055 / ADR-060). Never yank _start.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | After ID-field pan: present, MSR PAN, #1; map an EL0-RO page; EL1 LDR must permission-fault with SPSR.PAN set; print pan: enabled / pan: el1-fault; leave PAN set; syscall uaccess clears PAN around copies. | Fail-closed if fault missing or SPSR.PAN clear. Preserves net/FAT/samples. |
| H2 (rejected) | Enable for the probe only, then clear PAN for the rest of boot. | Weaker than “enable”; uaccess wrapping is small and honest. |
| H3 (rejected) | Claim “EL0 isolated” from this mile alone. | ADR-055 §B still unmet (taken SError, _start stay, umbrella sponsor accept). |
| H4 (rejected) | Invent Verified without a caught fault / silent -cpu. | Honesty fail. |
Decision
- Enable.
src/pan.rsobserve_enable(afterframe::init) executesMSR PAN, #1on FEAT_PAN CPUs (ID_AA64MMFR1_EL1.PAN != 0). Serialpan: enabled. - EL1-vs-EL0 fault. Map an EL0-accessible RO page (
map_el0_ro); with PAN clear, EL1 preload must succeed; with PAN set, EL1LDRmust take a current-EL permission DABORT. Handler printspan: el1-faultand requires SPSR.PAN (bit 22) set at the fault (fail-closed otherwise). Note: under QEMU 10.0.13 TCG,MRS PANmay read 0 even while SPSR.PAN and the permission check prove enable — SPSR is the probe of record here. - Leave PAN set. After a successful probe, PSTATE.PAN stays 1.
pan::with_user_accessclears/restores around syscallcopy_user/copy_to_userso intentional EL0-accessible copies keep working. - Serial + smoke. Keep
pan: id=/pan: present(ADR-079). Requirepan: enabled/pan: el1-fault. Rejectpan: enable missed/ unexpectedpan: absent. Historical a57pan: absentrows remain historical. - Target feature.
aarch64-ctos.jsonadds+panso the assembler accepts thepanPSTATE name (still-cpu cortex-a76in scripts). - Still non-claim. Umbrella “EL0 isolated” unchanged. Taken SError → ADR-081 (blocked). Never yank
_start. No Guest Linux / containers marketing.
Honesty
Say: “on -cpu cortex-a76 (ADR-079), PSTATE.PAN is enabled (ADR-080); EL1 load of an EL0-accessible page faults (pan: el1-fault); umbrella EL0 isolation stays non-claim.” Do not say:
- “EL0 isolated” / “secure OS”
- taken lower-EL SError is Verified
- PAN worked on cortex-a57 without a CPU change
- Guest Linux / containers / immutable-OS
Consequences
src/pan.rs,src/exception.rs(arm + SPSR.PAN check),src/syscall.rs(uaccess),src/main.rs,scripts/qemu-smoke.sh,aarch64-ctos.json.- Docs: this ADR; ADR-054 / ADR-055 / ADR-060 / ADR-026 / ADR-079 cross-links; honesty ledger; roadmap P-SEC-3k; threat-model v1.55; SUMMARY; el0 / limits / security light touch.
- Preserve all existing net/FAT/sample smoke greps. CloudAgent HELD. Do not self-merge (ADR-002).
Follow-on note (2026-09-29, ADR-098)
ADR-098 found that PAN, once enabled here, was not held while the kernel handled exceptions from EL0 or after it returned from an EL0 trip: SCTLR_EL1.SPAN was left at 1 (so exception entry from EL0 did not set PSTATE.PAN), and return_from_el0 resumed EL1 with SPSR.PAN clear. The pan: el1-fault probe above runs once at boot, before any EL0 trip, so it could not see this. ADR-098 clears SPAN after a successful enable, sets SPSR.PAN on return_from_el0, and adds fail-closed live checks with a negative build. In that work MRS PAN read 1 whenever a fault proved PAN set, so the “MRS PAN may read 0” note in Decision 2 is not reproduced there. This ADR’s text is kept as the record of its time.
ADR-081 — Taken lower-EL SError reopen (ADR-053 hard-stop re-probe; still blocked)
- Status: Accepted (docs / evidence). #125 mile (3). Dated re-probe on default
-cpu cortex-a76+ documented virt options: no honest inject delivers ARM async SError to standing EL0 on QEMU 10virtTCG. Taken path stays deferred / non-goal (locked).el0: serror-parkstays Verified. Guest A-clear / EXPECT plumbing stays dormant prep. Do not mark taken SError Verified. - Date: 2026-09-19
- Tracks #125. Builds on ADR-053 reopen gate. CloudAgent HELD — agent-box / amd64 workstation / GHA only.
Context
ADR-053 hard-stopped taken lower-EL SError on QEMU 10 virt + -cpu cortex-a57 after inject-nmi / HMP nmi → machine does not provide NMIs. Reopen gate §4: (a) honest host inject on the accepted smoke machine or sponsor-approved machine/CPU/GIC change with its own ADR; (b) serial el0: serror under EXPECT; (c) new ADR with working evidence.
ADR-079 / ADR-080 changed the default probe CPU to cortex-a76 and Verified PAN enable — not an SError unlock. Sponsor / DSO unlocked mile (3): re-research any honest path; if none, document the blocker explicitly rather than invent Verified.
Do not invent FIQ / BRK / FEAT_NMI as SError. Do not silent machine invent. Keep _start at 0x4008_0000. Umbrella “EL0 isolated” stays non-claim (ADR-055 / ADR-060).
Research (agent-box, 2026-09-19 CEST)
Host: agent-box. qemu-system-aarch64 10.0.13 (Debian 1:10.0.13+ds-0+deb13u1). Accel: TCG only (query-kvm → present=false; -accel kvm → invalid accelerator kvm on this box). Full brief: research/daily-briefs/2026-09-19-adr-081-serror-inject-probe.md.
| Config | QMP inject-nmi | Notes |
|---|---|---|
virt + cortex-a76 (default smoke) | error machine does not provide NMIs | HMP nmi same text |
virt + cortex-a57 (historical) | same | Matches ADR-053 / ADR-045 |
virt,gic-version=3 + cortex-a76 | same | GICv3 alone ≠ TYPE_NMI |
virt,gic-version=3 + max | same | |
virt,virtualization=on + cortex-a76 | same | EL2 virt ≠ SError inject |
virt,gic-version=3,virtualization=on + cortex-a76 | same | |
virt,gic-version=3,secure=on + cortex-a76 | same | |
-cpu max,nmi=on | startup fail | Property 'max-arm-cpu.nmi' not found |
query-commands lists inject-nmi, but virt does not implement TYPE_NMI. CXL *-inject-* commands are not an ARM SError path. Live smoke still arms el0: serror-arm, logs the QMP error, and prints el0: serror-park (no el0: serror).
Options (re-evaluated)
| ID | Option | 2026-09-19 status |
|---|---|---|
| H1 | QMP/HMP nmi → ARM_CPU_SERROR | Still impossible on virt TCG (all rows above). 2020 qemu-devel “Simulate NMI Injection” series did not land in QEMU 10.0.13. |
| H2 | Guest-only RAS / poisoned load | Still rejected for Verified (ADR-044). |
| H3 | Treat sync/BRK/FIQ / FEAT_NMI as SError | Still forbidden. FEAT_NMI is a GICv3 interrupt story, not async SError. |
| H5 | KVM serror_pending / plugin inject | Still out of scope for TCG virt -kernel smoke; this agent-box has no KVM accel. |
Decision
- Hard-stop stands on the new default CPU. Taken lower-EL SError while standing on QEMU 10
virtTCG +-cpu cortex-a76(and the documented GICv3 /virtualization=on/secure=on/maxvariants probed above) remains deferred / non-goal and locked. ADR-053 reopen gate §4 is not met. - Park stays Verified. Smoke must keep requiring
el0: serror-park. Do not requireel0: serror. - A-clear plumbing stays dormant prep. Keep
el0: serror-arm/eret_to_el0_serror/ EXPECT / QMP attempt from ADR-045. Do not delete; do not claim taken. - Explicit blocker (what would unlock). Sponsor must approve one of:
- B1 — Upstream / pinned QEMU:
virtimplementsTYPE_NMIsuch thatinject-nmiraises async SError (ARM_CPU_SERROR) on the accepted smoke CPU/GIC; re-probe dated; serialel0: serrorunder EXPECT; new ADR citing working evidence; or - B2 — Sponsor smoke-machine ADR: a documented, CI-runnable machine/accel/tooling change that honestly delivers async SError (not FIQ/BRK/FEAT_NMI-as-SError; not silent invent); same EXPECT + evidence ADR; or
- B3 — Permanent non-goal: sponsor records that taken SError will not be pursued further under virt TCG (locks remain; park stays Verified).
- B1 — Upstream / pinned QEMU:
- Still non-claims: umbrella “EL0 isolated” (ADR-055); never yank
_start. PAN enable remains Verified separately (ADR-080).
Honesty
Say: “taken lower-EL SError stays hard-stopped on virt TCG after ADR-081 re-probe on cortex-a76; inject-nmi still returns machine does not provide NMIs; unlock needs sponsor B1/B2/B3.” Do not say:
- taken lower-EL SError is Verified
- “EL0 isolated” / “secure OS”
- GICv3 /
virtualization=on/ FEAT_NMI made inject work - guest-only RAS inject is Verified
- KVM was used on this CI smoke path
Follow-on (sponsor B1)
Sponsor chose B1 (2026-09-19 ~08:58 CEST). Follow-on research: ADR-082 — no shippable upstream/apt pin this session; virt through QEMU master still lacks TYPE_NMI→async SError. Park stays Verified; taken stays blocked. Not B2/B3.
Follow-on
ADR-083 (2026-09-19): pinned QEMU opt-in Verifies taken el0: serror; stock park path unchanged.
Follow-on (sponsor B2)
Sponsor chose B2 (2026-09-25 23:44 CEST). ADR-084: free GitHub-hosted runners expose no arm64 KVM/HVF/WHPX; stock TCG there still says machine does not provide NMIs. B2 stays Planned / blocked on a sponsor choice (self-hosted arm64 KVM, paid bare metal, or wait). B1 opt-in pin unchanged; B1 ≠ B2.
Consequences
- Docs: this ADR; ADR-053 / ADR-045 / ADR-055 / ADR-060 / ADR-079 cross-links; honesty ledger; roadmap P-SEC-3r; threat-model v1.56; SUMMARY; el0 / limits / security light touch; research brief under
research/daily-briefs/. - Smoke /
qemu-serial-inject.py: keep QMP attempt + park requirement; log cites ADR-081. Nosrc/change required (dormant prep already present). - Preserve all existing smoke greps. CloudAgent HELD. Do not self-merge (ADR-002).
ADR-082 — B1 follow-on: upstream/pinned QEMU TYPE_NMI → async SError (blocker)
- Status: Accepted (docs / research). Sponsor chose B1 (2026-09-19 ~08:58 CEST) after ADR-081. Dated upstream + agent-box research: no shippable QEMU pin this session makes
inject-nmideliver real taken SError onvirtTCG. Taken path stays deferred / non-goal (locked).el0: serror-parkstays Verified. Do not markel0: serrorVerified. Not B2/B3. - Date: 2026-09-19
- Tracks #125. CloudAgent HELD — agent-box / amd64 workstation / GHA only.
Context
ADR-081 re-probed taken lower-EL SError on default -cpu cortex-a76 and recorded unlock options B1/B2/B3. Sponsor selected B1 — Upstream / pinned QEMU so virt implements TYPE_NMI such that QMP inject-nmi / HMP nmi raises async SError (ARM_CPU_SERROR / taken path → serial el0: serror under EXPECT).
Do not invent FIQ / BRK / FEAT_NMI as SError. Do not silent machine invent. Keep _start at 0x4008_0000. Umbrella “EL0 isolated” stays non-claim (ADR-055 / ADR-060).
Research (agent-box, 2026-09-19 CEST)
Host: agent-box (<agent-vm-hostname>). qemu-system-aarch64 10.0.13 (Debian 1:10.0.13+ds-0+deb13u1). Accel: TCG only. Default smoke CPU: -cpu cortex-a76 (ADR-079).
Live inject matrix (paused -S; QMP only)
Brief: research/daily-briefs/2026-09-19-adr-082-b1-qemu-nmi.md. Full log: research/daily-briefs/2026-09-19-adr-082-b1-qemu-nmi-probe-script.txt. Repro: ./scripts/qemu-nmi-probe.py.
| Config | QMP inject-nmi |
|---|---|
virt + cortex-a76 (default smoke) | error machine does not provide NMIs |
virt + cortex-a57 (historical) | same |
virt,gic-version=3 + cortex-a76 | same |
virt,gic-version=3 + max | same |
virt,virtualization=on + cortex-a76 | same |
virt,gic-version=3,virtualization=on + cortex-a76 | same |
virt,gic-version=3,secure=on + cortex-a76 | same |
Same failure class as ADR-081. No machine/CPU flag on this package exposes NMI.
Upstream QEMU (source inspection, 2026-09-19)
| Tree | hw/arm/virt.c implements TYPE_NMI? | Notes |
|---|---|---|
| Debian / agent-box 10.0.13 | No | inject-nmi → machine does not provide NMIs |
| Upstream tag v10.0.0 | No | rg TYPE_NMI empty in virt.c |
| Upstream tag v10.1.0 | No | same |
| Upstream master (fetched 2026-09-19) | No | FEAT_NMI / GICv3 has-nmi wires ARM_CPU_NMI (architectural interrupt NMI), not async SError, and does not register TYPE_NMI for QMP inject-nmi |
Historical series that would have satisfied B1:
- 2019–2020 qemu-devel “Simulate NMI Injection” / “Support SError injection” (Gavin Shan et al.): proposed
TYPE_NMIon virt → raiseARM_CPU_SERROR. Never merged into virt. - 2024 FEAT_NMI / FEAT_GICv3_NMI series: different architecture (PPI/SPI NMI interrupts +
PSTATE.ALLINT). Not a substitute for async SError Verified. - 2026-08 qemu-devel
nmi_injectAPI cleanup: still “machine does not provide NMIs” when noTYPE_NMIobject exists.
CPU_INTERRUPT_VSERR / EXCP_VSERR in current target/arm is the virtual SError (EL2→EL1) path — not a host QMP inject into our EL1/EL0 standing TCG guest.
CI pin options evaluated
| Option | CI-runnable? | Delivers el0: serror? | 2026-09-19 verdict |
|---|---|---|---|
| Apt pin newer Debian qemu | Yes (GHA apt-get) | No — 10.0.x / published 10.1 packages still lack virt TYPE_NMI→SError | Rejected as unlock |
Build upstream master from source + cache | Possible (slow; multi-min) | No — master also lacks the wiring | Rejected as unlock |
Local QEMU patch (draft under research/qemu-nmi/) + build-from-source | Possible next session | Would, if patch verified under EXPECT | Not shipped this session — needs sponsor-reviewed patch + reproducible Dockerfile/GHA cache before smoke can require el0: serror |
Machine flag / -cpu *,nmi=on alone | N/A | No — Property 'max-arm-cpu.nmi' not found on 10.0.13; FEAT_NMI ≠ SError | Forbidden as Verified |
Draft local-patch sketch (research only, not CI-wired): research/qemu-nmi/README.md + research/qemu-nmi/0001-hw-arm-virt-TYPE_NMI-raise-SError.patch.
Decision
- B1 chosen; B1 not met this session. Sponsor selected B1. Upstream and packaged QEMU (through master as of 2026-09-19) do not implement virt
TYPE_NMI→ async SError. No apt/git pin alone unlocks taken SError. - Blocker (explicit). Unlock remains: wait for upstream virt
TYPE_NMI→ARM_CPU_SERROR(or equivalent honest QMP), or a sponsor-approved pinned QEMU build (Dockerfile / GHA cache) that carries a reviewed local patch provinginject-nmi→ serialel0: serrorunder EXPECT on agent-box + GHA. Until then, hard-stop from ADR-053/081 stands. - Park stays Verified. Smoke keeps requiring
el0: serror-park. Do not requireel0: serror. - A-clear plumbing stays dormant prep. Keep ADR-045 EXPECT /
el0: serror-arm/ QMP attempt. - Not B2/B3. This ADR does not approve a different smoke machine (B2) and does not declare permanent non-goal (B3).
- Still non-claims: umbrella “EL0 isolated”; never yank
_start; FEAT_NMI / FIQ / BRK are not SError.
Honesty
Say: “sponsor chose B1; ADR-082 research finds no upstream/packaged QEMU pin that makes inject-nmi deliver taken SError on virt TCG; park stays Verified; taken stays blocked pending upstream or a reviewed pinned build.” Do not say:
- taken lower-EL SError is Verified /
el0: serroris required - a newer apt qemu “should” unlock without probe
- FEAT_NMI / GICv3 NMI is the taken-SError path
- the research patch is CI-wired or proven
- “EL0 isolated” / “secure OS”
Consequences
- Docs: this ADR; cross-links on ADR-053/081/045/055/060; honesty ledger; roadmap P-SEC-3r; threat-model v1.57; SUMMARY; el0 / limits / security light touch; research brief + probe script + draft patch under
research/qemu-nmi/. - Smoke: unchanged fail-closed park requirement. No
src/change. - Follow-on: ADR-083 ships the pinned build + opt-in
el0: serrorVerified (2026-09-19). - CloudAgent HELD. Do not self-merge (ADR-002).
ADR-083 — B1 pinned QEMU: inject-nmi → taken el0: serror (Verified opt-in)
- Status: Accepted (implementation + evidence). Sponsor B1 (ADR-081/082) met via pinned QEMU build (not upstream merge). Dated proof: QMP
inject-nmisucceeds and guest serial showsel0: serrorunder EXPECT. Stock distro QEMU path unchanged (park Verified). Umbrella “EL0 isolated” stays non-claim. - Date: 2026-09-19
- Follow-on (2026-09-26): ADR-090 (sponsor D1/D5) made
qemu-nmi-pinunconditional (repo-variable gate removed, job renamed “QEMU NMI pin smoke (taken SError)”). This reverses §6’s “do not force the compile on every PR”. It is part of the umbrella’s taken-SError evidence class together with B2-P. - Tracks #125. CloudAgent HELD — agent-box proof; amd64 workstation preferred for heavy build but offline this session (
<machine-id>intent). Do not self-merge (ADR-002).
Context
ADR-082 found no upstream/packaged QEMU pin: virt through master lacks TYPE_NMI→async SError. DSO (2026-09-19) preferred a pinned custom QEMU over waiting upstream. Draft sketch lived under research/qemu-nmi/.
Decision
- Pin QEMU v10.0.0 (
7c949c53e936aa3a658d84ab53bae5cadaa5d59c) + reviewed local patchresearch/qemu-nmi/0001-hw-arm-virt-TYPE_NMI-raise-SError.patch. - Patch semantics (honest): add TCG
EXCP_SERROR/CPU_INTERRUPT_SERROR; virt implementsTYPE_NMIsoinject-nmiraises that line. Masked byPSTATE.A. Not FEAT_NMI / FIQ / BRK. - Install prefix:
tools/qemu-nmi/via./scripts/build-qemu-nmi.sh(bin/share gitignored). - Opt-in smoke:
CTOS_QEMU=…/qemu-system-aarch64+CTOS_REQUIRE_TAKEN_SERROR=1fail-closed requires bareel0: serror. Default (unset) still requiresel0: serror-park. - Host inject timing:
qemu-serial-inject.pyissues QMPstop→inject-nmi→contonel0: serror-armso TCG cannot race past the A-clear window. - CI/Docker: default stays distro QEMU. Dockerfile
ARG CTOS_BUILD_QEMU_NMI=0opt-in. GHA job snippet in docs/dev/qemu-nmi-pin-ci.md (OAuth push lacksworkflowscope this session — paste when available). Do not force multi-minute QEMU compile on every PR. - Still non-claims: umbrella EL0 isolated; never yank
_start; park remains Verified on stock QEMU.
Evidence (2026-09-19 CEST, agent-box)
| Probe | Result |
|---|---|
CTOS_QEMU=tools/qemu-nmi/bin/… ./scripts/qemu-nmi-probe.py | inject-nmi ok on virt + cortex-a76 / a57 / GICv3 / virt-on / secure / max |
Stock qemu-system-aarch64 10.0.13 probe | still machine does not provide NMIs |
CTOS_QEMU=… CTOS_REQUIRE_TAKEN_SERROR=1 ./scripts/qemu-smoke.sh | el0: serror present; qemu-smoke: ok |
| Stock inject (no pin) | inject-nmi error + el0: serror-park |
Machine tag: (agent-box); (amd64 workstation) intent only (CloudAgent HELD / arm64 workstation offline).
Honesty
Say: “B1 met with a pinned QEMU build; taken el0: serror Verified under opt-in CTOS_QEMU + CTOS_REQUIRE_TAKEN_SERROR=1; stock QEMU still parks.” Do not say taken is Verified on distro QEMU, or that FEAT_NMI is the path, or “EL0 isolated.”
Consequences
- Docs: this ADR; ADR-082/081/053/045 cross-links; honesty ledger; SUMMARY; threat-model bump;
research/qemu-nmi/+tools/qemu-nmi/PIN.txt. - Smoke/scripts:
CTOS_QEMUhonored; taken gate opt-in; stock park preserved. - Dockerfile:
CTOS_BUILD_QEMU_NMIARG. GHA: see docs/dev/qemu-nmi-pin-ci.md (workflow file unchanged this PR).
ADR-084 — B2 spike: taken SError on a different free GitHub-hosted runner (blocked — no arm64 KVM/HVF/WHPX)
- Status: Accepted (docs / evidence). Sponsor chose B2 (“Go with B2 to unlock 1 and 2”, DSO relay 2026-09-25 23:44 CEST). ADR-081 B2 spike on free GitHub-hosted runners: impossible this session. No free hosted runner exposes a hardware accelerator that can run an AArch64 guest (
/dev/kvmabsent on arm64 Linux, HVFHV_UNSUPPORTEDon macOS arm64, no VT/WHPX on Windows arm64). Stock QEMU TCG on those runners still returnsmachine does not provide NMIs. The B2 (non-B1) path for P-SEC-3r stays Planned / blocked pending a sponsor decision. No new Verified. B1 opt-in pin (ADR-083) is unchanged and is not B2. - Date: 2026-09-25
- Follow-on (2026-09-26): sponsor chose B2-P. ADR-085 ran ctos on an AWS Graviton3
c7g.metalunder KVM and Verified a taken lower-EL SError (opt-inb2-serrorprofile, one-off paid run, not CI). This ADR’s free-runner finding stands. - Tracks P-SEC-3r (taken lower-EL SError). Builds on ADR-081 §4 B2, ADR-053 reopen gate, ADR-083. CloudAgent not used (over cap until 2026-10-04). No self-hosted runner registered. No paid / larger runner used. Do not self-merge (ADR-002).
Context
ADR-081 §4 listed three unlocks for a taken lower-EL SError while standing: B1 pinned/upstream QEMU with TYPE_NMI→async SError, B2 a documented CI-runnable machine / accel / tooling change that honestly delivers async SError, B3 permanent non-goal. B1 landed as an opt-in pinned-QEMU job (qemu-nmi-pin, repo variable CTOS_BUILD_QEMU_NMI=1, ~12–13 min; ADR-082/083, PRs #129–#132). That is still stock-virt TCG with a patched emulator.
The sponsor asked for B2 as a genuinely different path: run on a different CI machine/runner/accelerator (for example real arm64 hardware + KVM) and reach a real taken SError there — not a re-run of the B1 pin. Hard rules for this spike: free runners only (flag before paid/larger runners), no self-hosted runner registration, no CloudAgents, and if B2 is impossible, record it with evidence and stop (no slide into B1/B3 variants).
B2 hypotheses (what a genuine B2 would need)
| ID | Candidate SError source | Needs | Notes |
|---|---|---|---|
| K1 | KVM KVM_SET_VCPU_EVENTS with exception.serror_pending=1 (KVM sets HCR_EL2.VSE → guest takes a virtual SError, masked by PSTATE.A) | arm64 host with /dev/kvm + a small VMM (or QEMU change) that injects on the el0: serror-arm cue | Stock QEMU only get/puts env->serror.pending for migration (target/arm/kvm.c, target/arm/machine.c); no QMP/HMP setter. So K1 is a custom VMM, not stock QEMU. |
| K2 | QEMU inject-nmi under -accel kvm | same host | inject-nmi is machine-level (hw/core/nmi.c walks for a TYPE_NMI object). No hw/arm/* implements TYPE_NMI on QEMU master (code search 2026-09-25: only hppa / i386 / m68k / ppc / s390x / xtensa / macio). So K2 fails regardless of accelerator. |
| K3 | Guest access to an unbacked / external-abort region reported asynchronously | real hardware or KVM | Under KVM, unbacked IPA → MMIO exit to the VMM; QEMU can only inject a synchronous external data abort (KVM_CAP_ARM_INJECT_EXT_DABT). Async reporting is implementation-defined and not reproducible on a CI VM. Also a guest-only source (ADR-044 H2 rejected for Verified). |
| H1’ | Stock TCG inject-nmi on a different runner (native arm64) | any runner | Same QEMU code as the agent box → expected machine does not provide NMIs. Control row. |
K1 is the only honest B2 SError source, and it requires an arm64 host that exposes a hypervisor to the job. So the spike’s first question is: does any free GitHub-hosted runner expose one?
Probe (GitHub Actions, 2026-09-25 CEST)
Repo artofdream/ctos is public. Standard GitHub-hosted runners (including ubuntu-24.04-arm, ubuntu-22.04-arm, macos-15, windows-11-arm) are free for public repos under GitHub’s Actions billing policy. No larger runner labels used. Temporary branch probe/adr-084-b2-kvm (workflow b2-serror-probe, research only, not a gate; copy kept at research/daily-briefs/2026-09-25-adr-084-b2-probe-workflow.yml.txt).
- Run 1 (
c198bf1, 23:47 CEST): 36193423604 — facts + KVM/HVF/Hyper-V checks (22.04-arm job failed only because the ADR-082 probe script races QEMU 6.2, which lackscortex-a76; not a finding). - Run 2 (
749cd32, 23:49 CEST): 36193613972 — all 5 jobs green; adds an all-machineTYPE_NMIsweep and an unconditional HVF boot attempt.
| Runner (image) | Host | Accelerator result (exact log line) | Stock TCG inject-nmi |
|---|---|---|---|
ubuntu-24.04-arm (20260920.129.1) job | Azure Hyper-V VM, Neoverse-N2 (Cobalt 100), kernel 6.17.0-1022-azure | dmesg kvm [1]: HYP mode not available; b2-probe: /dev/kvm=absent; QEMU 8.2.2 -accel kvm: Could not access KVM kernel module: No such file or directory / failed to initialize kvm | virt × a76/a57/GICv3/max/virt-on/secure: all machine does not provide NMIs; sweep machines-probed=118 inject-nmi-ok=0 |
ubuntu-22.04-arm (20260920.137.1) job | same class, kernel 6.8.0-1064-azure | kvm [1]: HYP mode not available; /dev/kvm=absent; QEMU 6.2 -accel kvm same failure | sweep machines-probed=101 inject-nmi-ok=0 (QEMU 6.2 wording this feature or command is not currently supported) |
macos-15 arm64 (20260907.0337.1) job | Apple M1 (Virtual), Darwin 24.6.0 RELEASE_ARM64_VMAPPLE | sysctl: unknown oid 'kern.hv_support'; kern.hv_vmm_present: 1; QEMU 11.1.1 -accel hvf: Error: ret = HV_UNSUPPORTED (0xfae9400f, at ../target/arm/hvf/hvf.c:1269) | n/a (HVF has no SError inject API anyway) |
windows-11-arm (20260920.164.1) job | Cobalt 100 Hyper-V guest | VirtualizationFirmwareEnabled : False, VMMonitorModeExtensions : False, HypervisorPlatform (WHPX) state = Disabled (enabling needs a reboot; no nested virt) | n/a |
ubuntu-24.04 x86_64 (control) job | AMD EPYC (7763 run 1, 9V74 run 2), Hyper-V | /dev/kvm=present (kvm_amd) but qemu-system-aarch64 -accel kvm: invalid accelerator kvm (x86 KVM cannot run an AArch64 guest) | same machine does not provide NMIs; sweep machines-probed=117 inject-nmi-ok=0 |
Sweep note: on ubuntu-24.04-arm, 92 machine models answered machine does not provide NMIs and 26 did not start without board-specific -cpu/firmware (for example sbsa-ref, xlnx-zcu102); the QEMU source search above shows none of them implements TYPE_NMI either.
Consistent with public upstream reports: actions/runner-images#14062 (“KVM on ARM runners” — Azure arm64 VMs lack nested virtualization).
Decision
- B2 on free GitHub-hosted runners is impossible (2026-09-25). No free hosted runner can run ctos under a hardware accelerator (KVM/HVF/WHPX) on AArch64, so the only honest B2 SError source (K1, KVM virtual SError) cannot be reached. TCG on a different runner is the same emulator code and still has no NMI/SError path (H1’ control).
- Stop here. No B2 CI job, no new smoke marker, no
src/change. Do not slide into a B1 variant (patched QEMU on the arm runner) or B3. Do not relabel the B1 pin as B2. - Status stays honest.
- Stock QEMU TCG: taken SError deferred / non-goal (hard-stopped); smoke keeps requiring
el0: serror-park(ADR-053/081). - B1 opt-in pin: taken
el0: serrorVerified (opt-in) only inqemu-nmi-pin(ADR-083). Unchanged. - B2 (different runner/accelerator): Planned / blocked — probe ran and lost on free runners (ledger row Failed for the free-runner probe).
- Stock QEMU TCG: taken SError deferred / non-goal (hard-stopped); smoke keeps requiring
- What would unlock B2 (sponsor decision required; none taken here):
- B2-S — self-hosted arm64 runner with KVM on sponsor hardware (bare-metal arm64 Linux booted at EL2, e.g. an Ampere/Graviton-metal box, Raspberry Pi 5, or an Apple-silicon machine running Linux natively). Security change: registering a self-hosted runner on a public repo needs explicit sponsor go (fork PRs must not reach it). Not done.
- B2-P — paid bare-metal arm64 (cloud
*.metalor a third-party runner vendor that exposes arm64 KVM). Costs money; flagged, not used. (WarpBuild documents nested virt on x64 only.) - B2-W — wait for GitHub/Azure to offer nested virtualization on arm64 hosted runners (runner-images#14062), then re-run this probe (
workflow_dispatchcopy in the research brief). Any of those still needs K1 tooling: a small KVM VMM (or reviewed QEMU change) that loads the ctos ELF at0x4008_0000, provides PL011 + GIC, and callsKVM_SET_VCPU_EVENTS(serror_pending=1, optional ESR viaKVM_CAP_ARM_INJECT_SERROR_ESR) onel0: serror-arm; plus a fail-closed smoke that requires bareel0: serrorand fails on absence. That would be a new milestone ADR.
- Still non-claims: umbrella “EL0 isolated” (P-SEC-3l; ADR-055 / ADR-060) — out of scope here; never yank
_start(0x4008_0000).
Honesty
Say: “ADR-084 B2 spike: free GitHub-hosted runners expose no arm64 KVM/HVF/WHPX (runs 36193423604 / 36193613972, 2026-09-25); TCG there still says machine does not provide NMIs; B2 is Planned / blocked on a sponsor choice (self-hosted arm64 KVM, paid bare metal, or wait). Taken SError stays Verified only via the B1 opt-in pin.” Do not say:
- B2 is Verified, or taken SError is Verified on a hosted arm64 runner
- KVM was used anywhere on ctos CI
- the B1 pin job is a B2 result
- FEAT_NMI / FIQ / BRK / a synchronous external abort is an SError
- “EL0 isolated” / “secure OS”
Consequences
- Docs: this ADR; ADR-081 follow-on; ADR-053 / ADR-055 / ADR-060 cross-links; honesty ledger; roadmap P-SEC-3r; threat model v1.59; SUMMARY; research brief + raw log excerpt + probe workflow copy under
research/daily-briefs/; session memory underresearch/random-thoughts/. - CI:
.github/workflows/unchanged onmainby this PR. The temporary probe branch/workflow lived only onprobe/adr-084-b2-kvm. Existing smoke (el0: serror-parkrequired) and opt-inqemu-nmi-pinunchanged. - Cost: all probe minutes on free standard runners of a public repo. No paid, larger, or self-hosted runner.
ADR-085 — B2-P: taken lower-EL SError under KVM on real arm64 (AWS Graviton3 c7g.metal) — Verified (opt-in, one-off paid run)
- Status: Accepted. Sponsor GO for B2-P (paid Graviton bare metal), DSO relay 2026-09-26 11:28 CEST. Follow-on to ADR-084 §4 (B2-P). Result: taken lower-EL SError Verified under KVM on real Graviton3 hardware in a one-off, opt-in
b2-serrorprofile (run 4, 2026-09-26 13:26–13:33 CEST). Not a CI gate: no ctos CI job runs KVM. Stock-QEMU TCG smoke still parks; the B1 opt-in pin (ADR-083) is unchanged and is not B2. - Date: 2026-09-26
- Tracks P-SEC-3r (taken lower-EL SError). Builds on ADR-081 §4 B2, ADR-084 K1, ADR-045 standing-EL0 A-clear window, ADR-053. CloudAgent not used. No self-hosted runner. Do not self-merge (ADR-002). Item 2 (umbrella “EL0 isolated”) not started. It stays gated.
- Follow-on (2026-09-26): run 4 is the B2-P leg of the umbrella’s taken-SError evidence class (ADR-090, sponsor D1). It is historical (at
159b178, before the M1/M2 boot-path changes). A KVM re-verify is ≈ $0.2 and is a sponsor option, not required.
Context
ADR-084 showed that no free GitHub-hosted runner exposes an arm64 hypervisor. The only honest B2 SError source is K1: a KVM host sets serror_pending through KVM_SET_VCPU_EVENTS, KVM raises a virtual SError (HCR_EL2.VSE), and the guest takes it when PSTATE.A is clear. That path needs real arm64 hardware booted at EL2. The sponsor chose B2-P under these constraints: a paid AWS Graviton .metal box, cheapest type/region, Spot, tags project=ctos / component=b2-spike on every resource, a self-terminate timer, no standing resources, stop if the estimate exceeds ~$20, and a sanctioned credentials path only.
Design (what is different from B1)
| B1 (ADR-083) | B2-P (this ADR) | |
|---|---|---|
| Where the guest runs | QEMU TCG (emulated CPU) | Real Neoverse-V1 (Graviton3), QEMU -accel kvm -cpu host |
| Who delivers the SError | QEMU TCG raises an emulated async SError (patch 0001, TYPE_NMI on virt) | KVM / hardware: patch 0002 turns inject-nmi under kvm_enabled() into env.serror.pending=1; QEMU’s resume path (kvm_arch_put_registers → kvm_put_vcpu_events) issues KVM_SET_VCPU_EVENTS{serror_pending=1}; KVM sets HCR_EL2.VSE; the CPU takes a virtual SError when EL0 runs with A clear |
| Host | GitHub ubuntu-24.04-arm (free) or agent box | AWS c7g.metal, eu-north-1b, Spot |
| Gate | opt-in CI job qemu-nmi-pin | one-off paid run; no CI job |
Pieces (branch isolation/adr-085-b2p-graviton-kvm-serror):
research/qemu-nmi/0002-hw-arm-virt-TYPE_NMI-KVM-serror-pending.patch, applied on top of B1’s 0001 on QEMU v10.0.0 viascripts/build-qemu-b2.sh(outputtools/qemu-b2/, gitignored).- Opt-in Cargo feature
b2-serror: after the normal paging/identity-teardown boot, it printsb2: boot, runs only the standing-EL0 SError window (el0::observe_b2_serror), thenb2: done. It skips GIC/timer/virtio/FAT, because the host KVM exposes GICv3 only (kvm [1]: disabling GICv2 emulation). It also adds raw early markersb2: e0/e1/e2/p0/p1and a longer outer spin (MOVZ X3,#0xffff) for native speed. The default build is unchanged apart from the TG1 fix below. scripts/b2-kvm-smoke.py: fail-closed. Exit 0 only if a bareel0: serrorline andb2: takenappear with no park marker. If the guest is silent for 15 s it dumps vCPU registers (QMPhuman-monitor-command info registers) as diagnostics. That dump is never a pass signal.scripts/b2-metal-userdata.sh: no-SSH user-data (hardshutdown -h +80;InstanceInitiatedShutdownBehavior=terminate). It clones the branch, builds patched QEMU + theb2-serrorguest natively, runs the smoke up to 3×, printsCTOSB2:lines to the serial console (read withec2 get-console-output --latest), then powers off.
Runs (2026-09-26, eu-north-1, all tagged project=ctos, component=b2-spike)
Credentials path: the sanctioned user-Aws-mcp aws___run_script (IAM user <iam-user>, account <aws-account-id>). Security group ctos-b2-spike had no ingress. No key pair and no IAM role were created.
| # | Instance | Type / market | Launch (CEST) | End | Result |
|---|---|---|---|---|---|
| 0 | <instance-id-run0> | m6g.metal (Graviton2) Spot | 12:22:26 | self-poweroff ≈12:28:50 | Launched by a batch RunInstances whose MCP response timed out; not noticed until flagged. HOME unset in cloud-init → cargo: command not found. Patched QEMU built with kvm on Graviton2; smoke failed closed (no ELF). |
| 1 | <instance-id-run1> | c7g.metal Spot | 12:23:50 | terminated by me 12:29:29 | Same HOME bug; terminated early (at most one metal box at a time). |
| 2 | <instance-id-run2> | c7g.metal Spot | 12:42:59 | self-poweroff ≈12:55:15 | Guest built. Under KVM: zero guest UART bytes in 3×90 s → FAIL (fail-closed). |
| 3 | <instance-id-run3> | c7g.metal Spot | 13:06:27 | self-poweroff ≈13:17:31 | Early markers reached b2: p1 mmu-on, then stalled. Register dump: PC=ffffff8040080a00 (high-alias VBAR 0xffffff8040080800 + 0x200), PSTATE=a00003c5 EL1h, identity LR → recursive abort on the vector fetch through TTBR1. A control keeping the pre-MMU spin locks behaved identically, so the “pre-MMU exclusives” hypothesis was refuted. |
| 4 | <instance-id-run4> | c7g.metal Spot | 13:26:02 | self-poweroff ≈13:33:33 | PASS — taken lower-EL SError under KVM (below). |
Early tries at 12:1x CEST got InsufficientInstanceCapacity for c6g.metal Spot in eu-north-1a/b/c. Spot placement scores (read-only) then pointed to c7g.metal. ap-south-1 was cheaper but its vCPU quota is 5, so unusable.
Root cause found on real hardware (run 3) → fix
TCR_EL1 was programmed without TG1, so TG1 stayed 0b00. That encoding is reserved for TG1 (01=16K, 10=4K, 11=64K; TG0 differs). QEMU TCG sanitises a reserved TG1 to 4K, so the TTBR1 high alias (ADR-016…ADR-019) always worked there. On Graviton3 the reserved value selected a different granule, so every high-VA walk failed. Fix: TCR_TG1_4K = 0b10 << 30 in src/paging.rs (commit 159b178). It applies globally because it is a latent real-hardware bug, and it is identical on TCG: default smoke at 159b178 was green on both runners (run 36238549525).
Run 4 evidence (serial console, verbatim)
CTOSB2: cpu Vendor ID: ARM;BIOS Vendor ID: Amazon EC2;Model name: Neoverse-V1;BIOS Model name: AWS Graviton3 N/A CPU @ 2.6GHz;
CTOSB2: dmesg [ 0.026567] CPU: All CPU(s) started at EL2
CTOSB2: dmesg [ 0.228100] kvm [1]: VHE mode initialized successfully
CTOSB2: /dev/kvm=present
CTOSB2: head 159b17818fb6d22497de0544c0693844876571cc
CTOSB2: qemu build-qemu-b2: accelerators: Accelerators supported in QEMU binary: kvm tcg
CTOSB2: b2-kvm-smoke: cmd: /root/ctos/tools/qemu-b2/bin/qemu-system-aarch64 -accel kvm -machine virt,gic-version=3 -cpu host -m 128M ... -kernel target-b2/aarch64-ctos/debug/ctos
CTOSB2: b2: p1 mmu-on
CTOSB2: ident: jump
CTOSB2: pan: enabled
CTOSB2: pan: el1-fault
CTOSB2: ident: ram lo=0x4026a000 hi=0x48000000 pages=32150
CTOSB2: b2: boot (ADR-085 KVM SError profile)
CTOSB2: el0: serror-armb2-kvm-smoke: qmp stop=>ok inject-nmi=>ok cont=>ok (ADR-085: KVM_SET_VCPU_EVENTS serror_pending)
CTOSB2: el0: standing
CTOSB2: el0: serror
CTOSB2: el0: restored
CTOSB2: b2: taken
CTOSB2: b2-kvm-smoke: summary armed=True injected=True taken_marker=True b2_taken=True park=False kvm_err=False
CTOSB2: b2-kvm-smoke: PASS taken lower-EL SError observed under accel=kvm ('el0: serror' + 'b2: taken')
CTOSB2: smoke attempt 1 rc=0
(el0: serror-arm and the host’s QMP log line share one console line because guest serial and host log are tee’d to the same console.) Full excerpts: research/daily-briefs/2026-09-26-adr-085-b2p-console.txt.
The bare el0: serror line is printed only by the lower-EL SError vector handler when the standing-EL0 expectation is armed (src/exception.rs). Under KVM the guest runs natively, so the only SError delivery path is KVM’s virtual SError that the host requested via serror_pending. Limitation: no host-side strace of the ioctl was captured. The mechanism is QEMU’s documented kvm_put_vcpu_events path, and the fail-closed marker was observed.
Cost
Spot, per-second billing. Total metal time across runs 0–4 ≈ 43 min: c7g.metal ≈ 36.6 min at $0.2476/h plus m6g.metal ≈ 6.4 min at ≈$0.26/h. Prices come from DescribeSpotPriceHistory in eu-north-1b, 2026-09-26. Metal ≈ $0.18. Add gp3 30 GB and public IPv4 for ≈0.7 h (< $0.01). Total ≈ $0.19, far under the $20 gate.
Decision
- B2 (different accelerator: real arm64 + KVM): Verified, opt-in, one-off. It rests on run 4 above, a probe from this session. It is not a standing CI gate. Re-running needs another paid metal run (user-data script in-tree) or a self-hosted arm64 KVM runner (B2-S, still not registered).
- Stock QEMU TCG smoke: unchanged. It still requires
el0: serror-park(ADR-053/081). - B1 pin: unchanged, Verified (opt-in, TCG). Still not B2.
- TG1=4K is a real correctness fix for real hardware. The TTBR1 alias was only “Verified” on TCG before this ADR.
- Still non-claims: umbrella “EL0 isolated” (P-SEC-3l; ADR-055/060) is not claimed and not started.
_startstays at0x4008_0000. This is not “ctos runs on real hardware” in general: only theb2-serrorprofile ran, with no GIC/timer/virtio/FAT.
Honesty
Say: “ADR-085 B2-P: on an AWS Graviton3 c7g.metal (eu-north-1, Spot, 2026-09-26), QEMU -accel kvm with patch 0002 set serror_pending via KVM_SET_VCPU_EVENTS on the el0: serror-arm cue, and the ctos b2-serror profile took the lower-EL SError while standing at EL0 (el0: serror / b2: taken, fail-closed smoke PASS). One-off paid run, not CI. Found and fixed a latent TCR_EL1.TG1 bug that TCG masked.” Do not say:
- taken SError is Verified on stock QEMU, on free runners, or in CI under KVM
- the default ctos image boots on real hardware (only the b2 profile ran; GIC/timer/virtio untested under KVM)
- the B1 pin job is B2 evidence
- “EL0 isolated” / “secure OS”
Consequences
- Code: opt-in
b2-serrorfeature (src/main.rs,src/el0.rs, early markers insrc/paging.rs); globalTCR_TG1_4K; patch 0002;scripts/build-qemu-b2.sh,scripts/b2-kvm-smoke.py,scripts/b2-metal-userdata.sh;.gitignore/tools/qemu-b2/. - CI:
.github/workflows/unchanged. Default smoke and opt-inqemu-nmi-pinstay green (run 36238549525). - Docs: this ADR; ADR-084 follow-on; honesty ledger; roadmap P-SEC-3r; threat model v1.60; SUMMARY; research brief + console excerpt; session memory.
- AWS: all resources deleted; tag audit in the research brief.
ADR-086 — Umbrella “EL0 isolated” checklist audit after B2-P (gap list + milestone plan)
- Status: Accepted (docs / audit). The umbrella “EL0 isolated” (P-SEC-3l) stays Planned / non-claim. This ADR audits every ADR-055 / ADR-060 checklist item against evidence gathered this session (2026-09-26). It publishes a measured TTBR0 identity inventory, a gap list, and a milestone plan (one milestone per branch/PR). It mints no new Verified for the umbrella, taken SError on the accepted smoke machine, or full identity teardown.
- Follow-on (2026-09-28, M6): B3 met: the sponsor accepted the ADR-091 revision-3 sentence and scope in writing (2026-09-28 16:41 CEST). ADR-091 is Accepted. “EL0 isolated” is Verified only as that sentence, with its limits. Text in this ADR that says the umbrella is Planned / non-claim is as of this ADR’s date.
- Date: 2026-09-26
- Trigger: sponsor relay (DSO, 2026-09-26 13:55 CEST): item 2 (umbrella EL0 isolated) is unlocked now that B2 has a result (ADR-085). “Claim it ONLY when every checklist item is Verified by a probe from this session.”
- Base: stacked on PR #137 (
915796f). #136 / #137 are not merged at the time of writing, andmain@4ea871ehas neither. - Numbering: next free after ADR-085.
Context
ADR-055 §B names three Unmet rows that block the umbrella: (1) taken lower-EL SError while standing, (2) full identity teardown including yank _start, (3) a written sponsor accept that the umbrella sentence is in scope. ADR-060 is the sponsor one-pager of the same locks. B1 (ADR-083) and B2-P (ADR-085) have since produced a taken el0: serror, but only on opt-in paths. The identity tears ADR-018…038 plus ADR-049 landed, while _start stays (ADR-042 / ADR-047 §3, ident: start-stay).
This ADR answers: which checklist items are Verified today, which are not, and what exactly is left.
Evidence gathered this session
| Probe | When (CEST) | What | Result |
|---|---|---|---|
| Agent-box default smoke | 2026-09-26 13:59–14:04 | scripts/qemu-smoke.sh on 915796f, stock QEMU 10.0.13 (Debian), virt -cpu cortex-a76 TCG, rustc 1.100.0-nightly 0fc141305 (repo pin; corrected in ADR-087 — this line first said 5ceaf6608, the box’s +nightly) | qemu-smoke: ok; force-fail exit 1 (fail-closed ok). All §A markers below present. inject-nmi=>error:machine does not provide NMIs → el0: serror-park. |
| GHA PR run 36239619910 (+ push run 36239617837) | 2026-09-26 13:41–13:53 | smoke.yml on 915796f: ubuntu-24.04-arm + ubuntu-24.04 default jobs; opt-in qemu-nmi-pin (B1) | All success. Default: park. qemu-nmi-pin job 108397549350: inject-nmi=>ok → bare el0: serror → qemu-smoke: taken 'el0: serror' present (ADR-083 pinned QEMU). |
| B2-P run 4 (ADR-085) | 2026-09-26 13:26–13:33 | AWS c7g.metal KVM, opt-in b2-serror profile, one-off paid | el0: serror / b2: taken / b2-kvm-smoke: PASS. Not CI. |
Scratch TTBR0 inventory (not committed to src/) | 2026-09-26 ~14:10 | 915796f + scratch walker patch: walks kernel and user TTBR0 L1→L3, coalesces leaves, prints just before Hello World! (after every tear). Same QEMU/rustc; smoke rc=0 | Output. Five identity (VA = PA) ranges remain in kernel TTBR0 and two in user TTBR0. See §Inventory. |
Inventory: what is still identity-mapped after every tear
Kernel TTBR0 (the hello boot; the other two boots in the smoke show the same classes, with boundaries shifted by text size):
| # | Range | Size | Attrs | What it is | Status vs “full identity teardown” |
|---|---|---|---|---|---|
| I1 | 0x0000_0000–0x4000_0000 | 1 GiB L1 block | Device-nGnRnE, EL1 RW, PXN+UXN | MMIO identity: GIC 0x0800_0000, PL011 0x0900_0000, virtio-mmio 0x0a00_0000 (all accessed through identity VAs) | Not torn. Needs high MMIO alias in TTBR1 first. |
| I2 | 0x4000_0000–0x4008_0000 | 512 KiB | Normal, EL1 RW, XN | RAM below the image (QEMU DTB hole). ctos never reads the DTB (frame.rs: “never handed out”) | Not torn. Nothing uses it. |
| I3 | 0x4008_0000–0x4008_1000 | 1 page | Normal, EL1 RO+X | Boot stub page: _start + exception_vectors (identity copy; VBAR is high) | Stays by decision (ident: start-stay, ADR-042 / ADR-047 §3). |
| I4 | [.ident_tear end, 0x4020_1000) | ≈ 1.06 MiB (e.g. 0x400f1000–0x40201000) | Normal, EL1 RO+X | Linker padding between the last torn RO section and .data (pinned at 0x4020_1000). No live code. | Not torn. The ADR-019/020/025 tears cover .text after the stub, .rodata, and .ident_tear only. |
| I5 | __kernel_end page (0x4025d000) | 1 page | Normal, EL1 RW, XN | The page between __fatal_stack_top / __kernel_end and the heap (ident: heap lo=0x4025e000) | Not torn. Falls between the ADR-037 data tear (hi=__kernel_end) and the ADR-038 heap tear. |
User TTBR0 (what EL0 runs on; all EL1-only, el0=false): I3 (boot stub, RO+X) and I4 (padding tail, RO+X). Kernel .text/.rodata/.data/heap/RAM are already absent from user TTBR0.
Everything else (live .text after the stub, .rodata, .ident_tear, .data/.bss/stacks, heap, and the frame pool up to 0x4800_0000) is torn, which matches the Verified tear rows. The inventory is a scratch probe: it is Documented evidence for this audit, not a committed fail-closed ratchet. That ratchet is milestone M1.
Checklist audit
Status words: Verified = backed by a probe from this session (box and/or CI above). CI-gated = scripts/qemu-smoke.sh fails closed without the marker on the default GHA jobs. Note that main has no branch protection (GET /branches/main/protection → 404), so “CI-gated” means the sponsor does not merge red. No check is GitHub-required.
ADR-055 §A — Verified miles (necessary, not sufficient)
| # | Item | Status | Probe / marker (this session) | Recorded | CI-gated? |
|---|---|---|---|---|---|
| A1 | EL0 first mile + NX kernel data | Verified | el0: ok, el0: nx kernel | Ledger ADR-013 rows; box 14:04 + run 36239619910 | Yes |
| A2 | User TTBR0 omits kernel data | Verified | el0: no kernel read (ADR-055 lists el0: no data, which is stale wording: the code and smoke use el0: no kernel read) | Ledger ADR-013 | Yes |
| A3 | Standing EL0 + standing task | Verified | el0: standing, el0: restored, el0: task-enter/active/exit/restored, el0: task-ok | Ledger ADR-013/024 | Yes |
| A4 | ASID isolation | Verified | asid: dual / asid: conflict / asid: ok | Ledger ASID row | Yes |
| A5 | TTBR1 private page + EL1 high fetch | Verified | ttbr1: el1 / ttbr1: no el0 / ttbr1: ok, ttbr1: el1 exec / ttbr1: vbar | Ledger ADR-016/017 | Yes |
| A6 | Identity tears text / rodata / data / heap / RAM | Verified (as specific tears) | ident: live/range/text, ident: rodata(+-fault/-high), ident: data(+…), ident: heap(+…), ident: ram(+-fault/-high), ident: ok; smoke rejects ident: heap-stay / ident: ram-stay | Ledger ADR-018…020/025/037/038/049 | Yes |
| A7 | EL0 entry without TLBI VMALLE1 | Verified | el0: no-vmalle1 + smoke greps src/exception.rs for no tlbi vmalle1 | Ledger ADR-039 | Yes |
| A8 | Lower-EL IRQ while standing + default I clear | Verified | el0: irq, el0: irq-default | Ledger ADR-040/041 | Yes |
| A9 | Lower-EL FIQ while standing | Verified | el0: fiq | Ledger ADR-043 | Yes |
| A10 | SError park honesty (default machine) | Verified | el0: serror-arm → inject-nmi=>error:machine does not provide NMIs → el0: serror-park | Ledger ADR-043/045/053 | Yes |
| A11 | PAN ID field | Verified | pan: id= / pan: present (cortex-a76) | Ledger ADR-079 | Yes |
| A12 | PAN enable + EL1-vs-EL0 fault | Verified | pan: enabled, pan: el1-fault | Ledger ADR-080 | Yes |
| A13 | _start stay honesty | Verified (honesty; not a teardown) | ident: start-stay lo=0x40080000 hi=0x40081000 | Ledger ADR-042 | Yes |
ADR-055 §B / ADR-060 — umbrella blockers
| # | Item | Status | Evidence | Honest reading | CI-gated? |
|---|---|---|---|---|---|
| B1 | Taken lower-EL SError while standing | Unmet as worded (Verified only on two opt-in, non-default paths) | Default machine: park (A10, this session). B1 pin: GHA qemu-nmi-pin job 108397549350 (this session) inject-nmi=>ok → bare el0: serror. B2-P: ADR-085 run 4 (this session) el0: serror under KVM. | ADR-060 requires “honest host inject that delivers async SError on the accepted smoke machine; serial el0: serror under EXPECT”. The accepted smoke machine is still stock distro QEMU virt TCG, which parks. B1 is CI-runnable but uses a locally patched emulator (0001 TYPE_NMI→SError). Its job runs only while repo variable CTOS_BUILD_QEMU_NMI=1, so it is opt-in, not a gate. B2-P is the only real-hardware source (KVM serror_pending), but it was one paid run on the b2 profile (no GIC/timer/virtio/FAT) and is not CI-runnable without paid metal or a self-hosted arm64 KVM runner. Neither meets the row as worded. It can count only if the sponsor redefines the accepted evidence (decision D1). | Row wording implies yes (default smoke + EXPECT). B1 is CI but opt-in. B2-P: no. |
| B2 | Full identity teardown including yank _start | Unmet, and unmeetable as worded | Scratch inventory: I1, I2, I4, I5 still identity-mapped in kernel TTBR0; I3 + I4 in user TTBR0. | The row demands yanking _start, while ADR-047 §3 decides “never yank _start” and the sponsor repeated “never yank _start at 0x4008_0000”. As written the row cannot be met. Decision D2 must redefine it. Independently of _start, four other identity ranges (I1, I2, I4, I5) are not torn, so even “full teardown except the stub” is Unmet today. | Needs a new fail-closed ratchet (M1). None exists today. |
| B3 | Written sponsor accept that the umbrella sentence is in scope | Unmet (intent Documented) | DSO relays 2026-09-25 23:44 (“Go with B2 to unlock 1 and 2”) and 2026-09-26 13:55 (“item 2 unlocked … claim ONLY when every item Verified”) | This is direction, not an ADR-048/052-style accept that names the exact sentence and its scope (machine, CPU, profile). Decision D4. | n/a (docs) |
| B4 | PAN enable (ADR-060 row) | Verified | = A12 | Met (ADR-054 gates 1–4 via ADR-079/080). | Yes |
| B5 | Yank _start (ADR-060 row) | Decided: never (not a Verified target) | ident: start-stay | Conflicts with B2 wording. See D2. | Yes (stay marker) |
| B6 | Guest Linux / containers / immutable-OS marketing | Non-goal | ADR-029 / ADR-036 | Out of scope. Not required for the umbrella, and not unlocked by it. | n/a |
Roadmap P-SEC-3l also paraphrases the umbrella as “standing + PAN + full TTBR1 / identity teardown”: standing (A3) and PAN (A12) are Verified. “Full TTBR1 / identity teardown” = B2 (Unmet).
Bottom line: 13 of 13 §A miles plus PAN enable are Verified this session. All three §B blockers are not Met. The umbrella stays Planned / non-claim.
Gap list (ordered; identity teardown first)
| Gap | What | Size | Needs sponsor? |
|---|---|---|---|
| G1 | No committed, fail-closed identity inventory ratchet. Today “what is left” is a scratch probe. | Small: walker + allowlist + smoke line + #[test_case] | No |
| G2 | I2 low RAM 0x4000_0000–0x4008_0000 (DTB hole) identity RW | Small (unused range) | No |
| G3 | I4 padding tail [.ident_tear end, 0x4020_1000) identity RO+X in kernel and user TTBR0 | Small (no live code) | No |
| G4 | I5 __kernel_end page identity RW | Small | No |
| G5 | I1 MMIO 1 GiB identity Device block: GIC / PL011 / virtio must move to a TTBR1 high alias before the tear | Medium (UART is used pre-MMU and in fault paths; the b2 profile shares the UART) | Scope confirm (D3) |
| G6 | I3 boot-stub page (_start + identity vectors) in kernel + user TTBR0 | Small code if allowed; policy-blocked | Yes (D2) |
| G7 | Taken SError on the accepted machine | Policy / CI | Yes (D1) |
| G8 | Umbrella accept ADR with the exact sentence | Docs | Yes (D4) |
| G9 | Doc drift: ADR-055 §A el0: no data ≠ code el0: no kernel read | Trivial | No (noted here; fix in M1) |
Milestone plan (one milestone per branch / PR; all TCG, no AWS)
| M | Branch (proposed) | Scope | Fail-closed markers | Gate |
|---|---|---|---|---|
| M0 (this PR) | isolation/adr-086-el0-isolated-audit | This audit + inventory evidence + plan + sponsor decisions. Docs only. | n/a | — |
| M1 | isolation/adr-087-identity-inventory-ratchet | Commit the TTBR0 walker as ident: inv + tear G2, G3, G4 (RAM-side leftovers, kernel and user TTBR0) + fix G9 | ident: inv k=<n> u=<n> + ident: inv-ok. Smoke rejects any identity leaf outside the allowlist {I1 MMIO block, I3 stub page} (ident: inv-leak …). ident: low / ident: tail / ident: kend tears with -fault probes. Keeps ident: start-stay. | None |
| M2 | isolation/adr-088-mmio-high-alias | Map GIC / PL011 / virtio-mmio in TTBR1 (Device, EL1-only), switch base VAs after the high split, tear I1 | ident: mmio-high, ident: mmio-fault (identity MMIO VA → DABORT), allowlist shrinks to {I3} | D3 (scope). Optional b2-profile KVM re-check is not required. If wanted: ≈ $0.2, report cost first. |
| M3 | isolation/adr-089-boot-stub-policy | Per D2: (a) docs ADR that defines “full identity teardown” as all identity except the _start stub page (start-stay remains), or (b) code: after the high jump, unmap the identity mapping of the stub page (the image and _start stay physically at 0x4008_0000, and QEMU -kernel only needs that PA at entry), print ident: stub-torn, retire ident: start-stay | (b): ident: stub-torn / ident: stub-fault, allowlist becomes {} → ident: inv k=0 u=0 | D2 |
| M4 | isolation/adr-090-serror-accepted-machine | Per D1: close ADR-055 §B row 1 under the sponsor-chosen evidence class (see D1). A workflow edit (e.g. making qemu-nmi-pin unconditional) is pushed from the amd64 workstation. | Taken el0: serror required under the chosen gate | D1 |
| M5 | docs/adr-091-umbrella-accept | ADR-048/052-style sponsor accept: exact sentence, machine/CPU/profile scope, the list of Met rows | n/a | D4 |
| M6 | isolation/adr-092-umbrella-flip | Only if M1–M5 are Met: fresh-session re-probe of every row, ledger/roadmap flip, sponsor merges | All of the above in one run | Sponsor merge (ADR-002) |
ADR numbers in M1–M6 are proposed. Each PR takes the next free number at open time.
Sponsor decisions needed
- D1 — Which taken-SError evidence counts for the umbrella? Options: (i) the B1 pinned-QEMU CI job as the accepted smoke machine for this row (make it unconditional; honest caveat: TCG emulator patched locally, not stock); (ii) the B2-P one-off real-hardware run as sufficient (not CI; b2 profile only); (iii) require a CI-gated real-KVM run (self-hosted arm64 KVM runner = security change, or paid metal per gate run ≈ $0.2 each, tagged + self-terminating); (iv) keep taken SError a non-goal → the umbrella is permanently non-claim. Agent recommendation: (i)+(ii) together as the evidence class, stated verbatim in the accept ADR, or (iv). Not a silent redefinition.
- D2 —
_startand “full identity teardown”. ADR-055 §B row 2 (yank_start) contradicts ADR-047 §3 (never yank). Options: (a) the umbrella accepts start-stay as the single documented identity exception; (b) authorize unmapping the stub’s identity mapping after boot while_startstays at0x4008_0000(no image move, no-kernelchange). ADR-047 §3 would then read as “never move or remove_start”. - D3 — Is device MMIO in scope of identity teardown? Recommendation: yes (I1 is the largest identity range left).
- D4 — Umbrella sentence + accept ADR. Exact wording and scope (e.g. “on QEMU
virt-cpu cortex-a76TCG smoke, plus [D1 evidence]”). - D5 — CI gating.
mainhas no branch protection. Should the flip require GitHub-required checks, or is “sponsor does not merge red” enough?
Honesty
Say: “ADR-086 audited the ADR-055/060 checklist on 2026-09-26: every §A mile and PAN enable are Verified this session. Taken SError is Verified only on opt-in paths (B1 pin, B2-P one-off), not on the accepted smoke machine. Identity teardown still leaves five identity ranges (one is _start by decision). No umbrella accept exists. ‘EL0 isolated’ stays Planned / non-claim.”
Do not say:
- “EL0 isolated” / “secure OS” / “hardened isolation complete”
- taken lower-EL SError Verified on the default smoke machine
- identity mappings fully torn down /
_startyanked / the kernel moved - that the scratch inventory is a CI-gated probe (it is Documented research until M1)
Consequences
- Ledger, roadmap P-SEC-3l, threat model v1.61, el0.md pointer, SUMMARY, second brain cite this audit.
- ADR-055 / ADR-060 get a follow-on pointer. Their decisions are not rewritten.
- No
src/change. No AWS. No CloudAgent. Do not self-merge.
ADR-087 — Identity inventory ratchet + RAM-side identity leftovers tear (M1)
- Status: Accepted (serial / tests / smoke). M1 of the ADR-086 plan. It unmaps identity leftovers I2 / I4 / I5 and commits a fail-closed TTBR0 identity inventory. Allowed set = {I1 MMIO block, I3
_startstub page}. A planted identity page must be caught in-boot, and a dedicatedinv-leak-probekernel must fail the check. The umbrella “EL0 isolated” stays Planned / non-claim. - Date: 2026-09-26
- Base: stacked on PR #138 (
3f4de00). #136 / #137 / #138 are unmerged. - Sponsor decisions (DSO relay ~14:35 CEST) arrived while this ran: D2 keeps I3 as the one documented identity exception, and D3 puts MMIO in scope (M2). This ADR does not act on them beyond keeping I1 / I3 allowed for now.
Context
The ADR-086 scratch walker found five identity (VA = PA) ranges left in kernel TTBR0 after the ADR-018…049 tears: I1 MMIO 1 GiB Device block, I2 low RAM 0x4000_0000–0x4008_0000, I3 boot-stub page 0x4008_0000, I4 RO+X linker padding [.ident_tear end, 0x4020_1000), and I5 the pre-heap frame at __kernel_end. User TTBR0 still held I3 + I4. Those tears were section-driven, so nothing ratcheted “what is still identity-mapped”.
Decision
- Tear I2 / I4 / I5 (
paging::tear_identity_leftovers, right aftertear_identity_ram) from kernel and user TTBR0. ASID-B shares the kernel RAM L2. The ranges arepaging::leftover_ranges():[VIRT_RAM_BASE, KERNEL_TEXT),[boot_stub_end, data_start), and[data_tear_end, heap_pa).pages=counts only pages that were still mapped. High twins stay. One boot-timeTLBI VMALLE1._startis never touched: the function refuses any range overlapping the stub page. - Fault probes. An EL1
LDRof a torn leftover VA must take a current-EL translation DABORT:ident: low-fault/ident: tail-fault/ident: kend-fault.is_torn_identity_valearns the three ranges. - Inventory ratchet (
paging::identity_inventory). It walks kernel, user and ASID-B TTBR0 (L1→L3), coalesces identity leaves, and prints each asident: inv-range(allowed) orident: inv-leak(not allowed, or EL0-accessible), thenident: inv k=<n> u=<n> a=<n> leaks=<n>.ident: oknow requiresleaks=0. - In-boot negative probe (
inventory_negative_probe). It plants an identity page at0x4000_0000in kernel TTBR0 and requires the walk to flag it, then does the same in user TTBR0. It then removes the plant and requires a clean walk:ident: inv-neg k caught/ident: inv-neg u caught/ident: inv-neg clean→ident: inv-ok allow=mmio,stub. - Smoke-level negative probe. Cargo feature
inv-leak-probeleaves the plant in place.scripts/qemu-smoke.shbuilds and boots it and fails unless the serial showsident: inv-leak k lo=0x40000000and noident: inv-ok/ident: ok. This is CI-gated without a workflow edit. - Smoke requires the markers in 1–4 plus
leaks=0and still requiresident: start-stay. It rejectsident: inv-leak,ident: inv missed,ident: inv-neg missed, andident: left missed.#[test_case]s:identity_leftovers_torn_stub_stays,identity_inventory_allowlist_only_catches_plant. - Doc drift fix: ADR-055 §A now says
el0: no kernel read, which is what code and smoke use. The earlier text saidel0: no data. ADR-086 also misstated the rustc on the box smoke: the repo pin is1.100.0-nightly 0fc141305, not5ceaf6608. This is corrected in place.
Evidence (this session)
Agent box, 2026-09-26 14:35–14:40 CEST, scripts/qemu-smoke.sh → qemu-smoke: ok (QEMU 10.0.13 Debian, virt -cpu cortex-a76 TCG, rustc 1.100.0-nightly 0fc141305). Hello boot:
ident: low lo=0x40000000 hi=0x40080000 pages=128 user=0
ident: tail lo=0x400f3000 hi=0x40201000 pages=270 user=270
ident: kend lo=0x4025d000 hi=0x4025e000 pages=1 user=0
ident: start-stay lo=0x40080000 hi=0x40081000
ident: low-fault va=0x40000000
ident: tail-fault va=0x40200000
ident: kend-fault va=0x4025d000
ident: inv-range k lo=0x0 hi=0x40000000 dev rw nx
ident: inv-range k lo=0x40080000 hi=0x40081000 mem ro x
ident: inv-range u lo=0x40080000 hi=0x40081000 mem ro x
ident: inv-range a lo=0x0 hi=0x40000000 dev rw nx
ident: inv-range a lo=0x40080000 hi=0x40081000 mem ro x
ident: inv k=2 u=1 a=2 leaks=0
ident: inv-neg k caught va=0x40000000
ident: inv-neg u caught va=0x40000000
ident: inv-neg clean
ident: inv-ok allow=mmio,stub
ident: ok
cargo test: identity_inventory_allowlist_only_catches_plant [ok], identity_leftovers_torn_stub_stays [ok]. inv-leak-probe kernel:
ident: inv-leak-probe planted va=0x40000000
ident: inv-leak k lo=0x40000000 hi=0x40001000 mem rw nx
ident: inv-leak a lo=0x40000000 hi=0x40001000 mem rw nx
ident: inv k=3 u=1 a=3 leaks=2
ident: inv missed
qemu-smoke: inv-leak-probe caught (fail-closed ok)
(ASID-B sees the plant because it shares the kernel RAM L2.)
Before → after (kernel TTBR0 identity ranges): 5 → 2 (I1 MMIO, I3 stub). User TTBR0: 2 → 1 (I3).
b2-serror profile under stock TCG (CTOS_B2_ACCEL=tcg): boots through ident: low/tail/kend to el0: serror-arm → el0: serror-park, the expected result without patch 0002. Not re-run on KVM: no metal was used in this phase.
Surprise: the pointer rewrite also rewrote the allowlist
The first run flagged the stub page as a leak. INV_ALLOW was promoted into .rodata, and the ADR-020/025 identity-pointer rewrite (ident: ro-reloc n=1418) treated the words 0x4008_0000 / 0x4008_1000 as identity text pointers and rewrote them to TTBR1 aliases. The check now compares identity_pa() of the loaded bounds. Lesson: any .rodata constant that looks like a kernel identity address is silently relocated. Security-relevant constants must be compared physically or encoded as immediates.
Honesty
Say: “RAM-side identity leftovers (low RAM, image padding tail, pre-heap frame) are unmapped from kernel and user TTBR0. A fail-closed walk proves only the MMIO block and the _start stub page remain identity-mapped, and a planted identity page is caught.” Do not say:
- identity mappings are fully torn down (MMIO + stub remain)
_startwas yanked / the kernel moved- “EL0 isolated” / “secure OS”
- the b2 profile was re-verified on KVM after this change
Consequences
- Allowed set is {I1, I3}. M2 (ADR-086 plan, D3 accepted) moves MMIO high and shrinks it to {I3}. M3 records I3 as the one documented exception (D2).
- Ledger, roadmap P-SEC-3l, threat model v1.62, el0.md, SUMMARY, second brain updated. Do not self-merge.
ADR-088 — Device MMIO through a TTBR1 high alias; identity MMIO block torn (M2)
- Status: Accepted (serial / tests / smoke). M2 of the ADR-086 plan. Sponsor D3 (DSO relay 2026-09-26 ~14:35 CEST): device MMIO is in scope for identity teardown. GIC / PL011 / virtio-mmio now use TTBR1 Device blocks. The identity MMIO 1 GiB L1 block (I1) is cleared. The identity inventory’s allowed set shrinks to {I3
_startstub page}. The umbrella “EL0 isolated” stays Planned / non-claim. - Date: 2026-09-26
- Base: stacked on PR #139 (ADR-087, M1).
Decision
- TTBR1 Device alias.
paging::initadds three 2 MiB Device-nGnRnE blocks (EL1 RW, PXN+UXN, no EL0 access) toL2_HIGHatTTBR1_BASE + pa:0x0800_0000(GICD + GICC),0x0900_0000(PL011),0x0a00_0000(virtio-mmio transports). That is 6 MiB instead of the 1 GiB identity hole. The block list is built from immediates, because the ADR-087 finding showed that.rodatatables of kernel-looking words get rewritten. - Driver switch.
paging::mmio_va(pa)returns the identity PA untilenable_mmio_high()and the high alias afterwards.uart,gicandvirtioregister accessors all go through it.enable_mmio_high()runs first thing inkernel_main_high(after the high jump, before any tear) and printsident: mmio-high gic=… uart=… virtio=…. The flag is a plain atomic load, so the UART is safe pre-MMU, in fatal paths, and whileTABLESis held. - Tear I1.
tear_identity_mmio()runs after the ADR-087 leftovers tear. It clears kernelL1[0]and the ASID-B copy; user TTBR0 never had it. One boot-timeTLBI VMALLE1, then it verifies the GIC / PL011 / virtio identity VAs are unmapped:ident: mmio lo=0x0 hi=0x40000000 l1=1.prepare_asid_b_tablesno longer requires an identity MMIO slot. - Fault probe. An EL1
LDRof the torn identityUARTFR(0x0900_0018) must take a translation DABORT before any device access:ident: mmio-fault va=0x9000018. - Allowlist shrink.
INV_ALLOW = {[0x4008_0000, 0x4008_1000)}. The smoke requiresident: inv k=1 u=1 a=1 leaks=0andident: inv-ok allow=stub, and rejects anyident: inv-range k|a lo=0x0. W^X keeps “MMIO is XN” viapaging::mmio_xn, which checks whichever alias is live. #[test_case]identity_mmio_torn_high_alias_xn.
Evidence (this session)
Agent box 2026-09-26 ~14:43–14:44 CEST scripts/qemu-smoke.sh → qemu-smoke: ok (QEMU 10.0.13, cortex-a76 TCG, rustc 0fc141305):
ident: jump
ident: mmio-high gic=0xffffff8008000000 uart=0xffffff8009000000 virtio=0xffffff800a000000
ident: low lo=0x40000000 hi=0x40080000 pages=128 user=0
ident: tail lo=0x400f3000 hi=0x40201000 pages=270 user=270
ident: kend lo=0x4025d000 hi=0x4025e000 pages=1 user=0
ident: mmio lo=0x0 hi=0x40000000 l1=1
Hello World!
ident: start-stay lo=0x40080000 hi=0x40081000
ident: mmio-fault va=0x9000018
ident: inv-range k lo=0x40080000 hi=0x40081000 mem ro x
ident: inv-range u lo=0x40080000 hi=0x40081000 mem ro x
ident: inv-range a lo=0x40080000 hi=0x40081000 mem ro x
ident: inv k=1 u=1 a=1 leaks=0
ident: inv-neg k caught va=0x40000000
ident: inv-neg u caught va=0x40000000
ident: inv-neg clean
ident: inv-ok allow=stub
ident: ok
All UART output after ident: mmio-high went through the TTBR1 alias, as did the virtio-blk / virtio-net / FAT / GIC timer and IRQ / FIQ probes. wx: ok and cargo test identity_mmio_torn_high_alias_xn [ok]. The inv-leak-probe kernel is still caught.
Identity ranges: kernel TTBR0 2 → 1, ASID-B 2 → 1, user 1 (unchanged). Only the _start stub page remains.
The b2-serror profile under stock TCG boots through ident: mmio-high / ident: mmio to el0: serror-arm → park, the expected result without patch 0002. It was not re-run on KVM. The ADR-085 B2-P evidence was taken at 159b178, before M1/M2 changed the boot path (tears + MMIO alias). A KVM re-run would cost about $0.2 (≈ 45 min of c7g.metal Spot). Not done in this phase.
Honesty
Say: “device MMIO is reached only through EL1-only TTBR1 Device blocks; the identity MMIO block is unmapped; the only identity mapping left in any TTBR0 is the _start stub page, and a fail-closed walk proves it.” Do not say:
- identity teardown is “complete”. I3 remains, as the documented exception (D2 / M3).
_startwas yanked / the kernel moved- “EL0 isolated” / “secure OS”
- the b2 profile was re-verified on KVM after M1/M2
Consequences
- ADR-055 §B row 2 (as reworded by M3 per D2) has its evidence: identity = {stub} only. M3 records the exception; the umbrella still needs M4 (D1 evidence), M5 (draft accept) + sponsor accept, and M6.
- Ledger, roadmap P-SEC-3l, threat model v1.63, el0.md, SUMMARY, second brain. Do not self-merge.
ADR-089 — _start stub page is the one documented identity exception (M3, sponsor D2)
- Status: Accepted (docs). Records sponsor decision D2 (DSO relay 2026-09-26 ~14:35 CEST): keep the
_startboot-stub page (I3) as the one documented identity exception, never yank_start, and reword ADR-047 §3 plus the checklist’s identity-teardown row to match. Nosrc/change. The umbrella “EL0 isolated” stays Planned / non-claim. - Follow-on (2026-09-28, M6): B3 met: the sponsor accepted the ADR-091 revision-3 sentence and scope in writing (2026-09-28 16:41 CEST). ADR-091 is Accepted. “EL0 isolated” is Verified only as that sentence, with its limits. Text in this ADR that says the umbrella is Planned / non-claim is as of this ADR’s date.
- Date: 2026-09-26
- Base: stacked on PR #140 (ADR-088, M2).
Context
ADR-055 §B row 2 read “Full identity teardown including yank _start”, while ADR-047 §3 decided “never yank _start”. ADR-086 flagged the row as unmeetable as worded and asked the sponsor (D2). After ADR-087 (M1) and ADR-088 (M2), a fail-closed walk shows that the stub page is the only identity mapping left in any TTBR0.
Decision
- The exception. Exactly one identity (VA = PA) mapping may remain: the page
[0x4008_0000, 0x4008_1000)that holds_startand the identity copy ofexception_vectors(VBAR is high). It must be EL1 read-only + executable, never EL0-accessible, and present in kernel, user and ASID-B TTBR0 as it is today.paging::INV_ALLOWencodes exactly this page. Any other identity leaf failsscripts/qemu-smoke.sh(ident: inv-leak). - Never yank or move
_start. QEMU-kernelenters at0x4008_0000; the image stays there. (ADR-086 D2 option (b), unmapping the stub’s identity mapping after boot, was not chosen.) - Reworded requirement (ADR-055 §B row 2, ADR-060 “Yank
_start” / umbrella rows): “Full identity teardown except the documented_startstub page (ADR-089): the fail-closed TTBR0 identity inventory reports only that page (ident: inv k=1 u=1 a=1 leaks=0+ident: inv-ok allow=stub) on the default smoke.” - ADR-047 §3 amended in place (history kept): “never yank
_start” stands; leftover identity RAM is no longer “optional”, since ADR-049 + ADR-087 tore it; the stub page is the one documented exception. - Row status after this ADR: reworded row 2 = Met on evidence from this session. That evidence is the ADR-088 box smoke 2026-09-26 ~14:43 CEST, with CI on #140 recorded in the ledger once green. That is not the umbrella: rows 1 (taken SError → M4 / D1) and 3 (sponsor accept → M5 draft, then the sponsor’s own accept) remain.
Future narrowing (not in scope, not decided)
The stub page is in user TTBR0 only because fill_user_map maps [KERNEL_TEXT, data) as EL1 text. VBAR is high, so EL0 exceptions do not fetch it through TTBR0. Removing it from user TTBR0 alone would leave kernel TTBR0 and _start untouched. That would need a sponsor call; this ADR does not act on it.
Honesty
Say: “the only identity mapping left is the documented _start stub page (EL1 RO+X), ratcheted fail-closed.” Do not say “identity fully torn down”, “_start yanked”, “kernel moved”, or “EL0 isolated”.
Consequences
- ADR-047 §3, ADR-055 §B, ADR-060 and ADR-042 honesty wording point here. Ledger, roadmap P-SEC-3l, threat model v1.64, el0.md, SUMMARY, MOC.
- Next: M4 (D1 evidence set), M5 (draft accept). Stop before M6.
ADR-090 — Taken-SError evidence class for the umbrella = B1 pin + B2-P (M4, sponsor D1)
- Status: Accepted (docs + CI workflow). Records sponsor decision D1 (DSO relay 2026-09-26 ~14:35 CEST). The taken lower-EL SError evidence that counts for the umbrella is the B1 pinned-QEMU CI job plus the B2-P run, stated verbatim below and copied into the draft accept ADR (ADR-091). Makes the
qemu-nmi-pinjob unconditional. Nosrc/logic change; one stale M2 doc comment insrc/teardown.rsis fixed (the allowlist has been stub-only since ADR-088). The umbrella “EL0 isolated” stays Planned / non-claim: the sponsor’s written accept (B3) is still absent. - Follow-on (2026-09-28, M6): B3 met: the sponsor accepted the ADR-091 revision-3 sentence and scope in writing (2026-09-28 16:41 CEST). ADR-091 is Accepted. “EL0 isolated” is Verified only as that sentence, with its limits. Text in this ADR that says the umbrella is Planned / non-claim is as of this ADR’s date.
- Date: 2026-09-26
- Base: stacked on PR #141 (ADR-089, M3).
Context
ADR-055 §B row 1 required “honest host inject that delivers async SError on the accepted smoke machine” (ADR-060). ADR-086 found two taken-SError sources, neither on that machine as worded. B1 (ADR-083) is a CI job on a locally patched QEMU, gated by repo variable CTOS_BUILD_QEMU_NMI=1, so it is opt-in. B2-P (ADR-085) is one paid real-hardware KVM run, not CI. ADR-086 asked D1: which evidence counts? The sponsor accepted the recommendation (i)+(ii): both together, stated verbatim, with no silent redefinition.
Decision
-
D1 evidence class (verbatim; ADR-091 must quote this paragraph unchanged):
Taken lower-EL SError while standing at EL0 is evidenced by two sources together. (i) B1: the GitHub Actions job
qemu-nmi-pin(“QEMU NMI pin smoke (taken SError)”) in.github/workflows/smoke.yml. It builds QEMU v10.0.0 (7c949c53e936aa3a658d84ab53bae5cadaa5d59c) with reviewed local patchresearch/qemu-nmi/0001-hw-arm-virt-TYPE_NMI-raise-SError.patch(TCG:virtTYPE_NMI→ async SError, masked byPSTATE.A), boots the default image onvirt -cpu cortex-a76TCG, issues QMPstop→inject-nmi→conton the guest’sel0: serror-armcue, and withCTOS_REQUIRE_TAKEN_SERROR=1fails closed unless the bareel0: serrorline (printed only by the lower-EL SError vector while the standing-EL0 expectation is armed) appears. It runs on every push and pull request. (ii) B2-P: ADR-085 run 4, 2026-09-26 13:26–13:33 CEST, AWS Graviton3c7g.metal(eu-north-1b, Spot), ctos commit159b178, QEMU-accel kvmwith patch 0002 (inject-nmi→KVM_SET_VCPU_EVENTS{serror_pending=1}→HCR_EL2.VSE), opt-inb2-serrorprofile:el0: serror+b2: taken, fail-closedb2-kvm-smoke.pyPASS. It is a one-off real-hardware run, not CI. Stock distro QEMU (the default smoke) cannot inject and continues to requireel0: serror-park. -
Make B1 unconditional. Remove
if: ${{ vars.CTOS_BUILD_QEMU_NMI == '1' }}fromqemu-nmi-pinand rename the job from “(opt-in)” to “QEMU NMI pin smoke (taken SError)”. Why: D5 makes the smoke a GitHub-required check before the flip. GitHub treats a job skipped byif:as passing a required check, so a variable-gated job would let a PR pass without taking an SError. This reverses ADR-083 §6 (“do not force multi-minute QEMU compile on every PR”) on purpose. Cost: ≈ 10–15 min on a free GitHub-hostedubuntu-24.04runner per push/PR. The repo is public, so standard runners are free. The repo variable becomes unused and can be deleted by the sponsor at any time; its value no longer matters. -
Reworded requirement (ADR-055 §B row 1, ADR-060 taken-SError row): “Taken lower-EL SError while standing, on the D1 evidence class (ADR-090): B1
qemu-nmi-pingreen, unconditional on every push/PR, plus the B2-P one-off KVM run (ADR-085 run 4).” -
Row status: reworded row 1 = Met (D1 evidence class). The B1 leg is green after the boot-path changes of this session: #139 (post-M1) pin jobs 108406107228 / 108406127586 and #140 (post-M2) pin job 108408283187 (
inject-nmi=>ok→ bareel0: serror, 2026-09-26 14:56–15:10 CEST). The first unconditional run is on this PR’s CI. This is not the umbrella: row 3 (sponsor accept) remains, see M5 → ADR-091. -
Not changed: the default smoke on stock QEMU still requires
el0: serror-park(ADR-053/081). ADR-053’s hard-stop stays true for stock virt TCG. DockerfileARG CTOS_BUILD_QEMU_NMI=0stays opt-in (local images only). No KVM in CI.
Caveats stated with the evidence (must travel with any quote of §1)
- B1 is a locally patched emulator. The SError is raised by QEMU TCG code this repo carries (patch 0001), not by stock upstream QEMU and not by hardware. It proves that the guest’s lower-EL SError path takes the exception; it does not prove any CPU’s RAS behaviour.
- B2-P is historical. It ran once at
159b178, before this session’s M1/M2 boot-path changes (identity leftover tears, MMIO high alias). Theb2-serrorprofile still TCG-boots through its markers after M2 (agent box, ~14:44 CEST), but it was not re-run under KVM. A re-verify is one more paidc7g.metalSpot run, ≈ $0.2 (ADR-085 cost basis). D1 does not require it; it is a sponsor option. - B2-P used a reduced profile (no GIC / timer / virtio / FAT), per ADR-085.
- Neither leg is real hardware running the full default image.
Honesty
Say: “per sponsor D1 (ADR-090), taken lower-EL SError is evidenced by the B1 pinned-QEMU CI job, which is unconditional and fail-closed on every push/PR, plus the one-off B2-P Graviton3 KVM run (ADR-085 run 4). Stock QEMU parks.” Do not say “taken SError on stock QEMU”, “taken SError Verified on real hardware in CI”, “SError hard-stop lifted” (it stands for stock TCG), or “EL0 isolated”.
Consequences
.github/workflows/smoke.yml:qemu-nmi-pinunconditional and renamed. docs/dev/qemu-nmi-pin-ci.md updated.- D5 / M6 required-check names (branch protection is not changed here; the sponsor/DSO sets it at M6):
QEMU aarch64 smoke (ubuntu-24.04-arm),QEMU aarch64 smoke (ubuntu-24.04),QEMU NMI pin smoke (taken SError). - ADR-055 §B row 1, ADR-060 and ADR-083 follow-ons; ledger, roadmap P-SEC-3r/3l, threat model v1.65, el0.md, SUMMARY, MOC.
- Next: M5 (ADR-091 draft accept). Stop before M6.
ADR-091 — Umbrella “EL0 isolated” accept (sentence + scope) — Accepted (M5 draft, sponsor D4; B3 met 2026-09-28; M6; revision 4 accepted 2026-10-02)
- Revision 4 (2026-10-02): accepted. The sponsor accepted ADR-091 revision 4 in writing (“I accept ADR-091 rev 4”, record below). Revision 4 changes exactly two things in the accepted text: the PAN clause of the sentence, and the limit “EL0 can read its own code bytes”, now “app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN)”. Every other clause, the scope table and the non-scope list are unchanged. The wording is ADR-097’s rev 4 proposal, copied character for character. Everything below that says the sentence is “unchanged from revision 3” describes the state before 2026-10-02 and is kept as history.
- Status (as of revision 3, kept for history): Accepted (2026-09-28, M6). The sponsor accepted the revision-3 sentence and scope in writing: B3 is met (record below). “EL0 isolated” is Verified only as the sentence below, read together with the non-scope list below. The sentence and the scope are unchanged from revision 3.
- Draft status (2026-09-26 to 2026-09-28, kept for history): Draft / pending sponsor accept. Not accepted. Not a claim. This file proposes the exact umbrella sentence and its scope for the sponsor to read (decision D4, DSO relay 2026-09-26 ~14:35 CEST). Checklist row B3 (written sponsor accept, ADR-055 §B row 3) is not Met and must not be counted until the sponsor has read this draft and accepted it in writing. Until then the umbrella stays Planned / non-claim. Nothing is flipped here: no ledger status change, no roadmap Verified. The flip is M6 and is out of scope for this PR.
- Date: 2026-09-26 (draft); revision 2: 2026-09-27 after sponsor option (b) (2026-09-27 00:09 CEST: “Close G1 and G2 first … then bring the sentence back to me”). ADR-092 closed G1 and G2 with CI-gated fail-closed probes and found G4 (below). Revision 3: 2026-09-27. The sponsor skipped the G4 question, so the DSO applied the standing order (close G4 before bringing the sentence back). ADR-094 closed G4 with CI-gated fail-closed probes. The “direct EL0 memory access only” carve-out is dropped. G3 is the only remaining limit, and it is optional. The sentence below is restated for the sponsor. Still Draft; B3 still not Met; nothing flipped.
- Limits update (2026-09-28, not a revision of the sentence): the sponsor asked for G3. ADR-095 re-ran the ADR-085 B2-P procedure on
main8107578on an AWS Graviton3c7g.metalunder KVM: PASS (el0: serror/b2: taken, fail-closed smoke, attempt 1). G3 is done. The revision-3 sentence and scope are unchanged and need no change. Still Draft; B3 still not Met; nothing flipped. - Correction (2026-09-29, not a revision of the sentence): the clause “PAN is enabled” held at boot but, from the ADR-080 enable and so also from the M6 accept until ADR-098, not while the kernel handled EL0 exceptions (
SCTLR_EL1.SPANwas 1) and not after EL0 trips (return_from_el0resumed with SPSR.PAN clear). The ADR-080 boot probe missed it because it ran before any EL0 trip. ADR-098 fixes both and adds fail-closed live checks (pan: live-syscall pstate-pan=1 sctlr-span=0,pan: after-el0 …,pan: ok held …). With the fix every clause is literally true again. The sentence and scope are unchanged. - Base: stacked on PR #142 (ADR-090, M4).
- Merge: sponsor only (ADR-002). Merging this draft does not constitute acceptance; acceptance is a separate written sponsor statement recorded by a follow-up edit (M6).
B3: the sponsor’s written accept (verbatim; met 2026-09-28)
- Quote (verbatim): “I accept the ADR-091 revision-3 sentence and scope as written. After M6, have ctos propose a short costed list of next milestones.”
- Date and time: 2026-09-28 16:41 CEST (Europe/Oslo).
- Relay path: sponsor wrote it in the DSO agent chat, DSO relayed it to ctos.
- Question answered: “Do you accept the ADR-091 revision-3 sentence and scope as written?” The DSO asked it after showing the sponsor the revision-3 sentence verbatim and the list of what it does not cover.
- B3 (ADR-055 §B row 3): Met. The accept covers the revision-3 sentence and the scope and non-scope below exactly as written. This edit does not change either.
- The second sentence of the quote is a request to ctos for after M6. It is not part of the claim.
B3 rev 4: the sponsor’s written accept (verbatim; 2026-10-02)
- Quote (verbatim): “I accept ADR-091 rev 4”
- Date: 2026-10-02 (Europe/Oslo). The relay did not give a time of day.
- Relay path: the sponsor wrote it in the DSO chat, DSO recorded it in the DSO vault, and DSO relayed it to ctos.
- Question answered: the exact wording proposed in ADR-097 (section “Revision-4 wording”, formerly “Proposed revision-4 wording for the sponsor”): the new PAN clause and the new code-bytes limit, everything else unchanged.
- What the accept covers: the two changes above, exactly as worded. It does not widen the claim, add a scope item or change the non-scope list.
- Implementing change: the XO app text change, PR #150 / ADR-097, merged as
8344797(2026-10-02, DSO merged it after the accept). The PAN-held-during-EL0-exceptions fix that the new PAN clause relies on is PR #151 / ADR-098, onmainas3c47c0c. - Rev 3’s accept (B3, above) stays on record for revision 3. This follow-up edit is docs only.
Accepted umbrella sentence (verbatim, revision 4; accepted 2026-10-02)
Revision 3 (accepted 2026-09-28) differed only in the PAN clause. Its text is kept in the History section.
“On the ctos default smoke machine (QEMU
virt,-cpu cortex-a76, TCG, one core), EL0 is isolated from the kernel at the architectural page-table / exception-level boundary: each EL0 task runs on its own TTBR0 with its own ASID; EL0 faults when it reads, writes or executes kernel memory; in every translation table (kernel, user and ASID-B TTBR0, and TTBR1) the only EL0-reachable pages are five allowlisted user slots backed by non-kernel frames, checked fail-closed while a user app runs and at rest; PAN is enabled, including while the kernel handles exceptions from EL0, and EL1 faults on EL0-readable memory (the execute-only app text page is not covered: this CPU has no FEAT_EPAN); lower-EL IRQ and FIQ are taken while EL0 stands; taken lower-EL SError is evidenced by the ADR-090 evidence class; the only identity (VA = PA) mapping left in any TTBR0 is the documented EL1-only_startstub page (ADR-089); and a system call copies from or to an EL0 pointer only if EL0 itself may read (for a copy from EL0) or write (for a copy to EL0) every page in the range. It is not a speculative-execution or side-channel claim, not a real-hardware claim, and not a certification.”
Scope: what the sentence covers, and the evidence for each clause
| # | Clause | Evidence (all CI-gated by scripts/qemu-smoke.sh unless noted) | ADR |
|---|---|---|---|
| S1 | Default smoke machine | .github/workflows/smoke.yml jobs QEMU aarch64 smoke (ubuntu-24.04-arm) / (ubuntu-24.04): distro QEMU virt -cpu cortex-a76 TCG, no -smp (one core) | ADR-079 |
| S2 | Own TTBR0 + ASID per EL0 task | el0: standing, el0: task-ok, asid: ok; EL0 entry without VMALLE1 (el0: no-vmalle1) | ADR-013 / 024 / 039 |
| S3 | EL0 faults on kernel read / write / execute | Read: el0: no kernel read (EL0 load of the torn identity alias of kernel .data → translation fault) and ttbr1: no el0 (EL0 load of the mapped TTBR1 private page → data abort). Write (G2, closed): el0: write-fault kernel va=0xffffff80… esr=0x9200004f far=<same> ec=0x24 wnr=1 dfsc=perm-l3 (store to kernel .data via TTBR1), el0: write-fault user-ro va=0x80009000 esr=0x9200004f far=0x80009000 ec=0x24 wnr=1 dfsc=perm-l3, el0: write-ok kernel,user-ro; the smoke rejects el0: write-succeeded / el0: write-bad, and the write-leak-probe build must report el0: write-succeeded. Execute: el0: nx kernel (EL0 fetch of kernel data → instruction abort) | ADR-013 / 016 / 092 |
| S8 | Every EL0-reachable page is an allowlisted user slot (G1, closed) | Fail-closed walk of kernel / user / ASID-B TTBR0 and TTBR1 (EL0-reachable = AP[1] set or UXN clear; rules: allowlisted VA, pool frame outside heap, not Device, not EL0 W+X, nothing in TTBR1): el0-reach: live k=3 u=3 a=0 h=0 pages=6 leaks=0 (hello-libctos standing at EL0), el0-reach: steady k=0 u=0 a=0 h=0 pages=0 leaks=0, el0-reach: neg {kernel-pa,va,ttbr1,el0-wx} caught …, el0-reach: neg clean, el0-reach: ok allow=app-hdr[0x80000000,0x80001000),app-text[0x80002000,0x80003000),crt-stack-pan[0x80003000,0x80004000),user-stack[0x80007000,0x80008000),store-ro[0x80009000,0x8000a000); the reach-leak-probe build must report el0-reach: leak … why=kernel-pa | ADR-092 |
| S4 | PAN enabled, EL1-vs-EL0 fault | pan: present, pan: enabled, pan: el1-fault. Added for revision 4: PAN held while the kernel handles EL0 exceptions and after EL0 trips (pan: live-syscall pstate-pan=1 sctlr-span=0, pan: after-el0 …, pan: ok held …); EL1 faults on EL0-readable memory (xo: el1-pan control app-hdr … fault=yes spsr-pan=1); the execute-only text page is not covered, pinned as a fact (xo: el1-pan text … fault=no … mmfr1-pan=2 epan=no) | ADR-079 / 080 / 097 / 098 |
| S5 | Lower-EL IRQ / FIQ while standing | el0: irq, el0: irq-default, el0: fiq | ADR-040 / 041 / 043 |
| S6 | Taken lower-EL SError | D1 evidence class, quoted verbatim from ADR-090 §1 (below). The default smoke itself parks (el0: serror-park). | ADR-083 / 085 / 090 |
| S9 | Syscall pointers need EL0 permission on every page (G4, closed) | el0: sys-ptr control rw-data n=6; el0: sys-ptr denied kernel-stub (SYS_UART_WRITE of the _start stub page); el0: sys-ptr denied kernel-window (kernel-only map-window page); el0: sys-ptr denied xo-text and el0: sys-ptr denied write-ro (SYS_NET_MAC copy_to_user into EL0’s own execute-only text / an EL0-RO page, target unchanged); el0: sys-ptr denied straddle; seven lines el0: sys-ptr denied sys=<name> nr=<n> … (every pointer-taking syscall: 17, 19, 20, 21, 22, 24, 27); el0: sys-ptr sweep syscalls=7 denied=7; el0: sys-ptr ok. The smoke rejects el0: sys-ptr leaked / bad / skip / probe missed. The sysptr-leak-probe build must report el0: sys-ptr leaked kernel-stub n=8 etc. and withhold ok | ADR-094 |
| S7 | Only identity mapping = _start stub page | Fail-closed TTBR0 inventory: ident: inv k=1 u=1 a=1 leaks=0, ident: inv-ok allow=stub, negative probes ident: inv-neg … caught, inv-leak-probe build caught; leftover/MMIO fault probes ident: {low,tail,kend,mmio}-fault. The stub page is EL1 RO+X with UXN set, so EL0 cannot fetch it | ADR-087 / 088 / 089 |
S6: D1 evidence paragraph (verbatim from ADR-090 §1)
Taken lower-EL SError while standing at EL0 is evidenced by two sources together. (i) B1: the GitHub Actions job
qemu-nmi-pin(“QEMU NMI pin smoke (taken SError)”) in.github/workflows/smoke.yml. It builds QEMU v10.0.0 (7c949c53e936aa3a658d84ab53bae5cadaa5d59c) with reviewed local patchresearch/qemu-nmi/0001-hw-arm-virt-TYPE_NMI-raise-SError.patch(TCG:virtTYPE_NMI→ async SError, masked byPSTATE.A), boots the default image onvirt -cpu cortex-a76TCG, issues QMPstop→inject-nmi→conton the guest’sel0: serror-armcue, and withCTOS_REQUIRE_TAKEN_SERROR=1fails closed unless the bareel0: serrorline (printed only by the lower-EL SError vector while the standing-EL0 expectation is armed) appears. It runs on every push and pull request. (ii) B2-P: ADR-085 run 4, 2026-09-26 13:26–13:33 CEST, AWS Graviton3c7g.metal(eu-north-1b, Spot), ctos commit159b178, QEMU-accel kvmwith patch 0002 (inject-nmi→KVM_SET_VCPU_EVENTS{serror_pending=1}→HCR_EL2.VSE), opt-inb2-serrorprofile:el0: serror+b2: taken, fail-closedb2-kvm-smoke.pyPASS. It is a one-off real-hardware run, not CI. Stock distro QEMU (the default smoke) cannot inject and continues to requireel0: serror-park.
The ADR-090 caveats travel with it: B1 is a locally patched emulator; B2-P is historical (at 159b178, before M1/M2) on a reduced profile; a KVM re-verify ≈ $0.2 is a sponsor option. Update 2026-09-28: the KVM re-verify was done (ADR-095): the B2-P leg passed again on main 8107578 (post-M1/M2, ADR-092, ADR-094), still on the reduced b2-serror profile and still a one-off, not CI.
Explicitly not in scope (the sentence must never be read as covering these)
- Speculative execution / side channels. No KPTI: the TTBR1 kernel mapping stays present (EL1-only) while EL0 runs. No Spectre/Meltdown/cache/timing analysis; QEMU TCG is the wrong lab (threat model: “not side-channel complete”).
- Real hardware. Only the emulated default machine, plus the one-off B2-P KVM run on the reduced
b2-serrorprofile. No board, no Graviton CI. (The B2-P run was repeated once on currentmainfor G3, ADR-095; still reduced profile, still not CI.) - Multi-core. One core only. No SMP TLB-shootdown or cross-core isolation story.
- EL0-to-EL0 (task-vs-task) isolation beyond the
asid: okTLB mile and per-task TTBR0. No probe that one EL0 task cannot read another task’s pages. - Hypervisor / EL2 / secure world, DMA / IOMMU, physical attacks. The virtio device can DMA anywhere; there is no SMMU.
- Certification, formal verification, or “secure OS”. Not a guest-Linux, container or immutable-OS claim (ADR-029 non-goals stand).
- Stock-QEMU taken SError. Stock TCG still parks. The ADR-053 hard-stop for stock virt TCG stands.
- Check-to-copy races on SMP. S9 is a software check on one core. Nothing changes the calling task’s user page tables during a syscall. No
AT S1E0R/W, noLDTR/STTRcopy, no fault-fixup copy (ADR-094).
Gaps (revision 3, 2026-09-27)
- G1: exhaustive EL0-permission walk. CLOSED by ADR-092 (S8 markers above). Correction to revision 1: its code-reading note missed
l3_page_el0_exec(EL0 execute-only,UXNclear). The walk covers bothAP[1]andUXN, which supersedes that note. - G2: EL0 write probe. CLOSED by ADR-092 (S3 write markers above).
- G3: B2-P re-verify on the post-M2 boot path. DONE (2026-09-28) by ADR-095: AWS Graviton3
c7g.metalSpot, QEMU v10.0.0 + 0001 + 0002-accel kvm, ctosmain8107578:el0: serror+b2: taken,b2-kvm-smoke: PASS, attempt 1, ≈ $0.04. Still the reducedb2-serrorprofile and a one-off, not CI. Revision-3 text, kept for history: “(≈ $0.2 paid run). Open, optional, not required by D1. This is the only remaining limit.” No known limit remains open; B3 (the sponsor’s written accept) is the only step left before M6. - G4: syscall copy helpers skip the EL0 permission check. CLOSED by ADR-094 (S9 markers above):
user_range_ok_maxnow requires EL0-readable (reads) / EL0-writable (writes) on every page, and the loader maps app text+rodata EL0 read-only + executable. The G1 allowlist is unchanged (five slots). Revision-2 note, kept for history: found by ADR-092; documented by code reading; not probed, not fixed at that time.syscall::user_range_ok_maxaccepts any page with a valid leaf in user TTBR0, andcopy_userthen clears PAN. So EL0 could askSYS_UART_WRITEto copy bytes of an EL1-only page mapped in user TTBR0: the_startstub page, or an EL1-only map-window page present while the app runs.copy_to_usercould also write the app’s own EL0 execute-only text. Fix: require EL0 read/write permission on the leaf (orAT S1E0R/W). The loader must then map app rodata EL0-readable (today it shares the execute-only text page). Revision-4 note (2026-10-02): ADR-097 later moved rodata to its own EL0-readable page and made app text EL0 execute-only again (still five slots).
The sponsor can:
- (a) accept the revision-3 sentence as it stands, with G3 (optional B2-P re-verify) as the only known limit;
- (b) require G3 (the ≈ $0.2 KVM re-verify) first; (done 2026-09-28, ADR-095: PASS; so (a) and (b) now lead to the same sentence)
- (c) narrow the sentence further.
Revision-2 options were (a) accept with G3/G4 as limits, (b) fix G4 first, (c) narrow. The sponsor skipped that question. The DSO applied the standing order, which is (b), done in ADR-094.
History
Revision 1 (2026-09-26) proposed: “… EL0 faults when it reads or executes kernel memory; PAN is enabled … (ADR-089). This is not a speculative-execution or side-channel claim, not a real-hardware claim, and not a certification.” Revision 2 adds “writes”, the all-tables EL0-reachability clause (S8) and the direct-access-only limit (G4). Revision 2’s sentence ended its clause list with “… _start stub page (ADR-089). This covers direct EL0 memory access only, not data the kernel copies on EL0’s behalf in system calls. It is not …”. Revision 3 replaces that carve-out with the S9 syscall-pointer clause (G4 closed by ADR-094).
Revision 4 (2026-10-02, accepted) changes only the PAN clause and one limit; the sponsor accepted it in writing (“I accept ADR-091 rev 4”, record above). Revision-3 text, kept for history:
- PAN clause, before (revision 3): “PAN is enabled and EL1 faults on EL0 memory”
- PAN clause, after (revision 4): “PAN is enabled, including while the kernel handles exceptions from EL0, and EL1 faults on EL0-readable memory (the execute-only app text page is not covered: this CPU has no FEAT_EPAN)”
- Limit, before (revision 3): “EL0 can read its own code bytes” (app text and rodata were one EL0 read-only + executable page, ADR-094)
- Limit, after (revision 4): “app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN)”
Why: ADR-097 (PR #150) makes app text EL0 execute-only. This CPU has no FEAT_EPAN, so with PAN set an EL1 load of that page does not fault, which made the literal revision-3 clause false for it. ADR-098 (PR #151, 3c47c0c) separately fixed PAN being clear while the kernel handled EL0 exceptions and after EL0 trips, which is why the new clause can say “including while the kernel handles exceptions from EL0”. All other clauses of the sentence are identical to revision 3.
M6 prerequisites (all met 2026-09-28; the flip is the M6 PR, branch isolation/m6-el0-isolated-flip)
- B3: the sponsor’s written accept of the revision-3 sentence and scope (option a/b/c above), recorded verbatim with date and relay path. Merging this PR is not acceptance. Met: 2026-09-28 16:41 CEST, recorded above.
- D5: before the flip, the sponsor/DSO sets branch protection on
mainto requireQEMU aarch64 smoke (ubuntu-24.04-arm),QEMU aarch64 smoke (ubuntu-24.04)andQEMU NMI pin smoke (taken SError). The agent does not change branch protection. Met: the three checks are required onmain(set by the sponsor/DSO). - The stack merged in order: #136–#138 are already merged (2026-09-26); #139 → #140 → #141 → #142 → #143 (this draft) → #144 (ADR-092) → ADR-094’s PR remain, with all three checks green on the merge tip. Met: #139–#147 are merged;
mainat68d4c8f(merge of #147, ADR-095) is green on all three checks. - The flip PR changes ADR-055 §B row 3, ADR-060, the ledger P-SEC-3l row, roadmap, el0.md, SUMMARY and the threat model only to the accepted wording, and cites this ADR (status → Accepted). Done in the M6 PR.
Honesty (Accepted, M6; revision 4 since 2026-10-02)
Say “EL0 isolated” only as the sentence above (revision 4): quote it verbatim or point to it, and never widen it. Keep its limits next to it: one core; the TCG default smoke machine; not a speculative-execution or side-channel claim; not a real-hardware claim; not a certification; app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN) (ADR-097); no multi-core / SMP race story. Do not read it as covering anything in the non-scope list above.
Honesty text while Draft (kept for history): Say: “ADR-091 is a draft of the umbrella sentence and scope, pending the sponsor’s written accept; ‘EL0 isolated’ stays Planned / non-claim.” Do not quote the proposed sentence as a statement of fact, count B3 as Met, or flip any status.
ADR-092 — G1 EL0-reachability walk + G2 EL0 write permission fault (sponsor option (b) on ADR-091)
- Status: Accepted (implementation + evidence). Closes ADR-091 gaps G1 and G2 with fail-closed probes that CI gates. Found a new gap, G4 (syscall copy helpers; below), which is recorded, not fixed. The umbrella “EL0 isolated” stays Planned / non-claim. ADR-091 stays Draft; B3 (the sponsor’s written accept) is not Met.
- Follow-on (2026-09-28, M6): B3 met: the sponsor accepted the ADR-091 revision-3 sentence and scope in writing (2026-09-28 16:41 CEST). ADR-091 is Accepted. “EL0 isolated” is Verified only as that sentence, with its limits. Text in this ADR that says the umbrella is Planned / non-claim is as of this ADR’s date.
- Date: 2026-09-27
- Sponsor decision: 2026-09-27 00:09 CEST, option (b) on ADR-091: “Close G1 and G2 first (full EL0-reachable walk plus an EL0 write test), then bring the sentence back to me.”
- Base: stacked on PR #143 (ADR-091 draft).
_start(0x4008_0000) untouched: no probe targets it, and the stub page stays the ADR-089 exception.
Context
ADR-091 rested its clause “EL0 faults when it reads or executes kernel memory” on two single-address probes plus a code-reading note. It listed G1 (no exhaustive EL0-permission walk) and G2 (no EL0 write probe). The sponsor asked for both before re-reading the sentence.
Decision
G1 — fail-closed EL0-reachability walk (paging::el0_reach, src/reach.rs)
-
What is walked: every valid leaf of four translation roots:
k: kernel TTBR0u: user TTBR0a: ASID-B TTBR0h: TTBR1, added beyond the request because EL0 can walk TTBR1 whileTCR_EL1.EPD1 = 0
-
EL0-reachable =
AP[1]set (EL0 load/store) orUXNclear (EL0 fetch). Table-levelAPTable/UXNTablebits are ignored. That can only over-count, and this kernel never sets them. -
Rules per coalesced run. Any failure is a leak, printed
el0-reach: leak … why=<rule>:ttbr1: no EL0-reachable leaf in TTBR1 at all.device: never Device memory.kernel-pa: backed only by frame-pool frames,[pool_start, pool_end). Never the image, linker stacks or static page tables belowpool_start, and never outside RAM.heap-pa: never the kernel heap run.el0-wx: never EL0-writable and EL0-executable at once.va: inside oneEL0_ALLOWrange.
-
EL0_ALLOW: the five 4 KiB map-window slots this kernel hands to EL0, taken from everymap_el0_*call site:Name VA Used by app-hdr[0x8000_0000, 0x8000_1000)loader: rust-lld ELF-header PT_LOAD(EL0-RO)app-text[0x8000_2000, 0x8000_3000)EL0_PAGE: loaded app text + rodata (EL0 execute-only), EL0 probe trampolinescrt-stack-pan[0x8000_3000, 0x8000_4000)libctos CRT stack; PAN probe page user-stack[0x8000_7000, 0x8000_8000)loader user stack (EL0-RW, NX) store-ro[0x8000_9000, 0x8000_a000)G2 read-only store target The rest of the 2 MiB window is not allowed. A larger guest image needs a new ADR, not a silent widening.
-
Three views:
- Live: armed in the loader probe and taken on hello-libctos’s first
SYS_YIELD, while the app stands at EL0 with every user page mapped. - Steady: after the boot probes. Nothing may be EL0-reachable.
- Negative: four plants (never accessed; removed, then
TLBI VAAE1). Each must be caught in the expected roots under the expected rule, and a clean walk must follow:kernel-pa: EL0-RO mapping of a kernel.datapage at the allowlisted0x8000_7000va: pool frame at off-allowlist0x8019_0000ttbr1: pool frame atTTBR1_BASE + 1 MiBel0-wx: EL0 RW+X pool frame at0x8000_2000
- Live: armed in the loader probe and taken on hello-libctos’s first
-
Leak build (
--features reach-leak-probe): a persistent EL0-RO mapping of kernel.dataat0x8000_7000before the steady walk. It must printel0-reach: leak k|u … why=kernel-paand withholdel0-reach: ok.
G2 — EL0 write permission fault (el0::el0_write_faults)
-
EL0 runs
MOVZ X1, #magic; STR X1, [X0]; BRK #0withX0 = target. The store must take a lower-EL data abort with:EC = 0x24WnR = 1DFSC= permission fault, level 1, 2 or 3FAR = targetELR= theSTR- the target unchanged afterwards
Reaching the
BRKmeans the store completed, and the probe printsel0: write-succeeded. Any other exception printsel0: write-bad. Either one withholdsel0: write-ok, and the smoke rejects both. -
Targets:
- (a) kernel
.datathrough its TTBR1 alias: mapped EL1-RW / EL0-none, and TTBR1 walks stay live while EL0 runs. - (b) an EL0-read-only user page at
0x8000_9000, filled through its TTBR1 RAM alias because PAN blocks EL1 access through the EL0 mapping. _startis deliberately not a target.
- (a) kernel
-
Negative build (
--features write-leak-probe): target (b) is mapped EL0-RW, so the store completes. The probe must printel0: write-succeeded user-roand withholdel0: write-ok. The smoke builds both leak features in one kernel (target/el0-leak-probe).
Evidence (agent box, 2026-09-27 00:20–00:23 CEST; QEMU 10.0.13 Debian, virt -cpu cortex-a76 TCG, rustc 1.100.0-nightly 0fc141305): qemu-smoke: ok
el0: write-fault kernel va=0xffffff8040201008 esr=0x9200004f far=0xffffff8040201008 ec=0x24 wnr=1 dfsc=perm-l3
el0: write-fault user-ro va=0x80009000 esr=0x9200004f far=0x80009000 ec=0x24 wnr=1 dfsc=perm-l3
el0: write-ok kernel,user-ro
el0-reach: range u lo=0x80000000 hi=0x80001000 pa=0x4025d000 mem el0-ro el0-nx
el0-reach: range u lo=0x80002000 hi=0x80003000 pa=0x4026e000 mem el0-none el0-x
el0-reach: range u lo=0x80007000 hi=0x80008000 pa=0x4026f000 mem el0-rw el0-nx
el0-reach: live k=3 u=3 a=0 h=0 pages=6 leaks=0
el0-reach: steady k=0 u=0 a=0 h=0 pages=0 leaks=0
el0-reach: neg kernel-pa caught va=0x80007000 roots=k,u why=kernel-pa
el0-reach: neg va caught va=0x80190000 roots=k,u why=va
el0-reach: neg ttbr1 caught va=0xffffff8000100000 roots=h why=ttbr1
el0-reach: neg el0-wx caught va=0x80002000 roots=k,u why=el0-wx
el0-reach: neg clean
el0-reach: ok allow=app-hdr[0x80000000,0x80001000),app-text[0x80002000,0x80003000),crt-stack-pan[0x80003000,0x80004000),user-stack[0x80007000,0x80008000),store-ro[0x80009000,0x8000a000)
(The live walk also prints the same three range k … lines; k and u share the map-window L3.) Leak build: el0: write-succeeded user-ro va=0x80009000 esr=0xf2000000 (the BRK), el0-reach: leak-probe planted va=0x80007000, el0-reach: leak k|u lo=0x80007000 hi=0x80008000 pa=0x40204000 mem el0-ro el0-nx why=kernel-pa, el0-reach: steady k=1 u=1 a=0 h=0 pages=2 leaks=2 → qemu-smoke: el0-leak-probe caught (fail-closed ok). cargo test: new cases el0_store_to_kernel_and_ro_page_faults and el0_reach_rest_clean_and_plants_caught pass (146 [ok]). CI: see the ledger row for this PR.
Surprises / findings
- Nothing is EL0-reachable at rest. Every probe and loader run unmaps its user pages, so a steady-state walk alone would never exercise the allowlist. Hence the live walk from inside a standing app.
- Only one existing read probe was a permission fault.
el0: no kernel readtargets the torn identity alias of kernel.data, so it is a translation fault.ttbr1: no el0(an EL0 load of the TTBR1 private page) was already a permission-class check on a mapped kernel page. G2(a) is the first store on a mapped kernel page. - ADR-091’s code-reading note was incomplete. It said only
l3_page_el0_rw/l3_page_el0_roset EL0 access and missedl3_page_el0_exec(AP = 00, UXN clear: EL0 execute-only). The walk covers bothAP[1]andUXN, so the note is superseded. - App rodata sits on an EL0 execute-only page. The loader unions hello’s R segment into the RX page, so EL0 cannot load its own rodata directly. The apps still work because the kernel reads their strings in syscalls (G4).
a = 0andh = 0: ASID-B’s window pages are EL1-only nG probe pages. The kernel TTBR0 carries the same EL0 pages as user TTBR0 (shared window L3), and EL0 only ever runs on user TTBR0.
New gap G4 (Documented by code reading; not probed, not fixed)
syscall::user_range_ok_max (used by copy_user / copy_to_user / path_from_user) accepts a user pointer if paging::user_mapped(page) && paging::is_mapped(page). user_mapped accepts any valid leaf in user TTBR0, whatever its AP bits. The copy then clears PSTATE.PAN. Consequences:
SYS_UART_WRITE(0x4008_0000, n)from EL0 would copy bytes of the EL1-only_startstub page to the UART.- Any EL1-only map-window page that exists while an app runs would be readable the same way, because the window L3 is shared by kernel and user TTBR0.
copy_to_usercould write an EL0 execute-only page (the app’s own text).
The direct EL0 access path is closed (G1/G2); the kernel-mediated path is not. The fix is to require EL0 read/write permission on the user leaf (or use AT S1E0R/W). It needs the loader to map app rodata EL0-readable (AP = 11 with UXN clear, i.e. EL0 R+X), which changes loader and trampoline permissions, so it is out of this milestone’s scope. ADR-091 lists G4 for the sponsor.
Honesty
Say: “ADR-092: a fail-closed walk of kernel, user and ASID-B TTBR0 and TTBR1 finds EL0-reachable pages only in five allowlisted user slots backed by non-kernel frames (live: 3 per shared window root while an app stands; at rest: none), and four planted leaks are caught; EL0 stores to kernel data and to an EL0-RO page take permission faults (ESR/FAR checked). Syscall copy helpers do not yet check EL0 permission (G4).” Do not say “EL0 isolated”, “EL0 cannot obtain kernel bytes by any path”, or “exhaustively verified” beyond these four roots on this machine.
Consequences
- Code:
src/paging.rs:el0_reach,EL0_ALLOW, plant helpers,EL0_STORE_RO_VA.src/reach.rs(new).src/exception.rs: store-probe capture.src/el0.rs: G2.src/syscall.rs/src/loader.rs: live-walk hook.src/main.rs.Cargo.toml:reach-leak-probe,write-leak-probe.scripts/qemu-smoke.sh: required markers, rejects, el0-leak build.
- Docs: ADR-091 revision 2 (G1/G2 closed, G4 added, sentence restated), ADR-055 §A rows, ledger, roadmap, threat model v1.67, el0.md, SUMMARY, MOC, daily brief, session memory.
ADR-093 — Public-identifier scrub + fail-closed guard
- Status: Accepted (security / compliance guardrail). Not a kernel change. No
src/change. - Date: 2026-09-27
- Trigger: sponsor-approved task relayed by DSO (2026-09-27): remove the main AWS account ID and other identifying data from the current tree of this public repo; add a cheap fail-closed guard if it fits.
- Base: branched from
main(c1d98c7), not stacked on the open EL0 stack (#139–#144). - Numbering: ADR-087–092 are taken by the open EL0 stack; this ADR takes 093 to avoid a collision on rebase.
Context
This repository is public. A read-only scan found a cloud account identifier and related cloud resource identifiers written into docs, research notes and one Route 53 recovery batch file. Most came from the 2026-09-11 docs/DNS session and the 2026-09-26 B2-P paid spike (ADR-085). A second scan found a local OS user name inside host paths and a WSL command line. No workflow, script logic or kernel code uses any of these values.
Decision
-
Replace, don’t delete. Each value becomes a typed placeholder so the prose keeps its meaning:
- prose / tables:
<aws-account-id>,<route53-zone-id>,<instance-id-run0>…<instance-id-run4>,<spot-request-id-run0>…<spot-request-id-run4>,<security-group-id>,<iam-user>,<user>(local OS user in paths). Run indices keep the ADR-085 tables cross-referenced. - copy-paste shell snippets:
"${AWS_ACCOUNT_ID:?set AWS_ACCOUNT_ID locally}"and"${ROUTE53_ZONE_ID:?set ROUTE53_ZONE_ID locally}". The real values stay in the operator’s local config. Nothing in CI needs them, so no new secret or variable is needed.
- prose / tables:
-
Counts (types only, no values). Current tree on
main:Identifier type Files Occurrences Where AWS account ID 11 14 4 docs (incl. README), 6 research notes, 1 script data file (Route 53 JSON comment); 0 workflows Route 53 hosted-zone ID 12 17 5 docs, 6 research notes, 1 script data file EC2 instance IDs (5 distinct) 4 27 2 docs, 2 research notes Spot request IDs (5 distinct) 1 5 1 research console log Security-group ID 1 2 1 research note IAM user name 1 1 1 ADR Local OS user in paths / WSL -u4 6 3 docs, 1 research note 17 content files changed in total (several carry more than one type).
-
Fail-closed guard.
scripts/check-public-ids.shscans every git-tracked text file for identifier shapes: 12-digit account IDs next to an account/owner word, in an ARN or in an ECR host; AWS access key IDs; EC2/VPC resource IDs; Spot request IDs; Route 53 hosted-zone IDs (after a zone word, or bareZ+ digit form). It holds generic patterns only, never a real value. Its allowlist holds public AWS-documentation example values only. On a hit it printsfile:line classand not the matched text, because CI logs are public.--self-testplants synthetic tokens of every class in a throwaway repo (built at runtime) and fails if any class is missed or if an allowlisted example or a context-free number is flagged. -
Wired into
qemu-smoke.shas its first step (self-test, then scan). That script already runs in both defaultsmoke.ymljobs and in Docker smoke, so the guard is CI-gated without touching.github/workflows/. Marker lines:public-ids: self-test ok classes=7,public-ids: ok files=N hits=0. -
History is not rewritten. The replaced values remain in earlier commits and in already-merged PR diffs. Rewriting history or force-pushing needs a separate sponsor decision after a risk evaluation. This ADR does not decide that.
Evidence
- Guard on this branch:
public-ids: self-test ok classes=7,public-ids: ok files=… hits=0(bash and dash). - Guard on unmodified
main(script copied into a scratch worktree): FAIL hits=77 (raw matches). Distinctfile:line classpairs: account-id-ctx 12, route53-zone-id 16, ec2-resource-id 26, spot-request-id 5. Every line that carried one of the values above was flagged, with no extra lines. - CI: see the honesty ledger row for this PR.
Limits (non-claims)
- A bare 12-digit number with no account/ARN/ECR context is not flagged. GitHub Actions job IDs have the same shape and appear throughout the ledger. The guard is a ratchet for the common shapes, not a DLP product.
- It does not scan git history, PR descriptions, issue comments or CI logs.
- Public, intended identifiers stay: the GitHub owner name in repo URLs, the public docs domain, git author metadata, public AWS region / instance-type names, GitHub run/job IDs, QEMU user-net addresses (10.0.2.x) and loopback.
- Not a “secure” or “compliant” claim.
Consequences
- The open EL0 stack PRs (#139–#144) must pass this guard after they rebase on a
mainthat contains it. - New cloud work must write placeholders or env vars, never literal account or resource IDs.
ADR-094 — G4: syscall user pointers require EL0 permission on every page
- Status: Accepted (implementation + evidence). Closes ADR-091 gap G4 with fail-closed probes that CI gates. The umbrella “EL0 isolated” stays Planned / non-claim. ADR-091 stays Draft; B3 (the sponsor’s written accept) is not Met.
- Follow-on (2026-09-28, M6): B3 met: the sponsor accepted the ADR-091 revision-3 sentence and scope in writing (2026-09-28 16:41 CEST). ADR-091 is Accepted. “EL0 isolated” is Verified only as that sentence, with its limits. Text in this ADR that says the umbrella is Planned / non-claim is as of this ADR’s date.
- Date: 2026-09-27
- Decision path: the sponsor did not pick among ADR-091 revision-2 options (a)/(b)/(c). The DSO applied the standing order “close gaps before bringing the sentence back” (relay 2026-09-27 ~12:06 CEST), i.e. option (b): fix G4 first.
- Base: stacked on PR #144 (ADR-092). ADR-093 is the public-identifier guard (PR #145).
_start(0x4008_0000) untouched. The stub page is only the target of a refused read. No code maps, writes or changes it.
Context
ADR-092 found G4 by code reading. syscall::user_range_ok_max accepted any page with a valid leaf in the user TTBR0. copy_user then cleared PAN and loaded the bytes. EL0 could therefore ask the kernel to do what EL0 itself cannot:
- Read:
SYS_UART_WRITEcould copy the EL1-only_startstub page, or an EL1-only map-window page present while the app runs, to the UART. - Write:
copy_to_user(SYS_FS_READ,SYS_NET_MAC) could write EL0’s own execute-only text.
That is a confused-deputy path around the page-table boundary. So ADR-091 revision 2 limited its sentence to direct EL0 access.
Decision
1. The check (src/syscall.rs)
user_range_ok_max(ptr, len, max, access)gains an access kind:UAccess::Read: the kernel copies from EL0 (copy_user_max:uart_write,fs_write, and paths forfs_create/fs_open/fs_mkdir).UAccess::Write: the kernel copies to EL0 (copy_to_user, plus the pre-checks inSYS_FS_READandSYS_NET_MAC).
- The existing guards stay: length cap, overflow, identity/canonical, and “mapped in user TTBR0 and kernel TTBR0”.
- New, for every 4 KiB page in the range:
paging::user_el0_access(page)reads the user-root leaf’s AP bits.- A read needs EL0-readable:
AP[1]=1. - A write needs EL0-writable:
AP[1]=1andAP[2]=0. - Execute-only (
AP[2:1]=00, UXN clear), kernel-only and unmapped pages fail.
- A read needs EL0-readable:
- On failure the syscall returns its existing error value, and no byte moves:
uart_write/fs_write/net_mac: 0fs_create/fs_open: fd 0fs_mkdir:FS_MKDIR_BAD_PATHfs_read:FS_ERR
- The map-window L3 is shared by the kernel and user roots. So the leaf the check reads is the leaf the copy uses under kernel TTBR0.
- A counter,
SYSPTR_DENIED, counts refusals made by the permission check itself. The probes require a delta of exactly 1 per call. So a refusal cannot come from some other guard (length, unmapped) by accident.
2. Loader: app rodata EL0-readable (src/loader.rs, src/libctos.rs, src/paging.rs)
- Apps link
.textand.rodatainto one R+XPT_LOADpage.Perm::Execused to map it EL0 execute-only (l3_page_el0_exec, AP=00). The app’s own strings therefore failed the new read rule. Perm::Execnow maps with the newpaging::map_el0_text/l3_page_el0_text:- AP[2:1]=11: EL0 read-only, EL1 read-only
- UXN clear, PXN set
- so EL0 can read and execute the page but not write it.
- Bytes are written through the frame’s TTBR1 alias, as before. Cache maintenance by the EL0 VA runs inside
pan::with_user_access, since the page is EL0-accessible now. - The A2
libctos::run_hellopath does the same. - No new slot, no split page, no app ELF change. The app image and its sha256 are byte-identical, so the A9 cross-update to the prior OS still boots the same ELF.
- The G1 walk still passes. The page was already EL0-reachable (UXN clear), and it is not EL0 W+X.
- Kernel-built probe payloads keep execute-only code pages. Their strings and buffers move to a new EL0-RW NX data page,
syscall::EL0_DATA_VA(window slot 3, the allowlistedcrt-stack-panslot).
3. Probes (syscall::observe_sys_ptr_probe, called from kernel_main_high after svc: ok)
| Line | What EL0 asks | Must be |
|---|---|---|
el0: sys-ptr control rw-data n=6 | SYS_NET_MAC into the EL0-RW data page | 6 bytes land, equal to the guest MAC (positive control: the NIC works, so a later 0 is the check) |
el0: sys-ptr denied kernel-stub | SYS_UART_WRITE(0x40080000, 8): the _start stub page | returns 0, nothing printed, 1 check refusal |
el0: sys-ptr denied kernel-window | SYS_UART_WRITE of a kernel-only map-window page (map_page, EL1-RW, EL0-none, slot 11) | returns 0, nothing printed, 1 check refusal |
el0: sys-ptr denied xo-text | SYS_NET_MAC writing its own execute-only code page (+0x800, sentinel) | returns 0, sentinel unchanged, 1 check refusal |
el0: sys-ptr denied write-ro | SYS_NET_MAC writing an EL0-RO page (the store-ro slot, sentinel set via alias) | returns 0, sentinel unchanged, 1 check refusal |
el0: sys-ptr denied straddle | SYS_UART_WRITE(data+4092, 8): EL0-RW page, then a kernel-only page | returns 0, nothing printed, 1 check refusal (the second page is checked) |
el0: sys-ptr denied sys=<name> nr=<n> {read kernel-stub | write xo-text} ×7 | every pointer-taking syscall: uart_write 17, fs_create 19, fs_open 20, fs_mkdir 27, fs_write 22 (reads of the stub page); fs_read 21, net_mac 24 (writes of own xo text) | each returns its error value, 1 check refusal |
el0: sys-ptr sweep syscalls=7 denied=7 | summary | all 7 |
el0: sys-ptr ok | all of the above | only then |
-
Fail-closed: the smoke requires all 15 lines. It rejects
el0: sys-ptr leaked,el0: sys-ptr bad,el0: sys-ptr skipandel0: sys-ptr probe missed. -
Leak-probe build: the
sysptr-leak-probefeature (built together withreach-leak-probe,write-leak-probeintotarget/el0-leak-probe) makes the permission check always pass, keeping only the old guards. That kernel must print:el0: sys-ptr leaked kernel-stub n=8: 8 bytes of the stub page go to the UARTel0: sys-ptr leaked kernel-window n=8el0: sys-ptr leaked xo-text n=6 intact=false checks=0: the kernel wrote EL0’s execute-only textel0: sys-ptr skip write-ro: an EL1 store to an AP[2]=1 page would take an EL1 permission fault, so the probe stops there- no
el0: sys-ptr okand noel0: sys-ptr denied
The smoke requires those and fails otherwise.
-
cargo testaddssys_ptr_requires_el0_permissionandabi_success_from_el0_data_page.
4. G1 allowlist
Unchanged: five slots, fail-closed (app-hdr, app-text, crt-stack-pan, user-stack, store-ro). No new EL0-reachable slot:
- The EL0 data page reuses
crt-stack-pan. - The write-ro target reuses
store-ro. - The kernel-window (slot 11) and straddle (slot 4) pages are EL0-none, so they are not EL0-reachable.
- All probe pages are unmapped before the walk’s steady pass.
el0-reach: live k=3 u=3 a=0 h=0 pages=6 leaks=0 and el0-reach: ok allow=… are byte-identical to ADR-092.
Consequences
- G4 is closed on the default smoke machine: no syscall that takes a user pointer can read a page EL0 cannot read, or write a page EL0 cannot write. Probes cover each such syscall. The ADR-091 draft drops its “direct EL0 memory access only” limit (revision 3). Only G3 (the optional B2-P re-verify) is left.
- App text is now EL0-readable as well as executable. That is the usual Unix-like model for a shared text+rodata page, and still not writable. An EL0 app can read its own code bytes. That is not a kernel-isolation property.
- Follow-on (2026-09-28, proposed): ADR-097 moves app
.rodataonto theapp-hdrpage and maps.textEL0 execute-only again, so an EL0 pointer into text is now refused for copies from EL0 (xo: sys-read text denied … checks=1). Merge requires a sponsor rev 4 accept of the ADR-091 PAN clause (no FEAT_EPAN). This ADR’s text is kept as the record of its time. Update 2026-10-02: the sponsor accepted ADR-091 revision 4 and ADR-097 was merged as8344797; app text is now EL0 execute-only and the limit “EL0 can read its own code bytes” no longer applies.
- Follow-on (2026-09-28, proposed): ADR-097 moves app
- Not covered (stated, not gaps in G4’s scope):
- TOCTOU between check and copy. Today there is one core, and nothing changes the calling task’s user page tables during a syscall. On SMP this would need
AT S1E0R/Wor a fault-fixup copy. AT-based checks.- Real hardware.
LDTR/STTRunprivileged loads/stores would enforce the same rule in hardware; that is a possible later hardening.
- TOCTOU between check and copy. Today there is one core, and nothing changes the calling task’s user page tables during a syscall. On SMP this would need
- Still not “EL0 isolated”, not side-channel, not real hardware, not certified.
Honesty
Say: “ADR-094 closes G4: syscall user pointers need EL0 read/write permission on every page, proven by refused kernel-stub / kernel-window reads, refused xo-text / RO writes, a straddle case, a seven-syscall sweep, and a leak-probe build that must fail.” Do not say “EL0 isolated”, “secure” or “B3 met”.
ADR-095 — G3: B2-P KVM taken-SError re-run on current main (AWS Graviton3 c7g.metal) — PASS
- Status: Accepted (evidence record, docs only). The sponsor asked for G3, the optional B2-P KVM re-run listed as the only remaining limit in ADR-091 revision 3 (DSO relay 2026-09-28). Result: PASS on the first smoke attempt. The taken lower-EL SError still happens under KVM on real Graviton3 hardware on current
main, the same way ADR-085 run 4 showed it at159b178. ADR-091 stays Draft; B3 (the sponsor’s written accept) is not Met; the umbrella “EL0 isolated” stays Planned / non-claim. Nothing is flipped here. - Follow-on (2026-09-28, M6): B3 met: the sponsor accepted the ADR-091 revision-3 sentence and scope in writing (2026-09-28 16:41 CEST). ADR-091 is Accepted. “EL0 isolated” is Verified only as that sentence, with its limits. Text in this ADR that says the umbrella is Planned / non-claim is as of this ADR’s date.
- Date: 2026-09-28 (launch 13:50:32 CEST, smoke PASS before 13:58:13 CEST, teardown verified ≈ 14:06 CEST).
- Tested commit:
mainat8107578a6c715a89c9e5861ebb4102460ba20e80(merge of PR #146, ADR-094). It contains the whole EL0-isolation stack: ADR-087 … ADR-092 and ADR-094, including the ADR-090 evidence class and the ADR-091 draft. mainmoved during review (2026-09-28): PR #145 (ADR-093) merged ase1ee4a8after this run, and this branch merged it. Between the tested8107578ande1ee4a8there is no change tosrc/,user/,libctos/,Cargo.*,build.rs,linker.ld, the target JSON,rust-toolchain.toml,research/qemu-nmi/or the three B2 scripts (emptygit diff), so the guest and QEMU tested here are byte-identical in source to the current tip. ADR-093 changed only docs,scripts/check-public-ids.shand a guard hook inscripts/qemu-smoke.sh.- No
src/,scripts/or.github/workflows/change. The in-treescripts/b2-metal-userdata.shis unchanged; the run used a copy with the small delta listed below._start(0x4008_0000) untouched. CloudAgent not used. Do not self-merge (ADR-002). - Public-identifier rule (ADR-093, PR #145): no account, instance, Spot request, security group, volume, network, image or user identifiers appear in this record. They are described in words.
Context
ADR-085 run 4 (2026-09-26) is the B2-P leg of the ADR-090 taken-SError evidence class. It ran at 159b178, before the M1/M2 boot-path changes (ADR-087 RAM-side identity tears, ADR-088 TTBR1 MMIO high alias + identity MMIO tear) and before ADR-092 / ADR-094. ADR-090 therefore carries the caveat “B2-P is historical”, and ADR-091 revision 3 lists G3: B2-P re-verify on the post-M2 boot path (≈ $0.2 paid run) as its only remaining limit. Since then the b2-serror profile was only checked under stock TCG (ADR-087 / ADR-088 / ADR-090), where it parks as expected.
What was run
Same procedure as ADR-085 run 4, adapted to current main:
| ADR-085 run 4 | ADR-095 G3 (this ADR) | |
|---|---|---|
| ctos commit | 159b178 (branch head) | 8107578 (main, detached checkout of the full SHA) |
| Host | AWS c7g.metal (Graviton3), eu-north-1b, one-time Spot, max price $0.60 | same |
| OS image | Ubuntu 24.04 arm64 | Canonical Ubuntu 24.04 LTS arm64 server (gp3), build 20260923 |
| Root volume | gp3 30 GB, delete on termination | same |
| Access | no SSH, no key pair, no IAM role; security group with no ingress; serial console only | same; IMDSv2 required |
| Tags (every resource) | project=ctos, component=b2-spike | project=ctos, component=g3-kvm-recheck on the instance, root volume, network interface, Spot request and security group |
| Hard stop | shutdown -h +80 + shutdown behaviour terminate | shutdown -h +60 + shutdown behaviour terminate; the user-data powers off right after the smoke |
| QEMU | v10.0.0 7c949c53 + patches 0001 + 0002 via scripts/build-qemu-b2.sh | same script, same pin |
| Guest | cargo build --features b2-serror --target-dir target-b2 | same |
| Smoke | scripts/b2-kvm-smoke.py (fail-closed), CTOS_B2_TIMEOUT=90, CTOS_B2_STALL=15, up to 3 attempts | same |
Credentials path: the sanctioned AWS MCP run_script tool with the ctos-scoped IAM user (no other project’s credentials). Each mutating call ran in its own script, and every launch re-checked that no project=ctos instance was pending, running, stopping, stopped or shutting down (ADR-085 lesson: never two metal boxes).
AWS call sequence (eu-north-1):
- Read-only:
DescribeInstances(guard: 0 active ctos instances),DescribeSpotPriceHistory(c7g.metal eu-north-1b $0.2476/h, unchanged from ADR-085),DescribeImages(latest Canonical Ubuntu 24.04 arm64),DescribeSubnets/DescribeVpcs(default VPC),DescribeSecurityGroups(no leftover ctos group). CreateSecurityGroupctos-g3-kvm-recheckin the default VPC, tagged at create. No ingress rule was added (default egress only).DescribeInstancesguard, then oneRunInstances:c7g.metal, one-time Spot, user-data below, tags on instance / volume / network interface / Spot request. Launch 13:50:32 CEST.- Read-only polling:
GetConsoleOutput(Latest=true) andDescribeInstances. - After the instance self-terminated:
DeleteSecurityGroup, then the verification describes in Teardown.
User-data = scripts/b2-metal-userdata.sh with this delta (nothing else changed):
- shutdown -h +80 "ctos b2 hard timer" || true
+ shutdown -h +60 "ctos g3 hard timer" || true
- echo "CTOSB2: uname $(uname -a)"
+ echo "CTOSB2: uname $(uname -srm)" # no hostname on the console
+ echo "CTOSB2: kvm-api $(python3 -c "import fcntl,os;print(fcntl.ioctl(os.open('/dev/kvm',os.O_RDWR),0xAE00))" 2>&1)" # KVM_GET_API_VERSION
- git clone --depth 1 --branch "__BRANCH__" https://github.com/artofdream/ctos.git ctos
+ git clone --branch main https://github.com/artofdream/ctos.git ctos && git -C ctos checkout --detach 8107578a6c715a89c9e5861ebb4102460ba20e80
+ echo "CTOSB2: qemu-version $(tools/qemu-b2/bin/qemu-system-aarch64 --version | head -1)"
Pre-flight (free, agent box, before paying): cargo build --features b2-serror at 8107578, then CTOS_B2_ACCEL=tcg python3 scripts/b2-kvm-smoke.py with stock QEMU 10.0.13. The guest booted through ident: mmio-high … ident: mmio to b2: boot → el0: serror-arm; stock QEMU answered inject-nmi=>error:machine does not provide NMIs, so the guest printed el0: serror-park / b2: not taken and the smoke failed closed, which is the expected stock-TCG result. It showed the profile still boots on current main; it is not SError evidence.
Versions (from the serial console)
- CPU:
Neoverse-V1/AWS Graviton3 … @ 2.6GHz;CPU: All CPU(s) started at EL2;kvm [1]: VHE mode initialized successfully. - Host kernel:
Linux 7.0.0-1013-aws aarch64(Ubuntu 24.04 AWS kernel). KVM is the in-kernel module of that kernel;KVM_GET_API_VERSION= 12; vGIC is GICv3 only (disabling GICv2 emulation),IPA Size Limit: 48 bits. - QEMU:
QEMU emulator version 10.0.0 (v10.0.0-dirty)(dirty = patches 0001 + 0002 applied),Accelerators supported in QEMU binary: kvm tcg. Guest command:-accel kvm -machine virt,gic-version=3 -cpu host -m 128M …. - Rust:
nightly-2026-09-12fromrust-toolchain.toml, auto-installed byrustup 1.29.1. Honest note: theCTOSB2: rustcline captured rustup’s “syncing channel updates for nightly-2026-09-12-aarch64-unknown-linux-gnu” message instead of arustc --versionstring; the toolchain path in the cargo warning confirms the same pin.cargo rc=0,qemu-build rc=0.
Evidence (serial console, verbatim)
CTOSB2: uname Linux 7.0.0-1013-aws aarch64
CTOSB2: cpu Vendor ID: ARM;BIOS Vendor ID: Amazon EC2;Model name: Neoverse-V1;BIOS Model name: AWS Graviton3 N/A CPU @ 2.6GHz;
CTOSB2: dmesg [ 0.026519] CPU: All CPU(s) started at EL2
CTOSB2: dmesg [ 0.227753] kvm [1]: VHE mode initialized successfully
CTOSB2: /dev/kvm=present
CTOSB2: kvm-api 12
CTOSB2: head 8107578a6c715a89c9e5861ebb4102460ba20e80
CTOSB2: qemu-version QEMU emulator version 10.0.0 (v10.0.0-dirty)
CTOSB2: qemu build-qemu-b2: accelerators: Accelerators supported in QEMU binary: kvm tcg
CTOSB2: ===== smoke attempt 1 =====
CTOSB2: b2-kvm-smoke: cmd: /root/ctos/tools/qemu-b2/bin/qemu-system-aarch64 -accel kvm -machine virt,gic-version=3 -cpu host -m 128M -display none -serial stdio -monitor none -qmp unix:/tmp/ctos-b2-yyvqz4_3/qmp.sock,server,nowait -kernel target-b2/aarch64-ctos/debug/ctos
CTOSB2: b2: p1 mmu-on
CTOSB2: ident: jump
CTOSB2: ident: mmio-high gic=0xffffff8008000000 uart=0xffffff8009000000 virtio=0xffffff800a000000
CTOSB2: pan: id=2
CTOSB2: pan: present
CTOSB2: pan: enabled
CTOSB2: pan: el1-fault
CTOSB2: ident: ram lo=0x4026b000 hi=0x48000000 pages=32149
CTOSB2: ident: low lo=0x40000000 hi=0x40080000 pages=128 user=0
CTOSB2: ident: tail lo=0x400ca000 hi=0x40201000 pages=311 user=311
CTOSB2: ident: kend lo=0x4025a000 hi=0x4025b000 pages=1 user=0
CTOSB2: ident: mmio lo=0x0 hi=0x40000000 l1=1
CTOSB2: b2: boot (ADR-085 KVM SError profile)
CTOSB2: el0: serror-armb2-kvm-smoke: qmp stop=>ok inject-nmi=>ok cont=>ok (ADR-085: KVM_SET_VCPU_EVENTS serror_pending)
CTOSB2:
CTOSB2: el0: standing
CTOSB2: el0: serror
CTOSB2: el0: restored
CTOSB2: b2: taken
CTOSB2: b2: doneb2-kvm-smoke: summary armed=True injected=True taken_marker=True b2_taken=True park=False kvm_err=False
CTOSB2: b2-kvm-smoke: PASS taken lower-EL SError observed under accel=kvm ('el0: serror' + 'b2: taken')
CTOSB2: smoke attempt 1 rc=0
CTOSB2: end 2026-09-28T11:58:13Z
(el0: serror-arm + host QMP line and b2: done + summary share console lines because guest serial and host log are tee’d to the same console, as in ADR-085.) Full CTOSB2: excerpt (66 lines, including the early markers b2: e0 … p1 and every ident: line): research/daily-briefs/2026-09-28-adr-095-g3-console.txt.
Reading: the guest ran natively on Graviton3 under KVM through the current boot path: ident: mmio-high (ADR-088 TTBR1 Device alias), ident: low / tail / kend (ADR-087 RAM-side tears), ident: mmio … l1=1 (ADR-088 identity MMIO tear), PAN enable and EL1 fault. On the el0: serror-arm cue QEMU (patch 0002) turned inject-nmi into KVM_SET_VCPU_EVENTS{serror_pending=1}; KVM raised a virtual SError (HCR_EL2.VSE), and the guest took it at the lower-EL SError vector while standing at EL0: the bare el0: serror line is printed only by that handler with the standing-EL0 expectation armed (src/exception.rs). No el0: serror-park, no b2: not taken, fail-closed smoke PASS on attempt 1.
Difference from TCG: Graviton3 reports pan: id=2 (ID_AA64MMFR1_EL1.PAN = 2); PAN enable and the EL1-vs-EL0 fault still pass. ident: ram … pages=32149 vs 32150 in run 4 follows the image growth since 159b178.
Cost
One c7g.metal Spot instance, per-second billing, $0.2476/h (DescribeSpotPriceHistory, eu-north-1b, flat on 2026-09-28). Running 13:50:32 → self-poweroff ≈ 13:58:35 CEST (console end 11:58:13Z + 20 s sleep) ≈ 8.1 min ≈ $0.033. The instance then sat in shutting-down and was terminated by ≈ 14:06 CEST; even billing that whole 15.5-minute window at the Spot rate gives only ≈ $0.064. gp3 30 GB and one public IPv4 for the same window: < $0.01. Total ≈ $0.04 (upper bound ≈ $0.07), under the ≈ $0.2 budget and far under the $20 stop gate.
Teardown
- The instance powered itself off after the smoke and terminated (
Client.InstanceInitiatedShutdown; shutdown behaviourterminate). The root volume was delete-on-termination; the network interface went with the instance; the one-time Spot request closed. - The security group was deleted after the instance reached
terminated. - Verification (≈ 14:06 CEST, read-only describes, eu-north-1):
project=ctosinstances pending / running / stopping / stopped / shutting-down → 0; the run’s instance →terminated;project=ctosvolumes → 0, and the run’s root volume by ID →InvalidVolume.NotFound; tagged network interfaces → 0;project=ctossecurity groups → 0, and the run’s group by ID →InvalidGroup.NotFound; tagged key pairs → 0; tagged snapshots → 0; tagged Elastic IPs → 0; the onecomponent=g3-kvm-recheckSpot request →closed/instance-terminated-by-user(a closed one-time request is not billable and ages out by itself). - No key pair, IAM role, instance profile, snapshot, AMI or Elastic IP was created.
Decision
- G3: done, PASS. The B2-P leg of the ADR-090 evidence class is re-verified on current
main(8107578) under KVM on real Graviton3 hardware, same host class, same QEMU pin and patches, same fail-closed smoke. ADR-091 gap G3 is closed with a pointer here. - ADR-090 §1 is unchanged. It is the sponsor’s verbatim D1 paragraph and still names ADR-085 run 4. This ADR is a supplementary re-run of that leg on the post-M2 boot path. The ADR-090 caveat “B2-P is historical (at
159b178)” is answered by this record; the caveats “reduced profile” and “not real hardware running the full default image” still stand. - ADR-091 sentence: no change needed. The revision-3 sentence says taken lower-EL SError “is evidenced by the ADR-090 evidence class” and that the sentence is “not a real-hardware claim”. Both stay true and complete. G3 was a limit on the evidence, not on the wording.
- Not changed: ADR-091 stays Draft; B3 not Met; umbrella Planned / non-claim; no CI job runs KVM; the default smoke on stock QEMU still requires
el0: serror-park; the B1 pin is unchanged.
What this does not show
- Reduced profile only.
b2-serrorskips GIC / timer / virtio / FAT, and it does not run the identity inventory (ident: inv), the EL0-reachability walk (ADR-092), the EL0 write-fault probes or the syscall-pointer probes (ADR-094). Those are CI-gated on TCG only. - One-off, not CI. Re-running needs another paid metal run or a self-hosted arm64 KVM runner (B2-S, not registered).
- No host-side
straceof the ioctl (same limitation as ADR-085); the mechanism is QEMU’skvm_put_vcpu_eventspath and the fail-closed marker was observed. - Not “the default ctos image runs on real hardware”, not “EL0 isolated”, not “secure OS”.
Honesty
Say: “ADR-095 (G3): on 2026-09-28 an AWS Graviton3 c7g.metal (eu-north-1b, Spot) re-ran the ADR-085 B2-P procedure on ctos main 8107578; QEMU -accel kvm with patch 0002 set serror_pending via KVM_SET_VCPU_EVENTS on the el0: serror-arm cue, and the b2-serror profile took the lower-EL SError while standing at EL0 (el0: serror / b2: taken, fail-closed smoke PASS, attempt 1). One-off paid run, not CI, reduced profile. ADR-091 G3 is closed; ADR-091 is still a draft.” Do not say:
- taken SError is Verified in CI under KVM, on stock QEMU, or on free runners
- the default image boots on real hardware
- “EL0 isolated”, “B3 met”, or “ADR-091 accepted”
Consequences
- Docs: this ADR; ADR-091 gap G3 marked done (pointer here), sentence untouched, status untouched; honesty-ledger rows; threat model v1.69 note; SUMMARY; research MOC; console excerpt research/daily-briefs/2026-09-28-adr-095-g3-console.txt.
- Code / scripts / CI: unchanged. CI on PR #147: honesty-ledger row “CI on GitHub (ADR-095 / PR #147)”.
- AWS: everything created for the run is deleted and verified (above).
ADR-096 — Task-to-task EL0 isolation probe (task A cannot read or write task B’s page)
- Status: Accepted (implementation + evidence). Adds a fail-closed probe that CI gates: an EL0 task on one TTBR0/ASID cannot read or write another EL0 task’s private page at the same VA, including right after the other task ran. This is additional evidence recorded next to the accepted ADR-091 sentence. It does not change that sentence or its scope. Task-vs-task isolation stays in ADR-091’s non-scope list. Any wording that adds it (a possible revision 4) goes to the sponsor in writing first.
- Date: 2026-09-28
- Decision path: sponsor-approved milestone (DSO relay, 2026-09-28 21:21 CEST): “task-to-task isolation probe”.
- Base:
mainat9a1c2bb(M6 flip, PR #148). _start(0x4008_0000) untouched. No probe maps, reads or writes it.
Context
ADR-091’s non-scope says: “EL0-to-EL0 (task-vs-task) isolation beyond the asid: ok TLB mile and per-task TTBR0. No probe that one EL0 task cannot read another task’s pages.” The asid: ok mile (ADR-013) runs at EL1: it switches TTBR0 between the kernel L1 tagged ASID 1 and the ASID-B L1 tagged ASID 2 and loads through each. No EL0 code runs in it.
The tree already has a second address space: the ASID-B root (L1_ASID_B → L2_ASID_B → its own map-window L3_ASID_B, ASID 2). Every loaded app uses the user root (L1_USER, ASID 1), which shares the kernel’s map-window L3.
Decision
1. Two EL0 tasks on two roots (src/t2t.rs)
| Task A | Task B | |
|---|---|---|
| TTBR0 | user root L1_USER, ASID 1 (paging::user_ttbr0()) | ASID-B root L1_ASID_B, ASID 2 (paging::asid_b_ttbr0()) |
| Code | EL0 execute-only page at 0x80002000 (app-text slot), shared window L3 | own frame, EL0 execute-only, nG, at 0x80002000 in L3_ASID_B |
| Private data | round 1: nothing at 0x80007000; round 2: own frame, EL0-RW NX, nG | own frame, EL0-RW NX, nG, at 0x80007000 (user-stack slot) |
- Both tasks run at EL0 through a masked
ERET. B uses the newexception::eret_to_el0_masked_ttbr(an explicit TTBR0 value, ASID included); A uses the existing path. The existingeret_to_el0_*helpers now call the same asm withpaging::user_ttbr0(), so their behaviour does not change. - A one-shot capture (
exception::arm_el0_capture/el0_capture_result) records ESR, FAR, ELR andx1of the task’s first lower-EL sync exception (a data abort, or theBRK #0that ends the payload) and returns to EL1. - Payloads are kernel-built:
LDR X1, [X0](read) orMOVZ/MOVK X1, #val; STR X1, [X0]; LDR X1, [X0](write), each ending inBRK #0, withX0 = 0x80007000. - Frames are written and checked through their TTBR1 alias (PAN blocks EL1 access through the EL0 VA).
- Sentinels: B
0x7a5cb0b0; A0x7a5ca0a0, then0x7a5ca2a2; A’s attempted store into B’s page0x7a5ceeee.
2. Probe steps (called from kernel_main_high after asid: ok)
Round 1 (A has no mapping at the VA):
- Walk: walk every valid leaf of A’s root and B’s root for B’s frame. A’s root must have no leaf at all that outputs to B’s frame; B’s root must have exactly one EL0-reachable leaf for it (control). Marker:
t2t: walk b-frame u-refs=0 u-el0=0 a-el0=1. - B runs (ASID 2): stores its sentinel, reloads it (
BRK,x1= sentinel, frame = sentinel). Marker:t2t: b-own ok asid=2 va=0x80007000 val=0x7a5cb0b0. - A reads (ASID 1) the same VA right after B, with no TLBI of
0x80007000in between (A’s code-page map/unmap invalidates only0x80002000). A must take a lower-EL translation fault: EC 0x24, WnR 0, FAR = the VA, ELR = theLDR,x1still 0. Marker:t2t: a-read fault asid=1 va=0x80007000 esr=0x92000007 far=0x80007000 ec=0x24 wnr=0 dfsc=trans-l3 got=0x0 after-b no-tlbi. - A writes the same VA: translation fault with WnR 1, FAR = the VA, ELR = the
STR; B’s frame still holds B’s sentinel. Marker:t2t: a-write fault asid=1 va=0x80007000 esr=0x92000047 far=0x80007000 ec=0x24 wnr=1 dfsc=trans-l3 b-intact=true.
Round 2 (same VA, A gets its own frame):
- Walk: A’s frame is EL0-reachable only in A’s root, B’s frame only in B’s root. Marker:
t2t: walk same-va a-frame u-el0=1 a-el0=0 b-frame u-el0=0 a-el0=1. - B runs again first. Then A reads the VA and must see its own sentinel, never B’s:
t2t: same-va a-sees-own asid=1 got=0x7a5ca0a0 b=0x7a5cb0b0. - A writes the VA: the value lands in A’s frame only; B’s frame is unchanged.
t2t: same-va a-write-own got=0x7a5ca2a2 b-intact=true. - B reads the VA after A’s trips and sees its own value; A’s frame is unchanged.
t2t: same-va b-sees-own asid=2 got=0x7a5cb0b0 a-intact=true. - Summary:
t2t: ok asid-a=1 asid-b=2 va=0x80007000 read,write,same-va.
Setup checks before step 1: the two TTBR0 values carry ASIDs 1 and 2 and different table bases; both window slots are empty in both roots; B’s leaf has nG. All probe pages are unmapped (and TLBI VAAE1, all ASIDs) before the ADR-092 steady walk. Both VAs are ADR-092 allowlisted slots, so the G1 allowlist does not grow; the steady walk still reports pages=0 leaks=0.
3. Fail-closed wiring (scripts/qemu-smoke.sh)
- The default smoke requires all nine
t2t:lines above (regex in the script: ESR low bits and table level may vary within “translation fault”). It rejectst2t: leak,t2t: badandt2t: probe missed, and any task-A line that shows B’s sentinel (got=0x7a5cb0b0). - Leak-probe build: new feature
t2t-leak-probe(built together withreach-leak-probe,write-leak-probe,sysptr-leak-probeintotarget/el0-leak-probe) maps B’s frame into A’s root at the same VA (a shared-frame bug). That kernel must printt2t: leak-probe planted b-frame in a-ttbr0 va=0x80007000,t2t: leak walk b-frame u-refs=1 u-el0=1,t2t: leak read asid=1 va=0x80007000 got=0x7a5cb0b0andt2t: leak write asid=1 va=0x80007000 b=0x7a5ceeee, and must not printt2t: ok. The smoke requires this. cargo testaddsel0_task_cannot_read_or_write_other_task.- The three required checks run
scripts/qemu-smoke.sh, so they enforce all of the above. - Checker negative test (local, 2026-09-28): the ADR-096 check block from
scripts/qemu-smoke.shwas run against a passing serial log (it passed) and against five tampered logs:t2t: okdeleted, B’s sentinel on the task-A same-VA line,got=on the read-fault line changed to B’s sentinel,t2t: probe missedappended, and the real leak-build serial. It exited 1 on each, naming the missing or forbidden line. A sixth log added an extra task-A line with B’s sentinel while keeping all required lines; it was rejected by the sentinel guard alone.
What this proves
On the default smoke machine (QEMU virt, -cpu cortex-a76, TCG, one core): an EL0 task on the user root (ASID 1) cannot read or write the private page of an EL0 task on the ASID-B root (ASID 2) at the same VA. This holds when A has nothing mapped there (architectural translation fault, B’s data never observed, B’s frame unchanged), and when A has its own page there (each task sees and changes only its own frame). A’s root has no translation of any kind to B’s frame. The kernel does not issue a TLB invalidate of that VA between B’s trip and A’s trip.
What this does not prove
- Not part of the ADR-091 sentence. The accepted sentence is unchanged; task-vs-task stays in its non-scope. A revision 4 would need the sponsor’s written accept.
- Stale-TLB angle on hardware. QEMU TCG flushes its own software TLB whenever a TTBR0 write changes the ASID (
vmsa_ttbr_writein QEMUtarget/arm/helper.c). So “no TLBI after B” shows only that the kernel does not rely on a flush it issues itself. It says nothing about how a hardware ASID-tagged TLB would behave. Theasid: okmile has the same limit. - Two loaded apps. Both tasks are kernel-built payloads. There is no scheduler that switches between two loaded apps; every loaded app still runs on the one user root (ASID 1), one at a time. The probe checks the page-table and ASID mechanism that such a switch would use, not a multi-app scheduler.
- One core. No SMP, no cross-core TLB shootdown.
- Shared kernel-mapped pages. The user root shares the kernel’s map-window L3 and maps the kernel image text EL1-only; those are covered by the ADR-092 walk (nothing else is EL0-reachable), not by this probe.
- Not side-channel, not real hardware, not DMA/IOMMU, not a certification.
Consequences
- Honesty ledger row (Verified, scoped as above), threat model v1.71, daily brief, SUMMARY and MOC entries.
el0.mdpoints to this ADR as evidence next to the ADR-091 claim. ADR-091 itself is not edited: its sentence, scope table and non-scope list stay exactly as the sponsor accepted them.
ADR-097 — EL0 app text execute-only (rodata moved to the app-hdr slot)
Accepted (2026-10-02). The sponsor accepted ADR-091 revision 4 in writing: “I accept ADR-091 rev 4”, given in answer to the exact wording in Revision-4 wording below. Written by the sponsor in the DSO chat, recorded in the DSO vault, relayed by DSO. This PR (#150) was merged as
8344797after the accept. ADR-091 was then edited to revision 4 (PAN clause and code-bytes limit only) by a docs-only follow-up PR; the record is in ADR-091 (“B3 rev 4”).Original notice, 2026-09-28 (kept as history; the condition it set was met on 2026-10-02): Merge requires sponsor rev 4 written accept. This change makes one clause of the accepted ADR-091 revision-3 sentence false as literally worded: “PAN is enabled and EL1 faults on EL0 memory”. The execute-only app text page is EL0 memory (EL0 can execute it), and on the default smoke machine (no FEAT_EPAN) an EL1 load of it does not fault with PAN set. ADR-091 is not edited here. Its sentence and scope stay exactly as accepted. A candidate revision-4 wording is proposed below for the sponsor to read. Until the sponsor accepts a revision 4 in writing, this PR must not be merged.
- Status: Accepted (2026-10-02). The sponsor accepted ADR-091 revision 4 in writing (“I accept ADR-091 rev 4”); merged as
8344797. Status as of 2026-09-28, kept for history: Proposed (implementation + evidence; merge blocked on sponsor rev 4 written accept). - Date: 2026-09-28
- Decision path: sponsor-approved milestone 3 (DSO relay, 2026-09-28): split the app’s read-only data out of the app code page so EL0 app text becomes execute-only; $0 (local QEMU + existing GitHub Actions checks).
- Also merges PR #151 (ADR-098,
isolation/pan-span-fix, based onmain) for the PAN fix. - Base: stacked on PR #149 (ADR-096, branch
isolation/t2t-probe@e6e755a). It reuses #149’s EL0 capture helpers (arm_el0_capture/eret_to_el0_masked/el0_capture_result) and extends the samescripts/qemu-smoke.shleak-build block. Merge #149 first. _start(0x4008_0000) untouched.
Context
ADR-094 closed G4 by making every syscall pointer check require EL0 permission. To keep app string literals printable through uart_write, it mapped the app code page (text and rodata, one page at 0x80002000) EL0 read-only + executable and said so openly: “An EL0 app can read its own code bytes.” ADR-091’s honesty section carries that as a limit.
Decision
Layout (every app, same five slots)
| Slot (label unchanged) | VA | Contents | Stage-1 leaf |
|---|---|---|---|
app-hdr | 0x80000000 | ELF headers + .rodata (all string literals and constants) | AP=11 (EL1 RO, EL0 RO), UXN=1, PXN=1 (unchanged: map_el0_ro) |
app-text | 0x80002000 | .text only | AP=10 (EL1 RO, EL0 none), UXN=0, PXN=1 (new map_el0_xo); was AP=11 |
crt-stack-pan, user-stack, store-ro | unchanged | unchanged | unchanged |
- All eight
user/*/linker.ldplace.rodataat0x80000000 + SIZEOF_HEADERSand start.textalone at0x80002000. Each app ELF now has exactly twoPT_LOADs:Rat0x80000000andR Eat0x80002000. The asm fallbackhello.Smoved its strings to.rodata. - No sixth slot. The allowlist, its labels and the at-rest / live inventory lines are byte-identical:
el0-reach: live k=3 u=3 a=0 h=0 pages=6 leaks=0andel0-reach: ok allow=app-hdr[0x80000000,0x80001000),app-text[0x80002000,0x80003000),crt-stack-pan[…),user-stack[…),store-ro[…). build.rsenforces it at build time: loads below the text page must be read-only and inside the0x80000000page; each app’s marker strings must be in that page and absent from the text segment.src/loader.rsenforces it at load time (host unit tests + boot):Romay only map at0x80000000,Execonly at0x80002000,Rwnever; a page that would combineExec+Rois rejected (ParseError::RoOnText, was a union to EL0 RX); anything else isParseError::OffSlot.Execpages map throughmap_el0_xo.- The A2
libctospath maps the hello ro page at the map window withmap_el0_roand its text withmap_el0_xo.
EL0 reachability (inventory unchanged, XO page still counted)
The ADR-092 walk already defines EL0-reachable as AP[1] set or UXN clear, so the XO page (AP[1]=0, UXN=0) is counted and must match the app-text allowlist entry. paging::leaf_view reports reachable from the same rule, and a new probe plants an XO leaf (EL0 none, UXN=0) at an unallowlisted VA (0x80190000) and requires the walk to catch it (why=va ap-el0=0 uxn=0). So the inventory cannot pass by only looking at AP.
Syscall pointers (ADR-094, strengthened)
user_range_ok_max(…, Read) already requires AP[1] on every page. A pointer into the XO text now fails it: uart_write(0x80002000, 8) from EL0 returns 0 with one permission-check denial, while uart_write of the rodata message in app-hdr still prints.
PAN / EPAN facts (probed, not assumed)
ID_AA64MMFR1_EL1.PAN= 2 on QEMU 10.0.13-cpu cortex-a76(FEAT_PAN2, no FEAT_EPAN) on the box and in every CI smoke job (pan: id=2).- Without FEAT_EPAN (and
SCTLR_EL1.EPAN), PAN only blocks EL1 data access to pages that are EL0 data-accessible (AP[1]=1). The XO page hasAP[1]=0, so PAN does not apply. Probe: with PAN set (MRSPAN=1, and a control load of the EL0-ROapp-hdrpage faulting withSPSR.PAN=1 after EL0 trips), an EL1 load of the XO text VA does not fault and returns the code word (xo: el1-pan text va=0x80002000 fault=no match=true mmfr1-pan=2 epan=no). An EL1 store to it faults, but becauseAP[2]=1(read-only), not because of PAN (xo: el1-write text … fault=yes intact=true). - Consequence for ADR-091: the clause “PAN is enabled and EL1 faults on EL0 memory” is not literally true for the XO app text page. Before this change the only standing EL0 pages were
AP[1]=1pages, so PAN covered all of them (kernel-built probe payload pages mapped EL0-execute-only viamap_el0_execare transient and already had the same property). This is why merge needs a rev 4 accept.
Pre-existing PAN gap: split out to ADR-098
While probing the above, this work found that PAN was not set while the kernel handled EL0 exceptions (SCTLR_EL1.SPAN was 1) or after it returned from an EL0 trip (return_from_el0 resumed with SPSR.PAN clear). ADR-080’s boot probe runs before any EL0 trip, so it never saw this. Because that made the accepted ADR-091 clause “PAN is enabled” false on main while handling EL0 exceptions, the fix was split out (DSO, 2026-09-28 23:54 CEST) into ADR-098 (PR #151, based on main, no rev 4 needed). This branch merges that branch (no force-push) and no longer carries its own copy of the fix: SPAN=0, SPSR.PAN on return, the quiet EL1 fault catcher, pan::pstate_pan / pan::sctlr_span and the pan-keep-leak-probe feature all come from ADR-098. The xo: pan live-syscall … line is kept; it now echoes the reading taken by ADR-098’s pan::on_live_syscall in the same handler (no second implementation). The xo: el1-pan control … after-el0-trips line stays as the xo image probe’s own calibration of that reading.
Probes and markers (all fail-closed in scripts/qemu-smoke.sh)
Live (real hello-libctos, inside its SYS_YIELD handler, called from the ADR-092 live walk):
xo: pan live-syscall pstate-pan=1 sctlr-span=0xo: live app=hello text va=0x80002000 ap=2 uxn=0 pxn=1 el0=x reach=1 hdr va=0x80000000 ap=3 uxn=1 el0=r rodata=0x80000120 in-text=0 sys-read text=deny rodata=allow(the app’s ownlibctos: histring is found in the hdr frame and not in the text frame; the ADR-094 read check denies the text VA and allows the rodata VA)
Image probe (an app-shaped ELF built in-kernel, loaded by the real loader, run at EL0 on the user TTBR0):
xo: layout loader app-hdr va=0x80000000 ap=3 uxn=1 el0=r app-text va=0x80002000 ap=2 uxn=0 pxn=1 el0=x reach=uxn— the inventory reports the new layoutxo: el0-exec ok entry=0x80002000 el0-rodata va=0x80000800 got=0x584f524441544131— EL0 executes its text and loads a rodata sentinel from its new locationxo: el0-read-text fault va=0x80002000 esr=0x9200000f far=0x80002000 ec=0x24 wnr=0 dfsc=perm-l3 got=0x0— an EL0 load of its own text takes a lower-EL permission faultxo: el0-uart rodata-str ok— printed by EL0 throughuart_writefrom a rodata stringxo: sys-read rodata ok va=0x80000810 n=27xo: sys-read text denied sys=uart_write nr=17 va=0x80002000 n=0 checks=1— a syscall given a text pointer is refusedxo: el1-pan control app-hdr va=0x80000800 mrs-pan=1 fault=yes spsr-pan=1 after-el0-tripsxo: el1-pan text va=0x80002000 fault=no match=true mmfr1-pan=2 epan=no— fact, not a pass condition in the security sense; the smoke pins it so a CPU-model change forces a re-readxo: el1-write text va=0x80002f00 fault=yes intact=truexo: reach xo-plant caught va=0x80190000 roots=k,u why=va ap-el0=0 uxn=0xo: ok text=el0-xo rodata=app-hdr el0-read=fault sys-read=deny slots=5
The smoke requires all thirteen lines (ESR low bits / fault level flexible within permission faults), rejects xo: leak / xo: bad / xo: probe missed / xo: live missed, and rejects any xo: el0-read-text line with a nonzero got=. The same probe runs again as #[test_case] el0_app_text_is_execute_only under cargo test (without the live part). New host unit tests: loader_plans_hello_as_ro_hdr_and_xo_text, loader_rejects_ro_on_text_page, loader_rejects_pages_outside_app_slots.
Negative tests
- Build: the existing
el0-leak-probesmoke build now also enablesxo-leak-probe(text mapped EL0 RX again:AP=11,UXN=0) and ADR-098’span-keep-leak-probe(the pre-fix PAN behaviour; its ownpan: lost …lines are checked by the ADR-098 block). The smoke requiresxo: bad pan live-syscall pstate-pan=0 sctlr-span=1,xo: leak live text ap=3 el0=rx sys-read=allow,xo: leak layout text ap=3 el0=rx,xo: leak el0-read-text va=0x80002000 got=<nonzero code word>,xo: leak sys-read text n=8 checks=0,xo: bad el1-pan control pan=true mrs-pan=0 fault=false spsr-pan=falseandxo: probe missed, and fails ifxo: ok,xo: el0-read-text faultorxo: sys-read text deniedappears. All the ADR-092/094/096 leak lines are still required from the same build. - Tampered logs (the ADR-097 smoke block run against edited copies of a good serial log; each must fail):
xo: okdeleted; the read-text fault replaced by a leak line; a translation fault instead of a permission fault; live text shown asap=3 el0=rx; the inventory showingreach=none;xo: probe missedappended; the text-pointer syscall withchecks=0; an EPAN CPU (fault=yes … mmfr1-pan=3 epan=yes); PAN clear in the handler; and the real leak-build serial. All ten failed with a named missing / forbidden line; the unedited log passed. Results are in the PR body and the honesty ledger.
ADR-091 clause check (sentence not edited)
State of 2026-09-28 (this PR did not edit ADR-091). On 2026-10-02, after the sponsor’s accept, ADR-091 was edited to revision 4; the table below is the check that led to it.
| Clause | Still literally true? |
|---|---|
Default smoke machine (QEMU virt, -cpu cortex-a76, TCG, one core) | Yes (unchanged). |
| Own TTBR0 + own ASID per EL0 task | Yes (unchanged; asid: ok, el0: standing). |
| EL0 faults when it reads, writes or executes kernel memory | Yes (unchanged markers). |
| Only EL0-reachable pages are five allowlisted user slots backed by non-kernel frames, checked live and at rest | Yes. Still five slots, same labels; the XO page is counted as reachable (UXN=0) and allowlisted; live/at-rest lines byte-identical. |
| PAN is enabled and EL1 faults on EL0 memory | No, not literally, for the XO app text page (no FEAT_EPAN: EL1 loads of it do not fault). PAN is enabled (and, with ADR-098 merged here, also during syscall handling and after EL0 trips); EL1 still faults on every EL0 data-accessible page. |
| Lower-EL IRQ and FIQ taken while EL0 stands | Yes (unchanged). |
| Taken lower-EL SError evidenced by the ADR-090 class | Yes (unchanged). |
Only identity mapping left in any TTBR0 is the EL1-only _start stub page | Yes (unchanged; ident: lines). |
| A syscall copies from/to an EL0 pointer only if EL0 may read/write every page | Yes, and stronger for text: EL0 can no longer read its text, so text pointers are refused for copies from EL0. |
| Not speculative / real-hardware / certification | Unchanged. |
Revision-4 wording (proposed 2026-09-28; accepted 2026-10-02)
Accepted 2026-10-02: the sponsor wrote “I accept ADR-091 rev 4” in answer to exactly this wording. ADR-091 now carries it, character for character (PAN clause and code-bytes limit; nothing else changed). As proposed on 2026-09-28:
Change only the PAN clause, from “PAN is enabled and EL1 faults on EL0 memory” to:
“PAN is enabled, including while the kernel handles exceptions from EL0, and EL1 faults on EL0-readable memory (the execute-only app text page is not covered: this CPU has no FEAT_EPAN)”
and replace the limit “EL0 can read its own code bytes” with “app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN)”. Everything else unchanged. (2026-09-28 text, history:) This is a proposal for the sponsor to read, not an edit of ADR-091. If the sponsor declines, the alternative is to close this PR (or keep only the separable PAN fix).
Limits
- One core, the TCG default smoke machine; not real hardware (no KVM re-run of this change), not side-channel, not a certification.
- Execute-only is enforced for EL0 only. The kernel can still read app text (no FEAT_EPAN, and PAN would not cover it anyway on this CPU). A future CPU with FEAT_EPAN would need
SCTLR_EL1.EPANset; the smoke pinsepan=noso that change is noticed. - The live check covers
hello-libctos; the other seven apps are covered by the build-time checks (build.rs), the loader rules (host tests) and the fact that they all run in the smoke with the same loader path. The image probe uses an in-kernel app-shaped ELF, not a separate user crate. - The A9 cross-update to the prior OS image still boots the prior loader; that loader maps
Rsegments withmap_el0_roandR Eas execute-only, so the new two-segment ELFs load there too (smoke green), but the prior OS has no ADR-097 probe. - Writable (
Rw) app segments are now refused at load time: no allowlisted slot takes them. None of the eight apps has one (their.data/.bssare empty), so nothing changes today, but an app that needs writable globals would need a design change (it would need a slot, which is an ADR-091 question). - The PAN fix (ADR-098) is proven by the live
MRS PANread and the post-trip checks, not by an EL1 load inside a handler (fatal by design).
ADR-098 — PAN held while the kernel handles EL0 exceptions (correction of a live-claim defect)
- Status: Accepted (implementation + evidence). Fixes a defect in the accepted ADR-091 claim. It does not change the ADR-091 sentence or scope.
- Date: 2026-09-29
- Decision path: DSO relay (2026-09-28 23:54 CEST). The defect was found while working on ADR-097 (PR #150, execute-only app text, which waits for a sponsor rev 4 accept). It is split out here so the fix can merge without waiting for that accept.
- Base:
mainat9a1c2bb(M6 flip, PR #148). Not stacked on #149 or #150. _start(0x4008_0000) untouched.
Context: the defect
ADR-091’s accepted sentence says “PAN is enabled and EL1 faults on EL0 memory”. The evidence is ADR-080’s boot probe (pan: enabled / pan: el1-fault). That probe runs once, early in boot, before any EL0 trip. Two things in the exception path undid PAN once EL0 had run:
SCTLR_EL1.SPANwas 1 (the QEMU reset value; the kernel never cleared it). With SPAN=1, taking an exception to EL1 leaves PSTATE.PAN unchanged. EL0 was entered by ERET with an SPSR whose PAN bit (22) is clear, so every exception from EL0 (SVC, IRQ, FIQ, abort) started its EL1 handler with PAN clear. PAN came back only if the handler called a copy helper (pan::with_user_accessrestores it after the copy).return_from_el0resumed the kernel with SPSR0x3C4(PAN clear). After any EL0 trip that ended by returning to the kernel continuation, the kernel kept running with PAN clear until the nextwith_user_accessset it again.
Measured before the fix (reproduced by the pan-keep-leak-probe build below): in the loaded app’s SYS_YIELD handler MRS PAN reads 0 and SCTLR_EL1.SPAN is 1; after the app exits, MRS PAN reads 0 and an EL1 load of an EL0-readable page does not fault. The boot probe still prints pan: enabled and pan: el1-fault in that same kernel.
Consequence for the claim: from the ADR-080 enable (2026-09-19), and so also from the M6 accept (2026-09-28 16:41 CEST) until this fix, the clause “PAN is enabled” held at boot, but not while the kernel handled EL0 exceptions, and not after EL0 trips until a copy helper ran. The boot probe missed this because it ran before any EL0 trip. EL0 itself could not exploit it directly (EL0 still could not reach kernel memory, and the syscall pointer checks of ADR-094 do not depend on PAN), but PAN’s purpose is to catch kernel bugs that dereference user pointers, and that net was down exactly when the kernel handles EL0 requests.
Decision
- After a successful ADR-080 enable, clear
SCTLR_EL1.SPAN(bit 23). Every exception taken to EL1 then sets PSTATE.PAN. return_from_el0setsSPSR.PAN(bit 22) when PAN is enabled, so the kernel resumes with PAN set after an EL0 trip.- Copy helpers are unchanged:
with_user_accessstill clears PAN around an intentional copy and restores it.
Probes (fail-closed in scripts/qemu-smoke.sh)
- Live, during an EL0 exception: called from the ADR-092 live walk in the
SYS_YIELDhandler of the loadedhello-libctosapp:pan: live-syscall pstate-pan=1 sctlr-span=0. An EL1 load cannot be the test here: once PAN is really set it takes a current-EL permission fault inside an exception handler, which the kernel treats as fatal by design. So the handler readsMRS PAN. - After EL0 trips: right after the loaded app exits:
pan: after-el0 trips=<n> mrs-pan=1 el1-load va=0x80003000 fault=yes spsr-pan=1(n ≥ 1 returns from EL0 so far; an EL0-readable page is mapped at the transientcrt-stack-panslot VA, loaded with the ADR-080 catcher armed silently, then unmapped). - Summary:
pan: ok held boot,live-syscall,after-el0, printed only if both checks passed. - Calibration of
MRS PAN: it reads 1 in the after-EL0 check, where the real EL1 load faults with SPSR.PAN set; it reads 0 in the negative build, where the load does not fault. (ADR-080 noted thatMRS PAN“may read 0” under QEMU 10.0.13 TCG; in this build, whenever a fault proved PAN set,MRS PANread 1.) #[test_case]pan_span_cleared_after_enable(SPAN = 0 andMRS PAN= 1 in kernel context undercargo test).- The smoke requires the three lines and rejects
pan: lost,pan: bad after-el0,pan: held missed,pan: held probe missed. Existingpan: id=/pan: present/pan: enabled/pan: el1-faultchecks are unchanged.
Negative tests
- Build: the existing
el0-leak-probesmoke build also enablespan-keep-leak-probe, which undoes both parts of the fix (SPAN stays 1;return_from_el0resumes with SPSR.PAN clear). The smoke requires, from that kernel:pan: enabledandpan: el1-fault(the boot probe still passes — the blind spot),pan: lost live-syscall pstate-pan=0 sctlr-span=1,pan: lost after-el0 trips=<n> mrs-pan=0 el1-load va=0x80003000 fault=no spsr-pan=0,pan: held missed live-taken=true live-ok=false after-el0=false; and fails ifpan: ok held,pan: live-syscall pstate-pan=1orpan: after-el0appears. - Tampered logs (the ADR-098 block against edited copies of a good box serial; each must fail):
pan: ok helddeleted; live line withpstate-pan=0 sctlr-span=1; after-EL0 line withfault=no spsr-pan=0;trips=0; live line deleted; apan: lostline appended; and the real negative-build serial. All seven failed with a named missing / forbidden line; the unedited log passed. The leak-build section passes on the negative-build serial and fails on the good serial.
ADR-091 clause check after this fix
Every clause of the accepted revision-3 sentence is literally true again on the default smoke machine, and now continuously for the PAN clause: PAN is enabled at boot, in EL0 exception handlers, and after EL0 trips; EL1 faults on EL0-readable memory (boot probe + after-EL0 check). The sentence is not edited. A dated correction note is added to ADR-091’s status list (the same place as its 2026-09-28 limits update), and a correction row to the honesty ledger.
Limits
- One core, QEMU TCG default smoke machine; not real hardware (no KVM re-run of this change), not side-channel, not a certification.
- In handlers the check is
MRS PAN(calibrated), not an EL1 load; the live check samples one exception type (SVCSYS_YIELD). IRQ/FIQ/abort entries get the same SPAN=0 behaviour architecturally but are not sampled separately. - The after-EL0 check samples one point (after the loaded app exits).
- Kernel paths that deliberately touch EL0 pages must keep using
with_user_access; with PAN now held, one that doesn’t would fault (and in a handler, park). The full smoke (all EL0 apps, FAT / net / TCP samples, A9 cross-update,cargo test) is green with the fix.
ADR-099 — Milestone item 2: the KVM re-check in CI via GitHub OIDC (design sketch, inert)
- Status: Proposed — design only; since 2026-10-13 also the exact spec for ONE supervised pilot run (sponsor decision 2026-10-13 20:49 CEST, DSO relay). Not recurring: manual dispatch only, no schedule. See Pilot spec.
- Original status line: Nothing in this ADR is implemented. No AWS call was made and no AWS resource was created or changed to write it. The workflow it adds,
.github/workflows/kvm-recheck.yml, is inert (workflow_dispatchonly, not scheduled, not a required check, fails closed while its role variable is unset). The sponsor and DSO decide; do not self-merge (ADR-002). - Date: 2026-10-02 (CEST).
- Follows: ADR-085 (first KVM run), ADR-090 (evidence class), ADR-091 (Accepted; its sentence is not touched), ADR-093 (no identifiers in public text), ADR-095 (G3: the manual run this automates).
- Not in this ADR: no
src/,user/,libctos/orCargo.*change;_start(0x4008_0000) untouched;smoke.yml,pages.ymland the in-treescripts/b2-metal-userdata.shuntouched; the three required checks are unchanged; CloudAgent not used. - No threat-model bump.
security.mdonmainwas v1.72 when this ADR was written (ADR-098, PR #151, merged by DSO on the sponsor’s instruction on 2026-10-02 13:01 CEST); #149 and #150 have merged since (v1.71, v1.73), and the next free number is taken by the ADR-091 revision 4 PR (#153). A bump here would collide with those numbers. The risk table below is written so DSO can attach it to the sponsor record; folding it intosecurity.mdis a follow-up after those PRs merge. - Public-identifier rule (ADR-093): every account, role, instance, image, subnet, security-group and zone identifier is a
<PLACEHOLDER>here and in the workflow. Real values belong in repository variables set by the sponsor-approved path, never in repo text, commits, PR bodies or logs.
Context
ADR-095 proved by hand (one c7g.metal Graviton3 Spot instance, about $0.04, reduced b2-serror profile, markers el0: serror + b2: taken) that the taken lower-EL SError still happens under KVM. That run is a point in time. Milestone item 2 asks whether the same check can run from CI so the claim can be re-checked on a later commit without a person at an AWS console.
GitHub-hosted runners cannot run KVM for this profile on Graviton metal, so CI would have to start a short-lived AWS instance. That needs AWS credentials in CI. The credential rule (rule 4) forbids stored keys. The sponsor approved one exception, tracked as PX0: a GitHub OIDC role with no stored keys (DSO relay, 2026-09-28 21:23 CEST). The exception is conditional. Before the role is created or used:
- (a) a CloudTrail trail is on;
- (b) the ctos branch protection on
mainis enforced for admins; - (c) how test results leave the instance is decided before asking for any extra permission such as
iam:PassRole.
Condition (c) is answered in this ADR (section 2). The conditions above are taken from the DSO relay text. The decision record in the DSO vault (01-second-brain/decisions/2026-09-28-ctos-ci-oidc-exception.md) was not read while writing this ADR: the file exists on the DSO machines, but the executor’s file reader refused a path outside its allowed root, and that refusal was not worked around. Nothing here quotes or paraphrases the record, and the text of credential rule 4 is not reproduced. DSO must reconcile this ADR with the record before it goes to the sponsor.
Observations made while writing (GitHub read-only, 2026-10-02 about 13:00 CEST; no AWS)
| Item | Observed | Meaning |
|---|---|---|
Branch protection on main, enforce for admins | On | Condition (b) is verified today. Re-verify at role-creation time. |
Required checks on main | The three smoke checks (ubuntu-24.04-arm, ubuntu-24.04, NMI pin) | The new workflow is not one of them and must not become one. |
Required approving reviews on main | None (only status checks are required); strict is off | A PR that passes the three checks can be merged by anyone with merge rights. In practice the sponsor merges (ADR-002), but the control is by convention, not by GitHub. Listed as a residual risk below. |
Force-push and branch deletion on main | Not allowed | Good. |
| Actions environments | Only copilot and github-pages. No KVM environment exists. | A workflow that names an environment that does not exist makes GitHub create it with no protection. The environment must be created (reviewers, main-only branch policy) before the role exists. |
| Repository variables | Only one build variable; no KVM_* variable | The workflow is inert: its guard fails on the unset role variable. |
OIDC sub claim customisation | The repository uses the default template with an immutable subject; the subject prefix carries the numeric owner and repository IDs | The sub GitHub emits is probably not the plain repo:<owner>/<repo>:… form that most AWS examples show. A trust policy written from an example would never match (fails closed, but also never works). The exact string must come from the documentation and the customisation setting at role-creation time, and be confirmed in the supervised pilot from the CloudTrail AssumeRoleWithWebIdentity event. Not verified here. |
| Repository visibility | Public | Actions logs are public. See “Identifier leakage” in the risk table. |
Decision (proposed)
Add an opt-in, manual, inert workflow that, once the sponsor path enables it, assumes a least-privilege OIDC role, launches one Spot c7g.metal instance, reads the result from the serial console only, and terminates the instance in an always() step, with three independent teardown layers and a hard cost cap. Only a result produced by a probe on the instance is ever reported as PASS.
1. Trust policy and permission policy (placeholders only)
1.1 Trust policy shape
StringEquals only. No StringLike, no wildcard. The audience is fixed, the subject names this repository and exactly one of: the main branch or one named environment. The environment form is recommended, because the environment can also require a human reviewer and a main-only branch policy.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "GitHubOidcOnlyThisRepoOnlyThisEnvironment",
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::<ACCOUNT_ID>:oidc-provider/token.actions.githubusercontent.com"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"token.actions.githubusercontent.com:aud": "sts.amazonaws.com",
"token.actions.githubusercontent.com:sub": "repo:artofdream@<OWNER_ID>/ctos@<REPO_ID>:environment:kvm-pilot"
}
}
}
]
}
<SUB_FOR_THIS_REPO_AND_ENVIRONMENT> is one exact string. Two shapes, by the claim template in force:
| Template in force | Environment form (recommended) | Branch form (alternative) |
|---|---|---|
| Classic | repo:<OWNER>/<REPO>:environment:<ENV_NAME> | repo:<OWNER>/<REPO>:ref:refs/heads/main |
| Immutable subject (what this repository reports) [unverified format] | repo:<OWNER>@<OWNER_ID>/<REPO>@<REPO_ID>:environment:<ENV_NAME> | repo:<OWNER>@<OWNER_ID>/<REPO>@<REPO_ID>:ref:refs/heads/main |
<OWNER_ID> and <REPO_ID> are GitHub’s numeric IDs, read from the repository settings by the person creating the role, and not written to the repo. Do not “fix” a mismatch by widening to StringLike or *. A mismatch is a fail-closed AccessDenied; fix the string.
Other properties: role maximum session 3600 s; the OIDC identity provider for token.actions.githubusercontent.com must already exist in the account (precondition P4); the audience is the default that aws-actions/configure-aws-credentials requests. Optional hardening: customise the subject or add a job_workflow_ref condition so only kvm-recheck.yml on main matches [verify the claim and the customisation in the AWS and GitHub docs before relying on it].
Why the environment form: a pull request from a fork, or from any branch, produces a different sub (a pull_request subject, or another ref) and does not match. A workflow_dispatch from main bound to the environment matches only after the environment’s protection rules pass.
1.2 Permission policy (least privilege)
Placeholders (region is eu-north-1, the one used in ADR-095; formerly <REGION>), <ACCOUNT_ID>, <AMI_ID> (pinned Ubuntu 24.04 arm64 image), <SUBNET_ID_1> … <SUBNET_ID_n> (one subnet per AZ), <SG_ID> (a security group pre-created by DSO: tagged, no ingress).
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "RunInstancesNewInstanceOnly",
"Effect": "Allow",
"Action": "ec2:RunInstances",
"Resource": "arn:aws:ec2:eu-north-1:<ACCOUNT_ID>:instance/*",
"Condition": {
"StringEquals": {
"ec2:InstanceType": "c7g.metal",
"ec2:InstanceMarketType": "spot",
"ec2:MetadataHttpTokens": "required",
"aws:RequestTag/project": "ctos",
"aws:RequestTag/component": "kvm-ci-pilot"
},
"NumericLessThanEquals": { "ec2:MetadataHttpPutResponseHopLimit": 1 },
"ForAllValues:StringEquals": { "aws:TagKeys": ["project", "component", "run"] }
}
},
{
"Sid": "RunInstancesNewChildResourcesMustBeTagged",
"Effect": "Allow",
"Action": "ec2:RunInstances",
"Resource": [
"arn:aws:ec2:eu-north-1:<ACCOUNT_ID>:volume/*",
"arn:aws:ec2:eu-north-1:<ACCOUNT_ID>:network-interface/*",
"arn:aws:ec2:eu-north-1:<ACCOUNT_ID>:spot-instances-request/*"
],
"Condition": {
"StringEquals": {
"aws:RequestTag/project": "ctos",
"aws:RequestTag/component": "kvm-ci-pilot"
}
}
},
{
"Sid": "RunInstancesPinnedInputsOnly",
"Effect": "Allow",
"Action": "ec2:RunInstances",
"Resource": [
"arn:aws:ec2:eu-north-1::image/<AMI_ID>",
"arn:aws:ec2:eu-north-1:<ACCOUNT_ID>:subnet/<SUBNET_ID_1>",
"arn:aws:ec2:eu-north-1:<ACCOUNT_ID>:subnet/<SUBNET_ID_n>"
]
},
{
"Sid": "RunInstancesTaggedNoIngressSecurityGroupOnly",
"Effect": "Allow",
"Action": "ec2:RunInstances",
"Resource": "arn:aws:ec2:eu-north-1:<ACCOUNT_ID>:security-group/<SG_ID>",
"Condition": { "StringEquals": { "ec2:ResourceTag/project": "ctos" } }
},
{
"Sid": "TagOnCreateOnly",
"Effect": "Allow",
"Action": "ec2:CreateTags",
"Resource": [
"arn:aws:ec2:eu-north-1:<ACCOUNT_ID>:instance/*",
"arn:aws:ec2:eu-north-1:<ACCOUNT_ID>:volume/*",
"arn:aws:ec2:eu-north-1:<ACCOUNT_ID>:network-interface/*",
"arn:aws:ec2:eu-north-1:<ACCOUNT_ID>:spot-instances-request/*"
],
"Condition": { "StringEquals": { "ec2:CreateAction": "RunInstances" } }
},
{
"Sid": "TerminateAndReadConsoleOfOurTaggedInstancesOnly",
"Effect": "Allow",
"Action": ["ec2:TerminateInstances", "ec2:GetConsoleOutput"],
"Resource": "arn:aws:ec2:eu-north-1:<ACCOUNT_ID>:instance/*",
"Condition": {
"StringEquals": {
"ec2:ResourceTag/project": "ctos",
"ec2:ResourceTag/component": "kvm-ci-pilot"
}
}
},
{
"Sid": "DescribeInstancesForPollAndSweep",
"Effect": "Allow",
"Action": "ec2:DescribeInstances",
"Resource": "*",
"Condition": { "StringEquals": { "aws:RequestedRegion": "eu-north-1" } }
},
{
"Sid": "DenyOtherRegions",
"Effect": "Deny",
"Action": "ec2:*",
"Resource": "*",
"Condition": { "StringNotEquals": { "aws:RequestedRegion": "eu-north-1" } }
},
{
"Sid": "DenyIamS3SsmExplicitly",
"Effect": "Deny",
"Action": ["iam:*", "s3:*", "ssm:*"],
"Resource": "*"
}
]
}
(The last Deny is belt and braces: nothing here allows those services, but an explicit Deny keeps a later, careless policy attachment from widening the role. It deliberately does not name kms:*: if account-default EBS encryption uses a customer-managed key, the launch may need KMS permissions, and that is a decision for the sponsor, not a hidden Deny.)
Exact actions to launch, tag-scope and terminate one Spot instance:
| Action | Needed for | Can the project=ctos tag restrict it? |
|---|---|---|
ec2:RunInstances | launch the one instance (with Spot market options, in one call) | Partly. On instance, volume, network-interface, spot-instances-request through aws:RequestTag/project (and component), i.e. the request must tag what it creates. On the security group through ec2:ResourceTag/project. Not on the image or the subnets (they are not tagged by us; pinned by ARN instead). |
ec2:CreateTags | tag at creation | Scoped, not by the tag value: ec2:CreateAction = RunInstances means tags can only be written during a launch, never onto an existing resource. |
ec2:TerminateInstances | teardown | Yes, ec2:ResourceTag/project + component on the instance. |
ec2:GetConsoleOutput | the result channel | Yes, same condition. |
ec2:DescribeInstances | poll state, Spot interruption reason, sweeper | No. Describe* has no resource-level scope; it is limited by region only. It leaks nothing the account’s EC2 console would not already show the role holder. |
iam:PassRole | an instance profile | Not requested. There is no instance profile. |
iam:CreateServiceLinkedRole | first-ever Spot use | Not requested. The Spot service-linked role must pre-exist (P8, unverified). |
ec2:CreateSecurityGroup, ec2:Authorize*, key pairs, SSM, S3, KMS | — | Not requested. The group is pre-created. |
ec2:DescribeImages, DescribeSubnets, DescribeSpotPriceHistory | — | Not requested. Image and subnets are pinned; MaxPrice is the cap. |
Service-reference check (public AWS documentation, fetched 2026-10-02, not an AWS API call): TerminateInstances and GetConsoleOutput act on the instance resource type, which supports ec2:ResourceTag/*; RunInstances touches the instance, volume, network-interface, spot-instances-request, security-group, subnet and image types; Describe* have no resource types. IAM condition behaviour is not exercised until a supervised pilot (P13); expect one or two adjustments, each of which goes back to the sponsor as a change to this ADR.
Unverified, flagged for the pilot: if the account’s default EBS encryption uses a customer-managed key, RunInstances may need extra KMS permissions; if so, that is a new permission request and goes to the sponsor under condition (c) logic, it is not added silently.
2. Results-out decision (condition (c))
The question: how does “the smoke passed” leave a box that has no credentials, no key pair and no inbound port?
| Option | What it needs | What it avoids / costs | Verdict |
|---|---|---|---|
A. Serial console, ec2:GetConsoleOutput | One tag-scoped Get on the instance (already in the policy). No instance profile. | Avoids iam:PassRole, any instance profile, any bucket, any agent, any credential on the instance. The instance cannot send anything except text to its own console. Cost: nothing. Limits: last 64 KiB, may lag, text only, boot noise on the console can carry the host name or addresses (so the workflow prints only ^CTOSB2: lines). | Recommended. It is exactly what ADR-085 and ADR-095 used. |
| B. SSM (Run Command / session output) | An instance profile with SSM core permissions, hence iam:PassRole, plus ssm:SendCommand, ssm:GetCommandInvocation, an agent on the box and an output path. | Adds a new principal on the instance, a new service, a bigger role. | Rejected. It buys a bigger channel than a one-line verdict needs. |
| C. S3 bucket (instance PUTs the result) | Either an instance profile (iam:PassRole) or a pre-signed PUT URL minted by the role (no PassRole, but s3:PutObject and a bucket). | A new resource class (bucket, lifecycle, public-block settings, cost), a URL that is a bearer secret on the instance and in user-data, a read path. | Not now. Kept as a fallback if console output proves unreliable in the pilot; that would be a new sponsor decision under (c). |
| D. Instance pushes to GitHub (status, artefact, comment, self-hosted runner) | A GitHub token (or runner registration token) on the instance. | Violates the credential rule’s intent: a long-lived or minted secret on a machine that downloads and builds third-party code. | Rejected. |
Sponsor accept (2026-10-02): the sponsor accepted this section, results out via tag-scoped ec2:GetConsoleOutput, no iam:PassRole, no instance profile, in writing on 2026-10-02. It is recorded in the DSO vault and was relayed by DSO. Only this section is accepted; the rest of the ADR is still a proposal.
Recommendation: A, console output. It needs the least (two tag-scoped EC2 read/terminate actions the role needs anyway), avoids the most (PassRole, instance profile, S3, SSM, any credential on the instance), and was already proven by hand twice.
How the workflow stays honest with a text-only channel:
- Result contract. The user-data prints
CTOSB2: result smoke-rc=<rc> sha=<commit> nonce=<random>last (rcstarts at 99 = “never ran”; a setup failure prints 98). The nonce is generated by the workflow and substituted into the user-data, so a stale console or another run’s text cannot be mistaken for this run. - PASS needs all of: the result line with
rc=0, the exact commit and nonce; an anchoredCTOSB2: b2-kvm-smoke: PASS taken lower-EL SError observed under accel=kvmline (anchored, so aset -xecho of the same words does not count);/dev/kvm=present;unamereportingaarch64; the KVM API-version line. Anything else is FAIL. No result line before the deadline is INCONCLUSIVE. - Live vs after termination (unverified). It is not verified here whether the console is readable live or only after the instance stops. The design handles both: the instance lingers about 90 s after printing the result; the workflow polls live, and if the instance is already terminated it makes a few more reads before giving up as INCONCLUSIVE.
- Local test of the parser (no AWS): the parse block was run against the real ADR-095 console excerpt plus synthetic result lines. PASS only with the right commit, nonce, smoke line and markers; wrong nonce, wrong commit, rc≠0, missing smoke line, KVM absent, an unanchored echo and a setup failure all give FAIL; no result gives INCONCLUSIVE. This tests the parser, not AWS.
- Public logs. The workflow prints only the filtered marker lines and AWS error codes; it never prints raw console text or raw AWS stderr. Instance IDs are masked with
::add-mask::as soon as they exist.
3. Fail-safe teardown and cost cap
Four layers, so no single failure leaves a metal instance running:
- Pre-flight refusal. If any
project=ctosinstance is active, the run stops withREFUSED-active-instancebefore spending anything. Concurrency groupkvm-recheck(cancel-in-progress: false) serialises runs. - On the instance:
shutdown -h +<LIFETIME_MIN>is the first line of the user-data (default 45 min; the guard refuses anything over 60), shutdown behaviour isterminate, and the script powers off after a 90 s linger. This works even if the runner dies. - Always-run terminate step.
if: always(), with fresh credentials (a secondconfigure-aws-credentials, guarded by the first one’s success, because the first session may have expired or been cancelled). It finds the run’s instances by tag (run=<run id>), so a launch that died afterRunInstancesbut before returning an ID is still cleaned up, terminates them, and waits forterminated. - Tag-based sweeper check (same step). After polling up to 20 min for
terminated, counts the instances this run launched (project=ctos,component=kvm-ci-pilot,run=<run id>). Any stillpending/running/stopping/stoppedgives LEAK; any stillshutting-downgives TEARDOWN-SLOW (red, not LEAK). Other activeproject=ctosinstances (for example the one aREFUSED-active-instancepre-flight found) only raise a warning; they never turn this run’s verdict into LEAK (that informational count cannot fail the step) (Bugbot finding, fixed 2026-10-13). This is a check inside the run, not a scheduled sweeper: a schedule would use the role with no human present and needs its own sponsor decision.
Job timeout-minutes is 60; the poll limit is 40 min.
Hard cost cap per run. Spot is billed at the market price, never above MaxPrice. The guard computes cap = MAX_PRICE_USD_HR × (LIFETIME_MIN + 5) / 60 and refuses to run if it exceeds RUN_COST_CAP_USD, or if the price is above the ceiling the sketch allows. Defaults: MaxPrice $0.40/h (the observed Spot price in ADR-095 was $0.2476/h, the manual run used a $0.60 ceiling), lifetime 45 min, cap $0.50; the computed worst case is about $0.33 per run [estimate]. EBS (30 GB gp3, minutes) adds under $0.01 [estimate]. A price above MaxPrice interrupts the instance, which ends the spend and gives INCONCLUSIVE, not PASS.
Typical cost [estimate]: $0.03–0.07 per run (ADR-095: about 8 minutes of metal at about $0.25/h, about $0.04; build time varies with Spot price and apt mirrors).
| Scenario | Runs / month | Monthly cost [estimate] |
|---|---|---|
| Manual, as needed | 4–8 | about $0.12–0.56 |
| Weekly schedule (not enabled, not proposed here) | about 4–5 | about $0.12–0.35 |
After every merge to main (not proposed) | about 20 | about $0.6–1.4 |
| Worst case: every run hits the full cap | n | about $0.34 × n |
CloudTrail cost for one management-events trail is small [unverified]. An AWS Budget alarm at a low monthly figure is recommended as a last backstop (P9, optional, unverified).
Residual: if the GitHub runner dies hard and the user-data failed to start, the instance runs until MaxPrice interrupts it or someone notices (up to about $0.40 per hour). The in-instance timer, the pre-flight refusal on the next run, and the budget alarm are the mitigations.
4. Spot capacity-failure story
KVM_SUBNET_IDS is a space-separated list, one subnet per AZ, tried in order. On each failed RunInstances the workflow reads only the AWS error code:
| Code class | Examples | Action |
|---|---|---|
| Capacity | InsufficientInstanceCapacity, SpotMaxPriceTooLow, capacity-not-available | try the next AZ |
| Quota / limit | MaxSpotInstanceCountExceeded, VcpuLimitExceeded | stop: SKIPPED-no-capacity (reason: quota) |
| Anything else (for example AccessDenied) | FAIL immediately; a permissions problem is not “no capacity” | |
| All AZs exhausted | SKIPPED-no-capacity |
A skip is never green. The job exits non-zero with a step summary that says “not a pass”. There is no on-demand fallback: a metal on-demand instance would break the cost cap. A mid-run Spot interruption (StateReason of Spot termination) is INCONCLUSIVE and is not retried automatically. The ledger records the last PASS commit and date; a skipped or inconclusive run never updates it.
Result words: PASS (the only exit 0), FAIL, INCONCLUSIVE, SKIPPED-no-capacity, REFUSED-active-instance, LEAK, TEARDOWN-SLOW (this run’s instance still shutting down when the teardown wait ended), ABORTED-after-aws-call (cancelled or crashed after the role was requested or an instance launched, with no result written; teardown still runs), NOT-RUN (guard refused or an earlier step failed; no AWS call was attempted).
5. Honesty: what a PASS proves
A PASS proves only this: at the printed commit on main, on one Graviton3 .metal host with /dev/kvm, under QEMU v10.0.0 + patches 0001/0002 with accel=kvm, the reduced b2-serror profile printed el0: serror and b2: taken (the lower-EL SError was taken).
It does not prove: the default image boots or behaves the same under KVM; the identity inventory, reach walk or syscall-pointer probes; anything about real hardware beyond this one profile; or that “EL0 isolated” is wider than the accepted ADR-091 sentence (that sentence is not changed here). It is also not a security claim. Status words in the summary and the ledger come from the probe’s result line only, never from the workflow’s own opinion. The code under test controls its own console output; that is inherent to the method, so every run records the commit it tested.
6. Precondition checklist (DSO and sponsor)
Nothing below is done by this PR. “State” is the state in this PR’s evidence; all items must be re-checked at the time they matter.
| # | Precondition | Owner | State (2026-10-02) |
|---|---|---|---|
| P1 | (a) A CloudTrail trail is on (management events, all regions or at least eu-north-1) | DSO | MET (verified by DSO from the arm64 workstation, 2026-10-13 about 20:56 CEST: a multi-region trail, home region eu-north-1; the agent made no AWS call) |
| P2 | (b) Branch protection on main enforced for admins | DSO / sponsor | Verified via the GitHub API. Note: no required approving review; strict is off. |
| P3 | (c) Results-out decided before any extra permission | Sponsor accepts this ADR | Decided in this ADR (console output). Section 2 was accepted by the sponsor in writing on 2026-10-02, recorded in the DSO vault (relayed by DSO; the verbatim wording is in the vault and is not reproduced here). No PassRole requested. |
| P4 | The GitHub OIDC identity provider exists in the account | DSO | Exists (DSO, 2026-10-13): already present and shared; managed by another project’s Terraform, not by ctos. |
| P5 | A GitHub environment (name kvm-pilot since 2026-10-13; was placeholder kvm-recheck) with required reviewers and a main-only deployment branch policy, created before the role | Sponsor / DSO | Created by DSO 2026-10-13 22:12-22:16 CEST (sponsor approval 22:11). Sponsor decision 22:15: sole required reviewer = the sponsor, prevent_self_review off, deployments from main only, admin bypass off. |
| P6 | The role created via a sponsor-approved path with exactly the trust and permission policies above, adjusted for the immutable subject | Sponsor / DSO | Created by DSO 2026-10-13 22:12-22:16 CEST on the sponsor’s 22:11 approval (supersedes the earlier “not by an agent” wording). Tagged project=ctos, component=kvm-ci-pilot; inline policy follows this ADR incl. the hop-limit fix (NumericLessThanEquals). Not verified by this agent (no AWS call). |
| P7 | Pinned inputs set as repository variables (never repo text): image, no-ingress tagged security group, one subnet per AZ, region, role ARN | DSO | Not done (all unset) |
| P8 | Spot service-linked role and Spot vCPU quota for one c7g.metal | DSO | NOT VERIFIED |
| P9 | AWS Budget alarm (optional backstop) | DSO | Set by DSO 2026-10-13: budget ctos-monthly-5, USD 5/month, filtered to project=ctos, alerts at 80% actual and 100% forecast. |
| P10 | Credential rule 4 and the PX0 exception record referenced in the PR that enables the workflow | DSO | Record not read for this ADR; rule text not reproduced here |
| P11 | KVM_ROLE_ARN is set last | DSO | Unset, so the workflow is inert (verified) |
| P12 | Third-party actions pinned by full commit SHA | Author, before enabling | Not done (tags in the sketch) |
| P13 | A supervised pilot run, then DSO reviews CloudTrail (who assumed what, which calls, which denies) | DSO + sponsor | Not done |
| P14 | The exact immutable-subject sub string confirmed | DSO | NOT VERIFIED |
| P15 | Public-log hygiene confirmed on the pilot (no identifier in the log or summary) | DSO | Not done |
7. Risk and impact evaluation (for the sponsor record)
Impact is “what an attacker or a mistake could cost”. Residual is after the listed mitigations, assuming P1–P15 are met.
| # | Threat | Likelihood × impact (before) | Mitigations | Residual risk |
|---|---|---|---|---|
| 1 | Another repo, branch, fork or pull request assumes the role | Low × High | Exact aud and sub (no wildcard); environment with required reviewers and main-only branch policy; fork PRs get a different sub and no id-token for secrets | Low. Depends on the exact sub being right (P14) and the environment existing first (P5). |
| 2 | A PR edits the workflow to abuse the role | Medium × High | Dispatch runs only the default-branch file; guard refuses any ref but main; environment gate; enforce_admins is on | Low–Medium. main requires no approving review (observed), so the control is the sponsor merging by convention. DSO may ask the sponsor to require one review on main or a code-owner rule for .github/workflows/. |
| 3 | Role is over-permissioned (miner, wrong instance type, other region) | Medium × High | Conditions on instance type, Spot market, IMDSv2, tags and region; pinned image/subnets/SG; 1 h session; no IAM, no PassRole; Spot quota; budget alarm | Low. IAM conditions are untested until the pilot (P13). |
| 4 | Tag spoofing (a resource tagged project=ctos to dodge the sweeper, or the terminate scope abused) | Low × Low | Tags only at creation (CreateAction); terminate only matches tags the role itself applied; the sweeper counts the tag broadly | Low. A person with other credentials could still tag things; out of scope for this role. |
| 5 | Privilege escalation through the instance | Low × High | No instance profile, no iam:PassRole, explicit Deny on iam:*; IMDSv2 required, hop limit 1; no credentials on the box | Low. |
| 6 | Secret in user-data | Low × Medium | User-data holds only a commit, a nonce and a timer; no keys, tokens or URLs with secrets | Very low. |
| 7 | Identifier leakage in public logs (instance ID, account ID, host name, IP) | Medium × Low–Medium | Account-ID masking; instance IDs masked at creation; only ^CTOSB2: lines and error codes are printed; no raw console or stderr; identifier guard on repo text | Low. Confirm on the pilot (P15). |
| 8 | Leaked running instance (cost, exposure) | Medium × Low | Pre-flight refusal; in-instance shutdown -h +45 and terminate-on-shutdown; always-run terminate with fresh credentials, by tag; wait for terminated; sweeper gives LEAK; budget alarm | Low. Worst case about $0.40/h until noticed if the runner and the user-data both fail. |
| 9 | Console text is forged or replayed | Low × Medium | Per-run nonce and commit in the result line; anchored PASS line; KVM and arch markers; terminate-by-tag so one run’s console is read once | Low. The code under test can print its own PASS (inherent; the commit is recorded and the tested commit must be on main). |
| 10 | Silent green on a skip or timeout | Medium × High (trust) | Only PASS exits 0; SKIPPED, INCONCLUSIVE, REFUSED, LEAK, TEARDOWN-SLOW, ABORTED-after-aws-call, NOT-RUN are red; guard fails (not skips) on an unset variable | Low. |
| 11 | Spot capacity or interruption | High × Low | Multi-AZ retry; honest SKIPPED / INCONCLUSIVE; no on-demand fallback | Accepted: the check is sometimes unavailable. |
| 12 | Third-party action supply chain (configure-aws-credentials, checkout) | Low × High | Pin by full commit SHA before enabling (P12); permissions: {} at top level; the job only gets id-token: write and contents: read | Low–Medium until pinned. |
| 13 | Supply chain on the instance (apt, the rustup install script, git clone, QEMU source) | Medium × Low | QEMU is built from the pinned v10.0.0 source plus the repo’s patches; the repo is cloned at a commit already on main; the instance holds no credentials, has no ingress, IMDSv2 hop limit 1, and lives under an hour | Low. Open egress remains (needed for apt/rustup/git). |
| 14 | Repo rename or transfer (subject reuse by a new owner) | Low × High | The immutable subject (owner and repo IDs) is what the trust policy pins | Low once P14 is confirmed. |
| 15 | Cost runaway | Low × Low | Cost-cap guard; MaxPrice; lifetime; concurrency 1; budget alarm | Low; about $0.33 worst case per run [estimate]. |
| 16 | CloudTrail off, so no audit trail | Unknown × High | Condition (a) is a hard precondition before the role is created or used | P1 met 2026-10-13 (multi-region trail, DSO-verified). |
| 17 | Unprotected environment auto-created by the workflow’s first reference | Medium × High | P5: create the environment with protections first; order matters | Low if the order is followed. |
| 18 | Unknown account facts (Spot service-linked role, quota, EBS default key) | Medium × Low | P8; pilot; any new permission goes to the sponsor, not added silently | Low. |
| 19 | Credential rule 4 is only relaxed for this role | Low × Medium | One role, one repository, one environment, no stored key, 1 h tokens; the exception record is referenced in the enabling PR (P10) | Low. Any second role or a schedule needs its own approval. |
8. The inert workflow
.github/workflows/kvm-recheck.yml and scripts/kvm-recheck-userdata.sh are the sketch.
- Triggers:
workflow_dispatchonly (one optional input, the commit to test, which must be onmain). Noschedule. It is not, and must not become, a required check. - Fails closed: the first step makes no AWS call and fails (red, summary “NOT-RUN”) unless the repository, the ref (
refs/heads/main), the role variable (and image, group, subnets, region variables), the role ARN shape and the cost arithmetic are all in order. WhileKVM_ROLE_ARNis unset, it cannot do anything else. - Permissions:
permissions: {}at the top; the job getsid-token: writeandcontents: readonly. - Pinned inputs come from variables, not repo text.
- Pre-run checks done locally (no AWS): YAML parse,
actionlintwithshellcheck,bash -nandshellcheckon the user-data, the guard run with simulated variables (unset, wrong ref, wrong repo, bad ARN, over-cap, hostile input), and the console parser run on the real ADR-095 excerpt (see section 2). - Sketch status: the workflow has never run; the
ubuntu-24.04runner’s AWS CLI behaviour, theaws-actionsinputs, and every AWS call are untested.
User-data is derived from the ADR-095 variant of scripts/b2-metal-userdata.sh: same QEMU build, same cargo build --features b2-serror, same scripts/b2-kvm-smoke.py with CTOS_B2_TIMEOUT=90 and CTOS_B2_STALL=15, up to three attempts; plus the timer first, the uname -srm (no host name), KVM API line, CTOSB2: result line with commit and nonce, and the linger-then-poweroff.
Consequences
- If accepted and later enabled: a re-check of the reduced profile can run on demand without a stored AWS key, with the role limited to one region, one instance type, one pinned image, tagged resources and no IAM, at roughly $0.03–0.07 per run [estimate].
- Now: nothing changes. The workflow is a file that fails closed. The honesty ledger gets a Planned row (documented sketch, not implemented), not Implemented.
- Cost of the choice: the console channel is narrow and text only; Spot capacity makes the check sometimes unavailable; the immutable-subject format and several account facts have to be verified by DSO before the first run.
- Not decided here: a schedule, a per-merge run, making it a required check, an S3 or SSM result channel, an on-demand fallback, a standalone sweeper. Each needs its own sponsor decision.
Pilot spec (one-off, supervised; 2026-10-13)
Sponsor decision 2026-10-13 20:49 CEST (DSO relay): prepare this PR as the exact spec for one supervised pilot run. Not recurring: the workflow stays workflow_dispatch only, with no schedule, and it is not a required check. The PR stays draft. Nothing here creates an AWS or GitHub resource.
Order of operations (DSO / sponsor; the agent does none of these):
- CloudTrail on (P1): met, verified by DSO from the arm64 workstation, 2026-10-13 about 20:56 CEST: a multi-region trail, home region
eu-north-1. - Create the GitHub environment
kvm-pilot(below), before the role. - Create the OIDC provider (if absent) and the role through the sponsor-approved path, with exactly the trust policy and permission policy in section 1 and role MaxSessionDuration = 3600 s.
- Set the repository variables
KVM_AMI_ID,KVM_SG_ID(pre-created, taggedproject=ctos, no ingress),KVM_SUBNET_IDS, and lastKVM_ROLE_ARN. - One supervised dispatch from
mainwithpilot_ack=supervised-one-off-pilot; a reviewer approves the environment deployment; DSO watches the run. - DSO reviews CloudTrail for the session, then unsets
KVM_ROLE_ARN(back to inert) unless the sponsor decides otherwise.
Exact sub (environment-scoped, immutable-subject form): repo:artofdream@<OWNER_ID>/ctos@<REPO_ID>:environment:kvm-pilot. The repository reports use_default: true, use_immutable_subject: true (read-only GitHub API, 2026-10-13); its subject prefix carries the numeric owner and repository IDs. <OWNER_ID> and <REPO_ID> are those numeric IDs: values supplied to DSO out of band, not written here. StringEquals, no wildcard.
Environment kvm-pilot (GitHub setting, $0; to be created by DSO / the sponsor):
| Setting | Value |
|---|---|
| Name | kvm-pilot (the workflow’s environment: and the sub must match exactly) |
| Required reviewers | the sponsor (and/or DSO’s account); at least one; “Prevent self-review” on |
| Wait timer | 0 |
| Deployment branches and tags | Selected branches and tags, one rule: branch main (no tags) |
| Allow administrators to bypass | off |
| Environment secrets / variables | none (no stored keys; repository variables above are enough) |
Pinned actions (full commit SHA, version in the comment; looked up from the release tags 2026-10-13):
| Action | Tag | Commit SHA |
|---|---|---|
actions/checkout | v7.0.1 | 3d3c42e5aac5ba805825da76410c181273ba90b1 |
aws-actions/configure-aws-credentials | v6.3.0 | e1253824e5c10ff9df46874f81ed3ec929e19cfd |
Region: eu-north-1 (hard-coded in the workflow and in the policies; not an identifier value).
Tags: project=ctos, component=kvm-ci-pilot, run=<run id> on the instance, volume, network interface and Spot request (RunInstances request-tag conditions); TerminateInstances and GetConsoleOutput require ec2:ResourceTag/project=ctos and ec2:ResourceTag/component=kvm-ci-pilot; CreateTags only with ec2:CreateAction=RunInstances.
Limits enforced by IAM condition keys: ec2:InstanceType=c7g.metal, ec2:InstanceMarketType=spot, ec2:MetadataHttpTokens=required, ec2:MetadataHttpPutResponseHopLimit ≤ 1 (NumericLessThanEquals), region via aws:RequestedRegion. Not enforceable by IAM: the Spot price. EC2 has no condition key for the Spot max price (AWS service reference, checked 2026-10-13), so the price cap is the workflow’s MaxPrice (MAX_PRICE_USD_HR, guard refuses above $0.60/h and above the per-run cap), plus the in-instance timer, plus an AWS Budget alarm (P9). Session: role MaxSessionDuration 3600 s (a role property, not a policy condition) and role-duration-seconds: 3600 in the workflow.
Results out: tag-scoped ec2:GetConsoleOutput only; no iam:PassRole, no instance profile (sponsor accepted 2026-10-02, recorded in the DSO vault).
CloudTrail (P1): met (verified by DSO from the arm64 workstation, 2026-10-13 about 20:56 CEST: a multi-region trail, home region eu-north-1). Still unverified: OIDC provider, Spot service-linked role and quota, EBS default key, live-vs-terminated console readability, IAM condition behaviour (first exercised in the pilot).
Setup recorded 2026-10-13 (DSO, sponsor-approved; docs only)
Relayed by DSO; this agent made no AWS call and set no repo variable. Text only, no identifier values.
- Who/when: on the sponsor’s approval at 22:11 CEST, DSO created the
kvm-pilotenvironment and the OIDC role between 22:12 and 22:16 CEST on 2026-10-13. This supersedes P6’s earlier “not by an agent” line. - Role: tagged
project=ctosandcomponent=kvm-ci-pilot; inline policy follows this ADR, including the hop-limit fix. - OIDC provider: already existed; shared, managed by another project’s Terraform.
- Image: Ubuntu 24.04 arm64 AMI dated 20261004 - not the image ADR-095 used, so a pilot result is not a byte-identical re-run of ADR-095.
- Security group: no ingress.
- Environment protection (sponsor decision 22:15): sole reviewer is the sponsor,
prevent_self_reviewoff, deployments frommainonly, admin bypass off. - Budget:
ctos-monthly-5, USD 5/month, filtered toproject=ctos, alerts at 80% actual and 100% forecast. - Still open: P7/P11 repository variables unset (workflow inert), P8, P10, P12-P15.
Pilot run 2026-10-13 (relayed by DSO)
- The sponsor set the repository variables and dispatched the one supervised pilot, workflow run 38468490030, at
main069fe83. - Smoke: PASS under
accel=kvmonc7g.metal(el0: serror+b2: taken; run nonce685780b675853a95). - The run’s final verdict was red, from a false LEAK: teardown’s 5-minute wait for
terminatedtimed out while the metal instance was still shutting down, and the sweep counted it. DSO verified the instance reachedterminated. Fixed by the slow-termination change below (item 4). RunInstancessucceeded with the policy as written (no snapshot resource).- Relayed facts; this agent made no AWS call. No instance or account identifiers here.
Review fixes 2026-10-13 (Cursor Bugbot findings on PR #152, plus pilot findings)
- High (Bugbot) - snapshot resource for RunInstances: kept out. AWS docs list
snapshotas aRunInstancesresource type (Service Authorization Reference; account-less ARN form), but the live pilot run launched without it (the AMI is public and its snapshot is not in this account). The policy is unchanged. Add a snapshot resource only if a concreteUnauthorizedOperationdenial on a snapshot appears. - Medium - a pre-flight refusal was reported as LEAK. The sweeper counted every active
project=ctosinstance, so the instance that triggeredREFUSED-active-instancemade teardown fail and the verdict became LEAK. The sweeper now counts only instances this run launched (tagrun=<run id>); others only warn. - Slow metal termination (pilot finding). The pilot’s teardown waited only 5 minutes for
terminated; ac7g.metalwas still shutting down, and the sweep reported a false LEAK. Teardown now polls up toTERM_WAIT_MIN(20 min; job timeout 75 min). At the end: a run-tagged instance stillpending/running/stopping/stoppedis LEAK; one stillshutting-downis TEARDOWN-SLOW (red, not a pass, not LEAK; the smoke result stays in the summary). A later run’s pre-flight still refuses on any instance that never terminates. - Subnet list parsing.
KVM_SUBNET_IDSmay be separated by spaces, commas, or both; an empty list after parsing fails the launch step. - Low - NOT-RUN after an AWS call. The verdict defaulted to NOT-RUN whenever no result file existed, including a cancel or crash after assume-role or launch. New word ABORTED-after-aws-call: used when the credential step ran (success, failure or cancelled) and no result was written. NOT-RUN now means no AWS call was attempted. Teardown is unchanged (
if: always()after credentials succeeded).
No identifier values. No AWS call, no repository variable set, no dispatch by the agent.
Open reconciliation points (for the enabling PR)
- CloudTrail (P1, condition (a)): met — verified by DSO from the arm64 workstation, 2026-10-13 about 20:56 CEST: a multi-region trail, home region
eu-north-1.
DSO’s reconciliation notes, 2026-10-02. They are open points for the PR that would enable this workflow, not done here. This sketch does not close any of them.
- Exact immutable-subject string (P14): stated 2026-10-13 in the Pilot spec, with the numeric IDs supplied to DSO out of band; still to be confirmed against the pilot’s CloudTrail event. This repository’s
subclaim uses the immutable-subject form (numeric owner and repository IDs), so the trust policy’ssubmust be copied from what GitHub actually emits, then confirmed in the supervised pilot from the CloudTrail event. Never widened to a wildcard. - The protected environment must be created first (P5): required reviewers and a main-only deployment branch policy, before the role and its trust policy exist. Today no such environment exists.
- Actions must be SHA-pinned (P12): done 2026-10-13 (see Pilot spec). Re-check against the tags before the run.
- Region:
eu-north-1. Since 2026-10-13 the policies and the workflow state it directly; the region iseu-north-1, the one the G3 re-check (ADR-095) used. The region name is not an identifier value. AZ, subnet, image and security-group values stay in repository variables. - Results-out (section 2) accepted. The sponsor accepted section 2 in writing on 2026-10-02 (results out via tag-scoped
ec2:GetConsoleOutput, noiam:PassRole, no instance profile), recorded in the DSO vault, relayed by DSO. This satisfies condition (c) for that section only. Anything beyond it (S3, SSM, a PassRole) needs a new decision. - Still open from the checklist and not touched by DSO’s notes: the OIDC provider (P4), the Spot service-linked role and quota (P8), the DSO vault decision record (P10; not read here), the supervised pilot (P13).
Merge notes
This PR is draft and must stay a draft until DSO and the sponsor decide. It was cut from main at 3c47c0c, then merged with main (a normal merge, no force-push) to stay mergeable after #149 and #150 merged. #149, #150 and #151 were merged by DSO on the sponsor’s instruction. The ADR-091 revision 4 PR (#153) also appends to docs/SUMMARY.md, research/moc.md and docs/framework/honesty-ledger.md; if it merges first, expect trivial append-only conflicts there: keep both sides.
Reproducibility
# Syntax and lint (no AWS)
python3 -c "import yaml;yaml.safe_load(open('.github/workflows/kvm-recheck.yml'))"
actionlint .github/workflows/kvm-recheck.yml # needs shellcheck on PATH
bash -n scripts/kvm-recheck-userdata.sh
sh scripts/check-public-ids.sh --self-test && sh scripts/check-public-ids.sh # hits=0
ADR-100 — The SMP race story: plan, global-state inventory, and S1 secondary bring-up smoke
- Status: Proposed (plan) + S1 implemented as an opt-in smoke in the same PR. The plan does not change any claim.
- Date: 2026-10-13
- Decision path: milestone item 5 (“the SMP race story”), sponsor approval 2026-10-13, relayed by DSO. $0 work: one local machine plus the existing GitHub Actions checks.
- Base:
mainataf5a9d1(ADR-091 rev 4 recorded, PR #153). _start(0x4008_0000) untouched. The secondary entry is a separate symbol,_secondary_entry, in the same boot-stub page.
Context
ctos runs on one core. The accepted claim, ADR-091 revision 4, scopes “EL0 isolated” to -cpu cortex-a76, TCG, one core. That sentence does not change in this ADR or in any PR of this story. A multi-core claim of any kind needs a new, separate written sponsor accept.
Every piece of kernel code was written on the one-core assumption: “nothing else runs while I hold this”. On two cores that is false. This ADR writes down, before any code relies on it, where the kernel keeps global mutable state, what each piece assumes, what goes wrong on a second core, and the order in which the gaps get closed, each with a probe that fails closed.
Inventory of global mutable kernel state (main af5a9d1)
Method: rg for static / static mut items under src/ (excluding the new src/smp.rs), then reading each owner module. 262 static items:
| Kind | Count | Where |
|---|---|---|
static mut (no lock at all) | 25 | paging.rs 19 (page-table storage L1/L2/L3/L2_RAM/L3_KERNEL[], *_USER, *_ASID_B, *_HIGH, *_HIGH_RAM, and three “used” bump counters); virtio.rs 3 (DMA, NET_RX, NET_TX); el0.rs, ro.rs, teardown.rs 1 each (bait / magic words) |
spin::Mutex-wrapped | 10 | UART, heap ALLOCATOR, frame ALLOC, SCHED, VFS FS + LAST_READ, FAT, virtio DEV + NET, and paging::TABLES: Mutex<()> (a convention lock: 44 TABLES.lock() sites guard the static mut tables, but the type system does not force it) |
| Atomic flags / counters | 208 | mostly probe bookkeeping (EXPECT_* / *_CAUGHT in exception.rs 61, paging.rs, syscall.rs, el0.rs, pan.rs, sched.rs, timer.rs) |
| Immutable / linker symbols | 19 | __stack_top etc., embedded ELF, magic constants |
Plus per-CPU hardware state that the kernel treats as global: one boot stack (__stack_top), one exception stack, one fatal stack (linker symbols), VBAR_EL1, TTBR0/1_EL1, TCR, SCTLR (incl. SPAN), PSTATE.PAN, TPIDR_EL1 (holds the saved kernel TTBR0 for the EL0 trampoline), DAIF, the generic timer, the GIC CPU interface.
What each piece assumes and how it races on two cores
| # | State | One-core assumption | Race on two cores | Risk |
|---|---|---|---|---|
| R1 | Page-table storage + bump counters (static mut, paging.rs) | TABLES is taken by convention (44 sites); not audited that every writer (early boot, tears, relocations, leak-probe plants) holds it; L3_*_USED = n + 1 is a plain read-modify-write | Any writer outside TABLES races a builder on the other core: same L3 table handed out twice, lost or crossed mappings, an EL0-visible leaf in the wrong root | High (isolation) |
| R2 | TLB maintenance (tlbi vmalle1, tlbi vaae1, no IS suffix anywhere; switch_ttbr0_no_tlbi) | Invalidating the local TLB is enough | A torn / unmapped page (identity tears, unmap, ASID-B private page) stays cached in the other core’s TLB; EL0 or EL1 on that core still reaches it | High (isolation; every “torn” claim) |
| R3 | ASIDs are fixed constants (1 = user root, 2 = ASID-B), no allocator, no generation | Only one task holds an ASID at a time on one TLB | Two cores running different roots under one ASID share TLB tags (stale translations across tasks); ADR-096 t2t “no-tlbi” argument holds per core only | High |
| R4 | TPIDR_EL1 = saved kernel TTBR0; one exception / fatal stack | One CPU, one nesting chain | Each core needs its own; sharing one exception stack is memory corruption on the first simultaneous exception | High (correctness) |
| R5 | el0.rs ACTIVE / USER_PC / USER_SP / USER_TTBR / KIND; exception.rs EXPECT_* / *_CAUGHT | “The” EL0 context and “the” expected fault | A fault on core B is accepted as core A’s expected probe fault (false pass) or the restore path resumes the wrong context | High (probe integrity) |
| R6 | SCHED (spin::Mutex) + TASK_A_SP / TASK_B_SP; unlock before context_switch | Lock is only against re-entry; one runner | Both cores pick and run the same task on one stack | High |
| R7 | Heap ALLOCATOR, frame ALLOC (spin::Mutex) | No IRQ path allocates; contention never happens | Mutual exclusion holds across cores (spin locks are SMP-safe), but not IRQ-safe: a lock taken in IRQ context on the same core deadlocks; frame free/alloc vs. page-table use needs TLB ordering (R2) | Medium |
| R8 | virtio DMA / NET_RX / NET_TX (static mut) and the descriptor rings | Single driver user; polled completion | Two cores post to one ring; double-used descriptors, corrupted DMA buffers | Medium (no isolation claim depends on it) |
| R9 | VFS FS, FAT, LAST_READ (spin::Mutex) | Whole-op serialisation by one caller | Lock-correct per call, but multi-call sequences (open → read → close, mkdir) are not atomic | Low/Medium |
| R10 | UART (spin::Mutex) + write_str_raw / raw() (no lock, used by fatal/exception paths and probes) | Raw writes never interleave | Interleaved bytes from two cores break line-based markers; a fatal path on one core while the other holds UART | Low for safety, High for evidence (smoke grep) |
| R11 | GIC distributor / timer init, timer tick counters | One CPU interface, one timer | Secondary needs its own GICC + timer init; SPI routing; tick counts are per-core | Medium |
| R12 | PSTATE.PAN, SCTLR.SPAN, TCR, VBAR | Set once at boot on the one core | A secondary that does not repeat ADR-080/098 setup runs with PAN off / wrong vectors | High (the ADR-091 PAN clause) |
Riskiest: R2 (local-only TLBI), R1 (unlocked page-table builders), R3 (fixed ASIDs), R5 (global “expected fault” flags), R12 (per-core PAN/SPAN/VBAR setup). Any of these alone would make a multi-core “EL0 isolated” claim false.
Plan
Locking / ownership
- Page tables (R1): make
TABLESthe only way in: move the tables and bump counters into theMutexpayload (type-enforced), mask IRQs while held, audit every currentstatic mutwriter. No lock-free page-table writes. - TLB (R2): every unmap, permission downgrade or tear that may be cached on another core uses the Inner-Shareable forms (
tlbi vae1is/vaae1is/vmalle1is/aside1is) followed bydsb ish; isb. Local forms stay only where the mapping was never visible to another core (documented per site). No IPI-based shootdown is needed for the ARMv8 broadcast case on QEMUvirt; an IPI path is only planned if a probe shows broadcast is insufficient. - ASIDs (R3): a small allocator with a generation counter; on roll-over,
tlbi vmalle1isand bump generation; each core checks its current task’s generation on switch. Until then: secondaries never run EL0 (S1–S3). - Per-CPU (R4, R5, R11, R12): a
PerCpublock per core (stacks, saved TTBR0, EL0 context, expected-fault flags, timer state), pointed to byTPIDR_EL1; the trampoline’s current use ofTPIDR_EL1moves into that block. Probe “expect/caught” flags move to per-CPU. - Scheduler (R6): one run-queue lock, task state
Running(cpu); a task is never picked while running elsewhere. - Heap / frames (R7): keep spin mutexes; mask IRQs while held; forbid allocation in IRQ handlers (documented check).
- Drivers / FS (R8, R9): stay owned by CPU 0 (other cores reach them only via a syscall that is serviced on CPU 0) until a probe needs more.
- UART (R10): one console lock that also covers
raw()writes in non-fatal paths; per-line atomic writes; fatal path may break the lock (documented).
Secondary bring-up
PSCI CPU_ON (function 0xC400_0003) via HVC #0 — QEMU virt with -kernel and no EL2/EL3 firmware exposes PSCI on the HVC conduit, with secondaries powered off. Entry is a physical address; the secondary starts at EL1 MMU off. Later slices give it its own stack, VBAR, TCR/MAIR, TTBR1 (shared kernel high map), SCTLR (incl. SPAN=0), PAN set, GICC + timer, then a park/run loop.
Isolation probes that must be re-proved per core (S4)
On each core: ADR-080/098 PAN enabled + held; ADR-013/039 EL0 cannot read/execute kernel; ADR-092 EL0-reach walk (shared tables, but walk from each core’s live TTBR0); ADR-094 syscall pointer checks; ADR-096 task-to-task; ADR-097 execute-only text; ADR-087/088 identity inventory (each core’s TTBR0); plus a new cross-core TLB probe: unmap on core 0, access on core 1 must fault (negative build: local-only TLBI must be caught).
PR sequence (each with a fail-closed acceptance probe)
| Slice | Content | Acceptance probe (fail-closed) | Negative test |
|---|---|---|---|
| S1 (this PR) | Plan + opt-in smp2-probe: CPU_ON one secondary, MMU off, print MPIDR, store it to a flag, park | smp: secondary up mpidr=0x0000000080000001 and smp: ok cpu1 … parked under -smp 2; default smoke must show no smp: line | Same ELF under -smp 1 → rc=-2, smp: cpu1 missed; checker must fail |
| S2 | Secondary MMU on: shared TTBR1, own stack/VBAR/SPAN=0/PAN; PerCpu via TPIDR_EL1; console lock | smp: cpu1 mmu-on pan=1 span=0 vbar=high | build that skips SPAN clear on cpu1 must print smp: cpu1 pan lost |
| S3 | Type-enforced TABLES + Inner-Shareable TLBI everywhere + cross-core unmap probe | smp: xtlb unmap cpu0 → cpu1 fault | local-tlbi-leak-probe build (old local TLBI) must report smp: xtlb leak |
| S4 | ASID allocator with generation; per-CPU EL0 context + expect flags; re-run each isolation probe on cpu1 (still kernel-driven, one EL0 task per core) | each probe’s existing … ok marker tagged cpu=1 | existing leak builds run on cpu1 must still be caught |
| S5 | Scheduler run-queue lock, EL0 tasks on both cores; docs-only proposal of a multi-core sentence for the sponsor | all of the above in one boot, under -smp 2 CI job | — |
The -smp 2 CI job stays non-required until the sponsor accepts a multi-core claim.
What will NOT be claimed
- Not “SMP support”, not “multi-core safe”, not “EL0 isolated on multiple cores”. ADR-091 rev 4 stays one core.
- S1 does not run EL0, the scheduler, the heap, page-table code, drivers or interrupts on the secondary. The secondary runs MMU off, prints, writes one word, parks.
- No claim about real hardware, KVM, other
-cpumodels, more than two cores, or PSCI firmware other than QEMU’s built-in HVC conduit. - No memory-ordering / data-race freedom claim for any existing module. The inventory is by code reading; it may miss state reached through raw pointers.
- No performance or scaling claim.
S1 decision (this PR)
- Cargo feature
smp2-probe(opt-in, default off). After the identity tears and before GIC/virtio (same spot asb2-serror), the primary printssmp: cpu0 mpidr=…, cleans the flag line, calls PSCICPU_ON(target=1, entry=PA(_secondary_entry), context=PA(flag)), waits up to 0.5 s polling the flag (withdc civac), then printssmp: cpu_on target=0x1 rc=…andsmp: ok cpu1 mpidr=… parkedorsmp: cpu1 missed seen=…, then parks. _secondary_entry(in.text.boot, after_start, PA0x4008_0048on this build): MMU off, raw PL011 writes ofsmp: secondary up mpidr=0x<16 hex>, store MPIDR to the context PA,dsb sy; sev,wfeloop. No stack, no Rust.scripts/qemu-smoke.sh:smp2_check+smp2_smoke.CTOS_SMP2_SMOKE=1adds it after the default run;CTOS_SMP2_ONLY=1runs only the id guard + this. Positive under-smp 2, negative under-smp 1(checker must fail andsmp: cpu1 missed seen=0x0must appear). The default run gains a check that nosmp:line appears.- CI: a new non-required job
QEMU -smp 2 bring-up (opt-in, ADR-100; not required)runsCTOS_SMP2_ONLY=1. The three required checks are unchanged.
Consequences
- Default kernel binary: no code from
src/smp.rs(module is#[cfg(feature = "smp2-probe")]). The default smoke proves it still passes and has nosmp:marker. _secondary_entryexists only in thesmp2-probebuild (it lives insrc/smp.rs), so the default boot-stub page is byte-identical in layout; the ADR-087 inventory on the default run still reports the one stub page (k=1 u=1 a=1).- Next: S2. Done as ADR-102 (opt-in; PerCpu via
TPIDR_EL1moved to S4). S3: ADR-103. S4: ADR-104. S5: ADR-105. Claim sentence (proposed): ADR-106.
ADR-101 — Forward-only scrub of host and user names + guard classes
- Status: Accepted on merge (security / compliance guardrail, sponsor decision 2026-10-13 20:49 CEST, relayed by DSO). Not a kernel change.
- Date: 2026-10-13
- Follows: ADR-093 (cloud identifiers +
scripts/check-public-ids.sh). ADR-099 is taken by the open draft PR #152; this ADR takes 100. - No history rewrite (sponsor: forward-only). The replaced names remain in earlier commits and merged PR diffs.
_start(0x4008_0000) untouched. No kernel logic change:src/andlinker.ldchange in comments only.
Context
ADR-093 scrubbed cloud identifiers and the local OS user name in host paths. It did not classify machine names (the nicknames of the two developer workstations, used in docs as if they were roles), host names that tools printed into committed logs (an agent VM host name, an EC2 private host name, GitHub-hosted runner host names inside uname -a lines) or a machine ID (a UUID, in full once and as an 8-hex prefix four times). This repository is public. Two scripts would also print a host name into future logs: scripts/b2-metal-userdata.sh (uname -a) and scripts/qemu-nmi-probe.py (socket.gethostname()).
Decision
- Replace with role names or placeholders (values are not listed here):
- the amd64 developer workstation → “amd64 workstation”; the arm64 developer workstation → “arm64 workstation” (with “the” where the grammar needs it);
- agent VM host name →
<agent-vm-hostname>; EC2 private host name →<ec2-private-hostname>; GitHub runner host names →<gha-runner-hostname>; machine ID →<machine-id>. “agent-box” stays: it is a role name, not a host name.
- Stop new leaks at the source:
b2-metal-userdata.shprintsuname -srm(as ADR-095 already did by hand);qemu-nmi-probe.pyprintshost-os=<sysname> <machine>instead of the host name. - Guard classes (fail closed, CI-gated through
scripts/qemu-smoke.shas before).scripts/check-public-ids.shgains eight classes:vm-hostname,ec2-private-hostname,uname-hostname(a host-name-shaped token betweenLinux/Darwinand a version),gha-runner-hostname,user-home-path(C:\Users\<name>,/Users/<name>,/home/<name>;/home/runnerand<placeholder>allowed),wsl-user(wsl … -u <name>;rootand<placeholder>allowed),uuid, andmachine-name. Machine names are matched by SHA-256 of the lower-cased token, so the guard does not spell them out. That keeps them out of plain-text search; a hash of a short name can be brute-forced, so it is not secrecy. On a hit the guard still prints onlypath:line class.--self-testnow plants a synthetic token for all 15 classes (including a runtime-hashed fake machine name in two cases) plus a line of allowed forms that must not be flagged; markerpublic-ids: self-test ok classes=15.
Counts (by class, no values)
Guard of this branch run on unmodified main af5a9d1 (distinct path:line class pairs) and a plain grep for occurrences:
| Class | Before (lines) | Before (occurrences) | After |
|---|---|---|---|
| machine-name (two workstation names) | 129 | 148 (88 + 60) | 0 |
| vm-hostname | 7 | 7 | 0 |
| ec2-private-hostname | 1 | 1 | 0 |
| uname-hostname (overlaps the two rows above and below) | 5 | 5 | 0 |
| gha-runner-hostname | 3 | 4 | 0 |
| uuid (machine ID, runner host) | 2 | 2 | 0 |
| machine-ID 8-hex prefix (not a guard class; grep) | 4 | 4 | 0 |
| hostname-printing code | 2 scripts | 2 | 0 |
New guard on main: FAIL hits=151. On this branch: public-ids: ok … hits=0 (bash and dash).
Limits (non-claims)
- Shapes and a short hashed name list, not DLP. A new machine name needs its hash added. Short role-like words are not flagged.
- Git history, PR descriptions, issue comments and CI logs are not scanned or changed.
- Git author metadata and the GitHub owner name in repo URLs stay (public and intended).
- Not a “secure” or “compliant” claim.
ADR-102 — SMP slice S2: secondary core MMU on, own stack and vectors, PAN/SPAN, console lock
- Status: Accepted for implementation as an opt-in probe (sponsor go for S2, 2026-10-13 about 22:45 CEST, relayed by DSO). Changes no claim.
- Date: 2026-10-13
- Follows: ADR-100 (plan + S1). Implements its slice S2, with the deviations listed below.
- Base:
mainata6cb41e(#154 merged). _start(0x4008_0000) untouched. The secondary still enters at_secondary_entryin the boot-stub page.- $0: one local machine plus the existing GitHub Actions checks. No cloud agent.
Scope (from the sponsor go)
Bring core 1 up properly: MMU on with the shared kernel tables, its own stack, its own VBAR_EL1 vectors, PAN/SPAN set the same way as core 0 (ADR-080 / ADR-098), and a console lock so the two cores do not interleave UART output. A fail-closed probe proves it; negative builds must be caught.
ADR-091 revision 4 is unchanged: “EL0 isolated” stays scoped to -cpu cortex-a76, TCG, one core. No SMP safety claim of any kind.
Decision
All of it stays behind the opt-in Cargo feature smp2-probe (default build: no src/smp.rs code; the default smoke still fails on any smp: line).
- Parameter block. Core 0 fills a 64-byte-aligned block with the values it is running on:
MAIR_EL1,TCR_EL1, the liveTTBR0_EL1(kernel root) andTTBR1_EL1(shared kernel high map), andSCTLR_EL1with SPAN forced back to its reset value 1 (so core 1 must clear it itself). Plus the high-VA top of a dedicated 16 KiB secondary stack, the high VA ofcpu1_main, and the block’s own high VA. Core 0 cleans the block to the point of coherency (dc civac), because core 1 reads it with the MMU off. _secondary_entry(MMU off). Unchanged S1 part: raw MPIDR line, MPIDR stored to the block’s first word,dsb sy; sev. New: program MAIR/TCR/TTBR0/TTBR1,tlbi vmalle1(local; core 1’s TLB has seen nothing), write SCTLR (MMU, caches, WXN on), switchspto the secondary stack, branch tocpu1_mainat the TTBR1 alias. The instruction after the SCTLR write runs from the boot-stub page, which stays identity-mapped executable in the kernel TTBR0 (the same exception_startalready relies on).cpu1_main(MMU on, high alias, own stack). Installs its own 2 KiB vector tablectos_cpu1_vectors(not core 0’s, so no exception on core 1 can land on core 0’s exception / fatal stacks, ADR-100 R4). ClearsSCTLR_EL1.SPAN, setsPSTATE.PAN. Reads back SCTLR,PSTATE.PAN,VBAR_EL1,spandpcand prints them.- Deliberate exception. Only if MMU, VBAR and PAN read back correctly. Core 1 clears PAN, executes
brk #0x5102, and its own current-EL SPx sync handler (assembly, on core 1’s stack) recordsESR_EL1,ELR_EL1,PSTATE.PANat entry and an entry count, stepsELRpast thebrkanderets. Core 1 then sets PAN again. Pass requires: one entry,ESR = 0xf2005102(EC=0x3C BRK, IL=1, imm=0x5102),ELR= thebrkaddress, PAN=1 inside the handler (SPAN=0 sets it on entry, the ADR-098 property), PAN=1 after. Any other vector slot records which slot fired and parks the core, so core 0 times out and reports a miss. - Console lock. After its raw MMU-off line, every
smp:line from either core goes through the existinguart::UARTspin lock, held for one whole line. Core 0 also stops using raw writes once core 1 may be printing (smp: doneincluded). After core 1’s checks, core 1 waits for agoword, then both cores print foursmp: tick cpuN ilines at the same time. - Verdict by core 0. Core 0 waits up to 2 s for core 1’s verdict word, re-reads what core 1’s handler recorded, and prints
smp: ok cpu1 s2 mmu-on pan=1 span=0 vbar=own exc=handled console=lockedonly if the verdict is OK, no stray vector fired, four ticks were counted and the recorded ESR / count match. Otherwisesmp: cpu1 s2 missed verdict=… bad=… ticks=….
Negative builds (each implies smp2-probe; checker must fail and the miss line must appear)
| Feature | What it breaks on core 1 | Caught by |
|---|---|---|
smp2-neg-pan | skips msr pan, #1 | smp: cpu1 pan lost (pstate pan=0), verdict 3, no exception taken |
smp2-neg-span | skips the SPAN clear | handler sees PAN=0: smp: cpu1 pan lost (handler pan=0 span=1), verdict 4 |
smp2-neg-vbar | installs core 0’s vectors instead of its own | smp: cpu1 vbar mismatch, verdict 5; the exception is not taken (would hit core 0’s stacks) |
Plus the S1 negative: the same S2 ELF under -smp 1 (rc=-2, smp: cpu1 missed seen=0x0).
Checker (scripts/qemu-smoke.sh, smp2_check)
Requires every S1/S2 marker line exactly, each of the eight tick lines exactly once, rejects any smp: line that is not one whole known line (a torn or interleaved line fails closed), and rejects any miss line. CTOS_SMP2_ONLY=1 (the existing non-required CI job) runs: positive -smp 2, -smp 1 negative, then the three negative builds. The job name and required checks are unchanged; the job stays not required.
Deviations from ADR-100’s S2 row
- PerCpu via
TPIDR_EL1is not done here. Core 1 does not useTPIDR_EL1(it never enters the EL0 trampoline). Moved to S4, where the per-CPU EL0 context and expect flags need it. - The planned marker
smp: cpu1 mmu-on pan=1 span=0 vbar=highbecame the more specificsmp: cpu1 regs …+smp: ok cpu1 s2 …lines; the planned single negative (“skip SPAN clear →pan lost”) issmp2-neg-span, plus two more. - S1’s
smp: ok cpu1 … parkedbecamesmp: ok cpu1 … up(core 1 no longer parks MMU off).
What the probe proves (QEMU virt, TCG, -cpu cortex-a76, -smp 2)
- Core 1 runs with
SCTLR_EL1.M=1, from the TTBR1 high alias, on its own stack, with the same MAIR/TCR/TTBR0/TTBR1 values as core 0 at that moment. - Core 1’s
VBAR_EL1is its own table; one synchronous exception taken on core 1 was handled by that table on core 1 and returned. - Core 1 has
PSTATE.PAN=1andSCTLR_EL1.SPAN=0; PAN was set by hardware on exception entry. Each of the three settings, when skipped, is caught. - With the lock, the observed output lines are whole on this run.
What it does not prove
- Not SMP support, not multi-core safety, not “EL0 isolated” on two cores. Core 1 never runs EL0, the scheduler, the heap, page-table builders, drivers, the GIC or the timer.
- Nothing about TLB coherence between cores (S3), ASIDs (S4), concurrent page-table writes (R1), or that the tables core 1 sees stay valid when core 0 changes them.
- The console lock is not shown to be necessary: there is no deterministic negative for interleaving (it depends on host thread timing), so only the positive “lines are whole” is checked. Fatal / raw paths still bypass the lock (ADR-100 R10).
- Core 1’s stack has no guard page. Core 1’s vectors handle exactly one synchronous slot; IRQ/FIQ/SError on core 1 park it.
- No claim about real hardware, KVM, other
-cpumodels, more than two cores, or other PSCI firmware.
Proposed S3 (for sponsor go; not started)
Type-enforced TABLES (tables and bump counters inside the Mutex payload, IRQs masked while held) and Inner-Shareable TLB maintenance (tlbi …is + dsb ish; isb) at every unmap / downgrade / tear site, with a cross-core probe: core 1 (MMU on, as in S2) reads a probe page; core 0 unmaps it with the broadcast form; core 1 must fault on its own vectors (smp: xtlb unmap cpu0 → cpu1 fault). Negative build local-tlbi-leak-probe uses the old local tlbi and must report smp: xtlb leak (core 1 still reads the page through its stale TLB entry). Still no EL0 on core 1; ADR-091 rev 4 unchanged.
Consequences
- Default kernel and smoke unchanged (no
smp:line; required checks unchanged). - The opt-in CI job now builds four ELFs (positive + three negatives); it stays within its 30-minute timeout.
ADR-103 — SMP slice S3: type-enforced page-table lock, broadcast TLB invalidation, cross-core unmap probe
- Status: Accepted for implementation (sponsor go for S3, 2026-10-13 about 23:37 CEST, effective on the #156 merge at 23:38 CEST; relayed by DSO). Changes no claim.
- Date: 2026-10-13
- Follows: ADR-100 (plan, R1/R2), ADR-102 (S2). Implements ADR-100’s slice S3 exactly as proposed in ADR-102.
- Base:
mainatbb3f0ee(#156 merged). _startuntouched; the secondary still enters at_secondary_entry. Core 1 still runs no EL0.- $0: the Linux box plus the existing GitHub Actions checks. No cloud agent.
ADR-091 revision 4 is unchanged (one core). No SMP safety claim.
Decision
1. TABLES is enforced by the type system (R1)
- All 19 page-table statics (
L1…L3_HIGH_RAMand the three bump counters) move into a private modulepaging::tables. Nothing outside it can name them. TABLESis nowMutex<Token>;Tokenhas a private field, so only that module can build one.lock_tables()masks IRQs (DAIF.I), then takes the spin lock, and returns aTablesGuardthat restoresDAIFon drop.- Every function that hands out a writable slot, allocates an L3, clears a root or writes a descriptor takes
&mut TablesGuard: the 13 slot accessors, 3 allocators, 5 clear helpers,l3_write, and (because they call those)fill_ram_wx,fill_user_map,install_stack_guards,clone_ram_tables_for_high,l3_pa_for_ram_block,l3_pa_for_user_block,unmap_identity_page,unmap_user_ram_page,set_identity_l3_writable,reach_plant_slot. Forgetting the lock is now a compile error, not a convention. 45 lock sites. - Taking a table’s physical address needs no guard (it writes nothing).
Limits: reads still go through desc_at / walkers without the lock (racy reads are only a problem once another core writes; core 1 writes no tables). A slot pointer returned under the guard is a raw pointer; it is used only inside the guard’s scope by reading, not enforced by a lifetime. Code outside paging.rs (for example frame allocation) is not covered.
2. Broadcast TLB invalidation at every unmap / tear site (R2)
All TLB maintenance in paging.rs goes through two helpers (24 call sites: every unmap, permission downgrade, identity tear, map-window change and the boot tlbi_all). Both now do dsb ishst → tlbi vmalle1is / tlbi vaae1is → dsb ish → isb. The dsb ishst before the TLBI is new: it makes the descriptor write visible to other cores’ table walkers before the invalidate.
The default single-core build therefore runs the Inner-Shareable forms. On one core they act like the local forms; the default smoke and every existing probe pass unchanged. Not changed: switch_ttbr0_no_tlbi (ADR-096 t2t “no TLBI” by design, ASID-tagged) and core 1’s own boot-time tlbi vmalle1 in _secondary_entry (its TLB has never been visible to anyone).
3. Cross-core unmap probe (opt-in smp2-probe, after S2 passes)
- Core 0 writes a magic word to a dedicated 4 KiB kernel page and maps it in the shared kernel TTBR0 map window (
paging::map_page, global, under the guard). - Core 1 reads it through that VA (its TLB now holds the translation) and signals.
- Core 0 unmaps it (
paging::unmap_page: descriptor cleared under the guard, broadcast TLBI) and signals. - Core 1 executes one
ldrfrom the same VA. Its own sync handler (S2 vectors) records ESR / ELR / FAR and skips the load. - Pass: exactly one new exception,
EC=0x25(data abort, current EL), DFSC a translation fault,FAR= the VA, the load’s register still holds its sentinel. Core 1 printssmp: xtlb cpu1 fault ec=0x25 dfsc=trans-l3 far=window own-vectors; core 0 printssmp: xtlb unmap cpu0 → cpu1 fault. If core 1 instead reads the old magic without a fault:smp: xtlb leak.
4. Negative build local-tlbi-leak-probe
Same as smp2-probe but the two helpers keep the old local-only forms (tlbi vmalle1, tlbi vaae1). It must print smp: xtlb leak and the checker must fail. The smoke also still runs the S1/S2 negatives (-smp 1, smp2-neg-pan, smp2-neg-span, smp2-neg-vbar).
TCG timing (the honesty question)
Under TCG a missing broadcast does not have to leave a stale entry: QEMU’s per-vCPU soft TLB could be evicted or flushed for other reasons. Observed on the Linux box (QEMU TCG, -cpu cortex-a76, -smp 2), 2026-10-13: the leak build printed smp: xtlb leak on 27 of 27 runs and the broadcast build printed smp: xtlb unmap cpu0 → cpu1 fault on 27 of 27 runs. So on this QEMU the negative is reliable, and the smoke treats it as required.
This is a property of QEMU’s TLB model, not of any CPU. If a future QEMU stops reproducing the leak, the smoke will fail loudly (the negative will not print smp: xtlb leak) rather than pass silently. The fallback then is to show the leak on KVM on real arm64 (the ADR-099 pilot path, with sponsor approval), or to have core 1 read the page in a tight loop while core 0 unmaps it and report whether any read succeeded after the unmap completed.
What this proves
- No code can write page tables, allocate tables or clear roots without holding
TABLESwith IRQs masked: the compiler rejects it. - On QEMU TCG
-smp 2: after core 0 unmaps a page with the broadcast TLBI, core 1 faults on its next access (its own vectors handle it); with local-only TLBI, core 1 keeps reading the unmapped page. So the old local TLBI was a real cross-core hole here, and the broadcast form closes it for this case.
What this does not prove
- Not SMP support or safety; not “EL0 isolated” on two cores. Core 1 runs no EL0, scheduler, heap, drivers or interrupts, and writes no page tables.
- One VA, one page, one unmap direction (core 0 → core 1). Not every tear site is exercised cross-core; the others use the same helper.
- Nothing about ASIDs / non-global entries (S4), lock-free reads of tables, or ordering on real hardware or KVM.
- Nothing about QEMU versions other than the one on the box and the CI runner.
Proposed S4 (for sponsor go; not started)
ASID allocator with a generation counter (roll-over: tlbi vmalle1is + generation bump); a PerCpu block via TPIDR_EL1 (deferred from S2) holding each core’s saved kernel TTBR0, EL0 context and expected-fault flags (R4/R5); then run EL0 on core 1 for the first time, kernel-driven, one task per core, and re-run each isolation probe on core 1 with its marker tagged cpu=1. The existing leak builds run on core 1 must still be caught. Still no scheduler across cores (S5).
Consequences
- Default kernel: page-table code now needs the guard (compile-time); TLBI uses the IS forms; behaviour and smoke output unchanged on one core.
- The opt-in CI job builds five ELFs (positive + four negatives); it stays not required and within its 30-minute timeout.
ADR-104 — SMP slice S4: PerCpu via TPIDR_EL1, ASID allocator with generations, first EL0 on core 1
- Status: Accepted for implementation (sponsor “Go on S4”, relayed by DSO on 2026-10-19 at 17:31 CEST; the stop-all hold is lifted for this item only). Changes no claim.
- Date: 2026-10-19
- Follows: ADR-100 (plan, R3/R4/R5), ADR-102 (S2), ADR-103 (S3). Implements the “Proposed S4” section of ADR-103 exactly.
- Base:
mainat3234668(#158 merged). _startuntouched; core 1 still enters at_secondary_entry. The default single-core build keeps its behaviour and its full smoke.- $0: the Linux box plus the existing GitHub Actions checks. No cloud agent. No workflow file changed.
ADR-091 revision 4 is unchanged (one core). No multi-core safety claim. Any multi-core sentence is S5 and needs a separate sponsor accept.
Decision
1. PerCpu via TPIDR_EL1 (R4, R5)
- New
src/percpu.rs: one 64-byte-alignedPerCpublock per core (NCPU = 2): saved kernel TTBR0 at offset 0, core index, last ASID generation seen.TPIDR_EL1holds the block’s TTBR1 high VA. TPIDR_EL1used to hold the saved kernel TTBR0 directly. The four lower-EL trampolines (sync, IRQ, FIQ, SError) now domrs x2, tpidr_el1; cbz; ldr x2, [x2]; cbz; msr ttbr0_el1, x2(one extra load).- Core 0 installs its block in
kernel_main_highafter every identity tear, before the first EL0 entry. Installing it earlier failed: the ADR-037 pointer rewrite covers[__data_start, __kernel_end), and the saved TTBR0 is the kernel L1’s PA, which lies in.bssand looks like an identity pointer, so it was rewritten to a high VA. The boot then faulted on the first EL0 return. Until the install,TPIDR_EL1is 0 and the trampolines skip the restore (no EL0 runs before then). percpu::Banked<T>gives each core its own copy of a static and derefs to the reading core’s copy. Banked: the 11 EL0-context statics inel0.rs(ENTERED,ACTIVE,USER_PC,USER_SP,USER_TTBR,KIND, …) and the 60 expected-fault / caught / EL0-return statics inexception.rs(EXPECT_*,*_CAUGHT,EL0_CONT,EL0_KSP, …). Code that uses them is unchanged. On one core only slot 0 is used.
2. ASID allocator with a generation counter (R3), smp2-probe only
New src/asidgen.rs. 8-bit ASIDs; 0 (kernel), 1 and 2 (the fixed ADR-013 / ADR-096 probe ASIDs) are never issued, so 3..=255 (253 per generation). A task holds (generation, asid). Allocation runs under a spin lock. ASIDs are never freed one at a time. A retired task’s tables may be torn down without a TLBI, because its ASID is not reissued until the next rollover.
Rollover (bitmap exhausted): generation + 1, clear the bitmap, keep each core’s running ASID reserved into the new generation (so a task live on another core never shares its ASID with a new task), then dsb ishst; tlbi vmalle1is; dsb ish; isb before any ASID is reissued. switch_to revalidates a slot: same generation, keep it; older generation but still this core’s reserved running ASID, keep it under the new generation; otherwise allocate a fresh one. Each core records the generation it last switched under in PerCpu::asid_gen_seen.
Not wired into the default build: the default single-core kernel still uses the fixed ASIDs 1 and 2.
3. Core 1 in the kernel’s normal mode; the first EL0 on core 1 (smp2-probe)
After S1–S3 pass, core 0 runs vfs::init, virtio::init, init_net, fat::init, sched::init and gic::init (distributor and its own CPU interface; no timer on core 0), then signals core 1 and waits without printing. Core 1:
- installs
PerCpu[1]inTPIDR_EL1; - takes core 0’s
SCTLR_EL1.SPAN(so a build that undoes the PAN fix behaves on core 1 as on core 0); - sets
VBAR_EL1to the kernel vector table (no longer its S2 table),SP_EL1to its own 16 KiB exception stack, andSP_EL0to its own 64 KiB thread stack; thenSPSel = 0(EL1t, like core 0’s kernel threads); - enables its own banked GIC CPU interface, timer PPI and CNTP (
timer::init); - runs the single-core boot’s EL0 isolation probes, in the same order, on core 1:
el0(SVC, NX, read / write faults, standing + IRQ + FIQ + SError-arm),svc,sys-ptr,libctos,loader(also arms the live reach and PAN checks),pan-held,asid,t2t,xo,ttbr1,ident(identity inventory),el0-reach,pan. After each one it printssmp: cpu1 probe <name> ok cpu=1or… missed cpu=1; the probes print their own markers as on one core.
One task per core: core 0 runs no EL0 in this build. The standing loader task (observe_standing_task) and the non-isolation samples (FAT / net demos) are not part of S4.
4. ASID-generation probes (both cores)
- Concurrent allocation: both cores allocate 150 ASIDs at the same time (300 > 253, so at least one rollover). Core 0 checks that every
(generation, asid)pair is unique, every ASID is in 3..=255, and that at least one rollover happened:smp: asidgen alloc cpu0+cpu1 n=300 unique range=3..=255 gens=2 rollovers=1. - Reissue after rollover: core 0 maps a magic page (non-global) in the kernel map window. On core 1, task T1 gets ASID x (generation N), switches TTBR0 to the kernel L1 tagged x, and reads the page (core 1’s TLB now holds an x-tagged entry). T1 exits. Core 0 then clears the descriptor without any TLBI (the generation rule) and allocates until x is issued again in a later generation. That forces a rollover, which flushes with
tlbi vmalle1is. Core 1 runs task T2 with that slot (same TTBR0 value) and does oneldrfrom the page. Pass: an EL1 translation fault on the kernel vectors, caught by a per-core expected-fault flag:smp: asidgen cpu1 fault ec=0x25 dfsc=trans-l3 gen-seen=3 cpu=1, thensmp: asidgen reissue cpu0 rollover → cpu1 fault. - Core 0 prints
smp: ok cpu1 s4 percpu=tpidr el0=cpu1 probes=13 asidgen=unique,rollover-isonly if core 1 entered kernel mode, all 13 probes passed on core 1, the allocation was unique, and the reissue faulted.
5. Negative build (one ELF, all leaks on core 1)
smp2-probe + the new asid-rollover-local-probe (rollover flush is the local tlbi vmalle1) + the existing leak features: inv-leak-probe, reach-leak-probe, write-leak-probe, sysptr-leak-probe, t2t-leak-probe, xo-leak-probe, pan-keep-leak-probe. The smoke runs the same log checks as the single-core leak builds (factored into inv_leak_check / el0_leak_check; the only change is that the xo miss line is core 1’s smp: cpu1 probe xo missed cpu=1 instead of main.rs’s xo: probe missed). It also requires:
smp: cpu1 probe {el0,sys-ptr,pan-held,t2t,xo,ident,el0-reach} missed cpu=1,smp: cpu1 probes missed failed=7 n=13;… reissued (flush=local),smp: asidgen cpu1 read the retired page under the reissued asid cpu=1,smp: asidgen leak;- no
smp: ok cpu1 s4, and the S1–S4 checker must fail.
6. Smoke
scripts/qemu-serial-inject.py: new opt-inCTOS_QEMU_SMP(adds-smp N) andCTOS_QEMU_STOP_AT(stop at a serial string). Both absent means the default runs exactly as before.smp2_checkadditionally requires all S4 lines, the 13 probe tags exactly once each, and the probes’ own ok markers. The console-lock allow-list covers the newsmp:lines.- The S4 positive run and the S4 leak run use the inject script (virtio-blk FAT image, virtio-net) with
-smp 2. The-smp 1,smp2-neg-*andlocal-tlbi-leak-proberuns are unchanged (they stop before S4). - The opt-in CI job (
CTOS_SMP2_ONLY=1) builds one more ELF (six in total). Everycargo buildalready publishes the app ELFs, so the workflow file is unchanged. Its comment “Does not run EL0 on the secondary” is now stale; fixing it needs a separate workflow-file push and is left for later.
TCG timing (the honesty question)
- QEMU TCG does not model ASID-tagged TLB retention across an ASID change. On a TTBR0 write that changes the ASID, QEMU flushes that vCPU’s soft TLB. So under TCG the existing
asid: ok(ADR-013) and the ASID-switch part of t2t pass trivially. For the same reason the reissue probe keeps core 1 on one TTBR0 value from T1 through T2 (T1 “exits” without switching back to ASID 0). On hardware the stale entry would also survive an ASID switch, so the probe still models the real hazard, but it does not show ASID tagging itself. - What the probe does show under TCG: the only thing between core 1’s cached entry and its next access is core 0’s rollover flush. With
tlbi vmalle1iscore 1 faults; with localtlbi vmalle1core 1 reads the retired page. - Observed on the Linux box, QEMU 10.0.13 TCG,
-cpu cortex-a76,-smp 2, 2026-10-19: the S4 positive build printedsmp: ok cpu1 s4 …(13/13 probes ok on core 1, reissue fault) on 20 of 20 runs. The S4 leak build printedsmp: asidgen leakandsmp: cpu1 probes missed failed=7 n=13on 20 of 20 runs. The S3local-tlbi-leak-probestill printedsmp: xtlb leakon 20 of 20 runs on this QEMU (S3’s 27/27 were measured on an earlier QEMU on the box). The fullCTOS_SMP2_SMOKE=1 ./scripts/qemu-smoke.shpassed (default smoke, single-core leak builds, S1–S4 positive and all negatives). - If a future QEMU stops reproducing a leak, the negative fails loudly instead of passing silently. The fallback is the same as in ADR-103: KVM on real arm64 (ADR-099 pilot path, sponsor approval needed).
What this proves
- Two cores each have their own
TPIDR_EL1→PerCpublock, own exception and thread stacks, and own copies of the EL0 context and expected-fault flags. The EL0 trampolines restore each core’s own kernel TTBR0 through it. - On QEMU TCG
-smp 2, every EL0 isolation probe of the single-core boot also passes when run on core 1 (kernel-driven EL0, IRQ / FIQ from core 1’s own timer via the lower-EL trampolines). Every existing leak build is still caught there, with its ok marker withheld. - The ASID allocator hands out unique
(generation, asid)pairs to two cores at once across a rollover. A rollover’s broadcast flush removes core 1’s stale entry for a reissued ASID; a local flush does not.
What this does not prove
- Not SMP support or safety; not “EL0 isolated” on two cores. ADR-091 rev 4 stays one core.
- The cores run probes one at a time. Core 0 waits while core 1 runs EL0. The globals that are not banked are the result / “ready” flags core 0 publishes at boot and the probe bookkeeping in
syscall.rs,pan.rs,loader.rs,reach.rs,t2t.rs,xo.rs,teardown.rs. So are the shared user tables (user_ttbr0, ASID-B), the shared EL0 probe page, and the scheduler. Running EL0 on both cores at once would race on these. - The fatal / nested-exception stack is still one linker stack shared by both cores. Core 1’s stacks have no guard pages. The
guard,wx,ro,sched, standing-task and sample probes were not run on core 1. - The allocator is not used by the default build or by the probes’ fixed ASIDs 1 and 2. There is no scheduler, so
switch_tois only exercised by the probe. The “reserved across rollover” path is not exercised by a probe (no task runs on core 1 during core 0’s rollovers in the reissue probe). - One reissue scenario, one direction (core 0 rolls over, core 1 faults). Nothing about real hardware, KVM or other QEMU versions.
- The SError-while-standing probe on core 1 parks (
el0: serror-park), as on one core with this QEMU:inject-nmiis not available on stock QEMU. The taken-SError path was not shown on core 1.
Proposed S5 (for sponsor go; not started)
Done as ADR-105 (sponsor go 2026-10-19 21:26 CEST).
Scheduler and concurrency: a run-queue lock and per-core current-task pointers (R6); EL0 tasks on both cores at the same time, each with an allocator ASID and its own user tables (no shared EL0 probe page); bank or retire the remaining global probe flags; a per-core fatal stack and guard pages for core 1; a probe that runs t2t / reach / xo on both cores at once with a rollover in between, plus a negative that drops the reserved-running-ASID rule. Then, docs only and only with a written sponsor accept, a proposal for the multi-core sentence (scope, the TCG limits above, and what KVM evidence it would need).
Consequences
- Default kernel: the TPIDR indirection costs one extra load per lower-EL exception. The 71 banked statics double in size. Behaviour and the full default smoke (including the single-core leak builds) are unchanged.
- The opt-in CI job builds six ELFs (the positive, also run under
-smp 1; three S2 negatives; the S3 leak; the S4 leak). It stays not required, within its 30-minute timeout.
ADR-105 — SMP slice S5: concurrent EL0 on both cores, run-queue lock, reserved ASID
- Status: Accepted for implementation (sponsor go for S5, relayed by DSO on 2026-10-19 at 21:26 CEST). Changes no claim.
- Date: 2026-10-19
- Follows: ADR-100 (plan, R3/R4/R5/R6), ADR-104 (S4). Implements the “Proposed S5” section of ADR-104.
- Base: branch
smp/s4-asid-percpu-el0(PR #160 / ADR-104), ormainonce #160 has merged. _startuntouched. The default single-core build keeps its behaviour and its full smoke.- $0: the Linux box plus the existing GitHub Actions checks. No cloud agent.
ADR-091 revision 4 is unchanged (one core). A separate docs-only PR proposes a multi-core claim sentence; it stays Proposed until the sponsor writes an accept.
Decision
1. Run-queue lock and per-core current task (R6)
New src/runq.rs (smp2-probe only): a spin-locked table of up to 8 tasks. pick returns the next runnable unowned task, marks this core as its owner and stores id+1 in PerCpu::current. put returns it (runnable or finished); an owner mismatch increments a violation counter. Round-robin, no preemption: each trip is one EL0 entry to SYS_EXIT.
2. Per-task address spaces
New paging::TaskSpace (smp2-probe only): three fresh frames (L1→L2→L3) mapping only the task’s own pages in the window — text (EL0 execute-only), data (EL0 RW), stack (EL0 RW), all nG. No kernel mapping in TTBR0: vectors, exception stacks and TPIDR_EL1→PerCpu are TTBR1. Writes take TABLES. task_space_walk collects every EL0-reachable leaf for the reach / t2t / xo check.
3. Remaining shared flags banked; per-core fatal stack; guard pages
- Banked (on top of S4’s 71): the run-local SVC result flags in
syscall.rs(18), the live PAN / xo / reach flags inpan.rs,xo.rs,reach.rs(3+2+3). Boot-published IDs (PAN_ID, …) stay global. - Core 1 gets its own fatal stack (8 KiB + 4 KiB guard).
fatal_enterreadsPerCpu::fatal_top(offset 24); 0 = the linker fatal stack (core 0 / default). On a secondary the fatal handler only records the stack check (no raw UART) so it cannot tear core 0’s locked lines. - Core 1’s thread / exception / fatal stacks each have a 4 KiB guard page, punched at boot with the linker-stack guards (before the TTBR1 RAM clone), so the high aliases have the same holes.
4. Concurrent EL0 + mid-run ASID rollover (smp2-probe, after S4)
- Core 0 builds two tasks (A, B), each with its own
TaskSpaceand an allocator ASID, checks reach/t2t/xo on both spaces and that core 1’s guard pages are unmapped, then signals. - Both cores
pickfrom the run queue and drive EL0 (s5_run_once: switch ASID,ERET, wait forSYS_EXIT, check the data magic). - Core 1’s first pick holds its ASID (still marked running) while core 0 forces at least one ASID-generation rollover. The reserved-running-ASID rule (ADR-104) must keep core 1’s ASID out of the allocator. Core 1 then finishes its run and drains; core 0 drains the rest.
- Live xo: each task’s EL0 load of its own text must fault (permission L3).
- Core 1 triggers the nested-BRK fatal probe; it must land on its own fatal stack.
- Pass line:
smp: ok cpu0+cpu1 s5 runq=locked asid=reserved concurrent=el0 reach,t2t,xo.
5. Negative build asid-reserve-drop-probe
Same as smp2-probe but the ASID rollover no longer keeps running ASIDs reserved (and switch_to will not keep an older generation’s running ASID). The probe must print smp: s5 asid reissued while running cpu=1 asid=… and must not print the S5 ok line; the checker must fail.
6. Smoke / CI
smp2_checkrequires the S5 lines. New S5 negative ELF in the opt-in smoke.- Workflow comment on the
qemu-smp2job updated (ADR-100..105; EL0 on the secondary is no longer denied). Job stays not required. Timeout stays 30 minutes.
TCG honesty
- Same ASID-tagging limit as ADR-104: QEMU flushes a vCPU’s soft TLB on an ASID change. The reserved-ASID proof is that the allocator did not reissue core 1’s ASID during the hold (positive) / did reissue it (negative), not that a stale TLB entry survived a switch.
- Concurrent EL0 on TCG is real (both vCPUs enter EL0 and take SVCs); scheduling quanta are TCG’s, not a wall-clock claim.
- Observed on the Linux box, QEMU 10.0.13 TCG,
-cpu cortex-a76,-smp 2, 2026-10-19: the S5 positive build printedsmp: ok cpu0+cpu1 s5 …on 10 of 10 runs;asid-reserve-drop-probeprintedsmp: s5 asid reissued while running cpu=1on 10 of 10 runs (no S5 ok line). FullCTOS_SMP2_SMOKE=1smoke:qemu-smoke: ok(default smoke, single-core leaks, S1–S5 positive and all negatives).
What this proves
- A locked run queue hands each task to one core at a time; owner mismatches stay at 0.
- Two tasks with private tables and allocator ASIDs run EL0 on both cores in one session; each space has exactly the three expected EL0 leaves (xo text, RW data, RW stack) and neither data frame appears in the other’s walk; EL0 load of own text faults.
- A mid-run ASID rollover keeps core 1’s running ASID reserved; dropping that rule reissues it and is caught.
- Core 1’s fatal nested BRK runs on its own fatal stack; its three guard pages are unmapped (identity and high).
What this does not prove
- Not a general-purpose SMP OS. No preemption, no IRQ-driven reschedule, no multi-priority queue, no user-facing API.
- The default build still uses fixed ASIDs 1 and 2; the allocator and run queue are opt-in only.
- Boot-published result flags, the shared user / ASID-B tables used by the single-core probes, virtio DMA rings and VFS locks are unchanged. The S5 tasks do not share the single-core EL0 probe page.
- Nothing about real hardware, KVM or other QEMU versions. ADR-091 rev 4 stays one core.
Proposed next (not started)
- Wire the allocator into the single-core loader path (retire fixed ASIDs 1/2 for real tasks).
- IRQ-driven preemption across cores; per-core timer ticks counted separately.
- Sponsor accept of the multi-core claim sentence (separate docs-only PR).
Consequences
- Default kernel: core 1’s stacks add ~96 KiB to
.bssin the opt-in build only (theGuardedstatics live insmp.rs, which is compiled only withsmp2-probe). Default build unchanged. - Opt-in CI job builds one more ELF (S5 reserve-drop negative). Still not required.
Fix-forward (CI)
The first CI run of the opt-in smp2 job failed: core 0’s unlocked el0: task-enter print tore core 1’s locked s5 cpu1 hold line. In the smp2-probe build the task-enter/active/exit/restored and svc: exit lines now go through the UART lock, one whole line at a time. The default build still uses the raw writer and is unchanged.
ADR-106 — Proposed multi-core claim sentence (docs only; not accepted)
- Status: Proposed, not accepted; needs the sponsor’s written acceptance.
- Date: 2026-10-19
- Follows: ADR-091 rev 4 (the current accepted claim), ADR-100–ADR-105 (the SMP race story, opt-in probes only).
- This PR changes no code and does not edit the ADR-091 rev 4 claim. Until the sponsor writes an accept, the accepted claim stays one core.
Proposed sentence
On QEMU TCG,
-cpu cortex-a76,-smp 2, the opt-insmp2-probebuild has demonstrated: (1) each core has its ownTPIDR_EL1→ PerCpu block, exception / thread / fatal stacks and expected-fault flags; (2) broadcast TLBI (tlbi …is) removes a mapping from the other core’s TLB; (3) an ASID allocator with generations keeps a running core’s ASID reserved across a rollover (tlbi vmalle1is), and dropping that rule is caught; (4) two tasks with private TTBR0 tables and allocator ASIDs run EL0 concurrently on both cores, with a reach / task-to-task / execute-only check over both spaces. This is not a claim that “EL0 is isolated” on two cores in the ADR-091 sense: the default build is still one core; the probes are opt-in; TCG flushes a vCPU’s soft TLB on any ASID change (so ASID tagging itself is not shown); there is no preemptive cross-core scheduler; and no KVM or real-hardware evidence is attached.
Why this shape
- Tight to what ADR-102..105 actually printed on the box and in CI.
- Keeps ADR-091 rev 4 as the accepted product claim until a written accept replaces or extends it.
- Names the TCG / ASID-tagging limit explicitly so a future reader cannot treat the rollover proof as a hardware ASID-tag proof.
Accept path
A written sponsor accept of this sentence (or a tightened revision of it). On accept: move this ADR to Accepted, add a one-line pointer from ADR-091 (without rewriting rev 4’s one-core sentence unless the accept says to), and add a ledger row. No code change is required for the accept itself.
ADR-107 — Track A — FAT32 read path beyond FAT16 (second virtio-blk)
- Status: Accepted (runtime Verified only when this tip’s
qemu-smokeprintsfat32: okand the broken-chain negative printsfat32: fail reason=bad-chain; otherwise Documented) - Date: 2026-10-23
Context
ADR-028 put FAT16 on virtio-blk behind the thin VFS, and later ADRs grew write, readdir, unlink, grow, mkdir and rmdir on it. fat.rs rejects any volume with 65525 or more clusters, so a FAT32 volume (the kind mkfs.fat makes for anything past a few hundred MiB) cannot be read at all. The sponsor asked for FAT32 support on the existing virtio-blk path, tested in QEMU against an image built by mkfs.fat -F 32, with a fail-closed probe and a negative.
FAT32 differs from FAT16 in three places that matter here:
- BPB.
BPB_RootEntCntandBPB_FATSz16are 0; the FAT size is the 32-bitBPB_FATSz32, and the root directory’s first cluster isBPB_RootClus.BPB_FSInfonames the FSInfo sector, which carries three signatures and a free-cluster hint. - FAT table. Entries are 32 bits wide, of which the low 28 are used. End-of-chain is
>= 0x0FFFFFF8;0x0FFFFFF7is a bad cluster. - Root directory. There is no fixed root region. The root is an ordinary cluster chain.
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | A separate read-only FAT32 reader (src/fat32.rs) on a second virtio-blk device holding a mkfs.fat -F 32 image. The FAT16 disk, fat.rs and the VFS stay unchanged. | Smallest honest step. FAT16 regressions are impossible by construction, and the FAT16 smoke still runs on the same boot. |
| H2 (rejected) | Widen fat.rs (u16 clusters everywhere) to FAT32 with write support in one PR. | Touches every FAT16 write path at once. Too large to review and test honestly in one step. |
| H3 (rejected) | Append the FAT32 volume to the FAT16 image at a fixed LBA. | The FAT16 host checks (mkfat16.py --check) pin the image size, and a magic offset is a hidden contract. |
| H4 (rejected) | Hand-roll the FAT32 image in Python. | The sponsor asked for mkfs.fat -F 32. A real tool also catches reader assumptions a home-made writer would share. |
Decision
-
Disk. QEMU gets a second
-drive …,readonly=on+-device virtio-blk-devicewithtarget/fat32.img, listed before the FAT16 disk. QEMU fills virtio-mmio slots from the top, so the FAT16 disk stays the guest’s first-found blk device, and older kernels (the cross-update prior OS) keep using it.virtio::init2()programs the second transport in scan order with its own queue memory; it exposesread_sector2only (no write). -
Image.
scripts/mkfat32.shrunsmkfs.fat -F 32 -s 1 -S 512on 34 MiB (about 68k clusters, above the 65525 floor), then usesmcopyto add:PROBE32.TXT(fat32-hi\n);BIG.BIN(3000 deterministic bytes, 6 clusters);- 20 small files, so the root directory needs more than one cluster;
LAST.TXT(fat32-root-chain\n), which therefore lands in a later root cluster.
mtoolsallocates forward, so the script then movesBIG.BIN’s last three clusters to a free run near cluster 40000 and relinks them in every FAT copy.fsck.fat -nmust pass on the result (a host check, not a guest probe). -
Reader (
fat32.rs). It does the following:- Mount checks the 0x55AA signature,
bps == 512, a power-of-twospc, andRootEntCnt == 0andFATSz16 == 0(otherwiseNotFat32). It requires a cluster count of at least 65525 (otherwiseNotFat32), checks that the FAT fits every cluster index and that the root cluster is in range, and checks the FSInfo lead/struct/trail signatures plus a free-count bound. - Chain walk masks entries to 28 bits and ends at
>= 0x0FFFFFF8. A free (0), reserved (1), bad (0x0FFFFFF7) or out-of-range link isBadLink. The walk is bounded by the cluster count, so a cycle isCycle. - File read requires the chain length to equal
ceil(size / cluster)exactly (otherwiseSizeMismatch). - Directory scan walks the root chain, stops at the first
0x00entry, and skips deleted, LFN and volume-label entries. It matches 8.3 names only.
- Mount checks the 0x55AA signature,
-
Probe (boot path, after the FAT16 probe). It prints:
fat32: mount clusters=N root=R fsinfo=okfat32: read /probe32.txt bytes=9fat32: chain /big.bin clusters=6 jumps=J bytes=3000, where J ≥ 1 non-adjacent links and the content matches byte for bytefat32: root-chain clusters=K last=found(K ≥ 2)fat32: ok
Any error prints
fat32: fail reason=<reason>and the kernel printsfat32: probe missed. -
Smoke (fail closed).
qemu-smoke.shrequires every marker above, plusclusters >= 65525,jumps >= 1and root clusters>= 2, and rejects anyfat32: failorprobe missed. Negative: a second boot of the same kernel with a copy of the image whoseFAT[root]is set to free (mkfat32.sh --corrupt) must printfat32: fail reason=bad-chain, must not printfat32: ok, and must still print FAT16fat: ok. -
cargo test(in QEMU). The tests are:- in-memory FAT32 images covering a good read through a two-cluster root, a free link, a cycle, bad and out-of-range links, a chain longer than the size, bad FSInfo, a FAT16-sized cluster count and a non-zero fixed root;
- the real-disk probe;
- a check that the FAT16 primary disk is rejected by the FAT32 mount (
NotFat32).
-
CI.
dosfstoolsandmtoolsare added to the smoke jobs’ apt lines and to the Dockerfile.
Not claimed
- No FAT32 write, create, delete or grow. No VFS mount of the FAT32 volume. No LFN decoding. No subdirectory walk on FAT32 (only the root chain). No FSInfo update. No FAT mirror comparison (copy 0 is read). No exFAT.
- Not a performance claim. Not a real-hardware claim (QEMU
virtonly). - FAT16 behaviour is unchanged. The FAT16 VFS is still the only mounted FAT.
Consequences
- QEMU runs (smoke,
cargo test, cross-update) now needmkfs.fat,mcopyandfsck.faton the host.mkfat32.shfails loudly if they are missing. - A later ADR can mount FAT32 behind the VFS or add write support, starting from this reader.
ADR-108 — Taken-SError QEMU spike: what triggers it on TCG (stock and pinned)
- Status: Accepted (spike / research). Sponsor go relayed by DSO on 2026-10-23 at 19:25 CEST (stop-all lifted for ctos only). One finding, one PR. No change to the kernel,
_start, the default smoke or the ADR-091 claim. - Date: 2026-10-23
- Base: main
c420fba(includes #160, #161, #162). - Machine: amd64 workstation (bare-metal Ubuntu, TCG only; no KVM for an aarch64 guest). No cloud, no CloudAgent. Builds ran in an
ubuntu:24.04container on that host because the host lacks QEMU build deps and root. - Number: ADR-107 is taken by open PR #163, so this is ADR-108.
Question
Find a QEMU setup that can trigger and observe a taken SError, as groundwork for the “EL0 isolated” claim, and say exactly what is and isn’t shown today.
What is shown today (before this spike)
| Evidence | Where | Shows | Does not show |
|---|---|---|---|
B1 QEMU NMI pin smoke (taken SError) CI job | ADR-083, ADR-090 | Pinned QEMU v10.0.0 + patch 0001; QMP inject-nmi → bare el0: serror while the full ctos image stands at EL0 on cpu0, -smp 1. Fail-closed (CTOS_REQUIRE_TAKEN_SERROR=1). | Stock QEMU; cpu1 or any secondary; an SError source other than the repo’s own QEMU patch; hardware RAS. |
| B2-P (one-off KVM, Graviton3) | ADR-085 | Historical run 4 at 159b178, reduced profile. | Current boot path; repeatability; default image. |
| Default smoke on stock QEMU | ADR-053/081 | el0: serror-park: stock virt refuses inject-nmi (“machine does not provide NMIs”). | Any taken SError. |
| SMP S4/S5 | ADR-104, ADR-105 | cpu1 runs EL0 probes. | No SError probe is taken on cpu1; it parks on stock QEMU. |
Options tried (exact versions, this session)
QEMUs:
- stock A: distro
qemu-system-aarch6410.2.1 (1:10.2.1+ds-1ubuntu3.2) on the host. - stock B: Ubuntu 24.04
qemu-system-arm8.2.2 (1:8.2.2+ds-0ubuntu1.18), the same version as the GHA runners. - upstream master:
b6c4b7e(2026-10-23,VERSION11.1.50), built from source. - pinned B1: v10.0.0
7c949c53+research/qemu-nmi/0001-…patch, via./scripts/build-qemu-nmi.sh.
| Route | Result |
|---|---|
QMP inject-nmi / HMP nmi on stock A, stock B, master | Refused: Error: machine does not provide NMIs. Master hw/arm/virt.c still has no TYPE_NMI. |
| Same on pinned B1 | Works, and targets a chosen vCPU: HMP nmi with QMP human-monitor-command cpu-index=1 raises the SError on cpu1 only. |
gdbstub (-gdb, monitor nmi) | It is the same HMP nmi (forwarded over qRcmd): refused on stock, taken on pinned. gdb has no SError injection of its own, and writing registers would be a fake. Rejected as a separate route. |
| Bad device access (EL0 load from unassigned PA 4 GiB) | Synchronous external abort: vector 0x400, EC 0x24, DFSC 0x10. It is not an SError. Upstream target/arm/tcg/translate-a64.c says: “QEMU does not have a source of physical SErrors, so we are only concerned with virtual SErrors”; arm_cpu_do_transaction_failed raises a sync abort. Rejected (negative row kept). |
Virtual SError (HCR_EL2.VSE + AMO, guest boots at EL2 with -M virt,virtualization=on) | Works on every QEMU tried, including stock: taken at EL1 from EL0, vector VBAR_EL1+0x580, ESR_EL1 0xbe000000 (EC 0x2f), on cpu0 and on cpu1 (PSCI CPU_ON, -smp 2). Architectural, no QEMU patch. |
RAS extension registers (ERX*) / FEAT_RAS error injection | Not tried in code: TCG has no physical SError source (see above), so there is nothing to drive. FEAT_NMI is an interrupt and does not count as an SError (ADR-082). |
| KVM | Not available: the host is amd64. |
Finding (one)
On TCG there are exactly two honest ways to get a taken SError: (a) the pinned B1 patch, which is physical and host-injected and can now target cpu1; and (b) the architectural virtual SError via HCR_EL2.VSE, which works on stock QEMU 8.2.2, 10.2.1 and master, on cpu0 and cpu1, but only if the guest owns EL2. Bad-PA accesses and the gdbstub add nothing new.
Evidence: research/serror-spike/. The matrix has 13 fail-closed rows per QEMU, 4 QEMUs, 52/52 ok. Log: research/serror-spike/2026-10-23-matrix-log.txt.
Negatives (all pass, on every QEMU):
- VSE without
AMOis not routed; AMOwithout VSE gives no SError;- VSE with EL0
PSTATE.Aset is masked; - with the pin, an
nmito cpu0 while cpu1 stands at EL0 does not reach cpu1, and annmiwith EL0 A-masked is not taken; - with stock QEMU, every inject row must show the refusal and no SError.
Decision
- Record the finding. Add the stand-alone spike image (
research/serror-spike/vse),serror-matrix.py(fail-closed, exit 1 on any mismatch) andgdb-nmi.sh. No CI job is added, and the required checks are unchanged. - No claim changes. The ADR-090 evidence class and the ADR-091 rev 4 sentence are untouched. This spike uses a stand-alone image, not the ctos kernel.
- Possible next steps, each needing a sponsor go:
- N1 (pin, cpu1): extend B1 to
-smp 2withcpu-index=1while ctos cpu1 stands at EL0 (S4/S5 path). Needs a cpu1el0: serrorarm/handler in ctos. - N2 (stock, virtual): a small EL2 shim, loaded with
-device loaderahead of the kernel and not touching_start, that setsHCR_EL2.{RW,AMO}, ERETs to ctos at EL1, and sets VSE on an HVC from ctos’sserror-armpoint. This would give a taken SError on stock QEMU with the full image. It is a virtual SError (EL2-injected), and its honesty wording must say so.
- N1 (pin, cpu1): extend B1 to
Honesty
Say: “On QEMU TCG, a taken lower-EL SError is produced only by (a) the repo’s pinned TYPE_NMI patch, which can target cpu1 too, or (b) an architectural virtual SError from EL2 (HCR_EL2.VSE), which works on stock QEMU; shown on a stand-alone test image with negatives.”
Do not say:
- that ctos takes SErrors on cpu1;
- that stock QEMU has a physical SError source;
- that bad device accesses produce SErrors on QEMU;
- that this changes the “EL0 isolated” claim.
Consequences
- New: this ADR,
research/serror-spike/(README, vse image source, matrix script, gdb script, log), a brief, a ledger row, SUMMARY and moc entries. - No kernel, workflow, Dockerfile or smoke change. The default single-core build is unchanged.
ADR-109 — Track N / N6 — inbound TCP (one passive open over QEMU hostfwd)
- Status: Accepted (runtime Verified only when this tip’s
qemu-smokeprintsnet: tcp-in-okand the host client printshostfwd-client: neg=reset pos=ok rx=19 eof=yes; otherwise Documented) - Date: 2026-10-23
Context
ADR-074 (N5, PR #120) gave the kernel a thin active-open TCP: SYN to a QEMU guestfwd echo at 10.0.2.4:7, one payload, one echo. ADR-076 exposed it to EL0, and ADR-078 timed it. ADR-074 listed listen/accept as a locked non-goal. The sponsor now asks for inbound TCP: the host connects, data is exchanged, the connection closes, with fail-closed checks and a negative, over whatever networking the smoke already uses (QEMU user-net).
With QEMU user-net, hostfwd=tcp:127.0.0.1:<host port>-:<guest port> makes SLIRP accept on host loopback and open a TCP connection to the guest (source 10.0.2.2, destination 10.0.2.15:
Hypotheses
| ID | Option | Notes |
|---|---|---|
| H1 (chosen) | Kernel-path passive open on one port (8080) via hostfwd. The host client in qemu-serial-inject.py waits for the guest cue net: tcp-in-listen, then runs the negative and the positive. | Deterministic in CI (host loopback only, no external net). It uses the same virtio-net path and SLIRP as N1–N5. |
| H2 (rejected) | A sockets ABI (listen/accept/recv/send SVCs) with a socket table. | Product-sized. An EL0 surface can follow in its own ADR. |
| H3 (rejected) | TAP/bridge networking for a “real” inbound path. | Needs host privileges in CI and changes the network the smoke uses. |
| H4 (rejected) | Connect blindly on a timer without a guest cue. | Racy: SLIRP would accept on the host before the guest polls, and the result would depend on retransmit timing. |
Decision
-
QEMU. The hello-boot runner (
qemu-serial-inject.py) addshostfwd=tcp:127.0.0.1:<p1>-:8080,hostfwd=tcp:127.0.0.1:<p2>-:8081to the existing user netdev.p1andp2are free loopback ports chosen at run time; nothing binds outside loopback. Guest port 8081 has no listener: it is the negative. -
Guest (
src/tcpin.rs, after the N1–N5 probe). Two small raw-frame helpers are added tovirtio.rs:net_raw_txandnet_raw_rx.net_raw_rxreuses a still-posted RX buffer rather than posting twice. The guest then does the following:-
Cue. Print
net: tcp-in-listen port=8080 closed=8081, then poll for up to 15 s. -
ARP. Answer ARP who-has 10.0.2.15.
-
Parse. Accept only IPv4/TCP to 10.0.2.15 with a valid TCP checksum. Bad-checksum segments are dropped and counted; fragments and other addresses are dropped.
-
Closed port. A SYN to 8081 gets an RFC 793 reset (
RST|ACK,ack = seq + 1) andnet: tcp-in-rst port=8081. Late segments of earlier outbound probes, which arrive at other ports, also get an RST but do not count as the negative. -
Port 8080, one peer.
- SYN → SYN-ACK (
net: tcp-in-syn; a retransmitted SYN gets the SYN-ACK again). - The ACK of our ISS+1 →
net: tcp-in-est. - In-order data up to a newline →
net: tcp-in-rx bytes=N. Out-of-order data is re-ACKed. - Reply
ctos-in:<line>(net: tcp-in-tx bytes=M), then the guest closes with FIN. - The host’s ACK of our FIN plus the host’s FIN, which we ACK →
net: tcp-in-fin closed.
A second 4-tuple to 8080 gets an RST. A peer RST is a failure.
- SYN → SYN-ACK (
-
Result.
net: tcp-in-okrequires both the completed exchange and the closed-port RST. Otherwise the guest printsnet: tcp-in-fail reason=…and the kernel printsnet: tcp-in probe missed.
-
-
Host client. It starts on the cue and runs two steps:
- Negative. Connect to
p2. It must read EOF or get a reset with zero bytes; a timeout or any data is a failure. - Positive. Connect to
p1, sendctos-hello\n, and read to EOF. The reply must be exactlyctos-in:ctos-hello\n(19 bytes). Then close.
It prints
hostfwd-client: neg=… pos=… rx=… eof=…after QEMU’s output. - Negative. Connect to
-
Smoke (fail closed).
qemu-smoke.shrejects anytcp-in-failorprobe missed. It requires all eight guest markers (includingrx bytes=11,tx bytes=19andrst port=8081) and the exact host linehostfwd-client: neg=reset pos=ok rx=19 eof=yes. -
cargo test(in QEMU). The tests cover:- parsing a SYN;
- a corrupted segment being rejected as
BadChecksum; - RFC 793 RST fields for a SYN and for an ACK, and never answering an RST;
- reply checksums being valid with addresses and ports swapped;
- fragments and foreign destinations being dropped.
The runtime exchange itself is only in the boot smoke, which is where the host client exists.
Not claimed
- No sockets API, no EL0 SVC, no backlog or socket table, no concurrent connections, no retransmission timer, no window management or options (MSS not sent), no urgent data, no IPv6, no TLS or HTTP.
- No real-network claim: SLIRP on host loopback only, QEMU
virtTCG only. Not a security boundary. - N5 outbound TCP (ADR-074/076/078) is unchanged.
Consequences
- Every hello boot through
qemu-serial-inject.pynow runs the inbound exchange; the host client is always present there.cargo testboots (qemu-aarch64.sh) have no hostfwd and do not run this boot-path probe. - A later ADR can put an EL0
accept/recv/sendsurface on top of this state machine.
ADR-110 — N1: taken SError on core 1 at EL0 (pinned QEMU, -smp 2, cpu-index=1)
- Status: Accepted (implementation + evidence, opt-in). Sponsor go relayed by DSO on 2026-10-23 at 20:17 CEST (N1 from ADR-108).
- Date: 2026-10-23
- Base: main
c420fba. ADR-107 (#163, FAT32), ADR-108 (#165, the spike) and ADR-109 (#164, inbound TCP) are taken, so this is ADR-110. - Machine: amd64 workstation (TCG; builds in an
ubuntu:24.04container). CI: theQEMU NMI pin smoke (taken SError)job.
Context
The B1 pinned QEMU (v10.0.0 + patch 0001, ADR-083) gave a taken lower-EL SError only on core 0, at -smp 1. On core 1 (S4/S5, ADR-104/ADR-105) there was no SError probe. The ADR-109 spike showed that the pinned build’s HMP nmi honours cpu-index.
Decision
-
Opt-in features:
smp2-serror-probe(impliessmp2-probe): after core 1’s 13 S4 probes, core 1 printssmp: cpu1 serror-arm masked=no cpu=1and ERETs to the same standing-EL0 spin as the single-core probe, with PSTATE.A clear and the banked per-core EXPECT flags.smp2-serror-maskedis the negative build: it ERETs with A set.
-
Handler (fail-closed): in that build, core 1’s lower-EL SError handler prints its own locked line,
smp: cpu1 el0 serror taken cpu=1. It never prints the core-0el0: serrorcue. The probe printssmp: cpu1 serror taken=yes …only if the handler ran and the standing trip restored. Otherwise it printssmp: cpu1 serror park taken=no …. -
Host:
qemu-serial-inject.py, on thesmp: cpu1 serror-armcue and only whenCTOS_QEMU_NMI_CPU1=<n>is set, does QMPstop→human-monitor-command nmiwithcpu-index=<n>→cont. QMPinject-nmihas no CPU argument. -
Check:
scripts/smp2-serror-pin.sh. Every row requires the hostnmi=>ok,smp: ok cpu1 s4, and no unhandled exception.Row Build nmi to Must see Must not see P probe cpu-index=1 cpu1 el0 serror takenandtaken=yesparkN0 probe cpu-index=0 park taken=no masked=nocpu1 el0 serror takenNM masked cpu-index=1 park taken=no masked=yescpu1 el0 serror taken -
CI: a new step in
qemu-nmi-pinafter the unchanged single-core step (CTOS_REQUIRE_TAKEN_SERROR=1).
Evidence (2026-10-23, amd64 workstation, pinned QEMU 10.0.0)
smp2-serror-pin.sh: P ok (nmi(cpu-index=1)=>ok,smp: cpu1 el0 serror taken cpu=1), N0 ok, NM ok.- The full stock smoke (
CTOS_SMP2_SMOKE=1) and the single-core pin smoke (CTOS_REQUIRE_TAKEN_SERROR=1) still pass. - CI results are on the PR.
Honesty
Say: “With the repo’s pinned QEMU, a host-injected SError aimed at core 1 (cpu-index=1) is taken by ctos core 1 while it stands at EL0, on -smp 2 TCG. Aimed at core 0, or with core 1’s A bit set, it is not taken.”
Do not say:
- that stock QEMU does this;
- that it is hardware RAS;
- that the “EL0 isolated” sentence changes. A claim change needs a separate sponsor accept.
Also not shown:
- whether core 0 takes the N0 SError later: it stays pending with A masked, and nothing after unmasks it;
- concurrency with S5 (core 1 runs the probe before S5);
- KVM.
Consequences
Cargo.tomlgains two features.- Code:
src/el0.rscpu1_serror_probe, a cfg branch insrc/exception.rsfor the handler, one cfg call insrc/smp.rs, andsmp::linebecomespub(crate). - Scripts:
scripts/qemu-serial-inject.py(hmp_nmi, env-gated) andscripts/smp2-serror-pin.sh. - Workflow: one step added to
qemu-nmi-pin. - Docs: ledger row, SUMMARY, moc, brief.
- The default build, default smoke and
_startare unchanged.
ADR-111 — N2: EL2 shim → taken virtual SError at EL0 on stock QEMU (opt-in)
- Status: Accepted (implementation + evidence, opt-in). Sponsor go relayed by DSO on 2026-10-23 at 20:17 CEST (N2 from ADR-108).
- Date: 2026-10-23
- Base: main
c420fba. ADR-107 (#163), ADR-108 (#165, the spike), ADR-109 (#164) and ADR-110 (#166, N1) are taken, so this is ADR-111. - Machine: amd64 workstation (TCG; builds in an
ubuntu:24.04container).
Context
ADR-108 found that stock QEMU has no physical SError source, but it does model the architectural virtual SError (HCR_EL2.VSE) for a guest that owns EL2. ctos runs at EL1 and _start (0x4008_0000) must not change. So a tiny EL2 shim must sit in front of it.
Decision
-
tools/el2-shim: a stand-alone ELF linked at PA 0, the virt flash window, away from RAM and the ctos image. It is loaded with-device loader,file=<shim>,cpu-num=0, which also points core 0’s reset PC at it; QEMU still loads the ctos ELF with-kernel.- At EL2 it sets
HCR_EL2 = RW | AMO. IRQ/FIQ stay at EL1. - It also sets
CNTHCTL_EL2 = 3,CNTVOFF_EL2 = 0,CPTR_EL2 = 0x33ff,MDCR_EL2 = 0, and VPIDR/VMPIDR mirroring MIDR/MPIDR. - It prints
el2-shim: el1and ERETs to 0x4008_0000 at EL1h with DAIF masked and x0..x3 as QEMU set them. SCTLR_EL1 keeps its reset value. - It uses no RAM; its only scratch is
TPIDR_EL2. - HVC ABI (
hvc #0xC7, only x0 changes): 0 = query (returns magic0x5E2E1D00), 1 = setHCR_EL2.VSE, 2 = clear it. Any other HVC (ctos PSCIhvc #0) is forwarded assmc #0to QEMU’s PSCI. Any other EL2 exception printsel2-shim: unexpectedand parks.
- At EL2 it sets
-
ctos feature
el2-vserror(opt-in). In the standing-EL0 SError window, afterel0: serror-arm:- It checks
ID_AA64PFR0_EL1.EL2, and only then queries the shim. With no EL2 it issues no HVC. - It asks for VSE, prints
el0: virtual serror ask el2-shim=… vse=… masked=…, and ERETs with A clear. - The handler counts the SError only if ESR EC is 0x2f, and prints
el0: virtual serror taken (el2 vse) ec=0x2f esr=…. The defaultel0: serrorcue is never printed in this build. - VSE is cleared after the trip.
el2-vserror-maskedis the negative build: it ERETs with A set.
- It checks
-
qemu-serial-inject.py: opt-inCTOS_QEMU_MACHINEandCTOS_QEMU_EXTRA. When they are unset the argv is unchanged. -
Check:
scripts/el2-vserror-smoke.sh(fail-closed). Every row must reachel0: okwith no unhandled exception.Row Build QEMU Must see Must not see P el2-vserrorvirt,virtualization=on+ shimask el2-shim=yes vse=set masked=no,virtual serror taken (el2 vse) ec=0x2fserror-parkNA default same + shim el2-shim: el1,serror-parkvirtual serrorNM el2-vserror-maskedsame + shim vse=set masked=yes,serror-parktaken NS el2-vserrorplain virt(no EL2), no shimel2-shim=no vse=no,serror-parktaken, el2-shim: el1 -
CI: a new opt-in job,
QEMU EL2 shim virtual SError (opt-in, ADR-111; not required), on stock Ubuntu 24.04 QEMU.- The default build,
qemu-smoke.shand the required checks are unchanged. - The NA row also shows the default image runs fully under the shim.
- The default build,
Evidence (2026-10-23, amd64 workstation)
- Stock QEMU 8.2.2 (Ubuntu 24.04, container): P, NA, NM and NS all ok.
- Stock QEMU 10.2.1 (host distro): P, NA, NM and NS all ok. P line:
el0: virtual serror taken (el2 vse) ec=0x2f esr=0xbe000000. - CI results are on the PR.
Honesty
Say: “On stock QEMU (-M virt,virtualization=on) with the repo’s EL2 shim, the full ctos image takes a virtual SError (EL2-injected via HCR_EL2.VSE) at EL0, and it is not taken without the ask, with A masked, or without EL2.”
Do not say:
- that this is a physical SError or hardware RAS;
- that the default boot has an EL2;
- that it covers cpu1 (
-smp 1only; the shim does not wrap PSCICPU_ON); - that the “EL0 isolated” sentence changes. A claim change needs a separate sponsor accept.
Consequences
- New:
tools/el2-shim/andsrc/el2.rs(cfg). - Changed, all opt-in: cfg branches in
src/el0.rsandsrc/exception.rs, and two features inCargo.toml. - Scripts: env-gated
qemu-serial-inject.py, plusscripts/el2-vserror-smoke.sh. - Workflow: one new opt-in job.
- Docs: ledger row, SUMMARY, moc, brief.
_start, the default build and the default smoke are unchanged.
CI for ADR-083 pinned QEMU (unconditional since ADR-090)
The default qemu-smoke jobs in .github/workflows/smoke.yml stay on distro QEMU (park path).
Job qemu-nmi-pin (“QEMU NMI pin smoke (taken SError)”) builds the pinned QEMU and runs the smoke with CTOS_REQUIRE_TAKEN_SERROR=1 on every push and pull request (ADR-090, sponsor D1/D5). Until ADR-090 it was gated by repo variable CTOS_BUILD_QEMU_NMI == '1'. That gate was removed because GitHub treats a job skipped by if: as passing a required check. The variable is now unused; the Dockerfile build-arg of the same name is unrelated and stays opt-in.
Workflow edits need a credential with the OAuth workflow scope (amd64 workstation gh login). The box OAuth app has gist, read:org, repo only.
Local / amd64 workstation (no GHA variable needed)
./scripts/build-qemu-nmi.sh
export CTOS_QEMU="$PWD/tools/qemu-nmi/bin/qemu-system-aarch64"
export CTOS_REQUIRE_TAKEN_SERROR=1
./scripts/qemu-smoke.sh
Dockerfile already supports --build-arg CTOS_BUILD_QEMU_NMI=1.
Job as landed (reference; see smoke.yml for the source of truth)
qemu-nmi-pin:
name: QEMU NMI pin smoke (taken SError)
runs-on: ubuntu-24.04
timeout-minutes: 90
steps:
- uses: actions/checkout@v4
with: { fetch-depth: 0 }
- uses: dtolnay/rust-toolchain@nightly
with: { components: rust-src, llvm-tools-preview }
- run: |
sudo apt-get update
sudo apt-get install -y ninja-build meson pkg-config python3-venv flex bison \
libglib2.0-dev libpixman-1-dev libfdt-dev libslirp-dev
- run: ./scripts/build-qemu-nmi.sh
- env:
CTOS_QEMU: ${{ github.workspace }}/tools/qemu-nmi/bin/qemu-system-aarch64
CTOS_REQUIRE_TAKEN_SERROR: "1"
run: ./scripts/qemu-smoke.sh
Roadmap
Each row is one loop unit: one milestone, one branch, one GitHub PR. Do not stack a later stage onto an unclosed earlier PR. Cite frozen FR/NFR IDs in the PR when touching behavior; do not invent new IDs in chat.
Primary ISA is AArch64 (ADR-003). M0/M1 probes are UART + qemu-system-aarch64, not VGA 0xb8000.
| ID | Milestone | Probe that closes it | Status |
|---|---|---|---|
| M0 | Kernel tree + harness scaffold on GitHub | Files on the default-target PR; ledger started | In harness PRs #2/#3 (source). Merge is a human/MRC job. |
| M1 | QEMU virt boots UART “Hello World!” | qemu-system-aarch64 serial shows the string; see honesty ledger | Verified in the 2026-09-08 cloud probe (not CI). x86 VGA probe is historical only. |
| M2 | Integration test harness | QEMU virt ARM semihosting exit + #[test_case]; scripts/qemu-smoke.sh fail-closed | Verified in the 2026-09-08 cloud probe (not CI). Sponsor stacked this on the ISA PR. |
| M3 | Exception vectors + breakpoint | Test or QEMU serial proof the handler runs | Verified: 2026-09-09 cloud qemu-smoke + GHA smoke.yml on the M3 PR (see honesty ledger). |
| M4 | Fatal exception stack | Fatal path does not silently lock the VM | Verified: 2026-09-09 cloud qemu-smoke (see honesty ledger). GHA on the M4 PR branch still Unknown. |
| M5 | Hardware interrupts (GIC + timer) | Timer tick observable (serial or test) | Verified: 2026-09-09 cloud qemu-smoke + GHA smoke.yml on the M5 PR (see honesty ledger). |
| M6 | Input | Injected PL011 RX byte observable on serial (or test) | Verified: 2026-09-09 cloud qemu-smoke (see honesty ledger). GHA on the M6 PR branch still Unknown. |
| M7 | Paging + frame allocator | Map/unmap or allocator test | Verified: 2026-09-09 cloud qemu-smoke (see honesty ledger). GHA on the M7 PR branch still Unknown. |
| M8 | Heap (alloc) | Box/vec smoke on the heap | Verified: 2026-09-09 cloud qemu-smoke + GHA smoke.yml on the M8 PR (see honesty ledger). |
| M9 | Cooperative scheduler | Two tasks observed to run | Verified: 2026-09-09 cloud qemu-smoke + GHA smoke.yml (see honesty ledger). |
M0–M9 are on main (M9 = merge of PR #15 / FR-11). Pillar work through ADR-020 (#17–#28) plus docs #30/#51/#29, A1–A6 (ADR-021–ADR-027) are on main. This PR is Track A / A7 (issue #38, parent issue #31): ADR-028 virtio-blk + FAT16. Merge is still a human/MRC job (ADR-002).
Pillars (post-M9)
Bring-up M0–M9 stays one loop unit each. After M9, work is grouped under the three pillars.
| ID | Work | Probe that closes it | Status |
|---|---|---|---|
| P-SEC-1 | Threat-model v1.1 slice (NFR-10) | Read security.md; no “secure OS” claim | Verified (file + review). v1.1 adds linker stacks, guards, EL0 first mile, remaining W^X gaps. |
| P-SEC-2 | W^X / NX heap + coop stacks | Page-table PXN on heap + caught execute-from-heap IABORT (wx: ok) | Verified: 2026-09-10 cloud qemu-smoke (see honesty ledger). Map: ADR-012. Linker stack pages still X. |
| P-SEC-2b | Linker-stack guard pages | Unmapped 4 KiB holes; store faults (guard: ok) | Verified: 2026-09-10 cloud qemu-smoke (see honesty ledger). ADR-014. Not kernel W^X. |
| P-PERF-1 | Baseline CNTPCT probe (NFR-07) | Serial perf: cntpct and/or #[test_case]; not a published bench | Verified: 2026-09-09 cloud qemu-smoke + GHA (see honesty ledger). |
| P-PERF-2 | IRQ-to-handler CNTPCT delta | Serial perf: irq-delta + samples max >= min; not a latency budget | Verified: 2026-09-10 cloud qemu-smoke (see honesty ledger). |
| P-PERF-3 | Host debug ELF size (NFR-08) | Host marker perf: elf-size bytes=<n>; not a budget | Verified: 2026-09-10 cloud qemu-smoke host print (see honesty ledger). |
| P-SEC-3 | EL0 first mile | el0: svc + el0: nx kernel + el0: ok | First mile Verified: 2026-09-10 cloud qemu-smoke (see honesty ledger). |
| P-SEC-2c | RO+NX text/data (ADR-015) | ro: nx data + ro: write fault + ro: ok | Verified: 2026-09-11 cloud qemu-smoke (see honesty ledger). Identity image W^X on virt; not “secure OS.” |
| P-PERF-4 | Boot-to-ready CNTPCT (NFR-08) | perf: boot-delta ticks=<n>; not a budget | Verified: 2026-09-11 cloud qemu-smoke (see honesty ledger). |
| P-PERF-5 | A9 slot-disconnect CNTPCT pair (ADR-046) | perf: app-load + perf: embed-load + perf: slot-delta app=<a> embed=<b>; not a percent | Verified when this tip’s qemu-smoke prints the pair (see honesty ledger). QEMU TCG lab. Production slot stays FAT-only. |
| P-PERF-6 | FAT16 vs memfs write CNTPCT pair (ADR-051) | perf: fat-write + perf: memfs-write + perf: fs-write-delta fat=<a> memfs=<b>; not a percent | Verified when this tip’s qemu-smoke prints the pair (see honesty ledger). QEMU TCG lab. Keep fat: write / fat: ok. |
| P-PERF-7 | FAT16 vs memfs read CNTPCT pair (ADR-065) | perf: fat-read + perf: memfs-read + perf: fs-read-delta fat=<a> memfs=<b>; not a percent | Verified when this tip’s qemu-smoke prints the pair (see honesty ledger). QEMU TCG lab. Keep fat: read / fat: ok / ADR-051 write pair. |
| P-PERF-8 | Net-ping path CNTPCT (ADR-069) | perf: net-ping ticks=<n>; smoke greps prefix only; not a percent | Verified when this tip’s qemu-smoke prints the prefix (see honesty ledger). QEMU TCG lab. Keep N1/N2/N4. |
| P-PERF-9 | UDP DNS path CNTPCT (ADR-072) | perf: udp-dns ticks=<n>; smoke greps prefix only; not a percent | Verified when this tip’s qemu-smoke prints the prefix (see honesty ledger). QEMU TCG lab. Keep N1/N2/N3/N3.x/N4 + ADR-069. |
| P-PERF-10 | Thin TCP echo path CNTPCT (ADR-078) | perf: tcp-echo ticks=<n>; smoke greps prefix only; not a percent | Verified when this tip’s qemu-smoke prints the prefix (see honesty ledger). QEMU TCG lab. Keep N1–N5 / N5.x + ADR-069/072. |
| P-SEC-3b | User TTBR0 + EL0 cannot read kernel .data | el0: no kernel read; user table omits .data | Verified: 2026-09-11 cloud qemu-smoke (see honesty ledger). Isolation / PAN stay Planned. |
| P-SEC-3c | ASID-tagged TLB isolation | asid: dual + asid: conflict + asid: ok; no TLBI VMALLE1 on the switch | Verified: 2026-09-11 cloud qemu-smoke (see honesty ledger). Umbrella isolation / PAN stay Planned. |
| P-SEC-3d | Standing EL0 context | el0: standing + el0: restored; is_active() true only while standing | Verified: 2026-09-11 cloud qemu-smoke (see honesty ledger). Not POSIX. Default ERET clears IRQ mask (ADR-041). |
| P-SEC-3e | TTBR1 kernel-private page (ADR-016 first cut) | ttbr1: el1 + ttbr1: no el0 + ttbr1: ok | Verified: 2026-09-11 cloud qemu-smoke (see honesty ledger). |
| P-SEC-3f | EL1 fetch from TTBR1 high VA (ADR-017) | ttbr1: el1 exec + ttbr1: vbar + existing ttbr1: ok | Verified: 2026-09-11 cloud qemu-smoke (see honesty ledger). Identity boot stub stays. |
| P-SEC-3g | Identity-tear first cut (ADR-018) | ident: split + ident: fault + ident: high + ident: no el0 + ident: ok | Verified: 2026-09-11 cloud qemu-smoke (see honesty ledger). Full identity teardown / PAN / umbrella isolation were Planned at this mile (umbrella: P-SEC-3l). Do not claim “the kernel moved.” |
| P-SEC-3h | High-VA jump + 16 KiB identity text range (ADR-019) | ident: jump + ident: range + ident: text + existing ident: ok | Verified: 2026-09-11 cloud qemu-smoke + arm64 workstation Docker on 24d94e6 (see honesty ledger). Do not claim “the kernel moved.” |
| P-SEC-3i | High-VA vtable rewrite + live identity .text tear (ADR-020) | ident: reloc + ident: live + existing ident: ok; println! after the tear | Verified: 2026-09-11 cloud qemu-smoke + arm64 workstation Docker on e80dc93 + GHA merge-commit 34651404108 (see honesty ledger). .rodata / .data / heap stay. Do not claim “the kernel moved.” |
| P-SEC-3j | Identity .rodata / .data / heap tear | .rodata + .data/stacks + heap unmapped + high-only access. | .rodata Verified (ADR-025). .data/stacks Verified (ADR-037). Heap Verified (ADR-038) on this branch qemu-smoke (ident: heap-reloc / ident: heap / ident: heap-fault / ident: heap-high; heap-stay gone). Do not claim “the kernel moved.” |
| P-SEC-3k | PAN enable on virt | ID_AA64MMFR1_EL1.PAN != 0 and an EL1-vs-EL0 access fault | Verified (ADR-080 / #125 mile 2): default -cpu cortex-a76, pan: present + pan: enabled + pan: el1-fault. Historical a57 pan: absent (ADR-026 / ADR-054 / ADR-079). One mile; alone it is not the ADR-091 umbrella claim. |
| P-SEC-3m | EL0 entry without TLBI VMALLE1 (ADR-039) | el0: no-vmalle1; src/exception.rs has no tlbi vmalle1 | Verified: 2026-09-12 agent-box qemu-smoke (see honesty ledger). One mile; alone it is not the ADR-091 umbrella claim. PAN enable was a non-goal on the then-default a57 (enabled on a76 since ADR-080); umbrella: P-SEC-3l. |
| P-SEC-3n | Lower-EL IRQ while standing (ADR-040) | el0: irq; #[test_case] lower_el_irq_while_standing | Verified when this tip’s qemu-smoke prints el0: irq (see honesty ledger). FIQ/SError still park. One mile; alone it is not the ADR-091 umbrella claim. |
| P-SEC-3o | IRQ-unmasked default standing/task ERET (ADR-041) | el0: irq-default; #[test_case] lower_el_irq_default_eret | Verified when this tip’s qemu-smoke prints el0: irq-default (see honesty ledger). Short non-standing probes stay masked. FIQ/SError still park. One mile; alone it is not the ADR-091 umbrella claim. |
| P-SEC-3p | Identity _start stay honesty (ADR-042 / ADR-047) | ident: start-stay; #[test_case] identity_boot_stub_stays_while_live_torn | Verified honesty. Never yank _start. One mile; alone it is not the ADR-091 umbrella claim. |
| P-SEC-3s | Leftover identity RAM tear (ADR-049) | ident: ram / ident: ram-fault / ident: ram-high; reject ident: ram-stay | Verified on this branch when smoke passes. Optional mile from ADR-047. One mile; alone it is not the ADR-091 umbrella claim. Not “kernel moved.” |
| P-SEC-3q | Lower-EL FIQ while standing + SError park (ADR-043) | el0: fiq + el0: serror-park; #[test_case] lower_el_fiq_while_standing | Verified (FIQ taken + park honesty). Taken SError = deferred/non-goal (ADR-047). One mile; alone it is not the ADR-091 umbrella claim. |
| P-SEC-3r | Taken lower-EL SError QMP attempt (ADR-045) / hard-stop (ADR-053) / reopen (ADR-081) / B1 pin (ADR-083) / B2 spike (ADR-084) / B2-P KVM (ADR-085) | Guest A-clear path + el0: serror-arm + QMP inject-nmi log; el0: serror-park if not taken; bare el0: serror under opt-in pin | Stock virt TCG: deferred / non-goal (hard-stopped) (ADR-053 / ADR-081 / ADR-047); park Verified; do not require el0: serror on default smoke. B1: Verified (opt-in pin only) — ADR-083 pinned QEMU TYPE_NMI→SError, GHA job qemu-nmi-pin (CTOS_BUILD_QEMU_NMI=1). B2 (different runner/accelerator): Planned / blocked — ADR-084 2026-09-25 free-runner spike: no arm64 KVM/HVF/WHPX on any free GitHub-hosted runner (runs 36193423604 / 36193613972); needs sponsor choice (self-hosted arm64 KVM, paid bare metal, or wait). B2-P: Verified (opt-in, one-off paid run) — ADR-085 2026-09-26: AWS Graviton3 c7g.metal, QEMU -accel kvm + 0002 KVM_SET_VCPU_EVENTS serror_pending → bare el0: serror + b2: taken (fail-closed smoke PASS, run 4). Not a CI gate. Also fixed TCR_EL1.TG1 (reserved → 4K). Evidence class (D1): ADR-090 2026-09-26: B1 pin job made unconditional (every push/PR; a skipped job would pass a required check) + B2-P run 4 = the umbrella’s taken-SError evidence. Stock TCG hard-stop unchanged. One mile; alone it is not the ADR-091 umbrella claim. |
| P-SEC-3l | Umbrella EL0 isolation | Standing + PAN + full TTBR1 / identity teardown | Verified (scoped: the accepted ADR-091 sentence only; M6, 2026-09-28). History: was Planned / non-claim until checklist (ADR-055 / ADR-047 / sponsor closure ADR-060). Constraints: PAN enable Verified separately (ADR-080) but not the umbrella; _start stays; taken SError hard-stopped (ADR-081). Specific miles are not this row. 2026-09-26 audit ADR-086: all ADR-055 §A miles Verified this session; §B not Met (taken SError only opt-in B1/B2-P; five identity ranges remain in kernel TTBR0 incl. the _start stub; no umbrella accept). Milestones M1 (inventory ratchet + RAM-side tears) → M2 (MMIO high alias) → M3 (stub policy, D2) → M4 (SError evidence class, D1) → M5 (accept, D4) → M6 (flip). M1 done (ADR-087): I2/I4/I5 torn from kernel + user TTBR0; fail-closed ident: inv walk allows only {MMIO block, _start stub}; planted identity page caught (ident: inv-neg, inv-leak-probe). Sponsor D1–D5 answered 2026-09-26 ~14:35 CEST (D4 accept still pending the sponsor reading the draft). M2 done (ADR-088): GIC/PL011/virtio via TTBR1 Device blocks; identity MMIO block torn; inventory k=1 u=1 a=1 = _start stub only (ident: inv-ok allow=stub). M3 (ADR-089): sponsor D2: _start stub page = the one documented identity exception; checklist identity row reworded and Met (ADR-087/088 evidence). M4 (ADR-090): sponsor D1: taken-SError evidence class = B1 qemu-nmi-pin (now unconditional) + B2-P run 4; checklist row 1 reworded and Met on that class. M5 (ADR-091, Draft): proposed umbrella sentence + scope, pending the sponsor’s written accept (B3 not Met); M6 flip not started. ADR-092 (2026-09-27, sponsor option (b)): G1 EL0-reachability walk (k/u/ASID-B TTBR0 + TTBR1, allowlist = five user slots, fail-closed + leak build) and G2 EL0 write permission fault closed; new gap G4 (syscall copy helpers skip the EL0 permission check); ADR-091 revision 2 sentence back with the sponsor (B3 not Met). ADR-094 (2026-09-27): G4 closed; ADR-091 revision 3. ADR-095 (2026-09-28): G3 KVM re-check PASS. M6 (2026-09-28): sponsor’s written accept (B3, 16:41 CEST); ADR-091 Accepted; row flipped. Say “EL0 isolated” only as the ADR-091 sentence, with its limits (one core, the TCG default smoke machine; not a speculative-execution or side-channel claim, not a real-hardware claim, not a certification; app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN); no multi-core race story). |
Hub: pillars.md. ABI contract: syscall.md.
Track A — freestanding app hosting (#31)
A1 is the SVC ABI mile. A2 is the CRT / libctos mile. A3 is the guest ELF PT_LOAD loader mile. A4 is standing EL0 as normal mode for a loaded image. A5 is isolation completion (identity .rodata / .data / heap tears + EL0 entry without VMALLE1 + PAN ID-field; PAN enable was a non-goal on the then-default a57 per ADR-047, enabled on a76 since ADR-080). A6 is thin VFS + in-RAM memfs. A7 is virtio-blk + FAT16. A8 is documented sample rebuild recipes (no new runtime). A9 is the OS/app slot first cut plus leftover cross-update / embed-off (ADR-032). Track A children A1–A9 have a first cut; sample catalog deepened with fs-libctos (ADR-059), fat-libctos (ADR-061), and yield-libctos (ADR-062); product “app hosting is done” is Verified under ADR-048 criteria Met×5 (ADR-052). Not Linux containers. Not glibc. One child issue → one PR. Do not round A1–A9 miles alone into Linux/POSIX claims.
| ID | Work | Probe that closes it | Status |
|---|---|---|---|
| A1 | Stable SVC ABI + docs (#32, ADR-021) | svc: yield + svc: user-hi + svc: uart + svc: exit + svc: ok; #[test_case] | ABI mile Verified: 2026-09-11 cloud qemu-smoke on 8846bc5 (onto aa46219 / #29). App hosting stays Planned. |
| A2 | Freestanding CRT / libctos (#33, ADR-022) | libctos: hi + libctos: ok + libctos: linked; #[test_case] | CRT mile Verified: 2026-09-12 cloud qemu-smoke on c9b292b. App hosting stays Planned. |
| A3 | ELF (or raw image) loader into user TTBR0 (#34, ADR-023) | loader: mapped + loader: ok; payload libctos: hi / libctos: ok; #[test_case] | Loader mile Verified: 2026-09-12 cloud qemu-smoke (see honesty ledger). Not a Linux ABI. App hosting stays Planned. |
| A4 | Standing EL0 as normal mode (#35, ADR-024) | el0: task-enter + el0: task-active + el0: task-exit + el0: task-restored + el0: restore-fail + el0: task-ok; #[test_case] | Standing-task mile Verified: 2026-09-12 cloud qemu-smoke (see honesty ledger). Not isolation. App hosting stays Planned. |
| A5 | Isolation completion (#36, ADR-025 / ADR-026 / ADR-032 / ADR-037 / ADR-038 / ADR-039 / ADR-040 / ADR-041 / ADR-042 / ADR-043 / ADR-044 / ADR-045 / ADR-047 / ADR-049 / ADR-053 / ADR-054 / ADR-055 / ADR-060 / ADR-079 / ADR-080 / ADR-081 / #125) | ident: rodata + ident: data + ident: heap + ident: ram; el0: no-vmalle1; el0: irq + el0: irq-default + el0: fiq + el0: serror-park; pan: id= + pan: present + pan: enabled + pan: el1-fault (cortex-a76) | Prior A5 + PAN enable (ADR-079/080). Locks: taken SError still hard-stopped after ADR-081 re-probe (ADR-053 / ADR-081 / ADR-082); umbrella: P-SEC-3l (ADR-055); never yank _start. One mile; alone it is not the ADR-091 umbrella claim. |
| A6 | Thin VFS + memfs (#37, ADR-027) | fs: create + fs: write + fs: read + fs: el0 + fs: ok; #[test_case] | memfs mile Verified (see honesty ledger). Not POSIX. App hosting stays Planned. |
| A7 | virtio-blk + FAT16 (#38, ADR-028) | blk: virtio + blk: cap + blk: rw + blk: ok; fat: mount + fat: read + fat: ok; #[test_case] | block + FAT mile Verified when this tip’s qemu-smoke passes (see honesty ledger). Same VFS open. Not POSIX. Not FAT32. App hosting stays Planned. |
| — | FAT16 write depth (Future slice 3, ADR-050) | fat: write + fat: rewrite + fat: create + host --check-write; keep fat: ok / slot: ok | Verified when this tip’s qemu-smoke passes (see honesty ledger). Not POSIX. Not FAT32. App hosting stays Planned. |
| — | FAT vs memfs write CNTPCT (Future slice 4, ADR-051 / P-PERF-6) | perf: fat-write + perf: memfs-write + perf: fs-write-delta; keep fat: ok / slot: ok | Verified when this tip’s qemu-smoke prints the pair (see honesty ledger). Not a bench. Not a percent. App hosting stays Planned. |
| — | FAT vs memfs read CNTPCT (perf slice 3, ADR-065 / P-PERF-7) | perf: fat-read + perf: memfs-read + perf: fs-read-delta; keep fat: ok / write pair / slot: ok | Verified when this tip’s qemu-smoke prints the pair (see honesty ledger). Not a bench. Not a percent. App hosting stays Planned. |
| — | Net-ping CNTPCT (perf slice, ADR-069 / P-PERF-8) | perf: net-ping ticks=<n>; prefix grep; keep N1/N2/N4 | Verified when this tip’s qemu-smoke prints the prefix (see honesty ledger). Not a bench. Not a percent. N3 UDP unlocked by ADR-070 (not BSD sockets). |
| — | UDP DNS CNTPCT (perf slice, ADR-072 / P-PERF-9) | perf: udp-dns ticks=<n>; prefix grep; keep N1/N2/N3/N3.x/N4 + ADR-069 | Verified when this tip’s qemu-smoke prints the prefix (see honesty ledger). Not a bench. Not a percent. Not TCP. |
| — | Thin TCP echo CNTPCT (perf slice, ADR-078 / P-PERF-10) | perf: tcp-echo ticks=<n>; prefix grep; keep N1–N5 / N5.x + ADR-069/072 | Verified when this tip’s qemu-smoke prints the prefix (see honesty ledger). Not a bench. Not a percent. Not sockets. EL0 surface (not kernel N5 timer). |
| — | FAT16 mkdir depth (ADR-073 / sponsor mile 2) | fat: mkdir; keep fat: ok / write / readdir / delete / grow / slot: ok | Verified when this tip’s qemu-smoke passes (see honesty ledger). Not POSIX mkdir. Not a directory tree. Not TCP (mile 3). |
| — | Thin TCP (ADR-074 / N5 / sponsor mile 3) | net: tcp-ok (+ tcp-syn/tcp-est/tcp-tx/tcp-rx); guestfwd 10.0.2.4:7; keep N1–N3 + blk/FAT | Verified when this tip’s qemu-smoke prints net: tcp-ok (see honesty ledger). Not BSD sockets. Not listen/accept. Not TLS/HTTP. |
| — | FAT16 readdir depth (ADR-056) | fat: readdir + fat: entries; keep fat: ok / write / slot: ok | Verified when this tip’s qemu-smoke passes (see honesty ledger). Not POSIX getdents. Not FAT32. Not Linux/containers. |
| — | FAT16 delete depth (ADR-057) | fat: delete; keep fat: ok / write / readdir / slot: ok | Verified when this tip’s qemu-smoke passes (see honesty ledger). Not POSIX unlink. Not FAT32. Not Linux/containers. |
| — | VFS prefix mounts (ADR-058) | vfs: mount + vfs: mounts; keep fs: ok / fat: ok / slot: ok | Verified when this tip’s qemu-smoke passes (see honesty ledger). Not Linux mount(2). Not a new disk FS. |
| A8 | Documented sample apps (#39) | recipes cite only existing serial markers; docs-build | Recipes Verified (docs; smoke markers unchanged). Hub: what-can-run.md. No new ADR. Not app hosting. |
| A9 | OS–app slots (#48, ADR-030 / ADR-032) | host OS ELF + hello-libctos.elf; FAT /hello; slot: ok; same ELF on this OS and ba6541c | First cut + leftover when this tip’s qemu-smoke prints cross-update lines (see honesty ledger). A2–A4 load FAT (no embed). Product claim Verified under ADR-048/052 (this mile alone ≠ Linux hosting). |
| — | Freestanding fs-libctos catalog (ADR-059) | libctos: fs-ok + fsdemo: ok; keep slot: ok / /hello | Verified when this tip’s qemu-smoke passes (see honesty ledger). Memfs via ADR-058 /mem. Not POSIX. Not a reopen of ADR-048/052. |
| — | Freestanding fat-libctos catalog (ADR-061) | libctos: fat-ok + fatdemo: ok; keep /hello / /fsdemo | Verified when this tip’s qemu-smoke passes (see honesty ledger). FAT /probe via thin VFS. Not POSIX. Not a reopen of ADR-048/052. |
| — | Freestanding yield-libctos catalog (ADR-062) | libctos: yld-ok + ≥3× libctos: beat + yldemo: ok; keep prior three | Verified when this tip’s qemu-smoke passes (see honesty ledger). Cooperative yield rounds. Not preemption. Not a reopen of ADR-048/052. |
| — | FAT16 nested mkdir + empty rmdir (ADR-077) | fat: nested + fat: rmdir; keep fat: mkdir / grow / delete / net / samples | Verified when this tip’s qemu-smoke passes (see honesty ledger). Not POSIX. One nest level only. No new SVC. Not TCP CNTPCT. |
| — | Freestanding mkdir-libctos catalog (ADR-075) | libctos: mkdir-ok + mkdemo: ok; keep fat: mkdir / prior six | Verified when this tip’s qemu-smoke passes (see honesty ledger). EL0 fs_mkdir (27). Not POSIX. Not nested trees. Not EL0 TCP. |
| — | Freestanding tcp-libctos catalog (ADR-076) | libctos: tcp-ok + tcpdemo: ok; keep net: tcp-ok / prior seven | Verified when this tip’s qemu-smoke passes (see honesty ledger). EL0 net_tcp_echo (28). Not BSD sockets. Not listen/accept. |
On-disk filesystem work started as A7 (FAT16 read, not xv6-like). FAT16 write depth is ADR-050. FAT16 root listing is ADR-056. FAT16 root delete is ADR-057. FAT16 root mkdir is ADR-073. Nested mkdir + empty rmdir is ADR-077. EL0 fs_mkdir is ADR-075. EL0 net_tcp_echo is ADR-076. Thin VFS prefix mounts are ADR-058. Freestanding catalog samples on the same volume: ADR-059 / ADR-061 / ADR-062 / ADR-068 / ADR-071 / ADR-075 / ADR-076. See Filesystem: new vs extend.
Tracks (subordinate to principles)
Track A (freestanding apps, #31), Track B (Linux-compat research, #40), and Track N (network foundation, ADR-063) do not override principles.md. A9 is after Track A ABI/loader. Track B B1: Linux-compat subset research frame (ADR-031) — not claiming Linux userspace yet. Track B B2: Linux AArch64 vs ctos SVC gap map (linux-aarch64-syscall-gap.md) — inspection only; no Linux numbers in src/. Track B B3: standing EL0 stays the ctos process model (ADR-035) — Linux clone/execve/wait4 is a gap; do not add those syscalls. Track B B4: Linux ELF / auxv / PT_INTERP vs the freestanding loader (ADR-033) — gap ADR; Track A loader stays reusable; not claiming dynamic Linux ELF. Track B B5: Linux VFS vs thin ctos VFS (ADR-034) — concept compare only; no POSIX flags / dentries. Track B B6: never Linux-compat implementation from this ladder (ADR-036). Track B B7: guest OCI/Docker is a non-goal (ADR-029).
Track N — network foundation (ADR-063)
Separate track from A/B. Hub: track-n.md. N0 scope: ADR-063. N1 first frame: ADR-066. N2 ICMP ping: ADR-067. N3 UDP transport: ADR-070. N3.x EL0 UDP SVC: ADR-071. N4 EL0 net sample: ADR-068. N5 thin TCP: ADR-074. Reuse virtio-mmio patterns from blk; do not break FAT/blk smoke. Still not product: BSD sockets, TCP stack as product / listen-accept, DHCP/DNS as product, Wi‑Fi, Linux net stack, “has networking / sockets OS” marketing.
| ID | Work | Probe that closes it | Status |
|---|---|---|---|
| N0 | Scope ADR + docs wiring (ADR-063) | Docs-build; ledger Planned; no Verified invent | Accepted (docs) — this PR |
| N1 | Link bring-up / first frame (virtio-net mmio, ADR-066) | Discover + TX ARP + RX SLIRP gateway ARP reply; -netdev user,id=net0 -device virtio-net-device,netdev=net0; net: ok; keep blk/FAT green | Verified when this tip’s qemu-smoke prints net: ok |
| N2 | ARP + ICMP ping (kernel-path, ADR-067) | After ARP: ICMP echo vs 10.0.2.2; net: icmp-tx / net: icmp-rx / net: ping-ok; QEMU args unchanged | Verified when this tip’s qemu-smoke prints net: ping-ok |
| N3 | Minimal UDP transport (ADR-070) | UDP DNS query/reply vs SLIRP 10.0.2.3:53; net: udp-tx / net: udp-rx / net: udp-ok; keep N1/N2 | Verified when this tip’s qemu-smoke prints net: udp-ok |
| N4 | Freestanding sample using net SVCs (ADR-068) | libctos: net-ok / netdemo: ok; keep N1/N2/N3/samples | Verified when tip prints those markers |
| N3.x | EL0 UDP DNS SVC + udpdemo (ADR-071) | libctos: udp-ok / udpdemo: ok; keep N1/N2/N3/N4/samples | Verified when tip prints those markers |
| N5 | Thin TCP active-open + one payload (ADR-074) | net: tcp-ok (+ syn/est/tx/rx); guestfwd 10.0.2.4:7; keep N1–N3 | Verified when tip prints net: tcp-ok |
Docs website
mdBook + GitHub Pages (not a kernel milestone, not a new FR/NFR ID). https://ctos.artof.link HTTPS is Verified after #30 (deploy 34653046584 + HTTPS 200). Route 53 CNAME remains in place (zone <route53-zone-id>). Overview: what can run today. Publish notes: website.md.
Isolation leftovers (decided — ADR-047 / sponsor closure ADR-060)
- Taken SError: deferred / non-goal (hard-stopped) on virt TCG (ADR-053 / ADR-081 / ADR-082). Re-probed 2026-09-19 on cortex-a76 — still
machine does not provide NMIs. Unlock = ADR-081 B1/B2/B3; B1 research ADR-082, then B1 opt-in pin Verified (ADR-083,qemu-nmi-pin). B2 free-runner spike blocked (ADR-084, 2026-09-25: no arm64 KVM/HVF/WHPX on free hosted runners) — sponsor then chose B2-P: Verified (opt-in, one-off) on AWS Graviton3 under KVM (ADR-085, 2026-09-26). Park Verified; A-clear = dormant prep (ADR-045). - PAN: CPU reopen + enable Verified (ADR-079 / ADR-080; default
-cpu cortex-a76). Historical a57pan: absent. - Yank
_start: never while QEMU-kernelneeds0x4008_0000.ident: start-stayVerified honesty. Leftover identity RAM after heap: Verified optional mile (ADR-049; not umbrella-required). - Umbrella “EL0 isolated”: Verified only as the accepted ADR-091 sentence (M6, 2026-09-28; revision 4 accepted 2026-10-02; checklist ADR-055 all Met). Limits: one core, the TCG default smoke machine; not a speculative-execution or side-channel claim, not a real-hardware claim, not a certification; app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN); no multi-core race story. Sponsor one-pager: ADR-060.
- Product app hosting “done”: Verified under ADR-048 / ADR-052 (freestanding slots; not Linux/POSIX/containers).
Track A — freestanding app hosting
Tracker epic: issue #31. Child issues A1–A9.
Driving force (non-negotiable)
This track is subordinate to ctos core principles (principles.md). Do not skip a probe, drop a ratchet, or invent a “secure/fast app runtime” to finish a row faster.
- Honesty ledger — Verified only with a probe
- Antifragility — fail-closed smoke + ratchets
- Security — threat model; probed mitigations only
- Performance — measure first; no invented benches
- Document-first / one milestone → one branch → one PR
A9 (OS/app slot disconnect) sits after A1–A4. It does not override the list above. The first cut plus ADR-032 leftover mile: cross-update on ba6541c, no hello embed. Product “app hosting is done” is Verified under ADR-048 / ADR-052 (not Linux/POSIX/containers).
Goal
A freestanding (non-POSIX) application can be loaded and run with honest probes. Not Linux containers. Not glibc.
Ordered work
One loop unit each. Each row needs a fail-closed ledger probe.
| ID | Work | Status (2026-09-12) |
|---|---|---|
| A1 | Stable SVC / syscall ABI + docs | ABI mile Verified on 8846bc5 (#32, ADR-021, syscall.md). Not app hosting. |
| A2 | Freestanding CRT / libctos | CRT mile Verified on c9b292b (#33, ADR-022). Not app hosting. |
| A3 | ELF (or raw) loader into user TTBR0 | Loader mile Verified (#34, ADR-023). Not a Linux ABI. Not app hosting. |
| A4 | Standing EL0 as normal mode | Standing-task mile Verified on this tip (#35, ADR-024). Loaded app until exit; fail-closed restore. Not isolation. Not app hosting. |
| A5 | Isolation completion (remaining identity tear; PAN only if CPU + ADR) | Identity tears Verified; PAN CPU reopen Accepted (ADR-079 / #125; -cpu cortex-a76, pan: present). PAN enable still locked (ADR-080). Umbrella “EL0 isolated”: Verified only as the accepted ADR-091 sentence (M6); this mile alone is not it. |
| A6 | Thin VFS + memfs | memfs mile Verified (#37, ADR-027). In-RAM named buffers; create / write / read / close. Not POSIX. Not app hosting. |
| A7 | virtio-blk + FAT16 | block + FAT mile Verified on this tip when the ledger has blk: ok / fat: ok (#38, ADR-028). Same VFS open. FAT write depth: ADR-050. Root listing: ADR-056. Root delete: ADR-057. Not POSIX. Not a Linux FS. |
| A8 | Documented sample apps | Recipes Verified (docs + existing smoke markers) (#39). Rebuild recipes: what-can-run.md, apps-today.md, in-tree user/README.md. Coop UART, RX echo, standing EL0 / libctos hello; optional memfs + FAT16. No new ADR (docs-only). No new runtime markers. Not app hosting. |
| A9 | Disconnect OS image from app payloads | First cut (#48, ADR-030) + leftover (ADR-032): OS ELF + published hello-libctos.elf + FAT /hello. A2–A4 load that file (no production embed). Cross-update: same ELF on this OS and ba6541c when qemu-smoke prints both cross-update … slot:ok lines. Slot-load CNTPCT pair (ADR-046): perf: app-load + probe-only perf: embed-load + perf: slot-delta (QEMU TCG lab; not a percent). Not OTA. Product claim Verified under ADR-048 / ADR-052 (not Linux/containers). |
Out of scope for Track A
OCI/Docker containers (non-goal, ADR-029), glibc/musl ports, SMP, networking (Track N — ADR-063; Planned until N1). See host-apps.md.
Samples today: site what-can-run.md (extra: apps-today.md). Porting: site porting.md (extra: building-or-porting.md).
Track B — Linux-compat research (keep ctos specificity)
Tracker epic: issue #40. Child issues B1–B7.
Driving force (non-negotiable)
This track is subordinate to ctos core principles (principles.md). Linux-compat research must not abandon honesty, fail-closed sensors, the three pillars, or virt learning scope. Do not override principles for a “runs Linux” headline.
- Honesty ledger — Verified only with a probe
- Antifragility — fail-closed smoke + ratchets
- Security — threat model; probed mitigations only
- Performance — measure first; no invented benches
- Document-first / one milestone → one branch → one PR
Track A (track-a.md) should largely complete before deep Linux ABI work. Reuse A1–A9 where they help. Full Linux ABI is not a promise. Not claiming Linux userspace yet (ADR-031). Guest OCI/Docker/k8s is a non-goal (ADR-029) — not a later Planned row on this track.
Goal
Explore what Linux userspace / ABI-subset work would take without dropping ctos specificity. Frame: ADR-031. This is an ADR/research ladder, not a distro.
Keep
Honesty ledger, fail-closed smoke (including Docker/arm64 workstation ratchets), pillars (antifragility / security / performance), ADR-gated ISA/memory/EL0 decisions, QEMU virt until an ADR widens it.
Ordered research
| ID | Work | Status (2026-09-12) |
|---|---|---|
| B1 | ADR: Linux-compat goals & non-goals | Documented (#41, ADR-031). ABI subset research; keep ctos specificity; not claiming Linux userspace yet. Cites ADR-029; does not reopen containers as Planned. |
| B2 | Syscall surface map (Linux aarch64 vs ctos SVC) | Documented (#42, linux-aarch64-syscall-gap.md). Inspection map only. No Linux numbers in src/. Not claiming Linux userspace. |
| B3 | Process model vs Linux (fork/exec/wait) | Documented (#43, ADR-035). Standing EL0 stays the ctos model. Linux clone/execve/wait4 is a gap. Do not add those syscalls to src/. Not claiming Linux userspace. |
| B4 | Linux ELF / auxv / PT_INTERP vs freestanding loader | Documented (#44, ADR-033). Gap ADR only. Track A loader stays reusable. Does not accept PT_INTERP. Not claiming dynamic Linux ELF. |
| B5 | Linux VFS concepts vs thin ctos VFS | Documented (#45, ADR-034). Concept compare only. No POSIX flags / dentries in src/. Not claiming a Linux filesystem. |
| B6 | Decision: compat layer vs reimplement vs never | Decided (never) (#46, ADR-036). Research ladder complete. No Linux-compat implementation epic from B1–B5. Track A freestanding path stays. |
| B7 | Containers remain a non-goal (OCI needs a Linux host) | Documented (#47, ADR-029). Site: hosting-apps.md (extra: host-apps.md). Ledger: absence Verified; not Planned. |
B1 — Linux-compat frame
ADR-031 is the Track B frame. Linux-compat here means research into a possible Linux AArch64 ABI subset, not a promise to run a distro, glibc, musl, a shell, or Linux userspace.
Keep: honesty ledger, fail-closed smoke (including host Docker/arm64 workstation ratchets), three pillars, the freestanding Track A path (A1–A9 first cuts), QEMU virt until an ADR widens it.
Out: full Linux ABI as a promise; guest containers (ADR-029); replacing SVC #n / libctos with Linux x8 numbers in a research mile; claiming userspace because A3 loads ELF64 or A9 reads FAT /hello.
B2 is Documented (syscall gap table). B3 is Documented (process-model stance). B4 is Documented (ELF / auxv / PT_INTERP gap). B5 is Documented (VFS concept compare). B6 is Decided (never) (ADR-036): no Linux-compat layer or reimplementation from this ladder. File presence of these ADRs is not a Linux userspace probe.
B2 — syscall gap map
linux-aarch64-syscall-gap.md maps a representative Linux AArch64 set (exit, write, read, openat, close, brk/mmap, clone/execve/wait4, ioctl, plus CRT/namespace neighbors) onto today’s ctos SVC surface (reserved 0–2, public 16–23).
Keep: no Linux svc #0 / x8 numbers in src/ this mile. Track A SVC #<n> stays. No row is present (convention + number space both miss). Related Track A SVCs are partial. Namespace/mount stay never-per-ADR-031 (ADR-029).
B3–B5 must not treat this table as an implementation backlog. B6 chose never (ADR-036).
B3 — process model vs standing EL0
ADR-035 compares Linux fork / exec / wait to ctos standing EL0 (ADR-024) and the EL1 cooperative scheduler (ADR-010).
Keep: standing EL0 is one loaded trip until exit. EL1 coop workers are not Linux threads. A1–A9 probes stay. Do not add clone / execve / wait* to src/ this mile.
Out: treating A4 or A9 FAT /hello as execve; treating SYS_EXIT as exit_group; a pid table; CLONE_NEW* (already never-per-ADR-031).
On Linux AArch64, userspace fork() is clone. There is no fork syscall. B4 is Documented (ADR-033). B5 is Documented (ADR-034). B6 is Decided (never) (ADR-036). File presence of ADR-035 is not a Linux process table.
B4 — Linux ELF / auxv / PT_INTERP vs the freestanding loader
ADR-033 compares Linux exec of an ELF (interpreter + auxv + PT_DYNAMIC) with today’s A3 loader (ADR-023).
Keep: the Track A parser reusable (ET_EXEC, PT_LOAD, reject PT_INTERP, reject W+X). In-tree libctos apps stay on that path.
Out: accepting PT_INTERP in src/; claiming musl/glibc or dynamic Linux ELF; treating A9 FAT /hello as execve.
Static musl still needs a Linux stack/auxv and the B2 syscall surface. Dynamic musl/glibc also need an interpreter. Not claiming dynamic Linux ELF. B3 is Documented. B5 is Documented. B6 is Decided (never) (ADR-036).
B5 — Linux VFS vs thin ctos VFS
ADR-034 compares Linux VFS objects (dentry / inode / path / mount) to today’s thin VFS (ADR-027 / ADR-028).
Keep: no POSIX open flags, dentries, cwd, or mount(2) in src/ this mile. Track A VfsOps stays five calls. No Linux VFS object is present. Path strings and handles are partial. Mount / overlay stay never-per-ADR-031. File-descriptor numbers stay on the B2 gap map (openat / read / write / close partial).
B6 chose never (ADR-036) — this compare is not an implementation backlog. B3 and B4 are Documented. Not claiming a Linux filesystem.
B7 — containers are a non-goal
ctos guests do not host containers. OCI / Docker / Kubernetes need a Linux kernel (namespaces, cgroups, a Linux ABI, usually overlay). This tree has none of that in src/.
Today the arrow is reversed: a Linux host may run Docker/docker-smoke.sh to build and QEMU-smoke the ctos image. That is a host harness (NFR-04 / NFR-05), not a guest runtime.
Do not claim a Docker/OCI host. Do not treat this as far-later work waiting on Track B. Reopen only with a new GitHub epic plus a new ADR — not a Track B “win,” and not a new FR/NFR ID.
B6 — decision (never)
ADR-036 records the Track B outcome: never start a Linux-compat layer or Linux-like reimplementation from B1–B5.
Keep: Track A freestanding path; B1–B5 / B7 as the honesty record of the gap; principles over tracks.
Out: treating the research tables as a backlog; accepting PT_INTERP / Linux x8 / clone/execve / POSIX VFS / guest containers “later on Track B.”
Reopen only with a new epic + ADR + named workload probe.
Track N — network foundation (virtio-net ARP + ICMP + UDP + thin TCP + EL0 net/UDP samples)
Scope ADR: ADR-063. Separate from Track A (freestanding apps) and Track B (Linux-compat research — never implementation).
Driving force (non-negotiable)
This track is subordinate to ctos core principles (principles.md). Networking must not abandon honesty, fail-closed sensors, the three pillars, or virt learning scope. Do not override principles for a “has networking” headline.
- Honesty ledger — Verified only with a probe; N0 is Planned for runtime claims
- Antifragility — fail-closed smoke + ratchets; do not break blk/FAT markers
- Security — threat model names the I/O surface; no “secure OS”
- Performance — measure first; no invented benches
- Document-first / one milestone → one branch → one PR
Goal
Bring up a minimal guest network path on QEMU virt: virtio-mmio virtio-net, discover the device, TX/RX one raw Ethernet frame (N1), an ICMP echo to the SLIRP gateway (N2), a UDP datagram to SLIRP DNS (N3), then a thin TCP active-open + one payload vs QEMU guestfwd echo (N5), with serial markers and fail-closed smoke. Reuse virtio-mmio patterns from blk (ADR-028). N1: ADR-066. N2: ADR-067. N3: ADR-070. N3.x EL0 UDP SVC: ADR-071. N5 thin TCP: ADR-074. EL0 TCP SVC: ADR-076.
Keep
Honesty ledger, fail-closed qemu-smoke, existing virtio-blk + FAT16 + sample catalog markers, QEMU virt + AArch64 until an ADR widens it, Track A freestanding path.
Mile ladder
| ID | Work | Probe that closes it | Status |
|---|---|---|---|
| N0 | Scope ADR + docs wiring | Docs-build; ledger Planned row; no Verified invent | Accepted (docs) — ADR-063 |
| N1 | Link bring-up / first frame | Discover + TX/RX one raw Ethernet frame; QEMU -netdev / -device virtio-net-device; smoke greps; keep blk/FAT green | Verified when this tip’s qemu-smoke prints net: ok (ADR-066) |
| N2 | ARP + ICMP ping (kernel-path) | After ARP: ICMP echo request/reply vs 10.0.2.2; net: icmp-tx / net: icmp-rx / net: ping-ok; keep N1 + blk/FAT green | Verified when this tip’s qemu-smoke prints net: ping-ok (ADR-067) |
| N3 | Transport — minimal UDP (IPv4) | UDP TX/RX vs SLIRP DNS 10.0.2.3:53; net: udp-tx / net: udp-rx / net: udp-ok; keep N1/N2 + blk/FAT green | Verified when this tip’s qemu-smoke prints net: udp-ok (ADR-070) |
| N4 | Freestanding sample using net SVCs | After N1+ Verified; EL0 net_mac/net_ping + FAT /netdemo; smoke greps | Verified when this tip’s qemu-smoke prints libctos: net-ok / netdemo: ok (ADR-068) |
| N3.x | EL0 UDP DNS SVC + freestanding sample | After N3 Verified; EL0 net_udp_dns + FAT /udpdemo; smoke greps | Verified when this tip’s qemu-smoke prints libctos: udp-ok / udpdemo: ok (ADR-071) |
| N5 | Thin TCP (active open + one payload) | After N3; TCP SYN/est + one payload echo vs guestfwd 10.0.2.4:7; net: tcp-syn / net: tcp-est / net: tcp-tx / net: tcp-rx / net: tcp-ok; keep N1–N3 + blk/FAT green | Verified when this tip’s qemu-smoke prints net: tcp-ok (ADR-074) |
| N5.x | EL0 thin TCP SVC + sample | After N5; SYS_NET_TCP_ECHO (28) quiet guestfwd echo; FAT /tcpdemo; libctos: tcp-ok / tcpdemo: ok; keep net: tcp-ok | Verified when this tip’s qemu-smoke prints libctos: tcp-ok / tcpdemo: ok (ADR-076) |
Explicit non-goals (locked until new ADR + sponsor)
BSD sockets as product; TCP stack as product / listen/accept / multi-connection socket table (N5 is one active-open + one payload only — ADR-074); TLS; HTTP stack; DHCP/DNS as product features (N3/N3.x may use SLIRP DNS as probe bait only); Wi‑Fi; virtio-pci-only foundation stories; Linux net stack; “has networking / sockets OS” marketing. N3.x adds EL0 net_udp_dns only (ADR-071); N5.x adds EL0 net_tcp_echo only (ADR-076); general udp_send/udp_recv / listen-accept / sockets ABI need another ADR. N4 EL0 net ABI remains the tiny net_mac/net_ping surface (ADR-068).
Honesty
Do not say the guest “has networking” or “has sockets.” N0 is scope (ADR-063). N1 first-frame is ADR-066 (net: ok). N2 ICMP ping is ADR-067 — Verified only with net: ping-ok on a named tip. N3 UDP is ADR-070 — Verified only with net: udp-ok on a named tip. N3.x EL0 UDP SVC sample is ADR-071 — Verified only with libctos: udp-ok / udpdemo: ok on a named tip. N4 EL0 net sample is ADR-068 — Verified only with libctos: net-ok / netdemo: ok on a named tip. N5 thin TCP is ADR-074 — Verified only with net: tcp-ok on a named tip. N5.x EL0 TCP SVC sample is ADR-076 — Verified only with libctos: tcp-ok / tcpdemo: ok on a named tip. Still not BSD sockets / TCP product / DHCP/DNS product. CloudAgent HELD; evidence tags agent-box / amd64 workstation. No self-merge (ADR-002).
B2 — Linux AArch64 syscalls vs ctos SVC
Track B child issue #42 (parent #40). Frame: ADR-031. Freestanding ABI: ADR-021 / syscall.md.
This page is a gap map. It does not add Linux syscall numbers to src/. It does not decide compat layer vs reimplement vs never (B6 / #46). Not claiming Linux userspace. File presence of this note is not a Linux ABI.
Probe (what Verified means here)
| Claim | Probe | Status |
|---|---|---|
| Linux AArch64 numbers in the table | Read include/uapi/asm-generic/unistd.h from Linux v6.10 (torvalds/linux). AArch64 uses that generic table (svc #0, number in x8). | Document inspection |
| ctos public / reserved SVC surface | Read src/syscall.rs (SYS_* 16–23) + ADR-021 + ADR-027 + reserved 0–2 in ADR-013 on this tree | Document inspection |
| Linux userspace / musl / Alpine on the guest | No such serial marker | Not claimed. Ledger “Guest runs host apps” stays Planned. |
A later kernel qemu-smoke is a different row. This mile did not edit src/.
Status words
| Status | Means |
|---|---|
| present | Same Linux AArch64 number and comparable semantics, callable by a Linux svc #0 / x8 binary with no translation. |
| partial | A Track A SVC exists with a related effect. Number, calling convention, args, or meaning still differ. |
| absent | No ctos analog. B3–B6 may still discuss a named subset. Not a promise to implement. |
| never-per-ADR-031 | Would imply a full Linux host / distro / container promise that ADR-031 already ruled out (guest OCI also ADR-029). |
No Linux AArch64 syscall is present. The convention and the number space both miss.
Calling convention (the first gap)
| Item | Linux AArch64 | ctos (ADR-021) |
|---|---|---|
| Instruction | svc #0 (immediate is always 0) | SVC #<n> — n is the number |
| Number register | x8 | Immediate (not x8) |
| Args | x0–x5 | x0, x1, x2 only |
| Return | x0; negative Linux errno | x0; 0 or u64::MAX reject (not errno) |
| Unknown number | -ENOSYS | Unhandled immediate parks (fail-closed) |
Linux svc #0 collides with the ADR-013 first-mile probe (SVC #0). ADR-021 rejected adopting the Linux convention for that reason. A translation shim would be a later epic after B6 — not this page.
Public ctos numbers 16–26 are also not the Linux meaning of those integers:
Immediate / x8 | Linux AArch64 (v6.10 generic) | ctos |
|---|---|---|
| 0 | io_setup | ADR-013 first-mile return (reserved) |
| 1 | io_destroy | ADR-013 standing announce (reserved) |
| 2 | io_cancel | ADR-013 standing restore (reserved) |
| 16 | fremovexattr | exit |
| 17 | getcwd | uart_write |
| 18 | lookup_dcookie | yield |
| 19 | eventfd2 | fs_create |
| 20 | epoll_create1 | fs_open |
| 21 | epoll_ctl | fs_read |
| 22 | epoll_pwait | fs_write |
| 23 | dup | fs_close |
| 24 | dup3 | net_mac |
| 25 | fcntl | net_ping |
| 26 | inotify_init1 | net_udp_dns |
You cannot “just accept Linux numbers” in the SVC immediate, and you cannot treat today’s 16–26 as Linux.
ctos surface (inspection)
Reserved 0–2 are probe-only. Public:
| ctos # | Name | Related Linux name(s) | Why not present |
|---|---|---|---|
| 16 | exit | exit (93), exit_group (94) | Ends the standing EL0 trip; no process table; not exit_group. |
| 17 | uart_write | write (64) to a TTY | PL011 only; no fd; cap 64; \n → \r\n. |
| 18 | yield | sched_yield (124) | Dispatch-only hint; does not run EL1 sched::yield_now (ADR-010). |
| 19 | fs_create | openat (56) + O_CREAT | Flat memfs name; no dirfd, flags, or mode. |
| 20 | fs_open | openat (56) | Existing name only; FAT /probe and /hello are the known on-disk files. |
| 21 | fs_read | read (63) | Handle 1–4; cap 64; no stdin. UART RX is kernel-side, not this SVC. |
| 22 | fs_write | write (64) | memfs only (FAT write is ReadOnly); cap 64; not fd 1/2. |
| 23 | fs_close | close (57) | Drops the handle; the file stays. Four handles max. |
| 24 | net_mac | — | Guest MAC copy; not Linux dup3 (24). Kernel owns NIC (ADR-068). |
| 25 | net_ping | — | Quiet ICMP echo; not Linux fcntl (25). |
| 26 | net_udp_dns | — | Quiet UDP DNS probe; not Linux inotify_init1 (26). DNS bait only (ADR-071). |
Unknown immediates park. Not POSIX. Not Linux VFS.
Gap table (representative Linux AArch64 set)
Numbers from Linux v6.10 asm-generic/unistd.h. AArch64 has no fork, open, creat, dup2, pipe, poll, or select syscalls — those are clone / openat / dup3 / pipe2 / ppoll / pselect6. __NR_syscalls in that header is 463. This table is a slice, not the full list.
Process and lifetime (B3)
| Linux # | Name | ctos status | Notes |
|---|---|---|---|
| 93 | exit | partial | Analog SYS_EXIT 16. Halts the EL0 trip and returns to the EL1 caller. Status is recorded. Not a thread/process exit. |
| 94 | exit_group | absent | No process group. musl/glibc CRT usually ends here. |
| — | fork | absent | Not in the Linux AArch64 table. Userspace fork() is clone. Stance: ADR-035 / B3 / #43. |
| 220 | clone | absent | No child address space, no CLONE_*. Standing EL0 (ADR-024) is one loaded trip, not clone (ADR-035). CLONE_NEW* is never-per-ADR-031 (see namespace rows). |
| 435 | clone3 | absent | Same class as clone. |
| 221 | execve | absent | A3 maps freestanding PT_LOAD and rejects PT_INTERP. A9 FAT /hello is an EL1 load path, not execve (ADR-035). ELF / auxv / interpreter gap: ADR-033 (B4). |
| 281 | execveat | absent | Same class as execve. |
| 260 | wait4 | absent | No child to wait for. ADR-035. |
| 95 | waitid | absent | ADR-035. |
| 129 | kill | absent | No pid / signal delivery. |
| 172 | getpid | absent | No process table. |
| 178 | gettid | absent | No tid. |
| 96 | set_tid_address | absent | Typical musl/glibc _start requirement. |
File descriptors and I/O (B5 concept compare: ADR-034)
| Linux # | Name | ctos status | Notes |
|---|---|---|---|
| 56 | openat | partial | fs_open 20 / fs_create 19. Path is / + [a-z0-9_-] (1–15). No dirfd, O_*, or mode. At most 8 files. ADR-027. |
| 437 | openat2 | absent | No open_how. |
| 57 | close | partial | fs_close 23. File stays after close. |
| 63 | read | partial | fs_read 21. Cap 64. No fd 0 / line discipline. |
| 64 | write | partial | uart_write 17 (console, no fd) or fs_write 22 (memfs). Neither is POSIX write(1, …). |
| 65 / 66 | readv / writev | absent | No iovec. |
| 67 | pread64 | absent | Handle offset exists inside VFS; no positioned SVC. |
| 62 | lseek | absent | No seek SVC. |
| 29 | ioctl | absent | No TTY (TCGETS / TIOCGWINSZ), no block ioctl. PL011 is kernel-owned. |
| 25 | fcntl | absent | No F_GETFL / F_DUPFD. |
| 23 / 24 | dup / dup3 | absent | Handles 1–4 only; no dup. Linux 23 is dup; ctos 23 is fs_close. |
| 59 | pipe2 | absent | No pipes. |
| 61 | getdents64 | absent | No directory listing. Flat namespace. |
| 79 / 80 | newfstatat / fstat | absent | No stat. |
| 291 | statx | absent | No stat. |
| 48 | faccessat | absent | |
| 34 | mkdirat | absent | No directories. |
| 35 | unlinkat | absent | |
| 38 | renameat | absent | |
| 49 | chdir | absent | No cwd. |
| 17 | getcwd | absent | Linux 17 is getcwd; ctos 17 is uart_write. |
Memory (loader / libc heap)
| Linux # | Name | ctos status | Notes |
|---|---|---|---|
| 214 | brk | absent | Kernel first-fit heap is EL1 (ADR-009). No user program break. |
| 222 | mmap | absent | A3 maps PT_LOAD at load time into the user window. No mmap SVC. Anonymous / file maps are out. |
| 215 | munmap | absent | |
| 226 | mprotect | absent | Image W^X is a kernel policy (ADR-015), not an EL0 syscall. |
| 233 | madvise | absent |
Scheduling, time, sync, identity
| Linux # | Name | ctos status | Notes |
|---|---|---|---|
| 124 | sched_yield | partial | Analog yield 18 is a recorded hint and ERET to EL0. It does not switch EL1 tasks. |
| 101 | nanosleep | absent | CNTP tick is EL1. |
| 113 | clock_gettime | absent | No guest clock SVC. CNTPCT probes are kernel/NFR-07. |
| 98 | futex | absent | No user futex. |
| 99 | set_robust_list | absent | Typical libc _start. |
| 134 / 135 / 139 | rt_sigaction / rt_sigprocmask / rt_sigreturn | absent | Lower-EL IRQ still parks. No POSIX signals. |
| 73 | ppoll | absent | |
| 160 | uname | absent | |
| 167 | prctl | absent | |
| 163 / 261 | getrlimit / prlimit64 | absent | |
| 278 | getrandom | absent |
Sockets (not a Track B promise)
| Linux # | Name | ctos status | Notes |
|---|---|---|---|
| 198 | socket | absent | No virtio-net, no stack. Same class as the host-apps “sockets” gap. Not a B2 implementation item. |
Namespace / mount — never on this track
| Linux # | Name | ctos status | Notes |
|---|---|---|---|
| 97 | unshare | never-per-ADR-031 | Linux namespaces. Guest OCI is a non-goal (ADR-029). |
| 268 | setns | never-per-ADR-031 | Same. |
| 40 | mount | never-per-ADR-031 | Linux mount/overlay class. A7 FAT is not mount(2). Concept compare: ADR-034 (B5). Must not grow POSIX mount. |
| 39 | umount2 | never-per-ADR-031 | Same. |
| 41 | pivot_root | never-per-ADR-031 | Same. |
clone with CLONE_NEWNS / CLONE_NEWPID / CLONE_NEWNET is the same never-per-ADR-031 class. A later clone subset, if B6 ever names one, would still exclude those flags.
If a subset is ever useful (not a decision)
Issue #42 asked to prioritize a minimal subset if any. This is a research hint for B6, not an implementation list.
- In-tree
libctosapps need no Linux numbers. A1–A9 already haveexit/uart_write/yield/ memfs / FAT/hello. Keep that path. - A static musl “hello” is not a small subset. Besides convention translation, typical CRT wants
set_tid_address,exit_group,brkormmap, oftenwriteon fd 1,uname/geteuid-class probes, and signal/prctlstubs. That is already a libc-shaped surface. ADR-031 rejected “runs musl / Alpine” as a Track B promise. - The smallest named research slice (only if B6 chooses a compat path, and only with a probe) would still be: convention translation +
exit/exit_group+writeto a console fd.openat/read/closeare the next file slice.brk/mmapare the next heap slice.clone/execve/wait4are the process gap (ADR-035); they are not a small add-on. - Do not start with ioctl, sockets, or mount. ioctl is a device encyclopedia. Sockets need a stack. mount/unshare stay never-per-ADR-031.
B3 is Documented (ADR-035). B4 is Documented (ADR-033). B5 is Documented (ADR-034). They must not add Linux numbers to src/. B6 may choose never.
Honesty
- Analysis Verified means: this file was read against
src/syscall.rsand Linux v6.10unistd.h. It is not ABI Verified and not Linux userspace. - Do not say “ctos implements
write(2)” becauseuart_writeorfs_writeexists. - Do not mint FR-16+ / NFR-15+. Frozen set: fr-nfr.md.
- Guest containers stay non-goal. Host
docker-smoke.shis unrelated.
Roadmap: track-b.md. Extra stance: host-apps.md.
Core principles (driving force)
The project name is ctos. ctsOS is an optional display nickname only — use ctos in docs, issues, and code.
These principles are the driving force. They are non-negotiable. Track A, Track B, and A9 are subordinate: they never override principles for speed or marketing.
| Principle | What it means | Home |
|---|---|---|
| Honesty ledger | Status words need a probe. Unprobed stays Unknown. No rounding up. | honesty-ledger.md |
| Antifragility | Fail-closed sensors. Repeated failures become ratchets, not README advice. | antifragility.md / NFR-05 |
| Security | Written threat model. Mitigations are claimed only when probed. No “secure OS.” | security.md / NFR-10 |
| Performance | Measure first. No invented benches. Optimize only after a probe. | performance.md / NFR-07 |
| Document-first / one-PR loops | One milestone → one branch → one GitHub PR. Author ≠ merger. | ADR-002, roadmap |
Landing page (same list, site SoT): Driving principles. Hub: overview.md. Pillars: pillars.md. Vision: product-vision.md.
ctos harness map
A reliable kernel session is:
Shared understanding + domain + outer harness.
That three-part split is common harness-engineering language (prior art: architecture.artof.link). On ctos it is not a product name. It is a map for this repo.
Shared understanding
The session’s reviewable model of intent and state:
- Vision, architecture, ADRs, roadmap
- Honesty ledger (what we claim vs what we probed)
- Daily briefs and random-thoughts (handoff, not source of truth)
If a claim is only in chat, it is not shared understanding.
After M9, shared understanding also names the three pillars (antifragility, security, performance). A “secure” or “faster” sentence without a ledger row is not shared understanding.
Domain
The thing that is allowed to decide what is true about the machine:
- Kernel source and the custom AArch64 target
- QEMU
virt(and later real hardware, if probed) - Kernel ELF loaded by
-kernel - CPU, UART, interrupt, and memory behavior
Agents interpret. The domain (a build, a QEMU probe, a test) decides. Do not “confirm boot” from a README sentence.
Outer harness (six layers, kernel-scaled)
| Layer | On ctos | Prevents |
|---|---|---|
| Guides | AGENTS.md, .cursor/rules/, .cursor/skills/ctos-* | Out-of-scope work (wrong crate era, shop content, GitLab SOPs) |
| Sensors | scripts/qemu-smoke.sh, cargo test semihosting exit, GHA smoke.yml, optional Docker | Rounding “source exists” up to “it boots” |
| Loop | One milestone → one branch → one GitHub PR | Sprawling PRs that mix UART, paging, and docs rewrites |
| Memory | research/ vaults | Session amnesia; stuffing raw chat into the next prompt |
| Permissions | Author ≠ merger (artofdream vs cursor[bot]); MRC writes COMMENT; no GitHub self-APPROVE (ADR-002) | Same-login stamp counted as a second review |
| Observability | Honesty ledger + PR text that lists probed vs Unknown | Status theater |
Fail closed: if QEMU was not run, boot status is Unknown. Do not write Verified.
ctos honesty ledger
Status words are claims. Each row needs a probe. Unprobed = Unknown. Never round Unknown up to Verified because the file exists, a PR is open, or a README teaches cargo run.
Allowed flags: Verified (probe passed), Unknown (no probe or probe blocked), Planned (not built yet), Failed (probe ran and lost).
| Claim | Probe | Status | Notes |
|---|---|---|---|
UART hello source present (println!("Hello World!"); + PL011 writer) | Read src/main.rs and src/uart.rs on this branch | Verified | Source inspection. Boot is a separate row. |
.cargo/config.toml present (build-std, aarch64-ctos.json, qemu runner, json-target-spec) | Read .cargo/config.toml | Verified | Presence. Build success is the next row. |
cargo build for aarch64-ctos.json | cargo +nightly build on 2026-09-08 (rustc 1.100.0-nightly cea272fa3) | Verified | First attempt failed: invalid aarch64 ABI combination until the JSON had both "abi": "softfloat" and "rustc-abi": "softfloat". Then the ELF built (target/aarch64-ctos/debug/ctos, AArch64, entry 0x40080000). |
| QEMU aarch64 serial shows Hello World | qemu-system-aarch64 8.2.2, -machine virt -cpu cortex-a57 -display none -serial stdio -kernel target/aarch64-ctos/debug/ctos; timeout 4 | Verified | Serial printed Hello World! then the VM was killed (exit 124). Same string on -machine virt,gic-version=3 and -cpu max. One cloud environment, not CI. Not a Raspberry Pi probe. |
| x86_64 VGA / bootimage path | Historical probes on 2026-09-08 (PR #2) | Historical | Path removed by ADR-003. Those Verified rows do not apply to this tree. |
| CI on GitHub | Workflow file exists and a run is green | Verified | .github/workflows/smoke.yml: push run 34285784458 success (ubuntu-24.04 52s, ubuntu-24.04-arm 1m9s). PR run 34285786641 success. ubuntu-24.04-arm label is available on this repo. |
Docker smoke (Dockerfile / scripts/docker-smoke.sh) | the arm64 workstation Docker Desktop linux/arm64, 2026-09-09: docker build -t ctos-smoke . && docker run --rm ctos-smoke → exit 0 | Verified | After three Failed runs, ratchets landed in git and the full smoke passed on the arm64 workstation: serial Hello World! (hello QEMU timeout 124, expected); cargo test [ok]; force-fail exit 1; qemu-smoke: ok. Failures that were ratcheted: (1) CRLF shebang → exec ./scripts/qemu-smoke.sh: no such file or directory (.gitattributes *.sh/Dockerfile eol=lf, image sed, CMD bash); (2) linker cc not found on compiler_builtins (build-essential); (3) failed to find romfile "efi-virtio.rom" (qemu-efi-aarch64 + ipxe-qemu). One sponsor host, not CI and not this cloud VM (still no Docker engine here). |
Second-brain vaults (research/) | Paths exist; README explains Procedure / Correction / Relationship / Daily Brief | Verified | Structure present. Not a claim that vaults are richly filled. Optional Obsidian UI is structure-only (PR #3). |
Thin ctos-* roles | AGENTS.md + .cursor/skills/ctos-*/SKILL.md exist | Verified | Four roles. No aea-* names. |
Frozen FR/NFR IDs (FR-01–FR-15, NFR-01–NFR-14) | Read docs/02-requirements/fr-nfr.md; IDs present; ISA text revised under ADR-003 | Verified | Frozen IDs. Text now AArch64/UART. Do not invent extra FR/NFR IDs in chat. File presence is not a claim that every Now row is implemented. |
PR identity split (author ≠ merger; artofdream vs cursor[bot]) | Read ADR-002 (amended 2026-09-12), AGENTS.md, .cursor/rules/pr-identity-no-self-merge.mdc | Verified | Docs present. Principle reused from Café Fausse pr-coordinator (identity only). Going-forward rule: owner-opened → cursor[bot] merges; bot-opened → artofdream merges. File presence of the rule is not a practiced split. Distinct-identity merge on this repo (author ≠ mergedBy): Unknown until gh shows it on a post-amendment PR. cursor[bot] merge / App APPROVE on this repo: Unknown until probed on artofdream/ctos. Track A same-login merges are the Historical row below, not this probe. |
| Track A A1–A9 same-login merge (ADR-002 historical exception) | gh pr view --json author,mergedBy on 2026-09-12 for #49, #52–#57, #59, #60 | Historical | Each PR: author=artofdream and mergedBy=artofdream. Commits typically cursoragent + Co-authored-by: artofdream. Named exception (2026-09-11–2026-09-12); do not rewrite those PRs. Not a kernel-boot claim. Not permission to repeat. |
| Primary ISA is AArch64 | Read ADR-003; x86_64-ctos.json / src/vga_buffer.rs absent | Verified | Decision + file tree. QEMU boot is a separate row. |
| Integration tests / QEMU test exit (M2) | cargo +nightly test → QEMU virt + -semihosting; #[test_case]; QEMU host exit 0 | Verified | 2026-09-08 cloud: Running 2 tests / [ok]. 2026-09-09 M3: 4 tests. 2026-09-09 M4 re-probe: Running 6 tests / all [ok] (adds SPSel + stack-range cases). |
| Fail-closed test panic | cargo +nightly test --features force-fail | Verified | 2026-09-08 and 2026-09-09 cloud: panic force-fail, QEMU/host exit 1. |
scripts/qemu-smoke.sh | Ran on this cloud VM | Verified | 2026-09-09 M4 (qemu-system-aarch64 8.2.2, rustc 1.100.0-nightly 4aa1fbcf4): hello (timeout 124), exception: sync BRK (esr=0xf2000000), exception: fatal nested (kind=0x200), cargo test 0 (6 cases), force-fail 1, qemu-smoke: ok. First hello run Failed: MSR SP_EL1 at EL1 is UNDEF (esr=0x2000000) before Hello World! — fixed by setting SP_EL1 via mov sp while SPSel=1 (ADR-005). Not Docker. GHA on this PR is a separate row. |
VBAR_EL1 installed at 2 KiB-aligned exception_vectors (M3 / FR-06) | cargo +nightly test vbar_el1_points_at_table | Verified | 2026-09-09 cloud: [ok]. CurrentEL==1 and mrs vbar_el1 equals exception_vectors. ELF also had the table at 0x40080800. |
Current-EL sync BRK handler runs and returns (M3 / FR-06) | cargo +nightly test breakpoint_from_current_el; hello-kernel serial | Verified | 2026-09-09 cloud: test [ok]; hello serial exception: sync BRK esr=0xf2000000 elr=0x40081190 then timeout 124 (wfe loop). Other sync classes and lower-EL slots are parks — taking those is unprobed. |
| qemu-smoke requires BRK handler string | scripts/qemu-smoke.sh hello phase greps exception: sync BRK | Verified | 2026-09-09 cloud: qemu-smoke: BRK handler string present. Extends the M2 sensor (FR-06 / NFR-04). |
| Lower-EL AArch64 sync is live (SVC / IABORT); IRQ while standing live (ADR-040/041); FIQ while standing live (ADR-043); SError park | Source read of vector table; taken path is the EL0 first-mile / ADR-040 / ADR-041 / ADR-043 rows | Verified | Source: sync_lower_el + irq_lower_el + fiq_lower_el; default eret_to_el0 SPSR 0x340. SError lower-EL is standing-aware (ADR-045 EXPECT) but no working host inject on virt/a57; AArch32 still parks. Taken lower-EL FIQ: see ADR-043 row. Taken SError: see ADR-045 row. |
| CI on GitHub (M3 PR / this branch) | .github/workflows/smoke.yml on cursor/m3-vbar-el1-exceptions-bd73 | Verified | Push 34389110709 success (ubuntu-24.04 + ubuntu-24.04-arm). PR 34389132580 success. Same qemu-smoke.sh as the cloud probe (hello + BRK string + 4 tests + force-fail). Commit 72ecd37. |
Dedicated SP_EL1 exception stack + SPSel=0 thread stack (M4 / FR-07) | cargo +nightly test spsel_uses_thread_stack + stacks_are_distinct_and_aligned | Verified | 2026-09-09 cloud: both [ok]. SPSel==0 and current SP in __stack_*; three stack ranges 16-byte aligned and disjoint. |
| Nested current-EL exception uses fatal stack and is observable (M4 / FR-07) | Hello-kernel serial exception: fatal nested via scripts/qemu-smoke.sh | Verified | 2026-09-09 cloud: after two exception: sync BRK lines, serial exception: fatal nested then exception: fatal esr=0xf2000000 elr=0x400816cc kind=0x200. Nested AArch64 BRK after near-empty thread SP (ADR-005). Not an MMU overflow fault. Not GIC (M5). |
| qemu-smoke requires fatal nested string | scripts/qemu-smoke.sh hello phase greps exception: fatal nested and rejects fatal probe missed | Verified | 2026-09-09 cloud: qemu-smoke: fatal nested string present. Extends the M3 sensor (FR-07 / NFR-04). |
| CI on GitHub (M4 PR / this branch) | .github/workflows/smoke.yml on cursor/m4-fatal-exception-stack-c8b7 | Unknown | Not a green GHA URL on the post-fix revision before merge. M4 merged as PR #7 (180dbf2). |
| GICv2 init + CNTP PPI 30 tick observable (M5 / FR-08) | Hello-kernel serial timer: tick via scripts/qemu-smoke.sh; #[test_case] timer_tick_is_observable | Verified | 2026-09-09 cloud (QEMU 8.2.2, rustc 1.100.0-nightly 4aa1fbcf4): after Hello World!, serial timer: tick, then M3/M4 BRK + fatal. cargo test Running 9 tests all [ok] including gicd_typer_readable, cntfrq_is_nonzero, timer_tick_is_observable. Not GICv3. Not UART input (M6). |
| qemu-smoke requires timer tick string | scripts/qemu-smoke.sh hello phase greps timer: tick and rejects timer: tick missed | Verified | 2026-09-09 cloud: qemu-smoke: timer tick string present. Extends the M4 sensor (FR-08 / NFR-04). |
| CI on GitHub (M5 PR / this branch) | .github/workflows/smoke.yml on cursor/m5-hardware-interrupts-d5a6 | Verified | Push/PR run 34391557761 success (ubuntu-24.04 + ubuntu-24.04-arm) on the cloud-probe commit. Implementation commit run 34391552019 also success. Same qemu-smoke.sh (hello + tick + BRK + fatal + 9 tests + force-fail). Commit e28e4bb. |
PL011 RX of injected 0x41 observable (M6 / FR-08 input) | Hello-kernel serial input: rx 0x41 via scripts/qemu-smoke.sh + qemu-serial-inject.py | Verified | 2026-09-09 cloud (QEMU 8.2.2, rustc 1.100.0-nightly 4aa1fbcf4): after Hello World! and timer: tick, serial input: rx 0x41, then M3/M4 BRK + fatal. Not virtio-keyboard. Not QEMU LBE (unimplemented on 8.2). |
| qemu-smoke requires UART RX string | scripts/qemu-smoke.sh hello phase greps input: rx 0x41 and rejects input: rx missed | Verified | 2026-09-09 cloud: qemu-smoke: UART RX string present. Extends the M5 sensor (FR-08 input / NFR-04). |
Empty RX FIFO under cargo test (M6) | #[test_case] uart_rx_fifo_empty_without_host_byte | Verified | 2026-09-09 cloud: cargo test Running 10 tests all [ok]. Character proof is the serial row. |
| CI on GitHub (M6 PR / this branch) | .github/workflows/smoke.yml on cursor/m6-uart-rx-input-7ef6 | Unknown | No green GHA URL on the cloud-probe commit yet. |
| EL1 identity map + MMU on (M7 / FR-09) | Hello-kernel serial paging: ok via scripts/qemu-smoke.sh; #[test_case] mmu_is_enabled | Verified | 2026-09-09 cloud (QEMU 8.2.2, rustc 1.100.0-nightly 4aa1fbcf4): after Hello World!, serial paging: ok, then M5/M6/M3/M4 markers. cargo test mmu_is_enabled [ok]. Identity 1 GiB blocks (ADR-008). Not a DTB walk. |
| Frame alloc + map/unmap window (M7 / FR-09) | #[test_case] frame_alloc_aligned_and_distinct + map_unmap_roundtrip; serial marker from paging::observe_probe | Verified | 2026-09-09 cloud: both [ok]. Write-through window VA 0x8000_0000 matches identity PA. |
| qemu-smoke requires paging string | scripts/qemu-smoke.sh hello phase greps paging: ok and rejects paging: probe missed | Verified | 2026-09-09 cloud: qemu-smoke: paging string present. Extends the M6 sensor (FR-09 / NFR-04). |
| CI on GitHub (M7 PR / this branch) | .github/workflows/smoke.yml on cursor/m7-paging-frame-allocator-b567 | Unknown | No green GHA URL on the cloud-probe commit yet. |
GlobalAlloc + Box/Vec on identity-mapped frames (M8 / FR-10) | Hello-kernel serial heap: ok via scripts/qemu-smoke.sh; #[test_case] box_alloc_roundtrip + vec_grows + heap_lives_in_frame_pool | Verified | 2026-09-09 cloud (QEMU 8.2.2, rustc 1.100.0-nightly 4aa1fbcf4): after paging: ok, serial heap: ok, then M5/M6/M3/M4 markers. cargo test Running 17 tests all [ok]. First-fit + coalesce (ADR-009). Not a growing heap. Not M9. |
| qemu-smoke requires heap string | scripts/qemu-smoke.sh hello phase greps heap: ok and rejects heap: probe missed | Verified | 2026-09-09 cloud: qemu-smoke: heap string present. Extends the M7 sensor (FR-10 / NFR-04). |
| CI on GitHub (M8 PR / this branch) | .github/workflows/smoke.yml on cursor/m8-heap-globalalloc-0ee8 | Verified | Push run 34394697056 success (ubuntu-24.04-arm 59s, ubuntu-24.04 1m36s) on the cloud-probe commit ba6096a. Same qemu-smoke.sh (hello + paging + heap + tick + RX + BRK + fatal + 17 tests + force-fail). |
| Cooperative two-task yield (M9 / FR-11) | Hello-kernel serial sched: task a / sched: task b / sched: ok via scripts/qemu-smoke.sh; #[test_case] two_tasks_run_on_distinct_heap_stacks + yield_round_robin_resumes_both | Verified | 2026-09-09 cloud (QEMU 8.2.2, rustc 1.100.0-nightly 4aa1fbcf4): after heap: ok, serial sched: task a, sched: task b, sched: ok, then M5/M6/M3/M4 markers. cargo test Running 19 tests all [ok]. Cooperative EL1 yield (ADR-010). Not preemptive. Not SMP. |
| qemu-smoke requires scheduler strings | scripts/qemu-smoke.sh hello phase greps sched: ok and both task markers; rejects sched: probe missed | Verified | 2026-09-09 cloud: qemu-smoke: scheduler strings present. Extends the M8 sensor (FR-11 / NFR-04). |
| qemu-smoke requires CNTPCT perf string | scripts/qemu-smoke.sh hello phase greps perf: cntpct and rejects perf: probe missed | Verified | 2026-09-09 cloud: qemu-smoke: CNTPCT perf string present. Extends the M9 sensor (NFR-07 / NFR-04). |
| CI on GitHub (M9 PR / this branch) | .github/workflows/smoke.yml on cursor/m9-cooperative-scheduler-7da9 | Verified | Folded from #16 (not re-probed here): push 34395786658 and PR 34395791288 success (ubuntu-24.04-arm + ubuntu-24.04) on 88a9305. Merge-commit push 34396135202 success on 4e3b732. Same qemu-smoke.sh (hello + paging + heap + two-task sched + tick + RX + BRK + fatal + 19 tests + force-fail). |
| Three pillars accepted (ADR-011); NFR-05 / NFR-07 / NFR-10 text revised in place | Read ADR-011 and fr-nfr.md; IDs still NFR-05/NFR-07/NFR-10 | Verified | Document inspection. Not a kernel boot claim. |
| Threat-model v1 exists (NFR-10) | Read security.md (assets, adversaries, trust boundaries, non-goals, mitigations→probes) | Verified | File + review (2026-09-10, #18). Replaces the ADR-011 stub. “Secure OS” / “hardened”: unclaimed. |
| Threat-model v1.1 slice (NFR-10) | Read security.md (linker stacks, guards, EL0 first mile, remaining W^X gaps) | Verified | File + review. v1.1 update, not “secure.” |
| Threat-model v1.5 slice (NFR-10) | Read security.md (standing EL0, TTBR1 first cut, EL1 high-VA fetch, PAN/identity teardown still Planned) | Verified | File + PR #25. v1.5 update, not “secure.” |
| Threat-model v1.6 slice (NFR-10) | Read security.md (identity-tear first cut, PAN/full teardown still Planned) | Verified | File + PR #26. v1.6 update, not “secure.” |
| Threat-model v1.7 slice (NFR-10) | Read security.md (high-VA jump + 16 KiB dedicated identity text range, live .text stays, PAN/full teardown still Planned) | Verified | File + PR #27 (on main 24d94e6). v1.7 update, not “secure.” |
| Threat-model v1.8 slice (NFR-10) | Read security.md (high-VA vtable rewrite + live identity .text tear, .rodata/.data/heap stay, PAN/full teardown still Planned) | Verified | File + PR #28 (on main e80dc93). v1.8 update, not “secure.” |
| W^X / NX heap + coop stacks (ADR-012) | Hello-kernel serial wx: nx heap + wx: ok via scripts/qemu-smoke.sh; #[test_case] heap_and_mmio_are_pxn + kernel_text_is_executable + execute_from_heap_is_caught | Verified | 2026-09-10 cloud (QEMU 8.2.2, rustc 1.100.0-nightly a36d05efa): after sched: ok, serial wx: nx heap then wx: ok. cargo test Running 25 tests all [ok]. L2/L3 PXN on [__kernel_end, RAM end) (ADR-012). Linker SP_EL0 / SP_EL1 / fatal stacks stay in the executable image. Device MMIO L1 is XN. Not a “secure OS” or “the kernel is W^X” claim. |
| qemu-smoke requires W^X string | scripts/qemu-smoke.sh hello phase greps wx: ok and rejects wx: probe missed | Verified | 2026-09-10 cloud: qemu-smoke: W^X string present. Extends the pillars sensor (NFR-10 / NFR-04). |
| Baseline CNTPCT loop probe (NFR-07) | Hello-kernel serial perf: cntpct via scripts/qemu-smoke.sh; #[test_case] cntpct_advances_over_loop | Verified | 2026-09-10 cloud re-probe (QEMU 8.2.2, rustc 1.100.0-nightly a36d05efa): after wx: ok, serial perf: cntpct delta=22968. Prior 2026-09-09 pillars probe also Verified. Counter-advances baseline. Not a published bench. |
| IRQ-to-handler CNTPCT delta (NFR-07) | Hello-kernel serial perf: irq-delta via scripts/qemu-smoke.sh; #[test_case] irq_delta_samples_recorded | Verified | 2026-09-10 cloud (QEMU 8.2.2): after timer: tick, serial perf: irq-delta min=12973 max=35391 spread=22418 n=8. cargo test irq_delta_samples_recorded [ok]. CNTPCT−CVAL spread on this QEMU virt guest. Not a latency budget. Not “faster than X.” |
| qemu-smoke requires IRQ-delta string | scripts/qemu-smoke.sh hello phase greps perf: irq-delta and rejects perf: irq-delta missed | Verified | 2026-09-10 cloud: qemu-smoke: IRQ-delta perf string present. Extends the NFR-07 sensor (NFR-04). |
| Linker-stack guard pages (ADR-014) | Hello-kernel serial guard: fault + guard: ok via scripts/qemu-smoke.sh; #[test_case] linker_stack_guards_unmapped + store_to_thread_guard_is_caught | Verified | 2026-09-10 cloud (QEMU 8.2.2, rustc 1.100.0-nightly a36d05efa): after wx: ok, serial guard: fault then guard: ok. cargo test both guard cases [ok]. Unmapped 4 KiB holes (ADR-014). Live stack pages stay executable. Not “the kernel is W^X.” |
| qemu-smoke requires guard string | scripts/qemu-smoke.sh hello phase greps guard: ok and rejects guard: probe missed | Verified | 2026-09-10 cloud: qemu-smoke: stack-guard string present. |
| Host debug ELF size (NFR-08) | scripts/qemu-smoke.sh prints perf: elf-size bytes=<n> and rejects < 4096 | Verified | 2026-09-10 cloud: host perf: elf-size bytes=3752056 after cargo +nightly build. Measurement only. Not a budget. Not a bench. |
| EL0 entered and returned (SVC) | Hello-kernel serial el0: svc + el0: ok; #[test_case] el0_svc_roundtrip | Verified | 2026-09-10 cloud: after guard: ok, serial el0: svc then el0: ok. cargo test el0_svc_roundtrip [ok]. is_active() stays false. Not userspace. |
| EL0 cannot execute kernel data | Hello-kernel serial el0: nx kernel; #[test_case] el0_cannot_execute_kernel_data | Verified | 2026-09-10 cloud: serial el0: nx kernel. cargo test [ok]. Lower-EL UXN IABORT on kernel .data. 2026-09-11 hello re-probe (this branch) still prints el0: nx kernel (now also a translation IABORT under user TTBR0). Not isolation. |
| EL0 isolation (P-SEC-3) | Standing EL0 + PAN enable + full TTBR1/higher-half teardown; since M6: every ADR-055 §A/§B row Met + sponsor’s written accept (B3) + M6 PR CI re-probe | Verified (scoped: the accepted ADR-091 sentence only) | M6 (2026-09-28): ADR-091 Accepted; “EL0 isolated” means only that sentence, with its limits (one core, the TCG default smoke machine; not a speculative-execution or side-channel claim, not a real-hardware claim, not a certification; app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN); no multi-core race story). Revision 4 (2026-10-02): the sponsor accepted ADR-091 rev 4 in writing (PAN clause + that limit changed; see the revision-4 row near the end of this file). Earlier notes: Direction ADR-013; decisions ADR-047; checklist ADR-055; sponsor closure ADR-060. Specific miles are different rows. PAN enable Verified on a76 (ADR-080; historical a57 lock ADR-054); taken SError hard-stopped (ADR-053 / ADR-081); never yank _start. Do not widen “EL0 isolated” past the ADR-091 sentence. |
| RO+NX text/data (ADR-015) | Hello-kernel serial ro: nx data + ro: write fault + ro: ok; #[test_case] execute-from-.data + write-to-RO-text | Verified | 2026-09-11 cloud qemu-smoke (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e, -cpu cortex-a57): after guard: ok, serial ro: nx data then ro: write fault then ro: ok. cargo test both RO cases [ok]. Identity image W^X on this virt guest (text RO+X, data/stacks/heap RW+NX, SCTLR.WXN on). Not a “secure OS.” |
| qemu-smoke requires RO+NX strings | scripts/qemu-smoke.sh hello phase greps ro: ok / ro: nx data / ro: write fault | Verified | 2026-09-11 cloud: qemu-smoke: RO+NX string present. |
| Boot-to-ready CNTPCT (NFR-08) | Hello-kernel serial perf: boot-delta; #[test_case] boot_delta_sample_exists | Verified | 2026-09-11 cloud qemu-smoke: perf: boot-delta ticks=111019 (after paging::init → after Hello World!). First attempt Failed: pre-MMU .bss store lost on the test image. Measurement only. Not a budget. Not a bench. |
| qemu-smoke requires boot-delta string | scripts/qemu-smoke.sh hello phase greps perf: boot-delta | Verified | 2026-09-11 cloud: qemu-smoke: boot-delta perf string present. |
User TTBR0 omits kernel .data | #[test_case] user_ttbr0_omits_kernel_data; walk L1_USER | Verified | 2026-09-11 cloud cargo test: user_ttbr0_omits_kernel_data [ok]. ASID=1 programmed. Not ASID isolation. |
EL0 cannot read kernel .data | Hello-kernel serial el0: no kernel read; #[test_case] el0_cannot_read_kernel_data | Verified | 2026-09-11 cloud qemu-smoke: after el0: nx kernel, serial el0: no kernel read then el0: ok. cargo test [ok]. User TTBR0 omits .data. is_active() stays false. Not isolation. Not PAN. |
| PAN enable on virt | ID_AA64MMFR1_EL1.PAN != 0 and an EL1-vs-EL0 access fault | Verified (ADR-080) | Default -cpu cortex-a76 (ADR-079 / ADR-080 / #125): pan: present + pan: enabled + pan: el1-fault (SPSR.PAN). Historical a57 pan: absent (ADR-026). Not “EL0 isolated.” |
| ASID isolation | Hello-kernel serial asid: dual + asid: conflict + asid: ok; #[test_case] asid_isolation_without_vmalle1; no TLBI VMALLE1 on the switch | Verified | 2026-09-11 cloud qemu-smoke (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e, -cpu cortex-a57): after el0: ok, serial asid: dual then asid: conflict then asid: ok. cargo test asid_isolation_without_vmalle1 [ok] (Running 39 tests). Dual TTBR0 (ASID 1 vs 2) with nG pages; switch is MSR TTBR0 + ISB. At probe time the EL0 trampoline still TLBI VMALLE1 (.data leaves were global); ADR-039 later drops trampoline VMALLE1. Not “EL0 isolated.” Not PAN. |
| Standing EL0 context | Hello-kernel serial el0: standing + el0: restored; #[test_case] standing_el0_enter_leave; is_active() true only while standing | Verified | 2026-09-11 cloud qemu-smoke (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e, -cpu cortex-a57): after el0: no kernel read, serial el0: standing then el0: restored then el0: ok. cargo test standing_el0_enter_leave [ok] (Running 42 tests). Dual-SVC on user TTBR0 (not a trampoline flag). Default ERET clears IRQ mask (ADR-041). Not POSIX. Not isolation. |
| TTBR1 kernel-private page (ADR-016 first cut) | Hello-kernel serial ttbr1: el1 + ttbr1: no el0 + ttbr1: ok; #[test_case] ttbr1_el1_sees_priv_el0_does_not | Verified | 2026-09-11 cloud qemu-smoke: after asid: ok, serial ttbr1: el1 then ttbr1: no el0 then ttbr1: ok. cargo test ttbr1_el1_sees_priv_el0_does_not [ok]. EL1-only high page. Identity teardown Planned. Not a relocated kernel. Not “EL0 isolated.” |
| qemu-smoke requires standing EL0 strings | scripts/qemu-smoke.sh hello phase greps el0: standing / el0: restored | Verified | 2026-09-11 cloud: qemu-smoke: EL0 first-mile + read-mile + standing strings present. |
| qemu-smoke requires TTBR1 strings | scripts/qemu-smoke.sh hello phase greps ttbr1: ok / ttbr1: el1 / ttbr1: no el0; rejects ttbr1: leaked and ttbr1: probe missed | Verified | 2026-09-11 cloud: qemu-smoke: TTBR1 private-page strings present. |
| EL1 fetch from TTBR1 high VA (ADR-017) | Hello-kernel serial ttbr1: el1 exec + ttbr1: vbar; #[test_case] el1_executes_from_ttbr1_high_va + high_alias_maps_kernel_text | Verified | 2026-09-11 cloud qemu-smoke (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e, -cpu cortex-a57): after asid: ok, serial ttbr1: el1 exec then ttbr1: vbar then the ADR-016 private-page lines. cargo test both new cases [ok] (Running 44 tests). _start stays at 0x4008_0000. Nested fatal ELR can be the high alias (elr=0xffffff80400839e4). Not a relocated kernel. Not “EL0 isolated.” |
| qemu-smoke requires TTBR1 high-VA exec strings | scripts/qemu-smoke.sh hello phase greps ttbr1: el1 exec / ttbr1: vbar; rejects ttbr1: exec missed | Verified | 2026-09-11 cloud: qemu-smoke: TTBR1 private-page + high-VA exec strings present. |
| Full higher-half / identity teardown | Remaining identity RAM after the heap plus identity _start unmapped | Decided partial | ADR-020/025/037/038/049 tear live image + leftover frame RAM. Never yank _start while -kernel needs 0x4008_0000 (ADR-047). ident: start-stay Verified honesty. Leftover frame RAM after heap: Verified optional mile (ident: ram*, ADR-049). Do not claim the kernel moved. |
Identity .rodata tear (ADR-025) | Serial ident: rodata / ident: rodata-fault / ident: rodata-high | Verified | See earlier ADR-025 rows. |
Identity .data / stack tear (ADR-037) | Hello-kernel serial ident: data-reloc + ident: data + ident: data-fault + ident: data-high; heap stay; #[test_case] data-torn + heap-still-mapped | Verified | 2026-09-12 agent-box qemu-smoke (QEMU, nightly): ident: data-reloc n=241 / ident: data lo=0x40201000 hi=0x40258000 pages=87 / ident: data-fault / ident: data-high / ident: heap-stay / ident: ok; Running 85 tests all [ok] incl. identity_data_torn_high_stays + identity_heap_still_mapped; qemu-smoke: ok. (amd64 workstation) checkout target noted; this probe ran on the agent box. Heap stay is Historical after ADR-038. Not “EL0 isolated.” |
| Identity heap still mapped | Hello-kernel serial ident: heap-stay; #[test_case] identity_heap_still_mapped | Verified | Same smoke as ADR-037 row. Honesty marker, not a tear. Historical: superseded by ADR-038 on later tips. |
| Identity heap tear (ADR-038) | Hello-kernel serial ident: heap-reloc + ident: heap + ident: heap-fault + ident: heap-high; #[test_case] heap-torn | Verified | 2026-09-12 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on ident-heap-tear after dde185f: hello ident: heap-reloc n=0 / ident: heap lo=0x40258000 hi=0x40268000 pages=16 / ident: heap-fault / ident: heap-high / ident: ok; no ident: heap-stay; Running 86 tests all [ok] incl. identity_heap_torn_high_stays + identity_heap_unmapped_high_stays; A9 cross-update both slot:ok; force-fail exit 1; qemu-smoke: ok. (amd64 workstation) checkout target noted; this probe ran on the agent box — Docker on the amd64 workstation not re-run (CloudAgent HELD). Not “EL0 isolated.” Not “the kernel moved.” |
| Threat-model v1.18 slice (NFR-10) | Read security.md (identity heap tear; PAN enable Planned) | Verified | File + this PR. v1.18 update, not “secure.” |
| Docs site at https://ctos.artof.link | HTTP(S) fetch of the published Pages site on that host | Verified | 2026-09-11 after #30: curl -sL https://ctos.artof.link/ → HTTP 200; HTML title/menu ctos (ctsOS); landing #driving-principles. This VM repeat: same 200 + title + heading. Also 200: overview/what-can-run.html, overview/porting.html, framework/honesty-ledger.html. Deploy cited on the published-site row. Not a kernel boot claim. |
High-VA vtable rewrite + live identity .text tear (ADR-020) | Hello-kernel serial ident: reloc + ident: live + existing ident: ok; #[test_case] identity_fn_ptrs_rewritten_high + live_identity_text_unmapped_boot_stub_stays | Verified | 2026-09-11 cloud qemu-smoke (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e, -cpu cortex-a57): ident: jump then ident: reloc n=12 then ident: range lo=0x400ae000 hi=0x400b2000 pages=4 then ident: live lo=0x40081000 hi=0x400a6000 pages=37 then Hello World! then later ident: split / ident: fault / ident: high / ident: text / ident: no el0 / ident: ok. Hello BRK ELRs high (elr=0xffffff8040082990). cargo test Running 50 tests all [ok] (ident: reloc n=74, ident: live pages=45). Not a relocated kernel. Not “EL0 isolated.” .rodata/.data/heap stay. Full teardown Planned. |
| Identity text range tear (ADR-019) | Hello-kernel serial ident: jump + ident: range + ident: text + existing ident: ok; #[test_case] identity_text_range_unmapped_boot_stub_stays | Verified | 2026-09-11 cloud qemu-smoke (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e, -cpu cortex-a57): ident: jump then ident: range lo=0x400ac000 hi=0x400b0000 pages=4 then later ident: split / ident: fault / ident: high / ident: text / ident: no el0 / ident: ok. Hello BRK ELRs high (elr=0xffffff8040082758). cargo test identity_text_range_unmapped_boot_stub_stays [ok] (Running 48 tests). Live .text stays. First attempt to yank live .text after the stub Failed (unhandled sync on println! — rustc dyn Write vtables). Not a relocated kernel. Not “EL0 isolated.” Full teardown Planned. |
| Identity-tear first cut (ADR-018) | Hello-kernel serial ident: split + ident: fault + ident: high + ident: no el0 + ident: ok; #[test_case] high_ram_tables_are_independent + identity_tear_page_unmapped_high_stays + identity_tear_el1_faults_high_stays | Verified | 2026-09-11 cloud qemu-smoke (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e, -cpu cortex-a57): after ttbr1: ok, serial ident: split then ident: fault then ident: high then ident: no el0 then ident: ok. cargo test all three new cases [ok] (Running 47 tests). _start stays at 0x4008_0000. Nested fatal ELR can be the high alias (elr=0xffffff8040083bec). Not a relocated kernel. Not “EL0 isolated.” Full teardown Planned. |
| qemu-smoke requires identity-tear strings | scripts/qemu-smoke.sh hello phase greps ident: ok / ident: split / ident: fault / ident: high / ident: no el0; rejects ident: leaked and ident: probe missed | Verified | 2026-09-11 cloud: qemu-smoke: identity-tear strings present. |
| Obsidian open-vault checklist | Read research/obsidian-checklist.md; .gitignore has .obsidian/ | Verified | Structure only. .obsidian/ not committed. Not a claim that Obsidian is installed or synced. |
| CI on GitHub (pillars PR / ADR-011) | .github/workflows/smoke.yml on cursor/pillars-adr-011-325f | Verified | Cloud-probe commit cdf75eb: push 34396574483 and PR 34396590545 success. Ledger follow-up d716599: push 34396774022 and PR 34396779239 success. Same qemu-smoke.sh (hello + paging + heap + sched + CNTPCT + tick + RX + BRK + fatal + 20 tests + force-fail). |
scripts/qemu-smoke.sh (pillars follow-up / #18) | Ran on that cloud VM | Verified | 2026-09-10 (QEMU 8.2.2, rustc 1.100.0-nightly a36d05efa): hello (timeout 124) with paging: ok, heap: ok, sched: ok, wx: nx heap / wx: ok, perf: cntpct delta=22968, timer: tick, perf: irq-delta min=12973 max=35391 spread=22418 n=8, input: rx 0x41, two exception: sync BRK, exception: fatal nested / kind=0x200. cargo test Running 25 tests all [ok]. force-fail exit 1. qemu-smoke: ok. Not Docker. |
scripts/qemu-smoke.sh (pillar deepen / this branch) | Ran on this cloud VM | Verified | 2026-09-10 (QEMU 8.2.2, rustc 1.100.0-nightly a36d05efa): host perf: elf-size bytes=3752056; hello (timeout 124) with paging: ok, heap: ok, sched: ok, wx: nx heap / wx: ok, guard: fault / guard: ok, el0: svc / el0: nx kernel / el0: ok, perf: cntpct delta=20592, timer: tick, perf: irq-delta min=14477 max=24149 spread=9672 n=8, input: rx 0x41, two exception: sync BRK, exception: fatal nested / kind=0x200. cargo test Running 29 tests all [ok]. force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this PR is a separate row. |
| CI on GitHub (pillars follow-up PR / #18) | .github/workflows/smoke.yml on cursor/pillars-sec-perf-el0-9bc0 | Verified | Cloud-probe commit 8ae63bb: push 34541457222 and PR 34541473721 success (ubuntu-24.04 + ubuntu-24.04-arm). Ledger follow-up 4c610a2: push 34541534491 and PR 34541537733 success. Same qemu-smoke.sh (hello + paging + heap + sched + W^X + CNTPCT + IRQ-delta + tick + RX + BRK + fatal + 25 tests + force-fail). |
| CI on GitHub (pillar deepen PR / #19) | .github/workflows/smoke.yml on cursor/pillar-loops-deepen-48ae | Unknown | No green GHA URL on the cloud-probe commit recorded here. Merged as #19. |
scripts/qemu-smoke.sh (RO+NX / boot-delta / user TTBR0 / PR #20) | Ran on that cloud VM | Verified | 2026-09-11 (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e): host perf: elf-size bytes=3785520; hello (timeout 124) with paging: ok, heap: ok, sched: ok, wx: nx heap / wx: ok, guard: fault / guard: ok, ro: nx data / ro: write fault / ro: ok, el0: svc / el0: nx kernel / el0: no kernel read / el0: ok, perf: cntpct delta=20506, perf: boot-delta ticks=111019, timer: tick, perf: irq-delta min=4885 max=26301 spread=21416 n=8, input: rx 0x41, two exception: sync BRK, exception: fatal nested / kind=0x200. cargo test Running 36 tests all [ok]. force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on that PR is a separate row. |
| CI on GitHub (RO+NX / boot-delta / user TTBR0 / PR #20) | .github/workflows/smoke.yml on cursor/pillar-deepen-ro-boot-el0-aadc | Verified | Cloud-probe commit 6606be8: push 34561716967 and PR 34561719542 success (ubuntu-24.04 + ubuntu-24.04-arm). Host ELF there ~3.69 MiB — smaller than the arm64 workstation Docker 3786384. Do not copy that Verified onto the Docker host. |
Docker smoke on main b2bbb99 (post PR #20) | the arm64 workstation Docker Desktop linux/arm64 ./scripts/docker-smoke.sh after sync to b2bbb99 | Failed | Sponsor serial: perf: elf-size bytes=3786384; Hello World!; paging: probe missed; heap: probe missed; sched: probe missed; wx: probe missed; guard: fault / guard: ok; ro: nx data / ro: write fault / ro: ok; el0: probe missed; then CNTPCT / boot-delta / tick / irq-delta / RX / BRK / fatal still printed. qemu-smoke: missing 'paging: ok' EXIT 1. Guard/ro are current-EL identity walks — they do not need frame::ALLOC or USER_MAP_OK. Same class as the PR #20 pre-MMU .bss miss, plus a single shared L3 / first-2-MiB user map that a larger nightly layout can also break. Antifragility: this host found what GHA missed. Not this cloud VM. |
scripts/qemu-smoke.sh (layout L3 pool / post-MMU frame init / this branch) | Ran on this cloud VM | Verified | 2026-09-11 (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e): host perf: elf-size bytes=3837736; hello (timeout 124) with paging: layout data=0x40201000 end=0x4023c000 pool=0x4023c000 user=1, paging: ok, heap: ok, sched: ok, wx: nx heap / wx: ok, guard: fault / guard: ok, ro: nx data / ro: write fault / ro: ok, el0: svc / el0: nx kernel / el0: no kernel read / el0: ok, perf: cntpct delta=21094, perf: boot-delta ticks=115251, timer: tick, perf: irq-delta min=12053 max=24899 spread=12846 n=8, input: rx 0x41, two exception: sync BRK, exception: fatal nested / kind=0x200. cargo test Running 38 tests all [ok] (adds layout_stress_crosses_first_ram_l2 + frame_allocator_ready_after_mmu). force-fail exit 1. qemu-smoke: ok. Loaded layout larger than the arm64 workstation Docker’s 3.78 MiB class (__data_start in the second RAM 2 MiB). Not Docker. GHA on this PR is a separate row. |
| CI on GitHub (layout L3 pool / PR #21) | .github/workflows/smoke.yml on cursor/layout-l3-user-map-c915 | Verified | Cloud-probe commit c04b84b: push 34563006005 and PR 34563008516 success (ubuntu-24.04 + ubuntu-24.04-arm). Merge-commit push 34563104020 success on 71ee15f. MRC grepped both-matrix serial: paging: layout data=0x40201000 … user=1, paging: ok, heap: ok, sched: ok, wx: ok, el0: ok, 38 tests, force-fail fail-closed. Do not copy that Verified onto arm64 workstation Docker. |
Docker smoke after the layout fix (main 71ee15f) | the arm64 workstation Docker Desktop linux/arm64 ./scripts/docker-smoke.sh after #21 merge | Verified | Sponsor re-run on 71ee15f: perf: elf-size bytes=3838592; Hello World!; paging: layout data=0x40201000 end=0x4023c000 pool=0x4023c000 user=1; paging: ok; heap: ok; sched: ok; wx: ok; guard: ok; ro: ok; el0: svc / el0: nx kernel / el0: no kernel read / el0: ok; perf: cntpct …; perf: boot-delta ticks=217103; perf: irq-delta …; input: rx 0x41; exception: sync BRK … / fatal nested …; Running 38 tests all [ok]; force-fail exit 1; qemu-smoke: ok; EXIT 0. Keep the b2bbb99 Failed row (antifragility history). Not this cloud VM. |
| qemu-smoke requires ASID isolation strings | scripts/qemu-smoke.sh hello phase greps asid: ok / asid: dual / asid: conflict; rejects asid: stale and asid: probe missed | Verified | 2026-09-11 cloud: qemu-smoke: ASID isolation strings present. |
scripts/qemu-smoke.sh (ASID isolation / this branch) | Ran on this cloud VM | Verified | 2026-09-11 (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e): host perf: elf-size bytes=3853144; hello (timeout 124) with paging: layout data=0x40201000 end=0x4023f000 pool=0x4023f000 user=1, paging: ok, heap: ok, sched: ok, wx: nx heap / wx: ok, guard: fault / guard: ok, ro: nx data / ro: write fault / ro: ok, el0: svc / el0: nx kernel / el0: no kernel read / el0: ok, asid: dual / asid: conflict / asid: ok, perf: cntpct delta=21093, perf: boot-delta ticks=115983, timer: tick, perf: irq-delta min=9499 max=29721 spread=20222 n=8, input: rx 0x41, two exception: sync BRK, exception: fatal nested / kind=0x200. cargo test Running 39 tests all [ok]. force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this PR is a separate row. |
| CI on GitHub (ASID isolation / PR #23) | .github/workflows/smoke.yml on cursor/el0-asid-isolation-f98d | Unknown | No green GHA URL on the cloud-probe commit recorded here. Merged as #23 (fb050b7). |
scripts/qemu-smoke.sh (standing EL0 + TTBR1 / this branch) | Ran on this cloud VM | Verified | 2026-09-11 (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e): host perf: elf-size bytes=3872120; hello (timeout 124) with paging: layout data=0x40201000 end=0x40243000 pool=0x40243000 user=1, paging: ok, heap: ok, sched: ok, wx: nx heap / wx: ok, guard: fault / guard: ok, ro: nx data / ro: write fault / ro: ok, el0: svc / el0: nx kernel / el0: no kernel read / el0: standing / el0: restored / el0: ok, asid: dual / asid: conflict / asid: ok, ttbr1: el1 / ttbr1: no el0 / ttbr1: ok, perf: cntpct delta=23782, perf: boot-delta ticks=127971, timer: tick, perf: irq-delta min=11604 max=27616 spread=16012 n=8, input: rx 0x41, two exception: sync BRK, exception: fatal nested / kind=0x200. cargo test Running 42 tests all [ok]. force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this PR is a separate row. |
| CI on GitHub (standing EL0 + TTBR1 / PR #24) | .github/workflows/smoke.yml on cursor/el0-standing-ttbr1-5ce0 | Unknown | No green GHA URL on the cloud-probe commit recorded here. Merged as #24 (8745939). |
scripts/qemu-smoke.sh (TTBR1 high-VA exec / this branch) | Ran on this cloud VM | Verified | 2026-09-11 (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e): host perf: elf-size bytes=3877712; hello (timeout 124) with paging: layout data=0x40201000 end=0x40243000 pool=0x40243000 user=1, paging: ok, heap: ok, sched: ok, wx: nx heap / wx: ok, guard: fault / guard: ok, ro: nx data / ro: write fault / ro: ok, el0: svc / el0: nx kernel / el0: no kernel read / el0: standing / el0: restored / el0: ok, asid: dual / asid: conflict / asid: ok, ttbr1: el1 exec / ttbr1: vbar / ttbr1: el1 / ttbr1: no el0 / ttbr1: ok, perf: cntpct delta=20622, perf: boot-delta ticks=117383, timer: tick, perf: irq-delta min=11983 max=21932 spread=9949 n=8, input: rx 0x41, two exception: sync BRK, exception: fatal nested / kind=0x200 (fatal elr=0xffffff80400839e4). cargo test Running 44 tests all [ok]. force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this PR is a separate row. |
| CI on GitHub (TTBR1 high-VA exec / PR #25) | .github/workflows/smoke.yml on cursor/ttbr1-high-el1-exec-136c | Unknown | No green GHA URL on the cloud-probe commit recorded here. Merged as #25 (008f7c7). |
scripts/qemu-smoke.sh (identity-tear first cut / this branch) | Ran on this cloud VM | Verified | 2026-09-11 (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e): host perf: elf-size bytes=3895632; hello (timeout 124) with paging: layout data=0x40201000 end=0x40255000 pool=0x40255000 user=1, paging: ok, heap: ok, sched: ok, wx: nx heap / wx: ok, guard: fault / guard: ok, ro: nx data / ro: write fault / ro: ok, el0: svc / el0: nx kernel / el0: no kernel read / el0: standing / el0: restored / el0: ok, asid: dual / asid: conflict / asid: ok, ttbr1: el1 exec / ttbr1: vbar / ttbr1: el1 / ttbr1: no el0 / ttbr1: ok, ident: split / ident: fault / ident: high / ident: no el0 / ident: ok, perf: cntpct delta=25355, perf: boot-delta ticks=130631, timer: tick, perf: irq-delta min=14217 max=34342 spread=20125 n=8, input: rx 0x41, two exception: sync BRK, exception: fatal nested / kind=0x200 (fatal elr=0xffffff8040083bec). cargo test Running 47 tests all [ok]. force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this PR is a separate row. |
| CI on GitHub (identity-tear first cut / PR #26) | .github/workflows/smoke.yml on cursor/identity-teardown-c0fd | Unknown | No green GHA URL on the cloud-probe commit recorded here. Merged as #26 (b0f0ee5). |
Docker smoke on main b0f0ee5 (post PR #26 / ADR-018) | the arm64 workstation Docker Desktop linux/arm64 ./scripts/docker-smoke.sh after #26 merge | Verified | Sponsor re-run on b0f0ee5: 47 tests, ident: ok, fatal nested ELR high alias. Not this cloud VM. Keep prior Failed b2bbb99 row (antifragility history). |
qemu-smoke requires identity .text range-tear strings | scripts/qemu-smoke.sh hello phase greps ident: jump / ident: range / ident: text; rejects ident: range missed | Verified | 2026-09-11 cloud: qemu-smoke: identity-tear strings present (includes jump/range/text). |
scripts/qemu-smoke.sh (identity text range tear / this branch) | Ran on this cloud VM | Verified | 2026-09-11 3e710e1 (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e): host perf: elf-size bytes=3905712; hello (timeout 124) with ident: jump, ident: range lo=0x400ac000 hi=0x400b0000 pages=4, paging: layout data=0x40201000 end=0x40255000 pool=0x40255000 user=1, paging: ok, heap: ok, sched: ok, wx: ok, guard: ok, ro: ok, el0: ok, asid: ok, ttbr1: ok, ident: split / ident: fault / ident: high / ident: text / ident: no el0 / ident: ok, perf: cntpct delta=20293, perf: boot-delta ticks=92466, timer: tick, perf: irq-delta min=12711 max=26328 spread=13617 n=8, input: rx 0x41, two exception: sync BRK (ELRs 0xffffff8040082758 / 0xffffff8040082730), exception: fatal nested / kind=0x200 (fatal elr=0xffffff8040083bf4). cargo test Running 48 tests all [ok]. force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this PR is a separate row. |
CI on GitHub (identity text range tear / PR #27 ac3ad0b) | .github/workflows/smoke.yml on cursor/identity-tear-text-range-ea15 @ ac3ad0b | Failed | Both matrices: hello printed ident: jump / ident: range then ident: probe missed (init still required a 4 KiB __ident_tear_*; ADR-019 is 16 KiB). Keep this Failed row. |
CI on GitHub (identity text range tear / main 24d94e6) | .github/workflows/smoke.yml merge-commit push on 24d94e6 | Verified | Push 34640667227 success (ubuntu-24.04 + ubuntu-24.04-arm). Grepped both jobs: hello ident: jump / ident: range lo=0x400ac000 hi=0x400b0000 pages=4 / ident: text / ident: ok; BRK/fatal ELRs high; Running 48 tests; force-fail exit 1; qemu-smoke: ok. |
Docker smoke on main 24d94e6 (post PR #27 / ADR-019) | the arm64 workstation Docker Desktop linux/arm64 ./scripts/docker-smoke.sh after #27 merge | Verified | Sponsor re-run on 24d94e6: ident: jump, 16 KiB range tear, ident: ok, BRK/fatal on high ELRs, 48 tests + force-fail. Not this cloud VM. Keep the b2bbb99 Failed row. |
qemu-smoke requires identity reloc + live .text strings | scripts/qemu-smoke.sh hello phase greps ident: reloc / ident: live; rejects ident: reloc missed / ident: live missed | Verified | 2026-09-11 cloud: qemu-smoke: identity-tear strings present (includes reloc/live). |
scripts/qemu-smoke.sh (ADR-020 live .text tear / this branch) | Ran on this cloud VM | Verified | 2026-09-11 0fda690 (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e): host perf: elf-size bytes=3988408; hello (timeout 124) with ident: jump, ident: reloc n=12, ident: range lo=0x400ae000 hi=0x400b2000 pages=4, ident: live lo=0x40081000 hi=0x400a6000 pages=37, paging: layout data=0x40201000 end=0x40255000 pool=0x40255000 user=1, paging: ok, heap: ok, sched: ok, wx: nx heap / wx: ok, guard: fault / guard: ok, ro: nx data / ro: write fault / ro: ok, el0: svc / el0: nx kernel / el0: no kernel read / el0: standing / el0: restored / el0: ok, asid: dual / asid: conflict / asid: ok, ttbr1: el1 exec / ttbr1: vbar / ttbr1: el1 / ttbr1: no el0 / ttbr1: ok, ident: split / ident: fault / ident: high / ident: text / ident: no el0 / ident: ok, perf: cntpct delta=21002, perf: boot-delta ticks=94258, timer: tick, perf: irq-delta min=10686 max=27025 spread=16339 n=8, input: rx 0x41, two exception: sync BRK (ELRs 0xffffff8040082990 / 0xffffff8040082968), exception: fatal nested / kind=0x200 (fatal elr=0xffffff8040083e2c). cargo test Running 50 tests all [ok]. force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this PR is a separate row. |
CI on GitHub (ADR-020 live .text tear / this branch) | .github/workflows/smoke.yml on cursor/adr-020-live-text-reloc-322c | Unknown | No green GHA URL on the cloud-probe commit recorded here. Merged as #28 (e80dc93). |
CI on GitHub (ADR-020 live .text tear / main e80dc93) | .github/workflows/smoke.yml merge-commit push on e80dc93 | Verified | Push 34651404108 success (ubuntu-24.04 + ubuntu-24.04-arm). Grepped both jobs: hello ident: jump / ident: reloc n=12 / ident: live lo=0x40081000 hi=0x400a6000 pages=37 / ident: ok; Hello World! after the tear; Running 50 tests; force-fail exit 1; qemu-smoke: ok. |
Docker smoke on main e80dc93 (post PR #28 / ADR-020) | the arm64 workstation Docker Desktop linux/arm64 ./scripts/docker-smoke.sh after #28 merge | Verified | Sponsor re-run on e80dc93: 50 tests, ident: reloc n=12, live pages=37, force-fail ok. Not this cloud VM. Keep the earlier 24d94e6 Docker Verified row and the b2bbb99 Failed row. |
| mdBook docs site builds locally | ./scripts/docs-build.sh (mdBook 0.5.4 + mdbook-mermaid 0.17.1) writes book/ and book/CNAME is ctos.artof.link | Verified | 2026-09-11 this cloud VM (plain-English / mermaid branch): mdbook v0.5.4; mdbook-mermaid 0.17.1; docs-build: ok; mermaid class="mermaid" nodes on landing + overview + pillars. Generator only. Live URL is a separate row. Re-probe on this Darwin-pin rebase is a separate row. |
mdBook PATH pin compare strips clap v | PATH mdbook v0.5.4; ./scripts/docs-build.sh does not print the mismatch warning | Verified | 2026-09-12 this cloud VM on 8d14d08 rebase tip: real binary mdbook --version = mdbook v0.5.4; ${have#v} = 0.5.4; script output had no using PATH binary anyway and no re-download. Same strip on a bare 0.5.4 stays 0.5.4. Not a Darwin install probe. |
| docs-build auto-install on Linux (this rebase) | ./scripts/docs-build.sh with no PATH mdbook / mdbook-mermaid | Verified | 2026-09-12 this cloud VM (Linux/x86_64): downloaded x86_64-unknown-linux-gnu for mdBook 0.5.4 and mdbook-mermaid 0.17.1; docs-build: ok; mermaid class="mermaid" + book/CNAME = ctos.artof.link. Generator only. |
| docs-build auto-install on Darwin | Run ./scripts/docs-build.sh with no PATH mdbook / mdbook-mermaid on macOS (uname -s = Darwin) | Unknown | Source maps Darwin:arm64 / Darwin:x86_64 to published aarch64-apple-darwin / x86_64-apple-darwin assets for mdBook 0.5.4 and mdbook-mermaid 0.17.1 (GitHub release API listed those tarballs). This VM is Linux; Darwin download/extract was not run. |
| mdBook docs site builds locally (B7 / this branch) | ./scripts/docs-build.sh on cursor/b7-containers-nongoal-6b58 | Verified | 2026-09-12 this cloud VM: mdbook v0.5.4; mdbook-mermaid 0.17.1; docs-build: ok; book/CNAME is ctos.artof.link; generated hosting-apps.html / track-b.html contain ADR-029 + non-goal. Generator only. Pages GHA on this PR is a separate row. |
| Pages workflow builds this PR (B7 containers non-goal) | .github/workflows/pages.yml on cursor/b7-containers-nongoal-6b58 @ 1dd0a4f | Verified | PR run 34688560928 success (mdBook build 7s). Deploy job skipped on pull_request (expected). Not a new live-site deploy. Later docs-only SHAs on this branch are not this probe. |
| Pages workflow file exists | Read .github/workflows/pages.yml | Verified | Workflow present: PR builds the book; main uploads + actions/deploy-pages. Not a live URL. |
| Pages workflow builds this PR | .github/workflows/pages.yml on cursor/docs-pages-site-c371 | Verified | PR run 34651704287 success (mdBook build 7s). Deploy job skipped on pull_request (expected). Not a live site. |
| Pages workflow builds this PR (plain-English / mermaid) | .github/workflows/pages.yml on cursor/docs-plain-english-diagrams-e4a9 | Verified | PR run 34654325467 success (mdBook build 5s). Deploy job skipped on pull_request (expected). Not a new live-site deploy. |
| Docs website published (GitHub Pages) | Green pages workflow on main and an HTTPS fetch of https://ctos.artof.link | Verified | 2026-09-11 this cloud VM after #30: main deploy 34653046584 success (mdBook build + Deploy GitHub Pages). curl -sSI https://ctos.artof.link HTTP 200; body includes Driving principles. https://artofdream.github.io/ctos (no trailing slash) 301s to the custom domain; .../ctos/ 404’d — not a second live tree. From #51; not re-probed on this A1 rebase. |
Public CNAME ctos.artof.link → artofdream.github.io | dig CNAME ctos.artof.link +short | Verified | 2026-09-11 this cloud VM (repeat): artofdream.github.io. DNS is in place at the public resolver. Serving the book is the Verified docs-site / reachability rows. From #51; not re-probed here. |
Route 53 ctos CNAME in account <aws-account-id> zone <route53-zone-id> | Sponsor stated CREATE done (zone <route53-zone-id>); API list-resource-record-sets as that account | Unknown | This environment still has no AWS CLI / credentials. Sponsor named the zone and said the record exists. Do not CREATE again. Public dig is the Verified row above, not this API probe. |
Custom domain ctos.artof.link reachability | Pages Settings/API lists the hostname and curl -sSI https://ctos.artof.link is HTTP 200 with a cert for that name | Verified | 2026-09-11 this cloud VM: Pages API cname=ctos.artof.link, https_enforced=true, cert state=approved (expires 2026-12-10). HTTPS 200; TLS CN=ctos.artof.link (Let’s Encrypt). Route 53 API LIST stays the Unknown row above. From #51; not re-probed here. |
| Guest filesystem (virtio-blk + FAT16) | Hello-kernel serial blk: ok + fat: ok; VFS /probe | Verified | This tip’s qemu-smoke (see A7 rows below). Host -drive alone is not the probe. memfs is a separate A6 row. Do not say “supports FAT” as a product. |
| Guest runs host apps (Linux ELF / shell / Python) | No exec, libc, or app-load serial marker | Planned | Site: docs/overview/hosting-apps.md. Extra: host-apps.md. Today: in-tree samples only. A1 is a kernel ABI mile, not this row. |
| Guest is a container host (OCI / Docker / k8s) | Source: no OCI, runc, cgroup, or namespace code in src/ | Verified | Absence + non-goal (ADR-029, Track B B7 / #47). Site: hosting-apps.md. Host docker-smoke.sh is a build harness, not a guest runtime. Not Planned. Reopen only with a new epic + ADR. |
| Product “immutable OS” | Marketing sentence vs immutability.md | Planned | Non-claim. Absolute immutability is incompatible (heap/PTEs/devices mutate). Scoped RO is ADR-015 / ADR-020. Site: advantages.md. File presence is not Verified. |
| OS image vs app payloads disconnected (A9 first cut) | Host OS ELF + target/hello-libctos.elf; FAT /hello; hello serial slot: fat + slot: mapped + slot: ok; #[test_case] slot_load_from_fat_erets | Verified | 2026-09-12 this cloud VM (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305): host OS perf: elf-size bytes=4392040 + app 8624 bytes; FAT /hello; serial slot: fat bytes=8624 / slot: mapped segs=3 entry=0x80002000 / slot: ok / perf: app-load ticks=1134019. A1–A8 markers still present. First-cut note: A2–A4 still include_bytes! on that SHA. Leftover embed-off is a later row. Not “apps update independently.” |
| A9 cross-update (same app on OS n and n+1) | Same published hello-libctos.elf (sha256sum) on this OS tip and documented prior OS ba6541c (A9 merge; earliest main tip with the slot/FAT path); scripts/qemu-smoke.sh greps slot: ok on both boots | Verified | 2026-09-12 this cloud VM (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305): app sha256=142dedf1c3f6452555fe3af86c8bfafa260f083fecfdc81159063c433872d865 bytes=8624; host cross-update prior-os=ba6541c8e17c75c89c42451cd721735eddb36c2f slot:ok + cross-update this-os=800f52d183d0d58839f4c7363ac9d069414e55fc slot:ok. Both boots slot: fat bytes=8624 / slot: mapped / slot: ok. Prior kernel built from that SHA (no stored blob). Not “apps update independently.” |
| A9 slot-disconnect performance delta (ADR-046) | Hello serial perf: app-load ticks=<n> + perf: embed-load ticks=<n> + perf: slot-delta app=<a> embed=<b>; #[test_case] embed CNTPCT + bytes match FAT; smoke rejects slot: embed / percent marketing | Verified | 2026-09-13 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on a9/slot-perf-delta bb0379d: hello slot: fat bytes=8624 / slot: mapped / slot: ok / perf: app-load ticks=1079247 / perf: embed-load ticks=62805 / perf: slot-delta app=1079247 embed=62805 (no slot: embed); Running 92 tests all [ok] incl. slot_embed_probe_bytes_match_fat; force-fail exit 1; A9 cross-update both slot:ok; qemu-smoke: ok. Probe-only include_bytes! in src/slot.rs only. QEMU TCG lab — not a published bench, not a percent. Isolation leftovers (PAN enable / taken SError / yank _start / umbrella) unchanged Planned. (amd64 workstation) intent; CloudAgent HELD — probe on agent box clone. ADR-046. |
| Threat-model v1.9 slice (NFR-10) | Read security.md (SVC ABI mile; app hosting Planned) | Verified | File + A1 PR. v1.9 update, not “secure.” |
| Threat-model v1.10 slice (NFR-10) | Read security.md (libctos CRT mile; app hosting Planned) | Verified | File + A2 PR. v1.10 update, not “secure.” |
| Threat-model v1.11 slice (NFR-10) | Read security.md (guest ELF PT_LOAD loader mile; app hosting Planned) | Verified | File + this PR. v1.11 update, not “secure.” |
SVC ABI (exit / uart_write / yield) (ADR-021 / Track A A1) | Hello-kernel serial svc: yield + svc: user-hi + svc: uart + svc: exit + svc: ok; #[test_case] el0_svc_abi_yield_uart_exit + el0_uart_write_rejects_kernel_data + el0_uart_write_rejects_high_alias | Verified | 2026-09-11 tip 09425c0 (onto aa46219 / #29). Serial below. App hosting stays Planned. Not Linux. |
| qemu-smoke requires SVC ABI strings | scripts/qemu-smoke.sh hello phase greps svc: ok / svc: yield / svc: user-hi / svc: uart / svc: exit; rejects svc: probe missed | Verified | 2026-09-11 09425c0: qemu-smoke: SVC ABI strings present. |
scripts/qemu-smoke.sh (SVC ABI / this branch) | Ran on this cloud VM | Verified | 2026-09-11 09425c0 (QEMU 8.2.2, rustc 1.100.0-nightly 67eda617e): host perf: elf-size bytes=4020520; hello (timeout 124) with ident: jump / ident: reloc n=12 / ident: live / paging: ok / heap: ok / sched: ok / wx: ok / guard: ok / ro: ok / el0: standing / el0: restored / el0: ok / svc: yield / svc: user-hi / svc: uart n=13 / svc: exit / svc: ok / asid: ok / ttbr1: ok / ident: ok / perf: cntpct delta=20326 / perf: boot-delta ticks=104297 / timer: tick / input: rx 0x41 / exception: fatal nested. cargo test Running 54 tests all [ok] (includes el0_uart_write_rejects_high_alias). force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this tip is a separate row. |
| CI on GitHub (SVC ABI / this branch) | .github/workflows/smoke.yml on cursor/svc-syscall-abi-0dc5 | Unknown | Prior GHA is not this tip (09425c0). |
| Track A app hosting (slots) | Product “apps run independently of the OS” | Verified | Parent issue #31. Claim gate ADR-048 criteria 1–5 Met on tip be789c5+; sponsor accept ADR-052 (sponsor: “address 1. app hosting” after still-Planned #5-only review). Evidence: A8 recipes Verified; A9 cross-update cross-update … slot:ok on this OS + ba6541c; no production embed (probe-only include_bytes! in src/slot.rs / ADR-046; smoke rejects slot: embed); honesty — not Linux/POSIX/containers (ADR-029). Not Linux userspace, POSIX, glibc/musl, guest OCI, “EL0 isolated,” PAN enable, taken SError Verified, immutable-OS marketing, or OTA. |
| Documented sample apps (Track A A8) | Read what-can-run.md, apps-today.md, in-tree user/README.md; recipes cite only existing serial markers; ./scripts/docs-build.sh | Verified | 2026-09-12 this cloud VM: file read of the recipes + docs-build after rebase onto main c63077e (B7 / ADR-029; next row). Coop UART (sched:*), RX echo (input: rx 0x41), standing EL0 / libctos hello (libctos:* / loader:* / el0: task-*), optional memfs (fs:*) + FAT16 (blk:* / fat:*). No new runtime markers. No ADR (docs-only). App hosting stays Planned. A9 first cut is a later row. Guest containers stay non-goal. Kernel qemu-smoke not re-run (no src/ edit). |
scripts/docs-build.sh (A8 recipes / this branch) | Ran on this cloud VM | Verified | 2026-09-12 after rebase onto c63077e (B7 / ADR-029 + Darwin pin): mdbook v0.5.4; mdbook-mermaid 0.17.1; Linux auto-install x86_64-unknown-linux-gnu; docs-build: ok; book/CNAME is ctos.artof.link. Generator only. Live URL is a separate row. Kernel qemu-smoke not re-run (no src/ edit). |
| Standing EL0 as normal mode (ADR-024 / Track A A4) | Hello-kernel serial el0: task-enter + el0: task-active + el0: task-exit + el0: task-restored + el0: restore-fail + el0: task-ok; #[test_case] standing_task_runs_until_exit + standing_task_fault_restores | Verified | 2026-09-12 this cloud VM (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305): after loader: ok, serial el0: task-enter / el0: task-active / payload libctos: hi / libctos: ok / svc: exit / el0: task-exit / el0: task-restored / second el0: task-enter / el0: restore-fail / el0: task-ok. A1–A3 markers still present. Not isolation. App hosting stays Planned. |
| qemu-smoke requires standing-task strings | scripts/qemu-smoke.sh hello phase greps el0: task-ok / el0: task-enter / el0: task-active / el0: task-exit / el0: task-restored / el0: restore-fail; rejects el0: task missed | Verified | 2026-09-12 this cloud VM: qemu-smoke: standing-task (A4) strings present. |
scripts/qemu-smoke.sh (standing-as-normal / this branch) | Ran on this cloud VM | Verified | 2026-09-12 this tip (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305): host perf: elf-size bytes=4108960; hello (timeout 124) with ident: jump / ident: reloc n=12 / ident: live lo=0x40081000 hi=0x400ac000 pages=43 / paging: ok / heap: ok / sched: ok / wx: ok / guard: ok / ro: ok / el0: standing / el0: restored / el0: ok / svc: yield / svc: user-hi / svc: uart n=13 / svc: exit / svc: ok / libctos: hi / libctos: ok / libctos: linked / loader: mapped segs=3 entry=0x80002000 / loader: ok / el0: task-enter / el0: task-active / el0: task-exit / el0: task-restored / el0: restore-fail / el0: task-ok / asid: ok / ttbr1: ok / ident: ok / perf: cntpct delta=20161 / perf: boot-delta ticks=94742 / timer: tick / input: rx 0x41 / exception: fatal nested. cargo test Running 66 tests all [ok] (adds standing_task_runs_until_exit + standing_task_fault_restores). force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this tip is a separate row. |
| CI on GitHub (standing-as-normal / this branch) | .github/workflows/smoke.yml on cursor/a4-standing-el0-normal-1e14 | Unknown | No green GHA URL on the cloud-probe commit recorded here. |
| Threat-model v1.12 slice (NFR-10) | Read security.md (standing-as-normal mile; app hosting Planned) | Verified | File + A4 PR. v1.12 update, not “secure.” |
| Threat-model v1.13 slice (NFR-10) | Read security.md (identity .rodata tear + PAN ID-field; app hosting Planned) | Verified | File + A5 PR. v1.13 update, not “secure.” |
| Threat-model v1.14 slice (NFR-10) | Read security.md (thin VFS + memfs mile; app hosting Planned) | Verified | File + this PR. v1.14 update, not “secure.” |
| Thin VFS + memfs (ADR-027 / Track A A6) | Hello-kernel serial fs: create + fs: write + fs: read + fs: el0 + fs: ok; #[test_case] vfs_memfs_create_write_read_close + el0_fs_svc_roundtrip + el0_fs_write_rejects_kernel_data | Verified | 2026-09-12 this cloud VM (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57): after el0: task-ok, serial fs: create / fs: write / fs: read / svc: exit / fs: el0 / fs: ok. cargo test all three new cases [ok]. Not POSIX. Not FAT. App hosting stays Planned. |
| qemu-smoke requires memfs strings | scripts/qemu-smoke.sh hello phase greps fs: ok / fs: create / fs: write / fs: read / fs: el0; rejects fs: probe missed | Verified | 2026-09-12 this cloud VM: qemu-smoke: thin VFS + memfs (A6) strings present. |
scripts/qemu-smoke.sh (thin VFS + memfs / this branch) | Ran on this cloud VM | Verified | 2026-09-12 this tip (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305): host perf: elf-size bytes=4291384; hello (timeout 124) with ident: jump / ident: reloc n=12 / ident: live lo=0x40081000 hi=0x400b4000 pages=51 / ident: ro-reloc n=683 / ident: rodata lo=0x400b4000 hi=0x400c0000 pages=12 / pan: id=0 / pan: absent / paging: ok / heap: ok / sched: ok / wx: ok / guard: ok / ro: ok / el0: standing / el0: restored / el0: ok / svc: ok / libctos: ok / loader: ok / el0: task-ok / fs: create / fs: write / fs: read / fs: el0 / fs: ok / asid: ok / ttbr1: ok / ident: split / ident: fault / ident: high / ident: text / ident: no el0 / ident: rodata-fault / ident: rodata-high / ident: ok / perf: cntpct delta=21003 / perf: boot-delta ticks=104836 / timer: tick / input: rx 0x41 / exception: fatal nested. cargo test Running 73 tests all [ok] (adds vfs_memfs_create_write_read_close + vfs_open_missing_is_err + el0_fs_svc_roundtrip + el0_fs_write_rejects_kernel_data + vfs_paths_and_caps_are_documented). force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this tip is a separate row. |
| CI on GitHub (thin VFS + memfs / this branch) | .github/workflows/smoke.yml on cursor/a6-thin-vfs-memfs-766a | Unknown | No green GHA URL on this tip yet. |
Identity .rodata tear (ADR-025 / Track A A5) | Hello-kernel serial ident: ro-reloc + ident: rodata + ident: rodata-fault + ident: rodata-high + existing ident: ok; #[test_case] identity_rodata_unmapped_high_stays | Verified | 2026-09-12 this cloud VM (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57): after ident: live, serial ident: ro-reloc n=589 / ident: rodata lo=0x400ad000 hi=0x400b8000 pages=11 then later ident: rodata-fault / ident: rodata-high / ident: ok. Hello World! after the tear. cargo test identity_rodata_unmapped_high_stays [ok]. .data/heap stay. Not “EL0 isolated.” |
| PAN capability ID field (ADR-026 / Track A A5) | Hello-kernel serial pan: id= + pan: absent; #[test_case] pan_unimplemented_on_probe_cpu | Verified | 2026-09-12 this cloud VM: pan: id=0 / pan: absent. cargo test pan_unimplemented_on_probe_cpu [ok]. Enable stays Planned. Do not switch -cpu. |
qemu-smoke requires identity .rodata + PAN strings | scripts/qemu-smoke.sh hello phase greps ident: rodata / ident: rodata-fault / ident: rodata-high / pan: id= / pan: absent; rejects ident: rodata missed / pan: probe missed / pan: enabled | Verified | 2026-09-12 this cloud VM: qemu-smoke: identity-tear strings present + qemu-smoke: PAN capability strings present. |
scripts/qemu-smoke.sh (A5 identity .rodata + PAN / this branch) | Ran on this cloud VM | Verified | 2026-09-12 this tip (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305): host perf: elf-size bytes=4121360; hello (timeout 124) with ident: jump / ident: reloc n=12 / ident: live lo=0x40081000 hi=0x400ad000 pages=44 / ident: ro-reloc n=589 / ident: rodata lo=0x400ad000 hi=0x400b8000 pages=11 / pan: id=0 / pan: absent / paging: ok / heap: ok / sched: ok / wx: ok / guard: ok / ro: ok / el0: standing / el0: restored / el0: ok / svc: ok / libctos: ok / loader: ok / el0: task-ok / asid: ok / ttbr1: ok / ident: split / ident: fault / ident: high / ident: text / ident: no el0 / ident: rodata-fault / ident: rodata-high / ident: ok / perf: cntpct delta=21378 / perf: boot-delta ticks=96439 / timer: tick / input: rx 0x41 / exception: fatal nested. cargo test Running 68 tests all [ok] (adds identity_rodata_unmapped_high_stays + pan_unimplemented_on_probe_cpu). force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this tip is a separate row. |
| CI on GitHub (A5 / this branch) | .github/workflows/smoke.yml on cursor/a5-identity-tear-pan-0355 | Verified | PR run 34678808063 success on 726a4a2 (ubuntu-24.04 job 103513419325 + ubuntu-24.04-arm job 103513419189). Both matrices grepped: hello ident: ro-reloc n=588 / ident: rodata lo=0x400ad000 hi=0x400b8000 pages=11 / ident: rodata-fault / ident: rodata-high / pan: id=0 / pan: absent / ident: live / el0: task-ok / loader: ok / libctos: ok / svc: ok; Running 68 tests; qemu-smoke: identity-tear strings present / qemu-smoke: PAN capability strings present / qemu-smoke: ok. Push run 34678806604 success. mdBook 34678808079 success (Pages deploy skipped on PR). Cursor Bugbot pass on 726a4a2. Later docs-only SHAs on this branch are not this probe. |
| Guest ELF PT_LOAD loader (ADR-023 / Track A A3) | Hello-kernel serial loader: mapped + loader: ok; payload libctos: hi / libctos: ok; #[test_case] loader_maps_hello_and_erets | Verified | 2026-09-12 this cloud VM (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305): loader: mapped segs=3 entry=0x80002000 then loader: ok. Payload printed libctos: hi / libctos: ok on the loaded trip (after the A2 memcpy trip). Not a Linux ABI. App hosting stays Planned. |
| qemu-smoke requires guest loader strings | scripts/qemu-smoke.sh hello phase greps loader: ok / loader: mapped; rejects loader: probe missed | Verified | 2026-09-12 this cloud VM: qemu-smoke: guest ELF loader strings present. |
scripts/qemu-smoke.sh (guest ELF loader / this branch) | Ran on this cloud VM | Verified | 2026-09-12 this tip (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305): host perf: elf-size bytes=4101056; hello (timeout 124) with ident: jump / ident: reloc n=12 / ident: live lo=0x40081000 hi=0x400ac000 pages=43 / paging: ok / heap: ok / sched: ok / wx: ok / guard: ok / ro: ok / el0: standing / el0: restored / el0: ok / svc: yield / svc: user-hi / svc: uart n=13 / svc: exit / svc: ok / libctos: hi / libctos: ok / libctos: linked / loader: mapped segs=3 entry=0x80002000 / loader: ok / asid: ok / ttbr1: ok / ident: ok / perf: cntpct delta=21608 / perf: boot-delta ticks=98798 / timer: tick / input: rx 0x41 / exception: fatal nested. cargo test Running 64 tests all [ok] (adds loader_maps_hello_and_erets + parse / PT_INTERP / W+X / RX+R union). force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this tip is a separate row. |
| CI on GitHub (guest ELF loader / this branch) | .github/workflows/smoke.yml on cursor/a3-elf-loader-ttbr0-49f0 | Unknown | No green GHA URL on the cloud-probe commit recorded here. |
libctos CRT (exit / uart_write / yield wrappers) (ADR-022 / Track A A2) | Hello-kernel serial libctos: hi + libctos: ok + libctos: linked; #[test_case] libctos_hello_yield_uart_exit + libctos_payload_encodes_public_svc_only | Verified | 2026-09-12 tip c9b292b. Serial below. App hosting stays Planned. Not a guest ELF loader. Not Linux. |
| qemu-smoke requires libctos CRT strings | scripts/qemu-smoke.sh hello phase greps libctos: hi / libctos: ok / libctos: linked; rejects libctos: probe missed | Verified | 2026-09-12 this cloud VM: qemu-smoke: libctos CRT strings present. |
scripts/qemu-smoke.sh (libctos CRT / this branch) | Ran on this cloud VM | Verified | 2026-09-12 c9b292b (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305): host perf: elf-size bytes=4024936; hello (timeout 124) with ident: jump / ident: reloc n=12 / ident: live / paging: ok / heap: ok / sched: ok / wx: ok / guard: ok / ro: ok / el0: standing / el0: restored / el0: ok / svc: yield / svc: user-hi / svc: uart n=13 / svc: exit / svc: ok / libctos: hi / libctos: ok / libctos: linked / asid: ok / ttbr1: ok / ident: ok / perf: cntpct delta=38969 / perf: boot-delta ticks=153560 / timer: tick / input: rx 0x41 / exception: fatal nested. cargo test Running 57 tests all [ok] (adds libctos_hello_yield_uart_exit + payload SVC-only + ABI number match). force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this tip is a separate row. |
| CI on GitHub (libctos CRT / this branch) | .github/workflows/smoke.yml on cursor/libctos-crt-a2-590e | Unknown | No green GHA URL on the cloud-probe commit recorded here. |
| Threat-model v1.15 slice (NFR-10) | Read security.md (virtio-blk + FAT16 mile; app hosting Planned) | Verified | File + A7 PR. v1.15 update, not “secure.” |
| Threat-model v1.16 slice (NFR-10) | Read security.md (OS/app slot first cut; cross-update Planned) | Verified | File + A9 PR. v1.16 update, not “secure.” |
| Threat-model v1.17 slice (NFR-10) | Read security.md (leftover cross-update + FAT-only hello + stay markers; PAN enable Planned) | Verified | File + this PR. v1.17 update, not “secure.” |
A2–A4 hello without include_bytes! (ADR-032) | src/ has no include_bytes!.*hello-libctos; hello serial libctos: linked / loader: ok / el0: task-ok still from FAT /hello; build.rs still publishes target/hello-libctos.elf | Verified | 2026-09-12 this cloud VM: host qemu-smoke: kernel does not include_bytes! hello-libctos + kernel rebuild compiled app payload; serial libctos: linked / loader: ok / el0: task-ok / slot: ok. Same FAT /hello 8624 bytes. |
| virtio-blk (ADR-028 / Track A A7) | Hello-kernel serial blk: virtio + blk: cap + blk: rw + blk: ok; #[test_case] virtio_blk_capacity_and_rw | Verified | 2026-09-12 this cloud VM (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57): after fs: ok, serial blk: virtio / blk: cap sectors=8192 / blk: rw / blk: ok. cargo test virtio_blk_capacity_and_rw + virtio_blk_boot_sector_is_fat [ok]. QEMU virt virtio-mmio legacy v1. Host -drive is not this row. |
| FAT16 via thin VFS (ADR-028 / Track A A7) | Hello-kernel serial fat: mount + fat: read + fat: ok; VFS /probe = fat-hi; #[test_case] fat16_vfs_open_probe | Verified | 2026-09-12 this cloud VM: fat: mount / fat: read / fat: ok. Same vfs::open as memfs. cargo test fat16_vfs_open_probe + fat16_missing_and_readonly + vfs_fat_probe_is_same_open [ok]. Not POSIX. Not FAT32. Not writeable FAT. |
| qemu-smoke requires virtio-blk + FAT16 strings | scripts/qemu-smoke.sh hello phase greps blk: ok / fat: ok; rejects blk: probe missed / fat: probe missed | Verified | 2026-09-12 this cloud VM: qemu-smoke: virtio-blk (A7) strings present + qemu-smoke: FAT16 (A7) strings present. |
scripts/qemu-smoke.sh (virtio-blk + FAT16 / this branch) | Ran on this cloud VM | Verified | 2026-09-12 435bf2d cache-sync follow-up (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305): host perf: elf-size bytes=4379520; hello (timeout 124) with blk: virtio / blk: cap sectors=8192 / blk: rw / blk: ok / fat: mount / fat: read / fat: ok plus A1–A6 markers (svc: ok / libctos: ok / loader: ok / el0: task-ok / fs: ok / ident: ok / pan: id=0). cargo test Running 79 tests all [ok]. force-fail exit 1. qemu-smoke: ok. Prior b4903e8 cloud + GHA rows stay. Not Docker. GHA on 435bf2d is a separate row. |
CI on GitHub (virtio-blk + FAT16 / b4903e8) | .github/workflows/smoke.yml on cursor/a7-virtio-blk-fat-a587 @ b4903e8 | Verified | PR run 34687722316 success (ubuntu-24.04 job 103537544581 + ubuntu-24.04-arm job 103537544698). Both matrices grepped: hello blk: virtio / blk: cap sectors=8192 / blk: rw / blk: ok / fat: mount / fat: read / fat: ok / fs: ok / el0: task-ok / loader: ok / libctos: ok / svc: ok / ident: ok / pan: id=0 / pan: absent; qemu-smoke: virtio-blk (A7) strings present / qemu-smoke: FAT16 (A7) strings present / Running 79 tests / qemu-smoke: ok. Push run 34687720964 success. mdBook 34687722361 success (Pages deploy skipped on PR). First 79c6432 PR smoke 34687537221 Failed (modern-only / blk: probe missed). |
| CI on GitHub (virtio-blk + FAT16 / cache-sync tip) | .github/workflows/smoke.yml on cursor/a7-virtio-blk-fat-a587 @ 435bf2d | Unknown | Cloud qemu-smoke Verified on this SHA (row above). No green gh run URL on 435bf2d yet. |
| OS/app slot first cut (ADR-030 / Track A A9) | Host OS ELF + target/hello-libctos.elf; FAT /hello; hello serial slot: fat + slot: mapped + slot: ok + perf: app-load; #[test_case] slot_fat_hello_is_elf + slot_load_from_fat_erets | Verified | Same 2026-09-12 cloud qemu-smoke as the row above. cargo test both new cases [ok]. A2–A4 still embed. Not cross-update. Not app hosting. |
| qemu-smoke requires A9 slot strings | scripts/qemu-smoke.sh greps slot: ok / slot: fat / slot: mapped / perf: app-load / perf: embed-load / perf: slot-delta; rejects slot: probe missed / slot: embed / percent marketing | Verified | 2026-09-12 this cloud VM: qemu-smoke: OS/app slot (A9) strings present. ADR-046 extends greps on this PR tip. |
scripts/qemu-smoke.sh (A9 slots / this branch) | Ran on this cloud VM | Verified | 2026-09-12 9ca7372 (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305): host perf: elf-size bytes=4392040; app target/hello-libctos.elf 8624 bytes; hello (timeout 124) with slot: fat bytes=8624 / slot: mapped segs=3 entry=0x80002000 / slot: ok / perf: app-load ticks=1134019 plus A1–A8 markers (svc: ok / libctos: ok / loader: ok / el0: task-ok / fs: ok / blk: ok / fat: ok / ident: ok / pan: id=0). cargo test Running 82 tests all [ok] (adds slot_fat_hello_is_elf + slot_load_from_fat_erets + slot_path_is_not_probe). force-fail exit 1. qemu-smoke: ok. Not Docker. GHA on this tip is a separate row. |
| CI on GitHub (A9 slots / this branch) | .github/workflows/smoke.yml on cursor/a9-os-app-slots-cf00 | Unknown | Cloud qemu-smoke Verified on 9ca7372. No green gh run URL on this tip yet. |
| mdBook docs site builds locally (A9 / this branch) | ./scripts/docs-build.sh on cursor/a9-os-app-slots-cf00 | Verified | 2026-09-12 this cloud VM: mdbook v0.5.4; mdbook-mermaid 0.17.1; docs-build: ok; book/CNAME is ctos.artof.link. Generator only. Pages GHA on this PR is a separate row. |
| mdBook docs site builds locally (ADR-002 Track A merge-hat / this branch) | ./scripts/docs-build.sh on cursor/adr-002-track-a-merge-hat-0e11 | Verified | 2026-09-12 this cloud VM: mdbook v0.5.4; mdbook-mermaid 0.17.1; Linux auto-install x86_64-unknown-linux-gnu; docs-build: ok. Generator only. Kernel qemu-smoke not run (no src/ edit). Distinct-identity merge stays Unknown. |
| Linux-compat goals documented (ADR-031 / Track B B1) | Read ADR-031 and track-b.md | Verified | Document inspection. ABI subset research frame. Not claiming Linux userspace. Guest containers stay non-goal (ADR-029). B2–B6 stay Planned. File presence is not a Linux ABI. Kernel qemu-smoke not run (no src/ edit). |
| mdBook docs site builds locally (B1 / this branch) | ./scripts/docs-build.sh on cursor/b1-linux-compat-goals-9374 | Verified | 2026-09-12 this cloud VM after rebase onto 4459b93 (#61): mdbook v0.5.4; mdbook-mermaid 0.17.1; Linux auto-install x86_64-unknown-linux-gnu; docs-build: ok; book/CNAME is ctos.artof.link; generated ADR-031 + track-b contain not claiming Linux userspace; ADR-002 page still has the Track A historical exception. Generator only. Pages GHA on this PR is a separate row. |
| mdBook docs site builds locally (mobile theme / this branch) | ./scripts/docs-build.sh on cursor/docs-mobile-responsive-81c4 | Verified | 2026-09-12 this cloud VM: mdbook v0.5.4; mdbook-mermaid 0.17.1; Linux auto-install x86_64-unknown-linux-gnu; docs-build: ok; book/CNAME is ctos.artof.link; *theme.css present in book/. Generator only. Pages GHA on this PR is a separate row. |
| Docs site mobile-compliant (narrow viewport) | ./scripts/docs-mobile-probe.sh after ./scripts/docs-build.sh (viewport meta + theme.css + Chrome iframe scrollWidth vs clientWidth) | Verified | 2026-09-12 this cloud VM (Google Chrome 148.0.7778.96): landing index.html overflow=no at 320 (client=320 scroll=320), 768, and 1280; framework/honesty-ledger.html @ 320 overflow=no; website.html @ 320 overflow=no. Wide tables/pre may scroll inside their widget. Mermaid CDN was blocked for this numeric probe (source text). Not a live https://ctos.artof.link mobile claim until a post-merge Pages fetch. |
scripts/qemu-smoke.sh (Track A leftovers / this branch) | Ran on this cloud VM | Verified | 2026-09-12 800f52d (QEMU 8.2.2, rustc 1.100.0-nightly 0fc141305): host OS perf: elf-size bytes=4410056; app 8624 bytes; hello (timeout 124) with A1–A9 markers (svc: ok / libctos: ok / loader: ok / el0: task-ok / fs: ok / blk: ok / fat: ok / slot: ok / ident: data-stay / ident: heap-stay / ident: ok / pan: id=0 / pan: absent); host embed ratchet; same-app cross-update on ba6541c + this tip (sha256=142dedf1…); cargo test Running 83 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Re-probe on 17778ca and post-rebase 6abcb1b (onto e6255c0; same markers + qemu-smoke: ok). Re-probe on 7f52572 after the Docker-git / portable-sha256 ratchet: same A1–A9 + stay + sha256=142dedf1… + both cross-update … slot:ok + 83 tests + force-fail 1 + qemu-smoke: ok. Re-probe on 4a6b4f7 after moving /tmp/ctos-prior-os-* aside (stale .git/worktrees registration): git worktree add prior OS ba6541c then both cross-update … slot:ok + 83 tests + force-fail 1 + qemu-smoke: ok. Not Docker. GHA on 7b68d08 / 7f52572 are separate Verified rows. GHA on 4a6b4f7 / 70058db is Unknown until grepped. |
CI on GitHub (Track A leftovers / 17778ca) | .github/workflows/smoke.yml on cursor/track-a-leftovers-d2ee @ 17778ca | Verified | Push run 34701455001 success (ubuntu-24.04 job 103573837236 + ubuntu-24.04-arm job 103573837300). Both matrices grepped: kernel does not include_bytes! hello-libctos; hello slot: fat bytes=8624 / slot: ok / ident: data-stay / ident: heap-stay / svc: ok / libctos: ok / loader: ok / el0: task-ok / fs: ok / blk: ok / fat: ok / pan: absent; cross-update prior-os=ba6541c… slot:ok + cross-update this-os=17778ca… slot:ok; Running 83 tests; force-fail exited 1; qemu-smoke: ok. Prior ef9ff56 Failed closed (shallow clone missing ba6541c). |
CI on GitHub (Track A leftovers / 7b68d08) | .github/workflows/smoke.yml on cursor/track-a-leftovers-d2ee @ 7b68d08 (rebase onto 522c543 / #63) | Verified | Push run 34702218168 success (ubuntu-24.04-arm job 103575880564 + ubuntu-24.04 job 103575880766). PR run 34702220491 success (merge-ref ebdcc8d). Both matrices grepped: kernel does not include_bytes! hello-libctos; hello slot: fat bytes=8624 / slot: ok / ident: data-stay / ident: heap-stay / ident: ok / svc: ok / libctos: ok / loader: ok / el0: task-ok / fs: ok / blk: ok / fat: ok / pan: absent; cross-update prior-os=ba6541c… slot:ok + cross-update this-os=7b68d08… slot:ok (PR event this-os=ebdcc8d); Running 83 tests; force-fail exited 1; qemu-smoke: ok. Later ledger-only SHA is not this probe. |
| mdBook docs site builds locally (leftovers / this branch) | ./scripts/docs-build.sh on cursor/track-a-leftovers-d2ee | Verified | 2026-09-12 this cloud VM: mdbook v0.5.4; mdbook-mermaid 0.17.1; docs-build: ok. Generator only. Pages GHA on this PR is a separate row. |
| qemu-smoke requires A9 cross-update + stay + embed-off | scripts/qemu-smoke.sh greps both cross-update … slot:ok lines, ident: data-stay / ident: heap-stay, and rejects src/ hello include_bytes! | Verified | 2026-09-12 this cloud VM: qemu-smoke: A9 cross-update … strings present + identity-tear strings (includes stay) + kernel does not include_bytes! hello-libctos. |
| Leftover docker-smoke (git + prior SHA) | Dockerfile installs git; .dockerignore keeps .git objects but drops .git/worktrees; smoke git worktree prunes before add | Unknown | Source ratchet after Bugbot Mediums on 7b68d08 / 7b296e2. This cloud VM: command -v docker empty. File presence is not docker-smoke. |
| Leftover app sha256 pin (NFR-03) | qemu-smoke hashes via sha256sum / shasum -a 256 / openssl dgst -sha256 | Unknown | Source ratchet after Bugbot Medium on 7b68d08. This Linux tip used sha256sum (sha256=142dedf1… on 7f52572). Darwin shasum path not probed here. Same-app proof remains the cp of the published ELF. |
CI on GitHub (Track A leftovers / 7f52572) | .github/workflows/smoke.yml on cursor/track-a-leftovers-d2ee @ 7f52572 | Verified | Push run 34702699720 success (ubuntu-24.04 job 103577150895 + ubuntu-24.04-arm job 103577151020). PR run 34702702359 success (merge-ref 709bafd). Both matrices grepped: embed-off, A9 slot: + A1–A8 + stay, both cross-update … slot:ok (push this-os=7f52572), 83 tests, force-fail 1, qemu-smoke: ok. Later ledger-only SHA is not this probe. |
| Linux AArch64 vs ctos SVC gap documented (Track B B2) | Read linux-aarch64-syscall-gap.md, src/syscall.rs (SYS_* 16–23), and Linux v6.10 include/uapi/asm-generic/unistd.h | Verified | Document inspection. Representative table + convention gap. Not claiming Linux userspace. No Linux numbers in src/. B3–B6 stay Planned. File presence is not a Linux ABI. Kernel qemu-smoke not run (no src/ edit). |
| mdBook docs site builds locally (B2 / this branch) | ./scripts/docs-build.sh on cursor/b2-linux-syscall-gap-c35b | Verified | 2026-09-12 this cloud VM: mdbook v0.5.4; mdbook-mermaid 0.17.1; Linux auto-install x86_64-unknown-linux-gnu; docs-build: ok; book/CNAME is ctos.artof.link; generated gap page + track-b contain never-per-ADR-031 and not claiming Linux userspace. Generator only. Pages GHA on this PR is a separate row. Kernel qemu-smoke not run (no src/ edit). |
| CI on GitHub (B2 / this branch) | .github/workflows/smoke.yml on cursor/b2-linux-syscall-gap-c35b @ 60478ac | Verified | PR run 34701938481 success (ubuntu-24.04 job 103575136780 + ubuntu-24.04-arm job 103575136921). Both matrices grepped: hello slot: fat bytes=8624 / slot: mapped / slot: ok / svc: ok / libctos: ok / loader: ok / el0: task-ok / fs: ok / blk: ok / fat: ok; Running 82 tests; force-fail exit 1; qemu-smoke: ok. Push run 34701935451 success. Docs-only; not a new kernel mile. Later ledger SHAs on this branch are not this probe. |
| Pages workflow builds this PR (B2 syscall gap) | .github/workflows/pages.yml on cursor/b2-linux-syscall-gap-c35b @ 60478ac | Verified | PR run 34701938331 success (mdBook build job 103575136524). Deploy job skipped on pull_request (expected). Not a new live-site deploy. |
Linux ELF / auxv / PT_INTERP vs freestanding loader documented (ADR-033 / Track B B4) | Read ADR-033, src/loader.rs (PT_INTERP reject, ET_EXEC / PT_LOAD only), Linux v6.10 include/uapi/linux/elf.h + auxvec.h | Verified | Document inspection. Gap ADR. Track A loader stays reusable. Does not accept PT_INTERP. Not claiming dynamic Linux ELF. No src/ change. B3 / B5 / B6 stay Planned. File presence is not a Linux ELF ABI. Kernel qemu-smoke not run (no src/ edit). |
| mdBook docs site builds locally (B4 / this branch) | ./scripts/docs-build.sh on cursor/b4-elf-auxv-pt-interp-51bc | Verified | 2026-09-12 this cloud VM: mdbook v0.5.4; mdbook-mermaid 0.17.1; Linux auto-install x86_64-unknown-linux-gnu; docs-build: ok; book/CNAME is ctos.artof.link; generated ADR-033 + track-b contain stay-rejected and not claiming dynamic Linux ELF. Generator only. Pages GHA on this PR is a separate row. Kernel qemu-smoke not run (no src/ edit). |
| Process model vs standing EL0 documented (ADR-035 / Track B B3) | Read ADR-035 and track-b.md | Verified | Document inspection. Standing EL0 stays the ctos model. Linux clone/execve/wait4 is a gap. Not claiming Linux userspace. Do not add those syscalls to src/. B4 is Documented on main (774c672 / #68). B5 / B6 stay Planned. File presence is not a Linux process table. Kernel qemu-smoke not run (no src/ edit). |
| mdBook docs site builds locally (B3 / this branch) | ./scripts/docs-build.sh on cursor/b3-process-model-3443 | Verified | 2026-09-12 this cloud VM after rebase onto 774c672 (#68 / ADR-033 ELF): mdbook v0.5.4; mdbook-mermaid 0.17.1; Linux auto-install x86_64-unknown-linux-gnu; docs-build: ok; book/CNAME is ctos.artof.link; generated ADR-035 + track-b contain Standing EL0 stays the ctos model and not claiming Linux userspace; generated ADR-033 page contains stay-rejected and not claiming dynamic Linux ELF. No ADR-033-process-model or ADR-034-process-model page. Generator only. Pages GHA on this PR is a separate row. Kernel qemu-smoke not run (no src/ edit). |
| Linux VFS vs thin ctos VFS documented (ADR-034 / Track B B5) | Read ADR-034, src/vfs.rs (VfsOps + valid_path), src/fat.rs, and linux-aarch64-syscall-gap.md (openat / read / write / close) | Verified | Document inspection. Concept compare only. Not claiming a Linux filesystem. No POSIX flags / dentries in src/. B6 stays Planned. B3 is Documented (ADR-035). B4 is Documented (ADR-033). File presence is not Linux VFS. Kernel qemu-smoke not run (no src/ edit). |
| mdBook docs site builds locally (B5 / this branch) | ./scripts/docs-build.sh on cursor/b5-linux-vfs-vs-thin-ctos-f294 | Verified | 2026-09-12 this cloud VM: mdbook v0.5.4; mdbook-mermaid 0.17.1; Linux auto-install x86_64-unknown-linux-gnu; docs-build: ok; book/CNAME is ctos.artof.link; generated ADR-034 + track-b contain Not claiming a Linux filesystem and No Linux VFS object is present. Generator only. Pages GHA on this PR is a separate row. Kernel qemu-smoke not run (no src/ edit). |
CI on GitHub (B5 / prior SHA fc4bf89) | .github/workflows/smoke.yml on cursor/b5-linux-vfs-vs-thin-ctos-f294 @ fc4bf89 (then titled ADR-034) | Verified | MRC-grepped on #66: push 34713366059 (this-os=fc4bf89… slot:ok) + PR 34713368698 (merge-ref this-os=c0492c3… slot:ok). Both matrices: slot: fat / mapped / ok + stay + qemu-smoke: ok. Docs-only; not a new kernel mile. Later rename SHAs on this branch are not this probe. |
CI on GitHub (B5 / prior SHA ea2469e) | .github/workflows/smoke.yml on cursor/b5-linux-vfs-vs-thin-ctos-f294 @ ea2469e | Verified | Pre-rebase tip grepped: push 34713639443 (ubuntu-24.04 job 103606766099 + ubuntu-24.04-arm job 103606766239; this-os=ea2469e… slot:ok). PR 34713642019 success (merge-ref this-os=178d7fd… slot:ok). Both matrices: embed-off, slot: fat / mapped / ok, stay, prior-os=ba6541c… slot:ok, 83 tests, force-fail 1, qemu-smoke: ok. Docs-only; not a new kernel mile. Later rebase SHAs are not this probe. |
Pages workflow builds this PR (B5 VFS compare / ea2469e) | .github/workflows/pages.yml on cursor/b5-linux-vfs-vs-thin-ctos-f294 @ ea2469e | Verified | PR run 34713642021 success (mdBook build job 103606772101). Deploy job skipped on pull_request (expected). Not a new live-site deploy. Later rebase SHAs are not this probe. |
mdBook docs site builds locally (B5 / rebase onto 774c672) | ./scripts/docs-build.sh on cursor/b5-linux-vfs-vs-thin-ctos-f294 after rebase onto #68 | Verified | 2026-09-12 this cloud VM: mdbook v0.5.4; mdbook-mermaid 0.17.1; Linux auto-install x86_64-unknown-linux-gnu; docs-build: ok; book/CNAME is ctos.artof.link; generated ADR-033 ELF + ADR-034 VFS both present; track-b has B4 Documented and B5 Documented with Not claiming a Linux filesystem and No Linux VFS object is present. Generator only. Kernel qemu-smoke not run (no src/ edit). |
CI on GitHub (B5 / rebase tip 0cd62c2) | .github/workflows/smoke.yml on cursor/b5-linux-vfs-vs-thin-ctos-f294 @ 0cd62c2 (onto 774c672) | Verified | This session grepped: push 34714163309 (ubuntu-24.04 job 103608155056 + ubuntu-24.04-arm job 103608154963; this-os=0cd62c2… slot:ok). PR 34714165232 success (merge-ref this-os=20a544f6… slot:ok). Both matrices: embed-off, slot: fat / mapped / ok, stay, prior-os=ba6541c… slot:ok, 83 tests, force-fail 1, qemu-smoke: ok. Docs-only; not a new kernel mile. Later ledger SHAs are not this probe. |
Pages workflow builds this PR (B5 VFS compare / 0cd62c2) | .github/workflows/pages.yml on cursor/b5-linux-vfs-vs-thin-ctos-f294 @ 0cd62c2 | Verified | PR run 34714165116 success (mdBook build job 103608159994). Deploy job skipped on pull_request (expected). Not a new live-site deploy. |
mdBook docs site builds locally (B5 / rebase onto d81a53a) | ./scripts/docs-build.sh on cursor/b5-linux-vfs-vs-thin-ctos-f294 after rebase onto #67 | Verified | 2026-09-12 this cloud VM: mdbook v0.5.4; mdbook-mermaid 0.17.1; Linux auto-install x86_64-unknown-linux-gnu; docs-build: ok; book/CNAME is ctos.artof.link; generated ADR-033 ELF + ADR-034 VFS + ADR-035 process-model all present; track-b has B3/B4/B5 Documented with Not claiming a Linux filesystem. Generator only. This-tip GHA Unknown until grepped. Kernel qemu-smoke not run (no src/ edit). |
| qemu-smoke requires identity heap tear strings | scripts/qemu-smoke.sh hello phase greps ident: heap-reloc / ident: heap lo= / ident: heap-fault / ident: heap-high; rejects ident: heap-stay | Verified | 2026-09-12 agent-box: qemu-smoke: identity-tear strings present and heap-stay absent. |
scripts/qemu-smoke.sh (ADR-038 identity heap tear / this branch) | Ran on this agent box | Verified | 2026-09-12 ident-heap-tear after dde185f (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305): host perf: elf-size bytes=4432632; hello (timeout 124) with ident: heap-reloc n=0 / ident: heap lo=0x40258000 hi=0x40268000 pages=16 / ident: heap-fault / ident: heap-high plus A1–A9 + data tear markers; no ident: heap-stay; cargo test Running 86 tests all [ok]; force-fail exit 1; A9 cross-update prior-os=ba6541c… slot:ok + this-os=dde185f… slot:ok; qemu-smoke: ok. Not Docker. GHA on this PR is a separate row. |
| CI on GitHub (ADR-038 / this branch) | .github/workflows/smoke.yml on ident-heap-tear | Unknown | No green GHA URL on the cloud-probe commit yet. |
the arm64 workstation / amd64 workstation Docker smoke on dde185f (ADR-037 tip) | (amd64 workstation) 2026-09-13 amd64 Docker: ident: data-reloc n=241, ident: data … pages=87, ident: data-fault / ident: data-high, ident: heap-stay, slot: ok, A9 cross-update this-os=dde185f, qemu-smoke: ok | Verified | Host bash/WSL still missing — docker build/docker run directly. Heap stay is Historical after ADR-038. PAN enable / umbrella isolation still Planned. |
| Threat-model v1.19 slice (NFR-10) | Read security.md (EL0 trampoline without VMALLE1; PAN enable Planned) | Verified | File + this PR. v1.19 update, not “secure.” |
EL0 entry without TLBI VMALLE1 (ADR-039) | Hello-kernel serial el0: no-vmalle1; src/exception.rs has no tlbi vmalle1; #[test_case] el0_entry_without_vmalle1 | Verified | 2026-09-12 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on isolation/adr-039-el0-tlbi-without-vmalle1 after f2c0c5a: hello el0: no kernel read / el0: standing / el0: restored / el0: no-vmalle1 / el0: ok / el0: task-ok / asid: ok; qemu-smoke: exception.rs has no tlbi vmalle1; Running 87 tests all [ok] incl. el0_entry_without_vmalle1; force-fail exit 1; A9 cross-update both slot:ok; qemu-smoke: ok. What TLBI remains: paging::init VMALLE1 at MMU enable + TLBI VAAE1 on unmaps. Not “EL0 isolated.” Not PAN. (amd64 workstation) intent; CloudAgent HELD — probe on agent box clone. |
qemu-smoke requires el0: no-vmalle1 + exception.rs ratchet | scripts/qemu-smoke.sh greps el0: no-vmalle1 and rejects tlbi vmalle1 in src/exception.rs | Verified | 2026-09-12 agent-box: qemu-smoke: exception.rs has no tlbi vmalle1 + EL0 first-mile + read-mile + standing + no-vmalle1 strings present. |
scripts/qemu-smoke.sh (ADR-039 EL0 no-VMALLE1 / this branch) | Ran on this agent box | Verified | 2026-09-12 after f2c0c5a (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305): host perf: elf-size bytes=4432656; hello (timeout 124) with el0: no-vmalle1 plus A1–A9 + identity data/heap tear markers; Running 87 tests all [ok]; force-fail exit 1; A9 cross-update prior-os=ba6541c… slot:ok + this-os=f2c0c5a… slot:ok; qemu-smoke: ok. Not Docker. GHA on this PR is a separate row. |
| CI on GitHub (ADR-039 / this branch) | .github/workflows/smoke.yml on this branch | Unknown | No green GHA URL yet. |
| Threat-model v1.20 slice (NFR-10) | Read security.md (lower-EL IRQ while standing; FIQ/SError still park; PAN enable Planned) | Verified | File + this PR. v1.20 update, not “secure.” |
| Lower-EL IRQ while standing (ADR-040) | Hello-kernel serial el0: irq; #[test_case] lower_el_irq_while_standing; vector irq_lower_el | Verified | 2026-09-13 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on isolation/adr-040-lower-el-irq-standing after 28d3755: hello el0: standing / el0: restored / el0: standing / timer: tick / el0: irq / el0: restored / el0: no-vmalle1 / el0: ok; Running 88 tests all [ok] incl. lower_el_irq_while_standing; force-fail exit 1; A9 cross-update both slot:ok; qemu-smoke: ok. What still parks: lower-EL FIQ/SError. Default ERET I-clear is ADR-041. Not “EL0 isolated.” Not PAN. (amd64 workstation) intent; CloudAgent HELD — probe on agent box clone. |
qemu-smoke requires el0: irq | scripts/qemu-smoke.sh greps el0: irq | Verified | 2026-09-13 agent-box: qemu-smoke: EL0 first-mile + read-mile + standing + irq + no-vmalle1 strings present. |
scripts/qemu-smoke.sh (ADR-040 lower-EL IRQ / this branch) | Ran on this agent box | Verified | 2026-09-13 after 28d3755 (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305): host perf: elf-size bytes=4438008; hello (timeout 124) with el0: irq plus A1–A9 + identity data/heap tear + no-vmalle1 markers; Running 88 tests all [ok]; force-fail exit 1; A9 cross-update prior-os=ba6541c… slot:ok + this-os=28d3755… slot:ok (pre-commit tip); qemu-smoke: ok. Not Docker. GHA on this PR is a separate row. |
| CI on GitHub (ADR-040 / this branch) | .github/workflows/smoke.yml on this branch | Unknown | No green GHA URL yet. |
| Threat-model v1.21 slice (NFR-10) | Read security.md (IRQ-unmasked default ERET; FIQ/SError still park; PAN enable Planned) | Verified | File + this PR. v1.21 update, not “secure.” |
| IRQ-unmasked default standing/task ERET (ADR-041) | Hello-kernel serial el0: irq-default (+ el0: irq); #[test_case] lower_el_irq_default_eret; default eret_to_el0 SPSR 0x340 | Verified | 2026-09-13 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on isolation/adr-041-irq-unmasked-standing after 473a73d: hello el0: standing / el0: restored / el0: standing / el0: irq / el0: restored / el0: irq-default / el0: no-vmalle1 / el0: ok; Running 89 tests all [ok] incl. lower_el_irq_default_eret + lower_el_irq_while_standing; force-fail exit 1; A9 cross-update both slot:ok; qemu-smoke: ok. Entry 0x40080000. Short non-standing probes use eret_to_el0_masked. What still parks: lower-EL FIQ/SError. Not “EL0 isolated.” Not PAN. (amd64 workstation) intent; CloudAgent HELD — probe on agent box clone. |
qemu-smoke requires el0: irq-default | scripts/qemu-smoke.sh greps el0: irq-default | Verified | 2026-09-13 agent-box: qemu-smoke: EL0 first-mile + read-mile + standing + irq + irq-default + no-vmalle1 strings present. |
scripts/qemu-smoke.sh (ADR-041 default IRQ-unmasked ERET / this branch) | Ran on this agent box | Verified | 2026-09-13 after 473a73d (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305): host perf: elf-size bytes=4438016; hello (timeout 124) with el0: irq-default plus A1–A9 + identity data/heap tear + no-vmalle1 markers; Running 89 tests all [ok]; force-fail exit 1; A9 cross-update prior-os=ba6541c… slot:ok + this-os=473a73d… slot:ok (pre-commit tip); qemu-smoke: ok. Not Docker. GHA on this PR is a separate row. |
| CI on GitHub (ADR-041 / this branch) | .github/workflows/smoke.yml on this branch | Unknown | No green GHA URL yet. |
| Threat-model v1.22 slice (NFR-10) | Read security.md (start-stay / ram-stay honesty; FIQ/SError still park; PAN enable Planned; umbrella Planned) | Verified | File + this PR. v1.22 update, not “secure.” |
Identity _start / leftover-RAM stay honesty (ADR-042) | Hello-kernel serial ident: start-stay + ident: ram-stay; #[test_case] identity_boot_stub_stays_while_live_torn | Verified | 2026-09-13 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on isolation/adr-042-isolation-wrap after 75feef9: hello ident: start-stay lo=0x40080000 hi=0x40081000 / ident: ram-stay lo=0x40268000 hi=0x48000000 / ident: ok; Running 90 tests all [ok] incl. identity_boot_stub_stays_while_live_torn; force-fail exit 1; A9 cross-update both slot:ok; qemu-smoke: ok. Entry 0x40080000. Still Planned: PAN enable; lower-EL FIQ/SError; yank _start; umbrella “EL0 isolated.” Not “the kernel moved.” (amd64 workstation) intent; CloudAgent HELD — probe on agent box clone. |
| Threat-model v1.23 slice (NFR-10) | Read security.md (lower-EL FIQ while standing; SError park honesty; PAN enable Planned; umbrella Planned) | Verified | File + this PR. v1.23 update, not “secure.” |
| Lower-EL FIQ while standing + SError park (ADR-043) | Hello-kernel serial el0: fiq + el0: serror-park; #[test_case] lower_el_fiq_while_standing; vector fiq_lower_el | Verified | 2026-09-13 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on isolation/adr-043-lower-el-fiq-serror after 472d78e: hello el0: standing / el0: irq / el0: fiq / el0: irq-default / el0: serror-park / el0: no-vmalle1 / el0: ok; Running 91 tests all [ok] incl. lower_el_fiq_while_standing; force-fail exit 1; A9 cross-update both slot:ok; qemu-smoke: ok. FIQ via temporary GICC FIQEn + SPSR 0x380. Taken SError still Planned (no safe inject). Entry 0x40080000. Not “EL0 isolated.” Not PAN. (amd64 workstation) intent; CloudAgent HELD — probe on agent box clone. |
| qemu-smoke requires ident: start-stay / ident: ram-stay | scripts/qemu-smoke.sh greps ident: start-stay and ident: ram-stay | Verified | 2026-09-13 agent-box: qemu-smoke: identity-tear strings present (includes start-stay / ram-stay). |
| scripts/qemu-smoke.sh (ADR-042 isolation leftover wrap / this branch) | Ran on this agent box | Verified | 2026-09-13 after 75feef9 (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305): host perf: elf-size bytes=4440560; hello (timeout 124) with ident: start-stay lo=0x40080000 hi=0x40081000 / ident: ram-stay plus ADR-037…041 markers; Running 90 tests all [ok]; force-fail exit 1; A9 cross-update prior-os=ba6541c… slot:ok + this-os=75feef9… slot:ok (pre-commit tip); qemu-smoke: ok. Not Docker. GHA on this PR is a separate row. |
| CI on GitHub (ADR-042 / this branch) | .github/workflows/smoke.yml on this branch | Unknown | No green GHA URL yet. |
How to update
- Run or cite the probe (command, file path + revision, or
gh runURL). - Change only the rows you probed.
- If you could not run QEMU, leave boot Unknown and say so in the PR. A prior Verified row is one environment and one boot path; do not copy the 2026-09-08 x86 VGA probe forward.
| Track B B6 decision documented (ADR-036 / never) | Read ADR-036 and track-b.md | Verified | Document inspection. Explicit never: no Linux-compat layer or reimplementation from B1–B5. Research ladder complete. Not claiming Linux userspace. Track A freestanding path stays. No src/ change. Kernel qemu-smoke not run. |
| mdBook docs site builds locally (B6 / Grok Bot docs path) | ./scripts/docs-build.sh when available on a host with mdBook; else Pages GHA on the PR | Unknown | Generator probe deferred to GHA/docs-build on this PR. File presence of ADR-036 is document inspection only. |
| the arm64 workstation / amd64 workstation Docker smoke on aeec16c (B6 tip) | (arm64 workstation) earlier on 00ebfc2: slot: ok + A9 cross-update; (amd64 workstation) 2026-09-13 amd64 Docker Desktop 29.7.2, 32 CPU / 31 GiB: docker build -t ctos-smoke . && docker run --rm ctos-smoke → slot: ok, cross-update prior ba6541c + this tip, qemu-smoke: ok | Verified | Host bash/WSL missing on the amd64 workstation — smoke via docker CLI directly (same as scripts/docker-smoke.sh). GHA also green on #69. |
| the amd64 workstation Docker smoke on f2c0c5a (ADR-038 tip) | (amd64 workstation) 2026-09-13 amd64 Docker: ident: heap-reloc n=0, ident: heap … pages=16, ident: heap-fault / ident: heap-high, ident: data-*, slot: ok, qemu-smoke: ok | Verified | Host bash/WSL still missing — docker CLI directly. PAN enable / umbrella isolation still Planned. |
| the amd64 workstation Docker smoke on 28d3755 (ADR-039 tip) | (amd64 workstation) 2026-09-13 amd64 Docker: el0: no-vmalle1, ident: heap-*, slot: ok, qemu-smoke: ok | Verified | Host bash/WSL missing — docker CLI. PAN enable / lower-EL IRQ while standing / umbrella still Planned. |
| the amd64 workstation Docker smoke on 473a73d (ADR-040 tip) | (amd64 workstation) 2026-09-13 amd64 Docker: el0: irq, el0: no-vmalle1, slot: ok, qemu-smoke: ok | Verified | Host bash/WSL missing — docker CLI. PAN / FIQ-SError / default IRQ-unmasked standing / umbrella still Planned. |
| the amd64 workstation Docker smoke on 75feef9 (ADR-041 tip) | (amd64 workstation) 2026-09-13 amd64 Docker: el0: irq / el0: irq-default, slot: ok, qemu-smoke: ok | Verified | Host bash/WSL missing — docker CLI. PAN / FIQ-SError / identity _start / umbrella still Planned. |
| the amd64 workstation Docker smoke on 472d78e (ADR-042 tip) | (amd64 workstation) 2026-09-13 amd64 Docker: ident: start-stay, ident: ram-stay, el0: irq-default, slot: ok, qemu-smoke: ok | Verified | Host bash/WSL missing — docker CLI. Isolation ladder wrap complete; PAN / FIQ-SError / full _start tear / umbrella stay Planned (ADR-042). |
| the amd64 workstation Docker smoke on 65cf515 (ADR-043 tip) | (amd64 workstation) 2026-09-13 amd64 Docker: el0: fiq, el0: serror-park, slot: ok, qemu-smoke: ok | Verified | Host bash/WSL missing — docker CLI. Taken SError / PAN enable / _start tear / umbrella still Planned. |
| Taken lower-EL SError research (ADR-044) | Read ADR-044; park marker still ADR-043 | Verified (research) | Docs-only. Candidate inject failed on QEMU 10 virt+a57 (ADR-045). Taken path = deferred/non-goal on this smoke machine (ADR-047) — do not copy Verified onto a taken serial. |
| the amd64 workstation Docker smoke on 3f7d3c5 (ADR-045 tip) | (amd64 workstation) 2026-09-13 amd64 Docker: el0: serror-arm, qmp inject-nmi => error:machine does not provide NMIs, el0: serror-park, qemu-smoke: ok | Verified | Taken el0: serror not printed — inject blocked on this QEMU virt. Taken SError / PAN / _start tear / umbrella still Planned. |
| the amd64 workstation Docker smoke on 608f643 (ADR-046 tip) | (amd64 workstation) 2026-09-13 amd64 Docker: perf: app-load, perf: embed-load, perf: slot-delta app=… embed=…, slot: ok, qemu-smoke: ok | Verified | Measurement pair only (QEMU TCG lab). No percent / faster-slower claim. Isolation leftovers decided in ADR-047 (this PR). |
| Taken lower-EL SError on stock smoke QEMU (ADR-047 / ADR-053 / ADR-081) | Stock distro QEMU: deferred/non-goal; park + A-clear prep | Deferred / non-goal (stock) | Stock path unchanged. Pin Verified: ADR-083; the CI job qemu-nmi-pin has been unconditional since ADR-090 (2026-09-26, D1 evidence class with B2-P). B2-P (real arm64 + KVM) Verified opt-in, one-off: ADR-085. Keep el0: serror-park on distro QEMU. Do not require taken without CTOS_REQUIRE_TAKEN_SERROR=1. B2 (different runner/accelerator) blocked on free hosted runners: ADR-084 (2026-09-25). |
| PAN enable on default cortex-a57 (ADR-047 / ADR-054) | Decision: enable non-goal locked; ID-field stays Verified | Non-goal (default probe CPU) — locked | ADR-054 / ADR-047 / ADR-026. Explicit reopen gate in ADR-054. Do not silent -cpu switch. |
| Never yank _start (ADR-047) | Decision while QEMU -kernel needs 0x4008_0000; ident: start-stay | Decided (honesty Verified) | ADR-047 / ADR-042. Leftover RAM after heap Verified optional (ADR-049). |
| App hosting claim criteria documented (ADR-048) | Read ADR-048; product row still Planned | Verified (docs) | Checklist only. Do not claim “app hosting is done.” |
| Threat-model v1.26 slice (NFR-10) | Read security.md (ADR-047 leftover decisions + ADR-048 claim gate) | Verified | File + this PR. v1.26 update, not “secure.” Not “EL0 isolated.” |
| Leftover identity RAM tear (ADR-049) | Hello-kernel serial ident: ram lo= + ident: ram-fault + ident: ram-high; reject ident: ram-stay; ident: start-stay still required; #[test_case] identity_ram_torn_high_stays / identity_ram_unmapped_high_stays | Verified | 2026-09-13 agent-box smoke on this branch (see smoke row). High twin stays. Never yank _start. Not “EL0 isolated.” Not “kernel moved.” ADR-049. |
| Threat-model v1.27 slice (NFR-10) | Read security.md (ADR-049 leftover identity RAM tear) | Verified | File + this PR. v1.27 update, not “secure.” Not “EL0 isolated.” |
| scripts/qemu-smoke.sh (ADR-049 leftover identity RAM tear / this branch) | Ran on this agent box | Verified | 2026-09-13 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/ident-ram-unmap: hello ident: ram lo=0x40268000 hi=0x48000000 pages=32152 / ident: ram-fault / ident: ram-high / ident: start-stay lo=0x40080000 hi=0x40081000 / ident: ok (no ident: ram-stay); Running 94 tests all [ok] incl. identity_ram_torn_high_stays / identity_ram_unmapped_high_stays / identity_boot_stub_stays_while_live_torn; force-fail exit 1; A9 cross-update both slot:ok; qemu-smoke: ok. Not Docker. Not “EL0 isolated.” Not “kernel moved.” (amd64 workstation) not re-run — CloudAgent HELD; evidence tagged agent-box. |
| ADR-048 progress checklist + stale embed/cross-update doc honesty (Future slice 1) | Read progress tables in this ledger + hosting-apps.md; stale “A2–A4 still embed” / “Cross-update stays Planned” / Planned delta wording aligned with Verified ADR-032/046 rows; product row still Planned | Verified (docs) | Tip b47bbfc audit on this PR. Docs-only; no src/ change; kernel smoke not re-run. Do not claim “app hosting is done.” |
| (amd64 workstation) Alpine WSL bash + Docker Desktop PATH hygiene | Documented host notes for later smoke (docs-only OK) | Verified (docs) | Alpine WSL2 default; install bash: wsl -d Alpine -u root -- apk add bash. Use wsl -d Alpine -u <user> -- bash (not WindowsApps bash stub). Docker Desktop WSL integration not enabled for Alpine — call docker.exe via PATH /mnt/c/Users/<user>/AppData/Local/Programs/DockerDesktop/resources/bin or cmd docker build/run. scripts/docker-smoke.sh is already #!/bin/sh. (amd64 workstation) intent for any later smoke; CloudAgent HELD — this PR is docs-only. |
| FAT16 write behind thin VFS (ADR-050 / Future slice 3) | Hello-kernel serial fat: write + fat: rewrite + fat: create + keep fat: ok; host mkfat16.py --check-write; #[test_case] fat16_vfs_write_and_restore + fat16_create_write_small_file | Verified | 2026-09-13 agent-box smoke on feat/fat16-write @ e539813 (see smoke row). Guest wrote FAT16 bytes through thin VFS; created /fwr=fat-nw; restored /probe=fat-hi. Not POSIX. Not FAT32. Not “supports FAT.” App hosting stays Planned. ADR-050. |
| Threat-model v1.28 slice (NFR-10) | Read security.md (ADR-050 FAT16 write) | Verified | File + this PR. v1.28 update, not “secure.” |
| scripts/qemu-smoke.sh (ADR-050 FAT16 write / this branch) | Ran on this agent box | Verified | 2026-09-13 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/fat16-write @ e539813: hello fat: mount / fat: read / fat: write / fat: rewrite / fat: create / fat: ok; host mkfat16: check-write ok file=/fwr bytes=fat-nw; slot: ok; A9 cross-update both slot:ok; Running 96 tests all [ok] incl. fat16_vfs_write_and_restore + fat16_create_write_small_file; force-fail exit 1; qemu-smoke: ok. Not Docker. Not POSIX. Not FAT32. (amd64 workstation) not re-run — CloudAgent HELD; evidence tagged agent-box. Tip e539813. |
| FAT16 vs memfs write CNTPCT pair (ADR-051 / Future slice 4 / P-PERF-6) | Hello serial perf: fat-write ticks=<n> + perf: memfs-write ticks=<n> + perf: fs-write-delta fat=<a> memfs=<b>; #[test_case] fat_memfs_write_cntpct_pair samples > 0; smoke rejects percent marketing | Verified | 2026-09-13 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/fat-write-cntpct @ 45dc9e1: hello fat: write / fat: rewrite / fat: create / perf: fat-write ticks=66638 / perf: memfs-write ticks=5303 / perf: fs-write-delta fat=66638 memfs=5303 / fat: ok / slot: ok; Running 97 tests all [ok] incl. fat_memfs_write_cntpct_pair; force-fail exit 1; qemu-smoke: ok. QEMU TCG lab — not a published bench, not a percent. Keep ADR-050 write mile. App hosting stays Planned. (amd64 workstation) intent; CloudAgent HELD — probe on agent box clone. ADR-051. |
| scripts/qemu-smoke.sh (ADR-051 FAT-vs-memfs write CNTPCT / this branch) | Ran on this agent box | Verified | 2026-09-13 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/fat-write-cntpct: markers above; host --check-write still ok; A9 cross-update both slot:ok; Running 97 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not a latency SLA. Tip 45dc9e1. |
| Companion / overview polish (Future slice 6) | Read overview + linked framework pages: no live “A2–A4 still embed,” “cross-update Planned,” ram-stay as the story, “FAT read-only” as the only FAT story; ADR-048 checklist pointer present; ADR-047…051 reflected; product hosting row still Planned | Verified (docs) | Tip ebc968e (includes ADR-051). Docs-only; ./scripts/docs-build.sh ok; no src/ change; kernel smoke not re-run. Do not claim “app hosting is done,” “EL0 isolated,” PAN enable, or taken SError Verified. |
| Sponsor accept product app hosting (ADR-048 #5 / ADR-052) | Sponsor signal + this flip PR; criteria 1–4 already Met on tip be789c5+; no self-merge | Verified (docs) | 2026-09-13: sponsor said “address 1. app hosting” after a still-Planned review that listed product app hosting as blocked only on criterion #5. ADR-052. Product row flipped Planned → Verified. Docs-only; ./scripts/docs-build.sh ok; no src/ change; kernel smoke not re-run (criteria 1–4 already Met). Not Linux userspace, POSIX, containers, “EL0 isolated,” PAN enable, taken SError Verified, immutable OS, or OTA. |
| Threat-model v1.29 slice (NFR-10) | Read security.md (ADR-048/052 product app-hosting claim Verified; non-claims unchanged) | Verified | File + this PR. v1.29 update, not “secure.” Not “EL0 isolated.” Not Linux/POSIX/containers. |
| Taken lower-EL SError hard-stop (ADR-053) | Dated QMP re-probe: inject-nmi / HMP nmi → machine does not provide NMIs; park stays Verified; taken locked deferred/non-goal | Deferred / non-goal (locked) | 2026-09-14 agent-box QEMU 10.0.13 -machine virt -cpu cortex-a57: QMP inject-nmi error machine does not provide NMIs; HMP nmi same. Prior smoke: amd64 workstation 3f7d3c5 / ADR-045. Reopen attempt ADR-081. Do not claim taken Verified. ADR-053. |
| PAN enable lock (ADR-054) | Enable non-goal locked on default cortex-a57; reopen gate documented; ID-field stays Verified | Non-goal (locked) | Docs-only 2026-09-14. Evidence: ADR-026 pan: absent; scripts keep -cpu cortex-a57. Reopen = sponsor-approved CPU + FEAT_PAN + enable+fault ADR. Do not claim PAN enabled. ADR-054. |
| Umbrella “EL0 isolated” checklist (ADR-055; audited 2026-09-26 in ADR-086) | Evidence table: Verified miles vs unmet (PAN enable, taken SError, yank _start, sponsor umbrella accept) | Verified (§B rows 1–3 Met; umbrella = the ADR-091 sentence only) | M6 (2026-09-28): row 3 Met (sponsor’s written accept, B3); see the P-SEC-3 row. Docs-only 2026-09-14: “Cannot become Verified under current constraints” (as of then). Do not round Verified miles into “EL0 isolated.” ADR-055. Sponsor one-pager ADR-060. |
| Threat-model v1.30 slice (NFR-10) | Read security.md (ADR-053/054/055 isolation-gap locks; non-claims unchanged) | Verified | File + this PR. v1.30 update, not “secure.” Not “EL0 isolated.” Not PAN enabled. Not taken SError Verified. |
| FAT16 readdir behind thin VFS (ADR-056) | Hello-kernel serial fat: readdir + fat: entries n=<k> + keep fat: ok / write / slot: ok; #[test_case] fat16_readdir_lists_root + fat16_readdir_fail_closed_cap | Verified | 2026-09-14 agent-box smoke on feat/fat16-readdir: hello fat: readdir / fat: entries n=3 / fat: ok; Running 99 tests all [ok] incl. readdir cases; force-fail exit 1; qemu-smoke: ok. Guest listed FAT16 root paths through thin VFS (/probe, /hello, /fwr). Not POSIX getdents/opendir. Not FAT32. Not “supports FAT.” Not EL0 isolated / PAN / taken SError. CloudAgent HELD — evidence tagged agent-box. Tip abe72cf. ADR-056. |
| Threat-model v1.31 slice (NFR-10) | Read security.md (ADR-056 FAT16 readdir) | Verified | File + this PR. v1.31 update, not “secure.” Not POSIX getdents. |
| scripts/qemu-smoke.sh (ADR-056 FAT16 readdir / this branch) | Ran on this agent box | Verified | 2026-09-14 agent-box qemu-smoke (QEMU 10.0.13, -cpu cortex-a57) on feat/fat16-readdir: markers fat: mount / fat: read / fat: write / fat: rewrite / fat: create / fat: readdir / fat: entries n=3 / fat: ok / slot: ok; host --check-write ok; A9 cross-update both slot:ok; Running 99 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not POSIX. Tip abe72cf. |
| FAT16 delete behind thin VFS (ADR-057) | Hello-kernel serial fat: delete + keep fat: ok / write / readdir / slot: ok; #[test_case] fat16_unlink_removes_file + fat16_unlink_missing_is_err | Verified | 2026-09-14 agent-box smoke on feat/fat16-delete: hello fat: delete / fat: readdir / fat: entries n=3 / fat: ok; Running 101 tests all [ok] incl. unlink cases; force-fail exit 1; qemu-smoke: ok. Guest deleted FAT16 root dirent /fdel through thin VFS (vfs::unlink). Not POSIX unlink/remove. Not FAT32. Not “supports FAT.” Not EL0 isolated / PAN / taken SError. CloudAgent HELD — evidence tagged agent-box. Tip c945ec8. ADR-057. |
| Threat-model v1.32 slice (NFR-10) | Read security.md (ADR-057 FAT16 delete) | Verified | File + this PR. v1.32 update, not “secure.” Not POSIX unlink. |
| scripts/qemu-smoke.sh (ADR-057 FAT16 delete / this branch) | Ran on this agent box | Verified | 2026-09-14 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/fat16-delete: markers fat: mount / fat: read / fat: write / fat: rewrite / fat: create / fat: readdir / fat: entries n=3 / fat: delete / fat: ok / slot: ok; host --check-write ok; A9 cross-update both slot:ok; Running 101 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not POSIX. Tip c945ec8. |
| Thin VFS prefix mounts (ADR-058) | Hello-kernel serial vfs: mount + vfs: mounts + keep fs: ok / fat: ok / slot: ok; #[test_case] vfs_prefix_mounts_route_backends + vfs_memfs_mount_create_stays_off_fat | Verified | 2026-09-14 agent-box smoke on feat/vfs-virtual-mounts @ 4814735: hello vfs: mount /kprobe -> memfs … vfs: mount / -> fat / vfs: mounts / fs: ok / fat: ok / slot: ok; Running 103 tests all [ok] incl. mount cases; force-fail exit 1; qemu-smoke: ok. Guest routed path prefixes through a thin VFS mount table (/mem+A6 names → memfs; / → FAT16). Not POSIX mount(2). Not Linux VFS. Not a new disk FS. Not EL0 isolated / PAN / taken SError. CloudAgent HELD — evidence tagged agent-box. ADR-058. |
| scripts/qemu-smoke.sh (ADR-058 VFS prefix mounts / this branch) | Ran on this agent box | Verified | 2026-09-14 agent-box qemu-smoke (QEMU 10.x, -cpu cortex-a57) on feat/vfs-virtual-mounts @ 4814735: markers vfs: mount / vfs: mounts / fs: ok / fat: mount / fat: read / fat: write / fat: rewrite / fat: create / fat: readdir / fat: entries n=3 / fat: delete / fat: ok / slot: ok; host --check-write ok; A9 cross-update both slot:ok; Running 103 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not POSIX. Tip 4814735. |
| Freestanding fs-libctos sample + catalog (ADR-059) | Hello-kernel serial libctos: fs-hi + libctos: fs-ok + fsdemo: ok + keep slot: ok / /hello; #[test_case] fsdemo_fat_is_elf + fsdemo_load_from_fat_erets + fsdemo_path_is_memdemo_mount | Verified | 2026-09-14 agent-box smoke on feat/fs-libctos-sample: hello libctos: fs-hi / libctos: fs-ok / fsdemo: fat / fsdemo: mapped / fsdemo: ok / slot: ok; Running 106 tests all [ok] incl. fsdemo cases; force-fail exit 1; qemu-smoke: ok. Guest loaded FAT /fsdemo and the payload exercised libctos VFS on /memdemo (memfs via ADR-058 /mem prefix; EL0 exec pages fetch-only — compare with immediates). Not POSIX. Not getdents. Not EL0 isolated / PAN / taken SError. Do not reopen ADR-048/052 app hosting. CloudAgent HELD — evidence tagged agent-box (amd64 workstation Computer tool unavailable this turn). ADR-059. |
| Threat-model v1.34 slice (NFR-10) | Read security.md (ADR-059 fs-libctos catalog) | Verified | File + this PR. v1.34 update, not “secure.” Not POSIX. |
| scripts/qemu-smoke.sh (ADR-059 fs-libctos / this branch) | Ran on this agent box | Verified | 2026-09-14 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/fs-libctos-sample: markers libctos: fs-hi / libctos: fs-ok / fsdemo: ok + preserved vfs: mounts / fs: ok / fat: ok / slot: ok / A9 cross-update both slot:ok; Running 106 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not POSIX. Tip ca707e7. |
| Sponsor isolation leftovers closure checklist (ADR-060) | Single table: taken SError / PAN enable / umbrella / never-yank _start — locks + reopen gates only | Docs lock (non-Verified) | Docs-only 2026-09-15. Cites ADR-047 / ADR-053 / ADR-054 / ADR-055 / #98. Does not invent Verified for taken SError / PAN enable / “EL0 isolated.” M6 (2026-09-28): its umbrella row now reads Verified, only as the accepted ADR-091 sentence; the other locks are unchanged. Does not silent -cpu, yank _start, or reopen Guest Linux/containers/immutable marketing. ADR-060. |
| Threat-model v1.35 slice (NFR-10) | Read security.md (ADR-060 sponsor leftovers closure; non-claims unchanged) | Verified | File + this PR. v1.35 update, not “secure.” Not “EL0 isolated.” Not PAN enabled. Not taken SError Verified. |
| Freestanding fat-libctos sample + catalog (ADR-061) | Hello-kernel serial libctos: fat-hi + libctos: fat-ok + fatdemo: ok + keep slot: ok / fsdemo: ok / /hello / /fsdemo; #[test_case] fatdemo_fat_is_elf + fatdemo_load_from_fat_erets + fatdemo_path_is_fat_probe | Verified | 2026-09-15 agent-box smoke on feat/fat-libctos-sample: hello libctos: fat-hi / libctos: fat-ok / fatdemo: fat / fatdemo: mapped / fatdemo: ok / slot: ok / fsdemo: ok; Running 109 tests all [ok] incl. fatdemo cases; force-fail exit 1; qemu-smoke: ok. Guest loaded FAT /fatdemo and the payload opened/read FAT /probe (fat-hi) via thin VFS (EL0 exec pages fetch-only — compare with immediates). Not POSIX. Not getdents. Not EL0 isolated / PAN / taken SError. Do not reopen ADR-048/052 app hosting. CloudAgent HELD — evidence tagged agent-box (amd64 workstation Computer/CloudAgent unavailable this turn). Tip 8bf25d2. ADR-061. |
| Threat-model v1.36 slice (NFR-10) | Read security.md (ADR-061 fat-libctos catalog) | Verified | File + this PR. v1.36 update, not “secure.” Not POSIX. |
| Freestanding yield-libctos sample + catalog (ADR-062) | Hello-kernel serial libctos: yld-hi + ≥3× libctos: beat + libctos: yld-ok + yldemo: ok + keep slot: ok / fsdemo: ok / fatdemo: ok / /hello / /fsdemo / /fatdemo; #[test_case] yldemo_fat_is_elf + yldemo_load_from_fat_erets + yldemo_path_is_fat | Verified | 2026-09-16 agent-box smoke on feat/yield-libctos-el0-sample: hello libctos: yld-hi / 3× libctos: beat / libctos: yld-ok / yldemo: fat / yldemo: mapped / yldemo: ok / slot: ok / fsdemo: ok / fatdemo: ok; Running 112 tests all [ok] incl. yldemo cases; force-fail exit 1; qemu-smoke: ok (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305). Guest loaded FAT /yldemo and the payload issued several cooperative yield_now() rounds (deeper than hello’s single yield). Not preemption. Not multi-task EL0. Not a process table. Not POSIX. Not EL0 isolated / PAN / taken SError. Do not reopen ADR-048/052 app hosting. CloudAgent HELD — evidence tagged agent-box. Tip 7a5a529. ADR-062. |
| Threat-model v1.37 slice (NFR-10) | Read security.md (ADR-062 yield-libctos catalog) | Verified | File + this PR. v1.37 update, not “secure.” Not preemption. |
| scripts/qemu-smoke.sh (ADR-061 fat-libctos / this branch) | Ran on this agent box | Verified | 2026-09-15 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/fat-libctos-sample: markers libctos: fat-hi / libctos: fat-ok / fatdemo: ok + preserved libctos: fs-ok / fsdemo: ok / vfs: mounts / fs: ok / fat: ok / slot: ok / A9 cross-update both slot:ok; Running 109 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not POSIX. Tip 8bf25d2. |
| Track N network foundation scope (ADR-063 / N0) | Read ADR-063 + track-n.md; SUMMARY / roadmap / limits wired; no virtio-net driver in this PR | Docs lock (non-Verified) | Docs-only 2026-09-16. Opens Track N. Runtime link/first-frame stays Planned. Do not invent Verified. Do not claim TCP/UDP/sockets/DHCP/DNS/Wi‑Fi/Linux net/“has networking.” ADR-063. |
| Track N virtio-net first frame (N1 / ADR-066) | Hello-kernel serial net: virtio + net: mac + net: tx + net: rx + net: ok; QEMU -netdev user,id=net0 -device virtio-net-device,netdev=net0; keep blk: ok / fat: ok; #[test_case] virtio_net_discover_and_arp | Verified | 2026-09-16 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/n1-virtio-net-first-frame: hello net: virtio / net: mac 52:54:00:12:34:56 / net: tx / net: rx / net: ok + optional perf: net-tx / perf: net-rx + keep blk: ok / fat: ok / slot: ok; Running 116 tests all [ok] incl. virtio_net_discover_and_arp; force-fail exit 1; qemu-smoke: ok. Probe = ARP request for SLIRP gateway 10.0.2.2 + reply. Not TCP/UDP/sockets/DHCP/DNS/Wi‑Fi/“has networking.” CloudAgent HELD — evidence tagged agent-box. ADR-066. |
| Threat-model v1.38 slice (NFR-10) | Read security.md (ADR-063 Track N Planned; I/O surface) | Verified | File + #109. v1.38 update, not “secure.” Not “has networking.” |
| FAT16 multi-cluster grow (ADR-064) | Hello-kernel serial fat: grow + libctos: fat-grow + keep fat: ok / fat: write / fatdemo: ok; #[test_case] fat16_multi_cluster_grow + fat16_grow_refuses_over_file_max | Verified | 2026-09-16 agent-box smoke on feat/fat-multi-cluster-grow (rebased on main post-#109): hello fat: grow / libctos: fat-grow / libctos: fat-ok / fatdemo: ok / fat: ok; Running 114 tests all [ok] incl. grow cases; force-fail exit 1; qemu-smoke: ok (QEMU 10.x, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57). Guest allocated a FAT chain and wrote 600 bytes across a 512-byte cluster boundary; EL0 chunked SVC write/read-back. FILE_MAX raised to 1024. Not POSIX write API. Not exFAT/FAT32. Not Track N. CloudAgent HELD — evidence tagged agent-box. Tip cb3a946. ADR-064. |
| Threat-model v1.39 slice (NFR-10) | Read security.md (ADR-064 FAT16 multi-cluster grow; keeps ADR-063 Track N Planned) | Verified | File + this PR. v1.39 update on tip post-#109 v1.38, not “secure.” Not POSIX. Not “has networking.” |
| scripts/qemu-smoke.sh (ADR-064 multi-cluster grow / this branch) | Ran on this agent box | Verified | 2026-09-16 agent-box qemu-smoke on feat/fat-multi-cluster-grow (rebased on b4efb5f): markers fat: grow / libctos: fat-grow + preserved fat: write / fat: delete / fat: ok / fatdemo: ok / slot: ok; Running 114 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not POSIX. Tip cb3a946. |
| FAT16 vs memfs read CNTPCT pair (ADR-065 / perf slice 3 / P-PERF-7) | Hello serial perf: fat-read ticks=<n> + perf: memfs-read ticks=<n> + perf: fs-read-delta fat=<a> memfs=<b>; #[test_case] fat_memfs_read_cntpct_pair samples > 0; smoke rejects percent marketing; /mrprobe memfs mount prefix | Verified | 2026-09-16 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/fat-memfs-read-cntpct @ 0abba31: hello perf: fat-read ticks=7752 / perf: memfs-read ticks=2210 / perf: fs-read-delta fat=7752 memfs=2210 + keep perf: fat-write / perf: fs-write-delta / fat: ok / slot: ok; Running 115 tests all [ok] incl. fat_memfs_read_cntpct_pair; force-fail exit 1; qemu-smoke: ok. QEMU TCG lab — not a published bench, not a percent. Keep ADR-051 write pair. App hosting stays Planned. CloudAgent HELD — probe on agent box. ADR-065. |
| Threat-model v1.40 slice (NFR-10) | Read security.md (ADR-065 FAT vs memfs read CNTPCT; keeps ADR-063 Track N Planned) | Verified | File + this PR. v1.40 update on tip post-#110 v1.39, not “secure.” Not a latency SLA. Not “has networking.” |
| scripts/qemu-smoke.sh (ADR-065 FAT-vs-memfs read CNTPCT / this branch) | Ran on this agent box | Verified | 2026-09-16 agent-box qemu-smoke on feat/fat-memfs-read-cntpct: markers above; ADR-051 write pair still present; Running 115 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not a latency SLA. Tip 0abba31. |
| Threat-model v1.41 slice (NFR-10) | Read security.md (ADR-066 N1 virtio-net first frame; I/O surface names blk+net) | Verified | File + this PR. v1.41 update on tip post-#111 v1.40, not “secure.” Not TCP/UDP. Not “has networking.” |
| scripts/qemu-smoke.sh (ADR-066 N1 virtio-net / this branch) | Ran on this agent box | Verified | 2026-09-16 agent-box qemu-smoke on feat/n1-virtio-net-first-frame: markers net: virtio / net: mac / net: tx / net: rx / net: ok + preserved blk: ok / fat: ok / slot: ok / ADR-065 read pair; qemu-smoke: virtio-net (N1) strings present; Running 116 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not a product stack. Tip b87ddba. |
| Track N virtio-net ICMP ping (N2 / ADR-067) | Hello-kernel serial net: icmp-tx + net: icmp-rx + net: ping-ok (+ keep N1 net: ok); QEMU -netdev user,id=net0 -device virtio-net-device,netdev=net0 unchanged; keep blk: ok / fat: ok; #[test_case] virtio_net_icmp_echo_ping | Verified | 2026-09-16 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/n2-icmp-ping: hello net: virtio / net: mac 52:54:00:12:34:56 / net: tx / net: rx / net: ok / net: icmp-tx / net: icmp-rx / net: ping-ok + optional perf: net-icmp-tx / perf: net-icmp-rx + keep blk: ok / fat: ok / slot: ok; Running 117 tests all [ok] incl. virtio_net_discover_and_arp + virtio_net_icmp_echo_ping; force-fail exit 1; qemu-smoke: ok. Probe = ICMP echo vs SLIRP gateway 10.0.2.2 after ARP. Not TCP/UDP/sockets/DHCP/DNS/Wi‑Fi/“has networking.” CloudAgent HELD — evidence tagged agent-box. ADR-067. |
| Threat-model v1.42 slice (NFR-10) | Read security.md (ADR-067 N2 ICMP ping; I/O surface still blk+net) | Verified | File + this PR. v1.42 update on tip post-#112 v1.41, not “secure.” Not TCP/UDP. Not “has networking.” |
| scripts/qemu-smoke.sh (ADR-067 N2 ICMP ping / this branch) | Ran on this agent box | Verified | 2026-09-16 agent-box qemu-smoke on feat/n2-icmp-ping: markers net: icmp-tx / net: icmp-rx / net: ping-ok + N1 net: ok + preserved blk: ok / fat: ok / slot: ok / ADR-065 read pair; qemu-smoke: virtio-net (N1+N2) strings present; Running 117 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not a product stack. Tip b71d7f4. |
| Track N EL0 net SVC + net-libctos (N4 / ADR-068) | Hello-kernel serial libctos: net-hi + libctos: net-mac + libctos: net-ok + netdemo: ok (+ keep N1 net: ok / N2 net: ping-ok / prior samples); #[test_case] netdemo_fat_is_elf + netdemo_load_from_fat_erets + netdemo_path_is_fat | Verified | 2026-09-16 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/n4-el0-net-svc: hello libctos: net-hi / libctos: net-mac / libctos: net-ok / netdemo: fat / netdemo: mapped / netdemo: ok + keep net: ok / net: ping-ok / blk: ok / fat: ok / slot: ok / yldemo: ok; Running 120 tests all [ok] incl. netdemo cases; force-fail exit 1; qemu-smoke: ok. SVC surface: net_mac=24, net_ping=25. Kernel owns virtio-net. Not TCP/UDP/sockets/DHCP/DNS/Wi‑Fi/“has networking.” CloudAgent HELD — evidence tagged agent-box. Tip f1942ab. ADR-068. |
| Threat-model v1.43 slice (NFR-10) | Read security.md (ADR-068 N4 EL0 net SVC sample; I/O surface still blk+net) | Verified | File + this PR. v1.43 update on tip post-#113 v1.42, not “secure.” Not TCP/UDP. Not “has networking.” |
| scripts/qemu-smoke.sh (ADR-068 N4 net-libctos / this branch) | Ran on this agent box | Verified | 2026-09-16 agent-box qemu-smoke on feat/n4-el0-net-svc: markers libctos: net-ok / netdemo: ok + preserved N1/N2/samples; qemu-smoke: ADR-068 net-libctos /netdemo strings present; Running 120 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not a product stack. Tip f1942ab. |
| Net-ping path CNTPCT (ADR-069 / P-PERF-8) | Hello serial perf: net-ping ticks=<n> (prefix grep; not exact ticks); #[test_case] el0_net_ping_cntpct_advances sample > 0; smoke rejects percent marketing; keep N1/N2/N4 | Verified | 2026-09-16 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/net-ping-cntpct: hello perf: net-ping ticks=20370 + keep net: ping-ok / libctos: net-ok / netdemo: ok / blk: ok / fat: ok / slot: ok; Running 121 tests all [ok] incl. el0_net_ping_cntpct_advances (perf: net-ping ticks=13052 in that case); force-fail exit 1; qemu-smoke: ok. QEMU TCG lab — not a published bench, not a percent, not a ping SLA. Window = quiet ARP+ICMP as SYS_NET_PING / el0_icmp_ping. CloudAgent HELD — evidence tagged agent-box. ADR-069. |
| Threat-model v1.44 slice (NFR-10) | Read security.md (ADR-069 net-ping CNTPCT; keeps N1/N2/N4) | Verified | File + this PR. v1.44 update on tip post-#114 v1.43, not “secure.” Not a latency SLA. Not “has networking.” |
| scripts/qemu-smoke.sh (ADR-069 net-ping CNTPCT / this branch) | Ran on this agent box | Verified | 2026-09-16 agent-box qemu-smoke on feat/net-ping-cntpct: marker perf: net-ping ticks=20370 + preserved N1/N2/N4/samples; qemu-smoke: ADR-069 net-ping CNTPCT marker present; Running 121 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not a product stack. Tip e20c852. |
| Track N UDP transport (N3 / ADR-070) | Hello-kernel serial net: udp-tx + net: udp-rx + net: udp-ok (+ keep N1 net: ok / N2 net: ping-ok / N4 samples / ADR-069); QEMU -netdev user,id=net0 -device virtio-net-device,netdev=net0 unchanged; #[test_case] virtio_net_udp_dns_probe | Verified | 2026-09-16 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/n3-udp-transport: hello net: udp-tx / net: udp-rx / net: udp-ok + keep net: ok / net: ping-ok / libctos: net-ok / netdemo: ok / perf: net-ping / blk: ok / fat: ok / slot: ok; Running 122 tests all [ok] incl. virtio_net_udp_dns_probe; force-fail exit 1; qemu-smoke: ok. Probe = UDP DNS query to SLIRP 10.0.2.3:53 + matching DNS response (QR=1). Not BSD sockets / TCP product / DHCP/DNS product / Wi‑Fi / “has networking.” CloudAgent HELD — evidence tagged agent-box. ADR-070. |
| Threat-model v1.45 slice (NFR-10) | Read security.md (ADR-070 N3 minimal UDP; I/O surface still blk+net) | Verified | File + this PR. v1.45 update on tip post-#115 v1.44, not “secure.” Not BSD sockets. Not “has networking.” |
| scripts/qemu-smoke.sh (ADR-070 N3 UDP / this branch) | Ran on this agent box | Verified | 2026-09-16 agent-box qemu-smoke on feat/n3-udp-transport: markers net: udp-tx / net: udp-rx / net: udp-ok + preserved N1/N2/N4/samples/ADR-069; qemu-smoke: virtio-net (N1+N2+N3) strings present; Running 122 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not a product stack. Tip 7e74752. |
| Track N EL0 UDP DNS SVC + udp-libctos (N3.x / ADR-071) | Hello-kernel serial libctos: udp-hi + libctos: udp-ok + udpdemo: ok (+ keep N1 net: ok / N2 net: ping-ok / N3 net: udp-ok / N4 libctos: net-ok / prior samples / ADR-069); #[test_case] udpdemo_fat_is_elf + udpdemo_load_from_fat_erets + udpdemo_path_is_fat + el0_udp_dns_probe | Verified | 2026-09-16 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/n3x-el0-udp-svc: hello libctos: udp-hi / libctos: udp-ok / udpdemo: fat / udpdemo: mapped / udpdemo: ok + keep net: ok / net: ping-ok / net: udp-ok / libctos: net-ok / netdemo: ok / perf: net-ping / blk: ok / fat: ok / slot: ok; Running 126 tests all [ok] incl. el0_udp_dns_probe + udpdemo cases; force-fail exit 1; qemu-smoke: ok. SVC surface: net_udp_dns=26 (keep 24/25). FAT /udpdemo. Kernel owns virtio-net. DNS probe bait only. Not TCP/sockets/DHCP/DNS product/Wi‑Fi/“has networking.” CloudAgent HELD — evidence tagged agent-box. Tip 4c9e8a8. ADR-071. |
| Threat-model v1.46 slice (NFR-10) | Read security.md (ADR-071 N3.x EL0 UDP DNS SVC; I/O surface still blk+net) | Verified | File + this PR. v1.46 update on tip post-#116 v1.45, not “secure.” Not BSD sockets. Not “has networking.” |
| scripts/qemu-smoke.sh (ADR-071 N3.x / this branch) | Ran on this agent box | Verified | 2026-09-16 agent-box qemu-smoke on feat/n3x-el0-udp-svc: markers libctos: udp-hi / libctos: udp-ok / udpdemo: ok + preserved N1/N2/N3/N4/samples/ADR-069; qemu-smoke: ADR-071 udp-libctos /udpdemo strings present; Running 126 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not a product stack. Tip 4c9e8a8. |
| UDP DNS path CNTPCT (ADR-072 / P-PERF-9) | Hello serial perf: udp-dns ticks=<n> (prefix grep; not exact ticks); #[test_case] el0_udp_dns_cntpct_advances sample > 0; smoke rejects percent marketing; keep N1/N2/N3/N3.x/N4 + ADR-069 | Verified | 2026-09-16 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/udp-dns-cntpct: hello perf: udp-dns ticks=30943 + keep net: udp-ok / libctos: udp-ok / udpdemo: ok / perf: net-ping / blk: ok / fat: ok / slot: ok; Running 127 tests all [ok] incl. el0_udp_dns_cntpct_advances (perf: udp-dns ticks=65700 in that case); force-fail exit 1; qemu-smoke: ok. QEMU TCG lab — not a published bench, not a percent, not a DNS SLA. Window = quiet ARP+UDP DNS as SYS_NET_UDP_DNS / el0_udp_dns. CloudAgent HELD — evidence tagged agent-box. Tip 4989d68. ADR-072. |
| Threat-model v1.47 slice (NFR-10) | Read security.md (ADR-072 UDP DNS CNTPCT; keeps N1–N4 + ADR-069) | Verified | File + this PR. v1.47 update on tip post-#117 v1.46, not “secure.” Not a latency SLA. Not “has networking.” |
| scripts/qemu-smoke.sh (ADR-072 UDP DNS CNTPCT / this branch) | Ran on this agent box | Verified | 2026-09-16 agent-box qemu-smoke on feat/udp-dns-cntpct: marker perf: udp-dns ticks=30943 + preserved N1/N2/N3/N3.x/N4/samples/ADR-069; qemu-smoke: ADR-072 udp-dns CNTPCT marker present; Running 127 tests all [ok]; force-fail exit 1; qemu-smoke: ok. Not Docker. Not a product stack. Tip 4989d68. |
| FAT16 mkdir behind thin VFS (ADR-073 / sponsor mile 2) | Hello-kernel serial fat: mkdir + keep fat: ok / write / readdir / delete / grow / slot: ok; #[test_case] fat16_mkdir_root_dir + fat16_mkdir_exists_and_bad_path | Verified | 2026-09-16 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/fat-mkdir: hello fat: mkdir + keep fat: grow / fat: delete / fat: ok / slot: ok / net+sample markers; Running 129 tests all [ok] incl. mkdir cases; force-fail exit 1; qemu-smoke: ok. Guest created FAT16 root dir /fdir (dirent + ./.. cluster) via thin VFS. Not POSIX mkdir. Not a directory tree. Not exFAT/FAT32. No new SVC (EL0 follow-up). CloudAgent HELD — evidence tagged agent-box. Tip b0aa139. ADR-073. |
| Threat-model v1.48 slice (NFR-10) | Read security.md (ADR-073 FAT16 mkdir; keeps N1–N4 + ADR-069/072) | Verified | File + this PR. v1.48 update on tip post-#118 v1.47, not “secure.” Not POSIX mkdir. Not TCP. |
| scripts/qemu-smoke.sh (ADR-073 FAT16 mkdir / this branch) | Ran on this agent box | Verified | 2026-09-16 agent-box qemu-smoke on feat/fat-mkdir: marker fat: mkdir + preserved grow/delete/ok/net/samples; qemu-smoke: FAT16 mkdir (ADR-073) strings present; force-fail exit 1; qemu-smoke: ok. Not Docker. Not POSIX. Tip b0aa139. |
ADR-048 progress checklist
Living audit of ADR-048 against tip main @ be789c5 (or newer). Site mirror: hosting-apps.md. Product “app hosting is done” is Verified under criteria Met×5 (ADR-052). Still not Linux/POSIX/containers/“EL0 isolated.”
| # | Criterion | Status | Tip evidence |
|---|---|---|---|
| 1 | Documented recipes (A8) | Met | Row “Documented sample apps (Track A A8)” Verified; what-can-run.md. |
| 2 | Slot cross-update | Met | Row “A9 cross-update (same app on OS n and n+1)” Verified (cross-update … slot:ok on this OS + ba6541c). Later tips re-probed (ADR-046 / amd64 workstation Docker rows). |
| 3 | No production embed | Met | Row “A2–A4 hello without include_bytes! (ADR-032)” Verified; smoke rejects slot: embed. Probe-only include_bytes! in src/slot.rs only (ADR-046). Tip src/ audit: production A2–A4 FAT-only. |
| 4 | Honesty: no Linux/POSIX/containers | Met | Containers non-goal Verified (ADR-029); docs refuse Linux ABI / POSIX / guest OCI. |
| 5 | Explicit sponsor accept | Met | Sponsor: “address 1. app hosting” after still-Planned review blocked only on #5. Record: ADR-052. Flip PR cites ADR-048 checklist; no self-merge (ADR-002). |
| Track N thin TCP (N5 / ADR-074) | Hello-kernel serial net: tcp-syn + net: tcp-est + net: tcp-tx + net: tcp-rx + net: tcp-ok (+ keep N1 net: ok / N2 net: ping-ok / N3 net: udp-ok / N3.x/N4 samples / FAT); QEMU -netdev user,id=net0,guestfwd=tcp:10.0.2.4:7-cmd:…/tcp-echo-stdio.sh -device virtio-net-device,netdev=net0; #[test_case] virtio_net_tcp_echo_probe | Verified | 2026-09-16 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/n5-thin-tcp: hello net: tcp-syn / net: tcp-est / net: tcp-tx / net: tcp-rx / net: tcp-ok + keep prior net/FAT/sample markers; Running 130 tests all [ok] incl. virtio_net_tcp_echo_probe; force-fail exit 1; qemu-smoke: ok. Probe = TCP active-open + one payload echo via QEMU guestfwd 10.0.2.4:7. Not BSD sockets / listen-accept / TLS/HTTP / DHCP/DNS product / Wi‑Fi / “has networking.” CloudAgent HELD — evidence tagged agent-box. Tip a780f67. ADR-074. |
| Threat-model v1.49 slice (NFR-10) | Read security.md (ADR-074 N5 thin TCP; I/O surface still blk+net) | Verified | File + this PR. v1.49 update on tip post-#119 v1.48, not “secure.” Not BSD sockets. Not “has networking.” |
| scripts/qemu-smoke.sh (ADR-074 N5 thin TCP / this branch) | Ran on this agent box | Verified | 2026-09-16 agent-box qemu-smoke on feat/n5-thin-tcp: markers net: tcp-syn / net: tcp-est / net: tcp-tx / net: tcp-rx / net: tcp-ok + preserved N1–N3/N3.x/N4/FAT/samples; qemu-smoke: virtio-net (N1+N2+N3+N5) strings present; force-fail exit 1; qemu-smoke: ok. Not Docker. Not a sockets product. Tip a780f67. |
| EL0 fs_mkdir + mkdir-libctos catalog (ADR-075) | Hello-kernel serial libctos: mkdir-hi + libctos: mkdir-ok + mkdemo: ok + keep fat: mkdir / prior six samples / net; #[test_case] mkdemo_fat_is_elf + mkdemo_load_from_fat_erets + mkdemo_path_is_fat | Verified | 2026-09-17 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/el0-fs-mkdir: hello libctos: mkdir-hi / libctos: mkdir-ok / mkdemo: fat / mkdemo: mapped / mkdemo: ok + keep fat: mkdir / sample+net markers; Running 133 tests all [ok] incl. mkdemo cases; force-fail exit 1; qemu-smoke: ok. Guest loaded FAT /mkdemo and EL0 issued fs_mkdir (27) creating /edir with Exists→1 / BadPath→2 mapping. Not POSIX mkdir. Not a nested tree. Not EL0 TCP. CloudAgent HELD — evidence tagged agent-box. Tip 1c9c4dd. ADR-075. |
| Threat-model v1.50 slice (NFR-10) | Read security.md (ADR-075 EL0 fs_mkdir; keeps N1–N5 + ADR-073) | Verified | File + this PR. v1.50 update on tip post-#120 v1.49, not “secure.” Not POSIX mkdir. Not nested trees. |
| scripts/qemu-smoke.sh (ADR-075 EL0 fs_mkdir / this branch) | Ran on this agent box | Verified | 2026-09-17 agent-box qemu-smoke on feat/el0-fs-mkdir: markers libctos: mkdir-ok / mkdemo: ok + preserved fat: mkdir / net/samples; qemu-smoke: ADR-075 mkdir-libctos /mkdemo strings present; force-fail exit 1; qemu-smoke: ok. Not Docker. Not POSIX. Tip 1c9c4dd. |
| EL0 net_tcp_echo + tcp-libctos catalog (ADR-076) | Hello-kernel serial libctos: tcp-hi + libctos: tcp-ok + tcpdemo: ok + keep net: tcp-ok / prior seven samples / FAT; #[test_case] tcpdemo_fat_is_elf + tcpdemo_load_from_fat_erets + tcpdemo_path_is_fat + el0_tcp_echo_probe | Verified | 2026-09-17 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/el0-tcp-svc: hello libctos: tcp-hi / libctos: tcp-ok / tcpdemo: fat / tcpdemo: mapped / tcpdemo: ok + keep net: tcp-ok / sample+FAT markers; Running 137 tests all [ok] incl. tcpdemo + el0_tcp_echo_probe; force-fail exit 1; qemu-smoke: ok. Guest loaded FAT /tcpdemo and EL0 issued net_tcp_echo (28) quiet guestfwd echo. Not BSD sockets. Not listen/accept. Not TLS. Not TCP CNTPCT. CloudAgent HELD — evidence tagged agent-box. Tip 3704f2a. ADR-076. |
| Threat-model v1.51 slice (NFR-10) | Read security.md (ADR-076 EL0 TCP echo SVC; keeps N1–N5 + ADR-075) | Verified | File + this PR. v1.51 update on tip post-#121 v1.50, not “secure.” Not BSD sockets. Not “has networking.” |
| scripts/qemu-smoke.sh (ADR-076 EL0 TCP SVC / this branch) | Ran on this agent box | Verified | 2026-09-17 agent-box qemu-smoke on feat/el0-tcp-svc: markers libctos: tcp-ok / tcpdemo: ok + preserved net: tcp-ok / net/samples/FAT; qemu-smoke: ADR-076 tcp-libctos /tcpdemo strings present; force-fail exit 1; qemu-smoke: ok. Not Docker. Not a sockets product. Tip 3704f2a. |
| FAT16 nested mkdir + empty rmdir (ADR-077) | Hello-kernel serial fat: nested + fat: rmdir + keep fat: mkdir / grow / delete / ok / net / samples; #[test_case] fat16_nested_mkdir_and_empty_rmdir + fat16_dir_path_grammar | Verified | 2026-09-17 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/fat-nested-rmdir: hello fat: nested / fat: rmdir + keep fat: mkdir / sample+net markers; Running 139 tests all [ok] incl. nested+rmdir + path-grammar cases; force-fail exit 1 (timeout 180s); qemu-smoke: ok. Guest created /fdir/nest (dirent + ./.. with parent cluster) and empty-rmdir’d it via thin VFS. Not POSIX mkdir/rmdir. Not arbitrary-depth trees. No new SVC. Not TCP CNTPCT. CloudAgent HELD — evidence tagged agent-box. Tip 0025721. ADR-077. |
| Threat-model v1.52 slice (NFR-10) | Read security.md (ADR-077 nested mkdir + empty rmdir; keeps N1–N5 + ADR-073/075/076) | Verified | File + this PR. v1.52 update on tip post-#122 v1.51, not “secure.” Not POSIX mkdir/rmdir. Not TCP CNTPCT. |
| scripts/qemu-smoke.sh (ADR-077 nested+rmdir / this branch) | Ran on this agent box | Verified | 2026-09-17 agent-box qemu-smoke on feat/fat-nested-rmdir: markers fat: nested / fat: rmdir + preserved fat: mkdir / net/samples; qemu-smoke: FAT16 nested+rmdir (ADR-077) strings present; force-fail exit 1; qemu-smoke: ok. Not Docker. Not POSIX. Tip 0025721. |
| Thin TCP echo path CNTPCT (ADR-078 / P-PERF-10) | Hello serial perf: tcp-echo ticks=<n> (prefix grep; not exact ticks); #[test_case] el0_tcp_echo_cntpct_advances sample > 0; smoke rejects percent marketing; keep N1–N5 / N5.x + ADR-069/072/076/077 | Verified | 2026-09-17 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a57) on feat/tcp-echo-cntpct: hello perf: tcp-echo ticks=1124937 + keep net: tcp-ok / libctos: tcp-ok / tcpdemo: ok / perf: net-ping / perf: udp-dns / fat: nested / fat: rmdir / samples; Running 140 tests all [ok] incl. el0_tcp_echo_cntpct_advances (perf: tcp-echo ticks=1028684 in that case); force-fail exit 1 (timeout 180s); qemu-smoke: ok. QEMU TCG lab — not a published bench, not a percent, not a TCP SLA. Window = quiet ARP+thin TCP guestfwd as SYS_NET_TCP_ECHO / el0_tcp_echo (EL0 surface; kernel N5 untimed). CloudAgent HELD — evidence tagged agent-box. Tip 8f08f88. ADR-078. |
| Threat-model v1.53 slice (NFR-10) | Read security.md (ADR-078 TCP echo CNTPCT; keeps N1–N5 + ADR-069/072/076/077) | Verified | File + this PR. v1.53 update on tip post-#123 v1.52, not “secure.” Not a latency SLA. Not “has networking.” |
| scripts/qemu-smoke.sh (ADR-078 TCP echo CNTPCT / this branch) | Ran on this agent box | Verified | 2026-09-17 agent-box qemu-smoke on feat/tcp-echo-cntpct: marker perf: tcp-echo ticks=1124937 + preserved net/samples/FAT/ADR-069/072/076/077; qemu-smoke: ADR-078 tcp-echo CNTPCT marker present; Running 140 tests all [ok]; force-fail exit 1 (timeout 180s); qemu-smoke: ok. Not Docker. Not a sockets product. Tip 8f08f88. |
| PAN CPU reopen foundation (ADR-079 / #125) | Default -cpu cortex-a76; hello serial pan: id= + pan: present; smoke rejects pan: enabled / unexpected pan: absent; #[test_case] pan_present_on_probe_cpu | Verified (ID-field on new default) | 2026-09-19 agent-box research + smoke (QEMU 10.0.13): short probe table in ADR-079; hello pan: present on cortex-a76. Enable not Verified (ADR-080). inject-nmi still machine does not provide NMIs (ADR-081). CloudAgent HELD — agent-box. Tip 634ed8d. |
| Threat-model v1.54 slice (NFR-10) | Read security.md (ADR-079 PAN CPU reopen; enable still locked) | Verified | File + this PR. v1.54 update on tip post-#124 v1.53, not “secure.” Not PAN enabled. Not “EL0 isolated.” |
| scripts/qemu-smoke.sh (ADR-079 PAN CPU reopen / this branch) | Ran on this agent box | Verified | 2026-09-19 agent-box qemu-smoke on isolation/adr-079-pan-cpu-reopen: markers pan: id= / pan: present + preserved net/FAT/samples; qemu-smoke: PAN capability strings present; force-fail exit 1; qemu-smoke: ok. Not Docker. Tip 634ed8d. |
| PAN enable + EL1-vs-EL0 fault (ADR-080 / #125 mile 2) | Hello serial pan: enabled + pan: el1-fault (+ keep pan: id= / pan: present); smoke requires enable markers; #[test_case] pan_enable_el1_fault_on_probe_cpu; syscall uaccess clears PAN around copies | Verified | 2026-09-19 agent-box (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a76): hello pan: id=2 / pan: present / pan: enabled / pan: el1-fault; SPSR.PAN at fault; net/FAT/samples preserved. Umbrella still non-claim. CloudAgent HELD — agent-box. Tip 765a8d0. |
| Threat-model v1.55 slice (NFR-10) | Read security.md (ADR-080 PAN enable + EL1-vs-EL0 fault; umbrella still non-claim) | Verified | File + this PR. v1.55 update on tip post-#126 v1.54, not “secure.” Not “EL0 isolated.” |
| scripts/qemu-smoke.sh (ADR-080 PAN enable / this branch) | Ran on this agent box | Verified | 2026-09-19 agent-box qemu-smoke on isolation/adr-080-pan-enable-fault: markers pan: enabled / pan: el1-fault + preserved net/FAT/samples; qemu-smoke: PAN capability + enable + el1-fault strings present; force-fail exit 1; qemu-smoke: ok. Not Docker. Tip 765a8d0. |
| Taken lower-EL SError reopen (ADR-081 / #125 mile 3) | Dated QMP/HMP matrix on virt TCG + cortex-a76 (+ GICv3 / virtualization=on / secure=on / max): inject-nmi → machine does not provide NMIs; KVM absent; FEAT_NMI ≠ SError | Deferred / non-goal (locked; reopen unmet) | 2026-09-19 agent-box QEMU 10.0.13. Research brief research/daily-briefs/2026-09-19-adr-081-serror-inject-probe.md. Unlock = ADR-081 B1/B2/B3; B1 research ADR-082 (2026-09-19): no shippable pin yet. Park Verified; A-clear dormant prep. Do not invent Verified. ADR-081. |
| Threat-model v1.56 slice (NFR-10) | Read security.md (ADR-081 taken-SError re-probe still blocked; umbrella still non-claim) | Verified | File + this PR. v1.56 update on tip post-#127 v1.55, not “secure.” Not taken SError Verified. Not “EL0 isolated.” |
| scripts/qemu-smoke.sh (ADR-081 docs / this branch) | Ran on this agent box | Verified | 2026-09-19 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a76) on isolation/adr-081-taken-serror-reopen: hello el0: serror-arm / qmp inject-nmi => error:machine does not provide NMIs (ADR-045/081 dormant; park if not taken) / el0: serror-park (no bare el0: serror); pan: enabled / pan: el1-fault; net/FAT/samples green; force-fail exit 1; qemu-smoke: ok. Tip 903c7448d508ae0338c64868aac2319941f3c4cb. |
| Taken lower-EL SError B1 QEMU pin research (ADR-082 / #125) | Sponsor chose B1; dated upstream+agent-box research: virt through QEMU master still has no TYPE_NMI→async SError; apt/git pin alone does not unlock; draft local patch research-only | Deferred / non-goal (locked; B1 unmet this session) | 2026-09-19 agent-box QEMU 10.0.13. Probe scripts/qemu-nmi-probe.py + brief research/daily-briefs/2026-09-19-adr-082-b1-qemu-nmi.md. Blocker: wait upstream or sponsor-reviewed pinned QEMU build with proven el0: serror. Park Verified; do not require el0: serror. Not B2/B3. ADR-082. |
| Taken lower-EL SError B1 pinned QEMU (ADR-083 / #125) | Pin QEMU v10.0.0 + TYPE_NMI→SError patch; CTOS_QEMU + CTOS_REQUIRE_TAKEN_SERROR=1 smoke shows bare el0: serror; stock QEMU still parks | Verified (pin; CI job unconditional since ADR-090, 2026-09-26) | 2026-09-19 agent-box: build 7c949c53 + patch; probe inject-nmi ok; smoke qemu-smoke: taken 'el0: serror' present + qemu-smoke: ok. Stock 10.0.13: inject error + el0: serror-park. (amd64 workstation) intent <machine-id>; CloudAgent HELD — proof on agent-box. Not umbrella isolated. ADR-083. |
| Threat-model v1.58 slice (NFR-10) | Read security.md (ADR-083 pinned QEMU opt-in taken SError; umbrella still non-claim) | Verified | File + this PR. v1.58 on tip post-#129 v1.57. Not “secure.” Not “EL0 isolated.” |
| scripts/qemu-smoke.sh (ADR-083 pin / this branch) | Ran on this agent box with pin | Verified | 2026-09-19 agent-box CTOS_QEMU=tools/qemu-nmi/bin/qemu-system-aarch64 CTOS_REQUIRE_TAKEN_SERROR=1: taken el0: serror; force-fail exit 1; qemu-smoke: ok. Default path (no env) still park. |
| Threat-model v1.57 slice (NFR-10) | Read security.md (ADR-082 B1 research blocker; umbrella still non-claim) | Verified | File + this PR. v1.57 update on tip post-#128 v1.56, not “secure.” Not taken SError Verified. Not “EL0 isolated.” |
| scripts/qemu-smoke.sh (ADR-082 docs / this branch) | Ran on this agent box | Verified | 2026-09-19 agent-box qemu-smoke (QEMU 10.0.13, rustc 1.100.0-nightly 0fc141305, -cpu cortex-a76) on isolation/adr-082-b1-qemu-nmi-research: expect el0: serror-arm / inject-nmi error / el0: serror-park (no bare el0: serror); pan: enabled / pan: el1-fault; net/FAT/samples green; force-fail exit 1; qemu-smoke: ok. Hello el0: serror-arm / el0: serror-park (no bare el0: serror); pan: enabled / pan: el1-fault; net/FAT green; force-fail exit 1; qemu-smoke: ok. Tip ac61efbdd97a84f20aef67568d772d7e6aaabf44. |
| Taken lower-EL SError via B2 on free GitHub-hosted runners (ADR-084; not the B1 pin) | Temp workflow b2-serror-probe on probe/adr-084-b2-kvm: /dev/kvm + -accel kvm on ubuntu-24.04-arm / ubuntu-22.04-arm; -accel hvf on macos-15; VT/WHPX on windows-11-arm; x86 ubuntu-24.04 control; stock TCG inject-nmi matrix + all-machine sweep | Failed | 2026-09-25 CEST GHA runs 36193423604 + 36193613972 (all 5 jobs green on run 2; the probe lost). arm64 Linux: kvm [1]: HYP mode not available, /dev/kvm=absent, Could not access KVM kernel module. macOS M1 (Virtual): HV_UNSUPPORTED. Windows arm64: VMMonitorModeExtensions : False, WHPX Disabled. x86: /dev/kvm present but invalid accelerator kvm for AArch64. TCG everywhere: machine does not provide NMIs; sweep inject-nmi-ok=0 (118/101/117 machines). Free (public repo, standard runners). No self-hosted / paid / larger runner. Raw lines research/daily-briefs/2026-09-25-adr-084-b2-probe.txt. ADR-084. |
| P-SEC-3r B2 path (non-B1: different runner/accelerator delivering async SError) | arm64 host exposing KVM + QEMU 0002 KVM_SET_VCPU_EVENTS serror_pending on el0: serror-arm + fail-closed smoke requiring bare el0: serror | Verified (opt-in, one-off paid run) | 2026-09-26: sponsor chose B2-P; ADR-085 run 4 on AWS Graviton3 c7g.metal (eu-north-1b, Spot) PASS. Not a CI gate: no ctos CI job runs KVM, and re-running needs paid metal or a self-hosted arm64 KVM runner (B2-S, not registered). Free-runner probe stays Failed (ADR-084). B1 opt-in pin is not B2. Not “EL0 isolated.” |
| Threat-model v1.59 slice (NFR-10) | Read security.md (ADR-084 B2 free-runner spike blocked; umbrella still non-claim) | Verified | File + this PR. v1.59 on tip post-#130 v1.58. Not “secure.” Not taken SError Verified via B2. Not “EL0 isolated.” |
| CI on GitHub (ADR-084 docs / this PR) | .github/workflows/smoke.yml + pages.yml on isolation/adr-084-b2-free-runner-serror @ 5c4c38b (PR #136) | Verified | 2026-09-26 CEST PR run 36193979804 success: ubuntu-24.04-arm job 108265450556 (native arm64 runner, TCG) inject-nmi=>error:machine does not provide NMIs + el0: serror-park + qemu-smoke: ok; ubuntu-24.04 job 108265450597 success; opt-in qemu-nmi-pin job 108265450350 (B1, ADR-083) inject-nmi=>ok + bare el0: serror + qemu-smoke: taken 'el0: serror' present. Pages run 36193979996 mdBook build success. The B1 pin job is not B2 evidence. |
| Taken lower-EL SError via B2-P: real arm64 + KVM (ADR-085; not the B1 pin) | AWS c7g.metal (Graviton3 Neoverse-V1, CPU: All CPU(s) started at EL2, /dev/kvm=present); QEMU v10.0.0 + 0001 + 0002 -accel kvm -machine virt,gic-version=3 -cpu host; cargo build --features b2-serror; scripts/b2-kvm-smoke.py (fail-closed) via no-SSH user-data; serial console via ec2 get-console-output --latest | Verified (opt-in, one-off paid run) | 2026-09-26 13:26–13:33 CEST run 4 <instance-id-run4> (eu-north-1b Spot, head 159b178): qmp stop=>ok inject-nmi=>ok cont=>ok (ADR-085: KVM_SET_VCPU_EVENTS serror_pending) → el0: standing / el0: serror / el0: restored / b2: taken → b2-kvm-smoke: PASS taken lower-EL SError observed under accel=kvm. Excerpt: research/daily-briefs/2026-09-26-adr-085-b2p-console.txt. Limits: b2 profile only (no GIC/timer/virtio/FAT under KVM); no host ioctl trace; not CI. Not “EL0 isolated.” |
| B2-P metal runs 0–3 (ADR-085 bring-up) | Same host class; earlier heads | Failed | Run 0 m6g.metal <instance-id-run0> and run 1 c7g.metal <instance-id-run1>: cloud-init HOME unset, so cargo: command not found (patched QEMU did build with kvm on Graviton2). Run 2 <instance-id-run2>: zero guest UART bytes (3×90 s). Run 3 <instance-id-run3>: reached b2: p1 mmu-on, then PC=ffffff8040080a00 (high VBAR+0x200) PSTATE=a00003c5 EL1h. A pre-MMU-spin-lock control behaved the same, so that hypothesis was refuted. Root cause: TCR_EL1.TG1 reserved. All fail-closed, none claimed. |
| TCR_EL1.TG1 = 4 KiB explicit (ADR-085; real-hardware fix) | src/paging.rs TCR_TG1_4K = 0b10 << 30 (was reserved 0b00; TCG sanitised it to 4K) | Verified | Real Graviton3 run 3 stalled on the first TTBR1 fetch; run 4 with the fix printed ident: jump … ident: ram … pages=32150 under KVM. TCG unchanged: GHA run 36238549525 at 159b178 green (ubuntu-24.04-arm + ubuntu-24.04 el0: serror-park + qemu-smoke: ok; qemu-nmi-pin bare el0: serror). |
| Threat-model v1.60 slice (NFR-10) | Read security.md (ADR-085 B2-P Verified opt-in on Graviton3 KVM; TG1 fix; umbrella still non-claim) | Verified | File + this PR. v1.60 on the ADR-084 v1.59 branch. Not “secure.” Not “EL0 isolated.” Not “runs on real hardware” beyond the b2 profile. |
| AWS B2-P spike resources (ADR-085) | Tag audit project=ctos in eu-north-1 after runs: DescribeInstances (non-terminated) + resourcegroupstaggingapi GetResources + SG/key-pair/IAM check | Verified | See research/daily-briefs/2026-09-26-adr-085-b2p-graviton-kvm.md §Cleanup. Metal ≈43 min Spot, ≈ $0.19 total. |
| CI on GitHub (ADR-085 / PR #137) | .github/workflows/smoke.yml on isolation/adr-085-b2p-graviton-kvm-serror @ 159b178 (code tip incl. TG1 fix) | Verified | 2026-09-26 CEST push run 36238549525 success: ubuntu-24.04-arm job 108394690860 and ubuntu-24.04 job 108394690937 inject-nmi=>error:machine does not provide NMIs + el0: serror-park + qemu-smoke: ok; opt-in qemu-nmi-pin job 108394690974 (B1) inject-nmi=>ok + qemu-smoke: taken 'el0: serror' present. The docs-only commit on top is covered by the PR checks. CI never runs KVM, so it is not B2 evidence. |
| Umbrella checklist audit (ADR-086) | Every ADR-055 / ADR-060 row vs this-session probes: agent-box qemu-smoke on 915796f (2026-09-26 13:59–14:04 CEST, QEMU 10.0.13, cortex-a76 TCG, qemu-smoke: ok) + GHA PR run 36239619910 (13:41–13:53 CEST, all jobs success) + ADR-085 run 4 | Verified (audit) / umbrella Planned / non-claim | ADR-086: §A 13/13 miles + PAN enable Verified, CI-gated by qemu-smoke.sh. §B not Met: taken SError only on opt-in paths (B1 qemu-nmi-pin needs repo var CTOS_BUILD_QEMU_NMI=1; B2-P one-off, not CI), and the accepted smoke machine still parks. Full identity teardown Unmet and unmeetable as worded (yank _start vs ADR-047 never-yank). No umbrella accept ADR. Sponsor decisions D1–D5. Not “EL0 isolated.” |
| Residual TTBR0 identity inventory (ADR-086 scratch probe) | 915796f + scratch walker patch (not committed to src/); walks kernel + user TTBR0 before Hello World! | Documented (scratch; ratchet Planned M1) | 2026-09-26 ~14:10 CEST box run (output): kernel TTBR0 identity = MMIO 1 GiB Device block 0x0–0x4000_0000; low RAM 0x4000_0000–0x4008_0000 RW; stub page 0x4008_0000 RO+X (start-stay); padding tail [.ident_tear end, 0x4020_1000) RO+X; __kernel_end page RW. User TTBR0: stub page + padding tail (EL1-only). Not a CI gate until M1 commits ident: inv. |
| Threat-model v1.61 slice (NFR-10) | Read security.md (ADR-086 audit; residual identity inventory; umbrella still non-claim) | Verified | File + this PR. v1.61 on the ADR-085 v1.60 branch. Not “secure.” Not “EL0 isolated.” |
| CI on GitHub (ADR-086 docs / PR #138) | .github/workflows/smoke.yml + pages.yml on isolation/adr-086-el0-isolated-audit @ 3f4de00 | Verified | 2026-09-26 14:13–14:26 CEST PR run 36241262508 + push run 36241255248 success: ubuntu-24.04-arm + ubuntu-24.04 default smoke (park + qemu-smoke: ok), opt-in qemu-nmi-pin (B1) pass, Pages mdBook build 36241262505 pass. Docs-only. |
| Identity leftovers I2/I4/I5 tear + TTBR0 identity inventory ratchet (ADR-087, M1) | Hello serial ident: low/tail/kend lo=… + ident: low/tail/kend-fault + ident: inv-range … + ident: inv k=2 u=1 a=2 leaks=0 + ident: inv-neg k/u caught + ident: inv-neg clean + ident: inv-ok allow=mmio,stub; smoke rejects ident: inv-leak; inv-leak-probe kernel must be caught; #[test_case] identity_leftovers_torn_stub_stays + identity_inventory_allowlist_only_catches_plant | Verified | 2026-09-26 14:35–14:40 CEST agent-box qemu-smoke: ok (QEMU 10.0.13, cortex-a76 TCG, rustc 0fc141305): low 128 pages, tail 270 (+270 user), kend 1; kernel TTBR0 identity ranges 5 → 2 (MMIO block, _start stub), user 2 → 1 (stub). inv-leak-probe: ident: inv-leak k lo=0x40000000 + no ident: inv-ok → qemu-smoke: inv-leak-probe caught. MMIO + stub still identity-mapped: not full teardown. Not “EL0 isolated.” |
| Threat-model v1.62 slice (NFR-10) | Read security.md (ADR-087 residual identity = MMIO block + stub; ratchet) | Verified | File + this PR. v1.62 on the ADR-086 v1.61 branch. Not “secure.” Not “EL0 isolated.” |
| CI on GitHub (ADR-087 M1 / PR #139) | .github/workflows/smoke.yml on isolation/adr-087-identity-inventory-ratchet @ 6fcdc4e | Verified | 2026-09-26 14:41–14:52 CEST push run 36242707571 success: ubuntu-24.04-arm job 108406107386 + ubuntu-24.04 job 108406107365 ident: inv k=2 u=1 a=2 leaks=0 / ident: inv-ok allow=mmio,stub / qemu-smoke: inv-leak-probe caught (fail-closed ok) / qemu-smoke: ok; opt-in qemu-nmi-pin job 108406107228 inv-ok + taken el0: serror. PR run 36242713730 default jobs success. |
| Device MMIO TTBR1 alias + identity MMIO block tear (ADR-088, M2) | Hello serial ident: mmio-high gic=0xffffff8008000000 uart=0xffffff8009000000 virtio=0xffffff800a000000 + ident: mmio lo=0x0 hi=0x40000000 l1=1 + ident: mmio-fault va=0x9000018 + ident: inv k=1 u=1 a=1 leaks=0 + ident: inv-ok allow=stub; smoke rejects any inv-range k|a lo=0x0; #[test_case] identity_mmio_torn_high_alias_xn; wx: ok via mmio_xn | Verified | 2026-09-26 ~14:43 CEST agent-box qemu-smoke: ok. Only the _start stub page stays identity-mapped (kernel, user, ASID-B; EL1 RO+X). b2 profile TCG-booted only; B2-P KVM evidence predates M1/M2 (re-run ≈ $0.2, not done). Not “EL0 isolated.” |
| Threat-model v1.63 slice (NFR-10) | Read security.md (ADR-088 MMIO high alias; identity = stub only) | Verified | File + this PR. Not “secure.” Not “EL0 isolated.” |
| ADR-055 §B row 2 reworded: full identity teardown except the _start stub page (ADR-089, M3, sponsor D2) | Default-smoke inventory ident: inv k=1 u=1 a=1 leaks=0 + ident: inv-ok allow=stub + ident: start-stay (ADR-087/088) | Met (this session; not the umbrella) | Sponsor D2 relayed 2026-09-26 ~14:35 CEST. Box smoke ~14:43 CEST on ADR-088; CI on #140 recorded separately. ADR-047 §3 amended in place. Umbrella still needs taken-SError evidence class (M4) + sponsor accept (M5 draft → sponsor). Not “EL0 isolated.” |
| Threat-model v1.64 slice (NFR-10) | Read security.md (ADR-089 stub exception) | Verified | File + this PR. Not “secure.” Not “EL0 isolated.” |
| CI on GitHub (ADR-088 M2 / PR #140) | .github/workflows/smoke.yml on isolation/adr-088-mmio-high-alias @ 09076d4 | Verified | 2026-09-26 14:56–15:10 CEST push run 36243493356 + PR run 36243501093 all success. ubuntu-24.04-arm job 108408303774: ident: mmio-high gic=0xffffff8008000000 uart=0xffffff8009000000 virtio=0xffffff800a000000 / ident: mmio-fault va=0x9000018 / ident: inv k=1 u=1 a=1 leaks=0 / ident: inv-ok allow=stub / qemu-smoke: inv-leak-probe caught (fail-closed ok) / qemu-smoke: ok. Pin job 108408283187: inject-nmi=>ok → bare el0: serror + inv-ok allow=stub. Not “EL0 isolated.” |
| CI on GitHub (ADR-089 M3 docs / PR #141) | smoke.yml + pages.yml on docs/adr-089-stub-exception @ 8093721 | Verified | 2026-09-26 14:56–15:10 CEST push run 36243529890 all success (both default jobs + pin job 108408379969); mdBook build success. Not “EL0 isolated.” |
| ADR-055 §B row 1 reworded: taken lower-EL SError on the D1 evidence class (ADR-090, M4, sponsor D1) | B1 qemu-nmi-pin (pinned QEMU v10.0.0 + patch 0001, CTOS_REQUIRE_TAKEN_SERROR=1, bare el0: serror fail-closed), now unconditional on every push/PR, + B2-P ADR-085 run 4 (Graviton3 KVM, el0: serror + b2: taken, one-off at 159b178) | Met (D1 evidence class; not the umbrella) | Sponsor D1 relayed 2026-09-26 ~14:35 CEST. Workflow if: vars.CTOS_BUILD_QEMU_NMI removed (a skipped job would pass a D5 required check); job renamed “QEMU NMI pin smoke (taken SError)”. Stock QEMU still parks (el0: serror-park). B2-P predates M1/M2 (KVM re-verify ≈ $0.2, not done). First unconditional run: CI on this PR. Umbrella still needs the sponsor’s written accept (B3). Not “EL0 isolated.” |
| Threat-model v1.65 slice (NFR-10) | Read security.md (ADR-090 taken-SError evidence class) | Verified | File + this PR. Not “secure.” Not “EL0 isolated.” |
| Umbrella accept draft (ADR-091, M5, sponsor D4) — ADR-055 §B row 3 | Proposed umbrella sentence + scope S1–S7 + non-scope + gaps G1–G3 + M6 prerequisites, quoting ADR-090 §1 verbatim | Documented (Draft); row 3 not Met; umbrella Planned / non-claim | Drafted 2026-09-26. B3 needs the sponsor’s written accept after reading this draft; merging the draft is not acceptance. Found while drafting: no exhaustive EL0-permission walk (G1) and no EL0 write probe (G2); B2-P re-verify optional (G3, ≈ $0.2). Not “EL0 isolated.” → Accepted 2026-09-28 (M6 row below). |
| Threat-model v1.66 slice (NFR-10) | Read security.md (ADR-091 draft scope / non-scope) | Verified | File + this PR. Not “secure.” Not “EL0 isolated.” |
| CI on GitHub (ADR-090 M4 / PR #142) | smoke.yml on isolation/adr-090-serror-accepted-machine @ be829fe | Verified | 2026-09-26 15:12–15:28 CEST push run 36244384336 + PR run 36244391658 all success. First unconditional pin job 108410713515 (“QEMU NMI pin smoke (taken SError)”, 15:12–15:26): inject-nmi=>ok → bare el0: serror + ident: inv-ok allow=stub + qemu-smoke: ok. Default jobs 108410713544 (arm) / 108410713462 (x86). Not “EL0 isolated.” |
| CI on GitHub (ADR-091 M5 draft / PR #143) | smoke.yml + pages.yml on docs/adr-091-el0-isolated-accept-draft @ b6f324a | Verified | 2026-09-26 15:13–15:28 CEST PR run 36244433643 (arm 108410849837, x86 108410849682, pin 108410849707) + push run 36244425884 all success; mdBook build (run 36244433661) success. Not “EL0 isolated.” |
| EL0-reachability walk, fail-closed (ADR-092 G1) | Every valid leaf of kernel / user / ASID-B TTBR0 + TTBR1; EL0-reachable = AP[1] or UXN clear; allowlist = five 4 KiB user slots; pool frame outside heap; not Device; not EL0 W+X; nothing in TTBR1. Live (hello-libctos standing) + steady + 4 plants + reach-leak-probe build | Verified | Agent box 2026-09-27 00:20–00:23 CEST qemu-smoke: ok: el0-reach: live k=3 u=3 a=0 h=0 pages=6 leaks=0, el0-reach: steady k=0 u=0 a=0 h=0 pages=0 leaks=0, el0-reach: neg kernel-pa/va/ttbr1/el0-wx caught, el0-reach: neg clean, el0-reach: ok allow=…; leak build el0-reach: leak k/u lo=0x80007000 … why=kernel-pa → caught. CI: row below. Not “EL0 isolated.” |
| EL0 write → permission fault (ADR-092 G2) | EL0 STR to kernel .data via TTBR1 and to an EL0-RO user page: lower-EL data abort, EC 0x24, WnR 1, DFSC perm, FAR = target, ELR = STR, target unchanged; write-leak-probe build must report the completed store | Verified | Agent box 2026-09-27 00:20–00:23 CEST: el0: write-fault kernel va=0xffffff8040201008 esr=0x9200004f far=0xffffff8040201008 ec=0x24 wnr=1 dfsc=perm-l3, el0: write-fault user-ro va=0x80009000 esr=0x9200004f far=0x80009000 …, el0: write-ok kernel,user-ro; leak build el0: write-succeeded user-ro va=0x80009000 esr=0xf2000000 → caught. _start not targeted. Not “EL0 isolated.” |
| Syscall copy helpers check EL0 permission (ADR-092 G4) | user_range_ok_max would need EL0 read/write permission on each user leaf | Documented gap (code reading; not probed, not fixed) → closed by ADR-094 (next row) | Today it accepts any valid user-TTBR0 leaf and copy_user clears PAN. So SYS_UART_WRITE could copy the EL1-only _start stub page or an EL1-only window page; copy_to_user could write the app’s execute-only text. ADR-091 revision 2 limits the sentence to direct EL0 access and lists G4 for the sponsor. |
| Threat-model v1.67 slice (NFR-10) | Read security.md (ADR-092 EL0 reach walk + write fault; G4) | Verified | File + this PR. Not “secure.” Not “EL0 isolated.” |
| CI on GitHub (ADR-092 / PR #144) | smoke.yml + pages.yml on isolation/adr-092-el0-reach-walk @ ab158cb | Verified | 2026-09-27 00:25–00:40 CEST push run 36276236937 + PR run 36276246462 all success; mdBook (run 36276246435) success. ubuntu-24.04-arm job 108499427750, ubuntu-24.04 job 108499427771 and pin job 108499427726 (“QEMU NMI pin smoke (taken SError)”, also bare el0: serror) each show el0: write-fault kernel va=0xffffff8040203008 esr=0x9200004f far=0xffffff8040203008 ec=0x24 wnr=1 dfsc=perm-l3, el0: write-fault user-ro va=0x80009000 …, el0: write-ok kernel,user-ro, el0-reach: live k=3 u=3 a=0 h=0 pages=6 leaks=0, el0-reach: steady k=0 u=0 a=0 h=0 pages=0 leaks=0, el0-reach: neg clean, el0-reach: ok allow=…, qemu-smoke: el0-leak-probe caught (fail-closed ok) (leak build: el0: write-succeeded user-ro …, el0-reach: steady k=1 u=1 a=0 h=0 pages=2 leaks=2), qemu-smoke: ok. Not “EL0 isolated.” |
| Syscall pointers need EL0 permission on every page (ADR-094 G4) | user_range_ok_max requires EL0-readable (AP[1]) for copies from EL0 and EL0-writable (AP[1], !AP[2]) for copies to EL0, per page; loader maps app text+rodata EL0 RO+X (map_el0_text); probes: control + kernel-stub / kernel-window read + xo-text / RO write + straddle + 7-syscall sweep, each needing exactly one refusal by the check; sysptr-leak-probe build | Verified | Agent box 2026-09-27 (CEST) local qemu-smoke: ok: el0: sys-ptr control rw-data n=6, el0: sys-ptr denied kernel-stub, … kernel-window, … xo-text, … write-ro, … straddle, el0: sys-ptr denied sys=uart_write nr=17 read kernel-stub … sys=net_mac nr=24 write xo-text (7), el0: sys-ptr sweep syscalls=7 denied=7, el0: sys-ptr ok; leak build el0: sys-ptr leaked kernel-stub n=8, … kernel-window n=8, … xo-text n=6 intact=false checks=0, el0: sys-ptr skip write-ro → caught. G1 allowlist unchanged (5 slots); el0-reach: live k=3 u=3 a=0 h=0 pages=6 leaks=0 unchanged. App ELF sha256 unchanged (A9 cross-update ok). CI: row below. _start only a refused read target. Not “EL0 isolated.” |
| Umbrella accept draft revision 3 (ADR-091, after ADR-094) | Sentence drops “direct EL0 memory access only”; adds syscall-pointer clause (S9); G3 the only remaining optional limit | Documented (Draft); row 3 not Met; umbrella Planned / non-claim | Standing order applied (sponsor skipped the G4 question). Merging is not acceptance. Not “EL0 isolated.” → Accepted 2026-09-28 (M6 row below). |
| Threat-model v1.68 slice (NFR-10) | Read security.md (ADR-094 syscall pointer EL0 permission) | Verified | File + this PR. Not “secure.” Not “EL0 isolated.” |
| CI on GitHub (ADR-094 / PR #146) | smoke.yml + pages.yml on isolation/adr-094-sys-ptr-perm @ 77cae2a | Verified | 2026-09-27 13:25–13:41 CEST PR run 36315704190 (ubuntu-24.04 job 108609879034 13:25–13:35, ubuntu-24.04-arm job 108609879082 13:25–13:36, pin job 108609879083 “QEMU NMI pin smoke (taken SError)” 13:25–13:40) + push run 36315678386 (ubuntu-24.04 108609807133 13:25–13:35, ubuntu-24.04-arm 108609807340 13:25–13:36, pin 108609807322 13:25–13:41) all success; mdBook (run 36315704200, 13:25–13:26) success. Arm job log shows all 15 el0: sys-ptr … lines (control, 5 denied, 7-syscall sweep, sweep syscalls=7 denied=7, ok), qemu-smoke: syscall pointer EL0-permission strings present (ADR-094), qemu-smoke: el0-leak-probe caught (fail-closed ok; reach + write + sys-ptr), qemu-smoke: ok. Not “EL0 isolated.” |
| Public-identifier scrub (ADR-093) | git grep for the replaced values on this branch + scripts/check-public-ids.sh | Verified | ADR-093: account ID replaced in 11 files, plus hosted-zone, instance, Spot, SG, IAM-user and local-path identifiers (17 content files). Values remain in git history; cleanup is a separate sponsor decision. Not “secure,” not “compliant.” |
| Public-identifier guard is fail-closed (ADR-093) | sh scripts/check-public-ids.sh --self-test (planted synthetic tokens, 7 classes) + scan of unmodified main must FAIL | Verified | Self-test ok; unmodified main FAIL hits=77 (every value line, no extras); this branch hits=0. Wired as first step of qemu-smoke.sh (CI + Docker). Bare 12-digit numbers without context are not flagged (see ADR limits). |
| CI on GitHub (ADR-093 / PR #145) | .github/workflows/smoke.yml + pages.yml on security/scrub-public-ids @ d53eb29 | Verified | PR smoke 36310073042 + push smoke 36310067233: ubuntu-24.04 / ubuntu-24.04-arm / NMI pin all success, 2026-09-27 11:39–11:52 CEST; logs show public-ids: self-test ok classes=7 + public-ids: ok … hits=0 before qemu-smoke: ok. Pages build 36310073058 success (deploy skipped on PR). Docs-only follow-up commit adds this row. |
| CI on GitHub (ADR-093 / PR #145, after merging main @ 8107578) | smoke.yml + pages.yml on security/scrub-public-ids @ 4d644d6 (merge of main into the branch; conflicts in SUMMARY / ledger / MOC resolved by keeping both sides) | Verified | 2026-09-28 13:30-13:45 CEST PR run 36416030136 (ubuntu-24.04 job 108907277988 13:30-13:40, ubuntu-24.04-arm job 108907278179 13:30-13:42, pin job 108907277537 “QEMU NMI pin smoke (taken SError)” 13:30-13:45) + push run 36416025183 (ubuntu-24.04 108907261648 13:30-13:38, ubuntu-24.04-arm 108907261918 13:30-13:40, pin 108907261888 13:30-13:44) all success; mdBook (run 36416030182, job 108907277345, 13:30) success, deploy skipped on PR. First run with the three D5-required checks on this PR. Pin job log: public-ids: self-test ok classes=7, public-ids: ok files=451 hits=0 (whole merged tree, including ADR-087..094 files from the stack; no new scrubs needed), inject-nmi=>ok, bare el0: serror, qemu-smoke: ok. Docs-only follow-up commit adds this row. |
| Taken lower-EL SError via B2-P re-run on current main (ADR-095, G3; not the B1 pin) | Same as ADR-085 run 4: AWS c7g.metal (Graviton3 Neoverse-V1, CPU: All CPU(s) started at EL2, /dev/kvm=present, KVM API 12, host kernel 7.0.0-1013-aws); QEMU v10.0.0 + 0001 + 0002 -accel kvm -machine virt,gic-version=3 -cpu host; cargo build --features b2-serror at main 8107578; scripts/b2-kvm-smoke.py (fail-closed) via no-SSH user-data (in-tree script + small delta in ADR-095); serial console via ec2 get-console-output --latest | Verified (opt-in, one-off paid run) | 2026-09-28 13:50–13:58 CEST (eu-north-1b Spot, all resources tagged project=ctos / component=g3-kvm-recheck, deleted and verified): CTOSB2: head 8107578a6c715a89c9e5861ebb4102460ba20e80 → ident: mmio-high … / ident: low|tail|kend … / ident: mmio lo=0x0 hi=0x40000000 l1=1 / pan: id=2 / pan: enabled / pan: el1-fault → b2: boot → qmp stop=>ok inject-nmi=>ok cont=>ok (ADR-085: KVM_SET_VCPU_EVENTS serror_pending) → el0: standing / el0: serror / el0: restored / b2: taken → b2-kvm-smoke: PASS taken lower-EL SError observed under accel=kvm (attempt 1). Cost ≈ $0.04. Excerpt: research/daily-briefs/2026-09-28-adr-095-g3-console.txt. Closes ADR-091 G3. Limits: b2 profile only (no GIC/timer/virtio/FAT; no inventory / reach / write / sys-ptr probes under KVM); no host ioctl trace; not CI. ADR-091 still Draft, B3 not Met. Not “EL0 isolated.” |
| Threat-model v1.69 slice (NFR-10) | Read security.md (ADR-095 B2-P KVM re-run on current main) | Verified | File + this PR. No posture change. Not “secure.” Not “EL0 isolated.” |
| CI on GitHub (ADR-095 / PR #147) | smoke.yml + pages.yml on isolation/adr-095-g3-kvm-recheck @ 31cdb1d (ADR-095 commit 5af2fd3 + merge of main e1ee4a8, ADR-093) | Verified | 2026-09-28 14:19–14:37 CEST PR run 36421105682 (ubuntu-24.04 job 108923761266 14:19–14:29, ubuntu-24.04-arm job 108923761452 14:19–14:30, pin job 108923761116 “QEMU NMI pin smoke (taken SError)” 14:19–14:36) + push run 36421101861 (ubuntu-24.04 108923747720 14:19–14:28, ubuntu-24.04-arm 108923747783 14:19–14:30, pin 108923747527 14:19–14:33) all success; mdBook (run 36421105718, job 108923761567, 14:19) success. Arm job: public-ids: self-test ok classes=7 / public-ids: ok files=453 hits=0, el0: serror-park, qemu-smoke: ok. PR pin job: qmp stop=>ok inject-nmi=>ok cont=>ok → bare el0: serror → qemu-smoke: taken 'el0: serror' present (ADR-083 pinned QEMU) → qemu-smoke: ok. The first head 5af2fd3 also passed its push runs (36419823465: jobs 108919578949 / 108919579243 / 108919579461; 36419825002: jobs 108919583154 / 108919583418 / 108919583519; 14:07–14:25 CEST); its PR run did not start because main had moved (ADR-093 merge) and the PR conflicted in the append-only ledger, resolved by merging main. Docs only. Not “EL0 isolated.” |
| Umbrella “EL0 isolated” (M6 flip; ADR-091 Accepted; ADR-055 §B row 3) | Sponsor’s written accept (B3) of the revision-3 sentence and scope + ADR-055 §A/§B all Met + D5 branch protection (three required checks) + #139–#147 merged (main 68d4c8f, push CI green) + the M6 PR CI run as the one-session re-probe | Verified (scoped: the accepted ADR-091 sentence only) | B3 verbatim: “I accept the ADR-091 revision-3 sentence and scope as written. After M6, have ctos propose a short costed list of next milestones.” 2026-09-28 16:41 CEST; relay path: sponsor wrote it in the DSO agent chat, DSO relayed it to ctos. The claim is only this sentence: “On the ctos default smoke machine (QEMU virt, -cpu cortex-a76, TCG, one core), EL0 is isolated from the kernel at the architectural page-table / exception-level boundary: each EL0 task runs on its own TTBR0 with its own ASID; EL0 faults when it reads, writes or executes kernel memory; in every translation table (kernel, user and ASID-B TTBR0, and TTBR1) the only EL0-reachable pages are five allowlisted user slots backed by non-kernel frames, checked fail-closed while a user app runs and at rest; PAN is enabled and EL1 faults on EL0 memory; lower-EL IRQ and FIQ are taken while EL0 stands; taken lower-EL SError is evidenced by the ADR-090 evidence class; the only identity (VA = PA) mapping left in any TTBR0 is the documented EL1-only _start stub page (ADR-089); and a system call copies from or to an EL0 pointer only if EL0 itself may read (for a copy from EL0) or write (for a copy to EL0) every page in the range. It is not a speculative-execution or side-channel claim, not a real-hardware claim, and not a certification.” Limits: one core, the TCG default smoke machine; not a speculative-execution or side-channel claim, not a real-hardware claim, not a certification; EL0 can read its own code bytes; no multi-core race story. ADR-091 non-scope stands. Earlier rows that say “Not ‘EL0 isolated’” record their own PR’s scope at the time. ADR-091. Note 2026-10-02: this row records revision 3 as accepted at M6. The current claim is the revision-4 sentence (revision-4 row below); only the PAN clause and the code-bytes limit differ. |
| Threat-model v1.70 slice (NFR-10) | Read security.md (M6: umbrella “EL0 isolated” Verified only as the ADR-091 sentence) | Verified | File + this PR. Not “secure.” Not wider than the ADR-091 sentence. |
| CI on GitHub (M6 / PR #148, one-session re-probe of the ADR-091 sentence) | smoke.yml + pages.yml on isolation/m6-el0-isolated-flip @ 6956d31 (the M6 flip commit on main 68d4c8f) | Verified | 2026-09-28 16:59–17:16 CEST PR run 36440169580 (ubuntu-24.04 job 108988297915 16:59–17:07, ubuntu-24.04-arm job 108988297602 16:59–17:11, pin job 108988298105 “QEMU NMI pin smoke (taken SError)” 16:59–17:16) + push run 36440144452 (ubuntu-24.04 108988211209 16:59–17:10, ubuntu-24.04-arm 108988211425 16:59–17:10, pin 108988211602 16:59–17:16) all success; mdBook (run 36440169616, job 108988298049, 16:59) success, deploy skipped on PR. Every smoke job: ident: inv k=1 u=1 a=1 leaks=0 / ident: inv-ok allow=stub, el0-reach: live k=3 u=3 a=0 h=0 pages=6 leaks=0 / el0-reach: ok allow=…, el0: write-fault kernel … / el0: write-fault user-ro … / el0: write-ok kernel,user-ro, pan: enabled / pan: el1-fault, el0: sys-ptr sweep syscalls=7 denied=7 / el0: sys-ptr ok, qemu-smoke: ok. Default jobs: el0-reach: neg clean, ident: inv-neg clean, el0: serror-park, public-ids: self-test ok classes=7 / public-ids: ok files=454 hits=0. Pin jobs: el0: serror-arm, inject-nmi=>ok, bare el0: serror. Docs only; this row is a follow-up docs-only commit. |
| Task-to-task EL0 isolation probe (ADR-096) | Serial t2t: walk b-frame u-refs=0 u-el0=0 a-el0=1 / t2t: b-own ok asid=2 … / t2t: a-read fault asid=1 va=0x80007000 … got=0x0 after-b no-tlbi / t2t: a-write fault … b-intact=true / t2t: same-va … ×4 / t2t: ok asid-a=1 asid-b=2 va=0x80007000 read,write,same-va; #[test_case] el0_task_cannot_read_or_write_other_task; t2t-leak-probe build must report t2t: leak walk / t2t: leak read … got=0x7a5cb0b0 / t2t: leak write and withhold t2t: ok | Verified (scoped) | Task B (ASID-B TTBR0, ASID 2) writes sentinel 0x7a5cb0b0 to its private page at 0x80007000. Task A (user TTBR0, ASID 1) then loads and stores the same VA with no TLBI of it in between. Both take a lower-EL translation fault (FAR = the VA), A never sees the sentinel, and B’s frame is unchanged. A’s root has no leaf for B’s frame. With A’s own frame at the same VA, each task reads and writes only its own frame. Fail-closed in scripts/qemu-smoke.sh: every marker is required, t2t: leak / t2t: bad / t2t: probe missed are rejected, and the sentinel on any task-A line is rejected. A local negative test of the check block (missing t2t: ok, sentinel on an A line, got= changed on the read line, probe missed, the leak-build serial) failed each time. Limits: one core on the TCG default smoke machine. QEMU TCG flushes its whole software TLB when a TTBR write changes the ASID, so this is not a hardware stale-TLB/ASID-tagging test. It uses two kernel-built tasks, not two loaded apps under a scheduler. It is not a side-channel or speculative claim. Additional evidence only: not part of the ADR-091 sentence. Task-vs-task stays in ADR-091’s non-scope, and a revision 4 needs the sponsor in writing. |
| Threat-model v1.71 slice (NFR-10) | Read security.md (ADR-096 task-to-task probe section + mitigation row) | Verified | File + this PR. Not “secure.” The ADR-091 sentence is unchanged. |
| CI on GitHub (ADR-096 / PR #149, task-to-task EL0 isolation probe) | smoke.yml + pages.yml on isolation/t2t-probe @ 7439607 | Verified | PR run 36476725257: ubuntu-24.04 job 109112172166 (22:05–22:16 CEST), ubuntu-24.04-arm job 109112171989 (22:05–22:16), pin job 109112172193 “QEMU NMI pin smoke (taken SError)” (22:05–22:22). Push run 36476682111: ubuntu-24.04 109112035715 (22:05–22:15), ubuntu-24.04-arm 109112035659 (22:05–22:16), pin 109112035424 (22:05–22:21). All success. mdBook (run 36476725338, job 109112172352, 22:05) success; deploy skipped on PR. Every smoke job printed all nine t2t: lines at boot and again in cargo test (el0_task_cannot_read_or_write_other_task), then qemu-smoke: task-to-task EL0 isolation strings present (ADR-096). The leak build printed t2t: leak walk b-frame u-refs=1 u-el0=1, t2t: leak read asid=1 va=0x80007000 got=0x7a5cb0b0, t2t: leak write … and t2t: probe missed, then qemu-smoke: el0-leak-probe caught (fail-closed ok; reach + write + sys-ptr + t2t). Every job also printed public-ids: ok … hits=0 and qemu-smoke: ok. Local: the full smoke passed in QEMU TCG on a Linux box. A Docker Desktop smoke on the Windows workstation printed all nine t2t: lines, but stopped earlier at net probe missed (UDP step); main 9a1c2bb fails the same way there, so it is not caused by this change. Docs only; this row is a follow-up docs-only commit. |
| CI on GitHub (ADR-096 / PR #149 refreshed onto main 3c47c0c, i.e. after ADR-098 / #151) | smoke.yml + pages.yml on isolation/t2t-probe @ e6e7574 (plain merge of main into the branch; no rebase, no force-push) | Verified | PR run 37000471274: ubuntu-24.04 job 110816770656 (13:20-13:30 CEST), ubuntu-24.04-arm job 110816771290 (13:20-13:31), pin job 110816770893 “QEMU NMI pin smoke (taken SError)” (13:20-13:37). Push run 37000465958: ubuntu-24.04 110816754112 (13:20-13:31), ubuntu-24.04-arm 110816754218 (13:20-13:31), pin 110816754276 (13:20-13:37). All success. mdBook (run 37000471286, job 110816771395) success; deploy skipped on PR. The merged smoke script requires both the nine t2t: lines and the ADR-098 pan: live-syscall / pan: after-el0 / pan: ok held lines. The combined el0-leak-probe build enables t2t-leak-probe and pan-keep-leak-probe together; the t2t: leak ... lines and the pan: lost ... lines are each required individually and t2t: ok / pan: ok held are rejected there. Local: full ./scripts/qemu-smoke.sh on a Linux box (QEMU 10.0.13 TCG) printed qemu-smoke: ok, public-ids: ok files=459 hits=0, all nine t2t: lines, the three pan: lines and both leak families caught. Tampered-log check: deleting the t2t: ok line, and separately the pan: ok held line, each made the checker fail. Docs only; this row is a follow-up docs-only commit. |
| Correction: ADR-091 clause “PAN is enabled” was not true while the kernel handled EL0 exceptions (ADR-098) | Pre-fix behaviour reproduced by the pan-keep-leak-probe build: pan: enabled + pan: el1-fault (boot) and pan: lost live-syscall pstate-pan=0 sctlr-span=1 + pan: lost after-el0 trips=… mrs-pan=0 … fault=no spsr-pan=0 | Verified (defect found and fixed) | From the ADR-080 enable (2026-09-19), and so also from the M6 accept (2026-09-28 16:41 CEST) until this fix, the “PAN is enabled” clause held at boot but not while handling EL0 exceptions: SCTLR_EL1.SPAN was 1, so every exception from EL0 entered EL1 with PAN clear, and return_from_el0 resumed the kernel with SPSR 0x3C4 (PAN clear), so PAN stayed clear after EL0 trips until the next with_user_access. The boot probe missed this because it ran before any EL0 trip. The M6 umbrella row stays Verified only with this correction; the sentence is unchanged and true again with ADR-098. |
| PAN held during EL0 exception handling + after EL0 trips (ADR-098) | Serial pan: live-syscall pstate-pan=1 sctlr-span=0 (in the loaded app’s SYS_YIELD handler) / pan: after-el0 trips=<n≥1> mrs-pan=1 el1-load va=0x80003000 fault=yes spsr-pan=1 / pan: ok held boot,live-syscall,after-el0; #[test_case] pan_span_cleared_after_enable; pan-keep-leak-probe build must report pan: lost … and withhold pan: ok held | Verified (scoped) | Fix: SCTLR_EL1.SPAN=0 after the ADR-080 enable; SPSR.PAN set on return_from_el0. MRS PAN calibrated both ways (1 with a real EL1 fault, 0 in the negative build without one). Tampered-log tests: 7/7 edited logs fail, unedited passes. One core, TCG; one exception type and one post-trip point sampled. |
| Threat-model v1.72 slice (NFR-10) | Read security.md (ADR-098 PAN correction section + mitigation row) | Verified | File + this PR. v1.71 is ADR-096 (PR #149), numbered before this slice and merged after it. Not “secure.” ADR-091 sentence unchanged. |
| EL0 execute-only app text (ADR-097; merge requires sponsor rev 4 written accept) | Serial xo: live app=hello text va=0x80002000 ap=2 uxn=0 pxn=1 el0=x reach=1 hdr … rodata=0x80000120 in-text=0 sys-read text=deny rodata=allow / xo: layout loader app-hdr … app-text va=0x80002000 ap=2 uxn=0 pxn=1 el0=x reach=uxn / xo: el0-exec ok … got=0x584f524441544131 / xo: el0-read-text fault va=0x80002000 esr=0x9200000f far=0x80002000 ec=0x24 wnr=0 dfsc=perm-l3 got=0x0 / xo: el0-uart rodata-str ok / xo: sys-read rodata ok … / xo: sys-read text denied sys=uart_write nr=17 va=0x80002000 n=0 checks=1 / xo: el1-pan … ×2 / xo: el1-write text … fault=yes intact=true / xo: reach xo-plant caught … ap-el0=0 uxn=0 / xo: ok text=el0-xo rodata=app-hdr el0-read=fault sys-read=deny slots=5; #[test_case] el0_app_text_is_execute_only; host tests loader_plans_hello_as_ro_hdr_and_xo_text / loader_rejects_ro_on_text_page / loader_rejects_pages_outside_app_slots; xo-leak-probe build must report xo: leak … and withhold xo: ok | Verified (scoped; branch only until merged; update 2026-10-02: accepted by the sponsor as ADR-091 rev 4 and merged as 8344797) | App .rodata moved onto the app-hdr page (EL0 RO + NX); .text alone on app-text, mapped EL1 RO / EL0 none / UXN=0. Still five slots; allowlist and el0-reach: lines byte-identical. EL0 loads of its text fault, text pointers are refused by the ADR-094 check. EPAN finding: ID_AA64MMFR1_EL1.PAN=2 (no FEAT_EPAN) on QEMU 10.0.13 cortex-a76 (box and CI); with PAN set an EL1 load of the XO page does not fault (fault=no match=true). So the ADR-091 clause “PAN is enabled and EL1 faults on EL0 memory” is not literally true for that page: merge needs a sponsor rev 4 written accept. ADR-091 not edited. Tampered-log tests: 10/10 edited logs fail, unedited passes (ADR-097 §Negative tests). One core, TCG; not real hardware. |
| Threat-model v1.73 slice (NFR-10) | Read security.md (ADR-097 execute-only app text section + mitigation row) | Verified | File + this PR (was v1.72 before the ADR-098 PAN fix was split out and took v1.72). Not “secure.” ADR-091 sentence unchanged; merge requires sponsor rev 4 written accept. Update 2026-10-02: accepted and merged as 8344797; threat model is v1.74 (row below). |
| CI on GitHub (ADR-097 / PR #150, EL0 execute-only app text; stacked on #149; merge requires sponsor rev 4 written accept) | smoke.yml + pages.yml on isolation/xo-text @ e1135b6 | Verified | PR run 36485127669: ubuntu-24.04 job 109140067947 (23:17–23:29 CEST), ubuntu-24.04-arm job 109140068167 (23:17–23:29), pin job 109140068292 “QEMU NMI pin smoke (taken SError)” (23:17–23:35). Push run 36485081546: ubuntu-24.04 109139912849 (23:17–23:27), ubuntu-24.04-arm 109139913176 (23:17–23:28), pin 109139913104 (23:17–23:32). All success. mdBook (run 36485127464, job 109140067363, 23:17) success; deploy skipped on PR. Every smoke job: public-ids: ok files=460 hits=0; all thirteen xo: lines at boot (incl. xo: el0-read-text fault va=0x80002000 esr=0x9200000f … dfsc=perm-l3 got=0x0, xo: el1-pan text … fault=no match=true mmfr1-pan=2 epan=no, xo: pan live-syscall pstate-pan=1 sctlr-span=0, xo: ok … slots=5) and again in cargo test (152 tests, el0_app_text_is_execute_only); qemu-smoke: EL0 execute-only text strings present (ADR-097); the leak build printed xo: leak live text ap=3 el0=rx …, xo: leak el0-read-text va=0x80002000 got=0xf9400001d2800001, xo: leak sys-read text n=8 checks=0, xo: bad pan live-syscall pstate-pan=0 sctlr-span=1, xo: probe missed and was caught (… + t2t + xo)); qemu-smoke: ok. |
| CI on GitHub (ADR-097 / PR #150 refreshed onto main 3c47c0c via the refreshed #149 branch; merge requires sponsor rev 4 written accept) | smoke.yml + pages.yml on isolation/xo-text @ c63d712 (plain merge of isolation/t2t-probe @ e6e7574; no rebase, no force-push) | Verified | PR run 37001810303: ubuntu-24.04 job 110820981036 (13:35-13:45 CEST), ubuntu-24.04-arm job 110820980926 (13:35-13:45), pin job 110820980975 “QEMU NMI pin smoke (taken SError)” (13:35-13:48). Push run 37001811973: ubuntu-24.04 110820986072 (13:35-13:44), ubuntu-24.04-arm 110820985824 (13:35-13:45), pin 110820986091 (13:35-13:51). All success. mdBook (run 37001810402, job 110820981516) success; deploy skipped on PR. The merged smoke script requires the t2t:, xo: and ADR-098 pan: lines; the combined el0-leak-probe build enables t2t-leak-probe, xo-leak-probe and pan-keep-leak-probe together, and each family’s leak lines are required individually (t2t: ok, xo: ok and pan: ok held are rejected there). Local: full ./scripts/qemu-smoke.sh on a Linux box (QEMU 10.0.13 TCG, 153 tests) printed qemu-smoke: ok, public-ids: ok files=462 hits=0, all t2t:, xo: (incl. xo: pan live-syscall pstate-pan=1 sctlr-span=0) and pan: lines, and all three leak families caught (. + t2t + xo + pan)). Tampered-log check: deleting t2t: ok, xo: ok, xo: pan live-syscall . and pan: ok held (one at a time) each made the checker fail. Docs only; this row is a follow-up docs-only commit. |
| Umbrella “EL0 isolated” revision 4 (ADR-091 rev 4 accepted; ADR-097 Accepted; docs only) | Sponsor’s written accept, verbatim “I accept ADR-091 rev 4” (2026-10-02; written in the DSO chat, recorded in the DSO vault, relayed by DSO; answer to the exact wording in ADR-097) + PR #150 (ADR-097, XO app text) merged as 8344797 + PR #151 (ADR-098, PAN held in EL0 exception handling) on main as 3c47c0c. No code change in this PR; the xo: and pan: probes were already CI-gated by those PRs | Verified (scoped: the revision-4 ADR-091 sentence only) | Only the PAN clause and the code-bytes limit changed from revision 3. The claim is only this sentence: “On the ctos default smoke machine (QEMU virt, -cpu cortex-a76, TCG, one core), EL0 is isolated from the kernel at the architectural page-table / exception-level boundary: each EL0 task runs on its own TTBR0 with its own ASID; EL0 faults when it reads, writes or executes kernel memory; in every translation table (kernel, user and ASID-B TTBR0, and TTBR1) the only EL0-reachable pages are five allowlisted user slots backed by non-kernel frames, checked fail-closed while a user app runs and at rest; PAN is enabled, including while the kernel handles exceptions from EL0, and EL1 faults on EL0-readable memory (the execute-only app text page is not covered: this CPU has no FEAT_EPAN); lower-EL IRQ and FIQ are taken while EL0 stands; taken lower-EL SError is evidenced by the ADR-090 evidence class; the only identity (VA = PA) mapping left in any TTBR0 is the documented EL1-only _start stub page (ADR-089); and a system call copies from or to an EL0 pointer only if EL0 itself may read (for a copy from EL0) or write (for a copy to EL0) every page in the range. It is not a speculative-execution or side-channel claim, not a real-hardware claim, and not a certification.” Limits: one core, the TCG default smoke machine; not a speculative-execution or side-channel claim, not a real-hardware claim, not a certification; app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN); no multi-core race story. ADR-091 non-scope stands. The status rests on the existing probes (pan: live-syscall …, pan: ok held …, xo: ok …, xo: el1-pan text … fault=no … epan=no), not on this PR’s text; this PR’s CI is the re-probe. ADR-091, ADR-097. |
| Threat-model v1.74 slice (NFR-10) | Read security.md (ADR-091 revision 4: PAN clause + code-bytes limit; v1.73 section marked accepted) | Verified | File + this PR. Docs only. Not “secure.” Not wider than the revision-4 sentence. |
| KVM re-check in CI via GitHub OIDC (ADR-099, milestone item 2) | ADR-099: trust and permission policy shape (placeholders), results-out decision (console output, no PassRole), teardown and cost cap, Spot capacity story, risk table; inert .github/workflows/kvm-recheck.yml + scripts/kvm-recheck-userdata.sh | Planned | Documented design sketch, not implemented. The workflow has never run: dispatch-only, not scheduled, not a required check, fails closed while KVM_ROLE_ARN is unset. No AWS call was made. CloudTrail, the OIDC provider, the Spot service-linked role and the exact immutable-subject sub format are not verified; the DSO vault decision record was not read. Nothing about EL0 isolation changes. |
| ADR-099 files present (docs only) | ADR-099, .github/workflows/kvm-recheck.yml, scripts/kvm-recheck-userdata.sh in this PR; YAML parse, actionlint + shellcheck, bash -n, guard run with simulated variables (no AWS) | Verified | File presence and local lint only. Says nothing about AWS behaviour. Not “secure.” |
| ADR-099 pilot spec (2026-10-13, sponsor decision 20:49 CEST via DSO) | ADR-099 “Pilot spec” | One supervised, manual-dispatch run only; environment kvm-pilot; component tag kvm-ci-pilot; region eu-north-1; actions SHA-pinned; exact immutable sub with numeric IDs supplied to DSO out of band. Spot price is not IAM-enforceable (no condition key); capped by MaxPrice guard and budget. No AWS call by the agent; environment and role not created. CloudTrail (P1) met: DSO verified a multi-region trail, home region eu-north-1, from the arm64 workstation on 2026-10-13 about 20:56 CEST. |
| SMP race-story plan + global-state inventory (ADR-100) | rg of static items under src/ on main af5a9d1 + reading the owner modules | Documented (code reading; not a probe of race freedom) | 262 statics: 25 static mut, 10 spin-mutex, 208 atomics, 19 immutable. Riskiest: local-only TLBI, page-table static mut under a convention lock, fixed ASIDs, global EL0/expect flags, per-core PAN/SPAN/VBAR. Nothing here is “SMP-safe”. |
| Secondary CPU bring-up smoke, opt-in (ADR-100 S1, --features smp2-probe, QEMU -smp 2) | Serial smp: cpu0 mpidr=0x0000000080000000 / smp: secondary up mpidr=0x0000000080000001 / smp: cpu_on target=0x1 rc=0 / smp: ok cpu1 mpidr=0x0000000080000001 parked; negative: same ELF under -smp 1 → rc=-2, smp: cpu1 missed seen=0x0, checker fails; default run: no smp: line | Verified (local Linux box, QEMU 10.0.13 TCG, 2026-10-13; CI row below) | Secondary runs MMU off, prints, stores MPIDR to one flag word, parks in wfe. No EL0, no page tables, no heap/scheduler/drivers on cpu1. Not SMP support. ADR-091 rev 4 stays one core. |
| Threat-model v1.75 slice (NFR-10) | Read security.md (ADR-100 SMP plan; one-core non-goal; S1 mitigation row) | Verified | File + this PR. No posture change. Not “secure.” ADR-091 sentence unchanged. |
| CI on GitHub (ADR-100 / PR #154, SMP race story S1) | smoke.yml + pages.yml on smp/adr-100-s1 @ 6d0c324 | Verified | 2026-10-13 ~21:07-21:25 CEST PR run 38452394317 + push run 38452386420: both QEMU aarch64 smoke jobs (ubuntu-24.04-arm, ubuntu-24.04) green with qemu-smoke: default one-core (no smp: markers, ADR-100); QEMU NMI pin smoke (taken SError) green; non-required QEMU -smp 2 bring-up green with smp: secondary up mpidr=0x0000000080000001 / smp: ok cpu1 ... parked and the -smp 1 negative caught (rc=-2, smp: cpu1 missed seen=0x0); mdBook build green. Not SMP support; ADR-091 rev 4 stays one core. |
| Host / user name scrub + guard classes (ADR-101) | ADR-101; sh scripts/check-public-ids.sh --self-test → public-ids: self-test ok classes=15; scan → public-ids: ok … hits=0; same guard on unmodified main af5a9d1 → FAIL hits=151 | Verified (scoped: current tree only) | Forward-only, no history rewrite (sponsor 2026-10-13). Machine names, agent VM / EC2 private / GitHub runner host names and a machine ID replaced by role names or placeholders (counts by class in ADR-101, no values). Two scripts no longer print a host name. Not DLP; history, PR bodies and CI logs not scanned. |
| Repo variable CTOS_BUILD_QEMU_NMI deleted (sponsor 2026-10-13) | gh variable list before: one variable; workflows on main and on #152’s branch grep clean (no vars.CTOS_BUILD_QEMU_NMI since ADR-090 made the pin job unconditional); gh variable delete; gh variable list after: empty (2026-10-13 20:51 CEST) | Verified | GitHub setting, $0. The Dockerfile --build-arg CTOS_BUILD_QEMU_NMI is a separate build argument and is unchanged. |
| ADR-099 pilot setup recorded (DSO, 2026-10-13 22:12-22:16 CEST, on the sponsor’s 22:11 approval) | DSO relay to the agent; text only, no identifier values | Documented (relayed; not probed by this agent, no AWS call) | kvm-pilot environment + OIDC role created by DSO (role tagged project=ctos / component=kvm-ci-pilot, inline policy per ADR-099 incl. hop-limit fix); OIDC provider pre-existing and shared (another project’s Terraform); AMI Ubuntu 24.04 arm64 dated 20261004 (differs from ADR-095); SG no ingress; env: sole reviewer the sponsor, prevent_self_review off, main only, admin bypass off (sponsor 22:15); budget ctos-monthly-5 USD 5/month on project=ctos, alerts 80% actual / 100% forecast. Repo variables still unset; workflow inert. |
| ADR-099 review fixes (Bugbot on PR #152 + pilot findings, 2026-10-13) | Code/text reading of .github/workflows/kvm-recheck.yml + ADR-099; teardown logic exercised with a mocked aws (clean / slow / leak) | Documented (not run on AWS by the agent) | (1) snapshot resource kept out: AWS docs list it for RunInstances, but the live pilot launched without it (public AMI); add only on a concrete denial; (2) sweeper/LEAK counts only this run’s instances, so REFUSED-active-instance is not overwritten; (3) new ABORTED-after-aws-call, NOT-RUN = no AWS call; (4) teardown polls up to 20 min, shutting-down at the end = TEARDOWN-SLOW (red, not LEAK); (5) KVM_SUBNET_IDS accepts spaces and/or commas. |
| KVM pilot smoke under accel=kvm (ADR-099 pilot, relayed by DSO) | Workflow run 38468490030 at main 069fe83, c7g.metal; console markers el0: serror + b2: taken; nonce 685780b675853a95 | Verified (relayed by DSO; smoke PASS) | Run verdict red only from a false LEAK (5-min teardown wait timed out while the metal instance was shutting down); DSO verified the instance terminated. RunInstances worked without a snapshot resource. Reduced b2-serror profile only; not the default image. |
| SMP slice S2 opt-in probe (ADR-102) | ADR-102; CTOS_SMP2_ONLY=1 ./scripts/qemu-smoke.sh on the Linux box: -smp 2 → smp: ok cpu1 s2 mmu-on pan=1 span=0 vbar=own exc=handled console=locked; -smp 1, smp2-neg-pan, smp2-neg-span, smp2-neg-vbar each fail the checker with their miss line | Verified (QEMU TCG, -cpu cortex-a76, -smp 2, opt-in build only) | Core 1 MMU on with shared kernel tables, own stack and vectors, PAN=1 / SPAN=0, one brk handled on core 1. No SMP safety claim; core 1 runs no EL0, scheduler, heap, page-table code, drivers or interrupts. ADR-091 rev 4 (one core) unchanged. Console lock: positive only (no deterministic interleave negative). |
| SMP slice S3 (ADR-103): type-enforced TABLES, broadcast TLBI, cross-core unmap probe | ADR-103; page-table writers need &mut TablesGuard (compile-time); TLBI helpers use dsb ishst; tlbi …is; dsb ish; isb; CTOS_SMP2_SMOKE=1 ./scripts/qemu-smoke.sh on the Linux box: -smp 2 → smp: xtlb unmap cpu0 → cpu1 fault; local-tlbi-leak-probe → smp: xtlb leak (checker fails) | Verified (QEMU TCG, -cpu cortex-a76, -smp 2; leak 27/27 and fault 27/27 runs on the box) | One page, one direction (core 0 unmaps, core 1 faults). Reads of tables stay unlocked. No SMP safety claim; core 1 runs no EL0. ADR-091 rev 4 (one core) unchanged. The leak is a QEMU soft-TLB property, not a hardware claim. |
| Threat-model v1.76 slice (NFR-10) | Read security.md (new section “KVM re-check via GitHub OIDC (ADR-099)”: ADR-099 risk table folded in, as on main after PR #157) | Verified | File + this PR. Docs only. No posture change. Not “secure.” ADR-091 sentence unchanged. |
| SMP slice S4 (ADR-104): PerCpu via TPIDR_EL1, ASID allocator with generations, first EL0 on core 1 | ADR-104; CTOS_SMP2_SMOKE=1 ./scripts/qemu-smoke.sh on the Linux box: -smp 2 → 13 × smp: cpu1 probe <name> ok cpu=1, smp: asidgen alloc cpu0+cpu1 n=300 unique …, smp: asidgen reissue cpu0 rollover → cpu1 fault, smp: ok cpu1 s4 …; S4 leak build (all existing leak features + asid-rollover-local-probe) → every leak reported on cpu1, smp: asidgen leak, checker fails | Verified (QEMU 10.0.13 TCG, -cpu cortex-a76, -smp 2, opt-in build only; positive 20/20, leak 20/20, S3 xtlb leak 20/20) | Probes run on one core at a time (core 0 waits); result flags, user tables and the fatal stack stay shared; allocator not used by the default build. TCG flushes the TLB on any ASID change, so ASID tagging itself is not shown. No multi-core safety claim; ADR-091 rev 4 (one core) unchanged. |
| SMP slice S5 (ADR-105): concurrent EL0, run-queue lock, reserved ASID | ADR-105; CTOS_SMP2_SMOKE=1 ./scripts/qemu-smoke.sh on the Linux box: both cores run EL0; mid-run rollover keeps core 1’s ASID; asid-reserve-drop-probe → smp: s5 asid reissued while running cpu=1; cpu1 fatal on own stack; guards unmapped | Verified (QEMU 10.0.13 TCG, -cpu cortex-a76, -smp 2, opt-in; positive 10/10, reserve-drop 10/10) | No preemption; allocator opt-in only; TCG ASID-tag limit as in ADR-104. No multi-core safety claim; ADR-091 rev 4 unchanged. Multi-core sentence is a separate Proposed ADR. |
| Proposed multi-core claim sentence (ADR-106, docs only) | ADR-106; docs-only PR; no code; does not edit ADR-091 rev 4 | Proposed (not accepted) | Needs the sponsor’s written acceptance. Scope: QEMU TCG, cortex-a76, -smp 2, opt-in `smp2-probe`, with explicit TCG ASID-tag / no-preemption / no-KVM limits. |
| FAT32 read path beyond FAT16 (ADR-107): BPB + FSInfo, FAT32 chains, root as cluster chain, on a second read-only virtio-blk | ADR-107; ./scripts/qemu-smoke.sh: mkfs.fat -F 32 image → fat32: mount clusters=N≥65525 … fsinfo=ok, fat32: read /probe32.txt bytes=9, fat32: chain /big.bin clusters=6 jumps≥1 bytes=3000, fat32: root-chain clusters≥2 last=found, fat32: ok; negative boot with FAT[root] freed → fat32: fail reason=bad-chain, no fat32: ok, FAT16 fat: ok still; cargo test in-memory negatives (free / bad / out-of-range link, cycle, size mismatch, bad FSInfo, FAT16 geometry) | Verified (QEMU virt TCG, this PR’s CI smoke jobs) | Read-only; root directory only; 8.3 names only (LFN skipped); not mounted in the VFS; no write / FSInfo update / FAT mirror check. FAT16 path unchanged. Not hardware. |
| Taken-SError QEMU spike (ADR-108, research) | ADR-108; stand-alone image, research/serror-spike/serror-matrix.py 52/52 ok on stock 8.2.2 / 10.2.1 / master and pinned B1 | Shown: virtual SError (HCR_EL2.VSE, EL2-owned guest) taken from EL0 on cpu0/cpu1 on stock QEMU; pinned nmi targets cpu1. Not shown: any of this in the ctos kernel; physical SError on stock QEMU; bad-PA → SError (it is a sync abort). No claim change. |
| Inbound TCP over QEMU hostfwd (ADR-109 / N6): one passive open on guest port 8080, line → reply → close, RST on closed port 8081 | ADR-109; ./scripts/qemu-smoke.sh: net: tcp-in-listen port=8080 closed=8081, net: tcp-in-rst port=8081, net: tcp-in-syn, net: tcp-in-est, net: tcp-in-rx bytes=11, net: tcp-in-tx bytes=19, net: tcp-in-fin closed, net: tcp-in-ok; host hostfwd-client: neg=reset pos=ok rx=19 eof=yes; cargo test parse / checksum / RST / fragment unit cases | Verified (QEMU virt TCG, SLIRP on host loopback, this PR’s CI smoke jobs) | Kernel path only; one connection; no sockets API / EL0 SVC / retransmit timer / options / IPv6. Not a real-network or security claim. N5 outbound unchanged. |
| N1: taken SError on core 1 at EL0 (ADR-110, opt-in) | ADR-110; scripts/smp2-serror-pin.sh in the qemu-nmi-pin CI job | Shown: pinned QEMU, -smp 2, HMP nmi cpu-index=1 → core 1 takes it at EL0; nmi to cpu0 and A-masked are not taken (fail-closed). Not shown: stock QEMU, hardware, concurrency with S5. No claim change. |
| N2: EL2 shim, taken virtual SError at EL0 on stock QEMU (ADR-111, opt-in) | ADR-111; scripts/el2-vserror-smoke.sh, opt-in CI job | Shown: full image, stock QEMU 8.2.2/10.2.1 virt,virtualization=on + shim → virtual SError (HCR_EL2.VSE, EC 0x2f) taken at EL0; not taken without ask, A-masked, or without EL2. Not shown: physical SError, cpu1, hardware. Default boot has no EL2. No claim change. |
Three pillars
After the cooperative scheduler (M9), ctos treats antifragility, security, and performance as first-class pillars (ADR-011). In everyday words: we turn repeated misses into sensors, we prove security slices instead of saying “secure OS,” and we measure before we tune. They share one rule: a status word needs a probe (honesty ledger).
Principles drive; pillars sit under them; tracks sit under both. Visitor-facing list: landing — Driving principles. In-repo list: principles.md. This page is the deep hub, not a second marketing copy. Extra stance: overview.md, apps-today.md, building-or-porting.md, immutability.md.
flowchart TD P["Core principles<br/>honesty · antifragility · security<br/>performance · document-first"] L["Three pillars — this page<br/>NFR-05 · NFR-10 · NFR-07"] T["Tracks A / B<br/>subordinate workstreams"] P --> L --> T
Tracks (loader/ABI, later slots/FS) do not outrank a principle.
| Pillar | Frozen ID | Home | What “done” looks like |
|---|---|---|---|
| Antifragility | NFR-05 | antifragility.md | Repeated failures become sensors/gates, not extra README advice |
| Security | NFR-10 | security.md | Threat-model v1.29 + probes before any “secure OS” claim. EL0 miles: el0.md (umbrella “EL0 isolated” Verified only as the ADR-091 sentence; standing + TTBR1 first cut + EL1 high-VA fetch + identity .text range tear + live .text tear + identity .rodata/.data/heap tears + PAN ID-field + ASID + SVC ABI + libctos CRT + memfs + FAT16 + A9 slot first cut / cross-update / embed-off are separate rows). App hosting: syscall.md / ADR-048 / ADR-052 (product claim Verified under Met×5; not Linux/POSIX/containers) |
| Performance | NFR-07 | performance.md | Measurable CNTPCT + IRQ-delta + host ELF size + boot-delta; optimize only with a probe |
Bring-up M0–M9 stays on the roadmap. Pillar work after M9 is listed there as a separate section so a docs PR does not pretend to close paging or a scheduler.
Immutability is the same rule: scoped RO+NX / text-tear probes are fine; absolute “immutable OS” is not. Overview: Advantages — Immutability.
Do not import florist / AEA role names. These pillars are ctos-native.
Antifragility SOP
First-class pillar (ADR-011, NFR-05). Hub: pillars.md.
When the same failure happens twice, strengthen the strongest layer (a sensor or a gate), not another paragraph of advice.
Fail closed
- Unprobed boot → Unknown, not Verified.
- Missing nightly / rust-src / QEMU → say the environment blocked the probe; do not invent success.
- MRC / author conflict → do not merge. GitHub
author==mergedByis a process miss after the 2026-09-12 ADR-002 amendment (Track A A1–A9 is a named historical exception, not a going-forward allowance).
Ratchet
- Name the failure once in
research/random-thoughts/(what broke, command, error). - If it repeats, add a sensor: a script, a
#[test_case], a CI check, or a ledger row with a real probe command. - If people keep skipping the sensor, add a gate: CI required, or MRC refuses the PR.
- Only then tighten a guide (
AGENTS.mdor a skill). Guides without sensors rot.
Recent ratchets (keep Failed history)
These are history rows, not a claim that the current tip is broken.
- arm64 workstation Docker on
b2bbb99Failed after PR #20 (paging: probe missed/heap: probe missed/sched: probe missed/el0: probe missedwhile guard/RO still printed). Same class as the PR #20 pre-MMU.bssmiss plus a single shared L3 / first-2-MiB user map. Sensor: layout L3 pool + post-MMU frame init (#21). Docker on71ee15fthen Verified. Keep the Failed row in the honesty ledger. - Live identity
.textyank Failed (unhandled sync on firstprintln!— rustcdyn Writevtables are identity fn pointers). Sensor / cut: ADR-019 kept live.textand tore a 16 KiB dedicated range. ADR-020 then rewrote those vtables to high aliases and tore live.text. The arm64 workstation Docker + GHA one80dc93Verified that cut (ident: reloc n=12, live pages=37, 50 tests). Keep the Failed first yank. - PR #27
ac3ad0bGHA Failed (ident: rangethenident: probe missed— init still required a 4 KiB tear page). Sensor: 16 KiB range + relaxed init. Merge24d94e6GHA 34640667227 Verified. Keep the Failed SHA. - Track A same-login merge (A1–A9, 2026-09-11–2026-09-12).
gh pr view --json author,mergedByon #49, #52–#57, #59, #60 showedartofdreamon both hats (commits typicallycursoragent+Co-authored-by: artofdream). Sensor / docs ratchet: named historical exception in ADR-002; MRC must read the GitHubauthorfield, not a brief’s “expected bot.” Gate going forward: owner-opened →cursor[bot]merges; bot-opened →artofdreammerges. Same-login merge is a miss. Keep this history row; do not rewrite those PRs.
No self-approval / no self-merge
The author does not APPROVE or merge their own PR. Same GitHub login is not a second identity. Co-authored-by / cursoragent on commits does not change the GitHub author.
Going forward: GitHub shows artofdream as author → cursor[bot] merges (owner must not). GitHub shows cursor[bot] as author → artofdream merges (bot must not). MRC writes COMMENT and names author / reviewer / merger from gh, not from chat. See ADR-002 and .cursor/skills/ctos-mr-coordinator/SKILL.md. Do not enable GitHub author self-APPROVE to make the gate “count.”
Historical (Track A A1–A9 only): same-login merges listed in ADR-002. Named exception; not a license to repeat.
Security — threat model v1.76 (NFR-10 / ADR-011 / ADR-012 / ADR-013 / ADR-014 / ADR-015 / ADR-016 / ADR-017 / ADR-018 / ADR-019 / ADR-020 / ADR-021 / ADR-022 / ADR-023 / ADR-024 / ADR-025 / ADR-026 / ADR-027 / ADR-028 / ADR-030 / ADR-032 / ADR-037 / ADR-038 / ADR-039 / ADR-040 / ADR-041 / ADR-042 / ADR-043 / ADR-044 / ADR-045 / ADR-047 / ADR-048 / ADR-049 / ADR-050 / ADR-052 / ADR-053 / ADR-054 / ADR-055 / ADR-056 / ADR-057 / ADR-058 / ADR-059 / ADR-060 / ADR-061 / ADR-062 / ADR-063 / ADR-064 / ADR-065 / ADR-066 / ADR-067 / ADR-068 / ADR-069 / ADR-070 / ADR-071 / ADR-072 / ADR-073 / ADR-074 / ADR-075 / ADR-076 / ADR-077 / ADR-078 / ADR-079 / ADR-080 / ADR-081 / ADR-082 / ADR-083 / ADR-084 / ADR-085 / ADR-086 / ADR-087 / ADR-088 / ADR-089 / ADR-090 / ADR-091 / ADR-092 / ADR-094 / ADR-095 / ADR-096 / ADR-097 / ADR-098 / ADR-099 / ADR-100)
This is a written threat model for a QEMU virt learning kernel. It is not a certification, not an audit, and not a “secure OS” / “hardened” claim. File presence is not W^X. Image W^X is a separate ledger row that needs a QEMU probe.
Version: v1.76 (2026-10-19). Slice/update of v1.75. Docs only, no code change. Folds the ADR-099 risk and impact table (KVM re-check of the reduced b2-serror profile via GitHub OIDC on a Graviton metal Spot instance) into this threat model as section “KVM re-check via GitHub OIDC (ADR-099)”. Source: ADR-099 on main after PR #157 (merged 2026-10-14), so the table includes the #157 changes (run-scoped sweeper, TEARDOWN-SLOW, ABORTED-after-aws-call, snapshot resource kept out). No posture change for the kernel. Not “secure.” ADR-091 rev 4 sentence unchanged.
Prior v1.75 (2026-10-13). Slice/update of v1.74. ADR-100 (milestone item 5, the SMP race story) writes down the multi-core threat: an inventory of global mutable kernel state (262 statics: 25 static mut, 10 spin-mutex-wrapped, 208 atomics, 19 immutable) and the races each would have on a second core. The riskiest are local-only TLB invalidation (no …IS TLBI anywhere), page-table static mut storage guarded only by a convention lock, fixed ASIDs, global EL0-context / expected-fault flags, and per-core PAN/SPAN/VBAR setup. No posture change: the default kernel is one core and the ADR-091 rev 4 sentence (“-cpu cortex-a76, TCG, one core”) is unchanged. S1 adds an opt-in smp2-probe build that powers one secondary on via PSCI CPU_ON (MMU off, prints its MPIDR, parks; no EL0, no page tables, no shared kernel state beyond one flag word). Any multi-core claim needs a new written sponsor accept. Not “secure.”
Prior v1.74 (2026-10-02). Slice/update of v1.73. Docs only, no code change. The sponsor accepted ADR-091 revision 4 in writing (“I accept ADR-091 rev 4”, 2026-10-02; written in the DSO chat, recorded in the DSO vault, relayed by DSO). ADR-097 (PR #150) is Accepted and merged as 8344797; ADR-098 (PR #151) is on main as 3c47c0c. ADR-091 now carries revision 4: only the PAN clause (“including while the kernel handles exceptions from EL0, and EL1 faults on EL0-readable memory (the execute-only app text page is not covered: this CPU has no FEAT_EPAN)”) and the limit (“app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN)”, was “EL0 can read its own code bytes”) changed; every other clause is as accepted at revision 3. Threat-model effect: none on the mechanism; the written claim now matches it (no FEAT_EPAN, so EL1 can still read the execute-only app text page; PAN is held during EL0 exception handling). See below. Not “secure.”
Prior v1.73 (2026-09-29). Slice/update of v1.72 (ADR-098 PAN correction, PR #151), which builds on v1.71 (#149 ADR-096). ADR-097 (proposed at the time; merge required sponsor rev 4 written accept; accepted 2026-10-02, see v1.74): EL0 app text is execute-only. Every app’s .rodata moved onto the ELF-header page (app-hdr slot, EL0 RO + NX); .text alone on app-text is mapped EL1 RO / EL0 none / UXN=0. Still five slots. An EL0 load of its own text takes a permission fault, a syscall given a text pointer is refused, rodata still reads and prints, and the inventory still counts the UXN=0 page. Two findings: (1) the CPU has no FEAT_EPAN (ID_AA64MMFR1_EL1.PAN=2), so with PAN set an EL1 load of the XO text page does not fault; the ADR-091 clause “PAN is enabled and EL1 faults on EL0 memory” is therefore not literally true for that page, which is why merge waits for a sponsor rev 4 accept (ADR-091 is not edited). (2) The pre-existing PAN gap found here (PAN clear during EL0 exception handling and after EL0 trips) is split out and fixed by ADR-098 (v1.72); this branch merges it. Leak builds for both are caught.
Prior v1.72 (2026-09-29). Slice/update of v1.70 (#148 M6); v1.71 is ADR-096 (PR #149), numbered before this slice and merged after it. ADR-098 (correction): PAN was not held while the kernel handled EL0 exceptions (SCTLR_EL1.SPAN was 1) or after EL0 trips (return_from_el0 resumed with SPSR.PAN clear). The ADR-080 boot probe ran before any EL0 trip and missed it, so from the ADR-080 enable (and from the M6 accept) until this fix the ADR-091 clause “PAN is enabled” held at boot but not while handling EL0 exceptions. Fixed (SPAN=0 after enable; SPSR.PAN on return) with fail-closed live checks (pan: live-syscall pstate-pan=1 sctlr-span=0, pan: after-el0 …, pan: ok held …) and a negative build that undoes the fix (boot probe still passes; live checks report pan: lost …). Every clause of the ADR-091 sentence is literally true again; the sentence is unchanged.
Prior v1.71 (2026-09-28). Slice/update of v1.70 (#148 M6). ADR-096: a fail-closed task-to-task probe. An EL0 task on the user root (ASID 1) cannot read or write another EL0 task’s private page on the ASID-B root (ASID 2) at the same VA, right after that task ran (no TLBI of the VA in between). A’s root has no leaf for B’s frame. With its own page at that VA, each task sees and changes only its own frame. A leak build that maps B’s frame into A’s root is caught. Evidence next to the ADR-091 sentence, not part of it: the accepted sentence and its non-scope (task-vs-task) are unchanged. Limits: one core; QEMU TCG flushes its software TLB on an ASID change, so this is not a hardware stale-TLB test; two kernel-built tasks, not two loaded apps.
Prior v1.70 (2026-09-28). Slice/update of v1.69 (#147 ADR-095). M6: the sponsor accepted the ADR-091 revision-3 sentence and scope in writing (B3, 2026-09-28 16:41 CEST). ADR-091 is Accepted. Umbrella “EL0 isolated” is Verified, only as that sentence (below), with its limits: one core, the TCG default smoke machine; not a speculative-execution or side-channel claim, not a real-hardware claim, not a certification; EL0 can read its own code bytes; no multi-core race story. Docs only: no code change; the M6 PR CI run is the re-probe. (Revision 4, 2026-10-02: the limit and the PAN clause were reworded, see v1.74.)
Prior v1.69 (2026-09-28). Slice/update of v1.68 (#146 ADR-094). ADR-095 (G3): the ADR-085 B2-P procedure was re-run on main 8107578 on an AWS Graviton3 c7g.metal under KVM and passed (el0: serror / b2: taken, fail-closed smoke, attempt 1). The B2-P leg of the ADR-090 evidence class now also holds on the post-M1/M2 boot path. No posture change: reduced b2-serror profile, one-off, not CI. Umbrella still Planned / non-claim; ADR-091 is still Draft (B3 not Met).
Prior v1.68 (2026-09-27). Slice/update of v1.67 (#144 ADR-092). ADR-094 closes G4. Syscall user pointers now need EL0 read (copy from EL0) or write (copy to EL0) permission on every page. App text+rodata is mapped EL0 read-only + executable. Probes: refused kernel-stub / kernel-window reads, refused xo-text / RO writes, a straddle, a seven-syscall sweep, and a sysptr-leak-probe build that must fail. G1 allowlist unchanged. Umbrella still Planned / non-claim; ADR-091 revision 3 is still Draft (B3 not Met); G3 is optional.
Prior v1.67 (2026-09-26). Slice/update of v1.66 (#143 ADR-091). ADR-092 (sponsor option (b)): a fail-closed EL0-reachability walk over kernel / user / ASID-B TTBR0 and TTBR1, plus an EL0 store permission-fault probe, close ADR-091 gaps G1/G2. New gap G4: syscall copy helpers skip the EL0 permission check. Umbrella still Planned / non-claim.
Prior v1.66 (2026-09-26). Slice/update of v1.65 (#142 ADR-090). ADR-091 (M5, D4) is a draft umbrella sentence and scope, pending the sponsor’s written accept. No posture change. The umbrella stays Planned / non-claim.
Prior v1.65 (2026-09-26). Slice/update of v1.64 (#141 ADR-089). ADR-090 (M4, sponsor D1): the umbrella’s taken-SError evidence class is the B1 pinned-QEMU CI job (now unconditional) plus the B2-P one-off KVM run. Stock TCG still parks. Not the umbrella: the sponsor accept (B3) is still absent.
Prior v1.64 (2026-09-26). Slice/update of v1.63 (#140 ADR-088). ADR-089 (M3, sponsor D2): the _start stub page is the one documented identity exception, the checklist identity row is reworded accordingly and Met on this session’s evidence. Docs only. Still not “EL0 isolated.”
Prior v1.63 (2026-09-26). Slice/update of v1.62 (#139 ADR-087). ADR-088 (M2, sponsor D3): GIC / PL011 / virtio-mmio are reached only through three EL1-only, XN TTBR1 Device blocks (6 MiB); the identity 1 GiB MMIO block is cleared from kernel and ASID-B TTBR0. The fail-closed inventory now allows only the _start stub page (ident: inv k=1 u=1 a=1 leaks=0). Still not “EL0 isolated.”
Prior v1.62 (2026-09-26). Slice/update of v1.61 (#138 ADR-086). ADR-087 (M1): the RAM-side identity leftovers (low RAM below the image, the RO+X image padding tail, the pre-heap frame) are unmapped from kernel and user TTBR0. A committed fail-closed walk over kernel / user / ASID-B TTBR0 allows only the MMIO 1 GiB Device block and the _start stub page, and a planted identity page is caught (in-boot + inv-leak-probe kernel). The EL1-executable identity surface while EL0 runs shrinks from ≈ 1 MiB + 1 page to the single stub page. Still not “EL0 isolated.”
Prior v1.61 (2026-09-26). Slice/update of v1.60 (#137 ADR-085). ADR-086: sponsor unlocked the umbrella item. The ADR-055/060 checklist audit found every §A mile and PAN enable Verified this session (CI-gated by qemu-smoke.sh). §B is not Met: taken SError is Verified only on opt-in paths (B1 pin, B2-P one-off), while the accepted smoke machine parks. A scratch TTBR0 walker shows five identity ranges left in kernel TTBR0 (MMIO 1 GiB Device block, low RAM below the image, _start stub page, image padding tail RO+X, __kernel_end page) and two in user TTBR0 (stub + tail, EL1-only). No umbrella accept exists. Still not “EL0 isolated.”
Prior v1.60 (2026-09-26). Slice/update of v1.59 (#136 ADR-084). ADR-085: sponsor B2-P. On an AWS Graviton3 c7g.metal (eu-north-1, Spot), QEMU -accel kvm with patch 0002 set serror_pending via KVM_SET_VCPU_EVENTS, and the opt-in b2-serror profile took the lower-EL SError while standing at EL0 (el0: serror / b2: taken; fail-closed smoke PASS). That makes B2 Verified (opt-in, one-off paid run, not CI). Real hardware also exposed a latent TCR_EL1.TG1 reserved-encoding bug that TCG masked; it is fixed (TG1=4K). Stock distro QEMU still parks. B1 pin unchanged. Still not “EL0 isolated”, not “runs on real hardware” beyond the b2 profile, not “secure.”
Prior v1.59 (2026-09-25): Slice/update of v1.58 (#130 ADR-083). ADR-084: sponsor B2 spike (different runner/accelerator) on free GitHub-hosted runners is blocked — no arm64 KVM/HVF/WHPX; stock TCG still machine does not provide NMIs. B2 stays Planned / blocked on a sponsor decision. B1 opt-in pin (ADR-083) unchanged. Stock distro QEMU still parks. Still not “EL0 isolated” / BSD sockets / Linux/containers. Not a v2 model and not “secure.”
Prior v1.58 (2026-09-19): Slice/update of v1.57 (#129 ADR-082). ADR-083 ships opt-in pinned QEMU v10.0.0 + TYPE_NMI→async SError; CTOS_QEMU + CTOS_REQUIRE_TAKEN_SERROR=1 Verifies taken el0: serror. Stock distro QEMU still parks. Default CI/Docker stay on distro QEMU. Keep PAN enable + N1–N5 + samples. Still not “EL0 isolated” / BSD sockets / Linux/containers. Not a v2 model and not “secure.”
Scope
In: the ctos guest as built for qemu-system-aarch64 -machine virt (EL1, identity map + TTBR1 private page + TTBR1 RAM alias for EL1 fetch, PL011, GICv2, CNTP, first-fit heap, cooperative EL1 tasks, a deliberate EL0 first mile and a bounded standing context). The git repo (no secrets).
Out: Raspberry Pi or other boards, a second ISA, multi-tenant hosting, secure / measured boot, physical side channels, a hostile hypervisor. Networking: N1 ARP + N2 ICMP ping + N3 minimal UDP + N3.x EL0 UDP DNS SVC + N4 EL0 net SVCs + N5 thin TCP + N5.x EL0 TCP echo SVC (ADR-066 / ADR-067 / ADR-070 / ADR-071 / ADR-068 / ADR-074 / ADR-076); not a BSD sockets / TCP product stack.
QEMU and the host are the TCB we do not defend against. If the emulator or the CI runner is hostile, the guest cannot recover.
Assets
| Asset | Why it matters | Where it lives |
|---|---|---|
| Kernel integrity | Text, VBAR_EL1 table, page tables. Corruption ends the learning probe. | EL1 identity map (0x4008_0000…__kernel_end) plus TTBR1 RAM alias / high VBAR (ADR-017) |
| Secrets-none | Credentials in git would be a host/CI incident, not a guest exploit. | Repo policy (NFR-10). The guest stores no keys. |
| Availability of the virt guest | Smoke / cargo test must still print markers and exit. A silent lockup hides bugs. | UART, GIC, timer, fail-closed scripts/qemu-smoke.sh |
| Heap + cooperative stacks | Writable RAM used by Box/Vec and M9 workers. Execute-from-here is the W^X question. | Frame pool after __kernel_end (ADR-012) |
| Linker stacks (SP_EL0 / SP_EL1 / fatal) | Thread, first-level exception, and nested-fatal stacks. Live pages are RW+NX with .data (ADR-015). | Linker holes after .bss (ADR-005) |
| Guard pages (when mapped invalid) | 4 KiB unmapped holes below each linker stack. Downward overflow should become a translation fault, not a silent smash of the previous image bytes. | __stack_guard / __exc_stack_guard / __fatal_stack_guard (ADR-014) |
| EL0 trampoline + bait + standing context | One UXN-clear map-window page and a kernel .data bait. Proves enter/return, “cannot execute kernel data,” “cannot read kernel .data,” and a bounded standing dual-SVC. Not POSIX, not isolation. | paging::EL0_PAGE + KERNEL_DATA_BAIT + L1_USER (ADR-013) |
| TTBR1 private page | One EL1-only high page. Proves EL0 cannot load it. Not a relocated kernel. | paging::TTBR1_PRIV (ADR-016) |
| TTBR1 RAM alias / high VBAR | EL1 can fetch .text at identity + TTBR1_BASE. VBAR_EL1 is that alias. Identity -kernel stub stays. | L1_HIGH[1] → cloned L2_HIGH_RAM (ADR-017, ADR-018) |
| Identity-tear page | One dedicated identity text page unmapped from TTBR0. High twin stays. Not a relocated kernel. | __ident_tear_* (ADR-018) |
| Identity text range | Dedicated 16 KiB identity text range unmapped. High twins stay. Not a relocated kernel. | __ident_tear_* 16 KiB (ADR-019) |
Live identity .text | rustc vtables rewritten to high aliases; live identity .text after _start unmapped. High twins stay. Not a relocated kernel. | [0x4008_1000, __text_end) (ADR-020) |
Identity .rodata | Identity .rodata unmapped after a pointer rewrite. High twin stays. Not a relocated kernel. | [__rodata_start, __ident_tear_start) (ADR-025) |
Identity .data / stacks | Identity .data/.bss/linker stacks unmapped after SP relocate + pointer rewrite. High twin stays. Not a relocated kernel. | [__data_start, __kernel_end) (ADR-037) |
| Identity heap | Identity heap unmapped after high GlobalAlloc VAs. High twin stays. Remaining frames after the heap stay identity-mapped. Not a relocated kernel. | 64 KiB first-fit pool (ADR-038) |
| PAN capability + enable | ID_AA64MMFR1_EL1.PAN printed; PSTATE.PAN enabled on cortex-a76; EL1-vs-EL0 fault. Historical a57 enable non-goal (ADR-047). | src/pan.rs (ADR-026 / ADR-079 / ADR-080) |
| Remaining isolation gaps | Shared boot-stub text stays by decision (ident: start-stay; never yank _start). PAN enable Verified on cortex-a76 (ADR-080). Leftover identity RAM torn (ident: ram*, ADR-049). Taken lower-EL SError hard-stopped on stock QEMU after ADR-081 re-probe (el0: serror-park; A-clear dormant prep; ADR-053 / ADR-081 / ADR-082); Verified only under the B1 opt-in pin (ADR-083); B2 free-runner spike blocked (ADR-084). | User TTBR0 + ASID + TTBR1 + torn live .text / .rodata / .data/stacks / heap / leftover frame RAM + EL0 entry without VMALLE1 + lower-EL IRQ while standing + default IRQ-unmasked ERET + lower-EL FIQ while standing + SError park honesty + A-clear dormant prep + start-stay honesty + PAN enable + SVC ABI; umbrella “EL0 isolated” Verified only as the accepted ADR-091 sentence (M6; checklist ADR-055 all Met); sponsor closure table ADR-060 |
| SVC ABI (exit / uart_write / yield) | Documented numbers 16–18. uart_write rejects a kernel .data pointer and a TTBR1 / non-canonical alias. Not Linux. Not app hosting. | src/syscall.rs (ADR-021) |
libctos CRT | Wrappers + _start CRT. Hello payload copied onto the standing EL0 page. Must not issue SVC #0–#2. | libctos/ + src/libctos.rs (ADR-022) |
| Guest ELF PT_LOAD loader | Guest parses embedded ELF64 ET_EXEC, maps PT_LOAD into the user map-window, ERETs to e_entry. Rejects PT_INTERP and W+X. Not a Linux ABI. | src/loader.rs (ADR-023) |
| Standing EL0 as normal mode | Loaded image stands until SYS_EXIT. is_active() is a task flag. Unexpected lower-EL sync restores fail-closed. Not a process table. | src/el0.rs (ADR-024) |
| Thin VFS + memfs | Named heap buffers. Create / open / read / write / close. User path/I/O pointers must be user-mapped and kernel-mapped. Not POSIX. | src/vfs.rs (ADR-027) |
| virtio-net (N1 ARP + N2 ICMP + N3 UDP + N5 thin TCP + N3.x/N4/N5.x EL0 SVC) | Guest programs a virtio-mmio net DMA master for ARP, ICMP echo, one UDP datagram, then one thin TCP active-open + payload vs QEMU SLIRP/guestfwd. EL0 uses tiny SVCs only (kernel owns NIC), including quiet TCP echo. Not a BSD sockets / TCP product. | src/virtio.rs + src/syscall.rs (ADR-066, ADR-067, ADR-070, ADR-071, ADR-068, ADR-074, ADR-076) |
| virtio-blk + FAT16 | Guest programs a virtio-mmio DMA master. Image is host-built FAT16. Guest may write/create small root files (ADR-050), grow a file across a cluster boundary (ADR-064), list root names (ADR-056), delete a root file (ADR-057), create a root directory (ADR-073), and create one nested subdirectory + empty-rmdir (ADR-077). Same VFS open/write/readdir/unlink/mkdir/rmdir behind prefix mounts (ADR-058). Not a trusted disk. | src/virtio.rs + src/fat.rs + src/vfs.rs (ADR-028, ADR-050, ADR-056, ADR-057, ADR-058, ADR-064, ADR-073, ADR-077) |
| OS/app slots | Host kernel ELF + published app ELF. Guest reads FAT /hello and maps PT_LOAD. A2–A4 use the same file (no embed). Cross-update is a host two-boot probe. Not a trusted disk. | src/slot.rs (ADR-030, ADR-032) |
| Console / sensors | PL011 is how we see whether a probe ran. | Device MMIO 0x0900_0000 |
Adversaries
| Adversary | In scope? | Notes |
|---|---|---|
Buggy kernel code (wrong store, bad unsafe, execute-from-heap, stack smash) | Yes | Primary adversary today. Mitigate with PXN on heap/frames, unmapped linker-stack guards, minimize unsafe (NFR-01), fail-closed smoke. |
| Malicious EL0 (standing or trampoline task executing kernel data or escalating via a bad map) | Named; standing context is bounded | First mile + user-TTBR0 read mile + ASID TLB mile + standing dual-SVC + standing task until exit + TTBR1 private page + EL1 high-VA fetch + a documented SVC ABI exist. Since M6 the umbrella “EL0 isolated” is Verified only as the accepted ADR-091 sentence (one core, TCG default smoke machine; not speculative-execution / side-channel, not real hardware, not a certification). The boot-stub identity page stays by decision (ident: start-stay, ADR-089); stock-QEMU taken SError still parks (el0: serror-park). Product freestanding hosting Verified under ADR-048/052; not Linux/POSIX/containers (ADR-048 / ADR-052). |
| Compromised device tree | Mostly out | M7 does not walk FDT. DTB sits at RAM base below the image. A hostile DTB is a QEMU/host problem until a walker exists; then it becomes an input-validation ADR. |
| DMA / virtio devices | Named; bounded | A7 programs virtio-mmio blk (one queue, poll used.idx, identity-PA DMA). N1–N3/N5 program virtio-mmio net (RX+TX queues, same poll/DMA pattern) for ARP, ICMP echo, one UDP datagram, then thin TCP (ADR-066 / ADR-067 / ADR-070 / ADR-074). QEMU is still TCB. A malicious virtio device is out (hostile hypervisor). No BSD sockets / TCP product. |
| Hostile QEMU or CI host | Out | Hypervisor / runner is trusted. Repo-secret leak is a host control (.gitignore, review), not a guest mitigation. |
| Network attacker | Named; bounded (link only) | N1–N3/N5 attach QEMU user netdev (+ N5 guestfwd echo) + guest virtio-net for ARP + ICMP + one UDP datagram + thin TCP (ADR-066 / ADR-067 / ADR-070 / ADR-074). No BSD sockets / TCP product. QEMU SLIRP/guestfwd is still TCB. Hostile on-wire peers beyond this probe are out of product scope. |
Trust boundaries
[ host / QEMU / CI ] --trusted config-- [ virt guest ]
|
EL1 kernel ← TCB today
|
EL0 first mile + standing ← entered / stood / left
| (not a POSIX user)
EL0 isolation ← Verified only as the ADR-091 sentence (M6)
- EL1 now. Page tables distinguish execute (RO+X text vs RW+NX data/stacks/heap vs Device XN), write (AP[2] on text), and presence (guard holes).
- EL0 first mile + user TTBR0 + ASID TLB + standing + standing-as-normal + TTBR1 first cut + EL1 high-VA fetch + identity-tear first cut + identity
.textrange tear + live.texttear + identity.rodatatear + EL0 entry withoutVMALLE1+ lower-EL IRQ while standing + IRQ-unmasked default ERET + lower-EL FIQ while standing + SError park honesty + PAN ID-field + SVC ABI + libctos CRT + guest PT_LOAD loader; umbrella “EL0 isolated” = the ADR-091 sentence only (M6). Lower-EL AArch64 sync is taken (SVC / IABORT / DABORT). Lower-EL IRQ while standing is taken and returns to EL0 (el0: irq); default standing/taskERETclears I (el0: irq-default); lower-EL FIQ while standing is taken when FIQEn is armed (el0: fiq); SError park honesty (el0: serror-park); ADR-045 A-clear + QMP attempt (inject failed). Short non-standing trampoline probes stay IRQ-masked. Dual ASID withoutTLBI VMALLE1is a probed mile (asid: ok). Standing/EL0 trampoline also switches withoutTLBI VMALLE1after identity.data/heap tear (el0: no-vmalle1). Standing dual-SVC flipsis_active(). A standing task runs a loaded image untilexit(el0: task-ok); unexpected sync restores fail-closed (el0: restore-fail). TTBR1 private page is EL1-only (ttbr1: ok). EL1 can fetch a real path from the high RAM alias (ttbr1: el1 exec);VBAR_EL1is that alias (ttbr1: vbar). A 16 KiB dedicated identity text range is unmapped (ident: range); live identity.textafter_startis unmapped after a vtable rewrite (ident: reloc/ident: live); identity.rodatais unmapped (ident: rodata); identity.data/stacks are unmapped (ident: data); identity heap is unmapped (ident: heap);_startstays (ident: start-stay); leftover identity RAM after the heap is torn (ident: ram).ID_AA64MMFR1_EL1.PANis 0 on-cpu cortex-a57(pan: absent). A documented SVC ABI (svc: ok) is an ABI mile. A linkedlibctoshello (libctos: ok) is a CRT mile. A guestPT_LOAD(loader: ok) is a loader mile. Standing-as-normal (el0: task-ok) is a mile; product freestanding hosting is Verified separately under ADR-048/052. Umbrella “EL0 isolated” is Verified only as the accepted ADR-091 sentence (M6; PAN enabled on default cortex-a76 per ADR-080; taken SError on the ADR-090 evidence class while stock QEMU still parks; never yank_start; leftover RAM torn via ADR-049). A caught load of.dataor ofTTBR1_PRIVis that mile, not the umbrella by itself. - Repo vs guest. “No secrets in repo” is a host boundary. It does not harden the UART.
PAN enable + EL1-vs-EL0 fault (v1.55)
ADR-080 / #125 mile 2: on default -cpu cortex-a76, MSR PAN + EL1 load of an EL0-accessible page faults (pan: enabled / pan: el1-fault; SPSR.PAN at the fault). Syscall uaccess clears PAN around copies. Umbrella “EL0 isolated” stays non-claim (ADR-055). Taken SError still hard-stopped (ADR-053).
Taken SError reopen re-probe (v1.56)
ADR-081 / #125 mile 3: dated agent-box re-probe on default -cpu cortex-a76 plus GICv3 / virtualization=on / secure=on / max — QMP inject-nmi / HMP nmi still machine does not provide NMIs. FEAT_NMI ≠ SError. KVM absent on this smoke host. Unlock = sponsor B1/B2/B3. Park Verified; A-clear dormant prep. Still not taken Verified / “EL0 isolated.”
Taken SError B2-P on real arm64 + KVM (v1.60)
ADR-085 (2026-09-26): a paid one-off run on AWS c7g.metal (Graviton3, CPU: All CPU(s) started at EL2, /dev/kvm=present). The SError source is KVM’s virtual SError (HCR_EL2.VSE), which the host requests through KVM_SET_VCPU_EVENTS{serror_pending=1} (QEMU v10.0.0 + 0001 + 0002, inject-nmi on the el0: serror-arm cue). The guest runs natively. Run 4 console: el0: standing / el0: serror / el0: restored / b2: taken / b2-kvm-smoke: PASS ... accel=kvm. Threat-model relevance: the lower-EL SError vector, standing-EL0 A-clear window and restore path now have evidence on real Neoverse-V1 hardware, not only TCG. Real hardware also found TCR_EL1.TG1=0b00 (reserved), which broke every TTBR1 high-VA walk on Graviton3; it is fixed to 4K. Limits: opt-in profile only (no GIC/timer/virtio/FAT under KVM), no CI gate, no host-side ioctl trace. Umbrella “EL0 isolated” still a non-claim.
Taken SError B2 free-runner spike (v1.59)
ADR-084 (2026-09-25): sponsor chose ADR-081 B2 — a genuinely different runner/accelerator, not the B1 pin. Free GitHub-hosted runners were probed (runs 36193423604 / 36193613972): arm64 Linux kvm [1]: HYP mode not available / /dev/kvm absent; macOS arm64 HVF HV_UNSUPPORTED; Windows arm64 no VT / WHPX disabled; x86 /dev/kvm cannot run an AArch64 guest. Stock TCG on those runners still machine does not provide NMIs. The honest B2 source would be a KVM virtual SError (serror_pending) injected by a small VMM on real arm64 hardware — Planned / blocked on a sponsor choice (self-hosted arm64 KVM runner, paid bare metal, or wait). No new Verified. Threat surface unchanged; still not “EL0 isolated.”
Residual identity surface + umbrella audit (v1.61)
ADR-086 (2026-09-26). Threat-model relevance of the residual identity map: EL0 cannot touch any of it (every leaf is EL1-only, el0=false). The EL1-side exposure is: (1) an identity RO+X padding tail (≈ 1 MiB, no image content) and the stub page stay executable at EL1 through TTBR0, including while EL0 runs (user TTBR0 keeps both), which widens the EL1 gadget/ret2 surface through low VAs; (2) the MMIO block and two small RW RAM ranges stay reachable at EL1 through identity VAs, so a stray EL1 pointer into low VAs still hits memory instead of faulting. Mitigation plan: M1 ratchet + RAM-side tears, M2 MMIO high alias, M3 stub policy (sponsor D2). Taken SError evidence class is a sponsor decision (D1). Umbrella stays non-claim.
Identity inventory ratchet (v1.62)
ADR-087 (2026-09-26). Residual identity after M1: I1 MMIO block (EL1 RW, XN, Device; kernel + ASID-B only) and I3 _start stub page (EL1 RO+X; kernel + user + ASID-B). The ratchet turns “what we tore” into “what is still mapped”, so a regression that re-maps any identity page fails CI. Found while building it: the ADR-020/025 .rodata pointer rewrite silently relocated the allowlist constants themselves (compare physically). M2 removes I1; I3 stays as the one documented exception (sponsor D2).
MMIO high alias (v1.63)
ADR-088 (2026-09-26). Threat-model effect: a stray EL1 pointer into low VAs < 0x4000_0000 now faults instead of touching a device, and the device surface reachable at EL1 shrinks from a 1 GiB identity hole to three 2 MiB TTBR1 blocks. Residual identity: the _start stub page (EL1 RO+X) in kernel / user / ASID-B TTBR0, the documented exception (sponsor D2). B2-P KVM evidence predates this boot-path change.
_start stub identity exception (v1.64)
ADR-089 (2026-09-26). The residual identity page is EL1 RO+X (never EL0-accessible) in kernel / user / ASID-B TTBR0; it holds _start and the identity copy of the vectors. Any additional identity leaf fails CI. A possible later narrowing (drop it from user TTBR0 only) is noted, not decided.
Taken-SError evidence class (v1.65)
ADR-090 (2026-09-26). Threat-model effect: the lower-EL SError vector path (standing EL0, PSTATE.A clear) is exercised on every push/PR by a patched-QEMU TCG job; it is no longer opt-in. What this evidence does not show: RAS/hardware error semantics, SError on stock QEMU, or a CI-gated real-hardware run (B2-P was one paid run on a reduced profile at 159b178).
Umbrella accept draft: scope and non-scope (v1.66)
ADR-091 (2026-09-26, Draft). If accepted, the umbrella would cover only the architectural EL0→kernel page-table / exception-level boundary on the default QEMU TCG machine (one core). It explicitly excludes speculative execution / side channels (no KPTI: TTBR1 stays mapped EL1-only while EL0 runs), real hardware, multi-core, task-vs-task isolation beyond the ASID mile, DMA/IOMMU and EL2. Gaps recorded for the sponsor: no exhaustive EL0-permission walk (G1), no EL0 write probe (G2).
EL0 reachability + EL0 write fault (v1.67)
ADR-092 (2026-09-27). Threat-model effect: the EL0 attack surface is now enumerated rather than assumed. While an app stands, the only EL0-reachable pages are its header, text and stack slots (3 pages per shared window root); at rest there are none; nothing in TTBR1 is EL0-reachable. Any new EL0 leaf outside five allowlisted slots, or backed by kernel frames, fails CI. EL0 stores to kernel data fault. Open (G4): the kernel’s syscall copy path trusts “mapped in user TTBR0” instead of “EL0 may access”, a confused-deputy route to EL1-only pages mapped in user TTBR0 (the _start stub page; transient EL1-only window pages).
Syscall pointers need EL0 permission (v1.68)
ADR-094 (2026-09-27). Threat-model effect: the confused-deputy route recorded as G4 in v1.67 is closed on the default smoke machine. user_range_ok_max now checks the user leaf’s AP bits on every page in the range: AP[1] for a copy from EL0, AP[1] with !AP[2] for a copy to EL0. So no syscall can make the kernel read a page EL0 cannot read, or write a page EL0 cannot write. That covers the _start stub, kernel-only window pages, EL0 execute-only text and EL0-RO pages. CI gates every pointer-taking syscall (17, 19, 20, 21, 22, 24, 27), and a build without the check must fail. App text is now EL0-readable, the normal model for a shared text+rodata page, and still not writable. Residual: a check-then-copy software rule on one core. No AT S1E0R/W or LDTR/STTR copy, and no SMP TOCTOU story.
Taken SError B2-P re-run on current main (v1.69)
ADR-095 (2026-09-28). Threat-model effect: none on the posture; it removes a staleness caveat on the evidence. ADR-090’s B2-P leg was taken at 159b178, before the RAM-side identity tears (ADR-087), the TTBR1 MMIO alias and identity MMIO tear (ADR-088), ADR-092 and ADR-094. On main 8107578 the b2-serror profile booted natively on Graviton3 under KVM through ident: mmio-high, ident: low/tail/kend, ident: mmio, PAN enable and EL1 fault (Graviton3 reports pan: id=2), and took the KVM virtual SError (KVM_SET_VCPU_EVENTS{serror_pending=1} → HCR_EL2.VSE) at the lower-EL SError vector while standing at EL0. Residual: reduced profile (no GIC / timer / virtio / FAT, no inventory / reach / write / sys-ptr probes under KVM); one-off paid run, not CI; no host ioctl trace; not a real-hardware claim for the default image.
Umbrella “EL0 isolated” accepted (v1.70)
ADR-091 (2026-09-28, Accepted, M6). The sponsor accepted the revision-3 sentence and scope in writing (B3, 2026-09-28 16:41 CEST; sponsor wrote it in the DSO agent chat, DSO relayed it to ctos). Threat-model effect: the umbrella row moves from Planned / non-claim to Verified, and only as the sentence below. Revision 4 (2026-10-02, v1.74): the sponsor accepted ADR-091 rev 4 in writing (“I accept ADR-091 rev 4”); the sentence below is revision 4. Revision 3 differed only in its PAN clause (“PAN is enabled and EL1 faults on EL0 memory”) and in the limit “EL0 can read its own code bytes”:
“On the ctos default smoke machine (QEMU
virt,-cpu cortex-a76, TCG, one core), EL0 is isolated from the kernel at the architectural page-table / exception-level boundary: each EL0 task runs on its own TTBR0 with its own ASID; EL0 faults when it reads, writes or executes kernel memory; in every translation table (kernel, user and ASID-B TTBR0, and TTBR1) the only EL0-reachable pages are five allowlisted user slots backed by non-kernel frames, checked fail-closed while a user app runs and at rest; PAN is enabled, including while the kernel handles exceptions from EL0, and EL1 faults on EL0-readable memory (the execute-only app text page is not covered: this CPU has no FEAT_EPAN); lower-EL IRQ and FIQ are taken while EL0 stands; taken lower-EL SError is evidenced by the ADR-090 evidence class; the only identity (VA = PA) mapping left in any TTBR0 is the documented EL1-only_startstub page (ADR-089); and a system call copies from or to an EL0 pointer only if EL0 itself may read (for a copy from EL0) or write (for a copy to EL0) every page in the range. It is not a speculative-execution or side-channel claim, not a real-hardware claim, and not a certification.”
Limits that stay with it: one core; the TCG default smoke machine; not speculative-execution / side-channel; not real hardware; not a certification; app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN) (ADR-097); no multi-core race story. The ADR-091 non-scope list stands (no task-vs-task isolation beyond ASIDs, no DMA / IOMMU / EL2, stock-QEMU taken SError still parks). No code change: the M6 PR CI run on main plus this change is the one-session re-probe. Older version notes and sections in this file that say “not ‘EL0 isolated’” record the posture at their version.
Task-to-task EL0 isolation probe (v1.71)
ADR-096 (2026-09-28). Threat-model effect: the “malicious EL0 task reads or corrupts another EL0 task” case now has a probe on the page-table/ASID mechanism, not only the EL1-side asid: ok mile. Task B (ASID-B root, ASID 2) stores a sentinel at 0x80007000; task A (user root, ASID 1) then loads and stores the same VA and takes a lower-EL translation fault (FAR = the VA) without seeing B’s value; B’s frame is unchanged; a walk of A’s root finds no leaf for B’s frame. With A’s own frame at the VA, each task reads and writes only its own frame. Markers t2t: … / t2t: ok, fail-closed in scripts/qemu-smoke.sh, plus a t2t-leak-probe build that must be caught. Not part of the ADR-091 sentence (task-vs-task stays in its non-scope; a revision 4 needs the sponsor in writing). Limits: one core; TCG flushes its own TLB on an ASID change, so no hardware stale-TLB claim; kernel-built tasks, not two loaded apps under a scheduler; not side-channel.
PAN held during EL0 exception handling (v1.72, correction)
ADR-098 (2026-09-29). Threat-model effect: PAN is the net for kernel bugs that dereference a user pointer outside the copy helpers. That net was down at the start of every EL0 exception handler and after EL0 trips (until the next copy helper restored PAN), i.e. exactly while the kernel serves EL0. The boot probe (pan: el1-fault) could not see it. Now SCTLR_EL1.SPAN=0 and return_from_el0 sets SPSR.PAN; the smoke checks PAN live inside the loaded app’s SYS_YIELD handler (MRS PAN, calibrated) and after the app exits (an EL1 load of an EL0-readable page faults with SPSR.PAN set). A pan-keep-leak-probe build that undoes the fix is caught. Limits: one core, TCG; one exception type and one post-trip point sampled.
EL0 execute-only app text (v1.73; accepted 2026-10-02)
ADR-097 (2026-09-28; accepted 2026-10-02 as ADR-091 revision 4, “I accept ADR-091 rev 4”; merged as 8344797; at v1.73 it was proposed and merge required that accept). Threat-model effect: a compromised or buggy EL0 app can no longer read its own code bytes (e.g. to harvest gadgets or leak code through a syscall): EL0 loads of 0x80002000 take a lower-EL permission fault (xo: el0-read-text fault … dfsc=perm-l3 got=0x0) and uart_write of a text pointer is refused (xo: sys-read text denied … checks=1). Rodata lives on the app-hdr page (EL0 RO + NX). Still five EL0-reachable slots; the fail-closed walk counts UXN=0 pages as reachable (reach=uxn) and catches an unallowlisted XO plant. What gets weaker in the wording, not in the mechanism: without FEAT_EPAN, PAN does not stop EL1 from reading the XO page (xo: el1-pan text … fault=no … epan=no), so “EL1 faults on EL0 memory” no longer covered every EL0 page; revision 4 words this exactly (“EL1 faults on EL0-readable memory (the execute-only app text page is not covered: this CPU has no FEAT_EPAN)”). PAN fix split out: PAN being clear during EL0 exception handling and after EL0 trips is fixed by ADR-098 (v1.72, merged into this branch); the xo live line echoes its reading (xo: pan live-syscall pstate-pan=1 sctlr-span=0). Leak builds xo-leak-probe / pan-keep-leak-probe are caught. Limits: one core, TCG, not real hardware; live check on hello-libctos only (other apps by build/loader checks).
KVM re-check via GitHub OIDC (ADR-099)
Folded from ADR-099 §7 (risk and impact evaluation, as on main after PR #157) in v1.76. ADR-099 stays the source of truth for the trust and permission policies, preconditions P1–P15 and the workflow; if the two disagree, ADR-099 wins and this section must be re-synced.
Asset and boundary. A GitHub Actions job (.github/workflows/kvm-recheck.yml, manual dispatch only, from main, behind the kvm-pilot environment) assumes one AWS role through GitHub OIDC (no stored keys) and starts one Graviton metal Spot instance that runs the reduced b2-serror profile under KVM, then terminates it. New trust boundary: GitHub’s OIDC token → the AWS role. New assets: the role, the cloud account’s spend, and the public CI log.
Status (2026-10-13 pilot, relayed by DSO; see the honesty ledger). One supervised run: smoke PASS under accel=kvm; the run verdict was red only from a false LEAK (teardown wait too short for metal), fixed by PR #157. Not a recurring check, not a required check.
Impact is “what an attacker or a mistake could cost”. Residual is after the listed mitigations, assuming ADR-099 P1–P15 are met.
| # | Threat | Likelihood × impact (before) | Mitigations | Residual risk |
|---|---|---|---|---|
| 1 | Another repo, branch, fork or pull request assumes the role | Low × High | Exact aud and sub (no wildcard); environment with required reviewers and main-only branch policy; fork PRs get a different sub and no id-token for secrets | Low. Depends on the exact sub being right (P14) and the environment existing first (P5). |
| 2 | A PR edits the workflow to abuse the role | Medium × High | Dispatch runs only the default-branch file; guard refuses any ref but main; environment gate; enforce_admins is on | Low–Medium. main requires no approving review (observed), so the control is the sponsor merging by convention. DSO may ask the sponsor to require one review on main or a code-owner rule for .github/workflows/. |
| 3 | Role is over-permissioned (miner, wrong instance type, other region) | Medium × High | Conditions on instance type, Spot market, IMDSv2, tags and region; pinned image/subnets/SG; 1 h session; no IAM, no PassRole; Spot quota; budget alarm | Low. IAM conditions are untested until the pilot (P13). |
| 4 | Tag spoofing (a resource tagged project=ctos to dodge the sweeper, or the terminate scope abused) | Low × Low | Tags only at creation (CreateAction); terminate only matches tags the role itself applied; the sweeper counts the tag broadly | Low. A person with other credentials could still tag things; out of scope for this role. |
| 5 | Privilege escalation through the instance | Low × High | No instance profile, no iam:PassRole, explicit Deny on iam:*; IMDSv2 required, hop limit 1; no credentials on the box | Low. |
| 6 | Secret in user-data | Low × Medium | User-data holds only a commit, a nonce and a timer; no keys, tokens or URLs with secrets | Very low. |
| 7 | Identifier leakage in public logs (instance ID, account ID, host name, IP) | Medium × Low–Medium | Account-ID masking; instance IDs masked at creation; only ^CTOSB2: lines and error codes are printed; no raw console or stderr; identifier guard on repo text | Low. Confirm on the pilot (P15). |
| 8 | Leaked running instance (cost, exposure) | Medium × Low | Pre-flight refusal; in-instance shutdown -h +45 and terminate-on-shutdown; always-run terminate with fresh credentials, by tag; wait for terminated; sweeper gives LEAK; budget alarm | Low. Worst case about $0.40/h until noticed if the runner and the user-data both fail. |
| 9 | Console text is forged or replayed | Low × Medium | Per-run nonce and commit in the result line; anchored PASS line; KVM and arch markers; terminate-by-tag so one run’s console is read once | Low. The code under test can print its own PASS (inherent; the commit is recorded and the tested commit must be on main). |
| 10 | Silent green on a skip or timeout | Medium × High (trust) | Only PASS exits 0; SKIPPED, INCONCLUSIVE, REFUSED, LEAK, TEARDOWN-SLOW, ABORTED-after-aws-call, NOT-RUN are red; guard fails (not skips) on an unset variable | Low. |
| 11 | Spot capacity or interruption | High × Low | Multi-AZ retry; honest SKIPPED / INCONCLUSIVE; no on-demand fallback | Accepted: the check is sometimes unavailable. |
| 12 | Third-party action supply chain (configure-aws-credentials, checkout) | Low × High | Pin by full commit SHA before enabling (P12); permissions: {} at top level; the job only gets id-token: write and contents: read | Low–Medium until pinned. |
| 13 | Supply chain on the instance (apt, the rustup install script, git clone, QEMU source) | Medium × Low | QEMU is built from the pinned v10.0.0 source plus the repo’s patches; the repo is cloned at a commit already on main; the instance holds no credentials, has no ingress, IMDSv2 hop limit 1, and lives under an hour | Low. Open egress remains (needed for apt/rustup/git). |
| 14 | Repo rename or transfer (subject reuse by a new owner) | Low × High | The immutable subject (owner and repo IDs) is what the trust policy pins | Low once P14 is confirmed. |
| 15 | Cost runaway | Low × Low | Cost-cap guard; MaxPrice; lifetime; concurrency 1; budget alarm | Low; about $0.33 worst case per run [estimate]. |
| 16 | CloudTrail off, so no audit trail | Unknown × High | Condition (a) is a hard precondition before the role is created or used | P1 met 2026-10-13 (multi-region trail, DSO-verified). |
| 17 | Unprotected environment auto-created by the workflow’s first reference | Medium × High | P5: create the environment with protections first; order matters | Low if the order is followed. |
| 18 | Unknown account facts (Spot service-linked role, quota, EBS default key) | Medium × Low | P8; pilot; any new permission goes to the sponsor, not added silently | Low. |
| 19 | Credential rule 4 is only relaxed for this role | Low × Medium | One role, one repository, one environment, no stored key, 1 h tokens; the exception record is referenced in the enabling PR (P10) | Low. Any second role or a schedule needs its own approval. |
Not claimed here: the default image, real-hardware boot of the default image, or anything beyond the reduced b2-serror profile on one KVM host per run.
Non-goals
-
Not a certified secure OS (no Common Criteria, no PSA, no “hardened”).
-
Not side-channel complete (no cache/timing/Spectre story; QEMU TCG is the wrong lab).
-
Not a product “the kernel is W^X” sentence. The identity image on this virt guest is RO+X / RW+NX with WXN (ADR-015). Future mappings are not automatically covered. Guard pages remain holes.
-
Not ASAN / canaries / heap-stack guards. Coop worker stacks have no unmapped holes. Overflow there is still image-adjacent PXN RAM.
-
Not secure boot, PAN enable on historical cortex-a57 (non-goal on that CPU, ADR-047; enable Verified on cortex-a76 / ADR-080), or a fully torn-down identity map (ADR-020/025/037/038/049 unmap live
.text/.rodata/.data+stacks / heap / leftover frame RAM;_startstays at0x4008_0000by decision). -
Not multi-core. Every isolation claim is one core (ADR-091 rev 4). The ADR-100
smp2-probesecondary bring-up is a smoke, not SMP support.
Mitigations mapped to probes
| Mitigation | Probe | Ledger |
|---|---|---|
| Secondary CPU powered on, parked, never runs kernel state or EL0 (ADR-100 S1, opt-in) | smp2-probe under -smp 2: smp: secondary up mpidr=0x0000000080000001 + smp: ok cpu1 … parked; -smp 1 negative must print smp: cpu1 missed; default run must show no smp: line | Verified locally 2026-10-13 + this PR’s non-required CI job. Not SMP support. |
| Threat-model v1.23 written | Read this file | Verified (file + review). Still no “secure OS”. |
| Device MMIO XN (L1 block 0) | pxn_for(0x0900_0000) == Some(true) | Covered by the W^X tests when they run. |
| Heap + coop stacks PXN | Serial wx: ok; #[test_case] flags + execute-from-heap IABORT | Verified: 2026-09-10 cloud qemu-smoke (honesty ledger). |
| Kernel text still executable | is_executable(0x4008_0000) | Same W^X probe. |
| Execute-from-writable heap forbidden | Armed permission IABORT → wx: nx heap then wx: ok | Fail-closed in scripts/qemu-smoke.sh. |
| Linker-stack guard holes | Serial guard: fault / guard: ok; store to __stack_guard | Verified: 2026-09-10 cloud qemu-smoke (honesty ledger). |
Minimize unsafe | Review (NFR-01). Count is not a proof. | Policy. |
| Fail-closed smoke | scripts/qemu-smoke.sh greps + force-fail exit ≠ 0 | NFR-04 / NFR-05. |
| No secrets in repo | Policy + .gitignore (including .obsidian/) | Host control. |
| IRQ least privilege | IRQ path does not allocate or yield_now | Convention. Ratchet if it fails twice. |
| EL0 entered and returned | Serial el0: svc / el0: ok; #[test_case] | Verified first mile — not isolation. |
| EL0 cannot execute kernel data | Serial el0: nx kernel; lower-EL IABORT | Verified first mile. |
EL0 cannot read kernel .data | Serial el0: no kernel read; user TTBR0 omits .data | Verified read mile. One mile; alone it is not the ADR-091 umbrella claim. |
ASID isolation (no VMALLE1) | Serial asid: dual / asid: conflict / asid: ok | Specific mile. One mile; alone it is not the ADR-091 umbrella claim. |
| Task-to-task EL0 isolation (ADR-096) | Serial t2t: walk b-frame … / t2t: a-read fault … / t2t: a-write fault … / t2t: same-va … / t2t: ok …; t2t-leak-probe build caught; #[test_case] el0_task_cannot_read_or_write_other_task | Verified (one core, TCG, kernel-built tasks). Not part of the ADR-091 sentence. |
| EL0 execute-only app text (ADR-097, accepted 2026-10-02) | Serial xo: live … / xo: layout loader … / xo: el0-exec ok … / xo: el0-read-text fault … / xo: sys-read text denied … / xo: el1-pan … / xo: pan live-syscall pstate-pan=1 sctlr-span=0 / xo: ok …; xo-leak-probe + pan-keep-leak-probe build caught; #[test_case] el0_app_text_is_execute_only | Verified (one core, TCG); merged as 8344797 after the sponsor accepted ADR-091 rev 4 (2026-10-02). |
EL0 entry without VMALLE1 | Serial el0: no-vmalle1; src/exception.rs has no tlbi vmalle1 | Trampoline cut (ADR-039). Not isolation. |
| Lower-EL IRQ while standing | Serial el0: irq | Taken IRQ from EL0 returns to standing (ADR-040). Not isolation. |
| IRQ-unmasked default ERET | Serial el0: irq-default | Default standing/task ERET clears I (ADR-041). Short probes stay masked. Not isolation. |
| Lower-EL FIQ while standing | Serial el0: fiq | Taken FIQ from EL0 returns to standing (ADR-043). Not isolation. |
| SError park honesty | Serial el0: serror-park | No working SError inject on virt TCG (ADR-043 / ADR-045 / ADR-081). Taken path deferred/non-goal (ADR-047). |
| Taken SError QMP attempt | Serial el0: serror-arm + QMP log | Guest A-clear path (ADR-045); inject-nmi fails on QEMU 10 virt+a76 (ADR-081). |
Identity _start stay | Serial ident: start-stay | Honesty (ADR-042 / ADR-047). Not isolation. |
| Leftover identity RAM tear | Serial ident: ram / ident: ram-fault / ident: ram-high | Optional mile (ADR-049). High twin stays. Not isolation. |
| Standing EL0 context | Serial el0: standing / el0: restored; is_active() flips | Specific mile. Not POSIX. Not isolation. |
| Standing EL0 as normal mode | Serial el0: task-enter / el0: task-active / el0: task-exit / el0: task-restored / el0: restore-fail / el0: task-ok | Loaded image until exit. Fail-closed restore. Not isolation. |
| TTBR1 private page | Serial ttbr1: el1 / ttbr1: no el0 / ttbr1: ok | First cut (ADR-016). |
| EL1 fetch from TTBR1 high VA | Serial ttbr1: el1 exec / ttbr1: vbar | Exec mile (ADR-017). |
| Identity-tear first cut | Serial ident: split / ident: fault / ident: high / ident: no el0 / ident: ok | First cut (ADR-018). Yank _start decided never; leftover RAM torn (ADR-049). |
| Identity text range tear | Serial ident: jump / ident: range / ident: text | Range cut (ADR-019). |
High-VA vtable rewrite + live .text tear | Serial ident: reloc / ident: live | Live .text cut (ADR-020). |
Identity .rodata tear | Serial ident: rodata / ident: rodata-fault / ident: rodata-high | .rodata cut (ADR-025). |
Identity .data / stack tear | Serial ident: data / ident: data-fault / ident: data-high | .data/stacks cut (ADR-037). |
| Identity heap tear | Serial ident: heap-reloc / ident: heap / ident: heap-fault / ident: heap-high | Heap cut (ADR-038). |
| PAN ID field | Serial pan: id= / pan: present | Capability probe (ADR-026 / ADR-079). |
| PAN enable + EL1-vs-EL0 fault | Serial pan: enabled / pan: el1-fault | ADR-080. One mile; alone it is not the ADR-091 umbrella claim. |
| PAN held during EL0 exceptions + after EL0 trips (ADR-098) | Serial pan: live-syscall pstate-pan=1 sctlr-span=0 / pan: after-el0 trips=… mrs-pan=1 el1-load … fault=yes spsr-pan=1 / pan: ok held boot,live-syscall,after-el0; pan-keep-leak-probe build caught; #[test_case] pan_span_cleared_after_enable | Verified (one core, TCG). Correction of the ADR-091 PAN clause’s evidence; sentence unchanged. |
SVC ABI (exit / uart_write / yield) | Serial svc: ok; user buffer + kernel-.data / TTBR1-alias reject | ABI mile (ADR-021). Not app hosting. |
libctos CRT | Serial libctos: hi / libctos: ok / libctos: linked | CRT mile (ADR-022). Not app hosting. |
| Guest ELF PT_LOAD loader | Serial loader: mapped / loader: ok | Loader mile (ADR-023). Not a Linux ABI. Not app hosting. |
| Thin VFS + memfs | Serial fs: create / fs: write / fs: read / fs: el0 / fs: ok | memfs mile (ADR-027). Not POSIX. Not app hosting. |
| virtio-net ARP (N1) + ICMP ping (N2) + UDP (N3) + thin TCP (N5) | Serial net: virtio / net: mac / net: tx / net: rx / net: ok + net: icmp-tx / net: icmp-rx / net: ping-ok + net: udp-tx / net: udp-rx / net: udp-ok + net: tcp-syn / net: tcp-est / net: tcp-tx / net: tcp-rx / net: tcp-ok | ADR-066 / ADR-067 / ADR-070 / ADR-074. ARP + ICMP + one UDP + thin TCP vs QEMU SLIRP/guestfwd only. Not BSD sockets / TCP product. Host -netdev alone is not the probe. |
| EL0 net SVC sample (N4) | Serial libctos: net-hi / libctos: net-mac / libctos: net-ok / netdemo: ok | ADR-068. SVCs only; kernel owns NIC. Not sockets. |
| EL0 UDP DNS SVC sample (N3.x) | Serial libctos: udp-hi / libctos: udp-ok / udpdemo: ok | ADR-071. net_udp_dns only; DNS is probe bait. Not sockets. |
| virtio-blk + FAT16 | Serial blk: ok / fat: ok / fat: write / fat: create / fat: grow / fat: mkdir / fat: nested / fat: rmdir / fat: readdir / fat: entries / fat: delete; VFS /probe | block + FAT mile (ADR-028) + write depth (ADR-050) + multi-cluster grow (ADR-064) + mkdir (ADR-073) + nested+rmdir (ADR-077) + readdir (ADR-056) + delete (ADR-057). Host -drive alone is not the probe. Not POSIX getdents/unlink/mkdir/rmdir/write API. |
| OS/app slot first cut | Serial slot: fat / slot: mapped / slot: ok | Slot mile (ADR-030). A2–A4 load FAT (no embed). |
| A9 cross-update | Host smoke: same app sha256 on this OS and ba6541c | Leftover (ADR-032). Not “apps update independently.” |
| RO+NX text/data | Serial ro: ok; execute-from-.data + write-to-RO-text | Verified: 2026-09-11 cloud qemu-smoke (honesty ledger). |
Claim gate
A PR may say “threat-model v1.22 exists” after a file read. It may say “heap NX” / “identity image is W^X on this virt guest” only when the honesty ledger has a matching Verified QEMU probe. It may say “linker-stack guard faults” only with a matching translation-abort probe. It may say “EL0 entered and returned,” “EL0 cannot execute kernel data,” “EL0 cannot read kernel .data,” “standing EL0 context,” “ASID isolation,” “TTBR1 private page,” “EL1 fetched from a TTBR1 high VA,” “one identity text page was unmapped,” “a 16 KiB dedicated identity text range was unmapped,” “rustc vtables were rewritten to high aliases,” “live identity .text after the boot stub was unmapped,” “identity .rodata was unmapped,” “identity .data/stacks were unmapped,” “ID_AA64MMFR1_EL1.PAN is 0 on -cpu cortex-a57,” “EL0 issued the documented SVC ABI,” “a program linked against libctos issued that ABI,” “the guest mapped PT_LOAD and ERETed to e_entry,” or “a loaded image stood at EL0 until exit with fail-closed restore,” or “the guest created, wrote, and read a named in-RAM file,” or “the guest programmed virtio-blk and read a known FAT16 file through the thin VFS,” or “the guest wrote FAT16 bytes through the thin VFS,” or “the guest listed FAT16 root directory entries through the thin VFS,” or “the guest loaded a separate app ELF from FAT /hello into user TTBR0,” or “the same published app ELF loaded on this OS and on ba6541c,” or “identity heap was unmapped,” or “standing/EL0 trampoline switches without TLBI VMALLE1,” or “taken lower-EL IRQ while standing returned to EL0,” or “default standing/task ERET enters with IRQ unmasked,” or “_start stays mapped while live image ranges are torn” only for the mile that actually passed.
It may not say “secure OS,” “hardened,” “EL0 works,” “the kernel moved,” “immutable OS,” Linux app host, POSIX host, or container host. It may say the freestanding product app-hosting claim is Verified under ADR-048 when the ledger product row is Verified (ADR-052). It may say “EL0 isolated” only as the accepted ADR-091 sentence, quoted verbatim or pointed to, with its limits (M6). Scoped RO is immutability.md. Unprobed stays Unknown. https://ctos.artof.link HTTPS is Verified (2026-09-11 after #30; see the honesty ledger).
Taken SError research + QMP attempt (v1.25)
Research (ADR-044); implementation attempt (ADR-045): guest A-clear standing path + host QMP inject-nmi. On QEMU 10 virt+-cpu cortex-a57, QMP returns machine does not provide NMIs (no ARM_CPU_SERROR via nmi; FEAT_NMI needs a different CPU/GIC). Park marker remains Verified.
Isolation leftovers decisions + app-hosting claim gate (v1.26)
ADR-047: taken SError = deferred/non-goal on this smoke machine (A-clear dormant prep); PAN enable = non-goal on default cortex-a57; never yank _start; leftover identity RAM torn via ADR-049 (ident: ram*); umbrella “EL0 isolated” stays Planned/non-claim. ADR-048 (at v1.26): A1–A9 first cuts ≠ product “app hosting done”; claim stayed Planned until criteria Met + sponsor accept.
Product app-hosting claim Verified (v1.29)
ADR-048 criteria 1–5 Met; sponsor accept ADR-052. Product freestanding “apps run independently / hosting done” is Verified. Still not Linux userspace, POSIX, glibc/musl, guest OCI/Docker/k8s, “EL0 isolated,” PAN enable, taken SError Verified, immutable-OS marketing, or OTA
Isolation-gap locks (v1.30)
ADR-053: taken lower-EL SError hard-stopped on QEMU 10 virt+cortex-a57 (dated 2026-09-14 re-probe: inject-nmi / HMP nmi → machine does not provide NMIs); park Verified; A-clear dormant prep. ADR-054: PAN enable locked non-goal on default cortex-a57; explicit reopen gate (sponsor CPU + FEAT_PAN + enable+fault ADR). ADR-055: umbrella “EL0 isolated” non-claim until checklist — Verified miles listed; unmet blockers include PAN enable, taken SError, never-yank _start. Still not Guest Linux / containers / immutable-OS marketing..
FAT16 readdir (v1.31)
ADR-056: guest lists FAT16 root dirents as thin-VFS paths (fat: readdir / fat: entries). Not POSIX getdents / opendir. Keep write / /hello / slot: ok. Still not Linux/containers/“EL0 isolated.”
FAT16 delete (v1.32)
ADR-057: guest deletes a FAT16 root file via thin-VFS unlink (fat: delete). Not POSIX unlink / remove. Keep write / readdir / /hello / slot: ok. Still not Linux/containers/“EL0 isolated.”
VFS prefix mounts (v1.33)
ADR-058: guest routes path prefixes through a thin VFS mount table (vfs: mount / vfs: mounts). Not POSIX mount(2) / Linux vfsmount. Keep memfs / FAT write / readdir / delete / /hello / slot: ok. Still not Linux/containers/“EL0 isolated.”
Freestanding sample catalog (v1.34)
Two freestanding EL0 samples on FAT: /hello (hello-libctos, UART+yield) and /fsdemo (fs-libctos, libctos VFS create/open/read/write/close on /memdemo). Catalog docs list both. Not POSIX. Not getdents. Not a reopen of product app hosting.
Sponsor isolation leftovers closure (v1.35)
ADR-060: single sponsor-facing table for taken SError / PAN enable / umbrella “EL0 isolated” / never-yank _start — locks + reopen gates only. Cites ADR-047 / ADR-053 / ADR-054 / ADR-055 / #98. Does not invent Verified. Does not reopen Guest Linux / containers / immutable-OS marketing.
Freestanding FAT-via-VFS sample (v1.36)
Three freestanding EL0 samples on FAT: /hello, /fsdemo (memfs /memdemo), and /fatdemo (fat-libctos, open/read FAT /probe via thin VFS / → FAT16). Catalog docs list all three (ADR-061). Not POSIX. Not getdents. Not a reopen of product app hosting.
FAT16 multi-cluster grow (v1.39)
ADR-064: guest grows a FAT16 root file across a cluster boundary (fat: grow); EL0 fat-libctos chunked SVC write/read-back (libctos: fat-grow). Not POSIX write API. Not exFAT. Keep write / readdir / delete / /hello / slot: ok. Still not Linux/containers/“EL0 isolated.” Track N untouched.
FAT vs memfs read CNTPCT (v1.40)
ADR-065: same-boot CNTPCT pair around FAT /probe read vs memfs /mrprobe read (perf: fat-read / perf: memfs-read / perf: fs-read-delta). Measure-first lab ticks only — not a latency SLA, percent, or product KPI. Keep ADR-051 write pair / fat: ok / Track N untouched. Still not “secure.”
Freestanding cooperative-yield sample (v1.37)
ADR-062: fourth freestanding EL0 sample yield-libctos on FAT /yldemo. Several cooperative yield_now() rounds (libctos: yld-hi / libctos: beat / libctos: yld-ok). Not preemption. Not multi-task EL0. Not a process table. Does not reopen ADR-048/052.
Track N network foundation scope (v1.38)
ADR-063: new Track N (network), separate from A/B. N0 = docs scope. Non-goals locked: TCP/UDP/sockets/DHCP/DNS product, Wi‑Fi, virtio-pci-only foundation, Linux net stack, “has networking” marketing, EL0 net ABI before deeper net miles.
Track N virtio-net first frame (v1.41)
ADR-066: N1 discovers virtio-net, programs RX+TX, TX ARP for 10.0.2.2, RX SLIRP reply (net: virtio / net: mac / net: tx / net: rx / net: ok). Host -netdev user,id=net0 -device virtio-net-device,netdev=net0. I/O surface: virtio-blk and virtio-net are programmed DMA masters. Still not TCP/UDP/sockets/DHCP/DNS/Wi‑Fi/“has networking.”
Track N virtio-net ICMP ping (v1.42)
ADR-067: after N1 ARP, guest TX ICMP echo request to 10.0.2.2 and RX echo reply (net: icmp-tx / net: icmp-rx / net: ping-ok). QEMU args unchanged. Kernel-path only. Still not TCP/UDP/sockets/DHCP/DNS/Wi‑Fi/“has networking.”
Track N EL0 net SVC sample (v1.43)
ADR-068: SYS_NET_MAC (24) + SYS_NET_PING (25); freestanding net-libctos on FAT /netdemo (libctos: net-hi / libctos: net-mac / libctos: net-ok / netdemo: ok). Kernel still owns virtio-net. Still not TCP product / sockets / DHCP/DNS product / Wi‑Fi / “has networking.”
Net-ping CNTPCT (v1.44)
ADR-069: CNTPCT around EL0 net_ping quiet ARP+ICMP (perf: net-ping ticks=<n>). Measure-first lab ticks only — not a latency SLA, percent, or product KPI. Smoke greps the marker prefix (tick values vary). Keep N1/N2/N4 / FAT / slot. Still not TCP/UDP/sockets/“has networking.” Still not “secure.”
Track N UDP transport (v1.45)
ADR-070: after N2 ICMP, guest TX one UDP DNS query to SLIRP 10.0.2.3:53 and RX a matching DNS response (net: udp-tx / net: udp-rx / net: udp-ok). QEMU args unchanged. Kernel-path only this mile. DNS is probe bait, not a guest DNS product. Still not BSD sockets / TCP product / DHCP/DNS product / Wi‑Fi / “has networking / sockets OS.” Still not “secure.”
Track N EL0 UDP DNS SVC (v1.46)
ADR-071: SYS_NET_UDP_DNS (26); freestanding udp-libctos on FAT /udpdemo (libctos: udp-hi / libctos: udp-ok / udpdemo: ok). Quiet kernel-path ARP + UDP DNS to SLIRP 10.0.2.3:53 (same bait as ADR-070). Kernel still owns virtio-net. Still not BSD sockets / TCP product / DHCP/DNS product / Wi‑Fi / “has networking.” Still not “secure.”
UDP DNS path CNTPCT (v1.47)
ADR-072: CNTPCT around EL0 net_udp_dns quiet ARP+UDP DNS (perf: udp-dns ticks=<n>). Measure-first lab ticks only — not a latency SLA, percent, or product KPI. Smoke greps the marker prefix (tick values vary). Keep N1/N2/N3/N3.x/N4 / ADR-069 / FAT / slot. Still not TCP/sockets/DNS product/“has networking.” Still not “secure.”
FAT16 mkdir (v1.48)
ADR-073: guest creates a FAT16 root directory through the thin VFS (fat: mkdir); dirent + empty cluster with . / ... Not POSIX mkdir. Not a directory tree. Not exFAT. No new SVC this mile (EL0 follow-up). Keep write / readdir / delete / grow / /hello / slot: ok / Track N markers. Still not Linux/containers/“EL0 isolated.” Track N / TCP untouched.
Thin TCP N5 (v1.49)
ADR-074: after N3 UDP, guest active-opens TCP to QEMU guestfwd 10.0.2.4:7, sends one payload ctos-tcp\n, requires matching echo (net: tcp-syn / net: tcp-est / net: tcp-tx / net: tcp-rx / net: tcp-ok). QEMU -netdev gains guestfwd=tcp:10.0.2.4:7-cmd:…/tcp-echo-stdio.sh. Kernel-path N5. One connection; no listen/accept. ADR-076 adds EL0 net_tcp_echo (28) + FAT /tcpdemo quiet re-run of the same probe. Still not BSD sockets / TCP product / TLS/HTTP / DHCP/DNS product / Wi‑Fi / “has networking / sockets OS.” Still not “secure.”
FAT16 nested mkdir + empty rmdir (v1.52)
ADR-077: guest creates one nested FAT16 subdirectory (/fdir/nest) and removes an empty directory through the thin VFS (fat: nested / fat: rmdir). Path grammar: flat /name or one /parent/child; deeper → BadPath. Not POSIX mkdir/rmdir. Not arbitrary-depth trees. Not recursive rm. No new SVC this mile. Keep fat: mkdir / write / readdir / delete / grow / /hello / slot: ok / Track N markers. Still not Linux/containers/“EL0 isolated.” TCP CNTPCT untouched.
Thin TCP echo CNTPCT (v1.53)
ADR-078: CNTPCT around the quiet EL0 net_tcp_echo path (perf: tcp-echo ticks=<n>). Measure-first lab probe on QEMU TCG — not a latency SLA, not SPEC, not “has networking.” Chosen surface is EL0 SVC (same as ADR-069/072); kernel N5 net: tcp-ok stays transport proof only. Keep N1–N5 / N5.x / ADR-069/072 / FAT nested+rmdir / samples. Still not BSD sockets / listen-accept / TLS/HTTP. Still not “secure.”
PAN CPU reopen foundation (v1.54)
ADR-079 / #125: sponsor unlock + default smoke CPU -cpu cortex-a76 + Verified ID-field pan: present (ADR-054 gate items 1–2). Historical cortex-a57 pan: absent retained. Enable+fault is ADR-080 (v1.55). Does not unlock taken SError — confirmed blocked in ADR-081. Umbrella “EL0 isolated” stays non-claim. Never yank _start. Still not “secure.”
EL0 isolation (P-SEC-3 / ADR-013)
Isolation: “EL0 isolated” is Verified only as the accepted ADR-091 sentence (M6, 2026-09-28; quoted with its limits below). History (2026-09-26 audit + gap list: ADR-086; M1 identity inventory ratchet: ADR-087 ; M2 MMIO high alias: ADR-088 — only the _start stub page remains identity-mapped, the one documented exception per ADR-089; M4 taken-SError evidence class B1 pin + B2-P per ADR-090; M5 umbrella sentence drafted in ADR-091, accepted in writing by the sponsor 2026-09-28 (B3, M6); G1 EL0-reachability walk + G2 EL0 write fault closed: ADR-092; G4 syscall pointers need EL0 permission on every page, closed: ADR-094; ADR-091 revision 3 drops the direct-access-only carve-out, G3 optional only) (ADR-047 / ADR-055; sponsor closure ADR-060). A first mile, a user-TTBR0 read mile, an ASID TLB mile, a standing EL0 context, a TTBR1 private-page first cut, an EL1 high-VA fetch mile, an identity-tear first cut, an identity .text range tear, a live identity .text tear after a high-VA vtable rewrite, an identity .rodata tear, a PAN ID-field probe (pan: present on cortex-a76 / ADR-079) and PAN enable + EL1-vs-EL0 fault (pan: enabled / pan: el1-fault / ADR-080), a documented SVC ABI mile, a libctos CRT mile, a guest ELF PT_LOAD loader mile, a standing-task (normal mode) mile, a thin VFS + memfs mile, a virtio-blk + FAT16 mile, and an OS/app slot first cut exist. Those miles alone are not the product claim. A8 is documented rebuild recipes (what-can-run.md). A9 is two artifacts + FAT /hello (ADR-030); leftover cross-update / embed-off landed (ADR-032). Product “app hosting is done” is Verified under ADR-048 / ADR-052 (freestanding slots). Do not claim Linux userspace, POSIX or containers, and do not widen “EL0 isolated” past the ADR-091 sentence. Standing enter/leave as a guest sample: apps-today.md. CRT + loader + standing-as-normal: building-or-porting.md.
Direction: ADR-013. TTBR1 first cut: ADR-016. EL1 fetch mile: ADR-017. Identity-tear first cut: ADR-018. Identity .text range tear: ADR-019. Live .text tear: ADR-020. Identity .rodata tear: ADR-025. Identity .data tear: ADR-037. Identity heap tear: ADR-038. Leftover identity RAM tear: ADR-049. EL0 entry without VMALLE1: ADR-039. Lower-EL IRQ while standing: ADR-040. IRQ-unmasked default ERET: ADR-041. Isolation leftover wrap: ADR-042. Lower-EL FIQ + SError park: ADR-043. Taken SError QMP attempt: ADR-045. Isolation leftovers decisions: ADR-047. Taken SError hard-stop: ADR-053. Taken SError reopen (still blocked): ADR-081 / ADR-082. PAN enable lock: ADR-054. Umbrella checklist: ADR-055. Sponsor leftovers closure: ADR-060. App hosting claim criteria: ADR-048. PAN capability: ADR-026. SVC ABI: ADR-021 / syscall.md. CRT / libctos: ADR-022. Guest loader: ADR-023. Standing-as-normal: ADR-024. Thin VFS + memfs: ADR-027. Threat model: security.md. Code: src/el0.rs, src/syscall.rs, src/libctos.rs, src/loader.rs, src/vfs.rs, libctos/, src/asid.rs, src/ttbr1.rs, src/teardown.rs, src/pan.rs.
flowchart TD EL1["EL1 kernel<br/>TTBR · SVC · VFS"] EL0["Standing EL0 task<br/>freestanding image"] SVC["SVC ABI<br/>exit · uart · yield · fs_*"] EL0 -->|"asks"| SVC --> EL1 EL1 -->|"ERET"| EL0
Standing EL0 + SVC is a Verified mile class. It is not a Linux process; the umbrella claim is only the ADR-091 sentence.
flowchart LR M["Verified miles<br/>standing · tears · IRQ/FIQ<br/>PAN ID-field · SError park"] L["Limits kept<br/>stock-TCG SError parks · never-yank _start<br/>one core · not side-channel"] U["Umbrella “EL0 isolated”<br/>Verified: ADR-091 sentence only<br/>ADR-055 checklist Met"] M -.-> U L -.-> U
Miles alone are not the umbrella. “EL0 isolated” means only the accepted ADR-091 sentence (M6), never wider.
What exists today
- Kernel runs at EL1 (
SPSel = 0). - One map-window page (
paging::EL0_PAGE) can be UXN-clear / PXN for a trampoline or standing payload. ERETto EL0 switches to a user TTBR0 (L1_USER, ASID=1) that maps kernel text/rodata + the exception stack + the trampoline window (every 2 MiB those ranges occupy), and omits.data/.bss/ heap. The lower-EL handler restores kernel TTBR0 fromTPIDR_EL1before touching kernel data. After identity.data/heap tear, that path isMSR TTBR0+ISBonly — noTLBI VMALLE1(ADR-039; serialel0: no-vmalle1).- Dual EL1 ASIDs (1 vs 2) with
nGprobe pages switch withoutTLBI VMALLE1(src/asid.rs). - A bounded standing user context on that TTBR0:
SVC #1stays at EL0 (el0: standing), userMOVZruns,SVC #2restores EL1 (el0: restored).is_active()is true only for that lifetime. That dual-SVC is the ADR-013 probe. - A standing task (A4 / ADR-024) is the supported path for a loaded freestanding image: the A3 loader maps
PT_LOAD,is_active()is true while it runs,yield/uart_writestay at EL0 (el0: task-active),exitrestores (el0: task-exit/el0: task-restored). An unexpected lower-EL sync restores fail-closed (el0: restore-fail) instead of parking. That is the ctos process model (ADR-035) — not Linuxfork/execve/wait. - TTBR1 walks are enabled. One kernel-private high page (
TTBR1_PRIV) is EL1-only; EL0 load faults. Identity RAM is aliased atva + TTBR1_BASEvia cloned RAM tables; EL1 can fetch a real path there andVBAR_EL1is the high alias. After a high-VA jump, rustc vtables are rewritten to high aliases (ident: reloc) and live identity.textafter the boot stub is unmapped (ident: live), plus the dedicated 16 KiB range (ident: range/ident: ok). Identity.rodatais unmapped after a pointer rewrite (ident: rodata/ident: rodata-fault/ident: rodata-high). Identity.data/stacks are unmapped after SP relocate (ident: data). Identity heap is unmapped after highGlobalAllocVAs (ident: heap)._start/ QEMU-kernelstay at0x4008_0000(ident: start-stay, ADR-042). Leftover identity frames after the heap are unmapped (ident: ram/ident: ram-fault/ident: ram-high, ADR-049); high twins stay. Never yank_startwhile-kernelneeds that address (ADR-047). PAN ID field is printed (pan: presenton-cpu cortex-a76, ADR-079; historical a57pan: absent); PAN enable + EL1-vs-EL0 fault is Verified (ADR-080). - Lower-EL AArch64 sync handles
SVC(ADR-013 probes#0/#1/#2plus public ABI#16–#23), a kernel-data IABORT, a kernel-data DABORT, and the TTBR1 private-page DABORT, then returns to EL1t (or stays at EL0 on standingSVC #1/SYS_YIELD/SYS_UART_WRITE/ memfs SVCs). Lower-EL IRQ while standing handles the timer (or other) IRQ, then returns to EL0 (el0: irq, ADR-040); default standing/taskERETclears IRQ mask (el0: irq-default, ADR-041). Lower-EL FIQ while standing is taken when GICC FIQEn is armed (el0: fiq, ADR-043). ADR-045 wires an A-clear standing SError probe (el0: serror-arm/eret_to_el0_serror); QEMU 10 virt TCG (a57 and a76) has no working host inject (ADR-081 / ADR-082), so hello keepsel0: serror-park. Short non-standing trampoline probes stay IRQ-masked. Default standing SPSR still masks FIQ and SError except the dedicated probes. - Public SVC ABI (syscall.md):
exit/uart_write/yieldplus memfsfs_*(19–23). AlibctosCRT wraps those numbers; a hello payload is copied onto the standing EL0 page (A2) and also parsed as ELF64PT_LOADon the guest (A3). A4 runs that loaded image as a standing task untilexit. A6 is in-RAM memfs. ABI + CRT + loader + standing-task + memfs miles. App hosting Planned. - Syscall pointers (ADR-094, G4 closed). Every pointer the kernel copies from EL0 must be EL0-readable (
AP[1]=1) on every page. Every pointer it copies to EL0 must be EL0-writable (AP[1]=1,AP[2]=0). Execute-only, kernel-only and unmapped pages return the syscall’s error value, and no byte moves. The loader maps app text+rodata (one R+X page) EL0 read-only + executable (map_el0_text: AP=11, UXN clear, PXN set), so apps can pass their own string literals. Kernel-built probe payloads keep execute-only code and put their data on an EL0-RW NX page. Smoke:el0: sys-ptr denied {kernel-stub,kernel-window,xo-text,write-ro,straddle}, a seven-syscall sweep,el0: sys-ptr ok, and asysptr-leak-probebuild that must printel0: sys-ptr leaked …. ADR-097 (proposed): app text is EL0 execute-only, so a pointer into it now fails the copy-from-EL0 check (xo: sys-read text denied).
Probed miles
| Probe | What closes it | Honesty |
|---|---|---|
| EL0 entered and returned | Serial el0: svc + el0: ok; #[test_case] el0_svc_roundtrip | Entered and left. Not a user process. |
| EL0 cannot execute kernel data | Serial el0: nx kernel; #[test_case] el0_cannot_execute_kernel_data | IABORT on .data (UXN and/or unmapped). Not isolation. |
User TTBR0 omits kernel .data | Walk L1_USER; #[test_case] user_ttbr0_omits_kernel_data | Distinct user table. Text is still mapped so the handler can run. |
EL0 cannot read kernel .data | Serial el0: no kernel read; #[test_case] el0_cannot_read_kernel_data | Translation/permission DABORT. Not PAN. |
| Standing EL0 context | Serial el0: standing / el0: restored; #[test_case] standing_el0_enter_leave | Real enter/leave on user TTBR0. is_active() true only while standing. Not POSIX. Default ERET clears IRQ mask (ADR-041). |
| Standing EL0 as normal mode | Serial el0: task-enter / el0: task-active / el0: task-exit / el0: task-restored / el0: restore-fail / el0: task-ok; #[test_case] standing_task_runs_until_exit + standing_task_fault_restores | Loaded A3 image until exit. Fail-closed restore. Not POSIX. Not isolation. |
| ASID field programmed | user_ttbr0() >> 48 == 1 | Programming fact on the EL0 trampoline. Not the isolation mile. |
| ASID isolation | Serial asid: dual / asid: conflict / asid: ok; #[test_case] asid_isolation_without_vmalle1 | Dual TTBR0 without TLBI VMALLE1. Stale ASID-1 data under ASID 2 is Failed. One mile; alone it is not the ADR-091 umbrella claim. |
| TTBR1 private page | Serial ttbr1: el1 / ttbr1: no el0 / ttbr1: ok; #[test_case] ttbr1_el1_sees_priv_el0_does_not | EL1-only high page. Not a relocated kernel. |
| EL1 fetch from TTBR1 high VA | Serial ttbr1: el1 exec / ttbr1: vbar; #[test_case] el1_executes_from_ttbr1_high_va | Real EL1 path + high VBAR. Identity boot stub stays. |
| Identity-tear first cut | Serial ident: split / ident: fault / ident: high / ident: no el0 / ident: ok; #[test_case] identity_tear_el1_faults_high_stays | One identity text page unmapped; high twin still fetches. Not a relocated kernel. Full teardown Planned. |
| Identity text range tear | Serial ident: jump / ident: range / ident: text; #[test_case] identity_text_range_unmapped_boot_stub_stays | High-VA continuation + 16 KiB dedicated range unmapped; high twins still fetch. Not a relocated kernel. |
High-VA vtable rewrite + live .text tear | Serial ident: reloc / ident: live; #[test_case] identity_fn_ptrs_rewritten_high + live_identity_text_unmapped_boot_stub_stays | rustc dyn Write / fmt tables patched to high aliases; live identity .text after _start unmapped; println! still runs. Not a relocated kernel. |
Identity .rodata tear | Serial ident: ro-reloc / ident: rodata / ident: rodata-fault / ident: rodata-high; #[test_case] identity_rodata_unmapped_high_stays | Identity .rodata unmapped; high twin still loads. Not a relocated kernel. |
Identity .data / stack tear | Serial ident: data-reloc / ident: data / ident: data-fault / ident: data-high; #[test_case] identity_data_torn_high_stays | Identity .data/stacks unmapped; SP high-only (ADR-037). Not a relocated kernel. |
| Identity heap tear | Serial ident: heap-reloc / ident: heap / ident: heap-fault / ident: heap-high; #[test_case] identity_heap_torn_high_stays | Identity heap unmapped; GlobalAlloc returns high VAs (ADR-038). Not a relocated kernel. |
| EL0 entry without full TLBI | Serial el0: no-vmalle1; #[test_case] el0_entry_without_vmalle1; src/exception.rs has no tlbi vmalle1 | Standing/EL0 trampoline is MSR TTBR0 + ISB only after ADR-037/038 (ADR-039). One mile; alone it is not the ADR-091 umbrella claim. |
| Lower-EL IRQ while standing | Serial el0: irq; #[test_case] lower_el_irq_while_standing | Timer IRQ taken from standing EL0 and returned (ADR-040). FIQ/SError still park. One mile; alone it is not the ADR-091 umbrella claim. |
| IRQ-unmasked default ERET | Serial el0: irq-default; #[test_case] lower_el_irq_default_eret | Standing/task default ERET clears I (ADR-041). Short non-standing probes stay masked. One mile; alone it is not the ADR-091 umbrella claim. |
| Lower-EL FIQ while standing | Serial el0: fiq; #[test_case] lower_el_fiq_while_standing | Timer as FIQ via temporary GICC FIQEn (ADR-043). Default SPSR still masks F. One mile; alone it is not the ADR-091 umbrella claim. |
| SError park honesty | Serial el0: serror-park | No working SError inject on virt TCG (ADR-043 / ADR-045 / ADR-081 / ADR-082). Taken path deferred/non-goal (ADR-047). One mile; alone it is not the ADR-091 umbrella claim. |
| Taken SError QMP attempt | Serial el0: serror-arm + host QMP log; taken el0: serror only if inject works | Guest A-clear path wired (ADR-045). QMP inject-nmi fails on QEMU 10 virt+a76 (ADR-081). No #[test_case]. One mile; alone it is not the ADR-091 umbrella claim. |
Identity _start stay | Serial ident: start-stay; #[test_case] identity_boot_stub_stays_while_live_torn | Honesty (ADR-042 / ADR-047): stub stays while live image + heap + leftover RAM are torn. One mile; alone it is not the ADR-091 umbrella claim. |
| Leftover identity RAM tear | Serial ident: ram / ident: ram-fault / ident: ram-high; #[test_case] identity_ram_torn_high_stays | Optional mile taken (ADR-049). High twin stays. Not a relocated kernel. One mile; alone it is not the ADR-091 umbrella claim. |
| PAN ID field | Serial pan: id= / pan: present; #[test_case] pan_present_on_probe_cpu | ID_AA64MMFR1_EL1.PAN != 0 on -cpu cortex-a76 (ADR-079). Historical a57 pan: absent. |
| PAN enable + EL1-vs-EL0 fault | Serial pan: enabled / pan: el1-fault; #[test_case] pan_enable_el1_fault_on_probe_cpu | PSTATE.PAN set; EL1 load of EL0-accessible page faults (ADR-080). One mile; alone it is not the ADR-091 umbrella claim. |
SVC ABI (exit / uart_write / yield) | Serial svc: yield / svc: user-hi / svc: uart / svc: exit / svc: ok; #[test_case] el0_svc_abi_yield_uart_exit + kernel-.data / TTBR1-alias reject | Documented numbers 16–18. Not Linux. Not app hosting. |
libctos CRT | Serial libctos: hi / libctos: ok / libctos: linked; #[test_case] libctos_hello_yield_uart_exit | Linked wrappers, not a hand-encoded trampoline. Not app hosting. |
| Guest ELF PT_LOAD loader | Serial loader: mapped / loader: ok; #[test_case] loader_maps_hello_and_erets | Guest parse + map into user TTBR0 + ERET to e_entry. Not a Linux ABI. Not app hosting. |
| Thin VFS + memfs | Serial fs: create / fs: write / fs: read / fs: el0 / fs: ok; #[test_case] vfs_memfs_create_write_read_close + el0_fs_svc_roundtrip | In-RAM named buffers. Not POSIX. Not FAT. Not app hosting. |
| OS/app slot first cut | Serial slot: fat / slot: mapped / slot: ok; #[test_case] slot_load_from_fat_erets | A9 (ADR-030). FAT /hello + A3 map. A2–A4 load FAT (no production embed, ADR-032). Product freestanding hosting Verified under ADR-048/052 (this mile alone ≠ Linux hosting). |
Still Planned / decided leftovers (isolation) — ADR-047 / locks ADR-053–ADR-055 / sponsor closure ADR-060
| Item | Status | Notes |
|---|---|---|
| PAN enable | Verified (ADR-080) | One clause of the ADR-091 umbrella sentence, not the umbrella by itself (ADR-055). |
Yank _start | Decided: never while QEMU -kernel needs 0x4008_0000 | ident: start-stay Verified honesty (ADR-042). |
| Leftover identity RAM after heap | Verified (optional mile) | ident: ram* (ADR-049). Not required for the umbrella row. |
| Taken lower-EL SError on this smoke machine | Deferred / non-goal (hard-stopped) | Park Verified; A-clear dormant prep (ADR-045). Hard-stop ADR-053; reopen re-probe ADR-081 / ADR-082. Unlock = B1/B2/B3; B1 research ADR-082 (no shippable pin 2026-09-19). Do not claim taken on stock QEMU; the umbrella clause rests on the ADR-090 evidence class (B1 pin + B2-P). |
| Umbrella “EL0 isolated” | Verified (scoped: the accepted ADR-091 sentence only; M6, 2026-09-28; revision 4 accepted 2026-10-02) | ADR-091 Accepted (B3 met 2026-09-28 16:41 CEST); checklist ADR-055 §A/§B all Met; sponsor closure ADR-060. Limits: one core, the TCG default smoke machine; not a speculative-execution or side-channel claim, not a real-hardware claim, not a certification; app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN); no multi-core race story. Specific miles alone are not this row. |
| Product app hosting “done” | Verified | ADR-048 Met×5 / ADR-052. Freestanding slots only — not Linux/POSIX/containers. |
Unprobed stays Unknown. Do not say “EL0 works,” “Linux app host,” or “the kernel moved.” Say “EL0 isolated” only as the accepted ADR-091 sentence (below). The freestanding ADR-048 product claim is Verified separately.
Taken SError research + QMP attempt + hard-stop + reopen (ADR-044 / ADR-045 / ADR-047 / ADR-053 / ADR-081)
Taken lower-EL SError on this smoke machine is deferred / non-goal (hard-stopped) (ADR-053 / ADR-081 / ADR-082 / ADR-047). Research (ADR-044) recommended QMP/nmi; ADR-045 wired the guest A-clear path and attempted QMP — QEMU 10 virt reports machine does not provide NMIs. Re-probed 2026-09-14 (a57) and 2026-09-19 (default a76 + GICv3/virtualization=on/max): same error. Unlock = ADR-081 B1/B2/B3; B1 research ADR-082 (no shippable pin 2026-09-19). el0: serror-park stays Verified; A-clear stays dormant prep. Do not claim taken SError Verified on stock QEMU. 2026-09-26 (sponsor D1, ADR-090): the umbrella’s taken-SError evidence class is the B1 pinned-QEMU CI job qemu-nmi-pin (unconditional, fail-closed bare el0: serror) plus the B2-P one-off Graviton3 KVM run (ADR-085 run 4). ADR-055 §B row 1 is Met on that class; stock TCG still parks.
Umbrella “EL0 isolated” accepted (ADR-091, M6)
Since M6 (2026-09-28) umbrella “EL0 isolated” is Verified, and only as the sentence the sponsor accepted in writing in ADR-091 (B3, 2026-09-28 16:41 CEST). Revision 4 (2026-10-02): the sponsor accepted ADR-091 rev 4 in writing (“I accept ADR-091 rev 4”); only the PAN clause and the code-bytes limit changed. Verbatim, revision 4:
“On the ctos default smoke machine (QEMU
virt,-cpu cortex-a76, TCG, one core), EL0 is isolated from the kernel at the architectural page-table / exception-level boundary: each EL0 task runs on its own TTBR0 with its own ASID; EL0 faults when it reads, writes or executes kernel memory; in every translation table (kernel, user and ASID-B TTBR0, and TTBR1) the only EL0-reachable pages are five allowlisted user slots backed by non-kernel frames, checked fail-closed while a user app runs and at rest; PAN is enabled, including while the kernel handles exceptions from EL0, and EL1 faults on EL0-readable memory (the execute-only app text page is not covered: this CPU has no FEAT_EPAN); lower-EL IRQ and FIQ are taken while EL0 stands; taken lower-EL SError is evidenced by the ADR-090 evidence class; the only identity (VA = PA) mapping left in any TTBR0 is the documented EL1-only_startstub page (ADR-089); and a system call copies from or to an EL0 pointer only if EL0 itself may read (for a copy from EL0) or write (for a copy to EL0) every page in the range. It is not a speculative-execution or side-channel claim, not a real-hardware claim, and not a certification.”
Limits that stay with it:
- One core. No multi-core / SMP race story (no SMP TLB-shootdown story; syscall pointer checks are a one-core check-then-copy, ADR-094).
- The TCG default smoke machine only. Not a real-hardware claim (the B2-P KVM runs are a reduced one-off profile, ADR-085 / ADR-095).
- Not a speculative-execution or side-channel claim. Not a certification.
- app text is EL0 execute-only; EL1 can still read it (no FEAT_EPAN). ADR-097 moves rodata to the
app-hdrpage and maps app text EL0 execute-only (still five slots); accepted as ADR-091 revision 4 on 2026-10-02. Because this CPU has no FEAT_EPAN, PAN does not stop EL1 from reading that page, which is why the revision-4 PAN clause says the execute-only app text page is not covered. History: before 2026-10-02 the limit read “EL0 can read its own code bytes” (app text+rodata EL0 read-only + executable, ADR-094). The PAN-held fix is ADR-098. - Also outside it (ADR-091 non-scope): task-vs-task isolation beyond ASIDs, DMA / IOMMU / EL2, and taken SError on stock QEMU (still parks). Separate evidence (not part of the sentence): ADR-096 probes that an EL0 task on ASID 1 cannot read or write another EL0 task’s page on ASID 2 at the same VA (
t2t: ok; one core, TCG, kernel-built tasks).
Correction (2026-09-29): until ADR-098 the “PAN is enabled” clause held at boot but not while the kernel handled EL0 exceptions or after EL0 trips (the boot probe ran before any EL0 trip). ADR-098 fixes it and checks it live; the sentence was unchanged and true again. Update 2026-10-02: revision 4 now says “including while the kernel handles exceptions from EL0” in the PAN clause itself.
Checklist: ADR-055 (§A and §B all Met). Sponsor closure: ADR-060. Do not round single miles into the umbrella, and do not widen the sentence.
Before M6 (kept for history): Umbrella isolation stays Planned / non-claim until checklist. Verified miles vs unmet requirements: ADR-055. Sponsor-facing closure table (locks + reopen gates only): ADR-060. Unmet blockers include taken SError (ADR-053 / ADR-081 / ADR-082), never-yank _start, and umbrella sponsor accept (PAN enable is Met separately via ADR-080). Do not round Verified miles into “EL0 isolated.”
SVC syscall ABI (Track A / A1 / ADR-021)
ABI + CRT + loader + standing-task + memfs + FAT16 + slot first-cut miles. Sample rebuild recipes are A8 (docs). A9 is two artifacts + FAT /hello (ADR-030) plus leftover cross-update / FAT-only hello (ADR-032). Product “app hosting is done” is Verified under ADR-048 / ADR-052 (freestanding slots). Not Linux. Not POSIX.
Kernel contract: ADR-021. CRT / libctos: ADR-022. Guest loader: ADR-023. Standing-as-normal: ADR-024. Thin VFS + memfs: ADR-027. virtio-blk + FAT16: ADR-028. Parent plan: issue #31. A1: issue #32. A2: issue #33. A3: issue #34. A4: issue #35. A6: issue #37. Code: src/syscall.rs, src/vfs.rs, libctos/, user/hello-libctos/, src/loader.rs, src/el0.rs.
Calling convention
AArch64 SVC #<n> where n is the syscall number. Arguments in x0, x1, x2. Return in x0.
Reserved 0–2 are ADR-013 probes (#0 first-mile return, #1 standing, #2 restore). libctos must not issue them. Public numbers below are the freestanding contract; Linux ABI is out of scope.
Public numbers
| Number | Name | Args | Effect |
|---|---|---|---|
| 16 | exit | x0 = status | Halt the EL0 trip; return to the EL1 caller. Status is recorded. Not a process table. |
| 17 | uart_write | x0 = pointer, x1 = length | Write up to 64 user-mapped and kernel-mapped bytes to the virt PL011. \n → \r\n. Reject (including non-canonical / TTBR1 aliases) → x0 = 0. |
| 18 | yield | none | Dispatch-only hint; return to EL0. Does not switch EL1 tasks (ADR-010). |
| 19 | fs_create | x0 = path, x1 = length | Create an empty memfs name and a handle. Path / + [a-z0-9_-]. |
| 20 | fs_open | x0 = path, x1 = length | Open an existing memfs name. |
| 21 | fs_read | x0 = fd, x1 = buf, x2 = length | Copy from the handle offset into a user-mapped and kernel-mapped range (cap 64). |
| 22 | fs_write | x0 = fd, x1 = buf, x2 = length | Copy into the file at the handle offset (cap 64, file cap 256). |
| 23 | fs_close | x0 = fd | Drop the handle. The file stays. |
| 24 | net_mac | x0 = buf, x1 = length (≥6) | Copy guest virtio-net MAC (6 bytes). Kernel owns the NIC (ADR-068). |
| 25 | net_ping | none | Kernel-path ARP + ICMP echo to SLIRP 10.0.2.2. Not sockets. |
| 26 | net_udp_dns | none | Kernel-path ARP + UDP DNS probe vs SLIRP 10.0.2.3:53 (ADR-071). DNS is probe bait only. Not sockets. Not a DNS product. |
| 27 | fs_mkdir | x0 = path, x1 = length | Create a FAT16 root directory via thin VFS (ADR-075). Memfs / nested paths fail closed. Not POSIX mkdir. |
| 28 | net_tcp_echo | none | Kernel-path thin TCP echo vs QEMU guestfwd 10.0.2.4:7 (ADR-076). Same probe as ADR-074, quiet. Not sockets. Not listen/accept. |
Unknown SVC immediates park (fail-closed). Not POSIX. Not Linux VFS.
Track B B2 maps these numbers against Linux AArch64 (svc #0, x8): linux-aarch64-syscall-gap.md. Track B B4 maps ELF / auxv / PT_INTERP against the freestanding loader: ADR-033. Inspection only. Not claiming Linux userspace. Not claiming dynamic Linux ELF. Do not retarget 16–28 to Linux x8.
Probe
Serial svc: yield / svc: user-hi / svc: uart / svc: exit / svc: ok. #[test_case] el0_svc_abi_yield_uart_exit + el0_uart_write_rejects_kernel_data + el0_uart_write_rejects_high_alias. File presence is not that probe.
libctos (A2)
A no_std crate wraps the three public numbers. A hello payload linked against it is copied onto the standing EL0 page (A2) and parsed as ELF64 PT_LOAD on the guest (A3). Serial libctos: hi / libctos: ok / libctos: linked. File presence is not that probe.
Guest loader (A3)
The kernel parses the hello ELF, maps each PT_LOAD into the user map-window, and ERETs to e_entry (ADR-023). Serial loader: mapped / loader: ok. Not a Linux ELF ABI. Not PT_INTERP. A2–A4 and A9 read FAT /hello. Linux ELF/auxv gaps: ADR-033. File presence is not that probe.
Standing task (A4)
The A3 loader is how a payload appears. A4 makes standing EL0 the supported path: is_active() is true for the loaded task until exit; an unexpected fault restores fail-closed (ADR-024). Serial el0: task-enter / el0: task-active / el0: task-exit / el0: task-restored / el0: restore-fail / el0: task-ok. Not isolation. Not a process table. Linux fork/exec/wait is a documented gap (ADR-035). File presence is not that probe.
memfs (A6)
In-RAM named buffers behind a thin VFS (ADR-027, filesystem.md). Serial fs: create / fs: write / fs: read / fs: el0 / fs: ok. File presence is not that probe.
FAT16 (A7)
Same VFS open on virtio-blk (ADR-028). Serial blk: ok / fat: ok. /probe is the known FAT16 file. Write depth: ADR-050 (fat: write / fat: create). Root listing: ADR-056 (fat: readdir / fat: entries) — kernel/VFS probe, no new SVC. Root delete: ADR-057 (fat: delete) — kernel/VFS probe, no new SVC. Root mkdir kernel probe: ADR-073 (fat: mkdir). EL0 fs_mkdir (27): ADR-075. EL0 net_tcp_echo (28): ADR-076. Not POSIX getdents / unlink / mkdir. File presence is not that probe.
Sample recipes (A8)
Rebuild recipes for the probed classes: what-can-run.md, in-tree user/README.md. Coop UART, RX echo, standing EL0 / libctos hello; optional memfs + FAT16. N4 adds 24–25 (ADR-068). N3.x adds 26 (ADR-071). ADR-075 adds 27 (fs_mkdir). ADR-076 adds 28 (net_tcp_echo). File presence is not a new runtime.
OS/app slots (A9)
Host kernel ELF + published hello-libctos.elf. Guest reads FAT /hello through the same VFS and maps it with the A3 loader (ADR-030). Serial slot: fat / slot: mapped / slot: ok / perf: app-load. No production embed on this path (ADR-032): A2–A4 load the same FAT file. Cross-update (same published ELF on this OS and ba6541c) is ledger-Verified. Probe-only include_bytes! in src/slot.rs is ADR-046 measurement only — never print slot: embed. File presence is not that probe.
Product claim + remaining non-goals (Track A)
Product “app hosting is done” is Verified under ADR-048 / ADR-052 for freestanding OS/app slots. Isolation enable (PAN) is non-goal on default cortex-a57 (ADR-047). Identity .data/heap tears are Verified (ADR-037/038). Still not Linux/POSIX/containers; “EL0 isolated” only as the ADR-091 sentence.
Performance (NFR-07 / ADR-011)
Performance is a first-class pillar, not a Later learning-only note. That does not authorize fake benches.
Rules
- A latency, cycle, or “faster than” sentence is a claim. It needs a probe in the honesty ledger.
- Optimize only after a probe shows a cost. Do not rewrite the scheduler or heap “for speed” on a hunch.
- QEMU
virtnumbers are one environment. They are not a Raspberry Pi probe and not a published SPEC run.
Baseline probe (this tree)
CNTPCT_EL0 is already used to timeout the M5 timer observe window. The pillars ratchet (when landed) measures a fixed trivial loop:
- Serial marker
perf: cntpct delta=<n>(fail closed onperf: probe missed). #[test_case]asserts the counter advanced.
That probe proves the physical counter is readable and moves. It does not claim a microsecond budget, interrupt latency, or a comparison to other kernels.
IRQ-to-handler probe (this tree)
When CNTP fires, the handler records CNTPCT − CNTP_CVAL before rearm. After several ticks the hello kernel prints:
- Serial marker
perf: irq-delta min=<a> max=<b> spread=<b-a> n=<n>(fail closed onperf: irq-delta missed). #[test_case]asserts samples exist andmax >= min.
That is a spread of IRQ-to-handler counter deltas on this QEMU virt guest. It is not a latency budget, not “faster than X,” and not a published bench. QEMU TCG jitter is one environment.
Host debug ELF size (NFR-08, this tree)
scripts/qemu-smoke.sh prints the host byte size of target/aarch64-ctos/debug/ctos after cargo build:
- Host marker
perf: elf-size bytes=<n>(fail closed if missing or< 4096). - This is a measurement, not a size budget and not a “smaller is better” claim.
It does not time QEMU boot.
Boot-to-ready CNTPCT (NFR-08, this tree)
kernel_main samples CNTPCT_EL0 after paging::init (MMU + D-cache on) and again after Hello World! (init complete):
- Serial marker
perf: boot-delta ticks=<n>(fail closed onperf: boot-delta missed). #[test_case]asserts a sample exists and the counter advanced.
That is kernel_main-entry to after-init on this QEMU virt guest. It is not a latency budget, not QEMU startup time, not a published bench, and not criterion.
OS/app slot disconnect (A9) — compared pair (QEMU TCG lab)
A9 #48 / ADR-030 loads a separate app payload from FAT /hello after the OS image (immutability.md, site measure.md). The load path prints perf: app-load ticks=<n>. ADR-046 adds a probe-only linked-in trip of the same hello-libctos.elf bytes (include_bytes! in src/slot.rs only) that prints perf: embed-load ticks=<n>, then an honest pair perf: slot-delta app=<a> embed=<b>. Production A9 stays FAT-only (slot: embed must never print). That is a measurement pair, not a bench and not a percent.
| Class | What we measure | Honesty |
|---|---|---|
| FAT path | VFS/FAT read + ELF parse + user map + ERET | perf: app-load ticks=<n> |
| Probe-only embed | Same bytes already in the kernel image; parse + map + ERET (no FAT) | perf: embed-load ticks=<n> — not slot: embed |
| Pair | Raw tick counts on one boot | perf: slot-delta app=<a> embed=<b>. No “faster/slower.” No invented percent. QEMU TCG jitter. |
| Operational | Smaller OS updates without rebuilding apps | Operational ≠ measured latency. A2–A4 stay FAT-only (ADR-032). |
| Gate | Keep perf: boot-delta. Fail closed on missing perf: app-load / perf: embed-load / perf: slot-delta | Verified only when the ledger has serial evidence on a named tip. |
QEMU TCG jitter is still one lab. No criterion crate. No “slot disconnect is free.”
FAT16 vs memfs write (ADR-051) — compared pair (QEMU TCG lab)
ADR-050 writes a small payload through the thin VFS onto the FAT16 volume. ADR-051 samples CNTPCT around that single vfs::write (perf: fat-write ticks=<n>), then on the same boot times a memfs vfs::write of the same bytes (perf: memfs-write ticks=<n>), and prints perf: fs-write-delta fat=<a> memfs=<b>. That is a measurement pair, not a bench and not a percent.
| Class | What we measure | Honesty |
|---|---|---|
| FAT write | One VFS write of the /probe rewrite payload (fat-wr) | perf: fat-write ticks=<n> |
| Memfs write | One VFS write of the same bytes on /mwprobe | perf: memfs-write ticks=<n> |
| Pair | Raw tick counts on one boot | perf: fs-write-delta fat=<a> memfs=<b>. No “faster/slower.” No invented percent. QEMU TCG jitter. |
| Gate | Keep fat: write / fat: ok / A9 markers. Fail closed on missing pair markers | Verified only when the ledger has serial evidence on a named tip. |
QEMU TCG jitter is still one lab. No criterion crate. No latency SLA for FAT write.
FAT16 vs memfs read (ADR-065) — compared pair (QEMU TCG lab)
The guest already reads known FAT16 /probe bytes through the thin VFS. ADR-065 samples CNTPCT around that single vfs::read (perf: fat-read ticks=<n>), then on the same boot seeds memfs /mrprobe with the same bytes and times vfs::read (perf: memfs-read ticks=<n>), and prints perf: fs-read-delta fat=<a> memfs=<b>. That is a measurement pair, not a bench and not a percent. Symmetric to ADR-051 (write).
| Class | What we measure | Honesty |
|---|---|---|
| FAT read | One VFS read of /probe (fat-hi) | perf: fat-read ticks=<n> |
| Memfs read | One VFS read of the same bytes on /mrprobe | perf: memfs-read ticks=<n> |
| Pair | Raw tick counts on one boot | perf: fs-read-delta fat=<a> memfs=<b>. No “faster/slower.” No invented percent. QEMU TCG jitter. |
| Gate | Keep fat: read / fat: ok / ADR-051 write pair / A9 markers. Fail closed on missing pair markers | Verified only when the ledger has serial evidence on a named tip. |
QEMU TCG jitter is still one lab. No criterion crate. No latency SLA for FAT read.
Net-ping path CNTPCT (ADR-069) — single path (QEMU TCG lab)
ADR-068 EL0 net_ping (SVC 25) already runs a quiet kernel-owned ARP + ICMP echo vs SLIRP 10.0.2.2. ADR-069 samples CNTPCT around that full quiet path and prints perf: net-ping ticks=<n>. That is a measurement, not a bench and not a latency SLA. Related optional N2 crumbs (perf: net-icmp-tx / perf: net-icmp-rx) stay as ADR-067 lab notes.
| Class | What we measure | Honesty |
|---|---|---|
| EL0 net_ping | Quiet ARP + ICMP echo RTT as SYS_NET_PING / el0_icmp_ping | perf: net-ping ticks=<n> |
| Gate | Smoke greps the prefix perf: net-ping (not an exact tick count). Reject faster/slower/percent. Keep N1/N2/N4 / FAT / slot markers. | Verified only when the ledger has serial evidence on a named tip. |
QEMU TCG jitter is still one lab. Tick values vary — do not invent a budget. No criterion crate. No sockets / TCP/UDP this mile.
UDP DNS path CNTPCT (ADR-072) — single path (QEMU TCG lab)
ADR-071 EL0 net_udp_dns (SVC 26) already runs a quiet kernel-owned ARP + UDP DNS query vs SLIRP 10.0.2.3:53. ADR-072 samples CNTPCT around that full quiet path and prints perf: udp-dns ticks=<n>. That is a measurement, not a bench and not a latency SLA. Same pattern as ADR-069 on net_ping — EL0 SVC surface, not a separate kernel-only N3 timer.
| Class | What we measure | Honesty |
|---|---|---|
| EL0 net_udp_dns | Quiet ARP + UDP DNS RTT as SYS_NET_UDP_DNS / el0_udp_dns | perf: udp-dns ticks=<n> |
| Gate | Smoke greps the prefix perf: udp-dns (not an exact tick count). Reject faster/slower/percent. Keep N1/N2/N3/N3.x/N4 / ADR-069 / FAT / slot markers. | Verified only when the ledger has serial evidence on a named tip. |
QEMU TCG jitter is still one lab. Tick values vary — do not invent a budget. No criterion crate. No TCP / FAT mkdir this mile.
Thin TCP echo path CNTPCT (ADR-078) — single path (QEMU TCG lab)
ADR-076 EL0 net_tcp_echo (SVC 28) already runs a quiet kernel-owned ARP + thin TCP active-open + one payload vs QEMU guestfwd 10.0.2.4:7. ADR-078 samples CNTPCT around that full quiet path and prints perf: tcp-echo ticks=<n>. That is a measurement, not a bench and not a latency SLA. Same pattern as ADR-069 / ADR-072 — EL0 SVC surface, not a separate kernel-only N5 timer. Kernel net: tcp-ok markers stay as ADR-074 transport proof.
| Class | What we measure | Honesty |
|---|---|---|
| EL0 net_tcp_echo | Quiet ARP + thin TCP guestfwd RTT as SYS_NET_TCP_ECHO / el0_tcp_echo | perf: tcp-echo ticks=<n> |
| Gate | Smoke greps the prefix perf: tcp-echo (not an exact tick count). Reject faster/slower/percent. Keep N1–N5 / N5.x / ADR-069/072 / FAT / slot markers. | Verified only when the ledger has serial evidence on a named tip. |
QEMU TCG jitter is still one lab. Tick values vary — do not invent a budget. No criterion crate. No listen/accept / sockets / TLS this mile.
Later probes (Planned)
- A tighter “first instruction of
_start” sample if someone maps a.dataslot that BSS-clear will not wipe.
Do not add a host criterion crate or a “bench.yml” that prints invented numbers.
Overview (plain English)
Core principles drive ctos. Track A / Track B / A9 are subordinate and never override this list for speed or marketing. Full list: principles.md.
- Honesty ledger — Verified only with a probe
- Antifragility — fail-closed sensors + ratchets
- Security — threat model; probed mitigations only
- Performance — measure first; no invented benches
- Document-first / one milestone → one PR
ctos is a learning AArch64 kernel for QEMU virt. It is not a desktop, not POSIX, and not a “secure OS.” Status words need a probe in the honesty ledger. This note does not invent latency, size, or “faster than” numbers. Measured markers live in the ledger; they are one environment each.
Vision: product-vision.md. Pillars: pillars.md. Frozen IDs: fr-nfr.md. Samples: apps-today.md. Porting: building-or-porting.md. Immutability: immutability.md (scoped only). Tracks: A / B / N (subordinate; N1 ADR-066, N2 ADR-067, N3 ADR-070, N3.x ADR-071, N4 ADR-068, N5 ADR-074).
Site chapters (source of truth for the published book): Overview. HTTPS at https://ctos.artof.link is Verified (2026-09-11 after #30). See website.md.
KPIs (what we actually measure)
These are sensors, not product SLOs. A missing marker is a fail. A printed number is not a budget.
Performance (NFR-07 / performance.md)
| What we watch | Probe (do not invent a target) |
|---|---|
| Physical counter moves | Serial perf: cntpct delta=<n> + #[test_case] |
| IRQ-to-handler spread on this virt guest | Serial perf: irq-delta min=… max=… spread=… n=… |
| Debug ELF byte size | Host perf: elf-size bytes=<n> (measurement, not “smaller is better”) |
kernel_main after-MMU → after-init | Serial perf: boot-delta ticks=<n> |
| App-load after OS/app split (A9) | Verified when ledger has perf: app-load (+ ADR-046 / ADR-051 pairs). Not a published bench. |
QEMU TCG jitter is one lab. These are not Raspberry Pi numbers and not a published bench. Read the ledger row for the SHA you care about. A9 expected costs (boot/load, SVC, ASID/TTBR, later COW) vs steady EL0 compute: performance.md. No Verified delta — still one ELF.
Antifragility (NFR-05 / antifragility.md)
| What we watch | Probe |
|---|---|
| Hello + milestone strings still print | scripts/qemu-smoke.sh greps; missing string → fail |
| Tests stay fail-closed | cargo test exit 0; --features force-fail must be non-zero |
| Same failure twice | Becomes a sensor/gate, not another README paragraph |
| Failed history kept | Example: Docker b2bbb99 Failed, then layout fix Verified — the Failed row stays |
Unprobed boot stays Unknown. File presence is not QEMU boot.
Application-support scope
Do not say “applications run on ctos.” First-class samples and the cannot-run list live in apps-today.md. Porting stance: building-or-porting.md.
- Can run (probed): coop EL1 UART workers (
sched: task a/b/ok); one-byte UART RX (input: rx 0x41); standing EL0 stub (el0: standing/el0: restored); a loadedlibctoshello as a standing task untilexit(el0: task-ok). A heartbeat/counter variant is the same shape — not in tree until a probe greps it. - Cannot run: Linux ELF, shell, Python, network servers (Track N is ARP+ICMP+UDP + tiny EL0 net/UDP SVC samples — not sockets), POSIX disk apps, SMP, a POSIX-style userspace. “EL0 isolated” is Verified only as the accepted ADR-091 sentence (one core, TCG default smoke machine; limits there); PAN is enabled on default cortex-a76 (ADR-080). Identity
.data/heap/leftover-RAM tears are ADR-037/038/049. Filesystem stance: filesystem.md (memfs + FAT16 read/write). Gaps to host apps + containers: non-goal: host-apps.md.
Building or porting
First-class page: building-or-porting.md. Short honesty:
- Easiest = in-tree
no_stdcoop EL1 onaarch64-ctos.json, proven withcargo/qemu-smoke/docker-smoke. - POSIX / glibc = not easy, not started.
- SVC ABI +
libctos+ guestPT_LOAD+ standing-as-normal for freestanding EL0 = A1 ABI + A2 CRT + A3 loader + A4 standing-task miles (dual-SVC stub still exists; not a syscall table; not isolation).
OS image vs app payloads (A9)
The sponsor goal for “immutability” here is not a frozen kernel. It is to disconnect OS updates from apps: one OS image artifact, separate app payloads, so you can replace the kernel without rebuilding apps and replace apps without rebuilding the kernel.
- First cut + leftover: two host artifacts (
target/aarch64-ctos/debug/ctos+target/hello-libctos.elf) and a FAT16/helloload path (ADR-030, ADR-032, A9 #48). A2–A4 load that file. Stance: immutability.md. - Cross-update: same app ELF on this OS and documented prior OS
ba6541c. Product freestanding app hosting is Verified under ADR-048/052. Do not say OTA / “immutable OS” marketing / Linux hosting. - Do not say “immutable OS.”
- Performance:
perf: app-loadmeasures the FAT load path; ADR-046 adds probe-onlyperf: embed-load+perf: slot-delta(QEMU TCG lab pair — not a percent). ADR-051 addsperf: fat-write/perf: memfs-write/perf: fs-write-delta(same-boot write pair — not a percent). Expected costs: boot/load, SVC, ASID/TTBR, optional later COW. Details: performance.md.
Prerequisites
- Nightly Rust (
rust-toolchain.toml),rust-src,llvm-tools-preview qemu-system-aarch64(Debian package is oftenqemu-system-arm)- Optional: Docker linux/arm64 (
./scripts/docker-smoke.sh) — do not pinlinux/amd64 - To change the kernel: one milestone → one branch → one GitHub PR; author ≠ merger (ADR-002)
A machine that has not run scripts/qemu-smoke.sh (or Docker/GHA equivalent) has Unknown boot.
Advantages
- Small, readable
no_stdtree on one ISA (AArch64 / QEMUvirt/ PL011 UART) - Fail-closed smoke: serial markers,
cargo test, force-fail - Honesty ledger: Verified / Unknown / Planned / Failed, with a probe each
- Failures become sensors (layout miss, live-
.textvtable yank, CRLF shebang) - Three pillars named up front — security and speed still need probes
Drawbacks
- QEMU
virtonly. No Raspberry Pi or board claim - Not POSIX, not multi-tenant, not a product runtime. memfs + FAT16 read/write (filesystem.md; ADR-050). Not a container host (non-goal, ADR-029); host
docker-smoke≠ guest Docker. - Umbrella “EL0 isolated” is Verified only as the accepted ADR-091 sentence (M6; history ADR-047). Live identity
.text/.rodata/.data/ heap / leftover RAM are torn (ADR-020/025/037/038/049);_startstays; PAN is enabled on the default-cpu cortex-a76(ADR-080; historical a57pan: absent) - Scoped immutability only (immutability.md): RO+NX / WXN / live
.texttear are probed. Absolute “immutable OS” is incompatible (heap/PTEs/devices must mutate). OS/app slot first cut (A9 / ADR-030) is two artifacts + FAT/hello. Leftover (ADR-032): A2–A4 load FAT; same ELF on this OS andba6541c. - Performance numbers are guest counter deltas, not a latency budget
- Docs website / custom domain: HTTPS serving the book is Verified (website.md)
- Same-login cannot self-merge; a second identity has to land the PR
Honesty
Do not say “secure OS,” “the kernel moved,” “immutable OS,” or “apps update independently of the OS.” Point at the ledger for any number you quote. Say “EL0 isolated” only as the accepted ADR-091 sentence, never wider. The live docs URL is a separate Verified row — do not invent other site claims.
What can run today
Do not say “applications run on ctos.” These are the probed guest examples on QEMU virt. Each needs a matching honesty ledger row. POSIX and a userspace ABI stay Planned. “EL0 isolated” is Verified only as the accepted ADR-091 sentence (one core, TCG default smoke machine; limits there).
Hub: overview.md. How to add something: building-or-porting.md. Frozen IDs: FR-08 (UART RX), FR-11 (coop yield).
Site source of truth: What can run today (A8 rebuild recipes). In-tree index: user/README.md. This page is extra stance (walkthroughs). HTTPS at https://ctos.artof.link is Verified (2026-09-11 after #30).
One rebuild for every sample: cargo build then ./scripts/qemu-smoke.sh. There is no separate app runner. A8 documents that path. A9 first cut adds slot:* when FAT /hello loads. Cross-update (same ELF on this OS and ba6541c) is ledger-Verified (ADR-032). Product freestanding app hosting is Verified (ADR-048 / ADR-052); not Linux/POSIX/containers.
Sample: cooperative EL1 UART workers (M9)
What it is. Two heap-backed EL1 workers take turns. Each prints one UART line, then the idle thread sees both flags and prints sched: ok. Cooperative only — a task runs until it returns or calls yield_now(). Not a timer slice. Not SMP. Not EL0.
Where. src/sched.rs (task_a / task_b / run_two_tasks). Decision: ADR-010.
Rebuild. Edit those functions. cargo build then ./scripts/qemu-smoke.sh. Recipe hub: what-can-run.md.
How it works.
- Heap must already be up (
src/heap.rs). Worker stacks are 8 KiBVec<u8>boxes, not the linkerSP_EL0stack. spawn(task_a)andspawn(task_b)install AAPCS64 callee-saved frames. After ADR-020 the firstretlands on the high-VA alias of the trampoline.- Idle (
kernel_main/ the test runner) callsyield_now()until both atomics are set, or a bound is hit. - Each worker stores its SP, writes a raw UART line, sets its flag, and returns (
Done).
Probe (do not invent extra markers).
| Serial | Meaning |
|---|---|
sched: task a | Worker A ran on a heap stack |
sched: task b | Worker B ran on a heap stack |
sched: ok | Both ran; smoke greps all three |
#[test_case] two_tasks_run_on_distinct_heap_stacks and yield_round_robin_resumes_both close the same claim. Idle tip Docker on main e80dc93: those strings still print on the hello path (50 tests). File presence is not that probe.
Heartbeat / counter variant (not in tree)
Same shape as task_a / task_b, not a new subsystem. A worker that loops, increments an AtomicU64, prints sched: beat n=<n>, and yields would be a counter / heartbeat gadget.
That variant is not in src/sched.rs today. There is no sched: beat smoke string. Adding it is the easiest in-tree exercise (building-or-porting.md): one extra fn, one spawn, one milestone → one PR. Until a probe greps the new marker, treat it as Planned, not Verified.
Sample: UART RX echo gadget (M6)
What it is. The guest polls the virt PL011 RX FIFO and prints the first byte the host injects. On the smoke path that byte is 0x41 ('A'). That is one-shot observe, not a line editor and not a shell.
Where. src/uart.rs (observe_rx / observe_probe_byte). Host inject: scripts/qemu-serial-inject.py (default CTOS_INPUT_BYTE=0x41). Decision: ADR-007.
Rebuild. Same cargo build + ./scripts/qemu-smoke.sh (the script injects 0x41). Recipe: what-can-run.md.
How it works.
- After the hello path has printed earlier markers,
observe_probe_bytespins onUARTFR.RXFEuntil a byte arrives orCNTPCTtimes out. - IRQs stay masked on this path. There is no UART IRQ in M6. Do not
wfiwaiting for RX. - The expected byte prints
input: rx 0x41. Any other byte printsinput: rx 0xNN (unexpected)and fails the probe. Timeout printsinput: rx missed.
Probe.
| Serial | Meaning |
|---|---|
input: rx 0x41 | Host-injected 0x41 landed in the FIFO |
input: rx missed | Fail-closed (smoke rejects this) |
#[test_case] uart_rx_fifo_empty_without_host_byte only proves the FIFO is empty under cargo test (no inject). Character proof is the hello serial row.
This is not interactive echo, virtio-keyboard, or a TTY. A later line-oriented gadget would be a new milestone with its own marker.
Sample: standing EL0 stub (SVC enter / leave)
What it is. A bounded user context on the user TTBR0. The kernel ERETs to EL0, the payload announces with SVC #1, runs one user MOVZ, then SVC #2 restores EL1. el0::is_active() is true only for that lifetime.
Where. src/el0.rs (standing install) and the lower-EL sync path in src/exception.rs. Direction: ADR-013. Narrative: el0.md.
How it works.
- First miles already proved enter/return (
el0: svc), UXN fetch (el0: nx kernel), and no kernel.dataread (el0: no kernel read). - Standing payload on the EL0 window:
SVC #1→ serialel0: standing(stay at EL0) →MOVZ X1, #0x51A4→SVC #2→ serialel0: restored(back to EL1). - Saved user PC / SP / TTBR0 live only while
is_active()is true. The handler clears that flag on restore.
Probe.
| Serial | Meaning |
|---|---|
el0: standing | SVC #1 taken; still at EL0 |
el0: restored | SVC #2 taken; back to EL1 |
el0: ok | First-mile bundle including standing |
#[test_case] standing_el0_enter_leave closes enter/leave. After A1 the standing stub also runs the documented ABI trip (svc: yield / svc: user-hi / svc: uart / svc: exit / svc: ok) — syscall.md. After A2 a hello linked against libctos prints libctos: hi / libctos: ok (ADR-022). After A3 the same ELF is guest-parsed (loader: ok, ADR-023). After A4 that loaded image is a standing task until exit (el0: task-ok, ADR-024); an unexpected fault restores fail-closed (el0: restore-fail). Lower-EL IRQ still parks. PAN is enabled on the default -cpu cortex-a76 (ADR-080; PAN enable was a non-goal on the historical -cpu cortex-a57, ADR-047; a57 ID-field pan: absent Verified). This is not a user process and not POSIX; “EL0 isolated” means only the accepted ADR-091 sentence. The ABI + CRT + loader + standing-task miles are not app hosting (ADR-048).
Rebuild the hello. Source is user/hello-libctos/ (user/hello-libctos/README.md). A kernel cargo build publishes it via build.rs onto FAT /hello. Standalone:
cargo build --release \
--manifest-path user/hello-libctos/Cargo.toml \
--target user/hello-libctos/aarch64-ctos-user.json
That ELF still has to land on FAT /hello to run. The hello does not call fs_open. A second freestanding sample (user/fs-libctos, FAT /fsdemo) does exercise create/open/read/write/close on /memdemo (ADR-059; markers libctos: fs-hi / libctos: fs-ok / fsdemo: ok). A third (user/fat-libctos, FAT /fatdemo) opens/reads FAT /probe (fat-hi) via thin VFS (ADR-061; markers libctos: fat-hi / libctos: fat-ok / fatdemo: ok). A fourth (user/yield-libctos, FAT /yldemo) exercises several cooperative yield_now() rounds (ADR-062; markers libctos: yld-hi / libctos: beat / libctos: yld-ok / yldemo: ok). A fifth (user/net-libctos, FAT /netdemo) exercises EL0 net SVCs (ADR-068; markers libctos: net-hi / libctos: net-mac / libctos: net-ok / netdemo: ok). A sixth (user/udp-libctos, FAT /udpdemo) exercises EL0 net_udp_dns (ADR-071; markers libctos: udp-hi / libctos: udp-ok / udpdemo: ok). A seventh (user/mkdir-libctos, FAT /mkdemo) exercises EL0 fs_mkdir (ADR-075; markers libctos: mkdir-hi / libctos: mkdir-ok / mkdemo: ok). An eighth (user/tcp-libctos, FAT /tcpdemo) exercises EL0 net_tcp_echo (ADR-076; markers libctos: tcp-hi / libctos: tcp-ok / tcpdemo: ok). The kernel /eprobe trampoline (fs: el0) remains a separate Verified trip. Not POSIX. Not getdents. Not preemption. Not sockets.
Sample: freestanding libctos VFS (fs-libctos, ADR-059)
What it is. Second freestanding EL0 ELF linked against libctos. Creates /memdemo on memfs (ADR-058 /mem prefix), writes fs-hi, reads it back, prints libctos: fs-ok. Loaded from FAT /fsdemo.
Where. user/fs-libctos/, src/fsdemo.rs, scripts/mkfat16.py --app2. Decision: ADR-059.
Rebuild. Same cargo build + ./scripts/qemu-smoke.sh.
Probe. libctos: fs-hi / libctos: fs-ok / fsdemo: fat / fsdemo: mapped / fsdemo: ok. Keep slot: ok / /hello. Not POSIX. Not getdents.
Sample: freestanding libctos FAT (fat-libctos, ADR-061)
What it is. Third freestanding EL0 ELF linked against libctos. Opens FAT /probe, reads fat-hi through thin VFS (/ → FAT16), prints libctos: fat-ok. Loaded from FAT /fatdemo. Deeper than fs-libctos (memfs only).
Where. user/fat-libctos/, src/fatdemo.rs, scripts/mkfat16.py --app3. Decision: ADR-061.
Rebuild. Same cargo build + ./scripts/qemu-smoke.sh.
Probe. libctos: fat-hi / libctos: fat-ok / fatdemo: fat / fatdemo: mapped / fatdemo: ok. Keep slot: ok / fsdemo: ok / /hello / /fsdemo. Not POSIX. Not getdents. Fourth sample: Recipe yield below.
Sample: freestanding libctos yield rounds (yield-libctos, ADR-062)
What it is. Fourth freestanding EL0 ELF linked against libctos. Prints libctos: yld-hi, then several libctos: beat lines across yield_now() rounds (deeper than hello’s single yield), then libctos: yld-ok. Loaded from FAT /yldemo. Cooperative only — not preemption, not multi-task EL0, not a process table.
Where. user/yield-libctos/, src/yldemo.rs, scripts/mkfat16.py --app4. Decision: ADR-062.
Rebuild. Same cargo build + ./scripts/qemu-smoke.sh.
Probe. libctos: yld-hi / ≥3× libctos: beat / libctos: yld-ok / yldemo: fat / yldemo: mapped / yldemo: ok. Keep slot: ok / fsdemo: ok / fatdemo: ok / /hello / /fsdemo / /fatdemo.
Sample: freestanding libctos net SVCs (net-libctos, ADR-068)
What it is. Fifth freestanding EL0 ELF linked against libctos. Prints libctos: net-hi, reads guest MAC (libctos: net-mac), asks the kernel for an ICMP echo to SLIRP (net_ping), then libctos: net-ok. Loaded from FAT /netdemo. Kernel owns virtio-net — no guest driver in EL0.
Where. user/net-libctos/, src/netdemo.rs, scripts/mkfat16.py --app5. Decision: ADR-068.
Probe. libctos: net-hi / libctos: net-mac / libctos: net-ok / netdemo: fat / netdemo: mapped / netdemo: ok. Keep slot: ok / fsdemo: ok / fatdemo: ok / yldemo: ok / N1 net: ok / N2 net: ping-ok / N3 net: udp-ok. Not TCP product. Not sockets. Not “has networking.”
Sample: freestanding libctos UDP DNS SVC (udp-libctos, ADR-071)
What it is. Sixth freestanding EL0 ELF linked against libctos. Prints libctos: udp-hi, asks the kernel for a quiet ARP + UDP DNS probe vs SLIRP (net_udp_dns), then libctos: udp-ok. Loaded from FAT /udpdemo. Kernel owns virtio-net — no guest driver in EL0. DNS is probe bait only.
Where. user/udp-libctos/, src/udpdemo.rs, scripts/mkfat16.py --app6. Decision: ADR-071.
Probe. libctos: udp-hi / libctos: udp-ok / udpdemo: fat / udpdemo: mapped / udpdemo: ok. Keep slot: ok / fsdemo: ok / fatdemo: ok / yldemo: ok / netdemo: ok / N1 net: ok / N2 net: ping-ok / N3 net: udp-ok. Not TCP product. Not sockets. Not a DNS product. Not “has networking.”
Freestanding mkdir-libctos (EL0 fs_mkdir)
What it is. Seventh freestanding EL0 ELF linked against libctos. Prints libctos: mkdir-hi, creates FAT root /edir via fs_mkdir (SVC 27), proves Exists/BadPath mapping, then libctos: mkdir-ok. Loaded from FAT /mkdemo (8.3 — not /mkdirdemo).
Where. user/mkdir-libctos/, src/mkdemo.rs, scripts/mkfat16.py --app7. Decision: ADR-075.
Probe. libctos: mkdir-hi / libctos: mkdir-ok / mkdemo: fat / mkdemo: mapped / mkdemo: ok. Keep fat: mkdir / prior six samples / net. Not POSIX mkdir. Not a nested tree.
Freestanding tcp-libctos (EL0 net_tcp_echo)
What it is. Eighth freestanding EL0 ELF linked against libctos. Prints libctos: tcp-hi, runs quiet thin TCP guestfwd echo via net_tcp_echo (SVC 28), then libctos: tcp-ok. Loaded from FAT /tcpdemo.
Where. user/tcp-libctos/, src/tcpdemo.rs, scripts/mkfat16.py --app8. Decision: ADR-076.
Probe. libctos: tcp-hi / libctos: tcp-ok / tcpdemo: fat / tcpdemo: mapped / tcpdemo: ok. Keep net: tcp-ok / prior seven samples. Not BSD sockets. Not listen/accept. Not TLS.
Sample: memfs named-buffer probe (A6)
What it is. Thin VFS + in-RAM buffers. Kernel creates /kprobe, writes, reads. An EL0 trampoline does the same for /eprobe (memfs-el0). Not POSIX. Not a directory tree.
Where. src/vfs.rs, src/syscall.rs (el0_fs_svc_roundtrip). Decision: ADR-027.
Rebuild. Same cargo build + ./scripts/qemu-smoke.sh.
Probe. fs: create / fs: write / fs: read / fs: el0 / fs: ok. File presence is not that probe.
Sample: FAT16 /probe on virtio-blk (A7)
What it is. Guest programs virtio-mmio block and vfs::open("/probe") reads fat-hi from a host-built FAT16 image. Same open / write as memfs. Write depth + small create: ADR-050. Same-boot CNTPCT vs memfs write: ADR-051.
Where. src/virtio.rs, src/fat.rs, scripts/mkfat16.py. Decision: ADR-028.
Rebuild. Same smoke. The script writes target/fat16.img and attaches -drive if=none,file=…,id=hd0 -device virtio-blk-device,drive=hd0. Host -drive without fat: ok is not the probe.
Probe. blk: virtio / blk: cap / blk: rw / blk: ok and fat: mount / fat: read / fat: write / fat: create / fat: ok, plus perf: fat-write / perf: memfs-write / perf: fs-write-delta when the ledger has ADR-051. Do not say “supports FAT.”
Also on the same hello path
These are kernel sensors, not apps. Cite the ledger SHA if you quote a number. Recent A7 smoke still printed:
- UART
Hello World! paging: ok,heap: ok(Box/Vecon the first-fit heap)timer: tick(GICv2 + CNTP)wx: ok,guard: ok,ro: okident: reloc/ident: live/ident: rodata/ident: oksvc: ok/libctos: ok/loader: ok/el0: task-okfs: ok/blk: ok/fat: okexception: sync BRKthenexception: fatal nested
What cannot run today
Explicit no. Do not paper over these with a “porting guide.”
| Want | Why not |
|---|---|
| A Linux ELF / glibc / musl binary | No exec, no ELF loader, no syscall table. Custom target is os: none. |
A shell (sh, bash) or line-oriented TTY | RX probe is one injected byte. No line discipline. |
| Python, Node, or any hosted interpreter | Needs a process ABI, heap policy, and usually a filesystem. |
| Network / sockets / HTTP | No virtio-net, no stack, no sockets. |
| POSIX / Linux disk apps | memfs (A6) + FAT16 on virtio-blk (A7 / ADR-028 + write ADR-050). Not POSIX. Not FAT32. Stance: filesystem.md. |
| SMP / a second CPU / preemptive threads | M9 is cooperative EL1 on one vCPU. |
| Isolated userspace / “an app you compile and exec” | Standing EL0 is a stub. “EL0 isolated” is Verified only as the accepted ADR-091 sentence (one core, TCG; limits there): PAN enabled on a76 (ADR-080); _start stays; leftover identity RAM torn (ADR-049). Identity .rodata/.data/heap are torn (ADR-025/037/038). Gaps: host-apps.md. |
| OCI / Docker / k8s in the guest | No — non-goal (ADR-029). Host docker-smoke.sh only builds the kernel. |
Raspberry Pi or any board other than QEMU virt | Unprobed. Do not copy virt Verified onto hardware. |
| GPU / desktop / windowing / virtio devices | Out of scope on this horizon. |
Umbrella “EL0 isolated” is Verified only as the accepted ADR-091 sentence (M6). PAN is enabled on default cortex-a76 (ADR-080). Identity .rodata / .data / heap / leftover RAM tears are probed (ADR-025/037/038/049); _start stays. See el0.md.
Building or porting
Honesty first: there is a tiny libctos CRT for the A1 SVC ABI, a guest PT_LOAD loader for that in-tree ELF, and no libc. ctos is a freestanding no_std kernel on a custom target. Status words need a probe in the honesty ledger.
What already runs: apps-today.md. Recipes: What can run today, in-tree user/README.md. KPIs and trade-offs: overview.md.
Site source of truth: Building or porting. This page is extra stance. HTTPS at https://ctos.artof.link is Verified (2026-09-11 after #30).
Easiest path — in-tree no_std cooperative EL1
The only path that matches today’s probes is another kernel task in this repo, built for aarch64-ctos.json, proven with the same smoke you already have.
Target and build
| Piece | What it is |
|---|---|
aarch64-ctos.json | Custom rustc target: os: none, abort, soft-float, static, rust-lld. Not aarch64-unknown-linux-gnu. |
.cargo/config.toml | Pins that JSON, build-std (core / alloc / compiler_builtins), -Tlinker.ld, QEMU runner. |
rust-toolchain.toml | Nightly + rust-src / llvm-tools-preview. |
cargo build # ELF at target/aarch64-ctos/debug/ctos
cargo run # qemu-system-aarch64 -machine virt
./scripts/qemu-smoke.sh # hello greps + cargo test + force-fail
./scripts/docker-smoke.sh # same smoke in linux/arm64 (do not pin amd64)
A machine that has not run qemu-smoke.sh (or Docker / GHA equivalent) has Unknown boot. Idle tip: sponsor Docker is Verified on main e80dc93 (50 tests, ident: reloc n=12, live pages=37, force-fail). That is one host, not this VM and not a Pi.
Add a cooperative EL1 task
Same shape as apps-today.md (src/sched.rs task_a / task_b):
- Write a
fn my_task()that usesalloc/ UART /sched::yield_now()only. Nostd, no files, no sockets. spawn(my_task)next to the existing workers (heap must be up; stacks are heapVecs).- Print a new serial marker (
sched: beat n=…or similar). Teachscripts/qemu-smoke.shto grep it. Fail closed on miss. - One milestone → one branch → one GitHub PR. Author ≠ merger (ADR-002).
A heartbeat / counter loop is the natural variant. It is not in the tree until that PR lands. Do not claim it is Verified from this paragraph.
Stay on AArch64 QEMU virt / PL011 (ADR-003). Do not restore bootloader 0.9 or VGA as primary.
POSIX / glibc port — not easy, not started
A Linux, musl, or glibc binary will not run. Missing, among other things:
exec/ ELF loader / dynamic linker- syscall table (
read/write/open/mmap/clone/ …) - filesystem as Linux defines it (thin VFS + memfs + FAT16 is not POSIX
open— filesystem.md), signals, sockets,environ, TLS as Linux defines them - a C runtime (
crt0, libgcc helpers as a POSIX process)
Do not publish a “port busybox / musl to ctos” guide that skips those gaps. That work would be many ADRs, not a weekend #ifdef. Frozen Out list: fr-nfr.md (userspace processes, POSIX, networking).
libctos (A2) — freestanding EL0 CRT
Standing EL0 is still a dual-SVC stub (SVC #1 stay / SVC #2 restore) plus first-mile SVC #0, plus the A1 public ABI trip. A2 adds a no_std crate you can link:
| Piece | What it is |
|---|---|
libctos/ | Wrappers exit / uart_write / yield_now over SVC #16 / #17 / #18. Must not issue #0–#2. |
libctos/src/crt0.S | _start → main → ctos_exit. No argv / environ. |
user/hello-libctos/ | Hello that prints libctos: hi / libctos: ok via uart_write. Recipe: user/hello-libctos/README.md. |
That is not glibc. Not exec of a Linux ELF. The hello image is host-built. A2 copies the flattened PT_LOAD onto the standing EL0 page (ADR-022). A3 parses the same ELF on the guest and maps PT_LOAD into user TTBR0 (ADR-023). File presence is not the probe — see the ledger for libctos: ok and loader: ok.
Guest loader (A3)
src/loader.rs walks ELF64 LE AArch64 ET_EXEC program headers, maps PT_LOAD pages in the user map-window, and ERETs to e_entry. Rejects PT_INTERP and W+X. A2–A4 and A9 read FAT /hello (ADR-030, ADR-032). Not a Linux ABI. Linux ELF / auxv / PT_INTERP gaps: ADR-033. Do not claim dynamic Linux ELF.
Standing EL0 as normal mode (A4)
A4 keeps the A3 loader as the way a payload appears. install_task + ERET is the supported path: the loaded image runs until exit; is_active() is true only for that lifetime; an unexpected EL0 fault restores fail-closed (ADR-024). Serial el0: task-ok. File presence is not that probe.
That is not a process and not POSIX.
Still Planned / decided:
- Umbrella “EL0 isolated” is Verified only as the accepted ADR-091 sentence (M6; one core, TCG default smoke machine; not speculative-execution / side-channel, not real hardware, not a certification). PAN is enabled on default
cortex-a76(ADR-080); leftover identity RAM after the heap is torn (ADR-049); never yank_start. A5 took.rodata(ADR-025),.data(ADR-037), heap (ADR-038), and the PAN ID-field cut. - Product freestanding app hosting is Verified (ADR-048 / ADR-052). Leftover cross-update is Verified host smoke on this OS and
ba6541c(ADR-032). A2–A4 load FAT/hello(no production embed). Not Linux/POSIX/containers.
“Write a user program for ctos” still means: link libctos in-tree, publish target/hello-libctos.elf, and put it on FAT /hello — or add an EL1 task. The easiest thing you can do today remains an in-tree EL1 task.
The OS image vs app payload split (A9 #48) has a first cut plus leftover two-boot smoke. Product freestanding app hosting is Verified under ADR-048/052 (not Linux/POSIX). See overview.md and immutability.md.
Do not invent
- A porting guide that assumes POSIX, a shell, Python, or containers (host-apps.md: containers are a non-goal)
- A claim about the docs URL that skips the ledger (HTTPS is Verified as of 2026-09-11; do not invent extra site KPIs)
- “Secure OS,” “the kernel moved,” or “EL0 isolated” wider than the ADR-091 sentence
Cite the ledger for any Verified SHA you quote.
Filesystem stance
Thin VFS with two backends and prefix mounts. memfs (A6 / ADR-027) is in-RAM named buffers. FAT16 on virtio-blk (A7 / ADR-028) is an on-disk volume; write depth is ADR-050; multi-cluster grow is ADR-064; root listing is ADR-056; root delete is ADR-057; root mkdir is ADR-073; nested mkdir + empty rmdir is ADR-077; EL0 fs_mkdir is ADR-075. Path prefixes route through a mount table (ADR-058). Same vfs::open / read / write / close (+ readdir / unlink / mkdir / rmdir for FAT). No POSIX mount / getdents / unlink / rmdir, no FAT32, no xv6-like inode FS. Linux VFS objects vs this thin surface: ADR-034 (Track B B5). Not claiming a Linux filesystem.
Do not say “ctos has files” as if it were a Linux volume. Do not mint a new FR/NFR ID in chat. Frozen Out list already names POSIX / userspace as later (fr-nfr.md). Hub: overview.md. Samples that do run: apps-today.md.
Site source of truth: Filesystem. This page is extra stance. HTTPS at https://ctos.artof.link is Verified (2026-09-11 after #30).
Order
Each step is one milestone → one branch → one PR.
| Step | What it is | Why this order | Status |
|---|---|---|---|
| 1. memfs | In-RAM named buffers on the first-fit heap (src/heap.rs). Create / lookup / read / write of a path without DMA. | Same “easiest in-tree” shape as a coop EL1 task. | Verified (A6 / ADR-027). Serial fs: ok. |
| 2. virtio-blk | QEMU virt virtio-mmio block: virtqueues, DMA, sector R/W. | Host -drive is not a probe. | Verified when the ledger has blk: ok (A7 / ADR-028). |
| 3. FAT16 | Host-visible 4 MiB raw image; guest opens /probe (PROBE) and reads fat-hi. Same volume also stores freestanding samples /hello (A9), /fsdemo (ADR-059), /fatdemo (ADR-061), /yldemo (ADR-062), /netdemo (ADR-068), /udpdemo (ADR-071), /mkdemo (ADR-075). | Chosen over xv6-like so the host can inspect the image (scripts/mkfat16.py --check). FAT32 / xv6 rejected this mile. | Verified when the ledger has fat: ok. Write depth: ADR-050. Multi-cluster grow: ADR-064. Root listing: ADR-056. Root delete: ADR-057. Sample paths are catalog slots, not a second FS. |
| 4. Prefix mounts | Mount table routes /mem (+ A6 names) → memfs and / → FAT16. | Product-depth virtual mounts without a new disk format. | Verified when the ledger has vfs: mounts (ADR-058). Not mount(2). |
fs: create / fs: write / fs: read / fs: el0 / fs: ok stay fail-closed. A7 adds blk: virtio / blk: cap / blk: rw / blk: ok and fat: mount / fat: read / fat: ok. ADR-050 adds fat: write / fat: rewrite / fat: create. ADR-064 adds fat: grow. ADR-056 adds fat: readdir / fat: entries. ADR-057 adds fat: delete. ADR-073 adds fat: mkdir. ADR-077 adds fat: nested / fat: rmdir. ADR-075 adds EL0 fs_mkdir / libctos: mkdir-ok / mkdemo: ok. ADR-058 adds vfs: mount / vfs: mounts.
QEMU flags (smoke + cargo runner):
-drive if=none,file=target/fat16.img,format=raw,id=hd0 -device virtio-blk-device,drive=hd0
What this is not
- Not POSIX
open/stat/mount. Handles are a thin VFS fd, not a Linux fd table (building-or-porting.md). - Not virtio-net, 9p, or a Linux rootfs.
- Not “we have a disk because QEMU can attach one.” Host
-drivewithout guest virtio + a VFS read is not a probe. - Not POSIX writeable volumes. Guest FAT write is a thin-VFS depth mile (ADR-050); multi-cluster grow is ADR-064; say “wrote / grew FAT16 bytes” when markers pass. Not a POSIX write API.
- Not POSIX
getdents/opendir. Guest FAT root listing is a thin-VFS depth mile (ADR-056); say “listed FAT16 root entries” when markers pass. - Not POSIX
unlink/remove. Guest FAT root delete is a thin-VFS depth mile (ADR-057); say “deleted a FAT16 directory entry” when markers pass. - Not POSIX
mount(2)/ Linux vfsmount. Prefix mounts are a thin-VFS depth mile (ADR-058); say “routed path prefixes through a mount table” when markers pass. - Not app hosting. A8 is documented sample rebuild recipes (what-can-run.md).
Honesty
| Claim | Probe | Status |
|---|---|---|
| memfs create/write/read/close | Serial fs: ok + #[test_case] | Verified (A6; honesty ledger) |
| virtio-blk works | Serial blk: ok + sector R/W #[test_case] | Verified on this tip when the ledger has the probe |
| FAT16 on that device | Serial fat: ok; VFS /probe = fat-hi | Verified on this tip when the ledger has the probe |
| FAT16 write + small create | Serial fat: write / fat: create; host --check-write | Verified on this tip when the ledger has the probe (ADR-050) |
| FAT16 multi-cluster grow | Serial fat: grow / libctos: fat-grow | Verified on this tip when the ledger has the probe (ADR-064) |
| FAT16 root mkdir | Serial fat: mkdir | Verified on this tip when the ledger has the probe (ADR-073) |
EL0 fs_mkdir | Serial libctos: mkdir-ok / mkdemo: ok | Verified on this tip when the ledger has the probe (ADR-075) |
| FAT16 nested mkdir + empty rmdir | Serial fat: nested / fat: rmdir | Verified on this tip when the ledger has the probe (ADR-077) |
| FAT16 root readdir | Serial fat: readdir / fat: entries | Verified on this tip when the ledger has the probe (ADR-056) |
| FAT16 root delete | Serial fat: delete | Verified on this tip when the ledger has the probe (ADR-057) |
| Thin VFS prefix mounts | Serial vfs: mount / vfs: mounts | Verified on this tip when the ledger has the probe (ADR-058) |
Unprobed stays Unknown. File presence is not that probe. See the honesty ledger.
Gaps to host apps — and no containers
Do not say “you can run host apps on ctos.” A host app here means a program you already run on Linux or macOS: a shell, Python, a browser, a Docker/OCI container, a glibc ELF you exec. Today’s guest can run the probed samples only. Closing the gap is Planned in pieces, not a product claim.
./scripts/docker-smoke.sh is a host harness (build the kernel inside a Linux container, then QEMU). It is not a guest container runtime. ctos is not a container host.
Hub: overview.md. Porting: building-or-porting.md. Filesystem: filesystem.md.
Site source of truth: Hosting apps / containers. Product “hosting done” claim: Verified under ADR-048 / ADR-052 (freestanding slots; containers remain non-goal). This page is extra stance. HTTPS at https://ctos.artof.link is Verified (2026-09-11 after #30).
What a host app needs vs what exists
| Need (typical host app) | On ctos today | Status |
|---|---|---|
A process you exec | Standing EL0 + A1 ABI + A2 libctos + A3 PT_LOAD + A4 standing task + A9 FAT /hello first cut | Planned as Linux exec. Stance: ADR-035. A9 is two artifacts + a FAT load path, not a process table. |
| POSIX / glibc / musl | Custom aarch64-ctos.json, os: none, no libc | Not easy, not started. |
Files (open / a disk) | Thin VFS: memfs + FAT16 (read + write depth, ADR-050) | memfs + FAT miles (A6/A7 + ADR-050). Not POSIX. |
| Sockets / HTTP | No virtio-net, no stack | Planned at best; not a Now mile |
| Shell / TTY / Python | One injected UART byte; no interpreter | No until ABI + FS + line discipline |
| Isolated userspace | Standing miles + identity tears incl. leftover RAM (ADR-049); PAN enabled on a76 (ADR-080); _start stays | Umbrella “EL0 isolated” Verified only as the accepted ADR-091 sentence (one core, TCG; limits there) |
| Preemption / SMP | Cooperative EL1, one vCPU | No on this horizon |
| Containers (OCI / Docker / k8s as the guest) | Nothing | Non-goal. See below. |
The easiest thing you can add today is still an in-tree no_std coop EL1 task — not a host binary.
Containers: non-goal
ctos will not run containers as a guest feature. This is a non-goal, not “later on this horizon.” ADR-029. Site SoT: hosting-apps.md.
- No OCI image pull, no
runc, no cgroups, no Linux namespaces, no overlay FS, no containerd/CRI. - A Linux Docker/Podman host that builds this kernel is unrelated. That smoke does not make QEMU
virta container host. - “Run Alpine on ctos” / “k8s node” is the same class of claim as “POSIX port is easy.” It is not.
Do not write a container roadmap. Track A gaps (ABI, FS, isolation) are not a path to OCI. Do not mint a new FR ID for containers. Reopen only with a new GitHub epic plus a new ADR.
Honesty
| Claim | Probe | Status |
|---|---|---|
| Guest is not a container host | Source: no OCI/runc/cgroup/namespace code in src/ | Verified (absence) |
Host docker-smoke.sh builds the kernel | Ledger Docker rows (sponsor / GHA) | Separate claim — host harness only |
| A host app (Linux ELF, shell, Python) runs in the guest | No such serial marker | Planned |
| Guest is a container host | No OCI/runc/cgroup code | Verified absence; non-goal (ADR-029; not a later Planned feature) |
| Linux-compat goals documented (Track B B1) | Read ADR-031 | Documented frame. Not Linux userspace. B6 stays Planned research. |
| Linux AArch64 vs ctos SVC gap (Track B B2) | Read linux-aarch64-syscall-gap.md | Documented inspection map. No Linux numbers in src/. Not a Linux ABI. |
| Process model vs standing EL0 (Track B B3) | Read ADR-035 | Documented stance. Standing EL0 stays the ctos model. Not fork/execve/wait. |
Linux ELF / auxv / PT_INTERP vs freestanding loader (Track B B4) | Read ADR-033 | Documented gap ADR. Track A loader stays reusable. Does not accept PT_INTERP. Not dynamic Linux ELF. |
| Linux VFS vs thin ctos VFS (Track B B5) | Read ADR-034 | Documented concept compare. No POSIX flags / dentries in src/. Not a Linux filesystem. |
File presence of this note is not an app runtime. See the honesty ledger.
Immutability (scoped, not absolute)
Compatible with ctos principles only as scoped immutability: code and other RO regions stay non-writable after a probed lock-down; later, loaded app images can be RO too. That lines up with security (NFR-10), honesty (NFR-06), and antifragility (NFR-05) — a region is RO when a probe says so, and a Failed write-to-RO stays in the ledger.
Incompatible if absolute. A kernel must mutate heap, page tables, device MMIO, and task state. “Immutable OS” as marketing is a status inflation. Do not write it.
Hub: overview.md. Site SoT: Advantages — Immutability. Pillars: pillars.md. Threat model: security.md.
Plan issues (open on 2026-09-11): Track A #31 / in-repo track-a.md, Track B #40 / track-b.md. A9: #48. Tracks are subordinate to principles.md.
Already practiced (probed)
These are scoped cuts on QEMU virt. They are not an immutable kernel.
| Scope | Probe | ADR |
|---|---|---|
Identity .text/.rodata RO+X; .data/heap RW+NX | ro: nx data / ro: write fault / ro: ok | ADR-015 |
SCTLR.WXN on | same RO+NX smoke | ADR-015 |
Live identity .text torn after high-VA jump + vtable rewrite | ident: reloc / ident: live / ident: ok | ADR-020 |
Heap, PTEs, UART/GIC, and coop stacks stay writable on purpose. Identity .rodata / .data / heap tears are ledger-Verified (ADR-025 / ADR-037 / ADR-038). Leftover identity RAM after the heap is torn (ident: ram*, ADR-049); smoke rejects ident: ram-stay. _start stays mapped (ident: start-stay, ADR-047).
Track A — RO app payloads, then A9 slot disconnect
The goal of this stance (sponsor clarification) is to disconnect OS updates from apps: a separate OS image vs app payloads. Update/replace the kernel without rebuilding apps, and the reverse. That is A9 #48. It is not an “immutable OS” product sentence.
Track A #31 landed A1–A8 first. A1 is a kernel SVC ABI mile (syscall.md). A2 is a libctos CRT mile (ADR-022). A3 is a guest ELF64 PT_LOAD loader (ADR-023). A4 is standing EL0 as normal mode for that loaded image (ADR-024). RO app payloads are that loader mapping an image RO+X (R-only PT_LOAD already uses map_el0_ro; RX text stays Exec). A9 is the slot first cut (ADR-030), not an absolute immutable OS.
Today (Verified freestanding hosting under ADR-048/052): two host artifacts (kernel ELF + target/hello-libctos.elf) and a FAT16 /hello load path. A2–A4 load that file (no production include_bytes!). Cross-update is the host two-boot probe on this OS and ba6541c (ADR-032). Product claim: ADR-048 / ADR-052. Do not say OTA / “immutable OS” marketing / Linux hosting. Performance: perf: app-load plus probe-only perf: embed-load / perf: slot-delta are a QEMU TCG measurement pair (ADR-046, performance.md) — not a percent and not “slots are free.”
Track B #40 must not use Linux-compat research to claim an immutable or container host. Frame: ADR-031 (not claiming Linux userspace yet). Containers stay a non-goal (ADR-029).
Claim gate (ADR-style, no new ADR here)
Same rule as ADR-001 / ADR-011:
- Name the scope (which pages, which image).
- Cite a probe (serial +
#[test_case], or source absence). - Only then Verified. Unprobed stays Unknown. Future work stays Planned.
- Do not say “immutable OS,” “W^X kernel” as a product sentence, or “secure because RO.”
File presence of this note is not that probe. See the honesty ledger.
Docs website + DNS
The published site is this mdBook (book.toml, src = "docs"). The website and the repo are the same markdown. Visitor-facing source of truth is the landing plus docs/overview/*. docs/framework/* is for deep links (ledger, pillars, threat model). Do not keep a second marketing copy.
Required site chapters (sidebar + landing). A missing file fails mdbook build (create-missing = false) and scripts/docs-build.sh:
| Website page | Source file |
|---|---|
| What can run today | docs/overview/what-can-run.md |
| Building or porting | docs/overview/porting.md |
| Filesystem (memfs + FAT16) | docs/overview/filesystem.md |
| Hosting apps / containers | docs/overview/hosting-apps.md (guest OCI/Docker: non-goal, ADR-029) |
| KPIs / how we measure | docs/overview/measure.md |
| Prerequisites | docs/overview/prerequisites.md |
| Advantages | docs/overview/advantages.md |
| Drawbacks / limits | docs/overview/limits.md |
Frozen IDs touched by the publish path: document-first (FR-14), honesty (NFR-06), written acceptance matches what is proven (NFR-13). No new FR/NFR IDs.
Diagrams use mermaid fences. mdbook-mermaid 0.17.1 wraps them; docs/mermaid-init.js loads mermaid 11.6.0 from jsDelivr in the browser. A local mdbook build without mdbook-mermaid on PATH fails (book.toml lists the preprocessor). Use ./scripts/docs-build.sh.
Local build
You need mdBook 0.5.4 and mdbook-mermaid 0.17.1 (pinned in .github/workflows/pages.yml and scripts/docs-build.sh).
./scripts/docs-build.sh # downloads both binaries if missing, then `mdbook build`
# or, if both are already on PATH:
mdbook build # writes ./book/
mdbook serve # http://localhost:3000
Auto-install keys on uname -s:uname -m. Linux stays x86_64-unknown-linux-gnu / aarch64-unknown-linux-musl. macOS uses the published x86_64-apple-darwin / aarch64-apple-darwin tarballs for mdBook and mdbook-mermaid. Other hosts: install both pins yourself. mdbook --version prints mdbook v0.5.4; the script strips the leading v before comparing to pin 0.5.4.
create-missing is off: a SUMMARY.md link to a missing file fails the build. Output directory book/ is gitignored.
A successful local mdbook build is a generator probe. It is not a new “the website is published” claim. The live URL is a separate ledger row.
Mobile / narrow viewport
Sponsor bar (same as architecture.artof.link): viewport, readable type, usable nav on a phone, no page-wide horizontal overflow. Tables, pre, and Mermaid may scroll inside their widget.
Stock mdBook 0.5.4 already has (audited on the live HTML 2026-09-12):
<meta name="viewport" content="width=device-width, initial-scale=1">- Sidebar toggle (
#mdbook-sidebar-toggle) plus--sidebar-width: min(300px, 80vw) #mdbook-menu-bar { flex-wrap: wrap }.table-wrapper { overflow-x: auto }around markdown tables.content img { max-width: 100% }
Gaps closed in docs/theme.css + mermaid extras (this tree):
pre/ code:overflow-x: autoso long lines do not widen the page- Table cells:
overflow-wrap; wide tables keep amin-widthand scroll - Touch: 44px minimum on sidebar links, menu icons, and mobile chapter arrows (≤768px)
- Body type stays 16px; heading strings may wrap
- Mermaid: on ≤768px,
useMaxWidthis off so diagrams stay readable and the.mermaidbox scrolls (docs/mermaid.css,docs/mermaid-init.js)
Desktop content column stays --content-max-width: 750px. Do not treat this section as a live-site deploy probe.
Narrow-viewport probe
After ./scripts/docs-build.sh, ./scripts/docs-mobile-probe.sh serves book/ and uses headless Chrome. An iframe forces the chapter width so --dump-dom cannot fake a wider layout. It records whether documentElement.scrollWidth exceeds clientWidth on the landing, the honesty ledger, and this page.
2026-09-12 this cloud VM (Google Chrome 148.0.7778.96): landing overflow=no at 320 / 768 / 1280 (client = scroll); ledger and this page overflow=no at 320. Internal .table-wrapper / pre / .mermaid scroll is allowed. The numeric probe maps cdn.jsdelivr.net to 127.0.0.1 so mermaid.js cannot hang Chrome (diagrams stay as source text for that run). A green local probe is not “https://ctos.artof.link is mobile-verified” until a post-merge Pages fetch repeats it.
Production URLs
| URL | Role | Status word |
|---|---|---|
https://ctos.artof.link | Custom domain | Verified. HTTPS 200, cert for this name, landing includes Driving principles. Main deploy 34653046584 after #30. Pages API: cname=ctos.artof.link, https_enforced=true, cert approved (expires 2026-12-10). |
https://artofdream.github.io/ctos | Project-site path (no trailing slash) | Redirect (301) to the custom domain on the 2026-09-11 probe. A trailing-slash .../ctos/ 404’d. Do not treat github.io as a second live tree. |
Do not write Verified on a new deploy from file presence alone. Re-fetch after the next main Pages run if you change the claim.
GitHub Pages workflow
.github/workflows/pages.yml:
- Pull requests:
mdbook build+ check that the outputCNAMEisctos.artof.link. Does not deploy. - Push to
main: build, uploadactions/upload-pages-artifact, deploy withactions/deploy-pages.
Settings already in use after #30: Source = GitHub Actions, custom domain ctos.artof.link, Enforce HTTPS on. Do not flip Source away from Actions. A CNAME file in the artifact (from book.toml cname = "ctos.artof.link", also the repo-root CNAME) should stay in lockstep.
See GitHub: publishing with Actions.
DNS — Amazon Route 53 (CNAME in place)
artof.link is hosted on Amazon Route 53. The sponsor states the ctos CNAME is already created. Do not run CREATE again.
| Field | Value |
|---|---|
| AWS account | <aws-account-id> |
| Region for CLI | us-east-1 (Route 53 API is global) |
| Hosted zone name | artof.link |
| Hosted zone ID | <route53-zone-id> (sponsor-stated) |
| Record | CNAME ctos.artof.link. → artofdream.github.io. (trailing dot) |
This is a project site on a subdomain. Do not CNAME to artofdream.github.io/ctos. Do not touch the artof.link apex.
Public dig CNAME ctos.artof.link returns artofdream.github.io. DNS is in place at the resolver. That is not a Route 53 API list-resource-record-sets from this environment (no AWS CLI / credentials here). Next session may LIST zone <route53-zone-id> to confirm; skip change-resource-record-sets unless the row is missing.
export AWS_REGION=us-east-1
aws sts get-caller-identity --query Account --output text # expect "$AWS_ACCOUNT_ID" (set locally, not in repo)
aws route53 list-resource-record-sets --hosted-zone-id "${ROUTE53_ZONE_ID:?set ROUTE53_ZONE_ID locally}" \
--query "ResourceRecordSets[?Name=='ctos.artof.link.']"
scripts/route53-ctos-cname.json is a leftover CREATE batch for recovery only.
Probes
dig CNAME ctos.artof.link +short # expect artofdream.github.io.
curl -sSI https://ctos.artof.link # expect HTTP 200 (Verified 2026-09-11)
Honesty
| Claim | Probe | Until then |
|---|---|---|
| mdBook builds this tree | ./scripts/docs-build.sh (or mdbook build with mermaid preprocessor) exit 0 | — |
| Narrow viewport / mobile layout | ./scripts/docs-mobile-probe.sh after a local build (320 / 768 / 1280; landing + ledger + this page) | Verified locally 2026-09-12 (iframe overflow=no). Not a live-site mobile claim until a post-merge Pages fetch. |
| Pages workflow exists | Read .github/workflows/pages.yml | File presence only |
| Pages workflow builds a PR | Green pages run on this branch (build job; deploy skipped) | See honesty ledger |
| Docs website published | Green pages workflow on main and HTTPS fetch of https://ctos.artof.link | Verified — deploy 34653046584 + HTTPS 200 + Driving principles |
Public CNAME ctos.artof.link | dig CNAME ctos.artof.link +short | Verified — DNS in place (artofdream.github.io.) |
Route 53 API row in <aws-account-id> / <route53-zone-id> | list-resource-record-sets as that account | Unknown here (no AWS CLI). Sponsor states CREATE already done. Do not CREATE again. |
| Custom domain reachability | Pages lists the hostname and HTTPS 200 with a matching cert | Verified — Pages API cname + https_enforced + cert approved; curl -sSI https://ctos.artof.link HTTP 200 |
Do not say “secure OS,” or that QEMU boot was proven by this docs PR. Say “EL0 isolated” only as the accepted ADR-091 sentence, with its limits.
Second brain
Session memory lives in git under research/, not on a wiki. This page is a curated landing for the published book. Daily briefs and scratch notes stay on GitHub so the site does not pretend every session log is product documentation.
Published Track B research in the book (not session scratch): Linux AArch64 syscall gap (B2) (inspection only), ADR-035 (B3 process-model stance), ADR-033 (B4 ELF / auxv / PT_INTERP gap), and ADR-034 (B5 VFS concept compare). Not claiming Linux userspace. Not claiming dynamic Linux ELF. Not claiming a Linux filesystem.
| Vault | Where | Job |
|---|---|---|
| Procedure | .cursor/skills/ctos-*/ | How to do a repeatable job |
| Correction | .cursor/rules/ + hard constraints in AGENTS.md | Mistakes we must not repeat |
| Relationship | Docs cross-links (vision → architecture → ADR → roadmap → ledger) | How pieces connect |
| Daily Brief | research/daily-briefs/ | Handoff: where we stopped |
Session scratch: research/random-thoughts/. Do not treat scratch as the honesty ledger.
Tracker is GitHub. Optional Obsidian UI is local-only (.obsidian/ is gitignored). See research/README.md on GitHub.
Route 53: CNAME already created (zone <route53-zone-id>). Playbook / LIST-only: research/dns-route53-ctos.md. https://ctos.artof.link HTTPS is Verified after #30 (website.md).