logic @ 94db62486 · security @ c05e43f66
Auto-generated STPA safety model: unsafe states this system can reach and control actions that get it there. Single or multiple lenses: logic (mission failure), security (deliberate attack), governance (data safety).
<details>
<summary><b>How to read this</b>: STPA primer and diagram legend</summary>
STPA (System-Theoretic Process Analysis, Leveson) treats the system as controllers issuing control actions to controlled processes, with feedback flowing back. Instead of "what component fails," it asks "what control action, given or withheld at the wrong time, drives the system into an unsafe state?" Unsafe means a violation of this system's reason to exist. Multiple lenses (logic, security, governance) analyze the same control structure for different safety questions: mission failure, deliberate attack, data-processing violations. A finding appearing in multiple lenses is expected and correct.
Read top-down: Losses are outcomes we must prevent; Hazards are states leading to loss; diagrams show control structure (solid = control action, dashed = feedback); Unsafe Control Actions table is the core; Unsafe Feedback covers data channels (absent, stale, corrupted, spoofed). Every claim cites path:line; unbuilt elements are marked. Semantic stable IDs mean regenerating changes only findings that changed.
Control structure
Logical view
flowchart TD
subgraph control["control"]
session-manager["Serialized session lifecycle and recovery admission"]
end
subgraph control_plane["control_plane"]
dispatcher["Admission, quota gate, placement, dispatch"]
node-registry["Dial-home registry + reconcile inventory"]
s3-warmth-gc["Artifact TTL sweep (abort on partial listing)"]
stateful-manager["Stateful wake/bank/blessing/quarantine lifecycle"]
end
subgraph ingress["ingress"]
api["HTTP API + TokenReview auth"]
end
subgraph store["store"]
op-log["Durable ordered journal (book of record)"]
end
api -- "api.admit_task" --> dispatcher
caller -- "api.read-historical-cessation" --> api
api -- "api.recovery_fence" --> session-manager
dispatcher -- "dispatcher.quota_gate" --> op-log
session-manager -- "session-manager.commit_recovery_fence" --> op-log
stateful-manager -- "stateful-manager.bless_generation" --> op-log
stateful-manager -- "stateful-manager.destroy_orphan" --> node-registry
node-registry -. "reported primed pool inventory (adopt_inventory)" .-> dispatcher
node-registry -. "reported stateful VM inventory (dial-home NodeStatus), authoritative per invariant 5" .-> stateful-manager
op-log -. "Complete retained positive parking/eviction history and absence of surviving successor/restore authority" .-> api
op-log -. "Fresh durable restore authority: recovery-fenced, unknown lineage, or retained unfenced holder; read errors deny admission" .-> session-manager
Physical view
flowchart TD
subgraph compute["compute"]
egress-proxy["Brick-local egress credential proxy"]
guest-vm["Firecracker guest VM"]
node-envoy["node Envoy serving relay (DNAT)"]
noded["noded brick daemon (privileged, /dev/kvm)"]
shotter-chromium["Headless Chromium (content-driven fetches)"]
shotter-guest-init["Shotter guest-init (PID 1): launches Chromium, hosts the proxy, serves /shim/ready"]
shotter-proxy["In-guest egress allowlist proxy (image-baked policy)"]
end
subgraph control_plane["control_plane"]
control-plane["Elixir control plane pod (1 replica, Recreate)"]
end
subgraph data["data"]
s3-store["S3-compatible object store (SeaweedFS gateway)"]
end
subgraph external["external"]
caller["External task/session/serving caller"]
external-host["External destination (e.g. GitHub API)"]
end
subgraph secret_delivery["secret_delivery"]
egress-ca-volume["Optional projected egress CA Secret"]
end
control-plane -- "control-plane.grpc_command" --> noded
egress-proxy -- "egress-proxy.inject_credential" --> external-host
egress-proxy -- "egress-proxy.load_initial_ca" --> egress-proxy
node-envoy -- "node-envoy.dnat_route" --> guest-vm
noded -- "noded.artifact_verb" --> s3-store
noded -- "noded.refuse_if_silenced" --> noded
noded -- "noded.vsock_dispatch" --> guest-vm
shotter-chromium -- "shotter-chromium.fetch_subresource" --> shotter-proxy
shotter-proxy -- "shotter-proxy.forward_allowed" --> egress-proxy
control-plane -. "contact signal behind the silence gate: a 2xx dial-home Register response and every successful WatchNode NodeStatus send refresh lastContact, which refuseIfSilenced compares against the 21600s bound" .-> noded
egress-ca-volume -. "Configured certificate and key files from the optional Secret; missing, partial or malformed pairs fail loading" .-> egress-proxy
noded -. "dial-home registration {node, pod_uid, address, boot_id} + NodeStatus" .-> control-plane
s3-store -. "fetched artifact bytes on RestoreArtifact" .-> noded
shotter-guest-init -. "GET /shim/ready 200 (CDP /json/version answered and one real trial capture produced non-empty PNG bytes)" .-> noded
Control actions
| ID |
View |
Control action |
Controller → Process |
Maturity |
Evidence |
api.admit_task |
logical |
Admit task workload |
api → dispatcher |
built |
projects/embervm/control/lib/embervm/router.ex:33 |
api.read-historical-cessation |
logical |
Read exact bounded original-dispatch cessation proof without restoring execution |
caller → api |
built |
projects/embervm/control/lib/embervm/router.ex |
api.recovery_fence |
logical |
Seal exact verified terminal lineage against restoration |
api → session-manager |
built |
projects/embervm/control/lib/embervm/router.ex |
control-plane.grpc_command |
physical |
gRPC lifecycle command (BuildBase, Prime, Assign, Destroy, DeleteVolume, RestoreArtifact) |
control-plane → noded |
built |
projects/embervm/proto/embervm/node/v1/node.proto:57 |
dispatcher.quota_gate |
logical |
Gate quota on admission |
dispatcher → op-log |
built |
projects/embervm/control/lib/embervm/dispatcher.ex:258 |
egress-proxy.inject_credential |
physical |
Inject credential into request |
egress-proxy → external-host |
built |
projects/embervm/ARCHITECTURE.md:710 |
egress-proxy.load_initial_ca |
physical |
Retry initial CA load and retain the first successful keypair |
egress-proxy → egress-proxy |
built |
projects/firecracker/substrate/egress-proxy/cmd/swap.go:801-839 |
node-envoy.dnat_route |
physical |
DNAT route to guest |
node-envoy → guest-vm |
built |
projects/embervm/ARCHITECTURE.md:193 |
noded.artifact_verb |
physical |
Access artifact in store (get, put, delete, list, tag) |
noded → s3-store |
built |
projects/embervm/proto/embervm/node/v1/node.proto:463 |
noded.refuse_if_silenced |
physical |
Refuse local authority if control-plane contact stale |
noded → noded |
built |
projects/embervm/noded/server/server.go:619 |
noded.vsock_dispatch |
physical |
vsock dispatch to guest |
noded → guest-vm |
built |
projects/embervm/ARCHITECTURE.md:234 |
session-manager.commit_recovery_fence |
logical |
Compare-and-set immutable recovery fence and journal audit |
session-manager → op-log |
built |
projects/embervm/control/lib/embervm/session_recovery_fence.ex |
shotter-chromium.fetch_subresource |
physical |
Fetch subresource from rendered page |
shotter-chromium → shotter-proxy |
built |
projects/embervm/runtimes/shotter/guest-init/cmd/main.go:168 |
shotter-proxy.forward_allowed |
physical |
Forward request to allowlisted destination |
shotter-proxy → egress-proxy |
built |
projects/embervm/runtimes/shotter/guest-init/cmd/proxy.go:391-405 |
stateful-manager.bless_generation |
logical |
Issue blessed generation |
stateful-manager → op-log |
built |
projects/embervm/control/lib/embervm/stateful_manager.ex:676 |
stateful-manager.destroy_orphan |
logical |
Destroy orphaned VM |
stateful-manager → node-registry |
built |
projects/embervm/control/lib/embervm/stateful_manager.ex:2027 |
Lens: logic
Scope. EmberVM is a Firecracker microVM platform: an Elixir control plane admits and dispatches task, session, serving, and stateful workloads onto brick pods, each run by a privileged Go noded daemon that supervises guest VMs over vsock and dial-homes to register with the control plane.
Maturity detail
- Built: task/session/serving/stateful/composite lifecycle managers, admission quota gate (fail-closed per-principal when a budget is configured), generation blessing (CP-issued pre-dispatch and checkpoint-abort self-heal via durable checkpoint_dispatched), delegated generation leases (BlessingLease, bounded and monotonic), orphan-destroy reconcile with an ACTIVATOR-origin adoption guard that runs before the destroy pass, S3 warmth GC that aborts the whole sweep on a partial listing, dial-home registration, brick-local egress credential injection scoped by egressTo, the shotter task-class guest (ADR embervm/035): a warm headless Chromium snapshot base behind an in-guest, image-baked, hard-allowlisted egress proxy, whose readiness is now gated on one real trial capture before /shim/ready flips, the noded gRPC control channel, authenticated by a static bearer token enabled in production (#4693); the inert Cilium ingress policy was removed in #5816 and supplies no network isolation, SigV4-authenticated access to the S3 artifact store enforced by the gateway (enableAuth: true), enabled in production (#4708); the surviving embervm identity is one credential shared by every brick and the control plane, the brick silence timeout (ADR embervm/037) bounding node-local authority (activator wakes, group wakes, blessing-lease self-advance) once control-plane contact goes stale, armed at 21600s in production and dev, the warmth ownership transition guard (#4962): every brick refreshes a .alive claim per warmth segment and startup GC reaps only unclaimed pre-heartbeat directories or claims older than WarmthStaleAfter, principal-artifact envelope encryption and capability-gated restore in noded, reader path unconditional, writer and enforcement armed in production and dev as of 2026-08 (ADR embervm/033, #4691; status updated 2026-08-23 (#5233))
- Designed-only: mTLS/SPIFFE certificate-based mutual transport auth for noded (#4693 delivered bearer-token authentication; the inert policy was removed in #5816), node-local activator for stateful/composite wake beyond cold boot (partially landed, stateful activator currently only cold-boots), audience-scoped guest token, request-scoped GitHub tool mediation replacing host-keyed injection (ADR 055)
- Note: Focused egress CA startup-recovery review for #7187; live recovery remains unverified. Policy-removal corrections retain the #5816 review and unrelated findings retain their earlier evidence. Focused native AGENTS.md context-loading review against a37f9ce with the accompanying runtime diff; this changes worker instructions without changing the modeled platform control actions. Focused historical cessation proof against 434907e17 with this diff: bounded read-only journal snapshot, explicit original identity, complete creation coverage and serialized current ownership checks. Proof assumes the hardcoded one-replica Recreate authority; live recovery is not claimed. Focused retained-terminal recovery fencing against 94db62486 with this diff: exact interrupted dispatch and positive parked history, fresh complete successor checks, immutable CAS audit and durable restore exclusion; original result and accounting stay unknown.
Losses
| ID |
Loss |
L.integrity-loss |
Stored or served state (volume, snapshot, generation, artifact) diverges from truth and is acted on as correct |
L.liveness-loss |
The fleet, a brick, or a workload becomes unable to admit or serve legitimate work |
L.provenance-loss |
State, credentials, or execution history become attributed to the wrong principal, brick, or generation |
L.secret-exposure |
Credential or key material becomes reachable by an unauthorized principal or guest |
L.silent-incorrectness |
A caller receives a served or resumed result derived from stale, forged, or wrong state with no signal |
L.unauthorized-access |
An actor obtains a capability, credential reach, or execution path beyond its principal or role |
Hazards
| ID |
View |
Hazard (unsafe state) |
→ Losses |
Maturity |
egress-ca-startup-latch |
physical |
CA-dependent guest bootstrap and credential injection remain unavailable after a delayed optional Secret supplies a usable keypair because the sidecar retains its initial CA-load failure |
L.liveness-loss |
built |
egress-workload-derivation-gap |
physical |
the per-workload egress forwarder allowlist noded receives is a hand-written if-chain over named workloads in the chart template, not a true derivation, so a future task/session/stateful workload that enables egress but is not added to this chain gets no forwarder and dial-times-out inside the guest instead of a loud, diagnosable deny |
L.liveness-loss |
built |
host-keyed-credential-overreach |
physical |
host-keyed egress credential injection authorizes by destination host only, so a prompt-injected or compromised guest can shape any request to an allowlisted host and have the credential attached to it |
L.secret-exposure, L.unauthorized-access |
built |
identity-hijack |
physical |
an actor holding the shared noded ServiceAccount token can re-register an existing brick's (node, pod_uid) at an address it controls and become the dial-home source the control plane treats as authoritative for that brick |
L.unauthorized-access, L.integrity-loss, L.provenance-loss |
built |
shared-egress-allowlist-widening |
physical |
egress.internal.allowlist is global to the shared sidecar rather than scoped per workload, so shotter's two new frontend destinations become reachable by every other egress-enabled workload (today the claude runtime, later pi if granted egress) with no per-workload authorization check |
L.unauthorized-access |
built |
shotter-broken-base-snapshot |
physical |
shotter's base snapshot is cut when /shim/ready first returns 200; readiness now proves more than protocol liveness because guest-init must answer CDP /json/version and then complete one real trial capture (navigate about:blank, screenshot, non-empty PNG bytes within 15s) before flipping ready, so a globally broken renderer or GL stack fails loudly at warm-up instead of baking into every clone, but the trial exercises only about:blank, so a page-specific rendering failure can still be baked into the base and fail per-invocation instead of failing once, loudly, at BuildBase |
L.liveness-loss, L.silent-incorrectness |
built |
shotter-memmib-tmpfs-coupling |
physical |
the shotter workload's memMib budget and its guest-init tmpfs size are a YAML integer and a Go string literal in different directories with nothing in the build enforcing their relation, so raising either alone silently trades a legible ENOSPC on /tmp for an opaque guest OOM kill under load, or leaves /tmp undersized without freeing any more memMib |
L.liveness-loss, L.silent-incorrectness |
built |
shotter-policy-drift |
physical |
the guest's baked destination policy (/etc/shotter-egress.json, a checked-in file copied verbatim into the image) and the sidecar's chart-driven egress.internal.allowlist are two independently hand-maintained sources of egress truth with no build-time check that they agree, so they can diverge in either direction: the guest could deny a destination the chart already permits (silent functional failure), or a future image rebuild could widen the baked allowlist beyond what any ADR or chart review anticipated |
L.unauthorized-access, L.silent-incorrectness |
built |
store-credential-unscoped |
physical |
the S3 gateway now requires a valid SigV4-signed identity, but the embervm identity's credential is one secret rendered into every noded pod and the control plane, with bucket-wide read/write/list/tag reach over both the embervm and embervm-dev buckets; restore authorization is still storage-ACL-only, not scoped by (principal, lineage, brick, workload, generation, lease), so any brick holding the shared credential can write, substitute, or delete another principal's artifacts, bypassing noded's per-request artifact-verb checks entirely |
L.integrity-loss, L.silent-incorrectness |
built |
unmodeled-checkpoint-abort |
logical |
the interruptible-bank checkpoint commit/abort protocol (generation-advance-then-delete-temp-then-resume ordering, resolve-timeout auto-abort, blessed-vs-self-bump discrimination) governs whether a resumed VM's generation pairing stays trustworthy, but is verified only by code comments, unlike the bank/relight pairing invariant it sits underneath |
L.integrity-loss, L.silent-incorrectness |
built |
Unsafe control actions
| ID |
View |
Control action |
Guideword |
Unsafe condition |
Severity |
→ Hazards |
Evidence |
egress-proxy.inject_credential.providing |
physical |
egress-proxy.inject_credential |
providing |
the proxy injects a real credential into any request whose destination host is allowlisted, regardless of what the guest-originated request actually asks that host to do, so a prompt-injected guest can direct the credentialed call |
medium |
host-keyed-credential-overreach |
projects/embervm/ARCHITECTURE.md:745 |
egress-proxy.load_initial_ca.not-providing |
physical |
egress-proxy.load_initial_ca |
not-providing |
withholding a new load after the initial failure outlives Secret publication; the repair retries on later CA-dependent requests after a five-second cooldown under a mutex and retains the first successful keypair, with live recovery still pending under #7187 |
high |
egress-ca-startup-latch |
projects/firecracker/substrate/egress-proxy/cmd/swap.go:801-839; projects/firecracker/substrate/egress-proxy/cmd/main.go:299-310,414-428; #7187 |
Unsafe feedback
Data channels (absent, stale, corrupted, spoofed) whose failure drives a controller into a hazard.
| ID |
View |
Channel |
Guideword |
Unsafe condition |
Severity |
→ Hazards |
Evidence |
dial-home.unauthorized-source |
physical |
noded → control-plane: dial-home registration {node, pod_uid, address, boot_id} + NodeStatus |
unauthorized-source |
re-registering an existing (node, pod_uid) at a different address is accepted unconditionally and expires the prior instance, so identity is self-asserted under a ServiceAccount shared by every brick rather than bound to the actual brick |
high |
identity-hijack |
projects/embervm/control/lib/embervm/node_registry.ex:1320 |
shotter-readiness.corrupted |
physical |
shotter-guest-init → noded: GET /shim/ready 200 (CDP /json/version answered and one real trial capture produced non-empty PNG bytes) |
corrupted |
readiness no longer gates on protocol liveness alone: guest-init runs one real trial capture (navigate about:blank over CDP, screenshot, non-empty PNG bytes required within 15s) before /shim/ready flips, so a globally non-rendering browser now fails BuildBase loudly; the residual staleness is that the trial page is static about:blank, so readiness still cannot prove rendering of real fetched pages, and a page-specific break baked into the base surfaces per invocation instead of once at BuildBase |
low |
shotter-broken-base-snapshot |
projects/embervm/runtimes/shotter/guest-init/cmd/main.go:127-139,183-206 |
warmth-fetch.unauthorized-source |
physical |
s3-store → noded: fetched artifact bytes on RestoreArtifact |
unauthorized-source |
the gateway now enforces SigV4 (enableAuth: true) and rejects requests signed by no identity, but the embervm identity's access key is one credential shared by every brick and the control plane rather than scoped per principal, lineage, brick, workload, or generation, so an object noded restores as warmth may have been written by any brick holding that shared credential, not necessarily the legitimate owner of the lineage being restored; the finer per-principal binding (envelope encryption plus capability-gated restore) is built and enforced in production and dev (#5233 status update) |
medium |
store-credential-unscoped |
projects/embervm/chart/templates/_noded-pod.tpl:353 |
<details>
<summary><b>Not UCAs</b>: 19 examined and rejected</summary>
- a fleet-wide freeze of new session/serving/stateful placement during a genuine control-plane outage longer than the 6h brick silence bound (ADR embervm/037): a deliberate, accepted trade-off (ADR embervm/037 consequences): the gate only refuses NEW work, never banks or destroys anything, so live VMs keep running and held warmth stays intact; normal service resumes immediately once contact returns (noded/server/server.go:608-625), and 6h was sized specifically to exceed a routine control-plane roll
- a leaked Chromium CDP target accumulating across shotter invocations: each invocation restores its own CoW clone that is destroyed after the response, so a leak cannot outlive one clone; within a clone, closeCDPTarget runs on every return path via defer immediately after target creation, including navigation and capture errors (projects/embervm/chart/templates/workload-shotter.yaml, guest-init/cmd/cdp.go:467-481)
- a pruned session projection authorizing recovery solely because GET returns404: The owner-scoped verifier requires positive original parking/parked/evicted history above compaction, complete successor coverage, no surviving lineage or dispatch-capable restore authority, and fails closed on missing or bounded-out evidence. An orphan Prime after manager death cannot register, Assign or invoke; no zero-orphan-resource claim is made (projects/embervm/control/lib/embervm/session_cessation_evidence.ex; projects/embervm/control/lib/embervm/session_manager.ex; projects/embervm/control/test/embervm/session_manager_test.exs).
- a redirect or subresource on a mapped page escaping the shotter-proxy allowlist: the proxy is the only egress path (--proxy-server with loopback-only bypass, no direct-network fallback), so a redirect or a subresource fetch to an unmapped host issues a fresh CONNECT/absolute request through the same proxy, is re-parsed and refused per connection by handleConnection, and is checked by the same resolve() call as any other request, never trusted because it originated from an already-allowed page (projects/embervm/runtimes/shotter/guest-init/cmd/main.go:168-175, guest-init/cmd/proxy.go:293-313,391-405)
- a restored clone resuming a page or CDP target left over from a prior invocation: createCDPTarget opens a fresh about:blank target per invocation specifically so an hours-later restore never inherits a stale page or target (projects/embervm/runtimes/shotter/guest-init/cmd/cdp.go:46-47)
- a retained terminal holder restoring after an accepted recovery proof or uncertain fence response: Preview has no recovery authority. Exact owner-scoped apply repeats bounded journal, projection, successor and pending-work checks under manager serialization, then commits only an immutable recovery fence and audit. Every restoring create reads the durable fence; missing durable holders and read errors refuse, including stale ETS after pruning. Restart/replay tests preserve old invocation data and prove duplicate applies do not append again (projects/embervm/control/lib/embervm/session_recovery_fence.ex; projects/embervm/control/lib/embervm/session_manager.ex; projects/embervm/control/test/embervm/session_recovery_fence_test.exs).
- a successful initial CA recovery silently widening destination access or rotating running guests onto a new trust anchor: normal destinations pass route and credential resolution before requesting a minter; the reserved host serves only the public CA, and a loaded minter is returned without rereading replacement files for the process lifetime (projects/firecracker/substrate/egress-proxy/cmd/main.go:299-310,344-365,414-428; swap.go:816-828)
- checkpoint-abort auto-bump producing an unblessed generation: the resolve-timeout auto-abort lane (blessedGeneration: 0) is the ONLY case that produces it, and StatefulStore correctly quarantines it as a fail-closed signal rather than treating it as a bug (noded/server/stateful.go:605-610)
- enabling native Pi repository context also enabling arbitrary extension, skill or prompt-template discovery: Pi loads checkout guidance from its workspace while retaining --no-extensions, --no-skills and --no-prompt-templates. The factory role prompt still passes through compose_system_prompt and only image-owned extensions are explicitly loaded; the spawn regression asserts these arguments together (projects/embervm/runtimes/claude/shim.py:5779; projects/embervm/runtimes/claude/shim_test.py:9389).
- no per-principal daily budget configured in the reference deployment: spend is still bounded by admission caps and concurrency, not unbounded; cutoff is an admission action by design (invariant 4), and quota fails closed the moment a budget IS set (control/lib/embervm/dispatcher.ex:265-269, deploy/values.yaml)
- node-local activator on stateful defaults BlessedGeneration to 0 during control-plane absence: a zero generation fails pairing and forces a cold boot rather than an incorrect relight, matching invariant 4's fail-open-to-cold-boot rule (noded/server/stateful_activator.go:361)
- orphan-destroy racing a live node-woken (ACTIVATOR-origin) stateful VM: adopt_activator_stateful_vms runs on the same reconcile pass before the orphan-destroy loop, and the loop explicitly skips activator_origin? vms as belt-and-suspenders (control/lib/embervm/stateful_manager.ex:2031-2038)
- retaining an optional egress CA volume during first certificate issuance: the optional volume is retained because a required Secret previously blocked the brick rollout while cert-manager issuance waited in the same ArgoCD sync; bounded request-driven initial-load recovery removes the permanent startup latch without restoring that dependency cycle (projects/embervm/chart/templates/_noded-pod.tpl:957-975; projects/embervm/ARCHITECTURE.md:1789-1802)
- shotter-proxy refusing every destination when /etc/shotter-egress.json is missing or malformed: LoadProxyConfig returns a zero-value config on any read or parse error, and the zero value's nil maps make resolve() refuse everything; the failure mode is fail-closed and surfaces as a bounded per-request refusal, not a silent bypass (projects/embervm/runtimes/shotter/guest-init/cmd/proxy.go:78-99)
- shotter-proxy.forward_allowed checked against the requested top-level host instead of the actually-dialled destination: resolve() applies the host mapping first and validates the resulting mapped destination against the allowlist for both CONNECT targets and Host-header-derived absolute-form requests, so the value checked and the value dialled are always the same one; a request for an unmapped or mismatched host fails resolve() before any vsock dial (projects/embervm/runtimes/shotter/guest-init/cmd/proxy.go:211-247)
- temporary CA-dependent failure while an optional egress CA Secret is missing, partial, malformed or inside the initial-load retry cooldown: the loader returns no minter until the configured certificate and matching key parse successfully, permits another request-driven attempt after five seconds, and shares the first successful minter across concurrent callers; the guest CA response contains only the public certificate and plaintext injection remains independent (projects/firecracker/substrate/egress-proxy/cmd/swap.go:816-828,855-880,925-928; main.go:299-310,401-428)
- the S3 artifact-store gateway accepting anonymous, unsigned requests (formerly hazard anonymous-store-access): closed in production: SeaweedFS S3 auth is enabled (enableAuth: true, projects/platform/seaweedfs/values.yaml:181) with the embervm identity policy (s3-identities.json), so an unsigned request is now rejected at the gateway; the residual gap, that the surviving identity is shared fleet-wide rather than scoped per principal, is tracked separately as hazard store-credential-unscoped (#4708, closed)
- the brick silence gate firing while the control plane is actually alive, wrongly refusing activator wakes, group wakes, or blessing-lease self-advancement (ADR embervm/037 wrong-timing): rejected: lastContact advances only on authenticated control-plane contact through two independent channels, a 2xx dial-home POST on the roughly 30s jittered register loop (noded/server/register.go:96-98,198-203) and every successful WatchNode NodeStatus send on the 2s liveness interval (noded/server/server.go:70,1801), compared with time.Now's monotonic reading so an NTP step cannot arm or disarm the gate (noded/server/server.go:595-597); tripping it requires both channels to fail simultaneously for the full 21600s bound (deploy/values.yaml:195), which is itself the brick partition the gate exists to catch, and even a false trip is bounded refusal of NEW work with live VMs, banks, and held warmth untouched until contact returns
- the noded gRPC control channel accepting BuildBase/Prime/Assign/Destroy/DeleteVolume/RestoreArtifact from any pod-network caller (formerly hazard open-node-control-channel): authenticated in production: EMBERVM_NODED_BEARER_TOKEN gates unaryAuthInterceptor and streamAuthInterceptor (noded/cmd/auth.go:49-75, noded/cmd/main.go:320-326), with bearerTokenSecret.enabled true in deploy/values.yaml (#4693). The inert Cilium policy was removed in #5816; bearer authentication does not provide network-level caller isolation.
Open questions
- Before closing #7187, verify that a running sidecar recovers after delayed egress CA Secret projection and that S8 receives the recovered public CA. Successful loading intentionally does not rotate the CA; expiry, replacement and already-running guest trust remain outside this initial-load repair.
- How should a native-policy successor cover every noded listener, including the stateful/composite activator ranges (5400-5419), after #5816 removed the inert Cilium template? The historical 2026-08-22 omission dropped a cold wake to noded:5401 for 14 minutes; deleting that template supplies no replacement enforcement.
- Whether egress.internal.allowlist should become scoped per egress-enabled workload rather than global to the sidecar (ADR embervm/035 open question 1); shared-egress-allowlist-widening tracks the safety consequence of leaving it global.
- Whether the store-credential-unscoped residual (a shared, bucket-wide S3 identity) should be re-scoped per principal now that the ADR embervm/033 rollout is armed end to end in production and dev (kekRoot, EMBERVM_ARTIFACT_ENCRYPTION, store.encrypt, requireRestoreCapability; #4691, status updated 2026-08-23 (#5233)): what survives of the residual is write/delete reach and key custody, since reads of encrypted warmth yield ciphertext.
Lens: security
Scope. Deliberate attacks on EmberVM's control structure: an adversary who forges, withholds, or replays control actions or feedback. Same control structure as the logic lens, analyzed for attack rather than honest failure.
Maturity detail
- Built: bearer-token authentication on every unary and streaming gRPC call (#4693); the inert Cilium ingress policy was removed in #5816 and supplies no network isolation; SigV4-authenticated access to S3 artifact store enforced by gateway (#4708); brick dial-home registration bound to the noded pod's projected token claim identity (#4707, closed by #5049); per-principal envelope encryption of mutable state and capability-gated restore authorization (#4691, armed in production and dev 2026-08); the brick silence timeout (ADR embervm/037) bounding node-local authority once control-plane contact goes stale, armed at 21600s in production and dev; warmth ownership transition guard (#4962) refreshing .alive claims and reaping unclaimed pre-heartbeat directories; the Firecracker jailer (per-VM chroot, a uid and gid from a per-daemon leased block, a cgroup leaf with memory.oom.group, its own PID namespace), fail-closed at startup and per launch, shipped off until each stage arms it (#5255, #6965)
- Designed-only: mTLS/SPIFFE certificate-based mutual transport auth for noded, request-scoped GitHub tool mediation replacing host-keyed injection (ADR agents/055), digest-verified manifests for restored artifact integrity (ADR embervm/033 decision 3), per-workload egress allowlist scoping (ADR embervm/035)
- Note: Jailer hazard refreshed for #5255; policy-removal corrections for #5816; other findings retain their earlier review stamp.
Losses
| ID |
Loss |
L.capacity-exhaustion |
A malicious tenant or unauthorized actor exhausts shared placement capacity, quota, or concurrency |
L.credential-theft |
Egress credential, store credential, or bearer token is obtained and used by an unauthorized actor |
L.host-compromise |
A brick pod is compromised via a VMM escape or noded vulnerability and holds privileged access, store credential, and bearer token |
L.tenant-data-corruption |
Another tenant's mutable state is written, deleted, or substituted by an unauthorized principal or brick |
L.tenant-data-disclosure |
Another tenant's memory snapshot, session workspace, or state is read by an unauthorized principal |
Hazards
| ID |
View |
Hazard (unsafe state) |
→ Losses |
Maturity |
Status |
Issue |
firecracker-no-jailer |
physical |
in production noded execs the firecracker binary directly, as root, inside a privileged pod; the only per-VM containment added is a mount namespace for vsock isolation. Firecracker's built-in seccomp filter is active, but a VMM escape lands in a process holding /dev/kvm, the fleet-shared store credential, and the noded bearer token, becoming the compromised-brick adversary instantly. The jailer (per-VM chroot, leased uid and gid, cgroup leaf, PID namespace) is built and shipped off: once armed, a brick that cannot contain guests exits before registering and conformance S7 fails on any unjailed launch; embervm-dev arms it first and production after a dev soak (#6965). |
L.host-compromise |
built |
shipped-off |
#5255 |
host-keyed-credential-overreach |
physical |
host-keyed egress credential injection authorizes by destination host only, so a prompt-injected or compromised guest can shape any request to an allowlisted host and have the credential attached to it |
L.credential-theft |
built |
designed |
|
identity-hijack |
physical |
an actor holding the shared noded ServiceAccount token can re-register an existing brick's (node, pod_uid) at an address it controls and become the dial-home source the control plane treats as authoritative for that brick |
L.credential-theft, L.host-compromise |
built |
enforced-prod |
|
shared-egress-allowlist-widening |
physical |
egress.internal.allowlist is global to the shared sidecar rather than scoped per workload, so shotter's two new frontend destinations become reachable by every other egress-enabled workload (today the claude runtime, later pi if granted egress) with no per-workload authorization check |
L.credential-theft |
built |
none |
#5320 |
store-credential-unscoped |
physical |
the S3 gateway now requires a valid SigV4-signed identity, but the embervm identity's credential is one secret rendered into every noded pod and the control plane, with bucket-wide read/write/list/tag reach over both the embervm and embervm-dev buckets; restore authorization is still storage-ACL-only, not scoped by (principal, lineage, brick, workload, generation, lease), so any brick holding the shared credential can write, substitute, or delete another principal's artifacts, bypassing noded's per-request artifact-verb checks entirely |
L.tenant-data-corruption, L.tenant-data-disclosure |
built |
none |
#4691 |
Unsafe control actions
| ID |
View |
Control action |
Guideword |
Unsafe condition |
Severity |
→ Hazards |
Status |
Issue |
Evidence |
api.admit_task.providing |
logical |
api.admit_task |
providing |
a malicious principal submits a workload that references another principal's artifact ref or lineage, bypassing the invariant 3 boundary (no mutable VM or snapshot lineage crosses a principal); the principal-scoped keyspace for the shared prefix remains designed but not implemented |
high |
store-credential-unscoped |
enforced-prod |
#4691 |
projects/embervm/control/lib/embervm/dispatcher.ex:130 |
control-plane.grpc_command.providing |
physical |
control-plane.grpc_command |
providing |
an attacker on the cluster network calls noded's gRPC surface (BuildBase, Assign, Destroy, RestoreArtifact) directly from anywhere on the pod network without a legitimate identity |
high |
identity-hijack |
enforced-prod |
#4693 |
projects/embervm/noded/cmd/auth.go:49-75; bearer authentication remains enabled, while #5816 removed the inert Cilium policy and supplies no network-level caller restriction |
egress-proxy.inject_credential.providing |
physical |
egress-proxy.inject_credential |
providing |
a prompt-injected guest gets a real credential attached to a request it authored, not one the operator intended; the decided direction (request-scoped tool mediation for at least the git credential class) is drafted in ADR agents/055 and not yet the default for every credential |
high |
host-keyed-credential-overreach |
enforced-prod |
|
projects/embervm/ARCHITECTURE.md:745 |
noded.artifact_verb.providing |
physical |
noded.artifact_verb |
providing |
a compromised brick or a credential holder can write, evict, or substitute another principal's artifact; the store credential is still bucket-wide and fleet-shared, so a compromised brick can delete or overwrite any principal's artifacts |
high |
store-credential-unscoped |
enforced-prod |
#4691 |
projects/embervm/chart/templates/_noded-pod.tpl:353 |
Unsafe feedback
Data channels (absent, stale, corrupted, spoofed) whose failure drives a controller into a hazard.
| ID |
View |
Channel |
Guideword |
Unsafe condition |
Severity |
→ Hazards |
Status |
Issue |
Evidence |
dial-home.unauthorized-source |
physical |
noded → control-plane: dial-home registration {node, pod_uid, address, boot_id} + NodeStatus |
unauthorized-source |
an attacker re-registers an existing (node, pod_uid) at a different address using the shared ServiceAccount every brick already holds, attempting to become the dial-home source the control plane treats as authoritative for that brick; the register route now refuses a body whose pod_uid or node does not match the bound projected token's own claims, a token with no pod binding, or an unset noded SA |
high |
identity-hijack |
enforced-prod |
|
projects/embervm/control/lib/embervm/router.ex:909-913 |
warmth-fetch.unauthorized-source |
physical |
s3-store → noded: fetched artifact bytes on RestoreArtifact |
unauthorized-source |
a memory snapshot or artifact noded restores as warmth may have been written by any brick holding that shared credential, not necessarily the legitimate owner of the lineage being restored; the credential itself is still one identity with bucket-wide write, list, and delete reach |
high |
store-credential-unscoped |
enforced-prod |
#4691 |
projects/embervm/chart/templates/_noded-pod.tpl:353 |
<details>
<summary><b>Not UCAs</b>: 1 examined and rejected</summary>
- a guest with no cluster credential accessing the store directly: noded is the sole store client; guests hold no cluster credential by construction, and have no network path to the store (ARCHITECTURE.md section 9)
Open questions
- Whether digest-verified manifests for artifact integrity (ADR embervm/033 decision 3) should be implemented to detect deliberate artifact substitution by a credential holder
- Whether per-workload egress allowlist scoping (ADR embervm/035 open question 1) should be implemented to prevent shared-egress-allowlist-widening
- Whether request-scoped GitHub tool mediation (ADR agents/055) should replace host-keyed egress credential injection
- shared egress allowlist provides no per-workload isolation (ADR embervm/035), making destinations added for one workload reachable by all egress-enabled workloads with no per-workload authorization check