Three-State Pattern
What It Is
The three-state pattern is FortrOS's universal model for tracking operations that have uncertainty. Instead of binary (success/failure), every operation has three states:
| State | Meaning |
|---|---|
| Pending | Initiated but not yet verified. The system intends this to happen but hasn't confirmed the outcome. |
| Confirmed | Verified. The operation succeeded and the result has been observed. |
| Rejected/Failed | The operation did not produce the desired outcome. Either it failed, was revoked, or conflicted with another change. |
FortrOS is honest about uncertainty. A change that was sent but not verified is "pending," not "done." A generation that booted but hasn't been health- checked is "untested," not "ok."
Why It Matters
Binary tracking (done/not done) hides uncertainty. If you send a config change and mark it "done" before verifying, you don't know whether it actually took effect. If a node boots a new generation and you mark it "healthy" before the watchdog runs, you're guessing.
The three-state pattern forces the system (and the admin) to distinguish between "we intend this" and "we verified this." The pending state is not a temporary inconvenience -- it's valuable information about what hasn't been confirmed yet.
Where It Appears
Node Enrollment
| State | Meaning |
|---|---|
| Pending | Preboot authenticated, credentials delivered. WireGuard peered but not gossiped. |
| Confirmed | Main OS presented nonce, cert issued, membership gossiped to org. |
| Revoked | Hash removed from org state. Node rejected on all future connections. |
Generation Health
| State | Meaning |
|---|---|
| Untested | Generation deployed, not yet booted or boot not yet verified. |
| Ok | Boot watchdog confirmed maintainer healthy. |
| Failed | Watchdog timed out. Generation marked bad, preboot falls back. |
Configuration Changes
| State | Meaning |
|---|---|
| Pending | Change written to CRDT, propagating via gossip. Not yet enacted on target. |
| Confirmed | Enacted on target, result verified. |
| Conflicted | Two partitions made incompatible changes. Preconditions no longer match. Surfaced to admin. |
Conflict Resolution (Locality-Wins)
The three-state pattern IS the locality-wins mechanism: a local partition's change is confirmed (enacted, verified). A remote partition's change is pending (intended, not verified). On merge, the confirmed change wins and the pending change is rejected because its precondition no longer holds. The originator is informed.
See 08 Cluster Formation.
Managed Infrastructure Drift
| State | Meaning |
|---|---|
| Confirmed | Device config matches org desired state (last clobber verified). |
| Pending | Config just clobbered, not yet re-exported to verify. |
| Drifted | Exported config doesn't match desired state. Security signal. |
Rolling Upgrades
| State | Meaning |
|---|---|
| Pending | Node scheduled for upgrade, not yet started. |
| Upgrading | Node is draining workloads, rebooting, verifying. |
| Confirmed | OS healthy + workloads healthy. Proceed to next node. |
| Failed | OS or workloads unhealthy. Roll back, stop upgrade. |
The Pattern as a Primitive
The three-state pattern is not specific to any one feature -- it's a reusable primitive available to any org service via the state tree's resolution schemas. A service that registers a state tree and picks the precondition-based schema gets three-state tracking for free: proposed changes are pending until verified, verified changes are confirmed, conflicting changes are surfaced.
Links
- CRDTs -- Machine-resolvable conflict resolution underlying the pattern
- 08 Cluster Formation -- Resolution schemas
- Monitoring and Self-Observation -- Pending/confirmed states visible in the admin interface