Autonomy you can audit
Automated remediation is coming to infrastructure. The question is not whether to trust the model — it is how to check every action it proposes.
Every infrastructure team has automation it does not quite trust. Scripts that only one person runs. Auto-remediation that is enabled in staging and disabled in production. And now, a growing number of AI systems that can suggest — and sometimes execute — changes.
The instinct is to ask whether the model is good enough to trust. We think that is the wrong question.
Trust the check, not the proposer
An AI system that can propose and execute actions has a different security and safety surface from one that only alerts a human. No model will be right every time. What can be right every time is the check that stands between a proposal and production.
Our verification research makes the point concretely. Standard runbook checks approved 98.4% of unsafe maintenance actions in our test set. An exact graph check, run against live state and in-flight work, approved none — and verified a 14,680-device network in under a fifth of a second.
That check does not care whether the action came from an engineer, a script or a model. That is exactly the property you want as the proposers become more automated.
Four layers of guardrails
- Verification against live state: nothing runs that would reduce resilience below its requirement.
- Pacing within churn limits: fleet-wide changes start slowly and never exceed what the control plane can absorb.
- Isolation: every execution runs in its own Firecracker microVM, restored from a clean snapshot.
- Record: proposal, check, approval, execution and outcome are logged together with their evidence.
Progressive, not all-or-nothing
Autonomy should be earned per action type, not granted per system. Start read-only. Move to recommendations with verification attached. Let engineers approve with one click when the check passes. Automate the well-understood cases — inside agreed limits — only once the record shows they are safe.
That is slower than switching automation on. It is much faster than recovering from the incident automation caused.
More from At Raincurve
Why infrastructure needs a reasoning layer
We have spent twenty years getting better at collecting telemetry. The bottleneck has moved to understanding it.
The incident is cross-layer. The tooling isn't.
Organizations are divided by layer. Failures are not. What changes when the unit of analysis is the dependency, not the component.
Why alarm correlation needs a causal model, not a time window
Rule engines group alarms by closeness in time and space. We look at what that misses when background noise rises, and what a self-exciting model does differently.