Why alarm correlation needs a causal model, not a time window

Every network operations center has a version of the same rule. If two alarms fire within a few minutes of each other, on devices within a hop or two, they are probably the same incident. Group them, open one ticket, move on.

It is a good rule. It is intuitive, cheap to run and easy to explain. It also has a failure mode that shows up at exactly the wrong moment: when the network is noisy. This post explains why, and describes the alternative we studied in Turning Alarm Floods into Incidents with Hawkes Processes.

What a time window actually assumes

A time-and-topology window makes one assumption: alarms that are close together are related. That is true for the alarms caused by a fault. It is also true for every unrelated alarm that happens to fire nearby at the same time.

At quiet hours, that second group is small, so the rule looks accurate. But background noise in a real network is not constant. Polling cycles, CPU spikes, interface flaps, maintenance windows and storms all raise the rate of alarms that have nothing to do with the incident in front of you. As that rate rises, a window pulls more and more of them in.

Operators respond by tightening the window. Now noise is excluded — and so are the slower downstream effects of the real fault, which get split into separate tickets. Tuning moves the error between two buckets; it does not remove it.

Asking a different question

The model we used, a Hawkes process, starts from a different question. Instead of asking whether two alarms are close, it asks: is this alarm better explained by something that happened just before it, or by the normal background rate of alarms like it?

Hawkes processes were introduced in 1971 to model earthquake aftershocks. Each event is either a background event — something that happens on its own — or a triggered event, caused by an earlier one with an influence that fades over time. Network faults behave the same way. An optic degrades; the link goes down; routing adjacency drops; hosts become unreachable; applications time out.

We made two changes that matter for networks:

• Topology constraints. An alarm can only be triggered by alarms on physically connected devices within two hops. Faults propagate along cables, not across the room.
• Two timescales. Influence decays on a fast clock of roughly ten seconds for hop-by-hop propagation, and a slow clock of roughly two minutes for repeated polling cycles.

The model learns base rates for each alarm type by device role and influence strengths between alarm types at each distance, using expectation-maximization. It needs no labeled incidents. Fitting 24 hours of alarms — 13,638 events — took 1.6 seconds on a laptop.

Why noise stops hurting

Consider a CPU spike alarm on a core router. It happens all the time. A time window will attach it to any incident nearby. The Hawkes model has learned that this alarm type has a high background rate, so a CPU spike near an incident is well explained as background — and stays out.

The effect shows up clearly as noise rises. In our simulated 54-node network, we compared the topology-aware Hawkes model against a topology-window rule engine that had been tuned on labeled data:

The rule engine does not degrade gracefully; it collapses. At 30 alarms per minute it is effectively merging everything. The causal model loses accuracy too, but slowly, because it is separating cause from coincidence rather than proximity from distance.

Finding the origin, not just the group

Grouping is only half the job. Operators also need to know which alarm started it. Because every alarm in a Hawkes model has a most-likely parent, you can follow the chain back to an alarm with no parent inside the incident. That is the origin.

At default noise, the topology-aware model identified the root-cause device in 75% of incidents, compared with 42% for the tuned rule engine and 17% for simple time-gap grouping. It also produced fewer tickets per incident — 1.8 versus 2.9 for the rules.

Topology is necessary, but not sufficient

One result surprised us. Removing the topology constraint from the Hawkes model dropped F1 from 0.85 to 0.61, and under heavy noise it collapsed to 0.01. Topology clearly matters.

But the rule engine used the exact same topology map and still scored far lower. The map tells you which alarms could be related; the learned causal model tells you which ones are. You need both.

An auditable model

A frequent objection to statistical correlation is that operators cannot see why it made a decision. Hawkes models are unusually transparent here. The learned excitation strengths form a small table — which alarm types trigger which, at which distance — that you can read directly.

Without any labels, the strongest learned relationship was routing adjacency loss triggering host-unreachable alarms one hop away. The second was routing adjacency loss triggering application timeouts. It is the diagram an experienced network engineer would draw on a whiteboard.

What we have not shown yet

These results come from a simulator, deliberately built not to follow Hawkes assumptions so the model could not win by construction — but a simulator nonetheless, built by the same author as the model. Real networks will be messier. Incidents still fragment to 1.8 tickets rather than one, and results were measured on 54 devices.

Next, we are testing on real alarm streams with validated root causes and on topologies above 5,000 devices. The method, results and limits are in the full research note.

The question is not whether two alarms are close. It is whether one of them explains the other.