1. Check your tool’s switch
2. Is the correlation identifier stable?
Even when the recovery is sent, it only closes the right incident if the identifier is the same as the trigger’s:Datadog using event_id
Datadog using event_id
The template must emit
$ALERT_CYCLE_KEY. The event identifier ($ID) changes per event and never matches trigger to recovery.New Relic using an event identifier
New Relic using an event identifier
Use
issueId. Event or violation identifiers vary within the same issue.Prometheus with unstable labels
Prometheus with unstable labels
The
fingerprint is derived from the labels. If labels change between triggers (for example by including a timestamp), each send becomes a different alert.3. Was the incident already resolved or closed?
A recovery arriving for an already resolved or closed incident is ignored — it does not reopen or modify anything. If someone resolved it manually first, that is expected behavior.4. The New Relic special case
triggerEvent = INCIDENT_CLOSED does not resolve the incident. In the Issues model, an issue can contain several sub-events, and closing one of them does not mean the issue is over. What resolves it is state = CLOSED.
How to validate after fixing
1
Fire the alert
Force the condition in the tool and confirm the incident is created.
2
Force the recovery
Bring the condition back to normal, or use the tool’s force-state feature.
3
Confirm resolution
The incident should move to
resolved, with the auto-resolution event on the timeline.