On-Call
Escalation policies
An escalation policy is the chain EvoHub follows when an alert arrives: who to notify first, how long to wait for a response, and who to try next. Every integration points at one escalation policy. This page explains each setting and how the chain runs.
How a policy runs
- An alert arrives through an integration and the integration's policy starts.
- Step 1 runs after its delay (which can be 0 (now)). It notifies its target.
- If nobody acknowledges, takes over or resolves the alert, the next step runs after its own delay. Each step's delay counts from the step before it.
- After the last step, the policy either stops or, if Repeat escalation is on, waits and starts again from step 1.
The escalation stops as soon as someone:
- acknowledges the alert (unless the policy has an acknowledgement timeout, see below),
- takes over the alert,
- resolves the alert, or the source resolves it,
- suppresses it from a voice call.
Redirect on an alert stops the current escalation and starts the selected policy from the top. See Alerts and incidents.
An alert that arrives through an integration with no escalation policy is recorded, but nobody is paged.
Create a policy
Add steps
Under Escalation Steps, set when each step notifies (Notify after … min) and choose its target: a Schedule (and optionally one of its layers) or a User. Add more steps with the add button.
The New Policy dialog offers the common settings. The policy's own page (Edit) adds per-step notification methods, retries, webhook targets, Slack channels and the acknowledgement timeout.
Step settings
| Setting | Options | What it does |
|---|---|---|
| Notify after | First step: 0 (now), 1, 2, 3, 5, 10, 15, 30 or 60 minutes. Later steps: 1 to 60 from the same list. | How long to wait before this step runs, counted from the previous step (the first step from when the alert opened). |
| via | Default, Push, Call, Email | Default uses each person's own enabled channels ("their preferences"). Choosing a method forces that one channel for this step. |
| Try | once up to 10 times, waiting 1 to 60 min between | Notifies the same target again before the policy moves on. "Call three times, two minutes apart, then wake the backup" is one step with Try 3 times, waiting 2 min between. |
| Target | Schedule, User, Webhook | Who or what this step reaches. |
| Chat → Slack | Send to Slack | Also posts the alert to a Slack channel when this step runs, in addition to paging the target. Available once Slack is connected. See Slack. |
Targets
-
Schedule pages whoever is on call on that schedule when the step runs. Pick a specific layer, or Any layer (round-robin) to let the step's position choose among the layers on duty. See Layers and escalation. If nobody is on call at that moment, the step reaches nobody and the alert's timeline says "No one was on call — this step reached nobody".
-
User pages one person directly. Only people who can respond to alerts can be selected.
-
Webhook sends an HTTP
POSTto the URL you enter, so another system can react. The body is JSON:{ "alert_id": "6f1c2a4e-1b7d-4c55-9a0e-2f8b7d1c9e10", "title": "Database primary is down", "severity": "critical", "summary": "Replication lag over 30s, primary not answering on 5432", "step": 1 }summarycarries the alert's description. The request has a 10-second timeout and is sent withContent-Type: application/jsonandUser-Agent: EvoHub-OnCall/1.0.
Notification method and personal schedules
When a step forces a method, that channel is used even if the person has turned it off in their own preferences. A person's Notification schedule still applies, though: if their schedule does not allow the forced channel at that time, EvoHub uses the channels their schedule does allow instead, and if none are allowed, nothing is sent to them and the escalation carries on. See Notifications.
SMS is not available as a method. A step saved with SMS in the past is shown as SMS (no longer available) and is reset to Default when you edit the policy.
Policy settings
Repeat escalation
Turn on Repeat escalation to run the whole chain again when the last step has run and nobody responded. Choose 1 to 5 times and how many minutes to wait between cycles (1, 2, 3, 5, 10, 15, 30 or 60).
If acknowledged but not resolved
By default, acknowledging an alert ends its escalation. That can leave an alert open with nobody watching it if the person who acknowledged gets pulled away. To guard against that, set If acknowledged but not resolved in to a time (5 minutes to 4 hours). If the alert is still unresolved when the time runs out, the escalation resumes and the timeline records it. Choose escalate again, up to 1 to 5 times so it cannot ring all night, and choose and call:
- Someone else — skips whoever acknowledged and looks for another pair of hands.
- Whoever acknowledged, then someone else — reminds them once. If that goes unanswered, the next repeat escalates past them.
Never (the default) turns this off. The setting is read each time, so turning it off also stops the reminders for alerts that are already acknowledged.
Responding by phone
During a voice call, the person called can press 4 to acknowledge, 3 to escalate to the next step right away, or 6 to suppress the alert. See Notifications.
Delete a policy
Open the policy and delete it; the Delete Policy dialog asks you to confirm. Escalations of the policy that are still running stop.
Warning
Before you delete a policy, open On-Call → Integrations and point every integration that uses it at another policy (or — None —). An integration left pointing at a deleted policy can no longer open alerts.
Related
Was this page helpful?
