# Prometheus Alertmanager

EvoHub receives alerts from Prometheus through Alertmanager's webhook receiver. Each firing alert opens an EvoHub alert, and when Alertmanager reports it resolved, EvoHub resolves it too.

## Set it up

:::steps
### Create the integration
In EvoHub, go to **On-Call → Integrations → + Add Integration**, choose **Prometheus**, pick an **Escalation Policy** and click **Create Integration**. Copy the **Webhook URL**.
### Add a receiver to Alertmanager
Open your Alertmanager configuration (`alertmanager.yml`) and add a receiver that uses the URL. Keep `send_resolved: true` so EvoHub can resolve alerts automatically.
### Route alerts to it
Point a route at the `evohub` receiver — the top-level route, or a sub-route for the alerts that should page — and reload Alertmanager.
:::

```yaml
route:
  receiver: evohub
  # Or keep your current default receiver and add a sub-route:
  # routes:
  #   - receiver: evohub
  #     matchers:
  #       - severity=~"critical|warning"

receivers:
  - name: evohub
    webhook_configs:
      - url: "https://evohub.io/ingest/prometheus?key=YOUR_INTEGRATION_KEY"
        send_resolved: true
```

To keep another destination during a migration, add a second route or receiver for it; Alertmanager can notify several receivers.

## What EvoHub reads

Alertmanager sends a group of alerts in one request. EvoHub handles each alert in the group separately:

| EvoHub alert | Taken from |
| --- | --- |
| Title | The `alertname` label (or "Prometheus Alert" if it is missing). |
| Description | The `summary` annotation, or the `description` annotation if there is no summary. |
| Severity | The `severity` label — see below. |
| Labels and annotations | All of the alert's labels and annotations, shown on the alert and usable in the voice template. |
| Fingerprint | Alertmanager's fingerprint for the alert (or one computed from its labels). |

### Severity

| `severity` label | EvoHub severity |
| --- | --- |
| `critical` | critical |
| `warning` | high |
| `info`, `informational` | info |
| anything else, or missing | medium |

Add a `severity` label to your alerting rules to control how urgent each alert is.

## Resolve and deduplication

- An alert with status `firing` opens an EvoHub alert. If the same alert (same fingerprint) is still open in EvoHub, Alertmanager's repeat notifications are recorded as **Retriggered** and nobody is paged again.
- An alert with status `resolved` resolves the matching EvoHub alert. This needs `send_resolved: true`.
- Because the fingerprint comes from the alert's labels, an alert whose labels change (for example a label with a changing value) is treated as a new alert. Keep frequently changing values in annotations, not labels.

## Test it

Send a test alert to Alertmanager with `amtool`:

```bash
amtool alert add alertname=EvoHubTest severity=critical \
  --annotation=summary="Test alert from Alertmanager" \
  --alertmanager.url=http://localhost:9093
```

After Alertmanager's group wait, the alert appears in **On-Call → Alerts** and the integration's **Last Event** updates.

## Troubleshooting

- **Nothing arrives**: check Alertmanager's logs for webhook errors. A `401` means the key in the URL is wrong or the integration is disabled.
- **Alerts never resolve**: make sure `send_resolved: true` is set on the webhook config.
- **Alert arrives but nobody is paged**: check that the integration has an escalation policy and that someone is on call on the schedule it targets.

## Related

- [Integrations overview](https://docs-dev.evohub.io/integrations-overview.md)
- [Grafana](https://docs-dev.evohub.io/grafana.md)
- [Escalation policies](https://docs-dev.evohub.io/escalation-policies.md)