# Datadog

> Connect the Datadog API with read-only keys for monitors, metrics, events and logs, and send monitor notifications to the workspace webhook with the paste-in template.

_How-to · Updated 2026-10-17 · Describes release 0.1.164_

## Overview {#overview}

Datadog is two integrations in one. The connection on this page is the read side: the **Incident** agent reads monitors, metric queries, events and logs through it (alerting, metrics and logs are three of the categories that agent needs), the **SecOps** agent reads its logs, the **FinOps** agent reads its metrics, and the estate x-ray and the insights chat use it too. The other half is alerting: each monitor notifies a Datadog webhook that posts to this workspace, see [Alerts in](#alerts-in). Either half works on its own.

## Setup {#setup}

### Before you start {#before-you-start}

- The `datasources.manage` permission; the built-in owner and operator roles carry it.
- A Datadog API key and a scoped application key owned by a reader account ([Installation](#installation)).

### Installation {#installation}

In Datadog, under **Organization Settings → API Keys**, create a named API key for this connection. Under **Organization Settings → Application Keys**, create a scoped application key owned by a dedicated reader or service account with `monitors_read` and `metrics_read`, adding only the extra read scopes the APIs you use need; an application key with no scopes inherits its owner's full permissions. The platform refuses writes through this connection whatever the keys allow, so there is no reason to grant any.

### Configuration {#configuration}

1. Open **Settings · Data in → Data sources** and select **Connect datasource**.
2. Under **Type**, choose **Datadog**. Enter a **Connection name**; the connected list shows it, and per-connection agent tools are named after it.
3. Fill the fields; the TLS-skip field sits under **Advanced connection settings**:

| Field (portal label) | API key | Value |
|---|---|---|
| Datadog API URL (e.g. https://api.datadoghq.com) | `url` | `https://api.datadoghq.com` for US1; the regional API URL for every other site |
| API key | `api_key` | Sent as `DD-API-KEY`; stored write-only |
| Application key | `app_key` | Sent as `DD-APPLICATION-KEY`; stored write-only |
| Skip TLS verify for a self-signed cert (optional — type: true) | `insecure` | Leave blank |

4. Optionally, open **Optional · describe this environment** and say in **What should AIOps know about it?** what this system is and what matters.
5. Under **Use this connection for**, choose **Investigate only** and select **Connect and verify**.

The same connection through the API (`POST /api/aiops/{tenant_id}/datasources`, requires `datasources.manage`; both keys go in `secret_values`):

```json
{
  "type_key": "datadog",
  "name": "datadog",
  "description": "Monitors and APM for the web tier; services are tagged service:<name>, env:prod.",
  "config": {
    "url": "https://api.datadoghq.com"
  },
  "secret_values": {
    "api_key": "<API key>",
    "app_key": "<application key>"
  },
  "role": "read",
  "enabled": true
}
```

### Validation {#validation}

**Connect and verify**, and **Test** on the row later, sends a GET to `/api/v1/validate` with both headers. **You should see** a dialog titled `<connection name> is connected` with **Connected — verified from the control plane**, and later in the row's **Last test** `ok · HTTP 200 /api/v1/validate — {"valid":true}`. That endpoint validates the API key only: a wrong or under-scoped application key still passes **Test** and fails on the agent's first monitor or metric read. Confirm the application key can list your monitors and read a known metric before relying on the coverage.

### If something fails {#if-something-fails}

The dialog shows **Needs a fix**, what to fix and the check's own detail; the row is saved either way. Select **Edit**, correct the field the detail names and save again, or select **Retry** once a cause outside the form (network, vendor-side permission) is fixed. Every verdict this connector produces, and what to do about it, is in [Troubleshooting](#troubleshooting) below.

## Data collected {#data-collected}

### Tools the agent gets {#tools}

- `<name>_request` — the HTTP tool against the API URL with the two key headers: `method`, `path`, `query`, `body`, `content_type`. Monitors come from `GET /api/v1/monitor`, metric points from `GET /api/v1/query` (also accepted over `POST`) and the v2 `POST /api/v2/query/timeseries` and `/api/v2/query/scalar`, events from `POST /api/v2/events/search`, log aggregates from `POST /api/v2/logs/analytics/aggregate`.

There is no native Datadog tool. Responses are capped at 20,000 characters per call.

### Read surface {#read-surface}

`GET` and `HEAD` run freely, and so do `POST` requests to `api/v1/query`, `api/v2/query/timeseries`, `api/v2/query/scalar`, `events/search` and `analytics/aggregate`, matched on whole trailing path segments. Every other `POST`, `PUT`, `PATCH` and `DELETE` — creating or muting a monitor, scheduling a downtime, posting an event — is refused in an investigation, never offered for approval: `Investigation connections are read-only. This operation was not executed; prepare an incident change using a write connection and obtain approval.` The full rule table is under **The read/write gate in detail** in [Connections and remediation access](/help/read-and-write-roles.md#read-role-vs-write-role).

### Alerts in {#alerts-in}

Datadog sends nothing useful until a payload template is pasted in. In Datadog open **Integrations → Webhooks → New**, put `https://<your-host>/webhook/datadog/<workspace-slug>` in **URL**, the token in **Custom Headers** as `{"X-Webhook-Token": "<webhook-token>"}`, and paste this as the payload:

```json
{
  "alert_id": "$ALERT_ID",
  "id": "$ID",
  "alert_scope": "$ALERT_SCOPE",
  "title": "$ALERT_TITLE",
  "body": "$EVENT_MSG",
  "alert_type": "$ALERT_TYPE",
  "alert_transition": "$ALERT_TRANSITION",
  "tags": "$TAGS",
  "date": "$DATE"
}
```

Then mention `@webhook-<name>` in each monitor's message, for the alert and for the recovery. How the adapter reads it:

- `tag:value` pairs in `$TAGS` become labels, so a monitor tagged `service:checkout` and `env:prod` carries the identity the platform groups on for free. Severity comes only from `alert_type`: `error` is `critical`, `warning` is `warning`, `info` and `success` are `info`, anything else `warning`; a `severity:` tag is ignored.
- The alert's identity is `$ALERT_ID` plus `$ALERT_SCOPE`, so one group of a multi-alert monitor recovering does not close the incident for the other groups. Keep `$ALERT_SCOPE` in the template.
- A transition containing `recovered` resolves the alert; `$DATE` is when the monitor fired and never part of the identity.
- A payload with no title and no id is ignored.

**Alert intake & tokens** has a **Send a test alert** card for the `datadog` source that produces a sample payload and the curl command to post it. [Alerts from your observability platform](/help/external-observability.md) walks through the setup; the [webhook reference](/help/webhook-reference.md#url-forms-and-authentication) has the endpoint and the token forms. Field by field, with the response counters and error strings: [webhook reference](/help/webhook-reference.md#datadog).

## Troubleshooting {#troubleshooting}

| What you see | Cause | What to do |
|---|---|---|
| `failed · HTTP 403 /api/v1/validate` | The API key is wrong, or belongs to another Datadog site than the URL. | Check the key and use the API URL of the site that owns it. |
| **Test** is green, but monitor or metric reads answer `HTTP 403` | `/api/v1/validate` checks the API key only; the application key is wrong, or lacks `monitors_read` / `metrics_read`. | Replace the application key on **Edit** with one that has the read scopes. |
| `failed · ConnectError: …` or `ConnectTimeout` | The platform has no route to the Datadog API. | Check egress from the platform. |
| A `POST /api/v1/monitor`, `PATCH /api/v1/monitor/<id>` or `POST /api/v1/downtime` answers `Investigation connections are read-only.` | The write gate. | Expected. |
| Datadog's webhook test answers `401 invalid webhook credentials` | Wrong token, wrong workspace slug or a disabled workspace; the endpoint never says which. | Copy the exact URL and token from **Alert intake & tokens**; the header wins over a token in the path. |
| Monitors fire but the incident has no service | The monitor is not tagged `service:<name>`, or `$TAGS` is missing from the template. | Tag the monitor `service:<name>`; keep `$TAGS` in the payload. |
| An incident never resolves although the monitor recovered | `alert_transition` is missing from the template, or the recovery notification does not mention the webhook. | Keep `alert_transition` in the template and `@webhook-<name>` in the recovery message. |
| A recovery closes the incident while another group of the same monitor is still alerting | `$ALERT_SCOPE` is missing, so every group shares one identity. | Add `alert_scope` to the template. |

The generic verdict strings are explained in [Test a connection and read the failure](/help/manage-connections.md#troubleshooting); alert-intake symptoms are in [Alert sources: troubleshooting](/help/setup-alert-sources.md#troubleshooting). Every connection status word is in the [status and error reference](/help/status-and-error-reference.md#connection-statuses).

## Next steps {#next-steps}

- Select **Discover this connection** in the success dialog to find the services and dependencies behind it: [Launch a scan](/help/estate-harvest-and-xray.md#launch-a-scan).
- Send Datadog monitor alerts in through the workspace webhook: [Alerts in](#alerts-in).
- See what else each agent needs in [Choose your connection](/help/choose-your-connection.md#agent-needs).