# Loki

> Connect Grafana Loki for LogQL reads, address a multi-tenant Loki with one connection, and know which log endpoints the agent may call.

_How-to · Updated 2026-10-23 · Describes release 0.1.164_

## Overview {#overview}

A Loki connection gives the agents log search. The **Incident** agent correlates errors and timelines in it during diagnosis (logs is one of the five categories that agent needs) and the **SecOps** agent reads it for security evidence; the estate x-ray and the insights chat use the same connection, and a Lakehouse collection can pull logs from it on a schedule.

One connection can address every tenant of a shared, multi-tenant Loki: declare the tenant ids and the agent picks one per query. The platform sends `X-Scope-OrgID` for you; it is a routing header, not authentication, so restrict the authorised tenants at your gateway.

## Setup {#setup}

### Before you start {#before-you-start}

- The `datasources.manage` permission; the built-in owner and operator roles carry it.
- No credential in the form; a Loki behind authentication takes one through the REST API (see [Installation](#installation)).
- The query API exposed to the platform or to a [private-network agent](/help/private-access.md); [Installation](#installation) lists the paths.

### Installation {#installation}

Expose the query API to the platform or to a [private-network agent](/help/private-access.md): `loki/api/v1/query`, `loki/api/v1/query_range`, `loki/api/v1/labels`, `loki/api/v1/label/<name>/values` and `loki/api/v1/series`, plus `/ready` for the connection check. Keep `loki/api/v1/push` out of the exposed surface; the platform refuses it anyway.

The form has no credential field. A Loki behind authentication takes its credential through the REST API: `secret_values.api_key` is sent as `Authorization: Bearer <token>`, or `secret_values.username` and `secret_values.password` as HTTP Basic. Both are stored write-only.

### Configuration {#configuration}

1. Open **Settings · Data in → Data sources** and select **Connect datasource**.
2. Under **Type**, choose **Loki**. Enter a **Connection name**; the connected list shows it, and per-connection agent tools are named after it.
3. Fill the fields; the TLS-skip field sits under **Advanced connection settings**:

| Field (portal label) | API key | Value |
|---|---|---|
| Loki base URL | `url` | The query service, for example `https://loki.example.com` or `http://loki.logging.svc:3100` inside the cluster |
| X-Scope-OrgID tenant id(s), comma-separated (multi-tenant Loki; leave blank for single-tenant) | `org_ids` | `team-a, team-b`; blank for `auth_enabled: false` |
| Skip TLS verify for a self-signed cert (optional — type: true) | `insecure` | `true` only for a self-signed certificate |

4. Optionally, open **Optional · describe this environment** and fill **What should AIOps know about it?** with the label that names a service in your logs (`app`, `service`, `container`).
5. Under **Use this connection for**, choose **Investigate only** and select **Connect and verify**.

The same connection through the API (`POST /api/aiops/{tenant_id}/datasources`, requires `datasources.manage`):

```json
{
  "type_key": "loki",
  "name": "prod-loki",
  "description": "Prod cluster logs; services are labelled app=<name>, errors carry level=error.",
  "config": {
    "url": "https://loki.example.com",
    "org_ids": "team-a, team-b"
  },
  "role": "read",
  "enabled": true
}
```

### Validation {#validation}

**Connect and verify**, and **Test** on the row later, sends a GET to `/ready` and then to `/loki/api/v1/status/buildinfo`, with no tenant header. **You should see** a dialog titled `<connection name> is connected` with **Connected — verified from the control plane**, and later in the row's **Last test** `ok · HTTP 200 /ready — ready`. Readiness proves reachability and, where the gateway enforces it, authentication; it does not prove that a tenant's logs are queryable. Run a narrow LogQL query for a known service through the agent, or in your own log viewer, and confirm recent lines appear.

### If something fails {#if-something-fails}

The dialog shows **Needs a fix**, what to fix and the check's own detail; the row is saved either way. Select **Edit**, correct the field the detail names and save again, or select **Retry** once a cause outside the form (network, vendor-side permission) is fixed. Every verdict this connector produces, and what to do about it, is in [Troubleshooting](#troubleshooting) below.

## Data collected {#data-collected}

### Tools the agent gets {#tools}

- `<name>_request` — the HTTP tool against the base URL: `method`, `path`, `query`, `body`, `content_type`, and `org_id` whenever tenant ids are declared (one of the declared ids; the first is the default). A typical read is `GET /loki/api/v1/query_range` with `query`, `start`, `end` and `limit` as query parameters, for example `{"query": "{app=\"checkout\"} |= \"error\"", "limit": "50"}`. Label discovery uses `/loki/api/v1/labels` and `/loki/api/v1/label/<name>/values`.

There is no native Loki tool; LogQL rides the request tool. Responses are capped at 20,000 characters per call, so the agent is told to bound queries with `limit` and a time range.

### Read surface {#read-surface}

`GET` and `HEAD` run freely, and so does `POST` to `loki/api/v1/series`. Every other `POST`, `PUT`, `PATCH` and `DELETE` is refused in an investigation, never offered for approval; `POST /loki/api/v1/push` is the one to expect. The refusal reads `Investigation connections are read-only. This operation was not executed; prepare an incident change using a write connection and obtain approval.` The full rule table is under **The read/write gate in detail** in [Connections and remediation access](/help/read-and-write-roles.md#read-role-vs-write-role).

### What makes the evidence usable {#usable-evidence}

An investigation correlates on the incident's `service` label. Logs that carry the same value under a label the agent can name — `app`, `service`, `container`, or whatever your pipeline sets — are what the agent finds first, so say which label it is in the connection notes. The notes ride into the tool description the agent reads.

## Troubleshooting {#troubleshooting}

| What you see | Cause | What to do |
|---|---|---|
| `failed · SSRF guard: '<host>' resolves to non-public address … — blocked. The agent can still use this endpoint — only the Test probe is blocked by the SSRF guard's secure default.` | A cluster-internal or private address. The agent's own tools are not SSRF-guarded. | Accept it, add the host to the workspace's `ssrf_allow_hosts`, or turn the workspace SSRF guard off under Guardrails. |
| `failed · HTTP 404 /ready` then `HTTP 404 /loki/api/v1/status/buildinfo` | The URL points at a gateway path or a UI, not the Loki query service. | Use the query frontend's base URL without a path. |
| `failed · HTTP 401 /ready` or `HTTP 403` | The gateway wants a credential; the form has none. | Add `secret_values.api_key`, or `username` and `password`, through the API. |
| `failed · ConnectError: …` or `ConnectTimeout` | The platform cannot reach the URL. | Use an address the platform reaches, or install a [private-network agent](/help/private-access.md) where Loki lives. |
| A certificate error in the detail | A self-signed certificate. | Set **Skip TLS verify** to `true`, or install a trusted certificate. |
| **Test** is green, but every query answers `HTTP 401` | A multi-tenant Loki refuses a query with no tenant header, and the tool sends `X-Scope-OrgID` only when tenant ids are declared. | Fill the tenant ids. |
| Queries return nothing for a service you can see in Grafana | The wrong tenant, or another label names the service. | Pass the right `org_id`; read `/loki/api/v1/labels` to find the label, and name it in the connection notes. |
| A `POST` to `loki/api/v1/push` answers `Investigation connections are read-only.` | The write gate. | Expected. |
| The agent never uses the connection | Disabled, write role, a pillar allowlist that excludes the agent, or a type platform staff disabled. | Check **State**, **Use this connection for** and **Pillar allowlist** on **Edit**. |

The generic verdict strings and the SSRF guard are explained in [Test a connection and read the failure](/help/manage-connections.md#troubleshooting). Every connection status word is in the [status and error reference](/help/status-and-error-reference.md#connection-statuses).

## Next steps {#next-steps}

- Select **Discover this connection** in the success dialog to find the services and dependencies behind it: [Launch a scan](/help/estate-harvest-and-xray.md#launch-a-scan).
- Give the agent the metrics side too: [Prometheus](/help/integration-prometheus.md).
- See what else each agent needs in [Choose your connection](/help/choose-your-connection.md#agent-needs).