Loki

Connect Grafana Loki for LogQL reads, address a multi-tenant Loki with one connection, and know which log endpoints the agent may call.

How-to · Updated 2026-10-23 · Describes release 0.1.164

Overview

A Loki connection gives the agents log search. The Incident agent correlates errors and timelines in it during diagnosis (logs is one of the five categories that agent needs) and the SecOps agent reads it for security evidence; the estate x-ray and the insights chat use the same connection, and a Lakehouse collection can pull logs from it on a schedule.

One connection can address every tenant of a shared, multi-tenant Loki: declare the tenant ids and the agent picks one per query. The platform sends X-Scope-OrgID for you; it is a routing header, not authentication, so restrict the authorised tenants at your gateway.

Setup

Before you start

Installation

Expose the query API to the platform or to a private-network agent: loki/api/v1/query, loki/api/v1/query_range, loki/api/v1/labels, loki/api/v1/label/<name>/values and loki/api/v1/series, plus /ready for the connection check. Keep loki/api/v1/push out of the exposed surface; the platform refuses it anyway.

The form has no credential field. A Loki behind authentication takes its credential through the REST API: secret_values.api_key is sent as Authorization: Bearer <token>, or secret_values.username and secret_values.password as HTTP Basic. Both are stored write-only.

Configuration

  1. Open Settings · Data in → Data sources and select Connect datasource.
  2. Under Type, choose Loki. Enter a Connection name; the connected list shows it, and per-connection agent tools are named after it.
  3. Fill the fields; the TLS-skip field sits under Advanced connection settings:
Field (portal label) API key Value
Loki base URL url The query service, for example https://loki.example.com or http://loki.logging.svc:3100 inside the cluster
X-Scope-OrgID tenant id(s), comma-separated (multi-tenant Loki; leave blank for single-tenant) org_ids team-a, team-b; blank for auth_enabled: false
Skip TLS verify for a self-signed cert (optional — type: true) insecure true only for a self-signed certificate
  1. Optionally, open Optional · describe this environment and fill What should AIOps know about it? with the label that names a service in your logs (app, service, container).
  2. Under Use this connection for, choose Investigate only and select Connect and verify.

The same connection through the API (POST /api/aiops/{tenant_id}/datasources, requires datasources.manage):

{
  "type_key": "loki",
  "name": "prod-loki",
  "description": "Prod cluster logs; services are labelled app=<name>, errors carry level=error.",
  "config": {
    "url": "https://loki.example.com",
    "org_ids": "team-a, team-b"
  },
  "role": "read",
  "enabled": true
}

Validation

Connect and verify, and Test on the row later, sends a GET to /ready and then to /loki/api/v1/status/buildinfo, with no tenant header. You should see a dialog titled <connection name> is connected with Connected — verified from the control plane, and later in the row's Last test ok · HTTP 200 /ready — ready. Readiness proves reachability and, where the gateway enforces it, authentication; it does not prove that a tenant's logs are queryable. Run a narrow LogQL query for a known service through the agent, or in your own log viewer, and confirm recent lines appear.

If something fails

The dialog shows Needs a fix, what to fix and the check's own detail; the row is saved either way. Select Edit, correct the field the detail names and save again, or select Retry once a cause outside the form (network, vendor-side permission) is fixed. Every verdict this connector produces, and what to do about it, is in Troubleshooting below.

Data collected

Tools the agent gets

There is no native Loki tool; LogQL rides the request tool. Responses are capped at 20,000 characters per call, so the agent is told to bound queries with limit and a time range.

Read surface

GET and HEAD run freely, and so does POST to loki/api/v1/series. Every other POST, PUT, PATCH and DELETE is refused in an investigation, never offered for approval; POST /loki/api/v1/push is the one to expect. The refusal reads Investigation connections are read-only. This operation was not executed; prepare an incident change using a write connection and obtain approval. The full rule table is under The read/write gate in detail in Connections and remediation access.

What makes the evidence usable

An investigation correlates on the incident's service label. Logs that carry the same value under a label the agent can name — app, service, container, or whatever your pipeline sets — are what the agent finds first, so say which label it is in the connection notes. The notes ride into the tool description the agent reads.

Troubleshooting

What you see Cause What to do
failed · SSRF guard: '<host>' resolves to non-public address … — blocked. The agent can still use this endpoint — only the Test probe is blocked by the SSRF guard's secure default. A cluster-internal or private address. The agent's own tools are not SSRF-guarded. Accept it, add the host to the workspace's ssrf_allow_hosts, or turn the workspace SSRF guard off under Guardrails.
failed · HTTP 404 /ready then HTTP 404 /loki/api/v1/status/buildinfo The URL points at a gateway path or a UI, not the Loki query service. Use the query frontend's base URL without a path.
failed · HTTP 401 /ready or HTTP 403 The gateway wants a credential; the form has none. Add secret_values.api_key, or username and password, through the API.
failed · ConnectError: … or ConnectTimeout The platform cannot reach the URL. Use an address the platform reaches, or install a private-network agent where Loki lives.
A certificate error in the detail A self-signed certificate. Set Skip TLS verify to true, or install a trusted certificate.
Test is green, but every query answers HTTP 401 A multi-tenant Loki refuses a query with no tenant header, and the tool sends X-Scope-OrgID only when tenant ids are declared. Fill the tenant ids.
Queries return nothing for a service you can see in Grafana The wrong tenant, or another label names the service. Pass the right org_id; read /loki/api/v1/labels to find the label, and name it in the connection notes.
A POST to loki/api/v1/push answers Investigation connections are read-only. The write gate. Expected.
The agent never uses the connection Disabled, write role, a pillar allowlist that excludes the agent, or a type platform staff disabled. Check State, Use this connection for and Pillar allowlist on Edit.

The generic verdict strings and the SSRF guard are explained in Test a connection and read the failure. Every connection status word is in the status and error reference.

Next steps