# How the platform finds the last-known change

> Which recorded change caused an incident, if any — the step in the flow, the suspect and proven-good criteria, the change sources per onboarding option and the exact prompt section.

_Concept · Updated 2026-10-26 · Describes release 0.1.164_

Most incidents follow a change. Deciding which change caused one, if any, is split between fixed code and the analysis. Code records the changes, sets the window, ranks every change in it and decides what is *proven good*. The analysis tests the top few against evidence and names one, or none. Code then checks that pick against its own ranking before anything is shown or acted on. The result is the incident's **Probable cause** card ([Read an incident: Probable cause](/help/read-an-incident.md#probable-cause)).

## The step in the incident flow {#step}

The step is part of the analysis (L4), before its summary:

1. **Error start** (code) — from the alert itself: CloudWatch's first breaching datapoint, the Azure Monitor evaluation window, the sender's start time, else its arrival.
2. **Healthy since** (code) — the latest of: where the previous error on the same correlation key ended, where the alarm's own history last recovered, and when the fix in a verified last-known-good record of the key or resource was verified. With none of these, the look-back cap (`aiops_lkg_lookback_h`, 72 h).
3. **Candidates** (code) — every applied change the ledger recorded between those two points: the platform's own changes and every change a source below reported. Denied attempts, changes that never ran and changes made after the error started (the platform's own fixes included) are never candidates.
4. **Ranking** (code) — each candidate scored and given a verdict (below); the top 5 (`aiops_lkg_top_n`) reach the analysis, the rest are counted.
5. **The prompt section** (code → analysis) — the ranked list goes into the analysis's first prompt as one section, fenced as estate data and PII-scrubbed ([the exact text](#prompt)).
6. **The test** (analysis) — the analysis cannot finish until it has loaded the `last-change-check` skill. It reads the failing resource's logs, tests each candidate from the top, and names one change, or none, in its wrap-up.
7. **The check** (code) — a pick outside the ranked list is flagged and never acted on; the card shows the analysis's pick only when it is in the top 5, else the platform's #1 as a ranking.

Switch: `aiops_lkg_cause_block` (on by default) puts the section in the prompt; off, the analysis works from `change_lookup` alone. Every setting named here is in the [Settings reference](/help/settings-reference.md).

## Good or bad: the criteria {#criteria}

Code gives each candidate one verdict:

| Verdict | Rule | What follows |
|---|---|---|
| `suspect` | Related to the failing resource by a factor of 0.4 or more, and made within `aiops_lkg_suspect_h` (2 h by default) before the error start | Can be the cause. With a recorded before-state, **Draft revert of this change** restores it |
| `proven_good` | The resource's CloudWatch alarm evaluated real data and stayed OK from before the change until `AIOPS_LKG_PROVEN_GOOD_H` (24 h, a platform value) after it | Never the cause; ranked last. Not possible on Azure (its alert history has no OK state), nor on an alarm that counts missing data as OK |
| `possible` | Any other candidate | Weighed by its score; it can still be the cause when a mechanism explains the error |

The score is `100 × relation × time band × kind`, every factor shown to the analysis:

- **Relation** — named by the incident 1.0 · the same resource 1.0 · the same service 1.0 · attached to the resource 0.8 · one hop upstream 0.7 · two hops upstream 0.5 · the same resource group 0.4 · downstream 0.2 · time only 0.1.
- **Time band** (before the error start) — within 15 minutes 1.0 · within 1 hour 0.8 · within the suspect window 0.6 · within 24 hours 0.3 · older 0.1.
- **Kind** — a change that only edits tags × 0.3.

What makes the analysis name a change, from the `last-change-check` skill:

- a **mechanism**: how this change could cause this error (a revoked port gives timeouts on that port; a changed setting is named by the first error);
- the **timing**: the error started minutes after it;
- the **state now against the before-state**: the change is still in place;
- only a change inside the window counts; an older change needs a named delayed mechanism (an expiry, a TTL, a scheduled job, a cache flush);
- `revert` is set only when it is confident and the change's before-state is recorded; every candidate ruled out is listed with why.

A good change is one the evidence clears: `proven_good`, or tested and ruled out with a reason. An empty list is not proof that nothing changed: what the ledger cannot see is listed in the section itself ([What is never captured](/help/changes-reference.md#not-captured)).

## Sources per onboarding option {#sources}

A change can be ranked only when a source recorded it. What each onboarding option adds:

| Onboarding option | Change source | What it records | Before-state |
|---|---|---|---|
| Any workspace | The platform's own changes | Every write the platform ran, with its pre-change read | The pre-change state, read just before the write |
| AWS read connection | CloudTrail | Completed write events in the declared regions plus `us-east-1`, every 15 minutes (`aiops_change_harvest_interval_s`) and when an incident opens | None, unless the next row applies |
| AWS read connection with `config_history: true` | AWS Config | The configuration AWS Config recorded before and after each harvested write (the recorder must be on for the resource type) | Yes, up to 28 days back |
| AWS read connection with `k8s_audit: true` | EKS control-plane audit log | Kubernetes writes by people and by the CI/CD service accounts of `k8s_ci_namespaces` (the cluster's `audit` log type must be on) | None |
| Azure read connection | Activity Log + Change Analysis | Write operations per subscription, joined with the resource's property diff | Yes, from the diff |
| Source control (GitHub, GitLab, Azure DevOps, Gitea) with `repos` | Watched repositories, or the provider's webhook | Pushes and merged pull requests on up to 25 repositories per connection | The parent commit |
| Argo CD | `/webhook/argocd/…` | Syncs and drift | None |
| Self-managed Kubernetes | `/webhook/kubernetes-audit/…` | The API server's audit events, the same people and CI/CD rules as EKS | None |
| ITSM (ServiceNow, Jira) | `/webhook/servicenow/…`, `/webhook/jira/…` | Change tickets, linked to a service by their configuration item; an approved ticket is also the reference the unauthorized-change check matches an audit-log write against | None |
| Any tool | Change webhook `/webhook/change/…` | Deploys, configuration and pushes the sender reports | Reported, never a restore source |

Set-up for each: [Report changes to the change webhook](/help/change-webhook.md) (Kubernetes, Argo CD, repositories, and what the Probable cause card needs), [Set up AWS](/help/setup-aws.md), [Set up Azure](/help/setup-azure.md) and [Ticket systems](/help/ticket-systems.md). The harvest's caps and dedupe are in the [Changes reference](/help/changes-reference.md#harvest).

## The prompt section the analysis reads {#prompt}

The section sits in the analysis's first prompt after the alert data and the service relationships, at most 2,900 characters; the ranked lines are budgeted first, the optional lines are cut to what is left. Its fixed lines, as the analysis reads them (the suspect hours are the workspace's own `aiops_lkg_suspect_h`; the proven-good hours are the platform's `AIOPS_LKG_PROVEN_GOOD_H`):

```
CHANGE CANDIDATES — ranked by the platform (deterministic: relation x time x kind). Name at most ONE as suspect_change, or none.
Verdicts: suspect = related x0.4 or more and at most 2 h before the error; proven_good = its alarm evaluated data and stayed OK from before it to 24 h after — never the cause, ranked last; possible = the rest. Name one only when a mechanism explains THIS error, the timing fits and its effect is still in place (the resource now vs its recorded before-state); otherwise none.
Error start: <time> (<basis>); alert fired <time>.
Window: <from> to <to> — <how the window was bounded>.
Audit log read: <connection> through <time> (<n> min after the error start) — cloud audit logs land 5-20 min after a write, so a newer change may be missing.
```

Then, fenced as estate data, one line per candidate, factors first:

```
1. CHG-1a2b3c4d · score 100 suspect · 8 min before the error · same resource as the alert x1.0 · within 15 min x1.0 · before-state recorded · RevokeSecurityGroupIngress on sg-0abc by ops-admin (aws:prod)
+2 more; 1 denied attempts ignored
```

After the list, each only when it applies: the changes made during the incident ("never the cause"), earlier alerts on related resources, the alarm's log groups, the restore-first advice, and what the ledger cannot see ("absence here is no proof").

The finish gate, when the analysis tries to finish without the skill:

> You have not correlated this incident with recent changes. Load the 'last-change-check' skill and check deploys/commits/change tickets against the alert start time, then name the one change that caused it — or none — in suspect_change. If no change datasource is connected, state that explicitly in last_change_checked.

The answer is two fields of the wrap-up: `last_change_checked` (what was checked and the conclusion, or why it was not checkable) and `suspect_change` (`change_id` exactly as listed or empty, `confidence`, `reasoning`, `alternatives`, `revert`). To drill in, the analysis calls `change_lookup` with a resource or a service and a window; its default window is the section's.


## Next steps

- Next: [Storms, unmapped and flapping signals](/help/storms-and-unmapped-signals.md)
- [All Work with the agents pages](/help/work-with-the-agents/llms.txt)
- [Documentation home](/help/)