# Post-upgrade checks, 0.1.144 to 0.1.149

> The operator follow-up and the evidence to keep for every release from 0.1.144 to 0.1.149, for an install older than 0.1.150: the second part of the table in Upgrade and release verification, moved…

_Reference · Updated 2026-10-29 · Describes release 0.1.168_

The operator follow-up and the evidence to keep for every release from 0.1.144 to 0.1.149, for an install older than 0.1.150: the second part of the table in [Upgrade and release verification](/help/ops-upgrade-and-verify.md#post-upgrade-checks-by-release), moved here when that page reached its size ceiling again. An install older than 0.1.144 reads [Post-upgrade checks, 0.1.109 to 0.1.143](/help/ops-upgrade-checks-0-1-109-to-0-1-143.md) first. Read every release between the installed version and the target, oldest first, then continue with 0.1.150 and later on that page. The release notes are the authority; three versions are never installed (**0.1.106**, **0.1.115** and **0.1.127** — see that page).

| Release crossed | Operator follow-up | Evidence to retain |
|---|---|---|
| 0.1.144 | Schema, additive and created at start: new table `compliancereport` (one row per workspace per UTC day) and the JSON column `investigation.triage`. Re-run the AWS CloudShell setup for every account it set up (a role from the `deploy/aws` module: re-apply it). The read role gains `config:DescribeConformancePacks`, `config:DescribeConformancePackCompliance`, `ssm:DescribeInstanceInformation`, `ssm:DescribeInstancePatchStates`, `backup:ListBackupJobs`, `ec2:DescribeVpcEndpoints`, `ec2:DescribeTransitGateways`, `ec2:DescribeTransitGatewayAttachments`, `ec2:DescribeTransitGatewayVpcAttachments`, `ec2:DescribeTransitGatewayRouteTables` and `ec2:SearchTransitGatewayRoutes` (a role on `ReadOnlyAccess` already has them). The write role gains `ec2:CreateImage`, `ec2:CreateSnapshot` and `ec2:CreateReplaceRootVolumeTask` on instances and volumes tagged `aiops-managed=true`, the same three plus `ec2:CreateTags` on images, snapshots and restore tasks created with the `aiops:change` tag, and `ec2:DescribeImages`, `ec2:DescribeSnapshots` and `ec2:DescribeReplaceRootVolumeTasks`; nothing that deletes. Re-run the Azure Cloud Shell setup for a write connection: its role gains `Microsoft.Compute/disks/read`, `Microsoft.Compute/snapshots/read` and `Microsoft.Compute/snapshots/write`. The guided read role also gains `health:DescribeEvents` for the automated triage's cloud-health step (a role on `ReadOnlyAccess` already has it); until the setup is re-run, that step reads *not available*. Until the setups are re-run, **Scores** lists the new reads as denied and a reboot stops at its backup step (`UnauthorizedOperation`) before anything changes. Defaults that change behaviour on upgrade: a backup before every reboot-type step (`AIOPS_BACKUP_BEFORE_CHANGE`, on; kept 7 days by tag, never deleted by the platform; a reboot-type plan over a CLI write connection is refused at review); a certified step runbook carries its own operations (`AIOPS_RUNBOOK_OWNS_OPERATIONS`, on): **Pre-approved operations** covers single-call routes only, a fetched skill alone no longer makes a change standard, and pre-approving a fix waits until its incident is resolved; a person may only raise a change class, and a P1 a certified runbook covers is released under the standard rules (requests recorded before keep their stamped rules); denied access attempts open SecOps incidents while the SecOps agent is enabled (`AIOPS_DENIED_ACCESS_TO_SECOPS`, on); a scanner HIGH maps to `error` (P2) instead of `critical` (P1) (register rows change on the scanner's next send, open incidents keep their priority); a P3/P4 finding with a prepared fix waits for **Analyse**; the automated triage reads cloud health and known issues before each incident analysis and adds `## Fault domain` to its report (`AIOPS_TRIAGE_CHECKLIST`, on); the compliance reports are read once a day (`AIOPS_COMPLIANCE_REPORT_INTERVAL_S` = 86400). Off until a workspace turns them on: unauthorized-change incidents (`AIOPS_UNAUTHORIZED_CHANGE_INCIDENTS`; fill `change_authorized_principals` first), several runbooks per change (`AIOPS_RUNBOOK_MULTI_COVER`), the change management agent (`AIOPS_CHANGE_AGENT_CR`) and its fix drafting (`AIOPS_CHANGE_AGENT_OWNS_PLAN`); `AIOPS_PATCH_WITHIN_DAYS` is 0 (no rule). A private-network agent in database mode needs this release for the triage steps on its runs (in API mode the control plane does them); turning on the change management agent's fix drafting needs every private-network agent on this release. | The re-run setup output; the first **Scores** report with no denied call; `GET /guardrails` `switch_state` per workspace; one incident's **Automated triage** card |
| 0.1.145 | No schema change. Re-run the AWS CloudShell setup for every write connection it set up: the write role gains `iam:SimulatePrincipalPolicy` on its own ARN only, with an explicit deny on every other principal (a role from the `deploy/aws` module is unchanged and its permission check keeps reading *unknown*). Until the check can answer, the step-runbook automatic lane does not run a runbook on its own; a person releases it. Behaviour that changes on upgrade: an unconfirmed permission check stops an automatic step-runbook run; a change request the platform executed stops covering outside writes 60 seconds after its execution; STS calls are no longer harvested as changes; a fix whose alerts cleared at the source and whose validation plan reads as expected clears with no agent run; the agreed comparison table's sha256 changes (A.8.8 and A.8.13 notes), so a recorded acceptance reads as changed since and needs a new acceptance; a reviewed revert that re-creates a security-group rule with a description needs a new review (its digest now carries the description). | The re-run setup output; a step runbook's **Check runbook** showing the permission check passed; `GET /guardrails` `switch_state` per workspace |
| 0.1.146 | No schema change. Re-run the AWS CloudShell setup for every account it set up (a role from the `deploy/aws` module: re-apply it). The read role gains `lambda:ListFunctions`, `ecr:DescribeRepositories`, `ecr:DescribeImages` and `ecr:ListTagsForResource` (a role on `ReadOnlyAccess` already has them); until the setup is re-run, the harvest's **Coverage** lists `lambda:ListFunctions` as denied. The write role gains `ssm:SendCommand` with the `AWS-RunPatchBaseline` document only on instances tagged `aiops-managed=true`, `ssm:GetCommandInvocation`, `ssm:ListCommandInvocations` and `ssm:DescribeInstanceInformation`, and Reachability Analyzer (`ec2:CreateNetworkInsightsPath` and `ec2:StartNetworkInsightsAnalysis` only with the `aiops:change` tag on create, `ec2:StartNetworkInsightsAnalysis` on a path and `ec2:DeleteNetworkInsightsPath` and `ec2:DeleteNetworkInsightsAnalysis` only on what carries it, account-wide `ec2:Describe*` (instance and launch-template user data included), `ec2:GetManagedPrefixListEntries`, `ec2:GetTransitGatewayRouteTablePropagations`, `ec2:SearchTransitGatewayRoutes`, `elasticloadbalancing:Describe*`, `network-firewall:Describe*` / `List*` and `tiros:CreateQuery`, `ExtendQuery`, `GetQueryAnswer`, `GetQueryExplanation`, `GetQueryExtensionAccounts`); nothing uses those yet. `ssm:GetCommandInvocation` and `ListCommandInvocations` are on `*` and return any command's output in the region, and `AWS-RunPatchBaseline` takes a `BaselineOverride` and an `InstallOverrideList` IAM cannot restrict, so the write role can install packages on a tagged instance. The backup's `ec2:CreateTags` is now bound to the backup's own create calls (`ec2:CreateAction` = `CreateImage`, `CreateSnapshot`, `CreateReplaceRootVolumeTask`) and to the tag keys `aiops:change` and `aiops:retain-until`. A `deploy/aws` write role also gains `iam:SimulatePrincipalPolicy` on its own ARN and a region pin on its `aiops:change`-scoped statements. Defaults that change behaviour on upgrade: the scheduled Security Hub / Defender pull is on every 15 minutes (`AIOPS_SCANNER_SYNC_INTERVAL_S` = 900; a workspace that stored `0` stays off), whatever the SOC 2 / ISO 27001 switch says, and its Inspector CVE read is on unless a connection says `pull_vulnerabilities: false`. With **Analyse P1/P2 findings automatically** on, every HIGH or CRITICAL CVE starts one paid analysis (measured USD 1.04 to 2.92 each): before the first pull, set each workspace's `daily_cost_limit_usd`, set **Scanner sync** to off where the pull is not wanted, and give an AWS read connection into a shared account **Only findings on resources tagged** (`finding_tag_scope`, one `key=value`; any other value fails that connection with `invalid_tag_scope`, it never pulls wider). The change management agent drafts every change request (`AIOPS_CHANGE_AGENT_CR`, on): one change-agent run of at most 8 steps per change request, about USD 0.33 measured. A workspace that set `aiops_change_agent_owns_plan: true` without setting `aiops_change_agent_cr` now gets Phase 2 (the agent drafts the plan); set `aiops_change_agent_cr` or `aiops_change_agent_owns_plan` explicitly before the upgrade if that is not wanted. Screens and names: the change page's "Verification" card is **Validation plan** and "Safety net" is **Rollback plan**; "Alert correlation" is **Service correlation** in the sidebar and on L2 (skill id `alert-correlation`, the `/alerts` route and the `unmapped` filter are unchanged); the first firewall harvest lists deny rules as added once. API and MCP: the automated triage's `triage_checklist[*].step` numbers follow the new order (1 dependency map, 2 known issues, 3 cloud health, 4 recent changes, 5 signals, 6 summary, 7 fault domain); match a step by its `title`, not its number. | The re-run setup output; the next harvest's **Coverage** with no denied `lambda:ListFunctions`; a step runbook's **Check runbook** permission check; `GET /findings/sync` (`interval_s`, `interval_inherited`) and `GET /guardrails` `switch_state` per workspace |
| 0.1.147 | Schema, additive and created at start: new table `syntheticcheck`, the columns `incident.merged_into_id` and `investigation.scope_deferred` (JSON), and the index `ix_incident_tenant_opened` on `incident (tenant_id, opened_at)`. No new permission. New routes: `POST /api/aiops/{tenant_id}/changes/{change_id}/reject` (`approvals.decide`), `GET` and `POST /api/aiops/{tenant_id}/monitors`, `PATCH` and `DELETE …/monitors/{check_id}`, `POST …/monitors/{check_id}/run` and `POST …/monitors/derive` (`datasources.view` / `datasources.manage`), with the MCP tools `reject_change`, `list_monitors`, `create_monitor`, `update_monitor`, `delete_monitor`, `run_monitor` and `derive_monitors`. The storm gate is removed: `aiops_storm_threshold` and `aiops_storm_window_s` (a workspace setting, or `AIOPS_STORM_THRESHOLD` / `AIOPS_STORM_WINDOW_S` on the control plane) are accepted and ignored, and a burst of unrelated services now opens one incident each (the shared-alert-name fold, `aiops_fold_shared_cause_threshold` 3, stays). Grouping reads the new windows `aiops_storm_group_window_s` (300 s, at most 3600) and `aiops_storm_spread_window_s` (600 s, at most 7200), whole seconds, platform `AIOPS_STORM_GROUP_WINDOW_S` / `AIOPS_STORM_SPREAD_WINDOW_S` or a workspace setting; `aiops_storm_agent` keeps its meaning (the grouping agent, on). A `__storm__` incident left open is not split any more: no alert joins it, it resolves when its alerts clear or a person resolves it, and an alert on one of its services opens that service's own incident beside it. A dashboard or alert on `aiops_webhooks_total{outcome="storm"}` must read `outcome="grouped"`. SOC 2 / ISO 27001 are off in every workspace (`AIOPS_COMPLIANCE_FRAMEWORKS`, off): the first scheduler tick after the upgrade resolves every open platform-check finding (`control:<id>`, `by: scope-change`; one a person holds stays), and a compliance scan answers `409`. `AIOPS_COMPLIANCE_FRAMEWORKS` is platform-wide: `true` (helm `extraEnv`) turns SOC 2 / ISO 27001, EST-01 to EST-06 and the platform alarms back on in every workspace whose `compliance.enabled` is not an explicit `false`, customer workspaces included. Set it only once the customer has agreed the comparison table, never for a Test Lab twin on a shared control plane; Test Lab cases that expect the platform's own checks get no scan while it is off. Defaults that change behaviour on upgrade: the known-issue fast path (`AIOPS_KNOWN_ISSUE_FAST_PATH`, on: an alert-started analysis rated 100 % relevant runs capped at 3 steps); post-change validation in 3 rounds 300 s apart (a rule's `minutes` no longer times anything: `validation_rounds: 1` with `validation_interval_s` = minutes × 60 reads about the old single window); validation signals derived from the change request's plan when no rule matches (`AIOPS_VALIDATION_DERIVED`, on); a 10-minute hold on the automatic verify after a start, reboot or restart; daily synthetic-check suggestions (`AIOPS_MONITOR_DERIVE`, on): up to 20 internet-facing load balancers and API Gateway APIs per workspace are probed from the control plane, `GET /` every 300 s, in trial (nothing opens) until a person keeps or discards them; an API Gateway call is billed and can invoke its integration, so turn `aiops_monitor_derive` off where that is unwanted; backup findings from the daily AWS Backup read (`AIOPS_BACKUP_FINDINGS`, on: one P3 incident per workspace while SOC 2 / ISO 27001 are off); CI/CD service-account writes recorded as changes (`AIOPS_K8S_CI_NAMESPACES`: `flux-system`, `jenkins`, `gitlab-runner`, `tekton`); Azure DevOps `workitem.updated` deliveries that change the state are signals, so a service hook that already sends them now resolves incidents on Done, Closed, Resolved or Removed, and a state change on a work item the platform has not seen opens an incident with an analysis (any other update is ignored; an incoming ticket missing required fields is not held yet); AWS Config findings key on their own rule (`control:<rule-name>`): a finding already open in the shared incident stays there until it clears, new ones open one incident per rule. Roll private-network agents to this release: an agent in database mode needs it for the service's resources in the prompt, a run's resource selection and the vulnerable-package plan per resource type. | `GET /guardrails` per workspace: `switch_state.known_issue_fast_path`, `verification.validation_derived` and `compliance_frameworks_available: false`; the platform-check findings resolved `by: scope-change`; the first `incident.grouped` audit row; **Synthetic checks** after the first daily suggestion; agent version |
| 0.1.148 | No schema change or permission. New workspace setting `aiops_grouping_wait_s` (whole seconds `0`–`600`, default 300, `0` = no grouping wait; `PATCH /api/aiops/{tenant_id}`), separate from `aiops_storm_group_window_s`: a workspace that raised the group window keeps a 300 s wait. New audit action `change.approval_lapsed`. New route `POST /api/aiops/{tenant_id}/changes/{change_id}/close-without-verdict` (`approvals.decide`, body `{"reason": "…"}`; `409` until the change has gone 24 hours without a verdict), with the MCP tool `close_change_without_verdict`. An approved change released by a person who did not vote runs again; a change the 0.1.147 dispatch re-check set to `rejected` stays rejected, so its incident needs a new change request. Behaviour that changes on upgrade: an incident on a service that depends, over a dependency allowed to fold, on another service holds its analysis for the grouping wait (`aiops_grouping_wait_s`, 300 s) while its ticket and notification go out as usual, and an alert that clears inside the wait resolves its incident with no analysis (when it comes back it is analysed at once); with a reopen window set, an incident a person closed as known noise is no longer reopened (the next alert on its key opens a new incident, analysed as usual), and a caller whose resolved incident would reopen joins its root's open incident instead. One-shot clean-up of the connection tag scope (**Only findings on resources tagged**): AWS Backup records already open for a resource outside it clear at the next daily read of their region (one `compliance.backup_out_of_scope` audit row; a tag none of the resources read carries clears nothing and the **Backup** card says so); suggested synthetic checks nobody kept whose resource is outside it are removed at the next daily suggestion run, also where suggestions are turned off (named in the `monitor.derived` row); built-in detection pairs already in alarm on such a resource resolve (*outside the workspace scope or the connection's tag scope*), except under a scope that cannot be read or that none of the connection's resources carries, which resolves nothing. Security Hub findings of DynamoDB.1, the deletion-protection controls, ElastiCache.1/.2, FSx.1/.2 and Redshift.3 move family when Security Hub next sends them (daily). The rollback sweep no longer escalates a change that stopped at its backup. The fixes run in the control plane; roll private-network agents as usual. | An approved change released by a second approver reaching `change.executed`; a dispatch refusal's `change.release_refused` row naming its reason; the first `compliance.backup_out_of_scope` row and the **Backup** card's count; a caller incident reading **Held for triage — grouping window: …**; agent version |
| 0.1.149 | Image and chart bump, no values change; no new table or column. The start's schema step no longer scans tables: each statement waits at most 3 s for a table lock and retries (5 tries, the last waiting up to 60 s); with leader election off the start still resumes the runs the restart interrupted before it serves. Once the app serves, the control plane builds seven new indexes with `CREATE INDEX CONCURRENTLY` (`ix_auditevent_action_target_at`, `ix_auditevent_tenant_target_at`, `ix_incident_tenant_key_status`, `ix_incident_tenant_status_resolved`, `ix_investigation_tenant_started`, `ix_knowledgeitem_tenant_name_version`, `ix_changerecord_live_status`; minutes on a large audit trail, writes continue) and runs `migration:null-defaults-v1` once ([Schema changes after the pod is ready](/help/ops-upgrade-and-verify.md#schema-changes-after-the-pod-is-ready)); both need a direct database connection, not a pooler in transaction mode. A workspace without a PII salt gets one at the first start. The chart adds a startup probe (60 × 5 s) before liveness counts. A webhook body over 1 MiB is answered `413` (Alertmanager, Grafana and the generic intake: 8 MiB; `kubernetes-audit`: 16 MiB) and the sender does not retry it: where a source posts larger groups, raise `AIOPS_WEBHOOK_MAX_BODY_BYTES` through `extraEnv` or set Alertmanager's `max_alerts` ([Webhook body cap](/help/ops-upgrade-and-verify.md#webhook-body-cap)). `aiops_change_records` exports open statuses only: a dashboard or alert on `completed`, `rolled_back`, `rejected` or `recorded` reads nothing. New series: `aiops_scheduler_subtick_seconds`, `aiops_migration_pending`. Rollback: the previous image with this chart kept ([Rollback](/help/ops-roll-back-a-release.md)); nothing in the database is undone and the new indexes stay. | The `migrations` block of `GET /api/aiops/admin/overview` with `pending: 0` (every index `valid`, `null-defaults-v1` `done`); `/healthz` version; the pod's time from start to Ready (under 30 s when no run was interrupted); every `413` line in the control-plane log, with its source |


## Next steps

- Next: [Rollback](/help/ops-roll-back-a-release.md)
- [All Install and operate the platform pages](/help/install-and-operate-the-platform/llms.txt)
- [Documentation home](/help/)