Release notes 0.1.116 to 0.1.119

Archived release notes, newest first: 0.1.119 (13 October 2026) back to 0.1.116 (12 October 2026).

Release notes · Updated 2026-10-28 · Describes release 0.1.164

0.1.119 — 13 October 2026

• Pre-approved routes — demoted after one rollback, and an incident whose fix waits stays open (two switches, off by default): with Demote a pre-approved route after one rollback on (Guardrails & cost → Pre-approved auto-fix (L1), aiops_demote_on_rollback), the first rollback of a run through a pre-approved route takes the route's pre-approval away once the rollback is released — its restore pull request opened, its inverse run, or an operator's report that it ran — so the route stops running automatically. The route says when it was demoted, Needs you lists it as Demoted to people who may edit routes (datasources.manage), and the audit log records Pre-approved route demoted after a rollback; tick Pre-approved again, or press Keep it manual. With Keep an incident open while its fix waits for a decision on (Auto-close stale escalated incidents, aiops_hold_for_approval), the quiet-window close skips an escalated incident whose change request is still drafted, proposed, waiting for approval or approved but not run, and its timeline says Kept open — a fix waits for a decision once. Off, a rollback leaves the route pre-approved and the quiet window closes the incident.
• IMDSv2 fix for Security Hub EC2.8, and no platform write without a before-state (switch, off by default): EC2.8 (EC2 instances should use Instance Metadata Service Version 2 (IMDSv2)) gains a prepared fix in every workspace, Require IMDSv2 on an EC2 instance (aws.ec2.require_imdsv2, HttpTokens=required on one instance), released by a person like every direct cloud write (tier L2). The platform first reads the instance's metadata options through the write connection and stores them as a snapshot; the rollback, Restore EC2 instance metadata options, puts them back. Each captured pre-state on a change page now shows its SHA-256 checksum and, when it was stored before the write, says so. With Refuse a change when its before-state cannot be captured on (Guardrails & cost → Change safety, aiops_require_before_state), a direct write the platform cannot snapshot first is never requested, approved or released for platform execution: its review reads Manual route only and lists the commands to run with your own access (a legacy direct route needs a declared snapshot read). A snapshot read that fails while the change runs writes nothing, rejects the change with the reason and keeps the step on the manual route until the write connection is tested again. The read is the provider's own API, so AWS Config need not be on; pull requests and rollbacks are exempt. Off, a write with no before-state still runs and says so.
• Singapore regulated (MAS/PDPA) — model residency without prefixes (switch, off by default): ticked on Guardrails & cost → Model residency (aiops_sg_regulated), it implies a model route policy: every model call must stay in APAC — a Bedrock apac. inference profile, or a single-region Bedrock model in ap-southeast-1 (the ARN's region, else the aws_region_name the gateway states). A global. profile, any other geography or region, an unstated region, an application inference profile and any non-Bedrock model are refused before the call: the run fails closed and its incident goes to a person. Prefixes you wrote still apply. Under the flag, model.routed adds the region and reason for the analysis and RCA models, model.denied adds the region, a run resumed after an approval pause or a restart records its route again (resumed: true), each run of an incident shows its route (investigations[].route on the incident, route on the investigation), and the residency table gains a Region column. It proves the route the platform used, not legal compliance; an apac. profile proves APAC, not Singapore. Whenever a route policy is in force — the flag, or prefixes you wrote — every model call obeys it: a configured fallback (AIOPS_FALLBACK_MODELS) is offered only if the policy allows it, an alias the gateway balances over differing deployments must pass on each of them, and Test on a refused tier makes no call.
• Paging on-call through PagerDuty or Opsgenie — one page per incident (no switch: nothing pages until a target is added): Notifications & on-call adds PagerDuty (Events API v2) and Opsgenie targets beside Teams, Slack, email and webhooks, each taking open, escalate and resolve. The incident id is the page's key, so alerts that join an incident never page again and its escalation re-triggers the same page. The page follows the incident: every close resolves it — a bulk close, a merge, a storm split, a risk acceptance and a passing control too, which send nothing to chat — and a reopen triggers it again. The routing or API key is write-only, kept in the workspace secret store; the portal and the API say only whether one is set. Events go to the vendor's fixed US or EU endpoint, PII-scrubbed like every notification and never held for the chat digest, and each send is in the audit log.
• Alerts that name no service wait in a triage queue (switch, off by default; on by default since 0.1.133): with Send alerts that name no service to the triage queue on (Alert intake & tokens → Intake rules, aiops_unmapped_triage), an alert without a service label — an app, namespace or alert name does not count — opens its own unmapped incident: never grouped with another alert or counted toward an alert storm, and with no analysis, ticket, notification, page or automatic fix. The same alert firing again joins it, or reopens it inside the reopen window, still held. It waits under Needs you → Triage and the Incidents list's Triage filter until someone presses Analyse, which files and announces it as an open would, or Map this source (members.manage): for that webhook source only, a later alert that names no service takes the chosen label as its service — a copy, so the label stays and an alert's own service wins; an account, a region or another place is refused, and the Intake rules card lists each mapping with Remove. New: POST /api/aiops/{tid}/incidents/{id}/map-source (MCP map_incident_source) and the setting source_service_labels. Off, such an alert is grouped by its alert name or namespace and analysed.
• Post-change validation by metric, automatic cloud rollback, a higher severity and one correlation ID (Change safety, all off by default; the higher severity on by default since 0.1.133): Post-change validation rules (aiops_validation_metrics, up to 20) watch a Prometheus, VictoriaMetrics or CloudWatch metric for 1 to 120 minutes after a matching change's write lands (a direct write when it runs, a pull request when it merges); no agent verify runs on that fix meanwhile. A reading above the threshold in every minute of the window (every period for CloudWatch, read three minutes after it closes) fails the fix: the change fails, the rollback rule applies and the incident goes back to a person. The latest reading at or below the threshold validates the fix; no data, a gap or a mixed series ending above it hands the fix to the usual verify. Roll back a failed cloud fix automatically (aiops_auto_rollback_cloud: off, "L1" or "L2") then restores the captured before-state of a prepared cloud fix up to that tier without a person — only after its metric window failed, never at L3, and not while agents are paused, the workspace is deactivated or the licence blocks runs. Reopen a failed fix one severity level higher (aiops_reopen_raise_severity) re-derives the priority and records Severity raised — the fix failed validation (an unranked severity such as page becomes warning). Stamp one correlation ID on every audit row of an incident lifecycle (aiops_audit_correlation) puts the incident id — a change's own id when it has no incident — on its change request, approval (the agent's approval gate included), execution, validation, rollback and reopen rows; GET /api/aiops/{tid}/audit?correlation=<id> filters on it, and the incident and change timelines link there while the switch is on. Rows written earlier carry none.
• Flapping incidents (switch, off by default): with Mark an incident flapping when it keeps reopening on (Intake rules, aiops_flap_threshold: the box writes 3, 0 = off) and a reopen window set (Reopen window (hours), aiops_reopen_window_h, now on the same card; 0 = off by default), the reopen that reaches the count inside the window marks the incident flapping: it keeps its id, no pre-approved automatic fix is released for it (Marked flapping — automatic fix held back), and a problem record is proposed for a person to confirm or dismiss under Problems and Needs you → Problem — or the incident joins the problem already open for its key. Only the alarm firing again counts; a verification that finds the condition still present does not. New: POST /api/aiops/{tid}/problems/{id}/confirm (MCP confirm_problem), and a problem's status can read proposed.
• Risk exceptions — no self-approval and an expiry that acts (two switches, off by default): on Guardrails & cost → Risk exceptions, Never let a requester approve their own exception (aiops_risk_no_self_approval) stops a requester approving their own risk request even as the workspace's only approver (the self-approve button disappears; the API answers 409). Act when an exception expires (aiops_risk_expiry_sweep) has the scheduler act within a minute of the expiry: the decision is stamped expired, the alerts the acceptance quieted fire again, the finding's closed incident goes back to a person (Needs you reads exception expired), a platform check's incident reopens while its evidence still lists the resource, a Security Hub or Defender suppression the acceptance wrote is lifted, and the posture before and after is recorded on the Risk exception expired audit row and the decision. It never touches an active incident, a finding the scanner cleared, a fix awaiting the scanner or a resource another live exception covers, and its reopen never marks a last-known-good record as not held. Off, a sole approver may approve their own request (recorded as self-approved) and an expired exception waits for the scanner's next re-send. The posture page now shows the managed-risk score (scanner-cleared or excluded) beside the scanner-cleared score.
• Post-implementation review owed from submission (switch, off by default; on by default since 0.1.133): with Schedule the post-implementation review when an emergency change is submitted on (Change safety, aiops_pir_at_submission), an emergency request's review is scheduled when it is submitted — requested, voted on or released — and the request shows its due date (AIOPS_PIR_DUE_H, 72 hours by default). The review stays due if the change never lands and records that, can be closed only once the change has landed or been refused, and is withdrawn when the request is reclassified away from emergency before it lands. Off, the review opens when the write lands.
• Roll back the pinned change first (switch, off by default; on by default since 0.1.133): with Recommend rolling back the pinned change first on (Guardrails & cost → Suspect change across a declared dependency, aiops_rollback_first), an incident pinned to a suspected change that can be rolled back leads with that rollback: the summary's next step reads Roll back CHG-… first, a Recommended first card sits above the pipeline and on the change request, and the rollback heads the Probable cause card, the stored report, the stored plan and the agent's instructions. When the change cannot be rolled back the card says why; once the rollback ran or the incident resolved, it steps aside. With the switch on, a harvested ModifyDBParameterGroup or ModifyDBClusterParameterGroup also records each parameter's before value from the newest earlier recorded write of it (never across a reset of the group), and its rollback is the aws rds modify-db-parameter-group (or modify-db-cluster-parameter-group) command with JSON --parameters that sets those values back; a value no write recorded, or one the privacy filter redacted, is named, never guessed. A ModifyDBInstance rollback from its AWS Config before-state lists each changed value and sets back the ones a single flag can. Off, rolling back comes first only for a high-criticality resource when the analysis is unsure.
• Standard change for a runbook route (switch, off by default; on by default since 0.1.133): with Standard change for a runbook route's operations on (Guardrails & cost → Pre-approved operations, aiops_runbook_standard_change), a pre-approved route named remediate-…, scoped by a selector that matches the incident and declaring operations that cover every typed step of the fix — each step also ticked on Pre-approved operations — records the change request as Standard — runbook route, even for a P1 or critical incident. Recording it standard never lowers who must release it: a P1 or critical fix keeps the emergency rules (L3 with two approvers, the release through a closed window at full quorum, the post-implementation review), and the request says so. A human-run or raw command or a file change beside the typed steps keeps a plan out of scope, and a route the first rollback demoted stops counting. In every workspace, a change request an analysis proposes now carries one line saying why it has its class; Reclassify replaces it with the person's note, and the class then reads — set by a person. Off, a P1 or critical fix stays an emergency change.
• Observed and unmanaged changes (two switches, off by default; both on by default since 0.1.133): every change the platform recorded but did not execute (the Changes filter External is now Observed) is worded by who made it: unmanaged (a principal outside the platform, or a change a webhook or operator reported), platform write (the platform's credential, accounted for by a platform record), platform credential (no such record) or platform deployment (a sync, merge or push of a revision a platform change merged); the change page leads with the matching sentence, and change.recorded carries principal, receipt and deploys. A write with one of the workspace's own credentials (a connection's role ARN, access key ID or Azure client ID) is matched within 10 minutes to a platform change on the resource, a route change naming it or its service, an action approved inside an analysis, or a scanner suppression written or lifted at the source. On Guardrails & cost → Unmanaged changes, Flag the platform's credential used without a platform receipt (aiops_change_principal_check) flags such a write that nothing accounts for as unauthorized, and Re-check every control an unmanaged change touches (aiops_change_recheck_mapped) re-runs every compliance check its resource kind maps to (EST-03 for a security group or NSG, EST-02 a bucket or storage account, EST-01 and EST-04 a database, EST-04 a volume, disk or KMS key, EST-01 a VM, EST-05 a trail or diagnostic setting, EST-06 an IAM principal or role assignment) plus every rule it matched, shown as re-checked · EST-0x · <verdict> chips; a re-check for the resource kind alone never names the change as the cause. Off, such a write is recorded with its principal, not flagged, and only its first matched rule is checked.
• A prepared fix stays on the cloud its incident came from (on in every workspace; aiops_cross_cloud_plans, off by default, lets a fix cross clouds): before any write connection is matched, each step of a proposed fix — a Draft revert included — is compared with the clouds and accounts the incident's alerts name: an alert's cloud label, else its source (CloudWatch and Security Hub are AWS, Azure Monitor and Defender are Azure — the incident list's Cloud filter rule), with the account from its account or subscription label, for GCP its project_id. An alert that names no cloud — a ticket, a raised concern, another monitor on the same service — does not move it. A step on another cloud is refused with the reason (Step 1 (azure.vm.start) targets Azure subscription …, but this incident came from AWS account …), even when a write connection covers it, unless aiops_cross_cloud_plans is true; the step then carries a warning. A step on another account or subscription of the same cloud stays ready with a warning above the release button, bound into the reviewed action and shown on the request and approval audit rows. Incidents whose alerts name no cloud, platform checks, Kubernetes and repository steps, and change requests raised from no incident are unchanged.

Upgrade notes

• Every new switch is off by default: for the stories above, off is 0.1.118's behaviour; aiops_cross_cloud_plans is the exception (off, the new cloud guard refuses — see below). A workspace owner (members.manage) turns a switch on from Guardrails & cost, the Intake rules card or PATCH /api/aiops/{tid}/guardrails, whose field is the setting's name without aiops_ (JSON true or false; null restores the default). Six follow a platform default, all off: AIOPS_DEMOTE_ON_ROLLBACK, AIOPS_HOLD_FOR_APPROVAL, AIOPS_ROLLBACK_FIRST, AIOPS_RUNBOOK_STANDARD_CHANGE, AIOPS_UNMAPPED_TRIAGE and AIOPS_FLAP_THRESHOLD=0. Since 0.1.133, AIOPS_ROLLBACK_FIRST, AIOPS_RUNBOOK_STANDARD_CHANGE and AIOPS_UNMAPPED_TRIAGE default on, as do the severity raise, the review at submission and the two unmanaged-change switches (see 0.1.133).
• What each switch does when on: aiops_demote_on_rollback un-pre-approves a route at the first rollback of its run; aiops_hold_for_approval keeps the quiet-window close off an incident whose fix waits for a decision; aiops_require_before_state allows no platform write without a captured before-state; aiops_sg_regulated admits only APAC model routes; aiops_unmapped_triage holds alerts that name no service for triage; aiops_validation_metrics lets a metric window judge a fix; aiops_auto_rollback_cloud ("L1" or "L2") restores a cloud fix whose window failed; aiops_reopen_raise_severity raises a failed fix's incident one severity level; aiops_audit_correlation stamps one correlation ID per incident lifecycle; aiops_flap_threshold (with aiops_reopen_window_h) marks an incident that reopens that often flapping; aiops_risk_no_self_approval refuses a self-approved risk request; aiops_risk_expiry_sweep reopens a finding when its exception expires; aiops_pir_at_submission owes an emergency change's review from submission; aiops_rollback_first puts a pinned change's rollback first; aiops_runbook_standard_change records a runbook route's fix as standard; aiops_change_principal_check flags the platform's credential used with no platform record; and aiops_change_recheck_mapped re-checks every control an unmanaged write touches. aiops_cross_cloud_plans lets a fix act on another cloud; it has no /guardrails field.
• Trying the stories on one workspace: an owner sends one PATCH /api/aiops/{tid}/guardrails with true for demote_on_rollback, hold_for_approval, require_before_state, unmapped_triage, reopen_raise_severity, audit_correlation, risk_no_self_approval, risk_expiry_sweep, pir_at_submission, rollback_first, runbook_standard_change, change_principal_check and change_recheck_mapped, 24 for reopen_window_h, 3 for flap_threshold and "L2" for auto_rollback_cloud; the same fields with null restore the defaults. Tick sg_regulated only once the Model residency table allows every tier: a tier whose model is not an APAC route fails every run that needs it. Three stories need more than a value: a validation rule on the Change safety card, naming an enabled Prometheus, VictoriaMetrics or AWS read connection; a PagerDuty or Opsgenie target with its key under Notifications & on-call; and the runbook route's operations ticked on Pre-approved operations. notifications and preapproved_operations each replace the whole list, so build them from what GET returned. No self-approval needs a second member with approvals.decide.
• PATCH /api/aiops/{tid} checks the new keys: the boolean switches and aiops_cross_cloud_plans take only JSON true, false or null (400 otherwise), aiops_auto_rollback_cloud only "L1", "L2" or null, and aiops_validation_metrics and source_service_labels are validated; a save that flips a boolean switch names it, with its old and new value, in the Workspace settings changed audit row.
• The cloud guard is on from the first start: a prepared fix or revert draft whose step targets another cloud than its incident's alerts name is refused; set aiops_cross_cloud_plans: true where a workspace fixes across clouds. A Defender multicloud or Arc finding counts as Azure, so a fix on the AWS resource it names is refused unless the setting is on, and an account label that names a monitoring scope (a central Log Analytics workspace, a CloudWatch monitoring account) warns on a fix in the resource's own account.
• Route policies you wrote now cover every model call: a configured fallback the policy refuses is no longer offered, an alias the gateway balances over differing deployments is refused when one deployment fails the policy or the gateway does not list them, and Test refuses a refused tier. A fallback the gateway's own router configures is invisible to the platform: keep that router inside the policy's geography. With no policy and no flag, nothing changes.
• Roll private-network agents to 0.1.119 before ticking Singapore regulated: the flag is enforced where a run is built, and the runner configuration now forwards it, but an agent older than 0.1.119 does not know it and enforces only prefixes you wrote. Agents report no version, so while the flag is on the Model residency card names a live private agent for you to check.
• Update the AWS write role for the IMDSv2 fix: re-run the CloudShell setup, or update deploy/aws/policies/aiops-write-inline.json, for ec2:ModifyInstanceMetadataOptions on instances tagged aiops-managed=true and ec2:DescribeInstances (the snapshot read); until then the fix cannot run.
• Review AWS changes in flight again: the two new AWS operations change every AWS write connection's capability digest, so an AWS change request waiting for its second approval, or an AWS review open in a browser, must be reviewed and approved again after the upgrade, as with earlier operation additions.
• EC2.8 findings start analysis runs: with a prepared fix, a pulled EC2.8 finding below P2 is no longer held as "no prepared fix"; where automatic compliance analysis is on it starts one paid analysis run, as S3.1 and IAM.7 do.
• PagerDuty and Opsgenie keys are tenant secrets: the PagerDuty routing key (Events API v2 integration key) or Opsgenie API key is stored write-only in the workspace secret store under the target, like connection secrets (encrypted when AIOPS_TENANT_SECRET_KEY is set), never in the settings or an API answer; a target sent back with its id and no key keeps its key, and a removed target's key is deleted. Only the control plane sends pages, so it needs outbound HTTPS to events.pagerduty.com (EU events.eu.pagerduty.com) or api.opsgenie.com (EU api.eu.opsgenie.com). Atlassian has announced Opsgenie's retirement; the target works while its Alerts API does.
• Observed changes are reworded in every workspace: the filter and badges above, meta.principal and meta.deploys on observed rows and the principal, receipt and deploys keys on change.recorded change no decision. A later sighting never fills in who made a write, so the first harvest after the upgrade does not re-judge rows recorded before it.
• API clients: Needs you adds the types unmapped_signal, problem_proposal and route_demoted and the reason exception_expired, and GET /guardrails answers the new fields; a client that switches over them must take the new values.
• Schema: additive, no backfill — columns remediationroute.demoted, riskdecision.expired_at and incident.flapping; problemrecord.status takes the new value proposed.

0.1.118 — 13 October 2026

• Model prices — the publisher's list price for every model not served through Bedrock: Fetch list prices (was Fetch official prices) and the daily sync now also read models.dev, an open MIT-licensed dataset of model prices, for every model the platform does not call through Amazon Bedrock; Bedrock models keep the AWS Price List. Only each publisher's own entry is read (OpenAI, Anthropic, Google, DeepSeek, Z.ai, Mistral, xAI, Moonshot AI, Alibaba Cloud, Meta), never a reseller, a host, a $0 subscription plan or a CNY list. Each priced publisher model becomes a dated, append-only catalogue entry, and an alias reaches it through a dated mapping, so one publisher change re-prices every alias mapped to it from that read on. The reviewed router table maps the platform gateway's cx/ (OpenAI), glm/ (Z.ai), ds/ (DeepSeek) and cc/ (Anthropic) aliases by itself and follows a re-pointed backing. A failed read keeps the last good prices and retries within the hour; a $0 is never applied and nothing is re-booked.
• Model prices — every other model is mapped by a person: Needs attention → Map suggests the publisher model and preselects only what the gateway's own configuration says (the router table, model_info.base_model, a native backing), which is also all Map all suggested applies; what an endpoint said served the calls, or a match by name, is listed for you to check and never picked. A change beyond 2× either way in any rate, or a first mapping that would move an alias's price that far, waits for Confirm. A manual price with a source note still outranks both sources; Replace with the list price (was Replace with the AWS price) hands the model back.
• Model prices — gaps are flagged, not filled in: DeepSeek lines (models.dev lists only the off-peak rate; DeepSeek bills twice it at peak hours), lines whose publisher bills reasoning at another rate and a call over a long-context threshold book as estimate. Dated prices models.dev cannot hold, such as GPT-5.6 Sol's promotion "at least through November 21, 2026", are a reviewed watch list that Needs attention raises from its review date.
• Model prices — a workspace's own AI endpoint: its models are priced at their publisher's list price through a mapping the platform operator records (or the operator's Own rates, or No token charge), never at what the endpoint reports, $0 included, and no longer at the rate card the workspace typed, which keeps pricing only the runs from before the upgrade. That holds for a model the endpoint reaches through Bedrock too: the platform cannot see where an endpoint sends a call. An unmapped model books $0 as Not priced yet, unseen by the per-run cost cap until it is mapped. Workspace endpoints on the price map lists each such workspace's endpoint models with their runs, money and the model the endpoint said served them, with Map and Re-price history (dry run); the workspace reads each model's price on AI models, and each mapping lands in its audit log without the operator's note. prices sent with an endpoint save are stored but price nothing.
• Model prices — what a workspace reads: cost lines name their source — OpenAI list price · gpt-5.6-sol · models.dev, read 14 Oct 2026, No token charge (set by the platform operator), Your rate card (runs before …) — and a DeepSeek or long-context estimate reads as a list price that states part of the bill. The statement footnote names the month's price sources once, Re-priced on says when its history was last re-priced, and a workspace on its own endpoint reads Served by your endpoint. The ops Billing page flags tokens with no price (Not priced — resolve before invoicing), and every billing CSV ends with a new unpriced_tokens column.
• Model prices — re-pricing and the API: POST /api/aiops/admin/rebook-costs takes models (only those lines move), include_test (Test Lab runs too) and, with tenant_id, include_lab_boxes; a workspace on its own endpoint is no longer skipped but re-prices through its mappings (prices_source replaces byo_skipped). New: POST /api/aiops/admin/model-prices/mappings (MCP record_model_mappings); the sync takes confirm. Fill from gateway pricing is gone: PUT /admin/model-prices refuses openrouter-estimate, and the gateway model list carries ids only. Usage lines record the model the endpoint said served them, as evidence for a mapping, never a price.
• Gateway entries that pin a model id: for a gateway entry that pins litellm_params.model_id (an application inference profile or a provisioned-throughput ARN), the wrap-up's output cap, the Claude check that turns prompt caching on and the input window the overflow guard reads now come from the model the entry serves (litellm_params.model). In 0.1.117 such an entry lost its model: a Claude 3.5 Sonnet v2 alias was asked for 32,768 output tokens against its 8,192 limit, and neither cache marks nor the overflow guard applied. Pricing and the model route policy still judge the pinned target.
• A gateway's own estimate is named as the gateway's: a cost litellm takes from the platform LiteLLM gateway's x-litellm-response-cost header is still booked as litellm-estimate, but its price_ref now reads gateway x-litellm-response-cost instead of this release's litellm version: the gateway priced it from its own table, not the one this release pins.
• Built-in detection — stopping to watch is not a clear: when detection stops watching an alert (Turn off, a rule switched off, a connection opted out, a resource leaving the workspace scope), its incident still resolves, now with a resolution note saying nothing was seen to clear. 0.1.117 read those resolves as the source clearing: the incident got a recovery time, quiet expiry and the post-close verification could act on it, and the ticket write-back moved the ticket to Done. Now no clear time is recorded, the scheduled post-close verification skips the incident, and the write-back says why and leaves the ticket open: a ticket system preset adds the note, and an advanced route waits for a person (Write back). The note belongs to that close only: once the incident leaves resolved, a later real clear closes the ticket as usual.
• Built-in detection — a held warning stays off a ticket filed earlier: when a held warning re-opens an incident that was already announced and ticketed (it had an analysis), each clear of that warning no longer writes back to (and re-closes) the ticket, and Analyse or a lifted hold no longer files or announces the incident a second time. Only an incident that was never announced is filed and pushed when its hold lifts.
• Tickets — a lifted hold's filing survives a restart: Analyse on a quietly held built-in incident now records the pending filing in the same commit, as ingest does, and the five-minute ticket sweep ages a pending filing or an uncertain failed one by its own timestamp instead of the incident's open time. A warning held for days and then lifted is still filed when the control plane restarts before the filing runs.
• The incident page says a held warning is why there is no ticket: L3 reads No ticket yet: this built-in warning is held — no ticket and no notification until someone presses Analyse or an unheld alert joins it. Analyse files the ticket. instead of saying the incident was opened another way. The incident API's ticket_cover gains the value held.
• MCP — tools brought up to date: the platform server's runner-token tools (list_runner_tokens, mint_runner_token, revoke_runner_token) failed on every call and work again, taking tenant like every other tool. Uploads carry the documented 16 MiB (the transport cut anything over about 3 MiB with a bare 413). tenant also reaches your Test Lab workspaces and, on the organization tools, an organization's workspaces for its admin, as REST does. New tool export_xray_ontology returns a scan's ontology as Turtle; list_findings passes X-Accounts through; export_history_golden_set is marked a write (it records an audit row); serverInfo.version names the build; several descriptions now say what their route does (list_pending_approvals points at needs_you, a patch_datasource config replaces the whole config). Creating a Test Lab workspace (REST and MCP) no longer returns its raw settings with their PII salt.
• Members & roles — a role grid you can read at a glance: in Role definitions an allowed permission is a check on a solid disc and a missing one a grey cross on a pale disc — two shapes at one size, readable in greyscale, in both themes and printed — and each mark's tooltip names the role and the permission.
• Guides: the incident guide lists three moves out of resolved (a held built-in warning firing again inside the reopen window re-opens the incident into open, held again, with no analysis), and the ticket system and data source guides state the two built-in detection exceptions: a held warning files its ticket only once the hold lifts, and an incident detection stopped watching keeps its ticket open. The platform console's gateway chapter explains publisher list prices, mappings, gaps, workspace endpoints, the re-book filters and how prompt caching is priced on Bedrock; the portal guides say how an endpoint's models are priced and what Not priced yet and Re-priced on mean; the cost-management guide and the installation runbook name Fetch list prices and the models.dev egress. The MCP reference is brought up to 0.1.117 (release rights, needs_you, ticket tools, connection job switches, prompt settings, retention) and says both servers speak Streamable HTTP only: a client set to sse waits until its connect timeout. The connect guide also explains a 413 from the transport.

Upgrade notes

• Press Fetch list prices after the deploy: the daily sync keeps 0.1.117's clock, so otherwise the first models.dev read can wait a day. It records the catalogue and maps the router table's aliases; a mapping that would move a price beyond 2× or off $0 waits for Confirm, an alias with a manual price keeps it until Replace with the list price, and until an alias is mapped whatever priced it in 0.1.117 still does.
• Map what the table does not name from Needs attention → Not mapped to a publisher model, or record a manual price with its source. A mapping cannot start before its model's first catalogue entry (the first read): to price earlier runs at the publisher's rate, record a manual entry from the publisher's date, press Replace with the list price, then re-book.
• Tell workspaces on their own endpoint before the upgrade: the first 0.1.118 control-plane start retires their typed rate cards (each keeps pricing the runs from before that moment). After it every endpoint model books $0 as Not priced yet, unseen by the per-run cost cap, until it is mapped under Workspace endpoints; then re-price with Re-price history (dry run). A line an older agent books at the endpoint's own figure lands at $0.
• Re-pricing stays explicit: nothing moves booked runs by itself; run POST /api/aiops/admin/rebook-costs as a dry run first, with models (and include_test where the Test Lab ran them). A script that read byo_skipped reads prices_source now.
• Egress: the control plane needs HTTPS to models.dev as well as pricing.us-east-1.amazonaws.com (runners call neither); without it, set AIOPS_MODELS_DEV_PRICES=false (the AWS part keeps running) and record manual prices.
• Gateway-listed prices are refused: PUT /api/aiops/admin/model-prices answers 400 to source: "openrouter-estimate", and the gateway model list drops pricing and prices; record manual prices with a note instead.
• Billing CSVs end with unpriced_tokens (empty for a month sealed before it was counted): a parser that counts columns must take it.
• Agents older than 0.1.118 book DeepSeek, reasoning-rate and long-context lines at the plain list price, not as estimates, until they are upgraded.
• A close that stopped watching records no clear time: the report's time to resolution reads such an incident's resolve time, and an escalated incident detection stops watching stays with a person (quiet expiry no longer closes it). Tickets that 0.1.117 already closed this way stay closed.
• Schema: additive, no backfill — table modelmapping (append-only); the platform setting byo_cards_retired_at is stamped once at startup.

0.1.117 — 13 October 2026

• Ticket systems — Jira Cloud, ServiceNow and Azure DevOps without hand-written routes: a Jira, ServiceNow or Azure DevOps connection gains a Tickets action (a green Test offers Use for tickets). Check reads the identity, project, permissions, issue or work item type, priorities and close path without writing anything, and Send test ticket files a real ticket and closes it within seconds. Each Incident-agent incident then gets one ticket for its life, with a link back (once the public URL is set) and the system's own priority; on resolve, the RCA, the pull requests and the validation verdicts are written back and the ticket is closed the way the system expects — a Jira internal comment, then the workflow's own Done transition; a ServiceNow resolve by sys_id, read back; the Azure DevOps Resolved (else Completed) state. Every resolve now closes it, not only a person's resolve and the post-fix check: the source clearing, quiet-expiry, the agent's own resolve, a merge and a storm split, with a five-minute sweep as the backstop. Failures read Ticket filing failed: <reason> with Retry; a write that may have landed is searched for before any retry, so a ticket is never filed twice. The platform's own tickets coming back through an inbound webhook are dropped as ticket_echo, and aiops_ticket_operations_total counts filings and write-backs. Test on Jira and ServiceNow now proves the credential, advanced routes keep working with new placeholders (${incident_priority}, ${incident_url}, ${incident_rca_wiki} and more), and computer-assist mode writes nothing to a ticket system. Guide: Ticket systems — Jira Cloud, ServiceNow, Azure DevOps.
• Ticket systems — a customer's own ticket keeps its status: when an incident came in from a ticket (an inbound Jira, ServiceNow or Azure DevOps webhook), the platform writes the analysis to that ticket as an internal note and never transitions, resolves or closes it. An advanced route on such a ticket, one request that may close it, runs only when a person resolves the incident (Mark resolved, bulk close, accepting a merge proposal) or the post-fix verification confirms the fix; any other resolve sends nothing, and the incident says the ticket was raised by its owner and offers Write back.
• Built-in detection (AWS) — alerts for accounts with no alarms, off by default: a workspace owner turns it on under Datasources → Built-in detection (AWS): 25 deterministic CloudWatch rules (R01–R25) over what the estate inventory lists — EC2 and EBS, RDS, load balancers and target groups, Lambda, API Gateway, SQS, DynamoDB, EFS, OpenSearch and NAT gateways — each linked to the source of its threshold. It only reads (cloudwatch:ListMetrics, cloudwatch:GetMetricData), creates nothing in the account, and no model takes part until an incident opens. A 24-hour shadow period records would-have-fired episodes and opens nothing; then the workspace arms itself, and what is already in alarm opens with no analysis (a critical one still files its ticket and notifies). After that, critical rules open an incident and start the analysis; warning rules open it held — no analysis, ticket, push or paid recheck — until someone presses Analyse, a critical rule or another source's alert joins, or the owner lets warnings analyse. A rate needs 10 requests a minute, an episode clears after three clean polls, alerts carry the rule, the resource and the CloudWatch dimensions as labels, and last-known-good reads the rule's own datapoints. The owner can Arm now, Keep shadow, switch rules off or Turn off, which resolves every built-in alert still active; platform staff have a freeze switch on Platform defaults. The AWS inventory now also lists EFS file systems, OpenSearch domains, DynamoDB tables, SQS queues, NAT gateways and API Gateway APIs, tagged or not. Guide: Built-in detection (AWS).
• Model prices — official prices with their source, estimates labelled: Platform defaults → Model price map → Fetch official prices reads the public AWS Price List for a Bedrock model the gateway serves — in the region its gateway entry names, global or regional rows by inference profile — for the models a reviewed table maps (today Claude Sonnet 5 and 5.5, Claude 3.5 Sonnet v2 and Amazon Nova Pro); any other model takes a manual price with a source note. A daily sync repeats the read for the models in use and never replaces a manual price unless asked. Every price keeps its source, who set it and the date it applies from, and each run is priced at the entry in force when it started; to correct the past, date a price back and run the re-book, which reports moved money apart from relabelled runs and never re-prices what a workspace's own gateway booked. Needs attention lists the models in use without a trusted price or a recorded source (Record source). A model with no published or contract price is still costed, so budgets and the per-run cap keep working, but Usage & billing, the incident Spend card and the report say which part of a figure is an estimate; Billing marks each workspace's estimated part, every billing CSV gains estimated_usd, and the model library's own price table is now the one the release ships. The platform gateway's bedrock/claude-sonnet alias is now priced in the deployment's map at the AWS list rate for ap-southeast-1 (2.00 in, 10.00 out, 0.20 cache read per million tokens).
• Cache writes: Claude on Bedrock charges about 1.25 times the input rate to write a prompt into its cache. Cache-write tokens are now recorded and booked at the cache-write rate (at the input rate, flagged, when none is set), shown as Cache writes in every cost split and as a column of Usage & billing → By model, beside a Price column naming each model's price source.
• Prompt cost — caching, an overflow guard and a measured diet: on by default, each agent loop call to a Claude route marks the system prompt and the newest step with cache_control, so the next call reads that prefix at the cache-read rate (0.1× input on Claude Sonnet 5); the text the model reads does not change, apart from a repeated-call reminder that becomes a user note. Claude is judged on the model the gateway routes an alias to; OpenAI-family routes, a workspace's own gateway and a route with a non-Claude fallback send no marks, calls nobody reads again are never marked, and a provider that refuses the marks gets the call again without them. A prompt too large for the routed model's known window now gets the diet's budgets for that run instead of failing, with a routing note in the transcript. The prompt diet itself — bounded skill catalog, estate, routes, blast radius and planning blocks, with a search_skills tool — stays off until a Test Lab A/B passes: a benchmark run takes prompt_diet: true or false, and compare says when the diet is the only difference. Also fixed: knowledge and routes render in a stable order, a runbook's frontmatter description: shows in the skill catalog, and an estate over 100,000 characters is cut at a line end with an [estate cut] line and an estate_lookup tool instead of mid-line.
• Remote runners follow the platform default tier map: a workspace with no tier map of its own now runs on Platform defaults → Default model tiers on the platform's own runners too: an api-mode runner receives the map with every job, and a db-mode runner re-reads it, with the price card, each time it polls, so a save applies to the next job. A model route policy that denies the platform model fails the run closed there as on the control plane. A private-network agent the customer runs keeps the workspace's tier map, else its llm.model.
• Review fixes — incidents, findings and approvals: an alert held at intake for missing identity labels keeps its incident open, says held at intake, and is rechecked from its own alerts at no cost. Alerts of the webhook call that opened an incident no longer count as late, and L4 flags only late alerts the finished run's report does not name. A released posture fix stamps its scanner findings at the write, so an early re-send no longer reopens the incident (a still-present verdict or a rollback takes the stamp back), and a platform rollback's audit-log echo confirms its change instead of appearing as an outside change. Under a closed change window the first of two approvers can vote from the incident page, and that vote reads Approval recorded; L5 says The approved change ran for a prepared plan, and a requested plan keeps its Reused from the incident investigation's wrap-up line. An expired risk request leaves Needs you and no longer offers Approve.
• Review fixes — what the incident page says: an alert that fires again inside the reopen window no longer reads as a failed verification (the Timeline says Reopened and the alert fired again inside the <n> h reopen window), later alert-started runs are badged alert re-fired, and a reopened incident's recurring chip no longer links to itself. A close by the post-fix check reads The post-fix check confirmed the fix; a source-closed incident says its next alert opens a new incident or re-opens this one inside the reopen window; a skipped hand-off reads skipped, not Handed to a human. A primary_alert returned as an object or a list reads <alertname> · <service>, and the open-time monitor read is labelled Monitor history checked.
• Guides: new Ticket systems and Built-in detection (AWS) guides; the incident guide covers the identity hold, the reopen window and the release stamp; the Alert correlation columns and the grouped alerts (now L1 and L2) match the page; the platform console guides cover official prices, estimates, token reconciliation (python -m app.aiops.price_sources reconcile) and prompt caching.

Upgrade notes

• The platform's model gateway restarts once: this release's gateway configuration states aws_region_name on the Bedrock Claude Sonnet entry and moves its config-rev, so the LiteLLM gateway restarts for a few seconds when it syncs; let it sync between runs. A gateway you run yourself must state aws_region_name on each Bedrock entry for that model to be priced from AWS; with no gateway, a direct bedrock/… model needs AWS_REGION on the control plane.
• Price the models after the deploy: the leader runs the first official price sync by itself and then daily (a failed one retries after an hour); a superadmin can press Fetch official prices at any time. Then record a source (Record source) for the rates the deployment's own price map still sets, and run the re-book as a dry run first: the first re-book relabels the runs that map priced (unattributed) and moves no money unless you back-dated a price. Without outbound HTTPS to pricing.us-east-1.amazonaws.com, set AIOPS_PRICE_SYNC_INTERVAL_S=0 and enter manual prices.
• A manual price needs a source note: the editor and PUT /api/aiops/admin/model-prices refuse a changed, new or removed price without a note, an empty map and an effective_from in the future; a PUT that sends the version from GET is refused (409) when the card changed since; the answer carries priced and version instead of registered. A script that changes the map must send a note.
• Estimates show on past months where runs were booked from the model library's own table, and cache writes book at the input rate until a model's price has a cache-write rate.
• Prompt caching is on by default for Claude routes (AIOPS_PROMPT_CACHE=true, workspace aiops_prompt_cache): the cached prefix — the system prompt with its knowledge and estate, and the newest steps — stays in the provider's cache for its lifetime (five minutes on Bedrock). Set aiops_prompt_cache: false on a workspace where that is not acceptable.
• The prompt diet stays off (AIOPS_PROMPT_DIET=false, workspace aiops_prompt_diet) until a Test Lab A/B on one pinned model shows the same answers; do not turn it on for a workspace before that.
• Runners and older images: api-mode runners receive both prompt settings and the platform tier map with every job, and the runners the control plane provisions get the prompt settings in their environment; a runner you run yourself (the compose agents profile) needs them set, and an agent image older than 0.1.117 ignores them. The control plane and the runners now read the gateway's /model/info at every run start (cached for five minutes). A workspace with no tier map can change model on the platform's api-mode runners (the platform map instead of the runner's AIOPS_MODEL), and one whose route policy denies the platform model now fails closed: give it its own tier map.
• Test Lab fingerprints can change once: they hash the system prompt, which this release orders and corrects, so re-run a baseline before comparing with a run from an earlier release.
• Ticket routes you already have: an advanced ticket route keeps working and now also writes back on the source clearing, quiet expiry, a merge and a storm split — except on a customer's own ticket, where it still runs only on a person's resolve or a validated fix (Write back otherwise). An incident that adopted a ticket before 0.1.117 is recognised by its alert's ticket_ref or ticket_url, and a ticket filed before 0.1.117 still closes through a route whose connection points at the same site.
• ServiceNow inbound rule: add correlation_id and correlation_display to the record the business rule posts to /webhook/servicenow/<slug>; without them a ticket the platform is still filing can come back as a second incident. The ServiceNow preset is so far checked only against recorded ServiceNow API responses: prove each system with Send test ticket.
• Jira and ServiceNow Test now checks the credential, so a connection that showed green on a login page can turn red after the upgrade.
• Built-in detection is off and costs nothing until a workspace owner turns it on. The platform defaults are environment settings: AIOPS_BUILTIN_DETECTION_POLL_S=300, AIOPS_BUILTIN_DETECTION_MAX_RESOURCES=300, AIOPS_BUILTIN_DETECTION_DELAY_S=600 and AIOPS_BUILTIN_DETECTION_SHADOW_H=24. From the first bad minute to an alert takes the 10-minute delay, the rule's window and up to one poll — about 11 to 30 minutes (a native CloudWatch alarm takes about 5). A rate rule needs 10 requests a minute, so a quiet load balancer or function failing every request never fires; an NLB target group with no registered targets is not seen; and under the CloudShell read role only tagged Lambda functions are watched.
• Update the AWS read role: re-run the CloudShell setup, or update the inline policy deploy/aws/policies/aiops-read-inline.json, for the new inventory list reads (EFS, OpenSearch, DynamoDB, SQS, NAT gateways, API Gateway /restapis and /apis). The inline policy also names cloudwatch:ListMetrics and cloudwatch:GetMetricData and denies API Gateway key values, and the agent pauses for approval before reading them. Until the role is updated the harvest lists the reads it was refused, and the detection card names a missing CloudWatch permission; nothing else breaks. The first harvest after the upgrade shows every RDS instance and every newly listed resource as changed or new.
• Renamed labels and a stricter setting: the Timeline labels Approval released, Last-known-good recorded and Reopened — condition still present now read Approval recorded, Monitor history checked and Reopened (the audit action keys are unchanged). An identity-held incident stays open. PATCH takes aiops_require_identity_labels only as JSON true, false or null; a value saved earlier as a string is still read as on, so save it again.
• Schema: additive, no backfill — tables modelprice and detectionstatus, columns incident.external_ticket and benchmarkrun.prompt_diet.

0.1.116 — 12 October 2026

• The control plane no longer runs out of memory on a long-harvested estate: every inventory harvest and every absorbed cloud change saves a new version of the estate item, and each save loaded every earlier version of that item with its whole entity list. After a few hundred harvests of a busy account that was gigabytes per save, and the control plane restarted for memory about every 15 minutes. A save now reads only the version in use and the next version number. The estate x-ray context review, the knowledge export, the import preview and the coverage panel no longer load superseded versions either. Upgrade any installation that harvests a busy account to 0.1.116.
• Knowledge history lists the newest 20 superseded versions of each item: with Show inactive versions and drafts on, a history row carries its provenance and its Disable stamp but no longer the entity list; the version in use still carries everything. Older versions stay stored; only the list stops at 20 per item.

Next steps

• Next: Release notes 0.1.100 to 0.1.115
• All What's new pages
• Documentation home