SLO objectives on groups of operations

Operations guide: configure, deploy and observe the gateway. What the capability does, product side: product guide; calling it: developer guide.

Why - a team does not reason one operation at a time: it promises an availability per service ("the events API: 99.5 %"), holds a few critical operations, follows journeys ("sign up", "pay") and shows a headline figure. Since 1.6.0, an objective is declared on a group of operations, in the same slos file as an operation's: one budget, one burn rate and one slo-burn alert for the group, instead of twenty budgets without an overall view. Nothing changes in the gateway: no series, no label, no x-flexway-* extension more - the console reads the existing series with a filter on operation.

Not complete yet - group objectives are served: declaration, computation in both metric sources, slo-burn alert, console screens, covered by unit and contract tests. Still to come, with no announced version: the end-to-end proof that a group burns under an injected outage and that the alert arrives, and the measurement of the evaluation cost. Until then, the ceiling of 100 objectives is provisional.

# The console's values
slos:
  # 1.5.0, unchanged: a critical operation (its identifier is the operationId)
  - operation: createOrder
    latency: { p95: 300ms, target: 99% }
  # A service: every operation whose x-flexway-upstream names this upstream
  - name: events-api
    upstream: events-svc
    availability: 99.5%
  # An OpenAPI tag (the operation's tags), minus a probe
  - name: results
    tag: results
    exclude: [healthz]
    availability: 99%
  # A journey: an explicit list, the latency judged on each step
  - name: register
    operations: [createAccount, verifyEmail, createProfile]
    availability: 99.5%
    latency: { p95: 800ms, target: 99%, per: member }
  # The headline figure: every operation served, the built-in ones included (ai.*)
  - name: global
    all: true
    exclude: [healthz]
    availability: 99.9%
    window: 30d

Key	Rule	Default
Selector	Exactly one per entry: operation, or name and upstream, tag, operations or all: true - none or two refuse the file	-
name	Required for a group, refused with operation; [a-z0-9][a-z0-9-]{0,62}, unique among the names and the operations declared	-
upstream	An upstream of upstreams in flexway.yaml: its declared operations (x-flexway-upstream), never the series' upstream label - a cache hit counts there under _none	-
tag	An OpenAPI tag; a gRPC method has none	-
operations	2 to 50 operationId, no duplicate; a single one is written operation	-
all	true only: every operation served, ai.* included	-
exclude	With upstream, tag or all: at most 50 operationId taken out of the group (probes, administration)	none
latency.per	group: the group's p95 (buckets summed); member: each member with traffic, the slowest judges the minute	group
window	1 to 90 d	30d
Ceiling	100 objectives, one per signal declared; a group counts for one - provisional until the evaluation cost is measured	100

How it is computed - a group's availability is weighted by the requests (1 - Σ 5xx / Σ answers over all its members): a high-volume member weighs what its requests live through, a member without traffic counts neither for nor against. The latency is the share of minutes whose p95 holds the target - the group's, or the slowest member's with per: member. Budget, burn rate (1 h, 6 h, 3 d) and the slo-burn thresholds are an operation's objective's. The members are today's over the whole window: an operation added to the contract under the tag enters the group at the next reload of /config (journal console.alert.slo_members_changed), without history before it. A journey measures its requests, never the product of its steps' availabilities: set the target accordingly.

On screen - Traffic, Objectives (SLO) tab: the objectives by level (global, services, journeys, operations), a row opens the objective's panel - who spends the budget (each member's share), members and their last move, declaration, "Create an alert". The Overview carries the all objective at its head; the Operation screen lists the groups it is a member of. Admins only: an objective is computed over every tenant (403 before any read for a viewer).

Alerting - slo-burn already covers every group; its subjects are the objective identifiers (the operationId, or a group's name). To notify other webhooks for one objective:

alerts:
  rules:
    - { name: checkout-burn, kind: slo_burn, objective: register, notify: [ops-pager] }

Compatibility - a 1.5.0 file loads identically. Two changes, said in the release notes: an operation entry naming an operation absent from /config is now refused when a changed file loads (it was accepted and never computed); in the webhooks, slo-burn's target.kind is objective and each slos[] entry of a summary carries objective, operation being null for a group - a strict consumer must accept that null.

Symptom	Cause	What to do
Alerts or Objectives screen: Last version of the file refused with upstream ... is not in the gateway's configuration	The changed file names an upstream, a tag or an operation the gateway does not serve	Correct the name, or deploy the gateway first; the previous version applies meanwhile
An objective Invalid, in a banner	The gateway's configuration changed under an accepted file (upstream removed, operation renamed), or the group is empty once exclude applies	Correct slos or restore what it names; the objective is never computed on an amputated group
the group has no operation once exclude is applied	exclude removes every member	Shorten exclude
a group's name is an operation's too	A name equal to a declared operationId	Rename the group
The global objective does not move despite an operation's outage	It weighs little in the volume: the availability is weighted by the requests	Read each member's share in the panel, and declare an objective on the operation or a narrower group
Nothing at the head of the Overview	No all: true objective, or a viewer's session	Declare a global objective; a viewer never sees it