# SLO objectives on groups of operations

> Why - a team does not reason one operation at a time: it promises an availability per service ("the events API: 99.5 %"), holds a few critical operations…

Guide - Since version 1.6.0 - Flexway v1.6.3 - https://flexway.io/en/docs/operations-slo-groups/

> Operations guide: configure, deploy and observe the gateway. What the capability does, product side: [product guide](https://flexway.io/en/docs/product-overview/); calling it: [developer guide](https://flexway.io/en/docs/developer-overview/).

**Why** - a team does not reason one operation at a time: it promises an availability **per service** ("the
events API: 99.5 %"), holds a few **critical operations**, follows **journeys** ("sign up", "pay") and shows
a **headline figure**. Since `1.6.0`, an objective is declared on a **group of operations**,
in the same `slos` file as an operation's: one budget, one burn rate and one `slo-burn` alert for the group,
instead of twenty budgets without an overall view. Nothing changes in the gateway: no series, no label, no
`x-flexway-*` extension more - the console reads the existing series with a filter on `operation`.

> **Not complete yet** - group objectives are served: declaration, computation in both metric sources,
> `slo-burn` alert, console screens, covered by unit and contract tests. Still to come, with no announced
> version: the end-to-end proof that a group burns under an injected outage and that the alert arrives, and
> the measurement of the evaluation cost. Until then, the ceiling of 100 objectives is provisional.

```yaml
# The console's values
slos:
  # 1.5.0, unchanged: a critical operation (its identifier is the operationId)
  - operation: createOrder
    latency: { p95: 300ms, target: 99% }
  # A service: every operation whose x-flexway-upstream names this upstream
  - name: events-api
    upstream: events-svc
    availability: 99.5%
  # An OpenAPI tag (the operation's tags), minus a probe
  - name: results
    tag: results
    exclude: [healthz]
    availability: 99%
  # A journey: an explicit list, the latency judged on each step
  - name: register
    operations: [createAccount, verifyEmail, createProfile]
    availability: 99.5%
    latency: { p95: 800ms, target: 99%, per: member }
  # The headline figure: every operation served, the built-in ones included (ai.*)
  - name: global
    all: true
    exclude: [healthz]
    availability: 99.9%
    window: 30d
```

| Key | Rule | Default |
| --- | --- | --- |
| Selector | **Exactly one** per entry: `operation`, or `name` and `upstream`, `tag`, `operations` or `all: true` - none or two refuse the file | - |
| `name` | Required for a group, refused with `operation`; `[a-z0-9][a-z0-9-]{0,62}`, unique among the names **and** the operations declared | - |
| `upstream` | An upstream of `upstreams` in `flexway.yaml`: its **declared** operations (`x-flexway-upstream`), never the series' `upstream` label - a cache hit counts there under `_none` | - |
| `tag` | An OpenAPI tag; a gRPC method has none | - |
| `operations` | 2 to 50 `operationId`, no duplicate; a single one is written `operation` | - |
| `all` | `true` only: every operation served, `ai.*` included | - |
| `exclude` | With `upstream`, `tag` or `all`: at most 50 `operationId` taken out of the group (probes, administration) | none |
| `latency.per` | `group`: the group's p95 (buckets summed); `member`: each member with traffic, the slowest judges the minute | `group` |
| `window` | 1 to 90 d | `30d` |
| Ceiling | 100 objectives, one per signal declared; a group counts for one - provisional until the evaluation cost is measured | 100 |

**How it is computed** - a group's availability is **weighted by the requests** (`1 - Σ 5xx / Σ answers` over
all its members): a high-volume member weighs what its requests live through, a member without traffic counts
neither for nor against. The latency is the share of minutes whose p95 holds the target - the group's, or the
slowest member's with `per: member`. Budget, burn rate (1 h, 6 h, 3 d) and the `slo-burn` thresholds are an
operation's objective's. The members are **today's over the whole window**: an operation added to the
contract under the tag enters the group at the next reload of `/config` (journal
`console.alert.slo_members_changed`), without history before it. A journey measures **its requests**, never
the product of its steps' availabilities: set the target accordingly.

**On screen** - Traffic, **Objectives (SLO)** tab: the objectives by level (global, services, journeys,
operations), a row opens the objective's panel - who spends the budget (each member's share), members and
their last move, declaration, "Create an alert". The **Overview** carries the `all` objective at its head;
the **Operation** screen lists the groups it is a member of. Admins only: an objective is computed over every
tenant (`403` before any read for a viewer).

**Alerting** - `slo-burn` already covers every group; its subjects are the **objective identifiers** (the
`operationId`, or a group's `name`). To notify other webhooks for one objective:

```yaml
alerts:
  rules:
    - { name: checkout-burn, kind: slo_burn, objective: register, notify: [ops-pager] }
```

**Compatibility** - a `1.5.0` file loads identically. Two changes, said in the release notes: an `operation`
entry naming an operation absent from `/config` is now **refused** when a changed file loads (it was accepted
and never computed); in the webhooks, `slo-burn`'s `target.kind` is `objective` and each `slos[]` entry of a
summary carries `objective`, `operation` being `null` for a group - a strict consumer must accept that `null`.

| Symptom | Cause | What to do |
| --- | --- | --- |
| Alerts or Objectives screen: `Last version of the file refused` with `upstream ... is not in the gateway's configuration` | The changed file names an upstream, a tag or an operation the gateway does not serve | Correct the name, or deploy the gateway first; the previous version applies meanwhile |
| An objective `Invalid`, in a banner | The gateway's configuration changed **under** an accepted file (upstream removed, operation renamed), or the group is empty once `exclude` applies | Correct `slos` or restore what it names; the objective is never computed on an amputated group |
| `the group has no operation once exclude is applied` | `exclude` removes every member | Shorten `exclude` |
| `a group's name is an operation's too` | A `name` equal to a declared `operationId` | Rename the group |
| The global objective does not move despite an operation's outage | It weighs little in the volume: the availability is weighted by the requests | Read each member's share in the panel, and declare an objective on the operation or a narrower group |
| Nothing at the head of the Overview | No `all: true` objective, or a viewer's session | Declare a global objective; a viewer never sees it |