# Setting quotas

> Goal: limit per caller, not per address of the pod that relays, and keep a slow or dead upstream from becoming a general outage.

Guide - Since version 0.1.0 - Flexway v1.6.3 - https://flexway.io/en/docs/operations-quotas/

> Operations guide: configure, deploy and observe the gateway. What the capability does, product side: [product guide](https://flexway.io/en/docs/product-overview/); calling it: [developer guide](https://flexway.io/en/docs/developer-overview/).

**Goal**: limit **per caller**, not per address of the pod that relays, and keep a slow or dead
upstream from becoming a general outage.

**Command** - a `rate-limit` policy, attached globally or per host:

```yaml
policies:
  definitions:
    per-caller:
      type: rate-limit
      config:
        key: caller                  # the token's subject, else the client's address
        classes:
          read:  { rps: 50, burst: 100 }   # GET, HEAD, OPTIONS
          write: { rps: 10, burst: 20 }    # everything else
  chains:
    global: [per-caller]
server:
  trusted_proxies: [10.0.0.0/8]      # the edge: its X-Forwarded-For is believed, nobody else's
```

> **Behind a cloud load balancer, the edge may not see the client either.** A layer-4 load balancer
> connects to the edge from its own private address: every request, then, reaches the edge - and the
> gateway - from one address, and all anonymous callers share one bucket. Turn the proxy protocol on
> at both ends: the load balancer sends it (a Service annotation of the provider, for example
> `service.beta.kubernetes.io/scw-loadbalancer-proxy-protocol-v2: "*"` on Scaleway or
> `service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"` on AWS) and the edge's HTTPS
> listeners require it, which also stops a client from choosing its own address. With Envoy Gateway:
>
> ```yaml
> apiVersion: gateway.envoyproxy.io/v1alpha1
> kind: ClientTrafficPolicy
> metadata: { name: public-client, namespace: envoy-gateway-system }
> spec:
>   targetRefs:
>     - { group: gateway.networking.k8s.io, kind: Gateway, name: public, sectionName: https }
>   proxyProtocol: {}                  # the load balancer sends it; a connection without it is refused
> ```
>
> The two settings go together: one without the other drops the traffic. Check the result in the
> edge's access log - `x-forwarded-for` must show public addresses, not the load balancer's - and
> keep `server.trusted_proxies` on the edge's range only.

An operation changes class in the contract - the class must exist in the policy, or the gateway
refuses to start:

```yaml
      x-flexway-rate-class: write
```

**A bucket per operation** (`1.0.0`): by default (`scope: class`), a caller has **one**
bucket per class, which every operation of the class drains - a journey chaining three writes in the
same second spends three tokens of the same bucket. `scope: operation` gives each operation its own
bucket per caller, the class setting its size - what most gateways count. A class may
override the policy's scope; shared counters (`server.counters`) follow with nothing more to set.

```yaml
      config:
        scope: operation                  # default: class
        classes:
          read:  { rps: 50, burst: 100 }
          write: { rps: 10, burst: 20 }
          sensitive: { rps: 1, burst: 3, scope: class }   # one bucket for all the sensitive operations
```

**Behind a client server** (`1.0.0`): a BFF relays anonymous users, all from its own address. The
`header` key counts each one apart, by a header the BFF sets - **believed only when the peer proved
one of the `trusted` identities**. Otherwise the header is ignored and the address stays the key: a
key a caller chooses is no key. An authenticated subject always stays the key.

```yaml
policies:
  definitions:
    per-user:
      type: rate-limit
      config:
        key: header
        header: X-Client-Key
        trusted: [web-bff.web.serviceaccount.identity.linkerd.cluster.local]
        classes: { read: { rps: 20, burst: 40 } }
server:
  trusted_proxies: [local]                  # the mesh proxy: the pod's own addresses
  peer_identity_header: l5d-client-id       # the mTLS identity it verified
```

> **Not `127.0.0.1/32`**: the mesh proxy connects from the pod's IP, not from the loopback (measured
> with Linkerd). `local` names the pod's own addresses, resolved at load - never a CIDR of pods, which
> would trust every pod. Behind a mesh, count anonymous callers by peer identity as above, never by
> address: the mesh does not clean `X-Forwarded-For`.

The peer's identity comes from the certificate the gateway's own TLS listener verified, else - from a
peer of `server.trusted_proxies` only - from `server.peer_identity_header` when declared (and then
from it alone), else from `x-forwarded-client-cert`. **Precondition** of `peer_identity_header`: the local proxy **overwrites**
that header when a caller sends it; without that guarantee, any pod forges the key.

**Validation and errors** (`1.0.0`): the gateway validates parameters and body (`full`, the
default). A service that stays the authority on its body asks for `params` - the body is relayed
**unread**, neither validated nor bounded by `max_body` - or `off` - nothing is validated. No mode
opens the whitelist: an undeclared parameter is still refused. The errors' `type` points at your
portal:

```yaml
hosts:
  api.example.com: { default_upstream: orders, validation: params }   # the whole host
errors:
  type_base: https://docs.example.com/errors/    # type = <base><code>
  field_codes: { minLength: too_short, minimum: out_of_range }   # errors[].code in your own words
```

Each entry of `errors` carries `field`, `rule` (for a human) and `code`, what a client
switches on: the JSON Schema keyword broken (`required`, `type`, `enum`, `minLength`, `maximum`,
`additionalProperties`...), or `invalid` when no schema could be applied (an unreadable body).
`errors.field_codes` renames a keyword to the token of **your** contract; a keyword the table does
not list keeps its name. A missing or extra property is named as the field (`body.age`), and every
broken rule has its own entry.

```yaml
      x-flexway-validation: full     # in the contract: the operation wins over the host
```

Per upstream, the bulkhead and the circuit breaker (on by default, values shown are the defaults):

```yaml
upstreams:
  orders:
    url: http://orders:8080
    max_in_flight: 200              # no cap when absent
    circuit_breaker: { failure_ratio: 0.5, min_requests: 20, window: 10s, cooldown: 5s }
```

**Check**:

| Situation | Answer |
| --- | --- |
| Empty bucket | `429`, `Retry-After`, `RateLimit-Remaining: 0` - two users behind one server-side client have **separate** buckets |
| Any rate-limited response | `RateLimit-Limit`, `-Remaining`, `-Reset` |
| Body or parameter outside the schema | `422`, the `errors` extension names each field and its `code` - under `params`, the body is no longer examined |
| Method not declared on a declared path, `QUERY` or an extension token included | `405` with `Allow` |
| Body past `server.limits.max_body` (1 MiB) | `413` |
| Upstream down, circuit open | `503` **at once**, `Retry-After`, `Flexway-Degraded: circuit-open` |
| Upstream saturated | `503` at once, `Flexway-Degraded: upstream-saturated` |
| Tenant quota spent | `429` `quota_exceeded`, `Retry-After` until midnight or the end of the month (UTC) |
| Gateway saturated | `503` `overloaded` on the sheddable first, `Flexway-Degraded: shedding`; `critical` never refused |

**Share across replicas, sell plans, shed load** (`1.0.0`) - without `server.counters`, counters are
**local to each replica**: three replicas give up to three times the rate. With it, each replica
decides alone and reconciles with the store every `sync` - no round trip on the path, a limit held
within one `sync` window:

```yaml
server:
  counters: { store: "${COUNTERS_URL}", sync: 1s }   # Redis, Dragonfly or Valkey; not reloadable
  shedding: { target: 20ms }                         # optional; not reloadable
  in_flight: { max: 2048, queue: 0 }                 # optional - derived from the container's memory limit when absent; not reloadable
policies:
  definitions:
    plans:
      type: quota                                    # per tenant; needs server.counters
      config: { limits: { day: 100000, month: 2000000 } }
  chains:
    global: [per-caller, plans]
```

```yaml
      x-flexway-priority: sheddable     # in the contract: critical | normal (default) | sheddable
```

Store lost: the gateway serves, on local counters, and says so (`flexway_counters_store_up` at 0, a
`WARN` line). Shedding follows the Go runtime's scheduling latency - the gateway's own saturation,
not an upstream's slowness.