flexway
FRENTalk to an expert

Traffic

Refuse cleanly rather than fall over.

Rate per caller, tenant quotas per day and month, shedding under saturation, timeouts per operation, bounded retries and circuit breakers: every excess gets a clear answer, the upstream stays protected.

14 of the 14 shown delivered · since v0.1.0

What you expect from this domain

  • Control peaks, prevent abuseDelivered · since v0.1.0
  • Quotas per application, team, user or projectTenant and agent built; user and use case framed; application, team, project and cost centre are missingPartly · next in 1.8
  • Limit requests and tokensRequests per second per caller, tokens per day and per month; a token rate per minute against peaks is missingPartly · next in 1.8
Every market expectation
Traffic and latencyTraffic and latency
Traffic at a glance: requests, 5xx and 4xx errors, p95 latency and 429 refusals, against the previous period.Real console capture, demo data · v1.6.3

What it solves

Problem 1

One chatty client starves the others

A caller, a script or a looping agent takes all the throughput.

Flexway's answer: Token bucket per caller, never per pod; beyond it, 429 with Retry-After and RateLimit headers.

Problem 2

A slow upstream brings everything down

Requests pile up behind a service that no longer answers.

Flexway's answer: In-flight cap per upstream, circuit breaker with half-open state, timeout per operation.

Problem 3

The gateway dies under load

A saturated replica is killed and drags the others.

Flexway's answer: Adaptive shedding and a global cap: an overloaded replica refuses the excess instead of being killed.

How it works

Every step is a decision readable in the trace and in the console.

Your infrastructureflexwayAPI, AI and agent gatewayUnified Decision Chain (UDC)429beyond the rate: 42901 Load02 Caller03 Rate04 Quota05 UpstreamApplicationsAgentsPartnersServicesModelsYour infrastructureflexwayAPI, AI and agent gatewayUnified Decision Chain (UDC)42901 Loadin-flight cap, shedding before identity02 Calleridentity and rate class03 Ratetoken bucket per caller04 Quotatenant, day and month05 Upstreamtimeout, bulkhead, circuit breaker; announced degradationApplicationsAgentsPartnersServicesModels

Traffic - Domain diagram: who calls, the decision steps, what is called, and where a request is refused.

  • Applications
  • Agents
  • Partners
  1. Load
  2. Caller
  3. Rate
  4. Quota
  5. Upstream
  • Services
  • Models

beyond the rate: 429

Applications
The applications that call your APIs: each request goes through Flexway, which checks the contract and the identity before reaching your systems.
Agents
AI agents: they call MCP tools and models through Flexway, under the same identity, quota and audit as an API.
Partners
The partner companies that consume your APIs: identified by a client certificate or an API key, under a quota per tenant.
Load
Beyond the in-flight cap: 503, before routing; under saturation, adaptive shedding refuses non-critical traffic first, before identity.
Caller
Flexway identifies the caller and its rate class, declared per method or per operation.
Rate
A token bucket per caller limits its rate; beyond it: 429 with Retry-After and the RateLimit headers.
Quota
The tenant's daily and monthly quota applies across all its routes; beyond it: 429 quota_exceeded.
Upstream
Towards the upstream: deadline per operation, bulkhead, circuit breaker; any degradation is announced, never silent.
Services
Your REST and gRPC services: they receive only the requests Flexway let through, under deadline and circuit breaker.
Models
Your internal models, called through the OpenAI-compatible surface; a sensitive request is pinned to them.
  • Line: one route
  • Interchange: shared decision
  • Request
  • Refusal
  • Your infrastructure
  1. Loadin-flight cap, shedding before identity
  2. Calleridentity and rate class
  3. Ratetoken bucket per caller
  4. Quotatenant, day and month
  5. Upstreamtimeout, bulkhead, circuit breaker; announced degradation

In detail

The essentials of each topic, with its status in the published release; unfold a card for the full text.

Rate per caller

Delivered · v0.1.0

Rate is counted per verified caller, never per replica, and going over answers 429 with Retry-After and the RateLimit headers.

  • Token bucket per caller
  • Classes per method, per-operation override
  • 429 with Retry-After and RateLimit
See the detailHide the detail - Rate per caller

The rate is counted per verified caller - the identity's tenant and subject - never per replica or pod address: two replicas do not double a client's allowance. Rate classes are declared per method, and an operation can have its own.

Beyond it, the answer is an RFC 9457 429 with Retry-After and the RateLimit headers: a well-written client knows when to come back.

All the points

  • Token bucket per caller
  • Classes per method, per-operation override
  • 429 with Retry-After and RateLimit

Features and status

  • Rate per caller (token bucket)Delivered · v0.1.0
  • 429 with Retry-After and RateLimit headersDelivered · v0.1.0
  • Rate classes per method, per-operation overrideDelivered · v0.1.0
Read in the documentation : Rate limits, quotas and 429

Tenant quotas, per day and month

Delivered · v0.1.0

Each tenant has a quota per day or month, with counters shared across replicas and reconciled with no blocking call.

  • Day and month windows
  • Counters shared across replicas
See the detailHide the detail - Tenant quotas, per day and month

Beyond the instant rate, each tenant has a quota over a long window, per day or month. Across replicas, counters are shared and reconciled in the background, with no blocking call on the request path.

All the points

  • Day and month windows
  • Counters shared across replicas

Features and status

  • Quota per tenant per day and monthDelivered · v0.1.0
Read in the documentation : Setting quotas

Shedding rather than failure

Delivered · v0.2.0

Under saturation, the gateway refuses the excess with an explicit 503 rather than building an endless queue.

  • Adaptive shedding
  • Global in-flight cap
See the detailHide the detail - Shedding rather than failure

Under saturation, the gateway sheds load: it refuses the excess with an explicit 503 instead of building an endless queue. A global in-flight cap protects each replica: overloaded, it refuses rather than being killed for lack of memory.

All the points

  • Adaptive shedding
  • Global in-flight cap

Features and status

  • Adaptive load shedding under saturationDelivered · v0.1.0
  • Global in-flight cap: an overloaded replica sheds instead of being killedDelivered · v0.2.0
Read in the documentation : Traffic control and quotas

Timeouts, retries and circuit breakers

Delivered · v0.1.0

Each operation has its timeout budget, retries are kept to idempotent methods and a circuit breaker lets a sick upstream recover.

  • Timeout budget per operation
  • Idempotent retries only
  • Bulkhead and circuit breaker
See the detailHide the detail - Timeouts, retries and circuit breakers

Each operation has its timeout budget, never a global timeout. Retries are limited to idempotent methods. An in-flight cap per upstream isolates services from each other, and a half-open circuit breaker lets a sick upstream recover.

A degradation is always announced, in a header and in the trace, never silent.

All the points

  • Timeout budget per operation
  • Idempotent retries only
  • Bulkhead and circuit breaker
  • Announced degradation

Features and status

  • Timeout budget per operationDelivered · v0.1.0
  • In-flight request cap per upstream (bulkhead)Delivered · v0.1.0
  • Retries limited to idempotent methodsDelivered · v0.1.0
  • Circuit breaker with half-open stateDelivered · v0.1.0
  • Degradation announced (header and trace), never silentDelivered · v0.1.0
Read in the documentation : Retries and idempotency

Graceful shutdown

Delivered · v0.1.0

When a replica stops, requests and streams in flight are drained: a deployment cuts neither a call nor a stream.

  • Streams in flight drained
  • Deployment without a cut
See the detailHide the detail - Graceful shutdown

When a replica stops, requests and streams in flight are drained before closing: a deployment cuts neither a call nor an SSE or WebSocket stream.

All the points

  • Streams in flight drained
  • Deployment without a cut

Features and status

  • Graceful shutdown: streams in flight are drainedDelivered · v0.1.0
Read in the documentation : Observing

Configuration

The rate-limits example, as it ships with Flexway v1.6.3.

examples/rate-limits/flexway.yaml
# A token bucket per caller - the token's subject, else the client's address, read from
# X-Forwarded-For only behind a declared edge.
version: 1
contracts:
  openapi: ./openapi.yaml
upstreams:
  default:
    url: http://search.internal:8080
server:
  trusted_proxies: [10.0.0.0/8]                  # the edge; nobody else chooses their address
policies:
  definitions:
    per-caller:
      type: rate-limit
      config:
        key: caller
        classes:
          read:      { rps: 50, burst: 100 }
          write:     { rps: 10, burst: 20 }
          expensive: { rps: 1,  burst: 5 }
  chains:
    global: [per-caller]
examples/resilience/flexway.yaml
# Layered budgets, connection pools, a bulkhead and a circuit breaker per upstream. Retries are
# built in: once, for an idempotent method or an Idempotency-Key, within a retry budget.
version: 1
contracts:
  openapi: ./openapi.yaml
upstreams:
  default:
    url: http://billing.internal:8080
    timeout: 3s                                  # total budget of an exchange with this upstream
    pool: { max_idle: 100, max_conns: 200, idle_timeout: 90s }
    max_in_flight: 500                           # bulkhead: past it, 503 at once
    circuit_breaker: { failure_ratio: 0.5, min_requests: 20, window: 10s, cooldown: 5s }
  exports:
    url: http://exports.internal:8080
    max_in_flight: 20                            # a slow upstream uses up its own slots, never everyone's
server:
  timeouts:
    connect: 2s
    read: 60s                                    # silence between two bytes - what governs a stream
    total: 30s
  limits:
    max_body: 2MiB                               # a body read for validation; past it, 413
  shutdown:
    drain: 20s

Features

What is delivered in the published release v1.6.3, and what is coming; a feature is never shown as delivered before its release.

FeatureStatus
Rate per caller (token bucket)Delivered · v0.1.0
429 with Retry-After and RateLimit headersDelivered · v0.1.0
Rate classes per method, per-operation overrideDelivered · v0.1.0
Quota per tenant per day and monthDelivered · v0.1.0
Adaptive load shedding under saturationDelivered · v0.1.0
Timeout budget per operationDelivered · v0.1.0
In-flight request cap per upstream (bulkhead)Delivered · v0.1.0
Retries limited to idempotent methodsDelivered · v0.1.0
Circuit breaker with half-open stateDelivered · v0.1.0
Degradation announced (header and trace), never silentDelivered · v0.1.0
Graceful shutdown: streams in flight are drainedDelivered · v0.1.0
Global in-flight cap: an overloaded replica sheds instead of being killedDelivered · v0.2.0
Traffic per operation, upstream and tenantDelivered · v0.1.0
Service-level objectives per operation: remaining error budget, burn rate over several windows, built-in alert ruleDelivered · v1.3.0

Every feature

The other expectations of this domain

  • Refuse, queue or reroute on overrun429 refusal and the tenant's soft-cap switch; graduated actions for each budget level are missingPartly · next in 1.8
  • Link each call to an application, user, use caseTenant, agent, client; hashed subject in the audit; use case framed; application and cost centrePartly · next in 1.8
  • Govern AI in IDEs and CI/CDEvery client goes through the same identity, quota and audit; attribution per application (the client) with its budget is missingPartly · next in 1.8
  • Follow the AI calls of a featureUse case declared in the contract; a session is followed through the tracePartly · next in 1.8

The other domains

Let's talk about your gateway

Flexwiz is a young company: you talk directly with the person who founds it. Tell us what you want to run through Flexway. We talk about your infrastructure and your digital sovereignty requirements.

Online booking is coming soon; until then, an e-mail is enough.