# OpenAI-compatible inference

> /v1/chat/completions, /v1/embeddings and /v1/models in OpenAI's format: an existing SDK changes only its base URL. Product side: AI and inference; operations…

Guide - Since version 0.1.0 - Flexway v1.6.3 - https://flexway.io/en/docs/developer-inference/

> `/v1/chat/completions`, `/v1/embeddings` and `/v1/models` in OpenAI's format: an existing SDK changes only its base URL.
>
> Product side: [AI and inference](https://flexway.io/en/docs/product-ai-inference/); operations side: [Serving inference](https://flexway.io/en/docs/operations-inference/).

## Calling

```python
from openai import OpenAI
client = OpenAI(base_url="https://api.example.com/v1", api_key=token)  # an OIDC token, sent as Bearer
r = client.chat.completions.create(model="default", messages=[{"role": "user", "content": "Hello"}])
```

A Flexway API key goes in `Authorization: ApiKey`, so through a default header of the SDK
(`default_headers={"Authorization": f"ApiKey {key}"}`). The **model** actually called is chosen by the
operator's policy among those it declares; `stream: true` relays events as they come. Images as input
and embeddings are served when the chosen backend can do them.

## What you can get

Every response of `/v1/*`, gateway refusals included, is in **OpenAI's envelope**
(`{"error": {"type", "code", "message"}}`).

| Status | `code` | When |
| --- | --- | --- |
| `401` | `tenant_required` | No identity, or an identity without a tenant |
| `429` | `budget_exceeded` | The tenant's budget, in tokens or euros, would be exceeded - before any call |
| `403` | `sensitive_no_internal_backend` | A request marked sensitive with no internal backend |
| `502` | `backend_unavailable` | Backend and fallbacks down; during a stream, a last error event |
| `400` | `capability_unavailable` | The chosen backend does not serve an image or embeddings |

| Response header | Meaning |
| --- | --- |
| `Flexway-Budget: soft` | Your tenant passed the soft threshold: the response comes from a cheaper backend |
| `Flexway-Degraded: backend-fallback` | Served by a fallback backend |

A **sensitive** request is marked by the header the operator declares (for example
`X-Sensitive: true`): it never leaves an internal backend.