# Troubleshooting: asynchronous plane

> Publications, subscriptions, channel isolation and field encryption: the symptom, its cause and what to do.

Troubleshooting - Since version 1.5.0 - Flexway v1.7.1 - https://flexway.io/en/docs/operations-troubleshooting-async/

> Operations guide: configure, deploy and observe the gateway. What the feature does, product side: [product guide](https://flexway.io/en/docs/product-overview/); calling it: [developer guide](https://flexway.io/en/docs/developer-overview/).

The refusals and errors of the Kafka listener (native Kafka clients) have their own table: [Kafka clients: reference](https://flexway.io/en/docs/operations-kafka-reference/).

## Publish and consume

| Symptom | Cause | Action |
| --- | --- | --- |
| `POST` on an `x-flexway-publish` door: `504 upstream_timeout` | The broker did not acknowledge within the budget: **unknown outcome**, the message may have been written | Send again with the same `Idempotency-Key`; check the broker's health, widen `timeout` |
| `503 overloaded` on a publication | The broker's `max_in_flight` buffer is full: it acknowledges slower than callers publish | Read `flexway_async_publish_duration_seconds`; raise `max_in_flight` or the broker's capacity |
| `502 upstream_error` on a publication | The broker refused: ACL, missing topic (the gateway never creates one), message rejected; AMQP: no queue takes the message | Create the tenant's topic, grant the gateway's principal write access, bind a queue to the exchange |
| Start refused: `is consumed and isolates no tenant` | A consumption on a channel neither bound to the tenant nor filtered on the `tenant` header | `x-flexway-bind: tenant` on a channel parameter, or the `tenant` header in every message's `headers` |
| A WebSocket subscriber receives a message twice | At least once: no acknowledgement within `ack_timeout`, or a cut before it | Acknowledge each `id` (`{"ack": "<id>"}`); deduplicate on `source` + `id` |
| Start refused: `reads the schema registry's wire format and is isolated by the tenant attribute alone` | A consumption of a channel in the registry's wire format filtered on the `tenant` attribute, without a binding: an ordinary Kafka producer's messages have no attribute | Bind the channel to the tenant (`x-flexway-bind: tenant`, address `{tenant}.<topic>`), or declare it public (`x-flexway-public: true`) |

## Field encryption

| Symptom | Cause | Action |
| --- | --- | --- |
| Start refused: `kms.key "...": kms key absent or unusable` | The field-encryption key does not exist in the KMS (a typo) or cannot wrap (a Vault key not derived) | Fix `kms.key` or create the key - [Field encryption](https://flexway.io/en/docs/operations-kafka-encryption/) |
| Kafka consumers get `CORRUPT_MESSAGE` after `kms.key` changed | Records written before name the former key | Declare it in `kms.previous_keys` - [Field encryption](https://flexway.io/en/docs/operations-kafka-encryption/) |