IoT/doc/architecture/current-v0.2.3.md

# IoT Managed Service — Актуальная архитектура (v0.2.3)

> Дата: 2026-04-12
> Образ: naeel/iot-operator:v0.2.3
> Кластер: iot-naeel, namespace: sless

---

## Путь данных (data flow)

```
IoT Device (MQTT CONNECT)
    | username="{ns}_{deviceId}", password=hex(64)
    v
EMQX 5.5.1 (Pod emqx, namespace sless)
    | 1. POST /internal/mqtt/auth -> iot-operator:9090
    |    - Bridge auth: username=iot-bridge-internal -> allow
    |    - Device auth: username={ns}_{deviceId} -> Secret -> compare
    | 2. POST /internal/mqtt/acl -> iot-operator:9090
    |    - Device: pub/sub только {ns}/telemetry/{deviceId}
    |    - Bridge: sub на +/telemetry/+ (wildcard)
    v
    | MQTT PUBLISH -> topic: "{ns}/telemetry/{deviceId}"
    v
iot-mqtt-bridge (Pod, namespace sless)
    | paho.mqtt.golang, подписка на "+/telemetry/+"
    | Парсит topic -> namespace (segment 0), deviceId (segment 2)
    | AWS SDK SQS SendMessage -> shared-SQS
    v
shared-SQS (namespace shared-sqs)
    | Endpoint: https://qu.kube5s.ru
    | Tenant: iot-service (ID: t-96afe7e9f781f6ca)
    | Queue: iot-telemetry
    v
iot-sqs-consumer (Pod, namespace sless)
    | ReceiveMessage (WaitTimeSeconds=20, long polling)
    | Парсит envelope -> namespace, device_id, payload
    | EnsureTenantDB(namespace) -> CREATE DATABASE tenant_{ns}
    | INSERT INTO iot_telemetry
    | DeleteMessage (at-least-once)
    v
Managed PostgreSQL 17
    | Host: postgresqlk8s-master.dc5db45d-....svc.cluster.local
    | Per-tenant: DATABASE tenant_{namespace}
    | Таблица: iot_telemetry (id, device_id, payload JSONB, received_at, created_at)
    v
REST API (iot-operator:9090)
    | GET /v1/namespaces/{ns}/iot/telemetry?device=X&limit=N
    v
Пользователь
    | IoT Console: https://iot.kube5s.ru/console
    | WebSocket MQTT: wss://iot.kube5s.ru/mqtt
    | Terraform: sless_iot_device resource
```

---

## Компоненты

### iot-operator (cmd/iot-operator)

Единый бинарник: controller-manager + REST API сервер.

| Функция | Описание |
|---------|----------|
| IoTDevice Controller | Reconcile: создание Secret с MQTT credentials, OwnerReference |
| REST API :9090 | CRUD устройств, телеметрия, MQTT auth/acl, admin stats, UI |
| Health :8081 | /healthz, /readyz для k8s probes |

### iot-mqtt-bridge (cmd/mqtt-bridge)

MQTT subscriber -> SQS producer. Stateless.

| Параметр | Значение |
|----------|----------|
| MQTT broker | tcp://emqx.sless.svc:1883 |
| MQTT username | iot-bridge-internal (Secret iot-bridge-credentials) |
| MQTT subscription | +/telemetry/+ |
| SQS endpoint | https://qu.kube5s.ru |
| SQS queue | iot-telemetry |

Bridge auth (v0.2.3): operator проверяет BridgeUsername/BridgePassword ДО парсинга namespace_deviceId.

### iot-sqs-consumer (cmd/sqs-consumer)

SQS consumer -> Postgres writer. Stateless.

| Параметр | Значение |
|----------|----------|
| SQS endpoint | https://qu.kube5s.ru |
| SQS queue | iot-telemetry |
| Long polling | WaitTimeSeconds=20 |
| Postgres | IOT_PG_DSN из Secret |
| Семантика | at-least-once (DeleteMessage после INSERT) |

### EMQX 5.5.1

MQTT-брокер с HTTP auth backend.

| Параметр | Значение |
|----------|----------|
| Образ | emqx/emqx:5.5.1 |
| Порты | 1883 (MQTT), 8083 (WebSocket), 18083 (Dashboard) |
| Auth | HTTP POST -> iot-operator:9090/internal/mqtt/auth |
| ACL | HTTP POST -> iot-operator:9090/internal/mqtt/acl |
| Конфиг | HOCON emqx.conf через ConfigMap |

---

## CRD: IoTDevice (iot.kube5s.ru/v1alpha1)

### Spec

| Поле | Тип | Обязательное | Описание |
|------|-----|-------------|----------|
| deviceId | string | да | Pattern: ^[a-z0-9][a-z0-9-]*[a-z0-9]$ |
| enabled | bool | нет | default: true |
| metadata | map[string]string | нет | Произвольные метаданные |

### Status

| Поле | Описание |
|------|----------|
| phase | Active / Disabled / Pending / Error |
| mqttUsername | {namespace}_{deviceId} |
| secretName | iot-{deviceId} |
| topicPrefix | {namespace}/ |
| message | Сообщение об ошибке |

### Reconcile logic

1. Добавить finalizer iot.kube5s.ru/device-cleanup
2. Если Secret iot-{deviceId} не существует:
   - crypto/rand 32 bytes -> hex (64 символа) = пароль
   - Создать Secret с OwnerReference -> каскадное удаление
   - Keys: mqtt-username, mqtt-password, device-id
3. Status: phase=Active, mqttUsername={ns}_{deviceId}
4. Если enabled=false -> phase=Disabled (Secret НЕ удаляется)
5. DELETE: finalizer cleanup -> Secret удаляется каскадно

---

## Аутентификация

### REST API (/v1/)

- Middleware: Bearer JWT token
- authTestMode = true (текущий): любой непустой Bearer token принимается
- authTestMode = false (prod): JWT decode -> sub -> SHA256 -> namespace mapping

### MQTT Auth (/internal/mqtt/auth)

1. Если username == BridgeUsername -> проверить BridgePassword -> allow/deny
2. Иначе: парсить username по первому "_" -> namespace + deviceId
3. Найти Secret iot-{deviceId} в namespace
4. crypto/subtle.ConstantTimeCompare(password, secret.mqtt-password)
5. Всегда HTTP 200, body: {"result": "allow"} или {"result": "deny"}

### MQTT ACL (/internal/mqtt/acl)

- Bridge (clientid=sless-iot-bridge): allow subscribe +/telemetry/+
- Device: allow pub/sub только {namespace}/telemetry/{deviceId}
- Всё остальное: deny

---

## Хранение данных

### PostgreSQL (managed)

| Параметр | Значение |
|----------|----------|
| Версия | PG 17 |
| Host | postgresqlk8s-master.dc5db45d-f8b4-4fd0-ad33-ec4dd017f2d5.svc.cluster.local |
| User | super |
| Master DB | sqsdb |

Per-tenant изоляция: отдельная DATABASE tenant_{namespace}.

Таблица iot_telemetry:
- id SERIAL PRIMARY KEY
- device_id TEXT NOT NULL
- payload JSONB NOT NULL
- received_at TIMESTAMPTZ
- created_at TIMESTAMPTZ DEFAULT NOW()

### shared-SQS

| Параметр | Значение |
|----------|----------|
| Endpoint | https://qu.kube5s.ru |
| Tenant | iot-service (t-96afe7e9f781f6ca) |
| Queue | iot-telemetry |
| Протокол | AWS SQS API compatible |

---

## Docker образ

- Registry: Docker Hub naeel/iot-operator
- Базовый: gcr.io/distroless/static:nonroot
- Содержит 3 бинарника: /iot-operator, /mqtt-bridge, /sqs-consumer
- Выбор бинарника через command в Deployment YAML

---

## Сетевая схема

```
Internet
    |
    v
nginx-ingress (namespace ingress)
    | iot.kube5s.ru/mqtt -> emqx-ws:8083 (WebSocket)
    | iot.kube5s.ru/console -> iot-operator:9090 (UI)
    | iot.kube5s.ru/iot-admin -> iot-operator:9090 (Admin UI)
    v
namespace sless:
    emqx:1883 <-> iot-mqtt-bridge (MQTT)
    emqx:1883 <- IoT devices (MQTT)
    iot-operator:9090 <- emqx (auth/acl HTTP)
    iot-operator:9090 <- users (REST API)
    iot-mqtt-bridge -> shared-sqs (HTTPS, SQS API)
    iot-sqs-consumer <- shared-sqs (HTTPS, SQS API)
    iot-sqs-consumer -> managed-postgres (TCP 5432)
    iot-operator -> managed-postgres (TCP 5432, telemetry GET)
    iot-operator -> k8s API (CRD watch, Secret CRUD)

namespace shared-sqs:
    shared-sqs:9324 (SQS API)
    Ingress: qu.kube5s.ru -> shared-sqs

namespace dc5db45d-...:
    postgresqlk8s-0 (PG 17 managed)
```

---

## Секреты (namespace sless)

| Secret | Ключи | Используется |
|--------|-------|-------------|
| iot-bridge-credentials | MQTT_USERNAME, MQTT_PASSWORD | iot-operator, iot-mqtt-bridge |
| iot-sqs-credentials | SQS_ENDPOINT, SQS_ACCESS_KEY, SQS_SECRET_KEY | iot-mqtt-bridge, iot-sqs-consumer |
| iot-postgres-secret | IOT_PG_DSN | iot-sqs-consumer |

---

## Эволюция архитектуры

| Версия | Дата | Message bus | Postgres | Кластер |
|--------|------|------------|----------|---------|
| v0.1.50 | 2026-04-04 | RabbitMQ (AMQP) | emptyDir PVC | sless (общий) |
| v0.1.68 | 2026-04-06 | Kafka (segmentio/kafka-go) | emptyDir PVC | sless (общий) |
| v0.2.0 | 2026-04-12 | shared-SQS (AWS SDK) | Managed PG 17 | iot-naeel (новый) |
| v0.2.3 | 2026-04-12 | shared-SQS (AWS SDK) | Managed PG 17 | iot-naeel |