> ⛔⛔⛔ ЛЕГАСИ (2026-08-16) — СТАРЫЙ IoT (k8s-деплой). НЕ ПРИНИМАТЬ ВО ВНИМАНИЕ. Актуальное: HISTORY/2026-08-16-session-log.md # IoT Managed Service — Актуальная архитектура (v0.2.3) > Дата: 2026-04-12 > Образ: naeel/iot-operator:v0.2.3 > Кластер: iot-naeel, namespace: sless --- ## Путь данных (data flow) ``` IoT Device (MQTT CONNECT) | username="{ns}_{deviceId}", password=hex(64) v EMQX 5.5.1 (Pod emqx, namespace sless) | 1. POST /internal/mqtt/auth -> iot-operator:9090 | - Bridge auth: username=iot-bridge-internal -> allow | - Device auth: username={ns}_{deviceId} -> Secret -> compare | 2. POST /internal/mqtt/acl -> iot-operator:9090 | - Device: pub/sub только {ns}/telemetry/{deviceId} | - Bridge: sub на +/telemetry/+ (wildcard) v | MQTT PUBLISH -> topic: "{ns}/telemetry/{deviceId}" v iot-mqtt-bridge (Pod, namespace sless) | paho.mqtt.golang, подписка на "+/telemetry/+" | Парсит topic -> namespace (segment 0), deviceId (segment 2) | AWS SDK SQS SendMessage -> shared-SQS v shared-SQS (namespace shared-sqs) | Endpoint: https://qu.kube5s.ru | Tenant: iot-service (ID: t-96afe7e9f781f6ca) | Queue: iot-telemetry v iot-sqs-consumer (Pod, namespace sless) | ReceiveMessage (WaitTimeSeconds=20, long polling) | Парсит envelope -> namespace, device_id, payload | EnsureTenantDB(namespace) -> CREATE DATABASE tenant_{ns} | INSERT INTO iot_telemetry | DeleteMessage (at-least-once) v Managed PostgreSQL 17 | Host: postgresqlk8s-master.dc5db45d-....svc.cluster.local | Per-tenant: DATABASE tenant_{namespace} | Таблица: iot_telemetry (id, device_id, payload JSONB, received_at, created_at) v REST API (iot-operator:9090) | GET /v1/namespaces/{ns}/iot/telemetry?device=X&limit=N v Пользователь | IoT Console: https://iot.kube5s.ru/console | WebSocket MQTT: wss://iot.kube5s.ru/mqtt | Terraform: sless_iot_device resource ``` --- ## Компоненты ### iot-operator (cmd/iot-operator) Единый бинарник: controller-manager + REST API сервер. | Функция | Описание | |---------|----------| | IoTDevice Controller | Reconcile: создание Secret с MQTT credentials, OwnerReference | | REST API :9090 | CRUD устройств, телеметрия, MQTT auth/acl, admin stats, UI | | Health :8081 | /healthz, /readyz для k8s probes | ### iot-mqtt-bridge (cmd/mqtt-bridge) MQTT subscriber -> SQS producer. Stateless. | Параметр | Значение | |----------|----------| | MQTT broker | tcp://emqx.sless.svc:1883 | | MQTT username | iot-bridge-internal (Secret iot-bridge-credentials) | | MQTT subscription | +/telemetry/+ | | SQS endpoint | https://qu.kube5s.ru | | SQS queue | iot-telemetry | Bridge auth (v0.2.3): operator проверяет BridgeUsername/BridgePassword ДО парсинга namespace_deviceId. ### iot-sqs-consumer (cmd/sqs-consumer) SQS consumer -> Postgres writer. Stateless. | Параметр | Значение | |----------|----------| | SQS endpoint | https://qu.kube5s.ru | | SQS queue | iot-telemetry | | Long polling | WaitTimeSeconds=20 | | Postgres | IOT_PG_DSN из Secret | | Семантика | at-least-once (DeleteMessage после INSERT) | ### EMQX 5.5.1 MQTT-брокер с HTTP auth backend. | Параметр | Значение | |----------|----------| | Образ | emqx/emqx:5.5.1 | | Порты | 1883 (MQTT), 8083 (WebSocket), 18083 (Dashboard) | | Auth | HTTP POST -> iot-operator:9090/internal/mqtt/auth | | ACL | HTTP POST -> iot-operator:9090/internal/mqtt/acl | | Конфиг | HOCON emqx.conf через ConfigMap | --- ## CRD: IoTDevice (iot.kube5s.ru/v1alpha1) ### Spec | Поле | Тип | Обязательное | Описание | |------|-----|-------------|----------| | deviceId | string | да | Pattern: ^[a-z0-9][a-z0-9-]*[a-z0-9]$ | | enabled | bool | нет | default: true | | metadata | map[string]string | нет | Произвольные метаданные | ### Status | Поле | Описание | |------|----------| | phase | Active / Disabled / Pending / Error | | mqttUsername | {namespace}_{deviceId} | | secretName | iot-{deviceId} | | topicPrefix | {namespace}/ | | message | Сообщение об ошибке | ### Reconcile logic 1. Добавить finalizer iot.kube5s.ru/device-cleanup 2. Если Secret iot-{deviceId} не существует: - crypto/rand 32 bytes -> hex (64 символа) = пароль - Создать Secret с OwnerReference -> каскадное удаление - Keys: mqtt-username, mqtt-password, device-id 3. Status: phase=Active, mqttUsername={ns}_{deviceId} 4. Если enabled=false -> phase=Disabled (Secret НЕ удаляется) 5. DELETE: finalizer cleanup -> Secret удаляется каскадно --- ## Аутентификация ### REST API (/v1/) - Middleware: Bearer JWT token - authTestMode = true (текущий): любой непустой Bearer token принимается - authTestMode = false (prod): JWT decode -> sub -> SHA256 -> namespace mapping ### MQTT Auth (/internal/mqtt/auth) 1. Если username == BridgeUsername -> проверить BridgePassword -> allow/deny 2. Иначе: парсить username по первому "_" -> namespace + deviceId 3. Найти Secret iot-{deviceId} в namespace 4. crypto/subtle.ConstantTimeCompare(password, secret.mqtt-password) 5. Всегда HTTP 200, body: {"result": "allow"} или {"result": "deny"} ### MQTT ACL (/internal/mqtt/acl) - Bridge (clientid=sless-iot-bridge): allow subscribe +/telemetry/+ - Device: allow pub/sub только {namespace}/telemetry/{deviceId} - Всё остальное: deny --- ## Хранение данных ### PostgreSQL (managed) | Параметр | Значение | |----------|----------| | Версия | PG 17 | | Host | postgresqlk8s-master.dc5db45d-f8b4-4fd0-ad33-ec4dd017f2d5.svc.cluster.local | | User | super | | Master DB | sqsdb | Per-tenant изоляция: отдельная DATABASE tenant_{namespace}. Таблица iot_telemetry: - id SERIAL PRIMARY KEY - device_id TEXT NOT NULL - payload JSONB NOT NULL - received_at TIMESTAMPTZ - created_at TIMESTAMPTZ DEFAULT NOW() ### shared-SQS | Параметр | Значение | |----------|----------| | Endpoint | https://qu.kube5s.ru | | Tenant | iot-service (t-96afe7e9f781f6ca) | | Queue | iot-telemetry | | Протокол | AWS SQS API compatible | --- ## Docker образ - Registry: Docker Hub naeel/iot-operator - Базовый: gcr.io/distroless/static:nonroot - Содержит 3 бинарника: /iot-operator, /mqtt-bridge, /sqs-consumer - Выбор бинарника через command в Deployment YAML --- ## Сетевая схема ``` Internet | v nginx-ingress (namespace ingress) | iot.kube5s.ru/mqtt -> emqx-ws:8083 (WebSocket) | iot.kube5s.ru/console -> iot-operator:9090 (UI) | iot.kube5s.ru/iot-admin -> iot-operator:9090 (Admin UI) v namespace sless: emqx:1883 <-> iot-mqtt-bridge (MQTT) emqx:1883 <- IoT devices (MQTT) iot-operator:9090 <- emqx (auth/acl HTTP) iot-operator:9090 <- users (REST API) iot-mqtt-bridge -> shared-sqs (HTTPS, SQS API) iot-sqs-consumer <- shared-sqs (HTTPS, SQS API) iot-sqs-consumer -> managed-postgres (TCP 5432) iot-operator -> managed-postgres (TCP 5432, telemetry GET) iot-operator -> k8s API (CRD watch, Secret CRUD) namespace shared-sqs: shared-sqs:9324 (SQS API) Ingress: qu.kube5s.ru -> shared-sqs namespace dc5db45d-...: postgresqlk8s-0 (PG 17 managed) ``` --- ## Секреты (namespace sless) | Secret | Ключи | Используется | |--------|-------|-------------| | iot-bridge-credentials | MQTT_USERNAME, MQTT_PASSWORD | iot-operator, iot-mqtt-bridge | | iot-sqs-credentials | SQS_ENDPOINT, SQS_ACCESS_KEY, SQS_SECRET_KEY | iot-mqtt-bridge, iot-sqs-consumer | | iot-postgres-secret | IOT_PG_DSN | iot-sqs-consumer | --- ## Эволюция архитектуры | Версия | Дата | Message bus | Postgres | Кластер | |--------|------|------------|----------|---------| | v0.1.50 | 2026-04-04 | RabbitMQ (AMQP) | emptyDir PVC | sless (общий) | | v0.1.68 | 2026-04-06 | Kafka (segmentio/kafka-go) | emptyDir PVC | sless (общий) | | v0.2.0 | 2026-04-12 | shared-SQS (AWS SDK) | Managed PG 17 | iot-naeel (новый) | | v0.2.3 | 2026-04-12 | shared-SQS (AWS SDK) | Managed PG 17 | iot-naeel |