sqs-operator: docs + third test results (41 PASS, 0 OOM, 0 infra errors)

This commit is contained in:
Naeel
2026-04-07 20:44:56 +03:00
parent c132c68d74
commit c4efc5c960
3 changed files with 616 additions and 44 deletions
+425
View File
@@ -0,0 +1,425 @@
╔══════════════════════════════════════════════════╗
║ SQS Operator Test Suite v2.0 ║
║ 2026-04-07 19:31:20 ║
╚══════════════════════════════════════════════════╝
Tenant: test001 AK: SQSAK-test001-ae4dc8
Endpoint: https://sqs.kube5s.ru/sqs/test001
QS test001:
══════════════════════════════════════════════════
PHASE 0 — Provisioning Timing (новый тенант с нуля)
══════════════════════════════════════════════════
Тенант: timing-ts CR: test-tenant-timing-ts
Очищаем если остался от прошлого прогона...
namespace "sless-fn-timing-ts" deleted
Создаём QueueService...
queueservice.sqs.kube5s.ru/test-tenant-timing-ts created
[19:31:26] Apply complete, watching phases...
[19:31:59] Phase appeared: (+33s от apply)
[19:33:05] Phase: (+99s от apply)
❌ FAIL[P01_Phase_Ready]: timeout/error: timeout:Pending
❌ FAIL[P02_API_Ready]: API не ответил за 280s
─── Timing Summary ───────────────────────
apply → CR exists: 33s
apply → Phase=Ready: 280s
apply → первый API ответ: 280s
──────────────────────────────────────────
Ресурсы нового тенанта (kubectl top):
[metrics-server недоступен]
Оператор:
[metrics-server недоступен]
Удаляем timing-тенант...
queueservice.sqs.kube5s.ru "test-tenant-timing-ts" deleted from sqs-operator-system namespace
══════════════════════════════════════════════════
PHASE 1 — Базовые операции (T01–T11)
══════════════════════════════════════════════════
--- T01 ListQueues
✅ PASS[T01]
--- T02 CreateQueue
✅ PASS[T02 (https://sqs.kube5s.ru:443/sqs/test001/test001/t-basic)]
--- T03 CreateQueue idempotent
✅ PASS[T03]
--- T04 GetQueueUrl
✅ PASS[T04]
--- T05 SendMessage
✅ PASS[T05 (id=194e8609-89d2-404a-a637-4ff5330c712c)]
--- T06 ReceiveMessage
✅ PASS[T06 (body=hello-world)]
--- T07 DeleteMessage
✅ PASS[T07]
--- T08 ReceiveMessage == empty after delete
✅ PASS[T08]
--- T09 GetQueueAttributes
✅ PASS[T09]
--- T10 SetQueueAttributes
✅ PASS[T10]
--- T11 DeleteQueue
✅ PASS[T11]
══════════════════════════════════════════════════
PHASE 2 — Ошибочные параметры (E01–E09)
══════════════════════════════════════════════════
--- E01 CreateQueue invalid name
✅ PASS[E01]
--- E02 VisibilityTimeout > 43200
⚠️ WARN[E02]: accepted oversize timeout: <wrapper xmlns="http://queue.amazonaws.com/doc/2012-11-05/" name="SetQueueAttributesResponse">
<ResponseMetadata>
<RequestId>
00000000-0000-0000-0000-000000000000
</RequestId>
</ResponseMetadata>
</wrapper>
--- E03 ReceiveMessage non-existent
✅ PASS[E03]
--- E04 DeleteMessage invalid receipt
✅ PASS[E04]
--- E05 GetQueueUrl non-existent
✅ PASS[E05]
--- E06 DeleteMessage double-delete
✅ PASS[E06 (double-delete handled)]
--- E07 Wrong credentials (WONTFIX: ElasticMQ не верифицирует SigV4)
⏭️ SKIP[E07]: WONTFIX: изоляция через Keycloak в проде (не через SigV4)
--- E08 SendMessage empty body
✅ PASS[E08]
--- E09 SendMessage 300KB oversized
/home/naeel/terra/sless/sqs-operator/test_v2_suite.sh: line 392: /usr/bin/curl: Argument list too long
⚠️ WARN[E09]: oversized принято
══════════════════════════════════════════════════
PHASE 3 — Продвинутые фичи (A01–A09)
══════════════════════════════════════════════════
--- A01 VisibilityTimeout (5s return)
Взяли: 'batch-5', ждём 6с...
❌ FAIL[A01]: не вернулось: 'batch-6'
--- A02 ChangeMessageVisibility → 0
✅ PASS[A02]
--- A03 SendMessageBatch 10
✅ PASS[A03 (sent=10)]
--- A04 ReceiveMessageBatch MaxNumberOfMessages=10
✅ PASS[A04 (recv=10)]
--- A05 DeleteMessageBatch
✅ PASS[A05 (deleted=10)]
--- A06 Long polling 3s
⚠️ WARN[A06]: слишком быстро: 0s
--- A07 MessageAttributes
⚠️ WARN[A07]: атрибуты не вернулись
--- A08 PurgeQueue
✅ PASS[A08]
Сообщений после Purge: 0
--- A09 Dead Letter Queue
DLQ ARN: arn:aws:sqs:ru-msk-1:test001:t-dlq
✅ PASS[A09]
══════════════════════════════════════════════════
PHASE 4 — Multi-tenant изоляция (MT01MT05)
══════════════════════════════════════════════════
--- MT01 Создаём QueueService test002
queueservice.sqs.kube5s.ru/test-tenant-002 created
Ждём Ready (до 120с)...
✅ PASS[MT01 (tenant002 Ready за 38s)]
✅ PASS[MT01b creds OK (AK2=SQSAK-test002-83d83b)]
--- MT02 Одинаковое имя → разные URL
T1: https://sqs.kube5s.ru:443/sqs/test001/test001/shared-q
T2: https://sqs.kube5s.ru:443/sqs/test002/test002/shared-q
✅ PASS[MT02]
--- MT03 Cross-tenant message isolation [WONTFIX/Keycloak]
⏭️ SKIP[MT03]: WONTFIX: изоляция будет через Keycloak JWT в проде (не через SigV4); ElasticMQ не проверяет подпись
--- MT04 Операции tenant002 независимы
✅ PASS[MT04 (tenant002 работает независимо)]
--- MT05 Удаляем tenant002, ресурсы должны исчезнуть
queueservice.sqs.kube5s.ru "test-tenant-002" deleted from sqs-operator-system namespace
Ждём удаления namespace sless-fn-test002 (до 30с)...
⚠️ WARN[MT05]: namespace ещё существует: namespace/sless-fn-test002
══════════════════════════════════════════════════
PHASE 5 — Self-Healing (SH01SH06)
══════════════════════════════════════════════════
Текущие ресурсы test001:
pod/sqs-test001-675488cddd-k9jc5
service/sqs-svc-test001
deployment.apps/sqs-test001
replicaset.apps/sqs-test001-675488cddd
--- SH01 Delete Deployment → auto-recreate
deployment.apps "sqs-test001" deleted from sless-fn-test001 namespace
Ждём пересоздания pod...
✅ PASS[SH01 Deployment восстановлен за 49s]
--- SH02 Delete Service → auto-recreate (v0.1.5 fix)
service "sqs-svc-test001" deleted from sless-fn-test001 namespace
✅ PASS[SH02 Service восстановлен за 2s]
--- SH03 Delete ConfigMap → auto-recreate (v0.1.5 fix)
configmap "sqs-cfg-test001" deleted from sless-fn-test001 namespace
✅ PASS[SH03 ConfigMap восстановлен за 15s]
--- SH04 Delete Ingress → auto-recreate with auth-snippet
ingress.networking.k8s.io "sqs-ing-test001" deleted from sless-fn-test001 namespace
✅ PASS[SH04 Ingress восстановлен за 2s]
⚠️ WARN[SH04b]: auth-snippet отсутствует в Ingress (nginx-controller блокирует snippets?)
--- SH05 Удаляем Service+ConfigMap+Ingress одновременно
service "sqs-svc-test001" deleted from sless-fn-test001 namespace
Ждём восстановления всех трёх ресурсов...
✅ PASS[SH05 Все ресурсы восстановлены за 3s]
--- SH06 API доступен после self-healing
Ждём pod Ready после всех манипуляций...
pod/sqs-test001-675488cddd-t2k2s condition met
✅ PASS[SH06 SQS API доступен (Phase=)]
Self-Healing recovery times:
SH01 Deployment: 49s
SH02 Service: 2s
SH03 ConfigMap: 15s
SH04 Ingress: 2s
SH05 All-3: 3s
══════════════════════════════════════════════════
PHASE 6 — Ресурсы
══════════════════════════════════════════════════
--- R01 kubectl top: ElasticMQ pod (test001)
[metrics-server недоступен]
--- R02 kubectl top: Operator pod
[metrics-server недоступен]
--- R03 Limits/Requests ElasticMQ container
elasticmq {"limits":{"cpu":"500m","memory":"512Mi"},"requests":{"cpu":"10m","memory":"256Mi"}}
✅ PASS[R03]
--- R04 PVC usage
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE
sqs-data-test001 Bound pvc-008275fa-cb0e-409c-977d-23d1f156ff86 512Mi RWO local-path <unset> 4h57m
✅ PASS[R04 PVC Bound]
--- R05 QueueService Spec
✅ PASS[R05]
══════════════════════════════════════════════════
PHASE 7 — Concurrent load (параллельные запросы)
══════════════════════════════════════════════════
--- CL01 10 параллельных SendMessage
<SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
<SendMessageResult>
<MD5OfMessageBody>ad1f7b79b6f7dde21f9b4f03304f969c</MD5OfMessageBody>
<MessageId>c71f2f45-cd17-4676-bb7f-1b617ab168d6</MessageId>
</SendMessageResult>
<ResponseMetadata>
<RequestId>00000000-0000-0000-0000-000000000000</RequestId>
</ResponseMetadata>
</SendMessageResponse><SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
<SendMessageResult>
<MD5OfMessageBody>56ba32f49f0a5a89d0ba5d227a0eebc7</MD5OfMessageBody>
<MessageId>c1015e6a-20e4-459f-9fcd-76f82c9f24be</MessageId>
</SendMessageResult>
<ResponseMetadata>
<RequestId>00000000-0000-0000-0000-000000000000</RequestId>
</ResponseMetadata>
</SendMessageResponse><SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
<SendMessageResult>
<MD5OfMessageBody>ec2927ca5e5e0cc652b49f5b4c5b88e9</MD5OfMessageBody>
<MessageId>7ae22ab8-3762-4525-be2f-6b425da9471e</MessageId>
</SendMessageResult>
<ResponseMetadata>
<RequestId>00000000-0000-0000-0000-000000000000</RequestId>
</ResponseMetadata>
</SendMessageResponse><SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
<SendMessageResult>
<MD5OfMessageBody>ac43a05d51587e6b367dce5cd588ed68</MD5OfMessageBody>
<MessageId>5e696744-8f75-49b8-abd0-120a39698fa8</MessageId>
</SendMessageResult>
<ResponseMetadata>
<RequestId>00000000-0000-0000-0000-000000000000</RequestId>
</ResponseMetadata>
</SendMessageResponse><SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
<SendMessageResult>
<MD5OfMessageBody>0d47a9e731a0fb0f22fd2e896efd914e</MD5OfMessageBody>
<MessageId>0e265c08-74f6-4e22-b6fe-1bf64d33b059</MessageId>
</SendMessageResult>
<ResponseMetadata>
<RequestId>00000000-0000-0000-0000-000000000000</RequestId>
</ResponseMetadata>
</SendMessageResponse><SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
<SendMessageResult>
<MD5OfMessageBody>9586b8866255703a8a33beee866b14e9</MD5OfMessageBody>
<MessageId>823c5be3-e070-45ae-ac0f-15ba04556fd4</MessageId>
</SendMessageResult>
<ResponseMetadata>
<RequestId>00000000-0000-0000-0000-000000000000</RequestId>
</ResponseMetadata>
</SendMessageResponse><SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
<SendMessageResult>
<MD5OfMessageBody>2c18cb91c24e7d90a5a06f74038640b1</MD5OfMessageBody>
<MessageId>d473b5a1-d110-4ddf-b1e1-36bec830767c</MessageId>
</SendMessageResult>
<ResponseMetadata>
<RequestId>00000000-0000-0000-0000-000000000000</RequestId>
</ResponseMetadata>
</SendMessageResponse><SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
<SendMessageResult>
<MD5OfMessageBody>98ab682f9eee5bac6468b50121fd8437</MD5OfMessageBody>
<MessageId>df977795-3675-4594-b52a-fe2679bca678</MessageId>
</SendMessageResult>
<ResponseMetadata>
<RequestId>00000000-0000-0000-0000-000000000000</RequestId>
</ResponseMetadata>
</SendMessageResponse><SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
<SendMessageResult>
<MD5OfMessageBody>0969ec5135d74643ca08fb852239500a</MD5OfMessageBody>
<MessageId>acc834b5-24e0-4270-bb67-d1fc76b4d066</MessageId>
</SendMessageResult>
<ResponseMetadata>
<RequestId>00000000-0000-0000-0000-000000000000</RequestId>
</ResponseMetadata>
</SendMessageResponse><SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
<SendMessageResult>
<MD5OfMessageBody>5961eae6bda74d3ee8b2febe6ab93fac</MD5OfMessageBody>
<MessageId>baf93da7-0ae1-4ef0-a367-898b3819df9c</MessageId>
</SendMessageResult>
<ResponseMetadata>
<RequestId>00000000-0000-0000-0000-000000000000</RequestId>
</ResponseMetadata>
</SendMessageResponse> Отправлено 10 за 2s, в очереди: 40
✅ PASS[CL01 (в очереди=40 /expected=10)]
--- CL02 5 параллельных ReceiveMessage
Получили сообщений за 1s: 5
✅ PASS[CL02 (recv=5/5)]
--- CL03 Race: Одновременный send+receive (RPS throughput)
✅ PASS[CL03 система живая после concurrent send/recv (2s)]
Throughput estimates (crude):
SendMessage RPS: ~5/s (2s для 10 req)
══════════════════════════════════════════════════
PHASE 8 — Стресс-марафон 30 минут
══════════════════════════════════════════════════
Старт: 19:39:16
Конец: 20:09:16
Паттерны: normal-send, batch-send, batch-receive, batch-delete, purge, error-injection, attrs
[01:00 | iter=257 ] send=257 recv=274 del=274 err_infra=0 err_sqs=36 | q: α=0 β=41 | restarts=0 phase=
[02:00 | iter=638 ] send=638 recv=593 del=593 err_infra=0 err_sqs=91 | q: α=0 β=24 | restarts=0 phase=
[03:00 | iter=1031] send=1031 recv=926 del=926 err_infra=0 err_sqs=147 | q: α=0 β=21 | restarts=0 phase=
[04:00 | iter=1459] send=1459 recv=1292 del=1292 err_infra=0 err_sqs=208 | q: α=0 β=42 | restarts=0 phase=
[05:00 | iter=1881] send=1881 recv=1645 del=1645 err_infra=0 err_sqs=268 | q: α=1 β=55 | restarts=0 phase=
--- Resource snapshot [5min] ---
[metrics-server недоступен]
[06:00 | iter=2301] send=2301 recv=2003 del=2003 err_infra=0 err_sqs=328 | q: α=0 β=0 | restarts=0 phase=
[07:00 | iter=2710] send=2710 recv=2347 del=2347 err_infra=0 err_sqs=387 | q: α=0 β=9 | restarts=0 phase=
[08:00 | iter=3134] send=3134 recv=2705 del=2705 err_infra=0 err_sqs=447 | q: α=0 β=22 | restarts=0 phase=
[09:00 | iter=3559] send=3559 recv=3067 del=3067 err_infra=0 err_sqs=508 | q: α=0 β=42 | restarts=0 phase=
[10:00 | iter=3988] send=3988 recv=3423 del=3423 err_infra=0 err_sqs=569 | q: α=0 β=63 | restarts=0 phase=
--- Resource snapshot [10min] ---
[metrics-server недоступен]
[11:00 | iter=4426] send=4426 recv=3803 del=3803 err_infra=0 err_sqs=632 | q: α=0 β=20 | restarts=0 phase=
[12:00 | iter=4861] send=4861 recv=4166 del=4166 err_infra=0 err_sqs=694 | q: α=5 β=43 | restarts=0 phase=
[13:00 | iter=5298] send=5298 recv=4540 del=4540 err_infra=0 err_sqs=756 | q: α=0 β=65 | restarts=0 phase=
[14:00 | iter=5740] send=5740 recv=4910 del=4910 err_infra=0 err_sqs=820 | q: α=0 β=30 | restarts=0 phase=
[15:00 | iter=6172] send=6172 recv=5277 del=5277 err_infra=0 err_sqs=881 | q: α=1 β=46 | restarts=0 phase=
--- Resource snapshot [15min] ---
[metrics-server недоступен]
[16:00 | iter=6610] send=6610 recv=5646 del=5646 err_infra=0 err_sqs=944 | q: α=0 β=9 | restarts=0 phase=
[17:00 | iter=7043] send=7043 recv=6011 del=6011 err_infra=0 err_sqs=1006 | q: α=0 β=25 | restarts=0 phase=
[18:00 | iter=7483] send=7483 recv=6386 del=6386 err_infra=0 err_sqs=1069 | q: α=0 β=61 | restarts=0 phase=
[19:00 | iter=7932] send=7932 recv=6765 del=6765 err_infra=0 err_sqs=1133 | q: α=0 β=21 | restarts=0 phase=
[20:00 | iter=8373] send=8373 recv=7145 del=7145 err_infra=0 err_sqs=1196 | q: α=0 β=52 | restarts=0 phase=
--- Resource snapshot [20min] ---
[metrics-server недоступен]
[21:00 | iter=8808] send=8808 recv=7508 del=7508 err_infra=0 err_sqs=1258 | q: α=0 β=2 | restarts=0 phase=
[22:00 | iter=9260] send=9260 recv=7892 del=7892 err_infra=0 err_sqs=1322 | q: α=0 β=41 | restarts=0 phase=
[23:00 | iter=9686] send=9686 recv=8247 del=8247 err_infra=0 err_sqs=1383 | q: α=0 β=61 | restarts=0 phase=
[24:00 | iter=10121] send=10121 recv=8617 del=8617 err_infra=0 err_sqs=1445 | q: α=0 β=18 | restarts=0 phase=
[25:00 | iter=10558] send=10558 recv=8981 del=8981 err_infra=0 err_sqs=1508 | q: α=0 β=42 | restarts=0 phase=
--- Resource snapshot [25min] ---
[metrics-server недоступен]
[26:00 | iter=11001] send=11001 recv=9363 del=9363 err_infra=0 err_sqs=1571 | q: α=0 β=0 | restarts=0 phase=
[27:00 | iter=11416] send=11416 recv=9713 del=9713 err_infra=0 err_sqs=1630 | q: α=0 β=12 | restarts=0 phase=
[28:00 | iter=11859] send=11859 recv=10090 del=10090 err_infra=0 err_sqs=1694 | q: α=0 β=36 | restarts=0 phase=
[29:00 | iter=12305] send=12305 recv=10466 del=10466 err_infra=0 err_sqs=1757 | q: α=0 β=2 | restarts=0 phase=
─── Marathon Summary ─────────────────────
Итераций: 12733
Sent: 12733
Received: 10827
Deleted: 10827
Infra errors:0
SQS errors: 1819 (намеренные)
──────────────────────────────────────────
Финальные ресурсы после марафона:
[metrics-server недоступен]
Operator:
[metrics-server недоступен]
ElasticMQ pod restarts за тест: 0
✅ PASS[ST01 Marathon 12733iter/1800s — 0 инфра-ошибок]
✅ PASS[ST02 Нет pod restarts за марафон]
══════════════════════════════════════════════════
ИТОГОВЫЙ ОТЧЁТ
══════════════════════════════════════════════════
Завершено: 2026-04-07 20:09:16
Длительность: 2276s (37min 56sec)
✅ PASS: 41
❌ FAIL: 3
⚠️ WARN: 6
⏭️ SKIP: 2
TOTAL: 52
Провалившиеся тесты:
- P01_Phase_Ready: timeout/error: timeout:Pending
- P02_API_Ready: API не ответил за 280s
- A01: не вернулось: 'batch-6'
Предупреждения:
- E02: accepted oversize timeout: <wrapper xmlns="http://queue.amazonaws.com/doc/2012-11-05/" name="SetQueueAttributesResponse">
<ResponseMetadata>
<RequestId>
00000000-0000-0000-0000-000000000000
</RequestId>
</ResponseMetadata>
</wrapper>
- E09: oversized принято
- A06: слишком быстро: 0s
- A07: атрибуты не вернулись
- MT05: namespace ещё существует: namespace/sless-fn-test002
- SH04b: auth-snippet отсутствует в Ingress (nginx-controller блокирует snippets?)
╔══════════════════════════════════════════════════╗
║ ❌ ЕСТЬ ПРОВАЛЫ ║
╚══════════════════════════════════════════════════╝