sless/sqs-operator/test_results_v2_20260407.log

╔══════════════════════════════════════════════════╗
║   SQS Operator Test Suite v2.0                  ║
║   2026-04-07 18:33:37                        ║
╚══════════════════════════════════════════════════╝

  Tenant: test001  AK: SQSAK-test001-ae4dc8
  Endpoint: https://sqs.kube5s.ru/sqs/test001
  QS test001: 


══════════════════════════════════════════════════
  PHASE 0 — Provisioning Timing (новый тенант с нуля)
══════════════════════════════════════════════════
  Тенант: timing-ts  CR: test-tenant-timing-ts
  Очищаем если остался от прошлого прогона...
namespace "sless-fn-timing-ts" deleted
  Создаём QueueService...
queueservice.sqs.kube5s.ru/test-tenant-timing-ts created
  [18:33:43] Apply complete, watching phases...
  [18:34:16] Phase appeared:   (+33s от apply)
  [18:35:22] Phase:   (+99s от apply)
  ❌ FAIL[P01_Phase_Ready]: timeout/error: timeout:Failed
  ❌ FAIL[P02_API_Ready]: API не ответил за 280s

  ─── Timing Summary ───────────────────────
  apply → CR exists:         33s
  apply → Phase=Ready:      280s
  apply → первый API ответ: 280s
  ──────────────────────────────────────────

  Ресурсы нового тенанта (kubectl top):
    [metrics-server недоступен]
  Оператор:
    [metrics-server недоступен]

  Удаляем timing-тенант...
queueservice.sqs.kube5s.ru "test-tenant-timing-ts" deleted from sqs-operator-system namespace

══════════════════════════════════════════════════
  PHASE 1 — Базовые операции (T01–T11)
══════════════════════════════════════════════════
--- T01 ListQueues
  ✅ PASS[T01]
--- T02 CreateQueue
  ✅ PASS[T02 (https://sqs.kube5s.ru:443/sqs/test001/test001/t-basic)]
--- T03 CreateQueue idempotent
  ✅ PASS[T03]
--- T04 GetQueueUrl
  ✅ PASS[T04]
--- T05 SendMessage
  ✅ PASS[T05 (id=0cb35a74-c270-454e-b32e-ba1c510db024)]
--- T06 ReceiveMessage
  ✅ PASS[T06 (body=hello-world)]
--- T07 DeleteMessage
  ✅ PASS[T07]
--- T08 ReceiveMessage == empty after delete
  ✅ PASS[T08]
--- T09 GetQueueAttributes
  ✅ PASS[T09]
--- T10 SetQueueAttributes
  ✅ PASS[T10]
--- T11 DeleteQueue
  ✅ PASS[T11]

══════════════════════════════════════════════════
  PHASE 2 — Ошибочные параметры (E01–E09)
══════════════════════════════════════════════════
--- E01 CreateQueue invalid name
  ✅ PASS[E01]
--- E02 VisibilityTimeout > 43200
  ⚠️  WARN[E02]: accepted oversize timeout: <wrapper xmlns="http://queue.amazonaws.com/doc/2012-11-05/" name="SetQueueAttributesResponse">
            <ResponseMetadata>
              <RequestId>
                00000000-0000-0000-0000-000000000000
              </RequestId>
            </ResponseMetadata>
          </wrapper>
--- E03 ReceiveMessage non-existent
  ✅ PASS[E03]
--- E04 DeleteMessage invalid receipt
  ✅ PASS[E04]
--- E05 GetQueueUrl non-existent
  ✅ PASS[E05]
--- E06 DeleteMessage double-delete
  ✅ PASS[E06 (double-delete handled)]
--- E07 Wrong credentials (WONTFIX: ElasticMQ не верифицирует SigV4)
  ⏭️  SKIP[E07]: WONTFIX: изоляция через Keycloak в проде (не через SigV4)
--- E08 SendMessage empty body
  ✅ PASS[E08]
--- E09 SendMessage 300KB oversized
/home/naeel/terra/sless/sqs-operator/test_v2_suite.sh: line 392: /usr/bin/curl: Argument list too long
  ⚠️  WARN[E09]: oversized принято

══════════════════════════════════════════════════
  PHASE 3 — Продвинутые фичи (A01–A09)
══════════════════════════════════════════════════
--- A01 VisibilityTimeout (5s return)
    Взяли: 'purge1', ждём 6с...
  ❌ FAIL[A01]: не вернулось: 'purge2'
--- A02 ChangeMessageVisibility → 0
  ✅ PASS[A02]
--- A03 SendMessageBatch 10
  ✅ PASS[A03 (sent=10)]
--- A04 ReceiveMessageBatch MaxNumberOfMessages=10
  ✅ PASS[A04 (recv=10)]
--- A05 DeleteMessageBatch
  ✅ PASS[A05 (deleted=10)]
--- A06 Long polling 3s
  ⚠️  WARN[A06]: слишком быстро: 0s
--- A07 MessageAttributes
  ⚠️  WARN[A07]: атрибуты не вернулись
--- A08 PurgeQueue
  ✅ PASS[A08]
    Сообщений после Purge: 0
--- A09 Dead Letter Queue
    DLQ ARN: arn:aws:sqs:ru-msk-1:test001:t-dlq
  ✅ PASS[A09]

══════════════════════════════════════════════════
  PHASE 4 — Multi-tenant изоляция (MT01–MT05)
══════════════════════════════════════════════════
--- MT01 Создаём QueueService test002
queueservice.sqs.kube5s.ru/test-tenant-002 created
    Ждём Ready (до 120с)...
  ✅ PASS[MT01 (tenant002 Ready за 31s)]
  ✅ PASS[MT01b creds OK (AK2=SQSAK-test002-525921)]
--- MT02 Одинаковое имя → разные URL
    T1: https://sqs.kube5s.ru:443/sqs/test001/test001/shared-q
    T2: https://sqs.kube5s.ru:443/sqs/test002/test002/shared-q
  ✅ PASS[MT02]
--- MT03 Cross-tenant message isolation [WONTFIX/Keycloak]
  ✅ PASS[MT03 (tenant002 не прочитал сообщение tenant001)]
--- MT04 Операции tenant002 независимы
  ✅ PASS[MT04 (tenant002 работает независимо)]
--- MT05 Удаляем tenant002, ресурсы должны исчезнуть
queueservice.sqs.kube5s.ru "test-tenant-002" deleted from sqs-operator-system namespace
    Ждём удаления namespace sless-fn-test002 (до 30с)...
  ⚠️  WARN[MT05]: namespace ещё существует: namespace/sless-fn-test002

══════════════════════════════════════════════════
  PHASE 5 — Self-Healing (SH01–SH06)
══════════════════════════════════════════════════
  Текущие ресурсы test001:
pod/sqs-test001-84d46d5dcf-nngrp
service/sqs-svc-test001
deployment.apps/sqs-test001
replicaset.apps/sqs-test001-84d46d5dcf
--- SH01 Delete Deployment → auto-recreate
deployment.apps "sqs-test001" deleted from sless-fn-test001 namespace
    Ждём пересоздания pod...
  ✅ PASS[SH01 Deployment восстановлен за 48s]
--- SH02 Delete Service → auto-recreate (v0.1.5 fix)
service "sqs-svc-test001" deleted from sless-fn-test001 namespace
  ✅ PASS[SH02 Service восстановлен за 2s]
--- SH03 Delete ConfigMap → auto-recreate (v0.1.5 fix)
configmap "sqs-cfg-test001" deleted from sless-fn-test001 namespace
  ✅ PASS[SH03 ConfigMap восстановлен за 4s]
--- SH04 Delete Ingress → auto-recreate with auth-snippet
ingress.networking.k8s.io "sqs-ing-test001" deleted from sless-fn-test001 namespace
  ✅ PASS[SH04 Ingress восстановлен за 2s]
  ⚠️  WARN[SH04b]: auth-snippet отсутствует в Ingress (nginx-controller блокирует snippets?)
--- SH05 Удаляем Service+ConfigMap+Ingress одновременно
service "sqs-svc-test001" deleted from sless-fn-test001 namespace
    Ждём восстановления всех трёх ресурсов...
  ✅ PASS[SH05 Все ресурсы восстановлены за 2s]
--- SH06 API доступен после self-healing
    Ждём pod Ready после всех манипуляций...
pod/sqs-test001-84d46d5dcf-7b4s4 condition met
  ✅ PASS[SH06 SQS API доступен (Phase=)]

  Self-Healing recovery times:
    SH01 Deployment: 48s
    SH02 Service:    2s
    SH03 ConfigMap:  4s
    SH04 Ingress:    2s
    SH05 All-3:      2s

══════════════════════════════════════════════════
  PHASE 6 — Ресурсы
══════════════════════════════════════════════════
--- R01 kubectl top: ElasticMQ pod (test001)
    [metrics-server недоступен]

--- R02 kubectl top: Operator pod
    [metrics-server недоступен]

--- R03 Limits/Requests ElasticMQ container
elasticmq	{"limits":{"cpu":"500m","memory":"256Mi"},"requests":{"cpu":"10m","memory":"128Mi"}}
  ✅ PASS[R03]

--- R04 PVC usage
NAME               STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
sqs-data-test001   Bound    pvc-008275fa-cb0e-409c-977d-23d1f156ff86   512Mi      RWO            local-path     <unset>                 3h58m
  ✅ PASS[R04 PVC Bound]

--- R05 QueueService Spec
  ✅ PASS[R05]

══════════════════════════════════════════════════
  PHASE 7 — Concurrent load (параллельные запросы)
══════════════════════════════════════════════════
--- CL01 10 параллельных SendMessage
<SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
          <SendMessageResult>
            
            
            <MD5OfMessageBody>0d47a9e731a0fb0f22fd2e896efd914e</MD5OfMessageBody>
            <MessageId>655961ac-fda6-4d4a-beba-447f4f1a0aff</MessageId>
            
          </SendMessageResult>
          <ResponseMetadata>
            <RequestId>00000000-0000-0000-0000-000000000000</RequestId>
          </ResponseMetadata>
        </SendMessageResponse><SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
          <SendMessageResult>
            
            
            <MD5OfMessageBody>5961eae6bda74d3ee8b2febe6ab93fac</MD5OfMessageBody>
            <MessageId>a4f17a7b-7b27-4225-a345-4fc312422794</MessageId>
            
          </SendMessageResult>
          <ResponseMetadata>
            <RequestId>00000000-0000-0000-0000-000000000000</RequestId>
          </ResponseMetadata>
        </SendMessageResponse><SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
          <SendMessageResult>
            
            
            <MD5OfMessageBody>0969ec5135d74643ca08fb852239500a</MD5OfMessageBody>
            <MessageId>58707b2f-0f9f-4f10-92a2-1b061df71e3f</MessageId>
            
          </SendMessageResult>
          <ResponseMetadata>
            <RequestId>00000000-0000-0000-0000-000000000000</RequestId>
          </ResponseMetadata>
        </SendMessageResponse><SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
          <SendMessageResult>
            
            
            <MD5OfMessageBody>98ab682f9eee5bac6468b50121fd8437</MD5OfMessageBody>
            <MessageId>f8e903a5-80e4-490b-bf13-ac19053cb8af</MessageId>
            
          </SendMessageResult>
          <ResponseMetadata>
            <RequestId>00000000-0000-0000-0000-000000000000</RequestId>
          </ResponseMetadata>
        </SendMessageResponse><SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
          <SendMessageResult>
            
            
            <MD5OfMessageBody>ad1f7b79b6f7dde21f9b4f03304f969c</MD5OfMessageBody>
            <MessageId>35f6ad93-e201-4944-a3c3-b9b6facfcfbf</MessageId>
            
          </SendMessageResult>
          <ResponseMetadata>
            <RequestId>00000000-0000-0000-0000-000000000000</RequestId>
          </ResponseMetadata>
        </SendMessageResponse><SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
          <SendMessageResult>
            
            
            <MD5OfMessageBody>56ba32f49f0a5a89d0ba5d227a0eebc7</MD5OfMessageBody>
            <MessageId>fb75e01d-7f96-4f49-8813-e16a020c16d8</MessageId>
            
          </SendMessageResult>
          <ResponseMetadata>
            <RequestId>00000000-0000-0000-0000-000000000000</RequestId>
          </ResponseMetadata>
        </SendMessageResponse><SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
          <SendMessageResult>
            
            
            <MD5OfMessageBody>2c18cb91c24e7d90a5a06f74038640b1</MD5OfMessageBody>
            <MessageId>a333ec73-fc6a-466d-94bb-446b21f6caf0</MessageId>
            
          </SendMessageResult>
          <ResponseMetadata>
            <RequestId>00000000-0000-0000-0000-000000000000</RequestId>
          </ResponseMetadata>
        </SendMessageResponse><SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
          <SendMessageResult>
            
            
            <MD5OfMessageBody>9586b8866255703a8a33beee866b14e9</MD5OfMessageBody>
            <MessageId>0408c416-a83e-41a4-b202-2bc5c3d3b65d</MessageId>
            
          </SendMessageResult>
          <ResponseMetadata>
            <RequestId>00000000-0000-0000-0000-000000000000</RequestId>
          </ResponseMetadata>
        </SendMessageResponse><SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
          <SendMessageResult>
            
            
            <MD5OfMessageBody>ac43a05d51587e6b367dce5cd588ed68</MD5OfMessageBody>
            <MessageId>6fb7041e-0a3e-46cb-a5a2-9cfd5529e337</MessageId>
            
          </SendMessageResult>
          <ResponseMetadata>
            <RequestId>00000000-0000-0000-0000-000000000000</RequestId>
          </ResponseMetadata>
        </SendMessageResponse><SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
          <SendMessageResult>
            
            
            <MD5OfMessageBody>ec2927ca5e5e0cc652b49f5b4c5b88e9</MD5OfMessageBody>
            <MessageId>cff757c8-fca1-4f14-92eb-3840a1cf26e4</MessageId>
            
          </SendMessageResult>
          <ResponseMetadata>
            <RequestId>00000000-0000-0000-0000-000000000000</RequestId>
          </ResponseMetadata>
        </SendMessageResponse>    Отправлено 10 за 1s, в очереди: 10
  ✅ PASS[CL01 (в очереди=10 /expected=10)]
--- CL02 5 параллельных ReceiveMessage
    Получили сообщений за 1s: 5
  ✅ PASS[CL02 (recv=5/5)]
--- CL03 Race: Одновременный send+receive (RPS throughput)
  ✅ PASS[CL03 система живая после concurrent send/recv (3s)]

  Throughput estimates (crude):
  SendMessage RPS:    ~10/s (1s для 10 req)

══════════════════════════════════════════════════
  PHASE 8 — Стресс-марафон 30 минут
══════════════════════════════════════════════════
  Старт: 18:41:08
  Конец: 19:11:08
  Паттерны: normal-send, batch-send, batch-receive, batch-delete, purge, error-injection, attrs

  [01:00 | iter=298 ] send=298  recv=245  del=245  err_infra=0 err_sqs=42 | q: α=1 β=71 | restarts=0 phase=
  [02:00 | iter=667 ] send=667  recv=560  del=560  err_infra=0 err_sqs=95 | q: α=0 β=45 | restarts=0 phase=
  [03:00 | iter=1091] send=854  recv=717  del=717  err_infra=297 err_sqs=155 | q: α= β= | restarts=1 phase=
  [04:00 | iter=1381] send=1083 recv=922  del=922  err_infra=373 err_sqs=197 | q: α=5 β=55 | restarts=1 phase=
  [05:00 | iter=1736] send=1438 recv=1221 del=1221 err_infra=373 err_sqs=248 | q: α=0 β=28 | restarts=1 phase=
  --- Resource snapshot [5min] ---
    [metrics-server недоступен]
  [06:00 | iter=2116] send=1818 recv=1545 del=1545 err_infra=373 err_sqs=302 | q: α=0 β=12 | restarts=1 phase=
  [07:00 | iter=2522] send=2224 recv=1887 del=1887 err_infra=373 err_sqs=360 | q: α=5 β=12 | restarts=1 phase=
  [08:00 | iter=2911] send=2613 recv=2219 del=2219 err_infra=373 err_sqs=415 | q: α=5 β=9 | restarts=1 phase=
  [09:00 | iter=3321] send=3023 recv=2562 del=2562 err_infra=373 err_sqs=474 | q: α=0 β=12 | restarts=1 phase=
  [10:00 | iter=3739] send=3441 recv=2917 del=2917 err_infra=373 err_sqs=534 | q: α=0 β=24 | restarts=1 phase=
  --- Resource snapshot [10min] ---
    [metrics-server недоступен]
  [11:00 | iter=4159] send=3861 recv=3274 del=3274 err_infra=373 err_sqs=594 | q: α=1 β=42 | restarts=1 phase=
  [12:00 | iter=4592] send=4294 recv=3637 del=3637 err_infra=373 err_sqs=656 | q: α=5 β=64 | restarts=1 phase=
  [13:00 | iter=5020] send=4722 recv=4003 del=4003 err_infra=373 err_sqs=717 | q: α=0 β=18 | restarts=1 phase=
  [14:00 | iter=5446] send=5148 recv=4362 del=4362 err_infra=373 err_sqs=778 | q: α=1 β=33 | restarts=1 phase=
  [15:00 | iter=5873] send=5575 recv=4722 del=4722 err_infra=373 err_sqs=839 | q: α=0 β=46 | restarts=1 phase=
  --- Resource snapshot [15min] ---
    [metrics-server недоступен]
  [16:00 | iter=6267] send=5969 recv=5056 del=5056 err_infra=373 err_sqs=895 | q: α=0 β=49 | restarts=1 phase=
  [17:03 | iter=6692] send=6394 recv=5414 del=5414 err_infra=373 err_sqs=956 | q: α=5 β=64 | restarts=1 phase=
  [18:03 | iter=7096] send=6798 recv=5759 del=5759 err_infra=373 err_sqs=1013 | q: α=1 β=65 | restarts=1 phase=
  [19:03 | iter=7504] send=7206 recv=6103 del=6103 err_infra=373 err_sqs=1072 | q: α=0 β=2 | restarts=1 phase=
  [20:03 | iter=7931] send=7633 recv=6463 del=6463 err_infra=373 err_sqs=1133 | q: α=0 β=21 | restarts=1 phase=
  --- Resource snapshot [20min] ---
    [metrics-server недоступен]
  [21:03 | iter=8385] send=7931 recv=6717 del=6717 err_infra=569 err_sqs=1197 | q: α= β= | restarts=2 phase=
  [22:03 | iter=8674] send=8072 recv=6969 del=6969 err_infra=753 err_sqs=1239 | q: α=0 β=53 | restarts=2 phase=
  [23:03 | iter=9016] send=8414 recv=7254 del=7254 err_infra=753 err_sqs=1288 | q: α=0 β=11 | restarts=2 phase=
  [24:03 | iter=9400] send=8798 recv=7578 del=7578 err_infra=753 err_sqs=1342 | q: α=0 β=0 | restarts=2 phase=
  [25:03 | iter=9801] send=9199 recv=7919 del=7919 err_infra=753 err_sqs=1400 | q: α=1 β=0 | restarts=2 phase=
  --- Resource snapshot [25min] ---
    [metrics-server недоступен]
  [26:03 | iter=10208] send=9606 recv=8259 del=8259 err_infra=753 err_sqs=1458 | q: α=0 β=3 | restarts=2 phase=
  [27:03 | iter=10630] send=10028 recv=8618 del=8618 err_infra=753 err_sqs=1518 | q: α=0 β=21 | restarts=2 phase=
  [28:03 | iter=11026] send=10424 recv=8957 del=8957 err_infra=753 err_sqs=1575 | q: α=0 β=20 | restarts=2 phase=
  [29:03 | iter=11432] send=10830 recv=9294 del=9294 err_infra=753 err_sqs=1633 | q: α=5 β=22 | restarts=2 phase=

  ─── Marathon Summary ─────────────────────
  Итераций:    11815
  Sent:        11213
  Received:    9618
  Deleted:     9618
  Infra errors:753
  SQS errors:  1687 (намеренные)
  ──────────────────────────────────────────

  Финальные ресурсы после марафона:
    [metrics-server недоступен]
  Operator:
    [metrics-server недоступен]
  ElasticMQ pod restarts за тест: 2
  ❌ FAIL[ST01]: Marathon: 753/11815 ошибок (6%)
  ⚠️  WARN[ST02]: Pod перезапускался 2 раз

══════════════════════════════════════════════════
  ИТОГОВЫЙ ОТЧЁТ
══════════════════════════════════════════════════

  Завершено:  2026-04-07 19:11:08
  Длительность: 2251s (37min 31sec)

  ✅ PASS:  40
  ❌ FAIL:   4
  ⚠️  WARN:   7
  ⏭️  SKIP:   1
  TOTAL:    52

  Провалившиеся тесты:

  - P01_Phase_Ready: timeout/error: timeout:Failed
  - P02_API_Ready: API не ответил за 280s
  - A01: не вернулось: 'purge2'
  - ST01: Marathon: 753/11815 ошибок (6%)

  Предупреждения:

  - E02: accepted oversize timeout: <wrapper xmlns="http://queue.amazonaws.com/doc/2012-11-05/" name="SetQueueAttributesResponse">
            <ResponseMetadata>
              <RequestId>
                00000000-0000-0000-0000-000000000000
              </RequestId>
            </ResponseMetadata>
          </wrapper>
  - E09: oversized принято
  - A06: слишком быстро: 0s
  - A07: атрибуты не вернулись
  - MT05: namespace ещё существует: namespace/sless-fn-test002
  - SH04b: auth-snippet отсутствует в Ingress (nginx-controller блокирует snippets?)
  - ST02: Pod перезапускался 2 раз

╔══════════════════════════════════════════════════╗
║   ❌ ЕСТЬ ПРОВАЛЫ                               ║
╚══════════════════════════════════════════════════╝