Nail/SQS-service
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
56 Commits 6 Branches 0 Tags
6af635afeeacff9b06ca7b9a0761a2fd8ea2ff6c
Commit Graph
56 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
“Naeel” 6af635afee doc: план миграции shared-sqs на Nubes Managed 2026-08-13 14:03:09 +04:00
“Naeel” 83f1ec3910 Add GitHub rules: copilot-instructions.md and pravila.md 2026-05-08 14:20:07 +04:00
Naeel 11d03d6b10 feat: Helm chart для shared-sqs deployment + thinking logs 2026-04-13 18:50:28 +03:00
Naeel 2a31de8562 helm: hardcode replicas=1 (in-memory state, multi-replica not supported) 2026-04-13 06:49:06 +03:00
Naeel 1221d3303f helm: fix ingress port hardcode, remove imagePullSecrets block 2026-04-13 06:48:01 +03:00
Naeel 9272be5a20 helm: hardcode probes and strategy, remove from values.yaml 2026-04-13 06:46:25 +03:00
Naeel c86f865be7 helm: hardcode port 4100, remove service block from values.yaml 2026-04-13 06:45:45 +03:00
Naeel 494a8a84d4 helm: remove nubesEndpoint — internal Nubes integration, not for external users 2026-04-13 06:44:36 +03:00
Naeel 029eabe7d9 helm: remove internal nubesEndpoint, leave empty for external users 2026-04-13 06:42:53 +03:00
Naeel 8f76570512 helm: remove imagePullSecrets entirely 2026-04-13 06:41:02 +03:00
Naeel 15762b1eef helm: remove internal imagePullSecret, image is public on Docker Hub 2026-04-13 06:40:09 +03:00
Naeel 5707f279dc helm: rewrite values.yaml comments for clarity 2026-04-13 06:38:47 +03:00
Naeel d3a88adb5b helm: fill real values for our deployment 2026-04-13 06:22:33 +03:00
Naeel bd950db732 helm: fix values.yaml — remove hardcoded secrets, add billing fields, tag v0.1.24; fix .gitignore shared-sqs pattern 2026-04-13 06:21:34 +03:00
Naeel aa8968f221 doc: tenant isolation architecture 2026-04-13 06:06:56 +03:00
Naeel 8fd0a59c41 fix: remove server-snippet from ingress (blocked by nginx controller), doc stress-test results 2026-04-12 14:09:34 +03:00
Naeel 4cd5131f8d doc: add billing and metrics documentation 2026-04-12 12:34:53 +03:00
Naeel c870c09fff doc: update progress and thinking log for v0.1.23-v0.1.24 (billing + metrics) 2026-04-12 12:10:28 +03:00
Naeel 35d7985b87 fix: bump Go to 1.23 in Dockerfile (required by prometheus client) 2026-04-12 12:06:15 +03:00
Naeel 436060a83f infra: add VMServiceScrape for Victoria Metrics scraping 
VMAgent will scrape /metrics from shared-sqs Service every 30s.
Selector: app=shared-sqs, port: http (4100).
 2026-04-12 12:05:33 +03:00
Naeel a4c910a099 feat(metrics): integrate Prometheus metrics into router and main 
- /metrics endpoint via promhttp.Handler
- actionHandler: request counter, bytes counter, latency histogram, errors
- Gauge updater: queues/messages count per tenant every 15s
- All metrics labeled by tenant and/or operation
 2026-04-12 12:04:54 +03:00
Naeel 3d59b5500c feat(metrics): add Prometheus metrics package for Victoria Metrics 
Metrics exposed:
- sqs_requests_total{tenant, operation} — request counter
- sqs_request_bytes_total{tenant, operation} — traffic counter
- sqs_request_duration_seconds{operation} — latency histogram
- sqs_errors_total{operation} — error counter
- sqs_queues_count{tenant} — gauge, updated every 15s
- sqs_messages_count{tenant} — gauge, updated every 15s
 2026-04-12 12:03:16 +03:00
Naeel aade320400 feat(billing): add queue_name column to usage records 2026-04-12 11:56:53 +03:00
Naeel 1272388673 feat: billing — usage tracking per SQS operation in PostgreSQL 
- New package app/billing: Init/RecordUsage/Close with auto-migrate
- Integration in actionHandler: record tenant_id, operation, msg_count, msg_bytes
- Helm chart: billing section in values.yaml, secret-billing.yaml, env vars in deployment
- Optional: billing disabled by default (BILLING_PG_HOST not set = no-op)
- Table: sqs_usage_records with index on (tenant_id, recorded_at)
 2026-04-12 11:51:27 +03:00
Naeel d1af612b48 Deploy shared-sqs v0.1.22 2026-04-12 10:08:00 +03:00
Naeel d7e9d90453 Add demo UI showcase mode 2026-04-12 09:58:32 +03:00
Naeel dd08427424 docs: polish user benchmark summary 2026-04-12 09:00:09 +03:00
Naeel 8933b91aa5 docs: expand benchmark coverage summary 2026-04-12 08:56:44 +03:00
Naeel 4bf3794618 docs: simplify benchmark summary 2026-04-12 08:48:43 +03:00
Naeel 1a00ec9799 docs: rename benchmark summary 2026-04-12 08:47:40 +03:00
Naeel 4a83575491 feat: persist messages separately and update ingress docs 2026-04-12 08:44:46 +03:00
Naeel d5d3519836 docs: add 32kb benchmark comparison 2026-04-12 08:44:11 +03:00
Naeel 70cfc0d83d docs: add references for 64kb investigation 2026-04-12 08:02:37 +03:00
Naeel a1ff9c4e52 docs: finalize 64kb ingress investigation 2026-04-12 07:59:04 +03:00
Naeel eba01c9580 fix: 7 performance/correctness fixes — v0.1.20 
1. Убрано логирование тела сообщения (256KB I/O на каждый send — perf+security)
2. SentTimestamp исправлен: m.SentTime вместо time.Now() (баг)
3. MD5 не пересчитывается на ReceiveMessage — используется кэш из SendMessage
4. ChangeMessageVisibility/batch теперь персистит в Redis (баг — потеря данных)
5. MessageDoesNotExist error code: QueueExists → ReceiptHandleIsInvalid (copy-paste баг)
6. copystructure убран из GetQueueAttributes — простой map lookup
7. SNS dead code удалён (SnsErrors, SnsErrorType — не используется в SQS сервисе)

Tested: quick_test 31/31 PASS, deployed v0.1.20
 2026-04-11 20:39:08 +03:00
Naeel c580101f0c doc: анализ деградации на больших payload (3 root causes) 
- json.Marshal(queue) сериализует ВСЕ сообщения под Lock — O(N*msg_size)
- log.Infof логирует полное тело каждого сообщения (256KB на запись)
- CPU limit 500m вызывает K8s CFS throttling на тяжёлых операциях
- PeriodicTasks держит глобальный Lock каждую секунду
- Баг: MessageDoesNotExist возвращает Code=QueueExists
 2026-04-11 20:19:28 +03:00
Naeel 406ec69679 feat: add full comparative benchmark + stress test v2 
- benchmark_full.sh: 10-section comparative benchmark (shared-sqs vs Yandex MQ)
  All 17 SQS commands tested, latency, throughput, message sizes, batch, long poll
- stress_test_v2.sh: improved stress test
- Key findings: large payload (64KB+) degradation needs investigation
 2026-04-11 20:13:00 +03:00
Naeel b181b7ef0b test: compare_sqs.sh benchmark Yandex MQ vs shared-sqs + thinking log 2026-04-11 19:46:01 +03:00
Naeel f618f02e76 doc: stress testing report, progress update, thinking log — v0.1.19 assessment 2026-04-11 19:15:57 +03:00
Naeel eaed7bd6c2 test: уменьшаю нагрузку для SSH-стабильности (10msg/worker, 30/tenant, longpoll 20s) 2026-04-11 17:03:41 +03:00
Naeel 24103310c2 test: stress_test v2 — снижение параллелизма до 20 воркеров (SSH stability) 2026-04-11 16:52:31 +03:00
Naeel bd8303ca2e test: stress_test.sh v2 — 50 воркеров, 1000 msg, long-poll, race, batch, 30 очередей, memory, multi-kill, verbose progress 2026-04-11 16:41:07 +03:00
Naeel f937b7fccb fix: stress_test.sh — правильный формат queue URL, парсинг tenant API, subshell-safe массивы 2026-04-11 16:28:59 +03:00
Naeel 60931fd2fd test: stress_test.sh — конкурентность, multi-tenant изоляция, burst, pod kill, Redis disconnect 2026-04-11 16:21:08 +03:00
Naeel d633e59a50 fix: v0.1.19 — валидация VisibilityTimeout/WaitTimeSeconds/пустой MessageBody, fix hardcore_test LONG_Q length 2026-04-11 15:56:31 +03:00
Naeel 95d1dc1adb chore: удалить временный диагностический скрипт 2026-04-11 09:44:53 +03:00
Naeel 38194fb2fa feat: v0.1.18 — TagQueue/UntagQueue/ListQueueTags/ChangeMessageVisibilityBatch, JWT auth in UI API tests, hardcore_test JWT fix, Helm chart v0.1.18 2026-04-11 09:44:38 +03:00
Naeel fbecde72eb feat: add 4 missing SQS API commands for Yandex/AWS compatibility 
- ChangeMessageVisibilityBatch: batch visibility timeout (up to 10 msgs)
- TagQueue: add/update queue tags
- UntagQueue: remove queue tags by keys
- ListQueueTags: list all queue tags
- Added Tags field to Queue struct
- Request/Response models for all 4 commands
- Registered in router (17 total API commands now)
- Yandex Message Queue API reference doc
 2026-04-11 07:51:17 +03:00
Naeel c9d9231efc docs: record compatibility runs, decisions and session reasoning 2026-04-10 20:27:59 +03:00
Naeel ebcb20475a perf: optimize receive long polling and finalize formatting cleanup 2026-04-10 19:49:55 +03:00