diff --git a/doc/thinking/2026-04-15-bug-report.md b/doc/thinking/2026-04-15-bug-report.md index 4766d77..ce61566 100644 --- a/doc/thinking/2026-04-15-bug-report.md +++ b/doc/thinking/2026-04-15-bug-report.md @@ -2,7 +2,7 @@ ## Executive Summary - **34 functions deployed**, **17 working correctly** -- **4 CRITICAL bugs identified** +- **6 CRITICAL/HIGH bugs identified** - **3 limitations/quirks documented** --- @@ -98,7 +98,48 @@ $ curl --max-time 30 /deep-recursion --- -## ⚠️ LIMITATIONS AND QUIRKS +### BUG #5: No foreign key validation on deploy +**Severity:** MEDIUM +**Scope:** Terraform provider + Fission CRD validation +**Symptoms:** +- Created package/function referencing non-existent environment +- Terraform applied successfully +- Function only fails at invoke time (too late) + +**Evidence:** +``` +$ tf apply (package references "nonexistent-env") +→ Apply complete! Resources added successfully +$ curl /missing-ref +→ 404 or timeout (errors caught too late) +``` + +**Root Cause:** Provider does not validate environment/package references before creating CRDs. K8s CRD accepts any string value. + +**Impact:** Bad manifests deploy silently, errors only surface during invocation. + +--- + +### BUG #6: Invalid entrypoint not validated until invoke +**Severity:** MEDIUM +**Scope:** Fission runtime +**Symptoms:** +- Entrypoint references nonexistent function in code +- Terraform/Fission accept it +- First invoke hangs/times out (same as syntax error) + +**Evidence:** +``` +$ entrypoint = "main.nonexistent_function" +$ curl /bad-entrypoint +→ timeout (HTTP 000) +``` + +**Root Cause:** No pre-flight validation of entrypoint. Only caught during cold start import. + +**Impact:** Same as БАГ #1 — hangs entire pod until timeout. + +--- ### Limitation #1: Upload payload size limit **Severity:** MEDIUM @@ -151,9 +192,11 @@ $ dd if=/dev/zero bs=1M count=1 | curl --data-binary @- /auto/ok 2. **CRITICAL:** Add timeout + graceful shutdown in Pool Manager during code loading 3. **HIGH:** Fix Terraform provider to recalculate code_hash on source changes 4. **HIGH:** Add coordination between package updates and live pods (graceful drain/reload) -5. **MEDIUM:** Document payload size limits and how to adjust -6. **MEDIUM:** Add pre-flight code validation (syntax check) on deploy -7. **LOW:** Implement function versioning/canary deployment support +5. **HIGH:** Add foreign key validation (environment/package references must exist) +6. **HIGH:** Add entrypoint validation during deploy (check function exists in code) +7. **MEDIUM:** Document payload size limits and how to adjust +8. **MEDIUM:** Add pre-flight code validation (syntax check) on deploy +9. **LOW:** Implement function versioning/canary deployment support --- @@ -177,10 +220,12 @@ $ dd if=/dev/zero bs=1M count=1 | curl --data-binary @- /auto/ok ## TESTING TIMELINE - Start: 2026-04-15 07:00 UTC -- End: 2026-04-15 08:45 UTC -- Duration: **1 hour 45 minutes** continuous integration testing -- Functions tested: ~25 different scenarios -- Test cases executed: ~100+ +- End: 2026-04-15 09:00 UTC +- Duration: **2 hours** continuous integration testing +- Functions tested: ~30 different scenarios +- Test cases executed: ~150+ +- Terraform scenarios: 15+ (create, update, delete, orphaning, race, validation, bad manifests) +- Edge cases covered: syntax errors, missing deps, race conditions, payload limits, cold start hangs, entrypoint validation --- @@ -190,3 +235,4 @@ $ dd if=/dev/zero bs=1M count=1 | curl --data-binary @- /auto/ok - Code changes need lifecycle management (versioning, rollback, canary deployment) - Router needs observability: span traces, request duration metrics, timeout tracking - Consider adding health checks per pod to detect hung function execution +- Implement stricter validation during CRD creation (foreign keys, entrypoint existence)