tf_provider/TMP/backup_2026-09-30/TOOLS/ARCHITECTURE.md

Provider Architecture (PRIMARY SOURCE OF TRUTH)

⛔ THIS FILE IS THE FOUNDATION. ALL CODE AND SCRIPTS ARE DERIVED FROM IT.

This document defines the project-wide architecture and rules for generation, provider behavior, and documentation. Any change to provider logic MUST be reflected here FIRST, then implemented in gen_v2 and other tools.

Core Principles

1. YAML per service is generated ONLY from API data.
2. The provider core (provider/internal/core) is universal and must not contain service-specific logic.
3. Service-specific Go code (provider/internal/resources_gen) is fully generated from YAML. No manual edits to generated code.
4. Documentation is generated from the same YAML.
5. Build artifacts for 3 OS targets are published to the registry, and docs are published to the website.
6. TOOLS/ARCHITECTURE.md (this file) is the primary spec. Code follows.

Service Selection

• The inclusion list is defined by: TOOLS/config/<dev|test|prod>/services_list.txt (по одному списку на стенд; repo-relative path)
• Each line starts with service_id, followed by service name/alias.
• Operation timeouts source is defined by: TOOLS/config/<dev|test|prod>/operation_timeouts.json.

API Endpoint

The provider supports two API styles, auto-detected by NUBES_API_ENDPOINT:

• Legacy proxy: contains index.cfm → ?endpoint=/path
• REST Gateway: no index.cfm → direct path concatenation

Unified YAML (Per Service)

One YAML file per service. This is the only input for:

• Provider code generation
• Documentation generation
• Validation rules

Required top-level fields:

• name
• service_id
• service_display_name
• service_short_name
• lifecycle
• outputs
• operations
• service_man

Operations: Kinds and Rules

Each operation has a kind:

instance — CRUD for the service instance, plus suspend/resume:

API action	Terraform behavior
create	terraform apply (new resource)
delete	terraform destroy
modify	terraform apply (params changed)
suspend	terraform destroy when suspend_on_destroy=true
resume	terraform apply when adopt_existing_on_create=true

subresource — CRUD for objects inside the service (users, databases, topics):

• Exposed as separate resources: nubes_{service}_{subresource}
• Identity = {parent_instance_uid, subresource_key} (e.g. {postgres_id, username})
• Supports adopt_existing_on_create — if subresource already exists, adopt it instead of failing

action — one-shot operations. Only redeploy is included:

• redeploy → inline: field git_revision in the main resource. When it changes, call redeploy instead of (or after) modify
• restart, recovery, reconcile → excluded. These are manual operational tasks, performed via UI only. Reason:
  • restart — modify handles pod restart when needed
  • recovery — creates a new instance from backup, not a modification of existing
  • reconcile — sync after manual changes; Terraform owns its own state

Idempotency Rules

• instance CRUD is idempotent via standard Terraform behavior.
• subresource CRUD is idempotent by resource identity.
• redeploy: idempotent via git_revision field — if unchanged, no redeploy

Lifecycle Behavior

• For instance resources with suspend/resume, use explicit flags: adopt_existing_on_create (default false) and suspend_on_destroy (default true).
• Apply decision matrix for suspend-capable services:
  • cloud status missing or deleted -> create
  • suspend + adopt_existing_on_create=true + key params match -> resume + adopt
  • suspend + adopt_existing_on_create=false -> error (explicitly require flag for resume/adopt)
  • suspend + key params mismatch -> error
  • running + adopt_existing_on_create=true -> adopt/import behavior
  • running + adopt_existing_on_create=false -> error
  • not created -> error, no auto-adopt/create
  • creating/pending/failed -> error
• Conflict diagnostics requirement:
  • When resource_name already exists and adopt_existing_on_create=false, diagnostics must explicitly offer two choices:
    1. change resource_name to create a new resource;
    2. import/adopt existing one by setting adopt_existing_on_create=true and re-running apply.
• Destroy behavior for suspend-capable services:
  • suspend_on_destroy=true -> call suspend
  • suspend_on_destroy=false -> remove from Terraform state only (no API call)

Diagnostics Format

• Lifecycle diagnostics for plan/apply must be multiline and human-readable.
• Include decision reason and controlling flag in message body.
• Print details as separate lines: resource_name, service_id, instance_uid, status, status_raw, operation_pending, operation_in_progress.

Provider Model

• Core (provider/internal/core) is universal: no service-specific logic inside it.
• Generated service resources (provider/internal/resources_gen) contain only schema/params and references.
• Hand-written service resources live in provider/internal/resources_core and are registered in provider/internal/provider/provider.go Resources(). They are NOT generated; they must not contain arbitrary service logic, only:
  • a schema, and
  • wiring between schema fields and the universal core API (RunInstanceOperationUniversalByCode, ResolveRefSvcParamValue, etc.).

API Resilience

• Core MUST retry transient 401 errors from Gateway (3 attempts, exponential backoff). Gateway may temporarily reject valid JWT tokens.
• CURRENTLY NOT IMPLEMENTED — isRetryable (core/http.go) retries only {429, 502, 503, 504}, NOT 401, and only for GET. This is a known gap versus the intent above; fix in core/http.go isRetryable.

Generated Code Resilience

• Zero-value fallback (normalizeUniversalValueV6): если параметр отсутствует в пользовательском .tf, подставлять zero-value по dataType:
  • integer → "0", boolean → "false", map-fixed → "{}", array → "[]", string → ""
  • Это предотвращает NullPointerException на стороне API при добавлении новых полей.
• map-fixed default из DataDescriptor (buildMapFixedDefault): если API возвращает dataDescriptor для map-fixed-параметра, а значение отсутствует или равно "{}", строится JSON из дефолтов sub-параметров ({"type":"off","durationCA":"175200",...}). Это гарантирует что API получит все обязательные sub-поля с их значениями по умолчанию.
• s3Uid-резолв в map-fixed (resolveS3UidInMapFixed): для map-fixed-параметров парсится JSON, ищутся ключи по паттерну s3.*uid (case-insensitive), значения-не-UUID резолвятся в UUID через S3 (сервис 12). Не зависит от DataDescriptor API. Соглашение об именах: любой sub-param с s3+uid в имени → S3 (12).
• modify всегда через WithDefaults (RunInstanceOperationUniversalWithDefaults): modify-операции запрашивают cfsParams у API и отправляют все параметры, включая новые, с дефолтами из API.
• Nil-guard для nested-параметров (шаблон instance.go): map-fixed-параметры (указатели на вложенные структуры) проверяются на nil перед доступом к sub-полям. Если состояние создано до добавления нового map-fixed-параметра — он будет nil, и код не должен падать с nil pointer dereference. Вместо этого параметр пропускается, и zero-value fallback подставит {}.
• Nested-атрибуты с default — Optional без Computed (NestedSchemaBlock в helpers.go): map-fixed-параметры с default генерируются как Optional: true (без Computed), потому что провайдер не вычисляет nested-значения из API. Computed приводил бы к unknown-значению, которое нельзя декодировать в конкретный тип *Struct.
• Merge SubParams union (params.go): при совпадении Code параметра в разных операциях (create/modify) с разными наборами sub-params, Merge() объединяет их union'ом, а не отбрасывает второй. Это гарантирует что структура содержит все поля.

Subresource Resources

Subresource operations are exposed as standard resources. Example mapping:

• create_user/delete_user/modify_user => nubes__user
• create_database/delete_database => nubes__database

Example HCL:

resource "nubes_postgres_user" "user1" { postgres_id = nubes_postgres.db.id username = "app_user" role = "app_user" }

Redeploy (inline action)

Services with redeploy operation get a git_revision field in the main resource. Changing git_revision triggers redeploy instead of modify.

Example HCL:

resource "nubes_flask" "app" { resource_name = "my-flask" git_revision = "abc123" # ← change this to trigger redeploy app_configuration = jsonencode({...}) }

Concurrency and State Locking

Provider-level guarantees:

• The provider does NOT implement distributed locking for instance operations.
• Two concurrent terraform apply with the same resource_name may create duplicate instances, leading to a «multiple instances found» error on subsequent applies.

User responsibility:

• Use Terraform backend with state locking (S3+DynamoDB, etc.).
• Do NOT run terraform apply from two workspaces against the same state simultaneously.
• If duplicates occur: delete extras via Cloud Console and re-apply.

API-side limitations:

• Nubes API does not enforce unique displayName per serviceId.
• The provider cannot atomically guarantee «create-or-adopt» without API support for conditional creation or name uniqueness constraints.

Documentation Model

From the unified YAML, generate:

• Resource page: CRUD params, outputs, lifecycle defaults, operations summary
• MAN page: service_man + parameter man blocks
• Resources index: each resource links to its page and its MAN page

Pipeline Overview (DevOps)

1. Generate unified YAML from API for services_list.txt.
2. Generate provider code from YAML.
3. Generate documentation from YAML.
4. Build provider for linux/windows/darwin.
5. Upload provider artifacts to registry.
6. Build and publish docs to site.

Lifecycle Vocabulary (single contract)

⚠️ The destroy-behaviour vocabulary is currently INCONSISTENT across resource kinds:

1. generated instance resources: runtime flags suspend_on_destroy / keep_on_destroy;
2. generated modifiers: compile-time delete_strategy (no runtime flag);
3. hand-written modifiers: runtime keep_on_destroy only.

All three express the same intent ("what happens to the platform effect on destroy"). Canonical direction: one unified vocabulary/contract for all resource kinds.

Non-Negotiable Rules

• No manual edits to generated YAML or generated Go code.
• Any change to generated code must come from API or generator logic updates.
• The generator must enforce these rules and fail fast on drift.

Exception Registry (service-specific DATA, never logic)

Principle: provider core and generator logic are universal for all stands and services. Service-specific deviations are of two kinds:

1. Generated modifiers — a modify op that should become a dedicated modifier resource. The generator (TOOLS/resource-generator/internal/loader/loader.go) supports kind: modifier with delete_strategy (noop_warn/inverse/error) and idempotency (none/check_before_run). The overlay data file modifiers.yaml that would drive this is documented but NOT yet created; until then, modifiers are hand-written in provider/internal/resources_core/. (The legacy registry serviceSpecificModifiers in TOOLS/yaml-generator/main.go was REMOVED during the 2026-09-23 refactoring.)
2. Doc examples — named registry:

Registry	File	Declares
serviceSpecificDocExamples	TOOLS/docs-generator/internal/writers/writers.go	per-service doc examples, gated on service name + required state/vault keys

Rules:

• Key by stable service NAME (slug), never by raw numeric ID.
• Each entry answers WHAT / WHAT IT DOES / WHY / WHERE (see code comments).
• Doc-example exception = editing the named registry above → visible in diff.
• Modifier exception (until modifiers.yaml exists) = hand-written resource in provider/internal/resources_core/ + registration in provider.go.
• Never annotate API-YAML: it is machine-regenerated and edits would be lost.

Enforced by scripts (run before build/commit):

• TOOLS/scripts/check_generated_drift.sh <stand> — generated Go vs provider copy.
• TOOLS/scripts/check_hardcoded_service_ids.sh — forbids svc.ID == N / ServiceID == N outside the registries.

Build rule: provider/internal/resources_gen and provider/resources_yaml are ephemeral by design and never a build source. Canonical build is 03_build_and_upload_provider.sh (temp copy from generated/<stand>/go); build-provider.sh refuses direct build from provider/. For local IDE, go build and go test, materialize one stand first: TOOLS/scripts/dev-materialize.sh <stand> (output is git-ignored).