tf_provider/TOOLS/ARCHITECTURE.md

Provider Architecture (PRIMARY SOURCE OF TRUTH)

⛔ THIS FILE IS THE FOUNDATION. ALL CODE AND SCRIPTS ARE DERIVED FROM IT.

This document defines the project-wide architecture and rules for generation, provider behavior, and documentation. Any change to provider logic MUST be reflected here FIRST, then implemented in gen_v2 and other tools.

Core Principles

1. YAML per service is generated ONLY from API data.
2. The provider core is universal and must not contain service-specific logic.
3. Service-specific Go code is fully generated from YAML. No manual edits.
4. Documentation is generated from the same YAML.
5. Build artifacts for 3 OS targets are published to the registry, and docs are published to the website.
6. devops/ARCHITECTURE.md (this file) is the primary spec. Code follows.

Service Selection

• The inclusion list is defined by: devops/config/services_list.txt (repo-relative path)
• Each line starts with service_id, followed by service name/alias.
• Operation timeouts source is defined by: devops/config/operation_timeouts.json.

API Endpoint

The provider supports two API styles, auto-detected by NUBES_API_ENDPOINT:

• Legacy proxy: contains index.cfm → ?endpoint=/path
• REST Gateway: no index.cfm → direct path concatenation

Unified YAML (Per Service)

One YAML file per service. This is the only input for:

• Provider code generation
• Documentation generation
• Validation rules

Required top-level fields:

• name
• service_id
• service_display_name
• service_short_name
• lifecycle
• outputs
• operations
• service_man

Operations: Kinds and Rules

Each operation has a kind:

instance — CRUD for the service instance, plus suspend/resume:

API action	Terraform behavior
create	terraform apply (new resource)
delete	terraform destroy
modify	terraform apply (params changed)
suspend	terraform destroy when suspend_on_destroy=true
resume	terraform apply when adopt_existing_on_create=true

subresource — CRUD for objects inside the service (users, databases, topics):

• Exposed as separate resources: nubes_{service}_{subresource}
• Identity = {parent_instance_uid, subresource_key} (e.g. {postgres_id, username})
• Supports adopt_existing_on_create — if subresource already exists, adopt it instead of failing

action — one-shot operations. Only redeploy is included:

• redeploy → inline: field git_revision in the main resource. When it changes, call redeploy instead of (or after) modify
• restart, recovery, reconcile → excluded. These are manual operational tasks, performed via UI only. Reason:
  • restart — modify handles pod restart when needed
  • recovery — creates a new instance from backup, not a modification of existing
  • reconcile — sync after manual changes; Terraform owns its own state

Idempotency Rules

• instance CRUD is idempotent via standard Terraform behavior.
• subresource CRUD is idempotent by resource identity.
• redeploy: idempotent via git_revision field — if unchanged, no redeploy

Lifecycle Behavior

• For instance resources with suspend/resume, use explicit flags: adopt_existing_on_create (default false) and suspend_on_destroy (default true).
• Apply decision matrix for suspend-capable services:
  • cloud status missing or deleted -> create
  • suspend + adopt_existing_on_create=true + key params match -> resume + adopt
  • suspend + adopt_existing_on_create=false -> error (explicitly require flag for resume/adopt)
  • suspend + key params mismatch -> error
  • running + adopt_existing_on_create=true -> adopt/import behavior
  • running + adopt_existing_on_create=false -> error
  • not created -> error, no auto-adopt/create
  • creating/pending/failed -> error
• Conflict diagnostics requirement:
  • When resource_name already exists and adopt_existing_on_create=false, diagnostics must explicitly offer two choices:
    1. change resource_name to create a new resource;
    2. import/adopt existing one by setting adopt_existing_on_create=true and re-running apply.
• Destroy behavior for suspend-capable services:
  • suspend_on_destroy=true -> call suspend
  • suspend_on_destroy=false -> remove from Terraform state only (no API call)

Diagnostics Format

• Lifecycle diagnostics for plan/apply must be multiline and human-readable.
• Include decision reason and controlling flag in message body.
• Print details as separate lines: resource_name, service_id, instance_uid, status, status_raw, operation_pending, operation_in_progress.

Provider Model

• Core is universal: no service-specific logic inside the core.
• Generated service resources contain only schema/params and references.

API Resilience

• Core MUST retry transient 401 errors from Gateway (3 attempts, exponential backoff). Gateway may temporarily reject valid JWT tokens.
• GET operations (GetInstanceState, GetInstanceStateRaw) retry 401 with 2s/4s/8s backoff.
• doRequest treats 401 as retryable for GET requests (alongside 429, 502, 503, 504).

Subresource Resources

Subresource operations are exposed as standard resources. Example mapping:

• create_user/delete_user/modify_user => nubes__user
• create_database/delete_database => nubes__database

Example HCL:

resource "nubes_postgres_user" "user1" { postgres_id = nubes_postgres.db.id username = "app_user" role = "app_user" }

Redeploy (inline action)

Services with redeploy operation get a git_revision field in the main resource. Changing git_revision triggers redeploy instead of modify.

Example HCL:

resource "nubes_flask" "app" { resource_name = "my-flask" git_revision = "abc123" # ← change this to trigger redeploy app_configuration = jsonencode({...}) }

Concurrency and State Locking

Provider-level guarantees:

• The provider does NOT implement distributed locking for instance operations.
• Two concurrent terraform apply with the same resource_name may create duplicate instances, leading to a «multiple instances found» error on subsequent applies.

User responsibility:

• Use Terraform backend with state locking (S3+DynamoDB, etc.).
• Do NOT run terraform apply from two workspaces against the same state simultaneously.
• If duplicates occur: delete extras via Cloud Console and re-apply.

API-side limitations:

• Nubes API does not enforce unique displayName per serviceId.
• The provider cannot atomically guarantee «create-or-adopt» without API support for conditional creation or name uniqueness constraints.

Documentation Model

From the unified YAML, generate:

• Resource page: CRUD params, outputs, lifecycle defaults, operations summary
• MAN page: service_man + parameter man blocks
• Resources index: each resource links to its page and its MAN page

Pipeline Overview (DevOps)

1. Generate unified YAML from API for services_list.txt.
2. Generate provider code from YAML.
3. Generate documentation from YAML.
4. Build provider for linux/windows/darwin.
5. Upload provider artifacts to registry.
6. Build and publish docs to site.

Non-Negotiable Rules

• No manual edits to generated YAML or generated Go code.
• Any change must come from API or generator logic updates.
• The generator must enforce these rules and fail fast on drift.