1
This commit is contained in:
Repinoid
@@ -0,0 +1,277 @@
|
||||
# Provider Architecture (PRIMARY SOURCE OF TRUTH)
|
||||
|
||||
**⛔ THIS FILE IS THE FOUNDATION. ALL CODE AND SCRIPTS ARE DERIVED FROM IT.**
|
||||
|
||||
This document defines the project-wide architecture and rules for generation,
|
||||
provider behavior, and documentation. Any change to provider logic MUST be
|
||||
reflected here FIRST, then implemented in `gen_v2` and other tools.
|
||||
|
||||
## Core Principles
|
||||
|
||||
1) YAML per service is generated ONLY from API data.
|
||||
2) The provider core (`provider/internal/core`) is universal and must not contain
|
||||
service-specific logic.
|
||||
3) Service-specific Go code (`provider/internal/resources_gen`) is fully generated
|
||||
from YAML. No manual edits to generated code.
|
||||
4) Documentation is generated from the same YAML.
|
||||
5) Build artifacts for 3 OS targets are published to the registry, and docs are
|
||||
published to the website.
|
||||
6) `TOOLS/ARCHITECTURE.md` (this file) is the primary spec. Code follows.
|
||||
|
||||
## Service Selection
|
||||
|
||||
- The inclusion list is defined by: `TOOLS/config/<dev|test|prod>/services_list.txt`
|
||||
(по одному списку на стенд; repo-relative path)
|
||||
- Each line starts with service_id, followed by service name/alias.
|
||||
- Operation timeouts source is defined by: `TOOLS/config/<dev|test|prod>/operation_timeouts.json`.
|
||||
|
||||
## API Endpoint
|
||||
|
||||
The provider supports two API styles, auto-detected by `NUBES_API_ENDPOINT`:
|
||||
- **Legacy proxy**: contains `index.cfm` → `?endpoint=/path`
|
||||
- **REST Gateway**: no `index.cfm` → direct path concatenation
|
||||
|
||||
## Unified YAML (Per Service)
|
||||
|
||||
One YAML file per service. This is the only input for:
|
||||
- Provider code generation
|
||||
- Documentation generation
|
||||
- Validation rules
|
||||
|
||||
Required top-level fields:
|
||||
- name
|
||||
- service_id
|
||||
- service_display_name
|
||||
- service_short_name
|
||||
- lifecycle
|
||||
- outputs
|
||||
- operations
|
||||
- service_man
|
||||
|
||||
### Operations: Kinds and Rules
|
||||
|
||||
Each operation has a kind:
|
||||
|
||||
**instance** — CRUD for the service instance, plus suspend/resume:
|
||||
| API action | Terraform behavior |
|
||||
|---|---|
|
||||
| create | `terraform apply` (new resource) |
|
||||
| delete | `terraform destroy` |
|
||||
| modify | `terraform apply` (params changed) |
|
||||
| suspend | `terraform destroy` when `suspend_on_destroy=true` |
|
||||
| resume | `terraform apply` when `adopt_existing_on_create=true` |
|
||||
|
||||
**subresource** — CRUD for objects inside the service (users, databases, topics):
|
||||
- Exposed as separate resources: `nubes_{service}_{subresource}`
|
||||
- Identity = `{parent_instance_uid, subresource_key}` (e.g. `{postgres_id, username}`)
|
||||
- Supports `adopt_existing_on_create` — if subresource already exists, adopt it instead of failing
|
||||
|
||||
**action** — one-shot operations. **Only `redeploy` is included**:
|
||||
- `redeploy` → **inline**: field `git_revision` in the main resource. When it changes, call redeploy instead of (or after) modify
|
||||
- `restart`, `recovery`, `reconcile` → **excluded**. These are manual operational tasks, performed via UI only. Reason:
|
||||
- `restart` — modify handles pod restart when needed
|
||||
- `recovery` — creates a new instance from backup, not a modification of existing
|
||||
- `reconcile` — sync after manual changes; Terraform owns its own state
|
||||
|
||||
### Idempotency Rules
|
||||
|
||||
- instance CRUD is idempotent via standard Terraform behavior.
|
||||
- subresource CRUD is idempotent by resource identity.
|
||||
- `redeploy`: idempotent via `git_revision` field — if unchanged, no redeploy
|
||||
|
||||
### Lifecycle Behavior
|
||||
|
||||
- For `instance` resources with `suspend`/`resume`, use explicit flags:
|
||||
`adopt_existing_on_create` (default `false`) and `suspend_on_destroy` (default `true`).
|
||||
- Apply decision matrix for suspend-capable services:
|
||||
- cloud status `missing` or `deleted` -> `create`
|
||||
- `suspend` + `adopt_existing_on_create=true` + key params match -> `resume` + `adopt`
|
||||
- `suspend` + `adopt_existing_on_create=false` -> error (explicitly require flag for resume/adopt)
|
||||
- `suspend` + key params mismatch -> error
|
||||
- `running` + `adopt_existing_on_create=true` -> adopt/import behavior
|
||||
- `running` + `adopt_existing_on_create=false` -> error
|
||||
- `not created` -> error, no auto-adopt/create
|
||||
- `creating`/`pending`/`failed` -> error
|
||||
- Conflict diagnostics requirement:
|
||||
- When `resource_name` already exists and `adopt_existing_on_create=false`, diagnostics must explicitly offer two choices:
|
||||
1) change `resource_name` to create a new resource;
|
||||
2) import/adopt existing one by setting `adopt_existing_on_create=true` and re-running `apply`.
|
||||
- Destroy behavior for suspend-capable services:
|
||||
- `suspend_on_destroy=true` -> call `suspend`
|
||||
- `suspend_on_destroy=false` -> remove from Terraform state only (no API call)
|
||||
|
||||
### Diagnostics Format
|
||||
|
||||
- Lifecycle diagnostics for plan/apply must be multiline and human-readable.
|
||||
- Include decision reason and controlling flag in message body.
|
||||
- Print details as separate lines: `resource_name`, `service_id`, `instance_uid`,
|
||||
`status`, `status_raw`, `operation_pending`, `operation_in_progress`.
|
||||
|
||||
## Provider Model
|
||||
|
||||
- Core (`provider/internal/core`) is universal: no service-specific logic inside it.
|
||||
- Generated service resources (`provider/internal/resources_gen`) contain only
|
||||
schema/params and references.
|
||||
- Hand-written service resources live in `provider/internal/resources_core` and are
|
||||
registered in `provider/internal/provider/provider.go` `Resources()`. They are
|
||||
NOT generated; they must not contain arbitrary service logic, only:
|
||||
- a schema, and
|
||||
- wiring between schema fields and the universal core API
|
||||
(`RunInstanceOperationUniversalByCode`, `ResolveRefSvcParamValue`, etc.).
|
||||
|
||||
### API Resilience
|
||||
|
||||
- Core MUST retry transient 401 errors from Gateway (3 attempts, exponential backoff).
|
||||
Gateway may temporarily reject valid JWT tokens.
|
||||
- **CURRENTLY NOT IMPLEMENTED** — `isRetryable` (`core/http.go`) retries only
|
||||
{429, 502, 503, 504}, NOT 401, and only for GET. This is a known gap versus the
|
||||
intent above; fix in `core/http.go` `isRetryable`.
|
||||
|
||||
### Generated Code Resilience
|
||||
|
||||
- **Zero-value fallback** (`normalizeUniversalValueV6`): если параметр отсутствует
|
||||
в пользовательском `.tf`, подставлять zero-value по `dataType`:
|
||||
- `integer` → `"0"`, `boolean` → `"false"`, `map-fixed` → `"{}"`, `array` → `"[]"`, `string` → `""`
|
||||
- Это предотвращает NullPointerException на стороне API при добавлении новых полей.
|
||||
- **map-fixed default из DataDescriptor** (`buildMapFixedDefault`): если API возвращает
|
||||
`dataDescriptor` для `map-fixed`-параметра, а значение отсутствует или равно `"{}"`,
|
||||
строится JSON из дефолтов sub-параметров (`{"type":"off","durationCA":"175200",...}`).
|
||||
Это гарантирует что API получит все обязательные sub-поля с их значениями по умолчанию.
|
||||
- **s3Uid-резолв в map-fixed** (`resolveS3UidInMapFixed`): для map-fixed-параметров
|
||||
парсится JSON, ищутся ключи по паттерну `s3.*uid` (case-insensitive), значения-не-UUID
|
||||
резолвятся в UUID через S3 (сервис 12). Не зависит от DataDescriptor API.
|
||||
Соглашение об именах: любой sub-param с `s3`+`uid` в имени → S3 (12).
|
||||
- **modify всегда через WithDefaults** (`RunInstanceOperationUniversalWithDefaults`):
|
||||
modify-операции запрашивают `cfsParams` у API и отправляют все параметры,
|
||||
включая новые, с дефолтами из API.
|
||||
- **Nil-guard для nested-параметров** (шаблон `instance.go`): `map-fixed`-параметры
|
||||
(указатели на вложенные структуры) проверяются на nil перед доступом к sub-полям.
|
||||
Если состояние создано до добавления нового `map-fixed`-параметра — он будет nil,
|
||||
и код не должен падать с nil pointer dereference. Вместо этого параметр пропускается,
|
||||
и zero-value fallback подставит `{}`.
|
||||
- **Nested-атрибуты с default — Optional без Computed** (`NestedSchemaBlock` в `helpers.go`):
|
||||
`map-fixed`-параметры с default генерируются как `Optional: true` (без `Computed`),
|
||||
потому что провайдер не вычисляет nested-значения из API. `Computed` приводил бы к
|
||||
`unknown`-значению, которое нельзя декодировать в конкретный тип `*Struct`.
|
||||
- **Merge SubParams union** (`params.go`): при совпадении `Code` параметра в разных
|
||||
операциях (create/modify) с разными наборами sub-params, `Merge()` объединяет их
|
||||
union'ом, а не отбрасывает второй. Это гарантирует что структура содержит все поля.
|
||||
|
||||
### Subresource Resources
|
||||
|
||||
Subresource operations are exposed as standard resources.
|
||||
Example mapping:
|
||||
- create_user/delete_user/modify_user => nubes_<service>_user
|
||||
- create_database/delete_database => nubes_<service>_database
|
||||
|
||||
Example HCL:
|
||||
|
||||
resource "nubes_postgres_user" "user1" {
|
||||
postgres_id = nubes_postgres.db.id
|
||||
username = "app_user"
|
||||
role = "app_user"
|
||||
}
|
||||
|
||||
### Redeploy (inline action)
|
||||
|
||||
Services with `redeploy` operation get a `git_revision` field in the main resource.
|
||||
Changing `git_revision` triggers `redeploy` instead of `modify`.
|
||||
|
||||
Example HCL:
|
||||
|
||||
resource "nubes_flask" "app" {
|
||||
resource_name = "my-flask"
|
||||
git_revision = "abc123" # ← change this to trigger redeploy
|
||||
app_configuration = jsonencode({...})
|
||||
}
|
||||
|
||||
## Concurrency and State Locking
|
||||
|
||||
**Provider-level guarantees:**
|
||||
- The provider does NOT implement distributed locking for instance operations.
|
||||
- Two concurrent `terraform apply` with the same `resource_name` may create duplicate
|
||||
instances, leading to a «multiple instances found» error on subsequent applies.
|
||||
|
||||
**User responsibility:**
|
||||
- Use Terraform backend with state locking (S3+DynamoDB, etc.).
|
||||
- Do NOT run `terraform apply` from two workspaces against the same state simultaneously.
|
||||
- If duplicates occur: delete extras via Cloud Console and re-apply.
|
||||
|
||||
**API-side limitations:**
|
||||
- Nubes API does not enforce unique `displayName` per serviceId.
|
||||
- The provider cannot atomically guarantee «create-or-adopt» without API support
|
||||
for conditional creation or name uniqueness constraints.
|
||||
|
||||
## Documentation Model
|
||||
|
||||
From the unified YAML, generate:
|
||||
- Resource page: CRUD params, outputs, lifecycle defaults, operations summary
|
||||
- MAN page: service_man + parameter man blocks
|
||||
- Resources index: each resource links to its page and its MAN page
|
||||
|
||||
## Pipeline Overview (DevOps)
|
||||
|
||||
1) Generate unified YAML from API for services_list.txt.
|
||||
2) Generate provider code from YAML.
|
||||
3) Generate documentation from YAML.
|
||||
4) Build provider for linux/windows/darwin.
|
||||
5) Upload provider artifacts to registry.
|
||||
6) Build and publish docs to site.
|
||||
|
||||
## Lifecycle Vocabulary (single contract)
|
||||
|
||||
⚠️ The destroy-behaviour vocabulary is currently INCONSISTENT across resource kinds:
|
||||
|
||||
1. generated instance resources: runtime flags `suspend_on_destroy` / `keep_on_destroy`;
|
||||
2. generated modifiers: compile-time `delete_strategy` (no runtime flag);
|
||||
3. hand-written modifiers: runtime `keep_on_destroy` only.
|
||||
|
||||
All three express the same intent ("what happens to the platform effect on destroy").
|
||||
Canonical direction: one unified vocabulary/contract for all resource kinds.
|
||||
|
||||
## Non-Negotiable Rules
|
||||
|
||||
- No manual edits to generated YAML or generated Go code.
|
||||
- Any change to generated code must come from API or generator logic updates.
|
||||
- The generator must enforce these rules and fail fast on drift.
|
||||
|
||||
## Exception Registry (service-specific DATA, never logic)
|
||||
|
||||
Principle: provider core and generator logic are universal for all stands and
|
||||
services. Service-specific deviations are of two kinds:
|
||||
|
||||
1. **Generated modifiers** — a `modify` op that should become a dedicated modifier
|
||||
resource. The generator (`TOOLS/resource-generator/internal/loader/loader.go`)
|
||||
supports `kind: modifier` with `delete_strategy` (`noop_warn`/`inverse`/`error`)
|
||||
and `idempotency` (`none`/`check_before_run`). The overlay data file
|
||||
`modifiers.yaml` that would drive this is **documented but NOT yet created**;
|
||||
until then, modifiers are hand-written in `provider/internal/resources_core/`.
|
||||
(The legacy registry `serviceSpecificModifiers` in `TOOLS/yaml-generator/main.go`
|
||||
was REMOVED during the 2026-09-23 refactoring.)
|
||||
2. **Doc examples** — named registry:
|
||||
|
||||
| Registry | File | Declares |
|
||||
|---|---|---|
|
||||
| `serviceSpecificDocExamples` | `TOOLS/docs-generator/internal/writers/writers.go` | per-service doc examples, gated on service name + required state/vault keys |
|
||||
|
||||
Rules:
|
||||
|
||||
- Key by stable service NAME (slug), never by raw numeric ID.
|
||||
- Each entry answers WHAT / WHAT IT DOES / WHY / WHERE (see code comments).
|
||||
- Doc-example exception = editing the named registry above → visible in diff.
|
||||
- Modifier exception (until `modifiers.yaml` exists) = hand-written resource in
|
||||
`provider/internal/resources_core/` + registration in `provider.go`.
|
||||
- Never annotate API-YAML: it is machine-regenerated and edits would be lost.
|
||||
|
||||
Enforced by scripts (run before build/commit):
|
||||
|
||||
- `TOOLS/scripts/check_generated_drift.sh <stand>` — generated Go vs provider copy.
|
||||
- `TOOLS/scripts/check_hardcoded_service_ids.sh` — forbids `svc.ID == N` /
|
||||
`ServiceID == N` outside the registries.
|
||||
|
||||
Build rule: `provider/internal/resources_gen` and `provider/resources_yaml` are
|
||||
ephemeral by design and never a build source. Canonical build is
|
||||
`03_build_and_upload_provider.sh` (temp copy from `generated/<stand>/go`);
|
||||
`build-provider.sh` refuses direct build from `provider/`. For local IDE,
|
||||
`go build` and `go test`, materialize one stand first:
|
||||
`TOOLS/scripts/dev-materialize.sh <stand>` (output is git-ignored).
|
||||
Reference in New Issue
Block a user