docs: rewrite devops/README.md + add TOOLS/README.md

- Updated all paths from universal_rebuild to provider and TOOLS/ - Added pipeline diagram and script reference table - Added TOOLS/README.md with build instructions for each generator
2026-07-05 10:35:45 +04:00
parent f1a9b9b23c
commit d49f19460d
2 changed files with 108 additions and 79 deletions
@@ -0,0 +1,41 @@
+# TOOLS — Генераторы Terraform-провайдера Nubes
+
+Каждый инструмент — независимый Go-модуль.
+
+## yaml-generator
+
+API Nubes → `resources_yaml/*.yaml`
+
+```bash
+cd yaml-generator && go build -o ../bin/yaml-generator .
+./bin/yaml-generator
+```
+
+Структура: `main.go` + `internal/{client,config,normalize,spec,types}`.
+
+## resource-generator
+
+`resources_yaml/*.yaml` → `internal/resources_gen/*.go` + `registry.go`
+
+```bash
+cd resource-generator && go build -o ../bin/resource-generator .
+./bin/resource-generator
+```
+
+Структура: `main.go` + `internal/{helpers,loader,params,templates,types,writers}`.
+
+## docs-generator
+
+`resources_yaml/*.yaml` → Markdown-документация в `docs/30_registry/resources/`
+
+```bash
+cd docs-generator && go build -o ../bin/docs-generator .
+
+# Документация ресурсов
+./bin/docs-generator
+
+# Operations-документация  
+./bin/docs-generator --ops
+```
+
+Структура: `main.go` + `internal/{types,writers,ops}`.
@@ -1,103 +1,91 @@
-# DevOps Runbook: Provider Build Pipeline
+# DevOps — Pipeline сборки Terraform-провайдера Nubes

-This folder describes the full pipeline from API discovery to provider + docs publication.
+## Обзор

-## Read This First
+```
+API Nubes → YAML → Go-ресурсы → Провайдер → S3 Registry
+                    ↘ Markdown-доки → MkDocs-сайт
+```

-1) Start here: `devops/README.md` (this file).
-1a) Architecture: `devops/ARCHITECTURE.md`.
-1b) Mandatory suspend lifecycle policy: `docs/60_strategy/provider_philosophy.md` (sections 7-9).
-2) Service list: `devops/config/services_list.txt`.
-2a) Operation timeouts config: `devops/config/operation_timeouts.json`.
-3) Unified YAML generator: `universal_rebuild/tools/service_spec_gen`.
-4) Go resource generator: `universal_rebuild/tools/gen_v2`.
-5) Docs generator: `devops/02_generate_resources_and_docs_v2.sh`.
-6) Build + publish scripts: `devops/03_build_and_upload_provider.sh`, `devops/04_build_and_publish_docs.sh`.
+## Инструменты (TOOLS/)

-## Overview (end-to-end)
+| Бинарник | Исходники | Назначение |
+|---|---|---|
+| `TOOLS/bin/yaml-generator` | `TOOLS/yaml-generator/` | API → `resources_yaml/*.yaml` |
+| `TOOLS/bin/resource-generator` | `TOOLS/resource-generator/` | YAML → `internal/resources_gen/*.go` |
+| `TOOLS/bin/docs-generator` | `TOOLS/docs-generator/` | YAML → Markdown-документация |

-1) Fetch all service metadata from API
-2) Generate unified YAML specs from API
-3) Generate Go resources + documentation files from unified YAML
-4) Build provider binaries for 3 OS targets and upload to Registry
-5) Build and publish documentation site
+Каждый инструмент — независимый Go-модуль со своим `go.mod`.

-## Generator code locations
+## Скрипты пайплайна

- Unified YAML from API: `universal_rebuild/tools/service_spec_gen`
- Go resources from YAML: `universal_rebuild/tools/gen_v2`
- Docs generator: `devops/02_generate_resources_and_docs_v2.sh`
+| # | Скрипт | Что делает |
+|---|---|---|
+| 00 | `00_token_manager.sh` | Получение/обновление API-токенов |
+| 01 | `01_generate_yamls.sh` | API → YAML (требует `--profile`) |
+| 02 | `02_generate_resources_and_docs_v2.sh` | YAML → Go + Docs (требует `--profile`) |
+| 03 | `03_build_and_upload_provider.sh` | Сборка провайдера → GPG → S3 |
+| 04 | `04_build_and_publish_docs.sh` | MkDocs-сайт → публикация |
+| — | `10_yaml_stability_run.sh` | Тест стабильности (N запусков 01) |
+| — | `11_yaml_stability_run_latest.sh` | Авто-токен → 10 |
+| — | `12_generate_yamls_latest.sh` | Авто-токен → 01 |
+| — | `13_generate_yamls_clean.sh` | Очистка YAML + авто-токен → 01 |
+| — | `build-provider.sh` | Быстрая локальная сборка (без S3) |

-## Prerequisites
+## Профили стендов

- Go 1.22+
- `python3`
- `gpg`
- `mc` (MinIO/S3 client)
- Docker (for mkdocs build)
+```
+devops/profiles/
+├── test/profile.env    # lk-api-gateway-test.ngcloud.ru
+├── prod/profile.env    # продакшн
+└── dev/profile.env     # дев-стенд
+```

-## Shared settings
+Каждый профиль: свой endpoint, токен, версия, S3-бакет, GPG-ключи.

-S3 environment:
- `S3_ENDPOINT` (example: `https://s3.msk-1.ngcloud.ru`)
- `S3_ACCESS_KEY`
- `S3_SECRET_KEY`
-
-Tip: the local S3 config lives at `secrets/.s3cfg_registry`.
-
-Operation timeouts config:
- Source of truth: `devops/config/operation_timeouts.json`
- During provider build, this file is copied into embedded provider config by `devops/03_build_and_upload_provider.sh`.
-
-Provider naming defaults:
- `REGISTRY_HOSTNAME`: `terra.k8c.ru`
- `NAMESPACE`: `nubes`
- `NAME`: `nubes`
-
-## Stand profiles (TEST/PROD/DEV)
-
-To avoid overwrite between stands, use per-stand profiles:
-
- `devops/profiles/test`
- `devops/profiles/prod`
- `devops/profiles/dev`
-
-Each profile contains:
-
- `profile.env` (API URL, token file, versions, S3/GPG paths)
- `services_list.txt`
- `operation_timeouts.json`
- `generated/` (`resources_yaml`, `go`, `docs`, `json`, `tmp`)
-
-Run scripts with profile path:
+## Полный пайплайн (пример для test-стенда)

 ```bash
+# 1. Токен
+./devops/00_token_manager.sh --profile devops/profiles/test
+
+# 2. YAML из API
 ./devops/01_generate_yamls.sh --profile devops/profiles/test
+
+# 3. Go-ресурсы + документация
 ./devops/02_generate_resources_and_docs_v2.sh --profile devops/profiles/test
+
+# 4. Сборка и загрузка провайдера
 ./devops/03_build_and_upload_provider.sh --profile devops/profiles/test
+
+# 5. Публикация документации
 ./devops/04_build_and_publish_docs.sh --profile devops/profiles/test
 ```

-`--profile` is required for generation/build scripts in strict mode.
-Generated artifacts are stored only in `devops/profiles/<stand>/generated/*`.
-`universal_rebuild` is treated as code-only source directory.
+## Структура проекта

-## Step 1: Generate unified YAML from API
-
-Script: `01_generate_yamls.sh`
-
-Input list of services:
- `config/services_list.txt` (service_id only)
-
-Token options:
- `TOKEN_FILE=<repo_root>/HH-MM-SS.token`, or
- `NUBES_API_TOKEN` directly
-
-Example:
-```bash
-export TOKEN_FILE=<repo_root>/08-33-41.token
-./devops/01_generate_yamls.sh
 ```
+tf_provider/
+├── TOOLS/               # Генераторы (независимые Go-модули)
+│   ├── yaml-generator/
+│   ├── resource-generator/
+│   ├── docs-generator/
+│   └── bin/              # Скомпилированные бинарники
+├── provider/             # Terraform-провайдер (Go-модуль)
+│   ├── internal/
+│   │   ├── core/         # HTTP-клиент API
+│   │   ├── provider/     # Регистрация провайдера
+│   │   ├── resources_core/ # Общая логика CRUD
+│   │   └── resources_gen/  # Сгенерированные ресурсы
+│   └── main.go
+└── devops/               # Скрипты пайплайна + профили
+    ├── profiles/
+    └── config/
+```
+
+## Архитектура
+
+Подробно: `devops/ARCHITECTURE.md`.

 Outputs:
 - YAML files in `universal_rebuild/resources_yaml` (file names use `ID_name.yaml`)