terraform/tf_provider
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: rewrite devops/README.md + add TOOLS/README.md

Browse Source 
- Updated all paths from universal_rebuild to provider and TOOLS/
- Added pipeline diagram and script reference table
- Added TOOLS/README.md with build instructions for each generator
This commit is contained in:
“Naeel”
2026-07-05 10:35:45 +04:00
parent f1a9b9b23c
commit d49f19460d
2 changed files with 108 additions and 79 deletions
Download Patch File Download Diff File Expand all files Collapse all files
TOOLS/README.md
+41
View File
@@ -0,0 +1,41 @@
# TOOLS — Генераторы Terraform-провайдера Nubes
Каждый инструмент — независимый Go-модуль.
## yaml-generator
API Nubes → `resources_yaml/*.yaml`
```bash
cd yaml-generator && go build -o ../bin/yaml-generator .
./bin/yaml-generator
```
Структура: `main.go` + `internal/{client,config,normalize,spec,types}`.
## resource-generator
`resources_yaml/*.yaml` → `internal/resources_gen/*.go` + `registry.go`
```bash
cd resource-generator && go build -o ../bin/resource-generator .
./bin/resource-generator
```
Структура: `main.go` + `internal/{helpers,loader,params,templates,types,writers}`.
## docs-generator
`resources_yaml/*.yaml` → Markdown-документация в `docs/30_registry/resources/`
```bash
cd docs-generator && go build -o ../bin/docs-generator .
# Документация ресурсов
./bin/docs-generator
# Operations-документация
./bin/docs-generator --ops
```
Структура: `main.go` + `internal/{types,writers,ops}`.
devops/README.md
+67 -79
View File
@@ -1,103 +1,91 @@
# DevOps Runbook: Provider Build Pipeline # DevOps — Pipeline сборки Terraform-провайдера Nubes
This folder describes the full pipeline from API discovery to provider + docs publication. ## Обзор
## Read This First ```
API Nubes → YAML → Go-ресурсы → Провайдер → S3 Registry
↘ Markdown-доки → MkDocs-сайт
```
1) Start here: `devops/README.md` (this file). ## Инструменты (TOOLS/)
1a) Architecture: `devops/ARCHITECTURE.md`.
1b) Mandatory suspend lifecycle policy: `docs/60_strategy/provider_philosophy.md` (sections 7-9).
2) Service list: `devops/config/services_list.txt`.
2a) Operation timeouts config: `devops/config/operation_timeouts.json`.
3) Unified YAML generator: `universal_rebuild/tools/service_spec_gen`.
4) Go resource generator: `universal_rebuild/tools/gen_v2`.
5) Docs generator: `devops/02_generate_resources_and_docs_v2.sh`.
6) Build + publish scripts: `devops/03_build_and_upload_provider.sh`, `devops/04_build_and_publish_docs.sh`.
## Overview (end-to-end) | Бинарник | Исходники | Назначение |
|---|---|---|
| `TOOLS/bin/yaml-generator` | `TOOLS/yaml-generator/` | API → `resources_yaml/*.yaml` |
| `TOOLS/bin/resource-generator` | `TOOLS/resource-generator/` | YAML → `internal/resources_gen/*.go` |
| `TOOLS/bin/docs-generator` | `TOOLS/docs-generator/` | YAML → Markdown-документация |
1) Fetch all service metadata from API Каждый инструмент — независимый Go-модуль со своим `go.mod`.
2) Generate unified YAML specs from API
3) Generate Go resources + documentation files from unified YAML
4) Build provider binaries for 3 OS targets and upload to Registry
5) Build and publish documentation site
## Generator code locations ## Скрипты пайплайна
- Unified YAML from API: `universal_rebuild/tools/service_spec_gen` | # | Скрипт | Что делает |
- Go resources from YAML: `universal_rebuild/tools/gen_v2` |---|---|---|
- Docs generator: `devops/02_generate_resources_and_docs_v2.sh` | 00 | `00_token_manager.sh` | Получение/обновление API-токенов |
| 01 | `01_generate_yamls.sh` | API → YAML (требует `--profile`) |
| 02 | `02_generate_resources_and_docs_v2.sh` | YAML → Go + Docs (требует `--profile`) |
| 03 | `03_build_and_upload_provider.sh` | Сборка провайдера → GPG → S3 |
| 04 | `04_build_and_publish_docs.sh` | MkDocs-сайт → публикация |
| — | `10_yaml_stability_run.sh` | Тест стабильности (N запусков 01) |
| — | `11_yaml_stability_run_latest.sh` | Авто-токен → 10 |
| — | `12_generate_yamls_latest.sh` | Авто-токен → 01 |
| — | `13_generate_yamls_clean.sh` | Очистка YAML + авто-токен → 01 |
| — | `build-provider.sh` | Быстрая локальная сборка (без S3) |
## Prerequisites ## Профили стендов
- Go 1.22+ ```
- `python3` devops/profiles/
- `gpg` ├── test/profile.env # lk-api-gateway-test.ngcloud.ru
- `mc` (MinIO/S3 client) ├── prod/profile.env # продакшн
- Docker (for mkdocs build) └── dev/profile.env # дев-стенд
```
## Shared settings Каждый профиль: свой endpoint, токен, версия, S3-бакет, GPG-ключи.
S3 environment: ## Полный пайплайн (пример для test-стенда)
- `S3_ENDPOINT` (example: `https://s3.msk-1.ngcloud.ru`)
- `S3_ACCESS_KEY`
- `S3_SECRET_KEY`
Tip: the local S3 config lives at `secrets/.s3cfg_registry`.
Operation timeouts config:
- Source of truth: `devops/config/operation_timeouts.json`
- During provider build, this file is copied into embedded provider config by `devops/03_build_and_upload_provider.sh`.
Provider naming defaults:
- `REGISTRY_HOSTNAME`: `terra.k8c.ru`
- `NAMESPACE`: `nubes`
- `NAME`: `nubes`
## Stand profiles (TEST/PROD/DEV)
To avoid overwrite between stands, use per-stand profiles:
- `devops/profiles/test`
- `devops/profiles/prod`
- `devops/profiles/dev`
Each profile contains:
- `profile.env` (API URL, token file, versions, S3/GPG paths)
- `services_list.txt`
- `operation_timeouts.json`
- `generated/` (`resources_yaml`, `go`, `docs`, `json`, `tmp`)
Run scripts with profile path:
```bash ```bash
# 1. Токен
./devops/00_token_manager.sh --profile devops/profiles/test
# 2. YAML из API
./devops/01_generate_yamls.sh --profile devops/profiles/test ./devops/01_generate_yamls.sh --profile devops/profiles/test
# 3. Go-ресурсы + документация
./devops/02_generate_resources_and_docs_v2.sh --profile devops/profiles/test ./devops/02_generate_resources_and_docs_v2.sh --profile devops/profiles/test
# 4. Сборка и загрузка провайдера
./devops/03_build_and_upload_provider.sh --profile devops/profiles/test ./devops/03_build_and_upload_provider.sh --profile devops/profiles/test
# 5. Публикация документации
./devops/04_build_and_publish_docs.sh --profile devops/profiles/test ./devops/04_build_and_publish_docs.sh --profile devops/profiles/test
``` ```
`--profile` is required for generation/build scripts in strict mode. ## Структура проекта
Generated artifacts are stored only in `devops/profiles/<stand>/generated/*`.
`universal_rebuild` is treated as code-only source directory.
## Step 1: Generate unified YAML from API
Script: `01_generate_yamls.sh`
Input list of services:
- `config/services_list.txt` (service_id only)
Token options:
- `TOKEN_FILE=<repo_root>/HH-MM-SS.token`, or
- `NUBES_API_TOKEN` directly
Example:
```bash
export TOKEN_FILE=<repo_root>/08-33-41.token
./devops/01_generate_yamls.sh
``` ```
tf_provider/
├── TOOLS/ # Генераторы (независимые Go-модули)
│ ├── yaml-generator/
│ ├── resource-generator/
│ ├── docs-generator/
│ └── bin/ # Скомпилированные бинарники
├── provider/ # Terraform-провайдер (Go-модуль)
│ ├── internal/
│ │ ├── core/ # HTTP-клиент API
│ │ ├── provider/ # Регистрация провайдера
│ │ ├── resources_core/ # Общая логика CRUD
│ │ └── resources_gen/ # Сгенерированные ресурсы
│ └── main.go
└── devops/ # Скрипты пайплайна + профили
├── profiles/
└── config/
```
## Архитектура
Подробно: `devops/ARCHITECTURE.md`.
Outputs: Outputs:
- YAML files in `universal_rebuild/resources_yaml` (file names use `ID_name.yaml`) - YAML files in `universal_rebuild/resources_yaml` (file names use `ID_name.yaml`)