tf_provider/HOW_TO/DEVOPS_BUILD_PIPELINE.md

DevOps Runbook: Provider Build Pipeline

Перенесено из корневого README.md 2026-09-24 (в корне теперь — карта проекта). Пути и версии в тексте приведены к текущему состоянию репозитория.

Пайплайн сборки провайдера. Скрипты живут в TOOLS/scripts/ (НЕ в корне репозитория).

Overview

1. Generate YAML specs from API
2. Generate Go resources + documentation files from YAML
3. Build and upload provider binaries for 3 OS targets
4. Build and publish documentation site

Documentation publishing instructions

The verified documentation generation and publishing pipeline is documented in ../HISTORY/50_docs/2026-09-03_docs_upload_pipeline_verified.md. It covers the generated docs source, MkDocs build, the separate documentation S3 bucket, VM upload and mirror steps, stand-specific URLs, and the legacy script that must not be used.

Prerequisites

• Go 1.22+
• python3
• gpg
• mc (MinIO/S3 client)
• Docker (for mkdocs build)

Shared settings

S3 environment:

• S3_ENDPOINT (example: https://s3.msk-1.ngcloud.ru)
• S3_ACCESS_KEY
• S3_SECRET_KEY

Provider naming defaults:

• REGISTRY_HOSTNAME: tf-registry.containerk8s.services.ngcloud.ru
• NAMESPACE: nubes
• NAME: nubes

Step 1: Generate YAMLs from API

Script: TOOLS/scripts/01_generate_yamls.sh --profile TOOLS/config/<стенд>

Input list of services:

• TOOLS/config/<стенд>/services_list.txt (service_id + name; у каждого стенда свой список)

Token options:

• TOKEN_FILE=/home/naeel/terra/HH-MM-SS.token, or
• NUBES_API_TOKEN directly

Example:

export TOKEN_FILE=/home/naeel/terra/08-33-41.token
./TOOLS/scripts/01_generate_yamls.sh --profile TOOLS/config/dev

Step 2: Generate Go resources and docs

Script: TOOLS/scripts/02_generate_resources_and_docs_v2.sh

Example:

./TOOLS/scripts/02_generate_resources_and_docs_v2.sh --profile TOOLS/config/dev

Outputs:

• Go files in generated/<stand>/go
• Docs in generated/<stand>/docs

Important:

• The v2 script always rebuilds resource-generator and docs-generator from source before running.
• Do not invoke stale binaries from TOOLS/resource-generator/bin/ or TOOLS/docs-generator/bin/ directly.

Step 3: Build and upload provider

Script: 03_build_and_upload_provider.sh

Uses registry-server-build/build-provider.sh and signs with:

• secrets/private_key.asc (ignored by git)

Example:

export S3_ENDPOINT=https://s3.msk-1.ngcloud.ru
export S3_ACCESS_KEY=...
export S3_SECRET_KEY=...
./TOOLS/scripts/03_build_and_upload_provider.sh --profile TOOLS/config/dev 2.0.18

Step 4: Build and publish docs

Script: 04_build_and_publish_docs.sh

Example:

export S3_ENDPOINT=https://s3.msk-1.ngcloud.ru
export S3_ACCESS_KEY=...
export S3_SECRET_KEY=...
./TOOLS/scripts/04_build_and_publish_docs.sh --profile TOOLS/config/dev 2.0.18

Notes

• The GPG private key must remain stable across releases. Do not regenerate per build.
• If the key is regenerated, the registry server must be updated to serve the new public key.
• Terraform will fail with authentication signature from unknown issuer if the registry public key does not match the signing key.
• TOOLS/config/<стенд>/services_list.txt — источник правды по тому, какие сервисы генерируются (у каждого стенда свой).
• Если меняется версия провайдера — обновить provider/main.go (ранее universal_rebuild/main.go — устаревший путь).

One-time GPG bootstrap (do this once, keep the key stable)

1. Generate and export keys (no passphrase):

GPG_DIR=${ROOT_DIR}/secrets
GNUPGHOME=$(mktemp -d)
cat > /tmp/gpg_batch <<'EOF'
%no-protection
Key-Type: RSA
Key-Length: 4096
Subkey-Type: RSA
Subkey-Length: 4096
Name-Real: tazet@narod.ru
Name-Email: tazet@narod.ru
Expire-Date: 0
EOF
gpg --batch --homedir "$GNUPGHOME" --gen-key /tmp/gpg_batch
gpg --batch --homedir "$GNUPGHOME" --armor --export-secret-keys > "$GPG_DIR/private_key.asc"
gpg --batch --homedir "$GNUPGHOME" --armor --export > "$GPG_DIR/public_key.asc"
rm -rf "$GNUPGHOME" /tmp/gpg_batch

2. Update registry server public key (ASCII Armor) in:

• registry-server-build/main.go
• operator/cmd/registry/main.go

3. Rebuild and redeploy the registry server (see docs/50_history/00_system_mechanics.md).

4. Build and upload provider artifacts as usual.

check string