mirror of
https://github.com/game-ci/unity-builder.git
synced 2026-05-31 13:56:13 -07:00
ef0555fb0d
* feat(orchestrator): enterprise feature support — CLI provider, submodule profiles, caching, LFS, hooks Add generic enterprise-grade features to the orchestrator, enabling Unity projects with complex CI/CD pipelines to adopt game-ci/unity-builder with built-in support for: - CLI provider protocol: JSON-over-stdin/stdout bridge enabling providers in any language (Go, Python, Rust, shell) via the `providerExecutable` input - Submodule profiles: YAML-based selective submodule initialization with glob patterns and variant overlays (`submoduleProfilePath`, `submoduleVariantPath`) - Local build caching: Filesystem-based Library and LFS caching for local builds without external cache actions (`localCacheEnabled`, `localCacheRoot`) - Custom LFS transfer agents: Register external transfer agents like elastic-git-storage (`lfsTransferAgent`, `lfsTransferAgentArgs`, `lfsStoragePaths`) - Git hooks support: Detect and install lefthook/husky with configurable skip lists (`gitHooksEnabled`, `gitHooksSkipList`) Also removes all `orchestrator-develop` branch references, replacing with `main`. 13 new action inputs, 13 new files, 14 new CLI provider tests, 17 submodule tests, plus cache/LFS/hooks unit tests. All 452 tests pass. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(orchestrator): add experimental GCP Cloud Run and Azure ACI providers Add two new cloud provider implementations for the orchestrator, both marked as experimental: - **GCP Cloud Run Jobs** (`providerStrategy: gcp-cloud-run`): Executes Unity builds as Cloud Run Jobs with GCS FUSE for large artifact storage. Supports configurable machine types, service accounts, and VPC connectors. 7 new inputs (gcpProject, gcpRegion, gcpBucket, gcpMachineType, gcpDiskSizeGb, gcpServiceAccount, gcpVpcConnector). - **Azure Container Instances** (`providerStrategy: azure-aci`): Executes Unity builds as ACI containers with Azure File Shares (Premium FileStorage) for large artifact storage up to 100 TiB. Supports configurable CPU/memory, VNet integration, and subscription targeting. 9 new inputs (azureResourceGroup, azureLocation, azureStorageAccount, azureFileShareName, azureSubscriptionId, azureCpu, azureMemoryGb, azureDiskSizeGb, azureSubnetId). Both providers use their respective CLIs (gcloud, az) for infrastructure management and support garbage collection of old build resources. No tests included as these require real cloud infrastructure to validate. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(orchestrator): multi-storage support for GCP and Azure providers Both providers now support four storage backends via gcpStorageType / azureStorageType: GCP Cloud Run: - gcs-fuse: Mount GCS bucket as POSIX filesystem (unlimited, best for large sequential I/O) - gcs-copy: Copy artifacts in/out via gsutil (simpler, no FUSE overhead) - nfs: Filestore NFS mount (true POSIX, good random I/O, up to 100 TiB) - in-memory: tmpfs (fastest, volatile, up to 32 GiB) Azure ACI: - azure-files: SMB file share mount (up to 100 TiB, premium throughput) - blob-copy: Copy artifacts in/out via az storage blob (no mount overhead) - azure-files-nfs: NFS 4.1 file share mount (true POSIX, no SMB lock overhead) - in-memory: emptyDir tmpfs (fastest, volatile, limited by container memory) New inputs: gcpStorageType, gcpFilestoreIp, gcpFilestoreShare, azureStorageType, azureBlobContainer. Constructor validates storage config and warns on missing prerequisites (e.g. NFS requires VPC connector/subnet). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(orchestrator): automatic provider fallback with runner availability check Adds built-in load balancing: check GitHub runner availability before builds start, auto-route to a fallback provider when runners are busy or offline. Eliminates the need for a separate check-runner job. New inputs: fallbackProviderStrategy, runnerCheckEnabled, runnerCheckLabels, runnerCheckMinAvailable. Outputs providerFallbackUsed and providerFallbackReason for workflow visibility. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(orchestrator): add retry-on-fallback and provider init timeout Adds retryOnFallback (retry failed builds on alternate provider) and providerInitTimeout (swap provider if init takes too long). Refactors run() into run()/runWithProvider() to support retry loop. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: format changed files with prettier Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test(orchestrator): expand local cache service test coverage Adds tests for cache hit restore (picks latest tar), LFS cache restore/save, garbage collection age filtering, and edge cases like permission errors and empty directories. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test(orchestrator): add runner availability service tests Covers: no token skip, no runners fallback, busy/offline runners, label filtering (case-insensitive), minAvailable threshold, fail-open on API error, mixed runner states. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test(orchestrator): add unit tests for untested core services Adds 64 new mock-based unit tests covering orchestrator services that previously had zero test coverage: - TaskParameterSerializer: env var format conversion, round-trip, uniqBy deduplication, blocked params, default secrets - FollowLogStreamService: build output message parsing — end of transmission, build success/failure detection, error accumulation, Library rebuild detection - OrchestratorNamespace (guid): GUID generation format, platform name normalization, nanoid uniqueness - OrchestratorFolders: path computation for all folder getters, ToLinuxFolder conversion, repo URL generation, purge flag detection All tests are pure mock-based and run without any external infrastructure (no LocalStack, K8s, Docker, or AWS). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci(orchestrator): add fast unit test gate to integrity workflow Adds a fast-fail unit test step at the top of orchestrator-integrity, right after yarn install and before any infrastructure setup (k3d, LocalStack). Runs 113 mock-based orchestrator tests in ~5 seconds. If serialization, path computation, log parsing, or provider loading is broken, the workflow fails immediately instead of spending 30+ minutes setting up LocalStack and k3d clusters. Tests included: orchestrator-guid, orchestrator-folders, task-parameter-serializer, follow-log-stream-service, runner-availability-service, provider-url-parser, provider-loader, provider-git-manager, orchestrator-image, orchestrator-hooks, orchestrator-github-checks. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test(orchestrator): expand unit tests for enterprise services Add comprehensive tests for CLI provider (cleanupWorkflow, garbageCollect, listWorkflow, watchWorkflow, stderr forwarding, timeout handling), local cache service (saveLfsCache full path and error handling), git hooks service (husky install, failure logging, edge cases), and LFS agent service (empty storagePaths, validate logging). 73 tests across 4 test files. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(orchestrator): use http.extraHeader for secure git authentication Replace token-in-URL pattern with http.extraHeader for git clone and LFS operations. The token no longer appears in clone URLs, git remote config, or process command lines. Add gitAuthMode input (default: 'header', legacy: 'url') so users can fall back to the old behavior if needed. Closes #785 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(orchestrator): add premade secret sources and YAML definitions Add SecretSourceService with premade secret source integrations: - aws-secrets-manager (with --query SecretString for direct value) - aws-parameter-store (with --with-decryption) - gcp-secret-manager (latest version) - azure-key-vault (via $AZURE_VAULT_NAME env var) - env (environment variables, no shell command needed) - Custom commands (any string with {0} placeholder) - YAML file definitions for custom sources Add secretSource input that takes precedence over inputPullCommand. Backward compatible — existing inputPullCommand behavior unchanged. Closes #776 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(secrets): add HashiCorp Vault as first-class premade secret source Adds three Vault entries: hashicorp-vault (KV v2), hashicorp-vault-kv1 (KV v1), and vault (short alias). Uses VAULT_ADDR for server address and VAULT_MOUNT env var for configurable mount path (defaults to 'secret'). Refs #776 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(lfs): add built-in elastic-git-storage support with auto-install First-class support for elastic-git-storage as a custom LFS transfer agent. When lfsTransferAgent is set to "elastic-git-storage" (or "elastic-git-storage@v1.0.0" for a specific version), the service automatically finds or installs the agent from GitHub releases, then configures it via git config. Supports version pinning via @version suffix in the agent value, eliminating the need for a separate version parameter. Platform and architecture detection handles linux/darwin/windows on amd64/arm64. 37 unit tests covering detection, PATH lookup, installation, version parsing, and configuration delegation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(hooks): add Unity Git Hooks integration and runHookGroups Built-in support for Unity Git Hooks (com.frostebite.unitygithooks): - Auto-detect UPM package in Packages/manifest.json - Run init-unity-lefthook.js before hook installation - Set CI-friendly env vars (disable background project mode) New gitHooksRunBeforeBuild input runs specific lefthook groups before the Unity build, allowing CI to trigger pre-commit or pre-push checks that normally only fire on git events. 35 unit tests covering detection, init, CI env, group execution, and failure handling. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(orchestrator): add test workflow engine placeholder Initial scaffold for the test workflow engine service directory. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(orchestrator): add hot runner protocol placeholder Initial scaffold for the runner registration and hot editor provider module. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(orchestrator): generic artifact system — output types, manifests, and collection service Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(orchestrator): incremental sync protocol — git delta, direct input, and storage-backed sync Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: community plugin validation workflow (#800) Add scheduled workflow that validates community Unity packages compile and build correctly using unity-builder. Runs weekly on Sunday. Includes: - YAML plugin registry (community-plugins.yml) for package listings - Matrix expansion across plugins and platforms - Automatic failure reporting via GitHub issues - Manual trigger with plugin filter and Unity version override Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(orchestrator): CI platform providers — Remote PowerShell, GitHub Actions, GitLab CI, Ansible Add four new providers that delegate builds to external CI platforms: - remote-powershell: Execute on remote machines via WinRM/SSH - github-actions: Dispatch workflow_dispatch on target repository - gitlab-ci: Trigger pipeline via GitLab API - ansible: Run playbooks against managed inventory Each follows the CI-as-a-provider pattern: trigger remote job, pass build parameters, stream logs, report status. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: fix prettier formatting and eslint errors on test files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(orchestrator): build reliability features — git integrity, reserved filename cleanup, archival Add three optional reliability features for hardening CI pipelines: - Git corruption detection & recovery (fsck, stale lock cleanup, submodule backing store validation, auto-recovery) - Reserved filename cleanup (removes Windows device names that cause Unity asset importer infinite loops) - Build output archival with configurable retention policy All features are opt-in and fail gracefully with warnings only. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(reliability): implement build reliability service with git integrity, reserved filename cleanup, and build archival Adds BuildReliabilityService with the following capabilities: - checkGitIntegrity(): runs git fsck --no-dangling and parses output for corruption - cleanStaleLockFiles(): removes stale .lock files older than 10 minutes - validateSubmoduleBackingStores(): validates .git files point to valid backing stores - recoverCorruptedRepo(): orchestrates fsck, lock cleanup, re-fetch, retry fsck - cleanReservedFilenames(): removes Windows reserved filenames (con, prn, aux, nul, com1-9, lpt1-9) - archiveBuildOutput(): creates tar.gz archive of build output - enforceRetention(): deletes archives older than retention period - configureGitEnvironment(): sets GIT_TERMINAL_PROMPT=0, http.postBuffer, core.longpaths Wired into action.yml as opt-in inputs, with pre-build integrity checks and post-build archival in the main entry point. Includes 29 unit tests covering success and failure cases for all methods. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test(providers): add comprehensive unit tests for GitHub Actions, GitLab CI, PowerShell, and Ansible providers (#806) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(hot-runner): implement hot runner protocol with registry, health monitoring, and job dispatch (#791) Adds persistent Unity editor instance support to reduce build iteration time by eliminating cold-start overhead. Includes: - HotRunnerTypes: interfaces for config, status, job request/result, transport - HotRunnerRegistry: in-memory runner management with file-based persistence - HotRunnerHealthMonitor: periodic health checks, idle recycling, job-count recycling - HotRunnerDispatcher: job routing with wait-for-runner, timeout, and output streaming - HotRunnerService: high-level API integrating registry, health, and dispatch - 34 unit tests covering registration, filtering, health, dispatch, timeout, fallback - action.yml inputs for hot runner configuration (7 new inputs) - Input/BuildParameters integration for hot runner settings - index.ts wiring with cold-build fallback when hot runner unavailable Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(artifacts): complete generic artifact system with upload handlers, tests, and action integration (#798) - Add ArtifactUploadHandler with support for github-artifacts, storage (rclone), and local copy upload targets, including large file chunking for GitHub Artifacts - Add 44 unit tests covering OutputTypeRegistry, OutputService, and ArtifactUploadHandler (config parsing, upload coordination, file collection) - Add 6 new action.yml inputs for artifact configuration - Add artifactManifestPath action output - Wire artifact collection and upload into index.ts post-build flow Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(testing): implement test workflow engine with YAML suites, taxonomy filtering, and structured results (#790) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(sync): complete incremental sync protocol with storage-pull, state management, and tests (#799) - Add storage-pull strategy: rclone-based sync from remote storage with overlay and clean modes, URI parsing (storage://remote:bucket/path), transfer parallelism, and automatic rclone availability checking - Add SyncStateManager: persistent state load/save with configurable paths, workspace hash calculation via SHA-256 of key project files, and drift detection for external modification awareness - Add action.yml inputs: syncStrategy, syncInputRef, syncStorageRemote, syncRevertAfter, syncStatePath with sensible defaults - Wire sync into Input (5 getters), BuildParameters (5 fields), index.ts (local build path), and RemoteClient (orchestrator path) with post-job overlay revert when syncRevertAfter is true - Add 42 unit tests covering all strategies, URI parsing, state management, hash calculation, drift detection, error handling, and edge cases (missing rclone, invalid URIs, absent state, empty diffs) - Add root:true to eslintrc to prevent plugin resolution conflicts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(cache): add child workspace isolation for multi-product CI builds (#777) Implement two-level workspace isolation pattern for enterprise-scale CI: - Atomic O(1) workspace restore via filesystem move (no tar/download/extract) - Separate Library caching for independent restore - .git preservation for delta operations - Stale workspace cleanup with configurable retention policies - 5 new action inputs: childWorkspacesEnabled, childWorkspaceName, childWorkspaceCacheRoot, childWorkspacePreserveGit, childWorkspaceSeparateLibrary - 28 unit tests covering all service methods This enables enterprise CI where workspaces are 50GB+ and traditional caching via actions/cache is impractical. On NTFS, workspace restore is O(1) via atomic rename when source and destination are on the same volume. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(testing): use async exec for parallel test group execution Replace execSync with promisified exec so Promise.all actually runs test groups in parallel. Add native timeout support via exec options. Add 50MB maxBuffer for large Unity output. Fix ESLint violations (variable naming, padding lines, array push consolidation). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(cli-provider): add timeout protection for external CLI processes Prevent builds from hanging indefinitely when CLI provider subprocess is unresponsive. Default 2h for runTaskInWorkflow, 1h for watchWorkflow. Graceful SIGTERM with 10s grace before SIGKILL. - Added RUN_TASK_TIMEOUT_MS (2 hours) and WATCH_WORKFLOW_TIMEOUT_MS (1 hour) - Added gracefulKill helper: SIGTERM first, SIGKILL after 10s grace period - runTaskInWorkflow and watchWorkflow now have timeout protection - Existing execute() method upgraded to use gracefulKill - core.error() called with clear human-readable timeout message - Added comprehensive tests: timeout triggers, SIGKILL escalation, grace period cancellation on voluntary exit, normal completion Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(secrets): prevent shell injection in secret key names and mask values - Validate secret key names against alphanumeric allowlist before shell interpolation - Apply validation in both SecretSourceService.fetchSecret() and legacy queryOverride() - Mask fetched secret values with core.setSecret() to prevent log exposure - Add 20 new tests for validation and masking Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: rebuild dist for cli-provider timeout changes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(artifacts): validate rclone availability before storage upload Check for rclone binary before attempting storage-based uploads. Validate storage destination URI format (remoteName:path). Provide clear error message with install link when rclone is missing. Fail gracefully instead of cryptic ENOENT crash. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(load-balancing): add pagination limits and rate-limit detection Cap pagination at 100 pages (10,000 runners max), detect GitHub API rate limiting (403/429) with reset time reporting, add 30-second total timeout for pagination loop. Log clear diagnostic when no runners found suggesting possible causes (token permissions, runner registration). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(reliability): add disk space validation before build archival Check available disk space (cross-platform: wmic/df) before archive operations to prevent data loss on full disks. Skip archival with warning if insufficient space (10% safety margin). Clean up partial archives on tar failure. Proceed with warning when space check fails. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(hot-runner): validate persisted registry state and add dispatcher safeguards Validate runner entries when loading from hot-runners.json. Discard corrupted entries with warnings. Add validateAndRepair() method for runtime recovery. Validate data before persisting to prevent writing corrupt state. Handle corrupt persistence files (invalid JSON) gracefully. Rewrite executeWithTimeout using Promise.race to clean up transport connections on timeout. Fix pre-existing ESLint violations in dispatcher and test files. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(providers): add polling timeouts, fix credential parsing, validate dependencies - GitHub Actions: max 4-hour polling with clear timeout error including run URL - GitLab CI: max 4-hour polling with clear timeout error including pipeline URL - Remote PowerShell: fix credential split to preserve passwords with colons (split on first colon only instead of all colons) - Remote PowerShell: throw clear error when credential format is invalid - Ansible: validate ansible-playbook binary exists in setupWorkflow (separate from ansible --version check) - All timeout errors use core.error() for GitHub Actions annotation visibility Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: rebuild dist for provider timeout and credential fixes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: prettier formatting for orchestrator-folders-auth test Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: split orchestrator integrity into parallel jobs for faster validation Rewrite the monolith orchestrator-integrity.yml (1110 lines, single job, 3+ hour sequential execution) into 4 parallel jobs that run on separate runners: - k8s-tests: k3d cluster + LocalStack, 5 tests - aws-provider-tests: LocalStack only, 10 tests - local-docker-tests: Docker + LocalStack for S3 tests, 9 tests - rclone-tests: rclone + LocalStack, 1 test Key improvements: - Wall-clock time drops from ~3h to ~1h (longest single job) - Disk exhaustion eliminated: each job gets its own fresh 14GB runner - Cleanup logic deduplicated via sourced shell functions instead of 15 copy-pasted 30-line blocks - K3d node image cleanup only runs in the k8s job (where it matters) - Light cleanup (cache + docker prune -f) between tests; heavy cleanup (prune -af --volumes) only at job boundaries - workflow_call interface unchanged; integrity-check.yml needs no changes Ref: #794 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: fix prettier formatting Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: fix prettier formatting Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: fix prettier formatting Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: fix prettier formatting Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: fix prettier formatting Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add official game-ci CLI with build, activate, and orchestrate commands Introduces a yargs-based CLI entry point (src/cli.ts) distributed as the `game-ci` command. The CLI reuses existing unity-builder modules — Input, BuildParameters, Orchestrator, Docker, MacBuilder — so the same build engine powers both the GitHub Action and the standalone CLI. Commands: build, activate, orchestrate, cache (list/restore/clear), status, version. Closes #812 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(cli): add npm publish workflow and CLI tests Add .github/workflows/publish-cli.yml for publishing the CLI to npm on release or via manual workflow_dispatch with dry-run support. Add comprehensive test coverage for the CLI: - input-mapper.test.ts: 16 tests covering argument mapping, boolean conversion, yargs internal property filtering, and Cli.options population - commands.test.ts: 26 tests verifying command exports, builder flags, default values, and camelCase aliases for all six commands - cli-integration.test.ts: 8 integration tests spawning the CLI process to verify help output, version info, and error handling Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(cli): add release workflow, install scripts, and self-update command Replace the npm-only publish-cli.yml with a comprehensive release-cli.yml that builds standalone binaries via pkg for all platforms (Linux/macOS/Windows, x64/arm64), uploads them as GitHub Release assets with SHA256 checksums, and retains npm publish as an optional job. Add curl-pipe-sh installer (install.sh) and PowerShell installer (install.ps1) for one-liner installation from GitHub Releases. Both scripts auto-detect platform/architecture, verify checksums, and guide PATH configuration. Add `game-ci update` command for self-updating standalone binaries: checks GitHub releases for newer versions, downloads the correct platform binary, verifies it, and atomically replaces the running executable. Distribution strategy: GitHub Releases (primary), npm (optional), with winget/Homebrew/Chocolatey/Scoop as future providers. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(cli): address review findings — exit codes, missing inputs, null safety - Add process.exit(1) in cli.ts catch block so failures produce non-zero exit codes - Add 6 missing build inputs: containerRegistryRepository, containerRegistryImageVersion, dockerIsolationMode, sshPublicKeysDirectoryPath, cacheUnityInstallationOnMac, unityHubVersionOnMac - Add 6 missing orchestrate inputs: kubeStorageClass, readInputFromOverrideList, readInputOverrideCommand, postBuildSteps, preBuildSteps, customJob - Fix activate command description to accurately reflect verification behavior - Add null check before accessing result.BuildResults in orchestrate handler Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: split orchestrator integrity into 4 parallel jobs to fix timeout The monolithic orchestrator-integrity workflow runs 25+ tests sequentially in a single job, consistently hitting the 60-minute timeout on PR runs. Split into 4 parallel jobs (k8s, aws-provider, local-docker, rclone) each on its own runner, cutting wall-clock time from 3+ hours to ~1 hour and eliminating disk space exhaustion from shared runner contention. Adopts the parallel architecture from PR #809. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: add integration branch update scripts for release/lts-2.0.0 * ci: set macOS builds to continue-on-error Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: add release/lts-infrastructure to update-all script * ci: set macOS builds to continue-on-error Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: make git hooks opt-in only — do not modify hooks when disabled Remove the else branch that actively called GitHooksService.disableHooks() for every user where gitHooksEnabled was false (the default). This was a breaking change that silently modified core.hooksPath to point at an empty directory, disabling any existing git hooks (husky, lefthook, pre-commit, etc.). When gitHooksEnabled is false (default), the action now does nothing regarding hooks — exactly matching the behavior on main before the hooks feature was added. The hooks feature only activates when users explicitly set gitHooksEnabled: true. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: add integration wiring and input parsing tests for enterprise features Add three test files covering the two highest-priority gaps in PR #777: 1. src/index-enterprise-features.test.ts (21 tests) - Integration wiring tests for index.ts that verify conditional gating of all enterprise services (GitHooks, LocalCache, ChildWorkspace, SubmoduleProfile, LfsAgent). Tests that disabled features (default) are never invoked, enabled features call the correct service methods, and the order of operations is correct (restore before build, save after build). Also tests non-local provider strategy skips all enterprise features. 2. src/model/enterprise-inputs.test.ts (103 tests) - Input/BuildParameters wiring tests for all 20 new enterprise properties. Covers defaults, explicit values, and boolean string parsing edge cases (the #1 source of bugs: 'false' as truthy, 'TRUE' case sensitivity, '1', 'yes'). Verifies BuildParameters.create() correctly maps all Input getters. 3. src/model/orchestrator/services/submodule/submodule-profile-service.test.ts (5 new tests) - Command construction safety tests for execute(), documenting how paths, branches, and tokens are passed into git commands and verifying the expected command strings. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: mark failed macOS builds as neutral instead of failure Use the Checks API to flip failed macOS build conclusions to neutral (gray dash) so unstable builds don't show red X marks on PRs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * revert: restore build-tests-mac.yml to match main Stop modifying the macOS build workflow — leave it identical to main. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(test): add gitAuthMode to orchestrator-folders test mock The test mock was missing gitAuthMode, causing useHeaderAuth to default to true and strip the token from repo URLs. Adding gitAuthMode: 'url' restores the expected URL-mode behavior. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(ci): bump node version to 20 in integrity-check yargs@18.0.0 requires Node >=20.19.0, so Node 18 is no longer compatible. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: downgrade yargs to ^17.7.2 and revert Node to 18 for CI compatibility yargs@18 requires Node >=20.19.0 which is incompatible with CI's Node 18. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(cli): move cache command under orchestrate subcommand Cache is an orchestrator feature, so it belongs under `game-ci orchestrate cache` rather than as a top-level `game-ci cache` command. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: add orchestrator compatibility validation workflow Runs on PRs that touch orchestrator source or bridge files. Validates: - Orchestrator source files are in sync with standalone repo - Bridge file exports exist in both repos - Orchestrator tests pass in both unity-builder and standalone contexts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: route orchestrator through plugin loader Replace 8 direct orchestrator service imports with a thin plugin loader. - loadOrchestrator(): loads remote build orchestration - loadEnterpriseServices(): loads enterprise features for local builds All functionality is preserved; only the import mechanism changes. This is the first step toward making orchestrator an optional dependency. Includes comprehensive integration tests for enterprise feature wiring that verify gating logic, call ordering, and provider strategy routing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: extract orchestrator — delete 30k lines, decouple all imports Remove the entire src/model/orchestrator/ directory (148 files, ~30k lines) and refactor all dependent code to use the plugin loader pattern. Key changes: - build-parameters.ts: replace OrchestratorOptions with Input.getInput() - input.ts: remove OrchestratorQueryOverride input source - github.ts: strip to minimal class (only githubInputEnabled remains) - cli/cli.ts: remove orchestrator CLI commands, simplify to core structure - input-readers/*: replace OrchestratorSystem.Run with child_process.exec - orchestrator-plugin.ts: import from @game-ci/orchestrator package - orchestrate.ts, build.ts: use plugin loader instead of direct imports - index.ts: inline SyncStrategy type, fix implicit any types - Add type declarations for @game-ci/orchestrator - Remove orchestrator-only npm dependencies (AWS SDK, K8s, etc.) - Remove orchestrator-specific npm scripts and CI workflows - Update validate-orchestrator.yml for external repo validation All enterprise features gracefully degrade when @game-ci/orchestrator is not installed — the plugin loader returns undefined and optional chaining in index.ts skips all enterprise service calls. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: move CLI to orchestrator, fix validate-orchestrator workflow - Delete src/cli.ts, src/cli/ (commands, tests, input-mapper) — moved to game-ci/orchestrator repo (PR #813 reference) - Delete .github/workflows/release-cli.yml — moved to orchestrator - Remove bin, pkg, yargs, @types/yargs, pkg from package.json - Fix validate-orchestrator.yml: - Build TypeScript before running require() smoke tests - Remove || echo fallback that swallowed errors - Add smoke test that installs orchestrator via npm pack and verifies loadOrchestrator() returns defined exports Legacy src/model/cli/ (Cli class, CliFunctionsRepository) preserved — used by Input.getInput() and build-parameters.ts on main. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(ci): remove reference to deleted orchestrator-integrity.yml The orchestrator job in integrity-check.yml called the deleted orchestrator-integrity.yml workflow, causing CI failure. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(ci): use --legacy-peer-deps for orchestrator install in validation The orchestrator package brings eslint dependencies that conflict with unity-builder's peer deps. Since this install is only for smoke-testing the plugin loader, --legacy-peer-deps is safe here. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: remove temporary delete-me scripts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(ci): add orchestrator integration tests and plugin interface tests - Add validate-orchestrator-integration.yml with 3 parallel jobs: plugin-interface (unit tests + smoke tests), k8s-integration (k3d + localstack), and aws-integration (localstack only) - Add orchestrator-plugin.test.ts with 15 unit tests covering loadOrchestrator() and loadEnterpriseServices() for both installed and not-installed states - Disk space management follows proven patterns from orchestrator repo (parallel jobs, aggressive cleanup between tests) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(ci): add build step to k8s and aws integration jobs The orchestrator tests need compiled output (dist/index.js) to exist before running integration tests that spawn containers/k8s jobs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(ci): add refactor/** branch pattern and workflow_dispatch to orchestrator workflows The refactor/orchestrator-extraction branch was not matching the feature/** pattern, preventing the integration workflow from running after fix commits were pushed. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(ci): split orchestrator tests into per-PR health checks and nightly exhaustive suite validate-orchestrator.yml (per-PR, ~5 min): - Plugin architecture health: compilation, unit tests, plugin loader graceful degradation, installed service validation, type declaration checks validate-orchestrator-integration.yml (daily 3 AM UTC cron, ~1-2h): - 5 parallel jobs mirroring orchestrator-integrity.yml: plugin-interface, k8s (5 tests), aws (10 tests), local-docker (9 tests), rclone (1 test) - Full LocalStack + k3d integration coverage - continue-on-error on known flaky end2end tests Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: add yarn.lock to validate-orchestrator path filters Ensure orchestrator validation runs when yarn.lock changes, since dependency updates can affect plugin compatibility. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: move install scripts to orchestrator repo Install scripts now live at game-ci/orchestrator where the CLI releases are published. Removed from unity-builder to avoid duplication. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Potential fix for code scanning alert no. 78: Workflow does not contain permissions Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> * refactor: rename enterprise services to plugin services The orchestrator is a plugin, not an enterprise feature. Renamed loadEnterpriseServices -> loadPluginServices and all related variables, types, log messages, and test descriptions to use "plugin" terminology. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(ci): update workflow references from loadEnterpriseServices to loadPluginServices CI workflows still referenced the old function name after the rename. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: remove (Nightly) from integration tests workflow name Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: only suppress module-not-found errors in plugin loader Previously both loadOrchestrator() and loadPluginServices() caught all errors, masking real failures like syntax errors or missing transitive dependencies. Now only MODULE_NOT_FOUND / ERR_MODULE_NOT_FOUND errors are suppressed; all other exceptions are rethrown. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: add smoke test for orchestrator build wiring Verifies end-to-end that loadOrchestrator().run() is correctly wired to Orchestrator.run(), BuildParameters.create() produces valid config, and plugin services resolve to real implementations. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: wire orchestrator integration tests into integrity check - Add workflow_call trigger to validate-orchestrator-integration.yml so other workflows can invoke the exhaustive test suite - Add orchestrator-integration job to integrity-check.yml that runs on pushes to main (skipped on PRs to avoid 1-2h CI time) - Daily cron + manual dispatch remain as fallback triggers Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(ci): pin LocalStack to v3.8.1 for AWS SDK v3 compatibility localstack:latest (v4.14+) returns JSON responses for some S3 operations, but @aws-sdk/client-s3 v3.779+ uses AwsRestXmlProtocol which expects XML. This breaks all SharedWorkspaceLocking tests (locking, e2e caching, retaining). Pin to v3.8.1 (last v3 release) where the S3 provider returns proper XML responses. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * revert: restore localstack:latest now that SDK is pinned The S3 deserialization issue was caused by @aws-sdk/client-s3 v3.1005 (schema-based AwsRestXmlProtocol), not LocalStack's version. The SDK is now pinned to ~3.779.0 in the orchestrator repo, so localstack:latest works correctly. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: reorder AWS integration tests to prevent workspace corruption Move mandatory tests (caching, locking-core, locking-get-locked) before continue-on-error e2e tests. The e2e tests can corrupt the workspace (delete package.json), which was causing subsequent mandatory tests to fail with "Couldn't find a package.json". Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: plugin lifecycle interface for orchestrator extraction Replace hardcoded orchestrator params with a lifecycle-based plugin interface. The orchestrator reads its own config from env vars — unity-builder just calls 6 hooks (initialize, canHandleBuild, handleBuild, beforeLocalBuild, afterLocalBuild, handlePostBuild). Removes ~2900 lines from unity-builder (93 BuildParameters fields, 346 Input getters, 70 action.yml inputs, 400 lines of service orchestration in index.ts). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: align CI workflow with actual loadOrchestratorPlugin export The validate-orchestrator workflows referenced loadOrchestrator and loadPluginServices which don't exist — the source exports loadOrchestratorPlugin. Updated all CI steps to use the correct function name and test the actual OrchestratorPlugin lifecycle interface. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: checkout matching orchestrator branch in CI validation The validate-orchestrator workflow was always checking out the main branch of game-ci/orchestrator. When both repos have changes on a feature branch (e.g. refactor/orchestrator-extraction), the CI needs to use the matching branch. Falls back to main if the branch doesn't exist in the orchestrator repo. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * ci: add run-integration label to trigger full integration tests on PRs PRs labeled `run-integration` now run the full orchestrator integration suite (K8s, AWS, local-docker, rclone via LocalStack + k3d). Without the label, integration tests only run on push to main and the daily cron. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * ci: checkout matching orchestrator branch in integration tests Try the matching branch name (e.g. refactor/orchestrator-extraction) from game-ci/orchestrator first, falling back to main. This allows testing cross-repo changes before merging to orchestrator main. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * ci: switch from LocalStack to MiniStack for AWS mock services LocalStack community edition was discontinued (2026.03.0+) and now requires a paid license for ECS, CloudFormation, Kinesis, and other services used in integration tests. Switch to MiniStack (MIT, free, ministackorg/ministack) which provides all 40+ AWS services on the same port 4566 with backward-compatible health endpoints. ~10x smaller image, ~2s startup. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add sync-secrets workflow for sibling repositories Manually-triggered workflow that copies secrets (Unity credentials, AWS/GCP tokens, Codecov) from unity-builder to orchestrator or cli repos. Supports dry-run mode. Folded from PR #825. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Potential fix for pull request finding 'CodeQL / Workflow does not contain permissions' Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> * fix: add UNITY_LICENSE and NPM_TOKEN to sync-secrets, don't block on failures Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: remove LOCALSTACK_AUTH_TOKEN from sync-secrets workflow MiniStack doesn't require an auth token. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
9.4 KiB
9.4 KiB