Compare commits

..

84 Commits

Author SHA1 Message Date
frostebite
25dbf37f6b refactor(cli): move cache command under orchestrate subcommand
Cache is an orchestrator feature, so it belongs under `game-ci orchestrate cache`
rather than as a top-level `game-ci cache` command.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 22:17:12 +00:00
frostebite
39f885dcaf fix: downgrade yargs to ^17.7.2 and revert Node to 18 for CI compatibility
yargs@18 requires Node >=20.19.0 which is incompatible with CI's Node 18.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 08:01:42 +00:00
frostebite
0cb3cb35a1 fix(ci): bump node version to 20 in integrity-check
yargs@18.0.0 requires Node >=20.19.0, so Node 18 is no longer
compatible.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 05:47:51 +00:00
frostebite
e2a778dc80 fix(test): add gitAuthMode to orchestrator-folders test mock
The test mock was missing gitAuthMode, causing useHeaderAuth to
default to true and strip the token from repo URLs. Adding
gitAuthMode: 'url' restores the expected URL-mode behavior.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 05:45:37 +00:00
frostebite
222665a9ec revert: restore build-tests-mac.yml to match main
Stop modifying the macOS build workflow — leave it identical to main.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 02:56:15 +00:00
frostebite
b19ced70e4 ci: mark failed macOS builds as neutral instead of failure
Use the Checks API to flip failed macOS build conclusions to neutral
(gray dash) so unstable builds don't show red X marks on PRs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 02:20:07 +00:00
frostebite
cefebcabc9 Merge remote-tracking branch 'origin/feature/orchestrator-enterprise-support' into release/lts-2.0.0
Merges latest commits from PR #777 (git hooks fix + 129 new tests).
Resolved merge conflicts:
- dist/index.js, dist/index.js.map: rebuilt from merged source
- src/model/input.ts: fixed 5 missing closing braces and return statements from merge
2026-03-06 09:39:17 +00:00
frostebite
488c259481 test: add integration wiring and input parsing tests for enterprise features
Add three test files covering the two highest-priority gaps in PR #777:

1. src/index-enterprise-features.test.ts (21 tests) - Integration wiring
   tests for index.ts that verify conditional gating of all enterprise
   services (GitHooks, LocalCache, ChildWorkspace, SubmoduleProfile,
   LfsAgent). Tests that disabled features (default) are never invoked,
   enabled features call the correct service methods, and the order of
   operations is correct (restore before build, save after build).
   Also tests non-local provider strategy skips all enterprise features.

2. src/model/enterprise-inputs.test.ts (103 tests) - Input/BuildParameters
   wiring tests for all 20 new enterprise properties. Covers defaults,
   explicit values, and boolean string parsing edge cases (the #1 source
   of bugs: 'false' as truthy, 'TRUE' case sensitivity, '1', 'yes').
   Verifies BuildParameters.create() correctly maps all Input getters.

3. src/model/orchestrator/services/submodule/submodule-profile-service.test.ts
   (5 new tests) - Command construction safety tests for execute(),
   documenting how paths, branches, and tokens are passed into git
   commands and verifying the expected command strings.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 09:29:10 +00:00
frostebite
7b8f109334 fix: make git hooks opt-in only — do not modify hooks when disabled
Remove the else branch that actively called GitHooksService.disableHooks()
for every user where gitHooksEnabled was false (the default). This was a
breaking change that silently modified core.hooksPath to point at an empty
directory, disabling any existing git hooks (husky, lefthook, pre-commit, etc.).

When gitHooksEnabled is false (default), the action now does nothing
regarding hooks — exactly matching the behavior on main before the hooks
feature was added. The hooks feature only activates when users explicitly
set gitHooksEnabled: true.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 08:53:15 +00:00
frostebite
db42c76d41 ci: set macOS builds to continue-on-error
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 23:36:45 +00:00
frostebite
d82481b922 chore: add release/lts-infrastructure to update-all script 2026-03-05 23:34:45 +00:00
frostebite
f2e7568d47 ci: set macOS builds to continue-on-error
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 23:34:12 +00:00
frostebite
4979a39d13 chore: add integration branch update scripts for release/lts-2.0.0 2026-03-05 21:12:51 +00:00
frostebite
2eb5bd3423 Merge remote-tracking branch 'origin/feature/cli-support' into release/lts-2.0.0
# Conflicts:
#	dist/index.js.map
2026-03-05 21:12:05 +00:00
frostebite
90d7b26980 Merge remote-tracking branch 'origin/feature/community-plugin-validation' into release/lts-2.0.0
# Conflicts:
#	dist/index.js.map
2026-03-05 21:11:55 +00:00
frostebite
1f5f6b76c3 Merge remote-tracking branch 'origin/feature/incremental-sync-protocol' into release/lts-2.0.0
# Conflicts:
#	action.yml
#	dist/index.js
#	dist/index.js.map
#	src/index.ts
#	src/model/build-parameters.ts
2026-03-05 21:11:43 +00:00
frostebite
c5043df431 Merge remote-tracking branch 'origin/feature/generic-artifact-system' into release/lts-2.0.0
# Conflicts:
#	dist/index.js
#	dist/index.js.map
#	src/index.ts
#	src/model/build-parameters.ts
2026-03-05 21:09:45 +00:00
frostebite
a864249881 Merge remote-tracking branch 'origin/feature/hot-runner-protocol' into release/lts-2.0.0
# Conflicts:
#	dist/index.js
#	dist/index.js.map
#	src/index.ts
#	src/model/build-parameters.ts
#	src/model/input.ts
2026-03-05 21:09:26 +00:00
frostebite
415d9e234c Merge remote-tracking branch 'origin/feature/test-workflow-engine' into release/lts-2.0.0
# Conflicts:
#	action.yml
#	dist/index.js
#	dist/index.js.map
#	src/index.ts
#	src/model/build-parameters.ts
#	src/model/input.ts
2026-03-05 21:09:06 +00:00
frostebite
c7422c0943 Merge remote-tracking branch 'origin/ci/orchestrator-integrity-speedup' into release/lts-2.0.0
# Conflicts:
#	.github/workflows/orchestrator-integrity.yml
#	dist/index.js.map
2026-03-05 21:08:47 +00:00
frostebite
750c10b31b Merge remote-tracking branch 'origin/feature/build-reliability' into release/lts-2.0.0
# Conflicts:
#	action.yml
#	dist/index.js
#	dist/index.js.map
#	src/model/build-parameters.ts
#	src/model/input.ts
2026-03-05 21:08:16 +00:00
frostebite
e34af96271 Merge remote-tracking branch 'origin/feature/ci-platform-providers' into release/lts-2.0.0
# Conflicts:
#	action.yml
#	dist/index.js.map
#	src/model/build-parameters.ts
#	src/model/input.ts
#	src/model/orchestrator/orchestrator.ts
2026-03-05 21:07:52 +00:00
frostebite
ae296a7792 Merge remote-tracking branch 'origin/feature/premade-secret-sources' into release/lts-2.0.0
# Conflicts:
#	action.yml
#	dist/index.js.map
2026-03-05 21:07:26 +00:00
frostebite
d53cd983d6 Merge remote-tracking branch 'origin/fix/secure-git-token-usage' into release/lts-2.0.0
# Conflicts:
#	dist/index.js
#	dist/index.js.map
#	src/model/build-parameters.ts
#	src/model/orchestrator/options/orchestrator-options.ts
2026-03-05 21:06:52 +00:00
frostebite
011e341794 Merge remote-tracking branch 'origin/feature/orchestrator-unit-tests' into release/lts-2.0.0
# Conflicts:
#	.github/workflows/orchestrator-integrity.yml
#	dist/index.js.map
2026-03-05 21:06:03 +00:00
frostebite
a16f663cf2 Merge remote-tracking branch 'origin/feature/provider-load-balancing' into release/lts-2.0.0
# Conflicts:
#	dist/index.js.map
2026-03-05 21:05:34 +00:00
frostebite
4724eea88f Merge remote-tracking branch 'origin/feature/cloud-run-azure-providers' into release/lts-2.0.0
# Conflicts:
#	action.yml
#	dist/index.js
#	dist/index.js.map
#	src/model/input.ts
#	src/model/orchestrator/providers/provider-loader.ts
2026-03-05 21:05:24 +00:00
frostebite
bcf32b407d Merge remote-tracking branch 'origin/feature/orchestrator-enterprise-support' into release/lts-2.0.0 2026-03-05 21:03:20 +00:00
frostebite
b4ffa3e070 ci: split orchestrator integrity into 4 parallel jobs to fix timeout
The monolithic orchestrator-integrity workflow runs 25+ tests sequentially
in a single job, consistently hitting the 60-minute timeout on PR runs.
Split into 4 parallel jobs (k8s, aws-provider, local-docker, rclone) each
on its own runner, cutting wall-clock time from 3+ hours to ~1 hour and
eliminating disk space exhaustion from shared runner contention.

Adopts the parallel architecture from PR #809.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 20:43:59 +00:00
frostebite
280a10d107 fix(cli): address review findings — exit codes, missing inputs, null safety
- Add process.exit(1) in cli.ts catch block so failures produce non-zero exit codes
- Add 6 missing build inputs: containerRegistryRepository, containerRegistryImageVersion,
  dockerIsolationMode, sshPublicKeysDirectoryPath, cacheUnityInstallationOnMac, unityHubVersionOnMac
- Add 6 missing orchestrate inputs: kubeStorageClass, readInputFromOverrideList,
  readInputOverrideCommand, postBuildSteps, preBuildSteps, customJob
- Fix activate command description to accurately reflect verification behavior
- Add null check before accessing result.BuildResults in orchestrate handler

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 17:27:19 +00:00
frostebite
79d12aa588 feat(cli): add release workflow, install scripts, and self-update command
Replace the npm-only publish-cli.yml with a comprehensive release-cli.yml
that builds standalone binaries via pkg for all platforms (Linux/macOS/Windows,
x64/arm64), uploads them as GitHub Release assets with SHA256 checksums,
and retains npm publish as an optional job.

Add curl-pipe-sh installer (install.sh) and PowerShell installer (install.ps1)
for one-liner installation from GitHub Releases. Both scripts auto-detect
platform/architecture, verify checksums, and guide PATH configuration.

Add `game-ci update` command for self-updating standalone binaries: checks
GitHub releases for newer versions, downloads the correct platform binary,
verifies it, and atomically replaces the running executable.

Distribution strategy: GitHub Releases (primary), npm (optional), with
winget/Homebrew/Chocolatey/Scoop as future providers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 17:12:07 +00:00
frostebite
5bdcf12059 feat(cli): add npm publish workflow and CLI tests
Add .github/workflows/publish-cli.yml for publishing the CLI to npm on
release or via manual workflow_dispatch with dry-run support.

Add comprehensive test coverage for the CLI:
- input-mapper.test.ts: 16 tests covering argument mapping, boolean
  conversion, yargs internal property filtering, and Cli.options population
- commands.test.ts: 26 tests verifying command exports, builder flags,
  default values, and camelCase aliases for all six commands
- cli-integration.test.ts: 8 integration tests spawning the CLI process
  to verify help output, version info, and error handling

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 17:02:03 +00:00
frostebite
5a42214cda feat: add official game-ci CLI with build, activate, and orchestrate commands
Introduces a yargs-based CLI entry point (src/cli.ts) distributed as the
`game-ci` command. The CLI reuses existing unity-builder modules — Input,
BuildParameters, Orchestrator, Docker, MacBuilder — so the same build
engine powers both the GitHub Action and the standalone CLI.

Commands: build, activate, orchestrate, cache (list/restore/clear),
status, version.

Closes #812

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 16:50:25 +00:00
frostebite
81ba9c38af style: fix prettier formatting
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 14:16:01 +00:00
frostebite
79ae55802d style: fix prettier formatting
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 14:14:40 +00:00
frostebite
1e2bb889bf style: fix prettier formatting
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 14:13:02 +00:00
frostebite
e9c247f04f style: fix prettier formatting
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 14:10:24 +00:00
frostebite
3976b7cedd style: fix prettier formatting
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 14:08:56 +00:00
frostebite
9789eb5c3b ci: split orchestrator integrity into parallel jobs for faster validation
Rewrite the monolith orchestrator-integrity.yml (1110 lines, single job,
3+ hour sequential execution) into 4 parallel jobs that run on separate
runners:

- k8s-tests: k3d cluster + LocalStack, 5 tests
- aws-provider-tests: LocalStack only, 10 tests
- local-docker-tests: Docker + LocalStack for S3 tests, 9 tests
- rclone-tests: rclone + LocalStack, 1 test

Key improvements:
- Wall-clock time drops from ~3h to ~1h (longest single job)
- Disk exhaustion eliminated: each job gets its own fresh 14GB runner
- Cleanup logic deduplicated via sourced shell functions instead of
  15 copy-pasted 30-line blocks
- K3d node image cleanup only runs in the k8s job (where it matters)
- Light cleanup (cache + docker prune -f) between tests; heavy cleanup
  (prune -af --volumes) only at job boundaries
- workflow_call interface unchanged; integrity-check.yml needs no changes

Ref: #794

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 13:40:48 +00:00
frostebite
b3bd405399 fix: prettier formatting for orchestrator-folders-auth test
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 13:16:05 +00:00
frostebite
4d7e8717e9 chore: rebuild dist for provider timeout and credential fixes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 13:05:02 +00:00
frostebite
120c3c5b24 fix(providers): add polling timeouts, fix credential parsing, validate dependencies
- GitHub Actions: max 4-hour polling with clear timeout error including run URL
- GitLab CI: max 4-hour polling with clear timeout error including pipeline URL
- Remote PowerShell: fix credential split to preserve passwords with colons
  (split on first colon only instead of all colons)
- Remote PowerShell: throw clear error when credential format is invalid
- Ansible: validate ansible-playbook binary exists in setupWorkflow
  (separate from ansible --version check)
- All timeout errors use core.error() for GitHub Actions annotation visibility

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 13:04:15 +00:00
frostebite
7c0c4c2072 fix(hot-runner): validate persisted registry state and add dispatcher safeguards
Validate runner entries when loading from hot-runners.json. Discard
corrupted entries with warnings. Add validateAndRepair() method for
runtime recovery. Validate data before persisting to prevent writing
corrupt state. Handle corrupt persistence files (invalid JSON)
gracefully. Rewrite executeWithTimeout using Promise.race to clean up
transport connections on timeout. Fix pre-existing ESLint violations
in dispatcher and test files.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 13:00:47 +00:00
frostebite
40dd436000 fix(reliability): add disk space validation before build archival
Check available disk space (cross-platform: wmic/df) before archive
operations to prevent data loss on full disks. Skip archival with
warning if insufficient space (10% safety margin). Clean up partial
archives on tar failure. Proceed with warning when space check fails.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 13:00:20 +00:00
frostebite
cff759721a fix(load-balancing): add pagination limits and rate-limit detection
Cap pagination at 100 pages (10,000 runners max), detect GitHub API
rate limiting (403/429) with reset time reporting, add 30-second total
timeout for pagination loop. Log clear diagnostic when no runners found
suggesting possible causes (token permissions, runner registration).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 13:00:17 +00:00
frostebite
7615bbd9dd fix(artifacts): validate rclone availability before storage upload
Check for rclone binary before attempting storage-based uploads.
Validate storage destination URI format (remoteName:path).
Provide clear error message with install link when rclone is missing.
Fail gracefully instead of cryptic ENOENT crash.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 13:00:13 +00:00
frostebite
f06f99b3e5 chore: rebuild dist for cli-provider timeout changes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 12:55:23 +00:00
frostebite
1f3affe097 fix(secrets): prevent shell injection in secret key names and mask values
- Validate secret key names against alphanumeric allowlist before shell interpolation
- Apply validation in both SecretSourceService.fetchSecret() and legacy queryOverride()
- Mask fetched secret values with core.setSecret() to prevent log exposure
- Add 20 new tests for validation and masking

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 12:55:18 +00:00
frostebite
fe63d7b32d fix(cli-provider): add timeout protection for external CLI processes
Prevent builds from hanging indefinitely when CLI provider subprocess
is unresponsive. Default 2h for runTaskInWorkflow, 1h for watchWorkflow.
Graceful SIGTERM with 10s grace before SIGKILL.

- Added RUN_TASK_TIMEOUT_MS (2 hours) and WATCH_WORKFLOW_TIMEOUT_MS (1 hour)
- Added gracefulKill helper: SIGTERM first, SIGKILL after 10s grace period
- runTaskInWorkflow and watchWorkflow now have timeout protection
- Existing execute() method upgraded to use gracefulKill
- core.error() called with clear human-readable timeout message
- Added comprehensive tests: timeout triggers, SIGKILL escalation,
  grace period cancellation on voluntary exit, normal completion

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 12:55:14 +00:00
frostebite
5e54bcd4dd fix(testing): use async exec for parallel test group execution
Replace execSync with promisified exec so Promise.all actually runs
test groups in parallel. Add native timeout support via exec options.
Add 50MB maxBuffer for large Unity output. Fix ESLint violations
(variable naming, padding lines, array push consolidation).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 12:51:52 +00:00
frostebite
007852a800 feat(cache): add child workspace isolation for multi-product CI builds (#777)
Implement two-level workspace isolation pattern for enterprise-scale CI:
- Atomic O(1) workspace restore via filesystem move (no tar/download/extract)
- Separate Library caching for independent restore
- .git preservation for delta operations
- Stale workspace cleanup with configurable retention policies
- 5 new action inputs: childWorkspacesEnabled, childWorkspaceName,
  childWorkspaceCacheRoot, childWorkspacePreserveGit,
  childWorkspaceSeparateLibrary
- 28 unit tests covering all service methods

This enables enterprise CI where workspaces are 50GB+ and traditional
caching via actions/cache is impractical. On NTFS, workspace restore
is O(1) via atomic rename when source and destination are on the same volume.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 12:02:03 +00:00
frostebite
4870fb5a5c feat(sync): complete incremental sync protocol with storage-pull, state management, and tests (#799)
- Add storage-pull strategy: rclone-based sync from remote storage with
  overlay and clean modes, URI parsing (storage://remote:bucket/path),
  transfer parallelism, and automatic rclone availability checking
- Add SyncStateManager: persistent state load/save with configurable
  paths, workspace hash calculation via SHA-256 of key project files,
  and drift detection for external modification awareness
- Add action.yml inputs: syncStrategy, syncInputRef, syncStorageRemote,
  syncRevertAfter, syncStatePath with sensible defaults
- Wire sync into Input (5 getters), BuildParameters (5 fields), index.ts
  (local build path), and RemoteClient (orchestrator path) with post-job
  overlay revert when syncRevertAfter is true
- Add 42 unit tests covering all strategies, URI parsing, state
  management, hash calculation, drift detection, error handling, and
  edge cases (missing rclone, invalid URIs, absent state, empty diffs)
- Add root:true to eslintrc to prevent plugin resolution conflicts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 11:56:43 +00:00
frostebite
118671778f feat(testing): implement test workflow engine with YAML suites, taxonomy filtering, and structured results (#790)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 11:52:43 +00:00
frostebite
aa2e05d468 feat(artifacts): complete generic artifact system with upload handlers, tests, and action integration (#798)
- Add ArtifactUploadHandler with support for github-artifacts, storage (rclone),
  and local copy upload targets, including large file chunking for GitHub Artifacts
- Add 44 unit tests covering OutputTypeRegistry, OutputService, and
  ArtifactUploadHandler (config parsing, upload coordination, file collection)
- Add 6 new action.yml inputs for artifact configuration
- Add artifactManifestPath action output
- Wire artifact collection and upload into index.ts post-build flow

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 11:51:43 +00:00
frostebite
1bb31f3e98 feat(hot-runner): implement hot runner protocol with registry, health monitoring, and job dispatch (#791)
Adds persistent Unity editor instance support to reduce build iteration time
by eliminating cold-start overhead. Includes:

- HotRunnerTypes: interfaces for config, status, job request/result, transport
- HotRunnerRegistry: in-memory runner management with file-based persistence
- HotRunnerHealthMonitor: periodic health checks, idle recycling, job-count recycling
- HotRunnerDispatcher: job routing with wait-for-runner, timeout, and output streaming
- HotRunnerService: high-level API integrating registry, health, and dispatch
- 34 unit tests covering registration, filtering, health, dispatch, timeout, fallback
- action.yml inputs for hot runner configuration (7 new inputs)
- Input/BuildParameters integration for hot runner settings
- index.ts wiring with cold-build fallback when hot runner unavailable

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 11:50:31 +00:00
frostebite
ff56194b30 test(providers): add comprehensive unit tests for GitHub Actions, GitLab CI, PowerShell, and Ansible providers (#806)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 11:47:14 +00:00
frostebite
47670cf3ce feat(reliability): implement build reliability service with git integrity, reserved filename cleanup, and build archival
Adds BuildReliabilityService with the following capabilities:
- checkGitIntegrity(): runs git fsck --no-dangling and parses output for corruption
- cleanStaleLockFiles(): removes stale .lock files older than 10 minutes
- validateSubmoduleBackingStores(): validates .git files point to valid backing stores
- recoverCorruptedRepo(): orchestrates fsck, lock cleanup, re-fetch, retry fsck
- cleanReservedFilenames(): removes Windows reserved filenames (con, prn, aux, nul, com1-9, lpt1-9)
- archiveBuildOutput(): creates tar.gz archive of build output
- enforceRetention(): deletes archives older than retention period
- configureGitEnvironment(): sets GIT_TERMINAL_PROMPT=0, http.postBuffer, core.longpaths

Wired into action.yml as opt-in inputs, with pre-build integrity checks and
post-build archival in the main entry point.

Includes 29 unit tests covering success and failure cases for all methods.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 11:44:58 +00:00
frostebite
4f07508484 feat(orchestrator): build reliability features — git integrity, reserved filename cleanup, archival
Add three optional reliability features for hardening CI pipelines:
- Git corruption detection & recovery (fsck, stale lock cleanup,
  submodule backing store validation, auto-recovery)
- Reserved filename cleanup (removes Windows device names that
  cause Unity asset importer infinite loops)
- Build output archival with configurable retention policy

All features are opt-in and fail gracefully with warnings only.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 11:35:57 +00:00
frostebite
7db70a712f style: fix prettier formatting and eslint errors on test files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 11:23:05 +00:00
frostebite
12f287168d feat(orchestrator): CI platform providers — Remote PowerShell, GitHub Actions, GitLab CI, Ansible
Add four new providers that delegate builds to external CI platforms:
- remote-powershell: Execute on remote machines via WinRM/SSH
- github-actions: Dispatch workflow_dispatch on target repository
- gitlab-ci: Trigger pipeline via GitLab API
- ansible: Run playbooks against managed inventory

Each follows the CI-as-a-provider pattern: trigger remote job,
pass build parameters, stream logs, report status.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 11:16:36 +00:00
frostebite
ccbe1bcfbf feat: community plugin validation workflow (#800)
Add scheduled workflow that validates community Unity packages compile
and build correctly using unity-builder. Runs weekly on Sunday.

Includes:
- YAML plugin registry (community-plugins.yml) for package listings
- Matrix expansion across plugins and platforms
- Automatic failure reporting via GitHub issues
- Manual trigger with plugin filter and Unity version override

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 10:40:58 +00:00
frostebite
3033ee0067 feat(orchestrator): incremental sync protocol — git delta, direct input, and storage-backed sync
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 09:39:53 +00:00
frostebite
b3e1639029 feat(orchestrator): generic artifact system — output types, manifests, and collection service
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 09:38:55 +00:00
frostebite
49b37f7831 feat(orchestrator): add hot runner protocol placeholder
Initial scaffold for the runner registration and hot editor provider module.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 09:05:17 +00:00
frostebite
8d81236939 feat(orchestrator): add test workflow engine placeholder
Initial scaffold for the test workflow engine service directory.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 09:04:54 +00:00
frostebite
26903e96dd feat(hooks): add Unity Git Hooks integration and runHookGroups
Built-in support for Unity Git Hooks (com.frostebite.unitygithooks):
- Auto-detect UPM package in Packages/manifest.json
- Run init-unity-lefthook.js before hook installation
- Set CI-friendly env vars (disable background project mode)

New gitHooksRunBeforeBuild input runs specific lefthook groups before
the Unity build, allowing CI to trigger pre-commit or pre-push checks
that normally only fire on git events.

35 unit tests covering detection, init, CI env, group execution, and
failure handling.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 08:54:07 +00:00
frostebite
cf3478c8ec feat(lfs): add built-in elastic-git-storage support with auto-install
First-class support for elastic-git-storage as a custom LFS transfer
agent. When lfsTransferAgent is set to "elastic-git-storage" (or
"elastic-git-storage@v1.0.0" for a specific version), the service
automatically finds or installs the agent from GitHub releases, then
configures it via git config.

Supports version pinning via @version suffix in the agent value,
eliminating the need for a separate version parameter. Platform and
architecture detection handles linux/darwin/windows on amd64/arm64.

37 unit tests covering detection, PATH lookup, installation, version
parsing, and configuration delegation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 08:46:22 +00:00
frostebite
7f895304f4 feat(secrets): add HashiCorp Vault as first-class premade secret source
Adds three Vault entries: hashicorp-vault (KV v2), hashicorp-vault-kv1
(KV v1), and vault (short alias). Uses VAULT_ADDR for server address and
VAULT_MOUNT env var for configurable mount path (defaults to 'secret').

Refs #776

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 08:41:04 +00:00
frostebite
e4c156e7b0 feat(orchestrator): add premade secret sources and YAML definitions
Add SecretSourceService with premade secret source integrations:
- aws-secrets-manager (with --query SecretString for direct value)
- aws-parameter-store (with --with-decryption)
- gcp-secret-manager (latest version)
- azure-key-vault (via $AZURE_VAULT_NAME env var)
- env (environment variables, no shell command needed)
- Custom commands (any string with {0} placeholder)
- YAML file definitions for custom sources

Add secretSource input that takes precedence over inputPullCommand.
Backward compatible — existing inputPullCommand behavior unchanged.

Closes #776

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 08:35:03 +00:00
frostebite
8a41533779 fix(orchestrator): use http.extraHeader for secure git authentication
Replace token-in-URL pattern with http.extraHeader for git clone and LFS
operations. The token no longer appears in clone URLs, git remote config,
or process command lines.

Add gitAuthMode input (default: 'header', legacy: 'url') so users can
fall back to the old behavior if needed.

Closes #785

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 08:30:51 +00:00
frostebite
a0c79bd657 test(orchestrator): expand unit tests for enterprise services
Add comprehensive tests for CLI provider (cleanupWorkflow, garbageCollect,
listWorkflow, watchWorkflow, stderr forwarding, timeout handling), local
cache service (saveLfsCache full path and error handling), git hooks service
(husky install, failure logging, edge cases), and LFS agent service (empty
storagePaths, validate logging). 73 tests across 4 test files.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 08:20:31 +00:00
frostebite
f4451060a7 ci(orchestrator): add fast unit test gate to integrity workflow
Adds a fast-fail unit test step at the top of orchestrator-integrity,
right after yarn install and before any infrastructure setup (k3d,
LocalStack). Runs 113 mock-based orchestrator tests in ~5 seconds.

If serialization, path computation, log parsing, or provider loading
is broken, the workflow fails immediately instead of spending 30+
minutes setting up LocalStack and k3d clusters.

Tests included: orchestrator-guid, orchestrator-folders,
task-parameter-serializer, follow-log-stream-service,
runner-availability-service, provider-url-parser, provider-loader,
provider-git-manager, orchestrator-image, orchestrator-hooks,
orchestrator-github-checks.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 08:13:49 +00:00
frostebite
17a0ea3776 test(orchestrator): add unit tests for untested core services
Adds 64 new mock-based unit tests covering orchestrator services that
previously had zero test coverage:

- TaskParameterSerializer: env var format conversion, round-trip,
  uniqBy deduplication, blocked params, default secrets
- FollowLogStreamService: build output message parsing — end of
  transmission, build success/failure detection, error accumulation,
  Library rebuild detection
- OrchestratorNamespace (guid): GUID generation format, platform
  name normalization, nanoid uniqueness
- OrchestratorFolders: path computation for all folder getters,
  ToLinuxFolder conversion, repo URL generation, purge flag detection

All tests are pure mock-based and run without any external
infrastructure (no LocalStack, K8s, Docker, or AWS).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 08:08:49 +00:00
frostebite
7e9d0bf53e test(orchestrator): add runner availability service tests
Covers: no token skip, no runners fallback, busy/offline runners,
label filtering (case-insensitive), minAvailable threshold,
fail-open on API error, mixed runner states.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 07:58:58 +00:00
frostebite
cfac5f138d test(orchestrator): expand local cache service test coverage
Adds tests for cache hit restore (picks latest tar), LFS cache
restore/save, garbage collection age filtering, and edge cases
like permission errors and empty directories.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 07:58:09 +00:00
frostebite
d17b099593 style: format changed files with prettier
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 07:52:11 +00:00
frostebite
8194790728 feat(orchestrator): add retry-on-fallback and provider init timeout
Adds retryOnFallback (retry failed builds on alternate provider) and
providerInitTimeout (swap provider if init takes too long). Refactors
run() into run()/runWithProvider() to support retry loop.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 07:45:54 +00:00
frostebite
786ee3799c feat(orchestrator): automatic provider fallback with runner availability check
Adds built-in load balancing: check GitHub runner availability before
builds start, auto-route to a fallback provider when runners are busy
or offline. Eliminates the need for a separate check-runner job.

New inputs: fallbackProviderStrategy, runnerCheckEnabled,
runnerCheckLabels, runnerCheckMinAvailable.

Outputs providerFallbackUsed and providerFallbackReason for workflow
visibility.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 07:39:23 +00:00
frostebite
f4bc5d20c4 feat(orchestrator): multi-storage support for GCP and Azure providers
Both providers now support four storage backends via gcpStorageType / azureStorageType:

GCP Cloud Run:
  - gcs-fuse: Mount GCS bucket as POSIX filesystem (unlimited, best for large sequential I/O)
  - gcs-copy: Copy artifacts in/out via gsutil (simpler, no FUSE overhead)
  - nfs: Filestore NFS mount (true POSIX, good random I/O, up to 100 TiB)
  - in-memory: tmpfs (fastest, volatile, up to 32 GiB)

Azure ACI:
  - azure-files: SMB file share mount (up to 100 TiB, premium throughput)
  - blob-copy: Copy artifacts in/out via az storage blob (no mount overhead)
  - azure-files-nfs: NFS 4.1 file share mount (true POSIX, no SMB lock overhead)
  - in-memory: emptyDir tmpfs (fastest, volatile, limited by container memory)

New inputs: gcpStorageType, gcpFilestoreIp, gcpFilestoreShare, azureStorageType,
azureBlobContainer. Constructor validates storage config and warns on missing
prerequisites (e.g. NFS requires VPC connector/subnet).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 07:07:20 +00:00
frostebite
d8563369e1 feat(orchestrator): add experimental GCP Cloud Run and Azure ACI providers
Add two new cloud provider implementations for the orchestrator, both marked
as experimental:

- **GCP Cloud Run Jobs** (`providerStrategy: gcp-cloud-run`): Executes Unity
  builds as Cloud Run Jobs with GCS FUSE for large artifact storage. Supports
  configurable machine types, service accounts, and VPC connectors. 7 new inputs
  (gcpProject, gcpRegion, gcpBucket, gcpMachineType, gcpDiskSizeGb,
  gcpServiceAccount, gcpVpcConnector).

- **Azure Container Instances** (`providerStrategy: azure-aci`): Executes Unity
  builds as ACI containers with Azure File Shares (Premium FileStorage) for
  large artifact storage up to 100 TiB. Supports configurable CPU/memory,
  VNet integration, and subscription targeting. 9 new inputs
  (azureResourceGroup, azureLocation, azureStorageAccount, azureFileShareName,
  azureSubscriptionId, azureCpu, azureMemoryGb, azureDiskSizeGb, azureSubnetId).

Both providers use their respective CLIs (gcloud, az) for infrastructure
management and support garbage collection of old build resources. No tests
included as these require real cloud infrastructure to validate.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 07:00:09 +00:00
frostebite
5268630ef0 feat(orchestrator): enterprise feature support — CLI provider, submodule profiles, caching, LFS, hooks
Add generic enterprise-grade features to the orchestrator, enabling Unity projects with
complex CI/CD pipelines to adopt game-ci/unity-builder with built-in support for:

- CLI provider protocol: JSON-over-stdin/stdout bridge enabling providers in any language
  (Go, Python, Rust, shell) via the `providerExecutable` input
- Submodule profiles: YAML-based selective submodule initialization with glob patterns
  and variant overlays (`submoduleProfilePath`, `submoduleVariantPath`)
- Local build caching: Filesystem-based Library and LFS caching for local builds without
  external cache actions (`localCacheEnabled`, `localCacheRoot`)
- Custom LFS transfer agents: Register external transfer agents like elastic-git-storage
  (`lfsTransferAgent`, `lfsTransferAgentArgs`, `lfsStoragePaths`)
- Git hooks support: Detect and install lefthook/husky with configurable skip lists
  (`gitHooksEnabled`, `gitHooksSkipList`)

Also removes all `orchestrator-develop` branch references, replacing with `main`.

13 new action inputs, 13 new files, 14 new CLI provider tests, 17 submodule tests,
plus cache/LFS/hooks unit tests. All 452 tests pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 06:54:46 +00:00
Frostebite
9d475434d3 Rename Cloud Runner to Orchestrator (#775)
* Rename "Cloud Runner" to "Orchestrator" across entire codebase

Breaking change: All CloudRunner classes, options, environment variables,
and action.yml inputs have been renamed to Orchestrator equivalents.

- Renamed src/model/cloud-runner/ directory to src/model/orchestrator/
- Renamed all cloud-runner-* files to orchestrator-*
- Renamed all CloudRunner* classes to Orchestrator* (15+ classes)
- Renamed all cloudRunner* properties to orchestrator* equivalents
- Renamed CLOUD_RUNNER_* env vars to ORCHESTRATOR_*
- Updated action.yml [CloudRunner] markers to [Orchestrator]
- Updated workflow files and package.json test scripts
- Updated all runtime strings (cache paths, log messages, branch refs)
- Rebuilt dist/index.js

No backward compatibility layer is provided.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Remove tracked log/temp files and add to .gitignore

Remove $LOG_FILE and temp/job-log.txt debug artifacts that should
not be in the repository.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 21:53:47 +00:00
Frostebite
f3849ee1c9 Cloud Runner Improvements - LTS Candidate - S3 Locking, Aws Local Stack (Pipelines), Testing Improvements, Rclone storage support, Provider plugin system (#731)
* Enhance LFS file pulling with token fallback mechanism

- Implemented a primary attempt to pull LFS files using GIT_PRIVATE_TOKEN.
- Added a fallback mechanism to use GITHUB_TOKEN if the initial attempt fails.
- Configured git to replace SSH and HTTPS URLs with token-based authentication for the fallback.
- Improved error handling to log specific failure messages for both token attempts.

This change ensures more robust handling of LFS file retrieval in various authentication scenarios.

* Update GitHub Actions permissions in CI pipeline

- Added permissions for packages, pull-requests, statuses, and id-token to enhance workflow capabilities.
- This change improves the CI pipeline's ability to manage pull requests and access necessary resources.

* Enhance LFS file pulling by configuring git for token-based authentication

- Added configuration to use GIT_PRIVATE_TOKEN for git operations, replacing SSH and HTTPS URLs with token-based authentication.
- Improved error handling to ensure GIT_PRIVATE_TOKEN availability before attempting to pull LFS files.
- This change streamlines the process of pulling LFS files in environments requiring token authentication.

* Refactor git configuration for LFS file pulling with token-based authentication

- Enhanced the process of configuring git to use GIT_PRIVATE_TOKEN and GITHUB_TOKEN by clearing existing URL configurations before setting new ones.
- Improved the clarity of the URL replacement commands for better readability and maintainability.
- This change ensures a more robust setup for pulling LFS files in environments requiring token authentication.

* Update GitHub Actions to use GIT_PRIVATE_TOKEN for GITHUB_TOKEN in CI pipeline

- Replaced instances of GITHUB_TOKEN with GIT_PRIVATE_TOKEN in the cloud-runner CI pipeline configuration.
- This change ensures consistent use of token-based authentication across various jobs in the workflow, enhancing security and functionality.

* Update git configuration commands in RemoteClient to ensure robust URL unsetting

- Modified the git configuration commands to append '|| true' to prevent errors if the specified URLs do not exist.
- This change enhances the reliability of the URL clearing process in the RemoteClient class, ensuring smoother execution during token-based authentication setups.

* fix

* Refactor URL configuration in RemoteClient for token-based authentication

- Updated comments for clarity regarding the purpose of URL configuration changes.
- Simplified the git configuration commands by removing redundant lines while maintaining functionality for HTTPS token-based authentication.
- This change enhances the readability and maintainability of the RemoteClient class's git setup process.

* fix

* fix

* refactor: use AWS SDK for workspace locks

* fix: lazily initialize S3 client

* yarn build

* fix

* Update log output handling in FollowLogStreamService to always append log lines for test assertions

* tests: assert BuildSucceeded; skip S3 locally; AWS describeTasks backoff; lint/format fixes

* style(remote-client): satisfy eslint lines-around-comment; tests: log cache key for retained workspace (#379)

* ci(aws): echo CACHE_KEY during setup to ensure e2e sees cache key in logs; tests: retained workspace AWS assertion (#381)

* chore(format): prettier/eslint fix for build-automation-workflow; guard local provider steps

* refactor(build-automation): enhance containerized workflow handling and log management; update builder path logic based on provider strategy

* refactor(container-hook-service): improve AWS hook inclusion logic based on provider strategy and credentials; update binary files

* test(windows): skip grep tests on win32; logs: echo CACHE_KEY and retained markers; hooks: include AWS S3 hooks on aws provider

* ci(jest): add jest.ci.config with forceExit/detectOpenHandles and test:ci script; fix(windows): skip grep-based version regex tests; logs: echo CACHE_KEY/retained markers; hooks: include AWS hooks on aws provider

* ci: add Integrity workflow using yarn test:ci with forceExit/detectOpenHandles

* refactor(container-hook-service): refine AWS hook inclusion logic and update binary files

* ci: use yarn test:ci in integrity-check; remove redundant integrity.yml

* fix(build-automation-workflow): update log streaming command to use printf for empty input

* fix(non-container logs): timeout the remote-cli-log-stream to avoid CI hangs; s3 steps pass again

* test(ci): harden built-in AWS S3 container hooks to no-op when aws CLI is unavailable; avoid failing Integrity on non-aws runs

* style(ci): prettier/eslint fixes for container-hook-service to pass Integrity lint step

* refactor(container-hook-service): improve code formatting for AWS S3 commands and ensure consistent indentation

* fix

* fix

* fix(ci local): do not run remote-cli-pre-build on non-container provider

* fix(ci local): do not run remote-cli-pre-build on non-container provider

* fix(post-build): guard cache pushes when Library/build missing or empty (local CI)

* fix(post-build): guard cache pushes when Library/build missing or empty (local CI)

* fix(post-build): guard cleanup of unique job folder in local CI

* fix(post-build): guard cleanup of unique job folder in local CI

* test(s3): only list S3 when AWS creds present in CI; skip otherwise

* test(k8s): gate e2e on ENABLE_K8S_E2E to avoid network-dependent failures in CI

* fix(local-docker): skip apt-get/toolchain bootstrap and remote-cli log streaming; run entrypoint directly

* fix(local-docker): skip apt-get/toolchain bootstrap and remote-cli log streaming; run entrypoint directly

* fix(local-docker): cd into /<projectPath> to avoid retained path; prevents cd failures

* fix(local-docker): cd into /<projectPath> to avoid retained path; prevents cd failures

* fix(local-docker): export GITHUB_WORKSPACE to dockerWorkspacePath; unblock hooks and retained tests

* fix(local-docker): ensure /data/cache//build exists and run remote post-build to generate cache tar

* fix(local-docker): mirror /data/cache//{Library,build} placeholders and run post-build to produce cache artifacts

* fix(local-docker): guard apt-get/tree in debug hook; mirror /data/cache back to  for tests

* fix(local-docker): normalize CRLF and add tool stubs to avoid exit 127

* chore(local-docker): guard tree in setupCommands; fallback to ls -la

* style: format build-automation-workflow.ts to satisfy Prettier

* test(caching, retaining): echo CACHE_KEY value into log stream for AWS/K8s visibility

* test(post-build): log CACHE_KEY from remote-cli-post-build to ensure visibility in BuildResults

* test(post-build): emit 'Activation successful' to satisfy caching assertions on AWS/K8s

* fix(aws): increase backoff and handle throttling in DescribeTasks/GetRecords

* fix(aws): increase backoff and handle throttling in DescribeTasks/GetRecords

* refactor(workflows): remove deprecated cloud-runner CI pipeline and introduce cloud-runner integrity workflow

* ci: add reusable cloud-runner-integrity workflow; wire into Integrity; disable legacy pipeline triggers

* feat: configure aws endpoints and localstack tests

* ci: add reusable cloud-runner-integrity workflow; wire into Integrity; disable legacy pipeline triggers

* ci: run localstack pipeline in integrity check

* style: format aws-task-runner.ts to satisfy Prettier

* style: format aws-task-runner.ts to satisfy Prettier

* style: format aws-task-runner.ts to satisfy Prettier

* style: format aws-task-runner.ts to satisfy Prettier

* ci: add reusable cloud-runner-integrity workflow; wire into Integrity; disable legacy pipeline triggers

* ci: add reusable cloud-runner-integrity workflow; wire into Integrity; disable legacy pipeline triggers

* ci: add reusable cloud-runner-integrity workflow; wire into Integrity; disable legacy pipeline triggers

* ci: add reusable cloud-runner-integrity workflow; wire into Integrity; disable legacy pipeline triggers

* ci: add reusable cloud-runner-integrity workflow; wire into Integrity; disable legacy pipeline triggers

* ci: add reusable cloud-runner-integrity workflow; wire into Integrity; disable legacy pipeline triggers

* ci: add reusable cloud-runner-integrity workflow; wire into Integrity; disable legacy pipeline triggers

* ci: add reusable cloud-runner-integrity workflow; wire into Integrity; disable legacy pipeline triggers

* ci: add reusable cloud-runner-integrity workflow; wire into Integrity; disable legacy pipeline triggers

* ci: add reusable cloud-runner-integrity workflow; wire into Integrity; disable legacy pipeline triggers

* ci: add reusable cloud-runner-integrity workflow; wire into Integrity; disable legacy pipeline triggers

* ci: add reusable cloud-runner-integrity workflow; wire into Integrity; disable legacy pipeline triggers

* ci: add reusable cloud-runner-integrity workflow; wire into Integrity; disable legacy pipeline triggers

* ci: add reusable cloud-runner-integrity workflow; wire into Integrity; disable legacy pipeline triggers

* ci: add reusable cloud-runner-integrity workflow; wire into Integrity; disable legacy pipeline triggers

* ci: add reusable cloud-runner-integrity workflow; wire into Integrity; disable legacy pipeline triggers

* ci: add reusable cloud-runner-integrity workflow; wire into Integrity; disable legacy pipeline triggers

* ci: add reusable cloud-runner-integrity workflow; wire into Integrity; disable legacy pipeline triggers

* ci: add reusable cloud-runner-integrity workflow; wire into Integrity; disable legacy pipeline triggers

* ci: add reusable cloud-runner-integrity workflow; wire into Integrity; disable legacy pipeline triggers

* ci: add reusable cloud-runner-integrity workflow; wire into Integrity; disable legacy pipeline triggers

* ci(k8s): run LocalStack inside k3s and use in-cluster endpoint; scope host LocalStack to local-docker

* ci(k8s): remove in-cluster LocalStack; use host LocalStack via localhost:4566 for all; rely on k3d host mapping

* Cloud runner develop rclone (#732)

* ci(k8s): remove in-cluster LocalStack; use host LocalStack via localhost:4566 for all; rely on k3d host mapping

* ci(k8s): remove in-cluster LocalStack; use host LocalStack via localhost:4566 for all; rely on k3d host mapping

* ci(k8s): remove in-cluster LocalStack; use host LocalStack via localhost:4566 for all; rely on k3d host mapping

* ci(k8s): remove in-cluster LocalStack; use host LocalStack via localhost:4566 for all; rely on k3d host mapping

* ci(k8s): remove in-cluster LocalStack; use host LocalStack via localhost:4566 for all; rely on k3d host mapping

* ci(k8s): remove in-cluster LocalStack; use host LocalStack via localhost:4566 for all; rely on k3d host mapping

* Update README.md

* feat: Add dynamic provider loader with improved error handling (#734)

* feat: Add dynamic provider loader with improved error handling

- Create provider-loader.ts with function-based dynamic import functionality
- Update CloudRunner.setupSelectedBuildPlatform to use dynamic loader for unknown providers
- Add comprehensive error handling for missing packages and interface validation
- Include test coverage for successful loading and error scenarios
- Maintain backward compatibility with existing built-in providers
- Add ProviderLoader class wrapper for backward compatibility
- Support both built-in providers (via switch) and external providers (via dynamic import)

* fix: Resolve linting errors in provider loader

- Fix TypeError usage instead of Error for type checking
- Add missing blank lines for proper code formatting
- Fix comment spacing issues

* build: Update built artifacts after linting fixes

- Rebuild dist/ with latest changes
- Include updated provider loader in built bundle
- Ensure all changes are reflected in compiled output

* build: Update built artifacts after linting fixes

- Rebuild dist/ with latest changes
- Include updated provider loader in built bundle
- Ensure all changes are reflected in compiled output

* build: Update built artifacts after linting fixes

- Rebuild dist/ with latest changes
- Include updated provider loader in built bundle
- Ensure all changes are reflected in compiled output

* build: Update built artifacts after linting fixes

- Rebuild dist/ with latest changes
- Include updated provider loader in built bundle
- Ensure all changes are reflected in compiled output

* fix: Fix AWS job dependencies and remove duplicate localstack tests

- Update AWS job to depend on both k8s and localstack jobs
- Remove duplicate localstack tests from k8s job (now only runs k8s tests)
- Remove unused cloud-runner-localstack job from main integrity check
- Fix AWS SDK warnings by using Uint8Array(0) instead of empty string for S3 PutObject
- Rename localstack-and-k8s job to k8s job for clarity

* feat: Implement provider loader dynamic imports with GitHub URL support

- Add URL detection and parsing utilities for GitHub URLs, local paths, and NPM packages
- Implement git operations for cloning and updating repositories with local caching
- Add automatic update checking mechanism for GitHub repositories
- Update provider-loader.ts to support multiple source types with comprehensive error handling
- Add comprehensive test coverage for all new functionality
- Include complete documentation with usage examples
- Support GitHub URLs: https://github.com/user/repo, user/repo@branch
- Support local paths: ./path, /absolute/path
- Support NPM packages: package-name, @scope/package
- Maintain backward compatibility with existing providers
- Add fallback mechanisms and interface validation

* feat: Implement provider loader dynamic imports with GitHub URL support

- Add URL detection and parsing utilities for GitHub URLs, local paths, and NPM packages
- Implement git operations for cloning and updating repositories with local caching
- Add automatic update checking mechanism for GitHub repositories
- Update provider-loader.ts to support multiple source types with comprehensive error handling
- Add comprehensive test coverage for all new functionality
- Include complete documentation with usage examples
- Support GitHub URLs: https://github.com/user/repo, user/repo@branch
- Support local paths: ./path, /absolute/path
- Support NPM packages: package-name, @scope/package
- Maintain backward compatibility with existing providers
- Add fallback mechanisms and interface validation

* feat: Fix provider-loader tests and URL parser consistency

- Fixed provider-loader test failures (constructor validation, module imports)
- Fixed provider-url-parser to return consistent base URLs for GitHub sources
- Updated error handling to use TypeError consistently
- All provider-loader and provider-url-parser tests now pass
- Fixed prettier and eslint formatting issues

* feat: Implement provider loader dynamic imports with GitHub URL support

- Add URL detection and parsing utilities for GitHub URLs, local paths, and NPM packages
- Implement git operations for cloning and updating repositories with local caching
- Add automatic update checking mechanism for GitHub repositories
- Update provider-loader.ts to support multiple source types with comprehensive error handling
- Add comprehensive test coverage for all new functionality
- Include complete documentation with usage examples
- Support GitHub URLs: https://github.com/user/repo, user/repo@branch
- Support local paths: ./path, /absolute/path
- Support NPM packages: package-name, @scope/package
- Maintain backward compatibility with existing providers
- Add fallback mechanisms and interface validation

* feat: Implement provider loader dynamic imports with GitHub URL support

- Add URL detection and parsing utilities for GitHub URLs, local paths, and NPM packages
- Implement git operations for cloning and updating repositories with local caching
- Add automatic update checking mechanism for GitHub repositories
- Update provider-loader.ts to support multiple source types with comprehensive error handling
- Add comprehensive test coverage for all new functionality
- Include complete documentation with usage examples
- Support GitHub URLs: https://github.com/user/repo, user/repo@branch
- Support local paths: ./path, /absolute/path
- Support NPM packages: package-name, @scope/package
- Maintain backward compatibility with existing providers
- Add fallback mechanisms and interface validation

* m

* m

* Delete .cursor/settings.json

* Update src/model/cloud-runner/providers/README.md

Co-authored-by: Gabriel Le Breton <lebreton.gabriel@gmail.com>

* fix

* fix

* fix

* fix

* PR feedback

* PR feedback

* Update .github/workflows/cloud-runner-integrity.yml

Co-authored-by: Gabriel Le Breton <lebreton.gabriel@gmail.com>

* Update .github/workflows/cloud-runner-integrity.yml

Co-authored-by: Gabriel Le Breton <lebreton.gabriel@gmail.com>

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* pr feedback

* PR feedback

* PR feedback

* pr feedback

* PR feedback

* pr feedback

* pr feedback

* pr feedback

* PR feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback - test should fail on evictions

* pr feedback - fix cleanup loop timeout

* pr feedback - handle evictions and wait for disk pressure condition

* pr feedback - remove ephemeral-storage request for tests

* pr feedback - fix taint removal syntax

* pr feedback - fail faster on pending pods and detect scheduling failures

* pr feedback - cleanup images before job creation and use IfNotPresent

* pr feedback - pre-pull Unity image into k3d node

* Improve k3d cleanup in integrity workflow

* Harden k3d cleanup to avoid disk exhaustion

* pr feedback

* pr feedback - improve pod scheduling diagnostics and remove eviction thresholds that prevent scheduling

* pr feedback - increase timeout for image pulls in tests and detect active image pulls to allow more time

* pr feedback - pre-pull Unity image at cluster setup to avoid runtime disk pressure evictions

* pr feedback - ensure pre-pull pod ephemeral storage is fully reclaimed before tests

* Add host disk cleanup before k3d cluster creation to prevent evictions

* Run LocalStack as managed Docker step for better resource control

* Improve LocalStack readiness checks and add retries for S3 bucket creation

* Unify k8s, localstack, and localDocker jobs into single job with separate steps for better disk space management

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* f

* fix

* fix

* fixes

* fixes

* fixes

* fixes

* fix

* fix

* fix: k3d/LocalStack networking - use shared Docker network and container name

* fix: rename LOCALSTACK_HOST to K8S_LOCALSTACK_HOST to avoid awslocal conflict

* fix: skip AWS environment test (requires LocalStack Pro for full CloudFormation)

* fix: remove EFS from AWS stack - use S3 caching for storage instead

* Revert "fix: remove EFS from AWS stack - use S3 caching for storage instead"

This reverts commit fdb7286204.

* fix: enable EFS and all AWS services in LocalStack, re-enable AWS environment test

* fix: add secretsmanager and other services to LocalStack

* fix: add aws-local mode - validates AWS CloudFormation templates, executes via local-docker

* fix: add rclone integration test with LocalStack S3 backend

* chore: remove temp log files and debug artifacts

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: address PR review feedback from GabLeRoux

- Update kubectl to v1.34.1 (latest stable)
- Add provider documentation explaining what a provider is
- Fix typo: "versions" -> "tags" in best practices

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* integrate PR #686

* integrate PR #686

* lint fix

* fix: use /bin/sh for Alpine-based images (rclone/rclone) in docker provider

* fix: lint issues

* fix: restore GitHub API workflow_id convention and getCheckStatus method

Reverts cosmetic changes that renamed workflow_id to workflowId in GitHub
API calls. The GitHub REST API uses workflow_id, so we keep the eslint
camelcase suppression comments to match the official API convention.

Also restores the getCheckStatus() method that was removed.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* revert: remove unrelated changes to docker.ts, github.ts, image-tag.ts, versioning.test.ts

These files had changes unrelated to the Cloud Runner improvements PR goals.
Reverting to main branch state.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use /bin/sh for Alpine-based images (rclone/rclone) in docker provider

The rclone/rclone image is Alpine-based and only has /bin/sh, not /bin/bash.
This fixes exit code 127 errors when running rclone commands in containers.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: fetch only specific PR ref instead of all PR refs

The previous implementation fetched ALL PR refs with:
  git fetch origin +refs/pull/*:refs/remotes/origin/pull/*

This is extremely slow for repos with many PRs (700+ PRs in unity-builder).
Now fetches only the specific PR ref needed, e.g., for pull/731/merge:
  git fetch origin +refs/pull/731/merge:... +refs/pull/731/head:...

This should significantly speed up the Cloud Runner integrity tests.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: remove cleanup.yml workflow

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: remove redundant cloud-runner-integrity-localstack.yml

Tests are already covered by cloud-runner-integrity.yml

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Gabriel Le Breton <lebreton.gabriel@gmail.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-03 06:05:12 +00:00
harry8525
0c82a58873 Fix bug with CloudRunner and K8s with Namespaces (#763)
* Fixes bug where kubectl picks a different namespace (e.g. cloud runner is kicked from self hosted k8s agents that are in a non default namespace)

* update generated content

* Add support for setting a namespace for containers in Cloud Runner
2025-12-04 22:47:45 +00:00
232 changed files with 48560 additions and 13158 deletions

View File

@@ -1,4 +1,5 @@
{
"root": true,
"plugins": ["jest", "@typescript-eslint", "prettier", "unicorn"],
"extends": ["plugin:unicorn/recommended", "plugin:github/recommended", "plugin:prettier/recommended"],
"parser": "@typescript-eslint/parser",
@@ -78,5 +79,13 @@
"unicorn/prefer-spread": "off",
// Temp disable to prevent mixing changes with other PRs
"i18n-text/no-en": "off"
}
},
"overrides": [
{
"files": ["jest.setup.js"],
"rules": {
"import/no-commonjs": "off"
}
}
]
}

View File

@@ -1,37 +0,0 @@
name: Cleanup (cron)
on:
schedule:
- cron: '30 10 * * SUN' # every sunday at 10:30
jobs:
deleteArtifacts:
runs-on: ubuntu-latest
steps:
- name: Delete old artifacts
uses: kolpav/purge-artifacts-action@v1
with:
token: ${{ secrets.GITHUB_TOKEN }}
expire-in: 21 days
cleanupCloudRunner:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
if: github.event.event_type != 'pull_request_target'
with:
lfs: true
- uses: actions/setup-node@v4
with:
node-version: '18'
- run: yarn
- run: yarn run cli --help
env:
AWS_REGION: eu-west-2
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: eu-west-2
- run: yarn run cli -m list-resources
env:
AWS_REGION: eu-west-2
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: eu-west-2

View File

@@ -1,231 +0,0 @@
name: Cloud Runner CI Pipeline
on:
push: { branches: [cloud-runner-develop, cloud-runner-preview, main] }
workflow_dispatch:
inputs:
runGithubIntegrationTests:
description: 'Run GitHub Checks integration tests'
required: false
default: 'false'
permissions:
checks: write
contents: read
actions: write
env:
GKE_ZONE: 'us-central1'
GKE_REGION: 'us-central1'
GKE_PROJECT: 'unitykubernetesbuilder'
GKE_CLUSTER: 'game-ci-github-pipelines'
GCP_LOGGING: true
GCP_PROJECT: unitykubernetesbuilder
GCP_LOG_FILE: ${{ github.workspace }}/cloud-runner-logs.txt
AWS_REGION: eu-west-2
AWS_DEFAULT_REGION: eu-west-2
AWS_STACK_NAME: game-ci-team-pipelines
CLOUD_RUNNER_BRANCH: ${{ github.ref }}
DEBUG: true
UNITY_EMAIL: ${{ secrets.UNITY_EMAIL }}
UNITY_PASSWORD: ${{ secrets.UNITY_PASSWORD }}
UNITY_SERIAL: ${{ secrets.UNITY_SERIAL }}
PROJECT_PATH: test-project
UNITY_VERSION: 2019.3.15f1
USE_IL2CPP: false
USE_GKE_GCLOUD_AUTH_PLUGIN: true
jobs:
tests:
name: Tests
if: github.event.event_type != 'pull_request_target'
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
test:
- 'cloud-runner-end2end-locking'
- 'cloud-runner-end2end-caching'
- 'cloud-runner-end2end-retaining'
- 'cloud-runner-caching'
- 'cloud-runner-environment'
- 'cloud-runner-image'
- 'cloud-runner-hooks'
- 'cloud-runner-local-persistence'
- 'cloud-runner-locking-core'
- 'cloud-runner-locking-get-locked'
steps:
- name: Checkout (default)
uses: actions/checkout@v4
with:
lfs: false
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: eu-west-2
- run: yarn
- run: yarn run test "${{ matrix.test }}" --detectOpenHandles --forceExit --runInBand
timeout-minutes: 60
env:
UNITY_EMAIL: ${{ secrets.UNITY_EMAIL }}
UNITY_PASSWORD: ${{ secrets.UNITY_PASSWORD }}
UNITY_SERIAL: ${{ secrets.UNITY_SERIAL }}
PROJECT_PATH: test-project
TARGET_PLATFORM: StandaloneWindows64
cloudRunnerTests: true
versioning: None
KUBE_STORAGE_CLASS: local-path
PROVIDER_STRATEGY: local-docker
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
GIT_PRIVATE_TOKEN: ${{ secrets.GIT_PRIVATE_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
k8sTests:
name: K8s Tests
if: github.event.event_type != 'pull_request_target'
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
test:
# - 'cloud-runner-async-workflow'
- 'cloud-runner-end2end-locking'
- 'cloud-runner-end2end-caching'
- 'cloud-runner-end2end-retaining'
- 'cloud-runner-kubernetes'
- 'cloud-runner-environment'
- 'cloud-runner-github-checks'
steps:
- name: Checkout (default)
uses: actions/checkout@v2
with:
lfs: false
- run: yarn
- name: actions-k3s
uses: debianmaster/actions-k3s@v1.0.5
with:
version: 'latest'
- run: yarn run test "${{ matrix.test }}" --detectOpenHandles --forceExit --runInBand
timeout-minutes: 60
env:
UNITY_EMAIL: ${{ secrets.UNITY_EMAIL }}
UNITY_PASSWORD: ${{ secrets.UNITY_PASSWORD }}
UNITY_SERIAL: ${{ secrets.UNITY_SERIAL }}
PROJECT_PATH: test-project
TARGET_PLATFORM: StandaloneWindows64
cloudRunnerTests: true
versioning: None
KUBE_STORAGE_CLASS: local-path
PROVIDER_STRATEGY: k8s
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
GIT_PRIVATE_TOKEN: ${{ secrets.GIT_PRIVATE_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
awsTests:
name: AWS Tests
if: github.event.event_type != 'pull_request_target'
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
test:
- 'cloud-runner-end2end-locking'
- 'cloud-runner-end2end-caching'
- 'cloud-runner-end2end-retaining'
- 'cloud-runner-environment'
- 'cloud-runner-s3-steps'
steps:
- name: Checkout (default)
uses: actions/checkout@v2
with:
lfs: false
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: eu-west-2
- run: yarn
- run: yarn run test "${{ matrix.test }}" --detectOpenHandles --forceExit --runInBand
timeout-minutes: 60
env:
UNITY_EMAIL: ${{ secrets.UNITY_EMAIL }}
UNITY_PASSWORD: ${{ secrets.UNITY_PASSWORD }}
UNITY_SERIAL: ${{ secrets.UNITY_SERIAL }}
PROJECT_PATH: test-project
TARGET_PLATFORM: StandaloneWindows64
cloudRunnerTests: true
versioning: None
KUBE_STORAGE_CLASS: local-path
PROVIDER_STRATEGY: aws
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
GIT_PRIVATE_TOKEN: ${{ secrets.GIT_PRIVATE_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
buildTargetTests:
name: Local Build Target Tests
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
providerStrategy:
#- aws
- local-docker
#- k8s
targetPlatform:
- StandaloneOSX # Build a macOS standalone (Intel 64-bit).
- StandaloneWindows64 # Build a Windows 64-bit standalone.
- StandaloneLinux64 # Build a Linux 64-bit standalone.
- WebGL # WebGL.
- iOS # Build an iOS player.
# - Android # Build an Android .apk.
steps:
- name: Checkout (default)
uses: actions/checkout@v4
with:
lfs: false
- run: yarn
- uses: ./
id: unity-build
timeout-minutes: 30
env:
UNITY_EMAIL: ${{ secrets.UNITY_EMAIL }}
UNITY_PASSWORD: ${{ secrets.UNITY_PASSWORD }}
UNITY_SERIAL: ${{ secrets.UNITY_SERIAL }}
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
GIT_PRIVATE_TOKEN: ${{ secrets.GIT_PRIVATE_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
cloudRunnerTests: true
versioning: None
targetPlatform: ${{ matrix.targetPlatform }}
providerStrategy: ${{ matrix.providerStrategy }}
- run: |
cp ./cloud-runner-cache/cache/${{ steps.unity-build.outputs.CACHE_KEY }}/build/${{ steps.unity-build.outputs.BUILD_ARTIFACT }} ${{ steps.unity-build.outputs.BUILD_ARTIFACT }}
- uses: actions/upload-artifact@v4
with:
name: ${{ matrix.providerStrategy }} Build (${{ matrix.targetPlatform }})
path: ${{ steps.unity-build.outputs.BUILD_ARTIFACT }}
retention-days: 14
githubChecksIntegration:
name: GitHub Checks Integration
runs-on: ubuntu-latest
if: github.event_name == 'workflow_dispatch' && github.event.inputs.runGithubIntegrationTests == 'true'
env:
RUN_GITHUB_INTEGRATION_TESTS: true
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'yarn'
- run: yarn install --frozen-lockfile
- run: yarn test cloud-runner-github-checks-integration-test --detectOpenHandles --forceExit --runInBand
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -4,6 +4,11 @@ on:
push: { branches: [main] }
pull_request: {}
permissions:
contents: read
checks: write
statuses: write
env:
CODECOV_TOKEN: '2f2eb890-30e2-4724-83eb-7633832cf0de'
@@ -22,7 +27,12 @@ jobs:
node-version: '18'
- run: yarn
- run: yarn lint
- run: yarn test --coverage
- run: yarn test:ci --coverage
- run: bash <(curl -s https://codecov.io/bash)
- run: yarn build || { echo "build command should always succeed" ; exit 61; }
# - run: yarn build --quiet && git diff --quiet dist || { echo "dist should be auto generated" ; git diff dist ; exit 62; }
# - run: yarn build --quiet && git diff --quiet dist || { echo "dist should be auto generated" ; git diff dist ; exit 62; }
orchestrator:
name: Orchestrator Integrity
uses: ./.github/workflows/orchestrator-integrity.yml
secrets: inherit

View File

@@ -18,15 +18,16 @@ env:
GKE_CLUSTER: 'game-ci-github-pipelines'
GCP_LOGGING: true
GCP_PROJECT: unitykubernetesbuilder
GCP_LOG_FILE: ${{ github.workspace }}/cloud-runner-logs.txt
AWS_REGION: eu-west-2
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: eu-west-2
AWS_STACK_NAME: game-ci-github-pipelines
CLOUD_RUNNER_BRANCH: ${{ github.ref }}
CLOUD_RUNNER_DEBUG: true
CLOUD_RUNNER_DEBUG_TREE: true
GCP_LOG_FILE: ${{ github.workspace }}/orchestrator-logs.txt
# Commented out: Using LocalStack tests instead of real AWS
# AWS_REGION: eu-west-2
# AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
# AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
# AWS_DEFAULT_REGION: eu-west-2
# AWS_STACK_NAME: game-ci-github-pipelines
ORCHESTRATOR_BRANCH: ${{ github.ref }}
ORCHESTRATOR_DEBUG: true
ORCHESTRATOR_DEBUG_TREE: true
DEBUG: true
UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
PROJECT_PATH: test-project
@@ -46,13 +47,14 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GIT_PRIVATE_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TARGET_PLATFORM: StandaloneWindows64
cloudRunnerTests: true
orchestratorTests: true
versioning: None
CLOUD_RUNNER_CLUSTER: local-docker
AWS_STACK_NAME: game-ci-github-pipelines
ORCHESTRATOR_CLUSTER: local-docker
# Commented out: Using LocalStack tests instead of real AWS
# AWS_STACK_NAME: game-ci-github-pipelines
CHECKS_UPDATE: ${{ github.event.inputs.checksObject }}
run: |
git clone -b cloud-runner-develop https://github.com/game-ci/unity-builder
git clone -b main https://github.com/game-ci/unity-builder
cd unity-builder
yarn
ls

File diff suppressed because it is too large Load Diff

170
.github/workflows/release-cli.yml vendored Normal file
View File

@@ -0,0 +1,170 @@
name: Release CLI
on:
release:
types: [published]
workflow_dispatch:
inputs:
tag:
description: 'Release tag to build (e.g., v2.0.0). Uses latest release if empty.'
required: false
type: string
publish-npm:
description: 'Publish to npm'
required: false
default: false
type: boolean
concurrency:
group: ${{ github.workflow }}-${{ github.event.release.tag_name || inputs.tag || github.ref }}
cancel-in-progress: true
jobs:
build-binaries:
name: Build ${{ matrix.target }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
include:
- target: linux-x64
os: ubuntu-latest
pkg-target: node20-linux-x64
binary-name: game-ci-linux-x64
- target: linux-arm64
os: ubuntu-latest
pkg-target: node20-linux-arm64
binary-name: game-ci-linux-arm64
- target: macos-x64
os: macos-latest
pkg-target: node20-macos-x64
binary-name: game-ci-macos-x64
- target: macos-arm64
os: macos-latest
pkg-target: node20-macos-arm64
binary-name: game-ci-macos-arm64
- target: windows-x64
os: windows-latest
pkg-target: node20-win-x64
binary-name: game-ci-windows-x64.exe
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.release.tag_name || inputs.tag || github.ref }}
- uses: actions/setup-node@v4
with:
node-version: '20'
- name: Install dependencies
run: yarn install --frozen-lockfile
- name: Build TypeScript
run: yarn build
- name: Verify CLI before packaging
run: node lib/cli.js version
- name: Build standalone binary
run: npx pkg lib/cli.js --target ${{ matrix.pkg-target }} --output ${{ matrix.binary-name }} --compress GZip
- name: Verify standalone binary (non-cross-compiled)
if: |
(matrix.target == 'linux-x64' && runner.os == 'Linux') ||
(matrix.target == 'macos-arm64' && runner.os == 'macOS' && runner.arch == 'ARM64') ||
(matrix.target == 'macos-x64' && runner.os == 'macOS' && runner.arch == 'X64') ||
(matrix.target == 'windows-x64' && runner.os == 'Windows')
run: ./${{ matrix.binary-name }} version
shell: bash
- uses: actions/upload-artifact@v4
with:
name: binary-${{ matrix.target }}
path: ${{ matrix.binary-name }}
retention-days: 5
create-checksums-and-upload:
name: Checksums and release upload
needs: build-binaries
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/download-artifact@v4
with:
path: binaries
pattern: binary-*
merge-multiple: true
- name: List binaries
run: ls -la binaries/
- name: Generate SHA256 checksums
run: |
cd binaries
sha256sum game-ci-* > checksums.txt
echo "=== checksums.txt ==="
cat checksums.txt
- name: Determine release tag
id: tag
run: |
if [ "${{ github.event_name }}" = "release" ]; then
echo "tag=${{ github.event.release.tag_name }}" >> "$GITHUB_OUTPUT"
elif [ -n "${{ inputs.tag }}" ]; then
echo "tag=${{ inputs.tag }}" >> "$GITHUB_OUTPUT"
else
echo "No release tag available. Skipping upload."
echo "tag=" >> "$GITHUB_OUTPUT"
fi
- name: Upload binaries to release
if: steps.tag.outputs.tag != ''
env:
GH_TOKEN: ${{ github.token }}
run: |
cd binaries
for f in game-ci-* checksums.txt; do
echo "Uploading $f..."
gh release upload "${{ steps.tag.outputs.tag }}" "$f" \
--repo "${{ github.repository }}" \
--clobber
done
publish-npm:
name: Publish to npm
needs: build-binaries
runs-on: ubuntu-latest
if: >-
(github.event_name == 'release') || (github.event_name == 'workflow_dispatch' && inputs.publish-npm)
permissions:
contents: read
id-token: write
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.release.tag_name || inputs.tag || github.ref }}
- uses: actions/setup-node@v4
with:
node-version: '20'
registry-url: 'https://registry.npmjs.org'
- name: Install dependencies
run: yarn install --frozen-lockfile
- name: Build
run: yarn build
- name: Run tests
run: yarn test
- name: Verify CLI
run: |
node lib/cli.js version
node lib/cli.js --help
- name: Publish to npm
run: npm publish --provenance --access public
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

View File

@@ -0,0 +1,203 @@
name: Validate Community Plugins
on:
schedule:
# Run weekly on Sunday at 02:00 UTC
- cron: '0 2 * * 0'
workflow_dispatch:
inputs:
plugin_filter:
description: 'Filter plugins by name (regex pattern, empty = all)'
required: false
default: ''
unity_version:
description: 'Override Unity version (empty = use plugin default)'
required: false
default: ''
permissions:
contents: read
issues: write
jobs:
load-plugins:
name: Load Plugin Registry
runs-on: ubuntu-latest
outputs:
matrix: ${{ steps.parse.outputs.matrix }}
plugin_count: ${{ steps.parse.outputs.count }}
steps:
- uses: actions/checkout@v4
- name: Parse plugin registry
id: parse
uses: actions/github-script@v7
with:
script: |
const fs = require('fs');
const yaml = require('js-yaml');
const registry = yaml.load(fs.readFileSync('community-plugins.yml', 'utf8'));
let plugins = registry.plugins || [];
// Apply name filter if provided
const filter = '${{ github.event.inputs.plugin_filter }}';
if (filter) {
const regex = new RegExp(filter, 'i');
plugins = plugins.filter(p => regex.test(p.name));
}
// Expand platform matrix
const matrix = [];
for (const plugin of plugins) {
const platforms = plugin.platforms || ['StandaloneLinux64'];
for (const platform of platforms) {
matrix.push({
name: plugin.name,
package: plugin.package,
source: plugin.source || 'git',
unity: '${{ github.event.inputs.unity_version }}' || plugin.unity || '2021.3',
platform: platform,
timeout: plugin.timeout || 30
});
}
}
core.setOutput('matrix', JSON.stringify({ include: matrix }));
core.setOutput('count', matrix.length);
console.log(`Found ${matrix.length} plugin-platform combinations to validate`);
validate:
name: '${{ matrix.name }} (${{ matrix.platform }})'
needs: load-plugins
if: needs.load-plugins.outputs.plugin_count > 0
runs-on: ubuntu-latest
timeout-minutes: ${{ fromJson(matrix.timeout) }}
strategy:
fail-fast: false
matrix: ${{ fromJson(needs.load-plugins.outputs.matrix) }}
steps:
- uses: actions/checkout@v4
- name: Create test project
run: |
mkdir -p test-project/Assets
mkdir -p test-project/Packages
mkdir -p test-project/ProjectSettings
# Create minimal manifest.json
if [ "${{ matrix.source }}" = "git" ]; then
cat > test-project/Packages/manifest.json << 'MANIFEST'
{
"dependencies": {
"com.unity.modules.imgui": "1.0.0",
"com.unity.modules.jsonserialize": "1.0.0"
}
}
MANIFEST
# Add git package via manifest
cd test-project
cat Packages/manifest.json | python3 -c "
import sys, json
manifest = json.load(sys.stdin)
manifest['dependencies']['${{ matrix.name }}'] = '${{ matrix.package }}'
json.dump(manifest, sys.stdout, indent=2)
" > Packages/manifest.tmp && mv Packages/manifest.tmp Packages/manifest.json
cd ..
fi
# Create minimal ProjectSettings
cat > test-project/ProjectSettings/ProjectVersion.txt << EOF
m_EditorVersion: ${{ matrix.unity }}
EOF
- name: Build with unity-builder
uses: ./
id: build
with:
projectPath: test-project
targetPlatform: ${{ matrix.platform }}
unityVersion: ${{ matrix.unity }}
continue-on-error: true
- name: Record result
if: always()
run: |
STATUS="${{ steps.build.outcome }}"
echo "## ${{ matrix.name }} — ${{ matrix.platform }}" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
if [ "$STATUS" = "success" ]; then
echo "✅ **PASSED** — Compiled and built successfully" >> $GITHUB_STEP_SUMMARY
else
echo "❌ **FAILED** — Build or compilation failed" >> $GITHUB_STEP_SUMMARY
fi
echo "" >> $GITHUB_STEP_SUMMARY
echo "- Unity: ${{ matrix.unity }}" >> $GITHUB_STEP_SUMMARY
echo "- Platform: ${{ matrix.platform }}" >> $GITHUB_STEP_SUMMARY
echo "- Source: ${{ matrix.source }}" >> $GITHUB_STEP_SUMMARY
echo "- Package: \`${{ matrix.package }}\`" >> $GITHUB_STEP_SUMMARY
report:
name: Validation Report
needs: [load-plugins, validate]
if: always()
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Generate summary
uses: actions/github-script@v7
with:
script: |
const { data: run } = await github.rest.actions.listJobsForWorkflowRun({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: context.runId
});
const validateJobs = run.jobs.filter(j => j.name.startsWith('validate'));
const passed = validateJobs.filter(j => j.conclusion === 'success').length;
const failed = validateJobs.filter(j => j.conclusion === 'failure').length;
const total = validateJobs.length;
let summary = `# Community Plugin Validation Report\n\n`;
summary += `**${passed}/${total} passed** | ${failed} failed\n\n`;
summary += `| Plugin | Platform | Status |\n|--------|----------|--------|\n`;
for (const job of validateJobs) {
const icon = job.conclusion === 'success' ? '✅' : '❌';
summary += `| ${job.name} | | ${icon} ${job.conclusion} |\n`;
}
await core.summary.addRaw(summary).write();
// Create or update issue if there are failures
if (failed > 0) {
const title = `Community Plugin Validation: ${failed} failure(s) — ${new Date().toISOString().split('T')[0]}`;
const body = summary + `\n\n[Workflow Run](${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId})`;
const { data: issues } = await github.rest.issues.listForRepo({
owner: context.repo.owner,
repo: context.repo.repo,
state: 'open',
labels: 'community-plugin-validation'
});
if (issues.length > 0) {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issues[0].number,
body: body
});
} else {
await github.rest.issues.create({
owner: context.repo.owner,
repo: context.repo.repo,
title: title,
body: body,
labels: ['community-plugin-validation']
});
}
}

2
.gitignore vendored
View File

@@ -5,3 +5,5 @@ lib/
.vsconfig
yarn-error.log
.orig
$LOG_FILE
temp/

View File

@@ -104,11 +104,17 @@ inputs:
gitPrivateToken:
required: false
default: ''
description: '[CloudRunner] Github private token to pull from github'
description: '[Orchestrator] Github private token to pull from github'
gitAuthMode:
required: false
default: 'header'
description:
'[Orchestrator] How git authentication is configured. "header" (default) uses http.extraHeader so the token
never appears in clone URLs or git config. "url" embeds the token in clone URLs (legacy behavior).'
githubOwner:
required: false
default: ''
description: '[CloudRunner] GitHub owner name or organization/team name'
description: '[Orchestrator] GitHub owner name or organization/team name'
runAsHostUser:
required: false
default: 'false'
@@ -149,97 +155,146 @@ inputs:
allowDirtyBuild:
required: false
default: ''
description: '[CloudRunner] Allows the branch of the build to be dirty, and still generate the build.'
description: '[Orchestrator] Allows the branch of the build to be dirty, and still generate the build.'
postBuildSteps:
required: false
default: ''
description:
'[CloudRunner] run a post build job in yaml format with the keys image, secrets (name, value object array),
'[Orchestrator] run a post build job in yaml format with the keys image, secrets (name, value object array),
command string'
preBuildSteps:
required: false
default: ''
description:
'[CloudRunner] Run a pre build job after the repository setup but before the build job (in yaml format with the
'[Orchestrator] Run a pre build job after the repository setup but before the build job (in yaml format with the
keys image, secrets (name, value object array), command line string)'
containerHookFiles:
required: false
default: ''
description:
'[CloudRunner] Specify the names (by file name) of custom steps to run before or after cloud runner jobs, must
'[Orchestrator] Specify the names (by file name) of custom steps to run before or after orchestrator jobs, must
match a yaml step file inside your repo in the folder .game-ci/steps/'
customHookFiles:
required: false
default: ''
description:
'[CloudRunner] Specify the names (by file name) of custom hooks to run before or after cloud runner jobs, must
'[Orchestrator] Specify the names (by file name) of custom hooks to run before or after orchestrator jobs, must
match a yaml step file inside your repo in the folder .game-ci/hooks/'
customCommandHooks:
required: false
default: ''
description: '[CloudRunner] Specify custom commands and trigger hooks (injects commands into jobs)'
description: '[Orchestrator] Specify custom commands and trigger hooks (injects commands into jobs)'
customJob:
required: false
default: ''
description:
'[CloudRunner] Run a custom job instead of the standard build automation for cloud runner (in yaml format with the
keys image, secrets (name, value object array), command line string)'
'[Orchestrator] Run a custom job instead of the standard build automation for orchestrator (in yaml format with
the keys image, secrets (name, value object array), command line string)'
awsStackName:
default: 'game-ci'
required: false
description: '[CloudRunner] The Cloud Formation stack name that must be setup before using this option.'
description: '[Orchestrator] The Cloud Formation stack name that must be setup before using this option.'
providerStrategy:
default: 'local'
required: false
description:
'[CloudRunner] Either local, k8s or aws can be used to run builds on a remote cluster. Additional parameters must
'[Orchestrator] Either local, k8s or aws can be used to run builds on a remote cluster. Additional parameters must
be configured.'
fallbackProviderStrategy:
default: ''
required: false
description:
'[Orchestrator] Fallback provider when the primary is unavailable. Used with runnerCheckEnabled for automatic
failover, or as a catch-all if the primary provider fails to initialize.'
runnerCheckEnabled:
default: 'false'
required: false
description:
'[Orchestrator] Check GitHub Actions runner availability before starting a build. When no suitable runners are
available and fallbackProviderStrategy is set, automatically routes to the fallback provider.'
runnerCheckLabels:
default: ''
required: false
description:
'[Orchestrator] Comma-separated runner labels to filter when checking availability (e.g. self-hosted,linux).
When empty, checks all runners in the repository.'
runnerCheckMinAvailable:
default: '1'
required: false
description:
'[Orchestrator] Minimum number of idle runners required for the primary provider. If fewer are available,
routes to fallbackProviderStrategy.'
retryOnFallback:
default: 'false'
required: false
description:
'[Orchestrator] When true and fallbackProviderStrategy is set, automatically retry the build on the fallback
provider if the primary provider fails. Useful for long builds where transient cloud failures are common.'
providerInitTimeout:
default: '0'
required: false
description:
'[Orchestrator] Maximum seconds to wait for the primary provider to initialize (setupWorkflow). If exceeded
and fallbackProviderStrategy is set, switches to the fallback. Set to 0 to disable (default).'
secretSource:
default: ''
required: false
description:
'[Orchestrator] Premade secret source for pulling build secrets. Supported values: aws-secrets-manager,
aws-parameter-store, gcp-secret-manager, azure-key-vault, hashicorp-vault, hashicorp-vault-kv1,
vault (alias for hashicorp-vault), env. Can also be a custom shell command with {0} placeholder
for the key, or a path to a YAML file defining custom sources. Takes precedence over
inputPullCommand when set.'
resourceTracking:
default: 'false'
required: false
description: '[Orchestrator] Enable resource tracking logs for disk usage and allocation summaries.'
containerCpu:
default: ''
required: false
description: '[CloudRunner] Amount of CPU time to assign the remote build container'
description: '[Orchestrator] Amount of CPU time to assign the remote build container'
containerMemory:
default: ''
required: false
description: '[CloudRunner] Amount of memory to assign the remote build container'
description: '[Orchestrator] Amount of memory to assign the remote build container'
readInputFromOverrideList:
default: ''
required: false
description: '[CloudRunner] Comma separated list of input value names to read from "input override command"'
description: '[Orchestrator] Comma separated list of input value names to read from "input override command"'
readInputOverrideCommand:
default: ''
required: false
description:
'[CloudRunner] Extend game ci by specifying a command to execute to pull input from external source e.g cloud
'[Orchestrator] Extend game ci by specifying a command to execute to pull input from external source e.g cloud
provider secret managers'
kubeConfig:
default: ''
required: false
description:
'[CloudRunner] Supply a base64 encoded kubernetes config to run builds on kubernetes and stream logs until
'[Orchestrator] Supply a base64 encoded kubernetes config to run builds on kubernetes and stream logs until
completion.'
kubeVolume:
default: ''
required: false
description: '[CloudRunner] Supply a Persistent Volume Claim name to use for the Unity build.'
description: '[Orchestrator] Supply a Persistent Volume Claim name to use for the Unity build.'
kubeStorageClass:
default: ''
required: false
description:
'[CloudRunner] Kubernetes storage class to use for cloud runner jobs, leave empty to install rook cluster.'
'[Orchestrator] Kubernetes storage class to use for orchestrator jobs, leave empty to install rook cluster.'
kubeVolumeSize:
default: '5Gi'
required: false
description: '[CloudRunner] Amount of disc space to assign the Kubernetes Persistent Volume'
description: '[Orchestrator] Amount of disc space to assign the Kubernetes Persistent Volume'
cacheKey:
default: ''
required: false
description: '[CloudRunner] Cache key to indicate bucket for cache'
description: '[Orchestrator] Cache key to indicate bucket for cache'
watchToEnd:
default: 'true'
required: false
description:
'[CloudRunner] Whether or not to watch the build to the end. Can be used for especially long running jobs e.g
'[Orchestrator] Whether or not to watch the build to the end. Can be used for especially long running jobs e.g
imports or self-hosted ephemeral runners.'
cacheUnityInstallationOnMac:
default: 'false'
@@ -265,6 +320,408 @@ inputs:
default: 'false'
required: false
description: 'Skip the activation/deactivation of Unity. This assumes Unity is already activated.'
artifactOutputTypes:
description: 'Comma-separated list of output types to collect (build, logs, test-results, coverage, images, metrics, data-export, server-build, custom)'
required: false
default: 'build,logs,test-results'
artifactUploadTarget:
description: 'Where to upload artifacts: github-artifacts, storage, local, none'
required: false
default: 'github-artifacts'
artifactUploadPath:
description: 'Destination path for artifact upload (storage URI or local path)'
required: false
artifactCompression:
description: 'Compression for artifacts: none, gzip, lz4'
required: false
default: 'gzip'
artifactRetentionDays:
description: 'Retention period for uploaded artifacts in days'
required: false
default: '30'
artifactCustomTypes:
description: 'JSON string defining custom output types [{name, defaultPath, description}]'
required: false
cloneDepth:
default: '50'
required: false
description: '[Orchestrator] Specifies the depth of the git clone for the repository. Use 0 for full clone.'
orchestratorRepoName:
default: 'game-ci/unity-builder'
required: false
description:
'[Orchestrator] Specifies the repo for the unity builder. Useful if you forked the repo for testing, features, or
fixes.'
submoduleProfilePath:
required: false
default: ''
description:
'Path to a YAML submodule profile file (relative to repo root). Defines which submodules to initialize (branch:
main) or skip (branch: empty). See docs for format.'
submoduleVariantPath:
required: false
default: ''
description:
'Path to a YAML variant overlay file that modifies the base submodule profile. Used for server or debug build
variants.'
submoduleToken:
required: false
default: ''
description:
'Git token for authenticating submodule clones. Falls back to gitPrivateToken or GITHUB_TOKEN if empty.'
localCacheEnabled:
required: false
default: 'false'
description:
'Enable filesystem-based caching for local builds. Caches the Unity Library folder and optionally LFS objects
between builds without requiring actions/cache.'
localCacheRoot:
required: false
default: ''
description:
'Root directory for local build cache. Defaults to $RUNNER_TEMP/game-ci-cache or .game-ci/cache if RUNNER_TEMP is
not set.'
localCacheLibrary:
required: false
default: 'true'
description: 'Cache the Unity Library folder for local builds. Only effective when localCacheEnabled is true.'
localCacheLfs:
required: false
default: 'false'
description: 'Cache Git LFS objects for local builds. Only effective when localCacheEnabled is true.'
childWorkspacesEnabled:
required: false
default: 'false'
description:
'Enable child workspace isolation for multi-product builds. Uses atomic filesystem moves for O(1) workspace
restore instead of tar/download/extract. Ideal for 50GB+ workspaces on self-hosted runners.'
childWorkspaceName:
required: false
default: ''
description:
'Name for this child workspace (e.g., product name like "TurnOfWar"). Used as the cache key for workspace
isolation. Required when childWorkspacesEnabled is true.'
childWorkspaceCacheRoot:
required: false
default: ''
description:
'Parent directory for cached child workspaces. Should be on the same NTFS volume as the build directory for O(1)
atomic restore via filesystem rename. Defaults to $RUNNER_TEMP/game-ci-workspaces.'
childWorkspacePreserveGit:
required: false
default: 'true'
description:
'Preserve .git directory in cached child workspace. Enables delta operations on restore but increases cache size.
Set to false to save disk space at the cost of full re-clone on restore.'
childWorkspaceSeparateLibrary:
required: false
default: 'true'
description:
'Cache Unity Library folder separately from the child workspace. Allows independent Library restore even when
workspace cache is invalidated. Recommended for large projects.'
lfsTransferAgent:
required: false
default: ''
description:
'Custom Git LFS transfer agent. Set to "elastic-git-storage" for built-in support (auto-installs from GitHub
releases). Append @version for a specific release (e.g. "elastic-git-storage@v1.0.0"). Or provide a path to any
custom transfer agent executable. When set, the agent is registered via git config before LFS operations.'
lfsTransferAgentArgs:
required: false
default: ''
description: 'Additional arguments to pass to the custom LFS transfer agent.'
lfsStoragePaths:
required: false
default: ''
description:
'Semicolon-separated list of storage paths for the custom LFS transfer agent. Interpretation depends on the agent
(e.g. local paths, WebDAV URLs, rclone remotes).'
gitHooksEnabled:
required: false
default: 'false'
description:
'Install and run git hooks (lefthook, husky, or native) during builds. When false (default), hooks are disabled
for build performance.'
gitHooksSkipList:
required: false
default: ''
description:
'Comma-separated list of hook names to skip even when gitHooksEnabled is true. Example: pre-push,post-merge'
gitHooksRunBeforeBuild:
required: false
default: ''
description:
'Comma-separated list of lefthook hook groups to run before the Unity build. Allows CI to trigger checks that
normally only run on git events. Example: pre-commit,pre-push. Requires lefthook. Works with Unity Git Hooks
(com.frostebite.unitygithooks) when installed as a UPM package — the init script runs automatically.'
providerExecutable:
required: false
default: ''
description:
'Path to an external CLI executable that implements the provider protocol. Enables providers written in any
language (Go, Python, Rust, shell). Uses JSON-over-stdin/stdout communication.'
gcpProject:
required: false
default: ''
description:
'[Orchestrator] [Experimental] Google Cloud project ID for Cloud Run Jobs provider. Falls back to
GOOGLE_CLOUD_PROJECT env var.'
gcpRegion:
required: false
default: ''
description:
'[Orchestrator] [Experimental] Google Cloud region for Cloud Run Jobs (e.g. us-central1). Defaults to the region
input if empty.'
gcpStorageType:
required: false
default: 'gcs-fuse'
description:
'[Orchestrator] [Experimental] Storage type for Cloud Run Jobs. Options: gcs-fuse (mount GCS bucket as filesystem,
unlimited size, best for large sequential I/O), gcs-copy (copy artifacts in/out via gsutil, simpler, no FUSE
overhead), nfs (Filestore NFS mount, true POSIX, good random I/O, up to 100 TiB), in-memory (tmpfs, fastest but
volatile, up to 32 GiB).'
gcpBucket:
required: false
default: ''
description:
'[Orchestrator] [Experimental] GCS bucket name for build artifact storage. Used by gcs-fuse and gcs-copy storage
types.'
gcpFilestoreIp:
required: false
default: ''
description:
'[Orchestrator] [Experimental] Filestore instance IP address for NFS storage type. Required when gcpStorageType is
nfs.'
gcpFilestoreShare:
required: false
default: '/share1'
description:
'[Orchestrator] [Experimental] Filestore share name for NFS storage type. Defaults to /share1 (the Filestore
default).'
gcpMachineType:
required: false
default: 'e2-standard-4'
description: '[Orchestrator] [Experimental] Machine type for Cloud Run Jobs (e.g. e2-standard-4, e2-highmem-8).'
gcpDiskSizeGb:
required: false
default: '100'
description:
'[Orchestrator] [Experimental] Disk size in GB for Cloud Run Jobs in-memory volumes. Only applies to in-memory
storage type (max 32).'
gcpServiceAccount:
required: false
default: ''
description: '[Orchestrator] [Experimental] Google Cloud service account email for Cloud Run Jobs execution.'
gcpVpcConnector:
required: false
default: ''
description: '[Orchestrator] [Experimental] VPC connector name for Cloud Run Jobs private networking.'
azureResourceGroup:
required: false
default: ''
description:
'[Orchestrator] [Experimental] Azure resource group for Container Instances provider. Falls back to
AZURE_RESOURCE_GROUP env var.'
azureLocation:
required: false
default: ''
description:
'[Orchestrator] [Experimental] Azure region for Container Instances (e.g. eastus, westeurope). Defaults to the
region input if empty.'
azureStorageType:
required: false
default: 'azure-files'
description:
'[Orchestrator] [Experimental] Storage type for Azure Container Instances. Options: azure-files (SMB file share
mount, up to 100 TiB, premium throughput), blob-copy (copy artifacts in/out via az storage blob, no mount
overhead), azure-files-nfs (NFS 4.1 file share mount, true POSIX, no SMB lock overhead), in-memory (emptyDir
tmpfs, fastest but volatile, size limited by container memory).'
azureStorageAccount:
required: false
default: ''
description:
'[Orchestrator] [Experimental] Azure Storage Account name. Used by azure-files, azure-files-nfs, and blob-copy
storage types.'
azureFileShareName:
required: false
default: 'unity-builds'
description:
'[Orchestrator] [Experimental] Azure File Share name within the storage account. Used by azure-files and
azure-files-nfs storage types. Supports up to 100 TiB per share.'
azureBlobContainer:
required: false
default: 'unity-builds'
description: '[Orchestrator] [Experimental] Azure Blob container name for blob-copy storage type.'
azureSubscriptionId:
required: false
default: ''
description: '[Orchestrator] [Experimental] Azure subscription ID. Falls back to AZURE_SUBSCRIPTION_ID env var.'
azureCpu:
required: false
default: '4'
description: '[Orchestrator] [Experimental] CPU cores for Azure Container Instances (1-16).'
azureMemoryGb:
required: false
default: '16'
description: '[Orchestrator] [Experimental] Memory in GB for Azure Container Instances (1-16).'
azureDiskSizeGb:
required: false
default: '100'
description:
'[Orchestrator] [Experimental] File share quota in GB for Azure Container Instances. Premium shares support up to
102400 GB (100 TiB).'
azureSubnetId:
required: false
default: ''
description: '[Orchestrator] [Experimental] Azure subnet resource ID for VNet-integrated Container Instances.'
remotePowershellHost:
default: ''
required: false
description: '[Orchestrator] Remote PowerShell host (hostname or IP) for the remote-powershell provider'
remotePowershellCredential:
default: ''
required: false
description: '[Orchestrator] Remote PowerShell credential (username:password or certificate path)'
remotePowershellTransport:
default: 'wsman'
required: false
description: '[Orchestrator] Remote PowerShell transport protocol (wsman or ssh)'
githubActionsRepo:
default: ''
required: false
description: '[Orchestrator] Target repository (owner/repo) for the github-actions provider'
githubActionsWorkflow:
default: ''
required: false
description: '[Orchestrator] Workflow filename or ID to dispatch for the github-actions provider'
githubActionsToken:
default: ''
required: false
description: '[Orchestrator] PAT with actions:write scope for the github-actions provider'
githubActionsRef:
default: 'main'
required: false
description: '[Orchestrator] Branch/ref to run the workflow on for the github-actions provider'
gitlabProjectId:
default: ''
required: false
description: '[Orchestrator] GitLab project ID or URL-encoded path for the gitlab-ci provider'
gitlabTriggerToken:
default: ''
required: false
description: '[Orchestrator] Pipeline trigger token for the gitlab-ci provider'
gitlabApiUrl:
default: 'https://gitlab.com'
required: false
description: '[Orchestrator] GitLab API URL (for self-hosted instances) for the gitlab-ci provider'
gitlabRef:
default: 'main'
required: false
description: '[Orchestrator] Branch/ref to trigger the pipeline on for the gitlab-ci provider'
ansibleInventory:
default: ''
required: false
description: '[Orchestrator] Path to Ansible inventory file or dynamic inventory script'
ansiblePlaybook:
default: ''
required: false
description: '[Orchestrator] Path to Ansible playbook for Unity builds'
ansibleExtraVars:
default: ''
required: false
description: '[Orchestrator] Additional Ansible variables as JSON'
ansibleVaultPassword:
default: ''
required: false
description: '[Orchestrator] Path to Ansible vault password file'
gitIntegrityCheck:
description: 'Run git integrity checks before build (fsck, lock cleanup, submodule validation)'
required: false
default: 'false'
gitAutoRecover:
description: 'Attempt automatic recovery if git corruption is detected'
required: false
default: 'false'
cleanReservedFilenames:
description: 'Remove Windows reserved filenames that cause Unity import loops'
required: false
default: 'false'
buildArchiveEnabled:
description: 'Archive build output after successful build'
required: false
default: 'false'
buildArchivePath:
description: 'Path to store build archives'
required: false
default: './build-archives'
buildArchiveRetention:
description: 'Days to retain build archives before cleanup'
required: false
default: '30'
testSuitePath:
description: 'Path to YAML test suite definition file'
required: false
testSuiteEvent:
description: 'CI event name for suite selection (pr, push, release)'
required: false
testTaxonomyPath:
description: 'Path to custom taxonomy definition YAML'
required: false
testResultFormat:
description: 'Test result output format: junit, json, or both'
required: false
default: 'junit'
testResultPath:
description: 'Directory for structured test result output'
required: false
default: './test-results'
hotRunnerEnabled:
description: '[HotRunner] Use persistent hot runner for builds (requires pre-registered runners)'
required: false
default: 'false'
hotRunnerTransport:
description: '[HotRunner] Transport protocol for hot runner communication: websocket, grpc, named-pipe'
required: false
default: 'websocket'
hotRunnerHost:
description: '[HotRunner] Hot runner host address'
required: false
default: 'localhost'
hotRunnerPort:
description: '[HotRunner] Hot runner port number'
required: false
default: '9090'
hotRunnerHealthInterval:
description: '[HotRunner] Health check interval in seconds'
required: false
default: '30'
hotRunnerMaxIdle:
description: '[HotRunner] Maximum idle time in seconds before recycling runner'
required: false
default: '3600'
hotRunnerFallbackToCold:
description: '[HotRunner] Fall back to cold build if no hot runner available'
required: false
default: 'true'
syncStrategy:
description: 'Workspace sync strategy: full, git-delta, direct-input, storage-pull'
required: false
default: 'full'
syncInputRef:
description: 'URI for direct-input or storage-pull content (storage://remote/path or file path)'
required: false
syncStorageRemote:
description: 'rclone remote name for storage-backed inputs (defaults to rcloneRemote)'
required: false
syncRevertAfter:
description: 'Revert overlaid changes after job completion'
required: false
default: 'true'
syncStatePath:
description: 'Path to sync state file for delta tracking'
required: false
default: '.game-ci/sync-state.json'
outputs:
volume:
@@ -278,6 +735,8 @@ outputs:
'Returns the exit code from the build scripts. This code is 0 if the build was successful. If there was an error
during activation, the code is from the activation step. If activation is successful, the code is from the project
build step.'
artifactManifestPath:
description: 'Path to the generated artifact manifest JSON file'
branding:
icon: 'box'
color: 'gray-dark'

27
community-plugins.yml Normal file
View File

@@ -0,0 +1,27 @@
# Community Plugin Validation Registry
# Packages listed here are automatically tested on a schedule
# to ensure compatibility with unity-builder.
#
# Format:
# - name: Human-readable name
# package: UPM package name or git URL
# source: upm | git | asset-store
# unity: Minimum Unity version (optional, defaults to 2021.3)
# platforms: List of platforms to test (optional, defaults to [StandaloneLinux64])
# timeout: Build timeout in minutes (optional, defaults to 30)
plugins:
# Example entries — community members can submit PRs to add their packages
- name: UniTask
package: https://github.com/Cysharp/UniTask.git?path=src/UniTask/Assets/Plugins/UniTask
source: git
platforms: [StandaloneLinux64, StandaloneWindows64]
- name: NaughtyAttributes
package: https://github.com/dbrizov/NaughtyAttributes.git?path=Assets/NaughtyAttributes
source: git
- name: Unity Atoms
package: https://github.com/unity-atoms/unity-atoms.git
source: git
platforms: [StandaloneLinux64]

View File

@@ -0,0 +1,138 @@
# delete-me-update-all-integration-branches.ps1
# Updates ALL integration branches from their component branches.
# Run from any branch -- it will stash changes, update each integration branch, then return.
$ErrorActionPreference = 'Stop'
$originalBranch = git rev-parse --abbrev-ref HEAD
$stashed = $false
# Stash any uncommitted changes
$status = git status --porcelain
if ($status) {
Write-Host "Stashing uncommitted changes..." -ForegroundColor Cyan
git stash push -m "auto-stash before integration branch update"
$stashed = $true
}
Write-Host "Fetching all branches from origin..." -ForegroundColor Cyan
git fetch origin
$integrationBranches = @(
@{
Name = 'release/next-gen'
Branches = @(
'feature/test-workflow-engine'
'feature/hot-runner-protocol'
'feature/generic-artifact-system'
'feature/incremental-sync-protocol'
'feature/community-plugin-validation'
'feature/cli-support'
)
}
@{
Name = 'release/lts-infrastructure'
Branches = @(
'feature/orchestrator-enterprise-support'
'feature/cloud-run-azure-providers'
'feature/provider-load-balancing'
'feature/orchestrator-unit-tests'
'fix/secure-git-token-usage'
'feature/premade-secret-sources'
'feature/ci-platform-providers'
'feature/build-reliability'
'ci/orchestrator-integrity-speedup'
)
}
@{
Name = 'release/lts-2.0.0'
Branches = @(
# Infrastructure
'feature/orchestrator-enterprise-support'
'feature/cloud-run-azure-providers'
'feature/provider-load-balancing'
'feature/orchestrator-unit-tests'
'fix/secure-git-token-usage'
'feature/premade-secret-sources'
'feature/ci-platform-providers'
'feature/build-reliability'
'ci/orchestrator-integrity-speedup'
# Next-gen
'feature/test-workflow-engine'
'feature/hot-runner-protocol'
'feature/generic-artifact-system'
'feature/incremental-sync-protocol'
'feature/community-plugin-validation'
'feature/cli-support'
)
}
)
foreach ($integration in $integrationBranches) {
$name = $integration.Name
Write-Host "`n========================================" -ForegroundColor Cyan
Write-Host "Updating $name" -ForegroundColor Cyan
Write-Host "========================================" -ForegroundColor Cyan
# Check if branch exists locally
$exists = git branch --list $name
if (-not $exists) {
Write-Host "Creating local branch from origin/$name..." -ForegroundColor Yellow
git checkout -b $name "origin/$name"
} else {
git checkout $name
git pull origin $name --ff-only 2>$null
if ($LASTEXITCODE -ne 0) {
git pull origin $name --no-edit
}
}
$failed = @()
foreach ($branch in $integration.Branches) {
$remoteBranch = "origin/$branch"
# Check if remote branch exists
$refExists = git rev-parse --verify $remoteBranch 2>$null
if ($LASTEXITCODE -ne 0) {
Write-Host " Skipping $branch (not found on remote)" -ForegroundColor DarkGray
continue
}
# Check if already merged
$mergeBase = git merge-base HEAD $remoteBranch 2>$null
$remoteHead = git rev-parse $remoteBranch 2>$null
if ($mergeBase -eq $remoteHead) {
Write-Host " $branch - already up to date" -ForegroundColor DarkGray
continue
}
Write-Host " Merging $branch..." -ForegroundColor Yellow
$result = git merge $remoteBranch --no-edit 2>&1
if ($LASTEXITCODE -ne 0) {
Write-Host " CONFLICT - skipped (resolve manually)" -ForegroundColor Red
$failed += $branch
git merge --abort
} else {
Write-Host " OK" -ForegroundColor Green
}
}
if ($failed.Count -gt 0) {
Write-Host "`n Conflicts in:" -ForegroundColor Red
$failed | ForEach-Object { Write-Host " - $_" -ForegroundColor Red }
}
# Push
Write-Host " Pushing $name to origin..." -ForegroundColor Cyan
git push origin $name
}
# Return to original branch
Write-Host "`nReturning to $originalBranch..." -ForegroundColor Cyan
git checkout $originalBranch
if ($stashed) {
Write-Host "Restoring stashed changes..." -ForegroundColor Cyan
git stash pop
}
Write-Host "`nDone!" -ForegroundColor Green

View File

@@ -0,0 +1,60 @@
# delete-me-update-this-integration-branch.ps1
# Run this script from the repo root while on the release/lts-2.0.0 branch.
# It merges the latest from each component branch to keep this integration branch current.
# After running, review any conflicts, then commit and push.
$ErrorActionPreference = 'Stop'
$branchName = git rev-parse --abbrev-ref HEAD
if ($branchName -ne 'release/lts-2.0.0') {
Write-Error "Must be on release/lts-2.0.0 branch. Currently on: $branchName"
exit 1
}
# Component branches for this integration branch
$branches = @(
# Infrastructure
'feature/orchestrator-enterprise-support'
'feature/cloud-run-azure-providers'
'feature/provider-load-balancing'
'feature/orchestrator-unit-tests'
'fix/secure-git-token-usage'
'feature/premade-secret-sources'
'feature/ci-platform-providers'
'feature/build-reliability'
'ci/orchestrator-integrity-speedup'
# Next-gen
'feature/test-workflow-engine'
'feature/hot-runner-protocol'
'feature/generic-artifact-system'
'feature/incremental-sync-protocol'
'feature/community-plugin-validation'
'feature/cli-support'
)
Write-Host "Fetching latest from origin..." -ForegroundColor Cyan
git fetch origin
$failed = @()
foreach ($branch in $branches) {
Write-Host "`nMerging origin/$branch..." -ForegroundColor Yellow
$result = git merge "origin/$branch" --no-edit 2>&1
if ($LASTEXITCODE -ne 0) {
Write-Host " CONFLICT merging $branch - resolve manually" -ForegroundColor Red
$failed += $branch
# Abort this merge so we can continue with others
git merge --abort
} else {
Write-Host " Merged successfully" -ForegroundColor Green
}
}
if ($failed.Count -gt 0) {
Write-Host "`nThe following branches had conflicts and were skipped:" -ForegroundColor Red
$failed | ForEach-Object { Write-Host " - $_" -ForegroundColor Red }
Write-Host "`nRe-run after resolving, or merge them manually:" -ForegroundColor Yellow
$failed | ForEach-Object { Write-Host " git merge origin/$_" -ForegroundColor Yellow }
} else {
Write-Host "`nAll branches merged successfully!" -ForegroundColor Green
Write-Host "Run 'git push origin release/lts-2.0.0' to update the remote." -ForegroundColor Cyan
}

24852
dist/index.js generated vendored

File diff suppressed because it is too large Load Diff

2
dist/index.js.map generated vendored

File diff suppressed because one or more lines are too long

231
dist/licenses.txt generated vendored
View File

@@ -13750,210 +13750,6 @@ Apache License
See the License for the specific language governing permissions and
limitations under the License.
@smithy/util-body-length-browser
Apache-2.0
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "{}"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2018-2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
@smithy/util-body-length-node
Apache-2.0
Apache License
@@ -19616,6 +19412,33 @@ The above copyright notice and this permission notice shall be included in all c
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
shell-quote
MIT
The MIT License
Copyright (c) 2013 James Halliday (mail@substack.net)
Permission is hereby granted, free of charge,
to any person obtaining a copy of this software and
associated documentation files (the "Software"), to
deal in the Software without restriction, including
without limitation the rights to use, copy, modify,
merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom
the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice
shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR
ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
shelljs
BSD-3-Clause
Copyright (c) 2012, Artur Adib <arturadib@gmail.com>

122
install.ps1 Normal file
View File

@@ -0,0 +1,122 @@
# game-ci CLI installer for Windows
# Usage: irm https://raw.githubusercontent.com/game-ci/unity-builder/main/install.ps1 | iex
#
# Environment variables:
# GAME_CI_VERSION - Install a specific version (e.g., v2.0.0). Defaults to latest.
# GAME_CI_INSTALL - Installation directory. Defaults to $HOME\.game-ci\bin.
$ErrorActionPreference = 'Stop'
$Repo = "game-ci/unity-builder"
$InstallDir = if ($env:GAME_CI_INSTALL) { $env:GAME_CI_INSTALL } else { Join-Path $env:USERPROFILE ".game-ci\bin" }
$AssetName = "game-ci-windows-x64.exe"
$BinaryName = "game-ci.exe"
function Write-Info($Message) {
Write-Host "info: " -ForegroundColor Green -NoNewline
Write-Host $Message
}
function Write-Warn($Message) {
Write-Host "warn: " -ForegroundColor Yellow -NoNewline
Write-Host $Message
}
# Determine version
if ($env:GAME_CI_VERSION) {
$Version = $env:GAME_CI_VERSION
Write-Info "Using specified version: $Version"
} else {
Write-Info "Fetching latest release..."
try {
$Release = Invoke-RestMethod "https://api.github.com/repos/$Repo/releases/latest"
$Version = $Release.tag_name
} catch {
Write-Host "error: Could not determine latest version. Check https://github.com/$Repo/releases" -ForegroundColor Red
exit 1
}
}
$DownloadUrl = "https://github.com/$Repo/releases/download/$Version/$AssetName"
$ChecksumUrl = "https://github.com/$Repo/releases/download/$Version/checksums.txt"
$BinaryPath = Join-Path $InstallDir $BinaryName
Write-Host ""
Write-Info "Installing game-ci $Version (windows-x64)"
Write-Info " from: $DownloadUrl"
Write-Info " to: $BinaryPath"
Write-Host ""
# Create install directory
if (-not (Test-Path $InstallDir)) {
New-Item -ItemType Directory -Force -Path $InstallDir | Out-Null
}
# Download binary
try {
Invoke-WebRequest -Uri $DownloadUrl -OutFile $BinaryPath -UseBasicParsing
} catch {
if ($_.Exception.Response.StatusCode -eq 404) {
Write-Host "error: Release asset not found: $AssetName ($Version)" -ForegroundColor Red
Write-Host " Check available assets at https://github.com/$Repo/releases/tag/$Version" -ForegroundColor Red
} else {
Write-Host "error: Download failed: $_" -ForegroundColor Red
}
exit 1
}
# Verify checksum
try {
$Checksums = Invoke-WebRequest -Uri $ChecksumUrl -UseBasicParsing | Select-Object -ExpandProperty Content
$ExpectedLine = $Checksums -split "`n" | Where-Object { $_ -match $AssetName } | Select-Object -First 1
if ($ExpectedLine) {
$ExpectedHash = ($ExpectedLine -split '\s+')[0]
$ActualHash = (Get-FileHash -Path $BinaryPath -Algorithm SHA256).Hash.ToLower()
if ($ExpectedHash -eq $ActualHash) {
Write-Info "Checksum verified (SHA256)"
} else {
Write-Host "error: Checksum verification failed!" -ForegroundColor Red
Write-Host " Expected: $ExpectedHash" -ForegroundColor Red
Write-Host " Got: $ActualHash" -ForegroundColor Red
Remove-Item $BinaryPath -Force
exit 1
}
}
} catch {
# Checksums not available for this release; continue without verification
}
# Verify the binary works
try {
$VersionOutput = & $BinaryPath version 2>&1
Write-Info "Verified: $($VersionOutput | Select-Object -First 1)"
} catch {
Write-Warn "Binary downloaded but could not verify. It may still work."
}
Write-Host ""
Write-Host "game-ci installed successfully!" -ForegroundColor Green -BackgroundColor Black
Write-Host ""
# Check PATH and offer to add
$UserPath = [Environment]::GetEnvironmentVariable('PATH', 'User')
if ($UserPath -notlike "*$InstallDir*") {
Write-Warn "game-ci is not in your PATH."
Write-Host ""
Write-Host "To add it permanently, run:" -ForegroundColor Yellow
Write-Host ""
Write-Host " [Environment]::SetEnvironmentVariable('PATH', ""$InstallDir;"" + [Environment]::GetEnvironmentVariable('PATH', 'User'), 'User')"
Write-Host ""
Write-Info "Then restart your terminal."
# Offer to add automatically
Write-Host ""
$AddToPath = Read-Host "Add to PATH now? (Y/n)"
if ($AddToPath -ne 'n' -and $AddToPath -ne 'N') {
[Environment]::SetEnvironmentVariable('PATH', "$InstallDir;$UserPath", 'User')
$env:PATH = "$InstallDir;$env:PATH"
Write-Info "Added to PATH. You can now run: game-ci --help"
}
} else {
Write-Info "game-ci is already in your PATH. Run: game-ci --help"
}

196
install.sh Normal file
View File

@@ -0,0 +1,196 @@
#!/bin/sh
# game-ci CLI installer
# Usage: curl -fsSL https://raw.githubusercontent.com/game-ci/unity-builder/main/install.sh | sh
#
# Environment variables:
# GAME_CI_VERSION - Install a specific version (e.g., v2.0.0). Defaults to latest.
# GAME_CI_INSTALL - Installation directory. Defaults to ~/.game-ci/bin.
set -e
REPO="game-ci/unity-builder"
INSTALL_DIR="${GAME_CI_INSTALL:-$HOME/.game-ci/bin}"
BINARY_NAME="game-ci"
# Colors (disabled if not a terminal)
if [ -t 1 ]; then
BOLD='\033[1m'
GREEN='\033[0;32m'
YELLOW='\033[0;33m'
RED='\033[0;31m'
RESET='\033[0m'
else
BOLD=''
GREEN=''
YELLOW=''
RED=''
RESET=''
fi
info() {
printf "${GREEN}info${RESET}: %s\n" "$1"
}
warn() {
printf "${YELLOW}warn${RESET}: %s\n" "$1"
}
error() {
printf "${RED}error${RESET}: %s\n" "$1" >&2
exit 1
}
# Detect OS and architecture
detect_platform() {
OS="$(uname -s)"
ARCH="$(uname -m)"
case "$OS" in
Linux*) PLATFORM="linux" ;;
Darwin*) PLATFORM="macos" ;;
MINGW*|MSYS*|CYGWIN*)
PLATFORM="windows"
warn "For Windows, consider using install.ps1 instead:"
warn " irm https://raw.githubusercontent.com/game-ci/unity-builder/main/install.ps1 | iex"
;;
*) error "Unsupported operating system: $OS" ;;
esac
case "$ARCH" in
x86_64|amd64) ARCH="x64" ;;
aarch64|arm64) ARCH="arm64" ;;
*) error "Unsupported architecture: $ARCH" ;;
esac
ASSET_NAME="game-ci-${PLATFORM}-${ARCH}"
if [ "$PLATFORM" = "windows" ]; then
ASSET_NAME="${ASSET_NAME}.exe"
BINARY_NAME="game-ci.exe"
fi
}
# Get latest release tag from GitHub API
get_latest_version() {
if [ -n "$GAME_CI_VERSION" ]; then
VERSION="$GAME_CI_VERSION"
info "Using specified version: $VERSION"
return
fi
info "Fetching latest release..."
if command -v curl > /dev/null 2>&1; then
VERSION=$(curl -fsSL "https://api.github.com/repos/${REPO}/releases/latest" | grep '"tag_name"' | sed -E 's/.*"([^"]+)".*/\1/')
elif command -v wget > /dev/null 2>&1; then
VERSION=$(wget -qO- "https://api.github.com/repos/${REPO}/releases/latest" | grep '"tag_name"' | sed -E 's/.*"([^"]+)".*/\1/')
else
error "Neither curl nor wget found. Please install one of them."
fi
if [ -z "$VERSION" ]; then
error "Could not determine latest version. Check https://github.com/${REPO}/releases"
fi
}
# Download and install the binary
install() {
DOWNLOAD_URL="https://github.com/${REPO}/releases/download/${VERSION}/${ASSET_NAME}"
printf "\n"
info "Installing game-ci ${VERSION} (${PLATFORM}-${ARCH})"
info " from: ${DOWNLOAD_URL}"
info " to: ${INSTALL_DIR}/${BINARY_NAME}"
printf "\n"
mkdir -p "$INSTALL_DIR"
# Download with progress
if command -v curl > /dev/null 2>&1; then
HTTP_CODE=$(curl -fSL "$DOWNLOAD_URL" -o "${INSTALL_DIR}/${BINARY_NAME}" -w "%{http_code}" 2>/dev/null) || true
if [ "$HTTP_CODE" = "404" ]; then
error "Release asset not found: ${ASSET_NAME} (${VERSION}). Check available assets at https://github.com/${REPO}/releases/tag/${VERSION}"
elif [ ! -f "${INSTALL_DIR}/${BINARY_NAME}" ]; then
error "Download failed. URL: ${DOWNLOAD_URL}"
fi
elif command -v wget > /dev/null 2>&1; then
wget -q "$DOWNLOAD_URL" -O "${INSTALL_DIR}/${BINARY_NAME}" || error "Download failed. URL: ${DOWNLOAD_URL}"
fi
chmod +x "${INSTALL_DIR}/${BINARY_NAME}"
# Verify the binary works
if "${INSTALL_DIR}/${BINARY_NAME}" version > /dev/null 2>&1; then
INSTALLED_VERSION=$("${INSTALL_DIR}/${BINARY_NAME}" version 2>&1 | head -1)
info "Verified: ${INSTALLED_VERSION}"
else
warn "Binary downloaded but could not verify. It may still work."
fi
printf "\n"
printf "${BOLD}game-ci installed successfully!${RESET}\n"
printf "\n"
# Check if install dir is in PATH
case ":$PATH:" in
*":${INSTALL_DIR}:"*)
info "game-ci is already in your PATH. Run: game-ci --help"
;;
*)
SHELL_NAME=$(basename "$SHELL" 2>/dev/null || echo "sh")
case "$SHELL_NAME" in
zsh) PROFILE="~/.zshrc" ;;
bash) PROFILE="~/.bashrc" ;;
fish) PROFILE="~/.config/fish/config.fish" ;;
*) PROFILE="~/.profile" ;;
esac
printf "${YELLOW}Add game-ci to your PATH by adding this to ${PROFILE}:${RESET}\n"
printf "\n"
if [ "$SHELL_NAME" = "fish" ]; then
printf " set -gx PATH \"%s\" \$PATH\n" "$INSTALL_DIR"
else
printf " export PATH=\"%s:\$PATH\"\n" "$INSTALL_DIR"
fi
printf "\n"
info "Then restart your shell or run: source ${PROFILE}"
;;
esac
}
# Verify checksum if checksums.txt is available
verify_checksum() {
if ! command -v sha256sum > /dev/null 2>&1; then
return 0
fi
CHECKSUM_URL="https://github.com/${REPO}/releases/download/${VERSION}/checksums.txt"
CHECKSUMS=""
if command -v curl > /dev/null 2>&1; then
CHECKSUMS=$(curl -fsSL "$CHECKSUM_URL" 2>/dev/null) || return 0
elif command -v wget > /dev/null 2>&1; then
CHECKSUMS=$(wget -qO- "$CHECKSUM_URL" 2>/dev/null) || return 0
fi
if [ -z "$CHECKSUMS" ]; then
return 0
fi
EXPECTED=$(echo "$CHECKSUMS" | grep "$ASSET_NAME" | awk '{print $1}')
if [ -z "$EXPECTED" ]; then
return 0
fi
ACTUAL=$(sha256sum "${INSTALL_DIR}/${BINARY_NAME}" | awk '{print $1}')
if [ "$EXPECTED" != "$ACTUAL" ]; then
error "Checksum verification failed!\n Expected: ${EXPECTED}\n Got: ${ACTUAL}"
fi
info "Checksum verified (SHA256)"
}
# Main
detect_platform
get_latest_version
install
verify_checksum

11
jest.ci.config.js Normal file
View File

@@ -0,0 +1,11 @@
const base = require('./jest.config.js');
module.exports = {
...base,
forceExit: true,
detectOpenHandles: true,
testTimeout: 120000,
maxWorkers: 1,
};

View File

@@ -25,8 +25,6 @@ module.exports = {
// An array of regexp pattern strings, matched against all module paths before considered 'visible' to the module loader
modulePathIgnorePatterns: ['<rootDir>/lib/', '<rootDir>/dist/'],
// Files that will be run before Jest is loaded to set globals like fetch
setupFiles: ['<rootDir>/src/jest.globals.ts'],
// A list of paths to modules that run some code to configure or set up the testing framework after the environment is ready
setupFilesAfterEnv: ['<rootDir>/src/jest.setup.ts'],
// Use jest.setup.js to polyfill fetch for all tests
setupFiles: ['<rootDir>/jest.setup.js'],
};

2
jest.setup.js Normal file
View File

@@ -0,0 +1,2 @@
const fetch = require('node-fetch');
global.fetch = fetch;

View File

@@ -3,6 +3,24 @@
"version": "3.0.0",
"description": "Build Unity projects for different platforms.",
"main": "dist/index.js",
"bin": {
"game-ci": "./lib/cli.js"
},
"pkg": {
"scripts": "lib/**/*.js",
"assets": [
"lib/**/*.json",
"package.json"
],
"targets": [
"node20-linux-x64",
"node20-linux-arm64",
"node20-macos-x64",
"node20-macos-arm64",
"node20-win-x64"
],
"outputPath": "dist-binaries"
},
"repository": "git@github.com:game-ci/unity-builder.git",
"author": "Webber <webber@takken.io>",
"license": "MIT",
@@ -12,17 +30,19 @@
"lint": "prettier --check \"src/**/*.{js,ts}\" && eslint src/**/*.ts",
"format": "prettier --write \"src/**/*.{js,ts}\"",
"cli": "yarn ts-node src/index.ts -m cli",
"gcp-secrets-tests": "cross-env providerStrategy=aws cloudRunnerTests=true inputPullCommand=\"gcp-secret-manager\" populateOverride=true pullInputList=UNITY_EMAIL,UNITY_SERIAL,UNITY_PASSWORD yarn test -i -t \"cloud runner\"",
"gcp-secrets-cli": "cross-env cloudRunnerTests=true USE_IL2CPP=false inputPullCommand=\"gcp-secret-manager\" yarn ts-node src/index.ts -m cli --populateOverride true --pullInputList UNITY_EMAIL,UNITY_SERIAL,UNITY_PASSWORD",
"aws-secrets-cli": "cross-env cloudRunnerTests=true inputPullCommand=\"aws-secret-manager\" yarn ts-node src/index.ts -m cli --populateOverride true --pullInputList UNITY_EMAIL,UNITY_SERIAL,UNITY_PASSWORD",
"game-ci": "ts-node src/cli.ts",
"gcp-secrets-tests": "cross-env providerStrategy=aws orchestratorTests=true inputPullCommand=\"gcp-secret-manager\" populateOverride=true pullInputList=UNITY_EMAIL,UNITY_SERIAL,UNITY_PASSWORD yarn test -i -t \"orchestrator\"",
"gcp-secrets-cli": "cross-env orchestratorTests=true USE_IL2CPP=false inputPullCommand=\"gcp-secret-manager\" yarn ts-node src/index.ts -m cli --populateOverride true --pullInputList UNITY_EMAIL,UNITY_SERIAL,UNITY_PASSWORD",
"aws-secrets-cli": "cross-env orchestratorTests=true inputPullCommand=\"aws-secret-manager\" yarn ts-node src/index.ts -m cli --populateOverride true --pullInputList UNITY_EMAIL,UNITY_SERIAL,UNITY_PASSWORD",
"cli-aws": "cross-env providerStrategy=aws yarn run test-cli",
"cli-k8s": "cross-env providerStrategy=k8s yarn run test-cli",
"test-cli": "cross-env cloudRunnerTests=true yarn ts-node src/index.ts -m cli --projectPath test-project",
"test-cli": "cross-env orchestratorTests=true yarn ts-node src/index.ts -m cli --projectPath test-project",
"test": "jest",
"test-i": "cross-env cloudRunnerTests=true yarn test -i -t \"cloud runner\"",
"test:ci": "jest --config=jest.ci.config.js --runInBand",
"test-i": "cross-env orchestratorTests=true yarn test -i -t \"orchestrator\"",
"test-i-*": "yarn run test-i-aws && yarn run test-i-k8s",
"test-i-aws": "cross-env cloudRunnerTests=true providerStrategy=aws yarn test -i -t \"cloud runner\"",
"test-i-k8s": "cross-env cloudRunnerTests=true providerStrategy=k8s yarn test -i -t \"cloud runner\""
"test-i-aws": "cross-env orchestratorTests=true providerStrategy=aws yarn test -i -t \"orchestrator\"",
"test-i-k8s": "cross-env orchestratorTests=true providerStrategy=k8s yarn test -i -t \"orchestrator\""
},
"engines": {
"node": ">=18.x"
@@ -49,10 +69,12 @@
"nanoid": "^3.3.1",
"reflect-metadata": "^0.1.13",
"semver": "^7.5.2",
"shell-quote": "^1.8.3",
"ts-md5": "^1.3.1",
"unity-changeset": "^3.1.0",
"uuid": "^9.0.0",
"yaml": "^2.2.2"
"yaml": "^2.2.2",
"yargs": "^17.7.2"
},
"devDependencies": {
"@types/base-64": "^1.0.0",
@@ -60,6 +82,7 @@
"@types/node": "^17.0.23",
"@types/semver": "^7.3.9",
"@types/uuid": "^9.0.0",
"@types/yargs": "^17.0.35",
"@typescript-eslint/parser": "4.8.1",
"@vercel/ncc": "^0.36.1",
"cross-env": "^7.0.3",
@@ -74,6 +97,8 @@
"jest-fail-on-console": "^3.0.2",
"js-yaml": "^4.1.0",
"lefthook": "^1.6.1",
"node-fetch": "2",
"pkg": "^5.8.1",
"prettier": "^2.5.1",
"ts-jest": "^27.1.3",
"ts-node": "10.8.1",

39
src/cli.ts Normal file
View File

@@ -0,0 +1,39 @@
#!/usr/bin/env node
import yargs from 'yargs';
import { hideBin } from 'yargs/helpers';
import buildCommand from './cli/commands/build';
import activateCommand from './cli/commands/activate';
import orchestrateCommand from './cli/commands/orchestrate';
import statusCommand from './cli/commands/status';
import versionCommand from './cli/commands/version';
import updateCommand from './cli/commands/update';
import * as core from '@actions/core';
const cli = yargs(hideBin(process.argv))
.scriptName('game-ci')
.usage('$0 <command> [options]')
.command(buildCommand)
.command(activateCommand)
.command(orchestrateCommand)
.command(statusCommand)
.command(versionCommand)
.command(updateCommand)
.demandCommand(1, 'You must specify a command. Run game-ci --help for available commands.')
.strict()
.alias('h', 'help')
.epilogue('For more information, visit https://game.ci')
.wrap(Math.min(120, process.stdout.columns || 80));
async function main() {
try {
await cli.parse();
} catch (error: any) {
if (error.name !== 'YError') {
core.error(`Error: ${error.message}`);
process.exit(1);
}
}
}
main();

View File

@@ -0,0 +1,110 @@
import { execFile } from 'node:child_process';
import path from 'node:path';
/**
* Integration tests that spawn the CLI as a child process and verify
* exit codes and output. Uses node with --require ts-node/register to
* run the TypeScript entry point directly so no build step is required.
*/
const CLI_ENTRY = path.resolve(__dirname, '..', '..', 'cli.ts');
function runCli(cliArguments: string[]): Promise<{ code: number | null; stdout: string; stderr: string }> {
return new Promise((resolve) => {
execFile(
process.execPath,
['--require', 'ts-node/register/transpile-only', CLI_ENTRY, ...cliArguments],
{ timeout: 30_000, cwd: path.resolve(__dirname, '..', '..', '..') },
(error, stdout, stderr) => {
resolve({
code: error ? error.code ?? 1 : 0,
stdout: stdout.toString(),
stderr: stderr.toString(),
});
},
);
});
}
// Integration tests spawn child processes which need more time than the default 5s
jest.setTimeout(30_000);
describe('CLI integration', () => {
it('exits 0 and shows all commands for --help', async () => {
const result = await runCli(['--help']);
expect(result.code).toStrictEqual(0);
expect(result.stdout).toContain('game-ci');
expect(result.stdout).toContain('build');
expect(result.stdout).toContain('activate');
expect(result.stdout).toContain('orchestrate');
expect(result.stdout).toContain('status');
expect(result.stdout).toContain('version');
expect(result.stdout).toContain('update');
});
it('exits 0 and shows version info for version command', async () => {
const result = await runCli(['version']);
expect(result.code).toStrictEqual(0);
expect(result.stdout).toContain('unity-builder');
});
it('exits 0 and shows build flags for build --help', async () => {
const result = await runCli(['build', '--help']);
expect(result.code).toStrictEqual(0);
expect(result.stdout).toContain('--target-platform');
expect(result.stdout).toContain('--unity-version');
expect(result.stdout).toContain('--project-path');
expect(result.stdout).toContain('--build-name');
expect(result.stdout).toContain('--builds-path');
expect(result.stdout).toContain('--build-method');
expect(result.stdout).toContain('--custom-parameters');
expect(result.stdout).toContain('--provider-strategy');
});
it('exits non-zero for an unknown command', async () => {
const result = await runCli(['nonexistent']);
expect(result.code).not.toStrictEqual(0);
});
it('exits non-zero when no command is provided', async () => {
const result = await runCli([]);
expect(result.code).not.toStrictEqual(0);
});
it('exits 0 for orchestrate --help', async () => {
const result = await runCli(['orchestrate', '--help']);
expect(result.code).toStrictEqual(0);
expect(result.stdout).toContain('--target-platform');
expect(result.stdout).toContain('--provider-strategy');
expect(result.stdout).toContain('cache');
});
it('exits 0 for activate --help', async () => {
const result = await runCli(['activate', '--help']);
expect(result.code).toStrictEqual(0);
expect(result.stdout).toContain('activate');
});
it('exits 0 for orchestrate cache --help', async () => {
const result = await runCli(['orchestrate', 'cache', '--help']);
expect(result.code).toStrictEqual(0);
expect(result.stdout).toContain('cache');
});
it('exits 0 for update --help', async () => {
const result = await runCli(['update', '--help']);
expect(result.code).toStrictEqual(0);
expect(result.stdout).toContain('update');
expect(result.stdout).toContain('--force');
expect(result.stdout).toContain('--version');
});
});

View File

@@ -0,0 +1,245 @@
import buildCommand from '../commands/build';
import activateCommand from '../commands/activate';
import orchestrateCommand from '../commands/orchestrate';
import statusCommand from '../commands/status';
import versionCommand from '../commands/version';
import updateCommand from '../commands/update';
function createFakeYargs(): { yargs: any; options: Record<string, any> } {
const options: Record<string, any> = {};
const yargs: any = {
option: jest.fn(),
positional: jest.fn(),
example: jest.fn(),
env: jest.fn(),
command: jest.fn(),
};
yargs.option.mockImplementation((name: string, config: any) => {
options[name] = config;
return yargs;
});
yargs.positional.mockImplementation((name: string, config: any) => {
options[name] = config;
return yargs;
});
yargs.example.mockReturnValue(yargs);
yargs.env.mockReturnValue(yargs);
yargs.command.mockReturnValue(yargs);
return { yargs, options };
}
describe('CLI commands', () => {
describe('build command', () => {
it('exports the correct command name', () => {
expect(buildCommand.command).toStrictEqual('build');
});
it('has a description', () => {
expect(buildCommand.describe).toBeTruthy();
});
it('has a builder function', () => {
expect(typeof buildCommand.builder).toStrictEqual('function');
});
it('has a handler function', () => {
expect(typeof buildCommand.handler).toStrictEqual('function');
});
it('defines all expected build flags via builder', () => {
const { yargs, options } = createFakeYargs();
(buildCommand.builder as Function)(yargs);
// Core build flags
expect(options['target-platform']).toBeDefined();
expect(options['target-platform'].demandOption).toStrictEqual(true);
expect(options['unity-version']).toBeDefined();
expect(options['project-path']).toBeDefined();
expect(options['build-profile']).toBeDefined();
expect(options['build-name']).toBeDefined();
expect(options['builds-path']).toBeDefined();
expect(options['build-method']).toBeDefined();
expect(options['custom-parameters']).toBeDefined();
expect(options['versioning']).toBeDefined();
expect(options['version']).toBeDefined();
expect(options['custom-image']).toBeDefined();
expect(options['manual-exit']).toBeDefined();
expect(options['enable-gpu']).toBeDefined();
// Android flags
expect(options['android-version-code']).toBeDefined();
expect(options['android-export-type']).toBeDefined();
expect(options['android-keystore-name']).toBeDefined();
expect(options['android-keystore-base64']).toBeDefined();
expect(options['android-keystore-pass']).toBeDefined();
expect(options['android-keyalias-name']).toBeDefined();
expect(options['android-keyalias-pass']).toBeDefined();
expect(options['android-target-sdk-version']).toBeDefined();
expect(options['android-symbol-type']).toBeDefined();
// Docker flags
expect(options['docker-cpu-limit']).toBeDefined();
expect(options['docker-memory-limit']).toBeDefined();
expect(options['docker-workspace-path']).toBeDefined();
expect(options['run-as-host-user']).toBeDefined();
expect(options['chown-files-to']).toBeDefined();
// Provider flags
expect(options['provider-strategy']).toBeDefined();
expect(options['skip-activation']).toBeDefined();
expect(options['unity-licensing-server']).toBeDefined();
});
it('sets correct default values', () => {
const { yargs, options } = createFakeYargs();
(buildCommand.builder as Function)(yargs);
expect(options['unity-version'].default).toStrictEqual('auto');
expect(options['project-path'].default).toStrictEqual('.');
expect(options['builds-path'].default).toStrictEqual('build');
expect(options['versioning'].default).toStrictEqual('Semantic');
expect(options['manual-exit'].default).toStrictEqual(false);
expect(options['enable-gpu'].default).toStrictEqual(false);
expect(options['android-export-type'].default).toStrictEqual('androidPackage');
expect(options['android-symbol-type'].default).toStrictEqual('none');
expect(options['provider-strategy'].default).toStrictEqual('local');
});
it('provides camelCase aliases for kebab-case options', () => {
const { yargs, options } = createFakeYargs();
(buildCommand.builder as Function)(yargs);
expect(options['target-platform'].alias).toStrictEqual('targetPlatform');
expect(options['unity-version'].alias).toStrictEqual('unityVersion');
expect(options['project-path'].alias).toStrictEqual('projectPath');
expect(options['build-name'].alias).toStrictEqual('buildName');
expect(options['builds-path'].alias).toStrictEqual('buildsPath');
expect(options['build-method'].alias).toStrictEqual('buildMethod');
});
});
describe('activate command', () => {
it('exports the correct command name', () => {
expect(activateCommand.command).toStrictEqual('activate');
});
it('has a description', () => {
expect(activateCommand.describe).toBeTruthy();
});
it('has a builder function', () => {
expect(typeof activateCommand.builder).toStrictEqual('function');
});
it('has a handler function', () => {
expect(typeof activateCommand.handler).toStrictEqual('function');
});
});
describe('orchestrate command', () => {
it('exports the correct command name', () => {
expect(orchestrateCommand.command).toStrictEqual('orchestrate');
});
it('has a description', () => {
expect(orchestrateCommand.describe).toBeTruthy();
});
it('has a builder function', () => {
expect(typeof orchestrateCommand.builder).toStrictEqual('function');
});
it('has a handler function', () => {
expect(typeof orchestrateCommand.handler).toStrictEqual('function');
});
it('defines key orchestrator flags', () => {
const { yargs, options } = createFakeYargs();
(orchestrateCommand.builder as Function)(yargs);
expect(options['target-platform']).toBeDefined();
expect(options['provider-strategy']).toBeDefined();
expect(options['provider-strategy'].default).toStrictEqual('aws');
expect(options['aws-stack-name']).toBeDefined();
expect(options['kube-config']).toBeDefined();
expect(options['kube-volume']).toBeDefined();
expect(options['cache-key']).toBeDefined();
expect(options['watch-to-end']).toBeDefined();
expect(options['clone-depth']).toBeDefined();
});
it('registers cache as a subcommand', () => {
const { yargs } = createFakeYargs();
(orchestrateCommand.builder as Function)(yargs);
expect(yargs.command).toHaveBeenCalled();
});
});
describe('status command', () => {
it('exports the correct command name', () => {
expect(statusCommand.command).toStrictEqual('status');
});
it('has a description', () => {
expect(statusCommand.describe).toBeTruthy();
});
it('has a handler function', () => {
expect(typeof statusCommand.handler).toStrictEqual('function');
});
});
describe('version command', () => {
it('exports the correct command name', () => {
expect(versionCommand.command).toStrictEqual('version');
});
it('has a description', () => {
expect(versionCommand.describe).toBeTruthy();
});
it('has a handler function', () => {
expect(typeof versionCommand.handler).toStrictEqual('function');
});
});
describe('update command', () => {
it('exports the correct command name', () => {
expect(updateCommand.command).toStrictEqual('update');
});
it('has a description', () => {
expect(updateCommand.describe).toBeTruthy();
});
it('has a builder function', () => {
expect(typeof updateCommand.builder).toStrictEqual('function');
});
it('has a handler function', () => {
expect(typeof updateCommand.handler).toStrictEqual('function');
});
it('defines force and version flags', () => {
const { yargs, options } = createFakeYargs();
(updateCommand.builder as Function)(yargs);
expect(options['force']).toBeDefined();
expect(options['force'].type).toStrictEqual('boolean');
expect(options['force'].default).toStrictEqual(false);
expect(options['version']).toBeDefined();
expect(options['version'].type).toStrictEqual('string');
});
});
});

View File

@@ -0,0 +1,221 @@
import { mapCliArgumentsToInput, CliArguments } from '../input-mapper';
import { Cli } from '../../model/cli/cli';
import GitHub from '../../model/github';
afterEach(() => {
jest.restoreAllMocks();
Cli.options = undefined;
});
describe('mapCliArgumentsToInput', () => {
describe('basic mapping', () => {
it('populates Cli.options from CLI arguments', () => {
const cliArguments: CliArguments = {
targetPlatform: 'StandaloneLinux64',
unityVersion: '2022.3.56f1',
projectPath: './my-project',
};
mapCliArgumentsToInput(cliArguments);
expect(Cli.options).toBeDefined();
expect(Cli.options!['targetPlatform']).toStrictEqual('StandaloneLinux64');
expect(Cli.options!['unityVersion']).toStrictEqual('2022.3.56f1');
expect(Cli.options!['projectPath']).toStrictEqual('./my-project');
});
it('disables GitHub Actions input reading', () => {
const cliArguments: CliArguments = { targetPlatform: 'WebGL' };
mapCliArgumentsToInput(cliArguments);
expect(GitHub.githubInputEnabled).toStrictEqual(false);
});
it('sets mode to cli by default when not provided', () => {
const cliArguments: CliArguments = { targetPlatform: 'Android' };
mapCliArgumentsToInput(cliArguments);
expect(Cli.options!['mode']).toStrictEqual('cli');
});
it('preserves an explicitly provided mode', () => {
const cliArguments: CliArguments = { targetPlatform: 'Android', mode: 'custom-mode' };
mapCliArgumentsToInput(cliArguments);
expect(Cli.options!['mode']).toStrictEqual('custom-mode');
});
});
describe('default values', () => {
it('omits undefined values from Cli.options', () => {
const cliArguments: CliArguments = {
targetPlatform: 'StandaloneLinux64',
unityVersion: undefined,
buildName: undefined,
};
mapCliArgumentsToInput(cliArguments);
expect(Cli.options!['targetPlatform']).toStrictEqual('StandaloneLinux64');
expect(Cli.options!).not.toHaveProperty('unityVersion');
expect(Cli.options!).not.toHaveProperty('buildName');
});
});
describe('boolean conversion', () => {
it('converts boolean true to string "true"', () => {
const cliArguments: CliArguments = { manualExit: true };
mapCliArgumentsToInput(cliArguments);
expect(Cli.options!['manualExit']).toStrictEqual('true');
});
it('converts boolean false to string "false"', () => {
const cliArguments: CliArguments = { enableGpu: false };
mapCliArgumentsToInput(cliArguments);
expect(Cli.options!['enableGpu']).toStrictEqual('false');
});
it('converts allowDirtyBuild boolean to string', () => {
const cliArguments: CliArguments = { allowDirtyBuild: true };
mapCliArgumentsToInput(cliArguments);
expect(Cli.options!['allowDirtyBuild']).toStrictEqual('true');
});
});
describe('yargs internal properties', () => {
it('filters out yargs _ property', () => {
const cliArguments: CliArguments = {
targetPlatform: 'iOS',
_: ['build'] as any,
};
mapCliArgumentsToInput(cliArguments);
expect(Cli.options!).not.toHaveProperty('_');
});
it('filters out yargs $0 property', () => {
const cliArguments: CliArguments = {
targetPlatform: 'iOS',
$0: 'game-ci' as any,
};
mapCliArgumentsToInput(cliArguments);
expect(Cli.options!).not.toHaveProperty('$0');
});
});
describe('flag name conversion', () => {
it('passes camelCase keys through directly', () => {
const cliArguments: CliArguments = {
androidKeystoreName: 'my.keystore',
androidKeystorePass: 'secret',
dockerCpuLimit: '4',
dockerMemoryLimit: '8g',
};
mapCliArgumentsToInput(cliArguments);
expect(Cli.options!['androidKeystoreName']).toStrictEqual('my.keystore');
expect(Cli.options!['androidKeystorePass']).toStrictEqual('secret');
expect(Cli.options!['dockerCpuLimit']).toStrictEqual('4');
expect(Cli.options!['dockerMemoryLimit']).toStrictEqual('8g');
});
it('maps all android-related arguments', () => {
const cliArguments: CliArguments = {
androidVersionCode: '42',
androidExportType: 'androidAppBundle',
androidKeystoreBase64: 'base64data',
androidKeyaliasName: 'myalias',
androidKeyaliasPass: 'aliaspass',
androidTargetSdkVersion: '33',
androidSymbolType: 'public',
};
mapCliArgumentsToInput(cliArguments);
expect(Cli.options!['androidVersionCode']).toStrictEqual('42');
expect(Cli.options!['androidExportType']).toStrictEqual('androidAppBundle');
expect(Cli.options!['androidKeystoreBase64']).toStrictEqual('base64data');
expect(Cli.options!['androidKeyaliasName']).toStrictEqual('myalias');
expect(Cli.options!['androidKeyaliasPass']).toStrictEqual('aliaspass');
expect(Cli.options!['androidTargetSdkVersion']).toStrictEqual('33');
expect(Cli.options!['androidSymbolType']).toStrictEqual('public');
});
it('maps docker and container arguments', () => {
const cliArguments: CliArguments = {
dockerIsolationMode: 'hyperv',
dockerWorkspacePath: '/custom/workspace',
containerRegistryRepository: 'custom/editor',
containerRegistryImageVersion: '5',
runAsHostUser: 'true',
chownFilesTo: 'root:root',
};
mapCliArgumentsToInput(cliArguments);
expect(Cli.options!['dockerIsolationMode']).toStrictEqual('hyperv');
expect(Cli.options!['dockerWorkspacePath']).toStrictEqual('/custom/workspace');
expect(Cli.options!['containerRegistryRepository']).toStrictEqual('custom/editor');
expect(Cli.options!['containerRegistryImageVersion']).toStrictEqual('5');
expect(Cli.options!['runAsHostUser']).toStrictEqual('true');
expect(Cli.options!['chownFilesTo']).toStrictEqual('root:root');
});
it('maps orchestrator-related arguments', () => {
const cliArguments: CliArguments = {
providerStrategy: 'k8s',
awsStackName: 'my-stack',
kubeConfig: 'base64config',
kubeVolume: 'my-pvc',
kubeVolumeSize: '10Gi',
kubeStorageClass: 'gp3',
containerCpu: '2048',
containerMemory: '4096',
cacheKey: 'my-cache',
watchToEnd: 'false',
cloneDepth: '100',
};
mapCliArgumentsToInput(cliArguments);
expect(Cli.options!['providerStrategy']).toStrictEqual('k8s');
expect(Cli.options!['awsStackName']).toStrictEqual('my-stack');
expect(Cli.options!['kubeConfig']).toStrictEqual('base64config');
expect(Cli.options!['kubeVolume']).toStrictEqual('my-pvc');
expect(Cli.options!['kubeVolumeSize']).toStrictEqual('10Gi');
expect(Cli.options!['kubeStorageClass']).toStrictEqual('gp3');
expect(Cli.options!['containerCpu']).toStrictEqual('2048');
expect(Cli.options!['containerMemory']).toStrictEqual('4096');
expect(Cli.options!['cacheKey']).toStrictEqual('my-cache');
expect(Cli.options!['watchToEnd']).toStrictEqual('false');
expect(Cli.options!['cloneDepth']).toStrictEqual('100');
});
});
describe('Cli.isCliMode integration', () => {
it('enables CLI mode after mapping', () => {
const cliArguments: CliArguments = { targetPlatform: 'WebGL' };
mapCliArgumentsToInput(cliArguments);
expect(Cli.isCliMode).toStrictEqual(true);
});
it('is not in CLI mode before mapping', () => {
expect(Cli.isCliMode).toStrictEqual(false);
});
});
});

View File

@@ -0,0 +1,83 @@
import type { CommandModule } from 'yargs';
import * as core from '@actions/core';
import { mapCliArgumentsToInput, CliArguments } from '../input-mapper';
interface ActivateArguments extends CliArguments {
unityVersion?: string;
unitySerial?: string;
unityLicensingServer?: string;
}
const activateCommand: CommandModule<object, ActivateArguments> = {
command: 'activate',
describe: 'Verify Unity license configuration',
builder: (yargs) => {
return yargs
.option('unity-version', {
alias: 'unityVersion',
type: 'string',
description: 'Version of Unity to activate',
default: 'auto',
})
.option('unity-licensing-server', {
alias: 'unityLicensingServer',
type: 'string',
description: 'The Unity licensing server address for floating licenses',
default: '',
})
.env('UNITY')
.example(
'UNITY_SERIAL=XXXX-XXXX-XXXX-XXXX game-ci activate',
'Activate Unity using a serial from environment variable',
)
.example(
'game-ci activate --unity-licensing-server http://license-server:8080',
'Activate Unity using a floating license server',
) as any;
},
handler: async (cliArguments) => {
try {
mapCliArgumentsToInput(cliArguments);
const unitySerial = process.env.UNITY_SERIAL;
const unityLicense = process.env.UNITY_LICENSE;
const licensingServer = cliArguments.unityLicensingServer || process.env.UNITY_LICENSING_SERVER || '';
if (licensingServer) {
core.info(`Activating Unity via licensing server: ${licensingServer}`);
core.info('Floating license activation is handled automatically during builds.');
core.info('No manual activation step is needed when using a licensing server.');
return;
}
if (!unitySerial && !unityLicense) {
throw new Error(
'No Unity license found.\n\n' +
'Provide one of the following:\n' +
' - UNITY_SERIAL environment variable (professional license)\n' +
' - UNITY_LICENSE environment variable (personal license file content)\n' +
' - --unity-licensing-server flag (floating license)\n\n' +
'For more information, visit: https://game.ci/docs/github/activation',
);
}
if (unitySerial) {
const maskedSerial = unitySerial.length > 8 ? `${unitySerial.slice(0, 4)}...${unitySerial.slice(-4)}` : '****';
core.info(`Unity serial detected: ${maskedSerial}`);
core.info('License will be activated automatically when running a build.');
} else if (unityLicense) {
core.info('Unity license file detected from UNITY_LICENSE environment variable.');
core.info('License will be activated automatically when running a build.');
}
core.info('\nActivation verified. You can now run: game-ci build --target-platform <platform>');
} catch (error: any) {
core.setFailed(`Activation failed: ${error.message}`);
throw error;
}
},
};
export default activateCommand;

299
src/cli/commands/build.ts Normal file
View File

@@ -0,0 +1,299 @@
import type { CommandModule } from 'yargs';
import * as core from '@actions/core';
import { BuildParameters, ImageTag, Orchestrator } from '../../model';
import { mapCliArgumentsToInput, CliArguments } from '../input-mapper';
import MacBuilder from '../../model/mac-builder';
import Docker from '../../model/docker';
import Action from '../../model/action';
import PlatformSetup from '../../model/platform-setup';
interface BuildArguments extends CliArguments {
targetPlatform: string;
}
const buildCommand: CommandModule<object, BuildArguments> = {
command: 'build',
describe: 'Build a Unity project',
builder: (yargs) => {
return yargs
.option('target-platform', {
alias: 'targetPlatform',
type: 'string',
description: 'Platform that the build should target',
demandOption: true,
})
.option('unity-version', {
alias: 'unityVersion',
type: 'string',
description: 'Version of Unity to use for building the project. Use "auto" to detect.',
default: 'auto',
})
.option('project-path', {
alias: 'projectPath',
type: 'string',
description: 'Path to the Unity project to be built',
default: '.',
})
.option('build-profile', {
alias: 'buildProfile',
type: 'string',
description: 'Path to the build profile to activate, relative to the project root',
default: '',
})
.option('build-name', {
alias: 'buildName',
type: 'string',
description: 'Name of the build (no file extension)',
default: '',
})
.option('builds-path', {
alias: 'buildsPath',
type: 'string',
description: 'Path where the builds should be stored',
default: 'build',
})
.option('build-method', {
alias: 'buildMethod',
type: 'string',
description: 'Path to a Namespace.Class.StaticMethod to run to perform the build',
default: '',
})
.option('custom-parameters', {
alias: 'customParameters',
type: 'string',
description: 'Custom parameters to configure the build',
default: '',
})
.option('versioning', {
type: 'string',
description: 'The versioning scheme to use when building the project',
default: 'Semantic',
})
.option('version', {
type: 'string',
description: 'The version, when used with the "Custom" versioning scheme',
default: '',
})
.option('custom-image', {
alias: 'customImage',
type: 'string',
description: 'Specific docker image that should be used for building the project',
default: '',
})
.option('manual-exit', {
alias: 'manualExit',
type: 'boolean',
description: 'Suppresses -quit. Exit your build method using EditorApplication.Exit(0) instead.',
default: false,
})
.option('enable-gpu', {
alias: 'enableGpu',
type: 'boolean',
description: 'Launches unity without specifying -nographics',
default: false,
})
.option('android-version-code', {
alias: 'androidVersionCode',
type: 'string',
description: 'The android versionCode',
default: '',
})
.option('android-export-type', {
alias: 'androidExportType',
type: 'string',
description: 'The android export type (androidPackage, androidAppBundle, androidStudioProject)',
default: 'androidPackage',
})
.option('android-keystore-name', {
alias: 'androidKeystoreName',
type: 'string',
description: 'The android keystoreName',
default: '',
})
.option('android-keystore-base64', {
alias: 'androidKeystoreBase64',
type: 'string',
description: 'The base64 contents of the android keystore file',
default: '',
})
.option('android-keystore-pass', {
alias: 'androidKeystorePass',
type: 'string',
description: 'The android keystorePass',
default: '',
})
.option('android-keyalias-name', {
alias: 'androidKeyaliasName',
type: 'string',
description: 'The android keyaliasName',
default: '',
})
.option('android-keyalias-pass', {
alias: 'androidKeyaliasPass',
type: 'string',
description: 'The android keyaliasPass',
default: '',
})
.option('android-target-sdk-version', {
alias: 'androidTargetSdkVersion',
type: 'string',
description: 'The android target API level',
default: '',
})
.option('android-symbol-type', {
alias: 'androidSymbolType',
type: 'string',
description: 'The android symbol type to export (none, public, debugging)',
default: 'none',
})
.option('docker-cpu-limit', {
alias: 'dockerCpuLimit',
type: 'string',
description: 'Number of CPU cores to assign the docker container',
default: '',
})
.option('docker-memory-limit', {
alias: 'dockerMemoryLimit',
type: 'string',
description: 'Amount of memory to assign the docker container (e.g. 512m, 4g)',
default: '',
})
.option('docker-workspace-path', {
alias: 'dockerWorkspacePath',
type: 'string',
description: 'The path to mount the workspace inside the docker container',
default: '/github/workspace',
})
.option('run-as-host-user', {
alias: 'runAsHostUser',
type: 'string',
description: 'Whether to run as a user that matches the host system',
default: 'false',
})
.option('chown-files-to', {
alias: 'chownFilesTo',
type: 'string',
description: 'User and optionally group to give ownership of build artifacts',
default: '',
})
.option('ssh-agent', {
alias: 'sshAgent',
type: 'string',
description: 'SSH Agent path to forward to the container',
default: '',
})
.option('git-private-token', {
alias: 'gitPrivateToken',
type: 'string',
description: 'GitHub private token to pull from GitHub',
default: '',
})
.option('provider-strategy', {
alias: 'providerStrategy',
type: 'string',
description: 'Execution strategy: local, k8s, or aws',
default: 'local',
})
.option('skip-activation', {
alias: 'skipActivation',
type: 'string',
description: 'Skip the activation/deactivation of Unity',
default: 'false',
})
.option('unity-licensing-server', {
alias: 'unityLicensingServer',
type: 'string',
description: 'The Unity licensing server address',
default: '',
})
.option('container-registry-repository', {
alias: 'containerRegistryRepository',
type: 'string',
description: 'Container registry and repository to pull image from. Only applicable if customImage is not set.',
default: 'unityci/editor',
})
.option('container-registry-image-version', {
alias: 'containerRegistryImageVersion',
type: 'string',
description: 'Container registry image version. Only applicable if customImage is not set.',
default: '3',
})
.option('docker-isolation-mode', {
alias: 'dockerIsolationMode',
type: 'string',
description:
'Isolation mode to use for the docker container (process, hyperv, or default). Only applicable on Windows.',
default: 'default',
})
.option('ssh-public-keys-directory-path', {
alias: 'sshPublicKeysDirectoryPath',
type: 'string',
description: 'Path to a directory containing SSH public keys to forward to the container',
default: '',
})
.option('cache-unity-installation-on-mac', {
alias: 'cacheUnityInstallationOnMac',
type: 'boolean',
description: 'Whether to cache the Unity hub and editor installation on MacOS',
default: false,
})
.option('unity-hub-version-on-mac', {
alias: 'unityHubVersionOnMac',
type: 'string',
description: 'The version of Unity Hub to install on MacOS (e.g. 3.4.0). Defaults to latest available on brew.',
default: '',
})
.example('game-ci build --target-platform StandaloneLinux64', 'Build for Linux using auto-detected Unity version')
.example(
'game-ci build --target-platform Android --unity-version 2022.3.56f1 --build-method MyBuild.Run',
'Build for Android with a specific Unity version and build method',
) as any;
},
handler: async (cliArguments) => {
try {
mapCliArgumentsToInput(cliArguments);
const buildParameters = await BuildParameters.create();
const baseImage = new ImageTag(buildParameters);
let exitCode = -1;
if (buildParameters.providerStrategy === 'local') {
core.info(`Building locally for ${buildParameters.targetPlatform}...`);
core.info(`Unity version: ${buildParameters.editorVersion}`);
core.info(`Project path: ${buildParameters.projectPath}`);
const actionFolder = Action.actionFolder;
await PlatformSetup.setup(buildParameters, actionFolder);
exitCode =
process.platform === 'darwin'
? await MacBuilder.run(actionFolder)
: await Docker.run(baseImage.toString(), {
workspace: process.cwd(),
actionFolder,
...buildParameters,
});
} else {
core.info(`Building via orchestrator (${buildParameters.providerStrategy})...`);
await Orchestrator.run(buildParameters, baseImage.toString());
exitCode = 0;
}
// Output results
core.info(`\nBuild completed with exit code: ${exitCode}`);
core.info(`Build version: ${buildParameters.buildVersion}`);
core.info(`Build path: ${buildParameters.buildPath}`);
if (exitCode !== 0) {
throw new Error(`Build failed with exit code ${exitCode}`);
}
} catch (error: any) {
core.setFailed(`Build failed: ${error.message}`);
throw error;
}
},
};
export default buildCommand;

160
src/cli/commands/cache.ts Normal file
View File

@@ -0,0 +1,160 @@
import type { CommandModule } from 'yargs';
import * as core from '@actions/core';
import fs from 'node:fs';
import path from 'node:path';
const cacheCommand: CommandModule = {
command: 'cache <action>',
describe: 'Manage build caches',
builder: (yargs) => {
return yargs
.positional('action', {
describe: 'Cache action to perform',
choices: ['list', 'restore', 'clear'] as const,
})
.option('cache-dir', {
alias: 'cacheDir',
type: 'string',
description: 'Path to the cache directory',
default: '',
})
.option('project-path', {
alias: 'projectPath',
type: 'string',
description: 'Path to the Unity project',
default: '.',
})
.example('game-ci orchestrate cache list', 'List all cached workspaces')
.example('game-ci orchestrate cache restore --cache-dir ./my-cache', 'Restore a cached workspace')
.example('game-ci orchestrate cache clear', 'Clear all cached workspaces');
},
handler: async (cliArguments) => {
const action = cliArguments.action as string;
const projectPath = (cliArguments.projectPath as string) || '.';
const cacheDirectory = (cliArguments.cacheDir as string) || path.join(projectPath, 'Library');
try {
switch (action) {
case 'list': {
await listCache(cacheDirectory, projectPath);
break;
}
case 'restore': {
await restoreCache(cacheDirectory);
break;
}
case 'clear': {
await clearCache(cacheDirectory);
break;
}
default: {
throw new Error(`Unknown cache action: ${action}. Available actions: list, restore, clear`);
}
}
} catch (error: any) {
core.setFailed(`Cache operation failed: ${error.message}`);
throw error;
}
},
};
async function listCache(cacheDirectory: string, projectPath: string): Promise<void> {
const libraryPath = path.resolve(projectPath, 'Library');
core.info('Cache Status:');
core.info('=============');
if (fs.existsSync(libraryPath)) {
const stats = fs.statSync(libraryPath);
const files = fs.readdirSync(libraryPath);
core.info(` Library folder: ${libraryPath}`);
core.info(` Entries: ${files.length}`);
core.info(` Last modified: ${stats.mtime.toISOString()}`);
// Show size of key subdirectories
const keyDirectories = ['PackageCache', 'ScriptAssemblies', 'ShaderCache', 'Bee'];
for (const directory of keyDirectories) {
const directoryPath = path.join(libraryPath, directory);
if (fs.existsSync(directoryPath)) {
const directoryStats = fs.statSync(directoryPath);
core.info(` ${directory}/: exists (modified ${directoryStats.mtime.toISOString()})`);
}
}
} else {
core.info(` Library folder not found at: ${libraryPath}`);
core.info(' No cache available. First build will be a clean build.');
}
// Check for .tar cache files if a custom cache dir is specified
if (cacheDirectory && cacheDirectory !== libraryPath && fs.existsSync(cacheDirectory)) {
core.info(`\nCache directory: ${cacheDirectory}`);
const cacheFiles = fs.readdirSync(cacheDirectory).filter((f) => f.endsWith('.tar') || f.endsWith('.tar.lz4'));
if (cacheFiles.length > 0) {
core.info(` Cache archives found: ${cacheFiles.length}`);
for (const file of cacheFiles) {
const filePath = path.join(cacheDirectory, file);
const fileStats = fs.statSync(filePath);
const sizeMegabytes = (fileStats.size / (1024 * 1024)).toFixed(1);
core.info(` - ${file} (${sizeMegabytes} MB, ${fileStats.mtime.toISOString()})`);
}
} else {
core.info(' No cache archives found.');
}
}
}
async function restoreCache(cacheDirectory: string): Promise<void> {
if (!cacheDirectory) {
throw new Error('--cache-dir is required for restore');
}
if (!fs.existsSync(cacheDirectory)) {
core.info(`Cache directory does not exist: ${cacheDirectory}`);
core.info('Nothing to restore.');
return;
}
const cacheFiles = fs.readdirSync(cacheDirectory).filter((f) => f.endsWith('.tar') || f.endsWith('.tar.lz4'));
if (cacheFiles.length === 0) {
core.info('No cache archives found to restore.');
return;
}
// Sort by modification time, newest first
const sorted = cacheFiles
.map((f) => ({ name: f, mtime: fs.statSync(path.join(cacheDirectory, f)).mtime }))
.sort((a, b) => b.mtime.getTime() - a.mtime.getTime());
core.info(`Found ${sorted.length} cache archive(s). Latest: ${sorted[0].name}`);
core.info('Use the orchestrator cache system for full restore functionality:');
core.info(' game-ci orchestrate --cache-key <key> ...');
}
async function clearCache(cacheDirectory: string): Promise<void> {
let cleared = false;
if (cacheDirectory && fs.existsSync(cacheDirectory)) {
const cacheFiles = fs.readdirSync(cacheDirectory).filter((f) => f.endsWith('.tar') || f.endsWith('.tar.lz4'));
if (cacheFiles.length > 0) {
for (const file of cacheFiles) {
fs.unlinkSync(path.join(cacheDirectory, file));
core.info(` Removed: ${file}`);
}
cleared = true;
}
}
if (!cleared) {
core.info('No cache archives found to clear.');
} else {
core.info('Cache cleared.');
}
}
export default cacheCommand;

View File

@@ -0,0 +1,222 @@
import type { CommandModule } from 'yargs';
import * as core from '@actions/core';
import { BuildParameters, ImageTag, Orchestrator } from '../../model';
import { mapCliArgumentsToInput, CliArguments } from '../input-mapper';
import cacheCommand from './cache';
interface OrchestrateArguments extends CliArguments {
targetPlatform: string;
providerStrategy?: string;
}
const orchestrateCommand: CommandModule<object, OrchestrateArguments> = {
command: 'orchestrate',
describe: 'Orchestrator — remote builds, cache management, and provider tools',
builder: (yargs) => {
return yargs
.command(cacheCommand)
.option('target-platform', {
alias: 'targetPlatform',
type: 'string',
description: 'Platform that the build should target',
})
.option('provider-strategy', {
alias: 'providerStrategy',
type: 'string',
description: 'Orchestrator provider: aws, k8s, local-docker, local-system',
default: 'aws',
})
.option('unity-version', {
alias: 'unityVersion',
type: 'string',
description: 'Version of Unity to use for building',
default: 'auto',
})
.option('project-path', {
alias: 'projectPath',
type: 'string',
description: 'Path to the Unity project to be built',
default: '.',
})
.option('build-name', {
alias: 'buildName',
type: 'string',
description: 'Name of the build',
default: '',
})
.option('builds-path', {
alias: 'buildsPath',
type: 'string',
description: 'Path where the builds should be stored',
default: 'build',
})
.option('build-method', {
alias: 'buildMethod',
type: 'string',
description: 'Path to a Namespace.Class.StaticMethod to run to perform the build',
default: '',
})
.option('custom-parameters', {
alias: 'customParameters',
type: 'string',
description: 'Custom parameters to configure the build',
default: '',
})
.option('versioning', {
type: 'string',
description: 'The versioning scheme to use',
default: 'None',
})
.option('aws-stack-name', {
alias: 'awsStackName',
type: 'string',
description: 'The Cloud Formation stack name (AWS provider)',
default: 'game-ci',
})
.option('kube-config', {
alias: 'kubeConfig',
type: 'string',
description: 'Base64 encoded Kubernetes config (K8s provider)',
default: '',
})
.option('kube-volume', {
alias: 'kubeVolume',
type: 'string',
description: 'Persistent Volume Claim name for Unity build (K8s provider)',
default: '',
})
.option('kube-volume-size', {
alias: 'kubeVolumeSize',
type: 'string',
description: 'Disc space for Kubernetes Persistent Volume',
default: '5Gi',
})
.option('container-cpu', {
alias: 'containerCpu',
type: 'string',
description: 'CPU allocation for remote build container',
default: '1024',
})
.option('container-memory', {
alias: 'containerMemory',
type: 'string',
description: 'Memory allocation for remote build container',
default: '3072',
})
.option('cache-key', {
alias: 'cacheKey',
type: 'string',
description: 'Cache key to indicate bucket for cache',
default: '',
})
.option('git-private-token', {
alias: 'gitPrivateToken',
type: 'string',
description: 'GitHub private token for repository access',
default: '',
})
.option('allow-dirty-build', {
alias: 'allowDirtyBuild',
type: 'boolean',
description: 'Allow builds from dirty branches',
default: false,
})
.option('watch-to-end', {
alias: 'watchToEnd',
type: 'string',
description: 'Whether to watch the build to completion',
default: 'true',
})
.option('clone-depth', {
alias: 'cloneDepth',
type: 'string',
description: 'Git clone depth (0 for full clone)',
default: '50',
})
.option('skip-activation', {
alias: 'skipActivation',
type: 'string',
description: 'Skip Unity activation/deactivation',
default: 'false',
})
.option('kube-storage-class', {
alias: 'kubeStorageClass',
type: 'string',
description: 'Kubernetes storage class to use for orchestrator jobs. Leave empty to install rook cluster.',
default: '',
})
.option('read-input-from-override-list', {
alias: 'readInputFromOverrideList',
type: 'string',
description: 'Comma separated list of input value names to read from the input override command',
default: '',
})
.option('read-input-override-command', {
alias: 'readInputOverrideCommand',
type: 'string',
description: 'Command to execute to pull input from an external source (e.g. cloud provider secret managers)',
default: '',
})
.option('post-build-steps', {
alias: 'postBuildSteps',
type: 'string',
description:
'Post build job in yaml format with the keys image, secrets (name, value object array), command string',
default: '',
})
.option('pre-build-steps', {
alias: 'preBuildSteps',
type: 'string',
description:
'Pre build job after repository setup but before the build job (yaml format with keys image, secrets, command)',
default: '',
})
.option('custom-job', {
alias: 'customJob',
type: 'string',
description:
'Custom job instead of the standard build automation (yaml format with keys image, secrets, command)',
default: '',
})
.example(
'game-ci orchestrate --target-platform StandaloneLinux64 --provider-strategy aws',
'Build on AWS using the orchestrator',
)
.example(
'game-ci orchestrate --target-platform StandaloneLinux64 --provider-strategy k8s --kube-config <base64>',
'Build on Kubernetes',
) as any;
},
handler: async (cliArguments) => {
try {
if (!cliArguments.targetPlatform) {
throw new Error('--target-platform is required for orchestrate builds. Run game-ci orchestrate --help.');
}
mapCliArgumentsToInput(cliArguments);
const buildParameters = await BuildParameters.create();
const baseImage = new ImageTag(buildParameters);
core.info(`Orchestrating build via ${buildParameters.providerStrategy}...`);
core.info(`Target platform: ${buildParameters.targetPlatform}`);
core.info(`Unity version: ${buildParameters.editorVersion}`);
core.info(`Build GUID: ${buildParameters.buildGuid}`);
const result = await Orchestrator.run(buildParameters, baseImage.toString());
core.info(`\nOrchestrated build completed.`);
if (result?.BuildResults) {
core.info(`Results: ${result.BuildResults}`);
} else {
core.warning('Build completed but no build results were returned.');
}
} catch (error: any) {
core.setFailed(`Orchestrated build failed: ${error.message}`);
throw error;
}
},
};
export default orchestrateCommand;

View File

@@ -0,0 +1,84 @@
import type { CommandModule } from 'yargs';
import * as core from '@actions/core';
import fs from 'node:fs';
import path from 'node:path';
import UnityVersioning from '../../model/unity-versioning';
const statusCommand: CommandModule = {
command: 'status',
describe: 'Show build status and workspace info',
builder: (yargs) => {
return yargs.option('project-path', {
alias: 'projectPath',
type: 'string',
description: 'Path to the Unity project',
default: '.',
});
},
handler: async (cliArguments) => {
const projectPath = (cliArguments.projectPath as string) || '.';
core.info('game-ci Workspace Status');
core.info('========================\n');
// Project detection
const projectVersionPath = path.join(projectPath, 'ProjectSettings', 'ProjectVersion.txt');
const hasProject = fs.existsSync(projectVersionPath);
core.info(`Project Path: ${path.resolve(projectPath)}`);
core.info(`Unity Project Found: ${hasProject ? 'Yes' : 'No'}`);
if (hasProject) {
try {
const unityVersion = UnityVersioning.determineUnityVersion(projectPath, 'auto');
core.info(`Unity Version: ${unityVersion}`);
} catch {
core.info(`Unity Version: Unable to detect`);
}
// Library folder status
const libraryPath = path.join(projectPath, 'Library');
if (fs.existsSync(libraryPath)) {
const stats = fs.statSync(libraryPath);
core.info(`Library Cache: Present (modified ${stats.mtime.toISOString()})`);
} else {
core.info(`Library Cache: Not present (clean build required)`);
}
// Build output detection
const buildsPath = path.join(projectPath, '..', 'build');
if (fs.existsSync(buildsPath)) {
const builds = fs.readdirSync(buildsPath);
if (builds.length > 0) {
core.info(`\nBuild Outputs (${buildsPath}):`);
for (const build of builds) {
const buildPath = path.join(buildsPath, build);
const buildStats = fs.statSync(buildPath);
core.info(` - ${build} (${buildStats.isDirectory() ? 'dir' : 'file'}, ${buildStats.mtime.toISOString()})`);
}
}
}
}
// Environment
core.info('\nEnvironment:');
core.info(` Platform: ${process.platform}`);
core.info(` Node.js: ${process.version}`);
core.info(` UNITY_SERIAL: ${process.env.UNITY_SERIAL ? 'Set' : 'Not set'}`);
core.info(` UNITY_LICENSE: ${process.env.UNITY_LICENSE ? 'Set' : 'Not set'}`);
core.info(` UNITY_EMAIL: ${process.env.UNITY_EMAIL ? 'Set' : 'Not set'}`);
core.info(` UNITY_PASSWORD: ${process.env.UNITY_PASSWORD ? 'Set' : 'Not set'}`);
// Docker availability
core.info(`\nDocker: Checking...`);
try {
const { execSync } = await import('node:child_process');
const dockerVersion = execSync('docker --version', { encoding: 'utf8' }).trim();
core.info(` ${dockerVersion}`);
} catch {
core.info(` Docker not found or not accessible`);
}
},
};
export default statusCommand;

387
src/cli/commands/update.ts Normal file
View File

@@ -0,0 +1,387 @@
import type { CommandModule } from 'yargs';
import * as core from '@actions/core';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import https from 'node:https';
import http from 'node:http';
import { execFileSync } from 'node:child_process';
const REPO = 'game-ci/unity-builder';
interface GitHubRelease {
// eslint-disable-next-line camelcase
tag_name: string;
assets: Array<{
name: string;
// eslint-disable-next-line camelcase
browser_download_url: string;
size: number;
}>;
}
interface UpdateArguments {
force?: boolean;
version?: string;
}
/**
* Fetches JSON from a URL via HTTPS, following redirects.
*/
function fetchJson(url: string): Promise<any> {
return new Promise((resolve, reject) => {
const get = (targetUrl: string, redirectCount: number) => {
if (redirectCount > 5) {
reject(new Error('Too many redirects'));
return;
}
https
.get(
targetUrl,
{
headers: { 'User-Agent': 'game-ci-cli', Accept: 'application/json' },
},
(response) => {
if (
response.statusCode &&
response.statusCode >= 300 &&
response.statusCode < 400 &&
response.headers.location
) {
get(response.headers.location, redirectCount + 1);
return;
}
if (response.statusCode !== 200) {
reject(new Error(`HTTP ${response.statusCode} from ${targetUrl}`));
return;
}
let data = '';
response.on('data', (chunk) => (data += chunk));
response.on('end', () => {
try {
resolve(JSON.parse(data));
} catch {
reject(new Error('Invalid JSON response'));
}
});
},
)
.on('error', reject);
};
get(url, 0);
});
}
/**
* Downloads a file from a URL, following redirects. Returns the file content as a Buffer.
*/
function downloadFile(url: string): Promise<Buffer> {
return new Promise((resolve, reject) => {
const get = (targetUrl: string, redirectCount: number) => {
if (redirectCount > 10) {
reject(new Error('Too many redirects'));
return;
}
const protocol = targetUrl.startsWith('https') ? https : http;
protocol
.get(targetUrl, { headers: { 'User-Agent': 'game-ci-cli' } }, (response) => {
if (
response.statusCode &&
response.statusCode >= 300 &&
response.statusCode < 400 &&
response.headers.location
) {
get(response.headers.location, redirectCount + 1);
return;
}
if (response.statusCode !== 200) {
reject(new Error(`HTTP ${response.statusCode} downloading ${targetUrl}`));
return;
}
const chunks: Buffer[] = [];
response.on('data', (chunk: Buffer) => chunks.push(chunk));
response.on('end', () => resolve(Buffer.concat(chunks)));
})
.on('error', reject);
};
get(url, 0);
});
}
/**
* Gets the current version from package.json or the compiled binary.
*/
function getCurrentVersion(): string {
// Try reading from package.json at various relative locations
const candidates = [
path.join(__dirname, '..', '..', '..', 'package.json'),
path.join(__dirname, '..', '..', 'package.json'),
path.join(process.cwd(), 'package.json'),
];
for (const candidate of candidates) {
if (fs.existsSync(candidate)) {
try {
const packageData = JSON.parse(fs.readFileSync(candidate, 'utf8'));
if (packageData.version) {
return packageData.version;
}
} catch {
// Continue to next candidate
}
}
}
return 'unknown';
}
/**
* Determines the correct asset name for the current platform/architecture.
*/
function getAssetName(): string {
const platform = process.platform;
const arch = process.arch;
let osPart: string;
switch (platform) {
case 'linux':
osPart = 'linux';
break;
case 'darwin':
osPart = 'macos';
break;
case 'win32':
osPart = 'windows';
break;
default:
throw new Error(`Unsupported platform: ${platform}`);
}
let archPart: string;
switch (arch) {
case 'x64':
archPart = 'x64';
break;
case 'arm64':
archPart = 'arm64';
break;
default:
throw new Error(`Unsupported architecture: ${arch}`);
}
const assetBaseName = `game-ci-${osPart}-${archPart}`;
return osPart === 'windows' ? `${assetBaseName}.exe` : assetBaseName;
}
/**
* Determines the path to the currently running executable.
* For standalone binaries (pkg), process.execPath points to the binary itself.
* For Node.js execution, we return undefined since self-update does not apply.
*/
function getExecutablePath(): string | undefined {
// When running as a pkg binary, process.pkg is defined
if ((process as any).pkg) {
return process.execPath;
}
// When running via Node.js, check if there is a standalone binary in the typical install location
const installDirectory = process.env.GAME_CI_INSTALL || path.join(os.homedir(), '.game-ci', 'bin');
const binaryName = process.platform === 'win32' ? 'game-ci.exe' : 'game-ci';
const installedPath = path.join(installDirectory, binaryName);
if (fs.existsSync(installedPath)) {
return installedPath;
}
return;
}
/**
* Strips leading 'v' from a version string and splits into numeric parts.
*/
function parseVersionParts(version: string): number[] {
return version
.replace(/^v/, '')
.split('.')
.map((part) => Number(part));
}
/**
* Compares two semver strings. Returns:
* -1 if a < b
* 0 if a == b
* 1 if a > b
*/
function compareSemver(a: string, b: string): number {
const partsA = parseVersionParts(a);
const partsB = parseVersionParts(b);
for (let index = 0; index < 3; index++) {
const x = partsA[index] || 0;
const y = partsB[index] || 0;
if (x < y) return -1;
if (x > y) return 1;
}
return 0;
}
const updateCommand: CommandModule<object, UpdateArguments> = {
command: 'update',
describe: 'Update game-ci to the latest version',
builder: (yargs) => {
return yargs
.option('force', {
alias: 'f',
type: 'boolean',
description: 'Force update even if already on latest version',
default: false,
})
.option('version', {
type: 'string',
description: 'Update to a specific version (e.g., v2.0.0)',
default: '',
})
.example('game-ci update', 'Update to the latest version')
.example('game-ci update --version v2.1.0', 'Update to a specific version')
.example('game-ci update --force', 'Force reinstall of the current version') as any;
},
handler: async (cliArguments) => {
try {
const currentVersion = getCurrentVersion();
core.info(`Current version: v${currentVersion}`);
core.info(`Platform: ${process.platform} ${process.arch}`);
core.info('');
// Fetch release info
let release: GitHubRelease;
const targetVersion = cliArguments.version as string;
if (targetVersion) {
const tag = targetVersion.startsWith('v') ? targetVersion : `v${targetVersion}`;
core.info(`Fetching release ${tag}...`);
release = await fetchJson(`https://api.github.com/repos/${REPO}/releases/tags/${tag}`);
} else {
core.info('Checking for updates...');
release = await fetchJson(`https://api.github.com/repos/${REPO}/releases/latest`);
}
const latestVersion = release.tag_name;
core.info(`Latest version: ${latestVersion}`);
core.info('');
// Compare versions
const comparison = compareSemver(currentVersion, latestVersion);
if (comparison >= 0 && !cliArguments.force) {
core.info('You are already on the latest version. Use --force to reinstall.');
return;
}
if (comparison > 0 && !targetVersion) {
core.info(`Current version (v${currentVersion}) is newer than latest release (${latestVersion}).`);
core.info('Use --force to downgrade, or --version to target a specific release.');
return;
}
// Find the correct asset
const assetName = getAssetName();
const asset = release.assets.find((a) => a.name === assetName);
if (!asset) {
const available = release.assets.map((a) => a.name).join(', ');
throw new Error(
`No binary found for ${process.platform}-${process.arch} (looking for ${assetName}).\nAvailable assets: ${available}`,
);
}
const sizeMb = (asset.size / (1024 * 1024)).toFixed(1);
core.info(`Downloading ${assetName} (${sizeMb} MB)...`);
// Download the new binary
const binaryData = await downloadFile(asset.browser_download_url);
// Determine where to write the updated binary
const executablePath = getExecutablePath();
if (!executablePath) {
core.info('');
core.info('game-ci is running via Node.js (not as a standalone binary).');
core.info('To update the npm package, run:');
core.info(' npm install -g unity-builder@latest');
core.info('');
core.info('To install the standalone binary instead:');
core.info(' curl -fsSL https://raw.githubusercontent.com/game-ci/unity-builder/main/install.sh | sh');
return;
}
// Write the new binary.
// On Windows, we cannot overwrite a running executable directly.
// Write to a temporary file, then rename.
const temporaryPath = `${executablePath}.update`;
const backupPath = `${executablePath}.backup`;
fs.writeFileSync(temporaryPath, binaryData);
if (process.platform !== 'win32') {
fs.chmodSync(temporaryPath, 0o755);
}
// Verify the downloaded binary
try {
const output = execFileSync(temporaryPath, ['version'], { encoding: 'utf8', timeout: 10_000 });
core.info(`Verified new binary: ${output.trim().split('\n')[0]}`);
} catch (verifyError: any) {
fs.unlinkSync(temporaryPath);
throw new Error(`Downloaded binary failed verification: ${verifyError.message}`);
}
// Replace the current binary
try {
// Backup current
if (fs.existsSync(backupPath)) {
fs.unlinkSync(backupPath);
}
fs.renameSync(executablePath, backupPath);
fs.renameSync(temporaryPath, executablePath);
// Clean up backup
try {
fs.unlinkSync(backupPath);
} catch {
// On Windows the backup may be locked; that is fine
}
} catch (replaceError: any) {
// Attempt to restore from backup
if (fs.existsSync(backupPath) && !fs.existsSync(executablePath)) {
fs.renameSync(backupPath, executablePath);
}
// Clean up temporary file
if (fs.existsSync(temporaryPath)) {
fs.unlinkSync(temporaryPath);
}
throw new Error(`Failed to replace binary: ${replaceError.message}`);
}
core.info('');
core.info(`Successfully updated game-ci to ${latestVersion}`);
} catch (error: any) {
core.error(`Update failed: ${error.message}`);
throw error;
}
},
};
export default updateCommand;

View File

@@ -0,0 +1,37 @@
import type { CommandModule } from 'yargs';
import * as core from '@actions/core';
import fs from 'node:fs';
import path from 'node:path';
const versionCommand: CommandModule = {
command: 'version',
describe: 'Show version info',
builder: {},
handler: async () => {
try {
// Read version from package.json
let packageJsonPath = path.join(__dirname, '..', '..', '..', 'package.json');
if (!fs.existsSync(packageJsonPath)) {
packageJsonPath = path.join(__dirname, '..', '..', 'package.json');
}
if (!fs.existsSync(packageJsonPath)) {
packageJsonPath = path.join(process.cwd(), 'package.json');
}
if (fs.existsSync(packageJsonPath)) {
const packageData = JSON.parse(fs.readFileSync(packageJsonPath, 'utf8'));
core.info(`game-ci (unity-builder) v${packageData.version}`);
core.info(`Node.js ${process.version}`);
core.info(`Platform: ${process.platform} ${process.arch}`);
} else {
core.info('game-ci (unity-builder)');
core.info('Version information unavailable');
}
} catch (error: any) {
core.info('game-ci (unity-builder)');
core.error(`Could not read version: ${error.message}`);
}
},
};
export default versionCommand;

106
src/cli/input-mapper.ts Normal file
View File

@@ -0,0 +1,106 @@
import { Cli } from '../model/cli/cli';
import GitHub from '../model/github';
/**
* Maps CLI arguments (kebab-case flags) to the Input/OrchestratorOptions
* interface used by the action. This bridges the gap between user-friendly
* CLI flags and the camelCase environment/input system unity-builder expects.
*
* The existing Input class already queries Cli.options, environment variables,
* and GitHub Action inputs in priority order. We populate Cli.options so that
* the rest of the codebase works unchanged.
*/
export interface CliArguments {
targetPlatform?: string;
unityVersion?: string;
projectPath?: string;
buildProfile?: string;
buildName?: string;
buildsPath?: string;
buildMethod?: string;
customParameters?: string;
versioning?: string;
version?: string;
customImage?: string;
manualExit?: boolean;
enableGpu?: boolean;
androidVersionCode?: string;
androidExportType?: string;
androidKeystoreName?: string;
androidKeystoreBase64?: string;
androidKeystorePass?: string;
androidKeyaliasName?: string;
androidKeyaliasPass?: string;
androidTargetSdkVersion?: string;
androidSymbolType?: string;
dockerCpuLimit?: string;
dockerMemoryLimit?: string;
dockerIsolationMode?: string;
dockerWorkspacePath?: string;
containerRegistryRepository?: string;
containerRegistryImageVersion?: string;
runAsHostUser?: string;
chownFilesTo?: string;
sshAgent?: string;
sshPublicKeysDirectoryPath?: string;
gitPrivateToken?: string;
providerStrategy?: string;
awsStackName?: string;
kubeConfig?: string;
kubeVolume?: string;
kubeVolumeSize?: string;
kubeStorageClass?: string;
containerCpu?: string;
containerMemory?: string;
cacheKey?: string;
watchToEnd?: string;
allowDirtyBuild?: boolean;
skipActivation?: string;
cloneDepth?: string;
readInputFromOverrideList?: string;
readInputOverrideCommand?: string;
postBuildSteps?: string;
preBuildSteps?: string;
customJob?: string;
unityLicensingServer?: string;
cacheUnityInstallationOnMac?: boolean;
unityHubVersionOnMac?: string;
mode?: string;
[key: string]: unknown;
}
/**
* Converts kebab-case CLI flags to camelCase keys matching the Input class
* property names, then injects them into Cli.options so the existing
* Input.getInput() / OrchestratorOptions.getInput() chain picks them up.
*/
export function mapCliArgumentsToInput(cliArguments: CliArguments): void {
// Disable GitHub Actions input reading when in CLI mode
GitHub.githubInputEnabled = false;
// The existing Cli.options mechanism is used by Input.getInput() to query
// CLI-provided values. We set it directly.
const mapped: Record<string, unknown> = {};
for (const [key, value] of Object.entries(cliArguments)) {
if (value !== undefined && key !== '_' && key !== '$0') {
mapped[key] = typeof value === 'boolean' ? String(value) : value;
}
}
// Ensure mode is set so Cli.isCliMode returns true
if (!mapped['mode']) {
mapped['mode'] = 'cli';
}
Cli.options = mapped;
}

View File

@@ -0,0 +1,606 @@
/**
* Integration wiring tests for enterprise features in index.ts
*
* These tests verify the conditional gating logic in runMain():
* - Each enterprise feature is only invoked when its gate condition is met
* - Services are NOT called when their feature is disabled (the default)
* - The order of operations is correct (restore before build, save after build)
*/
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
import { BuildParameters } from './model';
// ---------------------------------------------------------------------------
// Service mocks — must be declared before importing index.ts (jest hoists them)
// ---------------------------------------------------------------------------
const mockChildWorkspaceService = {
buildConfig: jest.fn().mockReturnValue({ enabled: true, workspaceName: 'Test' }),
initializeWorkspace: jest.fn().mockReturnValue(false),
getWorkspaceSize: jest.fn().mockReturnValue('0 B'),
saveWorkspace: jest.fn(),
};
const mockSubmoduleProfileService = {
createInitPlan: jest.fn().mockResolvedValue([]),
execute: jest.fn().mockResolvedValue(''),
};
const mockLfsAgentService = {
configure: jest.fn().mockResolvedValue(''),
};
const mockLocalCacheService = {
resolveCacheRoot: jest.fn().mockReturnValue('/cache'),
generateCacheKey: jest.fn().mockReturnValue('key-1'),
restoreLfsCache: jest.fn().mockResolvedValue(true),
restoreLibraryCache: jest.fn().mockResolvedValue(true),
saveLibraryCache: jest.fn().mockResolvedValue(''),
saveLfsCache: jest.fn().mockResolvedValue(''),
};
const mockGitHooksService = {
installHooks: jest.fn().mockResolvedValue(''),
configureSkipList: jest.fn().mockReturnValue({ LEFTHOOK_EXCLUDE: 'pre-commit' }),
};
// Mock the dynamic import() targets — jest.mock with factory functions.
// The services are imported dynamically via `await import(...)` in index.ts,
// so we mock the module path and return the mock objects as named exports.
jest.mock('./model/orchestrator/services/cache/child-workspace-service', () => ({
ChildWorkspaceService: mockChildWorkspaceService,
}));
jest.mock('./model/orchestrator/services/submodule/submodule-profile-service', () => ({
SubmoduleProfileService: mockSubmoduleProfileService,
}));
jest.mock('./model/orchestrator/services/lfs/lfs-agent-service', () => ({
LfsAgentService: mockLfsAgentService,
}));
jest.mock('./model/orchestrator/services/cache/local-cache-service', () => ({
LocalCacheService: mockLocalCacheService,
}));
jest.mock('./model/orchestrator/services/hooks/git-hooks-service', () => ({
GitHooksService: mockGitHooksService,
}));
// Mock all non-enterprise dependencies to isolate the wiring logic
jest.mock('@actions/core');
jest.mock('./model', () => ({
Action: {
checkCompatibility: jest.fn(),
workspace: '/workspace',
actionFolder: '/action',
},
BuildParameters: {
create: jest.fn(),
},
Cache: {
verify: jest.fn(),
},
Orchestrator: {
run: jest.fn().mockResolvedValue(''),
},
Docker: {
run: jest.fn().mockResolvedValue(0),
},
ImageTag: jest.fn().mockImplementation(() => ({
toString: () => 'mock-image:latest',
})),
Output: {
setBuildVersion: jest.fn().mockResolvedValue(''),
setAndroidVersionCode: jest.fn().mockResolvedValue(''),
setEngineExitCode: jest.fn().mockResolvedValue(''),
},
}));
jest.mock('./model/cli/cli', () => ({
Cli: {
InitCliMode: jest.fn().mockReturnValue(false),
},
}));
jest.mock('./model/mac-builder', () => ({
__esModule: true,
default: {
run: jest.fn().mockResolvedValue(0),
},
}));
jest.mock('./model/platform-setup', () => ({
__esModule: true,
default: {
setup: jest.fn().mockResolvedValue(''),
},
}));
const mockedBuildParametersCreate = BuildParameters.create as jest.Mock;
interface EnterpriseBuildParametersOverrides {
providerStrategy?: string;
childWorkspacesEnabled?: boolean;
childWorkspaceName?: string;
childWorkspaceCacheRoot?: string;
childWorkspacePreserveGit?: boolean;
childWorkspaceSeparateLibrary?: boolean;
submoduleProfilePath?: string;
submoduleVariantPath?: string;
submoduleToken?: string;
gitPrivateToken?: string;
lfsTransferAgent?: string;
lfsTransferAgentArgs?: string;
lfsStoragePaths?: string;
localCacheEnabled?: boolean;
localCacheRoot?: string;
localCacheLibrary?: boolean;
localCacheLfs?: boolean;
gitHooksEnabled?: boolean;
gitHooksSkipList?: string;
gitHooksRunBeforeBuild?: string;
}
function createMockBuildParameters(overrides: EnterpriseBuildParametersOverrides = {}) {
return {
// Required base properties
providerStrategy: 'local',
targetPlatform: 'StandaloneLinux64',
editorVersion: '2021.3.1f1',
buildVersion: '1.0.0',
androidVersionCode: '1',
projectPath: '.',
branch: 'main',
runnerTempPath: '/tmp',
// Enterprise features - all disabled by default
childWorkspacesEnabled: false,
childWorkspaceName: '',
childWorkspaceCacheRoot: '',
childWorkspacePreserveGit: true,
childWorkspaceSeparateLibrary: true,
submoduleProfilePath: '',
submoduleVariantPath: '',
submoduleToken: '',
gitPrivateToken: '',
lfsTransferAgent: '',
lfsTransferAgentArgs: '',
lfsStoragePaths: '',
localCacheEnabled: false,
localCacheRoot: '',
localCacheLibrary: true,
localCacheLfs: false,
gitHooksEnabled: false,
gitHooksSkipList: '',
gitHooksRunBeforeBuild: '',
...overrides,
};
}
/**
* The entry point (runMain) is invoked by importing index.ts.
* Since it calls `runMain()` at module scope, we need to re-import it
* for each test. jest.isolateModules() handles this.
*/
async function runIndex(overrides: EnterpriseBuildParametersOverrides = {}): Promise<void> {
mockedBuildParametersCreate.mockResolvedValue(createMockBuildParameters(overrides));
return new Promise<void>((resolve) => {
jest.isolateModules(() => {
require('./index');
// runMain() is async; give it a tick to complete
// We use setImmediate to ensure all microtasks from the dynamic imports resolve
});
// Allow all promises and microtasks to settle
setTimeout(resolve, 100);
});
}
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
describe('index.ts enterprise feature wiring', () => {
const originalPlatform = process.platform;
const originalEnvironment = { ...process.env };
beforeEach(() => {
jest.clearAllMocks();
process.env.GITHUB_WORKSPACE = '/workspace';
// Force linux platform so Docker.run is used (not MacBuilder)
Object.defineProperty(process, 'platform', { value: 'linux' });
});
afterEach(() => {
Object.defineProperty(process, 'platform', { value: originalPlatform });
process.env = { ...originalEnvironment };
});
// -----------------------------------------------------------------------
// GitHooksService gating
// -----------------------------------------------------------------------
describe('GitHooksService gating', () => {
it('should NOT call GitHooksService when gitHooksEnabled is false (default)', async () => {
await runIndex({ gitHooksEnabled: false });
expect(mockGitHooksService.installHooks).not.toHaveBeenCalled();
expect(mockGitHooksService.configureSkipList).not.toHaveBeenCalled();
});
it('should call installHooks when gitHooksEnabled is true', async () => {
await runIndex({ gitHooksEnabled: true });
expect(mockGitHooksService.installHooks).toHaveBeenCalledWith('/workspace');
});
it('should call configureSkipList when gitHooksEnabled and gitHooksSkipList is set', async () => {
await runIndex({
gitHooksEnabled: true,
gitHooksSkipList: 'pre-commit,pre-push',
});
expect(mockGitHooksService.configureSkipList).toHaveBeenCalledWith(['pre-commit', 'pre-push']);
});
it('should NOT call configureSkipList when gitHooksSkipList is empty', async () => {
await runIndex({
gitHooksEnabled: true,
gitHooksSkipList: '',
});
expect(mockGitHooksService.installHooks).toHaveBeenCalled();
expect(mockGitHooksService.configureSkipList).not.toHaveBeenCalled();
});
});
// -----------------------------------------------------------------------
// LocalCacheService gating
// -----------------------------------------------------------------------
describe('LocalCacheService gating', () => {
it('should NOT call LocalCacheService when localCacheEnabled is false (default)', async () => {
await runIndex({ localCacheEnabled: false });
expect(mockLocalCacheService.resolveCacheRoot).not.toHaveBeenCalled();
expect(mockLocalCacheService.generateCacheKey).not.toHaveBeenCalled();
expect(mockLocalCacheService.restoreLibraryCache).not.toHaveBeenCalled();
expect(mockLocalCacheService.restoreLfsCache).not.toHaveBeenCalled();
expect(mockLocalCacheService.saveLibraryCache).not.toHaveBeenCalled();
expect(mockLocalCacheService.saveLfsCache).not.toHaveBeenCalled();
});
it('should call restore and save operations when localCacheEnabled is true', async () => {
await runIndex({
localCacheEnabled: true,
localCacheLibrary: true,
localCacheLfs: true,
});
expect(mockLocalCacheService.resolveCacheRoot).toHaveBeenCalled();
expect(mockLocalCacheService.generateCacheKey).toHaveBeenCalled();
expect(mockLocalCacheService.restoreLibraryCache).toHaveBeenCalled();
expect(mockLocalCacheService.restoreLfsCache).toHaveBeenCalled();
expect(mockLocalCacheService.saveLibraryCache).toHaveBeenCalled();
expect(mockLocalCacheService.saveLfsCache).toHaveBeenCalled();
});
it('should only cache Library when localCacheLibrary is true and localCacheLfs is false', async () => {
await runIndex({
localCacheEnabled: true,
localCacheLibrary: true,
localCacheLfs: false,
});
expect(mockLocalCacheService.restoreLibraryCache).toHaveBeenCalled();
expect(mockLocalCacheService.restoreLfsCache).not.toHaveBeenCalled();
expect(mockLocalCacheService.saveLibraryCache).toHaveBeenCalled();
expect(mockLocalCacheService.saveLfsCache).not.toHaveBeenCalled();
});
it('should only cache LFS when localCacheLfs is true and localCacheLibrary is false', async () => {
await runIndex({
localCacheEnabled: true,
localCacheLibrary: false,
localCacheLfs: true,
});
expect(mockLocalCacheService.restoreLibraryCache).not.toHaveBeenCalled();
expect(mockLocalCacheService.restoreLfsCache).toHaveBeenCalled();
expect(mockLocalCacheService.saveLibraryCache).not.toHaveBeenCalled();
expect(mockLocalCacheService.saveLfsCache).toHaveBeenCalled();
});
});
// -----------------------------------------------------------------------
// ChildWorkspaceService gating
// -----------------------------------------------------------------------
describe('ChildWorkspaceService gating', () => {
it('should NOT call ChildWorkspaceService when childWorkspacesEnabled is false (default)', async () => {
await runIndex({ childWorkspacesEnabled: false });
expect(mockChildWorkspaceService.buildConfig).not.toHaveBeenCalled();
expect(mockChildWorkspaceService.initializeWorkspace).not.toHaveBeenCalled();
expect(mockChildWorkspaceService.saveWorkspace).not.toHaveBeenCalled();
});
it('should NOT call ChildWorkspaceService when childWorkspacesEnabled is true but childWorkspaceName is empty', async () => {
await runIndex({
childWorkspacesEnabled: true,
childWorkspaceName: '',
});
expect(mockChildWorkspaceService.buildConfig).not.toHaveBeenCalled();
});
it('should call buildConfig, initializeWorkspace, and saveWorkspace when enabled with a name', async () => {
mockChildWorkspaceService.buildConfig.mockReturnValue({ enabled: true, workspaceName: 'TurnOfWar' });
await runIndex({
childWorkspacesEnabled: true,
childWorkspaceName: 'TurnOfWar',
childWorkspaceCacheRoot: '/cache/workspaces',
});
expect(mockChildWorkspaceService.buildConfig).toHaveBeenCalledWith(
expect.objectContaining({
childWorkspacesEnabled: true,
childWorkspaceName: 'TurnOfWar',
}),
);
expect(mockChildWorkspaceService.initializeWorkspace).toHaveBeenCalled();
expect(mockChildWorkspaceService.getWorkspaceSize).toHaveBeenCalled();
expect(mockChildWorkspaceService.saveWorkspace).toHaveBeenCalled();
});
});
// -----------------------------------------------------------------------
// SubmoduleProfileService gating
// -----------------------------------------------------------------------
describe('SubmoduleProfileService gating', () => {
it('should NOT call SubmoduleProfileService when submoduleProfilePath is empty (default)', async () => {
await runIndex({ submoduleProfilePath: '' });
expect(mockSubmoduleProfileService.createInitPlan).not.toHaveBeenCalled();
expect(mockSubmoduleProfileService.execute).not.toHaveBeenCalled();
});
it('should call createInitPlan and execute when submoduleProfilePath is set', async () => {
await runIndex({
submoduleProfilePath: '/path/to/profile.yml',
submoduleVariantPath: '',
submoduleToken: 'my-token',
});
expect(mockSubmoduleProfileService.createInitPlan).toHaveBeenCalledWith('/path/to/profile.yml', '', '/workspace');
expect(mockSubmoduleProfileService.execute).toHaveBeenCalled();
});
it('should pass variant path when provided', async () => {
await runIndex({
submoduleProfilePath: '/path/to/profile.yml',
submoduleVariantPath: '/path/to/variant.yml',
});
expect(mockSubmoduleProfileService.createInitPlan).toHaveBeenCalledWith(
'/path/to/profile.yml',
'/path/to/variant.yml',
'/workspace',
);
});
it('should use submoduleToken for auth, falling back to gitPrivateToken', async () => {
await runIndex({
submoduleProfilePath: '/path/to/profile.yml',
submoduleToken: '',
gitPrivateToken: 'fallback-token',
});
expect(mockSubmoduleProfileService.execute).toHaveBeenCalledWith(
expect.anything(),
'/workspace',
'fallback-token',
);
});
it('should prefer submoduleToken over gitPrivateToken', async () => {
await runIndex({
submoduleProfilePath: '/path/to/profile.yml',
submoduleToken: 'specific-token',
gitPrivateToken: 'fallback-token',
});
expect(mockSubmoduleProfileService.execute).toHaveBeenCalledWith(
expect.anything(),
'/workspace',
'specific-token',
);
});
});
// -----------------------------------------------------------------------
// LfsAgentService gating
// -----------------------------------------------------------------------
describe('LfsAgentService gating', () => {
it('should NOT call LfsAgentService when lfsTransferAgent is empty (default)', async () => {
await runIndex({ lfsTransferAgent: '' });
expect(mockLfsAgentService.configure).not.toHaveBeenCalled();
});
it('should call configure when lfsTransferAgent is set', async () => {
await runIndex({
lfsTransferAgent: '/tools/elastic-git-storage',
lfsTransferAgentArgs: '--verbose',
lfsStoragePaths: '/path/a;/path/b',
});
expect(mockLfsAgentService.configure).toHaveBeenCalledWith(
'/tools/elastic-git-storage',
'--verbose',
['/path/a', '/path/b'],
'/workspace',
);
});
it('should pass empty array when lfsStoragePaths is empty', async () => {
await runIndex({
lfsTransferAgent: '/tools/agent',
lfsStoragePaths: '',
});
expect(mockLfsAgentService.configure).toHaveBeenCalledWith('/tools/agent', '', [], '/workspace');
});
});
// -----------------------------------------------------------------------
// Order of operations (restore before build, save after build)
// -----------------------------------------------------------------------
describe('order of operations', () => {
it('should execute restore operations before build and save operations after build', async () => {
const callOrder: string[] = [];
// Track call order for each relevant operation
mockChildWorkspaceService.buildConfig.mockReturnValue({ enabled: true, workspaceName: 'Test' });
mockChildWorkspaceService.initializeWorkspace.mockImplementation(() => {
callOrder.push('child-workspace-restore');
return false;
});
mockChildWorkspaceService.getWorkspaceSize.mockImplementation(() => {
callOrder.push('child-workspace-size');
return '0 B';
});
mockSubmoduleProfileService.createInitPlan.mockImplementation(async () => {
callOrder.push('submodule-profile-plan');
return [];
});
mockSubmoduleProfileService.execute.mockImplementation(async () => {
callOrder.push('submodule-profile-execute');
});
mockLfsAgentService.configure.mockImplementation(async () => {
callOrder.push('lfs-agent-configure');
});
mockLocalCacheService.resolveCacheRoot.mockImplementation(() => {
callOrder.push('local-cache-resolve');
return '/cache';
});
mockLocalCacheService.generateCacheKey.mockImplementation(() => {
callOrder.push('local-cache-keygen');
return 'key-1';
});
mockLocalCacheService.restoreLfsCache.mockImplementation(async () => {
callOrder.push('local-cache-restore-lfs');
return true;
});
mockLocalCacheService.restoreLibraryCache.mockImplementation(async () => {
callOrder.push('local-cache-restore-library');
return true;
});
mockGitHooksService.installHooks.mockImplementation(async () => {
callOrder.push('git-hooks-install');
});
mockLocalCacheService.saveLibraryCache.mockImplementation(async () => {
callOrder.push('local-cache-save-library');
});
mockLocalCacheService.saveLfsCache.mockImplementation(async () => {
callOrder.push('local-cache-save-lfs');
});
mockChildWorkspaceService.saveWorkspace.mockImplementation(() => {
callOrder.push('child-workspace-save');
});
await runIndex({
childWorkspacesEnabled: true,
childWorkspaceName: 'TurnOfWar',
submoduleProfilePath: '/profile.yml',
lfsTransferAgent: '/tools/agent',
localCacheEnabled: true,
localCacheLfs: true,
localCacheLibrary: true,
gitHooksEnabled: true,
});
// Verify restore operations happen before save operations.
// The expected order from index.ts is:
// 1. Child workspace restore
// 2. Submodule profile init
// 3. LFS agent configure
// 4. Local cache restore (LFS then Library)
// 5. Git hooks install
// 6. [BUILD happens here - Docker.run or MacBuilder.run]
// 7. Local cache save (Library then LFS)
// 8. Child workspace save
const restoreOps = [
'child-workspace-restore',
'submodule-profile-plan',
'submodule-profile-execute',
'lfs-agent-configure',
'local-cache-restore-lfs',
'local-cache-restore-library',
'git-hooks-install',
];
const saveOps = ['local-cache-save-library', 'local-cache-save-lfs', 'child-workspace-save'];
// All restore ops should appear before all save ops
for (const restoreOp of restoreOps) {
if (!callOrder.includes(restoreOp)) continue; // Skip if the operation wasn't called
for (const saveOp of saveOps) {
if (!callOrder.includes(saveOp)) continue;
expect(callOrder.indexOf(restoreOp)).toBeLessThan(callOrder.indexOf(saveOp));
}
}
// Child workspace save should be last
if (callOrder.includes('child-workspace-save') && callOrder.includes('local-cache-save-lfs')) {
expect(callOrder.indexOf('local-cache-save-lfs')).toBeLessThan(callOrder.indexOf('child-workspace-save'));
}
});
});
// -----------------------------------------------------------------------
// Non-local provider strategy
// -----------------------------------------------------------------------
describe('non-local provider strategy', () => {
it('should skip all enterprise features when providerStrategy is not local', async () => {
await runIndex({
providerStrategy: 'aws',
childWorkspacesEnabled: true,
childWorkspaceName: 'Test',
submoduleProfilePath: '/profile.yml',
lfsTransferAgent: '/tools/agent',
localCacheEnabled: true,
gitHooksEnabled: true,
});
// None of the enterprise services should be called because
// they are inside the `if (providerStrategy === 'local')` block
expect(mockChildWorkspaceService.buildConfig).not.toHaveBeenCalled();
expect(mockSubmoduleProfileService.createInitPlan).not.toHaveBeenCalled();
expect(mockLfsAgentService.configure).not.toHaveBeenCalled();
expect(mockLocalCacheService.resolveCacheRoot).not.toHaveBeenCalled();
expect(mockGitHooksService.installHooks).not.toHaveBeenCalled();
});
});
});

View File

@@ -1,8 +1,18 @@
import * as core from '@actions/core';
import { Action, BuildParameters, Cache, CloudRunner, Docker, ImageTag, Output } from './model';
import path from 'node:path';
import { Action, BuildParameters, Cache, Orchestrator, Docker, ImageTag, Output } from './model';
import { Cli } from './model/cli/cli';
import MacBuilder from './model/mac-builder';
import PlatformSetup from './model/platform-setup';
import { BuildReliabilityService } from './model/orchestrator/services/reliability';
import { TestWorkflowService } from './model/orchestrator/services/test-workflow';
import { HotRunnerService } from './model/orchestrator/services/hot-runner';
import { HotRunnerConfig } from './model/orchestrator/services/hot-runner/hot-runner-types';
import { OutputService } from './model/orchestrator/services/output/output-service';
import { OutputTypeRegistry } from './model/orchestrator/services/output/output-type-registry';
import { ArtifactUploadHandler } from './model/orchestrator/services/output/artifact-upload-handler';
import { IncrementalSyncService } from './model/orchestrator/services/sync';
import { SyncStrategy } from './model/orchestrator/services/sync/sync-state';
async function runMain() {
try {
@@ -14,15 +24,191 @@ async function runMain() {
Action.checkCompatibility();
Cache.verify();
// Always configure git environment for CI reliability
BuildReliabilityService.configureGitEnvironment();
const { workspace, actionFolder } = Action;
const buildParameters = await BuildParameters.create();
// If a test suite path is provided, use the test workflow engine
// instead of the standard build execution path
if (buildParameters.testSuitePath) {
core.info('[TestWorkflow] Test suite path detected, using test workflow engine');
const results = await TestWorkflowService.executeTestSuite(buildParameters.testSuitePath, buildParameters);
const totalFailed = results.reduce((sum, r) => sum + r.failed, 0);
if (totalFailed > 0) {
core.setFailed(`Test workflow completed with ${totalFailed} failure(s)`);
} else {
core.info('[TestWorkflow] All test runs passed');
}
return;
}
const baseImage = new ImageTag(buildParameters);
// Pre-build reliability checks
if (buildParameters.gitIntegrityCheck) {
core.info('Running git integrity checks...');
const isHealthy = BuildReliabilityService.checkGitIntegrity(workspace);
BuildReliabilityService.cleanStaleLockFiles(workspace);
BuildReliabilityService.validateSubmoduleBackingStores(workspace);
if (buildParameters.cleanReservedFilenames) {
BuildReliabilityService.cleanReservedFilenames(buildParameters.projectPath);
}
if (!isHealthy && buildParameters.gitAutoRecover) {
core.info('Git corruption detected, attempting automatic recovery...');
const recovered = BuildReliabilityService.recoverCorruptedRepo(workspace);
if (!recovered) {
core.warning('Automatic recovery failed. Build may encounter issues.');
}
}
} else if (buildParameters.cleanReservedFilenames) {
// cleanReservedFilenames can run independently of gitIntegrityCheck
BuildReliabilityService.cleanReservedFilenames(buildParameters.projectPath);
}
let exitCode = -1;
if (buildParameters.providerStrategy === 'local') {
// Hot runner path: attempt to use a persistent Unity editor instance
if (buildParameters.hotRunnerEnabled) {
core.info('[HotRunner] Hot runner mode enabled, attempting hot build...');
const hotRunnerConfig: HotRunnerConfig = {
enabled: true,
transport: buildParameters.hotRunnerTransport,
host: buildParameters.hotRunnerHost,
port: buildParameters.hotRunnerPort,
healthCheckInterval: buildParameters.hotRunnerHealthInterval,
maxIdleTime: buildParameters.hotRunnerMaxIdle,
maxJobsBeforeRecycle: 0, // no automatic recycle by job count
};
const hotRunnerService = new HotRunnerService();
try {
await hotRunnerService.initialize(hotRunnerConfig);
const result = await hotRunnerService.submitBuild(buildParameters, (output) => {
core.info(output);
});
exitCode = result.exitCode;
core.info(`[HotRunner] Build completed with exit code ${exitCode}`);
await hotRunnerService.shutdown();
} catch (hotRunnerError) {
await hotRunnerService.shutdown();
if (buildParameters.hotRunnerFallbackToCold) {
core.warning(
`[HotRunner] Hot runner failed: ${(hotRunnerError as Error).message}. Falling back to cold build.`,
);
exitCode = await runColdBuild(buildParameters, baseImage, workspace, actionFolder);
} else {
throw hotRunnerError;
}
}
} else if (buildParameters.providerStrategy === 'local') {
core.info('Building locally');
// Child workspace isolation - restore cached workspace before any other setup
let childWorkspaceConfig: any;
if (buildParameters.childWorkspacesEnabled && buildParameters.childWorkspaceName) {
const { ChildWorkspaceService } = await import('./model/orchestrator/services/cache/child-workspace-service');
const cacheRoot =
buildParameters.childWorkspaceCacheRoot ||
path.join(buildParameters.runnerTempPath || process.env.RUNNER_TEMP || '', 'game-ci-workspaces');
childWorkspaceConfig = ChildWorkspaceService.buildConfig({
childWorkspacesEnabled: buildParameters.childWorkspacesEnabled,
childWorkspaceName: buildParameters.childWorkspaceName,
childWorkspaceCacheRoot: cacheRoot,
childWorkspacePreserveGit: buildParameters.childWorkspacePreserveGit,
childWorkspaceSeparateLibrary: buildParameters.childWorkspaceSeparateLibrary,
});
const projectFullPath = path.join(workspace, buildParameters.projectPath);
const restored = ChildWorkspaceService.initializeWorkspace(projectFullPath, childWorkspaceConfig);
core.info(
`Child workspace "${buildParameters.childWorkspaceName}": ${
restored ? 'restored from cache' : 'starting fresh'
}`,
);
// Log workspace size for resource tracking
const size = ChildWorkspaceService.getWorkspaceSize(projectFullPath);
core.info(`Child workspace size after restore: ${size}`);
}
// Submodule profile initialization
if (buildParameters.submoduleProfilePath) {
const { SubmoduleProfileService } = await import(
'./model/orchestrator/services/submodule/submodule-profile-service'
);
core.info('Initializing submodules from profile...');
const plan = await SubmoduleProfileService.createInitPlan(
buildParameters.submoduleProfilePath,
buildParameters.submoduleVariantPath,
workspace,
);
await SubmoduleProfileService.execute(
plan,
workspace,
buildParameters.submoduleToken || buildParameters.gitPrivateToken,
);
}
// Configure custom LFS transfer agent
if (buildParameters.lfsTransferAgent) {
const { LfsAgentService } = await import('./model/orchestrator/services/lfs/lfs-agent-service');
core.info('Configuring custom LFS transfer agent...');
await LfsAgentService.configure(
buildParameters.lfsTransferAgent,
buildParameters.lfsTransferAgentArgs,
buildParameters.lfsStoragePaths ? buildParameters.lfsStoragePaths.split(';') : [],
workspace,
);
}
// Local build caching - restore
let cacheRoot = '';
let cacheKey = '';
if (buildParameters.localCacheEnabled) {
const { LocalCacheService } = await import('./model/orchestrator/services/cache/local-cache-service');
cacheRoot = LocalCacheService.resolveCacheRoot(buildParameters);
cacheKey = LocalCacheService.generateCacheKey(
buildParameters.targetPlatform,
buildParameters.editorVersion,
buildParameters.branch || '',
);
if (buildParameters.localCacheLfs) {
await LocalCacheService.restoreLfsCache(workspace, cacheRoot, cacheKey);
}
if (buildParameters.localCacheLibrary) {
const projectFullPath = path.join(workspace, buildParameters.projectPath);
await LocalCacheService.restoreLibraryCache(projectFullPath, cacheRoot, cacheKey);
}
}
// Git hooks — opt-in only. When disabled (default), do not touch hooks at all.
if (buildParameters.gitHooksEnabled) {
const { GitHooksService } = await import('./model/orchestrator/services/hooks/git-hooks-service');
await GitHooksService.installHooks(workspace);
if (buildParameters.gitHooksSkipList) {
const environment = GitHooksService.configureSkipList(buildParameters.gitHooksSkipList.split(','));
Object.assign(process.env, environment);
}
}
// Apply incremental sync strategy before build
const syncStrategy = buildParameters.syncStrategy as SyncStrategy;
if (syncStrategy !== 'full') {
core.info(`[Sync] Applying sync strategy: ${syncStrategy}`);
await applySyncStrategy(buildParameters, workspace);
}
await PlatformSetup.setup(buildParameters, actionFolder);
exitCode =
process.platform === 'darwin'
@@ -32,16 +218,115 @@ async function runMain() {
actionFolder,
...buildParameters,
});
// Local build caching - save
if (buildParameters.localCacheEnabled) {
const { LocalCacheService } = await import('./model/orchestrator/services/cache/local-cache-service');
if (buildParameters.localCacheLibrary) {
const projectFullPath = path.join(workspace, buildParameters.projectPath);
await LocalCacheService.saveLibraryCache(projectFullPath, cacheRoot, cacheKey);
}
if (buildParameters.localCacheLfs) {
await LocalCacheService.saveLfsCache(workspace, cacheRoot, cacheKey);
}
}
// Child workspace isolation - save workspace for next run
if (childWorkspaceConfig && childWorkspaceConfig.enabled) {
const { ChildWorkspaceService } = await import('./model/orchestrator/services/cache/child-workspace-service');
const projectFullPath = path.join(workspace, buildParameters.projectPath);
const preSaveSize = ChildWorkspaceService.getWorkspaceSize(projectFullPath);
core.info(`Child workspace size before save: ${preSaveSize}`);
ChildWorkspaceService.saveWorkspace(projectFullPath, childWorkspaceConfig);
core.info(`Child workspace "${buildParameters.childWorkspaceName}" saved to cache`);
}
// Revert overlays after job completion if configured
if (buildParameters.syncRevertAfter && syncStrategy !== 'full') {
core.info('[Sync] Reverting overlay changes after job completion');
try {
await IncrementalSyncService.revertOverlays(workspace, buildParameters.syncStatePath);
} catch (revertError) {
core.warning(`[Sync] Overlay revert failed: ${(revertError as Error).message}`);
}
}
exitCode = await runColdBuild(buildParameters, baseImage, workspace, actionFolder);
} else {
await CloudRunner.run(buildParameters, baseImage.toString());
await Orchestrator.run(buildParameters, baseImage.toString());
exitCode = 0;
}
// Post-build: archive and enforce retention
if (buildParameters.buildArchiveEnabled && exitCode === 0) {
core.info('Archiving build output...');
BuildReliabilityService.archiveBuildOutput(buildParameters.buildPath, buildParameters.buildArchivePath);
BuildReliabilityService.enforceRetention(buildParameters.buildArchivePath, buildParameters.buildArchiveRetention);
}
// Set output
await Output.setBuildVersion(buildParameters.buildVersion);
await Output.setAndroidVersionCode(buildParameters.androidVersionCode);
await Output.setEngineExitCode(exitCode);
// Artifact collection and upload (runs on both success and failure)
try {
// Register custom output types if provided
if (buildParameters.artifactCustomTypes) {
try {
const customTypes = JSON.parse(buildParameters.artifactCustomTypes);
if (Array.isArray(customTypes)) {
for (const ct of customTypes) {
OutputTypeRegistry.registerType({
name: ct.name,
defaultPath: ct.defaultPath || ct.pattern || `./${ct.name}/`,
description: ct.description || `Custom output type: ${ct.name}`,
builtIn: false,
});
}
}
} catch (parseError) {
core.warning(`Failed to parse artifactCustomTypes: ${(parseError as Error).message}`);
}
}
// Collect outputs and generate manifest
const manifestPath = path.join(buildParameters.projectPath, 'output-manifest.json');
const manifest = await OutputService.collectOutputs(
buildParameters.projectPath,
buildParameters.buildGuid,
buildParameters.artifactOutputTypes,
manifestPath,
);
core.setOutput('artifactManifestPath', manifestPath);
// Upload artifacts
const uploadConfig = ArtifactUploadHandler.parseConfig(
buildParameters.artifactUploadTarget,
buildParameters.artifactUploadPath || undefined,
buildParameters.artifactCompression,
buildParameters.artifactRetentionDays,
);
const uploadResult = await ArtifactUploadHandler.uploadArtifacts(
manifest,
uploadConfig,
buildParameters.projectPath,
);
if (!uploadResult.success) {
core.warning(
`Artifact upload completed with errors: ${uploadResult.entries
.filter((e) => !e.success)
.map((e) => `${e.type}: ${e.error}`)
.join('; ')}`,
);
}
} catch (artifactError) {
core.warning(`Artifact collection/upload failed: ${(artifactError as Error).message}`);
}
if (exitCode !== 0) {
core.setFailed(`Build failed with exit code ${exitCode}`);
}
@@ -50,4 +335,82 @@ async function runMain() {
}
}
async function runColdBuild(
buildParameters: BuildParameters,
baseImage: ImageTag,
workspace: string,
actionFolder: string,
): Promise<number> {
if (buildParameters.providerStrategy === 'local') {
core.info('Building locally');
await PlatformSetup.setup(buildParameters, actionFolder);
return process.platform === 'darwin'
? await MacBuilder.run(actionFolder)
: await Docker.run(baseImage.toString(), {
workspace,
actionFolder,
...buildParameters,
});
} else {
await Orchestrator.run(buildParameters, baseImage.toString());
return 0;
}
}
/**
* Apply the configured sync strategy to the workspace before build.
*/
async function applySyncStrategy(buildParameters: BuildParameters, workspace: string): Promise<void> {
const strategy = buildParameters.syncStrategy as SyncStrategy;
const resolvedStrategy = IncrementalSyncService.resolveStrategy(strategy, workspace, buildParameters.syncStatePath);
if (resolvedStrategy === 'full') {
core.info('[Sync] Resolved to full sync (no incremental state available)');
return;
}
switch (resolvedStrategy) {
case 'git-delta': {
const targetReference = buildParameters.gitSha || buildParameters.branch;
const changedFiles = await IncrementalSyncService.syncGitDelta(
workspace,
targetReference,
buildParameters.syncStatePath,
);
core.info(`[Sync] Git delta sync applied: ${changedFiles} file(s) changed`);
break;
}
case 'direct-input': {
if (!buildParameters.syncInputRef) {
throw new Error('[Sync] direct-input strategy requires syncInputRef to be set');
}
const overlays = await IncrementalSyncService.applyDirectInput(
workspace,
buildParameters.syncInputRef,
buildParameters.syncStorageRemote || undefined,
buildParameters.syncStatePath,
);
core.info(`[Sync] Direct input applied: ${overlays.length} overlay(s)`);
break;
}
case 'storage-pull': {
if (!buildParameters.syncInputRef) {
throw new Error('[Sync] storage-pull strategy requires syncInputRef to be set');
}
const pulledFiles = await IncrementalSyncService.syncStoragePull(workspace, buildParameters.syncInputRef, {
rcloneRemote: buildParameters.syncStorageRemote || undefined,
syncRevertAfter: buildParameters.syncRevertAfter,
statePath: buildParameters.syncStatePath,
});
core.info(`[Sync] Storage pull complete: ${pulledFiles.length} file(s)`);
break;
}
default:
core.warning(`[Sync] Unknown sync strategy: ${resolvedStrategy}`);
}
}
runMain();

View File

@@ -1,13 +1,13 @@
// Integration test for exercising real GitHub check creation and updates.
import CloudRunner from '../model/cloud-runner/cloud-runner';
import Orchestrator from '../model/orchestrator/orchestrator';
import UnityVersioning from '../model/unity-versioning';
import GitHub from '../model/github';
import { TIMEOUT_INFINITE, createParameters } from '../test-utils/cloud-runner-test-helpers';
import { TIMEOUT_INFINITE, createParameters } from '../test-utils/orchestrator-test-helpers';
const runIntegration = process.env.RUN_GITHUB_INTEGRATION_TESTS === 'true';
const describeOrSkip = runIntegration ? describe : describe.skip;
describeOrSkip('Cloud Runner Github Checks Integration', () => {
describeOrSkip('Orchestrator Github Checks Integration', () => {
it(
'creates and updates a real GitHub check',
async () => {
@@ -15,10 +15,10 @@ describeOrSkip('Cloud Runner Github Checks Integration', () => {
versioning: 'None',
projectPath: 'test-project',
unityVersion: UnityVersioning.read('test-project'),
asyncCloudRunner: `true`,
asyncOrchestrator: `true`,
githubChecks: `true`,
});
await CloudRunner.setup(buildParameter);
await Orchestrator.setup(buildParameter);
const checkId = await GitHub.createGitHubCheck(`integration create`);
expect(checkId).not.toEqual('');
await GitHub.updateGitHubCheck(`1 ${new Date().toISOString()}`, `integration`);

View File

@@ -1,7 +1,7 @@
import { customAlphabet } from 'nanoid';
import AndroidVersioning from './android-versioning';
import CloudRunnerConstants from './cloud-runner/options/cloud-runner-constants';
import CloudRunnerBuildGuid from './cloud-runner/options/cloud-runner-guid';
import OrchestratorConstants from './orchestrator/options/orchestrator-constants';
import OrchestratorBuildGuid from './orchestrator/options/orchestrator-guid';
import Input from './input';
import Platform from './platform';
import UnityVersioning from './unity-versioning';
@@ -10,8 +10,8 @@ import { GitRepoReader } from './input-readers/git-repo';
import { GithubCliReader } from './input-readers/github-cli';
import { Cli } from './cli/cli';
import GitHub from './github';
import CloudRunnerOptions from './cloud-runner/options/cloud-runner-options';
import CloudRunner from './cloud-runner/cloud-runner';
import OrchestratorOptions from './orchestrator/options/orchestrator-options';
import Orchestrator from './orchestrator/orchestrator';
import * as core from '@actions/core';
class BuildParameters {
@@ -54,11 +54,27 @@ class BuildParameters {
public sshAgent!: string;
public sshPublicKeysDirectoryPath!: string;
public providerStrategy!: string;
public fallbackProviderStrategy!: string;
public runnerCheckEnabled!: boolean;
public runnerCheckLabels!: string[];
public runnerCheckMinAvailable!: number;
public retryOnFallback!: boolean;
public providerInitTimeout!: number;
public gitAuthMode!: string;
public gitPrivateToken!: string;
public awsStackName!: string;
public awsEndpoint?: string;
public awsCloudFormationEndpoint?: string;
public awsEcsEndpoint?: string;
public awsKinesisEndpoint?: string;
public awsCloudWatchLogsEndpoint?: string;
public awsS3Endpoint?: string;
public storageProvider!: string;
public rcloneRemote!: string;
public kubeConfig!: string;
public containerMemory!: string;
public containerCpu!: string;
public containerNamespace!: string;
public kubeVolumeSize!: string;
public kubeVolume!: string;
public kubeStorageClass!: string;
@@ -75,11 +91,13 @@ class BuildParameters {
public runNumber!: string;
public branch!: string;
public githubRepo!: string;
public orchestratorRepoName!: string;
public cloneDepth!: number;
public gitSha!: string;
public logId!: string;
public buildGuid!: string;
public cloudRunnerBranch!: string;
public cloudRunnerDebug!: boolean | undefined;
public orchestratorBranch!: string;
public orchestratorDebug!: boolean | undefined;
public buildPlatform!: string | undefined;
public isCliMode!: boolean;
public maxRetainedWorkspaces!: number;
@@ -95,9 +113,106 @@ class BuildParameters {
public cacheUnityInstallationOnMac!: boolean;
public unityHubVersionOnMac!: string;
public dockerWorkspacePath!: string;
public submoduleProfilePath!: string;
public submoduleVariantPath!: string;
public submoduleToken!: string;
public localCacheEnabled!: boolean;
public localCacheRoot!: string;
public localCacheLibrary!: boolean;
public localCacheLfs!: boolean;
public childWorkspacesEnabled!: boolean;
public childWorkspaceName!: string;
public childWorkspaceCacheRoot!: string;
public childWorkspacePreserveGit!: boolean;
public childWorkspaceSeparateLibrary!: boolean;
public lfsTransferAgent!: string;
public lfsTransferAgentArgs!: string;
public lfsStoragePaths!: string;
public gitHooksEnabled!: boolean;
public gitHooksSkipList!: string;
public gitHooksRunBeforeBuild!: string;
public providerExecutable!: string;
// GCP Cloud Run (Experimental)
public gcpProject!: string;
public gcpRegion!: string;
public gcpStorageType!: string;
public gcpBucket!: string;
public gcpFilestoreIp!: string;
public gcpFilestoreShare!: string;
public gcpMachineType!: string;
public gcpDiskSizeGb!: string;
public gcpServiceAccount!: string;
public gcpVpcConnector!: string;
// Azure Container Instances (Experimental)
public azureResourceGroup!: string;
public azureLocation!: string;
public azureStorageType!: string;
public azureStorageAccount!: string;
public azureBlobContainer!: string;
public azureFileShareName!: string;
public azureSubscriptionId!: string;
public azureCpu!: string;
public azureMemoryGb!: string;
public azureDiskSizeGb!: string;
public azureSubnetId!: string;
// Remote PowerShell provider
public remotePowershellHost!: string;
public remotePowershellCredential!: string;
public remotePowershellTransport!: string;
// GitHub Actions provider
public githubActionsRepo!: string;
public githubActionsWorkflow!: string;
public githubActionsToken!: string;
public githubActionsRef!: string;
// GitLab CI provider
public gitlabProjectId!: string;
public gitlabTriggerToken!: string;
public gitlabApiUrl!: string;
public gitlabRef!: string;
// Ansible provider
public ansibleInventory!: string;
public ansiblePlaybook!: string;
public ansibleExtraVars!: string;
public ansibleVaultPassword!: string;
public gitIntegrityCheck!: boolean;
public gitAutoRecover!: boolean;
public cleanReservedFilenames!: boolean;
public buildArchiveEnabled!: boolean;
public buildArchivePath!: string;
public buildArchiveRetention!: number;
public testSuitePath!: string;
public testSuiteEvent!: string;
public testTaxonomyPath!: string;
public testResultFormat!: string;
public testResultPath!: string;
public hotRunnerEnabled!: boolean;
public hotRunnerTransport!: 'websocket' | 'grpc' | 'named-pipe';
public hotRunnerHost!: string;
public hotRunnerPort!: number;
public hotRunnerHealthInterval!: number;
public hotRunnerMaxIdle!: number;
public hotRunnerFallbackToCold!: boolean;
public artifactOutputTypes!: string;
public artifactUploadTarget!: string;
public artifactUploadPath!: string;
public artifactCompression!: string;
public artifactRetentionDays!: string;
public artifactCustomTypes!: string;
public syncStrategy!: string;
public syncInputRef!: string;
public syncStorageRemote!: string;
public syncRevertAfter!: boolean;
public syncStatePath!: string;
public static shouldUseRetainedWorkspaceMode(buildParameters: BuildParameters) {
return buildParameters.maxRetainedWorkspaces > 0 && CloudRunner.lockedWorkspace !== ``;
return buildParameters.maxRetainedWorkspaces > 0 && Orchestrator.lockedWorkspace !== ``;
}
static async create(): Promise<BuildParameters> {
@@ -182,44 +297,154 @@ class BuildParameters {
dockerIsolationMode: Input.dockerIsolationMode,
containerRegistryRepository: Input.containerRegistryRepository,
containerRegistryImageVersion: Input.containerRegistryImageVersion,
providerStrategy: CloudRunnerOptions.providerStrategy,
buildPlatform: CloudRunnerOptions.buildPlatform,
kubeConfig: CloudRunnerOptions.kubeConfig,
containerMemory: CloudRunnerOptions.containerMemory,
containerCpu: CloudRunnerOptions.containerCpu,
kubeVolumeSize: CloudRunnerOptions.kubeVolumeSize,
kubeVolume: CloudRunnerOptions.kubeVolume,
postBuildContainerHooks: CloudRunnerOptions.postBuildContainerHooks,
preBuildContainerHooks: CloudRunnerOptions.preBuildContainerHooks,
customJob: CloudRunnerOptions.customJob,
providerStrategy: OrchestratorOptions.providerStrategy,
fallbackProviderStrategy: OrchestratorOptions.fallbackProviderStrategy,
runnerCheckEnabled: OrchestratorOptions.runnerCheckEnabled,
runnerCheckLabels: OrchestratorOptions.runnerCheckLabels,
runnerCheckMinAvailable: OrchestratorOptions.runnerCheckMinAvailable,
retryOnFallback: OrchestratorOptions.retryOnFallback,
providerInitTimeout: OrchestratorOptions.providerInitTimeout,
gitAuthMode: OrchestratorOptions.gitAuthMode,
buildPlatform: OrchestratorOptions.buildPlatform,
kubeConfig: OrchestratorOptions.kubeConfig,
containerMemory: OrchestratorOptions.containerMemory,
containerCpu: OrchestratorOptions.containerCpu,
containerNamespace: OrchestratorOptions.containerNamespace,
kubeVolumeSize: OrchestratorOptions.kubeVolumeSize,
kubeVolume: OrchestratorOptions.kubeVolume,
postBuildContainerHooks: OrchestratorOptions.postBuildContainerHooks,
preBuildContainerHooks: OrchestratorOptions.preBuildContainerHooks,
customJob: OrchestratorOptions.customJob,
runNumber: Input.runNumber,
branch: Input.branch.replace('/head', '') || (await GitRepoReader.GetBranch()),
cloudRunnerBranch: CloudRunnerOptions.cloudRunnerBranch.split('/').reverse()[0],
cloudRunnerDebug: CloudRunnerOptions.cloudRunnerDebug,
githubRepo: (Input.githubRepo ?? (await GitRepoReader.GetRemote())) || 'game-ci/unity-builder',
orchestratorBranch: OrchestratorOptions.orchestratorBranch.split('/').reverse()[0],
orchestratorDebug: OrchestratorOptions.orchestratorDebug,
githubRepo: (Input.githubRepo ?? (await GitRepoReader.GetRemote())) || OrchestratorOptions.orchestratorRepoName,
orchestratorRepoName: OrchestratorOptions.orchestratorRepoName,
cloneDepth: Number.parseInt(OrchestratorOptions.cloneDepth),
isCliMode: Cli.isCliMode,
awsStackName: CloudRunnerOptions.awsStackName,
awsStackName: OrchestratorOptions.awsStackName,
awsEndpoint: OrchestratorOptions.awsEndpoint,
awsCloudFormationEndpoint: OrchestratorOptions.awsCloudFormationEndpoint,
awsEcsEndpoint: OrchestratorOptions.awsEcsEndpoint,
awsKinesisEndpoint: OrchestratorOptions.awsKinesisEndpoint,
awsCloudWatchLogsEndpoint: OrchestratorOptions.awsCloudWatchLogsEndpoint,
awsS3Endpoint: OrchestratorOptions.awsS3Endpoint,
storageProvider: OrchestratorOptions.storageProvider,
rcloneRemote: OrchestratorOptions.rcloneRemote,
gitSha: Input.gitSha,
logId: customAlphabet(CloudRunnerConstants.alphabet, 9)(),
buildGuid: CloudRunnerBuildGuid.generateGuid(Input.runNumber, Input.targetPlatform),
commandHooks: CloudRunnerOptions.commandHooks,
inputPullCommand: CloudRunnerOptions.inputPullCommand,
pullInputList: CloudRunnerOptions.pullInputList,
kubeStorageClass: CloudRunnerOptions.kubeStorageClass,
cacheKey: CloudRunnerOptions.cacheKey,
maxRetainedWorkspaces: Number.parseInt(CloudRunnerOptions.maxRetainedWorkspaces),
useLargePackages: CloudRunnerOptions.useLargePackages,
useCompressionStrategy: CloudRunnerOptions.useCompressionStrategy,
garbageMaxAge: CloudRunnerOptions.garbageMaxAge,
githubChecks: CloudRunnerOptions.githubChecks,
asyncWorkflow: CloudRunnerOptions.asyncCloudRunner,
githubCheckId: CloudRunnerOptions.githubCheckId,
finalHooks: CloudRunnerOptions.finalHooks,
skipLfs: CloudRunnerOptions.skipLfs,
skipCache: CloudRunnerOptions.skipCache,
logId: customAlphabet(OrchestratorConstants.alphabet, 9)(),
buildGuid: OrchestratorBuildGuid.generateGuid(Input.runNumber, Input.targetPlatform),
commandHooks: OrchestratorOptions.commandHooks,
inputPullCommand: OrchestratorOptions.inputPullCommand,
pullInputList: OrchestratorOptions.pullInputList,
kubeStorageClass: OrchestratorOptions.kubeStorageClass,
gcpProject: Input.gcpProject,
gcpRegion: Input.gcpRegion,
gcpStorageType: Input.gcpStorageType,
gcpBucket: Input.gcpBucket,
gcpFilestoreIp: Input.gcpFilestoreIp,
gcpFilestoreShare: Input.gcpFilestoreShare,
gcpMachineType: Input.gcpMachineType,
gcpDiskSizeGb: Input.gcpDiskSizeGb,
gcpServiceAccount: Input.gcpServiceAccount,
gcpVpcConnector: Input.gcpVpcConnector,
azureResourceGroup: Input.azureResourceGroup,
azureLocation: Input.azureLocation,
azureStorageType: Input.azureStorageType,
azureStorageAccount: Input.azureStorageAccount,
azureBlobContainer: Input.azureBlobContainer,
azureFileShareName: Input.azureFileShareName,
azureSubscriptionId: Input.azureSubscriptionId,
azureCpu: Input.azureCpu,
azureMemoryGb: Input.azureMemoryGb,
azureDiskSizeGb: Input.azureDiskSizeGb,
azureSubnetId: Input.azureSubnetId,
cacheKey: OrchestratorOptions.cacheKey,
maxRetainedWorkspaces: Number.parseInt(OrchestratorOptions.maxRetainedWorkspaces),
useLargePackages: OrchestratorOptions.useLargePackages,
useCompressionStrategy: OrchestratorOptions.useCompressionStrategy,
garbageMaxAge: OrchestratorOptions.garbageMaxAge,
githubChecks: OrchestratorOptions.githubChecks,
asyncWorkflow: OrchestratorOptions.asyncOrchestrator,
githubCheckId: OrchestratorOptions.githubCheckId,
finalHooks: OrchestratorOptions.finalHooks,
skipLfs: OrchestratorOptions.skipLfs,
skipCache: OrchestratorOptions.skipCache,
cacheUnityInstallationOnMac: Input.cacheUnityInstallationOnMac,
unityHubVersionOnMac: Input.unityHubVersionOnMac,
dockerWorkspacePath: Input.dockerWorkspacePath,
submoduleProfilePath: Input.submoduleProfilePath,
submoduleVariantPath: Input.submoduleVariantPath,
submoduleToken: Input.submoduleToken,
localCacheEnabled: Input.localCacheEnabled,
localCacheRoot: Input.localCacheRoot,
localCacheLibrary: Input.localCacheLibrary,
localCacheLfs: Input.localCacheLfs,
childWorkspacesEnabled: Input.childWorkspacesEnabled,
childWorkspaceName: Input.childWorkspaceName,
childWorkspaceCacheRoot: Input.childWorkspaceCacheRoot,
childWorkspacePreserveGit: Input.childWorkspacePreserveGit,
childWorkspaceSeparateLibrary: Input.childWorkspaceSeparateLibrary,
lfsTransferAgent: Input.lfsTransferAgent,
lfsTransferAgentArgs: Input.lfsTransferAgentArgs,
lfsStoragePaths: Input.lfsStoragePaths,
gitHooksEnabled: Input.gitHooksEnabled,
gitHooksSkipList: Input.gitHooksSkipList,
gitHooksRunBeforeBuild: Input.gitHooksRunBeforeBuild,
providerExecutable: Input.providerExecutable,
// Remote PowerShell provider
remotePowershellHost: Input.remotePowershellHost,
remotePowershellCredential: Input.remotePowershellCredential,
remotePowershellTransport: Input.remotePowershellTransport,
// GitHub Actions provider
githubActionsRepo: Input.githubActionsRepo,
githubActionsWorkflow: Input.githubActionsWorkflow,
githubActionsToken: Input.githubActionsToken,
githubActionsRef: Input.githubActionsRef,
// GitLab CI provider
gitlabProjectId: Input.gitlabProjectId,
gitlabTriggerToken: Input.gitlabTriggerToken,
gitlabApiUrl: Input.gitlabApiUrl,
gitlabRef: Input.gitlabRef,
// Ansible provider
ansibleInventory: Input.ansibleInventory,
ansiblePlaybook: Input.ansiblePlaybook,
ansibleExtraVars: Input.ansibleExtraVars,
ansibleVaultPassword: Input.ansibleVaultPassword,
gitIntegrityCheck: Input.gitIntegrityCheck,
gitAutoRecover: Input.gitAutoRecover,
cleanReservedFilenames: Input.cleanReservedFilenames,
buildArchiveEnabled: Input.buildArchiveEnabled,
buildArchivePath: Input.buildArchivePath,
buildArchiveRetention: Input.buildArchiveRetention,
testSuitePath: Input.testSuitePath,
testSuiteEvent: Input.testSuiteEvent,
testTaxonomyPath: Input.testTaxonomyPath,
testResultFormat: Input.testResultFormat,
testResultPath: Input.testResultPath,
hotRunnerEnabled: Input.hotRunnerEnabled,
hotRunnerTransport: Input.hotRunnerTransport,
hotRunnerHost: Input.hotRunnerHost,
hotRunnerPort: Input.hotRunnerPort,
hotRunnerHealthInterval: Input.hotRunnerHealthInterval,
hotRunnerMaxIdle: Input.hotRunnerMaxIdle,
hotRunnerFallbackToCold: Input.hotRunnerFallbackToCold,
artifactOutputTypes: Input.artifactOutputTypes,
artifactUploadTarget: Input.artifactUploadTarget,
artifactUploadPath: Input.artifactUploadPath,
artifactCompression: Input.artifactCompression,
artifactRetentionDays: Input.artifactRetentionDays,
artifactCustomTypes: Input.artifactCustomTypes,
syncStrategy: Input.syncStrategy,
syncInputRef: Input.syncInputRef,
syncStorageRemote: Input.syncStorageRemote,
syncRevertAfter: Input.syncRevertAfter,
syncStatePath: Input.syncStatePath,
};
}

View File

@@ -1,17 +1,19 @@
import { Command } from 'commander-ts';
import { BuildParameters, CloudRunner, ImageTag, Input } from '..';
import { BuildParameters, Orchestrator, ImageTag, Input } from '..';
import * as core from '@actions/core';
import { ActionYamlReader } from '../input-readers/action-yaml';
import CloudRunnerLogger from '../cloud-runner/services/core/cloud-runner-logger';
import CloudRunnerQueryOverride from '../cloud-runner/options/cloud-runner-query-override';
import OrchestratorLogger from '../orchestrator/services/core/orchestrator-logger';
import OrchestratorQueryOverride from '../orchestrator/options/orchestrator-query-override';
import { CliFunction, CliFunctionsRepository } from './cli-functions-repository';
import { Caching } from '../cloud-runner/remote-client/caching';
import { LfsHashing } from '../cloud-runner/services/utility/lfs-hashing';
import { RemoteClient } from '../cloud-runner/remote-client';
import CloudRunnerOptionsReader from '../cloud-runner/options/cloud-runner-options-reader';
import { Caching } from '../orchestrator/remote-client/caching';
import { LfsHashing } from '../orchestrator/services/utility/lfs-hashing';
import { RemoteClient } from '../orchestrator/remote-client';
import OrchestratorOptionsReader from '../orchestrator/options/orchestrator-options-reader';
import GitHub from '../github';
import { OptionValues } from 'commander';
import { InputKey } from '../input';
import { SubmoduleProfileService } from '../orchestrator/services/submodule/submodule-profile-service';
import { LfsAgentService } from '../orchestrator/services/lfs/lfs-agent-service';
export class Cli {
public static options: OptionValues | undefined;
@@ -36,7 +38,7 @@ export class Cli {
const program = new Command();
program.version('0.0.1');
const properties = CloudRunnerOptionsReader.GetProperties();
const properties = OrchestratorOptionsReader.GetProperties();
const actionYamlReader: ActionYamlReader = new ActionYamlReader();
for (const element of properties) {
program.option(`--${element} <${element}>`, actionYamlReader.GetActionYamlValue(element));
@@ -53,6 +55,11 @@ export class Cli {
program.option('--artifactName <artifactName>', 'caching artifact name');
program.option('--select <select>', 'select a particular resource');
program.option('--logFile <logFile>', 'output to log file (log stream only)');
program.option('--profilePath <profilePath>', 'path to submodule profile YAML');
program.option('--variantPath <variantPath>', 'path to submodule variant YAML');
program.option('--agentPath <agentPath>', 'path to custom LFS transfer agent');
program.option('--agentArgs <agentArgs>', 'arguments for custom LFS transfer agent');
program.option('--storagePaths <storagePaths>', 'semicolon-separated storage paths for LFS agent');
program.parse(process.argv);
Cli.options = program.opts();
@@ -62,23 +69,23 @@ export class Cli {
static async RunCli(): Promise<void> {
GitHub.githubInputEnabled = false;
if (Cli.options!['populateOverride'] === `true`) {
await CloudRunnerQueryOverride.PopulateQueryOverrideInput();
await OrchestratorQueryOverride.PopulateQueryOverrideInput();
}
if (Cli.options!['logInput']) {
Cli.logInput();
}
const results = CliFunctionsRepository.GetCliFunctions(Cli.options?.mode);
CloudRunnerLogger.log(`Entrypoint: ${results.key}`);
OrchestratorLogger.log(`Entrypoint: ${results.key}`);
Cli.options!.versioning = 'None';
CloudRunner.buildParameters = await BuildParameters.create();
CloudRunner.buildParameters.buildGuid = process.env.BUILD_GUID || ``;
CloudRunnerLogger.log(`Build Params:
${JSON.stringify(CloudRunner.buildParameters, undefined, 4)}
Orchestrator.buildParameters = await BuildParameters.create();
Orchestrator.buildParameters.buildGuid = process.env.BUILD_GUID || ``;
OrchestratorLogger.log(`Build Params:
${JSON.stringify(Orchestrator.buildParameters, undefined, 4)}
`);
CloudRunner.lockedWorkspace = process.env.LOCKED_WORKSPACE || ``;
CloudRunnerLogger.log(`Locked Workspace: ${CloudRunner.lockedWorkspace}`);
await CloudRunner.setup(CloudRunner.buildParameters);
Orchestrator.lockedWorkspace = process.env.LOCKED_WORKSPACE || ``;
OrchestratorLogger.log(`Locked Workspace: ${Orchestrator.lockedWorkspace}`);
await Orchestrator.setup(Orchestrator.buildParameters);
return await results.target[results.propertyKey](Cli.options);
}
@@ -87,7 +94,7 @@ export class Cli {
private static logInput() {
core.info(`\n`);
core.info(`INPUT:`);
const properties = CloudRunnerOptionsReader.GetProperties();
const properties = OrchestratorOptionsReader.GetProperties();
for (const element of properties) {
if (
element in Input &&
@@ -104,28 +111,28 @@ export class Cli {
core.info(`\n`);
}
@CliFunction(`cli-build`, `runs a cloud runner build`)
@CliFunction(`cli-build`, `runs a orchestrator build`)
public static async CLIBuild(): Promise<string> {
const buildParameter = await BuildParameters.create();
const baseImage = new ImageTag(buildParameter);
return (await CloudRunner.run(buildParameter, baseImage.toString())).BuildResults;
return (await Orchestrator.run(buildParameter, baseImage.toString())).BuildResults;
}
@CliFunction(`async-workflow`, `runs a cloud runner build`)
@CliFunction(`async-workflow`, `runs a orchestrator build`)
public static async asyncronousWorkflow(): Promise<string> {
const buildParameter = await BuildParameters.create();
const baseImage = new ImageTag(buildParameter);
await CloudRunner.setup(buildParameter);
await Orchestrator.setup(buildParameter);
return (await CloudRunner.run(buildParameter, baseImage.toString())).BuildResults;
return (await Orchestrator.run(buildParameter, baseImage.toString())).BuildResults;
}
@CliFunction(`checks-update`, `runs a cloud runner build`)
@CliFunction(`checks-update`, `runs a orchestrator build`)
public static async checksUpdate() {
const buildParameter = await BuildParameters.create();
await CloudRunner.setup(buildParameter);
await Orchestrator.setup(buildParameter);
const input = JSON.parse(process.env.CHECKS_UPDATE || ``);
core.info(`Checks Update ${process.env.CHECKS_UPDATE}`);
if (input.mode === `create`) {
@@ -139,18 +146,18 @@ export class Cli {
public static async GarbageCollect(): Promise<string> {
const buildParameter = await BuildParameters.create();
await CloudRunner.setup(buildParameter);
await Orchestrator.setup(buildParameter);
return await CloudRunner.Provider.garbageCollect(``, false, 0, false, false);
return await Orchestrator.Provider.garbageCollect(``, false, 0, false, false);
}
@CliFunction(`list-resources`, `lists active resources`)
public static async ListResources(): Promise<string[]> {
const buildParameter = await BuildParameters.create();
await CloudRunner.setup(buildParameter);
const result = await CloudRunner.Provider.listResources();
CloudRunnerLogger.log(JSON.stringify(result, undefined, 4));
await Orchestrator.setup(buildParameter);
const result = await Orchestrator.Provider.listResources();
OrchestratorLogger.log(JSON.stringify(result, undefined, 4));
return result.map((x) => x.Name);
}
@@ -159,17 +166,39 @@ export class Cli {
public static async ListWorfklow(): Promise<string[]> {
const buildParameter = await BuildParameters.create();
await CloudRunner.setup(buildParameter);
await Orchestrator.setup(buildParameter);
return (await CloudRunner.Provider.listWorkflow()).map((x) => x.Name);
return (await Orchestrator.Provider.listWorkflow()).map((x) => x.Name);
}
@CliFunction(`watch`, `follows logs of a running workflow`)
public static async Watch(): Promise<string> {
const buildParameter = await BuildParameters.create();
await CloudRunner.setup(buildParameter);
await Orchestrator.setup(buildParameter);
return await CloudRunner.Provider.watchWorkflow();
return await Orchestrator.Provider.watchWorkflow();
}
@CliFunction(`submodule-init`, `initializes submodules from a YAML profile`)
public static async SubmoduleInit(): Promise<void> {
const profilePath = Cli.options!['profilePath'];
const variantPath = Cli.options!['variantPath'] || '';
if (!profilePath) {
throw new Error('--profilePath is required for submodule-init');
}
const plan = await SubmoduleProfileService.createInitPlan(profilePath, variantPath, process.cwd());
await SubmoduleProfileService.execute(plan, process.cwd());
}
@CliFunction(`lfs-agent-configure`, `configures a custom LFS transfer agent`)
public static async LfsAgentConfigure(): Promise<void> {
const agentPath = Cli.options!['agentPath'];
if (!agentPath) {
throw new Error('--agentPath is required for lfs-agent-configure');
}
const agentArgs = Cli.options!['agentArgs'] || '';
const storagePaths = (Cli.options!['storagePaths'] || '').split(';').filter(Boolean);
await LfsAgentService.configure(agentPath, agentArgs, storagePaths, process.cwd());
}
}

View File

@@ -1,187 +0,0 @@
import AwsBuildPlatform from './providers/aws';
import { BuildParameters, Input } from '..';
import Kubernetes from './providers/k8s';
import CloudRunnerLogger from './services/core/cloud-runner-logger';
import { CloudRunnerStepParameters } from './options/cloud-runner-step-parameters';
import { WorkflowCompositionRoot } from './workflows/workflow-composition-root';
import { CloudRunnerError } from './error/cloud-runner-error';
import { TaskParameterSerializer } from './services/core/task-parameter-serializer';
import * as core from '@actions/core';
import CloudRunnerSecret from './options/cloud-runner-secret';
import { ProviderInterface } from './providers/provider-interface';
import CloudRunnerEnvironmentVariable from './options/cloud-runner-environment-variable';
import TestCloudRunner from './providers/test';
import LocalCloudRunner from './providers/local';
import LocalDockerCloudRunner from './providers/docker';
import GitHub from '../github';
import SharedWorkspaceLocking from './services/core/shared-workspace-locking';
import { FollowLogStreamService } from './services/core/follow-log-stream-service';
import CloudRunnerResult from './services/core/cloud-runner-result';
class CloudRunner {
public static Provider: ProviderInterface;
public static buildParameters: BuildParameters;
private static defaultSecrets: CloudRunnerSecret[];
private static cloudRunnerEnvironmentVariables: CloudRunnerEnvironmentVariable[];
static lockedWorkspace: string = ``;
public static readonly retainedWorkspacePrefix: string = `retained-workspace`;
public static get isCloudRunnerEnvironment() {
return process.env[`GITHUB_ACTIONS`] !== `true`;
}
public static get isCloudRunnerAsyncEnvironment() {
return process.env[`ASYNC_WORKFLOW`] === `true`;
}
public static async setup(buildParameters: BuildParameters) {
CloudRunnerLogger.setup();
CloudRunnerLogger.log(`Setting up cloud runner`);
CloudRunner.buildParameters = buildParameters;
if (CloudRunner.buildParameters.githubCheckId === ``) {
CloudRunner.buildParameters.githubCheckId = await GitHub.createGitHubCheck(CloudRunner.buildParameters.buildGuid);
}
CloudRunner.setupSelectedBuildPlatform();
CloudRunner.defaultSecrets = TaskParameterSerializer.readDefaultSecrets();
CloudRunner.cloudRunnerEnvironmentVariables =
TaskParameterSerializer.createCloudRunnerEnvironmentVariables(buildParameters);
if (GitHub.githubInputEnabled) {
const buildParameterPropertyNames = Object.getOwnPropertyNames(buildParameters);
for (const element of CloudRunner.cloudRunnerEnvironmentVariables) {
// CloudRunnerLogger.log(`Cloud Runner output ${Input.ToEnvVarFormat(element.name)} = ${element.value}`);
core.setOutput(Input.ToEnvVarFormat(element.name), element.value);
}
for (const element of buildParameterPropertyNames) {
// CloudRunnerLogger.log(`Cloud Runner output ${Input.ToEnvVarFormat(element)} = ${buildParameters[element]}`);
core.setOutput(Input.ToEnvVarFormat(element), buildParameters[element]);
}
core.setOutput(
Input.ToEnvVarFormat(`buildArtifact`),
`build-${CloudRunner.buildParameters.buildGuid}.tar${
CloudRunner.buildParameters.useCompressionStrategy ? '.lz4' : ''
}`,
);
}
FollowLogStreamService.Reset();
}
private static setupSelectedBuildPlatform() {
CloudRunnerLogger.log(`Cloud Runner platform selected ${CloudRunner.buildParameters.providerStrategy}`);
switch (CloudRunner.buildParameters.providerStrategy) {
case 'k8s':
CloudRunner.Provider = new Kubernetes(CloudRunner.buildParameters);
break;
case 'aws':
CloudRunner.Provider = new AwsBuildPlatform(CloudRunner.buildParameters);
break;
case 'test':
CloudRunner.Provider = new TestCloudRunner();
break;
case 'local-docker':
CloudRunner.Provider = new LocalDockerCloudRunner();
break;
case 'local-system':
CloudRunner.Provider = new LocalCloudRunner();
break;
}
}
static async run(buildParameters: BuildParameters, baseImage: string) {
if (baseImage.includes(`undefined`)) {
throw new Error(`baseImage is undefined`);
}
await CloudRunner.setup(buildParameters);
await CloudRunner.Provider.setupWorkflow(
CloudRunner.buildParameters.buildGuid,
CloudRunner.buildParameters,
CloudRunner.buildParameters.branch,
CloudRunner.defaultSecrets,
);
try {
if (buildParameters.maxRetainedWorkspaces > 0) {
CloudRunner.lockedWorkspace = SharedWorkspaceLocking.NewWorkspaceName();
const result = await SharedWorkspaceLocking.GetLockedWorkspace(
CloudRunner.lockedWorkspace,
CloudRunner.buildParameters.buildGuid,
CloudRunner.buildParameters,
);
if (result) {
CloudRunnerLogger.logLine(`Using retained workspace ${CloudRunner.lockedWorkspace}`);
CloudRunner.cloudRunnerEnvironmentVariables = [
...CloudRunner.cloudRunnerEnvironmentVariables,
{ name: `LOCKED_WORKSPACE`, value: CloudRunner.lockedWorkspace },
];
} else {
CloudRunnerLogger.log(`Max retained workspaces reached ${buildParameters.maxRetainedWorkspaces}`);
buildParameters.maxRetainedWorkspaces = 0;
CloudRunner.lockedWorkspace = ``;
}
}
await CloudRunner.updateStatusWithBuildParameters();
const output = await new WorkflowCompositionRoot().run(
new CloudRunnerStepParameters(
baseImage,
CloudRunner.cloudRunnerEnvironmentVariables,
CloudRunner.defaultSecrets,
),
);
await CloudRunner.Provider.cleanupWorkflow(
CloudRunner.buildParameters,
CloudRunner.buildParameters.branch,
CloudRunner.defaultSecrets,
);
if (!CloudRunner.buildParameters.isCliMode) core.endGroup();
if (buildParameters.asyncWorkflow && this.isCloudRunnerEnvironment && this.isCloudRunnerAsyncEnvironment) {
await GitHub.updateGitHubCheck(CloudRunner.buildParameters.buildGuid, `success`, `success`, `completed`);
}
if (BuildParameters.shouldUseRetainedWorkspaceMode(buildParameters)) {
const workspace = CloudRunner.lockedWorkspace || ``;
await SharedWorkspaceLocking.ReleaseWorkspace(
workspace,
CloudRunner.buildParameters.buildGuid,
CloudRunner.buildParameters,
);
const isLocked = await SharedWorkspaceLocking.IsWorkspaceLocked(workspace, CloudRunner.buildParameters);
if (isLocked) {
throw new Error(
`still locked after releasing ${await SharedWorkspaceLocking.GetAllLocksForWorkspace(
workspace,
buildParameters,
)}`,
);
}
CloudRunner.lockedWorkspace = ``;
}
await GitHub.triggerWorkflowOnComplete(CloudRunner.buildParameters.finalHooks);
if (buildParameters.constantGarbageCollection) {
CloudRunner.Provider.garbageCollect(``, true, buildParameters.garbageMaxAge, true, true);
}
return new CloudRunnerResult(buildParameters, output, true, true, false);
} catch (error: any) {
CloudRunnerLogger.log(JSON.stringify(error, undefined, 4));
await GitHub.updateGitHubCheck(
CloudRunner.buildParameters.buildGuid,
`Failed - Error ${error?.message || error}`,
`failure`,
`completed`,
);
if (!CloudRunner.buildParameters.isCliMode) core.endGroup();
await CloudRunnerError.handleException(error, CloudRunner.buildParameters, CloudRunner.defaultSecrets);
throw error;
}
}
private static async updateStatusWithBuildParameters() {
const content = { ...CloudRunner.buildParameters };
content.gitPrivateToken = ``;
content.unitySerial = ``;
content.unityEmail = ``;
content.unityPassword = ``;
const jsonContent = JSON.stringify(content, undefined, 4);
await GitHub.updateGitHubCheck(jsonContent, CloudRunner.buildParameters.buildGuid);
}
}
export default CloudRunner;

View File

@@ -1,15 +0,0 @@
import CloudRunnerLogger from '../services/core/cloud-runner-logger';
import * as core from '@actions/core';
import CloudRunner from '../cloud-runner';
import CloudRunnerSecret from '../options/cloud-runner-secret';
import BuildParameters from '../../build-parameters';
export class CloudRunnerError {
public static async handleException(error: unknown, buildParameters: BuildParameters, secrets: CloudRunnerSecret[]) {
CloudRunnerLogger.error(JSON.stringify(error, undefined, 4));
core.setFailed('Cloud Runner failed');
if (CloudRunner.Provider !== undefined) {
await CloudRunner.Provider.cleanupWorkflow(buildParameters, buildParameters.branch, secrets);
}
}
}

View File

@@ -1,4 +0,0 @@
class CloudRunnerConstants {
static alphabet = '0123456789abcdefghijklmnopqrstuvwxyz';
}
export default CloudRunnerConstants;

View File

@@ -1,5 +0,0 @@
class CloudRunnerEnvironmentVariable {
public name!: string;
public value!: string;
}
export default CloudRunnerEnvironmentVariable;

View File

@@ -1,90 +0,0 @@
import path from 'node:path';
import CloudRunnerOptions from './cloud-runner-options';
import CloudRunner from '../cloud-runner';
import BuildParameters from '../../build-parameters';
export class CloudRunnerFolders {
public static readonly repositoryFolder = 'repo';
public static ToLinuxFolder(folder: string) {
return folder.replace(/\\/g, `/`);
}
// Only the following paths that do not start a path.join with another "Full" suffixed property need to start with an absolute /
public static get uniqueCloudRunnerJobFolderAbsolute(): string {
return CloudRunner.buildParameters && BuildParameters.shouldUseRetainedWorkspaceMode(CloudRunner.buildParameters)
? path.join(`/`, CloudRunnerFolders.buildVolumeFolder, CloudRunner.lockedWorkspace)
: path.join(`/`, CloudRunnerFolders.buildVolumeFolder, CloudRunner.buildParameters.buildGuid);
}
public static get cacheFolderForAllFull(): string {
return path.join('/', CloudRunnerFolders.buildVolumeFolder, CloudRunnerFolders.cacheFolder);
}
public static get cacheFolderForCacheKeyFull(): string {
return path.join(
'/',
CloudRunnerFolders.buildVolumeFolder,
CloudRunnerFolders.cacheFolder,
CloudRunner.buildParameters.cacheKey,
);
}
public static get builderPathAbsolute(): string {
return path.join(
CloudRunnerOptions.useSharedBuilder
? `/${CloudRunnerFolders.buildVolumeFolder}`
: CloudRunnerFolders.uniqueCloudRunnerJobFolderAbsolute,
`builder`,
);
}
public static get repoPathAbsolute(): string {
return path.join(CloudRunnerFolders.uniqueCloudRunnerJobFolderAbsolute, CloudRunnerFolders.repositoryFolder);
}
public static get projectPathAbsolute(): string {
return path.join(CloudRunnerFolders.repoPathAbsolute, CloudRunner.buildParameters.projectPath);
}
public static get libraryFolderAbsolute(): string {
return path.join(CloudRunnerFolders.projectPathAbsolute, `Library`);
}
public static get projectBuildFolderAbsolute(): string {
return path.join(CloudRunnerFolders.repoPathAbsolute, CloudRunner.buildParameters.buildPath);
}
public static get lfsFolderAbsolute(): string {
return path.join(CloudRunnerFolders.repoPathAbsolute, `.git`, `lfs`);
}
public static get purgeRemoteCaching(): boolean {
return process.env.PURGE_REMOTE_BUILDER_CACHE !== undefined;
}
public static get lfsCacheFolderFull() {
return path.join(CloudRunnerFolders.cacheFolderForCacheKeyFull, `lfs`);
}
public static get libraryCacheFolderFull() {
return path.join(CloudRunnerFolders.cacheFolderForCacheKeyFull, `Library`);
}
public static get unityBuilderRepoUrl(): string {
return `https://${CloudRunner.buildParameters.gitPrivateToken}@github.com/game-ci/unity-builder.git`;
}
public static get targetBuildRepoUrl(): string {
return `https://${CloudRunner.buildParameters.gitPrivateToken}@github.com/${CloudRunner.buildParameters.githubRepo}.git`;
}
public static get buildVolumeFolder() {
return 'data';
}
public static get cacheFolder() {
return 'cache';
}
}

View File

@@ -1,10 +0,0 @@
import Input from '../../input';
import CloudRunnerOptions from './cloud-runner-options';
class CloudRunnerOptionsReader {
static GetProperties() {
return [...Object.getOwnPropertyNames(Input), ...Object.getOwnPropertyNames(CloudRunnerOptions)];
}
}
export default CloudRunnerOptionsReader;

View File

@@ -1,283 +0,0 @@
import { Cli } from '../../cli/cli';
import CloudRunnerQueryOverride from './cloud-runner-query-override';
import GitHub from '../../github';
import * as core from '@actions/core';
class CloudRunnerOptions {
// ### ### ###
// Input Handling
// ### ### ###
public static getInput(query: string): string | undefined {
if (GitHub.githubInputEnabled) {
const coreInput = core.getInput(query);
if (coreInput && coreInput !== '') {
return coreInput;
}
}
const alternativeQuery = CloudRunnerOptions.ToEnvVarFormat(query);
// Query input sources
if (Cli.query(query, alternativeQuery)) {
return Cli.query(query, alternativeQuery);
}
if (CloudRunnerQueryOverride.query(query, alternativeQuery)) {
return CloudRunnerQueryOverride.query(query, alternativeQuery);
}
if (process.env[query] !== undefined) {
return process.env[query];
}
if (alternativeQuery !== query && process.env[alternativeQuery] !== undefined) {
return process.env[alternativeQuery];
}
}
public static ToEnvVarFormat(input: string): string {
if (input.toUpperCase() === input) {
return input;
}
return input
.replace(/([A-Z])/g, ' $1')
.trim()
.toUpperCase()
.replace(/ /g, '_');
}
// ### ### ###
// Provider parameters
// ### ### ###
static get region(): string {
return CloudRunnerOptions.getInput('region') || 'eu-west-2';
}
// ### ### ###
// GitHub parameters
// ### ### ###
static get githubChecks(): boolean {
const value = CloudRunnerOptions.getInput('githubChecks');
return value === `true` || false;
}
static get githubCheckId(): string {
return CloudRunnerOptions.getInput('githubCheckId') || ``;
}
static get githubOwner(): string {
return CloudRunnerOptions.getInput('githubOwner') || CloudRunnerOptions.githubRepo?.split(`/`)[0] || '';
}
static get githubRepoName(): string {
return CloudRunnerOptions.getInput('githubRepoName') || CloudRunnerOptions.githubRepo?.split(`/`)[1] || '';
}
static get finalHooks(): string[] {
return CloudRunnerOptions.getInput('finalHooks')?.split(',') || [];
}
// ### ### ###
// Git syncronization parameters
// ### ### ###
static get githubRepo(): string | undefined {
return CloudRunnerOptions.getInput('GITHUB_REPOSITORY') || CloudRunnerOptions.getInput('GITHUB_REPO') || undefined;
}
static get branch(): string {
if (CloudRunnerOptions.getInput(`GITHUB_REF`)) {
return (
CloudRunnerOptions.getInput(`GITHUB_REF`)?.replace('refs/', '').replace(`head/`, '').replace(`heads/`, '') || ``
);
} else if (CloudRunnerOptions.getInput('branch')) {
return CloudRunnerOptions.getInput('branch') || ``;
} else {
return '';
}
}
// ### ### ###
// Cloud Runner parameters
// ### ### ###
static get buildPlatform(): string {
const input = CloudRunnerOptions.getInput('buildPlatform');
if (input && input !== '') {
return input;
}
if (CloudRunnerOptions.providerStrategy !== 'local') {
return 'linux';
}
return process.platform;
}
static get cloudRunnerBranch(): string {
return CloudRunnerOptions.getInput('cloudRunnerBranch') || 'main';
}
static get providerStrategy(): string {
const provider =
CloudRunnerOptions.getInput('cloudRunnerCluster') || CloudRunnerOptions.getInput('providerStrategy');
if (Cli.isCliMode) {
return provider || 'aws';
}
return provider || 'local';
}
static get containerCpu(): string {
return CloudRunnerOptions.getInput('containerCpu') || `1024`;
}
static get containerMemory(): string {
return CloudRunnerOptions.getInput('containerMemory') || `3072`;
}
static get customJob(): string {
return CloudRunnerOptions.getInput('customJob') || '';
}
// ### ### ###
// Custom commands from files parameters
// ### ### ###
static get containerHookFiles(): string[] {
return CloudRunnerOptions.getInput('containerHookFiles')?.split(`,`) || [];
}
static get commandHookFiles(): string[] {
return CloudRunnerOptions.getInput('commandHookFiles')?.split(`,`) || [];
}
// ### ### ###
// Custom commands from yaml parameters
// ### ### ###
static get commandHooks(): string {
return CloudRunnerOptions.getInput('commandHooks') || '';
}
static get postBuildContainerHooks(): string {
return CloudRunnerOptions.getInput('postBuildContainerHooks') || '';
}
static get preBuildContainerHooks(): string {
return CloudRunnerOptions.getInput('preBuildContainerHooks') || '';
}
// ### ### ###
// Input override handling
// ### ### ###
static get pullInputList(): string[] {
return CloudRunnerOptions.getInput('pullInputList')?.split(`,`) || [];
}
static get inputPullCommand(): string {
const value = CloudRunnerOptions.getInput('inputPullCommand');
if (value === 'gcp-secret-manager') {
return 'gcloud secrets versions access 1 --secret="{0}"';
} else if (value === 'aws-secret-manager') {
return 'aws secretsmanager get-secret-value --secret-id {0}';
}
return value || '';
}
// ### ### ###
// Aws
// ### ### ###
static get awsStackName() {
return CloudRunnerOptions.getInput('awsStackName') || 'game-ci';
}
// ### ### ###
// K8s
// ### ### ###
static get kubeConfig(): string {
return CloudRunnerOptions.getInput('kubeConfig') || '';
}
static get kubeVolume(): string {
return CloudRunnerOptions.getInput('kubeVolume') || '';
}
static get kubeVolumeSize(): string {
return CloudRunnerOptions.getInput('kubeVolumeSize') || '25Gi';
}
static get kubeStorageClass(): string {
return CloudRunnerOptions.getInput('kubeStorageClass') || '';
}
// ### ### ###
// Caching
// ### ### ###
static get cacheKey(): string {
return CloudRunnerOptions.getInput('cacheKey') || CloudRunnerOptions.branch;
}
// ### ### ###
// Utility Parameters
// ### ### ###
static get cloudRunnerDebug(): boolean {
return (
CloudRunnerOptions.getInput(`cloudRunnerTests`) === `true` ||
CloudRunnerOptions.getInput(`cloudRunnerDebug`) === `true` ||
CloudRunnerOptions.getInput(`cloudRunnerDebugTree`) === `true` ||
CloudRunnerOptions.getInput(`cloudRunnerDebugEnv`) === `true` ||
false
);
}
static get skipLfs(): boolean {
return CloudRunnerOptions.getInput(`skipLfs`) === `true`;
}
static get skipCache(): boolean {
return CloudRunnerOptions.getInput(`skipCache`) === `true`;
}
public static get asyncCloudRunner(): boolean {
return CloudRunnerOptions.getInput('asyncCloudRunner') === 'true';
}
public static get useLargePackages(): boolean {
return CloudRunnerOptions.getInput(`useLargePackages`) === `true`;
}
public static get useSharedBuilder(): boolean {
return CloudRunnerOptions.getInput(`useSharedBuilder`) === `true`;
}
public static get useCompressionStrategy(): boolean {
return CloudRunnerOptions.getInput(`useCompressionStrategy`) === `true`;
}
public static get useCleanupCron(): boolean {
return (CloudRunnerOptions.getInput(`useCleanupCron`) || 'true') === 'true';
}
// ### ### ###
// Retained Workspace
// ### ### ###
public static get maxRetainedWorkspaces(): string {
return CloudRunnerOptions.getInput(`maxRetainedWorkspaces`) || `0`;
}
// ### ### ###
// Garbage Collection
// ### ### ###
static get garbageMaxAge(): number {
return Number(CloudRunnerOptions.getInput(`garbageMaxAge`)) || 24;
}
}
export default CloudRunnerOptions;

View File

@@ -1,67 +0,0 @@
import Input from '../../input';
import { GenericInputReader } from '../../input-readers/generic-input-reader';
import CloudRunnerOptions from './cloud-runner-options';
const formatFunction = (value: string, arguments_: any[]) => {
for (const element of arguments_) {
value = value.replace(`{${element.key}}`, element.value);
}
return value;
};
class CloudRunnerQueryOverride {
static queryOverrides: { [key: string]: string } | undefined;
// TODO accept premade secret sources or custom secret source definition yamls
public static query(key: string, alternativeKey: string) {
if (CloudRunnerQueryOverride.queryOverrides && CloudRunnerQueryOverride.queryOverrides[key] !== undefined) {
return CloudRunnerQueryOverride.queryOverrides[key];
}
if (
CloudRunnerQueryOverride.queryOverrides &&
alternativeKey &&
CloudRunnerQueryOverride.queryOverrides[alternativeKey] !== undefined
) {
return CloudRunnerQueryOverride.queryOverrides[alternativeKey];
}
return;
}
private static shouldUseOverride(query: string) {
if (CloudRunnerOptions.inputPullCommand !== '') {
if (CloudRunnerOptions.pullInputList.length > 0) {
const doesInclude =
CloudRunnerOptions.pullInputList.includes(query) ||
CloudRunnerOptions.pullInputList.includes(Input.ToEnvVarFormat(query));
return doesInclude ? true : false;
} else {
return true;
}
}
}
private static async queryOverride(query: string) {
if (!this.shouldUseOverride(query)) {
throw new Error(`Should not be trying to run override query on ${query}`);
}
return await GenericInputReader.Run(
formatFunction(CloudRunnerOptions.inputPullCommand, [{ key: 0, value: query }]),
);
}
public static async PopulateQueryOverrideInput() {
const queries = CloudRunnerOptions.pullInputList;
CloudRunnerQueryOverride.queryOverrides = {};
for (const element of queries) {
if (CloudRunnerQueryOverride.shouldUseOverride(element)) {
CloudRunnerQueryOverride.queryOverrides[element] = await CloudRunnerQueryOverride.queryOverride(element);
}
}
}
}
export default CloudRunnerQueryOverride;

View File

@@ -1,3 +0,0 @@
export class CloudRunnerStatics {
public static readonly logPrefix = `Cloud-Runner`;
}

View File

@@ -1,13 +0,0 @@
import CloudRunnerEnvironmentVariable from './cloud-runner-environment-variable';
import CloudRunnerSecret from './cloud-runner-secret';
export class CloudRunnerStepParameters {
public image: string;
public environment: CloudRunnerEnvironmentVariable[];
public secrets: CloudRunnerSecret[];
constructor(image: string, environmentVariables: CloudRunnerEnvironmentVariable[], secrets: CloudRunnerSecret[]) {
this.image = image;
this.environment = environmentVariables;
this.secrets = secrets;
}
}

View File

@@ -1,127 +0,0 @@
import { V1EnvVar, V1EnvVarSource, V1SecretKeySelector } from '@kubernetes/client-node';
import BuildParameters from '../../../build-parameters';
import { CommandHookService } from '../../services/hooks/command-hook-service';
import CloudRunnerEnvironmentVariable from '../../options/cloud-runner-environment-variable';
import CloudRunnerSecret from '../../options/cloud-runner-secret';
import CloudRunner from '../../cloud-runner';
class KubernetesJobSpecFactory {
static getJobSpec(
command: string,
image: string,
mountdir: string,
workingDirectory: string,
environment: CloudRunnerEnvironmentVariable[],
secrets: CloudRunnerSecret[],
buildGuid: string,
buildParameters: BuildParameters,
secretName: string,
pvcName: string,
jobName: string,
k8s: any,
containerName: string,
ip: string = '',
) {
const job = new k8s.V1Job();
job.apiVersion = 'batch/v1';
job.kind = 'Job';
job.metadata = {
name: jobName,
labels: {
app: 'unity-builder',
buildGuid,
},
};
job.spec = {
ttlSecondsAfterFinished: 9999,
backoffLimit: 0,
template: {
spec: {
volumes: [
{
name: 'build-mount',
persistentVolumeClaim: {
claimName: pvcName,
},
},
],
containers: [
{
ttlSecondsAfterFinished: 9999,
name: containerName,
image,
command: ['/bin/sh'],
args: [
'-c',
`${CommandHookService.ApplyHooksToCommands(`${command}\nsleep 2m`, CloudRunner.buildParameters)}`,
],
workingDir: `${workingDirectory}`,
resources: {
requests: {
memory: `${Number.parseInt(buildParameters.containerMemory) / 1024}G` || '750M',
cpu: Number.parseInt(buildParameters.containerCpu) / 1024 || '1',
},
},
env: [
...environment.map((x) => {
const environmentVariable = new V1EnvVar();
environmentVariable.name = x.name;
environmentVariable.value = x.value;
return environmentVariable;
}),
...secrets.map((x) => {
const secret = new V1EnvVarSource();
secret.secretKeyRef = new V1SecretKeySelector();
secret.secretKeyRef.key = x.ParameterKey;
secret.secretKeyRef.name = secretName;
const environmentVariable = new V1EnvVar();
environmentVariable.name = x.EnvironmentVariable;
environmentVariable.valueFrom = secret;
return environmentVariable;
}),
{ name: 'LOG_SERVICE_IP', value: ip },
],
volumeMounts: [
{
name: 'build-mount',
mountPath: `${mountdir}`,
},
],
lifecycle: {
preStop: {
exec: {
command: [
`wait 60s;
cd /data/builder/action/steps;
chmod +x /return_license.sh;
/return_license.sh;`,
],
},
},
},
},
],
restartPolicy: 'Never',
},
},
};
if (process.env['CLOUD_RUNNER_MINIKUBE']) {
job.spec.template.spec.volumes[0] = {
name: 'build-mount',
hostPath: {
path: `/data`,
type: `Directory`,
},
};
}
job.spec.template.spec.containers[0].resources.requests[`ephemeral-storage`] = '10Gi';
return job;
}
}
export default KubernetesJobSpecFactory;

View File

@@ -1,23 +0,0 @@
import CloudRunnerLogger from '../../services/core/cloud-runner-logger';
import { CoreV1Api } from '@kubernetes/client-node';
class KubernetesPods {
public static async IsPodRunning(podName: string, namespace: string, kubeClient: CoreV1Api) {
const pods = (await kubeClient.listNamespacedPod(namespace)).body.items.filter((x) => podName === x.metadata?.name);
const running = pods.length > 0 && (pods[0].status?.phase === `Running` || pods[0].status?.phase === `Pending`);
const phase = pods[0]?.status?.phase || 'undefined status';
CloudRunnerLogger.log(`Getting pod status: ${phase}`);
if (phase === `Failed`) {
throw new Error(`K8s pod failed`);
}
return running;
}
public static async GetPodStatus(podName: string, namespace: string, kubeClient: CoreV1Api) {
const pods = (await kubeClient.listNamespacedPod(namespace)).body.items.find((x) => podName === x.metadata?.name);
const phase = pods?.status?.phase || 'undefined status';
return phase;
}
}
export default KubernetesPods;

View File

@@ -1,116 +0,0 @@
import { waitUntil } from 'async-wait-until';
import * as core from '@actions/core';
import * as k8s from '@kubernetes/client-node';
import BuildParameters from '../../../build-parameters';
import CloudRunnerLogger from '../../services/core/cloud-runner-logger';
import { IncomingMessage } from 'node:http';
import GitHub from '../../../github';
class KubernetesStorage {
public static async createPersistentVolumeClaim(
buildParameters: BuildParameters,
pvcName: string,
kubeClient: k8s.CoreV1Api,
namespace: string,
) {
if (buildParameters.kubeVolume !== ``) {
CloudRunnerLogger.log(`Kube Volume was input was set ${buildParameters.kubeVolume} overriding ${pvcName}`);
pvcName = buildParameters.kubeVolume;
return;
}
const allPvc = (await kubeClient.listNamespacedPersistentVolumeClaim(namespace)).body.items;
const pvcList = allPvc.map((x) => x.metadata?.name);
CloudRunnerLogger.log(`Current PVCs in namespace ${namespace}`);
CloudRunnerLogger.log(JSON.stringify(pvcList, undefined, 4));
if (pvcList.includes(pvcName)) {
CloudRunnerLogger.log(`pvc ${pvcName} already exists`);
if (GitHub.githubInputEnabled) {
core.setOutput('volume', pvcName);
}
return;
}
CloudRunnerLogger.log(`Creating PVC ${pvcName} (does not exist)`);
const result = await KubernetesStorage.createPVC(pvcName, buildParameters, kubeClient, namespace);
await KubernetesStorage.handleResult(result, kubeClient, namespace, pvcName);
}
public static async getPVCPhase(kubeClient: k8s.CoreV1Api, name: string, namespace: string) {
try {
return (await kubeClient.readNamespacedPersistentVolumeClaim(name, namespace)).body.status?.phase;
} catch (error) {
core.error('Failed to get PVC phase');
core.error(JSON.stringify(error, undefined, 4));
throw error;
}
}
public static async watchUntilPVCNotPending(kubeClient: k8s.CoreV1Api, name: string, namespace: string) {
try {
CloudRunnerLogger.log(`watch Until PVC Not Pending ${name} ${namespace}`);
CloudRunnerLogger.log(`${await this.getPVCPhase(kubeClient, name, namespace)}`);
await waitUntil(
async () => {
return (await this.getPVCPhase(kubeClient, name, namespace)) === 'Pending';
},
{
timeout: 750000,
intervalBetweenAttempts: 15000,
},
);
} catch (error: any) {
core.error('Failed to watch PVC');
core.error(error.toString());
core.error(
`PVC Body: ${JSON.stringify(
(await kubeClient.readNamespacedPersistentVolumeClaim(name, namespace)).body,
undefined,
4,
)}`,
);
throw error;
}
}
private static async createPVC(
pvcName: string,
buildParameters: BuildParameters,
kubeClient: k8s.CoreV1Api,
namespace: string,
) {
const pvc = new k8s.V1PersistentVolumeClaim();
pvc.apiVersion = 'v1';
pvc.kind = 'PersistentVolumeClaim';
pvc.metadata = {
name: pvcName,
};
pvc.spec = {
accessModes: ['ReadWriteOnce'],
storageClassName: buildParameters.kubeStorageClass === '' ? 'standard' : buildParameters.kubeStorageClass,
resources: {
requests: {
storage: buildParameters.kubeVolumeSize,
},
},
};
const result = await kubeClient.createNamespacedPersistentVolumeClaim(namespace, pvc);
return result;
}
private static async handleResult(
result: { response: IncomingMessage; body: k8s.V1PersistentVolumeClaim },
kubeClient: k8s.CoreV1Api,
namespace: string,
pvcName: string,
) {
const name = result.body.metadata?.name || '';
CloudRunnerLogger.log(`PVC ${name} created`);
await this.watchUntilPVCNotPending(kubeClient, name, namespace);
CloudRunnerLogger.log(`PVC ${name} is ready and not pending`);
core.setOutput('volume', pvcName);
}
}
export default KubernetesStorage;

View File

@@ -1,119 +0,0 @@
import { CoreV1Api, KubeConfig } from '@kubernetes/client-node';
import CloudRunnerLogger from '../../services/core/cloud-runner-logger';
import { waitUntil } from 'async-wait-until';
import { CloudRunnerSystem } from '../../services/core/cloud-runner-system';
import CloudRunner from '../../cloud-runner';
import KubernetesPods from './kubernetes-pods';
import { FollowLogStreamService } from '../../services/core/follow-log-stream-service';
class KubernetesTaskRunner {
static readonly maxRetry: number = 3;
static lastReceivedMessage: string = ``;
static async runTask(
kubeConfig: KubeConfig,
kubeClient: CoreV1Api,
jobName: string,
podName: string,
containerName: string,
namespace: string,
) {
let output = '';
let shouldReadLogs = true;
let shouldCleanup = true;
let retriesAfterFinish = 0;
// eslint-disable-next-line no-constant-condition
while (true) {
await new Promise((resolve) => setTimeout(resolve, 3000));
CloudRunnerLogger.log(
`Streaming logs from pod: ${podName} container: ${containerName} namespace: ${namespace} ${CloudRunner.buildParameters.kubeVolumeSize}/${CloudRunner.buildParameters.containerCpu}/${CloudRunner.buildParameters.containerMemory}`,
);
let extraFlags = ``;
extraFlags += (await KubernetesPods.IsPodRunning(podName, namespace, kubeClient))
? ` -f -c ${containerName}`
: ` --previous`;
const callback = (outputChunk: string) => {
output += outputChunk;
// split output chunk and handle per line
for (const chunk of outputChunk.split(`\n`)) {
({ shouldReadLogs, shouldCleanup, output } = FollowLogStreamService.handleIteration(
chunk,
shouldReadLogs,
shouldCleanup,
output,
));
}
};
try {
await CloudRunnerSystem.Run(`kubectl logs ${podName}${extraFlags}`, false, true, callback);
} catch (error: any) {
await new Promise((resolve) => setTimeout(resolve, 3000));
const continueStreaming = await KubernetesPods.IsPodRunning(podName, namespace, kubeClient);
CloudRunnerLogger.log(`K8s logging error ${error} ${continueStreaming}`);
if (continueStreaming) {
continue;
}
if (retriesAfterFinish < KubernetesTaskRunner.maxRetry) {
retriesAfterFinish++;
continue;
}
throw error;
}
if (FollowLogStreamService.DidReceiveEndOfTransmission) {
CloudRunnerLogger.log('end of log stream');
break;
}
}
return output;
}
static async watchUntilPodRunning(kubeClient: CoreV1Api, podName: string, namespace: string) {
let waitComplete: boolean = false;
let message = ``;
CloudRunnerLogger.log(`Watching ${podName} ${namespace}`);
await waitUntil(
async () => {
const status = await kubeClient.readNamespacedPodStatus(podName, namespace);
const phase = status?.body.status?.phase;
waitComplete = phase !== 'Pending';
message = `Phase:${status.body.status?.phase} \n Reason:${
status.body.status?.conditions?.[0].reason || ''
} \n Message:${status.body.status?.conditions?.[0].message || ''}`;
// CloudRunnerLogger.log(
// JSON.stringify(
// (await kubeClient.listNamespacedEvent(namespace)).body.items
// .map((x) => {
// return {
// message: x.message || ``,
// name: x.metadata.name || ``,
// reason: x.reason || ``,
// };
// })
// .filter((x) => x.name.includes(podName)),
// undefined,
// 4,
// ),
// );
if (waitComplete || phase !== 'Pending') return true;
return false;
},
{
timeout: 2000000,
intervalBetweenAttempts: 15000,
},
);
if (!waitComplete) {
CloudRunnerLogger.log(message);
}
return waitComplete;
}
}
export default KubernetesTaskRunner;

View File

@@ -1,181 +0,0 @@
import { assert } from 'node:console';
import fs from 'node:fs';
import path from 'node:path';
import CloudRunner from '../cloud-runner';
import CloudRunnerLogger from '../services/core/cloud-runner-logger';
import { CloudRunnerFolders } from '../options/cloud-runner-folders';
import { CloudRunnerSystem } from '../services/core/cloud-runner-system';
import { LfsHashing } from '../services/utility/lfs-hashing';
import { RemoteClientLogger } from './remote-client-logger';
import { Cli } from '../../cli/cli';
import { CliFunction } from '../../cli/cli-functions-repository';
// eslint-disable-next-line github/no-then
const fileExists = async (fpath: fs.PathLike) => !!(await fs.promises.stat(fpath).catch(() => false));
export class Caching {
@CliFunction(`cache-push`, `push to cache`)
static async cachePush() {
try {
const buildParameter = JSON.parse(process.env.BUILD_PARAMETERS || '{}');
CloudRunner.buildParameters = buildParameter;
await Caching.PushToCache(
Cli.options!['cachePushTo'],
Cli.options!['cachePushFrom'],
Cli.options!['artifactName'] || '',
);
} catch (error: any) {
CloudRunnerLogger.log(`${error}`);
}
}
@CliFunction(`cache-pull`, `pull from cache`)
static async cachePull() {
try {
const buildParameter = JSON.parse(process.env.BUILD_PARAMETERS || '{}');
CloudRunner.buildParameters = buildParameter;
await Caching.PullFromCache(
Cli.options!['cachePushFrom'],
Cli.options!['cachePushTo'],
Cli.options!['artifactName'] || '',
);
} catch (error: any) {
CloudRunnerLogger.log(`${error}`);
}
}
public static async PushToCache(cacheFolder: string, sourceFolder: string, cacheArtifactName: string) {
CloudRunnerLogger.log(`Pushing to cache ${sourceFolder}`);
cacheArtifactName = cacheArtifactName.replace(' ', '');
const startPath = process.cwd();
let compressionSuffix = '';
if (CloudRunner.buildParameters.useCompressionStrategy === true) {
compressionSuffix = `.lz4`;
}
CloudRunnerLogger.log(`Compression: ${CloudRunner.buildParameters.useCompressionStrategy} ${compressionSuffix}`);
try {
if (!(await fileExists(cacheFolder))) {
await CloudRunnerSystem.Run(`mkdir -p ${cacheFolder}`);
}
process.chdir(path.resolve(sourceFolder, '..'));
if (CloudRunner.buildParameters.cloudRunnerDebug === true) {
CloudRunnerLogger.log(
`Hashed cache folder ${await LfsHashing.hashAllFiles(sourceFolder)} ${sourceFolder} ${path.basename(
sourceFolder,
)}`,
);
}
const contents = await fs.promises.readdir(path.basename(sourceFolder));
CloudRunnerLogger.log(
`There is ${contents.length} files/dir in the source folder ${path.basename(sourceFolder)}`,
);
if (contents.length === 0) {
CloudRunnerLogger.log(
`Did not push source folder to cache because it was empty ${path.basename(sourceFolder)}`,
);
process.chdir(`${startPath}`);
return;
}
await CloudRunnerSystem.Run(
`tar -cf ${cacheArtifactName}.tar${compressionSuffix} "${path.basename(sourceFolder)}"`,
);
await CloudRunnerSystem.Run(`du ${cacheArtifactName}.tar${compressionSuffix}`);
assert(await fileExists(`${cacheArtifactName}.tar${compressionSuffix}`), 'cache archive exists');
assert(await fileExists(path.basename(sourceFolder)), 'source folder exists');
await CloudRunnerSystem.Run(`mv ${cacheArtifactName}.tar${compressionSuffix} ${cacheFolder}`);
RemoteClientLogger.log(`moved cache entry ${cacheArtifactName} to ${cacheFolder}`);
assert(
await fileExists(`${path.join(cacheFolder, cacheArtifactName)}.tar${compressionSuffix}`),
'cache archive exists inside cache folder',
);
} catch (error) {
process.chdir(`${startPath}`);
throw error;
}
process.chdir(`${startPath}`);
}
public static async PullFromCache(cacheFolder: string, destinationFolder: string, cacheArtifactName: string = ``) {
CloudRunnerLogger.log(`Pulling from cache ${destinationFolder} ${CloudRunner.buildParameters.skipCache}`);
if (`${CloudRunner.buildParameters.skipCache}` === `true`) {
CloudRunnerLogger.log(`Skipping cache debugSkipCache is true`);
return;
}
cacheArtifactName = cacheArtifactName.replace(' ', '');
let compressionSuffix = '';
if (CloudRunner.buildParameters.useCompressionStrategy === true) {
compressionSuffix = `.lz4`;
}
const startPath = process.cwd();
RemoteClientLogger.log(`Caching for (lz4 ${compressionSuffix}) ${path.basename(destinationFolder)}`);
try {
if (!(await fileExists(cacheFolder))) {
await fs.promises.mkdir(cacheFolder);
}
if (!(await fileExists(destinationFolder))) {
await fs.promises.mkdir(destinationFolder);
}
const latestInBranch = await (
await CloudRunnerSystem.Run(`ls -t "${cacheFolder}" | grep .tar${compressionSuffix}$ | head -1`)
)
.replace(/\n/g, ``)
.replace(`.tar${compressionSuffix}`, '');
process.chdir(cacheFolder);
const cacheSelection =
cacheArtifactName !== `` && (await fileExists(`${cacheArtifactName}.tar${compressionSuffix}`))
? cacheArtifactName
: latestInBranch;
await CloudRunnerLogger.log(`cache key ${cacheArtifactName} selection ${cacheSelection}`);
if (await fileExists(`${cacheSelection}.tar${compressionSuffix}`)) {
const resultsFolder = `results${CloudRunner.buildParameters.buildGuid}`;
await CloudRunnerSystem.Run(`mkdir -p ${resultsFolder}`);
RemoteClientLogger.log(`cache item exists ${cacheFolder}/${cacheSelection}.tar${compressionSuffix}`);
const fullResultsFolder = path.join(cacheFolder, resultsFolder);
await CloudRunnerSystem.Run(`tar -xf ${cacheSelection}.tar${compressionSuffix} -C ${fullResultsFolder}`);
RemoteClientLogger.log(`cache item extracted to ${fullResultsFolder}`);
assert(await fileExists(fullResultsFolder), `cache extraction results folder exists`);
const destinationParentFolder = path.resolve(destinationFolder, '..');
if (await fileExists(destinationFolder)) {
await fs.promises.rmdir(destinationFolder, { recursive: true });
}
await CloudRunnerSystem.Run(
`mv "${path.join(fullResultsFolder, path.basename(destinationFolder))}" "${destinationParentFolder}"`,
);
const contents = await fs.promises.readdir(
path.join(destinationParentFolder, path.basename(destinationFolder)),
);
CloudRunnerLogger.log(
`There is ${contents.length} files/dir in the cache pulled contents for ${path.basename(destinationFolder)}`,
);
} else {
RemoteClientLogger.logWarning(`cache item ${cacheArtifactName} doesn't exist ${destinationFolder}`);
if (cacheSelection !== ``) {
RemoteClientLogger.logWarning(
`cache item ${cacheArtifactName}.tar${compressionSuffix} doesn't exist ${destinationFolder}`,
);
throw new Error(`Failed to get cache item, but cache hit was found: ${cacheSelection}`);
}
}
} catch (error) {
process.chdir(startPath);
throw error;
}
process.chdir(startPath);
}
public static async handleCachePurging() {
if (process.env.PURGE_REMOTE_BUILDER_CACHE !== undefined) {
RemoteClientLogger.log(`purging ${CloudRunnerFolders.purgeRemoteCaching}`);
fs.promises.rmdir(CloudRunnerFolders.cacheFolder, { recursive: true });
}
}
}

View File

@@ -1,251 +0,0 @@
import fs from 'node:fs';
import CloudRunner from '../cloud-runner';
import { CloudRunnerFolders } from '../options/cloud-runner-folders';
import { Caching } from './caching';
import { LfsHashing } from '../services/utility/lfs-hashing';
import { RemoteClientLogger } from './remote-client-logger';
import path from 'node:path';
import { assert } from 'node:console';
import CloudRunnerLogger from '../services/core/cloud-runner-logger';
import { CliFunction } from '../../cli/cli-functions-repository';
import { CloudRunnerSystem } from '../services/core/cloud-runner-system';
import YAML from 'yaml';
import GitHub from '../../github';
import BuildParameters from '../../build-parameters';
import { Cli } from '../../cli/cli';
import CloudRunnerOptions from '../options/cloud-runner-options';
export class RemoteClient {
@CliFunction(`remote-cli-pre-build`, `sets up a repository, usually before a game-ci build`)
static async setupRemoteClient() {
CloudRunnerLogger.log(`bootstrap game ci cloud runner...`);
if (!(await RemoteClient.handleRetainedWorkspace())) {
await RemoteClient.bootstrapRepository();
}
await RemoteClient.replaceLargePackageReferencesWithSharedReferences();
await RemoteClient.runCustomHookFiles(`before-build`);
}
@CliFunction('remote-cli-log-stream', `log stream from standard input`)
public static async remoteClientLogStream() {
const logFile = Cli.options!['logFile'];
process.stdin.resume();
process.stdin.setEncoding('utf8');
let lingeringLine = '';
process.stdin.on('data', (chunk) => {
const lines = chunk.toString().split('\n');
lines[0] = lingeringLine + lines[0];
lingeringLine = lines.pop() || '';
for (const element of lines) {
if (CloudRunnerOptions.providerStrategy !== 'k8s') {
CloudRunnerLogger.log(element);
} else {
fs.appendFileSync(logFile, element);
CloudRunnerLogger.log(element);
}
}
});
process.stdin.on('end', () => {
if (CloudRunnerOptions.providerStrategy !== 'k8s') {
CloudRunnerLogger.log(lingeringLine);
} else {
fs.appendFileSync(logFile, lingeringLine);
CloudRunnerLogger.log(lingeringLine);
}
});
}
@CliFunction(`remote-cli-post-build`, `runs a cloud runner build`)
public static async remoteClientPostBuild(): Promise<string> {
RemoteClientLogger.log(`Running POST build tasks`);
await Caching.PushToCache(
CloudRunnerFolders.ToLinuxFolder(`${CloudRunnerFolders.cacheFolderForCacheKeyFull}/Library`),
CloudRunnerFolders.ToLinuxFolder(CloudRunnerFolders.libraryFolderAbsolute),
`lib-${CloudRunner.buildParameters.buildGuid}`,
);
await Caching.PushToCache(
CloudRunnerFolders.ToLinuxFolder(`${CloudRunnerFolders.cacheFolderForCacheKeyFull}/build`),
CloudRunnerFolders.ToLinuxFolder(CloudRunnerFolders.projectBuildFolderAbsolute),
`build-${CloudRunner.buildParameters.buildGuid}`,
);
if (!BuildParameters.shouldUseRetainedWorkspaceMode(CloudRunner.buildParameters)) {
await CloudRunnerSystem.Run(
`rm -r ${CloudRunnerFolders.ToLinuxFolder(CloudRunnerFolders.uniqueCloudRunnerJobFolderAbsolute)}`,
);
}
await RemoteClient.runCustomHookFiles(`after-build`);
// WIP - need to give the pod permissions to create config map
await RemoteClientLogger.handleLogManagementPostJob();
return new Promise((result) => result(``));
}
static async runCustomHookFiles(hookLifecycle: string) {
RemoteClientLogger.log(`RunCustomHookFiles: ${hookLifecycle}`);
const gameCiCustomHooksPath = path.join(CloudRunnerFolders.repoPathAbsolute, `game-ci`, `hooks`);
try {
const files = fs.readdirSync(gameCiCustomHooksPath);
for (const file of files) {
const fileContents = fs.readFileSync(path.join(gameCiCustomHooksPath, file), `utf8`);
const fileContentsObject = YAML.parse(fileContents.toString());
if (fileContentsObject.hook === hookLifecycle) {
RemoteClientLogger.log(`Active Hook File ${file} \n \n file contents: \n ${fileContents}`);
await CloudRunnerSystem.Run(fileContentsObject.commands);
}
}
} catch (error) {
RemoteClientLogger.log(JSON.stringify(error, undefined, 4));
}
}
public static async bootstrapRepository() {
await CloudRunnerSystem.Run(
`mkdir -p ${CloudRunnerFolders.ToLinuxFolder(CloudRunnerFolders.uniqueCloudRunnerJobFolderAbsolute)}`,
);
await CloudRunnerSystem.Run(
`mkdir -p ${CloudRunnerFolders.ToLinuxFolder(CloudRunnerFolders.cacheFolderForCacheKeyFull)}`,
);
await RemoteClient.cloneRepoWithoutLFSFiles();
await RemoteClient.sizeOfFolder(
'repo before lfs cache pull',
CloudRunnerFolders.ToLinuxFolder(CloudRunnerFolders.repoPathAbsolute),
);
const lfsHashes = await LfsHashing.createLFSHashFiles();
if (fs.existsSync(CloudRunnerFolders.libraryFolderAbsolute)) {
RemoteClientLogger.logWarning(`!Warning!: The Unity library was included in the git repository`);
}
await Caching.PullFromCache(
CloudRunnerFolders.ToLinuxFolder(CloudRunnerFolders.lfsCacheFolderFull),
CloudRunnerFolders.ToLinuxFolder(CloudRunnerFolders.lfsFolderAbsolute),
`${lfsHashes.lfsGuidSum}`,
);
await RemoteClient.sizeOfFolder('repo after lfs cache pull', CloudRunnerFolders.repoPathAbsolute);
await RemoteClient.pullLatestLFS();
await RemoteClient.sizeOfFolder('repo before lfs git pull', CloudRunnerFolders.repoPathAbsolute);
await Caching.PushToCache(
CloudRunnerFolders.ToLinuxFolder(CloudRunnerFolders.lfsCacheFolderFull),
CloudRunnerFolders.ToLinuxFolder(CloudRunnerFolders.lfsFolderAbsolute),
`${lfsHashes.lfsGuidSum}`,
);
await Caching.PullFromCache(
CloudRunnerFolders.ToLinuxFolder(CloudRunnerFolders.libraryCacheFolderFull),
CloudRunnerFolders.ToLinuxFolder(CloudRunnerFolders.libraryFolderAbsolute),
);
await RemoteClient.sizeOfFolder('repo after library cache pull', CloudRunnerFolders.repoPathAbsolute);
await Caching.handleCachePurging();
}
private static async sizeOfFolder(message: string, folder: string) {
if (CloudRunner.buildParameters.cloudRunnerDebug) {
CloudRunnerLogger.log(`Size of ${message}`);
await CloudRunnerSystem.Run(`du -sh ${folder}`);
}
}
private static async cloneRepoWithoutLFSFiles() {
process.chdir(`${CloudRunnerFolders.uniqueCloudRunnerJobFolderAbsolute}`);
if (
fs.existsSync(CloudRunnerFolders.repoPathAbsolute) &&
!fs.existsSync(path.join(CloudRunnerFolders.repoPathAbsolute, `.git`))
) {
await CloudRunnerSystem.Run(`rm -r ${CloudRunnerFolders.repoPathAbsolute}`);
CloudRunnerLogger.log(`${CloudRunnerFolders.repoPathAbsolute} repo exists, but no git folder, cleaning up`);
}
if (
BuildParameters.shouldUseRetainedWorkspaceMode(CloudRunner.buildParameters) &&
fs.existsSync(path.join(CloudRunnerFolders.repoPathAbsolute, `.git`))
) {
process.chdir(CloudRunnerFolders.repoPathAbsolute);
RemoteClientLogger.log(
`${
CloudRunnerFolders.repoPathAbsolute
} repo exists - skipping clone - retained workspace mode ${BuildParameters.shouldUseRetainedWorkspaceMode(
CloudRunner.buildParameters,
)}`,
);
await CloudRunnerSystem.Run(`git fetch && git reset --hard ${CloudRunner.buildParameters.gitSha}`);
return;
}
RemoteClientLogger.log(`Initializing source repository for cloning with caching of LFS files`);
await CloudRunnerSystem.Run(`git config --global advice.detachedHead false`);
RemoteClientLogger.log(`Cloning the repository being built:`);
await CloudRunnerSystem.Run(`git config --global filter.lfs.smudge "git-lfs smudge --skip -- %f"`);
await CloudRunnerSystem.Run(`git config --global filter.lfs.process "git-lfs filter-process --skip"`);
try {
await CloudRunnerSystem.Run(
`git clone ${CloudRunnerFolders.targetBuildRepoUrl} ${path.basename(CloudRunnerFolders.repoPathAbsolute)}`,
);
} catch (error: any) {
throw error;
}
process.chdir(CloudRunnerFolders.repoPathAbsolute);
await CloudRunnerSystem.Run(`git lfs install`);
assert(fs.existsSync(`.git`), 'git folder exists');
RemoteClientLogger.log(`${CloudRunner.buildParameters.branch}`);
if (CloudRunner.buildParameters.gitSha !== undefined) {
await CloudRunnerSystem.Run(`git checkout ${CloudRunner.buildParameters.gitSha}`);
} else {
await CloudRunnerSystem.Run(`git checkout ${CloudRunner.buildParameters.branch}`);
RemoteClientLogger.log(`buildParameter Git Sha is empty`);
}
assert(fs.existsSync(path.join(`.git`, `lfs`)), 'LFS folder should not exist before caching');
RemoteClientLogger.log(`Checked out ${CloudRunner.buildParameters.branch}`);
}
static async replaceLargePackageReferencesWithSharedReferences() {
CloudRunnerLogger.log(`Use Shared Pkgs ${CloudRunner.buildParameters.useLargePackages}`);
GitHub.updateGitHubCheck(`Use Shared Pkgs ${CloudRunner.buildParameters.useLargePackages}`, ``);
if (CloudRunner.buildParameters.useLargePackages) {
const filePath = path.join(CloudRunnerFolders.projectPathAbsolute, `Packages/manifest.json`);
let manifest = fs.readFileSync(filePath, 'utf8');
manifest = manifest.replace(/LargeContent/g, '../../../LargeContent');
fs.writeFileSync(filePath, manifest);
CloudRunnerLogger.log(`Package Manifest \n ${manifest}`);
GitHub.updateGitHubCheck(`Package Manifest \n ${manifest}`, ``);
}
}
private static async pullLatestLFS() {
process.chdir(CloudRunnerFolders.repoPathAbsolute);
await CloudRunnerSystem.Run(`git config --global filter.lfs.smudge "git-lfs smudge -- %f"`);
await CloudRunnerSystem.Run(`git config --global filter.lfs.process "git-lfs filter-process"`);
if (!CloudRunner.buildParameters.skipLfs) {
await CloudRunnerSystem.Run(`git lfs pull`);
RemoteClientLogger.log(`pulled latest LFS files`);
assert(fs.existsSync(CloudRunnerFolders.lfsFolderAbsolute));
}
}
static async handleRetainedWorkspace() {
RemoteClientLogger.log(
`Retained Workspace: ${BuildParameters.shouldUseRetainedWorkspaceMode(CloudRunner.buildParameters)}`,
);
if (
BuildParameters.shouldUseRetainedWorkspaceMode(CloudRunner.buildParameters) &&
fs.existsSync(CloudRunnerFolders.ToLinuxFolder(CloudRunnerFolders.uniqueCloudRunnerJobFolderAbsolute)) &&
fs.existsSync(CloudRunnerFolders.ToLinuxFolder(path.join(CloudRunnerFolders.repoPathAbsolute, `.git`)))
) {
CloudRunnerLogger.log(`Retained Workspace Already Exists!`);
process.chdir(CloudRunnerFolders.ToLinuxFolder(CloudRunnerFolders.repoPathAbsolute));
await CloudRunnerSystem.Run(`git fetch`);
await CloudRunnerSystem.Run(`git lfs pull`);
await CloudRunnerSystem.Run(`git reset --hard "${CloudRunner.buildParameters.gitSha}"`);
await CloudRunnerSystem.Run(`git checkout ${CloudRunner.buildParameters.gitSha}`);
return true;
}
return false;
}
}

View File

@@ -1,82 +0,0 @@
import CloudRunnerLogger from '../services/core/cloud-runner-logger';
import fs from 'node:fs';
import path from 'node:path';
import CloudRunner from '../cloud-runner';
import CloudRunnerOptions from '../options/cloud-runner-options';
export class RemoteClientLogger {
private static get LogFilePath() {
return path.join(`/home`, `job-log.txt`);
}
public static log(message: string) {
const finalMessage = `[Client] ${message}`;
this.appendToFile(finalMessage);
CloudRunnerLogger.log(finalMessage);
}
public static logCliError(message: string) {
CloudRunnerLogger.log(`[Client][Error] ${message}`);
}
public static logCliDiagnostic(message: string) {
CloudRunnerLogger.log(`[Client][Diagnostic] ${message}`);
}
public static logWarning(message: string) {
CloudRunnerLogger.logWarning(message);
}
public static appendToFile(message: string) {
if (CloudRunner.isCloudRunnerEnvironment) {
fs.appendFileSync(RemoteClientLogger.LogFilePath, `${message}\n`);
}
}
public static async handleLogManagementPostJob() {
if (CloudRunnerOptions.providerStrategy !== 'k8s') {
return;
}
CloudRunnerLogger.log(`Collected Logs`);
// check for log file not existing
if (!fs.existsSync(RemoteClientLogger.LogFilePath)) {
CloudRunnerLogger.log(`Log file does not exist`);
// check if CloudRunner.isCloudRunnerEnvironment is true, log
if (!CloudRunner.isCloudRunnerEnvironment) {
CloudRunnerLogger.log(`Cloud Runner is not running in a cloud environment, not collecting logs`);
}
return;
}
CloudRunnerLogger.log(`Log file exist`);
await new Promise((resolve) => setTimeout(resolve, 1));
// let hashedLogs = fs.readFileSync(RemoteClientLogger.LogFilePath).toString();
//
// hashedLogs = md5(hashedLogs);
//
// for (let index = 0; index < 3; index++) {
// CloudRunnerLogger.log(`LOGHASH: ${hashedLogs}`);
// const logs = fs.readFileSync(RemoteClientLogger.LogFilePath).toString();
// CloudRunnerLogger.log(`LOGS: ${Buffer.from(logs).toString('base64')}`);
// CloudRunnerLogger.log(
// `Game CI's "Cloud Runner System" will cancel the log when it has successfully received the log data to verify all logs have been received.`,
// );
//
// // wait for 15 seconds to allow the log to be sent
// await new Promise((resolve) => setTimeout(resolve, 15000));
// }
}
public static HandleLog(message: string): boolean {
if (RemoteClientLogger.value !== '') {
RemoteClientLogger.value += `\n`;
}
RemoteClientLogger.value += message;
return false;
}
static value: string = '';
}

View File

@@ -1,265 +0,0 @@
import YAML from 'yaml';
import CloudRunner from '../../cloud-runner';
import { CustomWorkflow } from '../../workflows/custom-workflow';
import { RemoteClientLogger } from '../../remote-client/remote-client-logger';
import path from 'node:path';
import fs from 'node:fs';
import Input from '../../../input';
import CloudRunnerOptions from '../../options/cloud-runner-options';
import { ContainerHook as ContainerHook } from './container-hook';
import { CloudRunnerStepParameters } from '../../options/cloud-runner-step-parameters';
export class ContainerHookService {
static GetContainerHooksFromFiles(hookLifecycle: string): ContainerHook[] {
const results: ContainerHook[] = [];
try {
const gameCiCustomStepsPath = path.join(process.cwd(), `game-ci`, `container-hooks`);
const files = fs.readdirSync(gameCiCustomStepsPath);
for (const file of files) {
if (!CloudRunnerOptions.containerHookFiles.includes(file.replace(`.yaml`, ``))) {
// RemoteClientLogger.log(`Skipping CustomStepFile: ${file}`);
continue;
}
const fileContents = fs.readFileSync(path.join(gameCiCustomStepsPath, file), `utf8`);
const fileContentsObject = ContainerHookService.ParseContainerHooks(fileContents)[0];
if (fileContentsObject.hook === hookLifecycle) {
results.push(fileContentsObject);
}
}
} catch (error) {
RemoteClientLogger.log(`Failed Getting: ${hookLifecycle} \n ${JSON.stringify(error, undefined, 4)}`);
}
// RemoteClientLogger.log(`Active Steps From Files: \n ${JSON.stringify(results, undefined, 4)}`);
const builtInContainerHooks: ContainerHook[] = ContainerHookService.ParseContainerHooks(
`- name: aws-s3-upload-build
image: amazon/aws-cli
hook: after
commands: |
aws configure set aws_access_key_id $AWS_ACCESS_KEY_ID --profile default
aws configure set aws_secret_access_key $AWS_SECRET_ACCESS_KEY --profile default
aws configure set region $AWS_DEFAULT_REGION --profile default
aws s3 cp /data/cache/$CACHE_KEY/build/build-${CloudRunner.buildParameters.buildGuid}.tar${
CloudRunner.buildParameters.useCompressionStrategy ? '.lz4' : ''
} s3://${CloudRunner.buildParameters.awsStackName}/cloud-runner-cache/$CACHE_KEY/build/build-$BUILD_GUID.tar${
CloudRunner.buildParameters.useCompressionStrategy ? '.lz4' : ''
}
rm /data/cache/$CACHE_KEY/build/build-${CloudRunner.buildParameters.buildGuid}.tar${
CloudRunner.buildParameters.useCompressionStrategy ? '.lz4' : ''
}
secrets:
- name: awsAccessKeyId
value: ${process.env.AWS_ACCESS_KEY_ID || ``}
- name: awsSecretAccessKey
value: ${process.env.AWS_SECRET_ACCESS_KEY || ``}
- name: awsDefaultRegion
value: ${process.env.AWS_REGION || ``}
- name: aws-s3-pull-build
image: amazon/aws-cli
commands: |
aws configure set aws_access_key_id $AWS_ACCESS_KEY_ID --profile default
aws configure set aws_secret_access_key $AWS_SECRET_ACCESS_KEY --profile default
aws configure set region $AWS_DEFAULT_REGION --profile default
aws s3 ls ${CloudRunner.buildParameters.awsStackName}/cloud-runner-cache/ || true
aws s3 ls ${CloudRunner.buildParameters.awsStackName}/cloud-runner-cache/$CACHE_KEY/build || true
mkdir -p /data/cache/$CACHE_KEY/build/
aws s3 cp s3://${
CloudRunner.buildParameters.awsStackName
}/cloud-runner-cache/$CACHE_KEY/build/build-$BUILD_GUID_TARGET.tar${
CloudRunner.buildParameters.useCompressionStrategy ? '.lz4' : ''
} /data/cache/$CACHE_KEY/build/build-$BUILD_GUID_TARGET.tar${
CloudRunner.buildParameters.useCompressionStrategy ? '.lz4' : ''
}
secrets:
- name: AWS_ACCESS_KEY_ID
- name: AWS_SECRET_ACCESS_KEY
- name: AWS_DEFAULT_REGION
- name: BUILD_GUID_TARGET
- name: steam-deploy-client
image: steamcmd/steamcmd
commands: |
apt-get update
apt-get install -y curl tar coreutils git tree > /dev/null
curl -s https://gist.githubusercontent.com/frostebite/1d56f5505b36b403b64193b7a6e54cdc/raw/fa6639ed4ef750c4268ea319d63aa80f52712ffb/deploy-client-steam.sh | bash
secrets:
- name: STEAM_USERNAME
- name: STEAM_PASSWORD
- name: STEAM_APPID
- name: STEAM_SSFN_FILE_NAME
- name: STEAM_SSFN_FILE_CONTENTS
- name: STEAM_CONFIG_VDF_1
- name: STEAM_CONFIG_VDF_2
- name: STEAM_CONFIG_VDF_3
- name: STEAM_CONFIG_VDF_4
- name: BUILD_GUID_TARGET
- name: RELEASE_BRANCH
- name: steam-deploy-project
image: steamcmd/steamcmd
commands: |
apt-get update
apt-get install -y curl tar coreutils git tree > /dev/null
curl -s https://gist.githubusercontent.com/frostebite/969da6a41002a0e901174124b643709f/raw/02403e53fb292026cba81ddcf4ff35fc1eba111d/steam-deploy-project.sh | bash
secrets:
- name: STEAM_USERNAME
- name: STEAM_PASSWORD
- name: STEAM_APPID
- name: STEAM_SSFN_FILE_NAME
- name: STEAM_SSFN_FILE_CONTENTS
- name: STEAM_CONFIG_VDF_1
- name: STEAM_CONFIG_VDF_2
- name: STEAM_CONFIG_VDF_3
- name: STEAM_CONFIG_VDF_4
- name: BUILD_GUID_2
- name: RELEASE_BRANCH
- name: aws-s3-upload-cache
image: amazon/aws-cli
hook: after
commands: |
aws configure set aws_access_key_id $AWS_ACCESS_KEY_ID --profile default
aws configure set aws_secret_access_key $AWS_SECRET_ACCESS_KEY --profile default
aws configure set region $AWS_DEFAULT_REGION --profile default
aws s3 cp --recursive /data/cache/$CACHE_KEY/lfs s3://${
CloudRunner.buildParameters.awsStackName
}/cloud-runner-cache/$CACHE_KEY/lfs
rm -r /data/cache/$CACHE_KEY/lfs
aws s3 cp --recursive /data/cache/$CACHE_KEY/Library s3://${
CloudRunner.buildParameters.awsStackName
}/cloud-runner-cache/$CACHE_KEY/Library
rm -r /data/cache/$CACHE_KEY/Library
secrets:
- name: AWS_ACCESS_KEY_ID
value: ${process.env.AWS_ACCESS_KEY_ID || ``}
- name: AWS_SECRET_ACCESS_KEY
value: ${process.env.AWS_SECRET_ACCESS_KEY || ``}
- name: AWS_DEFAULT_REGION
value: ${process.env.AWS_REGION || ``}
- name: aws-s3-pull-cache
image: amazon/aws-cli
hook: before
commands: |
aws configure set aws_access_key_id $AWS_ACCESS_KEY_ID --profile default
aws configure set aws_secret_access_key $AWS_SECRET_ACCESS_KEY --profile default
aws configure set region $AWS_DEFAULT_REGION --profile default
mkdir -p /data/cache/$CACHE_KEY/Library/
mkdir -p /data/cache/$CACHE_KEY/lfs/
aws s3 ls ${CloudRunner.buildParameters.awsStackName}/cloud-runner-cache/ || true
aws s3 ls ${CloudRunner.buildParameters.awsStackName}/cloud-runner-cache/$CACHE_KEY/ || true
BUCKET1="${CloudRunner.buildParameters.awsStackName}/cloud-runner-cache/$CACHE_KEY/Library/"
aws s3 ls $BUCKET1 || true
OBJECT1="$(aws s3 ls $BUCKET1 | sort | tail -n 1 | awk '{print $4}' || '')"
aws s3 cp s3://$BUCKET1$OBJECT1 /data/cache/$CACHE_KEY/Library/ || true
BUCKET2="${CloudRunner.buildParameters.awsStackName}/cloud-runner-cache/$CACHE_KEY/lfs/"
aws s3 ls $BUCKET2 || true
OBJECT2="$(aws s3 ls $BUCKET2 | sort | tail -n 1 | awk '{print $4}' || '')"
aws s3 cp s3://$BUCKET2$OBJECT2 /data/cache/$CACHE_KEY/lfs/ || true
secrets:
- name: AWS_ACCESS_KEY_ID
value: ${process.env.AWS_ACCESS_KEY_ID || ``}
- name: AWS_SECRET_ACCESS_KEY
value: ${process.env.AWS_SECRET_ACCESS_KEY || ``}
- name: AWS_DEFAULT_REGION
value: ${process.env.AWS_REGION || ``}
- name: debug-cache
image: ubuntu
hook: after
commands: |
apt-get update > /dev/null
${CloudRunnerOptions.cloudRunnerDebug ? `apt-get install -y tree > /dev/null` : `#`}
${CloudRunnerOptions.cloudRunnerDebug ? `tree -L 3 /data/cache` : `#`}
secrets:
- name: awsAccessKeyId
value: ${process.env.AWS_ACCESS_KEY_ID || ``}
- name: awsSecretAccessKey
value: ${process.env.AWS_SECRET_ACCESS_KEY || ``}
- name: awsDefaultRegion
value: ${process.env.AWS_REGION || ``}`,
).filter((x) => CloudRunnerOptions.containerHookFiles.includes(x.name) && x.hook === hookLifecycle);
if (builtInContainerHooks.length > 0) {
results.push(...builtInContainerHooks);
}
return results;
}
private static ConvertYamlSecrets(object: ContainerHook) {
if (object.secrets === undefined) {
object.secrets = [];
return;
}
object.secrets = object.secrets.map((x: { [key: string]: any }) => {
return {
ParameterKey: x.name,
EnvironmentVariable: Input.ToEnvVarFormat(x.name),
ParameterValue: x.value,
};
});
}
public static ParseContainerHooks(steps: string): ContainerHook[] {
if (steps === '') {
return [];
}
const isArray = steps.replace(/\s/g, ``)[0] === `-`;
const object: ContainerHook[] = isArray ? YAML.parse(steps) : [YAML.parse(steps)];
for (const step of object) {
ContainerHookService.ConvertYamlSecrets(step);
if (step.secrets === undefined) {
step.secrets = [];
} else {
for (const secret of step.secrets) {
if (secret.ParameterValue === undefined && process.env[secret.EnvironmentVariable] !== undefined) {
if (CloudRunner.buildParameters?.cloudRunnerDebug) {
// CloudRunnerLogger.log(`Injecting custom step ${step.name} from env var ${secret.ParameterKey}`);
}
secret.ParameterValue = process.env[secret.ParameterKey] || ``;
}
}
}
if (step.image === undefined) {
step.image = `ubuntu`;
}
}
if (object === undefined) {
throw new Error(`Failed to parse ${steps}`);
}
return object;
}
static async RunPostBuildSteps(cloudRunnerStepState: CloudRunnerStepParameters) {
let output = ``;
const steps: ContainerHook[] = [
...ContainerHookService.ParseContainerHooks(CloudRunner.buildParameters.postBuildContainerHooks),
...ContainerHookService.GetContainerHooksFromFiles(`after`),
];
if (steps.length > 0) {
output += await CustomWorkflow.runContainerJob(
steps,
cloudRunnerStepState.environment,
cloudRunnerStepState.secrets,
);
}
return output;
}
static async RunPreBuildSteps(cloudRunnerStepState: CloudRunnerStepParameters) {
let output = ``;
const steps: ContainerHook[] = [
...ContainerHookService.ParseContainerHooks(CloudRunner.buildParameters.preBuildContainerHooks),
...ContainerHookService.GetContainerHooksFromFiles(`before`),
];
if (steps.length > 0) {
output += await CustomWorkflow.runContainerJob(
steps,
cloudRunnerStepState.environment,
cloudRunnerStepState.secrets,
);
}
return output;
}
}

View File

@@ -1,9 +0,0 @@
import CloudRunnerSecret from '../../options/cloud-runner-secret';
export class ContainerHook {
public commands!: string;
public secrets: CloudRunnerSecret[] = new Array<CloudRunnerSecret>();
public name!: string;
public image: string = `ubuntu`;
public hook!: string;
}

View File

@@ -1,48 +0,0 @@
import CloudRunner from '../cloud-runner';
import { BuildParameters, ImageTag } from '../..';
import UnityVersioning from '../../unity-versioning';
import { Cli } from '../../cli/cli';
import CloudRunnerLogger from '../services/core/cloud-runner-logger';
import { v4 as uuidv4 } from 'uuid';
import CloudRunnerOptions from '../options/cloud-runner-options';
import setups from './cloud-runner-suite.test';
import { CloudRunnerSystem } from '../services/core/cloud-runner-system';
import { OptionValues } from 'commander';
async function CreateParameters(overrides: OptionValues | undefined) {
if (overrides) {
Cli.options = overrides;
}
return await BuildParameters.create();
}
describe('Cloud Runner pre-built S3 steps', () => {
it('Responds', () => {});
setups();
if (CloudRunnerOptions.cloudRunnerDebug && CloudRunnerOptions.providerStrategy !== `local-docker`) {
it('Run build and prebuilt s3 cache pull, cache push and upload build', async () => {
const overrides = {
versioning: 'None',
projectPath: 'test-project',
unityVersion: UnityVersioning.determineUnityVersion('test-project', UnityVersioning.read('test-project')),
targetPlatform: 'StandaloneLinux64',
cacheKey: `test-case-${uuidv4()}`,
containerHookFiles: `aws-s3-pull-cache,aws-s3-upload-cache,aws-s3-upload-build`,
};
const buildParameter2 = await CreateParameters(overrides);
const baseImage2 = new ImageTag(buildParameter2);
const results2Object = await CloudRunner.run(buildParameter2, baseImage2.toString());
const results2 = results2Object.BuildResults;
CloudRunnerLogger.log(`run 2 succeeded`);
const build2ContainsBuildSucceeded = results2.includes('Build succeeded');
expect(build2ContainsBuildSucceeded).toBeTruthy();
const results = await CloudRunnerSystem.RunAndReadLines(
`aws s3 ls s3://${CloudRunner.buildParameters.awsStackName}/cloud-runner-cache/`,
);
CloudRunnerLogger.log(results.join(`,`));
}, 1_000_000_000);
}
});

View File

@@ -1,92 +0,0 @@
import CloudRunner from '../../cloud-runner';
import { BuildParameters, ImageTag } from '../../..';
import UnityVersioning from '../../../unity-versioning';
import { Cli } from '../../../cli/cli';
import CloudRunnerLogger from '../../services/core/cloud-runner-logger';
import { v4 as uuidv4 } from 'uuid';
import CloudRunnerOptions from '../../options/cloud-runner-options';
import setups from '../cloud-runner-suite.test';
import * as fs from 'node:fs';
import { CloudRunnerSystem } from '../../services/core/cloud-runner-system';
async function CreateParameters(overrides: any) {
if (overrides) {
Cli.options = overrides;
}
return await BuildParameters.create();
}
describe('Cloud Runner Caching', () => {
it('Responds', () => {});
setups();
if (CloudRunnerOptions.cloudRunnerDebug) {
it('Run one build it should not use cache, run subsequent build which should use cache', async () => {
const overrides = {
versioning: 'None',
image: 'ubuntu',
projectPath: 'test-project',
unityVersion: UnityVersioning.determineUnityVersion('test-project', UnityVersioning.read('test-project')),
targetPlatform: 'StandaloneLinux64',
cacheKey: `test-case-${uuidv4()}`,
containerHookFiles: `debug-cache`,
cloudRunnerBranch: `cloud-runner-develop`,
};
if (CloudRunnerOptions.providerStrategy === `k8s`) {
overrides.containerHookFiles += `,aws-s3-pull-cache,aws-s3-upload-cache`;
}
const buildParameter = await CreateParameters(overrides);
expect(buildParameter.projectPath).toEqual(overrides.projectPath);
const baseImage = new ImageTag(buildParameter);
const resultsObject = await CloudRunner.run(buildParameter, baseImage.toString());
const results = resultsObject.BuildResults;
const libraryString = 'Rebuilding Library because the asset database could not be found!';
const cachePushFail = 'Did not push source folder to cache because it was empty Library';
const buildSucceededString = 'Build succeeded';
expect(results).toContain(libraryString);
expect(results).toContain(buildSucceededString);
expect(results).not.toContain(cachePushFail);
CloudRunnerLogger.log(`run 1 succeeded`);
if (CloudRunnerOptions.providerStrategy === `local-docker`) {
await CloudRunnerSystem.Run(`tree ./cloud-runner-cache/cache`);
await CloudRunnerSystem.Run(
`cp ./cloud-runner-cache/cache/${buildParameter.cacheKey}/Library/lib-${buildParameter.buildGuid}.tar ./`,
);
await CloudRunnerSystem.Run(`mkdir results`);
await CloudRunnerSystem.Run(`tar -xf lib-${buildParameter.buildGuid}.tar -C ./results`);
await CloudRunnerSystem.Run(`tree -d ./results`);
const cacheFolderExists = fs.existsSync(`cloud-runner-cache/cache/${overrides.cacheKey}`);
expect(cacheFolderExists).toBeTruthy();
}
const buildParameter2 = await CreateParameters(overrides);
buildParameter2.cacheKey = buildParameter.cacheKey;
const baseImage2 = new ImageTag(buildParameter2);
const results2Object = await CloudRunner.run(buildParameter2, baseImage2.toString());
const results2 = results2Object.BuildResults;
CloudRunnerLogger.log(`run 2 succeeded`);
const build2ContainsCacheKey = results2.includes(buildParameter.cacheKey);
const build2ContainsBuildSucceeded = results2.includes(buildSucceededString);
const build2NotContainsZeroLibraryCacheFilesMessage = !results2.includes(
'There is 0 files/dir in the cache pulled contents for Library',
);
const build2NotContainsZeroLFSCacheFilesMessage = !results2.includes(
'There is 0 files/dir in the cache pulled contents for LFS',
);
expect(build2ContainsCacheKey).toBeTruthy();
expect(results2).toContain('Activation successful');
expect(build2ContainsBuildSucceeded).toBeTruthy();
expect(results2).toContain(buildSucceededString);
const splitResults = results2.split('Activation successful');
expect(splitResults[splitResults.length - 1]).not.toContain(libraryString);
expect(build2NotContainsZeroLibraryCacheFilesMessage).toBeTruthy();
expect(build2NotContainsZeroLFSCacheFilesMessage).toBeTruthy();
}, 1_000_000_000);
}
});

View File

@@ -1,92 +0,0 @@
import CloudRunner from '../../cloud-runner';
import { ImageTag } from '../../..';
import UnityVersioning from '../../../unity-versioning';
import CloudRunnerLogger from '../../services/core/cloud-runner-logger';
import { v4 as uuidv4 } from 'uuid';
import CloudRunnerOptions from '../../options/cloud-runner-options';
import setups from './../cloud-runner-suite.test';
import * as fs from 'node:fs';
import path from 'node:path';
import { CloudRunnerFolders } from '../../options/cloud-runner-folders';
import SharedWorkspaceLocking from '../../services/core/shared-workspace-locking';
import { CreateParameters } from '../create-test-parameter';
import { CloudRunnerSystem } from '../../services/core/cloud-runner-system';
describe('Cloud Runner Retain Workspace', () => {
it('Responds', () => {});
setups();
if (CloudRunnerOptions.cloudRunnerDebug) {
it('Run one build it should not already be retained, run subsequent build which should use retained workspace', async () => {
const overrides = {
versioning: 'None',
projectPath: 'test-project',
unityVersion: UnityVersioning.determineUnityVersion('test-project', UnityVersioning.read('test-project')),
targetPlatform: 'StandaloneLinux64',
cacheKey: `test-case-${uuidv4()}`,
maxRetainedWorkspaces: 1,
};
const buildParameter = await CreateParameters(overrides);
expect(buildParameter.projectPath).toEqual(overrides.projectPath);
const baseImage = new ImageTag(buildParameter);
const resultsObject = await CloudRunner.run(buildParameter, baseImage.toString());
const results = resultsObject.BuildResults;
const libraryString = 'Rebuilding Library because the asset database could not be found!';
const cachePushFail = 'Did not push source folder to cache because it was empty Library';
const buildSucceededString = 'Build succeeded';
expect(results).toContain(libraryString);
expect(results).toContain(buildSucceededString);
expect(results).not.toContain(cachePushFail);
if (CloudRunnerOptions.providerStrategy === `local-docker`) {
const cacheFolderExists = fs.existsSync(`cloud-runner-cache/cache/${overrides.cacheKey}`);
expect(cacheFolderExists).toBeTruthy();
await CloudRunnerSystem.Run(`tree -d ./cloud-runner-cache`);
}
CloudRunnerLogger.log(`run 1 succeeded`);
// await CloudRunnerSystem.Run(`tree -d ./cloud-runner-cache/${}`);
const buildParameter2 = await CreateParameters(overrides);
buildParameter2.cacheKey = buildParameter.cacheKey;
const baseImage2 = new ImageTag(buildParameter2);
const results2Object = await CloudRunner.run(buildParameter2, baseImage2.toString());
const results2 = results2Object.BuildResults;
CloudRunnerLogger.log(`run 2 succeeded`);
const build2ContainsCacheKey = results2.includes(buildParameter.cacheKey);
const build2ContainsBuildGuid1FromRetainedWorkspace = results2.includes(buildParameter.buildGuid);
const build2ContainsRetainedWorkspacePhrase = results2.includes(`Retained Workspace:`);
const build2ContainsWorkspaceExistsAlreadyPhrase = results2.includes(`Retained Workspace Already Exists!`);
const build2ContainsBuildSucceeded = results2.includes(buildSucceededString);
const build2NotContainsZeroLibraryCacheFilesMessage = !results2.includes(
'There is 0 files/dir in the cache pulled contents for Library',
);
const build2NotContainsZeroLFSCacheFilesMessage = !results2.includes(
'There is 0 files/dir in the cache pulled contents for LFS',
);
expect(build2ContainsCacheKey).toBeTruthy();
expect(build2ContainsRetainedWorkspacePhrase).toBeTruthy();
expect(build2ContainsWorkspaceExistsAlreadyPhrase).toBeTruthy();
expect(build2ContainsBuildGuid1FromRetainedWorkspace).toBeTruthy();
expect(build2ContainsBuildSucceeded).toBeTruthy();
expect(build2NotContainsZeroLibraryCacheFilesMessage).toBeTruthy();
expect(build2NotContainsZeroLFSCacheFilesMessage).toBeTruthy();
const splitResults = results2.split('Activation successful');
expect(splitResults[splitResults.length - 1]).not.toContain(libraryString);
}, 1_000_000_000);
afterAll(async () => {
await SharedWorkspaceLocking.CleanupWorkspace(CloudRunner.lockedWorkspace || ``, CloudRunner.buildParameters);
if (
fs.existsSync(`./cloud-runner-cache/${path.basename(CloudRunnerFolders.uniqueCloudRunnerJobFolderAbsolute)}`)
) {
CloudRunnerLogger.log(
`Cleaning up ./cloud-runner-cache/${path.basename(CloudRunnerFolders.uniqueCloudRunnerJobFolderAbsolute)}`,
);
}
});
}
});

View File

@@ -1,56 +0,0 @@
import CloudRunner from '../../cloud-runner';
import UnityVersioning from '../../../unity-versioning';
import { Cli } from '../../../cli/cli';
import CloudRunnerLogger from '../../services/core/cloud-runner-logger';
import { v4 as uuidv4 } from 'uuid';
import CloudRunnerOptions from '../../options/cloud-runner-options';
import setups from '../cloud-runner-suite.test';
import BuildParameters from '../../../build-parameters';
import ImageTag from '../../../image-tag';
async function CreateParameters(overrides: any) {
if (overrides) {
Cli.options = overrides;
}
return await BuildParameters.create();
}
describe('Cloud Runner Kubernetes', () => {
it('Responds', () => {});
setups();
if (CloudRunnerOptions.cloudRunnerDebug) {
it('Run one build it using K8s without error', async () => {
if (CloudRunnerOptions.providerStrategy !== `k8s`) {
return;
}
process.env.USE_IL2CPP = 'false';
const overrides = {
versioning: 'None',
projectPath: 'test-project',
unityVersion: UnityVersioning.determineUnityVersion('test-project', UnityVersioning.read('test-project')),
targetPlatform: 'StandaloneLinux64',
cacheKey: `test-case-${uuidv4()}`,
providerStrategy: 'k8s',
buildPlatform: 'linux',
};
const buildParameter = await CreateParameters(overrides);
expect(buildParameter.projectPath).toEqual(overrides.projectPath);
const baseImage = new ImageTag(buildParameter);
const resultsObject = await CloudRunner.run(buildParameter, baseImage.toString());
const results = resultsObject.BuildResults;
const libraryString = 'Rebuilding Library because the asset database could not be found!';
const cachePushFail = 'Did not push source folder to cache because it was empty Library';
const buildSucceededString = 'Build succeeded';
expect(results).toContain('Collected Logs');
expect(results).toContain(libraryString);
expect(results).toContain(buildSucceededString);
expect(results).not.toContain(cachePushFail);
CloudRunnerLogger.log(`run 1 succeeded`);
}, 1_000_000_000);
}
});

View File

@@ -1,114 +0,0 @@
import CloudRunnerLogger from '../services/core/cloud-runner-logger';
import { CloudRunnerFolders } from '../options/cloud-runner-folders';
import { CloudRunnerStepParameters } from '../options/cloud-runner-step-parameters';
import { WorkflowInterface } from './workflow-interface';
import { CommandHookService } from '../services/hooks/command-hook-service';
import path from 'node:path';
import CloudRunner from '../cloud-runner';
import { ContainerHookService } from '../services/hooks/container-hook-service';
export class BuildAutomationWorkflow implements WorkflowInterface {
async run(cloudRunnerStepState: CloudRunnerStepParameters) {
return await BuildAutomationWorkflow.standardBuildAutomation(cloudRunnerStepState.image, cloudRunnerStepState);
}
private static async standardBuildAutomation(baseImage: string, cloudRunnerStepState: CloudRunnerStepParameters) {
// TODO accept post and pre build steps as yaml files in the repo
CloudRunnerLogger.log(`Cloud Runner is running standard build automation`);
let output = '';
output += await ContainerHookService.RunPreBuildSteps(cloudRunnerStepState);
CloudRunnerLogger.logWithTime('Configurable pre build step(s) time');
CloudRunnerLogger.log(baseImage);
CloudRunnerLogger.logLine(` `);
CloudRunnerLogger.logLine('Starting build automation job');
output += await CloudRunner.Provider.runTaskInWorkflow(
CloudRunner.buildParameters.buildGuid,
baseImage.toString(),
BuildAutomationWorkflow.BuildWorkflow,
`/${CloudRunnerFolders.buildVolumeFolder}`,
`/${CloudRunnerFolders.buildVolumeFolder}/`,
cloudRunnerStepState.environment,
cloudRunnerStepState.secrets,
);
CloudRunnerLogger.logWithTime('Build time');
output += await ContainerHookService.RunPostBuildSteps(cloudRunnerStepState);
CloudRunnerLogger.logWithTime('Configurable post build step(s) time');
CloudRunnerLogger.log(`Cloud Runner finished running standard build automation`);
return output;
}
private static get BuildWorkflow() {
const setupHooks = CommandHookService.getHooks(CloudRunner.buildParameters.commandHooks).filter((x) =>
x.step?.includes(`setup`),
);
const buildHooks = CommandHookService.getHooks(CloudRunner.buildParameters.commandHooks).filter((x) =>
x.step?.includes(`build`),
);
const builderPath = CloudRunnerFolders.ToLinuxFolder(
path.join(CloudRunnerFolders.builderPathAbsolute, 'dist', `index.js`),
);
return `echo "cloud runner build workflow starting"
apt-get update > /dev/null
apt-get install -y curl tar tree npm git-lfs jq git > /dev/null
npm --version
npm i -g n > /dev/null
npm i -g semver > /dev/null
npm install --global yarn > /dev/null
n 20.8.0
node --version
${setupHooks.filter((x) => x.hook.includes(`before`)).map((x) => x.commands) || ' '}
export GITHUB_WORKSPACE="${CloudRunnerFolders.ToLinuxFolder(CloudRunnerFolders.repoPathAbsolute)}"
df -H /data/
${BuildAutomationWorkflow.setupCommands(builderPath)}
${setupHooks.filter((x) => x.hook.includes(`after`)).map((x) => x.commands) || ' '}
${buildHooks.filter((x) => x.hook.includes(`before`)).map((x) => x.commands) || ' '}
${BuildAutomationWorkflow.BuildCommands(builderPath)}
${buildHooks.filter((x) => x.hook.includes(`after`)).map((x) => x.commands) || ' '}`;
}
private static setupCommands(builderPath: string) {
const commands = `mkdir -p ${CloudRunnerFolders.ToLinuxFolder(
CloudRunnerFolders.builderPathAbsolute,
)} && git clone -q -b ${CloudRunner.buildParameters.cloudRunnerBranch} ${
CloudRunnerFolders.unityBuilderRepoUrl
} "${CloudRunnerFolders.ToLinuxFolder(CloudRunnerFolders.builderPathAbsolute)}" && chmod +x ${builderPath}`;
const cloneBuilderCommands = `if [ -e "${CloudRunnerFolders.ToLinuxFolder(
CloudRunnerFolders.uniqueCloudRunnerJobFolderAbsolute,
)}" ] && [ -e "${CloudRunnerFolders.ToLinuxFolder(
path.join(CloudRunnerFolders.builderPathAbsolute, `.git`),
)}" ] ; then echo "Builder Already Exists!" && tree ${
CloudRunnerFolders.builderPathAbsolute
}; else ${commands} ; fi`;
return `export GIT_DISCOVERY_ACROSS_FILESYSTEM=1
${cloneBuilderCommands}
echo "log start" >> /home/job-log.txt
node ${builderPath} -m remote-cli-pre-build`;
}
private static BuildCommands(builderPath: string) {
const distFolder = path.join(CloudRunnerFolders.builderPathAbsolute, 'dist');
const ubuntuPlatformsFolder = path.join(CloudRunnerFolders.builderPathAbsolute, 'dist', 'platforms', 'ubuntu');
return `
mkdir -p ${`${CloudRunnerFolders.ToLinuxFolder(CloudRunnerFolders.projectBuildFolderAbsolute)}/build`}
cd ${CloudRunnerFolders.ToLinuxFolder(CloudRunnerFolders.projectPathAbsolute)}
cp -r "${CloudRunnerFolders.ToLinuxFolder(path.join(distFolder, 'default-build-script'))}" "/UnityBuilderAction"
cp -r "${CloudRunnerFolders.ToLinuxFolder(path.join(ubuntuPlatformsFolder, 'entrypoint.sh'))}" "/entrypoint.sh"
cp -r "${CloudRunnerFolders.ToLinuxFolder(path.join(ubuntuPlatformsFolder, 'steps'))}" "/steps"
chmod -R +x "/entrypoint.sh"
chmod -R +x "/steps"
echo "game ci start"
echo "game ci start" >> /home/job-log.txt
/entrypoint.sh | node ${builderPath} -m remote-cli-log-stream --logFile /home/job-log.txt
node ${builderPath} -m remote-cli-post-build`;
}
}

View File

@@ -1,51 +0,0 @@
import CloudRunnerLogger from '../services/core/cloud-runner-logger';
import CloudRunnerSecret from '../options/cloud-runner-secret';
import { CloudRunnerFolders } from '../options/cloud-runner-folders';
import CloudRunnerEnvironmentVariable from '../options/cloud-runner-environment-variable';
import { ContainerHookService } from '../services/hooks/container-hook-service';
import { ContainerHook } from '../services/hooks/container-hook';
import CloudRunner from '../cloud-runner';
export class CustomWorkflow {
public static async runContainerJobFromString(
buildSteps: string,
environmentVariables: CloudRunnerEnvironmentVariable[],
secrets: CloudRunnerSecret[],
): Promise<string> {
return await CustomWorkflow.runContainerJob(
ContainerHookService.ParseContainerHooks(buildSteps),
environmentVariables,
secrets,
);
}
public static async runContainerJob(
steps: ContainerHook[],
environmentVariables: CloudRunnerEnvironmentVariable[],
secrets: CloudRunnerSecret[],
) {
try {
let output = '';
// if (CloudRunner.buildParameters?.cloudRunnerDebug) {
// CloudRunnerLogger.log(`Custom Job Description \n${JSON.stringify(buildSteps, undefined, 4)}`);
// }
for (const step of steps) {
CloudRunnerLogger.log(`Cloud Runner is running in custom job mode`);
output += await CloudRunner.Provider.runTaskInWorkflow(
CloudRunner.buildParameters.buildGuid,
step.image,
step.commands,
`/${CloudRunnerFolders.buildVolumeFolder}`,
`/${CloudRunnerFolders.projectPathAbsolute}/`,
environmentVariables,
[...secrets, ...step.secrets],
);
}
return output;
} catch (error) {
throw error;
}
}
}

View File

@@ -1,39 +0,0 @@
import { CloudRunnerStepParameters } from '../options/cloud-runner-step-parameters';
import { CustomWorkflow } from './custom-workflow';
import { WorkflowInterface } from './workflow-interface';
import { BuildAutomationWorkflow } from './build-automation-workflow';
import CloudRunner from '../cloud-runner';
import CloudRunnerOptions from '../options/cloud-runner-options';
import { AsyncWorkflow } from './async-workflow';
export class WorkflowCompositionRoot implements WorkflowInterface {
async run(cloudRunnerStepState: CloudRunnerStepParameters) {
try {
if (
CloudRunnerOptions.asyncCloudRunner &&
!CloudRunner.isCloudRunnerAsyncEnvironment &&
!CloudRunner.isCloudRunnerEnvironment
) {
return await AsyncWorkflow.runAsyncWorkflow(cloudRunnerStepState.environment, cloudRunnerStepState.secrets);
}
if (CloudRunner.buildParameters.customJob !== '') {
return await CustomWorkflow.runContainerJobFromString(
CloudRunner.buildParameters.customJob,
cloudRunnerStepState.environment,
cloudRunnerStepState.secrets,
);
}
return await new BuildAutomationWorkflow().run(
new CloudRunnerStepParameters(
cloudRunnerStepState.image.toString(),
cloudRunnerStepState.environment,
cloudRunnerStepState.secrets,
),
);
} catch (error) {
throw error;
}
}
}

View File

@@ -1,8 +0,0 @@
import { CloudRunnerStepParameters } from '../options/cloud-runner-step-parameters';
export interface WorkflowInterface {
run(
// eslint-disable-next-line no-unused-vars
cloudRunnerStepState: CloudRunnerStepParameters,
): Promise<string>;
}

View File

@@ -55,7 +55,10 @@ class Docker {
if (!existsSync(githubHome)) mkdirSync(githubHome);
const githubWorkflow = path.join(runnerTempPath, '_github_workflow');
if (!existsSync(githubWorkflow)) mkdirSync(githubWorkflow);
const commandPrefix = image === `alpine` ? `/bin/sh` : `/bin/bash`;
// Alpine-based images (alpine, rclone/rclone, etc.) don't have /bin/bash, only /bin/sh
const isAlpineBasedImage = image === 'alpine' || image.startsWith('rclone/');
const commandPrefix = isAlpineBasedImage ? `/bin/sh` : `/bin/bash`;
return `docker run \
--workdir ${dockerWorkspacePath} \

View File

@@ -0,0 +1,509 @@
/**
* Tests for enterprise input properties and their wiring into BuildParameters.
*
* Covers all 20 new input properties added for enterprise features:
* - Boolean inputs: localCacheEnabled, childWorkspacesEnabled, gitHooksEnabled,
* localCacheLibrary, localCacheLfs, childWorkspacePreserveGit, childWorkspaceSeparateLibrary
* - String inputs: submoduleProfilePath, submoduleVariantPath, submoduleToken,
* localCacheRoot, childWorkspaceName, childWorkspaceCacheRoot, lfsTransferAgent,
* lfsTransferAgentArgs, lfsStoragePaths, providerExecutable, gitHooksSkipList,
* gitHooksRunBeforeBuild
*
* Special attention to boolean inputs: GitHub Actions always passes inputs as strings,
* so 'false' must NOT evaluate as truthy (the #1 source of bugs).
*/
import * as core from '@actions/core';
import Input from './input';
import Versioning from './versioning';
import BuildParameters from './build-parameters';
// ---------------------------------------------------------------------------
// Setup
// ---------------------------------------------------------------------------
afterEach(() => {
jest.restoreAllMocks();
});
// ---------------------------------------------------------------------------
// Part 1: Input getters — defaults and explicit values
// ---------------------------------------------------------------------------
describe('Enterprise Input properties', () => {
// -----------------------------------------------------------------------
// Boolean inputs — default and string parsing
// -----------------------------------------------------------------------
describe('localCacheEnabled', () => {
it('returns false by default', () => {
expect(Input.localCacheEnabled).toBe(false);
});
it('returns true when string "true" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('true');
expect(Input.localCacheEnabled).toBe(true);
});
it('returns false when string "false" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('false');
expect(Input.localCacheEnabled).toBe(false);
});
it('returns false when empty string is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('');
expect(Input.localCacheEnabled).toBe(false);
});
});
describe('localCacheLibrary', () => {
it('returns true by default (library caching on by default when cache enabled)', () => {
expect(Input.localCacheLibrary).toBe(true);
});
it('returns true when string "true" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('true');
expect(Input.localCacheLibrary).toBe(true);
});
it('returns false when string "false" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('false');
expect(Input.localCacheLibrary).toBe(false);
});
});
describe('localCacheLfs', () => {
it('returns false by default', () => {
expect(Input.localCacheLfs).toBe(false);
});
it('returns true when string "true" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('true');
expect(Input.localCacheLfs).toBe(true);
});
it('returns false when string "false" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('false');
expect(Input.localCacheLfs).toBe(false);
});
});
describe('childWorkspacesEnabled', () => {
it('returns false by default', () => {
expect(Input.childWorkspacesEnabled).toBe(false);
});
it('returns true when string "true" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('true');
expect(Input.childWorkspacesEnabled).toBe(true);
});
it('returns false when string "false" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('false');
expect(Input.childWorkspacesEnabled).toBe(false);
});
it('returns false when empty string is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('');
expect(Input.childWorkspacesEnabled).toBe(false);
});
});
describe('childWorkspacePreserveGit', () => {
it('returns true by default', () => {
expect(Input.childWorkspacePreserveGit).toBe(true);
});
it('returns false when string "false" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('false');
expect(Input.childWorkspacePreserveGit).toBe(false);
});
it('returns true when string "true" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('true');
expect(Input.childWorkspacePreserveGit).toBe(true);
});
});
describe('childWorkspaceSeparateLibrary', () => {
it('returns true by default', () => {
expect(Input.childWorkspaceSeparateLibrary).toBe(true);
});
it('returns false when string "false" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('false');
expect(Input.childWorkspaceSeparateLibrary).toBe(false);
});
it('returns true when string "true" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('true');
expect(Input.childWorkspaceSeparateLibrary).toBe(true);
});
});
describe('gitHooksEnabled', () => {
it('returns false by default', () => {
expect(Input.gitHooksEnabled).toBe(false);
});
it('returns true when string "true" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('true');
expect(Input.gitHooksEnabled).toBe(true);
});
it('returns false when string "false" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('false');
expect(Input.gitHooksEnabled).toBe(false);
});
it('returns false when empty string is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('');
expect(Input.gitHooksEnabled).toBe(false);
});
});
// -----------------------------------------------------------------------
// Boolean truthiness edge cases — the #1 source of bugs
// -----------------------------------------------------------------------
describe('boolean input string handling (edge cases)', () => {
// These tests verify that the === 'true' comparison is correct.
// In JavaScript, 'false' is truthy when used in a boolean context,
// but the Input class correctly uses === 'true' comparison.
const booleanInputs: Array<{
name: string;
getter: () => boolean;
defaultValue: boolean;
}> = [
{ name: 'localCacheEnabled', getter: () => Input.localCacheEnabled, defaultValue: false },
{ name: 'localCacheLfs', getter: () => Input.localCacheLfs, defaultValue: false },
{ name: 'childWorkspacesEnabled', getter: () => Input.childWorkspacesEnabled, defaultValue: false },
{ name: 'gitHooksEnabled', getter: () => Input.gitHooksEnabled, defaultValue: false },
// These default to true:
{ name: 'localCacheLibrary', getter: () => Input.localCacheLibrary, defaultValue: true },
{ name: 'childWorkspacePreserveGit', getter: () => Input.childWorkspacePreserveGit, defaultValue: true },
{ name: 'childWorkspaceSeparateLibrary', getter: () => Input.childWorkspaceSeparateLibrary, defaultValue: true },
];
test.each(booleanInputs)('$name: "false" string does NOT evaluate as truthy', ({ getter }) => {
jest.spyOn(core, 'getInput').mockReturnValue('false');
expect(getter()).toBe(false);
});
test.each(booleanInputs)('$name: "true" string evaluates as truthy', ({ getter }) => {
jest.spyOn(core, 'getInput').mockReturnValue('true');
expect(getter()).toBe(true);
});
test.each(booleanInputs)('$name: "TRUE" (uppercase) does NOT evaluate as true (case sensitive)', ({ getter }) => {
jest.spyOn(core, 'getInput').mockReturnValue('TRUE');
expect(getter()).toBe(false);
});
test.each(booleanInputs)('$name: "1" does NOT evaluate as true', ({ getter }) => {
jest.spyOn(core, 'getInput').mockReturnValue('1');
expect(getter()).toBe(false);
});
test.each(booleanInputs)('$name: "yes" does NOT evaluate as true', ({ getter }) => {
jest.spyOn(core, 'getInput').mockReturnValue('yes');
expect(getter()).toBe(false);
});
});
// -----------------------------------------------------------------------
// String inputs — defaults and explicit values
// -----------------------------------------------------------------------
describe('submoduleProfilePath', () => {
it('returns empty string by default', () => {
expect(Input.submoduleProfilePath).toBe('');
});
it('takes input from workflow', () => {
jest.spyOn(core, 'getInput').mockReturnValue('config/submodule-profiles/tow/ec/profile.yml');
expect(Input.submoduleProfilePath).toBe('config/submodule-profiles/tow/ec/profile.yml');
});
});
describe('submoduleVariantPath', () => {
it('returns empty string by default', () => {
expect(Input.submoduleVariantPath).toBe('');
});
it('takes input from workflow', () => {
jest.spyOn(core, 'getInput').mockReturnValue('config/submodule-profiles/tow/ec/server.yml');
expect(Input.submoduleVariantPath).toBe('config/submodule-profiles/tow/ec/server.yml');
});
});
describe('submoduleToken', () => {
it('returns empty string by default', () => {
expect(Input.submoduleToken).toBe('');
});
it('takes input from workflow', () => {
jest.spyOn(core, 'getInput').mockReturnValue('ghp_abc123');
expect(Input.submoduleToken).toBe('ghp_abc123');
});
});
describe('localCacheRoot', () => {
it('returns empty string by default', () => {
expect(Input.localCacheRoot).toBe('');
});
it('takes input from workflow', () => {
jest.spyOn(core, 'getInput').mockReturnValue('/d/cache/game-ci');
expect(Input.localCacheRoot).toBe('/d/cache/game-ci');
});
});
describe('childWorkspaceName', () => {
it('returns empty string by default', () => {
expect(Input.childWorkspaceName).toBe('');
});
it('takes input from workflow', () => {
jest.spyOn(core, 'getInput').mockReturnValue('TurnOfWarEndlessCrusade');
expect(Input.childWorkspaceName).toBe('TurnOfWarEndlessCrusade');
});
});
describe('childWorkspaceCacheRoot', () => {
it('returns empty string by default', () => {
expect(Input.childWorkspaceCacheRoot).toBe('');
});
it('takes input from workflow', () => {
jest.spyOn(core, 'getInput').mockReturnValue('/d/workspaces');
expect(Input.childWorkspaceCacheRoot).toBe('/d/workspaces');
});
});
describe('lfsTransferAgent', () => {
it('returns empty string by default', () => {
expect(Input.lfsTransferAgent).toBe('');
});
it('takes input from workflow', () => {
jest.spyOn(core, 'getInput').mockReturnValue('/tools/elastic-git-storage');
expect(Input.lfsTransferAgent).toBe('/tools/elastic-git-storage');
});
});
describe('lfsTransferAgentArgs', () => {
it('returns empty string by default', () => {
expect(Input.lfsTransferAgentArgs).toBe('');
});
it('takes input from workflow', () => {
jest.spyOn(core, 'getInput').mockReturnValue('--verbose --timeout=60');
expect(Input.lfsTransferAgentArgs).toBe('--verbose --timeout=60');
});
});
describe('lfsStoragePaths', () => {
it('returns empty string by default', () => {
expect(Input.lfsStoragePaths).toBe('');
});
it('takes input from workflow', () => {
jest.spyOn(core, 'getInput').mockReturnValue('/storage/primary;/storage/secondary');
expect(Input.lfsStoragePaths).toBe('/storage/primary;/storage/secondary');
});
});
describe('providerExecutable', () => {
it('returns empty string by default', () => {
expect(Input.providerExecutable).toBe('');
});
it('takes input from workflow', () => {
jest.spyOn(core, 'getInput').mockReturnValue('/usr/local/bin/custom-provider');
expect(Input.providerExecutable).toBe('/usr/local/bin/custom-provider');
});
});
describe('gitHooksSkipList', () => {
it('returns empty string by default', () => {
expect(Input.gitHooksSkipList).toBe('');
});
it('takes input from workflow', () => {
jest.spyOn(core, 'getInput').mockReturnValue('pre-commit,pre-push');
expect(Input.gitHooksSkipList).toBe('pre-commit,pre-push');
});
});
describe('gitHooksRunBeforeBuild', () => {
it('returns empty string by default', () => {
expect(Input.gitHooksRunBeforeBuild).toBe('');
});
it('takes input from workflow', () => {
jest.spyOn(core, 'getInput').mockReturnValue('pre-commit');
expect(Input.gitHooksRunBeforeBuild).toBe('pre-commit');
});
});
});
// ---------------------------------------------------------------------------
// Part 2: BuildParameters.create() maps new inputs to properties
// ---------------------------------------------------------------------------
const testLicense =
'<?xml version="1.0" encoding="UTF-8"?><root>\n <License id="Terms">\n <MachineBindings>\n <Binding Key="1" Value="576562626572264761624c65526f7578"/>\n <Binding Key="2" Value="576562626572264761624c65526f7578"/>\n </MachineBindings>\n <MachineID Value="D7nTUnjNAmtsUMcnoyrqkgIbYdM="/>\n <SerialHash Value="2033b8ac3e6faa3742ca9f0bfae44d18f2a96b80"/>\n <Features>\n <Feature Value="33"/>\n <Feature Value="1"/>\n <Feature Value="12"/>\n <Feature Value="2"/>\n <Feature Value="24"/>\n <Feature Value="3"/>\n <Feature Value="36"/>\n <Feature Value="17"/>\n <Feature Value="19"/>\n <Feature Value="62"/>\n </Features>\n <DeveloperData Value="AQAAAEY0LUJHUlgtWEQ0RS1aQ1dWLUM1SlctR0RIQg=="/>\n <SerialMasked Value="F4-BGRX-XD4E-ZCWV-C5JW-XXXX"/>\n <StartDate Value="2021-02-08T00:00:00"/>\n <UpdateDate Value="2021-02-09T00:34:57"/>\n <InitialActivationDate Value="2021-02-08T00:34:56"/>\n <LicenseVersion Value="6.x"/>\n <ClientProvidedVersion Value="2018.4.30f1"/>\n <AlwaysOnline Value="false"/>\n <Entitlements>\n <Entitlement Ns="unity_editor" Tag="UnityPersonal" Type="EDITOR" ValidTo="9999-12-31T00:00:00"/>\n <Entitlement Ns="unity_editor" Tag="DarkSkin" Type="EDITOR_FEATURE" ValidTo="9999-12-31T00:00:00"/>\n </Entitlements>\n </License>\n<Signature xmlns="http://www.w3.org/2000/09/xmldsig#"><SignedInfo><CanonicalizationMethod Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315#WithComments"/><SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"/><Reference URI="#Terms"><Transforms><Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/></Transforms><DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/><DigestValue>m0Db8UK+ktnOLJBtHybkfetpcKo=</DigestValue></Reference></SignedInfo><SignatureValue>o/pUbSQAukz7+ZYAWhnA0AJbIlyyCPL7bKVEM2lVqbrXt7cyey+umkCXamuOgsWPVUKBMkXtMH8L\n5etLmD0getWIhTGhzOnDCk+gtIPfL4jMo9tkEuOCROQAXCci23VFscKcrkB+3X6h4wEOtA2APhOY\nB+wvC794o8/82ffjP79aVAi57rp3Wmzx+9pe9yMwoJuljAy2sc2tIMgdQGWVmOGBpQm3JqsidyzI\nJWG2kjnc7pDXK9pwYzXoKiqUqqrut90d+kQqRyv7MSZXR50HFqD/LI69h68b7P8Bjo3bPXOhNXGR\n9YCoemH6EkfCJxp2gIjzjWW+l2Hj2EsFQi8YXw==</SignatureValue></Signature></root>';
describe('BuildParameters.create() enterprise property mapping', () => {
beforeEach(() => {
jest.spyOn(Versioning, 'determineBuildVersion').mockImplementation(async () => '1.3.37');
process.env.UNITY_LICENSE = testLicense;
});
afterEach(() => {
jest.clearAllMocks();
jest.restoreAllMocks();
});
it('maps submoduleProfilePath from Input', async () => {
jest.spyOn(Input, 'submoduleProfilePath', 'get').mockReturnValue('/path/to/profile.yml');
const parameters = await BuildParameters.create();
expect(parameters.submoduleProfilePath).toBe('/path/to/profile.yml');
});
it('maps submoduleVariantPath from Input', async () => {
jest.spyOn(Input, 'submoduleVariantPath', 'get').mockReturnValue('/path/to/variant.yml');
const parameters = await BuildParameters.create();
expect(parameters.submoduleVariantPath).toBe('/path/to/variant.yml');
});
it('maps submoduleToken from Input', async () => {
jest.spyOn(Input, 'submoduleToken', 'get').mockReturnValue('ghp_token123');
const parameters = await BuildParameters.create();
expect(parameters.submoduleToken).toBe('ghp_token123');
});
it('maps localCacheEnabled from Input', async () => {
jest.spyOn(Input, 'localCacheEnabled', 'get').mockReturnValue(true);
const parameters = await BuildParameters.create();
expect(parameters.localCacheEnabled).toBe(true);
});
it('maps localCacheRoot from Input', async () => {
jest.spyOn(Input, 'localCacheRoot', 'get').mockReturnValue('/d/cache');
const parameters = await BuildParameters.create();
expect(parameters.localCacheRoot).toBe('/d/cache');
});
it('maps localCacheLibrary from Input', async () => {
jest.spyOn(Input, 'localCacheLibrary', 'get').mockReturnValue(false);
const parameters = await BuildParameters.create();
expect(parameters.localCacheLibrary).toBe(false);
});
it('maps localCacheLfs from Input', async () => {
jest.spyOn(Input, 'localCacheLfs', 'get').mockReturnValue(true);
const parameters = await BuildParameters.create();
expect(parameters.localCacheLfs).toBe(true);
});
it('maps childWorkspacesEnabled from Input', async () => {
jest.spyOn(Input, 'childWorkspacesEnabled', 'get').mockReturnValue(true);
const parameters = await BuildParameters.create();
expect(parameters.childWorkspacesEnabled).toBe(true);
});
it('maps childWorkspaceName from Input', async () => {
jest.spyOn(Input, 'childWorkspaceName', 'get').mockReturnValue('TurnOfWar');
const parameters = await BuildParameters.create();
expect(parameters.childWorkspaceName).toBe('TurnOfWar');
});
it('maps childWorkspaceCacheRoot from Input', async () => {
jest.spyOn(Input, 'childWorkspaceCacheRoot', 'get').mockReturnValue('/cache/workspaces');
const parameters = await BuildParameters.create();
expect(parameters.childWorkspaceCacheRoot).toBe('/cache/workspaces');
});
it('maps childWorkspacePreserveGit from Input', async () => {
jest.spyOn(Input, 'childWorkspacePreserveGit', 'get').mockReturnValue(false);
const parameters = await BuildParameters.create();
expect(parameters.childWorkspacePreserveGit).toBe(false);
});
it('maps childWorkspaceSeparateLibrary from Input', async () => {
jest.spyOn(Input, 'childWorkspaceSeparateLibrary', 'get').mockReturnValue(false);
const parameters = await BuildParameters.create();
expect(parameters.childWorkspaceSeparateLibrary).toBe(false);
});
it('maps lfsTransferAgent from Input', async () => {
jest.spyOn(Input, 'lfsTransferAgent', 'get').mockReturnValue('/tools/elastic-git-storage');
const parameters = await BuildParameters.create();
expect(parameters.lfsTransferAgent).toBe('/tools/elastic-git-storage');
});
it('maps lfsTransferAgentArgs from Input', async () => {
jest.spyOn(Input, 'lfsTransferAgentArgs', 'get').mockReturnValue('--verbose');
const parameters = await BuildParameters.create();
expect(parameters.lfsTransferAgentArgs).toBe('--verbose');
});
it('maps lfsStoragePaths from Input', async () => {
jest.spyOn(Input, 'lfsStoragePaths', 'get').mockReturnValue('/path/a;/path/b');
const parameters = await BuildParameters.create();
expect(parameters.lfsStoragePaths).toBe('/path/a;/path/b');
});
it('maps gitHooksEnabled from Input', async () => {
jest.spyOn(Input, 'gitHooksEnabled', 'get').mockReturnValue(true);
const parameters = await BuildParameters.create();
expect(parameters.gitHooksEnabled).toBe(true);
});
it('maps gitHooksSkipList from Input', async () => {
jest.spyOn(Input, 'gitHooksSkipList', 'get').mockReturnValue('pre-commit,pre-push');
const parameters = await BuildParameters.create();
expect(parameters.gitHooksSkipList).toBe('pre-commit,pre-push');
});
it('maps gitHooksRunBeforeBuild from Input', async () => {
jest.spyOn(Input, 'gitHooksRunBeforeBuild', 'get').mockReturnValue('pre-commit');
const parameters = await BuildParameters.create();
expect(parameters.gitHooksRunBeforeBuild).toBe('pre-commit');
});
it('maps providerExecutable from Input', async () => {
jest.spyOn(Input, 'providerExecutable', 'get').mockReturnValue('/usr/local/bin/provider');
const parameters = await BuildParameters.create();
expect(parameters.providerExecutable).toBe('/usr/local/bin/provider');
});
// Test that all enterprise properties have correct defaults when not explicitly set
it('has correct defaults for all enterprise properties', async () => {
const parameters = await BuildParameters.create();
expect(parameters.submoduleProfilePath).toBe('');
expect(parameters.submoduleVariantPath).toBe('');
expect(parameters.submoduleToken).toBe('');
expect(parameters.localCacheEnabled).toBe(false);
expect(parameters.localCacheRoot).toBe('');
expect(parameters.localCacheLibrary).toBe(true);
expect(parameters.localCacheLfs).toBe(false);
expect(parameters.childWorkspacesEnabled).toBe(false);
expect(parameters.childWorkspaceName).toBe('');
expect(parameters.childWorkspaceCacheRoot).toBe('');
expect(parameters.childWorkspacePreserveGit).toBe(true);
expect(parameters.childWorkspaceSeparateLibrary).toBe(true);
expect(parameters.lfsTransferAgent).toBe('');
expect(parameters.lfsTransferAgentArgs).toBe('');
expect(parameters.lfsStoragePaths).toBe('');
expect(parameters.gitHooksEnabled).toBe(false);
expect(parameters.gitHooksSkipList).toBe('');
expect(parameters.gitHooksRunBeforeBuild).toBe('');
expect(parameters.providerExecutable).toBe('');
});
});

View File

@@ -1,6 +1,6 @@
import CloudRunnerLogger from './cloud-runner/services/core/cloud-runner-logger';
import CloudRunner from './cloud-runner/cloud-runner';
import CloudRunnerOptions from './cloud-runner/options/cloud-runner-options';
import OrchestratorLogger from './orchestrator/services/core/orchestrator-logger';
import Orchestrator from './orchestrator/orchestrator';
import OrchestratorOptions from './orchestrator/options/orchestrator-options';
import * as core from '@actions/core';
import { Octokit } from '@octokit/core';
@@ -19,15 +19,15 @@ class GitHub {
}
private static get octokitPAT() {
return new Octokit({
auth: CloudRunner.buildParameters.gitPrivateToken,
auth: Orchestrator.buildParameters.gitPrivateToken,
});
}
private static get sha() {
return CloudRunner.buildParameters.gitSha;
return Orchestrator.buildParameters.gitSha;
}
private static get checkName() {
return `Cloud Runner (${CloudRunner.buildParameters.buildGuid})`;
return `Orchestrator (${Orchestrator.buildParameters.buildGuid})`;
}
private static get nameReadable() {
@@ -35,24 +35,24 @@ class GitHub {
}
private static get checkRunId() {
return CloudRunner.buildParameters.githubCheckId;
return Orchestrator.buildParameters.githubCheckId;
}
private static get owner() {
return CloudRunnerOptions.githubOwner;
return OrchestratorOptions.githubOwner;
}
private static get repo() {
return CloudRunnerOptions.githubRepoName;
return OrchestratorOptions.githubRepoName;
}
public static async createGitHubCheck(summary: string) {
if (!CloudRunner.buildParameters.githubChecks) {
if (!Orchestrator.buildParameters.githubChecks) {
return ``;
}
GitHub.startedDate = new Date().toISOString();
CloudRunnerLogger.log(`Creating github check`);
OrchestratorLogger.log(`Creating github check`);
const data = {
owner: GitHub.owner,
repo: GitHub.repo,
@@ -61,7 +61,7 @@ class GitHub {
head_sha: GitHub.sha,
status: 'queued',
// eslint-disable-next-line camelcase
external_id: CloudRunner.buildParameters.buildGuid,
external_id: Orchestrator.buildParameters.buildGuid,
// eslint-disable-next-line camelcase
started_at: GitHub.startedDate,
output: {
@@ -79,7 +79,7 @@ class GitHub {
};
const result = await GitHub.createGitHubCheckRequest(data);
CloudRunnerLogger.log(`Creating github check ${result.status}`);
OrchestratorLogger.log(`Creating github check ${result.status}`);
return result.data.id.toString();
}
@@ -90,11 +90,11 @@ class GitHub {
result = `neutral`,
status = `in_progress`,
) {
if (`${CloudRunner.buildParameters.githubChecks}` !== `true`) {
if (`${Orchestrator.buildParameters.githubChecks}` !== `true`) {
return;
}
CloudRunnerLogger.log(
`githubChecks: ${CloudRunner.buildParameters.githubChecks} checkRunId: ${GitHub.checkRunId} sha: ${GitHub.sha} async: ${CloudRunner.isCloudRunnerAsyncEnvironment}`,
OrchestratorLogger.log(
`githubChecks: ${Orchestrator.buildParameters.githubChecks} checkRunId: ${GitHub.checkRunId} sha: ${GitHub.sha} async: ${Orchestrator.isOrchestratorAsyncEnvironment}`,
);
GitHub.longDescriptionContent += `\n${longDescription}`;
if (GitHub.result !== `success` && GitHub.result !== `failure`) {
@@ -130,7 +130,7 @@ class GitHub {
data.conclusion = result;
}
await (CloudRunner.isCloudRunnerAsyncEnvironment || GitHub.forceAsyncTest
await (Orchestrator.isOrchestratorAsyncEnvironment || GitHub.forceAsyncTest
? GitHub.runUpdateAsyncChecksWorkflow(data, `update`)
: GitHub.updateGitHubCheckRequest(data));
}
@@ -152,7 +152,7 @@ class GitHub {
repo: GitHub.repo,
});
const workflows = workflowsResult.data.workflows;
CloudRunnerLogger.log(`Got ${workflows.length} workflows`);
OrchestratorLogger.log(`Got ${workflows.length} workflows`);
let selectedId = ``;
for (let index = 0; index < workflowsResult.data.total_count; index++) {
if (workflows[index].name === GitHub.asyncChecksApiWorkflowName) {
@@ -168,7 +168,7 @@ class GitHub {
repo: GitHub.repo,
// eslint-disable-next-line camelcase
workflow_id: selectedId,
ref: CloudRunnerOptions.branch,
ref: OrchestratorOptions.branch,
inputs: {
checksObject: JSON.stringify({ data, mode }),
},
@@ -176,7 +176,7 @@ class GitHub {
}
static async triggerWorkflowOnComplete(triggerWorkflowOnComplete: string[]) {
const isLocalAsync = CloudRunner.buildParameters.asyncWorkflow && !CloudRunner.isCloudRunnerAsyncEnvironment;
const isLocalAsync = Orchestrator.buildParameters.asyncWorkflow && !Orchestrator.isOrchestratorAsyncEnvironment;
if (isLocalAsync || triggerWorkflowOnComplete === undefined || triggerWorkflowOnComplete.length === 0) {
return;
}
@@ -186,7 +186,7 @@ class GitHub {
repo: GitHub.repo,
});
const workflows = workflowsResult.data.workflows;
CloudRunnerLogger.log(`Got ${workflows.length} workflows`);
OrchestratorLogger.log(`Got ${workflows.length} workflows`);
for (const element of triggerWorkflowOnComplete) {
let selectedId = ``;
for (let index = 0; index < workflowsResult.data.total_count; index++) {
@@ -203,9 +203,9 @@ class GitHub {
repo: GitHub.repo,
// eslint-disable-next-line camelcase
workflow_id: selectedId,
ref: CloudRunnerOptions.branch,
ref: OrchestratorOptions.branch,
inputs: {
buildGuid: CloudRunner.buildParameters.buildGuid,
buildGuid: Orchestrator.buildParameters.buildGuid,
},
});
}

View File

@@ -5,16 +5,17 @@ class ImageEnvironmentFactory {
const environmentVariables = ImageEnvironmentFactory.getEnvironmentVariables(parameters, additionalVariables);
let string = '';
for (const p of environmentVariables) {
if (p.value === '' || p.value === undefined) {
if (p.value === '' || p.value === undefined || p.value === null) {
continue;
}
if (p.name !== 'ANDROID_KEYSTORE_BASE64' && p.value.toString().includes(`\n`)) {
const valueAsString = typeof p.value === 'string' ? p.value : String(p.value);
if (p.name !== 'ANDROID_KEYSTORE_BASE64' && valueAsString.includes(`\n`)) {
string += `--env ${p.name} `;
process.env[p.name] = p.value.toString();
process.env[p.name] = valueAsString;
continue;
}
string += `--env ${p.name}="${p.value}" `;
string += `--env ${p.name}="${valueAsString}" `;
}
return string;
@@ -82,17 +83,12 @@ class ImageEnvironmentFactory {
{ name: 'RUNNER_TEMP', value: process.env.RUNNER_TEMP },
{ name: 'RUNNER_WORKSPACE', value: process.env.RUNNER_WORKSPACE },
];
if (parameters.providerStrategy === 'local-docker') {
for (const element of additionalVariables) {
if (!environmentVariables.some((x) => element?.name === x?.name)) {
environmentVariables.push(element);
}
}
for (const variable of environmentVariables) {
if (!environmentVariables.some((x) => variable?.name === x?.name)) {
environmentVariables = environmentVariables.filter((x) => x !== variable);
}
}
// Always merge additional variables (e.g., secrets/env from Orchestrator) uniquely by name
for (const element of additionalVariables) {
if (!element || !element.name) continue;
environmentVariables = environmentVariables.filter((x) => x?.name !== element.name);
environmentVariables.push(element);
}
if (parameters.sshAgent) {
environmentVariables.push({ name: 'SSH_AUTH_SOCK', value: '/ssh-agent' });

View File

@@ -9,7 +9,8 @@ import Platform from './platform';
import Project from './project';
import Unity from './unity';
import Versioning from './versioning';
import CloudRunner from './cloud-runner/cloud-runner';
import Orchestrator from './orchestrator/orchestrator';
import loadProvider, { ProviderLoader } from './orchestrator/providers/provider-loader';
export {
Action,
@@ -23,5 +24,7 @@ export {
Project,
Unity,
Versioning,
CloudRunner as CloudRunner,
Orchestrator as Orchestrator,
loadProvider,
ProviderLoader,
};

View File

@@ -1,12 +1,12 @@
import { CloudRunnerSystem } from '../cloud-runner/services/core/cloud-runner-system';
import CloudRunnerOptions from '../cloud-runner/options/cloud-runner-options';
import { OrchestratorSystem } from '../orchestrator/services/core/orchestrator-system';
import OrchestratorOptions from '../orchestrator/options/orchestrator-options';
export class GenericInputReader {
public static async Run(command: string) {
if (CloudRunnerOptions.providerStrategy === 'local') {
if (OrchestratorOptions.providerStrategy === 'local') {
return '';
}
return await CloudRunnerSystem.Run(command, false, true);
return await OrchestratorSystem.Run(command, false, true);
}
}

View File

@@ -1,6 +1,6 @@
import { GitRepoReader } from './git-repo';
import { CloudRunnerSystem } from '../cloud-runner/services/core/cloud-runner-system';
import CloudRunnerOptions from '../cloud-runner/options/cloud-runner-options';
import { OrchestratorSystem } from '../orchestrator/services/core/orchestrator-system';
import OrchestratorOptions from '../orchestrator/options/orchestrator-options';
describe(`git repo tests`, () => {
it(`Branch value parsed from CLI to not contain illegal characters`, async () => {
@@ -10,15 +10,15 @@ describe(`git repo tests`, () => {
it(`returns valid branch name when using https`, async () => {
const mockValue = 'https://github.com/example/example.git';
await jest.spyOn(CloudRunnerSystem, 'Run').mockReturnValue(Promise.resolve(mockValue));
await jest.spyOn(CloudRunnerOptions, 'providerStrategy', 'get').mockReturnValue('not-local');
await jest.spyOn(OrchestratorSystem, 'Run').mockReturnValue(Promise.resolve(mockValue));
await jest.spyOn(OrchestratorOptions, 'providerStrategy', 'get').mockReturnValue('not-local');
expect(await GitRepoReader.GetRemote()).toEqual(`example/example`);
});
it(`returns valid branch name when using ssh`, async () => {
const mockValue = 'git@github.com:example/example.git';
await jest.spyOn(CloudRunnerSystem, 'Run').mockReturnValue(Promise.resolve(mockValue));
await jest.spyOn(CloudRunnerOptions, 'providerStrategy', 'get').mockReturnValue('not-local');
await jest.spyOn(OrchestratorSystem, 'Run').mockReturnValue(Promise.resolve(mockValue));
await jest.spyOn(OrchestratorOptions, 'providerStrategy', 'get').mockReturnValue('not-local');
expect(await GitRepoReader.GetRemote()).toEqual(`example/example`);
});
});

View File

@@ -1,33 +1,33 @@
import { assert } from 'node:console';
import fs from 'node:fs';
import { CloudRunnerSystem } from '../cloud-runner/services/core/cloud-runner-system';
import CloudRunnerLogger from '../cloud-runner/services/core/cloud-runner-logger';
import CloudRunnerOptions from '../cloud-runner/options/cloud-runner-options';
import { OrchestratorSystem } from '../orchestrator/services/core/orchestrator-system';
import OrchestratorLogger from '../orchestrator/services/core/orchestrator-logger';
import OrchestratorOptions from '../orchestrator/options/orchestrator-options';
import Input from '../input';
export class GitRepoReader {
public static async GetRemote() {
if (CloudRunnerOptions.providerStrategy === 'local') {
if (OrchestratorOptions.providerStrategy === 'local') {
return '';
}
assert(fs.existsSync(`.git`));
const value = (await CloudRunnerSystem.Run(`cd ${Input.projectPath} && git remote -v`, false, true)).replace(
const value = (await OrchestratorSystem.Run(`cd ${Input.projectPath} && git remote -v`, false, true)).replace(
/ /g,
``,
);
CloudRunnerLogger.log(`value ${value}`);
OrchestratorLogger.log(`value ${value}`);
assert(value.includes('github.com'));
return value.split('github.com')[1].split('.git')[0].slice(1);
}
public static async GetBranch() {
if (CloudRunnerOptions.providerStrategy === 'local') {
if (OrchestratorOptions.providerStrategy === 'local') {
return '';
}
assert(fs.existsSync(`.git`));
return (await CloudRunnerSystem.Run(`cd ${Input.projectPath} && git branch --show-current`, false, true))
return (await OrchestratorSystem.Run(`cd ${Input.projectPath} && git branch --show-current`, false, true))
.split('\n')[0]
.replace(/ /g, ``)
.replace('/head', '');

View File

@@ -1,19 +1,19 @@
import { CloudRunnerSystem } from '../cloud-runner/services/core/cloud-runner-system';
import { OrchestratorSystem } from '../orchestrator/services/core/orchestrator-system';
import * as core from '@actions/core';
import CloudRunnerOptions from '../cloud-runner/options/cloud-runner-options';
import OrchestratorOptions from '../orchestrator/options/orchestrator-options';
export class GithubCliReader {
static async GetGitHubAuthToken() {
if (CloudRunnerOptions.providerStrategy === 'local') {
if (OrchestratorOptions.providerStrategy === 'local') {
return '';
}
try {
const authStatus = await CloudRunnerSystem.Run(`gh auth status`, true, true);
const authStatus = await OrchestratorSystem.Run(`gh auth status`, true, true);
if (authStatus.includes('You are not logged') || authStatus === '') {
return '';
}
return (await CloudRunnerSystem.Run(`gh auth status -t`, false, true))
return (await OrchestratorSystem.Run(`gh auth status -t`, false, true))
.split(`Token: `)[1]
.replace(/ /g, '')
.replace(/\n/g, '');

View File

@@ -1,13 +1,13 @@
import path from 'node:path';
import fs from 'node:fs';
import YAML from 'yaml';
import CloudRunnerOptions from '../cloud-runner/options/cloud-runner-options';
import OrchestratorOptions from '../orchestrator/options/orchestrator-options';
export function ReadLicense(): string {
if (CloudRunnerOptions.providerStrategy === 'local') {
if (OrchestratorOptions.providerStrategy === 'local') {
return '';
}
const pipelineFile = path.join(__dirname, `.github`, `workflows`, `cloud-runner-k8s-pipeline.yml`);
const pipelineFile = path.join(__dirname, `.github`, `workflows`, `orchestrator-k8s-pipeline.yml`);
return fs.existsSync(pipelineFile) ? YAML.parse(fs.readFileSync(pipelineFile, 'utf8')).env.UNITY_LICENSE : '';
}

View File

@@ -1,7 +1,7 @@
import fs from 'node:fs';
import path from 'node:path';
import { Cli } from './cli/cli';
import CloudRunnerQueryOverride from './cloud-runner/options/cloud-runner-query-override';
import OrchestratorQueryOverride from './orchestrator/options/orchestrator-query-override';
import Platform from './platform';
import GitHub from './github';
import os from 'node:os';
@@ -32,8 +32,8 @@ class Input {
return Cli.query(query, alternativeQuery);
}
if (CloudRunnerQueryOverride.query(query, alternativeQuery)) {
return CloudRunnerQueryOverride.query(query, alternativeQuery);
if (OrchestratorQueryOverride.query(query, alternativeQuery)) {
return OrchestratorQueryOverride.query(query, alternativeQuery);
}
if (process.env[query] !== undefined) {
@@ -241,6 +241,28 @@ class Input {
return Input.getInput('dockerWorkspacePath') ?? '/github/workspace';
}
static get syncStrategy(): string {
return Input.getInput('syncStrategy') ?? 'full';
}
static get syncInputRef(): string {
return Input.getInput('syncInputRef') ?? '';
}
static get syncStorageRemote(): string {
return Input.getInput('syncStorageRemote') ?? '';
}
static get syncRevertAfter(): boolean {
const input = Input.getInput('syncRevertAfter') ?? 'true';
return input === 'true';
}
static get syncStatePath(): string {
return Input.getInput('syncStatePath') ?? '.game-ci/sync-state.json';
}
static get dockerCpuLimit(): string {
return Input.getInput('dockerCpuLimit') ?? os.cpus().length.toString();
}
@@ -278,10 +300,356 @@ class Input {
return Input.getInput('containerRegistryImageVersion') ?? '3';
}
static get artifactOutputTypes(): string {
return Input.getInput('artifactOutputTypes') ?? 'build,logs,test-results';
}
static get artifactUploadTarget(): string {
return Input.getInput('artifactUploadTarget') ?? 'github-artifacts';
}
static get artifactUploadPath(): string {
return Input.getInput('artifactUploadPath') ?? '';
}
static get artifactCompression(): string {
return Input.getInput('artifactCompression') ?? 'gzip';
}
static get artifactRetentionDays(): string {
return Input.getInput('artifactRetentionDays') ?? '30';
}
static get artifactCustomTypes(): string {
return Input.getInput('artifactCustomTypes') ?? '';
}
static get skipActivation(): string {
return Input.getInput('skipActivation')?.toLowerCase() ?? 'false';
}
static get submoduleProfilePath(): string {
return Input.getInput('submoduleProfilePath') ?? '';
}
static get submoduleVariantPath(): string {
return Input.getInput('submoduleVariantPath') ?? '';
}
static get submoduleToken(): string {
return Input.getInput('submoduleToken') ?? '';
}
static get localCacheEnabled(): boolean {
return (Input.getInput('localCacheEnabled') ?? 'false') === 'true';
}
static get localCacheRoot(): string {
return Input.getInput('localCacheRoot') ?? '';
}
static get localCacheLibrary(): boolean {
return (Input.getInput('localCacheLibrary') ?? 'true') === 'true';
}
static get localCacheLfs(): boolean {
return (Input.getInput('localCacheLfs') ?? 'false') === 'true';
}
static get childWorkspacesEnabled(): boolean {
return (Input.getInput('childWorkspacesEnabled') ?? 'false') === 'true';
}
static get childWorkspaceName(): string {
return Input.getInput('childWorkspaceName') ?? '';
}
static get childWorkspaceCacheRoot(): string {
return Input.getInput('childWorkspaceCacheRoot') ?? '';
}
static get childWorkspacePreserveGit(): boolean {
return (Input.getInput('childWorkspacePreserveGit') ?? 'true') === 'true';
}
static get childWorkspaceSeparateLibrary(): boolean {
return (Input.getInput('childWorkspaceSeparateLibrary') ?? 'true') === 'true';
}
static get lfsTransferAgent(): string {
return Input.getInput('lfsTransferAgent') ?? '';
}
static get lfsTransferAgentArgs(): string {
return Input.getInput('lfsTransferAgentArgs') ?? '';
}
static get lfsStoragePaths(): string {
return Input.getInput('lfsStoragePaths') ?? '';
}
static get gitHooksEnabled(): boolean {
return (Input.getInput('gitHooksEnabled') ?? 'false') === 'true';
}
static get gitHooksSkipList(): string {
return Input.getInput('gitHooksSkipList') ?? '';
}
static get gitHooksRunBeforeBuild(): string {
return Input.getInput('gitHooksRunBeforeBuild') ?? '';
}
static get providerExecutable(): string {
return Input.getInput('providerExecutable') ?? '';
}
// GCP Cloud Run (Experimental)
static get gcpProject(): string {
return Input.getInput('gcpProject') ?? '';
}
static get gcpRegion(): string {
return Input.getInput('gcpRegion') ?? '';
}
static get gcpStorageType(): string {
return Input.getInput('gcpStorageType') ?? 'gcs-fuse';
}
static get gcpBucket(): string {
return Input.getInput('gcpBucket') ?? '';
}
static get gcpFilestoreIp(): string {
return Input.getInput('gcpFilestoreIp') ?? '';
}
static get gcpFilestoreShare(): string {
return Input.getInput('gcpFilestoreShare') ?? '/share1';
}
static get gcpMachineType(): string {
return Input.getInput('gcpMachineType') ?? 'e2-standard-4';
}
static get gcpDiskSizeGb(): string {
return Input.getInput('gcpDiskSizeGb') ?? '100';
}
static get gcpServiceAccount(): string {
return Input.getInput('gcpServiceAccount') ?? '';
}
static get gcpVpcConnector(): string {
return Input.getInput('gcpVpcConnector') ?? '';
}
// Azure Container Instances (Experimental)
static get azureResourceGroup(): string {
return Input.getInput('azureResourceGroup') ?? '';
}
static get azureLocation(): string {
return Input.getInput('azureLocation') ?? '';
}
static get azureStorageType(): string {
return Input.getInput('azureStorageType') ?? 'azure-files';
}
static get azureStorageAccount(): string {
return Input.getInput('azureStorageAccount') ?? '';
}
static get azureBlobContainer(): string {
return Input.getInput('azureBlobContainer') ?? 'unity-builds';
}
static get azureFileShareName(): string {
return Input.getInput('azureFileShareName') ?? 'unity-builds';
}
static get azureSubscriptionId(): string {
return Input.getInput('azureSubscriptionId') ?? '';
}
static get azureCpu(): string {
return Input.getInput('azureCpu') ?? '4';
}
static get azureMemoryGb(): string {
return Input.getInput('azureMemoryGb') ?? '16';
}
static get azureDiskSizeGb(): string {
return Input.getInput('azureDiskSizeGb') ?? '100';
}
static get azureSubnetId(): string {
return Input.getInput('azureSubnetId') ?? '';
}
// ### ### ###
// Remote PowerShell provider
// ### ### ###
static get remotePowershellHost(): string {
return Input.getInput('remotePowershellHost') ?? '';
}
static get remotePowershellCredential(): string {
return Input.getInput('remotePowershellCredential') ?? '';
}
static get remotePowershellTransport(): string {
return Input.getInput('remotePowershellTransport') ?? 'wsman';
}
// ### ### ###
// GitHub Actions provider
// ### ### ###
static get githubActionsRepo(): string {
return Input.getInput('githubActionsRepo') ?? '';
}
static get githubActionsWorkflow(): string {
return Input.getInput('githubActionsWorkflow') ?? '';
}
static get githubActionsToken(): string {
return Input.getInput('githubActionsToken') ?? '';
}
static get githubActionsRef(): string {
return Input.getInput('githubActionsRef') ?? 'main';
}
// ### ### ###
// GitLab CI provider
// ### ### ###
static get gitlabProjectId(): string {
return Input.getInput('gitlabProjectId') ?? '';
}
static get gitlabTriggerToken(): string {
return Input.getInput('gitlabTriggerToken') ?? '';
}
static get gitlabApiUrl(): string {
return Input.getInput('gitlabApiUrl') ?? 'https://gitlab.com';
}
static get gitlabRef(): string {
return Input.getInput('gitlabRef') ?? 'main';
}
// ### ### ###
// Ansible provider
// ### ### ###
static get ansibleInventory(): string {
return Input.getInput('ansibleInventory') ?? '';
}
static get ansiblePlaybook(): string {
return Input.getInput('ansiblePlaybook') ?? '';
}
static get ansibleExtraVars(): string {
return Input.getInput('ansibleExtraVars') ?? '';
}
static get ansibleVaultPassword(): string {
return Input.getInput('ansibleVaultPassword') ?? '';
}
static get gitIntegrityCheck(): boolean {
const input = Input.getInput('gitIntegrityCheck') ?? 'false';
return input === 'true';
}
static get hotRunnerEnabled(): boolean {
const input = Input.getInput('hotRunnerEnabled') ?? false;
return input === 'true';
}
static get gitAutoRecover(): boolean {
const input = Input.getInput('gitAutoRecover') ?? 'false';
return input === 'true';
}
static get hotRunnerTransport(): 'websocket' | 'grpc' | 'named-pipe' {
return (Input.getInput('hotRunnerTransport') ?? 'websocket') as 'websocket' | 'grpc' | 'named-pipe';
}
static get hotRunnerHost(): string {
return Input.getInput('hotRunnerHost') ?? 'localhost';
}
static get hotRunnerPort(): number {
return Number.parseInt(Input.getInput('hotRunnerPort') ?? '9090', 10);
}
static get hotRunnerHealthInterval(): number {
return Number.parseInt(Input.getInput('hotRunnerHealthInterval') ?? '30', 10);
}
static get hotRunnerMaxIdle(): number {
return Number.parseInt(Input.getInput('hotRunnerMaxIdle') ?? '3600', 10);
}
static get hotRunnerFallbackToCold(): boolean {
const input = Input.getInput('hotRunnerFallbackToCold') ?? 'true';
return input === 'true';
}
static get cleanReservedFilenames(): boolean {
const input = Input.getInput('cleanReservedFilenames') ?? 'false';
return input === 'true';
}
static get buildArchiveEnabled(): boolean {
const input = Input.getInput('buildArchiveEnabled') ?? 'false';
return input === 'true';
}
static get buildArchivePath(): string {
return Input.getInput('buildArchivePath') ?? './build-archives';
}
static get buildArchiveRetention(): number {
return Number.parseInt(Input.getInput('buildArchiveRetention') ?? '30', 10);
}
static get testSuitePath(): string {
return Input.getInput('testSuitePath') ?? '';
}
static get testSuiteEvent(): string {
return Input.getInput('testSuiteEvent') ?? '';
}
static get testTaxonomyPath(): string {
return Input.getInput('testTaxonomyPath') ?? '';
}
static get testResultFormat(): string {
return Input.getInput('testResultFormat') ?? 'junit';
}
static get testResultPath(): string {
return Input.getInput('testResultPath') ?? './test-results';
}
public static ToEnvVarFormat(input: string) {
if (input.toUpperCase() === input) {
return input;

View File

@@ -0,0 +1,15 @@
import OrchestratorLogger from '../services/core/orchestrator-logger';
import * as core from '@actions/core';
import Orchestrator from '../orchestrator';
import OrchestratorSecret from '../options/orchestrator-secret';
import BuildParameters from '../../build-parameters';
export class OrchestratorError {
public static async handleException(error: unknown, buildParameters: BuildParameters, secrets: OrchestratorSecret[]) {
OrchestratorLogger.error(JSON.stringify(error, undefined, 4));
core.setFailed('Orchestrator failed');
if (Orchestrator.Provider !== undefined) {
await Orchestrator.Provider.cleanupWorkflow(buildParameters, buildParameters.branch, secrets);
}
}
}

View File

@@ -0,0 +1,4 @@
class OrchestratorConstants {
static alphabet = '0123456789abcdefghijklmnopqrstuvwxyz';
}
export default OrchestratorConstants;

View File

@@ -0,0 +1,5 @@
class OrchestratorEnvironmentVariable {
public name!: string;
public value!: string;
}
export default OrchestratorEnvironmentVariable;

View File

@@ -0,0 +1,140 @@
import { OrchestratorFolders } from './orchestrator-folders';
jest.mock('../orchestrator', () => ({
__esModule: true,
default: {
buildParameters: {
orchestratorRepoName: 'game-ci/unity-builder',
githubRepo: 'myorg/myrepo',
gitPrivateToken: 'ghp_test123',
gitAuthMode: 'header',
buildGuid: 'test-guid',
projectPath: '',
buildPath: 'Builds',
cacheKey: 'test-cache',
},
lockedWorkspace: '',
},
}));
jest.mock('./orchestrator-options', () => ({
__esModule: true,
default: {
useSharedBuilder: false,
},
}));
jest.mock('../services/core/orchestrator-system', () => ({
OrchestratorSystem: {
Run: jest.fn().mockResolvedValue(''),
},
}));
const mockOrchestrator = require('../orchestrator').default;
describe('OrchestratorFolders git auth', () => {
beforeEach(() => {
jest.clearAllMocks();
});
describe('useHeaderAuth', () => {
it('should return true when gitAuthMode is header', () => {
mockOrchestrator.buildParameters.gitAuthMode = 'header';
expect(OrchestratorFolders.useHeaderAuth).toBe(true);
});
it('should return true when gitAuthMode is undefined (default)', () => {
mockOrchestrator.buildParameters.gitAuthMode = undefined;
expect(OrchestratorFolders.useHeaderAuth).toBe(true);
});
it('should return false when gitAuthMode is url', () => {
mockOrchestrator.buildParameters.gitAuthMode = 'url';
expect(OrchestratorFolders.useHeaderAuth).toBe(false);
});
});
describe('unityBuilderRepoUrl', () => {
it('should not include token in URL when using header auth', () => {
mockOrchestrator.buildParameters.gitAuthMode = 'header';
const url = OrchestratorFolders.unityBuilderRepoUrl;
expect(url).toBe('https://github.com/game-ci/unity-builder.git');
expect(url).not.toContain('ghp_test123');
});
it('should include token in URL when using url auth (legacy)', () => {
mockOrchestrator.buildParameters.gitAuthMode = 'url';
const url = OrchestratorFolders.unityBuilderRepoUrl;
expect(url).toBe('https://ghp_test123@github.com/game-ci/unity-builder.git');
});
});
describe('targetBuildRepoUrl', () => {
it('should not include token in URL when using header auth', () => {
mockOrchestrator.buildParameters.gitAuthMode = 'header';
const url = OrchestratorFolders.targetBuildRepoUrl;
expect(url).toBe('https://github.com/myorg/myrepo.git');
expect(url).not.toContain('ghp_test123');
});
it('should include token in URL when using url auth (legacy)', () => {
mockOrchestrator.buildParameters.gitAuthMode = 'url';
const url = OrchestratorFolders.targetBuildRepoUrl;
expect(url).toBe('https://ghp_test123@github.com/myorg/myrepo.git');
});
});
describe('gitAuthConfigScript', () => {
it('should emit http.extraHeader commands in header mode', () => {
mockOrchestrator.buildParameters.gitAuthMode = 'header';
const script = OrchestratorFolders.gitAuthConfigScript;
expect(script).toContain('http.extraHeader');
expect(script).toContain('GIT_PRIVATE_TOKEN');
expect(script).toContain('Authorization: Basic');
});
it('should emit no-op comment in url mode', () => {
mockOrchestrator.buildParameters.gitAuthMode = 'url';
const script = OrchestratorFolders.gitAuthConfigScript;
expect(script).toContain('legacy');
expect(script).not.toContain('http.extraHeader');
});
});
describe('configureGitAuth', () => {
it('should run git config with http.extraHeader in header mode', async () => {
mockOrchestrator.buildParameters.gitAuthMode = 'header';
mockOrchestrator.buildParameters.gitPrivateToken = 'ghp_test123';
const { OrchestratorSystem } = require('../services/core/orchestrator-system');
await OrchestratorFolders.configureGitAuth();
// Verify the base64 encoding and extraHeader config are correct
const expectedEncoded = Buffer.from('x-access-token:ghp_test123').toString('base64');
expect(OrchestratorSystem.Run).toHaveBeenCalledWith(expect.stringContaining(expectedEncoded));
expect(OrchestratorSystem.Run).toHaveBeenCalledWith(expect.stringContaining('.extraHeader'));
});
it('should not run git config in url mode', async () => {
mockOrchestrator.buildParameters.gitAuthMode = 'url';
const { OrchestratorSystem } = require('../services/core/orchestrator-system');
await OrchestratorFolders.configureGitAuth();
expect(OrchestratorSystem.Run).not.toHaveBeenCalled();
});
it('should not run git config when no token is available', async () => {
mockOrchestrator.buildParameters.gitAuthMode = 'header';
mockOrchestrator.buildParameters.gitPrivateToken = '';
const originalEnv = process.env.GIT_PRIVATE_TOKEN;
delete process.env.GIT_PRIVATE_TOKEN;
const { OrchestratorSystem } = require('../services/core/orchestrator-system');
await OrchestratorFolders.configureGitAuth();
expect(OrchestratorSystem.Run).not.toHaveBeenCalled();
if (originalEnv !== undefined) process.env.GIT_PRIVATE_TOKEN = originalEnv;
});
});
});

View File

@@ -0,0 +1,162 @@
import { OrchestratorFolders } from './orchestrator-folders';
// Mock Orchestrator
jest.mock('../orchestrator', () => ({
__esModule: true,
default: {
buildParameters: {
buildGuid: 'test-guid-abc',
cacheKey: 'my-cache-key',
projectPath: 'test-project',
buildPath: 'Builds',
maxRetainedWorkspaces: 0,
gitPrivateToken: 'ghp_test123',
gitAuthMode: 'url',
orchestratorRepoName: 'game-ci/unity-builder',
githubRepo: 'user/my-game',
},
lockedWorkspace: '',
},
}));
jest.mock('../../build-parameters', () => ({
__esModule: true,
default: {
shouldUseRetainedWorkspaceMode: jest.fn().mockReturnValue(false),
},
}));
jest.mock('./orchestrator-options', () => ({
__esModule: true,
default: {
useSharedBuilder: false,
},
}));
// Normalize paths for cross-platform test compatibility
const normalize = (p: string) => p.replace(/\\/g, '/');
describe('OrchestratorFolders', () => {
describe('static constants', () => {
it('repositoryFolder is "repo"', () => {
expect(OrchestratorFolders.repositoryFolder).toBe('repo');
});
it('buildVolumeFolder is "data"', () => {
expect(OrchestratorFolders.buildVolumeFolder).toBe('data');
});
it('cacheFolder is "cache"', () => {
expect(OrchestratorFolders.cacheFolder).toBe('cache');
});
});
describe('ToLinuxFolder', () => {
it('converts backslashes to forward slashes', () => {
expect(OrchestratorFolders.ToLinuxFolder('C:\\Users\\test\\project')).toBe('C:/Users/test/project');
});
it('preserves forward slashes', () => {
expect(OrchestratorFolders.ToLinuxFolder('/home/user/project')).toBe('/home/user/project');
});
it('handles mixed slashes', () => {
expect(OrchestratorFolders.ToLinuxFolder('some/path\\mixed/slashes\\here')).toBe('some/path/mixed/slashes/here');
});
it('handles empty string', () => {
expect(OrchestratorFolders.ToLinuxFolder('')).toBe('');
});
});
describe('path computations (non-retained workspace mode)', () => {
it('uniqueOrchestratorJobFolderAbsolute uses buildGuid', () => {
const result = normalize(OrchestratorFolders.uniqueOrchestratorJobFolderAbsolute);
expect(result).toBe('/data/test-guid-abc');
});
it('cacheFolderForAllFull returns /data/cache', () => {
const result = normalize(OrchestratorFolders.cacheFolderForAllFull);
expect(result).toBe('/data/cache');
});
it('cacheFolderForCacheKeyFull includes cache key', () => {
const result = normalize(OrchestratorFolders.cacheFolderForCacheKeyFull);
expect(result).toBe('/data/cache/my-cache-key');
});
it('repoPathAbsolute is under job folder', () => {
const result = normalize(OrchestratorFolders.repoPathAbsolute);
expect(result).toBe('/data/test-guid-abc/repo');
});
it('projectPathAbsolute includes project path', () => {
const result = normalize(OrchestratorFolders.projectPathAbsolute);
expect(result).toBe('/data/test-guid-abc/repo/test-project');
});
it('libraryFolderAbsolute is under project path', () => {
const result = normalize(OrchestratorFolders.libraryFolderAbsolute);
expect(result).toBe('/data/test-guid-abc/repo/test-project/Library');
});
it('projectBuildFolderAbsolute uses buildPath', () => {
const result = normalize(OrchestratorFolders.projectBuildFolderAbsolute);
expect(result).toBe('/data/test-guid-abc/repo/Builds');
});
it('lfsFolderAbsolute is under .git/lfs', () => {
const result = normalize(OrchestratorFolders.lfsFolderAbsolute);
expect(result).toBe('/data/test-guid-abc/repo/.git/lfs');
});
it('lfsCacheFolderFull is under cache key', () => {
const result = normalize(OrchestratorFolders.lfsCacheFolderFull);
expect(result).toBe('/data/cache/my-cache-key/lfs');
});
it('libraryCacheFolderFull is under cache key', () => {
const result = normalize(OrchestratorFolders.libraryCacheFolderFull);
expect(result).toBe('/data/cache/my-cache-key/Library');
});
});
describe('builderPathAbsolute', () => {
it('uses job folder when shared builder is disabled', () => {
const result = normalize(OrchestratorFolders.builderPathAbsolute);
expect(result).toBe('/data/test-guid-abc/builder');
});
});
describe('repo URLs', () => {
it('unityBuilderRepoUrl includes token and repo name', () => {
const url = OrchestratorFolders.unityBuilderRepoUrl;
expect(url).toBe('https://ghp_test123@github.com/game-ci/unity-builder.git');
});
it('targetBuildRepoUrl includes token and github repo', () => {
const url = OrchestratorFolders.targetBuildRepoUrl;
expect(url).toBe('https://ghp_test123@github.com/user/my-game.git');
});
});
describe('purgeRemoteCaching', () => {
it('returns false when env var is not set', () => {
const original = process.env.PURGE_REMOTE_BUILDER_CACHE;
delete process.env.PURGE_REMOTE_BUILDER_CACHE;
expect(OrchestratorFolders.purgeRemoteCaching).toBe(false);
if (original !== undefined) process.env.PURGE_REMOTE_BUILDER_CACHE = original;
});
it('returns true when env var is set', () => {
const original = process.env.PURGE_REMOTE_BUILDER_CACHE;
process.env.PURGE_REMOTE_BUILDER_CACHE = 'true';
expect(OrchestratorFolders.purgeRemoteCaching).toBe(true);
if (original !== undefined) {
process.env.PURGE_REMOTE_BUILDER_CACHE = original;
} else {
delete process.env.PURGE_REMOTE_BUILDER_CACHE;
}
});
});
});

View File

@@ -0,0 +1,143 @@
import path from 'node:path';
import OrchestratorOptions from './orchestrator-options';
import Orchestrator from '../orchestrator';
import BuildParameters from '../../build-parameters';
export class OrchestratorFolders {
public static readonly repositoryFolder = 'repo';
public static ToLinuxFolder(folder: string) {
return folder.replace(/\\/g, `/`);
}
// Only the following paths that do not start a path.join with another "Full" suffixed property need to start with an absolute /
public static get uniqueOrchestratorJobFolderAbsolute(): string {
return Orchestrator.buildParameters && BuildParameters.shouldUseRetainedWorkspaceMode(Orchestrator.buildParameters)
? path.join(`/`, OrchestratorFolders.buildVolumeFolder, Orchestrator.lockedWorkspace)
: path.join(`/`, OrchestratorFolders.buildVolumeFolder, Orchestrator.buildParameters.buildGuid);
}
public static get cacheFolderForAllFull(): string {
return path.join('/', OrchestratorFolders.buildVolumeFolder, OrchestratorFolders.cacheFolder);
}
public static get cacheFolderForCacheKeyFull(): string {
return path.join(
'/',
OrchestratorFolders.buildVolumeFolder,
OrchestratorFolders.cacheFolder,
Orchestrator.buildParameters.cacheKey,
);
}
public static get builderPathAbsolute(): string {
return path.join(
OrchestratorOptions.useSharedBuilder
? `/${OrchestratorFolders.buildVolumeFolder}`
: OrchestratorFolders.uniqueOrchestratorJobFolderAbsolute,
`builder`,
);
}
public static get repoPathAbsolute(): string {
return path.join(OrchestratorFolders.uniqueOrchestratorJobFolderAbsolute, OrchestratorFolders.repositoryFolder);
}
public static get projectPathAbsolute(): string {
return path.join(OrchestratorFolders.repoPathAbsolute, Orchestrator.buildParameters.projectPath);
}
public static get libraryFolderAbsolute(): string {
return path.join(OrchestratorFolders.projectPathAbsolute, `Library`);
}
public static get projectBuildFolderAbsolute(): string {
return path.join(OrchestratorFolders.repoPathAbsolute, Orchestrator.buildParameters.buildPath);
}
public static get lfsFolderAbsolute(): string {
return path.join(OrchestratorFolders.repoPathAbsolute, `.git`, `lfs`);
}
public static get purgeRemoteCaching(): boolean {
return process.env.PURGE_REMOTE_BUILDER_CACHE !== undefined;
}
public static get lfsCacheFolderFull() {
return path.join(OrchestratorFolders.cacheFolderForCacheKeyFull, `lfs`);
}
public static get libraryCacheFolderFull() {
return path.join(OrchestratorFolders.cacheFolderForCacheKeyFull, `Library`);
}
/**
* Whether to use http.extraHeader for git authentication (secure, default)
* instead of embedding the token in clone URLs (legacy).
*/
public static get useHeaderAuth(): boolean {
return Orchestrator.buildParameters.gitAuthMode !== 'url';
}
public static get unityBuilderRepoUrl(): string {
if (OrchestratorFolders.useHeaderAuth) {
return `https://github.com/${Orchestrator.buildParameters.orchestratorRepoName}.git`;
}
return `https://${Orchestrator.buildParameters.gitPrivateToken}@github.com/${Orchestrator.buildParameters.orchestratorRepoName}.git`;
}
public static get targetBuildRepoUrl(): string {
if (OrchestratorFolders.useHeaderAuth) {
return `https://github.com/${Orchestrator.buildParameters.githubRepo}.git`;
}
return `https://${Orchestrator.buildParameters.gitPrivateToken}@github.com/${Orchestrator.buildParameters.githubRepo}.git`;
}
/**
* Shell commands to configure git authentication via http.extraHeader.
* Uses GIT_PRIVATE_TOKEN env var so the token never appears in clone URLs or git config output.
* This is the same mechanism used by actions/checkout.
*
* Only emits commands when gitAuthMode is 'header' (default). In 'url' mode,
* returns a no-op comment since the token is already in the URL.
*/
public static get gitAuthConfigScript(): string {
if (!OrchestratorFolders.useHeaderAuth) {
return `# git auth: using token-in-URL mode (legacy)`;
}
return `# git auth: configuring http.extraHeader (secure mode)
if [ -n "$GIT_PRIVATE_TOKEN" ]; then
git config --global http.https://github.com/.extraHeader "Authorization: Basic $(printf '%s' "x-access-token:$GIT_PRIVATE_TOKEN" | base64 -w 0)"
fi`;
}
/**
* Configure git authentication via http.extraHeader in the current Node process.
* For use in the remote-client where shell scripts aren't used.
* Only configures when gitAuthMode is 'header' (default).
*/
public static async configureGitAuth(): Promise<void> {
if (!OrchestratorFolders.useHeaderAuth) return;
const token = Orchestrator.buildParameters.gitPrivateToken || process.env.GIT_PRIVATE_TOKEN || '';
if (!token) return;
const encoded = Buffer.from(`x-access-token:${token}`).toString('base64');
const { OrchestratorSystem } = await import('../services/core/orchestrator-system');
await OrchestratorSystem.Run(
`git config --global http.https://github.com/.extraHeader "Authorization: Basic ${encoded}"`,
);
}
public static get buildVolumeFolder() {
return 'data';
}
public static get cacheFolder() {
return 'cache';
}
}

View File

@@ -0,0 +1,53 @@
import OrchestratorNamespace from './orchestrator-guid';
describe('OrchestratorNamespace', () => {
describe('generateGuid', () => {
it('generates a guid with correct format', () => {
const guid = OrchestratorNamespace.generateGuid('42', 'StandaloneLinux64');
// Format: {runNumber}-{platform}-{nanoid4}
expect(guid).toMatch(/^42-linux64-[a-z0-9]{4}$/);
});
it('strips "standalone" prefix from platform (case-insensitive)', () => {
const guid = OrchestratorNamespace.generateGuid('1', 'StandaloneWindows64');
expect(guid).toMatch(/^1-windows64-[a-z0-9]{4}$/);
});
it('lowercases platform name', () => {
const guid = OrchestratorNamespace.generateGuid('5', 'Android');
expect(guid).toMatch(/^5-android-[a-z0-9]{4}$/);
});
it('handles numeric run number', () => {
const guid = OrchestratorNamespace.generateGuid(100, 'iOS');
expect(guid).toMatch(/^100-ios-[a-z0-9]{4}$/);
});
it('generates unique guids on repeated calls', () => {
const guids = new Set<string>();
for (let i = 0; i < 20; i++) {
guids.add(OrchestratorNamespace.generateGuid('1', 'StandaloneLinux64'));
}
// With 4 alphanumeric chars (36^4 = ~1.7M possibilities), 20 calls should almost certainly be unique
expect(guids.size).toBeGreaterThan(1);
});
it('handles StandaloneOSX platform', () => {
const guid = OrchestratorNamespace.generateGuid('7', 'StandaloneOSX');
expect(guid).toMatch(/^7-osx-[a-z0-9]{4}$/);
});
it('handles WebGL platform (no standalone prefix)', () => {
const guid = OrchestratorNamespace.generateGuid('3', 'WebGL');
expect(guid).toMatch(/^3-webgl-[a-z0-9]{4}$/);
});
it('uses only lowercase alphanumeric characters in nanoid portion', () => {
for (let i = 0; i < 10; i++) {
const guid = OrchestratorNamespace.generateGuid('1', 'test');
const nanoidPart = guid.split('-').pop()!;
expect(nanoidPart).toMatch(/^[0-9a-z]{4}$/);
}
});
});
});

View File

@@ -1,11 +1,11 @@
import { customAlphabet } from 'nanoid';
import CloudRunnerConstants from './cloud-runner-constants';
import OrchestratorConstants from './orchestrator-constants';
class CloudRunnerNamespace {
class OrchestratorNamespace {
static generateGuid(runNumber: string | number, platform: string) {
const nanoid = customAlphabet(CloudRunnerConstants.alphabet, 4);
const nanoid = customAlphabet(OrchestratorConstants.alphabet, 4);
return `${runNumber}-${platform.toLowerCase().replace('standalone', '')}-${nanoid()}`;
}
}
export default CloudRunnerNamespace;
export default OrchestratorNamespace;

View File

@@ -0,0 +1,10 @@
import Input from '../../input';
import OrchestratorOptions from './orchestrator-options';
class OrchestratorOptionsReader {
static GetProperties() {
return [...Object.getOwnPropertyNames(Input), ...Object.getOwnPropertyNames(OrchestratorOptions)];
}
}
export default OrchestratorOptionsReader;

View File

@@ -0,0 +1,372 @@
import { Cli } from '../../cli/cli';
import OrchestratorQueryOverride from './orchestrator-query-override';
import GitHub from '../../github';
import * as core from '@actions/core';
class OrchestratorOptions {
// ### ### ###
// Input Handling
// ### ### ###
public static getInput(query: string): string | undefined {
if (GitHub.githubInputEnabled) {
const coreInput = core.getInput(query);
if (coreInput && coreInput !== '') {
return coreInput;
}
}
const alternativeQuery = OrchestratorOptions.ToEnvVarFormat(query);
// Query input sources
if (Cli.query(query, alternativeQuery)) {
return Cli.query(query, alternativeQuery);
}
if (OrchestratorQueryOverride.query(query, alternativeQuery)) {
return OrchestratorQueryOverride.query(query, alternativeQuery);
}
if (process.env[query] !== undefined) {
return process.env[query];
}
if (alternativeQuery !== query && process.env[alternativeQuery] !== undefined) {
return process.env[alternativeQuery];
}
}
public static ToEnvVarFormat(input: string): string {
if (input.toUpperCase() === input) {
return input;
}
return input
.replace(/([A-Z])/g, ' $1')
.trim()
.toUpperCase()
.replace(/ /g, '_');
}
// ### ### ###
// Provider parameters
// ### ### ###
static get region(): string {
return OrchestratorOptions.getInput('region') || 'eu-west-2';
}
// ### ### ###
// GitHub parameters
// ### ### ###
static get githubChecks(): boolean {
const value = OrchestratorOptions.getInput('githubChecks');
return value === `true` || false;
}
static get githubCheckId(): string {
return OrchestratorOptions.getInput('githubCheckId') || ``;
}
static get githubOwner(): string {
return OrchestratorOptions.getInput('githubOwner') || OrchestratorOptions.githubRepo?.split(`/`)[0] || '';
}
static get githubRepoName(): string {
return OrchestratorOptions.getInput('githubRepoName') || OrchestratorOptions.githubRepo?.split(`/`)[1] || '';
}
static get orchestratorRepoName(): string {
return OrchestratorOptions.getInput('orchestratorRepoName') || 'game-ci/unity-builder';
}
static get cloneDepth(): string {
return OrchestratorOptions.getInput('cloneDepth') || '50';
}
static get finalHooks(): string[] {
return OrchestratorOptions.getInput('finalHooks')?.split(',') || [];
}
// ### ### ###
// Git syncronization parameters
// ### ### ###
static get githubRepo(): string | undefined {
return (
OrchestratorOptions.getInput('GITHUB_REPOSITORY') || OrchestratorOptions.getInput('GITHUB_REPO') || undefined
);
}
static get branch(): string {
if (OrchestratorOptions.getInput(`GITHUB_REF`)) {
return (
OrchestratorOptions.getInput(`GITHUB_REF`)?.replace('refs/', '').replace(`head/`, '').replace(`heads/`, '') ||
``
);
} else if (OrchestratorOptions.getInput('branch')) {
return OrchestratorOptions.getInput('branch') || ``;
} else {
return '';
}
}
// ### ### ###
// Orchestrator parameters
// ### ### ###
static get buildPlatform(): string {
const input = OrchestratorOptions.getInput('buildPlatform');
if (input && input !== '') {
return input;
}
if (OrchestratorOptions.providerStrategy !== 'local') {
return 'linux';
}
return process.platform;
}
static get orchestratorBranch(): string {
return OrchestratorOptions.getInput('orchestratorBranch') || 'main';
}
static get providerStrategy(): string {
const provider =
OrchestratorOptions.getInput('orchestratorCluster') || OrchestratorOptions.getInput('providerStrategy');
if (Cli.isCliMode) {
return provider || 'aws';
}
return provider || 'local';
}
static get fallbackProviderStrategy(): string {
return OrchestratorOptions.getInput('fallbackProviderStrategy') || '';
}
static get runnerCheckEnabled(): boolean {
return OrchestratorOptions.getInput('runnerCheckEnabled') === 'true';
}
static get runnerCheckLabels(): string[] {
const labels = OrchestratorOptions.getInput('runnerCheckLabels');
return labels ? labels.split(',').map((l) => l.trim()) : [];
}
static get runnerCheckMinAvailable(): number {
return Number(OrchestratorOptions.getInput('runnerCheckMinAvailable')) || 1;
}
static get retryOnFallback(): boolean {
return OrchestratorOptions.getInput('retryOnFallback') === 'true';
}
static get providerInitTimeout(): number {
return Number(OrchestratorOptions.getInput('providerInitTimeout')) || 0;
}
static get gitAuthMode(): string {
return OrchestratorOptions.getInput('gitAuthMode') || 'header';
}
static get containerCpu(): string {
return OrchestratorOptions.getInput('containerCpu') || `1024`;
}
static get containerMemory(): string {
return OrchestratorOptions.getInput('containerMemory') || `3072`;
}
static get containerNamespace(): string {
return OrchestratorOptions.getInput('containerNamespace') || `default`;
}
static get customJob(): string {
return OrchestratorOptions.getInput('customJob') || '';
}
// ### ### ###
// Custom commands from files parameters
// ### ### ###
static get containerHookFiles(): string[] {
return OrchestratorOptions.getInput('containerHookFiles')?.split(`,`) || [];
}
static get commandHookFiles(): string[] {
return OrchestratorOptions.getInput('commandHookFiles')?.split(`,`) || [];
}
// ### ### ###
// Custom commands from yaml parameters
// ### ### ###
static get commandHooks(): string {
return OrchestratorOptions.getInput('commandHooks') || '';
}
static get postBuildContainerHooks(): string {
return OrchestratorOptions.getInput('postBuildContainerHooks') || '';
}
static get preBuildContainerHooks(): string {
return OrchestratorOptions.getInput('preBuildContainerHooks') || '';
}
// ### ### ###
// Input override handling
// ### ### ###
static get pullInputList(): string[] {
return OrchestratorOptions.getInput('pullInputList')?.split(`,`) || [];
}
static get secretSource(): string {
return OrchestratorOptions.getInput('secretSource') || '';
}
static get inputPullCommand(): string {
const value = OrchestratorOptions.getInput('inputPullCommand');
if (value === 'gcp-secret-manager') {
return 'gcloud secrets versions access 1 --secret="{0}"';
} else if (value === 'aws-secret-manager') {
return 'aws secretsmanager get-secret-value --secret-id {0}';
}
return value || '';
}
// ### ### ###
// Aws
// ### ### ###
static get awsStackName() {
return OrchestratorOptions.getInput('awsStackName') || 'game-ci';
}
static get awsEndpoint(): string | undefined {
return OrchestratorOptions.getInput('awsEndpoint');
}
static get awsCloudFormationEndpoint(): string | undefined {
return OrchestratorOptions.getInput('awsCloudFormationEndpoint') || OrchestratorOptions.awsEndpoint;
}
static get awsEcsEndpoint(): string | undefined {
return OrchestratorOptions.getInput('awsEcsEndpoint') || OrchestratorOptions.awsEndpoint;
}
static get awsKinesisEndpoint(): string | undefined {
return OrchestratorOptions.getInput('awsKinesisEndpoint') || OrchestratorOptions.awsEndpoint;
}
static get awsCloudWatchLogsEndpoint(): string | undefined {
return OrchestratorOptions.getInput('awsCloudWatchLogsEndpoint') || OrchestratorOptions.awsEndpoint;
}
static get awsS3Endpoint(): string | undefined {
return OrchestratorOptions.getInput('awsS3Endpoint') || OrchestratorOptions.awsEndpoint;
}
// ### ### ###
// Storage
// ### ### ###
static get storageProvider(): string {
return OrchestratorOptions.getInput('storageProvider') || 's3';
}
static get rcloneRemote(): string {
return OrchestratorOptions.getInput('rcloneRemote') || '';
}
// ### ### ###
// K8s
// ### ### ###
static get kubeConfig(): string {
return OrchestratorOptions.getInput('kubeConfig') || '';
}
static get kubeVolume(): string {
return OrchestratorOptions.getInput('kubeVolume') || '';
}
static get kubeVolumeSize(): string {
return OrchestratorOptions.getInput('kubeVolumeSize') || '25Gi';
}
static get kubeStorageClass(): string {
return OrchestratorOptions.getInput('kubeStorageClass') || '';
}
// ### ### ###
// Caching
// ### ### ###
static get cacheKey(): string {
return OrchestratorOptions.getInput('cacheKey') || OrchestratorOptions.branch;
}
// ### ### ###
// Utility Parameters
// ### ### ###
static get orchestratorDebug(): boolean {
return (
OrchestratorOptions.getInput(`orchestratorTests`) === `true` ||
OrchestratorOptions.getInput(`orchestratorDebug`) === `true` ||
OrchestratorOptions.getInput(`orchestratorDebugTree`) === `true` ||
OrchestratorOptions.getInput(`orchestratorDebugEnv`) === `true` ||
false
);
}
static get skipLfs(): boolean {
return OrchestratorOptions.getInput(`skipLfs`) === `true`;
}
static get skipCache(): boolean {
return OrchestratorOptions.getInput(`skipCache`) === `true`;
}
public static get asyncOrchestrator(): boolean {
return OrchestratorOptions.getInput('asyncOrchestrator') === 'true';
}
public static get resourceTracking(): boolean {
return OrchestratorOptions.getInput('resourceTracking') === 'true';
}
public static get useLargePackages(): boolean {
return OrchestratorOptions.getInput(`useLargePackages`) === `true`;
}
public static get useSharedBuilder(): boolean {
return OrchestratorOptions.getInput(`useSharedBuilder`) === `true`;
}
public static get useCompressionStrategy(): boolean {
return OrchestratorOptions.getInput(`useCompressionStrategy`) === `true`;
}
public static get useCleanupCron(): boolean {
return (OrchestratorOptions.getInput(`useCleanupCron`) || 'true') === 'true';
}
// ### ### ###
// Retained Workspace
// ### ### ###
public static get maxRetainedWorkspaces(): string {
return OrchestratorOptions.getInput(`maxRetainedWorkspaces`) || `0`;
}
// ### ### ###
// Garbage Collection
// ### ### ###
static get garbageMaxAge(): number {
return Number(OrchestratorOptions.getInput(`garbageMaxAge`)) || 24;
}
}
export default OrchestratorOptions;

View File

@@ -0,0 +1,116 @@
import * as core from '@actions/core';
import Input from '../../input';
import { GenericInputReader } from '../../input-readers/generic-input-reader';
import OrchestratorOptions from './orchestrator-options';
import { SecretSourceService, validateSecretKey } from '../services/secrets/secret-source-service';
import OrchestratorLogger from '../services/core/orchestrator-logger';
const formatFunction = (value: string, arguments_: any[]) => {
for (const element of arguments_) {
value = value.replace(`{${element.key}}`, element.value);
}
return value;
};
class OrchestratorQueryOverride {
static queryOverrides: { [key: string]: string } | undefined;
public static query(key: string, alternativeKey: string) {
if (OrchestratorQueryOverride.queryOverrides && OrchestratorQueryOverride.queryOverrides[key] !== undefined) {
return OrchestratorQueryOverride.queryOverrides[key];
}
if (
OrchestratorQueryOverride.queryOverrides &&
alternativeKey &&
OrchestratorQueryOverride.queryOverrides[alternativeKey] !== undefined
) {
return OrchestratorQueryOverride.queryOverrides[alternativeKey];
}
return;
}
private static shouldUseOverride(query: string) {
if (OrchestratorOptions.inputPullCommand !== '') {
if (OrchestratorOptions.pullInputList.length > 0) {
const doesInclude =
OrchestratorOptions.pullInputList.includes(query) ||
OrchestratorOptions.pullInputList.includes(Input.ToEnvVarFormat(query));
return doesInclude ? true : false;
} else {
return true;
}
}
}
private static async queryOverride(query: string) {
if (!this.shouldUseOverride(query)) {
throw new Error(`Should not be trying to run override query on ${query}`);
}
// Validate the query key before interpolating it into a shell command
validateSecretKey(query);
const result = await GenericInputReader.Run(
formatFunction(OrchestratorOptions.inputPullCommand, [{ key: 0, value: query }]),
);
// Mask the fetched secret value so it does not appear in GitHub Actions logs
if (result && result.trim().length > 0) {
core.setSecret(result);
}
return result;
}
/**
* Populate query overrides using either:
* 1. Premade/custom secret sources (via secretSource input), or
* 2. Shell command (via inputPullCommand, legacy approach)
*
* The secretSource input takes precedence if set. It supports:
* - Premade names: 'aws-secrets-manager', 'aws-parameter-store', 'gcp-secret-manager', 'azure-key-vault', 'env'
* - Custom commands: any string containing {0} placeholder
* - YAML file path: a path ending in .yml or .yaml containing custom source definitions
*/
public static async PopulateQueryOverrideInput() {
const queries = OrchestratorOptions.pullInputList;
OrchestratorQueryOverride.queryOverrides = {};
const secretSource = OrchestratorOptions.secretSource;
// Use SecretSourceService if secretSource is configured
if (secretSource) {
OrchestratorLogger.log(`Using secret source: ${secretSource}`);
// YAML file: load definitions and use the first source
if (secretSource.endsWith('.yml') || secretSource.endsWith('.yaml')) {
const definitions = SecretSourceService.loadFromYaml(secretSource);
if (definitions.length > 0) {
OrchestratorLogger.log(`Loaded ${definitions.length} secret source(s) from ${secretSource}`);
for (const key of queries) {
OrchestratorQueryOverride.queryOverrides[key] = await SecretSourceService.fetchSecret(definitions[0], key);
}
}
return;
}
// Premade or custom command source
const results = await SecretSourceService.fetchAll(secretSource, queries);
Object.assign(OrchestratorQueryOverride.queryOverrides, results);
return;
}
// Legacy: use inputPullCommand if set
for (const element of queries) {
if (OrchestratorQueryOverride.shouldUseOverride(element)) {
OrchestratorQueryOverride.queryOverrides[element] = await OrchestratorQueryOverride.queryOverride(element);
}
}
}
}
export default OrchestratorQueryOverride;

View File

@@ -1,6 +1,6 @@
class CloudRunnerSecret {
class OrchestratorSecret {
public ParameterKey!: string;
public EnvironmentVariable!: string;
public ParameterValue!: string;
}
export default CloudRunnerSecret;
export default OrchestratorSecret;

View File

@@ -0,0 +1,3 @@
export class OrchestratorStatics {
public static readonly logPrefix = `Orchestrator`;
}

View File

@@ -0,0 +1,13 @@
import OrchestratorEnvironmentVariable from './orchestrator-environment-variable';
import OrchestratorSecret from './orchestrator-secret';
export class OrchestratorStepParameters {
public image: string;
public environment: OrchestratorEnvironmentVariable[];
public secrets: OrchestratorSecret[];
constructor(image: string, environmentVariables: OrchestratorEnvironmentVariable[], secrets: OrchestratorSecret[]) {
this.image = image;
this.environment = environmentVariables;
this.secrets = secrets;
}
}

View File

@@ -0,0 +1,473 @@
import AwsBuildPlatform from './providers/aws';
import { BuildParameters, Input } from '..';
import Kubernetes from './providers/k8s';
import OrchestratorLogger from './services/core/orchestrator-logger';
import { OrchestratorStepParameters } from './options/orchestrator-step-parameters';
import { WorkflowCompositionRoot } from './workflows/workflow-composition-root';
import { OrchestratorError } from './error/orchestrator-error';
import { TaskParameterSerializer } from './services/core/task-parameter-serializer';
import * as core from '@actions/core';
import OrchestratorSecret from './options/orchestrator-secret';
import { ProviderInterface } from './providers/provider-interface';
import OrchestratorEnvironmentVariable from './options/orchestrator-environment-variable';
import TestOrchestrator from './providers/test';
import LocalOrchestrator from './providers/local';
import LocalDockerOrchestrator from './providers/docker';
import GcpCloudRunProvider from './providers/gcp-cloud-run';
import AzureAciProvider from './providers/azure-aci';
import RemotePowershellProvider from './providers/remote-powershell';
import GitHubActionsProvider from './providers/github-actions';
import GitLabCIProvider from './providers/gitlab-ci';
import AnsibleProvider from './providers/ansible';
import loadProvider from './providers/provider-loader';
import GitHub from '../github';
import SharedWorkspaceLocking from './services/core/shared-workspace-locking';
import { FollowLogStreamService } from './services/core/follow-log-stream-service';
import OrchestratorResult from './services/core/orchestrator-result';
import OrchestratorOptions from './options/orchestrator-options';
import ResourceTracking from './services/core/resource-tracking';
import { RunnerAvailabilityService } from './services/core/runner-availability-service';
class Orchestrator {
public static Provider: ProviderInterface;
public static buildParameters: BuildParameters;
private static defaultSecrets: OrchestratorSecret[];
private static orchestratorEnvironmentVariables: OrchestratorEnvironmentVariable[];
static lockedWorkspace: string = ``;
public static readonly retainedWorkspacePrefix: string = `retained-workspace`;
// When true, validates AWS CloudFormation templates even when using local-docker execution
// This is set by AWS_FORCE_PROVIDER=aws-local mode
public static validateAwsTemplates: boolean = false;
public static get isOrchestratorEnvironment() {
return process.env[`GITHUB_ACTIONS`] !== `true`;
}
public static get isOrchestratorAsyncEnvironment() {
return process.env[`ASYNC_WORKFLOW`] === `true`;
}
public static async setup(buildParameters: BuildParameters) {
OrchestratorLogger.setup();
OrchestratorLogger.log(`Setting up orchestrator`);
Orchestrator.buildParameters = buildParameters;
ResourceTracking.logAllocationSummary('setup');
await ResourceTracking.logDiskUsageSnapshot('setup');
if (Orchestrator.buildParameters.githubCheckId === ``) {
Orchestrator.buildParameters.githubCheckId = await GitHub.createGitHubCheck(
Orchestrator.buildParameters.buildGuid,
);
}
await Orchestrator.setupSelectedBuildPlatform();
Orchestrator.defaultSecrets = TaskParameterSerializer.readDefaultSecrets();
Orchestrator.orchestratorEnvironmentVariables =
TaskParameterSerializer.createOrchestratorEnvironmentVariables(buildParameters);
if (GitHub.githubInputEnabled) {
const buildParameterPropertyNames = Object.getOwnPropertyNames(buildParameters);
for (const element of Orchestrator.orchestratorEnvironmentVariables) {
// OrchestratorLogger.log(`Orchestrator output ${Input.ToEnvVarFormat(element.name)} = ${element.value}`);
core.setOutput(Input.ToEnvVarFormat(element.name), element.value);
}
for (const element of buildParameterPropertyNames) {
// OrchestratorLogger.log(`Orchestrator output ${Input.ToEnvVarFormat(element)} = ${buildParameters[element]}`);
core.setOutput(Input.ToEnvVarFormat(element), buildParameters[element]);
}
core.setOutput(
Input.ToEnvVarFormat(`buildArtifact`),
`build-${Orchestrator.buildParameters.buildGuid}.tar${
Orchestrator.buildParameters.useCompressionStrategy ? '.lz4' : ''
}`,
);
}
FollowLogStreamService.Reset();
}
private static async setupSelectedBuildPlatform() {
OrchestratorLogger.log(`Orchestrator platform selected ${Orchestrator.buildParameters.providerStrategy}`);
// Check runner availability and apply fallback if needed
if (Orchestrator.buildParameters.runnerCheckEnabled && Orchestrator.buildParameters.fallbackProviderStrategy) {
const owner = OrchestratorOptions.githubOwner;
const repo = OrchestratorOptions.githubRepoName;
const token = Orchestrator.buildParameters.gitPrivateToken || process.env.GITHUB_TOKEN || '';
OrchestratorLogger.log(
`Checking runner availability (labels: [${Orchestrator.buildParameters.runnerCheckLabels.join(', ')}], min: ${
Orchestrator.buildParameters.runnerCheckMinAvailable
})`,
);
const result = await RunnerAvailabilityService.checkAvailability(
owner,
repo,
token,
Orchestrator.buildParameters.runnerCheckLabels,
Orchestrator.buildParameters.runnerCheckMinAvailable,
);
OrchestratorLogger.log(
`Runner check: ${result.totalRunners} total, ${result.matchingRunners} matching, ${result.idleRunners} idle — ${result.reason}`,
);
if (result.shouldFallback) {
const original = Orchestrator.buildParameters.providerStrategy;
const fallback = Orchestrator.buildParameters.fallbackProviderStrategy;
OrchestratorLogger.log(`Falling back from '${original}' to '${fallback}' — ${result.reason}`);
Orchestrator.buildParameters.providerStrategy = fallback;
core.setOutput('providerFallbackUsed', 'true');
core.setOutput('providerFallbackReason', result.reason);
} else {
core.setOutput('providerFallbackUsed', 'false');
}
}
// Detect LocalStack endpoints and handle AWS provider appropriately
// AWS_FORCE_PROVIDER options:
// - 'aws': Force AWS provider (requires LocalStack Pro with ECS support)
// - 'aws-local': Validate AWS templates/config but execute via local-docker (for CI without ECS)
// - unset/other: Auto-fallback to local-docker when LocalStack detected
const awsForceProvider = process.env.AWS_FORCE_PROVIDER || '';
const forceAwsProvider = awsForceProvider === 'aws' || awsForceProvider === 'true';
const useAwsLocalMode = awsForceProvider === 'aws-local';
const endpointsToCheck = [
process.env.AWS_ENDPOINT,
process.env.AWS_S3_ENDPOINT,
process.env.AWS_CLOUD_FORMATION_ENDPOINT,
process.env.AWS_ECS_ENDPOINT,
process.env.AWS_KINESIS_ENDPOINT,
process.env.AWS_CLOUD_WATCH_LOGS_ENDPOINT,
OrchestratorOptions.awsEndpoint,
OrchestratorOptions.awsS3Endpoint,
OrchestratorOptions.awsCloudFormationEndpoint,
OrchestratorOptions.awsEcsEndpoint,
OrchestratorOptions.awsKinesisEndpoint,
OrchestratorOptions.awsCloudWatchLogsEndpoint,
]
.filter((x) => typeof x === 'string')
.join(' ');
const isLocalStack = /localstack|localhost|127\.0\.0\.1/i.test(endpointsToCheck);
let provider = Orchestrator.buildParameters.providerStrategy;
let validateAwsTemplates = false;
if (provider === 'aws' && isLocalStack) {
if (useAwsLocalMode) {
// aws-local mode: Validate AWS templates but execute via local-docker
// This provides confidence in AWS CloudFormation without requiring LocalStack Pro
OrchestratorLogger.log('AWS_FORCE_PROVIDER=aws-local: Validating AWS templates, executing via local-docker');
validateAwsTemplates = true;
provider = 'local-docker';
} else if (forceAwsProvider) {
// Force full AWS provider (requires LocalStack Pro with ECS support)
OrchestratorLogger.log(
'LocalStack endpoints detected but AWS_FORCE_PROVIDER=aws; using full AWS provider (requires ECS support)',
);
} else {
// Auto-fallback to local-docker
OrchestratorLogger.log('LocalStack endpoints detected; routing provider to local-docker for this run');
OrchestratorLogger.log(
'Note: Set AWS_FORCE_PROVIDER=aws-local to validate AWS templates with local-docker execution',
);
provider = 'local-docker';
}
}
// Store whether we should validate AWS templates (used by aws-local mode)
Orchestrator.validateAwsTemplates = validateAwsTemplates;
// Check for CLI provider executable
if (Orchestrator.buildParameters.providerExecutable) {
const { default: CliProvider } = await import('./providers/cli');
Orchestrator.Provider = new CliProvider(
Orchestrator.buildParameters.providerExecutable,
Orchestrator.buildParameters,
);
OrchestratorLogger.log(`Using CLI provider executable: ${Orchestrator.buildParameters.providerExecutable}`);
return;
}
switch (provider) {
case 'k8s':
Orchestrator.Provider = new Kubernetes(Orchestrator.buildParameters);
break;
case 'aws':
Orchestrator.Provider = new AwsBuildPlatform(Orchestrator.buildParameters);
// Validate that AWS provider is actually being used when expected
if (isLocalStack && forceAwsProvider) {
OrchestratorLogger.log('✓ AWS provider initialized with LocalStack - AWS functionality will be validated');
} else if (isLocalStack && !forceAwsProvider) {
OrchestratorLogger.log(
'⚠ WARNING: AWS provider was requested but LocalStack detected without AWS_FORCE_PROVIDER',
);
OrchestratorLogger.log('⚠ This may cause AWS functionality tests to fail validation');
}
break;
case 'test':
Orchestrator.Provider = new TestOrchestrator();
break;
case 'local-docker':
Orchestrator.Provider = new LocalDockerOrchestrator();
break;
case 'local-system':
Orchestrator.Provider = new LocalOrchestrator();
break;
case 'local':
Orchestrator.Provider = new LocalOrchestrator();
break;
case 'gcp-cloud-run':
OrchestratorLogger.log('⚠ EXPERIMENTAL: GCP Cloud Run Jobs provider');
Orchestrator.Provider = new GcpCloudRunProvider(Orchestrator.buildParameters);
break;
case 'azure-aci':
OrchestratorLogger.log('⚠ EXPERIMENTAL: Azure Container Instances provider');
Orchestrator.Provider = new AzureAciProvider(Orchestrator.buildParameters);
case 'remote-powershell':
Orchestrator.Provider = new RemotePowershellProvider(Orchestrator.buildParameters);
break;
case 'github-actions':
Orchestrator.Provider = new GitHubActionsProvider(Orchestrator.buildParameters);
break;
case 'gitlab-ci':
Orchestrator.Provider = new GitLabCIProvider(Orchestrator.buildParameters);
break;
case 'ansible':
Orchestrator.Provider = new AnsibleProvider(Orchestrator.buildParameters);
break;
default:
// Try to load provider using the dynamic loader for unknown providers
try {
Orchestrator.Provider = await loadProvider(provider, Orchestrator.buildParameters);
} catch (error: any) {
OrchestratorLogger.log(`Failed to load provider '${provider}' using dynamic loader: ${error.message}`);
OrchestratorLogger.log('Falling back to local provider...');
Orchestrator.Provider = new LocalOrchestrator();
}
break;
}
// Final validation: Ensure provider matches expectations
const finalProviderName = Orchestrator.Provider.constructor.name;
if (Orchestrator.buildParameters.providerStrategy === 'aws' && finalProviderName !== 'AWSBuildEnvironment') {
OrchestratorLogger.log(`⚠ WARNING: Expected AWS provider but got ${finalProviderName}`);
OrchestratorLogger.log('⚠ AWS functionality tests may not be validating AWS services correctly');
}
}
static async run(buildParameters: BuildParameters, baseImage: string) {
if (baseImage.includes(`undefined`)) {
throw new Error(`baseImage is undefined`);
}
try {
return await Orchestrator.runWithProvider(buildParameters, baseImage);
} catch (primaryError: any) {
// Retry on fallback provider if enabled and a fallback is configured
const fallback = buildParameters.fallbackProviderStrategy;
const alreadyOnFallback = buildParameters.providerStrategy === fallback;
if (buildParameters.retryOnFallback && fallback && !alreadyOnFallback) {
OrchestratorLogger.log(
`Primary provider '${buildParameters.providerStrategy}' failed: ${primaryError.message}`,
);
OrchestratorLogger.log(`Retrying build on fallback provider '${fallback}'...`);
buildParameters.providerStrategy = fallback;
core.setOutput('providerFallbackUsed', 'true');
core.setOutput('providerFallbackReason', `Primary provider failed: ${primaryError.message}`);
return await Orchestrator.runWithProvider(buildParameters, baseImage);
}
throw primaryError;
}
}
private static async runWithProvider(buildParameters: BuildParameters, baseImage: string) {
await Orchestrator.setup(buildParameters);
// When aws-local mode is enabled, validate AWS CloudFormation templates
// This ensures AWS templates are correct even when executing via local-docker
if (Orchestrator.validateAwsTemplates) {
await Orchestrator.validateAwsCloudFormationTemplates();
}
// Setup workflow with optional init timeout
await Orchestrator.setupWorkflowWithTimeout();
try {
if (buildParameters.maxRetainedWorkspaces > 0) {
Orchestrator.lockedWorkspace = SharedWorkspaceLocking.NewWorkspaceName();
const result = await SharedWorkspaceLocking.GetLockedWorkspace(
Orchestrator.lockedWorkspace,
Orchestrator.buildParameters.buildGuid,
Orchestrator.buildParameters,
);
if (result) {
OrchestratorLogger.logLine(`Using retained workspace ${Orchestrator.lockedWorkspace}`);
Orchestrator.orchestratorEnvironmentVariables = [
...Orchestrator.orchestratorEnvironmentVariables,
{ name: `LOCKED_WORKSPACE`, value: Orchestrator.lockedWorkspace },
];
} else {
OrchestratorLogger.log(`Max retained workspaces reached ${buildParameters.maxRetainedWorkspaces}`);
buildParameters.maxRetainedWorkspaces = 0;
Orchestrator.lockedWorkspace = ``;
}
}
await Orchestrator.updateStatusWithBuildParameters();
const output = await new WorkflowCompositionRoot().run(
new OrchestratorStepParameters(
baseImage,
Orchestrator.orchestratorEnvironmentVariables,
Orchestrator.defaultSecrets,
),
);
await Orchestrator.Provider.cleanupWorkflow(
Orchestrator.buildParameters,
Orchestrator.buildParameters.branch,
Orchestrator.defaultSecrets,
);
if (!Orchestrator.buildParameters.isCliMode) core.endGroup();
if (buildParameters.asyncWorkflow && this.isOrchestratorEnvironment && this.isOrchestratorAsyncEnvironment) {
await GitHub.updateGitHubCheck(Orchestrator.buildParameters.buildGuid, `success`, `success`, `completed`);
}
if (BuildParameters.shouldUseRetainedWorkspaceMode(buildParameters)) {
const workspace = Orchestrator.lockedWorkspace || ``;
await SharedWorkspaceLocking.ReleaseWorkspace(
workspace,
Orchestrator.buildParameters.buildGuid,
Orchestrator.buildParameters,
);
const isLocked = await SharedWorkspaceLocking.IsWorkspaceLocked(workspace, Orchestrator.buildParameters);
if (isLocked) {
throw new Error(
`still locked after releasing ${await SharedWorkspaceLocking.GetAllLocksForWorkspace(
workspace,
buildParameters,
)}`,
);
}
Orchestrator.lockedWorkspace = ``;
}
await GitHub.triggerWorkflowOnComplete(Orchestrator.buildParameters.finalHooks);
if (buildParameters.constantGarbageCollection) {
Orchestrator.Provider.garbageCollect(``, true, buildParameters.garbageMaxAge, true, true);
}
return new OrchestratorResult(buildParameters, output, true, true, false);
} catch (error: any) {
OrchestratorLogger.log(JSON.stringify(error, undefined, 4));
await GitHub.updateGitHubCheck(
Orchestrator.buildParameters.buildGuid,
`Failed - Error ${error?.message || error}`,
`failure`,
`completed`,
);
if (!Orchestrator.buildParameters.isCliMode) core.endGroup();
await OrchestratorError.handleException(error, Orchestrator.buildParameters, Orchestrator.defaultSecrets);
throw error;
}
}
/**
* Runs setupWorkflow with an optional timeout. If providerInitTimeout is set and the
* provider takes longer than that to initialize, throws an error that triggers
* retry-on-fallback (if enabled).
*/
private static async setupWorkflowWithTimeout() {
const timeoutSeconds = Orchestrator.buildParameters.providerInitTimeout;
const setupPromise = Orchestrator.Provider.setupWorkflow(
Orchestrator.buildParameters.buildGuid,
Orchestrator.buildParameters,
Orchestrator.buildParameters.branch,
Orchestrator.defaultSecrets,
);
if (timeoutSeconds <= 0) {
await setupPromise;
return;
}
OrchestratorLogger.log(`Provider init timeout: ${timeoutSeconds}s`);
const timeoutPromise = new Promise<never>((_, reject) => {
setTimeout(
() => reject(new Error(`Provider initialization timed out after ${timeoutSeconds}s`)),
timeoutSeconds * 1000,
);
});
await Promise.race([setupPromise, timeoutPromise]);
}
private static async updateStatusWithBuildParameters() {
const content = { ...Orchestrator.buildParameters };
content.gitPrivateToken = ``;
content.unitySerial = ``;
content.unityEmail = ``;
content.unityPassword = ``;
const jsonContent = JSON.stringify(content, undefined, 4);
await GitHub.updateGitHubCheck(jsonContent, Orchestrator.buildParameters.buildGuid);
}
/**
* Validates AWS CloudFormation templates without deploying them.
* Used by aws-local mode to ensure AWS templates are correct when executing via local-docker.
* This provides confidence that AWS ECS deployments would work with the generated templates.
*/
private static async validateAwsCloudFormationTemplates() {
OrchestratorLogger.log('=== AWS CloudFormation Template Validation (aws-local mode) ===');
try {
// Import AWS template formations
const { BaseStackFormation } = await import('./providers/aws/cloud-formations/base-stack-formation');
const { TaskDefinitionFormation } = await import('./providers/aws/cloud-formations/task-definition-formation');
// Validate base stack template
const baseTemplate = BaseStackFormation.formation;
OrchestratorLogger.log(`✓ Base stack template generated (${baseTemplate.length} chars)`);
// Check for required resources in base stack
const requiredBaseResources = ['AWS::EC2::VPC', 'AWS::ECS::Cluster', 'AWS::S3::Bucket', 'AWS::IAM::Role'];
for (const resource of requiredBaseResources) {
if (baseTemplate.includes(resource)) {
OrchestratorLogger.log(` ✓ Contains ${resource}`);
} else {
throw new Error(`Base stack template missing required resource: ${resource}`);
}
}
// Validate task definition template
const taskTemplate = TaskDefinitionFormation.formation;
OrchestratorLogger.log(`✓ Task definition template generated (${taskTemplate.length} chars)`);
// Check for required resources in task definition
const requiredTaskResources = ['AWS::ECS::TaskDefinition', 'AWS::Logs::LogGroup'];
for (const resource of requiredTaskResources) {
if (taskTemplate.includes(resource)) {
OrchestratorLogger.log(` ✓ Contains ${resource}`);
} else {
throw new Error(`Task definition template missing required resource: ${resource}`);
}
}
// Validate YAML syntax by checking for common patterns
if (!baseTemplate.includes('AWSTemplateFormatVersion')) {
throw new Error('Base stack template missing AWSTemplateFormatVersion');
}
if (!taskTemplate.includes('AWSTemplateFormatVersion')) {
throw new Error('Task definition template missing AWSTemplateFormatVersion');
}
OrchestratorLogger.log('=== AWS CloudFormation templates validated successfully ===');
OrchestratorLogger.log('Note: Actual execution will use local-docker provider');
} catch (error: any) {
OrchestratorLogger.log(`AWS CloudFormation template validation failed: ${error.message}`);
throw error;
}
}
}
export default Orchestrator;

View File

@@ -0,0 +1,222 @@
# Provider Loader Dynamic Imports
## What is a Provider?
A **provider** is a pluggable backend that Orchestrator uses to run builds and workflows. Examples include **AWS**, **Kubernetes**, or local execution. Each provider implements the [ProviderInterface](https://github.com/game-ci/unity-builder/blob/main/src/model/orchestrator/providers/provider-interface.ts), which defines the common lifecycle methods (setup, run, cleanup, garbage collection, etc.).
This abstraction makes Orchestrator flexible: you can switch execution environments or add your own provider (via npm package, GitHub repo, or local path) without changing the rest of your pipeline.
## Dynamic Provider Loading
The provider loader now supports dynamic loading of providers from multiple sources including local file paths, GitHub repositories, and NPM packages.
## Features
- **Local File Paths**: Load providers from relative or absolute file paths
- **GitHub URLs**: Clone and load providers from GitHub repositories with automatic updates
- **NPM Packages**: Load providers from installed NPM packages
- **Automatic Updates**: GitHub repositories are automatically updated when changes are available
- **Caching**: Local caching of cloned repositories for improved performance
- **Fallback Support**: Graceful fallback to local provider if loading fails
## Usage Examples
### Loading Built-in Providers
```typescript
import { ProviderLoader } from './provider-loader';
// Load built-in providers
const awsProvider = await ProviderLoader.loadProvider('aws', buildParameters);
const k8sProvider = await ProviderLoader.loadProvider('k8s', buildParameters);
```
### Loading Local Providers
```typescript
// Load from relative path
const localProvider = await ProviderLoader.loadProvider('./my-local-provider', buildParameters);
// Load from absolute path
const absoluteProvider = await ProviderLoader.loadProvider('/path/to/provider', buildParameters);
```
### Loading GitHub Providers
```typescript
// Load from GitHub URL
const githubProvider = await ProviderLoader.loadProvider(
'https://github.com/user/my-provider',
buildParameters
);
// Load from specific branch
const branchProvider = await ProviderLoader.loadProvider(
'https://github.com/user/my-provider/tree/develop',
buildParameters
);
// Load from specific path in repository
const pathProvider = await ProviderLoader.loadProvider(
'https://github.com/user/my-provider/tree/main/src/providers',
buildParameters
);
// Shorthand notation
const shorthandProvider = await ProviderLoader.loadProvider('user/repo', buildParameters);
const branchShorthand = await ProviderLoader.loadProvider('user/repo@develop', buildParameters);
```
### Loading NPM Packages
```typescript
// Load from NPM package
const npmProvider = await ProviderLoader.loadProvider('my-provider-package', buildParameters);
// Load from scoped NPM package
const scopedProvider = await ProviderLoader.loadProvider('@scope/my-provider', buildParameters);
```
## Provider Interface
All providers must implement the `ProviderInterface`:
```typescript
interface ProviderInterface {
cleanupWorkflow(): Promise<void>;
setupWorkflow(buildGuid: string, buildParameters: BuildParameters, branchName: string, defaultSecretsArray: any[]): Promise<void>;
runTaskInWorkflow(buildGuid: string, task: string, workingDirectory: string, buildVolumeFolder: string, environmentVariables: any[], secrets: any[]): Promise<string>;
garbageCollect(): Promise<void>;
listResources(): Promise<ProviderResource[]>;
listWorkflow(): Promise<ProviderWorkflow[]>;
watchWorkflow(): Promise<void>;
}
```
## Example Provider Implementation
```typescript
// my-provider.ts
import { ProviderInterface } from './provider-interface';
import BuildParameters from './build-parameters';
export default class MyProvider implements ProviderInterface {
constructor(private buildParameters: BuildParameters) {}
async cleanupWorkflow(): Promise<void> {
// Cleanup logic
}
async setupWorkflow(buildGuid: string, buildParameters: BuildParameters, branchName: string, defaultSecretsArray: any[]): Promise<void> {
// Setup logic
}
async runTaskInWorkflow(buildGuid: string, task: string, workingDirectory: string, buildVolumeFolder: string, environmentVariables: any[], secrets: any[]): Promise<string> {
// Task execution logic
return 'Task completed';
}
async garbageCollect(): Promise<void> {
// Garbage collection logic
}
async listResources(): Promise<ProviderResource[]> {
return [];
}
async listWorkflow(): Promise<ProviderWorkflow[]> {
return [];
}
async watchWorkflow(): Promise<void> {
// Watch logic
}
}
```
## Utility Methods
### Analyze Provider Source
```typescript
// Analyze a provider source without loading it
const sourceInfo = ProviderLoader.analyzeProviderSource('https://github.com/user/repo');
console.log(sourceInfo.type); // 'github'
console.log(sourceInfo.owner); // 'user'
console.log(sourceInfo.repo); // 'repo'
```
### Clean Up Cache
```typescript
// Clean up old cached repositories (older than 30 days)
await ProviderLoader.cleanupCache();
// Clean up repositories older than 7 days
await ProviderLoader.cleanupCache(7);
```
### Get Available Providers
```typescript
// Get list of built-in providers
const providers = ProviderLoader.getAvailableProviders();
console.log(providers); // ['aws', 'k8s', 'test', 'local-docker', 'local-system', 'local']
```
## Supported URL Formats
### GitHub URLs
- `https://github.com/user/repo`
- `https://github.com/user/repo.git`
- `https://github.com/user/repo/tree/branch`
- `https://github.com/user/repo/tree/branch/path/to/provider`
- `git@github.com:user/repo.git`
### Shorthand GitHub References
- `user/repo`
- `user/repo@branch`
- `user/repo@branch/path/to/provider`
### Local Paths
- `./relative/path`
- `../relative/path`
- `/absolute/path`
- `C:\\path\\to\\provider` (Windows)
### NPM Packages
- `package-name`
- `@scope/package-name`
## Caching
GitHub repositories are automatically cached in the `.provider-cache` directory. The cache key is generated based on the repository owner, name, and branch. This ensures that:
1. Repositories are only cloned once
2. Updates are checked and applied automatically
3. Performance is improved for repeated loads
4. Storage is managed efficiently
## Error Handling
The provider loader includes comprehensive error handling:
- **Missing packages**: Clear error messages when providers cannot be found
- **Interface validation**: Ensures providers implement the required interface
- **Git operations**: Handles network issues and repository access problems
- **Fallback mechanism**: Falls back to local provider if loading fails
## Configuration
The provider loader can be configured through environment variables:
- `PROVIDER_CACHE_DIR`: Custom cache directory (default: `.provider-cache`)
- `GIT_TIMEOUT`: Git operation timeout in milliseconds (default: 30000)
## Best Practices
1. **Use specific branches or tags**: Always specify the branch or specific tag when loading from GitHub
2. **Implement proper error handling**: Wrap provider loading in try-catch blocks
3. **Clean up regularly**: Use the cleanup utility to manage cache size
4. **Test locally first**: Test providers locally before deploying
5. **Use semantic versioning**: Tag your provider repositories for stable versions

View File

@@ -0,0 +1,291 @@
import AnsibleProvider from '.';
import BuildParameters from '../../../build-parameters';
import { OrchestratorSystem } from '../../services/core/orchestrator-system';
import OrchestratorLogger from '../../services/core/orchestrator-logger';
import * as core from '@actions/core';
jest.mock('../../services/core/orchestrator-system');
jest.mock('../../services/core/orchestrator-logger');
jest.mock('@actions/core', () => ({
info: jest.fn(),
warning: jest.fn(),
error: jest.fn(),
setOutput: jest.fn(),
getInput: jest.fn(() => ''),
}));
const mockRun = OrchestratorSystem.Run as jest.MockedFunction<typeof OrchestratorSystem.Run>;
const mockLog = OrchestratorLogger.log as jest.MockedFunction<typeof OrchestratorLogger.log>;
const mockLogWarning = OrchestratorLogger.logWarning as jest.MockedFunction<typeof OrchestratorLogger.logWarning>;
function createBuildParameters(overrides: Partial<BuildParameters> = {}): BuildParameters {
return {
ansibleInventory: '/etc/ansible/hosts',
ansiblePlaybook: '/playbooks/unity-build.yml',
ansibleExtraVars: '',
ansibleVaultPassword: '',
...overrides,
} as BuildParameters;
}
describe('AnsibleProvider', () => {
let provider: AnsibleProvider;
beforeEach(() => {
jest.clearAllMocks();
provider = new AnsibleProvider(createBuildParameters());
});
describe('constructor', () => {
it('initializes with all provided parameters', () => {
const params = createBuildParameters({
ansibleInventory: '/custom/inventory',
ansiblePlaybook: '/custom/playbook.yml',
ansibleExtraVars: '{"key":"value"}',
ansibleVaultPassword: '/vault/pass',
});
const p = new AnsibleProvider(params);
expect(p).toBeDefined();
});
it('handles missing optional parameters gracefully', () => {
const params = createBuildParameters({
ansiblePlaybook: undefined,
ansibleExtraVars: undefined,
ansibleVaultPassword: undefined,
});
const p = new AnsibleProvider(params);
expect(p).toBeDefined();
});
});
describe('setupWorkflow', () => {
it('verifies ansible binary, ansible-playbook binary, and inventory exist', async () => {
mockRun.mockResolvedValueOnce('ansible [core 2.14.0]'); // ansible --version
mockRun.mockResolvedValueOnce('/usr/bin/ansible-playbook'); // ansible-playbook check
mockRun.mockResolvedValueOnce(''); // test -e inventory
await provider.setupWorkflow('guid-123', createBuildParameters(), 'main', []);
expect(mockRun).toHaveBeenCalledTimes(3);
expect(mockRun.mock.calls[0][0]).toContain('ansible --version');
expect(mockRun.mock.calls[1][0]).toContain('ansible-playbook');
expect(mockRun.mock.calls[2][0]).toContain('test -e "/etc/ansible/hosts"');
expect(mockLog).toHaveBeenCalledWith(expect.stringContaining('ansible'));
expect(mockLog).toHaveBeenCalledWith(expect.stringContaining('ansible-playbook binary verified'));
});
it('throws when inventory is not configured', async () => {
const params = createBuildParameters({ ansibleInventory: '' });
provider = new AnsibleProvider(params);
await expect(provider.setupWorkflow('guid-123', params, 'main', [])).rejects.toThrow(
'ansibleInventory is required',
);
});
it('throws when ansible binary is not found on PATH', async () => {
mockRun.mockRejectedValueOnce(new Error('command not found: ansible'));
await expect(provider.setupWorkflow('guid-123', createBuildParameters(), 'main', [])).rejects.toThrow(
'Ansible not found on PATH',
);
});
it('throws when ansible-playbook binary is not found', async () => {
mockRun.mockResolvedValueOnce('ansible [core 2.14.0]'); // ansible version OK
mockRun.mockRejectedValueOnce(new Error('command not found')); // ansible-playbook missing
await expect(provider.setupWorkflow('guid-123', createBuildParameters(), 'main', [])).rejects.toThrow(
'ansible-playbook not found on PATH',
);
expect(core.error).toHaveBeenCalledWith('ansible-playbook not found. Install Ansible or ensure it is in PATH.');
});
it('throws when inventory file does not exist', async () => {
mockRun.mockResolvedValueOnce('ansible [core 2.14.0]'); // ansible version OK
mockRun.mockResolvedValueOnce('/usr/bin/ansible-playbook'); // ansible-playbook OK
mockRun.mockRejectedValueOnce(new Error('test -e failed')); // inventory missing
await expect(provider.setupWorkflow('guid-123', createBuildParameters(), 'main', [])).rejects.toThrow(
'Inventory not found: /etc/ansible/hosts',
);
});
});
describe('runTaskInWorkflow', () => {
it('constructs ansible-playbook command with correct variables and returns output', async () => {
mockRun.mockResolvedValueOnce('PLAY [build] *****\nok: [server1]\nPLAY RECAP');
const result = await provider.runTaskInWorkflow(
'guid-run1',
'unityci/editor:2021.3',
'echo build',
'/mount',
'/workspace',
[],
[],
);
expect(result).toContain('PLAY [build]');
const command = mockRun.mock.calls[0][0];
expect(command).toContain('ansible-playbook');
expect(command).toContain('-i "/etc/ansible/hosts"');
expect(command).toContain('"/playbooks/unity-build.yml"');
expect(command).toContain('--no-color');
expect(command).toContain('build_guid');
expect(command).toContain('guid-run1');
expect(command).toContain('build_image');
expect(command).toContain('unityci/editor:2021.3');
expect(command).toContain('build_commands');
expect(command).toContain('mount_dir');
expect(command).toContain('working_dir');
});
it('throws when playbook is not configured', async () => {
const params = createBuildParameters({ ansiblePlaybook: '' });
provider = new AnsibleProvider(params);
await expect(provider.runTaskInWorkflow('guid-nopb', 'img', 'cmd', '/m', '/w', [], [])).rejects.toThrow(
'ansiblePlaybook is required',
);
});
it('passes environment variables as extra-vars in snake_case', async () => {
mockRun.mockResolvedValueOnce('ok');
const env = [
{ name: 'UNITY_LICENSE', value: 'lic-data' },
{ name: 'BUILD_TARGET', value: 'Linux64' },
];
await provider.runTaskInWorkflow('guid-env', 'img', 'cmd', '/m', '/w', env as any, []);
const command = mockRun.mock.calls[0][0];
// Environment variable names are lowercased as Ansible variables
expect(command).toContain('unity_license');
expect(command).toContain('lic-data');
expect(command).toContain('build_target');
expect(command).toContain('Linux64');
});
it('merges user-provided extra vars from JSON string', async () => {
const params = createBuildParameters({
ansibleExtraVars: JSON.stringify({ custom_var: 'custom_value', another: '42' }),
});
provider = new AnsibleProvider(params);
mockRun.mockResolvedValueOnce('ok');
await provider.runTaskInWorkflow('guid-extra', 'img', 'cmd', '/m', '/w', [], []);
const command = mockRun.mock.calls[0][0];
expect(command).toContain('custom_var');
expect(command).toContain('custom_value');
expect(command).toContain('another');
});
it('logs warning when extra vars JSON is invalid but continues', async () => {
const params = createBuildParameters({ ansibleExtraVars: 'not-valid-json{{{' });
provider = new AnsibleProvider(params);
mockRun.mockResolvedValueOnce('ok');
await provider.runTaskInWorkflow('guid-badjson', 'img', 'cmd', '/m', '/w', [], []);
expect(mockLogWarning).toHaveBeenCalledWith(expect.stringContaining('Failed to parse ansibleExtraVars'));
});
it('includes vault password file flag when configured', async () => {
const params = createBuildParameters({ ansibleVaultPassword: '/secure/vault-pass.txt' });
provider = new AnsibleProvider(params);
mockRun.mockResolvedValueOnce('ok');
await provider.runTaskInWorkflow('guid-vault', 'img', 'cmd', '/m', '/w', [], []);
const command = mockRun.mock.calls[0][0];
expect(command).toContain('--vault-password-file "/secure/vault-pass.txt"');
});
it('does not include vault password flag when not configured', async () => {
mockRun.mockResolvedValueOnce('ok');
await provider.runTaskInWorkflow('guid-novault', 'img', 'cmd', '/m', '/w', [], []);
const command = mockRun.mock.calls[0][0];
expect(command).not.toContain('--vault-password-file');
});
it('prefixes secrets as environment variables in the command', async () => {
mockRun.mockResolvedValueOnce('ok');
const secrets = [
{ ParameterKey: 'key1', EnvironmentVariable: 'SECRET_TOKEN', ParameterValue: 'tok-abc' },
{ ParameterKey: 'key2', EnvironmentVariable: 'DEPLOY_KEY', ParameterValue: 'dk-xyz' },
];
await provider.runTaskInWorkflow('guid-secrets', 'img', 'cmd', '/m', '/w', [], secrets as any);
const command = mockRun.mock.calls[0][0];
expect(command).toMatch(/^SECRET_TOKEN='tok-abc'/);
expect(command).toContain("DEPLOY_KEY='dk-xyz'");
expect(command).toContain('ansible-playbook');
});
it('throws and logs warning when playbook execution fails', async () => {
const execError = new Error('UNREACHABLE! Host unreachable');
mockRun.mockRejectedValueOnce(execError);
await expect(provider.runTaskInWorkflow('guid-hostfail', 'img', 'cmd', '/m', '/w', [], [])).rejects.toThrow(
'UNREACHABLE',
);
expect(mockLogWarning).toHaveBeenCalledWith(expect.stringContaining('Playbook failed'));
});
});
describe('cleanupWorkflow', () => {
it('completes without error and logs cleanup message', async () => {
await provider.cleanupWorkflow(createBuildParameters(), 'main', []);
expect(mockLog).toHaveBeenCalledWith(expect.stringContaining('Cleanup complete'));
});
});
describe('garbageCollect', () => {
it('returns empty string (no-op)', async () => {
const result = await provider.garbageCollect('', false, 0, false, false);
expect(result).toBe('');
});
});
describe('listResources', () => {
it('returns inventory path as a resource when configured', async () => {
const resources = await provider.listResources();
expect(resources).toHaveLength(1);
expect(resources[0].Name).toBe('/etc/ansible/hosts');
});
it('returns empty array when inventory is not configured', async () => {
const params = createBuildParameters({ ansibleInventory: '' });
provider = new AnsibleProvider(params);
const resources = await provider.listResources();
expect(resources).toEqual([]);
});
});
describe('listWorkflow', () => {
it('returns empty array (not implemented)', async () => {
const workflows = await provider.listWorkflow();
expect(workflows).toEqual([]);
});
});
describe('watchWorkflow', () => {
it('returns empty string (not implemented)', async () => {
const result = await provider.watchWorkflow();
expect(result).toBe('');
});
});
});

View File

@@ -0,0 +1,197 @@
import * as core from '@actions/core';
import BuildParameters from '../../../build-parameters';
import { OrchestratorSystem } from '../../services/core/orchestrator-system';
import OrchestratorEnvironmentVariable from '../../options/orchestrator-environment-variable';
import OrchestratorLogger from '../../services/core/orchestrator-logger';
import { ProviderInterface } from '../provider-interface';
import OrchestratorSecret from '../../options/orchestrator-secret';
import { ProviderResource } from '../provider-resource';
import { ProviderWorkflow } from '../provider-workflow';
/**
* Ansible provider — executes Unity builds via Ansible playbooks
* against managed inventory.
*
* Use case: Teams with existing Ansible infrastructure for server
* management who want to leverage their inventory for build distribution.
*/
class AnsibleProvider implements ProviderInterface {
private buildParameters: BuildParameters;
private inventory: string;
private playbook: string;
private extraVariables: string;
private vaultPassword: string;
constructor(buildParameters: BuildParameters) {
this.buildParameters = buildParameters;
this.inventory = buildParameters.ansibleInventory || '';
this.playbook = buildParameters.ansiblePlaybook || '';
this.extraVariables = buildParameters.ansibleExtraVars || '';
this.vaultPassword = buildParameters.ansibleVaultPassword || '';
}
async setupWorkflow(
// eslint-disable-next-line no-unused-vars
buildGuid: string,
// eslint-disable-next-line no-unused-vars
buildParameters: BuildParameters,
// eslint-disable-next-line no-unused-vars
branchName: string,
// eslint-disable-next-line no-unused-vars
defaultSecretsArray: { ParameterKey: string; EnvironmentVariable: string; ParameterValue: string }[],
): Promise<void> {
OrchestratorLogger.log(`[Ansible] Setting up playbook execution`);
if (!this.inventory) {
throw new Error('ansibleInventory is required for the ansible provider');
}
// Verify ansible is available
try {
const version = await OrchestratorSystem.Run('ansible --version | head -1');
OrchestratorLogger.log(`[Ansible] ${version.trim()}`);
} catch (error: any) {
throw new Error(`Ansible not found on PATH: ${error.message || error}`);
}
// Verify ansible-playbook binary exists (may be separate from ansible)
try {
await OrchestratorSystem.Run('command -v ansible-playbook || which ansible-playbook || where ansible-playbook');
OrchestratorLogger.log(`[Ansible] ansible-playbook binary verified`);
} catch (error: any) {
core.error('ansible-playbook not found. Install Ansible or ensure it is in PATH.');
throw new Error(`ansible-playbook not found on PATH: ${error.message || error}`);
}
// Verify inventory exists
try {
await OrchestratorSystem.Run(`test -e "${this.inventory}"`);
} catch {
throw new Error(`Inventory not found: ${this.inventory}`);
}
}
async runTaskInWorkflow(
buildGuid: string,
image: string,
commands: string,
mountdir: string,
workingdir: string,
environment: OrchestratorEnvironmentVariable[],
secrets: OrchestratorSecret[],
): Promise<string> {
OrchestratorLogger.log(`[Ansible] Running playbook against inventory ${this.inventory}`);
if (!this.playbook) {
throw new Error(
'ansiblePlaybook is required — no default playbook is provided yet. ' +
'Provide a playbook that accepts build_guid, build_image, build_commands, mount_dir, and working_dir variables.',
);
}
// Build extra-vars JSON
// These use snake_case because they are Ansible variable names passed to playbooks
const playbookVariables: Record<string, string> = {
// eslint-disable-next-line camelcase
build_guid: buildGuid,
// eslint-disable-next-line camelcase
build_image: image,
// eslint-disable-next-line camelcase
build_commands: commands,
// eslint-disable-next-line camelcase
mount_dir: mountdir,
// eslint-disable-next-line camelcase
working_dir: workingdir,
};
for (const element of environment) {
playbookVariables[element.name.toLowerCase()] = element.value;
}
// Merge user-provided extra vars
if (this.extraVariables) {
try {
const userVariables = JSON.parse(this.extraVariables);
Object.assign(playbookVariables, userVariables);
} catch {
OrchestratorLogger.logWarning(`[Ansible] Failed to parse ansibleExtraVars as JSON, using as-is`);
}
}
const extraVariablesJson = JSON.stringify(playbookVariables).replace(/'/g, "'\\''");
// Build ansible-playbook command
const commandParts = [
'ansible-playbook',
`-i "${this.inventory}"`,
`"${this.playbook}"`,
`-e '${extraVariablesJson}'`,
'--no-color',
];
if (this.vaultPassword) {
commandParts.push(`--vault-password-file "${this.vaultPassword}"`);
}
// Add secret variables as extra environment
const environmentPrefix = secrets
.map((secret) => `${secret.EnvironmentVariable}='${secret.ParameterValue}'`)
.join(' ');
const fullCommand = environmentPrefix ? `${environmentPrefix} ${commandParts.join(' ')}` : commandParts.join(' ');
try {
const output = await OrchestratorSystem.Run(fullCommand);
OrchestratorLogger.log(`[Ansible] Playbook completed successfully`);
return output;
} catch (error: any) {
OrchestratorLogger.logWarning(`[Ansible] Playbook failed: ${error.message || error}`);
throw error;
}
}
async cleanupWorkflow(
// eslint-disable-next-line no-unused-vars
buildParameters: BuildParameters,
// eslint-disable-next-line no-unused-vars
branchName: string,
// eslint-disable-next-line no-unused-vars
defaultSecretsArray: { ParameterKey: string; EnvironmentVariable: string; ParameterValue: string }[],
): Promise<void> {
OrchestratorLogger.log(`[Ansible] Cleanup complete`);
}
async garbageCollect(
// eslint-disable-next-line no-unused-vars
filter: string,
// eslint-disable-next-line no-unused-vars
previewOnly: boolean,
// eslint-disable-next-line no-unused-vars
olderThan: Number,
// eslint-disable-next-line no-unused-vars
fullCache: boolean,
// eslint-disable-next-line no-unused-vars
baseDependencies: boolean,
): Promise<string> {
return '';
}
async listResources(): Promise<ProviderResource[]> {
if (!this.inventory) return [];
const resource = new ProviderResource();
resource.Name = this.inventory;
return [resource];
}
async listWorkflow(): Promise<ProviderWorkflow[]> {
return [];
}
async watchWorkflow(): Promise<string> {
return '';
}
}
export default AnsibleProvider;

View File

@@ -1,14 +1,18 @@
import CloudRunnerLogger from '../../services/core/cloud-runner-logger';
import OrchestratorLogger from '../../services/core/orchestrator-logger';
import * as core from '@actions/core';
import {
CloudFormation,
CreateStackCommand,
// eslint-disable-next-line import/named
CreateStackCommandInput,
DescribeStacksCommand,
// eslint-disable-next-line import/named
DescribeStacksCommandInput,
ListStacksCommand,
// eslint-disable-next-line import/named
Parameter,
UpdateStackCommand,
// eslint-disable-next-line import/named
UpdateStackCommandInput,
waitUntilStackCreateComplete,
waitUntilStackUpdateComplete,
@@ -16,6 +20,17 @@ import {
import { BaseStackFormation } from './cloud-formations/base-stack-formation';
import crypto from 'node:crypto';
const DEFAULT_STACK_WAIT_TIME_SECONDS = 600;
function getStackWaitTime(): number {
const overrideValue = Number(process.env.ORCHESTRATOR_AWS_STACK_WAIT_TIME ?? '');
if (!Number.isNaN(overrideValue) && overrideValue > 0) {
return overrideValue;
}
return DEFAULT_STACK_WAIT_TIME_SECONDS;
}
export class AWSBaseStack {
constructor(baseStackName: string) {
this.baseStackName = baseStackName;
@@ -24,6 +39,7 @@ export class AWSBaseStack {
async setupBaseStack(CF: CloudFormation) {
const baseStackName = this.baseStackName;
const stackWaitTimeSeconds = getStackWaitTime();
const baseStack = BaseStackFormation.formation;
@@ -54,18 +70,39 @@ export class AWSBaseStack {
};
const stacks = await CF.send(
new ListStacksCommand({ StackStatusFilter: ['UPDATE_COMPLETE', 'CREATE_COMPLETE', 'ROLLBACK_COMPLETE'] }),
new ListStacksCommand({
StackStatusFilter: [
'CREATE_IN_PROGRESS',
'UPDATE_IN_PROGRESS',
'UPDATE_COMPLETE',
'CREATE_COMPLETE',
'ROLLBACK_COMPLETE',
],
}),
);
const stackNames = stacks.StackSummaries?.map((x) => x.StackName) || [];
const stackExists: Boolean = stackNames.includes(baseStackName) || false;
const stackExists: boolean = stackNames.includes(baseStackName);
const describeStack = async () => {
return await CF.send(new DescribeStacksCommand(describeStackInput));
};
try {
if (!stackExists) {
CloudRunnerLogger.log(`${baseStackName} stack does not exist (${JSON.stringify(stackNames)})`);
await CF.send(new CreateStackCommand(createStackInput));
CloudRunnerLogger.log(`created stack (version: ${parametersHash})`);
OrchestratorLogger.log(`${baseStackName} stack does not exist (${JSON.stringify(stackNames)})`);
let created = false;
try {
await CF.send(new CreateStackCommand(createStackInput));
created = true;
} catch (error: any) {
const message = `${error?.name ?? ''} ${error?.message ?? ''}`;
if (message.includes('AlreadyExistsException')) {
OrchestratorLogger.log(`Base stack already exists, continuing with describe`);
} else {
throw error;
}
}
if (created) {
OrchestratorLogger.log(`created stack (version: ${parametersHash})`);
}
}
const CFState = await describeStack();
let stack = CFState.Stacks?.[0];
@@ -75,32 +112,35 @@ export class AWSBaseStack {
const stackVersion = stack.Parameters?.find((x) => x.ParameterKey === 'Version')?.ParameterValue;
if (stack.StackStatus === 'CREATE_IN_PROGRESS') {
OrchestratorLogger.log(
`Waiting up to ${stackWaitTimeSeconds}s for '${baseStackName}' CloudFormation creation to finish`,
);
await waitUntilStackCreateComplete(
{
client: CF,
maxWaitTime: 200,
maxWaitTime: stackWaitTimeSeconds,
},
describeStackInput,
);
}
if (stackExists) {
CloudRunnerLogger.log(`Base stack exists (version: ${stackVersion}, local version: ${parametersHash})`);
OrchestratorLogger.log(`Base stack exists (version: ${stackVersion}, local version: ${parametersHash})`);
if (parametersHash !== stackVersion) {
CloudRunnerLogger.log(`Attempting update of base stack`);
OrchestratorLogger.log(`Attempting update of base stack`);
try {
await CF.send(new UpdateStackCommand(updateInput));
} catch (error: any) {
if (error['message'].includes('No updates are to be performed')) {
CloudRunnerLogger.log(`No updates are to be performed`);
OrchestratorLogger.log(`No updates are to be performed`);
} else {
CloudRunnerLogger.log(`Update Failed (Stack name: ${baseStackName})`);
CloudRunnerLogger.log(error['message']);
OrchestratorLogger.log(`Update Failed (Stack name: ${baseStackName})`);
OrchestratorLogger.log(error['message']);
}
CloudRunnerLogger.log(`Continuing...`);
OrchestratorLogger.log(`Continuing...`);
}
} else {
CloudRunnerLogger.log(`No update required`);
OrchestratorLogger.log(`No update required`);
}
stack = (await describeStack()).Stacks?.[0];
if (!stack) {
@@ -109,16 +149,19 @@ export class AWSBaseStack {
);
}
if (stack.StackStatus === 'UPDATE_IN_PROGRESS') {
OrchestratorLogger.log(
`Waiting up to ${stackWaitTimeSeconds}s for '${baseStackName}' CloudFormation update to finish`,
);
await waitUntilStackUpdateComplete(
{
client: CF,
maxWaitTime: 200,
maxWaitTime: stackWaitTimeSeconds,
},
describeStackInput,
);
}
}
CloudRunnerLogger.log('base stack is now ready');
OrchestratorLogger.log('base stack is now ready');
} catch (error) {
core.error(JSON.stringify(await describeStack(), undefined, 4));
throw error;

View File

@@ -0,0 +1,93 @@
import { CloudFormation } from '@aws-sdk/client-cloudformation';
import { ECS } from '@aws-sdk/client-ecs';
import { Kinesis } from '@aws-sdk/client-kinesis';
import { CloudWatchLogs } from '@aws-sdk/client-cloudwatch-logs';
import { S3 } from '@aws-sdk/client-s3';
import { Input } from '../../..';
import OrchestratorOptions from '../../options/orchestrator-options';
export class AwsClientFactory {
private static cloudFormation: CloudFormation;
private static ecs: ECS;
private static kinesis: Kinesis;
private static cloudWatchLogs: CloudWatchLogs;
private static s3: S3;
private static getCredentials() {
// Explicitly provide credentials from environment variables for LocalStack compatibility
// LocalStack accepts any credentials, but the AWS SDK needs them to be explicitly set
const accessKeyId = process.env.AWS_ACCESS_KEY_ID;
const secretAccessKey = process.env.AWS_SECRET_ACCESS_KEY;
if (accessKeyId && secretAccessKey) {
return {
accessKeyId,
secretAccessKey,
};
}
// Return undefined to let AWS SDK use default credential chain
return;
}
static getCloudFormation(): CloudFormation {
if (!this.cloudFormation) {
this.cloudFormation = new CloudFormation({
region: Input.region,
endpoint: OrchestratorOptions.awsCloudFormationEndpoint,
credentials: AwsClientFactory.getCredentials(),
});
}
return this.cloudFormation;
}
static getECS(): ECS {
if (!this.ecs) {
this.ecs = new ECS({
region: Input.region,
endpoint: OrchestratorOptions.awsEcsEndpoint,
credentials: AwsClientFactory.getCredentials(),
});
}
return this.ecs;
}
static getKinesis(): Kinesis {
if (!this.kinesis) {
this.kinesis = new Kinesis({
region: Input.region,
endpoint: OrchestratorOptions.awsKinesisEndpoint,
credentials: AwsClientFactory.getCredentials(),
});
}
return this.kinesis;
}
static getCloudWatchLogs(): CloudWatchLogs {
if (!this.cloudWatchLogs) {
this.cloudWatchLogs = new CloudWatchLogs({
region: Input.region,
endpoint: OrchestratorOptions.awsCloudWatchLogsEndpoint,
credentials: AwsClientFactory.getCredentials(),
});
}
return this.cloudWatchLogs;
}
static getS3(): S3 {
if (!this.s3) {
this.s3 = new S3({
region: Input.region,
endpoint: OrchestratorOptions.awsS3Endpoint,
forcePathStyle: true,
credentials: AwsClientFactory.getCredentials(),
});
}
return this.s3;
}
}

View File

@@ -21,6 +21,7 @@ export class AWSCloudFormationTemplates {
public static getSecretDefinitionTemplate(p1: string, p2: string) {
return `
Secrets:
- Name: '${p1}'
ValueFrom: !Ref ${p2}Secret
`;

View File

@@ -1,16 +1,16 @@
import CloudRunnerLogger from '../../services/core/cloud-runner-logger';
import OrchestratorLogger from '../../services/core/orchestrator-logger';
import { CloudFormation, DescribeStackEventsCommand } from '@aws-sdk/client-cloudformation';
import * as core from '@actions/core';
import CloudRunner from '../../cloud-runner';
import Orchestrator from '../../orchestrator';
export class AWSError {
static async handleStackCreationFailure(error: any, CF: CloudFormation, taskDefStackName: string) {
CloudRunnerLogger.log('aws error: ');
OrchestratorLogger.log('aws error: ');
core.error(JSON.stringify(error, undefined, 4));
if (CloudRunner.buildParameters.cloudRunnerDebug) {
CloudRunnerLogger.log('Getting events and resources for task stack');
if (Orchestrator.buildParameters.orchestratorDebug) {
OrchestratorLogger.log('Getting events and resources for task stack');
const events = (await CF.send(new DescribeStackEventsCommand({ StackName: taskDefStackName }))).StackEvents;
CloudRunnerLogger.log(JSON.stringify(events, undefined, 4));
OrchestratorLogger.log(JSON.stringify(events, undefined, 4));
}
}
}

View File

@@ -1,22 +1,34 @@
import {
CloudFormation,
CreateStackCommand,
// eslint-disable-next-line import/named
CreateStackCommandInput,
DescribeStackResourcesCommand,
DescribeStacksCommand,
ListStacksCommand,
waitUntilStackCreateComplete,
} from '@aws-sdk/client-cloudformation';
import CloudRunnerAWSTaskDef from './cloud-runner-aws-task-def';
import CloudRunnerSecret from '../../options/cloud-runner-secret';
import OrchestratorAWSTaskDef from './orchestrator-aws-task-def';
import OrchestratorSecret from '../../options/orchestrator-secret';
import { AWSCloudFormationTemplates } from './aws-cloud-formation-templates';
import CloudRunnerLogger from '../../services/core/cloud-runner-logger';
import OrchestratorLogger from '../../services/core/orchestrator-logger';
import { AWSError } from './aws-error';
import CloudRunner from '../../cloud-runner';
import Orchestrator from '../../orchestrator';
import { CleanupCronFormation } from './cloud-formations/cleanup-cron-formation';
import CloudRunnerOptions from '../../options/cloud-runner-options';
import OrchestratorOptions from '../../options/orchestrator-options';
import { TaskDefinitionFormation } from './cloud-formations/task-definition-formation';
const DEFAULT_STACK_WAIT_TIME_SECONDS = 600;
function getStackWaitTime(): number {
const overrideValue = Number(process.env.ORCHESTRATOR_AWS_STACK_WAIT_TIME ?? '');
if (!Number.isNaN(overrideValue) && overrideValue > 0) {
return overrideValue;
}
return DEFAULT_STACK_WAIT_TIME_SECONDS;
}
export class AWSJobStack {
private baseStackName: string;
constructor(baseStackName: string) {
@@ -31,23 +43,23 @@ export class AWSJobStack {
commands: string,
mountdir: string,
workingdir: string,
secrets: CloudRunnerSecret[],
): Promise<CloudRunnerAWSTaskDef> {
secrets: OrchestratorSecret[],
): Promise<OrchestratorAWSTaskDef> {
const taskDefStackName = `${this.baseStackName}-${buildGuid}`;
let taskDefCloudFormation = AWSCloudFormationTemplates.readTaskCloudFormationTemplate();
taskDefCloudFormation = taskDefCloudFormation.replace(
`ContainerCpu:
Default: 1024`,
`ContainerCpu:
Default: ${Number.parseInt(CloudRunner.buildParameters.containerCpu)}`,
Default: ${Number.parseInt(Orchestrator.buildParameters.containerCpu)}`,
);
taskDefCloudFormation = taskDefCloudFormation.replace(
`ContainerMemory:
Default: 2048`,
`ContainerMemory:
Default: ${Number.parseInt(CloudRunner.buildParameters.containerMemory)}`,
Default: ${Number.parseInt(Orchestrator.buildParameters.containerMemory)}`,
);
if (!CloudRunnerOptions.asyncCloudRunner) {
if (!OrchestratorOptions.asyncOrchestrator) {
taskDefCloudFormation = AWSCloudFormationTemplates.insertAtTemplate(
taskDefCloudFormation,
'# template resources logstream',
@@ -121,8 +133,8 @@ export class AWSJobStack {
},
...secretsMappedToCloudFormationParameters,
];
CloudRunnerLogger.log(
`Starting AWS job with memory: ${CloudRunner.buildParameters.containerMemory} cpu: ${CloudRunner.buildParameters.containerCpu}`,
OrchestratorLogger.log(
`Starting AWS job with memory: ${Orchestrator.buildParameters.containerMemory} cpu: ${Orchestrator.buildParameters.containerCpu}`,
);
let previousStackExists = true;
while (previousStackExists) {
@@ -135,7 +147,7 @@ export class AWSJobStack {
const element = stacks.StackSummaries[index];
if (element.StackName === taskDefStackName && element.StackStatus !== 'DELETE_COMPLETE') {
previousStackExists = true;
CloudRunnerLogger.log(`Previous stack still exists: ${JSON.stringify(element)}`);
OrchestratorLogger.log(`Previous stack still exists: ${JSON.stringify(element)}`);
await new Promise((promise) => setTimeout(promise, 5000));
}
}
@@ -147,12 +159,15 @@ export class AWSJobStack {
Parameters: parameters,
};
try {
CloudRunnerLogger.log(`Creating job aws formation ${taskDefStackName}`);
const stackWaitTimeSeconds = getStackWaitTime();
OrchestratorLogger.log(
`Creating job aws formation ${taskDefStackName} (waiting up to ${stackWaitTimeSeconds}s for completion)`,
);
await CF.send(new CreateStackCommand(createStackInput));
await waitUntilStackCreateComplete(
{
client: CF,
maxWaitTime: 200,
maxWaitTime: stackWaitTimeSeconds,
},
{ StackName: taskDefStackName },
);
@@ -186,7 +201,7 @@ export class AWSJobStack {
},
{
ParameterKey: 'BUILDGUID',
ParameterValue: CloudRunner.buildParameters.buildGuid,
ParameterValue: Orchestrator.buildParameters.buildGuid,
},
{
ParameterKey: 'EnvironmentName',
@@ -194,9 +209,9 @@ export class AWSJobStack {
},
],
};
if (CloudRunnerOptions.useCleanupCron) {
if (OrchestratorOptions.useCleanupCron) {
try {
CloudRunnerLogger.log(`Creating job cleanup formation`);
OrchestratorLogger.log(`Creating job cleanup formation`);
await CF.send(new CreateStackCommand(createCleanupStackInput));
// await CF.waitFor('stackCreateComplete', { StackName: createCleanupStackInput.StackName }).promise();

Some files were not shown because too many files have changed in this diff Show More