Compare commits

..

17 Commits

Author SHA1 Message Date
frostebite
cf00694232 revert: restore build-tests-mac.yml to match main
Stop modifying the macOS build workflow — leave it identical to main.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 02:54:08 +00:00
frostebite
b7f6ea9194 ci: use step-level continue-on-error for macOS builds
GitHub deprecated external check run conclusion updates (Feb 2025),
so the Checks API approach to mark builds as neutral no longer works.

Instead, move continue-on-error from the job level to the build step.
The job always succeeds (green check), failed builds emit a warning
annotation, and upload is skipped on failure.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 02:52:49 +00:00
frostebite
8ccbfad587 ci: mark failed macOS builds as neutral instead of failure
Use the Checks API to flip failed macOS build conclusions to neutral
(gray dash) so unstable builds don't show red X marks on PRs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 02:17:15 +00:00
frostebite
483bab04c7 feat(orchestrator): middleware pipeline — composable trigger-aware hooks
Add a middleware system that wraps around build pipeline phases with
before/after semantics, built on the existing command hook and container
hook fundamentals. Middleware supports rich trigger conditions (phase,
provider, platform, environment expressions), priority-based ordering
with wrapping semantics, and both inline YAML and file-based definitions
from game-ci/middleware/.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 22:57:20 +00:00
frostebite
488c259481 test: add integration wiring and input parsing tests for enterprise features
Add three test files covering the two highest-priority gaps in PR #777:

1. src/index-enterprise-features.test.ts (21 tests) - Integration wiring
   tests for index.ts that verify conditional gating of all enterprise
   services (GitHooks, LocalCache, ChildWorkspace, SubmoduleProfile,
   LfsAgent). Tests that disabled features (default) are never invoked,
   enabled features call the correct service methods, and the order of
   operations is correct (restore before build, save after build).
   Also tests non-local provider strategy skips all enterprise features.

2. src/model/enterprise-inputs.test.ts (103 tests) - Input/BuildParameters
   wiring tests for all 20 new enterprise properties. Covers defaults,
   explicit values, and boolean string parsing edge cases (the #1 source
   of bugs: 'false' as truthy, 'TRUE' case sensitivity, '1', 'yes').
   Verifies BuildParameters.create() correctly maps all Input getters.

3. src/model/orchestrator/services/submodule/submodule-profile-service.test.ts
   (5 new tests) - Command construction safety tests for execute(),
   documenting how paths, branches, and tokens are passed into git
   commands and verifying the expected command strings.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 09:29:10 +00:00
frostebite
7b8f109334 fix: make git hooks opt-in only — do not modify hooks when disabled
Remove the else branch that actively called GitHooksService.disableHooks()
for every user where gitHooksEnabled was false (the default). This was a
breaking change that silently modified core.hooksPath to point at an empty
directory, disabling any existing git hooks (husky, lefthook, pre-commit, etc.).

When gitHooksEnabled is false (default), the action now does nothing
regarding hooks — exactly matching the behavior on main before the hooks
feature was added. The hooks feature only activates when users explicitly
set gitHooksEnabled: true.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 08:53:15 +00:00
frostebite
f2e7568d47 ci: set macOS builds to continue-on-error
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 23:34:12 +00:00
frostebite
b4ffa3e070 ci: split orchestrator integrity into 4 parallel jobs to fix timeout
The monolithic orchestrator-integrity workflow runs 25+ tests sequentially
in a single job, consistently hitting the 60-minute timeout on PR runs.
Split into 4 parallel jobs (k8s, aws-provider, local-docker, rclone) each
on its own runner, cutting wall-clock time from 3+ hours to ~1 hour and
eliminating disk space exhaustion from shared runner contention.

Adopts the parallel architecture from PR #809.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 20:43:59 +00:00
frostebite
3976b7cedd style: fix prettier formatting
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 14:08:56 +00:00
frostebite
f06f99b3e5 chore: rebuild dist for cli-provider timeout changes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 12:55:23 +00:00
frostebite
fe63d7b32d fix(cli-provider): add timeout protection for external CLI processes
Prevent builds from hanging indefinitely when CLI provider subprocess
is unresponsive. Default 2h for runTaskInWorkflow, 1h for watchWorkflow.
Graceful SIGTERM with 10s grace before SIGKILL.

- Added RUN_TASK_TIMEOUT_MS (2 hours) and WATCH_WORKFLOW_TIMEOUT_MS (1 hour)
- Added gracefulKill helper: SIGTERM first, SIGKILL after 10s grace period
- runTaskInWorkflow and watchWorkflow now have timeout protection
- Existing execute() method upgraded to use gracefulKill
- core.error() called with clear human-readable timeout message
- Added comprehensive tests: timeout triggers, SIGKILL escalation,
  grace period cancellation on voluntary exit, normal completion

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 12:55:14 +00:00
frostebite
007852a800 feat(cache): add child workspace isolation for multi-product CI builds (#777)
Implement two-level workspace isolation pattern for enterprise-scale CI:
- Atomic O(1) workspace restore via filesystem move (no tar/download/extract)
- Separate Library caching for independent restore
- .git preservation for delta operations
- Stale workspace cleanup with configurable retention policies
- 5 new action inputs: childWorkspacesEnabled, childWorkspaceName,
  childWorkspaceCacheRoot, childWorkspacePreserveGit,
  childWorkspaceSeparateLibrary
- 28 unit tests covering all service methods

This enables enterprise CI where workspaces are 50GB+ and traditional
caching via actions/cache is impractical. On NTFS, workspace restore
is O(1) via atomic rename when source and destination are on the same volume.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 12:02:03 +00:00
frostebite
26903e96dd feat(hooks): add Unity Git Hooks integration and runHookGroups
Built-in support for Unity Git Hooks (com.frostebite.unitygithooks):
- Auto-detect UPM package in Packages/manifest.json
- Run init-unity-lefthook.js before hook installation
- Set CI-friendly env vars (disable background project mode)

New gitHooksRunBeforeBuild input runs specific lefthook groups before
the Unity build, allowing CI to trigger pre-commit or pre-push checks
that normally only fire on git events.

35 unit tests covering detection, init, CI env, group execution, and
failure handling.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 08:54:07 +00:00
frostebite
cf3478c8ec feat(lfs): add built-in elastic-git-storage support with auto-install
First-class support for elastic-git-storage as a custom LFS transfer
agent. When lfsTransferAgent is set to "elastic-git-storage" (or
"elastic-git-storage@v1.0.0" for a specific version), the service
automatically finds or installs the agent from GitHub releases, then
configures it via git config.

Supports version pinning via @version suffix in the agent value,
eliminating the need for a separate version parameter. Platform and
architecture detection handles linux/darwin/windows on amd64/arm64.

37 unit tests covering detection, PATH lookup, installation, version
parsing, and configuration delegation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 08:46:22 +00:00
frostebite
a0c79bd657 test(orchestrator): expand unit tests for enterprise services
Add comprehensive tests for CLI provider (cleanupWorkflow, garbageCollect,
listWorkflow, watchWorkflow, stderr forwarding, timeout handling), local
cache service (saveLfsCache full path and error handling), git hooks service
(husky install, failure logging, edge cases), and LFS agent service (empty
storagePaths, validate logging). 73 tests across 4 test files.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 08:20:31 +00:00
frostebite
cfac5f138d test(orchestrator): expand local cache service test coverage
Adds tests for cache hit restore (picks latest tar), LFS cache
restore/save, garbage collection age filtering, and edge cases
like permission errors and empty directories.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 07:58:09 +00:00
frostebite
5268630ef0 feat(orchestrator): enterprise feature support — CLI provider, submodule profiles, caching, LFS, hooks
Add generic enterprise-grade features to the orchestrator, enabling Unity projects with
complex CI/CD pipelines to adopt game-ci/unity-builder with built-in support for:

- CLI provider protocol: JSON-over-stdin/stdout bridge enabling providers in any language
  (Go, Python, Rust, shell) via the `providerExecutable` input
- Submodule profiles: YAML-based selective submodule initialization with glob patterns
  and variant overlays (`submoduleProfilePath`, `submoduleVariantPath`)
- Local build caching: Filesystem-based Library and LFS caching for local builds without
  external cache actions (`localCacheEnabled`, `localCacheRoot`)
- Custom LFS transfer agents: Register external transfer agents like elastic-git-storage
  (`lfsTransferAgent`, `lfsTransferAgentArgs`, `lfsStoragePaths`)
- Git hooks support: Detect and install lefthook/husky with configurable skip lists
  (`gitHooksEnabled`, `gitHooksSkipList`)

Also removes all `orchestrator-develop` branch references, replacing with `main`.

13 new action inputs, 13 new files, 14 new CLI provider tests, 17 submodule tests,
plus cache/LFS/hooks unit tests. All 452 tests pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 06:54:46 +00:00
36 changed files with 9646 additions and 672 deletions

File diff suppressed because it is too large Load Diff

View File

@@ -1,80 +0,0 @@
# Unity-Builder
GitHub Action and CLI that builds Unity projects for multiple platforms. Part of the [GameCI](https://game.ci) project.
## Quick Reference
```bash
yarn # install dependencies
yarn build # full build: tsc → ncc bundle (src/ → lib/ → dist/index.js)
yarn test # run all tests (jest)
yarn test:ci # run tests in CI mode (single-threaded, 2min timeout)
yarn lint # prettier + eslint check
yarn format # auto-format with prettier
```
## Architecture
**Entry point:** `src/index.ts` → decides between CLI mode and GitHub Action mode.
**Two execution paths:**
1. **Local builds** — Docker container or native macOS (`src/model/docker.ts`, `src/model/mac-builder.ts`)
2. **Orchestrator builds** — Remote execution on AWS ECS, Kubernetes, or other providers (`src/model/orchestrator/`)
**Key modules:**
| Path | Purpose |
|---|---|
| `src/model/build-parameters.ts` | Central config object — all build settings flow through here |
| `src/model/input.ts` | Input resolution with priority: Action inputs → CLI flags → env override → env vars |
| `src/model/orchestrator/orchestrator.ts` | Remote build orchestration — provider selection, workflow execution |
| `src/model/orchestrator/providers/` | Provider plugin system (AWS, K8s, Docker, Local, Test) |
| `src/model/orchestrator/remote-client/` | Code that runs inside remote containers (caching, hooks, artifacts) |
| `src/model/orchestrator/workflows/` | Build workflow types (standard, custom, async) |
| `src/model/orchestrator/services/` | Logging, locking, resource tracking |
| `src/model/cli/` | CLI mode using commander — dispatches to `@CliFunction`-decorated methods |
| `action.yml` | GitHub Action manifest — all inputs/outputs defined here |
| `dist/index.js` | Bundled output (committed to repo, used by action.yml at runtime) |
**Provider interface:** All providers implement `ProviderInterface` (`providers/provider-interface.ts`) with methods: `setupWorkflow`, `runTaskInWorkflow`, `cleanupWorkflow`, `garbageCollect`, `listResources`, `listWorkflow`, `watchWorkflow`.
**Provider loading:** Providers can be built-in, loaded from npm, cloned from GitHub repos, or loaded from local paths (`provider-loader.ts`).
## Build System
The build pipeline is: `yarn``tsc` (src/ → lib/) → `ncc build lib` (lib/ → dist/index.js).
- **dist/ is committed** — GitHub Actions loads `dist/index.js` directly, no install step on runners
- **Pre-commit hooks** (lefthook) auto-run formatting, linting, related tests, and `yarn build` to keep dist/ in sync
- Runtime: Node 20 (configured via Volta and action.yml `runs.using: node20`)
## Code Conventions
- **Files:** kebab-case (enforced by eslint `unicorn/filename-case`)
- **Code:** camelCase variables/functions, PascalCase classes/types
- **Formatting:** Prettier — 120 char width, single quotes, trailing commas, semicolons
- **Linting:** ESLint with unicorn, github, prettier, jest plugins
- **TypeScript:** strict mode, ES2020 target, CommonJS modules, experimental decorators enabled
- **Blank line before return statements** (enforced)
- **Blank line before block/line comments** (enforced)
- **No `for...in` loops** — use `for...of`
## Testing
- **Framework:** Jest 27 with ts-jest
- **Pattern:** `**/*.test.ts` files colocated with source
- **Orchestrator tests:** Concentrated in `src/model/orchestrator/tests/`
- **Run specific tests:** `yarn test -t "pattern"` or `yarn jest path/to/file.test.ts`
- **Orchestrator integration tests** require `orchestratorTests=true` env var: `cross-env orchestratorTests=true yarn test -i -t "orchestrator"`
## Security
- **Never log, output, or hardcode credentials** — cloud provider secrets (AWS, GCP, K8s), Unity serial keys, keystores, and private tokens must stay in secret inputs
- **Input validation matters** — user-supplied hook commands and custom parameters can be injection vectors; use `shell-quote` for shell escaping
- **Keystore/license data** is base64-encoded in inputs and written to temp files at build time
## CI Workflows
- `integrity-check.yml` — lint, test, build on every push/PR
- `build-tests-{ubuntu,windows,mac}.yml` — matrix builds across Unity versions and platforms
- `orchestrator-integrity.yml` / `orchestrator-async-checks.yml` — orchestrator-specific validation

View File

@@ -182,8 +182,8 @@ inputs:
required: false
default: ''
description:
'[Orchestrator] Run a custom job instead of the standard build automation for orchestrator (in yaml format with the
keys image, secrets (name, value object array), command line string)'
'[Orchestrator] Run a custom job instead of the standard build automation for orchestrator (in yaml format with
the keys image, secrets (name, value object array), command line string)'
awsStackName:
default: 'game-ci'
required: false
@@ -279,6 +279,129 @@ inputs:
description:
'[Orchestrator] Specifies the repo for the unity builder. Useful if you forked the repo for testing, features, or
fixes.'
submoduleProfilePath:
required: false
default: ''
description:
'Path to a YAML submodule profile file (relative to repo root). Defines which submodules to initialize (branch:
main) or skip (branch: empty). See docs for format.'
submoduleVariantPath:
required: false
default: ''
description:
'Path to a YAML variant overlay file that modifies the base submodule profile. Used for server or debug build
variants.'
submoduleToken:
required: false
default: ''
description:
'Git token for authenticating submodule clones. Falls back to gitPrivateToken or GITHUB_TOKEN if empty.'
localCacheEnabled:
required: false
default: 'false'
description:
'Enable filesystem-based caching for local builds. Caches the Unity Library folder and optionally LFS objects
between builds without requiring actions/cache.'
localCacheRoot:
required: false
default: ''
description:
'Root directory for local build cache. Defaults to $RUNNER_TEMP/game-ci-cache or .game-ci/cache if RUNNER_TEMP is
not set.'
localCacheLibrary:
required: false
default: 'true'
description: 'Cache the Unity Library folder for local builds. Only effective when localCacheEnabled is true.'
localCacheLfs:
required: false
default: 'false'
description: 'Cache Git LFS objects for local builds. Only effective when localCacheEnabled is true.'
childWorkspacesEnabled:
required: false
default: 'false'
description:
'Enable child workspace isolation for multi-product builds. Uses atomic filesystem moves for O(1) workspace
restore instead of tar/download/extract. Ideal for 50GB+ workspaces on self-hosted runners.'
childWorkspaceName:
required: false
default: ''
description:
'Name for this child workspace (e.g., product name like "TurnOfWar"). Used as the cache key for workspace
isolation. Required when childWorkspacesEnabled is true.'
childWorkspaceCacheRoot:
required: false
default: ''
description:
'Parent directory for cached child workspaces. Should be on the same NTFS volume as the build directory for O(1)
atomic restore via filesystem rename. Defaults to $RUNNER_TEMP/game-ci-workspaces.'
childWorkspacePreserveGit:
required: false
default: 'true'
description:
'Preserve .git directory in cached child workspace. Enables delta operations on restore but increases cache size.
Set to false to save disk space at the cost of full re-clone on restore.'
childWorkspaceSeparateLibrary:
required: false
default: 'true'
description:
'Cache Unity Library folder separately from the child workspace. Allows independent Library restore even when
workspace cache is invalidated. Recommended for large projects.'
lfsTransferAgent:
required: false
default: ''
description:
'Custom Git LFS transfer agent. Set to "elastic-git-storage" for built-in support (auto-installs from GitHub
releases). Append @version for a specific release (e.g. "elastic-git-storage@v1.0.0"). Or provide a path to any
custom transfer agent executable. When set, the agent is registered via git config before LFS operations.'
lfsTransferAgentArgs:
required: false
default: ''
description: 'Additional arguments to pass to the custom LFS transfer agent.'
lfsStoragePaths:
required: false
default: ''
description:
'Semicolon-separated list of storage paths for the custom LFS transfer agent. Interpretation depends on the agent
(e.g. local paths, WebDAV URLs, rclone remotes).'
gitHooksEnabled:
required: false
default: 'false'
description:
'Install and run git hooks (lefthook, husky, or native) during builds. When false (default), hooks are disabled
for build performance.'
gitHooksSkipList:
required: false
default: ''
description:
'Comma-separated list of hook names to skip even when gitHooksEnabled is true. Example: pre-push,post-merge'
gitHooksRunBeforeBuild:
required: false
default: ''
description:
'Comma-separated list of lefthook hook groups to run before the Unity build. Allows CI to trigger checks that
normally only run on git events. Example: pre-commit,pre-push. Requires lefthook. Works with Unity Git Hooks
(com.frostebite.unitygithooks) when installed as a UPM package — the init script runs automatically.'
providerExecutable:
required: false
default: ''
description:
'Path to an external CLI executable that implements the provider protocol. Enables providers written in any
language (Go, Python, Rust, shell). Uses JSON-over-stdin/stdout communication.'
middleware:
required: false
default: ''
description:
'[Orchestrator] Inline YAML middleware pipeline definitions. Middleware wraps around build phases with
before/after semantics, supports trigger conditions (phase, provider, platform, environment expressions), and
resolves to either command hooks (inline in build container) or container hooks (separate Docker containers).
Middleware executes in priority order — lower priority values run first for before phases and last for after
phases (wrapping pattern). See documentation for full YAML format.'
middlewareFiles:
required: false
default: ''
description:
'[Orchestrator] Comma-separated list of middleware file names (without extension) to load from the
game-ci/middleware/ directory in your repository. Each file should contain YAML middleware definitions.'
outputs:
volume:

2089
dist/index.js generated vendored

File diff suppressed because it is too large Load Diff

2
dist/index.js.map generated vendored

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,606 @@
/**
* Integration wiring tests for enterprise features in index.ts
*
* These tests verify the conditional gating logic in runMain():
* - Each enterprise feature is only invoked when its gate condition is met
* - Services are NOT called when their feature is disabled (the default)
* - The order of operations is correct (restore before build, save after build)
*/
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
import { BuildParameters } from './model';
// ---------------------------------------------------------------------------
// Service mocks — must be declared before importing index.ts (jest hoists them)
// ---------------------------------------------------------------------------
const mockChildWorkspaceService = {
buildConfig: jest.fn().mockReturnValue({ enabled: true, workspaceName: 'Test' }),
initializeWorkspace: jest.fn().mockReturnValue(false),
getWorkspaceSize: jest.fn().mockReturnValue('0 B'),
saveWorkspace: jest.fn(),
};
const mockSubmoduleProfileService = {
createInitPlan: jest.fn().mockResolvedValue([]),
execute: jest.fn().mockResolvedValue(''),
};
const mockLfsAgentService = {
configure: jest.fn().mockResolvedValue(''),
};
const mockLocalCacheService = {
resolveCacheRoot: jest.fn().mockReturnValue('/cache'),
generateCacheKey: jest.fn().mockReturnValue('key-1'),
restoreLfsCache: jest.fn().mockResolvedValue(true),
restoreLibraryCache: jest.fn().mockResolvedValue(true),
saveLibraryCache: jest.fn().mockResolvedValue(''),
saveLfsCache: jest.fn().mockResolvedValue(''),
};
const mockGitHooksService = {
installHooks: jest.fn().mockResolvedValue(''),
configureSkipList: jest.fn().mockReturnValue({ LEFTHOOK_EXCLUDE: 'pre-commit' }),
};
// Mock the dynamic import() targets — jest.mock with factory functions.
// The services are imported dynamically via `await import(...)` in index.ts,
// so we mock the module path and return the mock objects as named exports.
jest.mock('./model/orchestrator/services/cache/child-workspace-service', () => ({
ChildWorkspaceService: mockChildWorkspaceService,
}));
jest.mock('./model/orchestrator/services/submodule/submodule-profile-service', () => ({
SubmoduleProfileService: mockSubmoduleProfileService,
}));
jest.mock('./model/orchestrator/services/lfs/lfs-agent-service', () => ({
LfsAgentService: mockLfsAgentService,
}));
jest.mock('./model/orchestrator/services/cache/local-cache-service', () => ({
LocalCacheService: mockLocalCacheService,
}));
jest.mock('./model/orchestrator/services/hooks/git-hooks-service', () => ({
GitHooksService: mockGitHooksService,
}));
// Mock all non-enterprise dependencies to isolate the wiring logic
jest.mock('@actions/core');
jest.mock('./model', () => ({
Action: {
checkCompatibility: jest.fn(),
workspace: '/workspace',
actionFolder: '/action',
},
BuildParameters: {
create: jest.fn(),
},
Cache: {
verify: jest.fn(),
},
Orchestrator: {
run: jest.fn().mockResolvedValue(''),
},
Docker: {
run: jest.fn().mockResolvedValue(0),
},
ImageTag: jest.fn().mockImplementation(() => ({
toString: () => 'mock-image:latest',
})),
Output: {
setBuildVersion: jest.fn().mockResolvedValue(''),
setAndroidVersionCode: jest.fn().mockResolvedValue(''),
setEngineExitCode: jest.fn().mockResolvedValue(''),
},
}));
jest.mock('./model/cli/cli', () => ({
Cli: {
InitCliMode: jest.fn().mockReturnValue(false),
},
}));
jest.mock('./model/mac-builder', () => ({
__esModule: true,
default: {
run: jest.fn().mockResolvedValue(0),
},
}));
jest.mock('./model/platform-setup', () => ({
__esModule: true,
default: {
setup: jest.fn().mockResolvedValue(''),
},
}));
const mockedBuildParametersCreate = BuildParameters.create as jest.Mock;
interface EnterpriseBuildParametersOverrides {
providerStrategy?: string;
childWorkspacesEnabled?: boolean;
childWorkspaceName?: string;
childWorkspaceCacheRoot?: string;
childWorkspacePreserveGit?: boolean;
childWorkspaceSeparateLibrary?: boolean;
submoduleProfilePath?: string;
submoduleVariantPath?: string;
submoduleToken?: string;
gitPrivateToken?: string;
lfsTransferAgent?: string;
lfsTransferAgentArgs?: string;
lfsStoragePaths?: string;
localCacheEnabled?: boolean;
localCacheRoot?: string;
localCacheLibrary?: boolean;
localCacheLfs?: boolean;
gitHooksEnabled?: boolean;
gitHooksSkipList?: string;
gitHooksRunBeforeBuild?: string;
}
function createMockBuildParameters(overrides: EnterpriseBuildParametersOverrides = {}) {
return {
// Required base properties
providerStrategy: 'local',
targetPlatform: 'StandaloneLinux64',
editorVersion: '2021.3.1f1',
buildVersion: '1.0.0',
androidVersionCode: '1',
projectPath: '.',
branch: 'main',
runnerTempPath: '/tmp',
// Enterprise features - all disabled by default
childWorkspacesEnabled: false,
childWorkspaceName: '',
childWorkspaceCacheRoot: '',
childWorkspacePreserveGit: true,
childWorkspaceSeparateLibrary: true,
submoduleProfilePath: '',
submoduleVariantPath: '',
submoduleToken: '',
gitPrivateToken: '',
lfsTransferAgent: '',
lfsTransferAgentArgs: '',
lfsStoragePaths: '',
localCacheEnabled: false,
localCacheRoot: '',
localCacheLibrary: true,
localCacheLfs: false,
gitHooksEnabled: false,
gitHooksSkipList: '',
gitHooksRunBeforeBuild: '',
...overrides,
};
}
/**
* The entry point (runMain) is invoked by importing index.ts.
* Since it calls `runMain()` at module scope, we need to re-import it
* for each test. jest.isolateModules() handles this.
*/
async function runIndex(overrides: EnterpriseBuildParametersOverrides = {}): Promise<void> {
mockedBuildParametersCreate.mockResolvedValue(createMockBuildParameters(overrides));
return new Promise<void>((resolve) => {
jest.isolateModules(() => {
require('./index');
// runMain() is async; give it a tick to complete
// We use setImmediate to ensure all microtasks from the dynamic imports resolve
});
// Allow all promises and microtasks to settle
setTimeout(resolve, 100);
});
}
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
describe('index.ts enterprise feature wiring', () => {
const originalPlatform = process.platform;
const originalEnvironment = { ...process.env };
beforeEach(() => {
jest.clearAllMocks();
process.env.GITHUB_WORKSPACE = '/workspace';
// Force linux platform so Docker.run is used (not MacBuilder)
Object.defineProperty(process, 'platform', { value: 'linux' });
});
afterEach(() => {
Object.defineProperty(process, 'platform', { value: originalPlatform });
process.env = { ...originalEnvironment };
});
// -----------------------------------------------------------------------
// GitHooksService gating
// -----------------------------------------------------------------------
describe('GitHooksService gating', () => {
it('should NOT call GitHooksService when gitHooksEnabled is false (default)', async () => {
await runIndex({ gitHooksEnabled: false });
expect(mockGitHooksService.installHooks).not.toHaveBeenCalled();
expect(mockGitHooksService.configureSkipList).not.toHaveBeenCalled();
});
it('should call installHooks when gitHooksEnabled is true', async () => {
await runIndex({ gitHooksEnabled: true });
expect(mockGitHooksService.installHooks).toHaveBeenCalledWith('/workspace');
});
it('should call configureSkipList when gitHooksEnabled and gitHooksSkipList is set', async () => {
await runIndex({
gitHooksEnabled: true,
gitHooksSkipList: 'pre-commit,pre-push',
});
expect(mockGitHooksService.configureSkipList).toHaveBeenCalledWith(['pre-commit', 'pre-push']);
});
it('should NOT call configureSkipList when gitHooksSkipList is empty', async () => {
await runIndex({
gitHooksEnabled: true,
gitHooksSkipList: '',
});
expect(mockGitHooksService.installHooks).toHaveBeenCalled();
expect(mockGitHooksService.configureSkipList).not.toHaveBeenCalled();
});
});
// -----------------------------------------------------------------------
// LocalCacheService gating
// -----------------------------------------------------------------------
describe('LocalCacheService gating', () => {
it('should NOT call LocalCacheService when localCacheEnabled is false (default)', async () => {
await runIndex({ localCacheEnabled: false });
expect(mockLocalCacheService.resolveCacheRoot).not.toHaveBeenCalled();
expect(mockLocalCacheService.generateCacheKey).not.toHaveBeenCalled();
expect(mockLocalCacheService.restoreLibraryCache).not.toHaveBeenCalled();
expect(mockLocalCacheService.restoreLfsCache).not.toHaveBeenCalled();
expect(mockLocalCacheService.saveLibraryCache).not.toHaveBeenCalled();
expect(mockLocalCacheService.saveLfsCache).not.toHaveBeenCalled();
});
it('should call restore and save operations when localCacheEnabled is true', async () => {
await runIndex({
localCacheEnabled: true,
localCacheLibrary: true,
localCacheLfs: true,
});
expect(mockLocalCacheService.resolveCacheRoot).toHaveBeenCalled();
expect(mockLocalCacheService.generateCacheKey).toHaveBeenCalled();
expect(mockLocalCacheService.restoreLibraryCache).toHaveBeenCalled();
expect(mockLocalCacheService.restoreLfsCache).toHaveBeenCalled();
expect(mockLocalCacheService.saveLibraryCache).toHaveBeenCalled();
expect(mockLocalCacheService.saveLfsCache).toHaveBeenCalled();
});
it('should only cache Library when localCacheLibrary is true and localCacheLfs is false', async () => {
await runIndex({
localCacheEnabled: true,
localCacheLibrary: true,
localCacheLfs: false,
});
expect(mockLocalCacheService.restoreLibraryCache).toHaveBeenCalled();
expect(mockLocalCacheService.restoreLfsCache).not.toHaveBeenCalled();
expect(mockLocalCacheService.saveLibraryCache).toHaveBeenCalled();
expect(mockLocalCacheService.saveLfsCache).not.toHaveBeenCalled();
});
it('should only cache LFS when localCacheLfs is true and localCacheLibrary is false', async () => {
await runIndex({
localCacheEnabled: true,
localCacheLibrary: false,
localCacheLfs: true,
});
expect(mockLocalCacheService.restoreLibraryCache).not.toHaveBeenCalled();
expect(mockLocalCacheService.restoreLfsCache).toHaveBeenCalled();
expect(mockLocalCacheService.saveLibraryCache).not.toHaveBeenCalled();
expect(mockLocalCacheService.saveLfsCache).toHaveBeenCalled();
});
});
// -----------------------------------------------------------------------
// ChildWorkspaceService gating
// -----------------------------------------------------------------------
describe('ChildWorkspaceService gating', () => {
it('should NOT call ChildWorkspaceService when childWorkspacesEnabled is false (default)', async () => {
await runIndex({ childWorkspacesEnabled: false });
expect(mockChildWorkspaceService.buildConfig).not.toHaveBeenCalled();
expect(mockChildWorkspaceService.initializeWorkspace).not.toHaveBeenCalled();
expect(mockChildWorkspaceService.saveWorkspace).not.toHaveBeenCalled();
});
it('should NOT call ChildWorkspaceService when childWorkspacesEnabled is true but childWorkspaceName is empty', async () => {
await runIndex({
childWorkspacesEnabled: true,
childWorkspaceName: '',
});
expect(mockChildWorkspaceService.buildConfig).not.toHaveBeenCalled();
});
it('should call buildConfig, initializeWorkspace, and saveWorkspace when enabled with a name', async () => {
mockChildWorkspaceService.buildConfig.mockReturnValue({ enabled: true, workspaceName: 'TurnOfWar' });
await runIndex({
childWorkspacesEnabled: true,
childWorkspaceName: 'TurnOfWar',
childWorkspaceCacheRoot: '/cache/workspaces',
});
expect(mockChildWorkspaceService.buildConfig).toHaveBeenCalledWith(
expect.objectContaining({
childWorkspacesEnabled: true,
childWorkspaceName: 'TurnOfWar',
}),
);
expect(mockChildWorkspaceService.initializeWorkspace).toHaveBeenCalled();
expect(mockChildWorkspaceService.getWorkspaceSize).toHaveBeenCalled();
expect(mockChildWorkspaceService.saveWorkspace).toHaveBeenCalled();
});
});
// -----------------------------------------------------------------------
// SubmoduleProfileService gating
// -----------------------------------------------------------------------
describe('SubmoduleProfileService gating', () => {
it('should NOT call SubmoduleProfileService when submoduleProfilePath is empty (default)', async () => {
await runIndex({ submoduleProfilePath: '' });
expect(mockSubmoduleProfileService.createInitPlan).not.toHaveBeenCalled();
expect(mockSubmoduleProfileService.execute).not.toHaveBeenCalled();
});
it('should call createInitPlan and execute when submoduleProfilePath is set', async () => {
await runIndex({
submoduleProfilePath: '/path/to/profile.yml',
submoduleVariantPath: '',
submoduleToken: 'my-token',
});
expect(mockSubmoduleProfileService.createInitPlan).toHaveBeenCalledWith('/path/to/profile.yml', '', '/workspace');
expect(mockSubmoduleProfileService.execute).toHaveBeenCalled();
});
it('should pass variant path when provided', async () => {
await runIndex({
submoduleProfilePath: '/path/to/profile.yml',
submoduleVariantPath: '/path/to/variant.yml',
});
expect(mockSubmoduleProfileService.createInitPlan).toHaveBeenCalledWith(
'/path/to/profile.yml',
'/path/to/variant.yml',
'/workspace',
);
});
it('should use submoduleToken for auth, falling back to gitPrivateToken', async () => {
await runIndex({
submoduleProfilePath: '/path/to/profile.yml',
submoduleToken: '',
gitPrivateToken: 'fallback-token',
});
expect(mockSubmoduleProfileService.execute).toHaveBeenCalledWith(
expect.anything(),
'/workspace',
'fallback-token',
);
});
it('should prefer submoduleToken over gitPrivateToken', async () => {
await runIndex({
submoduleProfilePath: '/path/to/profile.yml',
submoduleToken: 'specific-token',
gitPrivateToken: 'fallback-token',
});
expect(mockSubmoduleProfileService.execute).toHaveBeenCalledWith(
expect.anything(),
'/workspace',
'specific-token',
);
});
});
// -----------------------------------------------------------------------
// LfsAgentService gating
// -----------------------------------------------------------------------
describe('LfsAgentService gating', () => {
it('should NOT call LfsAgentService when lfsTransferAgent is empty (default)', async () => {
await runIndex({ lfsTransferAgent: '' });
expect(mockLfsAgentService.configure).not.toHaveBeenCalled();
});
it('should call configure when lfsTransferAgent is set', async () => {
await runIndex({
lfsTransferAgent: '/tools/elastic-git-storage',
lfsTransferAgentArgs: '--verbose',
lfsStoragePaths: '/path/a;/path/b',
});
expect(mockLfsAgentService.configure).toHaveBeenCalledWith(
'/tools/elastic-git-storage',
'--verbose',
['/path/a', '/path/b'],
'/workspace',
);
});
it('should pass empty array when lfsStoragePaths is empty', async () => {
await runIndex({
lfsTransferAgent: '/tools/agent',
lfsStoragePaths: '',
});
expect(mockLfsAgentService.configure).toHaveBeenCalledWith('/tools/agent', '', [], '/workspace');
});
});
// -----------------------------------------------------------------------
// Order of operations (restore before build, save after build)
// -----------------------------------------------------------------------
describe('order of operations', () => {
it('should execute restore operations before build and save operations after build', async () => {
const callOrder: string[] = [];
// Track call order for each relevant operation
mockChildWorkspaceService.buildConfig.mockReturnValue({ enabled: true, workspaceName: 'Test' });
mockChildWorkspaceService.initializeWorkspace.mockImplementation(() => {
callOrder.push('child-workspace-restore');
return false;
});
mockChildWorkspaceService.getWorkspaceSize.mockImplementation(() => {
callOrder.push('child-workspace-size');
return '0 B';
});
mockSubmoduleProfileService.createInitPlan.mockImplementation(async () => {
callOrder.push('submodule-profile-plan');
return [];
});
mockSubmoduleProfileService.execute.mockImplementation(async () => {
callOrder.push('submodule-profile-execute');
});
mockLfsAgentService.configure.mockImplementation(async () => {
callOrder.push('lfs-agent-configure');
});
mockLocalCacheService.resolveCacheRoot.mockImplementation(() => {
callOrder.push('local-cache-resolve');
return '/cache';
});
mockLocalCacheService.generateCacheKey.mockImplementation(() => {
callOrder.push('local-cache-keygen');
return 'key-1';
});
mockLocalCacheService.restoreLfsCache.mockImplementation(async () => {
callOrder.push('local-cache-restore-lfs');
return true;
});
mockLocalCacheService.restoreLibraryCache.mockImplementation(async () => {
callOrder.push('local-cache-restore-library');
return true;
});
mockGitHooksService.installHooks.mockImplementation(async () => {
callOrder.push('git-hooks-install');
});
mockLocalCacheService.saveLibraryCache.mockImplementation(async () => {
callOrder.push('local-cache-save-library');
});
mockLocalCacheService.saveLfsCache.mockImplementation(async () => {
callOrder.push('local-cache-save-lfs');
});
mockChildWorkspaceService.saveWorkspace.mockImplementation(() => {
callOrder.push('child-workspace-save');
});
await runIndex({
childWorkspacesEnabled: true,
childWorkspaceName: 'TurnOfWar',
submoduleProfilePath: '/profile.yml',
lfsTransferAgent: '/tools/agent',
localCacheEnabled: true,
localCacheLfs: true,
localCacheLibrary: true,
gitHooksEnabled: true,
});
// Verify restore operations happen before save operations.
// The expected order from index.ts is:
// 1. Child workspace restore
// 2. Submodule profile init
// 3. LFS agent configure
// 4. Local cache restore (LFS then Library)
// 5. Git hooks install
// 6. [BUILD happens here - Docker.run or MacBuilder.run]
// 7. Local cache save (Library then LFS)
// 8. Child workspace save
const restoreOps = [
'child-workspace-restore',
'submodule-profile-plan',
'submodule-profile-execute',
'lfs-agent-configure',
'local-cache-restore-lfs',
'local-cache-restore-library',
'git-hooks-install',
];
const saveOps = ['local-cache-save-library', 'local-cache-save-lfs', 'child-workspace-save'];
// All restore ops should appear before all save ops
for (const restoreOp of restoreOps) {
if (!callOrder.includes(restoreOp)) continue; // Skip if the operation wasn't called
for (const saveOp of saveOps) {
if (!callOrder.includes(saveOp)) continue;
expect(callOrder.indexOf(restoreOp)).toBeLessThan(callOrder.indexOf(saveOp));
}
}
// Child workspace save should be last
if (callOrder.includes('child-workspace-save') && callOrder.includes('local-cache-save-lfs')) {
expect(callOrder.indexOf('local-cache-save-lfs')).toBeLessThan(callOrder.indexOf('child-workspace-save'));
}
});
});
// -----------------------------------------------------------------------
// Non-local provider strategy
// -----------------------------------------------------------------------
describe('non-local provider strategy', () => {
it('should skip all enterprise features when providerStrategy is not local', async () => {
await runIndex({
providerStrategy: 'aws',
childWorkspacesEnabled: true,
childWorkspaceName: 'Test',
submoduleProfilePath: '/profile.yml',
lfsTransferAgent: '/tools/agent',
localCacheEnabled: true,
gitHooksEnabled: true,
});
// None of the enterprise services should be called because
// they are inside the `if (providerStrategy === 'local')` block
expect(mockChildWorkspaceService.buildConfig).not.toHaveBeenCalled();
expect(mockSubmoduleProfileService.createInitPlan).not.toHaveBeenCalled();
expect(mockLfsAgentService.configure).not.toHaveBeenCalled();
expect(mockLocalCacheService.resolveCacheRoot).not.toHaveBeenCalled();
expect(mockGitHooksService.installHooks).not.toHaveBeenCalled();
});
});
});

View File

@@ -1,4 +1,5 @@
import * as core from '@actions/core';
import path from 'node:path';
import { Action, BuildParameters, Cache, Orchestrator, Docker, ImageTag, Output } from './model';
import { Cli } from './model/cli/cli';
import MacBuilder from './model/mac-builder';
@@ -23,6 +24,94 @@ async function runMain() {
if (buildParameters.providerStrategy === 'local') {
core.info('Building locally');
// Child workspace isolation - restore cached workspace before any other setup
let childWorkspaceConfig: any;
if (buildParameters.childWorkspacesEnabled && buildParameters.childWorkspaceName) {
const { ChildWorkspaceService } = await import('./model/orchestrator/services/cache/child-workspace-service');
const cacheRoot =
buildParameters.childWorkspaceCacheRoot ||
path.join(buildParameters.runnerTempPath || process.env.RUNNER_TEMP || '', 'game-ci-workspaces');
childWorkspaceConfig = ChildWorkspaceService.buildConfig({
childWorkspacesEnabled: buildParameters.childWorkspacesEnabled,
childWorkspaceName: buildParameters.childWorkspaceName,
childWorkspaceCacheRoot: cacheRoot,
childWorkspacePreserveGit: buildParameters.childWorkspacePreserveGit,
childWorkspaceSeparateLibrary: buildParameters.childWorkspaceSeparateLibrary,
});
const projectFullPath = path.join(workspace, buildParameters.projectPath);
const restored = ChildWorkspaceService.initializeWorkspace(projectFullPath, childWorkspaceConfig);
core.info(
`Child workspace "${buildParameters.childWorkspaceName}": ${
restored ? 'restored from cache' : 'starting fresh'
}`,
);
// Log workspace size for resource tracking
const size = ChildWorkspaceService.getWorkspaceSize(projectFullPath);
core.info(`Child workspace size after restore: ${size}`);
}
// Submodule profile initialization
if (buildParameters.submoduleProfilePath) {
const { SubmoduleProfileService } = await import(
'./model/orchestrator/services/submodule/submodule-profile-service'
);
core.info('Initializing submodules from profile...');
const plan = await SubmoduleProfileService.createInitPlan(
buildParameters.submoduleProfilePath,
buildParameters.submoduleVariantPath,
workspace,
);
await SubmoduleProfileService.execute(
plan,
workspace,
buildParameters.submoduleToken || buildParameters.gitPrivateToken,
);
}
// Configure custom LFS transfer agent
if (buildParameters.lfsTransferAgent) {
const { LfsAgentService } = await import('./model/orchestrator/services/lfs/lfs-agent-service');
core.info('Configuring custom LFS transfer agent...');
await LfsAgentService.configure(
buildParameters.lfsTransferAgent,
buildParameters.lfsTransferAgentArgs,
buildParameters.lfsStoragePaths ? buildParameters.lfsStoragePaths.split(';') : [],
workspace,
);
}
// Local build caching - restore
let cacheRoot = '';
let cacheKey = '';
if (buildParameters.localCacheEnabled) {
const { LocalCacheService } = await import('./model/orchestrator/services/cache/local-cache-service');
cacheRoot = LocalCacheService.resolveCacheRoot(buildParameters);
cacheKey = LocalCacheService.generateCacheKey(
buildParameters.targetPlatform,
buildParameters.editorVersion,
buildParameters.branch || '',
);
if (buildParameters.localCacheLfs) {
await LocalCacheService.restoreLfsCache(workspace, cacheRoot, cacheKey);
}
if (buildParameters.localCacheLibrary) {
const projectFullPath = path.join(workspace, buildParameters.projectPath);
await LocalCacheService.restoreLibraryCache(projectFullPath, cacheRoot, cacheKey);
}
}
// Git hooks — opt-in only. When disabled (default), do not touch hooks at all.
if (buildParameters.gitHooksEnabled) {
const { GitHooksService } = await import('./model/orchestrator/services/hooks/git-hooks-service');
await GitHooksService.installHooks(workspace);
if (buildParameters.gitHooksSkipList) {
const environment = GitHooksService.configureSkipList(buildParameters.gitHooksSkipList.split(','));
Object.assign(process.env, environment);
}
}
await PlatformSetup.setup(buildParameters, actionFolder);
exitCode =
process.platform === 'darwin'
@@ -32,6 +121,29 @@ async function runMain() {
actionFolder,
...buildParameters,
});
// Local build caching - save
if (buildParameters.localCacheEnabled) {
const { LocalCacheService } = await import('./model/orchestrator/services/cache/local-cache-service');
if (buildParameters.localCacheLibrary) {
const projectFullPath = path.join(workspace, buildParameters.projectPath);
await LocalCacheService.saveLibraryCache(projectFullPath, cacheRoot, cacheKey);
}
if (buildParameters.localCacheLfs) {
await LocalCacheService.saveLfsCache(workspace, cacheRoot, cacheKey);
}
}
// Child workspace isolation - save workspace for next run
if (childWorkspaceConfig && childWorkspaceConfig.enabled) {
const { ChildWorkspaceService } = await import('./model/orchestrator/services/cache/child-workspace-service');
const projectFullPath = path.join(workspace, buildParameters.projectPath);
const preSaveSize = ChildWorkspaceService.getWorkspaceSize(projectFullPath);
core.info(`Child workspace size before save: ${preSaveSize}`);
ChildWorkspaceService.saveWorkspace(projectFullPath, childWorkspaceConfig);
core.info(`Child workspace "${buildParameters.childWorkspaceName}" saved to cache`);
}
} else {
await Orchestrator.run(buildParameters, baseImage.toString());
exitCode = 0;

View File

@@ -106,6 +106,27 @@ class BuildParameters {
public cacheUnityInstallationOnMac!: boolean;
public unityHubVersionOnMac!: string;
public dockerWorkspacePath!: string;
public submoduleProfilePath!: string;
public submoduleVariantPath!: string;
public submoduleToken!: string;
public localCacheEnabled!: boolean;
public localCacheRoot!: string;
public localCacheLibrary!: boolean;
public localCacheLfs!: boolean;
public childWorkspacesEnabled!: boolean;
public childWorkspaceName!: string;
public childWorkspaceCacheRoot!: string;
public childWorkspacePreserveGit!: boolean;
public childWorkspaceSeparateLibrary!: boolean;
public lfsTransferAgent!: string;
public lfsTransferAgentArgs!: string;
public lfsStoragePaths!: string;
public gitHooksEnabled!: boolean;
public gitHooksSkipList!: string;
public gitHooksRunBeforeBuild!: string;
public providerExecutable!: string;
public middleware!: string;
public middlewareFiles!: string[];
public static shouldUseRetainedWorkspaceMode(buildParameters: BuildParameters) {
return buildParameters.maxRetainedWorkspaces > 0 && Orchestrator.lockedWorkspace !== ``;
@@ -242,6 +263,27 @@ class BuildParameters {
cacheUnityInstallationOnMac: Input.cacheUnityInstallationOnMac,
unityHubVersionOnMac: Input.unityHubVersionOnMac,
dockerWorkspacePath: Input.dockerWorkspacePath,
submoduleProfilePath: Input.submoduleProfilePath,
submoduleVariantPath: Input.submoduleVariantPath,
submoduleToken: Input.submoduleToken,
localCacheEnabled: Input.localCacheEnabled,
localCacheRoot: Input.localCacheRoot,
localCacheLibrary: Input.localCacheLibrary,
localCacheLfs: Input.localCacheLfs,
childWorkspacesEnabled: Input.childWorkspacesEnabled,
childWorkspaceName: Input.childWorkspaceName,
childWorkspaceCacheRoot: Input.childWorkspaceCacheRoot,
childWorkspacePreserveGit: Input.childWorkspacePreserveGit,
childWorkspaceSeparateLibrary: Input.childWorkspaceSeparateLibrary,
lfsTransferAgent: Input.lfsTransferAgent,
lfsTransferAgentArgs: Input.lfsTransferAgentArgs,
lfsStoragePaths: Input.lfsStoragePaths,
gitHooksEnabled: Input.gitHooksEnabled,
gitHooksSkipList: Input.gitHooksSkipList,
gitHooksRunBeforeBuild: Input.gitHooksRunBeforeBuild,
providerExecutable: Input.providerExecutable,
middleware: OrchestratorOptions.middleware,
middlewareFiles: OrchestratorOptions.middlewareFiles,
};
}

View File

@@ -12,6 +12,8 @@ import OrchestratorOptionsReader from '../orchestrator/options/orchestrator-opti
import GitHub from '../github';
import { OptionValues } from 'commander';
import { InputKey } from '../input';
import { SubmoduleProfileService } from '../orchestrator/services/submodule/submodule-profile-service';
import { LfsAgentService } from '../orchestrator/services/lfs/lfs-agent-service';
export class Cli {
public static options: OptionValues | undefined;
@@ -53,6 +55,11 @@ export class Cli {
program.option('--artifactName <artifactName>', 'caching artifact name');
program.option('--select <select>', 'select a particular resource');
program.option('--logFile <logFile>', 'output to log file (log stream only)');
program.option('--profilePath <profilePath>', 'path to submodule profile YAML');
program.option('--variantPath <variantPath>', 'path to submodule variant YAML');
program.option('--agentPath <agentPath>', 'path to custom LFS transfer agent');
program.option('--agentArgs <agentArgs>', 'arguments for custom LFS transfer agent');
program.option('--storagePaths <storagePaths>', 'semicolon-separated storage paths for LFS agent');
program.parse(process.argv);
Cli.options = program.opts();
@@ -172,4 +179,26 @@ export class Cli {
return await Orchestrator.Provider.watchWorkflow();
}
@CliFunction(`submodule-init`, `initializes submodules from a YAML profile`)
public static async SubmoduleInit(): Promise<void> {
const profilePath = Cli.options!['profilePath'];
const variantPath = Cli.options!['variantPath'] || '';
if (!profilePath) {
throw new Error('--profilePath is required for submodule-init');
}
const plan = await SubmoduleProfileService.createInitPlan(profilePath, variantPath, process.cwd());
await SubmoduleProfileService.execute(plan, process.cwd());
}
@CliFunction(`lfs-agent-configure`, `configures a custom LFS transfer agent`)
public static async LfsAgentConfigure(): Promise<void> {
const agentPath = Cli.options!['agentPath'];
if (!agentPath) {
throw new Error('--agentPath is required for lfs-agent-configure');
}
const agentArgs = Cli.options!['agentArgs'] || '';
const storagePaths = (Cli.options!['storagePaths'] || '').split(';').filter(Boolean);
await LfsAgentService.configure(agentPath, agentArgs, storagePaths, process.cwd());
}
}

View File

@@ -0,0 +1,509 @@
/**
* Tests for enterprise input properties and their wiring into BuildParameters.
*
* Covers all 20 new input properties added for enterprise features:
* - Boolean inputs: localCacheEnabled, childWorkspacesEnabled, gitHooksEnabled,
* localCacheLibrary, localCacheLfs, childWorkspacePreserveGit, childWorkspaceSeparateLibrary
* - String inputs: submoduleProfilePath, submoduleVariantPath, submoduleToken,
* localCacheRoot, childWorkspaceName, childWorkspaceCacheRoot, lfsTransferAgent,
* lfsTransferAgentArgs, lfsStoragePaths, providerExecutable, gitHooksSkipList,
* gitHooksRunBeforeBuild
*
* Special attention to boolean inputs: GitHub Actions always passes inputs as strings,
* so 'false' must NOT evaluate as truthy (the #1 source of bugs).
*/
import * as core from '@actions/core';
import Input from './input';
import Versioning from './versioning';
import BuildParameters from './build-parameters';
// ---------------------------------------------------------------------------
// Setup
// ---------------------------------------------------------------------------
afterEach(() => {
jest.restoreAllMocks();
});
// ---------------------------------------------------------------------------
// Part 1: Input getters — defaults and explicit values
// ---------------------------------------------------------------------------
describe('Enterprise Input properties', () => {
// -----------------------------------------------------------------------
// Boolean inputs — default and string parsing
// -----------------------------------------------------------------------
describe('localCacheEnabled', () => {
it('returns false by default', () => {
expect(Input.localCacheEnabled).toBe(false);
});
it('returns true when string "true" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('true');
expect(Input.localCacheEnabled).toBe(true);
});
it('returns false when string "false" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('false');
expect(Input.localCacheEnabled).toBe(false);
});
it('returns false when empty string is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('');
expect(Input.localCacheEnabled).toBe(false);
});
});
describe('localCacheLibrary', () => {
it('returns true by default (library caching on by default when cache enabled)', () => {
expect(Input.localCacheLibrary).toBe(true);
});
it('returns true when string "true" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('true');
expect(Input.localCacheLibrary).toBe(true);
});
it('returns false when string "false" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('false');
expect(Input.localCacheLibrary).toBe(false);
});
});
describe('localCacheLfs', () => {
it('returns false by default', () => {
expect(Input.localCacheLfs).toBe(false);
});
it('returns true when string "true" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('true');
expect(Input.localCacheLfs).toBe(true);
});
it('returns false when string "false" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('false');
expect(Input.localCacheLfs).toBe(false);
});
});
describe('childWorkspacesEnabled', () => {
it('returns false by default', () => {
expect(Input.childWorkspacesEnabled).toBe(false);
});
it('returns true when string "true" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('true');
expect(Input.childWorkspacesEnabled).toBe(true);
});
it('returns false when string "false" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('false');
expect(Input.childWorkspacesEnabled).toBe(false);
});
it('returns false when empty string is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('');
expect(Input.childWorkspacesEnabled).toBe(false);
});
});
describe('childWorkspacePreserveGit', () => {
it('returns true by default', () => {
expect(Input.childWorkspacePreserveGit).toBe(true);
});
it('returns false when string "false" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('false');
expect(Input.childWorkspacePreserveGit).toBe(false);
});
it('returns true when string "true" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('true');
expect(Input.childWorkspacePreserveGit).toBe(true);
});
});
describe('childWorkspaceSeparateLibrary', () => {
it('returns true by default', () => {
expect(Input.childWorkspaceSeparateLibrary).toBe(true);
});
it('returns false when string "false" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('false');
expect(Input.childWorkspaceSeparateLibrary).toBe(false);
});
it('returns true when string "true" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('true');
expect(Input.childWorkspaceSeparateLibrary).toBe(true);
});
});
describe('gitHooksEnabled', () => {
it('returns false by default', () => {
expect(Input.gitHooksEnabled).toBe(false);
});
it('returns true when string "true" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('true');
expect(Input.gitHooksEnabled).toBe(true);
});
it('returns false when string "false" is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('false');
expect(Input.gitHooksEnabled).toBe(false);
});
it('returns false when empty string is passed', () => {
jest.spyOn(core, 'getInput').mockReturnValue('');
expect(Input.gitHooksEnabled).toBe(false);
});
});
// -----------------------------------------------------------------------
// Boolean truthiness edge cases — the #1 source of bugs
// -----------------------------------------------------------------------
describe('boolean input string handling (edge cases)', () => {
// These tests verify that the === 'true' comparison is correct.
// In JavaScript, 'false' is truthy when used in a boolean context,
// but the Input class correctly uses === 'true' comparison.
const booleanInputs: Array<{
name: string;
getter: () => boolean;
defaultValue: boolean;
}> = [
{ name: 'localCacheEnabled', getter: () => Input.localCacheEnabled, defaultValue: false },
{ name: 'localCacheLfs', getter: () => Input.localCacheLfs, defaultValue: false },
{ name: 'childWorkspacesEnabled', getter: () => Input.childWorkspacesEnabled, defaultValue: false },
{ name: 'gitHooksEnabled', getter: () => Input.gitHooksEnabled, defaultValue: false },
// These default to true:
{ name: 'localCacheLibrary', getter: () => Input.localCacheLibrary, defaultValue: true },
{ name: 'childWorkspacePreserveGit', getter: () => Input.childWorkspacePreserveGit, defaultValue: true },
{ name: 'childWorkspaceSeparateLibrary', getter: () => Input.childWorkspaceSeparateLibrary, defaultValue: true },
];
test.each(booleanInputs)('$name: "false" string does NOT evaluate as truthy', ({ getter }) => {
jest.spyOn(core, 'getInput').mockReturnValue('false');
expect(getter()).toBe(false);
});
test.each(booleanInputs)('$name: "true" string evaluates as truthy', ({ getter }) => {
jest.spyOn(core, 'getInput').mockReturnValue('true');
expect(getter()).toBe(true);
});
test.each(booleanInputs)('$name: "TRUE" (uppercase) does NOT evaluate as true (case sensitive)', ({ getter }) => {
jest.spyOn(core, 'getInput').mockReturnValue('TRUE');
expect(getter()).toBe(false);
});
test.each(booleanInputs)('$name: "1" does NOT evaluate as true', ({ getter }) => {
jest.spyOn(core, 'getInput').mockReturnValue('1');
expect(getter()).toBe(false);
});
test.each(booleanInputs)('$name: "yes" does NOT evaluate as true', ({ getter }) => {
jest.spyOn(core, 'getInput').mockReturnValue('yes');
expect(getter()).toBe(false);
});
});
// -----------------------------------------------------------------------
// String inputs — defaults and explicit values
// -----------------------------------------------------------------------
describe('submoduleProfilePath', () => {
it('returns empty string by default', () => {
expect(Input.submoduleProfilePath).toBe('');
});
it('takes input from workflow', () => {
jest.spyOn(core, 'getInput').mockReturnValue('config/submodule-profiles/tow/ec/profile.yml');
expect(Input.submoduleProfilePath).toBe('config/submodule-profiles/tow/ec/profile.yml');
});
});
describe('submoduleVariantPath', () => {
it('returns empty string by default', () => {
expect(Input.submoduleVariantPath).toBe('');
});
it('takes input from workflow', () => {
jest.spyOn(core, 'getInput').mockReturnValue('config/submodule-profiles/tow/ec/server.yml');
expect(Input.submoduleVariantPath).toBe('config/submodule-profiles/tow/ec/server.yml');
});
});
describe('submoduleToken', () => {
it('returns empty string by default', () => {
expect(Input.submoduleToken).toBe('');
});
it('takes input from workflow', () => {
jest.spyOn(core, 'getInput').mockReturnValue('ghp_abc123');
expect(Input.submoduleToken).toBe('ghp_abc123');
});
});
describe('localCacheRoot', () => {
it('returns empty string by default', () => {
expect(Input.localCacheRoot).toBe('');
});
it('takes input from workflow', () => {
jest.spyOn(core, 'getInput').mockReturnValue('/d/cache/game-ci');
expect(Input.localCacheRoot).toBe('/d/cache/game-ci');
});
});
describe('childWorkspaceName', () => {
it('returns empty string by default', () => {
expect(Input.childWorkspaceName).toBe('');
});
it('takes input from workflow', () => {
jest.spyOn(core, 'getInput').mockReturnValue('TurnOfWarEndlessCrusade');
expect(Input.childWorkspaceName).toBe('TurnOfWarEndlessCrusade');
});
});
describe('childWorkspaceCacheRoot', () => {
it('returns empty string by default', () => {
expect(Input.childWorkspaceCacheRoot).toBe('');
});
it('takes input from workflow', () => {
jest.spyOn(core, 'getInput').mockReturnValue('/d/workspaces');
expect(Input.childWorkspaceCacheRoot).toBe('/d/workspaces');
});
});
describe('lfsTransferAgent', () => {
it('returns empty string by default', () => {
expect(Input.lfsTransferAgent).toBe('');
});
it('takes input from workflow', () => {
jest.spyOn(core, 'getInput').mockReturnValue('/tools/elastic-git-storage');
expect(Input.lfsTransferAgent).toBe('/tools/elastic-git-storage');
});
});
describe('lfsTransferAgentArgs', () => {
it('returns empty string by default', () => {
expect(Input.lfsTransferAgentArgs).toBe('');
});
it('takes input from workflow', () => {
jest.spyOn(core, 'getInput').mockReturnValue('--verbose --timeout=60');
expect(Input.lfsTransferAgentArgs).toBe('--verbose --timeout=60');
});
});
describe('lfsStoragePaths', () => {
it('returns empty string by default', () => {
expect(Input.lfsStoragePaths).toBe('');
});
it('takes input from workflow', () => {
jest.spyOn(core, 'getInput').mockReturnValue('/storage/primary;/storage/secondary');
expect(Input.lfsStoragePaths).toBe('/storage/primary;/storage/secondary');
});
});
describe('providerExecutable', () => {
it('returns empty string by default', () => {
expect(Input.providerExecutable).toBe('');
});
it('takes input from workflow', () => {
jest.spyOn(core, 'getInput').mockReturnValue('/usr/local/bin/custom-provider');
expect(Input.providerExecutable).toBe('/usr/local/bin/custom-provider');
});
});
describe('gitHooksSkipList', () => {
it('returns empty string by default', () => {
expect(Input.gitHooksSkipList).toBe('');
});
it('takes input from workflow', () => {
jest.spyOn(core, 'getInput').mockReturnValue('pre-commit,pre-push');
expect(Input.gitHooksSkipList).toBe('pre-commit,pre-push');
});
});
describe('gitHooksRunBeforeBuild', () => {
it('returns empty string by default', () => {
expect(Input.gitHooksRunBeforeBuild).toBe('');
});
it('takes input from workflow', () => {
jest.spyOn(core, 'getInput').mockReturnValue('pre-commit');
expect(Input.gitHooksRunBeforeBuild).toBe('pre-commit');
});
});
});
// ---------------------------------------------------------------------------
// Part 2: BuildParameters.create() maps new inputs to properties
// ---------------------------------------------------------------------------
const testLicense =
'<?xml version="1.0" encoding="UTF-8"?><root>\n <License id="Terms">\n <MachineBindings>\n <Binding Key="1" Value="576562626572264761624c65526f7578"/>\n <Binding Key="2" Value="576562626572264761624c65526f7578"/>\n </MachineBindings>\n <MachineID Value="D7nTUnjNAmtsUMcnoyrqkgIbYdM="/>\n <SerialHash Value="2033b8ac3e6faa3742ca9f0bfae44d18f2a96b80"/>\n <Features>\n <Feature Value="33"/>\n <Feature Value="1"/>\n <Feature Value="12"/>\n <Feature Value="2"/>\n <Feature Value="24"/>\n <Feature Value="3"/>\n <Feature Value="36"/>\n <Feature Value="17"/>\n <Feature Value="19"/>\n <Feature Value="62"/>\n </Features>\n <DeveloperData Value="AQAAAEY0LUJHUlgtWEQ0RS1aQ1dWLUM1SlctR0RIQg=="/>\n <SerialMasked Value="F4-BGRX-XD4E-ZCWV-C5JW-XXXX"/>\n <StartDate Value="2021-02-08T00:00:00"/>\n <UpdateDate Value="2021-02-09T00:34:57"/>\n <InitialActivationDate Value="2021-02-08T00:34:56"/>\n <LicenseVersion Value="6.x"/>\n <ClientProvidedVersion Value="2018.4.30f1"/>\n <AlwaysOnline Value="false"/>\n <Entitlements>\n <Entitlement Ns="unity_editor" Tag="UnityPersonal" Type="EDITOR" ValidTo="9999-12-31T00:00:00"/>\n <Entitlement Ns="unity_editor" Tag="DarkSkin" Type="EDITOR_FEATURE" ValidTo="9999-12-31T00:00:00"/>\n </Entitlements>\n </License>\n<Signature xmlns="http://www.w3.org/2000/09/xmldsig#"><SignedInfo><CanonicalizationMethod Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315#WithComments"/><SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"/><Reference URI="#Terms"><Transforms><Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/></Transforms><DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/><DigestValue>m0Db8UK+ktnOLJBtHybkfetpcKo=</DigestValue></Reference></SignedInfo><SignatureValue>o/pUbSQAukz7+ZYAWhnA0AJbIlyyCPL7bKVEM2lVqbrXt7cyey+umkCXamuOgsWPVUKBMkXtMH8L\n5etLmD0getWIhTGhzOnDCk+gtIPfL4jMo9tkEuOCROQAXCci23VFscKcrkB+3X6h4wEOtA2APhOY\nB+wvC794o8/82ffjP79aVAi57rp3Wmzx+9pe9yMwoJuljAy2sc2tIMgdQGWVmOGBpQm3JqsidyzI\nJWG2kjnc7pDXK9pwYzXoKiqUqqrut90d+kQqRyv7MSZXR50HFqD/LI69h68b7P8Bjo3bPXOhNXGR\n9YCoemH6EkfCJxp2gIjzjWW+l2Hj2EsFQi8YXw==</SignatureValue></Signature></root>';
describe('BuildParameters.create() enterprise property mapping', () => {
beforeEach(() => {
jest.spyOn(Versioning, 'determineBuildVersion').mockImplementation(async () => '1.3.37');
process.env.UNITY_LICENSE = testLicense;
});
afterEach(() => {
jest.clearAllMocks();
jest.restoreAllMocks();
});
it('maps submoduleProfilePath from Input', async () => {
jest.spyOn(Input, 'submoduleProfilePath', 'get').mockReturnValue('/path/to/profile.yml');
const parameters = await BuildParameters.create();
expect(parameters.submoduleProfilePath).toBe('/path/to/profile.yml');
});
it('maps submoduleVariantPath from Input', async () => {
jest.spyOn(Input, 'submoduleVariantPath', 'get').mockReturnValue('/path/to/variant.yml');
const parameters = await BuildParameters.create();
expect(parameters.submoduleVariantPath).toBe('/path/to/variant.yml');
});
it('maps submoduleToken from Input', async () => {
jest.spyOn(Input, 'submoduleToken', 'get').mockReturnValue('ghp_token123');
const parameters = await BuildParameters.create();
expect(parameters.submoduleToken).toBe('ghp_token123');
});
it('maps localCacheEnabled from Input', async () => {
jest.spyOn(Input, 'localCacheEnabled', 'get').mockReturnValue(true);
const parameters = await BuildParameters.create();
expect(parameters.localCacheEnabled).toBe(true);
});
it('maps localCacheRoot from Input', async () => {
jest.spyOn(Input, 'localCacheRoot', 'get').mockReturnValue('/d/cache');
const parameters = await BuildParameters.create();
expect(parameters.localCacheRoot).toBe('/d/cache');
});
it('maps localCacheLibrary from Input', async () => {
jest.spyOn(Input, 'localCacheLibrary', 'get').mockReturnValue(false);
const parameters = await BuildParameters.create();
expect(parameters.localCacheLibrary).toBe(false);
});
it('maps localCacheLfs from Input', async () => {
jest.spyOn(Input, 'localCacheLfs', 'get').mockReturnValue(true);
const parameters = await BuildParameters.create();
expect(parameters.localCacheLfs).toBe(true);
});
it('maps childWorkspacesEnabled from Input', async () => {
jest.spyOn(Input, 'childWorkspacesEnabled', 'get').mockReturnValue(true);
const parameters = await BuildParameters.create();
expect(parameters.childWorkspacesEnabled).toBe(true);
});
it('maps childWorkspaceName from Input', async () => {
jest.spyOn(Input, 'childWorkspaceName', 'get').mockReturnValue('TurnOfWar');
const parameters = await BuildParameters.create();
expect(parameters.childWorkspaceName).toBe('TurnOfWar');
});
it('maps childWorkspaceCacheRoot from Input', async () => {
jest.spyOn(Input, 'childWorkspaceCacheRoot', 'get').mockReturnValue('/cache/workspaces');
const parameters = await BuildParameters.create();
expect(parameters.childWorkspaceCacheRoot).toBe('/cache/workspaces');
});
it('maps childWorkspacePreserveGit from Input', async () => {
jest.spyOn(Input, 'childWorkspacePreserveGit', 'get').mockReturnValue(false);
const parameters = await BuildParameters.create();
expect(parameters.childWorkspacePreserveGit).toBe(false);
});
it('maps childWorkspaceSeparateLibrary from Input', async () => {
jest.spyOn(Input, 'childWorkspaceSeparateLibrary', 'get').mockReturnValue(false);
const parameters = await BuildParameters.create();
expect(parameters.childWorkspaceSeparateLibrary).toBe(false);
});
it('maps lfsTransferAgent from Input', async () => {
jest.spyOn(Input, 'lfsTransferAgent', 'get').mockReturnValue('/tools/elastic-git-storage');
const parameters = await BuildParameters.create();
expect(parameters.lfsTransferAgent).toBe('/tools/elastic-git-storage');
});
it('maps lfsTransferAgentArgs from Input', async () => {
jest.spyOn(Input, 'lfsTransferAgentArgs', 'get').mockReturnValue('--verbose');
const parameters = await BuildParameters.create();
expect(parameters.lfsTransferAgentArgs).toBe('--verbose');
});
it('maps lfsStoragePaths from Input', async () => {
jest.spyOn(Input, 'lfsStoragePaths', 'get').mockReturnValue('/path/a;/path/b');
const parameters = await BuildParameters.create();
expect(parameters.lfsStoragePaths).toBe('/path/a;/path/b');
});
it('maps gitHooksEnabled from Input', async () => {
jest.spyOn(Input, 'gitHooksEnabled', 'get').mockReturnValue(true);
const parameters = await BuildParameters.create();
expect(parameters.gitHooksEnabled).toBe(true);
});
it('maps gitHooksSkipList from Input', async () => {
jest.spyOn(Input, 'gitHooksSkipList', 'get').mockReturnValue('pre-commit,pre-push');
const parameters = await BuildParameters.create();
expect(parameters.gitHooksSkipList).toBe('pre-commit,pre-push');
});
it('maps gitHooksRunBeforeBuild from Input', async () => {
jest.spyOn(Input, 'gitHooksRunBeforeBuild', 'get').mockReturnValue('pre-commit');
const parameters = await BuildParameters.create();
expect(parameters.gitHooksRunBeforeBuild).toBe('pre-commit');
});
it('maps providerExecutable from Input', async () => {
jest.spyOn(Input, 'providerExecutable', 'get').mockReturnValue('/usr/local/bin/provider');
const parameters = await BuildParameters.create();
expect(parameters.providerExecutable).toBe('/usr/local/bin/provider');
});
// Test that all enterprise properties have correct defaults when not explicitly set
it('has correct defaults for all enterprise properties', async () => {
const parameters = await BuildParameters.create();
expect(parameters.submoduleProfilePath).toBe('');
expect(parameters.submoduleVariantPath).toBe('');
expect(parameters.submoduleToken).toBe('');
expect(parameters.localCacheEnabled).toBe(false);
expect(parameters.localCacheRoot).toBe('');
expect(parameters.localCacheLibrary).toBe(true);
expect(parameters.localCacheLfs).toBe(false);
expect(parameters.childWorkspacesEnabled).toBe(false);
expect(parameters.childWorkspaceName).toBe('');
expect(parameters.childWorkspaceCacheRoot).toBe('');
expect(parameters.childWorkspacePreserveGit).toBe(true);
expect(parameters.childWorkspaceSeparateLibrary).toBe(true);
expect(parameters.lfsTransferAgent).toBe('');
expect(parameters.lfsTransferAgentArgs).toBe('');
expect(parameters.lfsStoragePaths).toBe('');
expect(parameters.gitHooksEnabled).toBe(false);
expect(parameters.gitHooksSkipList).toBe('');
expect(parameters.gitHooksRunBeforeBuild).toBe('');
expect(parameters.providerExecutable).toBe('');
});
});

View File

@@ -282,6 +282,90 @@ class Input {
return Input.getInput('skipActivation')?.toLowerCase() ?? 'false';
}
static get submoduleProfilePath(): string {
return Input.getInput('submoduleProfilePath') ?? '';
}
static get submoduleVariantPath(): string {
return Input.getInput('submoduleVariantPath') ?? '';
}
static get submoduleToken(): string {
return Input.getInput('submoduleToken') ?? '';
}
static get localCacheEnabled(): boolean {
return (Input.getInput('localCacheEnabled') ?? 'false') === 'true';
}
static get localCacheRoot(): string {
return Input.getInput('localCacheRoot') ?? '';
}
static get localCacheLibrary(): boolean {
return (Input.getInput('localCacheLibrary') ?? 'true') === 'true';
}
static get localCacheLfs(): boolean {
return (Input.getInput('localCacheLfs') ?? 'false') === 'true';
}
static get childWorkspacesEnabled(): boolean {
return (Input.getInput('childWorkspacesEnabled') ?? 'false') === 'true';
}
static get childWorkspaceName(): string {
return Input.getInput('childWorkspaceName') ?? '';
}
static get childWorkspaceCacheRoot(): string {
return Input.getInput('childWorkspaceCacheRoot') ?? '';
}
static get childWorkspacePreserveGit(): boolean {
return (Input.getInput('childWorkspacePreserveGit') ?? 'true') === 'true';
}
static get childWorkspaceSeparateLibrary(): boolean {
return (Input.getInput('childWorkspaceSeparateLibrary') ?? 'true') === 'true';
}
static get lfsTransferAgent(): string {
return Input.getInput('lfsTransferAgent') ?? '';
}
static get lfsTransferAgentArgs(): string {
return Input.getInput('lfsTransferAgentArgs') ?? '';
}
static get lfsStoragePaths(): string {
return Input.getInput('lfsStoragePaths') ?? '';
}
static get gitHooksEnabled(): boolean {
return (Input.getInput('gitHooksEnabled') ?? 'false') === 'true';
}
static get gitHooksSkipList(): string {
return Input.getInput('gitHooksSkipList') ?? '';
}
static get gitHooksRunBeforeBuild(): string {
return Input.getInput('gitHooksRunBeforeBuild') ?? '';
}
static get providerExecutable(): string {
return Input.getInput('providerExecutable') ?? '';
}
static get middleware(): string {
return Input.getInput('middleware') ?? '';
}
static get middlewareFiles(): string {
return Input.getInput('middlewareFiles') ?? '';
}
public static ToEnvVarFormat(input: string) {
if (input.toUpperCase() === input) {
return input;

View File

@@ -166,6 +166,14 @@ class OrchestratorOptions {
return OrchestratorOptions.getInput('commandHookFiles')?.split(`,`) || [];
}
static get middleware(): string {
return OrchestratorOptions.getInput('middleware') || '';
}
static get middlewareFiles(): string[] {
return OrchestratorOptions.getInput('middlewareFiles')?.split(',') || [];
}
// ### ### ###
// Custom commands from yaml parameters
// ### ### ###

View File

@@ -129,6 +129,17 @@ class Orchestrator {
// Store whether we should validate AWS templates (used by aws-local mode)
Orchestrator.validateAwsTemplates = validateAwsTemplates;
// Check for CLI provider executable
if (Orchestrator.buildParameters.providerExecutable) {
const { default: CliProvider } = await import('./providers/cli');
Orchestrator.Provider = new CliProvider(
Orchestrator.buildParameters.providerExecutable,
Orchestrator.buildParameters,
);
OrchestratorLogger.log(`Using CLI provider executable: ${Orchestrator.buildParameters.providerExecutable}`);
return;
}
switch (provider) {
case 'k8s':
Orchestrator.Provider = new Kubernetes(Orchestrator.buildParameters);

View File

@@ -0,0 +1,20 @@
export interface CliProviderRequest {
command: CliProviderSubcommand;
params: Record<string, any>;
}
export interface CliProviderResponse {
success: boolean;
result?: any;
error?: string;
output?: string;
}
export type CliProviderSubcommand =
| 'setup-workflow'
| 'cleanup-workflow'
| 'run-task'
| 'garbage-collect'
| 'list-resources'
| 'list-workflow'
| 'watch-workflow';

View File

@@ -0,0 +1,532 @@
import { EventEmitter } from 'events';
import { ProviderLoader } from '../provider-loader';
// Mock child_process
jest.mock('child_process', () => ({
spawn: jest.fn(),
exec: jest.fn(),
}));
// Mock @actions/core to prevent GitHub Actions API calls
jest.mock('@actions/core', () => ({
info: jest.fn(),
warning: jest.fn(),
error: jest.fn(),
setOutput: jest.fn(),
getInput: jest.fn(() => ''),
}));
// Mock provider-git-manager (required by provider-loader)
jest.mock('../provider-git-manager');
import { spawn } from 'child_process';
import * as core from '@actions/core';
import CliProvider from './cli-provider';
const mockSpawn = spawn as jest.MockedFunction<typeof spawn>;
/**
* Creates a mock child process with stdin, stdout, stderr as EventEmitters.
*/
function createMockChildProcess() {
const child = new EventEmitter() as any;
child.stdin = { write: jest.fn(), end: jest.fn() };
child.stdout = new EventEmitter();
child.stderr = new EventEmitter();
child.kill = jest.fn();
return child;
}
describe('CliProvider', () => {
beforeEach(() => {
jest.clearAllMocks();
});
describe('constructor', () => {
it('validates that executable path is non-empty', () => {
expect(() => new CliProvider('', {} as any)).toThrow('executablePath must be a non-empty string');
});
it('validates that executable path is not just whitespace', () => {
expect(() => new CliProvider(' ', {} as any)).toThrow('executablePath must be a non-empty string');
});
it('accepts a valid executable path', () => {
const provider = new CliProvider('/usr/bin/my-provider', {} as any);
expect(provider).toBeDefined();
});
});
describe('request serialization', () => {
it('sends JSON request to stdin with correct command and params', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.listResources();
// Simulate successful response
child.stdout.emit('data', Buffer.from(JSON.stringify({ success: true, result: [] }) + '\n'));
child.emit('close', 0);
await promise;
expect(child.stdin.write).toHaveBeenCalledTimes(1);
const writtenData = child.stdin.write.mock.calls[0][0];
const parsed = JSON.parse(writtenData);
expect(parsed.command).toBe('list-resources');
expect(parsed.params).toEqual({});
expect(child.stdin.end).toHaveBeenCalled();
});
it('serializes setupWorkflow params correctly', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.setupWorkflow('guid-123', { editorVersion: '2022.3' } as any, 'main', []);
child.stdout.emit('data', Buffer.from(JSON.stringify({ success: true, result: {} }) + '\n'));
child.emit('close', 0);
await promise;
const writtenData = child.stdin.write.mock.calls[0][0];
const parsed = JSON.parse(writtenData);
expect(parsed.command).toBe('setup-workflow');
expect(parsed.params.buildGuid).toBe('guid-123');
expect(parsed.params.branchName).toBe('main');
});
});
describe('response parsing', () => {
it('resolves on successful JSON response', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.listResources();
const resources = [{ Name: 'resource-1' }, { Name: 'resource-2' }];
child.stdout.emit('data', Buffer.from(JSON.stringify({ success: true, result: resources }) + '\n'));
child.emit('close', 0);
const result = await promise;
expect(result).toEqual(resources);
});
it('rejects on error JSON response', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.garbageCollect('', false, 30, false, false);
child.stdout.emit('data', Buffer.from(JSON.stringify({ success: false, error: 'something went wrong' }) + '\n'));
child.emit('close', 1);
await expect(promise).rejects.toThrow('something went wrong');
});
it('rejects when process exits with non-zero code and no JSON response', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.listWorkflow();
child.stderr.emit('data', Buffer.from('segfault\n'));
child.emit('close', 139);
await expect(promise).rejects.toThrow('exited with code 139');
});
it('resolves when process exits with code 0 and no JSON response', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.listResources();
child.stdout.emit('data', Buffer.from('some plain text output\n'));
child.emit('close', 0);
const result = await promise;
// listResources falls back to empty array when result is missing
expect(result).toEqual([]);
});
it('rejects on spawn error', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/nonexistent/path', {} as any);
const promise = provider.listResources();
child.emit('error', new Error('ENOENT'));
await expect(promise).rejects.toThrow('failed to spawn executable');
});
});
describe('runTaskInWorkflow', () => {
it('forwards non-JSON stdout lines as build output and returns final response', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.runTaskInWorkflow('guid', 'image', 'cmd', '/mnt', '/work', [], []);
// Simulate build output followed by JSON response
child.stdout.emit('data', Buffer.from('Building project...\nCompiling scripts...\n'));
child.stdout.emit('data', Buffer.from(JSON.stringify({ success: true, output: 'Build succeeded' }) + '\n'));
child.emit('close', 0);
const result = await promise;
expect(result).toBe('Build succeeded');
});
it('rejects on run-task failure', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.runTaskInWorkflow('guid', 'image', 'cmd', '/mnt', '/work', [], []);
child.stdout.emit(
'data',
Buffer.from(JSON.stringify({ success: false, error: 'Build failed: compilation errors' }) + '\n'),
);
child.emit('close', 1);
await expect(promise).rejects.toThrow('Build failed: compilation errors');
});
it('returns collected output lines when no JSON response and exit code 0', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.runTaskInWorkflow('guid', 'image', 'cmd', '/mnt', '/work', [], []);
child.stdout.emit('data', Buffer.from('line 1\nline 2\n'));
child.emit('close', 0);
const result = await promise;
expect(result).toBe('line 1\nline 2');
});
it('rejects on spawn error', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/nonexistent/path', {} as any);
const promise = provider.runTaskInWorkflow('guid', 'image', 'cmd', '/mnt', '/work', [], []);
child.emit('error', new Error('ENOENT'));
await expect(promise).rejects.toThrow('failed to spawn executable');
});
});
describe('cleanupWorkflow', () => {
it('sends cleanup-workflow command and returns result', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.cleanupWorkflow({ editorVersion: '2022.3' } as any, 'main', []);
child.stdout.emit('data', Buffer.from(JSON.stringify({ success: true, result: 'cleaned' }) + '\n'));
child.emit('close', 0);
const result = await promise;
expect(result).toBe('cleaned');
const parsed = JSON.parse(child.stdin.write.mock.calls[0][0]);
expect(parsed.command).toBe('cleanup-workflow');
expect(parsed.params.branchName).toBe('main');
});
});
describe('garbageCollect', () => {
it('sends garbage-collect command with correct params', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.garbageCollect('filter*', true, 30, false, true);
child.stdout.emit('data', Buffer.from(JSON.stringify({ success: true, output: '3 items removed' }) + '\n'));
child.emit('close', 0);
const result = await promise;
expect(result).toBe('3 items removed');
const parsed = JSON.parse(child.stdin.write.mock.calls[0][0]);
expect(parsed.command).toBe('garbage-collect');
expect(parsed.params.filter).toBe('filter*');
expect(parsed.params.previewOnly).toBe(true);
expect(parsed.params.olderThan).toBe(30);
expect(parsed.params.baseDependencies).toBe(true);
});
it('returns empty string when response has no output', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.garbageCollect('', false, 7, false, false);
child.stdout.emit('data', Buffer.from(JSON.stringify({ success: true }) + '\n'));
child.emit('close', 0);
const result = await promise;
expect(result).toBe('');
});
});
describe('listWorkflow', () => {
it('returns workflow list from response', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.listWorkflow();
const workflows = [{ Name: 'wf-1' }];
child.stdout.emit('data', Buffer.from(JSON.stringify({ success: true, result: workflows }) + '\n'));
child.emit('close', 0);
const result = await promise;
expect(result).toEqual(workflows);
});
it('returns empty array when result is missing', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.listWorkflow();
child.stdout.emit('data', Buffer.from(JSON.stringify({ success: true }) + '\n'));
child.emit('close', 0);
const result = await promise;
expect(result).toEqual([]);
});
});
describe('watchWorkflow', () => {
it('forwards streaming output and resolves on success', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.watchWorkflow();
child.stdout.emit('data', Buffer.from('watching...\nstatus: running\n'));
child.stdout.emit('data', Buffer.from(JSON.stringify({ success: true, output: 'completed' }) + '\n'));
child.emit('close', 0);
const result = await promise;
expect(result).toBe('completed');
const parsed = JSON.parse(child.stdin.write.mock.calls[0][0]);
expect(parsed.command).toBe('watch-workflow');
});
it('rejects on watch-workflow failure', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.watchWorkflow();
child.stdout.emit('data', Buffer.from(JSON.stringify({ success: false, error: 'lost connection' }) + '\n'));
child.emit('close', 1);
await expect(promise).rejects.toThrow('lost connection');
});
it('rejects on spawn error', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.watchWorkflow();
child.emit('error', new Error('ENOENT'));
await expect(promise).rejects.toThrow('failed to spawn executable');
});
it('resolves with collected output when exit code 0 and no JSON', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.watchWorkflow();
child.stdout.emit('data', Buffer.from('line A\nline B\n'));
child.emit('close', 0);
const result = await promise;
expect(result).toBe('line A\nline B');
});
});
describe('stderr forwarding', () => {
it('forwards stderr lines to logger during execute', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.listResources();
child.stderr.emit('data', Buffer.from('warning: something\n'));
child.stdout.emit('data', Buffer.from(JSON.stringify({ success: true, result: [] }) + '\n'));
child.emit('close', 0);
await promise;
// stderr content included in error message if process fails
// Here it succeeds, so we just verify no rejection
});
it('includes stderr in error message when process fails without JSON', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.listResources();
child.stderr.emit('data', Buffer.from('fatal: segfault\n'));
child.emit('close', 1);
await expect(promise).rejects.toThrow('fatal: segfault');
});
});
describe('timeout handling', () => {
beforeEach(() => {
jest.useFakeTimers();
});
afterEach(() => {
jest.useRealTimers();
});
it('rejects and kills process when execute command times out', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.listResources();
// Advance past the 300s default timeout
jest.advanceTimersByTime(301_000);
await expect(promise).rejects.toThrow('timed out');
expect(child.kill).toHaveBeenCalledWith('SIGTERM');
});
it('rejects and kills process when runTaskInWorkflow times out', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.runTaskInWorkflow('guid', 'image', 'cmd', '/mnt', '/work', [], []);
// Advance past the 2-hour timeout (7_200_000ms)
jest.advanceTimersByTime(7_200_001);
await expect(promise).rejects.toThrow('run-task timed out');
expect(child.kill).toHaveBeenCalledWith('SIGTERM');
expect(core.error).toHaveBeenCalledWith(expect.stringContaining('CLI provider timed out after 120 minutes'));
});
it('rejects and kills process when watchWorkflow times out', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.watchWorkflow();
// Advance past the 1-hour timeout (3_600_000ms)
jest.advanceTimersByTime(3_600_001);
await expect(promise).rejects.toThrow('watch-workflow timed out');
expect(child.kill).toHaveBeenCalledWith('SIGTERM');
expect(core.error).toHaveBeenCalledWith(expect.stringContaining('CLI provider timed out after 60 minutes'));
});
it('escalates to SIGKILL after grace period on runTaskInWorkflow timeout', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.runTaskInWorkflow('guid', 'image', 'cmd', '/mnt', '/work', [], []);
// Trigger the timeout
jest.advanceTimersByTime(7_200_001);
await expect(promise).rejects.toThrow('timed out');
// SIGTERM was sent
expect(child.kill).toHaveBeenCalledWith('SIGTERM');
// Advance past the 10s grace period — SIGKILL should fire
jest.advanceTimersByTime(10_001);
expect(child.kill).toHaveBeenCalledWith('SIGKILL');
});
it('does not send SIGKILL if process exits before grace period', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.runTaskInWorkflow('guid', 'image', 'cmd', '/mnt', '/work', [], []);
// Trigger the timeout
jest.advanceTimersByTime(7_200_001);
await expect(promise).rejects.toThrow('timed out');
// Process exits voluntarily after SIGTERM
child.emit('close', 143);
// Advance past the grace period — SIGKILL should NOT fire because process already exited
jest.advanceTimersByTime(10_001);
expect(child.kill).toHaveBeenCalledWith('SIGTERM');
// SIGKILL should not have been called because the close event cleared the timer
expect(child.kill).not.toHaveBeenCalledWith('SIGKILL');
});
it('clears timeout when runTaskInWorkflow completes normally', async () => {
const child = createMockChildProcess();
mockSpawn.mockReturnValue(child);
const provider = new CliProvider('/path/to/exe', {} as any);
const promise = provider.runTaskInWorkflow('guid', 'image', 'cmd', '/mnt', '/work', [], []);
// Process completes before timeout
child.stdout.emit('data', Buffer.from(JSON.stringify({ success: true, output: 'done' }) + '\n'));
child.emit('close', 0);
const result = await promise;
expect(result).toBe('done');
// Advance far past timeout — should NOT reject
jest.advanceTimersByTime(8_000_000);
expect(child.kill).not.toHaveBeenCalled();
});
});
describe('available providers list', () => {
it('includes cli in the available providers', () => {
const providers = ProviderLoader.getAvailableProviders();
expect(providers).toContain('cli');
});
});
});

View File

@@ -0,0 +1,406 @@
import { spawn, ChildProcess } from 'child_process';
import * as core from '@actions/core';
import { ProviderInterface } from '../provider-interface';
import BuildParameters from '../../../build-parameters';
import OrchestratorEnvironmentVariable from '../../options/orchestrator-environment-variable';
import OrchestratorSecret from '../../options/orchestrator-secret';
import { ProviderResource } from '../provider-resource';
import { ProviderWorkflow } from '../provider-workflow';
import OrchestratorLogger from '../../services/core/orchestrator-logger';
import { CliProviderRequest, CliProviderResponse, CliProviderSubcommand } from './cli-provider-protocol';
const DEFAULT_TIMEOUT_MS = 300_000; // 300 seconds
const RUN_TASK_TIMEOUT_MS = 7_200_000; // 2 hours
const WATCH_WORKFLOW_TIMEOUT_MS = 3_600_000; // 1 hour
const SIGKILL_GRACE_MS = 10_000; // 10 seconds grace period before SIGKILL
/**
* Gracefully kill a child process: SIGTERM first, then SIGKILL after a grace period.
*/
function gracefulKill(child: ChildProcess, graceMs: number = SIGKILL_GRACE_MS): void {
child.kill('SIGTERM');
const forceKillTimer = setTimeout(() => {
try {
child.kill('SIGKILL');
} catch {
// Process may already be dead
}
}, graceMs);
// Clear the force-kill timer if the process exits on its own
child.on('close', () => {
clearTimeout(forceKillTimer);
});
}
class CliProvider implements ProviderInterface {
private readonly executablePath: string;
private readonly buildParameters: BuildParameters;
constructor(executablePath: string, buildParameters: BuildParameters) {
if (!executablePath || executablePath.trim() === '') {
throw new Error('CliProvider: executablePath must be a non-empty string');
}
this.executablePath = executablePath;
this.buildParameters = buildParameters;
}
async setupWorkflow(
buildGuid: string,
buildParameters: BuildParameters,
branchName: string,
defaultSecretsArray: { ParameterKey: string; EnvironmentVariable: string; ParameterValue: string }[],
): Promise<any> {
const response = await this.execute('setup-workflow', {
buildGuid,
buildParameters,
branchName,
defaultSecretsArray,
});
return response.result;
}
async cleanupWorkflow(
buildParameters: BuildParameters,
branchName: string,
defaultSecretsArray: { ParameterKey: string; EnvironmentVariable: string; ParameterValue: string }[],
): Promise<any> {
const response = await this.execute('cleanup-workflow', {
buildParameters,
branchName,
defaultSecretsArray,
});
return response.result;
}
async runTaskInWorkflow(
buildGuid: string,
image: string,
commands: string,
mountdir: string,
workingdir: string,
environment: OrchestratorEnvironmentVariable[],
secrets: OrchestratorSecret[],
): Promise<string> {
const request: CliProviderRequest = {
command: 'run-task',
params: {
buildGuid,
image,
commands,
mountdir,
workingdir,
environment,
secrets,
},
};
const timeoutMs = RUN_TASK_TIMEOUT_MS;
return new Promise<string>((resolve, reject) => {
const child = spawn(this.executablePath, ['run-task'], {
stdio: ['pipe', 'pipe', 'pipe'],
shell: process.platform === 'win32',
});
let lastJsonResponse: CliProviderResponse | undefined;
const outputLines: string[] = [];
let stderrOutput = '';
let timedOut = false;
// Set up timeout to prevent indefinite hangs
const timer = setTimeout(() => {
timedOut = true;
const minutes = Math.round(timeoutMs / 60_000);
const message = `CLI provider timed out after ${minutes} minutes. The external provider may be unresponsive.`;
core.error(message);
gracefulKill(child);
reject(new Error(`CliProvider run-task timed out after ${timeoutMs}ms`));
}, timeoutMs);
child.stdin.write(JSON.stringify(request));
child.stdin.end();
child.stdout.on('data', (data: Buffer) => {
const lines = data.toString().split('\n');
for (const line of lines) {
const trimmed = line.trim();
if (!trimmed) continue;
// Try to parse as JSON response
try {
const parsed = JSON.parse(trimmed);
if (typeof parsed === 'object' && parsed !== null && 'success' in parsed) {
lastJsonResponse = parsed as CliProviderResponse;
continue;
}
} catch {
// Not JSON — treat as build output
}
// Forward non-JSON lines as real-time build output
OrchestratorLogger.log(trimmed);
outputLines.push(trimmed);
}
});
child.stderr.on('data', (data: Buffer) => {
const text = data.toString();
stderrOutput += text;
// Forward stderr to logger
for (const line of text.split('\n')) {
const trimmed = line.trim();
if (trimmed) {
OrchestratorLogger.log(`[cli-provider stderr] ${trimmed}`);
}
}
});
child.on('error', (error: Error) => {
clearTimeout(timer);
if (!timedOut) {
reject(new Error(`CliProvider: failed to spawn executable '${this.executablePath}': ${error.message}`));
}
});
child.on('close', (code: number | null) => {
clearTimeout(timer);
if (timedOut) return;
if (lastJsonResponse) {
if (lastJsonResponse.success) {
resolve(lastJsonResponse.output || outputLines.join('\n'));
} else {
reject(
new Error(`CliProvider run-task failed: ${lastJsonResponse.error || 'Unknown error from CLI provider'}`),
);
}
} else if (code === 0) {
resolve(outputLines.join('\n'));
} else {
reject(
new Error(`CliProvider run-task exited with code ${code}${stderrOutput ? ': ' + stderrOutput.trim() : ''}`),
);
}
});
});
}
async garbageCollect(
filter: string,
previewOnly: boolean,
olderThan: Number,
fullCache: boolean,
baseDependencies: boolean,
): Promise<string> {
const response = await this.execute('garbage-collect', {
filter,
previewOnly,
olderThan,
fullCache,
baseDependencies,
});
return response.output || '';
}
async listResources(): Promise<ProviderResource[]> {
const response = await this.execute('list-resources', {});
return (response.result as ProviderResource[]) || [];
}
async listWorkflow(): Promise<ProviderWorkflow[]> {
const response = await this.execute('list-workflow', {});
return (response.result as ProviderWorkflow[]) || [];
}
async watchWorkflow(): Promise<string> {
const request: CliProviderRequest = {
command: 'watch-workflow',
params: {},
};
const timeoutMs = WATCH_WORKFLOW_TIMEOUT_MS;
return new Promise<string>((resolve, reject) => {
const child = spawn(this.executablePath, ['watch-workflow'], {
stdio: ['pipe', 'pipe', 'pipe'],
shell: process.platform === 'win32',
});
let lastJsonResponse: CliProviderResponse | undefined;
const outputLines: string[] = [];
let timedOut = false;
// Set up timeout to prevent indefinite hangs
const timer = setTimeout(() => {
timedOut = true;
const minutes = Math.round(timeoutMs / 60_000);
const message = `CLI provider timed out after ${minutes} minutes. The external provider may be unresponsive.`;
core.error(message);
gracefulKill(child);
reject(new Error(`CliProvider watch-workflow timed out after ${timeoutMs}ms`));
}, timeoutMs);
child.stdin.write(JSON.stringify(request));
child.stdin.end();
child.stdout.on('data', (data: Buffer) => {
const lines = data.toString().split('\n');
for (const line of lines) {
const trimmed = line.trim();
if (!trimmed) continue;
try {
const parsed = JSON.parse(trimmed);
if (typeof parsed === 'object' && parsed !== null && 'success' in parsed) {
lastJsonResponse = parsed as CliProviderResponse;
continue;
}
} catch {
// Not JSON
}
OrchestratorLogger.log(trimmed);
outputLines.push(trimmed);
}
});
child.stderr.on('data', (data: Buffer) => {
for (const line of data.toString().split('\n')) {
const trimmed = line.trim();
if (trimmed) {
OrchestratorLogger.log(`[cli-provider stderr] ${trimmed}`);
}
}
});
child.on('error', (error: Error) => {
clearTimeout(timer);
if (!timedOut) {
reject(new Error(`CliProvider: failed to spawn executable '${this.executablePath}': ${error.message}`));
}
});
child.on('close', (code: number | null) => {
clearTimeout(timer);
if (timedOut) return;
if (lastJsonResponse) {
if (lastJsonResponse.success) {
resolve(lastJsonResponse.output || outputLines.join('\n'));
} else {
reject(new Error(`CliProvider watch-workflow failed: ${lastJsonResponse.error || 'Unknown error'}`));
}
} else if (code === 0) {
resolve(outputLines.join('\n'));
} else {
reject(new Error(`CliProvider watch-workflow exited with code ${code}`));
}
});
});
}
/**
* Execute a CLI provider subcommand with a default timeout.
* Timeout applies a graceful SIGTERM followed by SIGKILL after a grace period.
*/
private execute(
command: CliProviderSubcommand,
params: Record<string, any>,
timeoutMs: number = DEFAULT_TIMEOUT_MS,
): Promise<CliProviderResponse> {
const request: CliProviderRequest = { command, params };
return new Promise<CliProviderResponse>((resolve, reject) => {
const child = spawn(this.executablePath, [command], {
stdio: ['pipe', 'pipe', 'pipe'],
shell: process.platform === 'win32',
});
let stdoutData = '';
let stderrData = '';
let timedOut = false;
// Set up timeout with graceful kill
const timer = setTimeout(() => {
timedOut = true;
gracefulKill(child);
reject(new Error(`CliProvider: command '${command}' timed out after ${timeoutMs}ms`));
}, timeoutMs);
child.stdin.write(JSON.stringify(request));
child.stdin.end();
child.stdout.on('data', (data: Buffer) => {
stdoutData += data.toString();
});
child.stderr.on('data', (data: Buffer) => {
const text = data.toString();
stderrData += text;
// Forward stderr to logger
for (const line of text.split('\n')) {
const trimmed = line.trim();
if (trimmed) {
OrchestratorLogger.log(`[cli-provider stderr] ${trimmed}`);
}
}
});
child.on('error', (error: Error) => {
clearTimeout(timer);
if (!timedOut) {
reject(new Error(`CliProvider: failed to spawn executable '${this.executablePath}': ${error.message}`));
}
});
child.on('close', (code: number | null) => {
clearTimeout(timer);
if (timedOut) return;
// Find the last JSON line in stdout
const lines = stdoutData.split('\n').filter((l) => l.trim());
let response: CliProviderResponse | undefined;
for (let i = lines.length - 1; i >= 0; i--) {
try {
const parsed = JSON.parse(lines[i].trim());
if (typeof parsed === 'object' && parsed !== null && 'success' in parsed) {
response = parsed as CliProviderResponse;
break;
}
} catch {
// Not valid JSON, skip
}
}
if (response) {
if (response.success) {
resolve(response);
} else {
reject(new Error(`CliProvider ${command} failed: ${response.error || 'Unknown error from CLI provider'}`));
}
} else if (code === 0) {
// No JSON response but exit code 0 — treat as success with raw output
resolve({ success: true, output: stdoutData.trim() });
} else {
reject(
new Error(
`CliProvider ${command} exited with code ${code}` +
(stderrData ? `: ${stderrData.trim()}` : '') +
(!stderrData && stdoutData ? `: ${stdoutData.trim()}` : ''),
),
);
}
});
});
}
}
export default CliProvider;

View File

@@ -0,0 +1 @@
export { default } from './cli-provider';

View File

@@ -58,6 +58,7 @@ export default async function loadProvider(
const providerModuleMap: Record<string, string> = {
aws: './aws',
k8s: './k8s',
cli: './cli',
test: './test',
'local-docker': './docker',
'local-system': './local',
@@ -136,7 +137,7 @@ export class ProviderLoader {
* @returns string[] - Array of available provider names
*/
static getAvailableProviders(): string[] {
return ['aws', 'k8s', 'test', 'local-docker', 'local-system', 'local'];
return ['aws', 'k8s', 'cli', 'test', 'local-docker', 'local-system', 'local'];
}
/**

View File

@@ -237,6 +237,23 @@ export class RemoteClient {
`mkdir -p ${OrchestratorFolders.ToLinuxFolder(OrchestratorFolders.cacheFolderForCacheKeyFull)}`,
);
await RemoteClient.cloneRepoWithoutLFSFiles();
// Initialize submodules from profile if configured
if (Orchestrator.buildParameters.submoduleProfilePath) {
const { SubmoduleProfileService } = await import('../services/submodule/submodule-profile-service');
RemoteClientLogger.log('Initializing submodules from profile...');
const plan = await SubmoduleProfileService.createInitPlan(
Orchestrator.buildParameters.submoduleProfilePath,
Orchestrator.buildParameters.submoduleVariantPath,
OrchestratorFolders.repoPathAbsolute,
);
await SubmoduleProfileService.execute(
plan,
OrchestratorFolders.repoPathAbsolute,
Orchestrator.buildParameters.submoduleToken || Orchestrator.buildParameters.gitPrivateToken,
);
}
await RemoteClient.sizeOfFolder(
'repo before lfs cache pull',
OrchestratorFolders.ToLinuxFolder(OrchestratorFolders.repoPathAbsolute),
@@ -251,6 +268,19 @@ export class RemoteClient {
`${lfsHashes.lfsGuidSum}`,
);
await RemoteClient.sizeOfFolder('repo after lfs cache pull', OrchestratorFolders.repoPathAbsolute);
// Configure custom LFS transfer agent if specified
if (Orchestrator.buildParameters.lfsTransferAgent) {
const { LfsAgentService } = await import('../services/lfs/lfs-agent-service');
RemoteClientLogger.log('Configuring custom LFS transfer agent...');
await LfsAgentService.configure(
Orchestrator.buildParameters.lfsTransferAgent,
Orchestrator.buildParameters.lfsTransferAgentArgs,
Orchestrator.buildParameters.lfsStoragePaths ? Orchestrator.buildParameters.lfsStoragePaths.split(';') : [],
OrchestratorFolders.repoPathAbsolute,
);
}
await RemoteClient.pullLatestLFS();
await RemoteClient.sizeOfFolder('repo before lfs git pull', OrchestratorFolders.repoPathAbsolute);
await Caching.PushToCache(

View File

@@ -0,0 +1,458 @@
import fs from 'node:fs';
import path from 'node:path';
import { ChildWorkspaceService, ChildWorkspaceConfig } from './child-workspace-service';
jest.mock('node:fs');
jest.mock('../core/orchestrator-logger', () => ({
__esModule: true,
default: {
log: jest.fn(),
logWarning: jest.fn(),
error: jest.fn(),
},
}));
const mockFs = fs as jest.Mocked<typeof fs>;
function createConfig(overrides: Partial<ChildWorkspaceConfig> = {}): ChildWorkspaceConfig {
return {
enabled: true,
workspaceName: 'TurnOfWar',
parentCacheRoot: '/cache/workspaces',
preserveGitDirectory: true,
separateLibraryCache: true,
...overrides,
};
}
describe('ChildWorkspaceService', () => {
beforeEach(() => {
jest.resetAllMocks();
});
describe('initializeWorkspace', () => {
it('should return false when no cached workspace exists', () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(false);
const result = ChildWorkspaceService.initializeWorkspace('/project', createConfig());
expect(result).toBe(false);
});
it('should return false when cached workspace is empty', () => {
(mockFs.existsSync as jest.Mock).mockImplementation(
(p: string) => String(p) === path.join('/cache/workspaces', 'TurnOfWar'),
);
(mockFs.readdirSync as jest.Mock).mockReturnValue([]);
const result = ChildWorkspaceService.initializeWorkspace('/project', createConfig());
expect(result).toBe(false);
expect(mockFs.rmSync).toHaveBeenCalledWith(path.join('/cache/workspaces', 'TurnOfWar'), {
recursive: true,
force: true,
});
});
it('should restore workspace via atomic move when cache exists', () => {
const cachedPath = path.join('/cache/workspaces', 'TurnOfWar');
(mockFs.existsSync as jest.Mock).mockImplementation((p: string) => {
if (String(p) === cachedPath) return true;
if (String(p) === '/project') return false;
if (String(p) === '/') return true;
return false;
});
(mockFs.readdirSync as jest.Mock).mockReturnValue(['Assets', '.git', 'Library']);
const config = createConfig({ separateLibraryCache: false });
const result = ChildWorkspaceService.initializeWorkspace('/project', config);
expect(result).toBe(true);
expect(mockFs.renameSync).toHaveBeenCalledWith(cachedPath, '/project');
});
it('should remove existing target path before restoring', () => {
const cachedPath = path.join('/cache/workspaces', 'TurnOfWar');
(mockFs.existsSync as jest.Mock).mockImplementation((p: string) => {
if (String(p) === cachedPath) return true;
if (String(p) === '/project') return true;
return false;
});
(mockFs.readdirSync as jest.Mock).mockReturnValue(['Assets']);
const config = createConfig({ separateLibraryCache: false });
const result = ChildWorkspaceService.initializeWorkspace('/project', config);
expect(result).toBe(true);
expect(mockFs.rmSync).toHaveBeenCalledWith('/project', { recursive: true, force: true });
});
it('should restore Library cache separately when configured', () => {
const cachedPath = path.join('/cache/workspaces', 'TurnOfWar');
const libraryBackupPath = path.join('/cache/workspaces', 'TurnOfWar-Library');
const libraryDestination = path.join('/project', 'Library');
(mockFs.existsSync as jest.Mock).mockImplementation((p: string) => {
if (String(p) === cachedPath) return true;
if (String(p) === '/project') return false;
if (String(p) === libraryBackupPath) return true;
if (String(p) === libraryDestination) return false;
return true; // parent dirs
});
(mockFs.readdirSync as jest.Mock).mockReturnValue(['Assets', 'ProjectSettings']);
const result = ChildWorkspaceService.initializeWorkspace('/project', createConfig());
expect(result).toBe(true);
// Should have been called twice: once for workspace, once for Library
expect(mockFs.renameSync).toHaveBeenCalledTimes(2);
expect(mockFs.renameSync).toHaveBeenCalledWith(cachedPath, '/project');
expect(mockFs.renameSync).toHaveBeenCalledWith(libraryBackupPath, libraryDestination);
});
it('should return false and log warning on error', () => {
(mockFs.existsSync as jest.Mock).mockImplementation(() => {
throw new Error('Access denied');
});
const result = ChildWorkspaceService.initializeWorkspace('/project', createConfig());
expect(result).toBe(false);
});
});
describe('saveWorkspace', () => {
it('should skip save when project path does not exist', () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(false);
ChildWorkspaceService.saveWorkspace('/project', createConfig());
expect(mockFs.renameSync).not.toHaveBeenCalled();
});
it('should save workspace via atomic move', () => {
const cachedPath = path.join('/cache/workspaces', 'TurnOfWar');
(mockFs.existsSync as jest.Mock).mockImplementation((p: string) => {
if (String(p) === '/project') return true;
if (String(p) === path.join('/project', 'Library')) return false;
if (String(p) === '/cache/workspaces') return true;
if (String(p) === cachedPath) return false;
return false;
});
const config = createConfig({ separateLibraryCache: false });
ChildWorkspaceService.saveWorkspace('/project', config);
expect(mockFs.renameSync).toHaveBeenCalledWith('/project', cachedPath);
});
it('should remove .git directory when preserveGit is false', () => {
const gitDirectory = path.join('/project', '.git');
(mockFs.existsSync as jest.Mock).mockImplementation((p: string) => {
if (String(p) === '/project') return true;
if (String(p) === gitDirectory) return true;
if (String(p) === path.join('/project', 'Library')) return false;
if (String(p) === '/cache/workspaces') return true;
return false;
});
const config = createConfig({ preserveGitDirectory: false, separateLibraryCache: false });
ChildWorkspaceService.saveWorkspace('/project', config);
expect(mockFs.rmSync).toHaveBeenCalledWith(gitDirectory, { recursive: true, force: true });
});
it('should not remove .git directory when preserveGit is true', () => {
(mockFs.existsSync as jest.Mock).mockImplementation((p: string) => {
if (String(p) === '/project') return true;
if (String(p) === path.join('/project', 'Library')) return false;
if (String(p) === '/cache/workspaces') return true;
return false;
});
const config = createConfig({ preserveGitDirectory: true, separateLibraryCache: false });
ChildWorkspaceService.saveWorkspace('/project', config);
// rmSync should not have been called with .git path
const rmSyncCalls = (mockFs.rmSync as jest.Mock).mock.calls;
const gitRmCalls = rmSyncCalls.filter((call: any[]) => String(call[0]).includes('.git'));
expect(gitRmCalls).toHaveLength(0);
});
it('should remove existing cached workspace before saving', () => {
const cachedPath = path.join('/cache/workspaces', 'TurnOfWar');
(mockFs.existsSync as jest.Mock).mockImplementation((p: string) => {
if (String(p) === '/project') return true;
if (String(p) === path.join('/project', 'Library')) return false;
if (String(p) === '/cache/workspaces') return true;
if (String(p) === cachedPath) return true;
return false;
});
const config = createConfig({ separateLibraryCache: false });
ChildWorkspaceService.saveWorkspace('/project', config);
expect(mockFs.rmSync).toHaveBeenCalledWith(cachedPath, { recursive: true, force: true });
expect(mockFs.renameSync).toHaveBeenCalledWith('/project', cachedPath);
});
it('should save Library separately when separateLibraryCache is true', () => {
const libraryPath = path.join('/project', 'Library');
const libraryBackupPath = path.join('/cache/workspaces', 'TurnOfWar-Library');
(mockFs.existsSync as jest.Mock).mockImplementation((p: string) => {
if (String(p) === '/project') return true;
if (String(p) === libraryPath) return true;
if (String(p) === libraryBackupPath) return false;
if (String(p) === '/cache/workspaces') return true;
return false;
});
(mockFs.readdirSync as jest.Mock).mockReturnValue(['ScriptAssemblies', 'ShaderCache']);
ChildWorkspaceService.saveWorkspace('/project', createConfig());
expect(mockFs.renameSync).toHaveBeenCalledWith(libraryPath, libraryBackupPath);
});
it('should handle save errors gracefully', () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.renameSync as jest.Mock).mockImplementation(() => {
throw new Error('Cross-device link');
});
(mockFs.readdirSync as jest.Mock).mockReturnValue([]);
// Should not throw
ChildWorkspaceService.saveWorkspace('/project', createConfig({ separateLibraryCache: false }));
});
});
describe('restoreLibraryCache', () => {
it('should return false when no Library backup exists', () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(false);
const result = ChildWorkspaceService.restoreLibraryCache('/project', createConfig());
expect(result).toBe(false);
});
it('should return false when Library backup is empty', () => {
const libraryBackup = path.join('/cache/workspaces', 'TurnOfWar-Library');
(mockFs.existsSync as jest.Mock).mockImplementation((p: string) => String(p) === libraryBackup);
(mockFs.readdirSync as jest.Mock).mockReturnValue([]);
const result = ChildWorkspaceService.restoreLibraryCache('/project', createConfig());
expect(result).toBe(false);
expect(mockFs.rmSync).toHaveBeenCalledWith(libraryBackup, { recursive: true, force: true });
});
it('should restore Library via atomic move', () => {
const libraryBackup = path.join('/cache/workspaces', 'TurnOfWar-Library');
const libraryDestination = path.join('/project', 'Library');
(mockFs.existsSync as jest.Mock).mockImplementation((p: string) => {
if (String(p) === libraryBackup) return true;
if (String(p) === libraryDestination) return false;
return false;
});
(mockFs.readdirSync as jest.Mock).mockReturnValue(['ScriptAssemblies']);
const result = ChildWorkspaceService.restoreLibraryCache('/project', createConfig());
expect(result).toBe(true);
expect(mockFs.renameSync).toHaveBeenCalledWith(libraryBackup, libraryDestination);
});
it('should use custom libraryBackupPath when provided', () => {
const customBackup = '/custom/library/cache';
const libraryDestination = path.join('/project', 'Library');
(mockFs.existsSync as jest.Mock).mockImplementation((p: string) => {
if (String(p) === customBackup) return true;
if (String(p) === libraryDestination) return false;
return false;
});
(mockFs.readdirSync as jest.Mock).mockReturnValue(['ScriptAssemblies']);
const config = createConfig({ libraryBackupPath: customBackup });
const result = ChildWorkspaceService.restoreLibraryCache('/project', config);
expect(result).toBe(true);
expect(mockFs.renameSync).toHaveBeenCalledWith(customBackup, libraryDestination);
});
it('should remove existing Library directory before restore', () => {
const libraryBackup = path.join('/cache/workspaces', 'TurnOfWar-Library');
const libraryDestination = path.join('/project', 'Library');
(mockFs.existsSync as jest.Mock).mockImplementation((p: string) => {
if (String(p) === libraryBackup) return true;
if (String(p) === libraryDestination) return true;
return false;
});
(mockFs.readdirSync as jest.Mock).mockReturnValue(['ScriptAssemblies']);
ChildWorkspaceService.restoreLibraryCache('/project', createConfig());
expect(mockFs.rmSync).toHaveBeenCalledWith(libraryDestination, { recursive: true, force: true });
});
});
describe('getWorkspaceSize', () => {
it('should return "0 B" for non-existent directory', () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(false);
const result = ChildWorkspaceService.getWorkspaceSize('/nonexistent');
expect(result).toBe('0 B');
});
it('should calculate and format directory size', () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.readdirSync as jest.Mock).mockReturnValue([
{ name: 'file1.txt', isDirectory: () => false, isFile: () => true },
{ name: 'file2.bin', isDirectory: () => false, isFile: () => true },
]);
(mockFs.statSync as jest.Mock).mockReturnValue({ size: 1024 * 1024 }); // 1 MB each
const result = ChildWorkspaceService.getWorkspaceSize('/workspace');
expect(result).toBe('2.00 MB');
});
it('should return "unknown" when existsSync throws', () => {
(mockFs.existsSync as jest.Mock).mockImplementation(() => {
throw new Error('Permission denied');
});
const result = ChildWorkspaceService.getWorkspaceSize('/workspace');
expect(result).toBe('unknown');
});
it('should recurse into subdirectories', () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
let callCount = 0;
(mockFs.readdirSync as jest.Mock).mockImplementation(() => {
callCount++;
if (callCount === 1) {
return [
{ name: 'subdir', isDirectory: () => true, isFile: () => false },
{ name: 'root.txt', isDirectory: () => false, isFile: () => true },
];
}
return [{ name: 'nested.txt', isDirectory: () => false, isFile: () => true }];
});
(mockFs.statSync as jest.Mock).mockReturnValue({ size: 512 });
const result = ChildWorkspaceService.getWorkspaceSize('/workspace');
expect(result).toBe('1.00 KB');
});
});
describe('cleanStaleWorkspaces', () => {
it('should skip when cache root does not exist', () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(false);
ChildWorkspaceService.cleanStaleWorkspaces('/nonexistent', 7);
expect(mockFs.readdirSync).not.toHaveBeenCalled();
});
it('should remove workspaces older than retention period', () => {
const now = Date.now();
const tenDaysAgo = now - 10 * 24 * 60 * 60 * 1000;
const oneDayAgo = now - 1 * 24 * 60 * 60 * 1000;
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.readdirSync as jest.Mock).mockImplementation((directoryPath: string) => {
if (String(directoryPath) === '/cache') {
return ['old-workspace', 'recent-workspace'];
}
return [];
});
(mockFs.statSync as jest.Mock).mockImplementation((filePath: string) => ({
isDirectory: () => true,
mtimeMs: String(filePath).includes('old') ? tenDaysAgo : oneDayAgo,
size: 0,
}));
ChildWorkspaceService.cleanStaleWorkspaces('/cache', 7);
expect(mockFs.rmSync).toHaveBeenCalledTimes(1);
expect(mockFs.rmSync).toHaveBeenCalledWith(path.join('/cache', 'old-workspace'), {
recursive: true,
force: true,
});
});
it('should not remove workspaces newer than retention period', () => {
const oneDayAgo = Date.now() - 1 * 24 * 60 * 60 * 1000;
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.readdirSync as jest.Mock).mockReturnValue(['recent-workspace']);
(mockFs.statSync as jest.Mock).mockReturnValue({
isDirectory: () => true,
mtimeMs: oneDayAgo,
});
ChildWorkspaceService.cleanStaleWorkspaces('/cache', 7);
expect(mockFs.rmSync).not.toHaveBeenCalled();
});
it('should handle errors during cleanup gracefully', () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.readdirSync as jest.Mock).mockReturnValue(['broken-workspace']);
(mockFs.statSync as jest.Mock).mockImplementation(() => {
throw new Error('Access denied');
});
// Should not throw
ChildWorkspaceService.cleanStaleWorkspaces('/cache', 7);
});
});
describe('buildConfig', () => {
it('should build config from build parameters', () => {
const config = ChildWorkspaceService.buildConfig({
childWorkspacesEnabled: true,
childWorkspaceName: 'Shell',
childWorkspaceCacheRoot: '/d/cache',
childWorkspacePreserveGit: false,
childWorkspaceSeparateLibrary: true,
});
expect(config).toEqual({
enabled: true,
workspaceName: 'Shell',
parentCacheRoot: '/d/cache',
preserveGitDirectory: false,
separateLibraryCache: true,
});
});
it('should build config with defaults from disabled state', () => {
const config = ChildWorkspaceService.buildConfig({
childWorkspacesEnabled: false,
childWorkspaceName: '',
childWorkspaceCacheRoot: '',
childWorkspacePreserveGit: true,
childWorkspaceSeparateLibrary: true,
});
expect(config.enabled).toBe(false);
});
});
});

View File

@@ -0,0 +1,373 @@
import fs from 'node:fs';
import path from 'node:path';
import OrchestratorLogger from '../core/orchestrator-logger';
export interface ChildWorkspaceConfig {
enabled: boolean;
/** Name for this child workspace, used as cache key (e.g., "TurnOfWar", "Shell") */
workspaceName: string;
/** Parent directory for cached child workspaces. Should be on same NTFS volume for O(1) restore. */
parentCacheRoot: string;
/** Keep .git directory in cached workspace for delta operations (default: true) */
preserveGitDirectory: boolean;
/** Cache Library/ independently from workspace (default: true) */
separateLibraryCache: boolean;
/** Override location for Library cache. Defaults to parentCacheRoot/<workspaceName>/Library-cache */
libraryBackupPath?: string;
}
/**
* Child workspace isolation service for enterprise-scale CI builds.
*
* Instead of building in the git checkout directory, this service:
* 1. Keeps the root workspace lean (no LFS files in the checkout dir)
* 2. Creates isolated child workspaces per product/build-target
* 3. Each child workspace gets its own submodule profile, LFS hydration, and Library folder
* 4. After build, the child workspace (.git preserved) is moved to a parent-level backup directory
* 5. On next CI run, the child workspace is restored via atomic filesystem move (O(1) on NTFS)
* 6. Library folders are cached separately for independent restore
*
* This is orders of magnitude faster than actions/cache for 50GB+ workspaces.
*/
export class ChildWorkspaceService {
/**
* Initialize child workspace by restoring from cache if available.
* Uses atomic filesystem move (rename) for O(1) restore on same volume.
*
* @param projectPath - Target path where the workspace should live during build
* @param config - Child workspace configuration
* @returns true if restored from cache, false if starting fresh
*/
static initializeWorkspace(projectPath: string, config: ChildWorkspaceConfig): boolean {
const cachedWorkspacePath = path.join(config.parentCacheRoot, config.workspaceName);
try {
if (!fs.existsSync(cachedWorkspacePath)) {
OrchestratorLogger.log(`[ChildWorkspace] No cached workspace found at ${cachedWorkspacePath}, starting fresh`);
return false;
}
// Verify the cached workspace has content
const entries = fs.readdirSync(cachedWorkspacePath);
if (entries.length === 0) {
OrchestratorLogger.log(`[ChildWorkspace] Cached workspace at ${cachedWorkspacePath} is empty, starting fresh`);
fs.rmSync(cachedWorkspacePath, { recursive: true, force: true });
return false;
}
// If the target path already exists, remove it to make way for the move
if (fs.existsSync(projectPath)) {
OrchestratorLogger.log(`[ChildWorkspace] Removing existing target path: ${projectPath}`);
fs.rmSync(projectPath, { recursive: true, force: true });
}
// Ensure parent directory of project path exists
const parentDirectory = path.dirname(projectPath);
if (!fs.existsSync(parentDirectory)) {
fs.mkdirSync(parentDirectory, { recursive: true });
}
// Atomic move (rename) - O(1) on NTFS when on same volume
OrchestratorLogger.log(`[ChildWorkspace] Restoring workspace: ${cachedWorkspacePath} -> ${projectPath}`);
fs.renameSync(cachedWorkspacePath, projectPath);
OrchestratorLogger.log(`[ChildWorkspace] Workspace restored via atomic move`);
// Restore Library cache separately if configured
if (config.separateLibraryCache) {
ChildWorkspaceService.restoreLibraryCache(projectPath, config);
}
return true;
} catch (error: any) {
OrchestratorLogger.logWarning(`[ChildWorkspace] Workspace restore failed: ${error.message}. Starting fresh.`);
return false;
}
}
/**
* Save child workspace after build for reuse on next CI run.
* Moves the entire workspace to the cache directory via atomic filesystem move.
*
* @param projectPath - Path to the workspace to save
* @param config - Child workspace configuration
*/
static saveWorkspace(projectPath: string, config: ChildWorkspaceConfig): void {
const cachedWorkspacePath = path.join(config.parentCacheRoot, config.workspaceName);
try {
if (!fs.existsSync(projectPath)) {
OrchestratorLogger.log(`[ChildWorkspace] Project path ${projectPath} does not exist, skipping save`);
return;
}
// Remove .git directory if not preserving it (saves space but loses delta capability)
if (!config.preserveGitDirectory) {
const gitDirectory = path.join(projectPath, '.git');
if (fs.existsSync(gitDirectory)) {
OrchestratorLogger.log(`[ChildWorkspace] Removing .git directory (preserveGit=false)`);
fs.rmSync(gitDirectory, { recursive: true, force: true });
}
}
// If separateLibraryCache, move Library/ to its own backup path before saving workspace
if (config.separateLibraryCache) {
ChildWorkspaceService.saveLibraryCache(projectPath, config);
}
// Ensure parent cache root exists
if (!fs.existsSync(config.parentCacheRoot)) {
fs.mkdirSync(config.parentCacheRoot, { recursive: true });
}
// Remove any existing cached workspace to make room
if (fs.existsSync(cachedWorkspacePath)) {
OrchestratorLogger.log(`[ChildWorkspace] Removing previous cached workspace: ${cachedWorkspacePath}`);
fs.rmSync(cachedWorkspacePath, { recursive: true, force: true });
}
// Atomic move (rename) - O(1) on NTFS when on same volume
OrchestratorLogger.log(`[ChildWorkspace] Saving workspace: ${projectPath} -> ${cachedWorkspacePath}`);
fs.renameSync(projectPath, cachedWorkspacePath);
OrchestratorLogger.log(`[ChildWorkspace] Workspace saved via atomic move`);
} catch (error: any) {
OrchestratorLogger.logWarning(`[ChildWorkspace] Workspace save failed: ${error.message}`);
}
}
/**
* Restore Library folder from separate cache location.
*
* @param projectPath - Path to the workspace where Library should be restored
* @param config - Child workspace configuration
* @returns true if Library was restored from cache
*/
static restoreLibraryCache(projectPath: string, config: ChildWorkspaceConfig): boolean {
const libraryBackup = ChildWorkspaceService.resolveLibraryBackupPath(config);
const libraryDestination = path.join(projectPath, 'Library');
try {
if (!fs.existsSync(libraryBackup)) {
OrchestratorLogger.log(`[ChildWorkspace] No Library cache found at ${libraryBackup}`);
return false;
}
const entries = fs.readdirSync(libraryBackup);
if (entries.length === 0) {
OrchestratorLogger.log(`[ChildWorkspace] Library cache at ${libraryBackup} is empty`);
fs.rmSync(libraryBackup, { recursive: true, force: true });
return false;
}
// Remove existing Library directory if present
if (fs.existsSync(libraryDestination)) {
fs.rmSync(libraryDestination, { recursive: true, force: true });
}
// Atomic move
OrchestratorLogger.log(`[ChildWorkspace] Restoring Library cache: ${libraryBackup} -> ${libraryDestination}`);
fs.renameSync(libraryBackup, libraryDestination);
OrchestratorLogger.log(`[ChildWorkspace] Library cache restored`);
return true;
} catch (error: any) {
OrchestratorLogger.logWarning(`[ChildWorkspace] Library cache restore failed: ${error.message}`);
return false;
}
}
/**
* Save Library folder to a separate cache location for independent restore.
* Moves Library/ out of the workspace before workspace save.
*
* @param projectPath - Path to the workspace containing Library/
* @param config - Child workspace configuration
*/
private static saveLibraryCache(projectPath: string, config: ChildWorkspaceConfig): void {
const libraryPath = path.join(projectPath, 'Library');
const libraryBackup = ChildWorkspaceService.resolveLibraryBackupPath(config);
try {
if (!fs.existsSync(libraryPath)) {
OrchestratorLogger.log(`[ChildWorkspace] No Library folder to cache`);
return;
}
const entries = fs.readdirSync(libraryPath);
if (entries.length === 0) {
OrchestratorLogger.log(`[ChildWorkspace] Library folder is empty, skipping cache`);
return;
}
// Ensure parent of backup path exists
const backupParent = path.dirname(libraryBackup);
if (!fs.existsSync(backupParent)) {
fs.mkdirSync(backupParent, { recursive: true });
}
// Remove existing Library backup
if (fs.existsSync(libraryBackup)) {
fs.rmSync(libraryBackup, { recursive: true, force: true });
}
// Atomic move
OrchestratorLogger.log(`[ChildWorkspace] Caching Library: ${libraryPath} -> ${libraryBackup}`);
fs.renameSync(libraryPath, libraryBackup);
OrchestratorLogger.log(`[ChildWorkspace] Library cached separately`);
} catch (error: any) {
OrchestratorLogger.logWarning(`[ChildWorkspace] Library cache save failed: ${error.message}`);
}
}
/**
* Calculate the total size of a directory in human-readable format.
*
* @param directoryPath - Path to the directory to measure
* @returns Human-readable size string (e.g., "1.23 GB", "456.78 MB")
*/
static getWorkspaceSize(directoryPath: string): string {
try {
if (!fs.existsSync(directoryPath)) {
return '0 B';
}
const totalBytes = ChildWorkspaceService.calculateDirectorySize(directoryPath);
return ChildWorkspaceService.formatBytes(totalBytes);
} catch (error: any) {
OrchestratorLogger.logWarning(`[ChildWorkspace] Failed to calculate workspace size: ${error.message}`);
return 'unknown';
}
}
/**
* Clean stale child workspaces that haven't been used within the retention period.
*
* @param parentCacheRoot - Root directory containing cached workspaces
* @param retentionDays - Maximum age in days before a workspace is considered stale
*/
static cleanStaleWorkspaces(parentCacheRoot: string, retentionDays: number): void {
try {
if (!fs.existsSync(parentCacheRoot)) {
OrchestratorLogger.log(`[ChildWorkspace] Cache root ${parentCacheRoot} does not exist, nothing to clean`);
return;
}
const now = Date.now();
const maxAgeMs = retentionDays * 24 * 60 * 60 * 1000;
const entries = fs.readdirSync(parentCacheRoot);
let removedCount = 0;
let freedBytes = 0;
for (const entry of entries) {
const entryPath = path.join(parentCacheRoot, entry);
try {
const stat = fs.statSync(entryPath);
if (stat.isDirectory() && now - stat.mtimeMs > maxAgeMs) {
const size = ChildWorkspaceService.calculateDirectorySize(entryPath);
fs.rmSync(entryPath, { recursive: true, force: true });
removedCount++;
freedBytes += size;
OrchestratorLogger.log(
`[ChildWorkspace] Cleaned stale workspace: ${entry} (age: ${Math.floor(
(now - stat.mtimeMs) / (24 * 60 * 60 * 1000),
)} days)`,
);
}
} catch (error: any) {
OrchestratorLogger.logWarning(`[ChildWorkspace] Failed to clean ${entryPath}: ${error.message}`);
}
}
OrchestratorLogger.log(
`[ChildWorkspace] Cleanup complete: ${removedCount} stale workspaces removed, ${ChildWorkspaceService.formatBytes(
freedBytes,
)} freed`,
);
} catch (error: any) {
OrchestratorLogger.logWarning(`[ChildWorkspace] Stale workspace cleanup failed: ${error.message}`);
}
}
/**
* Build a ChildWorkspaceConfig from build parameters and action inputs.
*/
static buildConfig(parameters: {
childWorkspacesEnabled: boolean;
childWorkspaceName: string;
childWorkspaceCacheRoot: string;
childWorkspacePreserveGit: boolean;
childWorkspaceSeparateLibrary: boolean;
}): ChildWorkspaceConfig {
return {
enabled: parameters.childWorkspacesEnabled,
workspaceName: parameters.childWorkspaceName,
parentCacheRoot: parameters.childWorkspaceCacheRoot,
preserveGitDirectory: parameters.childWorkspacePreserveGit,
separateLibraryCache: parameters.childWorkspaceSeparateLibrary,
};
}
/**
* Resolve the Library backup path from config, using a default if not overridden.
*/
private static resolveLibraryBackupPath(config: ChildWorkspaceConfig): string {
if (config.libraryBackupPath) {
return config.libraryBackupPath;
}
return path.join(config.parentCacheRoot, `${config.workspaceName}-Library`);
}
/**
* Recursively calculate total size of a directory in bytes.
*/
private static calculateDirectorySize(directoryPath: string): number {
let totalSize = 0;
try {
const entries = fs.readdirSync(directoryPath, { withFileTypes: true });
for (const entry of entries) {
const fullPath = path.join(directoryPath, entry.name);
if (entry.isDirectory()) {
totalSize += ChildWorkspaceService.calculateDirectorySize(fullPath);
} else if (entry.isFile()) {
totalSize += fs.statSync(fullPath).size;
}
}
} catch {
// Permission errors or race conditions — return what we have
}
return totalSize;
}
/**
* Format bytes into human-readable string.
*/
private static formatBytes(bytes: number): string {
if (bytes === 0) return '0 B';
const units = ['B', 'KB', 'MB', 'GB', 'TB'];
const k = 1024;
const index = Math.floor(Math.log(bytes) / Math.log(k));
const value = bytes / Math.pow(k, index);
return `${value.toFixed(2)} ${units[index]}`;
}
}

View File

@@ -0,0 +1,272 @@
import fs from 'node:fs';
import path from 'node:path';
import { LocalCacheService } from './local-cache-service';
jest.mock('node:fs');
jest.mock('../core/orchestrator-system', () => ({
OrchestratorSystem: {
Run: jest.fn().mockResolvedValue(''),
},
}));
jest.mock('../core/orchestrator-logger', () => ({
__esModule: true,
default: {
log: jest.fn(),
logWarning: jest.fn(),
error: jest.fn(),
},
}));
const mockFs = fs as jest.Mocked<typeof fs>;
describe('LocalCacheService', () => {
beforeEach(() => {
jest.clearAllMocks();
});
describe('generateCacheKey', () => {
it('should generate a key from platform, version, and branch', () => {
const key = LocalCacheService.generateCacheKey('StandaloneLinux64', '2021.3.1f1', 'main');
expect(key).toBe('StandaloneLinux64-2021_3_1f1-main');
});
it('should sanitize non-alphanumeric characters except hyphens', () => {
const key = LocalCacheService.generateCacheKey('WebGL', '2022.3.0f1', 'feature/my-branch');
expect(key).toBe('WebGL-2022_3_0f1-feature_my-branch');
});
it('should handle empty branch', () => {
const key = LocalCacheService.generateCacheKey('StandaloneWindows64', '2021.3.1f1', '');
expect(key).toBe('StandaloneWindows64-2021_3_1f1-');
});
it('should handle dots in version string', () => {
const key = LocalCacheService.generateCacheKey('Android', '6000.0.23f1', 'main');
expect(key).toBe('Android-6000_0_23f1-main');
});
it('should preserve hyphens in platform names', () => {
const key = LocalCacheService.generateCacheKey('Standalone-Linux64', '2021.3.1f1', 'main');
expect(key).toBe('Standalone-Linux64-2021_3_1f1-main');
});
});
describe('resolveCacheRoot', () => {
const originalEnv = process.env;
beforeEach(() => {
process.env = { ...originalEnv };
});
afterEach(() => {
process.env = originalEnv;
});
it('should use localCacheRoot when set', () => {
const result = LocalCacheService.resolveCacheRoot({ localCacheRoot: '/custom/cache' });
expect(result).toBe('/custom/cache');
});
it('should use RUNNER_TEMP when localCacheRoot is empty', () => {
process.env.RUNNER_TEMP = '/tmp/runner';
const result = LocalCacheService.resolveCacheRoot({ localCacheRoot: '' });
expect(result).toBe(path.join('/tmp/runner', 'game-ci-cache'));
});
it('should fall back to .game-ci/cache when neither is set', () => {
delete process.env.RUNNER_TEMP;
const result = LocalCacheService.resolveCacheRoot({ localCacheRoot: '' });
expect(result).toBe(path.join(process.cwd(), '.game-ci', 'cache'));
});
});
describe('restoreLibraryCache', () => {
it('should return false on cache miss (directory does not exist)', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(false);
const result = await LocalCacheService.restoreLibraryCache('/project', '/cache', 'key1');
expect(result).toBe(false);
});
it('should return false when cache directory has no tar files', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.readdirSync as jest.Mock).mockReturnValue(['readme.txt', 'info.json']);
const result = await LocalCacheService.restoreLibraryCache('/project', '/cache', 'key1');
expect(result).toBe(false);
});
it('should restore from the latest tar file on cache hit', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.readdirSync as jest.Mock).mockReturnValue(['lib-1000.tar', 'lib-2000.tar']);
(mockFs.statSync as jest.Mock).mockImplementation((filePath: string) => ({
mtimeMs: String(filePath).includes('lib-2000') ? 2000 : 1000,
}));
(mockFs.mkdirSync as jest.Mock).mockReturnValue(undefined);
const { OrchestratorSystem } = require('../core/orchestrator-system');
const result = await LocalCacheService.restoreLibraryCache('/project', '/cache', 'key1');
expect(result).toBe(true);
expect(OrchestratorSystem.Run).toHaveBeenCalledWith(expect.stringContaining('lib-2000.tar'), true);
});
it('should return false and log warning on error', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.readdirSync as jest.Mock).mockImplementation(() => {
throw new Error('Permission denied');
});
const result = await LocalCacheService.restoreLibraryCache('/project', '/cache', 'key1');
expect(result).toBe(false);
});
});
describe('saveLibraryCache', () => {
it('should skip save when Library folder does not exist', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(false);
await LocalCacheService.saveLibraryCache('/project', '/cache', 'key1');
expect(mockFs.mkdirSync).not.toHaveBeenCalled();
});
it('should skip save when Library folder is empty', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.readdirSync as jest.Mock).mockReturnValue([]);
await LocalCacheService.saveLibraryCache('/project', '/cache', 'key1');
expect(mockFs.mkdirSync).not.toHaveBeenCalled();
});
it('should create cache directory and save tar', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.readdirSync as jest.Mock).mockImplementation((dirPath: string) => {
if (String(dirPath).includes('Library') && !String(dirPath).includes('cache')) {
return ['file1.asset', 'file2.asset'];
}
return [];
});
(mockFs.statSync as jest.Mock).mockReturnValue({ mtimeMs: Date.now() });
(mockFs.mkdirSync as jest.Mock).mockReturnValue(undefined);
const { OrchestratorSystem } = require('../core/orchestrator-system');
OrchestratorSystem.Run.mockResolvedValue('');
await LocalCacheService.saveLibraryCache('/project', '/cache', 'key1');
expect(mockFs.mkdirSync).toHaveBeenCalledWith(path.join('/cache', 'key1', 'Library'), { recursive: true });
expect(OrchestratorSystem.Run).toHaveBeenCalledWith(expect.stringContaining('tar -cf'), true);
});
});
describe('restoreLfsCache', () => {
it('should return false on cache miss', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(false);
const result = await LocalCacheService.restoreLfsCache('/repo', '/cache', 'key1');
expect(result).toBe(false);
});
it('should return false when no tar files exist', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.readdirSync as jest.Mock).mockReturnValue(['readme.txt']);
const result = await LocalCacheService.restoreLfsCache('/repo', '/cache', 'key1');
expect(result).toBe(false);
});
it('should restore from latest tar on hit', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.readdirSync as jest.Mock).mockReturnValue(['lfs-100.tar', 'lfs-200.tar']);
(mockFs.statSync as jest.Mock).mockImplementation((filePath: string) => ({
mtimeMs: String(filePath).includes('lfs-200') ? 200 : 100,
}));
(mockFs.mkdirSync as jest.Mock).mockReturnValue(undefined);
const { OrchestratorSystem } = require('../core/orchestrator-system');
const result = await LocalCacheService.restoreLfsCache('/repo', '/cache', 'key1');
expect(result).toBe(true);
expect(OrchestratorSystem.Run).toHaveBeenCalledWith(expect.stringContaining('lfs-200.tar'), true);
});
});
describe('saveLfsCache', () => {
it('should skip when .git/lfs does not exist', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(false);
await LocalCacheService.saveLfsCache('/repo', '/cache', 'key1');
expect(mockFs.mkdirSync).not.toHaveBeenCalled();
});
it('should skip when .git/lfs is empty', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.readdirSync as jest.Mock).mockReturnValue([]);
await LocalCacheService.saveLfsCache('/repo', '/cache', 'key1');
expect(mockFs.mkdirSync).not.toHaveBeenCalled();
});
it('should create cache directory and save tar when lfs has content', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.readdirSync as jest.Mock).mockImplementation((dirPath: string) => {
if (String(dirPath).includes('lfs') && !String(dirPath).includes('cache')) {
return ['objects', 'tmp'];
}
return [];
});
(mockFs.statSync as jest.Mock).mockReturnValue({ mtimeMs: Date.now() });
(mockFs.mkdirSync as jest.Mock).mockReturnValue(undefined);
const { OrchestratorSystem } = require('../core/orchestrator-system');
OrchestratorSystem.Run.mockResolvedValue('');
await LocalCacheService.saveLfsCache('/repo', '/cache', 'key1');
expect(mockFs.mkdirSync).toHaveBeenCalledWith(path.join('/cache', 'key1', 'lfs'), { recursive: true });
expect(OrchestratorSystem.Run).toHaveBeenCalledWith(expect.stringContaining('tar -cf'), true);
});
it('should handle save errors gracefully', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.readdirSync as jest.Mock).mockImplementation(() => {
throw new Error('Disk full');
});
// Should not throw
await LocalCacheService.saveLfsCache('/repo', '/cache', 'key1');
});
});
describe('garbageCollect', () => {
it('should skip when cache root does not exist', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(false);
await LocalCacheService.garbageCollect('/nonexistent');
});
it('should remove directories older than maxAgeDays', async () => {
const now = Date.now();
const eightDaysAgo = now - 8 * 24 * 60 * 60 * 1000;
const oneDayAgo = now - 1 * 24 * 60 * 60 * 1000;
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.readdirSync as jest.Mock).mockReturnValue(['old-cache', 'recent-cache']);
(mockFs.statSync as jest.Mock).mockImplementation((filePath: string) => ({
isDirectory: () => true,
mtimeMs: String(filePath).includes('old') ? eightDaysAgo : oneDayAgo,
}));
(mockFs.rmSync as jest.Mock).mockReturnValue(undefined);
await LocalCacheService.garbageCollect('/cache', 7);
expect(mockFs.rmSync).toHaveBeenCalledTimes(1);
expect(mockFs.rmSync).toHaveBeenCalledWith(path.join('/cache', 'old-cache'), { recursive: true, force: true });
});
it('should not remove directories newer than maxAgeDays', async () => {
const oneDayAgo = Date.now() - 1 * 24 * 60 * 60 * 1000;
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.readdirSync as jest.Mock).mockReturnValue(['recent-cache']);
(mockFs.statSync as jest.Mock).mockReturnValue({
isDirectory: () => true,
mtimeMs: oneDayAgo,
});
await LocalCacheService.garbageCollect('/cache', 7);
expect(mockFs.rmSync).not.toHaveBeenCalled();
});
});
});

View File

@@ -0,0 +1,273 @@
import fs from 'node:fs';
import path from 'node:path';
import { OrchestratorSystem } from '../core/orchestrator-system';
import OrchestratorLogger from '../core/orchestrator-logger';
export class LocalCacheService {
/**
* Resolve the cache root directory based on build parameters and environment.
* Priority: localCacheRoot > RUNNER_TEMP/game-ci-cache > .game-ci/cache
*/
static resolveCacheRoot(buildParameters: { localCacheRoot: string }): string {
if (buildParameters.localCacheRoot) {
return buildParameters.localCacheRoot;
}
if (process.env.RUNNER_TEMP) {
return path.join(process.env.RUNNER_TEMP, 'game-ci-cache');
}
return path.join(process.cwd(), '.game-ci', 'cache');
}
/**
* Generate a sanitized cache key from build parameters.
* Non-alphanumeric characters (except hyphens) are replaced with underscores.
*/
static generateCacheKey(targetPlatform: string, unityVersion: string, branch: string): string {
const raw = `${targetPlatform}-${unityVersion}-${branch}`;
return raw.replace(/[^a-zA-Z0-9-]/g, '_');
}
/**
* Restore Unity Library cache from the local filesystem.
* Returns true if cache was restored, false on cache miss.
*/
static async restoreLibraryCache(projectPath: string, cacheRoot: string, cacheKey: string): Promise<boolean> {
const cachePath = path.join(cacheRoot, cacheKey, 'Library');
try {
if (!fs.existsSync(cachePath)) {
OrchestratorLogger.log(`[LocalCache] Library cache miss: ${cachePath}`);
return false;
}
const files = fs.readdirSync(cachePath).filter((f) => f.endsWith('.tar'));
if (files.length === 0) {
OrchestratorLogger.log(`[LocalCache] Library cache miss (no tar files): ${cachePath}`);
return false;
}
// Find the latest tar file by modification time
let latestFile = files[0];
let latestMtime = fs.statSync(path.join(cachePath, files[0])).mtimeMs;
for (let i = 1; i < files.length; i++) {
const mtime = fs.statSync(path.join(cachePath, files[i])).mtimeMs;
if (mtime > latestMtime) {
latestMtime = mtime;
latestFile = files[i];
}
}
const tarPath = path.join(cachePath, latestFile);
const libraryDest = path.join(projectPath, 'Library');
// Ensure destination exists
fs.mkdirSync(libraryDest, { recursive: true });
OrchestratorLogger.log(`[LocalCache] Library cache hit: restoring from ${tarPath}`);
await OrchestratorSystem.Run(`tar -xf "${tarPath}" -C "${projectPath}"`, true);
OrchestratorLogger.log(`[LocalCache] Library cache restored successfully`);
return true;
} catch (error: any) {
OrchestratorLogger.logWarning(`[LocalCache] Library cache restore failed: ${error.message}`);
return false;
}
}
/**
* Save Unity Library folder to the local cache as a tar archive.
* Keeps only the latest 2 cache entries.
*/
static async saveLibraryCache(projectPath: string, cacheRoot: string, cacheKey: string): Promise<void> {
const libraryPath = path.join(projectPath, 'Library');
try {
if (!fs.existsSync(libraryPath)) {
OrchestratorLogger.log(`[LocalCache] Library folder does not exist, skipping save`);
return;
}
const entries = fs.readdirSync(libraryPath);
if (entries.length === 0) {
OrchestratorLogger.log(`[LocalCache] Library folder is empty, skipping save`);
return;
}
const cachePath = path.join(cacheRoot, cacheKey, 'Library');
fs.mkdirSync(cachePath, { recursive: true });
const timestamp = Date.now();
const tarName = `lib-${timestamp}.tar`;
const tarPath = path.join(cachePath, tarName);
OrchestratorLogger.log(`[LocalCache] Saving Library cache to ${tarPath}`);
await OrchestratorSystem.Run(`tar -cf "${tarPath}" -C "${projectPath}" Library`, true);
OrchestratorLogger.log(`[LocalCache] Library cache saved successfully`);
// Clean up old entries - keep latest 2
await LocalCacheService.cleanupOldEntries(cachePath, 2);
} catch (error: any) {
OrchestratorLogger.logWarning(`[LocalCache] Library cache save failed: ${error.message}`);
}
}
/**
* Restore LFS cache from the local filesystem.
* Returns true if cache was restored, false on cache miss.
*/
static async restoreLfsCache(repoPath: string, cacheRoot: string, cacheKey: string): Promise<boolean> {
const cachePath = path.join(cacheRoot, cacheKey, 'lfs');
try {
if (!fs.existsSync(cachePath)) {
OrchestratorLogger.log(`[LocalCache] LFS cache miss: ${cachePath}`);
return false;
}
const files = fs.readdirSync(cachePath).filter((f) => f.endsWith('.tar'));
if (files.length === 0) {
OrchestratorLogger.log(`[LocalCache] LFS cache miss (no tar files): ${cachePath}`);
return false;
}
// Find the latest tar file by modification time
let latestFile = files[0];
let latestMtime = fs.statSync(path.join(cachePath, files[0])).mtimeMs;
for (let i = 1; i < files.length; i++) {
const mtime = fs.statSync(path.join(cachePath, files[i])).mtimeMs;
if (mtime > latestMtime) {
latestMtime = mtime;
latestFile = files[i];
}
}
const tarPath = path.join(cachePath, latestFile);
const lfsDest = path.join(repoPath, '.git', 'lfs');
// Ensure destination exists
fs.mkdirSync(lfsDest, { recursive: true });
OrchestratorLogger.log(`[LocalCache] LFS cache hit: restoring from ${tarPath}`);
await OrchestratorSystem.Run(`tar -xf "${tarPath}" -C "${path.join(repoPath, '.git')}"`, true);
OrchestratorLogger.log(`[LocalCache] LFS cache restored successfully`);
return true;
} catch (error: any) {
OrchestratorLogger.logWarning(`[LocalCache] LFS cache restore failed: ${error.message}`);
return false;
}
}
/**
* Save .git/lfs folder to the local cache as a tar archive.
* Keeps only the latest 2 cache entries.
*/
static async saveLfsCache(repoPath: string, cacheRoot: string, cacheKey: string): Promise<void> {
const lfsPath = path.join(repoPath, '.git', 'lfs');
try {
if (!fs.existsSync(lfsPath)) {
OrchestratorLogger.log(`[LocalCache] LFS folder does not exist, skipping save`);
return;
}
const entries = fs.readdirSync(lfsPath);
if (entries.length === 0) {
OrchestratorLogger.log(`[LocalCache] LFS folder is empty, skipping save`);
return;
}
const cachePath = path.join(cacheRoot, cacheKey, 'lfs');
fs.mkdirSync(cachePath, { recursive: true });
const timestamp = Date.now();
const tarName = `lfs-${timestamp}.tar`;
const tarPath = path.join(cachePath, tarName);
OrchestratorLogger.log(`[LocalCache] Saving LFS cache to ${tarPath}`);
await OrchestratorSystem.Run(`tar -cf "${tarPath}" -C "${path.join(repoPath, '.git')}" lfs`, true);
OrchestratorLogger.log(`[LocalCache] LFS cache saved successfully`);
// Clean up old entries - keep latest 2
await LocalCacheService.cleanupOldEntries(cachePath, 2);
} catch (error: any) {
OrchestratorLogger.logWarning(`[LocalCache] LFS cache save failed: ${error.message}`);
}
}
/**
* Remove cache entries older than maxAgeDays from the cache root.
*/
static async garbageCollect(cacheRoot: string, maxAgeDays: number = 7): Promise<void> {
try {
if (!fs.existsSync(cacheRoot)) {
OrchestratorLogger.log(`[LocalCache] Cache root does not exist, nothing to collect`);
return;
}
const now = Date.now();
const maxAgeMs = maxAgeDays * 24 * 60 * 60 * 1000;
const entries = fs.readdirSync(cacheRoot);
let removedCount = 0;
for (const entry of entries) {
const entryPath = path.join(cacheRoot, entry);
try {
const stat = fs.statSync(entryPath);
if (stat.isDirectory() && now - stat.mtimeMs > maxAgeMs) {
fs.rmSync(entryPath, { recursive: true, force: true });
removedCount++;
OrchestratorLogger.log(`[LocalCache] Garbage collected: ${entryPath}`);
}
} catch (error: any) {
OrchestratorLogger.logWarning(`[LocalCache] Failed to garbage collect ${entryPath}: ${error.message}`);
}
}
OrchestratorLogger.log(`[LocalCache] Garbage collection complete: ${removedCount} entries removed`);
} catch (error: any) {
OrchestratorLogger.logWarning(`[LocalCache] Garbage collection failed: ${error.message}`);
}
}
/**
* Clean up old tar files in a cache directory, keeping only the latest N.
*/
private static async cleanupOldEntries(cachePath: string, keepCount: number): Promise<void> {
try {
const files = fs
.readdirSync(cachePath)
.filter((f) => f.endsWith('.tar'))
.map((f) => ({
name: f,
mtime: fs.statSync(path.join(cachePath, f)).mtimeMs,
}))
.sort((a, b) => b.mtime - a.mtime);
if (files.length > keepCount) {
const toRemove = files.slice(keepCount);
for (const file of toRemove) {
const filePath = path.join(cachePath, file.name);
fs.unlinkSync(filePath);
OrchestratorLogger.log(`[LocalCache] Cleaned up old cache entry: ${filePath}`);
}
}
} catch (error: any) {
OrchestratorLogger.logWarning(`[LocalCache] Cleanup of old entries failed: ${error.message}`);
}
}
}

View File

@@ -0,0 +1,405 @@
import fs from 'node:fs';
import path from 'node:path';
import { GitHooksService } from './git-hooks-service';
// Mock dependencies
jest.mock('node:fs');
jest.mock('../core/orchestrator-system', () => ({
OrchestratorSystem: {
Run: jest.fn().mockResolvedValue(''),
},
}));
jest.mock('../core/orchestrator-logger', () => ({
__esModule: true,
default: {
log: jest.fn(),
logWarning: jest.fn(),
error: jest.fn(),
},
}));
const mockFs = fs as jest.Mocked<typeof fs>;
describe('GitHooksService', () => {
beforeEach(() => {
jest.clearAllMocks();
});
describe('detectHookFramework', () => {
it('should detect lefthook.yml', () => {
(mockFs.existsSync as jest.Mock).mockImplementation((filePath: string) => {
return String(filePath).includes('lefthook.yml') && !String(filePath).startsWith('.');
});
const result = GitHooksService.detectHookFramework('/repo');
expect(result).toBe('lefthook');
});
it('should detect .lefthook.yml', () => {
(mockFs.existsSync as jest.Mock).mockImplementation((filePath: string) => {
return String(filePath).includes('.lefthook.yml');
});
const result = GitHooksService.detectHookFramework('/repo');
expect(result).toBe('lefthook');
});
it('should detect .husky directory', () => {
(mockFs.existsSync as jest.Mock).mockImplementation((filePath: string) => {
return String(filePath).endsWith('.husky');
});
const result = GitHooksService.detectHookFramework('/repo');
expect(result).toBe('husky');
});
it('should return none when no framework is detected', () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(false);
const result = GitHooksService.detectHookFramework('/repo');
expect(result).toBe('none');
});
});
describe('detectUnityGitHooks', () => {
it('should return true when package is in manifest.json', () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.readFileSync as jest.Mock).mockReturnValue(
JSON.stringify({
dependencies: {
'com.frostebite.unitygithooks': 'https://github.com/frostebite/UnityGitHooks.git#1.0.5',
},
}),
);
expect(GitHooksService.detectUnityGitHooks('/repo')).toBe(true);
});
it('should return false when package is not in manifest.json', () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.readFileSync as jest.Mock).mockReturnValue(
JSON.stringify({
dependencies: {
'com.unity.textmeshpro': '3.0.6',
},
}),
);
expect(GitHooksService.detectUnityGitHooks('/repo')).toBe(false);
});
it('should return false when manifest.json does not exist', () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(false);
expect(GitHooksService.detectUnityGitHooks('/repo')).toBe(false);
});
it('should return false on read error', () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.readFileSync as jest.Mock).mockImplementation(() => {
throw new Error('Permission denied');
});
expect(GitHooksService.detectUnityGitHooks('/repo')).toBe(false);
});
});
describe('findUnityGitHooksPackagePath', () => {
it('should find versioned package directory', () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.readdirSync as jest.Mock).mockReturnValue([
'com.unity.textmeshpro@3.0.6',
'com.frostebite.unitygithooks@1.0.5',
'com.unity.ugui@1.0.0',
]);
const result = GitHooksService.findUnityGitHooksPackagePath('/repo');
expect(result).toContain('com.frostebite.unitygithooks@1.0.5');
});
it('should return empty string when package not in cache', () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.readdirSync as jest.Mock).mockReturnValue(['com.unity.textmeshpro@3.0.6']);
const result = GitHooksService.findUnityGitHooksPackagePath('/repo');
expect(result).toBe('');
});
it('should return empty string when PackageCache does not exist', () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(false);
const result = GitHooksService.findUnityGitHooksPackagePath('/repo');
expect(result).toBe('');
});
});
describe('initUnityGitHooks', () => {
it('should run the init script when found', async () => {
const { OrchestratorSystem } = require('../core/orchestrator-system');
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.readdirSync as jest.Mock).mockReturnValue(['com.frostebite.unitygithooks@1.0.5']);
await GitHooksService.initUnityGitHooks('/repo');
expect(OrchestratorSystem.Run).toHaveBeenCalledWith(expect.stringContaining('init-unity-lefthook.js'), true);
});
it('should skip when package not found in cache', async () => {
const { OrchestratorSystem } = require('../core/orchestrator-system');
(mockFs.existsSync as jest.Mock).mockReturnValue(false);
await GitHooksService.initUnityGitHooks('/repo');
expect(OrchestratorSystem.Run).not.toHaveBeenCalled();
});
it('should warn when init script does not exist', async () => {
const OrchestratorLogger = require('../core/orchestrator-logger').default;
(mockFs.existsSync as jest.Mock).mockImplementation((p: string) => {
// PackageCache dir exists, but init script doesn't
return !String(p).includes('init-unity-lefthook');
});
(mockFs.readdirSync as jest.Mock).mockReturnValue(['com.frostebite.unitygithooks@1.0.5']);
await GitHooksService.initUnityGitHooks('/repo');
expect(OrchestratorLogger.logWarning).toHaveBeenCalledWith(expect.stringContaining('init script not found'));
});
it('should log warning on init failure', async () => {
const { OrchestratorSystem } = require('../core/orchestrator-system');
const OrchestratorLogger = require('../core/orchestrator-logger').default;
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.readdirSync as jest.Mock).mockReturnValue(['com.frostebite.unitygithooks@1.0.5']);
OrchestratorSystem.Run.mockRejectedValue(new Error('node not found'));
await GitHooksService.initUnityGitHooks('/repo');
expect(OrchestratorLogger.logWarning).toHaveBeenCalledWith(expect.stringContaining('init failed'));
});
});
describe('configureUnityGitHooksCIEnv', () => {
it('should disable background project mode', () => {
const env = GitHooksService.configureUnityGitHooksCIEnv();
expect(env.UNITY_GITHOOKS_BACKGROUND_PROJECT_ENABLED).toBe('false');
});
it('should set CI=true', () => {
const env = GitHooksService.configureUnityGitHooksCIEnv();
expect(env.CI).toBe('true');
});
});
describe('installHooks', () => {
it('should run npx lefthook install when lefthook is detected', async () => {
(mockFs.existsSync as jest.Mock).mockImplementation((filePath: string) => {
return String(filePath).includes('lefthook.yml') && !String(filePath).startsWith('.');
});
const { OrchestratorSystem } = require('../core/orchestrator-system');
await GitHooksService.installHooks('/repo');
expect(OrchestratorSystem.Run).toHaveBeenCalledWith(`cd "/repo" && npx lefthook install`, true);
});
it('should init Unity Git Hooks before installing lefthook when detected', async () => {
const { OrchestratorSystem } = require('../core/orchestrator-system');
const OrchestratorLogger = require('../core/orchestrator-logger').default;
const callOrder: string[] = [];
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.readFileSync as jest.Mock).mockReturnValue(
`{"dependencies":{"com.frostebite.unitygithooks":"https://github.com/frostebite/UnityGitHooks.git"}}`,
);
(mockFs.readdirSync as jest.Mock).mockReturnValue(['com.frostebite.unitygithooks@1.0.5']);
OrchestratorSystem.Run.mockImplementation((cmd: string) => {
if (cmd.includes('init-unity-lefthook')) {
callOrder.push('init');
} else if (cmd.includes('lefthook install')) {
callOrder.push('install');
}
return Promise.resolve('');
});
await GitHooksService.installHooks('/repo');
// Init should happen before install
expect(callOrder).toEqual(['init', 'install']);
expect(OrchestratorLogger.log).toHaveBeenCalledWith(expect.stringContaining('Unity Git Hooks (UPM) detected'));
});
it('should set CI env vars when Unity Git Hooks detected', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
(mockFs.readFileSync as jest.Mock).mockReturnValue(`{"dependencies":{"com.frostebite.unitygithooks":"1.0.5"}}`);
(mockFs.readdirSync as jest.Mock).mockReturnValue(['com.frostebite.unitygithooks@1.0.5']);
await GitHooksService.installHooks('/repo');
expect(process.env.UNITY_GITHOOKS_BACKGROUND_PROJECT_ENABLED).toBe('false');
});
it('should run npx husky install when husky is detected', async () => {
(mockFs.existsSync as jest.Mock).mockImplementation((filePath: string) => {
return String(filePath).endsWith('.husky');
});
const { OrchestratorSystem } = require('../core/orchestrator-system');
await GitHooksService.installHooks('/repo');
expect(OrchestratorSystem.Run).toHaveBeenCalledWith(`cd "/repo" && npx husky install`, true);
});
it('should log and return when no framework is detected', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(false);
const { OrchestratorSystem } = require('../core/orchestrator-system');
const OrchestratorLogger = require('../core/orchestrator-logger').default;
await GitHooksService.installHooks('/repo');
expect(OrchestratorSystem.Run).not.toHaveBeenCalled();
expect(OrchestratorLogger.log).toHaveBeenCalledWith(expect.stringContaining('No hook framework detected'));
});
it('should log warning on installation failure', async () => {
(mockFs.existsSync as jest.Mock).mockImplementation((filePath: string) => {
return String(filePath).includes('lefthook.yml') && !String(filePath).startsWith('.');
});
const { OrchestratorSystem } = require('../core/orchestrator-system');
const OrchestratorLogger = require('../core/orchestrator-logger').default;
OrchestratorSystem.Run.mockRejectedValue(new Error('npx not found'));
await GitHooksService.installHooks('/repo');
expect(OrchestratorLogger.logWarning).toHaveBeenCalledWith(expect.stringContaining('Hook installation failed'));
});
});
describe('runHookGroups', () => {
it('should run each group via lefthook run', async () => {
const { OrchestratorSystem } = require('../core/orchestrator-system');
(mockFs.existsSync as jest.Mock).mockImplementation((filePath: string) => {
return String(filePath).includes('lefthook.yml') && !String(filePath).startsWith('.');
});
OrchestratorSystem.Run.mockResolvedValue('');
const results = await GitHooksService.runHookGroups('/repo', ['pre-commit', 'pre-push']);
expect(OrchestratorSystem.Run).toHaveBeenCalledWith(`cd "/repo" && npx lefthook run pre-commit`, true);
expect(OrchestratorSystem.Run).toHaveBeenCalledWith(`cd "/repo" && npx lefthook run pre-push`, true);
expect(results['pre-commit']).toBe(true);
expect(results['pre-push']).toBe(true);
});
it('should return empty results for empty groups', async () => {
const results = await GitHooksService.runHookGroups('/repo', []);
expect(results).toEqual({});
});
it('should warn and return empty if not using lefthook', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(false);
const OrchestratorLogger = require('../core/orchestrator-logger').default;
const results = await GitHooksService.runHookGroups('/repo', ['pre-commit']);
expect(results).toEqual({});
expect(OrchestratorLogger.logWarning).toHaveBeenCalledWith(expect.stringContaining('requires lefthook'));
});
it('should mark failed groups as false', async () => {
const { OrchestratorSystem } = require('../core/orchestrator-system');
(mockFs.existsSync as jest.Mock).mockImplementation((filePath: string) => {
return String(filePath).includes('lefthook.yml') && !String(filePath).startsWith('.');
});
OrchestratorSystem.Run.mockResolvedValueOnce('') // pre-commit passes
.mockRejectedValueOnce(new Error('tests failed')); // pre-push fails
const results = await GitHooksService.runHookGroups('/repo', ['pre-commit', 'pre-push']);
expect(results['pre-commit']).toBe(true);
expect(results['pre-push']).toBe(false);
});
it('should log each group result', async () => {
const { OrchestratorSystem } = require('../core/orchestrator-system');
const OrchestratorLogger = require('../core/orchestrator-logger').default;
(mockFs.existsSync as jest.Mock).mockImplementation((filePath: string) => {
return String(filePath).includes('lefthook.yml') && !String(filePath).startsWith('.');
});
OrchestratorSystem.Run.mockResolvedValueOnce('').mockRejectedValueOnce(new Error('check failed'));
await GitHooksService.runHookGroups('/repo', ['pre-commit', 'commit-msg']);
expect(OrchestratorLogger.log).toHaveBeenCalledWith(expect.stringContaining("'pre-commit' passed"));
expect(OrchestratorLogger.logWarning).toHaveBeenCalledWith(expect.stringContaining("'commit-msg' failed"));
});
});
describe('configureSkipList', () => {
it('should return empty object for empty skip list', () => {
const result = GitHooksService.configureSkipList([]);
expect(result).toEqual({});
});
it('should return LEFTHOOK_EXCLUDE with comma-separated hooks', () => {
const result = GitHooksService.configureSkipList(['pre-commit', 'pre-push']);
expect(result.LEFTHOOK_EXCLUDE).toBe('pre-commit,pre-push');
});
it('should set HUSKY=0 when hooks are skipped', () => {
const result = GitHooksService.configureSkipList(['pre-commit']);
expect(result.HUSKY).toBe('0');
});
it('should handle single hook in skip list', () => {
const result = GitHooksService.configureSkipList(['commit-msg']);
expect(result.LEFTHOOK_EXCLUDE).toBe('commit-msg');
expect(result.HUSKY).toBe('0');
});
});
describe('disableHooks', () => {
it('should set core.hooksPath to an empty directory', async () => {
(mockFs.mkdirSync as jest.Mock).mockReturnValue(undefined);
const { OrchestratorSystem } = require('../core/orchestrator-system');
await GitHooksService.disableHooks('/repo');
expect(OrchestratorSystem.Run).toHaveBeenCalledWith(
expect.stringContaining('git -C "/repo" config core.hooksPath'),
true,
);
});
it('should log warning on failure to disable hooks', async () => {
(mockFs.mkdirSync as jest.Mock).mockReturnValue(undefined);
const { OrchestratorSystem } = require('../core/orchestrator-system');
const OrchestratorLogger = require('../core/orchestrator-logger').default;
OrchestratorSystem.Run.mockRejectedValue(new Error('git config failed'));
await GitHooksService.disableHooks('/repo');
expect(OrchestratorLogger.logWarning).toHaveBeenCalledWith(expect.stringContaining('Failed to disable hooks'));
});
});
describe('constants', () => {
it('should have correct package name', () => {
expect(GitHooksService.UNITY_GIT_HOOKS_PACKAGE).toBe('com.frostebite.unitygithooks');
});
});
});

View File

@@ -0,0 +1,235 @@
import fs from 'node:fs';
import path from 'node:path';
import os from 'node:os';
import { OrchestratorSystem } from '../core/orchestrator-system';
import OrchestratorLogger from '../core/orchestrator-logger';
export class GitHooksService {
static readonly UNITY_GIT_HOOKS_PACKAGE = 'com.frostebite.unitygithooks';
/**
* Detect which git hook framework is configured in the repository.
* Checks for lefthook and husky configuration files.
*/
static detectHookFramework(repoPath: string): 'lefthook' | 'husky' | 'none' {
// Check for lefthook config files
if (fs.existsSync(path.join(repoPath, 'lefthook.yml')) || fs.existsSync(path.join(repoPath, '.lefthook.yml'))) {
return 'lefthook';
}
// Check for husky directory
if (fs.existsSync(path.join(repoPath, '.husky'))) {
return 'husky';
}
return 'none';
}
/**
* Detect if Unity Git Hooks (com.frostebite.unitygithooks) is installed as a UPM package.
* Checks Packages/manifest.json for the package dependency.
*/
static detectUnityGitHooks(repoPath: string): boolean {
const manifestPath = path.join(repoPath, 'Packages', 'manifest.json');
if (!fs.existsSync(manifestPath)) {
return false;
}
try {
const content = fs.readFileSync(manifestPath, 'utf8');
return content.includes(GitHooksService.UNITY_GIT_HOOKS_PACKAGE);
} catch {
return false;
}
}
/**
* Find the Unity Git Hooks package directory in the Library/PackageCache.
* Returns the path to the package directory, or empty string if not found.
*/
static findUnityGitHooksPackagePath(repoPath: string): string {
const packageCacheDir = path.join(repoPath, 'Library', 'PackageCache');
if (!fs.existsSync(packageCacheDir)) {
return '';
}
try {
const entries = fs.readdirSync(packageCacheDir);
const match = entries.find((entry) => entry.startsWith(GitHooksService.UNITY_GIT_HOOKS_PACKAGE));
if (match) {
return path.join(packageCacheDir, match);
}
} catch {
// PackageCache not available
}
return '';
}
/**
* Initialize Unity Git Hooks by running its init script.
* This installs the required npm modules that the hooks depend on.
* Should be called before installHooks() when Unity Git Hooks is detected.
*/
static async initUnityGitHooks(repoPath: string): Promise<void> {
const packagePath = GitHooksService.findUnityGitHooksPackagePath(repoPath);
if (!packagePath) {
OrchestratorLogger.log(`[GitHooks] Unity Git Hooks package not found in Library/PackageCache, skipping init`);
return;
}
const initScript = path.join(packagePath, '~js', 'init-unity-lefthook.js');
if (!fs.existsSync(initScript)) {
OrchestratorLogger.logWarning(`[GitHooks] Unity Git Hooks init script not found at ${initScript}`);
return;
}
OrchestratorLogger.log(`[GitHooks] Initializing Unity Git Hooks from ${packagePath}`);
try {
await OrchestratorSystem.Run(`cd "${repoPath}" && node "${initScript}"`, true);
OrchestratorLogger.log(`[GitHooks] Unity Git Hooks initialized successfully`);
} catch (error: any) {
OrchestratorLogger.logWarning(`[GitHooks] Unity Git Hooks init failed: ${error.message}`);
}
}
/**
* Configure CI-friendly environment variables for Unity Git Hooks.
* Disables background project mode (CI already has an isolated workspace)
* and sets other env vars appropriate for headless CI environments.
*/
static configureUnityGitHooksCIEnv(): Record<string, string> {
return {
UNITY_GITHOOKS_BACKGROUND_PROJECT_ENABLED: 'false',
CI: 'true',
};
}
/**
* Install git hooks using the detected framework.
* If Unity Git Hooks is detected, initializes it first.
* Errors are caught and logged as warnings - hook installation should not fail the build.
*/
static async installHooks(repoPath: string): Promise<void> {
const framework = GitHooksService.detectHookFramework(repoPath);
if (framework === 'none') {
OrchestratorLogger.log(`[GitHooks] No hook framework detected in ${repoPath}`);
return;
}
OrchestratorLogger.log(`[GitHooks] Detected hook framework: ${framework}`);
// If Unity Git Hooks is present, initialize it before installing hooks
if (framework === 'lefthook' && GitHooksService.detectUnityGitHooks(repoPath)) {
OrchestratorLogger.log(`[GitHooks] Unity Git Hooks (UPM) detected, running init`);
// Set CI-friendly env vars
const ciEnv = GitHooksService.configureUnityGitHooksCIEnv();
for (const [key, value] of Object.entries(ciEnv)) {
process.env[key] = value;
}
await GitHooksService.initUnityGitHooks(repoPath);
}
try {
if (framework === 'lefthook') {
await OrchestratorSystem.Run(`cd "${repoPath}" && npx lefthook install`, true);
OrchestratorLogger.log(`[GitHooks] Lefthook hooks installed`);
} else if (framework === 'husky') {
await OrchestratorSystem.Run(`cd "${repoPath}" && npx husky install`, true);
OrchestratorLogger.log(`[GitHooks] Husky hooks installed`);
}
} catch (error: any) {
OrchestratorLogger.logWarning(`[GitHooks] Hook installation failed: ${error.message}`);
}
}
/**
* Explicitly run specific lefthook hook groups before the build.
* This allows CI to trigger pre-commit, pre-push, or other checks
* that would normally only run on git events.
*
* @param repoPath - Path to the repository
* @param hookGroups - Lefthook group names to run (e.g., ['pre-commit', 'pre-push'])
* @returns Map of group name to success/failure
*/
static async runHookGroups(repoPath: string, hookGroups: string[]): Promise<Record<string, boolean>> {
const results: Record<string, boolean> = {};
if (hookGroups.length === 0) {
return results;
}
const framework = GitHooksService.detectHookFramework(repoPath);
if (framework !== 'lefthook') {
OrchestratorLogger.logWarning(`[GitHooks] runHookGroups requires lefthook, but detected: ${framework}`);
return results;
}
OrchestratorLogger.log(`[GitHooks] Running ${hookGroups.length} hook group(s): ${hookGroups.join(', ')}`);
for (const group of hookGroups) {
try {
await OrchestratorSystem.Run(`cd "${repoPath}" && npx lefthook run ${group}`, true);
OrchestratorLogger.log(`[GitHooks] Hook group '${group}' passed`);
results[group] = true;
} catch (error: any) {
OrchestratorLogger.logWarning(`[GitHooks] Hook group '${group}' failed: ${error.message}`);
results[group] = false;
}
}
return results;
}
/**
* Return environment variables that will skip the listed hooks.
* For lefthook: sets LEFTHOOK_EXCLUDE to a comma-separated list.
* For husky: sets HUSKY=0 to disable all hooks (husky does not support selective skipping).
* The caller is responsible for applying the returned env vars.
*/
static configureSkipList(skipList: string[]): Record<string, string> {
if (skipList.length === 0) {
return {};
}
// Return both lefthook and husky env vars so the caller can apply whichever is relevant.
// Lefthook supports selective hook exclusion.
const env: Record<string, string> = {
LEFTHOOK_EXCLUDE: skipList.join(','),
};
// Husky only supports full disable (HUSKY=0), not selective skipping.
// If any hooks are in the skip list, disable husky entirely.
env.HUSKY = '0';
OrchestratorLogger.log(`[GitHooks] Skip list configured: ${skipList.join(', ')}`);
return env;
}
/**
* Disable all git hooks by pointing core.hooksPath to an empty temporary directory.
* This prevents any hooks from running during the build.
*/
static async disableHooks(repoPath: string): Promise<void> {
try {
const emptyDir = path.join(os.tmpdir(), 'game-ci-empty-hooks');
fs.mkdirSync(emptyDir, { recursive: true });
await OrchestratorSystem.Run(`git -C "${repoPath}" config core.hooksPath "${emptyDir}"`, true);
OrchestratorLogger.log(`[GitHooks] Hooks disabled via core.hooksPath -> ${emptyDir}`);
} catch (error: any) {
OrchestratorLogger.logWarning(`[GitHooks] Failed to disable hooks: ${error.message}`);
}
}
}

View File

@@ -0,0 +1,422 @@
import { MiddlewareService } from './middleware-service';
import { Middleware } from './middleware';
// Mock dependencies
jest.mock('../../orchestrator', () => ({
__esModule: true,
default: {
buildParameters: {
providerStrategy: 'aws',
targetPlatform: 'StandaloneLinux64',
},
},
}));
jest.mock('../../options/orchestrator-options', () => ({
__esModule: true,
default: {
providerStrategy: 'aws',
middlewareFiles: ['code-signing', 'cache-optimizer'],
},
}));
jest.mock('../core/orchestrator-logger', () => ({
__esModule: true,
default: {
log: jest.fn(),
logWarning: jest.fn(),
},
}));
describe('MiddlewareService', () => {
const originalEnv = process.env;
beforeEach(() => {
process.env = { ...originalEnv };
});
afterEach(() => {
process.env = originalEnv;
});
describe('parseMiddleware', () => {
it('should parse a single middleware definition', () => {
const yaml = `
name: test-middleware
type: command
priority: 50
trigger:
phase: [build]
before:
commands: echo "before"
after:
commands: echo "after"
`;
const result = MiddlewareService.parseMiddleware(yaml);
expect(result).toHaveLength(1);
expect(result[0].name).toBe('test-middleware');
expect(result[0].type).toBe('command');
expect(result[0].priority).toBe(50);
expect(result[0].trigger.phase).toEqual(['build']);
expect(result[0].before?.commands).toBe('echo "before"');
expect(result[0].after?.commands).toBe('echo "after"');
});
it('should parse an array of middleware definitions', () => {
const yaml = `
- name: first
type: command
trigger:
phase: [setup]
before:
commands: echo "first"
- name: second
type: container
image: node:20
trigger:
phase: [build]
after:
commands: echo "second"
`;
const result = MiddlewareService.parseMiddleware(yaml);
expect(result).toHaveLength(2);
expect(result[0].name).toBe('first');
expect(result[0].type).toBe('command');
expect(result[1].name).toBe('second');
expect(result[1].type).toBe('container');
expect(result[1].image).toBe('node:20');
});
it('should handle string shorthand for before/after', () => {
const yaml = `
name: shorthand
type: command
trigger:
phase: [build]
before: echo "shorthand before"
after: echo "shorthand after"
`;
const result = MiddlewareService.parseMiddleware(yaml);
expect(result).toHaveLength(1);
expect(result[0].before?.commands).toBe('echo "shorthand before"');
expect(result[0].after?.commands).toBe('echo "shorthand after"');
});
it('should parse secrets', () => {
const yaml = `
name: with-secrets
type: container
trigger:
phase: [build]
secrets:
- name: MY_SECRET
value: secret-value
- name: ANOTHER_SECRET
after:
commands: echo "done"
`;
const result = MiddlewareService.parseMiddleware(yaml);
expect(result[0].secrets).toHaveLength(2);
expect(result[0].secrets[0].ParameterKey).toBe('MY_SECRET');
expect(result[0].secrets[0].ParameterValue).toBe('secret-value');
expect(result[0].secrets[1].ParameterKey).toBe('ANOTHER_SECRET');
});
it('should default priority to 100', () => {
const yaml = `
name: no-priority
type: command
trigger:
phase: [build]
before: echo "test"
`;
const result = MiddlewareService.parseMiddleware(yaml);
expect(result[0].priority).toBe(100);
});
it('should default type to command', () => {
const yaml = `
name: no-type
trigger:
phase: [build]
before: echo "test"
`;
const result = MiddlewareService.parseMiddleware(yaml);
expect(result[0].type).toBe('command');
});
it('should return empty array for empty input', () => {
expect(MiddlewareService.parseMiddleware('')).toEqual([]);
expect(MiddlewareService.parseMiddleware(' ')).toEqual([]);
});
it('should normalize scalar trigger values to arrays', () => {
const yaml = `
name: scalar-triggers
type: command
trigger:
phase: build
provider: aws
platform: StandaloneLinux64
before: echo "test"
`;
const result = MiddlewareService.parseMiddleware(yaml);
expect(result[0].trigger.phase).toEqual(['build']);
expect(result[0].trigger.provider).toEqual(['aws']);
expect(result[0].trigger.platform).toEqual(['StandaloneLinux64']);
});
it('should handle per-phase image override for container type', () => {
const yaml = `
name: multi-image
type: container
image: ubuntu
trigger:
phase: [build]
before:
image: node:20
commands: npm install
after:
image: python:3
commands: python verify.py
`;
const result = MiddlewareService.parseMiddleware(yaml);
expect(result[0].image).toBe('ubuntu');
expect(result[0].before?.image).toBe('node:20');
expect(result[0].after?.image).toBe('python:3');
});
});
describe('evaluateExpression', () => {
it('should evaluate equality expressions', () => {
process.env.TEST_VAR = 'hello';
expect(MiddlewareService.evaluateExpression("env.TEST_VAR == 'hello'")).toBe(true);
expect(MiddlewareService.evaluateExpression("env.TEST_VAR == 'world'")).toBe(false);
});
it('should evaluate inequality expressions', () => {
process.env.TEST_VAR = 'hello';
expect(MiddlewareService.evaluateExpression("env.TEST_VAR != 'world'")).toBe(true);
expect(MiddlewareService.evaluateExpression("env.TEST_VAR != 'hello'")).toBe(false);
});
it('should evaluate truthy expressions', () => {
process.env.DEFINED_VAR = 'yes';
expect(MiddlewareService.evaluateExpression('env.DEFINED_VAR')).toBe(true);
process.env.FALSE_VAR = 'false';
expect(MiddlewareService.evaluateExpression('env.FALSE_VAR')).toBe(false);
process.env.EMPTY_VAR = '';
expect(MiddlewareService.evaluateExpression('env.EMPTY_VAR')).toBe(false);
delete process.env.MISSING_VAR;
expect(MiddlewareService.evaluateExpression('env.MISSING_VAR')).toBe(false);
});
it('should evaluate falsy expressions', () => {
delete process.env.MISSING_VAR;
expect(MiddlewareService.evaluateExpression('!env.MISSING_VAR')).toBe(true);
process.env.DEFINED_VAR = 'yes';
expect(MiddlewareService.evaluateExpression('!env.DEFINED_VAR')).toBe(false);
});
it('should default to true for unknown expressions', () => {
expect(MiddlewareService.evaluateExpression('some unknown expression')).toBe(true);
});
it('should handle double-quoted values', () => {
process.env.TEST_VAR = 'hello';
expect(MiddlewareService.evaluateExpression('env.TEST_VAR == "hello"')).toBe(true);
});
});
describe('evaluateTrigger', () => {
it('should match when phase matches', () => {
const trigger = { phase: ['build'] };
expect(MiddlewareService.evaluateTrigger(trigger, 'build')).toBe(true);
expect(MiddlewareService.evaluateTrigger(trigger, 'setup')).toBe(false);
});
it('should match multiple phases', () => {
const trigger = { phase: ['setup', 'build'] };
expect(MiddlewareService.evaluateTrigger(trigger, 'setup')).toBe(true);
expect(MiddlewareService.evaluateTrigger(trigger, 'build')).toBe(true);
expect(MiddlewareService.evaluateTrigger(trigger, 'pre-build')).toBe(false);
});
it('should filter by provider', () => {
const trigger = { phase: ['build'], provider: ['k8s'] };
// Mock provider is 'aws', so this should not match
expect(MiddlewareService.evaluateTrigger(trigger, 'build')).toBe(false);
const matchingTrigger = { phase: ['build'], provider: ['aws', 'k8s'] };
expect(MiddlewareService.evaluateTrigger(matchingTrigger, 'build')).toBe(true);
});
it('should filter by platform', () => {
const trigger = { phase: ['build'], platform: ['StandaloneWindows64'] };
// Mock platform is 'StandaloneLinux64', so this should not match
expect(MiddlewareService.evaluateTrigger(trigger, 'build')).toBe(false);
const matchingTrigger = { phase: ['build'], platform: ['StandaloneLinux64'] };
expect(MiddlewareService.evaluateTrigger(matchingTrigger, 'build')).toBe(true);
});
it('should evaluate when expressions', () => {
process.env.FEATURE_FLAG = 'true';
const trigger = { phase: ['build'], when: "env.FEATURE_FLAG == 'true'" };
expect(MiddlewareService.evaluateTrigger(trigger, 'build')).toBe(true);
process.env.FEATURE_FLAG = 'false';
expect(MiddlewareService.evaluateTrigger(trigger, 'build')).toBe(false);
});
it('should require all conditions to pass (AND logic)', () => {
process.env.FEATURE_FLAG = 'true';
const trigger = {
phase: ['build'],
provider: ['k8s'], // won't match (mock provider is 'aws')
when: "env.FEATURE_FLAG == 'true'", // would match
};
expect(MiddlewareService.evaluateTrigger(trigger, 'build')).toBe(false);
});
});
describe('resolveCommandHooks', () => {
const middleware: Middleware[] = [
{
name: 'low-priority',
type: 'command',
priority: 10,
trigger: { phase: ['build'] },
image: 'ubuntu',
before: { commands: 'echo "low-before"' },
after: { commands: 'echo "low-after"' },
secrets: [],
allowFailure: false,
},
{
name: 'high-priority',
type: 'command',
priority: 90,
trigger: { phase: ['build'] },
image: 'ubuntu',
before: { commands: 'echo "high-before"' },
after: { commands: 'echo "high-after"' },
secrets: [],
allowFailure: false,
},
{
name: 'container-type',
type: 'container',
priority: 50,
trigger: { phase: ['build'] },
image: 'node:20',
before: { commands: 'echo "container"' },
secrets: [],
allowFailure: false,
},
];
it('should only return command type middleware', () => {
const hooks = MiddlewareService.resolveCommandHooks(middleware, 'build', 'before');
expect(hooks).toHaveLength(2);
expect(hooks.every((h) => h.name.startsWith('middleware:'))).toBe(true);
expect(hooks.find((h) => h.name.includes('container-type'))).toBeUndefined();
});
it('should order before hooks by ascending priority', () => {
const hooks = MiddlewareService.resolveCommandHooks(middleware, 'build', 'before');
expect(hooks[0].name).toBe('middleware:low-priority:before');
expect(hooks[1].name).toBe('middleware:high-priority:before');
});
it('should order after hooks by descending priority (wrapping)', () => {
const hooks = MiddlewareService.resolveCommandHooks(middleware, 'build', 'after');
expect(hooks[0].name).toBe('middleware:high-priority:after');
expect(hooks[1].name).toBe('middleware:low-priority:after');
});
it('should filter by phase', () => {
const hooks = MiddlewareService.resolveCommandHooks(middleware, 'setup', 'before');
expect(hooks).toHaveLength(0);
});
});
describe('resolveContainerHooks', () => {
const middleware: Middleware[] = [
{
name: 'container-a',
type: 'container',
priority: 20,
trigger: { phase: ['pre-build'] },
image: 'ubuntu',
before: { commands: 'echo "a-before"' },
after: { commands: 'echo "a-after"' },
secrets: [],
allowFailure: false,
},
{
name: 'container-b',
type: 'container',
priority: 80,
trigger: { phase: ['pre-build'] },
image: 'node:20',
before: { commands: 'echo "b-before"', image: 'python:3' },
secrets: [],
allowFailure: true,
},
];
it('should resolve container hooks with correct images', () => {
const hooks = MiddlewareService.resolveContainerHooks(middleware, 'pre-build', 'before');
expect(hooks).toHaveLength(2);
expect(hooks[0].image).toBe('ubuntu'); // default image from middleware
expect(hooks[1].image).toBe('python:3'); // per-phase override
});
it('should propagate allowFailure', () => {
const hooks = MiddlewareService.resolveContainerHooks(middleware, 'pre-build', 'before');
expect(hooks[0].allowFailure).toBe(false);
expect(hooks[1].allowFailure).toBe(true);
});
it('should only include phases that have definitions', () => {
// container-b has no after phase
const hooks = MiddlewareService.resolveContainerHooks(middleware, 'pre-build', 'after');
expect(hooks).toHaveLength(1);
expect(hooks[0].name).toBe('middleware:container-a:after');
});
});
describe('getMiddleware', () => {
it('should sort by priority ascending', () => {
const yaml = `
- name: high
type: command
priority: 200
trigger:
phase: [build]
before: echo "high"
- name: low
type: command
priority: 5
trigger:
phase: [build]
before: echo "low"
- name: medium
type: command
priority: 50
trigger:
phase: [build]
before: echo "medium"
`;
const result = MiddlewareService.getMiddleware(yaml);
expect(result[0].name).toBe('low');
expect(result[1].name).toBe('medium');
expect(result[2].name).toBe('high');
});
});
});

View File

@@ -0,0 +1,329 @@
import YAML from 'yaml';
import fs from 'node:fs';
import path from 'node:path';
import { Middleware, MiddlewareTrigger } from './middleware';
import { ContainerHook } from './container-hook';
import { CommandHook } from './command-hook';
import OrchestratorOptions from '../../options/orchestrator-options';
import Orchestrator from '../../orchestrator';
import OrchestratorLogger from '../core/orchestrator-logger';
import Input from '../../../input';
/**
* Service for loading, evaluating, and resolving middleware into hooks.
*
* Middleware is a higher-level composable abstraction over the existing
* command hook and container hook systems. Each middleware:
* - Wraps around pipeline phases with before/after semantics
* - Has trigger conditions (phase, provider, platform, expression)
* - Resolves to either CommandHooks or ContainerHooks at execution time
* - Executes in priority order (before: ascending, after: descending)
*/
export class MiddlewareService {
/**
* Load all active middleware from inline YAML + file-based definitions.
* Returns them sorted by priority (ascending).
*/
static getMiddleware(inlineYaml: string): Middleware[] {
const middleware: Middleware[] = [];
// Parse inline YAML definitions
if (inlineYaml && inlineYaml !== '') {
middleware.push(...MiddlewareService.parseMiddleware(inlineYaml));
}
// Load file-based definitions from game-ci/middleware/
middleware.push(...MiddlewareService.getMiddlewareFromFiles());
// Sort by priority (lower = earlier)
middleware.sort((a, b) => (a.priority ?? 100) - (b.priority ?? 100));
OrchestratorLogger.log(`Middleware: loaded ${middleware.length} definition(s)`);
return middleware;
}
/**
* Resolve middleware to CommandHooks for a given phase and timing.
* Filters by trigger conditions and converts to hooks.
*
* Before hooks: ascending priority (lowest priority runs first, closest to core phase).
* After hooks: descending priority (highest priority runs first, closest to core phase).
* This produces the wrapping pattern: outermost middleware's before runs first and after runs last.
*/
static resolveCommandHooks(middleware: Middleware[], phase: string, timing: 'before' | 'after'): CommandHook[] {
const applicable = middleware
.filter((m) => m.type === 'command')
.filter((m) => MiddlewareService.evaluateTrigger(m.trigger, phase));
// before: ascending priority; after: descending (wrapping order)
if (timing === 'after') {
applicable.reverse();
}
const hooks = applicable
.filter((m) => (timing === 'before' ? m.before : m.after))
.map((m) => {
const mPhase = timing === 'before' ? m.before! : m.after!;
const hook = new CommandHook();
hook.name = `middleware:${m.name}:${timing}`;
hook.commands = typeof mPhase.commands === 'string' ? [mPhase.commands] : [mPhase.commands];
hook.hook = [timing];
hook.step = [phase];
hook.secrets = m.secrets || [];
return hook;
});
if (hooks.length > 0) {
OrchestratorLogger.log(
`Middleware: resolved ${hooks.length} command hook(s) for ${phase}:${timing}${hooks
.map((h) => h.name)
.join(', ')}`,
);
}
return hooks;
}
/**
* Resolve middleware to ContainerHooks for a given phase and timing.
* Same ordering logic as resolveCommandHooks.
*/
static resolveContainerHooks(middleware: Middleware[], phase: string, timing: 'before' | 'after'): ContainerHook[] {
const applicable = middleware
.filter((m) => m.type === 'container')
.filter((m) => MiddlewareService.evaluateTrigger(m.trigger, phase));
// before: ascending priority; after: descending (wrapping order)
if (timing === 'after') {
applicable.reverse();
}
const hooks = applicable
.filter((m) => (timing === 'before' ? m.before : m.after))
.map((m) => {
const mPhase = timing === 'before' ? m.before! : m.after!;
const hook = new ContainerHook();
hook.name = `middleware:${m.name}:${timing}`;
hook.commands = typeof mPhase.commands === 'string' ? mPhase.commands : mPhase.commands;
hook.image = mPhase.image || m.image || 'ubuntu';
hook.hook = timing === 'before' ? 'before' : 'after';
hook.secrets = m.secrets || [];
hook.allowFailure = m.allowFailure ?? false;
return hook;
});
if (hooks.length > 0) {
OrchestratorLogger.log(
`Middleware: resolved ${hooks.length} container hook(s) for ${phase}:${timing}${hooks
.map((h) => h.name)
.join(', ')}`,
);
}
return hooks;
}
/**
* Evaluate whether a middleware's trigger conditions are met.
* All specified conditions must pass (AND logic).
*/
static evaluateTrigger(trigger: MiddlewareTrigger, currentPhase: string): boolean {
// Phase must match
if (!trigger.phase || !trigger.phase.includes(currentPhase)) {
return false;
}
// Provider filter (if specified)
if (trigger.provider && trigger.provider.length > 0) {
const currentProvider = Orchestrator.buildParameters?.providerStrategy || OrchestratorOptions.providerStrategy;
if (!trigger.provider.includes(currentProvider)) {
return false;
}
}
// Platform filter (if specified)
if (trigger.platform && trigger.platform.length > 0) {
const currentPlatform = Orchestrator.buildParameters?.targetPlatform || '';
if (!trigger.platform.includes(currentPlatform)) {
return false;
}
}
// Expression-based condition
if (trigger.when) {
if (!MiddlewareService.evaluateExpression(trigger.when)) {
return false;
}
}
return true;
}
/**
* Evaluate a simple expression string against environment variables.
*
* Supported formats:
* - env.VAR_NAME == 'value' — equality check
* - env.VAR_NAME != 'value' — inequality check
* - env.VAR_NAME — truthy check (defined, non-empty, not 'false')
* - !env.VAR_NAME — falsy check
*/
static evaluateExpression(expression: string): boolean {
const trimmed = expression.trim();
// Match: env.VAR == 'value' or env.VAR != 'value'
const comparisonMatch = trimmed.match(/^env\.(\w+)\s*(==|!=)\s*['"](.*)['"]$/);
if (comparisonMatch) {
const [, varName, operator, value] = comparisonMatch;
const envValue = process.env[varName] || '';
return operator === '==' ? envValue === value : envValue !== value;
}
// Match: !env.VAR (falsy check)
const falsyMatch = trimmed.match(/^!env\.(\w+)$/);
if (falsyMatch) {
const [, varName] = falsyMatch;
const envValue = process.env[varName];
return envValue === undefined || envValue === '' || envValue === 'false';
}
// Match: env.VAR (truthy check)
const truthyMatch = trimmed.match(/^env\.(\w+)$/);
if (truthyMatch) {
const [, varName] = truthyMatch;
const envValue = process.env[varName];
return envValue !== undefined && envValue !== '' && envValue !== 'false';
}
// Unknown expression format — log warning, default to true
OrchestratorLogger.logWarning(`Middleware: unknown expression format "${expression}", defaulting to true`);
return true;
}
/**
* Parse middleware definitions from a YAML string.
* Accepts both single-object and array format.
*/
static parseMiddleware(yamlString: string): Middleware[] {
if (!yamlString || yamlString.trim() === '') {
return [];
}
try {
const isArray = yamlString.replace(/\s/g, '')[0] === '-';
const parsed = isArray ? YAML.parse(yamlString) : [YAML.parse(yamlString)];
if (!parsed || !Array.isArray(parsed)) {
return [];
}
return parsed.map((m: any) => MiddlewareService.hydrateMiddleware(m));
} catch (error: any) {
OrchestratorLogger.logWarning(`Middleware: failed to parse YAML — ${error.message}`);
return [];
}
}
/**
* Hydrate a raw parsed YAML object into a Middleware instance.
*/
private static hydrateMiddleware(m: any): Middleware {
const middleware = new Middleware();
middleware.name = m.name || 'unnamed';
middleware.description = m.description;
middleware.type = m.type || 'command';
middleware.priority = m.priority ?? 100;
middleware.image = m.image || 'ubuntu';
middleware.allowFailure = m.allowFailure ?? false;
middleware.outputs = m.outputs;
// Parse trigger — normalize scalar values to arrays
middleware.trigger = {
phase: MiddlewareService.toStringArray(m.trigger?.phase),
provider: m.trigger?.provider ? MiddlewareService.toStringArray(m.trigger.provider) : undefined,
platform: m.trigger?.platform ? MiddlewareService.toStringArray(m.trigger.platform) : undefined,
when: m.trigger?.when,
};
// Parse before/after phases — accept string shorthand or object format
if (m.before) {
middleware.before = {
commands: typeof m.before === 'string' ? m.before : m.before.commands || '',
image: typeof m.before === 'string' ? undefined : m.before.image,
};
}
if (m.after) {
middleware.after = {
commands: typeof m.after === 'string' ? m.after : m.after.commands || '',
image: typeof m.after === 'string' ? undefined : m.after.image,
};
}
// Parse secrets
if (m.secrets && Array.isArray(m.secrets)) {
middleware.secrets = m.secrets.map((s: any) => ({
ParameterKey: s.name,
EnvironmentVariable: Input.ToEnvVarFormat(s.name),
ParameterValue: s.value ?? process.env[s.name] ?? process.env[Input.ToEnvVarFormat(s.name)] ?? '',
}));
}
return middleware;
}
/**
* Load middleware definitions from game-ci/middleware/ directory files.
* Only files whose base name appears in the middlewareFiles allowlist are loaded.
*/
static getMiddlewareFromFiles(): Middleware[] {
const results: Middleware[] = [];
const allowedFiles = OrchestratorOptions.middlewareFiles;
if (!allowedFiles || allowedFiles.length === 0) {
return results;
}
try {
const middlewarePath = path.join(process.cwd(), 'game-ci', 'middleware');
if (!fs.existsSync(middlewarePath)) {
return results;
}
const files = fs.readdirSync(middlewarePath);
for (const file of files) {
const baseName = file.replace(/\.ya?ml$/, '');
if (!allowedFiles.includes(baseName)) {
continue;
}
try {
const contents = fs.readFileSync(path.join(middlewarePath, file), 'utf8');
results.push(...MiddlewareService.parseMiddleware(contents));
} catch (error: any) {
OrchestratorLogger.logWarning(`Middleware: failed to parse file ${file}${error.message}`);
}
}
} catch {
// Directory doesn't exist or can't be read — not an error
}
return results;
}
/**
* Normalize a value to a string array. Accepts string, string[], or undefined.
*/
private static toStringArray(value: any): string[] {
if (!value) return [];
if (Array.isArray(value)) return value;
return [value];
}
}

View File

@@ -0,0 +1,85 @@
import OrchestratorSecret from '../../options/orchestrator-secret';
/**
* Trigger conditions that determine when a middleware activates.
* All specified conditions must be true (AND logic).
*/
export interface MiddlewareTrigger {
/** Pipeline phases this middleware applies to: 'setup', 'build', 'pre-build', 'post-build' */
phase: string[];
/** Restrict to specific providers. If omitted, applies to all providers. */
provider?: string[];
/** Restrict to specific build target platforms. If omitted, applies to all platforms. */
platform?: string[];
/** Expression-based condition. Supports: env.VAR == 'value', env.VAR != 'value', env.VAR (truthy) */
when?: string;
}
/**
* A single phase (before or after) of a middleware definition.
*/
export interface MiddlewarePhase {
/** Shell commands to execute */
commands: string;
/** Override image for this phase (container type only) */
image?: string;
}
/**
* Middleware — a composable, trigger-aware pipeline unit built on hooks.
*
* Middleware wraps around pipeline phases with before/after semantics.
* Each middleware resolves to either CommandHooks (inline in build container)
* or ContainerHooks (separate Docker containers) at execution time.
*
* Execution order: before phases run in ascending priority order,
* after phases run in descending priority order (wrapping pattern).
*
* Example YAML:
* ```yaml
* name: code-signing
* description: Signs build artifacts after successful build
* type: container
* priority: 50
* image: ubuntu:22.04
* trigger:
* phase: [build]
* provider: [aws, k8s]
* platform: [StandaloneWindows64]
* when: "env.SIGN_BUILDS == 'true'"
* before:
* commands: |
* echo "Preparing signing environment..."
* after:
* commands: |
* echo "Signing build artifacts..."
* secrets:
* - name: SIGNING_KEY
* - name: SIGNING_CERT
* allowFailure: false
* ```
*/
export class Middleware {
/** Unique name identifying this middleware */
public name!: string;
/** Human-readable description */
public description?: string;
/** Hook type: 'command' (inline in build container) or 'container' (separate Docker container) */
public type!: 'command' | 'container';
/** Execution priority. Lower values run first for before, last for after (wrapping order). Default: 100 */
public priority: number = 100;
/** Conditions that determine when this middleware activates */
public trigger!: MiddlewareTrigger;
/** Default Docker image for container type middleware */
public image: string = 'ubuntu';
/** Commands to run before the target phase */
public before?: MiddlewarePhase;
/** Commands to run after the target phase */
public after?: MiddlewarePhase;
/** Secrets injected as environment variables */
public secrets: OrchestratorSecret[] = [];
/** If true, failures warn but don't stop the build. Default: false */
public allowFailure: boolean = false;
/** Environment variable names this middleware exports to downstream middleware */
public outputs?: string[];
}

View File

@@ -0,0 +1,395 @@
import fs from 'node:fs';
import os from 'node:os';
import { ElasticGitStorageService } from './elastic-git-storage-service';
jest.mock('node:fs');
jest.mock('node:os');
jest.mock('../core/orchestrator-system', () => ({
OrchestratorSystem: {
Run: jest.fn().mockResolvedValue(''),
},
}));
jest.mock('../core/orchestrator-logger', () => ({
__esModule: true,
default: {
log: jest.fn(),
logWarning: jest.fn(),
error: jest.fn(),
},
}));
jest.mock('./lfs-agent-service', () => ({
LfsAgentService: {
configure: jest.fn().mockResolvedValue(undefined),
},
}));
const mockFs = fs as jest.Mocked<typeof fs>;
const mockOs = os as jest.Mocked<typeof os>;
describe('ElasticGitStorageService', () => {
beforeEach(() => {
jest.clearAllMocks();
});
describe('parseAgentValue', () => {
it('should parse name without version', () => {
const result = ElasticGitStorageService.parseAgentValue('elastic-git-storage');
expect(result.name).toBe('elastic-git-storage');
expect(result.version).toBe('latest');
});
it('should parse name@version', () => {
const result = ElasticGitStorageService.parseAgentValue('elastic-git-storage@v1.0.0');
expect(result.name).toBe('elastic-git-storage');
expect(result.version).toBe('v1.0.0');
});
it('should parse name@latest', () => {
const result = ElasticGitStorageService.parseAgentValue('elastic-git-storage@latest');
expect(result.name).toBe('elastic-git-storage');
expect(result.version).toBe('latest');
});
it('should handle trailing @ as latest', () => {
const result = ElasticGitStorageService.parseAgentValue('elastic-git-storage@');
expect(result.name).toBe('elastic-git-storage');
expect(result.version).toBe('latest');
});
it('should handle whitespace', () => {
const result = ElasticGitStorageService.parseAgentValue(' elastic-git-storage@v2.0.0 ');
expect(result.name).toBe('elastic-git-storage');
expect(result.version).toBe('v2.0.0');
});
});
describe('isElasticGitStorage', () => {
it('should match exact name', () => {
expect(ElasticGitStorageService.isElasticGitStorage('elastic-git-storage')).toBe(true);
});
it('should match with .exe extension', () => {
expect(ElasticGitStorageService.isElasticGitStorage('elastic-git-storage.exe')).toBe(true);
});
it('should match with @version suffix', () => {
expect(ElasticGitStorageService.isElasticGitStorage('elastic-git-storage@v1.0.0')).toBe(true);
});
it('should match with @latest suffix', () => {
expect(ElasticGitStorageService.isElasticGitStorage('elastic-git-storage@latest')).toBe(true);
});
it('should match forward-slash path', () => {
expect(ElasticGitStorageService.isElasticGitStorage('/usr/local/bin/elastic-git-storage')).toBe(true);
});
it('should match backslash path', () => {
expect(ElasticGitStorageService.isElasticGitStorage('C:\\tools\\elastic-git-storage')).toBe(true);
});
it('should match path with .exe', () => {
expect(ElasticGitStorageService.isElasticGitStorage('C:\\tools\\elastic-git-storage.exe')).toBe(true);
});
it('should be case-insensitive', () => {
expect(ElasticGitStorageService.isElasticGitStorage('Elastic-Git-Storage')).toBe(true);
});
it('should handle whitespace', () => {
expect(ElasticGitStorageService.isElasticGitStorage(' elastic-git-storage ')).toBe(true);
});
it('should not match other agent names', () => {
expect(ElasticGitStorageService.isElasticGitStorage('lfs-folderstore')).toBe(false);
});
it('should return false for empty string', () => {
expect(ElasticGitStorageService.isElasticGitStorage('')).toBe(false);
});
it('should not match partial names', () => {
expect(ElasticGitStorageService.isElasticGitStorage('my-elastic-git-storage-v2')).toBe(false);
});
});
describe('findInstalled', () => {
it('should find on PATH via which/where', async () => {
const { OrchestratorSystem } = require('../core/orchestrator-system');
mockOs.platform.mockReturnValue('linux');
OrchestratorSystem.Run.mockResolvedValue('/usr/local/bin/elastic-git-storage\n');
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
const result = await ElasticGitStorageService.findInstalled();
expect(result).toBe('/usr/local/bin/elastic-git-storage');
});
it('should use where on windows', async () => {
const { OrchestratorSystem } = require('../core/orchestrator-system');
mockOs.platform.mockReturnValue('win32');
OrchestratorSystem.Run.mockResolvedValue('C:\\tools\\elastic-git-storage.exe\n');
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
const result = await ElasticGitStorageService.findInstalled();
expect(result).toBe('C:\\tools\\elastic-git-storage.exe');
expect(OrchestratorSystem.Run).toHaveBeenCalledWith(expect.stringContaining('where'), false, true);
});
it('should check common install locations when not on PATH', async () => {
const { OrchestratorSystem } = require('../core/orchestrator-system');
mockOs.platform.mockReturnValue('linux');
mockOs.homedir.mockReturnValue('/home/runner');
OrchestratorSystem.Run.mockRejectedValue(new Error('not found'));
(mockFs.existsSync as jest.Mock)
.mockReturnValueOnce(false) // RUNNER_TOOL_CACHE
.mockReturnValueOnce(true); // /usr/local/bin
const result = await ElasticGitStorageService.findInstalled();
expect(result).toBe('/usr/local/bin/elastic-git-storage');
});
it('should return empty string when not found anywhere', async () => {
const { OrchestratorSystem } = require('../core/orchestrator-system');
mockOs.platform.mockReturnValue('linux');
mockOs.homedir.mockReturnValue('/home/runner');
OrchestratorSystem.Run.mockRejectedValue(new Error('not found'));
(mockFs.existsSync as jest.Mock).mockReturnValue(false);
const result = await ElasticGitStorageService.findInstalled();
expect(result).toBe('');
});
it('should check windows-specific locations on win32', async () => {
const { OrchestratorSystem } = require('../core/orchestrator-system');
mockOs.platform.mockReturnValue('win32');
mockOs.homedir.mockReturnValue('C:\\Users\\runner');
OrchestratorSystem.Run.mockRejectedValue(new Error('not found'));
const originalEnv = { ...process.env };
process.env.LOCALAPPDATA = 'C:\\Users\\runner\\AppData\\Local';
process.env.RUNNER_TOOL_CACHE = '';
(mockFs.existsSync as jest.Mock).mockImplementation((p: string) => {
return p.includes('AppData');
});
const result = await ElasticGitStorageService.findInstalled();
expect(result).toContain('elastic-git-storage.exe');
process.env = originalEnv;
});
});
describe('install', () => {
it('should download correct binary for linux amd64', async () => {
const { OrchestratorSystem } = require('../core/orchestrator-system');
mockOs.platform.mockReturnValue('linux');
mockOs.arch.mockReturnValue('x64');
mockOs.tmpdir.mockReturnValue('/tmp');
OrchestratorSystem.Run.mockResolvedValue('');
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
const result = await ElasticGitStorageService.install('latest');
expect(OrchestratorSystem.Run).toHaveBeenCalledWith(expect.stringContaining('elastic-git-storage_linux_amd64'));
expect(OrchestratorSystem.Run).toHaveBeenCalledWith(expect.stringContaining('chmod +x'));
expect(result).toContain('elastic-git-storage');
});
it('should download correct binary for darwin arm64', async () => {
const { OrchestratorSystem } = require('../core/orchestrator-system');
mockOs.platform.mockReturnValue('darwin');
mockOs.arch.mockReturnValue('arm64');
mockOs.tmpdir.mockReturnValue('/tmp');
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
await ElasticGitStorageService.install('v1.2.0');
expect(OrchestratorSystem.Run).toHaveBeenCalledWith(expect.stringContaining('elastic-git-storage_darwin_arm64'));
expect(OrchestratorSystem.Run).toHaveBeenCalledWith(expect.stringContaining('v1.2.0'));
});
it('should download .exe for windows', async () => {
const { OrchestratorSystem } = require('../core/orchestrator-system');
mockOs.platform.mockReturnValue('win32');
mockOs.arch.mockReturnValue('x64');
mockOs.tmpdir.mockReturnValue('C:\\temp');
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
await ElasticGitStorageService.install('latest');
expect(OrchestratorSystem.Run).toHaveBeenCalledWith(
expect.stringContaining('elastic-git-storage_windows_amd64.exe'),
);
// Should NOT chmod on windows
expect(OrchestratorSystem.Run).not.toHaveBeenCalledWith(expect.stringContaining('chmod'));
});
it('should use RUNNER_TOOL_CACHE for install dir when available', async () => {
const { OrchestratorSystem } = require('../core/orchestrator-system');
mockOs.platform.mockReturnValue('linux');
mockOs.arch.mockReturnValue('x64');
OrchestratorSystem.Run.mockResolvedValue('');
const originalCache = process.env.RUNNER_TOOL_CACHE;
process.env.RUNNER_TOOL_CACHE = '/opt/hostedtoolcache';
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
const result = await ElasticGitStorageService.install('latest');
expect(result).toContain('hostedtoolcache');
if (originalCache === undefined) {
delete process.env.RUNNER_TOOL_CACHE;
} else {
process.env.RUNNER_TOOL_CACHE = originalCache;
}
});
it('should use latest release URL when version is latest', async () => {
const { OrchestratorSystem } = require('../core/orchestrator-system');
mockOs.platform.mockReturnValue('linux');
mockOs.arch.mockReturnValue('x64');
mockOs.tmpdir.mockReturnValue('/tmp');
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
await ElasticGitStorageService.install('latest');
expect(OrchestratorSystem.Run).toHaveBeenCalledWith(expect.stringContaining('/releases/latest/download/'));
});
it('should use tagged release URL when version is specified', async () => {
const { OrchestratorSystem } = require('../core/orchestrator-system');
mockOs.platform.mockReturnValue('linux');
mockOs.arch.mockReturnValue('x64');
mockOs.tmpdir.mockReturnValue('/tmp');
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
await ElasticGitStorageService.install('v2.0.0');
expect(OrchestratorSystem.Run).toHaveBeenCalledWith(expect.stringContaining('/releases/download/v2.0.0/'));
});
it('should return empty string on download failure', async () => {
const { OrchestratorSystem } = require('../core/orchestrator-system');
mockOs.platform.mockReturnValue('linux');
mockOs.arch.mockReturnValue('x64');
mockOs.tmpdir.mockReturnValue('/tmp');
OrchestratorSystem.Run.mockRejectedValue(new Error('curl failed'));
const result = await ElasticGitStorageService.install('latest');
expect(result).toBe('');
});
it('should return empty string if binary not found after download', async () => {
const { OrchestratorSystem } = require('../core/orchestrator-system');
mockOs.platform.mockReturnValue('linux');
mockOs.arch.mockReturnValue('x64');
mockOs.tmpdir.mockReturnValue('/tmp');
(mockFs.existsSync as jest.Mock).mockReturnValue(false);
const result = await ElasticGitStorageService.install('latest');
expect(result).toBe('');
});
});
describe('ensureAndConfigure', () => {
it('should use existing installation if found', async () => {
const { LfsAgentService } = require('./lfs-agent-service');
const { OrchestratorSystem } = require('../core/orchestrator-system');
mockOs.platform.mockReturnValue('linux');
mockOs.homedir.mockReturnValue('/home/runner');
// findInstalled returns a result
OrchestratorSystem.Run.mockResolvedValue('/usr/local/bin/elastic-git-storage\n');
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
const result = await ElasticGitStorageService.ensureAndConfigure('latest', '--verbose', ['/mnt/lfs'], '/repo');
expect(result).toBe('/usr/local/bin/elastic-git-storage');
expect(LfsAgentService.configure).toHaveBeenCalledWith(
'/usr/local/bin/elastic-git-storage',
'--verbose',
['/mnt/lfs'],
'/repo',
);
});
it('should install when not found and configure', async () => {
const { LfsAgentService } = require('./lfs-agent-service');
const { OrchestratorSystem } = require('../core/orchestrator-system');
mockOs.platform.mockReturnValue('linux');
mockOs.arch.mockReturnValue('x64');
mockOs.homedir.mockReturnValue('/home/runner');
mockOs.tmpdir.mockReturnValue('/tmp');
// findInstalled finds nothing
OrchestratorSystem.Run.mockRejectedValueOnce(new Error('not found')) // which
.mockResolvedValueOnce('') // curl download
.mockResolvedValueOnce(''); // chmod
(mockFs.existsSync as jest.Mock)
.mockReturnValueOnce(false) // RUNNER_TOOL_CACHE
.mockReturnValueOnce(false) // /usr/local/bin
.mockReturnValueOnce(false) // ~/.local/bin
.mockReturnValueOnce(true); // after install
const result = await ElasticGitStorageService.ensureAndConfigure('v1.0.0', '', [], '/repo');
expect(result).toContain('elastic-git-storage');
expect(LfsAgentService.configure).toHaveBeenCalled();
});
it('should return empty string when install fails', async () => {
const { OrchestratorSystem } = require('../core/orchestrator-system');
mockOs.platform.mockReturnValue('linux');
mockOs.arch.mockReturnValue('x64');
mockOs.homedir.mockReturnValue('/home/runner');
mockOs.tmpdir.mockReturnValue('/tmp');
// findInstalled finds nothing
OrchestratorSystem.Run.mockRejectedValue(new Error('not found'));
(mockFs.existsSync as jest.Mock).mockReturnValue(false);
const result = await ElasticGitStorageService.ensureAndConfigure('latest', '', [], '/repo');
expect(result).toBe('');
});
it('should use default version when empty string passed', async () => {
const { OrchestratorSystem } = require('../core/orchestrator-system');
mockOs.platform.mockReturnValue('linux');
mockOs.arch.mockReturnValue('x64');
mockOs.homedir.mockReturnValue('/home/runner');
mockOs.tmpdir.mockReturnValue('/tmp');
// findInstalled finds nothing
OrchestratorSystem.Run.mockRejectedValueOnce(new Error('not found'))
.mockResolvedValueOnce('')
.mockResolvedValueOnce('');
(mockFs.existsSync as jest.Mock)
.mockReturnValueOnce(false)
.mockReturnValueOnce(false)
.mockReturnValueOnce(false)
.mockReturnValueOnce(true);
await ElasticGitStorageService.ensureAndConfigure('', '', [], '/repo');
expect(OrchestratorSystem.Run).toHaveBeenCalledWith(expect.stringContaining('/releases/latest/download/'));
});
});
describe('constants', () => {
it('should have correct repo owner', () => {
expect(ElasticGitStorageService.REPO_OWNER).toBe('frostebite');
});
it('should have correct repo name', () => {
expect(ElasticGitStorageService.REPO_NAME).toBe('elastic-git-storage');
});
it('should have correct agent name', () => {
expect(ElasticGitStorageService.AGENT_NAME).toBe('elastic-git-storage');
});
});
});

View File

@@ -0,0 +1,211 @@
import fs from 'node:fs';
import path from 'node:path';
import os from 'node:os';
import { OrchestratorSystem } from '../core/orchestrator-system';
import OrchestratorLogger from '../core/orchestrator-logger';
import { LfsAgentService } from './lfs-agent-service';
/**
* Built-in support for elastic-git-storage, a custom Git LFS transfer agent
* that supports multiple storage backends (local filesystem, WebDAV, rclone remotes).
*
* When `lfsTransferAgent` is set to 'elastic-git-storage', this service:
* 1. Checks if the agent is already installed on the system
* 2. If not, downloads it from the GitHub release page
* 3. Makes it executable
* 4. Delegates to LfsAgentService for git config setup
*
* @see https://github.com/frostebite/elastic-git-storage
*/
export class ElasticGitStorageService {
static readonly REPO_OWNER = 'frostebite';
static readonly REPO_NAME = 'elastic-git-storage';
static readonly AGENT_NAME = 'elastic-git-storage';
/**
* Parse an agent value that may include a version suffix.
* Supports formats like:
* - 'elastic-git-storage' → { name: 'elastic-git-storage', version: 'latest' }
* - 'elastic-git-storage@v1.0.0' → { name: 'elastic-git-storage', version: 'v1.0.0' }
* - 'elastic-git-storage@latest' → { name: 'elastic-git-storage', version: 'latest' }
*/
static parseAgentValue(agentValue: string): { name: string; version: string } {
const trimmed = agentValue.trim();
const atIndex = trimmed.indexOf('@');
if (atIndex > 0) {
return {
name: trimmed.substring(0, atIndex),
version: trimmed.substring(atIndex + 1) || 'latest',
};
}
return { name: trimmed, version: 'latest' };
}
/**
* Check if the given lfsTransferAgent value refers to elastic-git-storage.
* Matches the exact name (without path) or a path ending in the agent name.
* Also matches 'elastic-git-storage@version' format.
*/
static isElasticGitStorage(agentValue: string): boolean {
if (!agentValue) return false;
const { name } = ElasticGitStorageService.parseAgentValue(agentValue);
const normalized = name.trim().toLowerCase();
return (
normalized === 'elastic-git-storage' ||
normalized === 'elastic-git-storage.exe' ||
normalized.endsWith('/elastic-git-storage') ||
normalized.endsWith('\\elastic-git-storage') ||
normalized.endsWith('/elastic-git-storage.exe') ||
normalized.endsWith('\\elastic-git-storage.exe')
);
}
/**
* Resolve the full path to the elastic-git-storage executable.
*
* Search order:
* 1. The provided path (if it's a full path and exists)
* 2. $PATH lookup via `which` / `where`
* 3. Known install locations ($RUNNER_TOOL_CACHE, /usr/local/bin, ~/.local/bin)
*
* @returns Full path to the executable, or empty string if not found
*/
static async findInstalled(): Promise<string> {
// Check PATH
try {
const whichCmd = os.platform() === 'win32' ? 'where elastic-git-storage' : 'which elastic-git-storage';
const result = await OrchestratorSystem.Run(whichCmd, false, true);
const foundPath = result.trim().split('\n')[0].trim();
if (foundPath && fs.existsSync(foundPath)) {
return foundPath;
}
} catch {
// Not on PATH
}
// Check common install locations
const candidates = [
path.join(process.env.RUNNER_TOOL_CACHE || '', 'elastic-git-storage', 'elastic-git-storage'),
'/usr/local/bin/elastic-git-storage',
path.join(os.homedir(), '.local', 'bin', 'elastic-git-storage'),
];
if (os.platform() === 'win32') {
candidates.push(
path.join(process.env.RUNNER_TOOL_CACHE || '', 'elastic-git-storage', 'elastic-git-storage.exe'),
path.join(process.env.LOCALAPPDATA || '', 'elastic-git-storage', 'elastic-git-storage.exe'),
);
}
for (const candidate of candidates) {
if (candidate && fs.existsSync(candidate)) {
return candidate;
}
}
return '';
}
/**
* Install elastic-git-storage from GitHub releases.
*
* @param version - Version to install (e.g., 'v1.0.0', 'latest')
* @returns Path to the installed executable
*/
static async install(version: string = 'latest'): Promise<string> {
const platform = os.platform();
const arch = os.arch();
const osName = platform === 'win32' ? 'windows' : platform === 'darwin' ? 'darwin' : 'linux';
const archName = arch === 'x64' ? 'amd64' : arch === 'arm64' ? 'arm64' : 'amd64';
const ext = platform === 'win32' ? '.exe' : '';
const installDir = process.env.RUNNER_TOOL_CACHE
? path.join(process.env.RUNNER_TOOL_CACHE, 'elastic-git-storage')
: path.join(os.tmpdir(), 'elastic-git-storage');
const binaryName = `elastic-git-storage${ext}`;
const installPath = path.join(installDir, binaryName);
OrchestratorLogger.log(`[ElasticGitStorage] Installing to ${installPath}`);
// Create install directory
fs.mkdirSync(installDir, { recursive: true });
// Build download URL
const releaseTag = version === 'latest' ? 'latest' : version;
const assetName = `elastic-git-storage_${osName}_${archName}${ext}`;
let downloadUrl: string;
if (releaseTag === 'latest') {
downloadUrl = `https://github.com/${ElasticGitStorageService.REPO_OWNER}/${ElasticGitStorageService.REPO_NAME}/releases/latest/download/${assetName}`;
} else {
downloadUrl = `https://github.com/${ElasticGitStorageService.REPO_OWNER}/${ElasticGitStorageService.REPO_NAME}/releases/download/${releaseTag}/${assetName}`;
}
OrchestratorLogger.log(`[ElasticGitStorage] Downloading from ${downloadUrl}`);
try {
await OrchestratorSystem.Run(`curl -fsSL -o "${installPath}" "${downloadUrl}"`);
if (platform !== 'win32') {
await OrchestratorSystem.Run(`chmod +x "${installPath}"`);
}
// Verify installation
if (!fs.existsSync(installPath)) {
throw new Error(`Binary not found after download at ${installPath}`);
}
OrchestratorLogger.log(`[ElasticGitStorage] Successfully installed ${releaseTag} to ${installPath}`);
return installPath;
} catch (error: any) {
OrchestratorLogger.logWarning(
`[ElasticGitStorage] Failed to install: ${error.message}. Continuing without elastic-git-storage.`,
);
return '';
}
}
/**
* Ensure elastic-git-storage is available (find or install) and configure it.
*
* @param version - Version to install if not found ('latest' or a tag like 'v1.0.0')
* @param agentArgs - Additional arguments to pass to the agent
* @param storagePaths - Storage paths for the agent
* @param repoPath - Path to the git repository
* @returns Path to the configured executable, or empty string if setup failed
*/
static async ensureAndConfigure(
version: string,
agentArgs: string,
storagePaths: string[],
repoPath: string,
): Promise<string> {
OrchestratorLogger.log(`[ElasticGitStorage] Setting up elastic-git-storage (version: ${version || 'latest'})`);
// Try to find existing installation
let agentPath = await ElasticGitStorageService.findInstalled();
if (agentPath) {
OrchestratorLogger.log(`[ElasticGitStorage] Found existing installation at ${agentPath}`);
} else {
OrchestratorLogger.log(`[ElasticGitStorage] Not found on system, installing...`);
agentPath = await ElasticGitStorageService.install(version || 'latest');
if (!agentPath) {
return '';
}
}
// Delegate to LfsAgentService for git config setup
await LfsAgentService.configure(agentPath, agentArgs, storagePaths, repoPath);
return agentPath;
}
}

View File

@@ -0,0 +1,122 @@
import fs from 'node:fs';
import path from 'node:path';
import { LfsAgentService } from './lfs-agent-service';
// Mock dependencies
jest.mock('node:fs');
jest.mock('../core/orchestrator-system', () => ({
OrchestratorSystem: {
Run: jest.fn().mockResolvedValue(''),
},
}));
jest.mock('../core/orchestrator-logger', () => ({
__esModule: true,
default: {
log: jest.fn(),
logWarning: jest.fn(),
error: jest.fn(),
},
}));
const mockFs = fs as jest.Mocked<typeof fs>;
describe('LfsAgentService', () => {
beforeEach(() => {
jest.clearAllMocks();
});
describe('configure', () => {
it('should call correct git config commands when agent exists', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
const { OrchestratorSystem } = require('../core/orchestrator-system');
await LfsAgentService.configure(
'/usr/local/bin/elastic-git-storage',
'--verbose',
['/storage/path1', '/storage/path2'],
'/repo',
);
expect(OrchestratorSystem.Run).toHaveBeenCalledWith(
`git -C "/repo" config lfs.customtransfer.elastic-git-storage.path "/usr/local/bin/elastic-git-storage"`,
);
expect(OrchestratorSystem.Run).toHaveBeenCalledWith(
`git -C "/repo" config lfs.customtransfer.elastic-git-storage.args "--verbose"`,
);
expect(OrchestratorSystem.Run).toHaveBeenCalledWith(
`git -C "/repo" config lfs.standalonetransferagent elastic-git-storage`,
);
});
it('should set LFS_STORAGE_PATHS environment variable when storagePaths provided', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
await LfsAgentService.configure('/usr/local/bin/elastic-git-storage', '', ['/path/a', '/path/b'], '/repo');
expect(process.env.LFS_STORAGE_PATHS).toBe('/path/a;/path/b');
});
it('should log warning and return early when agent executable does not exist', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(false);
const { OrchestratorSystem } = require('../core/orchestrator-system');
await LfsAgentService.configure('/nonexistent/agent', '', [], '/repo');
expect(OrchestratorSystem.Run).not.toHaveBeenCalled();
});
it('should derive agent name from executable filename', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
const { OrchestratorSystem } = require('../core/orchestrator-system');
await LfsAgentService.configure('/tools/my-custom-agent.exe', '', [], '/repo');
expect(OrchestratorSystem.Run).toHaveBeenCalledWith(
`git -C "/repo" config lfs.customtransfer.my-custom-agent.path "/tools/my-custom-agent.exe"`,
);
});
});
describe('configure with empty storagePaths', () => {
it('should not set LFS_STORAGE_PATHS when storagePaths is empty', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
const originalValue = process.env.LFS_STORAGE_PATHS;
delete process.env.LFS_STORAGE_PATHS;
await LfsAgentService.configure('/usr/local/bin/agent', '', [], '/repo');
expect(process.env.LFS_STORAGE_PATHS).toBeUndefined();
if (originalValue !== undefined) {
process.env.LFS_STORAGE_PATHS = originalValue;
}
});
});
describe('validate', () => {
it('should return true when agent executable exists', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(true);
const result = await LfsAgentService.validate('/usr/local/bin/elastic-git-storage');
expect(result).toBe(true);
});
it('should return false when agent executable does not exist', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(false);
const result = await LfsAgentService.validate('/nonexistent/agent');
expect(result).toBe(false);
});
it('should log warning when agent does not exist', async () => {
(mockFs.existsSync as jest.Mock).mockReturnValue(false);
const OrchestratorLogger = require('../core/orchestrator-logger').default;
await LfsAgentService.validate('/nonexistent/agent');
expect(OrchestratorLogger.logWarning).toHaveBeenCalledWith(expect.stringContaining('Agent executable not found'));
});
});
});

View File

@@ -0,0 +1,59 @@
import fs from 'node:fs';
import path from 'node:path';
import { OrchestratorSystem } from '../core/orchestrator-system';
import OrchestratorLogger from '../core/orchestrator-logger';
export class LfsAgentService {
/**
* Configure a custom LFS transfer agent in a git repository.
* Sets up the git config entries and environment variables needed for the agent.
*/
static async configure(
agentPath: string,
agentArgs: string,
storagePaths: string[],
repoPath: string,
): Promise<void> {
// Validate the agent executable exists
if (!fs.existsSync(agentPath)) {
OrchestratorLogger.logWarning(
`[LfsAgent] Agent executable not found at ${agentPath}, continuing without custom LFS agent`,
);
return;
}
// Derive agent name from executable filename (without extension)
const agentName = path.basename(agentPath, path.extname(agentPath));
OrchestratorLogger.log(`[LfsAgent] Configuring custom LFS transfer agent: ${agentName}`);
OrchestratorLogger.log(`[LfsAgent] Path: ${agentPath}`);
OrchestratorLogger.log(`[LfsAgent] Args: ${agentArgs}`);
// Set git config entries for the custom transfer agent
await OrchestratorSystem.Run(`git -C "${repoPath}" config lfs.customtransfer.${agentName}.path "${agentPath}"`);
await OrchestratorSystem.Run(`git -C "${repoPath}" config lfs.customtransfer.${agentName}.args "${agentArgs}"`);
await OrchestratorSystem.Run(`git -C "${repoPath}" config lfs.standalonetransferagent ${agentName}`);
// Set storage paths environment variable if provided
if (storagePaths.length > 0) {
const storagePathsValue = storagePaths.join(';');
process.env.LFS_STORAGE_PATHS = storagePathsValue;
OrchestratorLogger.log(`[LfsAgent] Storage paths: ${storagePathsValue}`);
}
OrchestratorLogger.log(`[LfsAgent] Custom LFS transfer agent configured successfully`);
}
/**
* Validate that the LFS transfer agent executable exists.
*/
static async validate(agentPath: string): Promise<boolean> {
const exists = fs.existsSync(agentPath);
if (!exists) {
OrchestratorLogger.logWarning(`[LfsAgent] Agent executable not found: ${agentPath}`);
}
return exists;
}
}

View File

@@ -0,0 +1,385 @@
import fs from 'node:fs';
import { SubmoduleProfileService } from './submodule-profile-service';
import { OrchestratorSystem } from '../core/orchestrator-system';
jest.mock('node:fs');
jest.mock('../core/orchestrator-system');
jest.mock('../core/orchestrator-logger', () => ({
__esModule: true,
default: {
log: jest.fn(),
logWarning: jest.fn(),
error: jest.fn(),
},
}));
const mockedFs = fs as jest.Mocked<typeof fs>;
const mockedSystem = OrchestratorSystem as jest.Mocked<typeof OrchestratorSystem>;
describe('SubmoduleProfileService', () => {
beforeEach(() => {
jest.clearAllMocks();
});
describe('parseProfile', () => {
it('reads and parses a valid YAML profile', () => {
const profileYaml = `
primary_submodule: Assets/_Game/Submodules/TurnOfWarEndlessCrusade
product_name: Endless Crusade
submodules:
- name: TurnOfWar
branch: main
- name: TurnOfWarEndlessCrusade
branch: main
- name: AreaOfOperations
branch: empty
`;
mockedFs.existsSync.mockReturnValue(true);
mockedFs.readFileSync.mockReturnValue(profileYaml);
const profile = SubmoduleProfileService.parseProfile('/path/to/profile.yml');
expect(profile.primary_submodule).toBe('Assets/_Game/Submodules/TurnOfWarEndlessCrusade');
expect(profile.product_name).toBe('Endless Crusade');
expect(profile.submodules).toHaveLength(3);
expect(profile.submodules[0]).toEqual({ name: 'TurnOfWar', branch: 'main' });
expect(profile.submodules[1]).toEqual({ name: 'TurnOfWarEndlessCrusade', branch: 'main' });
expect(profile.submodules[2]).toEqual({ name: 'AreaOfOperations', branch: 'empty' });
});
it('throws if profile file does not exist', () => {
mockedFs.existsSync.mockReturnValue(false);
expect(() => SubmoduleProfileService.parseProfile('/missing/profile.yml')).toThrow('Submodule profile not found');
});
it('throws if YAML is missing submodules array', () => {
mockedFs.existsSync.mockReturnValue(true);
mockedFs.readFileSync.mockReturnValue('product_name: Test\n');
expect(() => SubmoduleProfileService.parseProfile('/path/to/bad.yml')).toThrow("expected 'submodules' array");
});
});
describe('mergeVariant', () => {
it('correctly overlays variant entries on base profile', () => {
const baseYaml = `
submodules:
- name: ModuleA
branch: main
- name: ModuleB
branch: main
`;
const variantYaml = `
product_name: Server Build
submodules:
- name: ModuleB
branch: empty
- name: ModuleC
branch: develop
`;
mockedFs.existsSync.mockReturnValue(true);
mockedFs.readFileSync.mockImplementation((filePath: any) => {
if (filePath === '/base.yml') return baseYaml;
if (filePath === '/variant.yml') return variantYaml;
return '';
});
const base = SubmoduleProfileService.parseProfile('/base.yml');
const merged = SubmoduleProfileService.mergeVariant(base, '/variant.yml');
expect(merged.product_name).toBe('Server Build');
expect(merged.submodules).toHaveLength(3);
const moduleA = merged.submodules.find((s) => s.name === 'ModuleA');
const moduleB = merged.submodules.find((s) => s.name === 'ModuleB');
const moduleC = merged.submodules.find((s) => s.name === 'ModuleC');
expect(moduleA?.branch).toBe('main');
expect(moduleB?.branch).toBe('empty');
expect(moduleC?.branch).toBe('develop');
});
});
describe('matchSubmodule', () => {
it('matches exact submodule name', () => {
expect(SubmoduleProfileService.matchSubmodule('TurnOfWar', 'TurnOfWar')).toBe(true);
});
it('matches exact leaf folder name against full path', () => {
expect(SubmoduleProfileService.matchSubmodule('Assets/_Game/Submodules/TurnOfWar', 'TurnOfWar')).toBe(true);
});
it('does not match unrelated names', () => {
expect(SubmoduleProfileService.matchSubmodule('TurnOfWar', 'AreaOfOperations')).toBe(false);
});
it('matches trailing wildcard against full path', () => {
expect(SubmoduleProfileService.matchSubmodule('Assets/_Engine/Submodules/PluginsFoo', 'Plugins*')).toBe(true);
});
it('matches trailing wildcard against full path prefix', () => {
expect(
SubmoduleProfileService.matchSubmodule(
'Assets/_Engine/Submodules/PluginsFoo',
'Assets/_Engine/Submodules/Plugins*',
),
).toBe(true);
});
it('does not match wildcard that does not align', () => {
expect(SubmoduleProfileService.matchSubmodule('Assets/_Engine/Submodules/SensorToolkit', 'Plugins*')).toBe(false);
});
});
describe('parseGitmodules', () => {
it('parses a typical .gitmodules file', () => {
const gitmodulesContent = `[submodule "Assets/_Game/Submodules/TurnOfWar"]
\tpath = Assets/_Game/Submodules/TurnOfWar
\turl = https://github.com/org/TurnOfWar.git
[submodule "Assets/_Game/Submodules/EndlessCrusade"]
\tpath = Assets/_Game/Submodules/EndlessCrusade
\turl = https://github.com/org/EndlessCrusade.git
[submodule "Assets/_Engine/Submodules/SensorToolkit"]
\tpath = Assets/_Engine/Submodules/SensorToolkit
\turl = https://github.com/org/SensorToolkit.git
`;
mockedFs.existsSync.mockReturnValue(true);
mockedFs.readFileSync.mockReturnValue(gitmodulesContent);
const result = SubmoduleProfileService.parseGitmodules('/repo');
expect(result.size).toBe(3);
expect(result.get('Assets/_Game/Submodules/TurnOfWar')).toBe('Assets/_Game/Submodules/TurnOfWar');
expect(result.get('Assets/_Game/Submodules/EndlessCrusade')).toBe('Assets/_Game/Submodules/EndlessCrusade');
expect(result.get('Assets/_Engine/Submodules/SensorToolkit')).toBe('Assets/_Engine/Submodules/SensorToolkit');
});
it('returns empty map when .gitmodules does not exist', () => {
mockedFs.existsSync.mockReturnValue(false);
const result = SubmoduleProfileService.parseGitmodules('/repo');
expect(result.size).toBe(0);
});
});
describe('createInitPlan', () => {
it('generates correct init and skip actions', async () => {
const profileYaml = `
submodules:
- name: TurnOfWar
branch: main
- name: EndlessCrusade
branch: main
- name: SensorToolkit
branch: empty
`;
const gitmodulesContent = `[submodule "Assets/_Game/Submodules/TurnOfWar"]
\tpath = Assets/_Game/Submodules/TurnOfWar
\turl = https://github.com/org/TurnOfWar.git
[submodule "Assets/_Game/Submodules/EndlessCrusade"]
\tpath = Assets/_Game/Submodules/EndlessCrusade
\turl = https://github.com/org/EndlessCrusade.git
[submodule "Assets/_Engine/Submodules/SensorToolkit"]
\tpath = Assets/_Engine/Submodules/SensorToolkit
\turl = https://github.com/org/SensorToolkit.git
[submodule "Assets/_Game/Submodules/Unmatched"]
\tpath = Assets/_Game/Submodules/Unmatched
\turl = https://github.com/org/Unmatched.git
`;
mockedFs.existsSync.mockReturnValue(true);
mockedFs.readFileSync.mockImplementation((filePath: any) => {
if (String(filePath).endsWith('profile.yml')) return profileYaml;
if (String(filePath).endsWith('.gitmodules')) return gitmodulesContent;
return '';
});
const plan = await SubmoduleProfileService.createInitPlan('/path/to/profile.yml', '', '/repo');
expect(plan).toHaveLength(4);
const turnOfWar = plan.find((a) => a.name === 'Assets/_Game/Submodules/TurnOfWar');
expect(turnOfWar?.action).toBe('init');
expect(turnOfWar?.branch).toBe('main');
const endlessCrusade = plan.find((a) => a.name === 'Assets/_Game/Submodules/EndlessCrusade');
expect(endlessCrusade?.action).toBe('init');
expect(endlessCrusade?.branch).toBe('main');
const sensorToolkit = plan.find((a) => a.name === 'Assets/_Engine/Submodules/SensorToolkit');
expect(sensorToolkit?.action).toBe('skip');
expect(sensorToolkit?.branch).toBe('empty');
const unmatched = plan.find((a) => a.name === 'Assets/_Game/Submodules/Unmatched');
expect(unmatched?.action).toBe('skip');
expect(unmatched?.branch).toBe('empty');
});
it('applies variant overlay when variantPath is provided', async () => {
const profileYaml = `
submodules:
- name: TurnOfWar
branch: main
- name: EndlessCrusade
branch: main
`;
const variantYaml = `
submodules:
- name: EndlessCrusade
branch: empty
`;
const gitmodulesContent = `[submodule "Assets/_Game/Submodules/TurnOfWar"]
\tpath = Assets/_Game/Submodules/TurnOfWar
\turl = https://github.com/org/TurnOfWar.git
[submodule "Assets/_Game/Submodules/EndlessCrusade"]
\tpath = Assets/_Game/Submodules/EndlessCrusade
\turl = https://github.com/org/EndlessCrusade.git
`;
mockedFs.existsSync.mockReturnValue(true);
mockedFs.readFileSync.mockImplementation((filePath: any) => {
const p = String(filePath);
if (p.endsWith('profile.yml')) return profileYaml;
if (p.endsWith('variant.yml')) return variantYaml;
if (p.endsWith('.gitmodules')) return gitmodulesContent;
return '';
});
const plan = await SubmoduleProfileService.createInitPlan(
'/path/to/profile.yml',
'/path/to/variant.yml',
'/repo',
);
expect(plan).toHaveLength(2);
const turnOfWar = plan.find((a) => a.name === 'Assets/_Game/Submodules/TurnOfWar');
expect(turnOfWar?.action).toBe('init');
const endlessCrusade = plan.find((a) => a.name === 'Assets/_Game/Submodules/EndlessCrusade');
expect(endlessCrusade?.action).toBe('skip');
});
});
describe('execute', () => {
it('runs init commands for init actions and deinit for skip actions', async () => {
mockedSystem.Run.mockResolvedValue('');
const plan = [
{ name: 'ModuleA', path: 'Assets/ModuleA', branch: 'main', action: 'init' as const },
{ name: 'ModuleB', path: 'Assets/ModuleB', branch: 'develop', action: 'init' as const },
{ name: 'ModuleC', path: 'Assets/ModuleC', branch: 'empty', action: 'skip' as const },
];
await SubmoduleProfileService.execute(plan, '/repo');
// ModuleA: init only (branch is main, no checkout needed)
expect(mockedSystem.Run).toHaveBeenCalledWith('git submodule update --init Assets/ModuleA');
// ModuleB: init + checkout develop
expect(mockedSystem.Run).toHaveBeenCalledWith('git submodule update --init Assets/ModuleB');
expect(mockedSystem.Run).toHaveBeenCalledWith('git -C Assets/ModuleB checkout develop');
// ModuleC: deinit
expect(mockedSystem.Run).toHaveBeenCalledWith('git submodule deinit -f Assets/ModuleC 2>/dev/null || true');
});
it('configures auth when token is provided', async () => {
mockedSystem.Run.mockResolvedValue('');
await SubmoduleProfileService.execute([], '/repo', 'my-secret-token');
expect(mockedSystem.Run).toHaveBeenCalledWith(
'git config url."https://my-secret-token@github.com/".insteadOf "https://github.com/"',
);
});
it('does not configure auth when no token is provided', async () => {
mockedSystem.Run.mockResolvedValue('');
await SubmoduleProfileService.execute([], '/repo');
expect(mockedSystem.Run).not.toHaveBeenCalledWith(expect.stringContaining('git config url'));
});
});
describe('execute — command construction safety', () => {
it('constructs expected git commands for a standard path', async () => {
mockedSystem.Run.mockResolvedValue('');
const plan = [
{
name: 'Assets/_Game/Submodules/TurnOfWar',
path: 'Assets/_Game/Submodules/TurnOfWar',
branch: 'main',
action: 'init' as const,
},
];
await SubmoduleProfileService.execute(plan, '/repo');
expect(mockedSystem.Run).toHaveBeenCalledWith('git submodule update --init Assets/_Game/Submodules/TurnOfWar');
});
it('passes path directly into git commands (paths with spaces are not quoted)', async () => {
mockedSystem.Run.mockResolvedValue('');
// This test documents current behavior: paths are passed as-is.
// If a path contained shell metacharacters, they would be passed through.
// This is acceptable because submodule paths come from .gitmodules (trusted source),
// not from user input. Still, this test documents the behavior for awareness.
const plan = [
{ name: 'Module With Spaces', path: 'Assets/Module With Spaces', branch: 'main', action: 'init' as const },
];
await SubmoduleProfileService.execute(plan, '/repo');
// The current implementation passes the path directly — no shell quoting
expect(mockedSystem.Run).toHaveBeenCalledWith('git submodule update --init Assets/Module With Spaces');
});
it('passes branch name directly into git checkout command', async () => {
mockedSystem.Run.mockResolvedValue('');
// Document that branch names are passed as-is into shell commands.
// Branch names come from the trusted profile YAML, not user input.
const plan = [{ name: 'ModuleX', path: 'Assets/ModuleX', branch: 'feature/my-branch', action: 'init' as const }];
await SubmoduleProfileService.execute(plan, '/repo');
expect(mockedSystem.Run).toHaveBeenCalledWith('git -C Assets/ModuleX checkout feature/my-branch');
});
it('constructs deinit command with error suppression for skip actions', async () => {
mockedSystem.Run.mockResolvedValue('');
const plan = [{ name: 'Unused', path: 'Assets/Unused', branch: 'empty', action: 'skip' as const }];
await SubmoduleProfileService.execute(plan, '/repo');
expect(mockedSystem.Run).toHaveBeenCalledWith('git submodule deinit -f Assets/Unused 2>/dev/null || true');
});
it('injects token into git config URL insteadOf pattern', async () => {
mockedSystem.Run.mockResolvedValue('');
// The token is embedded directly into the URL pattern.
// This is the standard Git credential approach for CI.
await SubmoduleProfileService.execute([], '/repo', 'ghp_abc123xyz');
const configCall = mockedSystem.Run.mock.calls.find((call: any[]) => String(call[0]).includes('git config url'));
expect(configCall).toBeDefined();
expect(configCall![0]).toBe('git config url."https://ghp_abc123xyz@github.com/".insteadOf "https://github.com/"');
});
});
});

View File

@@ -0,0 +1,226 @@
import fs from 'node:fs';
import path from 'node:path';
import YAML from 'yaml';
import { SubmoduleProfile, SubmoduleEntry, SubmoduleInitAction, SubmoduleInitPlan } from './submodule-profile-types';
import { OrchestratorSystem } from '../core/orchestrator-system';
import OrchestratorLogger from '../core/orchestrator-logger';
export class SubmoduleProfileService {
/**
* Parse a submodule profile YAML file and return the typed profile.
*/
static parseProfile(profilePath: string): SubmoduleProfile {
if (!fs.existsSync(profilePath)) {
throw new Error(`Submodule profile not found: ${profilePath}`);
}
const raw = fs.readFileSync(profilePath, 'utf8');
let parsed: any;
try {
parsed = YAML.parse(raw);
} catch (error: any) {
throw new Error(`Failed to parse submodule profile YAML at ${profilePath}: ${error.message}`);
}
if (!parsed || !Array.isArray(parsed.submodules)) {
throw new Error(`Invalid submodule profile: expected 'submodules' array in ${profilePath}`);
}
return {
primary_submodule: parsed.primary_submodule,
product_name: parsed.product_name,
submodules: parsed.submodules.map((entry: any) => ({
name: String(entry.name),
branch: String(entry.branch),
})),
};
}
/**
* Merge a variant profile on top of a base profile.
* Variant submodule entries override base entries matched by name.
* New variant entries are appended.
* Scalar fields (primary_submodule, product_name) are replaced by variant values.
*/
static mergeVariant(base: SubmoduleProfile, variantPath: string): SubmoduleProfile {
if (!fs.existsSync(variantPath)) {
throw new Error(`Submodule variant not found: ${variantPath}`);
}
const variant = SubmoduleProfileService.parseProfile(variantPath);
// Start with a copy of base submodules
const mergedEntries = new Map<string, SubmoduleEntry>();
for (const entry of base.submodules) {
mergedEntries.set(entry.name, { ...entry });
}
// Overlay variant entries
for (const entry of variant.submodules) {
mergedEntries.set(entry.name, { ...entry });
}
return {
primary_submodule: variant.primary_submodule ?? base.primary_submodule,
product_name: variant.product_name ?? base.product_name,
submodules: [...mergedEntries.values()],
};
}
/**
* Parse the .gitmodules file from a repository and return a map of submodule name -> path.
*/
static parseGitmodules(repoPath: string): Map<string, string> {
const gitmodulesPath = path.join(repoPath, '.gitmodules');
const result = new Map<string, string>();
if (!fs.existsSync(gitmodulesPath)) {
return result;
}
const content = fs.readFileSync(gitmodulesPath, 'utf8');
const lines = content.split('\n');
let currentName: string | undefined;
for (const line of lines) {
const trimmed = line.trim();
// Match [submodule "name"]
const submoduleMatch = trimmed.match(/^\[submodule\s+"(.+)"\]$/);
if (submoduleMatch) {
currentName = submoduleMatch[1];
continue;
}
// Match path = value
const pathMatch = trimmed.match(/^path\s*=\s*(.+)$/);
if (pathMatch && currentName) {
result.set(currentName, pathMatch[1].trim());
}
}
return result;
}
/**
* Match a submodule name/path against a profile pattern.
* Supports exact match and glob-like patterns (only `*` wildcard at end).
* Matches against both the full submodule path and the leaf folder name.
*/
static matchSubmodule(submoduleName: string, pattern: string): boolean {
// Check for trailing wildcard
if (pattern.endsWith('*')) {
const prefix = pattern.slice(0, -1);
// Match against full path
if (submoduleName.startsWith(prefix)) {
return true;
}
// Match against leaf folder name
const leaf = submoduleName.split('/').pop() || '';
if (leaf.startsWith(prefix)) {
return true;
}
return false;
}
// Exact match against full path
if (submoduleName === pattern) {
return true;
}
// Exact match against leaf folder name
const leaf = submoduleName.split('/').pop() || '';
if (leaf === pattern) {
return true;
}
return false;
}
/**
* Create an initialization plan by matching .gitmodules entries against profile rules.
* Unmatched submodules default to 'skip'.
*/
static async createInitPlan(profilePath: string, variantPath: string, repoPath: string): Promise<SubmoduleInitPlan> {
let profile = SubmoduleProfileService.parseProfile(profilePath);
if (variantPath) {
profile = SubmoduleProfileService.mergeVariant(profile, variantPath);
}
const gitmodules = SubmoduleProfileService.parseGitmodules(repoPath);
const plan: SubmoduleInitPlan = [];
for (const [name, submodulePath] of gitmodules) {
let matchedEntry: SubmoduleEntry | undefined;
for (const entry of profile.submodules) {
if (
SubmoduleProfileService.matchSubmodule(name, entry.name) ||
SubmoduleProfileService.matchSubmodule(submodulePath, entry.name)
) {
matchedEntry = entry;
break;
}
}
if (matchedEntry) {
const action: SubmoduleInitAction = {
name,
path: submodulePath,
branch: matchedEntry.branch,
action: matchedEntry.branch === 'empty' ? 'skip' : 'init',
};
plan.push(action);
} else {
// Unmatched submodules default to skip
plan.push({
name,
path: submodulePath,
branch: 'empty',
action: 'skip',
});
}
}
return plan;
}
/**
* Execute a submodule initialization plan.
* Configures auth if token is provided, then inits or deinits each submodule.
*/
static async execute(plan: SubmoduleInitPlan, repoPath: string, token?: string): Promise<void> {
if (token) {
OrchestratorLogger.log('Configuring git authentication for submodule initialization...');
await OrchestratorSystem.Run(`git config url."https://${token}@github.com/".insteadOf "https://github.com/"`);
}
for (const action of plan) {
const fullPath = path.posix.join(repoPath, action.path).replace(/\\/g, '/');
if (action.action === 'init') {
OrchestratorLogger.log(`Initializing submodule: ${action.name} (branch: ${action.branch})`);
await OrchestratorSystem.Run(`git submodule update --init ${action.path}`);
if (action.branch !== 'main') {
OrchestratorLogger.log(`Checking out branch '${action.branch}' for submodule: ${action.name}`);
await OrchestratorSystem.Run(`git -C ${action.path} checkout ${action.branch}`);
}
} else {
OrchestratorLogger.log(`Skipping submodule: ${action.name}`);
await OrchestratorSystem.Run(`git submodule deinit -f ${action.path} 2>/dev/null || true`);
}
}
OrchestratorLogger.log(
`Submodule initialization complete: ${plan.filter((a) => a.action === 'init').length} initialized, ${
plan.filter((a) => a.action === 'skip').length
} skipped`,
);
}
}

View File

@@ -0,0 +1,19 @@
export interface SubmoduleEntry {
name: string;
branch: string;
}
export interface SubmoduleProfile {
primary_submodule?: string;
product_name?: string;
submodules: SubmoduleEntry[];
}
export interface SubmoduleInitAction {
name: string;
path: string;
branch: string;
action: 'init' | 'skip';
}
export type SubmoduleInitPlan = SubmoduleInitAction[];

View File

@@ -6,6 +6,8 @@ import { CommandHookService } from '../services/hooks/command-hook-service';
import path from 'node:path';
import Orchestrator from '../orchestrator';
import { ContainerHookService } from '../services/hooks/container-hook-service';
import { MiddlewareService } from '../services/hooks/middleware-service';
import { CustomWorkflow } from './custom-workflow';
export class BuildAutomationWorkflow implements WorkflowInterface {
async run(orchestratorStepState: OrchestratorStepParameters) {
@@ -13,13 +15,41 @@ export class BuildAutomationWorkflow implements WorkflowInterface {
}
private static async standardBuildAutomation(baseImage: string, orchestratorStepState: OrchestratorStepParameters) {
// TODO accept post and pre build steps as yaml files in the repo
OrchestratorLogger.log(`Orchestrator is running standard build automation`);
let output = '';
// Load middleware pipeline
const middleware = MiddlewareService.getMiddleware(Orchestrator.buildParameters.middleware);
// --- Pre-build phase ---
// Middleware container hooks: pre-build before
const preBuildMiddlewareBefore = MiddlewareService.resolveContainerHooks(middleware, 'pre-build', 'before');
if (preBuildMiddlewareBefore.length > 0) {
output += await CustomWorkflow.runContainerJob(
preBuildMiddlewareBefore,
orchestratorStepState.environment,
orchestratorStepState.secrets,
);
OrchestratorLogger.logWithTime('Middleware pre-build:before container step(s) time');
}
// Standard pre-build container hooks
output += await ContainerHookService.RunPreBuildSteps(orchestratorStepState);
OrchestratorLogger.logWithTime('Configurable pre build step(s) time');
// Middleware container hooks: pre-build after
const preBuildMiddlewareAfter = MiddlewareService.resolveContainerHooks(middleware, 'pre-build', 'after');
if (preBuildMiddlewareAfter.length > 0) {
output += await CustomWorkflow.runContainerJob(
preBuildMiddlewareAfter,
orchestratorStepState.environment,
orchestratorStepState.secrets,
);
OrchestratorLogger.logWithTime('Middleware pre-build:after container step(s) time');
}
// --- Build phase ---
OrchestratorLogger.log(baseImage);
OrchestratorLogger.logLine(` `);
OrchestratorLogger.logLine('Starting build automation job');
@@ -35,9 +65,33 @@ export class BuildAutomationWorkflow implements WorkflowInterface {
);
OrchestratorLogger.logWithTime('Build time');
// --- Post-build phase ---
// Middleware container hooks: post-build before
const postBuildMiddlewareBefore = MiddlewareService.resolveContainerHooks(middleware, 'post-build', 'before');
if (postBuildMiddlewareBefore.length > 0) {
output += await CustomWorkflow.runContainerJob(
postBuildMiddlewareBefore,
orchestratorStepState.environment,
orchestratorStepState.secrets,
);
OrchestratorLogger.logWithTime('Middleware post-build:before container step(s) time');
}
// Standard post-build container hooks
output += await ContainerHookService.RunPostBuildSteps(orchestratorStepState);
OrchestratorLogger.logWithTime('Configurable post build step(s) time');
// Middleware container hooks: post-build after
const postBuildMiddlewareAfter = MiddlewareService.resolveContainerHooks(middleware, 'post-build', 'after');
if (postBuildMiddlewareAfter.length > 0) {
output += await CustomWorkflow.runContainerJob(
postBuildMiddlewareAfter,
orchestratorStepState.environment,
orchestratorStepState.secrets,
);
OrchestratorLogger.logWithTime('Middleware post-build:after container step(s) time');
}
OrchestratorLogger.log(`Orchestrator finished running standard build automation`);
return output;
@@ -50,6 +104,13 @@ export class BuildAutomationWorkflow implements WorkflowInterface {
const buildHooks = CommandHookService.getHooks(Orchestrator.buildParameters.commandHooks).filter((x) =>
x.step?.includes(`build`),
);
// Resolve middleware command hooks for setup and build phases
const middleware = MiddlewareService.getMiddleware(Orchestrator.buildParameters.middleware);
const middlewareSetupBefore = MiddlewareService.resolveCommandHooks(middleware, 'setup', 'before');
const middlewareSetupAfter = MiddlewareService.resolveCommandHooks(middleware, 'setup', 'after');
const middlewareBuildBefore = MiddlewareService.resolveCommandHooks(middleware, 'build', 'before');
const middlewareBuildAfter = MiddlewareService.resolveCommandHooks(middleware, 'build', 'after');
const isContainerized =
Orchestrator.buildParameters.providerStrategy === 'aws' ||
Orchestrator.buildParameters.providerStrategy === 'k8s' ||
@@ -71,6 +132,7 @@ export class BuildAutomationWorkflow implements WorkflowInterface {
? 'apt-get install -y curl tar tree npm git-lfs jq git > /dev/null || true\n npm --version || true\n npm i -g n > /dev/null || true\n npm i -g semver > /dev/null || true\n npm install --global yarn > /dev/null || true\n n 20.8.0 || true\n node --version || true'
: '# skipping toolchain setup in local-docker or non-container provider'
}
${middlewareSetupBefore.map((x) => x.commands).join('\n ') || ' '}
${setupHooks.filter((x) => x.hook.includes(`before`)).map((x) => x.commands) || ' '}
${
Orchestrator.buildParameters.providerStrategy === 'local-docker'
@@ -82,9 +144,12 @@ export class BuildAutomationWorkflow implements WorkflowInterface {
export LOG_FILE=${isContainerized ? '/home/job-log.txt' : '$(pwd)/temp/job-log.txt'}
${BuildAutomationWorkflow.setupCommands(builderPath, isContainerized)}
${setupHooks.filter((x) => x.hook.includes(`after`)).map((x) => x.commands) || ' '}
${middlewareSetupAfter.map((x) => x.commands).join('\n ') || ' '}
${middlewareBuildBefore.map((x) => x.commands).join('\n ') || ' '}
${buildHooks.filter((x) => x.hook.includes(`before`)).map((x) => x.commands) || ' '}
${BuildAutomationWorkflow.BuildCommands(builderPath, isContainerized)}
${buildHooks.filter((x) => x.hook.includes(`after`)).map((x) => x.commands) || ' '}`;
${buildHooks.filter((x) => x.hook.includes(`after`)).map((x) => x.commands) || ' '}
${middlewareBuildAfter.map((x) => x.commands).join('\n ') || ' '}`;
}
private static setupCommands(builderPath: string, isContainerized: boolean) {