Compare commits

..

2 Commits

Author SHA1 Message Date
frostebite
3d8edb0711 fix: replace orchestrator-develop branch references with main
The orchestrator-develop branch no longer exists. Update all fallback
clone commands and test fixtures to use main instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 19:54:16 +00:00
frostebite
9cf65cf5fc feat: add CLAUDE.md for AI-assisted development
Adds a minimal CLAUDE.md covering project architecture, build system,
code conventions, testing, security considerations, and CI workflows.
Provides Claude Code with the context needed to assist with builds,
debugging, orchestrator work, and contributions.

Closes #801

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:18:28 +00:00
11 changed files with 105 additions and 52 deletions

View File

@@ -54,7 +54,7 @@ jobs:
# AWS_STACK_NAME: game-ci-github-pipelines
CHECKS_UPDATE: ${{ github.event.inputs.checksObject }}
run: |
git clone -b orchestrator-develop https://github.com/game-ci/unity-builder
git clone -b main https://github.com/game-ci/unity-builder
cd unity-builder
yarn
ls

View File

@@ -91,7 +91,7 @@ jobs:
-e SERVICES=s3,cloudformation,ecs,kinesis,cloudwatch,logs,efs,ec2,iam,elasticfilesystem,secretsmanager,lambda,events,sts \
-e DEBUG=0 \
-e HOSTNAME_EXTERNAL=localstack-main \
localstack/localstack:4.4.0 || true
localstack/localstack:latest || true
# Wait for LocalStack to be ready - check both health endpoint and S3 service
echo "Waiting for LocalStack to be ready..."
MAX_ATTEMPTS=60

80
CLAUDE.md Normal file
View File

@@ -0,0 +1,80 @@
# Unity-Builder
GitHub Action and CLI that builds Unity projects for multiple platforms. Part of the [GameCI](https://game.ci) project.
## Quick Reference
```bash
yarn # install dependencies
yarn build # full build: tsc → ncc bundle (src/ → lib/ → dist/index.js)
yarn test # run all tests (jest)
yarn test:ci # run tests in CI mode (single-threaded, 2min timeout)
yarn lint # prettier + eslint check
yarn format # auto-format with prettier
```
## Architecture
**Entry point:** `src/index.ts` → decides between CLI mode and GitHub Action mode.
**Two execution paths:**
1. **Local builds** — Docker container or native macOS (`src/model/docker.ts`, `src/model/mac-builder.ts`)
2. **Orchestrator builds** — Remote execution on AWS ECS, Kubernetes, or other providers (`src/model/orchestrator/`)
**Key modules:**
| Path | Purpose |
|---|---|
| `src/model/build-parameters.ts` | Central config object — all build settings flow through here |
| `src/model/input.ts` | Input resolution with priority: Action inputs → CLI flags → env override → env vars |
| `src/model/orchestrator/orchestrator.ts` | Remote build orchestration — provider selection, workflow execution |
| `src/model/orchestrator/providers/` | Provider plugin system (AWS, K8s, Docker, Local, Test) |
| `src/model/orchestrator/remote-client/` | Code that runs inside remote containers (caching, hooks, artifacts) |
| `src/model/orchestrator/workflows/` | Build workflow types (standard, custom, async) |
| `src/model/orchestrator/services/` | Logging, locking, resource tracking |
| `src/model/cli/` | CLI mode using commander — dispatches to `@CliFunction`-decorated methods |
| `action.yml` | GitHub Action manifest — all inputs/outputs defined here |
| `dist/index.js` | Bundled output (committed to repo, used by action.yml at runtime) |
**Provider interface:** All providers implement `ProviderInterface` (`providers/provider-interface.ts`) with methods: `setupWorkflow`, `runTaskInWorkflow`, `cleanupWorkflow`, `garbageCollect`, `listResources`, `listWorkflow`, `watchWorkflow`.
**Provider loading:** Providers can be built-in, loaded from npm, cloned from GitHub repos, or loaded from local paths (`provider-loader.ts`).
## Build System
The build pipeline is: `yarn``tsc` (src/ → lib/) → `ncc build lib` (lib/ → dist/index.js).
- **dist/ is committed** — GitHub Actions loads `dist/index.js` directly, no install step on runners
- **Pre-commit hooks** (lefthook) auto-run formatting, linting, related tests, and `yarn build` to keep dist/ in sync
- Runtime: Node 20 (configured via Volta and action.yml `runs.using: node20`)
## Code Conventions
- **Files:** kebab-case (enforced by eslint `unicorn/filename-case`)
- **Code:** camelCase variables/functions, PascalCase classes/types
- **Formatting:** Prettier — 120 char width, single quotes, trailing commas, semicolons
- **Linting:** ESLint with unicorn, github, prettier, jest plugins
- **TypeScript:** strict mode, ES2020 target, CommonJS modules, experimental decorators enabled
- **Blank line before return statements** (enforced)
- **Blank line before block/line comments** (enforced)
- **No `for...in` loops** — use `for...of`
## Testing
- **Framework:** Jest 27 with ts-jest
- **Pattern:** `**/*.test.ts` files colocated with source
- **Orchestrator tests:** Concentrated in `src/model/orchestrator/tests/`
- **Run specific tests:** `yarn test -t "pattern"` or `yarn jest path/to/file.test.ts`
- **Orchestrator integration tests** require `orchestratorTests=true` env var: `cross-env orchestratorTests=true yarn test -i -t "orchestrator"`
## Security
- **Never log, output, or hardcode credentials** — cloud provider secrets (AWS, GCP, K8s), Unity serial keys, keystores, and private tokens must stay in secret inputs
- **Input validation matters** — user-supplied hook commands and custom parameters can be injection vectors; use `shell-quote` for shell escaping
- **Keystore/license data** is base64-encoded in inputs and written to temp files at build time
## CI Workflows
- `integrity-check.yml` — lint, test, build on every push/PR
- `build-tests-{ubuntu,windows,mac}.yml` — matrix builds across Unity versions and platforms
- `orchestrator-integrity.yml` / `orchestrator-async-checks.yml` — orchestrator-specific validation

34
dist/index.js generated vendored
View File

@@ -3398,7 +3398,7 @@ class AWSTaskRunner {
return { name: x.name, value };
});
}
static async runTask(taskDef, environment, secrets, commands) {
static async runTask(taskDef, environment, commands) {
const cluster = taskDef.baseResources?.find((x) => x.LogicalResourceId === 'ECSCluster')?.PhysicalResourceId || '';
const taskDefinition = taskDef.taskDefResources?.find((x) => x.LogicalResourceId === 'TaskDefinition')?.PhysicalResourceId || '';
const SubnetOne = taskDef.baseResources?.find((x) => x.LogicalResourceId === 'PublicSubnetOne')?.PhysicalResourceId || '';
@@ -3407,11 +3407,6 @@ class AWSTaskRunner {
const streamName = taskDef.taskDefResources?.find((x) => x.LogicalResourceId === 'KinesisStream')?.PhysicalResourceId || '';
// Transform localhost endpoints for container environment
const transformedEnvironment = AWSTaskRunner.transformEndpointsForContainer(environment);
// Merge secrets into environment as plain env vars, matching docker and k8s provider behavior.
// This ensures UNITY_EMAIL, UNITY_PASSWORD, UNITY_SERIAL reach the container reliably
// without depending on CloudFormation Secrets Manager resolution.
const secretsAsEnvironment = secrets.map((s) => ({ name: s.EnvironmentVariable, value: s.ParameterValue }));
const mergedEnvironment = [...transformedEnvironment, ...secretsAsEnvironment];
const runParameters = {
cluster,
taskDefinition,
@@ -3420,7 +3415,7 @@ class AWSTaskRunner {
containerOverrides: [
{
name: taskDef.taskDefStackName,
environment: mergedEnvironment,
environment: transformedEnvironment,
command: ['-c', command_hook_service_1.CommandHookService.ApplyHooksToCommands(commands, orchestrator_1.default.buildParameters)],
},
],
@@ -4454,7 +4449,7 @@ class AWSBuildEnvironment {
try {
const postSetupStacksTimeMs = Date.now();
orchestrator_logger_1.default.log(`Setup job time: ${Math.floor((postSetupStacksTimeMs - startTimeMs) / 1000)}s`);
const { output, shouldCleanup } = await aws_task_runner_1.default.runTask(taskDef, environment, secrets, commands);
const { output, shouldCleanup } = await aws_task_runner_1.default.runTask(taskDef, environment, commands);
postRunTaskTimeMs = Date.now();
orchestrator_logger_1.default.log(`Run job time: ${Math.floor((postRunTaskTimeMs - postSetupStacksTimeMs) / 1000)}s`);
if (shouldCleanup) {
@@ -9419,10 +9414,6 @@ class ContainerHookService {
fi
ENDPOINT_ARGS=""
if [ -n "$AWS_S3_ENDPOINT" ]; then ENDPOINT_ARGS="--endpoint-url $AWS_S3_ENDPOINT"; fi
# Skip uploading empty or near-empty tar files (< 1KB) these are leftover
# stubs with no real cache data and would poison the cache for the next build.
find /data/cache/$CACHE_KEY/lfs -name "*.tar*" -size -1k -delete 2>/dev/null || true
find /data/cache/$CACHE_KEY/Library -name "*.tar*" -size -1k -delete 2>/dev/null || true
aws $ENDPOINT_ARGS s3 cp --recursive /data/cache/$CACHE_KEY/lfs s3://${orchestrator_1.default.buildParameters.awsStackName}/orchestrator-cache/$CACHE_KEY/lfs || true
rm -r /data/cache/$CACHE_KEY/lfs || true
aws $ENDPOINT_ARGS s3 cp --recursive /data/cache/$CACHE_KEY/Library s3://${orchestrator_1.default.buildParameters.awsStackName}/orchestrator-cache/$CACHE_KEY/Library || true
@@ -9740,8 +9731,7 @@ if [ -n "$(git ls-remote --heads "$REPO" "$BRANCH" 2>/dev/null)" ]; then
git clone -q -b "$BRANCH" "$REPO" /builder
else
echo "Remote branch $BRANCH not found in $REPO; falling back to a known branch"
git clone -q -b orchestrator-develop "$REPO" /builder \
|| git clone -q -b main "$REPO" /builder \
git clone -q -b main "$REPO" /builder \
|| git clone -q "$REPO" /builder
fi
git clone -q -b ${orchestrator_1.default.buildParameters.branch} ${orchestrator_folders_1.OrchestratorFolders.targetBuildRepoUrl} /repo
@@ -9858,8 +9848,7 @@ if [ -n "$(git ls-remote --heads "$REPO" "$BRANCH" 2>/dev/null)" ]; then
git clone -q -b "$BRANCH" "$REPO" "$DEST"
else
echo "Remote branch $BRANCH not found in $REPO; falling back to a known branch"
git clone -q -b orchestrator-develop "$REPO" "$DEST" \
|| git clone -q -b main "$REPO" "$DEST" \
git clone -q -b main "$REPO" "$DEST" \
|| git clone -q "$REPO" "$DEST"
fi
chmod +x ${builderPath}`;
@@ -9916,14 +9905,13 @@ echo "CACHE_KEY=$CACHE_KEY"`;
if ! command -v yarn > /dev/null 2>&1; then printf '#!/bin/sh\nexit 0\n' > /usr/local/bin/yarn && chmod +x /usr/local/bin/yarn; fi
# Pipe entrypoint.sh output through log stream to capture Unity build output (including "Build succeeded")
{ echo "game ci start"; echo "game ci start" >> /home/job-log.txt; echo "CACHE_KEY=$CACHE_KEY"; echo "$CACHE_KEY"; if [ -n "$LOCKED_WORKSPACE" ]; then echo "Retained Workspace: true"; fi; if [ -n "$LOCKED_WORKSPACE" ] && [ -d "$GITHUB_WORKSPACE/.git" ]; then echo "Retained Workspace Already Exists!"; fi; /entrypoint.sh; } | node ${builderPath} -m remote-cli-log-stream --logFile /home/job-log.txt
# Ensure cache directories exist for post-build and S3 upload hooks.
# Do NOT create empty placeholder tars they waste S3 storage and on next
# build the pull-cache hook downloads them, giving Unity an empty Library
# (no caching benefit). The real tars are created by remote-cli-post-build
# via Caching.PushToCache(), and the S3 upload hooks use || true so missing
# files are handled gracefully.
mkdir -p "/data/cache/$CACHE_KEY/Library"
mkdir -p "/data/cache/$CACHE_KEY/build"
if [ ! -f "/data/cache/$CACHE_KEY/Library/lib-$BUILD_GUID.tar" ] && [ ! -f "/data/cache/$CACHE_KEY/Library/lib-$BUILD_GUID.tar.lz4" ]; then
tar -cf "/data/cache/$CACHE_KEY/Library/lib-$BUILD_GUID.tar" --files-from /dev/null || touch "/data/cache/$CACHE_KEY/Library/lib-$BUILD_GUID.tar"
fi
if [ ! -f "/data/cache/$CACHE_KEY/build/build-$BUILD_GUID.tar" ] && [ ! -f "/data/cache/$CACHE_KEY/build/build-$BUILD_GUID.tar.lz4" ]; then
tar -cf "/data/cache/$CACHE_KEY/build/build-$BUILD_GUID.tar" --files-from /dev/null || touch "/data/cache/$CACHE_KEY/build/build-$BUILD_GUID.tar"
fi
# Run post-build tasks and capture output
# Note: Post-build may clean up the builder directory, so we write output directly to log file
# Use set +e to allow the command to fail without exiting the script

2
dist/index.js.map generated vendored

File diff suppressed because one or more lines are too long

View File

@@ -1,7 +1,6 @@
import { DescribeTasksCommand, RunTaskCommand, waitUntilTasksRunning } from '@aws-sdk/client-ecs';
import { DescribeStreamCommand, GetRecordsCommand, GetShardIteratorCommand } from '@aws-sdk/client-kinesis';
import OrchestratorEnvironmentVariable from '../../options/orchestrator-environment-variable';
import OrchestratorSecret from '../../options/orchestrator-secret';
import * as core from '@actions/core';
import OrchestratorAWSTaskDef from './orchestrator-aws-task-def';
import * as zlib from 'node:zlib';
@@ -57,7 +56,6 @@ class AWSTaskRunner {
static async runTask(
taskDef: OrchestratorAWSTaskDef,
environment: OrchestratorEnvironmentVariable[],
secrets: OrchestratorSecret[],
commands: string,
): Promise<{ output: string; shouldCleanup: boolean }> {
const cluster = taskDef.baseResources?.find((x) => x.LogicalResourceId === 'ECSCluster')?.PhysicalResourceId || '';
@@ -75,12 +73,6 @@ class AWSTaskRunner {
// Transform localhost endpoints for container environment
const transformedEnvironment = AWSTaskRunner.transformEndpointsForContainer(environment);
// Merge secrets into environment as plain env vars, matching docker and k8s provider behavior.
// This ensures UNITY_EMAIL, UNITY_PASSWORD, UNITY_SERIAL reach the container reliably
// without depending on CloudFormation Secrets Manager resolution.
const secretsAsEnvironment = secrets.map((s) => ({ name: s.EnvironmentVariable, value: s.ParameterValue }));
const mergedEnvironment = [...transformedEnvironment, ...secretsAsEnvironment];
const runParameters = {
cluster,
taskDefinition,
@@ -89,7 +81,7 @@ class AWSTaskRunner {
containerOverrides: [
{
name: taskDef.taskDefStackName,
environment: mergedEnvironment,
environment: transformedEnvironment,
command: ['-c', CommandHookService.ApplyHooksToCommands(commands, Orchestrator.buildParameters)],
},
],

View File

@@ -125,7 +125,7 @@ class AWSBuildEnvironment implements ProviderInterface {
try {
const postSetupStacksTimeMs = Date.now();
OrchestratorLogger.log(`Setup job time: ${Math.floor((postSetupStacksTimeMs - startTimeMs) / 1000)}s`);
const { output, shouldCleanup } = await AwsTaskRunner.runTask(taskDef, environment, secrets, commands);
const { output, shouldCleanup } = await AwsTaskRunner.runTask(taskDef, environment, commands);
postRunTaskTimeMs = Date.now();
OrchestratorLogger.log(`Run job time: ${Math.floor((postRunTaskTimeMs - postSetupStacksTimeMs) / 1000)}s`);
if (shouldCleanup) {

View File

@@ -155,10 +155,6 @@ export class ContainerHookService {
fi
ENDPOINT_ARGS=""
if [ -n "$AWS_S3_ENDPOINT" ]; then ENDPOINT_ARGS="--endpoint-url $AWS_S3_ENDPOINT"; fi
# Skip uploading empty or near-empty tar files (< 1KB) — these are leftover
# stubs with no real cache data and would poison the cache for the next build.
find /data/cache/$CACHE_KEY/lfs -name "*.tar*" -size -1k -delete 2>/dev/null || true
find /data/cache/$CACHE_KEY/Library -name "*.tar*" -size -1k -delete 2>/dev/null || true
aws $ENDPOINT_ARGS s3 cp --recursive /data/cache/$CACHE_KEY/lfs s3://${
Orchestrator.buildParameters.awsStackName
}/orchestrator-cache/$CACHE_KEY/lfs || true

View File

@@ -30,7 +30,7 @@ describe('Orchestrator Caching', () => {
targetPlatform: 'StandaloneLinux64',
cacheKey: `test-case-${uuidv4()}`,
containerHookFiles: `debug-cache`,
orchestratorBranch: `orchestrator-develop`,
orchestratorBranch: `main`,
orchestratorDebug: true,
};

View File

@@ -33,8 +33,7 @@ if [ -n "$(git ls-remote --heads "$REPO" "$BRANCH" 2>/dev/null)" ]; then
git clone -q -b "$BRANCH" "$REPO" /builder
else
echo "Remote branch $BRANCH not found in $REPO; falling back to a known branch"
git clone -q -b orchestrator-develop "$REPO" /builder \
|| git clone -q -b main "$REPO" /builder \
git clone -q -b main "$REPO" /builder \
|| git clone -q "$REPO" /builder
fi
git clone -q -b ${Orchestrator.buildParameters.branch} ${OrchestratorFolders.targetBuildRepoUrl} /repo

View File

@@ -99,8 +99,7 @@ if [ -n "$(git ls-remote --heads "$REPO" "$BRANCH" 2>/dev/null)" ]; then
git clone -q -b "$BRANCH" "$REPO" "$DEST"
else
echo "Remote branch $BRANCH not found in $REPO; falling back to a known branch"
git clone -q -b orchestrator-develop "$REPO" "$DEST" \
|| git clone -q -b main "$REPO" "$DEST" \
git clone -q -b main "$REPO" "$DEST" \
|| git clone -q "$REPO" "$DEST"
fi
chmod +x ${builderPath}`;
@@ -170,14 +169,13 @@ echo "CACHE_KEY=$CACHE_KEY"`;
if ! command -v yarn > /dev/null 2>&1; then printf '#!/bin/sh\nexit 0\n' > /usr/local/bin/yarn && chmod +x /usr/local/bin/yarn; fi
# Pipe entrypoint.sh output through log stream to capture Unity build output (including "Build succeeded")
{ echo "game ci start"; echo "game ci start" >> /home/job-log.txt; echo "CACHE_KEY=$CACHE_KEY"; echo "$CACHE_KEY"; if [ -n "$LOCKED_WORKSPACE" ]; then echo "Retained Workspace: true"; fi; if [ -n "$LOCKED_WORKSPACE" ] && [ -d "$GITHUB_WORKSPACE/.git" ]; then echo "Retained Workspace Already Exists!"; fi; /entrypoint.sh; } | node ${builderPath} -m remote-cli-log-stream --logFile /home/job-log.txt
# Ensure cache directories exist for post-build and S3 upload hooks.
# Do NOT create empty placeholder tars — they waste S3 storage and on next
# build the pull-cache hook downloads them, giving Unity an empty Library
# (no caching benefit). The real tars are created by remote-cli-post-build
# via Caching.PushToCache(), and the S3 upload hooks use || true so missing
# files are handled gracefully.
mkdir -p "/data/cache/$CACHE_KEY/Library"
mkdir -p "/data/cache/$CACHE_KEY/build"
if [ ! -f "/data/cache/$CACHE_KEY/Library/lib-$BUILD_GUID.tar" ] && [ ! -f "/data/cache/$CACHE_KEY/Library/lib-$BUILD_GUID.tar.lz4" ]; then
tar -cf "/data/cache/$CACHE_KEY/Library/lib-$BUILD_GUID.tar" --files-from /dev/null || touch "/data/cache/$CACHE_KEY/Library/lib-$BUILD_GUID.tar"
fi
if [ ! -f "/data/cache/$CACHE_KEY/build/build-$BUILD_GUID.tar" ] && [ ! -f "/data/cache/$CACHE_KEY/build/build-$BUILD_GUID.tar.lz4" ]; then
tar -cf "/data/cache/$CACHE_KEY/build/build-$BUILD_GUID.tar" --files-from /dev/null || touch "/data/cache/$CACHE_KEY/build/build-$BUILD_GUID.tar"
fi
# Run post-build tasks and capture output
# Note: Post-build may clean up the builder directory, so we write output directly to log file
# Use set +e to allow the command to fail without exiting the script