mirror of
https://github.com/game-ci/unity-builder.git
synced 2026-06-02 06:46:15 -07:00
Compare commits
2 Commits
fix/skip-e
...
feat/claud
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
3d8edb0711 | ||
|
|
9cf65cf5fc |
@@ -54,7 +54,7 @@ jobs:
|
||||
# AWS_STACK_NAME: game-ci-github-pipelines
|
||||
CHECKS_UPDATE: ${{ github.event.inputs.checksObject }}
|
||||
run: |
|
||||
git clone -b orchestrator-develop https://github.com/game-ci/unity-builder
|
||||
git clone -b main https://github.com/game-ci/unity-builder
|
||||
cd unity-builder
|
||||
yarn
|
||||
ls
|
||||
|
||||
2
.github/workflows/orchestrator-integrity.yml
vendored
2
.github/workflows/orchestrator-integrity.yml
vendored
@@ -91,7 +91,7 @@ jobs:
|
||||
-e SERVICES=s3,cloudformation,ecs,kinesis,cloudwatch,logs,efs,ec2,iam,elasticfilesystem,secretsmanager,lambda,events,sts \
|
||||
-e DEBUG=0 \
|
||||
-e HOSTNAME_EXTERNAL=localstack-main \
|
||||
localstack/localstack:4.4.0 || true
|
||||
localstack/localstack:latest || true
|
||||
# Wait for LocalStack to be ready - check both health endpoint and S3 service
|
||||
echo "Waiting for LocalStack to be ready..."
|
||||
MAX_ATTEMPTS=60
|
||||
|
||||
80
CLAUDE.md
Normal file
80
CLAUDE.md
Normal file
@@ -0,0 +1,80 @@
|
||||
# Unity-Builder
|
||||
|
||||
GitHub Action and CLI that builds Unity projects for multiple platforms. Part of the [GameCI](https://game.ci) project.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
```bash
|
||||
yarn # install dependencies
|
||||
yarn build # full build: tsc → ncc bundle (src/ → lib/ → dist/index.js)
|
||||
yarn test # run all tests (jest)
|
||||
yarn test:ci # run tests in CI mode (single-threaded, 2min timeout)
|
||||
yarn lint # prettier + eslint check
|
||||
yarn format # auto-format with prettier
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
**Entry point:** `src/index.ts` → decides between CLI mode and GitHub Action mode.
|
||||
|
||||
**Two execution paths:**
|
||||
1. **Local builds** — Docker container or native macOS (`src/model/docker.ts`, `src/model/mac-builder.ts`)
|
||||
2. **Orchestrator builds** — Remote execution on AWS ECS, Kubernetes, or other providers (`src/model/orchestrator/`)
|
||||
|
||||
**Key modules:**
|
||||
|
||||
| Path | Purpose |
|
||||
|---|---|
|
||||
| `src/model/build-parameters.ts` | Central config object — all build settings flow through here |
|
||||
| `src/model/input.ts` | Input resolution with priority: Action inputs → CLI flags → env override → env vars |
|
||||
| `src/model/orchestrator/orchestrator.ts` | Remote build orchestration — provider selection, workflow execution |
|
||||
| `src/model/orchestrator/providers/` | Provider plugin system (AWS, K8s, Docker, Local, Test) |
|
||||
| `src/model/orchestrator/remote-client/` | Code that runs inside remote containers (caching, hooks, artifacts) |
|
||||
| `src/model/orchestrator/workflows/` | Build workflow types (standard, custom, async) |
|
||||
| `src/model/orchestrator/services/` | Logging, locking, resource tracking |
|
||||
| `src/model/cli/` | CLI mode using commander — dispatches to `@CliFunction`-decorated methods |
|
||||
| `action.yml` | GitHub Action manifest — all inputs/outputs defined here |
|
||||
| `dist/index.js` | Bundled output (committed to repo, used by action.yml at runtime) |
|
||||
|
||||
**Provider interface:** All providers implement `ProviderInterface` (`providers/provider-interface.ts`) with methods: `setupWorkflow`, `runTaskInWorkflow`, `cleanupWorkflow`, `garbageCollect`, `listResources`, `listWorkflow`, `watchWorkflow`.
|
||||
|
||||
**Provider loading:** Providers can be built-in, loaded from npm, cloned from GitHub repos, or loaded from local paths (`provider-loader.ts`).
|
||||
|
||||
## Build System
|
||||
|
||||
The build pipeline is: `yarn` → `tsc` (src/ → lib/) → `ncc build lib` (lib/ → dist/index.js).
|
||||
|
||||
- **dist/ is committed** — GitHub Actions loads `dist/index.js` directly, no install step on runners
|
||||
- **Pre-commit hooks** (lefthook) auto-run formatting, linting, related tests, and `yarn build` to keep dist/ in sync
|
||||
- Runtime: Node 20 (configured via Volta and action.yml `runs.using: node20`)
|
||||
|
||||
## Code Conventions
|
||||
|
||||
- **Files:** kebab-case (enforced by eslint `unicorn/filename-case`)
|
||||
- **Code:** camelCase variables/functions, PascalCase classes/types
|
||||
- **Formatting:** Prettier — 120 char width, single quotes, trailing commas, semicolons
|
||||
- **Linting:** ESLint with unicorn, github, prettier, jest plugins
|
||||
- **TypeScript:** strict mode, ES2020 target, CommonJS modules, experimental decorators enabled
|
||||
- **Blank line before return statements** (enforced)
|
||||
- **Blank line before block/line comments** (enforced)
|
||||
- **No `for...in` loops** — use `for...of`
|
||||
|
||||
## Testing
|
||||
|
||||
- **Framework:** Jest 27 with ts-jest
|
||||
- **Pattern:** `**/*.test.ts` files colocated with source
|
||||
- **Orchestrator tests:** Concentrated in `src/model/orchestrator/tests/`
|
||||
- **Run specific tests:** `yarn test -t "pattern"` or `yarn jest path/to/file.test.ts`
|
||||
- **Orchestrator integration tests** require `orchestratorTests=true` env var: `cross-env orchestratorTests=true yarn test -i -t "orchestrator"`
|
||||
|
||||
## Security
|
||||
|
||||
- **Never log, output, or hardcode credentials** — cloud provider secrets (AWS, GCP, K8s), Unity serial keys, keystores, and private tokens must stay in secret inputs
|
||||
- **Input validation matters** — user-supplied hook commands and custom parameters can be injection vectors; use `shell-quote` for shell escaping
|
||||
- **Keystore/license data** is base64-encoded in inputs and written to temp files at build time
|
||||
|
||||
## CI Workflows
|
||||
|
||||
- `integrity-check.yml` — lint, test, build on every push/PR
|
||||
- `build-tests-{ubuntu,windows,mac}.yml` — matrix builds across Unity versions and platforms
|
||||
- `orchestrator-integrity.yml` / `orchestrator-async-checks.yml` — orchestrator-specific validation
|
||||
34
dist/index.js
generated
vendored
34
dist/index.js
generated
vendored
@@ -3398,7 +3398,7 @@ class AWSTaskRunner {
|
||||
return { name: x.name, value };
|
||||
});
|
||||
}
|
||||
static async runTask(taskDef, environment, secrets, commands) {
|
||||
static async runTask(taskDef, environment, commands) {
|
||||
const cluster = taskDef.baseResources?.find((x) => x.LogicalResourceId === 'ECSCluster')?.PhysicalResourceId || '';
|
||||
const taskDefinition = taskDef.taskDefResources?.find((x) => x.LogicalResourceId === 'TaskDefinition')?.PhysicalResourceId || '';
|
||||
const SubnetOne = taskDef.baseResources?.find((x) => x.LogicalResourceId === 'PublicSubnetOne')?.PhysicalResourceId || '';
|
||||
@@ -3407,11 +3407,6 @@ class AWSTaskRunner {
|
||||
const streamName = taskDef.taskDefResources?.find((x) => x.LogicalResourceId === 'KinesisStream')?.PhysicalResourceId || '';
|
||||
// Transform localhost endpoints for container environment
|
||||
const transformedEnvironment = AWSTaskRunner.transformEndpointsForContainer(environment);
|
||||
// Merge secrets into environment as plain env vars, matching docker and k8s provider behavior.
|
||||
// This ensures UNITY_EMAIL, UNITY_PASSWORD, UNITY_SERIAL reach the container reliably
|
||||
// without depending on CloudFormation Secrets Manager resolution.
|
||||
const secretsAsEnvironment = secrets.map((s) => ({ name: s.EnvironmentVariable, value: s.ParameterValue }));
|
||||
const mergedEnvironment = [...transformedEnvironment, ...secretsAsEnvironment];
|
||||
const runParameters = {
|
||||
cluster,
|
||||
taskDefinition,
|
||||
@@ -3420,7 +3415,7 @@ class AWSTaskRunner {
|
||||
containerOverrides: [
|
||||
{
|
||||
name: taskDef.taskDefStackName,
|
||||
environment: mergedEnvironment,
|
||||
environment: transformedEnvironment,
|
||||
command: ['-c', command_hook_service_1.CommandHookService.ApplyHooksToCommands(commands, orchestrator_1.default.buildParameters)],
|
||||
},
|
||||
],
|
||||
@@ -4454,7 +4449,7 @@ class AWSBuildEnvironment {
|
||||
try {
|
||||
const postSetupStacksTimeMs = Date.now();
|
||||
orchestrator_logger_1.default.log(`Setup job time: ${Math.floor((postSetupStacksTimeMs - startTimeMs) / 1000)}s`);
|
||||
const { output, shouldCleanup } = await aws_task_runner_1.default.runTask(taskDef, environment, secrets, commands);
|
||||
const { output, shouldCleanup } = await aws_task_runner_1.default.runTask(taskDef, environment, commands);
|
||||
postRunTaskTimeMs = Date.now();
|
||||
orchestrator_logger_1.default.log(`Run job time: ${Math.floor((postRunTaskTimeMs - postSetupStacksTimeMs) / 1000)}s`);
|
||||
if (shouldCleanup) {
|
||||
@@ -9419,10 +9414,6 @@ class ContainerHookService {
|
||||
fi
|
||||
ENDPOINT_ARGS=""
|
||||
if [ -n "$AWS_S3_ENDPOINT" ]; then ENDPOINT_ARGS="--endpoint-url $AWS_S3_ENDPOINT"; fi
|
||||
# Skip uploading empty or near-empty tar files (< 1KB) — these are leftover
|
||||
# stubs with no real cache data and would poison the cache for the next build.
|
||||
find /data/cache/$CACHE_KEY/lfs -name "*.tar*" -size -1k -delete 2>/dev/null || true
|
||||
find /data/cache/$CACHE_KEY/Library -name "*.tar*" -size -1k -delete 2>/dev/null || true
|
||||
aws $ENDPOINT_ARGS s3 cp --recursive /data/cache/$CACHE_KEY/lfs s3://${orchestrator_1.default.buildParameters.awsStackName}/orchestrator-cache/$CACHE_KEY/lfs || true
|
||||
rm -r /data/cache/$CACHE_KEY/lfs || true
|
||||
aws $ENDPOINT_ARGS s3 cp --recursive /data/cache/$CACHE_KEY/Library s3://${orchestrator_1.default.buildParameters.awsStackName}/orchestrator-cache/$CACHE_KEY/Library || true
|
||||
@@ -9740,8 +9731,7 @@ if [ -n "$(git ls-remote --heads "$REPO" "$BRANCH" 2>/dev/null)" ]; then
|
||||
git clone -q -b "$BRANCH" "$REPO" /builder
|
||||
else
|
||||
echo "Remote branch $BRANCH not found in $REPO; falling back to a known branch"
|
||||
git clone -q -b orchestrator-develop "$REPO" /builder \
|
||||
|| git clone -q -b main "$REPO" /builder \
|
||||
git clone -q -b main "$REPO" /builder \
|
||||
|| git clone -q "$REPO" /builder
|
||||
fi
|
||||
git clone -q -b ${orchestrator_1.default.buildParameters.branch} ${orchestrator_folders_1.OrchestratorFolders.targetBuildRepoUrl} /repo
|
||||
@@ -9858,8 +9848,7 @@ if [ -n "$(git ls-remote --heads "$REPO" "$BRANCH" 2>/dev/null)" ]; then
|
||||
git clone -q -b "$BRANCH" "$REPO" "$DEST"
|
||||
else
|
||||
echo "Remote branch $BRANCH not found in $REPO; falling back to a known branch"
|
||||
git clone -q -b orchestrator-develop "$REPO" "$DEST" \
|
||||
|| git clone -q -b main "$REPO" "$DEST" \
|
||||
git clone -q -b main "$REPO" "$DEST" \
|
||||
|| git clone -q "$REPO" "$DEST"
|
||||
fi
|
||||
chmod +x ${builderPath}`;
|
||||
@@ -9916,14 +9905,13 @@ echo "CACHE_KEY=$CACHE_KEY"`;
|
||||
if ! command -v yarn > /dev/null 2>&1; then printf '#!/bin/sh\nexit 0\n' > /usr/local/bin/yarn && chmod +x /usr/local/bin/yarn; fi
|
||||
# Pipe entrypoint.sh output through log stream to capture Unity build output (including "Build succeeded")
|
||||
{ echo "game ci start"; echo "game ci start" >> /home/job-log.txt; echo "CACHE_KEY=$CACHE_KEY"; echo "$CACHE_KEY"; if [ -n "$LOCKED_WORKSPACE" ]; then echo "Retained Workspace: true"; fi; if [ -n "$LOCKED_WORKSPACE" ] && [ -d "$GITHUB_WORKSPACE/.git" ]; then echo "Retained Workspace Already Exists!"; fi; /entrypoint.sh; } | node ${builderPath} -m remote-cli-log-stream --logFile /home/job-log.txt
|
||||
# Ensure cache directories exist for post-build and S3 upload hooks.
|
||||
# Do NOT create empty placeholder tars — they waste S3 storage and on next
|
||||
# build the pull-cache hook downloads them, giving Unity an empty Library
|
||||
# (no caching benefit). The real tars are created by remote-cli-post-build
|
||||
# via Caching.PushToCache(), and the S3 upload hooks use || true so missing
|
||||
# files are handled gracefully.
|
||||
mkdir -p "/data/cache/$CACHE_KEY/Library"
|
||||
mkdir -p "/data/cache/$CACHE_KEY/build"
|
||||
if [ ! -f "/data/cache/$CACHE_KEY/Library/lib-$BUILD_GUID.tar" ] && [ ! -f "/data/cache/$CACHE_KEY/Library/lib-$BUILD_GUID.tar.lz4" ]; then
|
||||
tar -cf "/data/cache/$CACHE_KEY/Library/lib-$BUILD_GUID.tar" --files-from /dev/null || touch "/data/cache/$CACHE_KEY/Library/lib-$BUILD_GUID.tar"
|
||||
fi
|
||||
if [ ! -f "/data/cache/$CACHE_KEY/build/build-$BUILD_GUID.tar" ] && [ ! -f "/data/cache/$CACHE_KEY/build/build-$BUILD_GUID.tar.lz4" ]; then
|
||||
tar -cf "/data/cache/$CACHE_KEY/build/build-$BUILD_GUID.tar" --files-from /dev/null || touch "/data/cache/$CACHE_KEY/build/build-$BUILD_GUID.tar"
|
||||
fi
|
||||
# Run post-build tasks and capture output
|
||||
# Note: Post-build may clean up the builder directory, so we write output directly to log file
|
||||
# Use set +e to allow the command to fail without exiting the script
|
||||
|
||||
2
dist/index.js.map
generated
vendored
2
dist/index.js.map
generated
vendored
File diff suppressed because one or more lines are too long
@@ -1,7 +1,6 @@
|
||||
import { DescribeTasksCommand, RunTaskCommand, waitUntilTasksRunning } from '@aws-sdk/client-ecs';
|
||||
import { DescribeStreamCommand, GetRecordsCommand, GetShardIteratorCommand } from '@aws-sdk/client-kinesis';
|
||||
import OrchestratorEnvironmentVariable from '../../options/orchestrator-environment-variable';
|
||||
import OrchestratorSecret from '../../options/orchestrator-secret';
|
||||
import * as core from '@actions/core';
|
||||
import OrchestratorAWSTaskDef from './orchestrator-aws-task-def';
|
||||
import * as zlib from 'node:zlib';
|
||||
@@ -57,7 +56,6 @@ class AWSTaskRunner {
|
||||
static async runTask(
|
||||
taskDef: OrchestratorAWSTaskDef,
|
||||
environment: OrchestratorEnvironmentVariable[],
|
||||
secrets: OrchestratorSecret[],
|
||||
commands: string,
|
||||
): Promise<{ output: string; shouldCleanup: boolean }> {
|
||||
const cluster = taskDef.baseResources?.find((x) => x.LogicalResourceId === 'ECSCluster')?.PhysicalResourceId || '';
|
||||
@@ -75,12 +73,6 @@ class AWSTaskRunner {
|
||||
// Transform localhost endpoints for container environment
|
||||
const transformedEnvironment = AWSTaskRunner.transformEndpointsForContainer(environment);
|
||||
|
||||
// Merge secrets into environment as plain env vars, matching docker and k8s provider behavior.
|
||||
// This ensures UNITY_EMAIL, UNITY_PASSWORD, UNITY_SERIAL reach the container reliably
|
||||
// without depending on CloudFormation Secrets Manager resolution.
|
||||
const secretsAsEnvironment = secrets.map((s) => ({ name: s.EnvironmentVariable, value: s.ParameterValue }));
|
||||
const mergedEnvironment = [...transformedEnvironment, ...secretsAsEnvironment];
|
||||
|
||||
const runParameters = {
|
||||
cluster,
|
||||
taskDefinition,
|
||||
@@ -89,7 +81,7 @@ class AWSTaskRunner {
|
||||
containerOverrides: [
|
||||
{
|
||||
name: taskDef.taskDefStackName,
|
||||
environment: mergedEnvironment,
|
||||
environment: transformedEnvironment,
|
||||
command: ['-c', CommandHookService.ApplyHooksToCommands(commands, Orchestrator.buildParameters)],
|
||||
},
|
||||
],
|
||||
|
||||
@@ -125,7 +125,7 @@ class AWSBuildEnvironment implements ProviderInterface {
|
||||
try {
|
||||
const postSetupStacksTimeMs = Date.now();
|
||||
OrchestratorLogger.log(`Setup job time: ${Math.floor((postSetupStacksTimeMs - startTimeMs) / 1000)}s`);
|
||||
const { output, shouldCleanup } = await AwsTaskRunner.runTask(taskDef, environment, secrets, commands);
|
||||
const { output, shouldCleanup } = await AwsTaskRunner.runTask(taskDef, environment, commands);
|
||||
postRunTaskTimeMs = Date.now();
|
||||
OrchestratorLogger.log(`Run job time: ${Math.floor((postRunTaskTimeMs - postSetupStacksTimeMs) / 1000)}s`);
|
||||
if (shouldCleanup) {
|
||||
|
||||
@@ -155,10 +155,6 @@ export class ContainerHookService {
|
||||
fi
|
||||
ENDPOINT_ARGS=""
|
||||
if [ -n "$AWS_S3_ENDPOINT" ]; then ENDPOINT_ARGS="--endpoint-url $AWS_S3_ENDPOINT"; fi
|
||||
# Skip uploading empty or near-empty tar files (< 1KB) — these are leftover
|
||||
# stubs with no real cache data and would poison the cache for the next build.
|
||||
find /data/cache/$CACHE_KEY/lfs -name "*.tar*" -size -1k -delete 2>/dev/null || true
|
||||
find /data/cache/$CACHE_KEY/Library -name "*.tar*" -size -1k -delete 2>/dev/null || true
|
||||
aws $ENDPOINT_ARGS s3 cp --recursive /data/cache/$CACHE_KEY/lfs s3://${
|
||||
Orchestrator.buildParameters.awsStackName
|
||||
}/orchestrator-cache/$CACHE_KEY/lfs || true
|
||||
|
||||
@@ -30,7 +30,7 @@ describe('Orchestrator Caching', () => {
|
||||
targetPlatform: 'StandaloneLinux64',
|
||||
cacheKey: `test-case-${uuidv4()}`,
|
||||
containerHookFiles: `debug-cache`,
|
||||
orchestratorBranch: `orchestrator-develop`,
|
||||
orchestratorBranch: `main`,
|
||||
orchestratorDebug: true,
|
||||
};
|
||||
|
||||
|
||||
@@ -33,8 +33,7 @@ if [ -n "$(git ls-remote --heads "$REPO" "$BRANCH" 2>/dev/null)" ]; then
|
||||
git clone -q -b "$BRANCH" "$REPO" /builder
|
||||
else
|
||||
echo "Remote branch $BRANCH not found in $REPO; falling back to a known branch"
|
||||
git clone -q -b orchestrator-develop "$REPO" /builder \
|
||||
|| git clone -q -b main "$REPO" /builder \
|
||||
git clone -q -b main "$REPO" /builder \
|
||||
|| git clone -q "$REPO" /builder
|
||||
fi
|
||||
git clone -q -b ${Orchestrator.buildParameters.branch} ${OrchestratorFolders.targetBuildRepoUrl} /repo
|
||||
|
||||
@@ -99,8 +99,7 @@ if [ -n "$(git ls-remote --heads "$REPO" "$BRANCH" 2>/dev/null)" ]; then
|
||||
git clone -q -b "$BRANCH" "$REPO" "$DEST"
|
||||
else
|
||||
echo "Remote branch $BRANCH not found in $REPO; falling back to a known branch"
|
||||
git clone -q -b orchestrator-develop "$REPO" "$DEST" \
|
||||
|| git clone -q -b main "$REPO" "$DEST" \
|
||||
git clone -q -b main "$REPO" "$DEST" \
|
||||
|| git clone -q "$REPO" "$DEST"
|
||||
fi
|
||||
chmod +x ${builderPath}`;
|
||||
@@ -170,14 +169,13 @@ echo "CACHE_KEY=$CACHE_KEY"`;
|
||||
if ! command -v yarn > /dev/null 2>&1; then printf '#!/bin/sh\nexit 0\n' > /usr/local/bin/yarn && chmod +x /usr/local/bin/yarn; fi
|
||||
# Pipe entrypoint.sh output through log stream to capture Unity build output (including "Build succeeded")
|
||||
{ echo "game ci start"; echo "game ci start" >> /home/job-log.txt; echo "CACHE_KEY=$CACHE_KEY"; echo "$CACHE_KEY"; if [ -n "$LOCKED_WORKSPACE" ]; then echo "Retained Workspace: true"; fi; if [ -n "$LOCKED_WORKSPACE" ] && [ -d "$GITHUB_WORKSPACE/.git" ]; then echo "Retained Workspace Already Exists!"; fi; /entrypoint.sh; } | node ${builderPath} -m remote-cli-log-stream --logFile /home/job-log.txt
|
||||
# Ensure cache directories exist for post-build and S3 upload hooks.
|
||||
# Do NOT create empty placeholder tars — they waste S3 storage and on next
|
||||
# build the pull-cache hook downloads them, giving Unity an empty Library
|
||||
# (no caching benefit). The real tars are created by remote-cli-post-build
|
||||
# via Caching.PushToCache(), and the S3 upload hooks use || true so missing
|
||||
# files are handled gracefully.
|
||||
mkdir -p "/data/cache/$CACHE_KEY/Library"
|
||||
mkdir -p "/data/cache/$CACHE_KEY/build"
|
||||
if [ ! -f "/data/cache/$CACHE_KEY/Library/lib-$BUILD_GUID.tar" ] && [ ! -f "/data/cache/$CACHE_KEY/Library/lib-$BUILD_GUID.tar.lz4" ]; then
|
||||
tar -cf "/data/cache/$CACHE_KEY/Library/lib-$BUILD_GUID.tar" --files-from /dev/null || touch "/data/cache/$CACHE_KEY/Library/lib-$BUILD_GUID.tar"
|
||||
fi
|
||||
if [ ! -f "/data/cache/$CACHE_KEY/build/build-$BUILD_GUID.tar" ] && [ ! -f "/data/cache/$CACHE_KEY/build/build-$BUILD_GUID.tar.lz4" ]; then
|
||||
tar -cf "/data/cache/$CACHE_KEY/build/build-$BUILD_GUID.tar" --files-from /dev/null || touch "/data/cache/$CACHE_KEY/build/build-$BUILD_GUID.tar"
|
||||
fi
|
||||
# Run post-build tasks and capture output
|
||||
# Note: Post-build may clean up the builder directory, so we write output directly to log file
|
||||
# Use set +e to allow the command to fail without exiting the script
|
||||
|
||||
Reference in New Issue
Block a user