mirror of
https://github.com/game-ci/unity-builder.git
synced 2026-06-12 08:53:55 -07:00
feat: Add dynamic provider loader with improved error handling (#734)
* feat: Add dynamic provider loader with improved error handling - Create provider-loader.ts with function-based dynamic import functionality - Update CloudRunner.setupSelectedBuildPlatform to use dynamic loader for unknown providers - Add comprehensive error handling for missing packages and interface validation - Include test coverage for successful loading and error scenarios - Maintain backward compatibility with existing built-in providers - Add ProviderLoader class wrapper for backward compatibility - Support both built-in providers (via switch) and external providers (via dynamic import) * fix: Resolve linting errors in provider loader - Fix TypeError usage instead of Error for type checking - Add missing blank lines for proper code formatting - Fix comment spacing issues * build: Update built artifacts after linting fixes - Rebuild dist/ with latest changes - Include updated provider loader in built bundle - Ensure all changes are reflected in compiled output * build: Update built artifacts after linting fixes - Rebuild dist/ with latest changes - Include updated provider loader in built bundle - Ensure all changes are reflected in compiled output * build: Update built artifacts after linting fixes - Rebuild dist/ with latest changes - Include updated provider loader in built bundle - Ensure all changes are reflected in compiled output * build: Update built artifacts after linting fixes - Rebuild dist/ with latest changes - Include updated provider loader in built bundle - Ensure all changes are reflected in compiled output * fix: Fix AWS job dependencies and remove duplicate localstack tests - Update AWS job to depend on both k8s and localstack jobs - Remove duplicate localstack tests from k8s job (now only runs k8s tests) - Remove unused cloud-runner-localstack job from main integrity check - Fix AWS SDK warnings by using Uint8Array(0) instead of empty string for S3 PutObject - Rename localstack-and-k8s job to k8s job for clarity * feat: Implement provider loader dynamic imports with GitHub URL support - Add URL detection and parsing utilities for GitHub URLs, local paths, and NPM packages - Implement git operations for cloning and updating repositories with local caching - Add automatic update checking mechanism for GitHub repositories - Update provider-loader.ts to support multiple source types with comprehensive error handling - Add comprehensive test coverage for all new functionality - Include complete documentation with usage examples - Support GitHub URLs: https://github.com/user/repo, user/repo@branch - Support local paths: ./path, /absolute/path - Support NPM packages: package-name, @scope/package - Maintain backward compatibility with existing providers - Add fallback mechanisms and interface validation * feat: Implement provider loader dynamic imports with GitHub URL support - Add URL detection and parsing utilities for GitHub URLs, local paths, and NPM packages - Implement git operations for cloning and updating repositories with local caching - Add automatic update checking mechanism for GitHub repositories - Update provider-loader.ts to support multiple source types with comprehensive error handling - Add comprehensive test coverage for all new functionality - Include complete documentation with usage examples - Support GitHub URLs: https://github.com/user/repo, user/repo@branch - Support local paths: ./path, /absolute/path - Support NPM packages: package-name, @scope/package - Maintain backward compatibility with existing providers - Add fallback mechanisms and interface validation * feat: Fix provider-loader tests and URL parser consistency - Fixed provider-loader test failures (constructor validation, module imports) - Fixed provider-url-parser to return consistent base URLs for GitHub sources - Updated error handling to use TypeError consistently - All provider-loader and provider-url-parser tests now pass - Fixed prettier and eslint formatting issues * feat: Implement provider loader dynamic imports with GitHub URL support - Add URL detection and parsing utilities for GitHub URLs, local paths, and NPM packages - Implement git operations for cloning and updating repositories with local caching - Add automatic update checking mechanism for GitHub repositories - Update provider-loader.ts to support multiple source types with comprehensive error handling - Add comprehensive test coverage for all new functionality - Include complete documentation with usage examples - Support GitHub URLs: https://github.com/user/repo, user/repo@branch - Support local paths: ./path, /absolute/path - Support NPM packages: package-name, @scope/package - Maintain backward compatibility with existing providers - Add fallback mechanisms and interface validation * feat: Implement provider loader dynamic imports with GitHub URL support - Add URL detection and parsing utilities for GitHub URLs, local paths, and NPM packages - Implement git operations for cloning and updating repositories with local caching - Add automatic update checking mechanism for GitHub repositories - Update provider-loader.ts to support multiple source types with comprehensive error handling - Add comprehensive test coverage for all new functionality - Include complete documentation with usage examples - Support GitHub URLs: https://github.com/user/repo, user/repo@branch - Support local paths: ./path, /absolute/path - Support NPM packages: package-name, @scope/package - Maintain backward compatibility with existing providers - Add fallback mechanisms and interface validation * m * m
This commit is contained in:
Frostebite
@@ -0,0 +1,214 @@
|
||||
# Provider Loader Dynamic Imports
|
||||
|
||||
The provider loader now supports dynamic loading of providers from multiple sources including local file paths, GitHub repositories, and NPM packages.
|
||||
|
||||
## Features
|
||||
|
||||
- **Local File Paths**: Load providers from relative or absolute file paths
|
||||
- **GitHub URLs**: Clone and load providers from GitHub repositories with automatic updates
|
||||
- **NPM Packages**: Load providers from installed NPM packages
|
||||
- **Automatic Updates**: GitHub repositories are automatically updated when changes are available
|
||||
- **Caching**: Local caching of cloned repositories for improved performance
|
||||
- **Fallback Support**: Graceful fallback to local provider if loading fails
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Loading Built-in Providers
|
||||
|
||||
```typescript
|
||||
import { ProviderLoader } from './provider-loader';
|
||||
|
||||
// Load built-in providers
|
||||
const awsProvider = await ProviderLoader.loadProvider('aws', buildParameters);
|
||||
const k8sProvider = await ProviderLoader.loadProvider('k8s', buildParameters);
|
||||
```
|
||||
|
||||
### Loading Local Providers
|
||||
|
||||
```typescript
|
||||
// Load from relative path
|
||||
const localProvider = await ProviderLoader.loadProvider('./my-local-provider', buildParameters);
|
||||
|
||||
// Load from absolute path
|
||||
const absoluteProvider = await ProviderLoader.loadProvider('/path/to/provider', buildParameters);
|
||||
```
|
||||
|
||||
### Loading GitHub Providers
|
||||
|
||||
```typescript
|
||||
// Load from GitHub URL
|
||||
const githubProvider = await ProviderLoader.loadProvider(
|
||||
'https://github.com/user/my-provider',
|
||||
buildParameters
|
||||
);
|
||||
|
||||
// Load from specific branch
|
||||
const branchProvider = await ProviderLoader.loadProvider(
|
||||
'https://github.com/user/my-provider/tree/develop',
|
||||
buildParameters
|
||||
);
|
||||
|
||||
// Load from specific path in repository
|
||||
const pathProvider = await ProviderLoader.loadProvider(
|
||||
'https://github.com/user/my-provider/tree/main/src/providers',
|
||||
buildParameters
|
||||
);
|
||||
|
||||
// Shorthand notation
|
||||
const shorthandProvider = await ProviderLoader.loadProvider('user/repo', buildParameters);
|
||||
const branchShorthand = await ProviderLoader.loadProvider('user/repo@develop', buildParameters);
|
||||
```
|
||||
|
||||
### Loading NPM Packages
|
||||
|
||||
```typescript
|
||||
// Load from NPM package
|
||||
const npmProvider = await ProviderLoader.loadProvider('my-provider-package', buildParameters);
|
||||
|
||||
// Load from scoped NPM package
|
||||
const scopedProvider = await ProviderLoader.loadProvider('@scope/my-provider', buildParameters);
|
||||
```
|
||||
|
||||
## Provider Interface
|
||||
|
||||
All providers must implement the `ProviderInterface`:
|
||||
|
||||
```typescript
|
||||
interface ProviderInterface {
|
||||
cleanupWorkflow(): Promise<void>;
|
||||
setupWorkflow(buildGuid: string, buildParameters: BuildParameters, branchName: string, defaultSecretsArray: any[]): Promise<void>;
|
||||
runTaskInWorkflow(buildGuid: string, task: string, workingDirectory: string, buildVolumeFolder: string, environmentVariables: any[], secrets: any[]): Promise<string>;
|
||||
garbageCollect(): Promise<void>;
|
||||
listResources(): Promise<ProviderResource[]>;
|
||||
listWorkflow(): Promise<ProviderWorkflow[]>;
|
||||
watchWorkflow(): Promise<void>;
|
||||
}
|
||||
```
|
||||
|
||||
## Example Provider Implementation
|
||||
|
||||
```typescript
|
||||
// my-provider.ts
|
||||
import { ProviderInterface } from './provider-interface';
|
||||
import BuildParameters from './build-parameters';
|
||||
|
||||
export default class MyProvider implements ProviderInterface {
|
||||
constructor(private buildParameters: BuildParameters) {}
|
||||
|
||||
async cleanupWorkflow(): Promise<void> {
|
||||
// Cleanup logic
|
||||
}
|
||||
|
||||
async setupWorkflow(buildGuid: string, buildParameters: BuildParameters, branchName: string, defaultSecretsArray: any[]): Promise<void> {
|
||||
// Setup logic
|
||||
}
|
||||
|
||||
async runTaskInWorkflow(buildGuid: string, task: string, workingDirectory: string, buildVolumeFolder: string, environmentVariables: any[], secrets: any[]): Promise<string> {
|
||||
// Task execution logic
|
||||
return 'Task completed';
|
||||
}
|
||||
|
||||
async garbageCollect(): Promise<void> {
|
||||
// Garbage collection logic
|
||||
}
|
||||
|
||||
async listResources(): Promise<ProviderResource[]> {
|
||||
return [];
|
||||
}
|
||||
|
||||
async listWorkflow(): Promise<ProviderWorkflow[]> {
|
||||
return [];
|
||||
}
|
||||
|
||||
async watchWorkflow(): Promise<void> {
|
||||
// Watch logic
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Utility Methods
|
||||
|
||||
### Analyze Provider Source
|
||||
|
||||
```typescript
|
||||
// Analyze a provider source without loading it
|
||||
const sourceInfo = ProviderLoader.analyzeProviderSource('https://github.com/user/repo');
|
||||
console.log(sourceInfo.type); // 'github'
|
||||
console.log(sourceInfo.owner); // 'user'
|
||||
console.log(sourceInfo.repo); // 'repo'
|
||||
```
|
||||
|
||||
### Clean Up Cache
|
||||
|
||||
```typescript
|
||||
// Clean up old cached repositories (older than 30 days)
|
||||
await ProviderLoader.cleanupCache();
|
||||
|
||||
// Clean up repositories older than 7 days
|
||||
await ProviderLoader.cleanupCache(7);
|
||||
```
|
||||
|
||||
### Get Available Providers
|
||||
|
||||
```typescript
|
||||
// Get list of built-in providers
|
||||
const providers = ProviderLoader.getAvailableProviders();
|
||||
console.log(providers); // ['aws', 'k8s', 'test', 'local-docker', 'local-system', 'local']
|
||||
```
|
||||
|
||||
## Supported URL Formats
|
||||
|
||||
### GitHub URLs
|
||||
- `https://github.com/user/repo`
|
||||
- `https://github.com/user/repo.git`
|
||||
- `https://github.com/user/repo/tree/branch`
|
||||
- `https://github.com/user/repo/tree/branch/path/to/provider`
|
||||
- `git@github.com:user/repo.git`
|
||||
|
||||
### Shorthand GitHub References
|
||||
- `user/repo`
|
||||
- `user/repo@branch`
|
||||
- `user/repo@branch/path/to/provider`
|
||||
|
||||
### Local Paths
|
||||
- `./relative/path`
|
||||
- `../relative/path`
|
||||
- `/absolute/path`
|
||||
- `C:\\path\\to\\provider` (Windows)
|
||||
|
||||
### NPM Packages
|
||||
- `package-name`
|
||||
- `@scope/package-name`
|
||||
|
||||
## Caching
|
||||
|
||||
GitHub repositories are automatically cached in the `.provider-cache` directory. The cache key is generated based on the repository owner, name, and branch. This ensures that:
|
||||
|
||||
1. Repositories are only cloned once
|
||||
2. Updates are checked and applied automatically
|
||||
3. Performance is improved for repeated loads
|
||||
4. Storage is managed efficiently
|
||||
|
||||
## Error Handling
|
||||
|
||||
The provider loader includes comprehensive error handling:
|
||||
|
||||
- **Missing packages**: Clear error messages when providers cannot be found
|
||||
- **Interface validation**: Ensures providers implement the required interface
|
||||
- **Git operations**: Handles network issues and repository access problems
|
||||
- **Fallback mechanism**: Falls back to local provider if loading fails
|
||||
|
||||
## Configuration
|
||||
|
||||
The provider loader can be configured through environment variables:
|
||||
|
||||
- `PROVIDER_CACHE_DIR`: Custom cache directory (default: `.provider-cache`)
|
||||
- `GIT_TIMEOUT`: Git operation timeout in milliseconds (default: 30000)
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use specific branches**: Always specify the branch when loading from GitHub
|
||||
2. **Implement proper error handling**: Wrap provider loading in try-catch blocks
|
||||
3. **Clean up regularly**: Use the cleanup utility to manage cache size
|
||||
4. **Test locally first**: Test providers locally before deploying
|
||||
5. **Use semantic versioning**: Tag your provider repositories for stable versions
|
||||
Reference in New Issue
Block a user