mirror of
https://github.com/wshobson/agents.git
synced 2026-03-18 09:37:15 +00:00
feat: add 5 new specialized agents with 20 skills
Add domain expert agents with comprehensive skill sets: - service-mesh-expert (cloud-infrastructure): Istio/Linkerd patterns, mTLS, observability - event-sourcing-architect (backend-development): CQRS, event stores, projections, sagas - vector-database-engineer (llm-application-dev): embeddings, similarity search, hybrid search - monorepo-architect (developer-essentials): Nx, Turborepo, Bazel, pnpm workspaces - threat-modeling-expert (security-scanning): STRIDE, attack trees, security requirements Update all documentation to reflect correct counts: - 67 plugins, 99 agents, 107 skills, 71 commands
This commit is contained in:
Seth Hobson
44
plugins/developer-essentials/agents/monorepo-architect.md
Normal file
44
plugins/developer-essentials/agents/monorepo-architect.md
Normal file
@@ -0,0 +1,44 @@
|
||||
# Monorepo Architect
|
||||
|
||||
Expert in monorepo architecture, build systems, and dependency management at scale. Masters Nx, Turborepo, Bazel, and Lerna for efficient multi-project development. Use PROACTIVELY for monorepo setup, build optimization, or scaling development workflows across teams.
|
||||
|
||||
## Capabilities
|
||||
|
||||
- Monorepo tool selection (Nx, Turborepo, Bazel, Lerna)
|
||||
- Workspace configuration and project structure
|
||||
- Build caching (local and remote)
|
||||
- Dependency graph management
|
||||
- Affected/changed detection for CI optimization
|
||||
- Code sharing and library extraction
|
||||
- Task orchestration and parallelization
|
||||
|
||||
## When to Use
|
||||
|
||||
- Setting up a new monorepo from scratch
|
||||
- Migrating from polyrepo to monorepo
|
||||
- Optimizing slow CI/CD pipelines
|
||||
- Sharing code between multiple applications
|
||||
- Managing dependencies across projects
|
||||
- Implementing consistent tooling across teams
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Assess codebase size and team structure
|
||||
2. Select appropriate monorepo tooling
|
||||
3. Design workspace and project structure
|
||||
4. Configure build caching strategy
|
||||
5. Set up affected/changed detection
|
||||
6. Implement task pipelines
|
||||
7. Configure remote caching for CI
|
||||
8. Document conventions and workflows
|
||||
|
||||
## Best Practices
|
||||
|
||||
- Start with clear project boundaries
|
||||
- Use consistent naming conventions
|
||||
- Implement remote caching early
|
||||
- Keep shared libraries focused
|
||||
- Use tags for dependency constraints
|
||||
- Automate dependency updates
|
||||
- Document the dependency graph
|
||||
- Set up code ownership rules
|
||||
@@ -0,0 +1,385 @@
|
||||
---
|
||||
name: bazel-build-optimization
|
||||
description: Optimize Bazel builds for large-scale monorepos. Use when configuring Bazel, implementing remote execution, or optimizing build performance for enterprise codebases.
|
||||
---
|
||||
|
||||
# Bazel Build Optimization
|
||||
|
||||
Production patterns for Bazel in large-scale monorepos.
|
||||
|
||||
## When to Use This Skill
|
||||
|
||||
- Setting up Bazel for monorepos
|
||||
- Configuring remote caching/execution
|
||||
- Optimizing build times
|
||||
- Writing custom Bazel rules
|
||||
- Debugging build issues
|
||||
- Migrating to Bazel
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### 1. Bazel Architecture
|
||||
|
||||
```
|
||||
workspace/
|
||||
├── WORKSPACE.bazel # External dependencies
|
||||
├── .bazelrc # Build configurations
|
||||
├── .bazelversion # Bazel version
|
||||
├── BUILD.bazel # Root build file
|
||||
├── apps/
|
||||
│ └── web/
|
||||
│ └── BUILD.bazel
|
||||
├── libs/
|
||||
│ └── utils/
|
||||
│ └── BUILD.bazel
|
||||
└── tools/
|
||||
└── bazel/
|
||||
└── rules/
|
||||
```
|
||||
|
||||
### 2. Key Concepts
|
||||
|
||||
| Concept | Description |
|
||||
|---------|-------------|
|
||||
| **Target** | Buildable unit (library, binary, test) |
|
||||
| **Package** | Directory with BUILD file |
|
||||
| **Label** | Target identifier `//path/to:target` |
|
||||
| **Rule** | Defines how to build a target |
|
||||
| **Aspect** | Cross-cutting build behavior |
|
||||
|
||||
## Templates
|
||||
|
||||
### Template 1: WORKSPACE Configuration
|
||||
|
||||
```python
|
||||
# WORKSPACE.bazel
|
||||
workspace(name = "myproject")
|
||||
|
||||
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
|
||||
|
||||
# Rules for JavaScript/TypeScript
|
||||
http_archive(
|
||||
name = "aspect_rules_js",
|
||||
sha256 = "...",
|
||||
strip_prefix = "rules_js-1.34.0",
|
||||
url = "https://github.com/aspect-build/rules_js/releases/download/v1.34.0/rules_js-v1.34.0.tar.gz",
|
||||
)
|
||||
|
||||
load("@aspect_rules_js//js:repositories.bzl", "rules_js_dependencies")
|
||||
rules_js_dependencies()
|
||||
|
||||
load("@rules_nodejs//nodejs:repositories.bzl", "nodejs_register_toolchains")
|
||||
nodejs_register_toolchains(
|
||||
name = "nodejs",
|
||||
node_version = "20.9.0",
|
||||
)
|
||||
|
||||
load("@aspect_rules_js//npm:repositories.bzl", "npm_translate_lock")
|
||||
npm_translate_lock(
|
||||
name = "npm",
|
||||
pnpm_lock = "//:pnpm-lock.yaml",
|
||||
verify_node_modules_ignored = "//:.bazelignore",
|
||||
)
|
||||
|
||||
load("@npm//:repositories.bzl", "npm_repositories")
|
||||
npm_repositories()
|
||||
|
||||
# Rules for Python
|
||||
http_archive(
|
||||
name = "rules_python",
|
||||
sha256 = "...",
|
||||
strip_prefix = "rules_python-0.27.0",
|
||||
url = "https://github.com/bazelbuild/rules_python/releases/download/0.27.0/rules_python-0.27.0.tar.gz",
|
||||
)
|
||||
|
||||
load("@rules_python//python:repositories.bzl", "py_repositories")
|
||||
py_repositories()
|
||||
```
|
||||
|
||||
### Template 2: .bazelrc Configuration
|
||||
|
||||
```bash
|
||||
# .bazelrc
|
||||
|
||||
# Build settings
|
||||
build --enable_platform_specific_config
|
||||
build --incompatible_enable_cc_toolchain_resolution
|
||||
build --experimental_strict_conflict_checks
|
||||
|
||||
# Performance
|
||||
build --jobs=auto
|
||||
build --local_cpu_resources=HOST_CPUS*.75
|
||||
build --local_ram_resources=HOST_RAM*.75
|
||||
|
||||
# Caching
|
||||
build --disk_cache=~/.cache/bazel-disk
|
||||
build --repository_cache=~/.cache/bazel-repo
|
||||
|
||||
# Remote caching (optional)
|
||||
build:remote-cache --remote_cache=grpcs://cache.example.com
|
||||
build:remote-cache --remote_upload_local_results=true
|
||||
build:remote-cache --remote_timeout=3600
|
||||
|
||||
# Remote execution (optional)
|
||||
build:remote-exec --remote_executor=grpcs://remote.example.com
|
||||
build:remote-exec --remote_instance_name=projects/myproject/instances/default
|
||||
build:remote-exec --jobs=500
|
||||
|
||||
# Platform configurations
|
||||
build:linux --platforms=//platforms:linux_x86_64
|
||||
build:macos --platforms=//platforms:macos_arm64
|
||||
|
||||
# CI configuration
|
||||
build:ci --config=remote-cache
|
||||
build:ci --build_metadata=ROLE=CI
|
||||
build:ci --bes_results_url=https://results.example.com/invocation/
|
||||
build:ci --bes_backend=grpcs://bes.example.com
|
||||
|
||||
# Test settings
|
||||
test --test_output=errors
|
||||
test --test_summary=detailed
|
||||
|
||||
# Coverage
|
||||
coverage --combined_report=lcov
|
||||
coverage --instrumentation_filter="//..."
|
||||
|
||||
# Convenience aliases
|
||||
build:opt --compilation_mode=opt
|
||||
build:dbg --compilation_mode=dbg
|
||||
|
||||
# Import user settings
|
||||
try-import %workspace%/user.bazelrc
|
||||
```
|
||||
|
||||
### Template 3: TypeScript Library BUILD
|
||||
|
||||
```python
|
||||
# libs/utils/BUILD.bazel
|
||||
load("@aspect_rules_ts//ts:defs.bzl", "ts_project")
|
||||
load("@aspect_rules_js//js:defs.bzl", "js_library")
|
||||
load("@npm//:defs.bzl", "npm_link_all_packages")
|
||||
|
||||
npm_link_all_packages(name = "node_modules")
|
||||
|
||||
ts_project(
|
||||
name = "utils_ts",
|
||||
srcs = glob(["src/**/*.ts"]),
|
||||
declaration = True,
|
||||
source_map = True,
|
||||
tsconfig = "//:tsconfig.json",
|
||||
deps = [
|
||||
":node_modules/@types/node",
|
||||
],
|
||||
)
|
||||
|
||||
js_library(
|
||||
name = "utils",
|
||||
srcs = [":utils_ts"],
|
||||
visibility = ["//visibility:public"],
|
||||
)
|
||||
|
||||
# Tests
|
||||
load("@aspect_rules_jest//jest:defs.bzl", "jest_test")
|
||||
|
||||
jest_test(
|
||||
name = "utils_test",
|
||||
config = "//:jest.config.js",
|
||||
data = [
|
||||
":utils",
|
||||
"//:node_modules/jest",
|
||||
],
|
||||
node_modules = "//:node_modules",
|
||||
)
|
||||
```
|
||||
|
||||
### Template 4: Python Library BUILD
|
||||
|
||||
```python
|
||||
# libs/ml/BUILD.bazel
|
||||
load("@rules_python//python:defs.bzl", "py_library", "py_test", "py_binary")
|
||||
load("@pip//:requirements.bzl", "requirement")
|
||||
|
||||
py_library(
|
||||
name = "ml",
|
||||
srcs = glob(["src/**/*.py"]),
|
||||
deps = [
|
||||
requirement("numpy"),
|
||||
requirement("pandas"),
|
||||
requirement("scikit-learn"),
|
||||
"//libs/utils:utils_py",
|
||||
],
|
||||
visibility = ["//visibility:public"],
|
||||
)
|
||||
|
||||
py_test(
|
||||
name = "ml_test",
|
||||
srcs = glob(["tests/**/*.py"]),
|
||||
deps = [
|
||||
":ml",
|
||||
requirement("pytest"),
|
||||
],
|
||||
size = "medium",
|
||||
timeout = "moderate",
|
||||
)
|
||||
|
||||
py_binary(
|
||||
name = "train",
|
||||
srcs = ["train.py"],
|
||||
deps = [":ml"],
|
||||
data = ["//data:training_data"],
|
||||
)
|
||||
```
|
||||
|
||||
### Template 5: Custom Rule for Docker
|
||||
|
||||
```python
|
||||
# tools/bazel/rules/docker.bzl
|
||||
def _docker_image_impl(ctx):
|
||||
dockerfile = ctx.file.dockerfile
|
||||
base_image = ctx.attr.base_image
|
||||
layers = ctx.files.layers
|
||||
|
||||
# Build the image
|
||||
output = ctx.actions.declare_file(ctx.attr.name + ".tar")
|
||||
|
||||
args = ctx.actions.args()
|
||||
args.add("--dockerfile", dockerfile)
|
||||
args.add("--output", output)
|
||||
args.add("--base", base_image)
|
||||
args.add_all("--layer", layers)
|
||||
|
||||
ctx.actions.run(
|
||||
inputs = [dockerfile] + layers,
|
||||
outputs = [output],
|
||||
executable = ctx.executable._builder,
|
||||
arguments = [args],
|
||||
mnemonic = "DockerBuild",
|
||||
progress_message = "Building Docker image %s" % ctx.label,
|
||||
)
|
||||
|
||||
return [DefaultInfo(files = depset([output]))]
|
||||
|
||||
docker_image = rule(
|
||||
implementation = _docker_image_impl,
|
||||
attrs = {
|
||||
"dockerfile": attr.label(
|
||||
allow_single_file = [".dockerfile", "Dockerfile"],
|
||||
mandatory = True,
|
||||
),
|
||||
"base_image": attr.string(mandatory = True),
|
||||
"layers": attr.label_list(allow_files = True),
|
||||
"_builder": attr.label(
|
||||
default = "//tools/docker:builder",
|
||||
executable = True,
|
||||
cfg = "exec",
|
||||
),
|
||||
},
|
||||
)
|
||||
```
|
||||
|
||||
### Template 6: Query and Dependency Analysis
|
||||
|
||||
```bash
|
||||
# Find all dependencies of a target
|
||||
bazel query "deps(//apps/web:web)"
|
||||
|
||||
# Find reverse dependencies (what depends on this)
|
||||
bazel query "rdeps(//..., //libs/utils:utils)"
|
||||
|
||||
# Find all targets in a package
|
||||
bazel query "//libs/..."
|
||||
|
||||
# Find changed targets since commit
|
||||
bazel query "rdeps(//..., set($(git diff --name-only HEAD~1 | sed 's/.*/"&"/' | tr '\n' ' ')))"
|
||||
|
||||
# Generate dependency graph
|
||||
bazel query "deps(//apps/web:web)" --output=graph | dot -Tpng > deps.png
|
||||
|
||||
# Find all test targets
|
||||
bazel query "kind('.*_test', //...)"
|
||||
|
||||
# Find targets with specific tag
|
||||
bazel query "attr(tags, 'integration', //...)"
|
||||
|
||||
# Compute build graph size
|
||||
bazel query "deps(//...)" --output=package | wc -l
|
||||
```
|
||||
|
||||
### Template 7: Remote Execution Setup
|
||||
|
||||
```python
|
||||
# platforms/BUILD.bazel
|
||||
platform(
|
||||
name = "linux_x86_64",
|
||||
constraint_values = [
|
||||
"@platforms//os:linux",
|
||||
"@platforms//cpu:x86_64",
|
||||
],
|
||||
exec_properties = {
|
||||
"container-image": "docker://gcr.io/myproject/bazel-worker:latest",
|
||||
"OSFamily": "Linux",
|
||||
},
|
||||
)
|
||||
|
||||
platform(
|
||||
name = "remote_linux",
|
||||
parents = [":linux_x86_64"],
|
||||
exec_properties = {
|
||||
"Pool": "default",
|
||||
"dockerNetwork": "standard",
|
||||
},
|
||||
)
|
||||
|
||||
# toolchains/BUILD.bazel
|
||||
toolchain(
|
||||
name = "cc_toolchain_linux",
|
||||
exec_compatible_with = [
|
||||
"@platforms//os:linux",
|
||||
"@platforms//cpu:x86_64",
|
||||
],
|
||||
target_compatible_with = [
|
||||
"@platforms//os:linux",
|
||||
"@platforms//cpu:x86_64",
|
||||
],
|
||||
toolchain = "@remotejdk11_linux//:jdk",
|
||||
toolchain_type = "@bazel_tools//tools/jdk:runtime_toolchain_type",
|
||||
)
|
||||
```
|
||||
|
||||
## Performance Optimization
|
||||
|
||||
```bash
|
||||
# Profile build
|
||||
bazel build //... --profile=profile.json
|
||||
bazel analyze-profile profile.json
|
||||
|
||||
# Identify slow actions
|
||||
bazel build //... --execution_log_json_file=exec_log.json
|
||||
|
||||
# Memory profiling
|
||||
bazel build //... --memory_profile=memory.json
|
||||
|
||||
# Skip analysis cache
|
||||
bazel build //... --notrack_incremental_state
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Do's
|
||||
- **Use fine-grained targets** - Better caching
|
||||
- **Pin dependencies** - Reproducible builds
|
||||
- **Enable remote caching** - Share build artifacts
|
||||
- **Use visibility wisely** - Enforce architecture
|
||||
- **Write BUILD files per directory** - Standard convention
|
||||
|
||||
### Don'ts
|
||||
- **Don't use glob for deps** - Explicit is better
|
||||
- **Don't commit bazel-* dirs** - Add to .gitignore
|
||||
- **Don't skip WORKSPACE setup** - Foundation of build
|
||||
- **Don't ignore build warnings** - Technical debt
|
||||
|
||||
## Resources
|
||||
|
||||
- [Bazel Documentation](https://bazel.build/docs)
|
||||
- [Bazel Remote Execution](https://bazel.build/docs/remote-execution)
|
||||
- [rules_js](https://github.com/aspect-build/rules_js)
|
||||
@@ -0,0 +1,452 @@
|
||||
---
|
||||
name: nx-workspace-patterns
|
||||
description: Configure and optimize Nx monorepo workspaces. Use when setting up Nx, configuring project boundaries, optimizing build caching, or implementing affected commands.
|
||||
---
|
||||
|
||||
# Nx Workspace Patterns
|
||||
|
||||
Production patterns for Nx monorepo management.
|
||||
|
||||
## When to Use This Skill
|
||||
|
||||
- Setting up new Nx workspaces
|
||||
- Configuring project boundaries
|
||||
- Optimizing CI with affected commands
|
||||
- Implementing remote caching
|
||||
- Managing dependencies between projects
|
||||
- Migrating to Nx
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### 1. Nx Architecture
|
||||
|
||||
```
|
||||
workspace/
|
||||
├── apps/ # Deployable applications
|
||||
│ ├── web/
|
||||
│ └── api/
|
||||
├── libs/ # Shared libraries
|
||||
│ ├── shared/
|
||||
│ │ ├── ui/
|
||||
│ │ └── utils/
|
||||
│ └── feature/
|
||||
│ ├── auth/
|
||||
│ └── dashboard/
|
||||
├── tools/ # Custom executors/generators
|
||||
├── nx.json # Nx configuration
|
||||
└── workspace.json # Project configuration
|
||||
```
|
||||
|
||||
### 2. Library Types
|
||||
|
||||
| Type | Purpose | Example |
|
||||
|------|---------|---------|
|
||||
| **feature** | Smart components, business logic | `feature-auth` |
|
||||
| **ui** | Presentational components | `ui-buttons` |
|
||||
| **data-access** | API calls, state management | `data-access-users` |
|
||||
| **util** | Pure functions, helpers | `util-formatting` |
|
||||
| **shell** | App bootstrapping | `shell-web` |
|
||||
|
||||
## Templates
|
||||
|
||||
### Template 1: nx.json Configuration
|
||||
|
||||
```json
|
||||
{
|
||||
"$schema": "./node_modules/nx/schemas/nx-schema.json",
|
||||
"npmScope": "myorg",
|
||||
"affected": {
|
||||
"defaultBase": "main"
|
||||
},
|
||||
"tasksRunnerOptions": {
|
||||
"default": {
|
||||
"runner": "nx/tasks-runners/default",
|
||||
"options": {
|
||||
"cacheableOperations": [
|
||||
"build",
|
||||
"lint",
|
||||
"test",
|
||||
"e2e",
|
||||
"build-storybook"
|
||||
],
|
||||
"parallel": 3
|
||||
}
|
||||
}
|
||||
},
|
||||
"targetDefaults": {
|
||||
"build": {
|
||||
"dependsOn": ["^build"],
|
||||
"inputs": ["production", "^production"],
|
||||
"cache": true
|
||||
},
|
||||
"test": {
|
||||
"inputs": ["default", "^production", "{workspaceRoot}/jest.preset.js"],
|
||||
"cache": true
|
||||
},
|
||||
"lint": {
|
||||
"inputs": ["default", "{workspaceRoot}/.eslintrc.json"],
|
||||
"cache": true
|
||||
},
|
||||
"e2e": {
|
||||
"inputs": ["default", "^production"],
|
||||
"cache": true
|
||||
}
|
||||
},
|
||||
"namedInputs": {
|
||||
"default": ["{projectRoot}/**/*", "sharedGlobals"],
|
||||
"production": [
|
||||
"default",
|
||||
"!{projectRoot}/**/?(*.)+(spec|test).[jt]s?(x)?(.snap)",
|
||||
"!{projectRoot}/tsconfig.spec.json",
|
||||
"!{projectRoot}/jest.config.[jt]s",
|
||||
"!{projectRoot}/.eslintrc.json"
|
||||
],
|
||||
"sharedGlobals": [
|
||||
"{workspaceRoot}/babel.config.json",
|
||||
"{workspaceRoot}/tsconfig.base.json"
|
||||
]
|
||||
},
|
||||
"generators": {
|
||||
"@nx/react": {
|
||||
"application": {
|
||||
"style": "css",
|
||||
"linter": "eslint",
|
||||
"bundler": "webpack"
|
||||
},
|
||||
"library": {
|
||||
"style": "css",
|
||||
"linter": "eslint"
|
||||
},
|
||||
"component": {
|
||||
"style": "css"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Template 2: Project Configuration
|
||||
|
||||
```json
|
||||
// apps/web/project.json
|
||||
{
|
||||
"name": "web",
|
||||
"$schema": "../../node_modules/nx/schemas/project-schema.json",
|
||||
"sourceRoot": "apps/web/src",
|
||||
"projectType": "application",
|
||||
"tags": ["type:app", "scope:web"],
|
||||
"targets": {
|
||||
"build": {
|
||||
"executor": "@nx/webpack:webpack",
|
||||
"outputs": ["{options.outputPath}"],
|
||||
"defaultConfiguration": "production",
|
||||
"options": {
|
||||
"compiler": "babel",
|
||||
"outputPath": "dist/apps/web",
|
||||
"index": "apps/web/src/index.html",
|
||||
"main": "apps/web/src/main.tsx",
|
||||
"tsConfig": "apps/web/tsconfig.app.json",
|
||||
"assets": ["apps/web/src/assets"],
|
||||
"styles": ["apps/web/src/styles.css"]
|
||||
},
|
||||
"configurations": {
|
||||
"development": {
|
||||
"extractLicenses": false,
|
||||
"optimization": false,
|
||||
"sourceMap": true
|
||||
},
|
||||
"production": {
|
||||
"optimization": true,
|
||||
"outputHashing": "all",
|
||||
"sourceMap": false,
|
||||
"extractLicenses": true
|
||||
}
|
||||
}
|
||||
},
|
||||
"serve": {
|
||||
"executor": "@nx/webpack:dev-server",
|
||||
"defaultConfiguration": "development",
|
||||
"options": {
|
||||
"buildTarget": "web:build"
|
||||
},
|
||||
"configurations": {
|
||||
"development": {
|
||||
"buildTarget": "web:build:development"
|
||||
},
|
||||
"production": {
|
||||
"buildTarget": "web:build:production"
|
||||
}
|
||||
}
|
||||
},
|
||||
"test": {
|
||||
"executor": "@nx/jest:jest",
|
||||
"outputs": ["{workspaceRoot}/coverage/{projectRoot}"],
|
||||
"options": {
|
||||
"jestConfig": "apps/web/jest.config.ts",
|
||||
"passWithNoTests": true
|
||||
}
|
||||
},
|
||||
"lint": {
|
||||
"executor": "@nx/eslint:lint",
|
||||
"outputs": ["{options.outputFile}"],
|
||||
"options": {
|
||||
"lintFilePatterns": ["apps/web/**/*.{ts,tsx,js,jsx}"]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Template 3: Module Boundary Rules
|
||||
|
||||
```json
|
||||
// .eslintrc.json
|
||||
{
|
||||
"root": true,
|
||||
"ignorePatterns": ["**/*"],
|
||||
"plugins": ["@nx"],
|
||||
"overrides": [
|
||||
{
|
||||
"files": ["*.ts", "*.tsx", "*.js", "*.jsx"],
|
||||
"rules": {
|
||||
"@nx/enforce-module-boundaries": [
|
||||
"error",
|
||||
{
|
||||
"enforceBuildableLibDependency": true,
|
||||
"allow": [],
|
||||
"depConstraints": [
|
||||
{
|
||||
"sourceTag": "type:app",
|
||||
"onlyDependOnLibsWithTags": [
|
||||
"type:feature",
|
||||
"type:ui",
|
||||
"type:data-access",
|
||||
"type:util"
|
||||
]
|
||||
},
|
||||
{
|
||||
"sourceTag": "type:feature",
|
||||
"onlyDependOnLibsWithTags": [
|
||||
"type:ui",
|
||||
"type:data-access",
|
||||
"type:util"
|
||||
]
|
||||
},
|
||||
{
|
||||
"sourceTag": "type:ui",
|
||||
"onlyDependOnLibsWithTags": ["type:ui", "type:util"]
|
||||
},
|
||||
{
|
||||
"sourceTag": "type:data-access",
|
||||
"onlyDependOnLibsWithTags": ["type:data-access", "type:util"]
|
||||
},
|
||||
{
|
||||
"sourceTag": "type:util",
|
||||
"onlyDependOnLibsWithTags": ["type:util"]
|
||||
},
|
||||
{
|
||||
"sourceTag": "scope:web",
|
||||
"onlyDependOnLibsWithTags": ["scope:web", "scope:shared"]
|
||||
},
|
||||
{
|
||||
"sourceTag": "scope:api",
|
||||
"onlyDependOnLibsWithTags": ["scope:api", "scope:shared"]
|
||||
},
|
||||
{
|
||||
"sourceTag": "scope:shared",
|
||||
"onlyDependOnLibsWithTags": ["scope:shared"]
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Template 4: Custom Generator
|
||||
|
||||
```typescript
|
||||
// tools/generators/feature-lib/index.ts
|
||||
import {
|
||||
Tree,
|
||||
formatFiles,
|
||||
generateFiles,
|
||||
joinPathFragments,
|
||||
names,
|
||||
readProjectConfiguration,
|
||||
} from '@nx/devkit';
|
||||
import { libraryGenerator } from '@nx/react';
|
||||
|
||||
interface FeatureLibraryGeneratorSchema {
|
||||
name: string;
|
||||
scope: string;
|
||||
directory?: string;
|
||||
}
|
||||
|
||||
export default async function featureLibraryGenerator(
|
||||
tree: Tree,
|
||||
options: FeatureLibraryGeneratorSchema
|
||||
) {
|
||||
const { name, scope, directory } = options;
|
||||
const projectDirectory = directory
|
||||
? `${directory}/${name}`
|
||||
: `libs/${scope}/feature-${name}`;
|
||||
|
||||
// Generate base library
|
||||
await libraryGenerator(tree, {
|
||||
name: `feature-${name}`,
|
||||
directory: projectDirectory,
|
||||
tags: `type:feature,scope:${scope}`,
|
||||
style: 'css',
|
||||
skipTsConfig: false,
|
||||
skipFormat: true,
|
||||
unitTestRunner: 'jest',
|
||||
linter: 'eslint',
|
||||
});
|
||||
|
||||
// Add custom files
|
||||
const projectConfig = readProjectConfiguration(tree, `${scope}-feature-${name}`);
|
||||
const projectNames = names(name);
|
||||
|
||||
generateFiles(
|
||||
tree,
|
||||
joinPathFragments(__dirname, 'files'),
|
||||
projectConfig.sourceRoot,
|
||||
{
|
||||
...projectNames,
|
||||
scope,
|
||||
tmpl: '',
|
||||
}
|
||||
);
|
||||
|
||||
await formatFiles(tree);
|
||||
}
|
||||
```
|
||||
|
||||
### Template 5: CI Configuration with Affected
|
||||
|
||||
```yaml
|
||||
# .github/workflows/ci.yml
|
||||
name: CI
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
branches: [main]
|
||||
|
||||
env:
|
||||
NX_CLOUD_ACCESS_TOKEN: ${{ secrets.NX_CLOUD_ACCESS_TOKEN }}
|
||||
|
||||
jobs:
|
||||
main:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 20
|
||||
cache: 'npm'
|
||||
|
||||
- name: Install dependencies
|
||||
run: npm ci
|
||||
|
||||
- name: Derive SHAs for affected commands
|
||||
uses: nrwl/nx-set-shas@v4
|
||||
|
||||
- name: Run affected lint
|
||||
run: npx nx affected -t lint --parallel=3
|
||||
|
||||
- name: Run affected test
|
||||
run: npx nx affected -t test --parallel=3 --configuration=ci
|
||||
|
||||
- name: Run affected build
|
||||
run: npx nx affected -t build --parallel=3
|
||||
|
||||
- name: Run affected e2e
|
||||
run: npx nx affected -t e2e --parallel=1
|
||||
```
|
||||
|
||||
### Template 6: Remote Caching Setup
|
||||
|
||||
```typescript
|
||||
// nx.json with Nx Cloud
|
||||
{
|
||||
"tasksRunnerOptions": {
|
||||
"default": {
|
||||
"runner": "nx-cloud",
|
||||
"options": {
|
||||
"cacheableOperations": ["build", "lint", "test", "e2e"],
|
||||
"accessToken": "your-nx-cloud-token",
|
||||
"parallel": 3,
|
||||
"cacheDirectory": ".nx/cache"
|
||||
}
|
||||
}
|
||||
},
|
||||
"nxCloudAccessToken": "your-nx-cloud-token"
|
||||
}
|
||||
|
||||
// Self-hosted cache with S3
|
||||
{
|
||||
"tasksRunnerOptions": {
|
||||
"default": {
|
||||
"runner": "@nx-aws-cache/nx-aws-cache",
|
||||
"options": {
|
||||
"cacheableOperations": ["build", "lint", "test"],
|
||||
"awsRegion": "us-east-1",
|
||||
"awsBucket": "my-nx-cache-bucket",
|
||||
"awsProfile": "default"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Common Commands
|
||||
|
||||
```bash
|
||||
# Generate new library
|
||||
nx g @nx/react:lib feature-auth --directory=libs/web --tags=type:feature,scope:web
|
||||
|
||||
# Run affected tests
|
||||
nx affected -t test --base=main
|
||||
|
||||
# View dependency graph
|
||||
nx graph
|
||||
|
||||
# Run specific project
|
||||
nx build web --configuration=production
|
||||
|
||||
# Reset cache
|
||||
nx reset
|
||||
|
||||
# Run migrations
|
||||
nx migrate latest
|
||||
nx migrate --run-migrations
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Do's
|
||||
- **Use tags consistently** - Enforce with module boundaries
|
||||
- **Enable caching early** - Significant CI savings
|
||||
- **Keep libs focused** - Single responsibility
|
||||
- **Use generators** - Ensure consistency
|
||||
- **Document boundaries** - Help new developers
|
||||
|
||||
### Don'ts
|
||||
- **Don't create circular deps** - Graph should be acyclic
|
||||
- **Don't skip affected** - Test only what changed
|
||||
- **Don't ignore boundaries** - Tech debt accumulates
|
||||
- **Don't over-granularize** - Balance lib count
|
||||
|
||||
## Resources
|
||||
|
||||
- [Nx Documentation](https://nx.dev/getting-started/intro)
|
||||
- [Module Boundaries](https://nx.dev/core-features/enforce-module-boundaries)
|
||||
- [Nx Cloud](https://nx.app/)
|
||||
407
plugins/developer-essentials/skills/turborepo-caching/SKILL.md
Normal file
407
plugins/developer-essentials/skills/turborepo-caching/SKILL.md
Normal file
@@ -0,0 +1,407 @@
|
||||
---
|
||||
name: turborepo-caching
|
||||
description: Configure Turborepo for efficient monorepo builds with local and remote caching. Use when setting up Turborepo, optimizing build pipelines, or implementing distributed caching.
|
||||
---
|
||||
|
||||
# Turborepo Caching
|
||||
|
||||
Production patterns for Turborepo build optimization.
|
||||
|
||||
## When to Use This Skill
|
||||
|
||||
- Setting up new Turborepo projects
|
||||
- Configuring build pipelines
|
||||
- Implementing remote caching
|
||||
- Optimizing CI/CD performance
|
||||
- Migrating from other monorepo tools
|
||||
- Debugging cache misses
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### 1. Turborepo Architecture
|
||||
|
||||
```
|
||||
Workspace Root/
|
||||
├── apps/
|
||||
│ ├── web/
|
||||
│ │ └── package.json
|
||||
│ └── docs/
|
||||
│ └── package.json
|
||||
├── packages/
|
||||
│ ├── ui/
|
||||
│ │ └── package.json
|
||||
│ └── config/
|
||||
│ └── package.json
|
||||
├── turbo.json
|
||||
└── package.json
|
||||
```
|
||||
|
||||
### 2. Pipeline Concepts
|
||||
|
||||
| Concept | Description |
|
||||
|---------|-------------|
|
||||
| **dependsOn** | Tasks that must complete first |
|
||||
| **cache** | Whether to cache outputs |
|
||||
| **outputs** | Files to cache |
|
||||
| **inputs** | Files that affect cache key |
|
||||
| **persistent** | Long-running tasks (dev servers) |
|
||||
|
||||
## Templates
|
||||
|
||||
### Template 1: turbo.json Configuration
|
||||
|
||||
```json
|
||||
{
|
||||
"$schema": "https://turbo.build/schema.json",
|
||||
"globalDependencies": [
|
||||
".env",
|
||||
".env.local"
|
||||
],
|
||||
"globalEnv": [
|
||||
"NODE_ENV",
|
||||
"VERCEL_URL"
|
||||
],
|
||||
"pipeline": {
|
||||
"build": {
|
||||
"dependsOn": ["^build"],
|
||||
"outputs": [
|
||||
"dist/**",
|
||||
".next/**",
|
||||
"!.next/cache/**"
|
||||
],
|
||||
"env": [
|
||||
"API_URL",
|
||||
"NEXT_PUBLIC_*"
|
||||
]
|
||||
},
|
||||
"test": {
|
||||
"dependsOn": ["build"],
|
||||
"outputs": ["coverage/**"],
|
||||
"inputs": [
|
||||
"src/**/*.tsx",
|
||||
"src/**/*.ts",
|
||||
"test/**/*.ts"
|
||||
]
|
||||
},
|
||||
"lint": {
|
||||
"outputs": [],
|
||||
"cache": true
|
||||
},
|
||||
"typecheck": {
|
||||
"dependsOn": ["^build"],
|
||||
"outputs": []
|
||||
},
|
||||
"dev": {
|
||||
"cache": false,
|
||||
"persistent": true
|
||||
},
|
||||
"clean": {
|
||||
"cache": false
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Template 2: Package-Specific Pipeline
|
||||
|
||||
```json
|
||||
// apps/web/turbo.json
|
||||
{
|
||||
"$schema": "https://turbo.build/schema.json",
|
||||
"extends": ["//"],
|
||||
"pipeline": {
|
||||
"build": {
|
||||
"outputs": [".next/**", "!.next/cache/**"],
|
||||
"env": [
|
||||
"NEXT_PUBLIC_API_URL",
|
||||
"NEXT_PUBLIC_ANALYTICS_ID"
|
||||
]
|
||||
},
|
||||
"test": {
|
||||
"outputs": ["coverage/**"],
|
||||
"inputs": [
|
||||
"src/**",
|
||||
"tests/**",
|
||||
"jest.config.js"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Template 3: Remote Caching with Vercel
|
||||
|
||||
```bash
|
||||
# Login to Vercel
|
||||
npx turbo login
|
||||
|
||||
# Link to Vercel project
|
||||
npx turbo link
|
||||
|
||||
# Run with remote cache
|
||||
turbo build --remote-only
|
||||
|
||||
# CI environment variables
|
||||
TURBO_TOKEN=your-token
|
||||
TURBO_TEAM=your-team
|
||||
```
|
||||
|
||||
```yaml
|
||||
# .github/workflows/ci.yml
|
||||
name: CI
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
|
||||
env:
|
||||
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
|
||||
jobs:
|
||||
build:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 20
|
||||
cache: 'npm'
|
||||
|
||||
- name: Install dependencies
|
||||
run: npm ci
|
||||
|
||||
- name: Build
|
||||
run: npx turbo build --filter='...[origin/main]'
|
||||
|
||||
- name: Test
|
||||
run: npx turbo test --filter='...[origin/main]'
|
||||
```
|
||||
|
||||
### Template 4: Self-Hosted Remote Cache
|
||||
|
||||
```typescript
|
||||
// Custom remote cache server (Express)
|
||||
import express from 'express';
|
||||
import { createReadStream, createWriteStream } from 'fs';
|
||||
import { mkdir } from 'fs/promises';
|
||||
import { join } from 'path';
|
||||
|
||||
const app = express();
|
||||
const CACHE_DIR = './cache';
|
||||
|
||||
// Get artifact
|
||||
app.get('/v8/artifacts/:hash', async (req, res) => {
|
||||
const { hash } = req.params;
|
||||
const team = req.query.teamId || 'default';
|
||||
const filePath = join(CACHE_DIR, team, hash);
|
||||
|
||||
try {
|
||||
const stream = createReadStream(filePath);
|
||||
stream.pipe(res);
|
||||
} catch {
|
||||
res.status(404).send('Not found');
|
||||
}
|
||||
});
|
||||
|
||||
// Put artifact
|
||||
app.put('/v8/artifacts/:hash', async (req, res) => {
|
||||
const { hash } = req.params;
|
||||
const team = req.query.teamId || 'default';
|
||||
const dir = join(CACHE_DIR, team);
|
||||
const filePath = join(dir, hash);
|
||||
|
||||
await mkdir(dir, { recursive: true });
|
||||
|
||||
const stream = createWriteStream(filePath);
|
||||
req.pipe(stream);
|
||||
|
||||
stream.on('finish', () => {
|
||||
res.json({ urls: [`${req.protocol}://${req.get('host')}/v8/artifacts/${hash}`] });
|
||||
});
|
||||
});
|
||||
|
||||
// Check artifact exists
|
||||
app.head('/v8/artifacts/:hash', async (req, res) => {
|
||||
const { hash } = req.params;
|
||||
const team = req.query.teamId || 'default';
|
||||
const filePath = join(CACHE_DIR, team, hash);
|
||||
|
||||
try {
|
||||
await fs.access(filePath);
|
||||
res.status(200).end();
|
||||
} catch {
|
||||
res.status(404).end();
|
||||
}
|
||||
});
|
||||
|
||||
app.listen(3000);
|
||||
```
|
||||
|
||||
```json
|
||||
// turbo.json for self-hosted cache
|
||||
{
|
||||
"remoteCache": {
|
||||
"signature": false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
```bash
|
||||
# Use self-hosted cache
|
||||
turbo build --api="http://localhost:3000" --token="my-token" --team="my-team"
|
||||
```
|
||||
|
||||
### Template 5: Filtering and Scoping
|
||||
|
||||
```bash
|
||||
# Build specific package
|
||||
turbo build --filter=@myorg/web
|
||||
|
||||
# Build package and its dependencies
|
||||
turbo build --filter=@myorg/web...
|
||||
|
||||
# Build package and its dependents
|
||||
turbo build --filter=...@myorg/ui
|
||||
|
||||
# Build changed packages since main
|
||||
turbo build --filter='...[origin/main]'
|
||||
|
||||
# Build packages in directory
|
||||
turbo build --filter='./apps/*'
|
||||
|
||||
# Combine filters
|
||||
turbo build --filter=@myorg/web --filter=@myorg/docs
|
||||
|
||||
# Exclude package
|
||||
turbo build --filter='!@myorg/docs'
|
||||
|
||||
# Include dependencies of changed
|
||||
turbo build --filter='...[HEAD^1]...'
|
||||
```
|
||||
|
||||
### Template 6: Advanced Pipeline Configuration
|
||||
|
||||
```json
|
||||
{
|
||||
"$schema": "https://turbo.build/schema.json",
|
||||
"pipeline": {
|
||||
"build": {
|
||||
"dependsOn": ["^build"],
|
||||
"outputs": ["dist/**"],
|
||||
"inputs": [
|
||||
"$TURBO_DEFAULT$",
|
||||
"!**/*.md",
|
||||
"!**/*.test.*"
|
||||
]
|
||||
},
|
||||
"test": {
|
||||
"dependsOn": ["^build"],
|
||||
"outputs": ["coverage/**"],
|
||||
"inputs": [
|
||||
"src/**",
|
||||
"tests/**",
|
||||
"*.config.*"
|
||||
],
|
||||
"env": ["CI", "NODE_ENV"]
|
||||
},
|
||||
"test:e2e": {
|
||||
"dependsOn": ["build"],
|
||||
"outputs": [],
|
||||
"cache": false
|
||||
},
|
||||
"deploy": {
|
||||
"dependsOn": ["build", "test", "lint"],
|
||||
"outputs": [],
|
||||
"cache": false
|
||||
},
|
||||
"db:generate": {
|
||||
"cache": false
|
||||
},
|
||||
"db:push": {
|
||||
"cache": false,
|
||||
"dependsOn": ["db:generate"]
|
||||
},
|
||||
"@myorg/web#build": {
|
||||
"dependsOn": ["^build", "@myorg/db#db:generate"],
|
||||
"outputs": [".next/**"],
|
||||
"env": ["NEXT_PUBLIC_*"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Template 7: Root package.json Setup
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "my-turborepo",
|
||||
"private": true,
|
||||
"workspaces": [
|
||||
"apps/*",
|
||||
"packages/*"
|
||||
],
|
||||
"scripts": {
|
||||
"build": "turbo build",
|
||||
"dev": "turbo dev",
|
||||
"lint": "turbo lint",
|
||||
"test": "turbo test",
|
||||
"clean": "turbo clean && rm -rf node_modules",
|
||||
"format": "prettier --write \"**/*.{ts,tsx,md}\"",
|
||||
"changeset": "changeset",
|
||||
"version-packages": "changeset version",
|
||||
"release": "turbo build --filter=./packages/* && changeset publish"
|
||||
},
|
||||
"devDependencies": {
|
||||
"turbo": "^1.10.0",
|
||||
"prettier": "^3.0.0",
|
||||
"@changesets/cli": "^2.26.0"
|
||||
},
|
||||
"packageManager": "npm@10.0.0"
|
||||
}
|
||||
```
|
||||
|
||||
## Debugging Cache
|
||||
|
||||
```bash
|
||||
# Dry run to see what would run
|
||||
turbo build --dry-run
|
||||
|
||||
# Verbose output with hashes
|
||||
turbo build --verbosity=2
|
||||
|
||||
# Show task graph
|
||||
turbo build --graph
|
||||
|
||||
# Force no cache
|
||||
turbo build --force
|
||||
|
||||
# Show cache status
|
||||
turbo build --summarize
|
||||
|
||||
# Debug specific task
|
||||
TURBO_LOG_VERBOSITY=debug turbo build --filter=@myorg/web
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Do's
|
||||
- **Define explicit inputs** - Avoid cache invalidation
|
||||
- **Use workspace protocol** - `"@myorg/ui": "workspace:*"`
|
||||
- **Enable remote caching** - Share across CI and local
|
||||
- **Filter in CI** - Build only affected packages
|
||||
- **Cache build outputs** - Not source files
|
||||
|
||||
### Don'ts
|
||||
- **Don't cache dev servers** - Use `persistent: true`
|
||||
- **Don't include secrets in env** - Use runtime env vars
|
||||
- **Don't ignore dependsOn** - Causes race conditions
|
||||
- **Don't over-filter** - May miss dependencies
|
||||
|
||||
## Resources
|
||||
|
||||
- [Turborepo Documentation](https://turbo.build/repo/docs)
|
||||
- [Caching Guide](https://turbo.build/repo/docs/core-concepts/caching)
|
||||
- [Remote Caching](https://turbo.build/repo/docs/core-concepts/remote-caching)
|
||||
Reference in New Issue
Block a user