mirror of
https://github.com/wshobson/agents.git
synced 2026-03-18 09:37:15 +00:00
feat: add Conductor plugin for Context-Driven Development
Add comprehensive Conductor plugin implementing Context-Driven Development methodology with tracks, specs, and phased implementation plans. Components: - 5 commands: setup, new-track, implement, status, revert - 1 agent: conductor-validator - 3 skills: context-driven-development, track-management, workflow-patterns - 18 templates for project artifacts Documentation updates: - README.md: Updated counts (68 plugins, 100 agents, 110 skills, 76 tools) - docs/plugins.md: Added Conductor to Workflows section - docs/agents.md: Added conductor-validator agent - docs/agent-skills.md: Added Conductor skills section Also includes Prettier formatting across all project files.
This commit is contained in:
Seth Hobson
@@ -9,12 +9,14 @@ Systematic improvement of existing agents through performance analysis, prompt e
|
||||
Comprehensive analysis of agent performance using context-manager for historical data collection.
|
||||
|
||||
### 1.1 Gather Performance Data
|
||||
|
||||
```
|
||||
Use: context-manager
|
||||
Command: analyze-agent-performance $ARGUMENTS --days 30
|
||||
```
|
||||
|
||||
Collect metrics including:
|
||||
|
||||
- Task completion rate (successful vs failed tasks)
|
||||
- Response accuracy and factual correctness
|
||||
- Tool usage efficiency (correct tools, call frequency)
|
||||
@@ -25,6 +27,7 @@ Collect metrics including:
|
||||
### 1.2 User Feedback Pattern Analysis
|
||||
|
||||
Identify recurring patterns in user interactions:
|
||||
|
||||
- **Correction patterns**: Where users consistently modify outputs
|
||||
- **Clarification requests**: Common areas of ambiguity
|
||||
- **Task abandonment**: Points where users give up
|
||||
@@ -34,6 +37,7 @@ Identify recurring patterns in user interactions:
|
||||
### 1.3 Failure Mode Classification
|
||||
|
||||
Categorize failures by root cause:
|
||||
|
||||
- **Instruction misunderstanding**: Role or task confusion
|
||||
- **Output format errors**: Structure or formatting issues
|
||||
- **Context loss**: Long conversation degradation
|
||||
@@ -44,6 +48,7 @@ Categorize failures by root cause:
|
||||
### 1.4 Baseline Performance Report
|
||||
|
||||
Generate quantitative baseline metrics:
|
||||
|
||||
```
|
||||
Performance Baseline:
|
||||
- Task Success Rate: [X%]
|
||||
@@ -61,6 +66,7 @@ Apply advanced prompt optimization techniques using prompt-engineer agent.
|
||||
### 2.1 Chain-of-Thought Enhancement
|
||||
|
||||
Implement structured reasoning patterns:
|
||||
|
||||
```
|
||||
Use: prompt-engineer
|
||||
Technique: chain-of-thought-optimization
|
||||
@@ -74,6 +80,7 @@ Technique: chain-of-thought-optimization
|
||||
### 2.2 Few-Shot Example Optimization
|
||||
|
||||
Curate high-quality examples from successful interactions:
|
||||
|
||||
- **Select diverse examples** covering common use cases
|
||||
- **Include edge cases** that previously failed
|
||||
- **Show both positive and negative examples** with explanations
|
||||
@@ -81,6 +88,7 @@ Curate high-quality examples from successful interactions:
|
||||
- **Annotate examples** with key decision points
|
||||
|
||||
Example structure:
|
||||
|
||||
```
|
||||
Good Example:
|
||||
Input: [User request]
|
||||
@@ -98,6 +106,7 @@ Correct approach: [Fixed version]
|
||||
### 2.3 Role Definition Refinement
|
||||
|
||||
Strengthen agent identity and capabilities:
|
||||
|
||||
- **Core purpose**: Clear, single-sentence mission
|
||||
- **Expertise domains**: Specific knowledge areas
|
||||
- **Behavioral traits**: Personality and interaction style
|
||||
@@ -108,6 +117,7 @@ Strengthen agent identity and capabilities:
|
||||
### 2.4 Constitutional AI Integration
|
||||
|
||||
Implement self-correction mechanisms:
|
||||
|
||||
```
|
||||
Constitutional Principles:
|
||||
1. Verify factual accuracy before responding
|
||||
@@ -118,6 +128,7 @@ Constitutional Principles:
|
||||
```
|
||||
|
||||
Add critique-and-revise loops:
|
||||
|
||||
- Initial response generation
|
||||
- Self-critique against principles
|
||||
- Automatic revision if issues detected
|
||||
@@ -126,6 +137,7 @@ Add critique-and-revise loops:
|
||||
### 2.5 Output Format Tuning
|
||||
|
||||
Optimize response structure:
|
||||
|
||||
- **Structured templates** for common tasks
|
||||
- **Dynamic formatting** based on complexity
|
||||
- **Progressive disclosure** for detailed information
|
||||
@@ -140,6 +152,7 @@ Comprehensive testing framework with A/B comparison.
|
||||
### 3.1 Test Suite Development
|
||||
|
||||
Create representative test scenarios:
|
||||
|
||||
```
|
||||
Test Categories:
|
||||
1. Golden path scenarios (common successful cases)
|
||||
@@ -153,6 +166,7 @@ Test Categories:
|
||||
### 3.2 A/B Testing Framework
|
||||
|
||||
Compare original vs improved agent:
|
||||
|
||||
```
|
||||
Use: parallel-test-runner
|
||||
Config:
|
||||
@@ -164,6 +178,7 @@ Config:
|
||||
```
|
||||
|
||||
Statistical significance testing:
|
||||
|
||||
- Minimum sample size: 100 tasks per variant
|
||||
- Confidence level: 95% (p < 0.05)
|
||||
- Effect size calculation (Cohen's d)
|
||||
@@ -174,6 +189,7 @@ Statistical significance testing:
|
||||
Comprehensive scoring framework:
|
||||
|
||||
**Task-Level Metrics:**
|
||||
|
||||
- Completion rate (binary success/failure)
|
||||
- Correctness score (0-100% accuracy)
|
||||
- Efficiency score (steps taken vs optimal)
|
||||
@@ -181,6 +197,7 @@ Comprehensive scoring framework:
|
||||
- Response relevance and completeness
|
||||
|
||||
**Quality Metrics:**
|
||||
|
||||
- Hallucination rate (factual errors per response)
|
||||
- Consistency score (alignment with previous responses)
|
||||
- Format compliance (matches specified structure)
|
||||
@@ -188,6 +205,7 @@ Comprehensive scoring framework:
|
||||
- User satisfaction prediction
|
||||
|
||||
**Performance Metrics:**
|
||||
|
||||
- Response latency (time to first token)
|
||||
- Total generation time
|
||||
- Token consumption (input + output)
|
||||
@@ -197,6 +215,7 @@ Comprehensive scoring framework:
|
||||
### 3.4 Human Evaluation Protocol
|
||||
|
||||
Structured human review process:
|
||||
|
||||
- Blind evaluation (evaluators don't know version)
|
||||
- Standardized rubric with clear criteria
|
||||
- Multiple evaluators per sample (inter-rater reliability)
|
||||
@@ -210,6 +229,7 @@ Safe rollout with monitoring and rollback capabilities.
|
||||
### 4.1 Version Management
|
||||
|
||||
Systematic versioning strategy:
|
||||
|
||||
```
|
||||
Version Format: agent-name-v[MAJOR].[MINOR].[PATCH]
|
||||
Example: customer-support-v2.3.1
|
||||
@@ -220,6 +240,7 @@ PATCH: Bug fixes, minor adjustments
|
||||
```
|
||||
|
||||
Maintain version history:
|
||||
|
||||
- Git-based prompt storage
|
||||
- Changelog with improvement details
|
||||
- Performance metrics per version
|
||||
@@ -228,6 +249,7 @@ Maintain version history:
|
||||
### 4.2 Staged Rollout
|
||||
|
||||
Progressive deployment strategy:
|
||||
|
||||
1. **Alpha testing**: Internal team validation (5% traffic)
|
||||
2. **Beta testing**: Selected users (20% traffic)
|
||||
3. **Canary release**: Gradual increase (20% → 50% → 100%)
|
||||
@@ -237,6 +259,7 @@ Progressive deployment strategy:
|
||||
### 4.3 Rollback Procedures
|
||||
|
||||
Quick recovery mechanism:
|
||||
|
||||
```
|
||||
Rollback Triggers:
|
||||
- Success rate drops >10% from baseline
|
||||
@@ -256,6 +279,7 @@ Rollback Process:
|
||||
### 4.4 Continuous Monitoring
|
||||
|
||||
Real-time performance tracking:
|
||||
|
||||
- Dashboard with key metrics
|
||||
- Anomaly detection alerts
|
||||
- User feedback collection
|
||||
@@ -265,6 +289,7 @@ Real-time performance tracking:
|
||||
## Success Criteria
|
||||
|
||||
Agent improvement is successful when:
|
||||
|
||||
- Task success rate improves by ≥15%
|
||||
- User corrections decrease by ≥25%
|
||||
- No increase in safety violations
|
||||
@@ -275,6 +300,7 @@ Agent improvement is successful when:
|
||||
## Post-Deployment Review
|
||||
|
||||
After 30 days of production use:
|
||||
|
||||
1. Analyze accumulated performance data
|
||||
2. Compare against baseline and targets
|
||||
3. Identify new improvement opportunities
|
||||
@@ -284,9 +310,10 @@ After 30 days of production use:
|
||||
## Continuous Improvement Cycle
|
||||
|
||||
Establish regular improvement cadence:
|
||||
|
||||
- **Weekly**: Monitor metrics and collect feedback
|
||||
- **Monthly**: Analyze patterns and plan improvements
|
||||
- **Quarterly**: Major version updates with new capabilities
|
||||
- **Annually**: Strategic review and architecture updates
|
||||
|
||||
Remember: Agent optimization is an iterative process. Each cycle builds upon previous learnings, gradually improving performance while maintaining stability and safety.
|
||||
Remember: Agent optimization is an iterative process. Each cycle builds upon previous learnings, gradually improving performance while maintaining stability and safety.
|
||||
|
||||
@@ -3,9 +3,11 @@
|
||||
## Role: AI-Powered Multi-Agent Performance Engineering Specialist
|
||||
|
||||
### Context
|
||||
|
||||
The Multi-Agent Optimization Tool is an advanced AI-driven framework designed to holistically improve system performance through intelligent, coordinated agent-based optimization. Leveraging cutting-edge AI orchestration techniques, this tool provides a comprehensive approach to performance engineering across multiple domains.
|
||||
|
||||
### Core Capabilities
|
||||
|
||||
- Intelligent multi-agent coordination
|
||||
- Performance profiling and bottleneck identification
|
||||
- Adaptive optimization strategies
|
||||
@@ -13,7 +15,9 @@ The Multi-Agent Optimization Tool is an advanced AI-driven framework designed to
|
||||
- Cost and efficiency tracking
|
||||
|
||||
## Arguments Handling
|
||||
|
||||
The tool processes optimization arguments with flexible input parameters:
|
||||
|
||||
- `$TARGET`: Primary system/application to optimize
|
||||
- `$PERFORMANCE_GOALS`: Specific performance metrics and objectives
|
||||
- `$OPTIMIZATION_SCOPE`: Depth of optimization (quick-win, comprehensive)
|
||||
@@ -23,11 +27,13 @@ The tool processes optimization arguments with flexible input parameters:
|
||||
## 1. Multi-Agent Performance Profiling
|
||||
|
||||
### Profiling Strategy
|
||||
|
||||
- Distributed performance monitoring across system layers
|
||||
- Real-time metrics collection and analysis
|
||||
- Continuous performance signature tracking
|
||||
|
||||
#### Profiling Agents
|
||||
|
||||
1. **Database Performance Agent**
|
||||
- Query execution time analysis
|
||||
- Index utilization tracking
|
||||
@@ -44,6 +50,7 @@ The tool processes optimization arguments with flexible input parameters:
|
||||
- Core Web Vitals monitoring
|
||||
|
||||
### Profiling Code Example
|
||||
|
||||
```python
|
||||
def multi_agent_profiler(target_system):
|
||||
agents = [
|
||||
@@ -62,12 +69,14 @@ def multi_agent_profiler(target_system):
|
||||
## 2. Context Window Optimization
|
||||
|
||||
### Optimization Techniques
|
||||
|
||||
- Intelligent context compression
|
||||
- Semantic relevance filtering
|
||||
- Dynamic context window resizing
|
||||
- Token budget management
|
||||
|
||||
### Context Compression Algorithm
|
||||
|
||||
```python
|
||||
def compress_context(context, max_tokens=4000):
|
||||
# Semantic compression using embedding-based truncation
|
||||
@@ -82,12 +91,14 @@ def compress_context(context, max_tokens=4000):
|
||||
## 3. Agent Coordination Efficiency
|
||||
|
||||
### Coordination Principles
|
||||
|
||||
- Parallel execution design
|
||||
- Minimal inter-agent communication overhead
|
||||
- Dynamic workload distribution
|
||||
- Fault-tolerant agent interactions
|
||||
|
||||
### Orchestration Framework
|
||||
|
||||
```python
|
||||
class MultiAgentOrchestrator:
|
||||
def __init__(self, agents):
|
||||
@@ -112,6 +123,7 @@ class MultiAgentOrchestrator:
|
||||
## 4. Parallel Execution Optimization
|
||||
|
||||
### Key Strategies
|
||||
|
||||
- Asynchronous agent processing
|
||||
- Workload partitioning
|
||||
- Dynamic resource allocation
|
||||
@@ -120,12 +132,14 @@ class MultiAgentOrchestrator:
|
||||
## 5. Cost Optimization Strategies
|
||||
|
||||
### LLM Cost Management
|
||||
|
||||
- Token usage tracking
|
||||
- Adaptive model selection
|
||||
- Caching and result reuse
|
||||
- Efficient prompt engineering
|
||||
|
||||
### Cost Tracking Example
|
||||
|
||||
```python
|
||||
class CostOptimizer:
|
||||
def __init__(self):
|
||||
@@ -145,6 +159,7 @@ class CostOptimizer:
|
||||
## 6. Latency Reduction Techniques
|
||||
|
||||
### Performance Acceleration
|
||||
|
||||
- Predictive caching
|
||||
- Pre-warming agent contexts
|
||||
- Intelligent result memoization
|
||||
@@ -153,6 +168,7 @@ class CostOptimizer:
|
||||
## 7. Quality vs Speed Tradeoffs
|
||||
|
||||
### Optimization Spectrum
|
||||
|
||||
- Performance thresholds
|
||||
- Acceptable degradation margins
|
||||
- Quality-aware optimization
|
||||
@@ -161,6 +177,7 @@ class CostOptimizer:
|
||||
## 8. Monitoring and Continuous Improvement
|
||||
|
||||
### Observability Framework
|
||||
|
||||
- Real-time performance dashboards
|
||||
- Automated optimization feedback loops
|
||||
- Machine learning-driven improvement
|
||||
@@ -169,21 +186,24 @@ class CostOptimizer:
|
||||
## Reference Workflows
|
||||
|
||||
### Workflow 1: E-Commerce Platform Optimization
|
||||
|
||||
1. Initial performance profiling
|
||||
2. Agent-based optimization
|
||||
3. Cost and performance tracking
|
||||
4. Continuous improvement cycle
|
||||
|
||||
### Workflow 2: Enterprise API Performance Enhancement
|
||||
|
||||
1. Comprehensive system analysis
|
||||
2. Multi-layered agent optimization
|
||||
3. Iterative performance refinement
|
||||
4. Cost-efficient scaling strategy
|
||||
|
||||
## Key Considerations
|
||||
|
||||
- Always measure before and after optimization
|
||||
- Maintain system stability during optimization
|
||||
- Balance performance gains with resource consumption
|
||||
- Implement gradual, reversible changes
|
||||
|
||||
Target Optimization: $ARGUMENTS
|
||||
Target Optimization: $ARGUMENTS
|
||||
|
||||
Reference in New Issue
Block a user