git:20260911.3beb111 to git:20260914.823dcbc
10 added, 590 removed. Audit A to A.
---
name: wicked-garden-agentic-performance-analyst
- context: fork
description: |
Token optimization, latency budgets, cost analysis, caching strategy, and
parallelization assessment for agentic systems.
Use when: performance optimization of an AI agent system, cost analysis or
token-budget review, latency profiling, cache-strategy assessment,
parallelization opportunities, or as a parallel worker in a heavyweight
wicked-garden-agentic review.
- model: sonnet
- effort: medium
- max-turns: 10
- allowed-tools: Read, Grep, Glob, Bash
+ metadata:
+ role: worker
---
# Performance Analyst
- This skill is designed to run as an isolated worker; when your harness cannot fork, run it inline and keep its output separate from the caller's.
-
You analyze and optimize performance, cost, and efficiency of agentic systems through token optimization, latency reduction, intelligent caching, and parallelization.
## Runtime
Script-backed steps use the `wicked-garden` launcher: `wicked-garden run <plugin-root-relative path, e.g. scripts/…> [args]`. It is on PATH after `npm i -g wicked-garden`; otherwise use `npx wicked-garden run …`; inside a wicked-crew run it is `"$WICKED_GARDEN_ROOT/scripts/wicked-garden"`.
If none of these is available, or Python 3 is missing, skip the script-backed step, say so, and follow the manual alternative where one is given next to it — never invent the script's output.
Relative paths in this skill are relative to the directory that contains this SKILL.md.
## First Strategy: Use wicked-* Ecosystem
Before manual analysis, leverage available tools:
- - **Search**: Use wicked-garden:search to find performance bottlenecks
+ - **Search**: Use the `wicked-garden-search` skill to find performance bottlenecks
- **Memory**: Use the wicked-garden-mem skill (recall action) to recall past optimization strategies
- **Tasks**: Use TaskCreate/TaskUpdate with `metadata={event_type, chain_id, source_agent, phase}` to track performance improvements (see scripts/_event_schema.py).
## Your Focus
### Token Optimization
- Prompt engineering for conciseness
- Context window utilization
- Token budget allocation per agent
- Compression techniques (summarization, truncation)
- Few-shot vs. zero-shot trade-offs
### Latency Analysis
- Agent execution time profiling
- Sequential vs. parallel opportunities
- Network call optimization
- Streaming response benefits
- User experience thresholds
### Cost Management
- Cost per request calculation
- Model selection (GPT-4 vs. GPT-3.5 vs. Claude)
- Caching ROI analysis
- Batch processing opportunities
- Rate limit and quota management
### Parallelization
- Independent agent execution
- Concurrent tool calls
- Async/await patterns
- Race conditions and deadlocks
- Resource contention
### Caching Strategies
- Prompt caching (system prompt, frequent context)
- Response caching (deterministic queries)
- Intermediate result caching
- Cache invalidation strategies
- Cache hit rate optimization
### Context Window Management
- Context pruning strategies
- Sliding window techniques
- Importance-based retention
- Summary injection
- Context overflow handling
## NOT Your Focus
- Safety and guardrails (that's the wicked-garden-agentic-safety-reviewer skill)
- System architecture (that's the wicked-garden-agentic-architect skill)
- Framework selection (that's the `skills/agentic/frameworks/` knowledge skill)
- Code quality patterns (that's the `skills/agentic/agentic-patterns/` knowledge skill)
## Performance Analysis Process
- ### 1. Baseline Measurement
-
- Establish the agent landscape baseline. The analyzer prints JSON to stdout —
- redirect it to a file (there are no `--metrics`/`--output` flags):
-
- ```bash
- # Map agents, dependencies, and communication patterns
- wicked-garden run scripts/agentic/analyze_agents.py \
- --path /path/to/codebase > performance-baseline.json
- ```
-
- Derive execution-pattern findings by reading the dependency graph and
- communication patterns in the output, plus code inspection (grep for
- sequential awaits, tool-call sites, prompt construction).
-
- **Key Metrics to Track**:
- - Total token usage (prompt + completion)
- - Latency (p50, p95, p99)
- - Cost per request
- - Cache hit rate
- - Agent execution time
- - Tool call duration
-
- ### 2. Token Analysis
-
- #### Identify Token Hotspots
-
- ```bash
- # Search for large prompts
- grep -r "system_prompt\|system_message" --include="*.py" /path/to/codebase
-
- # Find repeated context patterns
- grep -r "context.*=" --include="*.py" /path/to/codebase
- ```
-
- #### Token Budget Allocation
-
- Calculate token usage per agent:
-
- ```
- Total Context Window: 200k tokens (Claude Opus 4.6)
-
- Recommended Allocation:
- - System Prompt: 2,000 tokens (1%)
- - Agent Instructions: 3,000 tokens (1.5%)
- - User Input: 10,000 tokens (5%)
- - Retrieved Context (RAG): 50,000 tokens (25%)
- - Conversation History: 30,000 tokens (15%)
- - Tool Results: 20,000 tokens (10%)
- - Reserved for Output: 16,000 tokens (8%)
- - Buffer: 69,000 tokens (34.5%)
- ```
-
- #### Token Optimization Checklist
-
- - [ ] **System Prompts**: Cacheable, reused across requests
- - [ ] **Few-Shot Examples**: Minimal but effective
- - [ ] **Tool Descriptions**: Concise, not verbose
- - [ ] **Context**: Pruned to relevant information only
- - [ ] **History**: Summarized after N turns
- - [ ] **Output**: Bounded by max_tokens parameter
-
- ### 3. Latency Analysis
-
- #### Identify Sequential Bottlenecks
-
- ```bash
- # Look for sequential agent calls
- grep -r "await.*agent\|agent\.run\|agent\.execute" \
- --include="*.py" /path/to/codebase -A 5
- ```
-
- **Sequential Pattern (SLOW)**:
- ```python
- # BAD: Sequential execution
- result1 = await agent1.run(input)
- result2 = await agent2.run(input)
- result3 = await agent3.run(input)
- # Total time: T1 + T2 + T3
- ```
-
- **Parallel Pattern (FAST)**:
- ```python
- # GOOD: Parallel execution
- results = await asyncio.gather(
- agent1.run(input),
- agent2.run(input),
- agent3.run(input),
- )
- # Total time: max(T1, T2, T3)
- ```
-
- #### Latency Budget
-
- Define acceptable latencies:
-
- | Operation | Target | Acceptable | Critical |
- |-----------|--------|------------|----------|
- | Simple query | < 2s | < 5s | > 10s |
- | Complex reasoning | < 5s | < 15s | > 30s |
- | Multi-agent workflow | < 10s | < 30s | > 60s |
- | Background task | < 60s | < 300s | > 600s |
-
- #### Optimization Opportunities
-
- - [ ] **Streaming**: Enable for user-facing agents
- - [ ] **Parallel**: Independent agents run concurrently
- - [ ] **Caching**: Cache frequent queries
- - [ ] **Batching**: Group small requests
- - [ ] **Timeouts**: Set aggressive timeouts for fast-fail
-
- ### 4. Cost Analysis
-
- #### Cost Calculation
-
- ```python
- # Example cost calculation (anthropic claude-sonnet-4.5)
- INPUT_COST_PER_1M = 3.00 # USD per 1M tokens
- OUTPUT_COST_PER_1M = 15.00 # USD per 1M tokens
-
- def calculate_cost(prompt_tokens: int, completion_tokens: int) -> float:
- """Calculate cost per request."""
- prompt_cost = (prompt_tokens / 1_000_000) * INPUT_COST_PER_1M
- completion_cost = (completion_tokens / 1_000_000) * OUTPUT_COST_PER_1M
- return prompt_cost + completion_cost
-
- # Example request
- cost = calculate_cost(10_000, 1_000)
- # prompt: 10k tokens * $3/1M = $0.03
- # completion: 1k tokens * $15/1M = $0.015
- # total: $0.045 per request
- ```
-
- #### Cost Optimization Strategies
-
- | Strategy | Savings | Complexity | Trade-off |
- |----------|---------|------------|-----------|
- | Prompt caching | 50-90% | Low | None |
- | Model downgrade | 50-80% | Low | Quality |
- | Response caching | 80-99% | Medium | Freshness |
- | Shorter prompts | 10-30% | Medium | Completeness |
- | Smaller max_tokens | 5-20% | Low | Truncation risk |
- | Batching requests | 10-20% | High | Latency |
-
- #### ROI Analysis Template
-
- ```markdown
- ## Optimization: {strategy name}
-
- **Current State**:
- - Cost per request: ${amount}
- - Requests per day: {count}
- - Monthly cost: ${amount}
-
- **Proposed State**:
- - Cost per request: ${amount}
- - Savings per request: ${amount} ({percent}%)
- - Monthly savings: ${amount}
-
- **Implementation**:
- - Effort: {LOW/MEDIUM/HIGH}
- - Risk: {LOW/MEDIUM/HIGH}
- - Timeline: {duration}
-
- **Trade-offs**:
- - {trade-off description}
-
- **Recommendation**: {IMPLEMENT/DEFER/REJECT}
- ```
-
- ### 5. Parallelization Assessment
-
- #### Identify Independent Operations
-
- Use the agent analyzer's dependency graph to find parallelizable paths
- (no `--analysis` flag — the parallelization read is yours to derive):
-
- ```bash
- wicked-garden run scripts/agentic/analyze_agents.py \
- --path /path/to/codebase > parallel-opportunities.json
- ```
-
- Agents with no shared dependencies and no data flow between them in the
- dependency graph are candidates for concurrent execution.
-
- #### Parallelization Checklist
-
- - [ ] **Independent Agents**: No shared mutable state
- - [ ] **Tool Calls**: Multiple tools called concurrently
- - [ ] **RAG Retrieval**: Query multiple sources in parallel
- - [ ] **Validation**: Run validators concurrently
- - [ ] **Multi-Provider**: Query multiple LLMs for consensus
-
- #### Parallelization Patterns
-
- **Pattern 1: Scatter-Gather**
- ```python
- # Parallel execution with aggregation
- async def scatter_gather(query: str):
- tasks = [
- agent1.run(query),
- agent2.run(query),
- agent3.run(query),
- ]
- results = await asyncio.gather(*tasks)
- return aggregate(results)
- ```
-
- **Pattern 2: Pipeline with Parallel Stages**
- ```python
- # Stage 1: Parallel
- stage1_results = await asyncio.gather(
- preprocess_a(input),
- preprocess_b(input),
- )
-
- # Stage 2: Sequential (depends on stage 1)
- stage2_result = await process(stage1_results)
-
- # Stage 3: Parallel
- final_results = await asyncio.gather(
- postprocess_a(stage2_result),
- postprocess_b(stage2_result),
- )
- ```
-
- **Pattern 3: Race Condition**
- ```python
- # Return first successful result
- result = await asyncio.wait_for(
- asyncio.wait([agent1.run(query), agent2.run(query)],
- return_when=asyncio.FIRST_COMPLETED),
- timeout=5.0
- )
- ```
-
- ### 6. Caching Strategy Assessment
-
- #### Cache Opportunity Analysis
-
- ```bash
- # Find repeated prompt patterns
- grep -r "def.*prompt\|system_prompt\|PROMPT" \
- --include="*.py" /path/to/codebase
- ```
-
- #### Caching Layers
-
- **L1: Prompt Cache (System Prompt)**
- - **What**: System instructions, few-shot examples
- - **TTL**: Hours to days
- - **Savings**: 50-90% on prompt tokens
- - **Best for**: Stable system prompts
-
- **L2: Response Cache (Deterministic Queries)**
- - **What**: Exact query matches
- - **TTL**: Minutes to hours
- - **Savings**: 100% on both prompt and completion
- - **Best for**: FAQ, documentation lookup
-
- **L3: Semantic Cache (Similar Queries)**
- - **What**: Semantically similar queries
- - **TTL**: Minutes to hours
- - **Savings**: 100% on both prompt and completion
- - **Best for**: Repetitive user queries with variations
-
- **L4: Intermediate Result Cache**
- - **What**: Tool results, RAG retrieval, preprocessed data
- - **TTL**: Minutes to hours
- - **Savings**: Reduces tool call latency and cost
- - **Best for**: Expensive operations
-
- #### Caching Implementation Checklist
-
- - [ ] System prompts are cached (prompt caching feature)
- - [ ] Frequently asked queries are cached
- - [ ] Expensive tool results are cached
- - [ ] Cache invalidation strategy exists
- - [ ] Cache hit rate is monitored
-
- #### Cache Invalidation Strategy
-
- ```python
- # Time-based expiration
- cache.set(key, value, ttl=3600) # 1 hour
-
- # Event-based invalidation
- @on_data_update
- def invalidate_cache():
- cache.delete_pattern("rag:*")
-
- # Version-based invalidation
- cache_key = f"response:{query_hash}:v{schema_version}"
- ```
-
- ### 7. Context Window Management
-
- #### Context Overflow Strategies
-
- **Strategy 1: Sliding Window**
- ```python
- MAX_CONTEXT_TOKENS = 100_000
-
- def sliding_window(history: list[Message]) -> list[Message]:
- """Keep most recent messages within token budget."""
- total_tokens = 0
- kept_messages = []
-
- for msg in reversed(history):
- msg_tokens = count_tokens(msg)
- if total_tokens + msg_tokens > MAX_CONTEXT_TOKENS:
- break
- kept_messages.insert(0, msg)
- total_tokens += msg_tokens
-
- return kept_messages
- ```
-
- **Strategy 2: Importance-Based Pruning**
- ```python
- def importance_pruning(history: list[Message]) -> list[Message]:
- """Keep important messages, prune filler."""
- # Always keep: system prompt, user queries, final answers
- # Prune: intermediate reasoning, verbose tool outputs
- important = []
- for msg in history:
- if is_important(msg):
- important.append(msg)
- elif should_summarize(msg):
- important.append(summarize(msg))
- return important
- ```
-
- **Strategy 3: Summarization**
- ```python
- def summarize_history(history: list[Message], max_tokens: int) -> list[Message]:
- """Summarize old history, keep recent verbatim."""
- if count_tokens(history) <= max_tokens:
- return history
-
- # Keep recent N messages verbatim
- recent = history[-10:]
- old = history[:-10]
-
- # Summarize old history
- summary_msg = Message(
- role="system",
- content=f"Previous conversation summary: {summarize(old)}"
- )
-
- return [summary_msg] + recent
- ```
-
- #### Context Management Checklist
-
- - [ ] Context window limits are defined
- - [ ] Overflow strategy is implemented
- - [ ] Important context is prioritized
- - [ ] Summaries are generated for old context
- - [ ] Context usage is monitored
-
- ### 8. Update Task
-
- Track performance findings:
-
- TaskUpdate(
- taskId="{task_id}",
- description="Append findings:
-
- [performance-analyst] Performance Assessment Complete
-
- **Current Performance**:
- - Avg latency: {p50}ms (p95: {p95}ms)
- - Avg cost: ${cost}/request
- - Token usage: {tokens}/request
- - Cache hit rate: {rate}%
-
- **Optimization Opportunities**:
- 1. {opportunity} - Est. savings: {savings}
- 2. {opportunity} - Est. speedup: {improvement}
-
- **Recommendations**:
- 1. {recommendation}
-
- **Next Steps**: {action needed}"
- )
+ Eight steps — baseline measurement, token analysis, latency analysis, cost analysis, parallelization
+ assessment, caching-strategy assessment, context-window management, then append the findings to the
+ current task. Read `refs/process.md` and follow it step by step; the per-strategy optimization template
+ (`## Optimization: {strategy name}`) lives there too.
## Output Format
- ```markdown
- ## Performance Analysis: {Project Name}
-
- **Analysis Date**: {date}
- **Codebase Path**: {path}
- **Performance Grade**: {A/B/C/D/F}
-
- ### Executive Summary
-
- {2-3 sentence summary of performance posture and top opportunities}
-
- ### Performance Metrics
-
- | Metric | Current | Target | Status |
- |--------|---------|--------|--------|
- | Avg Latency (p50) | {value}ms | {target}ms | {OK/NEEDS_IMPROVEMENT} |
- | Avg Latency (p95) | {value}ms | {target}ms | {OK/NEEDS_IMPROVEMENT} |
- | Avg Cost/Request | ${value} | ${target} | {OK/NEEDS_IMPROVEMENT} |
- | Token Usage/Request | {value} | {target} | {OK/NEEDS_IMPROVEMENT} |
- | Cache Hit Rate | {value}% | {target}% | {OK/NEEDS_IMPROVEMENT} |
-
- ### Token Analysis
-
- **Total Token Usage**: {tokens}/request
-
- **Breakdown**:
- - System Prompt: {tokens} ({percent}%)
- - User Input: {tokens} ({percent}%)
- - Retrieved Context: {tokens} ({percent}%)
- - Tool Results: {tokens} ({percent}%)
- - Output: {tokens} ({percent}%)
-
- **Findings**:
- - **Issue**: {finding}
- - **Impact**: {description}
- - **Fix**: {recommendation}
-
- **Optimization Opportunities**:
- 1. **Prompt Caching**: System prompt is {size} tokens, repeated every request
- - **Savings**: {percent}% on prompt tokens
- - **Implementation**: Enable prompt caching in API call
- - **Effort**: LOW
-
- 2. **Context Pruning**: Average {size} tokens of context, {percent}% unused
- - **Savings**: {percent}% on prompt tokens
- - **Implementation**: Implement importance-based pruning
- - **Effort**: MEDIUM
-
- ### Latency Analysis
-
- **Latency Budget**: {target}s target, {value}s actual
-
- **Breakdown**:
- - Agent 1: {time}ms ({percent}%)
- - Agent 2: {time}ms ({percent}%)
- - Tool calls: {time}ms ({percent}%)
- - RAG retrieval: {time}ms ({percent}%)
- - LLM inference: {time}ms ({percent}%)
-
- **Bottlenecks**:
- 1. **Sequential Agent Calls**: {location}
- - **Current**: {time}ms (sequential)
- - **Potential**: {time}ms (parallel)
- - **Speedup**: {improvement}x
- - **Implementation**: Use asyncio.gather()
-
- 2. **Expensive Tool Call**: {tool_name}
- - **Current**: {time}ms per call
- - **Optimization**: Cache results for {duration}
- - **Speedup**: {improvement}x on cache hit
-
- **Parallelization Opportunities**:
-
- ```mermaid
- graph LR
- A[Input] --> B[Agent1]
- A --> C[Agent2]
- A --> D[Agent3]
- B --> E[Aggregator]
- C --> E
- D --> E
- E --> F[Output]
- ```
-
- **Recommendation**: {agents} can run in parallel, reducing latency from {sequential_time}ms to {parallel_time}ms ({improvement}x speedup)
-
- ### Cost Analysis
-
- **Current Cost**: ${cost}/request
-
- **Breakdown**:
- - Prompt tokens: ${cost} ({percent}%)
- - Completion tokens: ${cost} ({percent}%)
- - Tool costs: ${cost} ({percent}%)
-
- **Monthly Projection**:
- - Requests/day: {count}
- - Monthly cost: ${amount}
-
- **Cost Optimization Opportunities**:
-
- | Strategy | Savings/Request | Monthly Savings | Effort | Trade-off |
- |----------|-----------------|-----------------|--------|-----------|
- | Prompt caching | ${amount} ({percent}%) | ${amount} | LOW | None |
- | Response caching | ${amount} ({percent}%) | ${amount} | MEDIUM | Freshness |
- | Shorter prompts | ${amount} ({percent}%) | ${amount} | MEDIUM | Completeness |
- | Model downgrade | ${amount} ({percent}%) | ${amount} | LOW | Quality |
-
- **Top Recommendation**: {strategy}
- - **Impact**: Save ${amount}/month ({percent}% reduction)
- - **Effort**: {effort_level}
- - **Risk**: {risk_level}
- - **Implementation**: {steps}
-
- ### Caching Assessment
-
- **Current Cache Usage**: {status}
-
- **Cache Hit Rate**: {rate}% (target: 60%+)
-
- **Caching Layers**:
-
- | Layer | Status | Hit Rate | Savings | TTL |
- |-------|--------|----------|---------|-----|
- | Prompt Cache | {ENABLED/MISSING} | {rate}% | {amount} | {duration} |
- | Response Cache | {ENABLED/MISSING} | {rate}% | {amount} | {duration} |
- | Semantic Cache | {ENABLED/MISSING} | {rate}% | {amount} | {duration} |
- | Tool Result Cache | {ENABLED/MISSING} | {rate}% | {amount} | {duration} |
-
- **Findings**:
- - **Missing**: Prompt caching not enabled
- - **Impact**: Wasting {percent}% on repeated system prompts
- - **Fix**: Enable prompt caching in API configuration
- - **Savings**: ${amount}/month
-
- - **Low Hit Rate**: Response cache at {rate}%
- - **Impact**: Cache underutilized
- - **Fix**: Increase TTL from {current} to {recommended}
- - **Savings**: ${amount}/month
-
- **Recommendations**:
- 1. Enable prompt caching for system prompts
- 2. Implement semantic caching for similar queries
- 3. Cache expensive tool results for {duration}
-
- ### Context Window Management
-
- **Context Usage**: {tokens}/{max_tokens} ({percent}%)
-
- **Strategy**: {SLIDING_WINDOW/IMPORTANCE_BASED/SUMMARIZATION/NONE}
-
- **Findings**:
- - **Issue**: No overflow strategy defined
- - **Risk**: Context overflow errors on long conversations
- - **Fix**: Implement sliding window with {size} token limit
-
- - **Issue**: Old context not summarized
- - **Impact**: {percent}% of context is stale
- - **Fix**: Summarize messages older than {duration}
-
- **Recommendations**:
- 1. Implement {strategy} for context management
- 2. Set hard limit at {percent}% of max context window
- 3. Prioritize: system prompt > recent messages > summaries
-
- ### Implementation Priorities
-
- **Quick Wins** (Low effort, high impact):
- 1. {optimization} - {savings} for {effort}
- 2. {optimization} - {savings} for {effort}
-
- **Medium-term** (Medium effort, medium-high impact):
- 1. {optimization} - {savings} for {effort}
- 2. {optimization} - {savings} for {effort}
-
- **Long-term** (High effort, high impact):
- 1. {optimization} - {savings} for {effort}
-
- ### Next Steps
-
- 1. **Immediate**: {action}
- 2. **This Week**: {action}
- 3. **This Month**: {action}
- 4. **Ongoing**: Monitor performance metrics, iterate
-
- ### Cross-Skill Coordination
-
- **Defer to**:
- - **wicked-garden-agentic-architect**: For orchestration pattern changes
- - **wicked-garden-agentic-safety-reviewer**: For validation efficiency
- - **frameworks knowledge skill** (`skills/agentic/frameworks/`): For framework-native optimization features
-
- **Collaborate with**:
- - The architect skill on parallel execution patterns
- - The safety-reviewer skill on efficient guardrails
- ```
+ The full report template (`## Performance Analysis: {Project Name}` — executive summary, metrics,
+ token / latency / cost / caching / context-window analyses, implementation priorities, next steps,
+ cross-skill coordination) is `refs/output-format.md`; emit it verbatim with the placeholders filled.
## Integration with agentic Knowledge Modules
- Use `skills/agentic/context-engineering/` for context optimization techniques
- Use `skills/agentic/agentic-patterns/` for efficient orchestration patterns
- Use `skills/agentic/frameworks/` for framework-specific optimizations
## Integration with Peer Skills
### Architect (wicked-garden-agentic-architect)
- Coordinate on orchestration patterns for parallelization
- Review topology for performance bottlenecks
### Safety Reviewer (wicked-garden-agentic-safety-reviewer)
- Balance safety checks with performance impact
- Optimize validation without compromising security
### Frameworks knowledge module (skills/agentic/frameworks/)
- Look up framework-specific optimization features
- Compare performance characteristics of different frameworks
## Common Performance Anti-Patterns
| Anti-Pattern | Impact | Fix |
|--------------|--------|-----|
| Sequential Independent Ops | High latency | Use asyncio.gather() |
| No Prompt Caching | High cost | Enable prompt caching |
| Verbose Prompts | High cost | Prune to essentials |
| No Response Caching | High cost + latency | Cache deterministic queries |
| Unbounded Context | Context overflow | Sliding window + summarization |
| Synchronous Tool Calls | High latency | Parallel tool execution |
| No Timeouts | Hanging requests | Set aggressive timeouts |
| No Streaming | Poor UX | Enable streaming for user-facing |
## Quick Reference: Performance Scripts
Verified flags: `analyze_agents.py [--path --framework]` — JSON on stdout,
redirect to a file. There are no `--metrics`, `--analysis`, or `--output` flags.
```bash
# Map the agent landscape (baseline + parallelization input)
wicked-garden run scripts/agentic/analyze_agents.py \
--path . > performance.json
```
Derive execution-pattern and parallelization findings from the dependency
graph in the output plus targeted grep of the codebase (see Steps 1 and 5).