AI Agents - Background and Orchestration

ai agents automation claude-code orchestration

---

Overview

AI agents can run autonomously, executing multi-step tasks, using tools, and coordinating with other agents. This guide covers running agents in the background and orchestrating multiple agents for complex workflows.

---

Claude Code Agents

Background Agents (Task Tool)

Claude Code can spawn sub-agents that run in the background:

Within Claude Code session:
> Run tests in the background while I continue working

The agent will:

1. Spawn a background process
2. Return an output_file path
3. Continue your main session
4. Notify when complete

Check background agent status:

> Check on the test agent
> Read the output file from the background task

Agent Types in Claude Code

Agent Type	Purpose	Tools Available
Bash	Command execution	Bash
general-purpose	Complex multi-step tasks	All tools
Explore	Codebase exploration	Read, Glob, Grep
Plan	Implementation planning	Read-only tools

Spawning Agents Programmatically

Using Claude Code’s Task tool:

Spawn an Explore agent to find all API endpoints

The agent runs with its own context and returns results.

---

Aider Multi-File Workflows

Architect Mode

Use a smart model to plan, cheaper model to implement:

# Claude as architect, GPT-3.5 as coder
aider --architect --model claude-3-5-sonnet-20241022 --editor-model gpt-3.5-turbo

Watch Mode

Auto-run on file changes:

# Watch for changes and auto-apply
aider --watch src/
 
# With auto-commit
aider --watch --auto-commits src/

---

Agent Orchestration Patterns

Pattern 1: Sequential Pipeline

┌─────────┐    ┌─────────┐    ┌─────────┐    ┌─────────┐
│ Analyze │───>│  Plan   │───>│ Execute │───>│ Verify  │
└─────────┘    └─────────┘    └─────────┘    └─────────┘

Example workflow:

1. Analyze Agent: Explore codebase, identify patterns
2. Plan Agent: Design implementation approach
3. Execute Agent: Write the code
4. Verify Agent: Run tests, check for issues

Pattern 2: Parallel Workers

                ┌──────────┐
            ┌──>│ Worker 1 │──┐
┌─────────┐ │   └──────────┘  │   ┌──────────┐
│ Manager │─┼──>│ Worker 2 │──┼──>│ Combiner │
└─────────┘ │   └──────────┘  │   └──────────┘
            └──>│ Worker 3 │──┘
                └──────────┘

Example:

In Claude Code:
> Run these three agents in parallel:
  1. Search for all usages of deprecated API
  2. Find the new API documentation
  3. List all test files affected

Pattern 3: Supervisor-Worker

┌────────────┐
│ Supervisor │
└─────┬──────┘
      │ Delegates
      ▼
┌─────────────────────────┐
│   Worker Pool           │
│ ┌───┐ ┌───┐ ┌───┐ ┌───┐│
│ │ W │ │ W │ │ W │ │ W ││
│ └───┘ └───┘ └───┘ └───┘│
└─────────────────────────┘

Supervisor assigns tasks, monitors progress, handles failures.

---

Building Custom Orchestration

Shell Script Orchestration

#!/bin/bash
# orchestrate.sh - Run multiple AI agents
 
# Phase 1: Analysis (parallel)
claude "analyze src/ for security issues" > analysis.md &
claude "check for deprecated dependencies" > deps.md &
wait
 
# Phase 2: Planning (sequential, uses phase 1 output)
claude "based on analysis.md and deps.md, create fix plan" > plan.md
 
# Phase 3: Execution (with confirmation)
echo "Review plan.md before proceeding"
read -p "Continue? (y/n) " confirm
if [ "$confirm" = "y" ]; then
    claude "execute the plan in plan.md"
fi

Python Orchestration

#!/usr/bin/env python3
"""orchestrate.py - Multi-agent workflow"""
 
import subprocess
import asyncio
from pathlib import Path
 
async def run_agent(prompt: str, output_file: str) -> str:
    """Run Claude Code agent and capture output."""
    proc = await asyncio.create_subprocess_exec(
        "claude", "-p", prompt,
        stdout=asyncio.subprocess.PIPE,
        stderr=asyncio.subprocess.PIPE
    )
    stdout, stderr = await proc.communicate()
 
    result = stdout.decode()
    Path(output_file).write_text(result)
    return result
 
async def parallel_analysis():
    """Run analysis agents in parallel."""
    tasks = [
        run_agent("find all TODO comments in src/", "todos.txt"),
        run_agent("identify unused imports", "unused.txt"),
        run_agent("check for type errors", "types.txt"),
    ]
    return await asyncio.gather(*tasks)
 
async def sequential_workflow():
    """Run agents in sequence."""
    # Step 1: Analyze
    analysis = await run_agent(
        "analyze the codebase structure",
        "analysis.md"
    )
 
    # Step 2: Plan (uses analysis)
    plan = await run_agent(
        f"based on this analysis, suggest improvements: {analysis}",
        "plan.md"
    )
 
    return plan
 
if __name__ == "__main__":
    asyncio.run(parallel_analysis())

Make-based Workflow

# Makefile for AI agent orchestration
 
.PHONY: analyze plan implement test all
 
analyze:
	claude "analyze src/ for code quality" > reports/analysis.md
	claude "find security vulnerabilities" > reports/security.md
 
plan: analyze
	claude "create implementation plan based on reports/" > reports/plan.md
 
implement: plan
	@echo "Review reports/plan.md before proceeding"
	@read -p "Continue? " confirm && [ "$$confirm" = "y" ]
	claude "implement the plan in reports/plan.md"
 
test: implement
	claude "run all tests and report failures"
 
all: test
	@echo "Workflow complete"

---

Agent Communication

File-Based Communication

Agents share context through files:

workspace/
├── context/
│   ├── requirements.md    # Input requirements
│   ├── analysis.md        # Agent 1 output
│   ├── plan.md           # Agent 2 output
│   └── implementation/   # Agent 3 output
└── agents/
    ├── analyzer.sh
    ├── planner.sh
    └── implementer.sh

Structured Output

Request JSON output for machine-readable communication:

> Analyze the codebase and output findings as JSON:
  {
    "issues": [...],
    "suggestions": [...],
    "priority": "high|medium|low"
  }

---

Error Handling and Recovery

Retry Logic

#!/bin/bash
max_retries=3
retry_count=0
 
while [ $retry_count -lt $max_retries ]; do
    if claude "complete the task" 2>/dev/null; then
        echo "Success"
        exit 0
    fi
    retry_count=$((retry_count + 1))
    echo "Retry $retry_count of $max_retries"
    sleep 5
done
 
echo "Failed after $max_retries attempts"
exit 1

Checkpoint and Resume

#!/bin/bash
# checkpoint.sh - Resume from last successful step
 
CHECKPOINT_FILE=".agent_checkpoint"
 
get_checkpoint() {
    [ -f "$CHECKPOINT_FILE" ] && cat "$CHECKPOINT_FILE" || echo "0"
}
 
set_checkpoint() {
    echo "$1" > "$CHECKPOINT_FILE"
}
 
step=$(get_checkpoint)
 
case $step in
    0)
        claude "step 1: analyze" && set_checkpoint 1
        ;&  # Fall through
    1)
        claude "step 2: plan" && set_checkpoint 2
        ;&
    2)
        claude "step 3: implement" && set_checkpoint 3
        ;&
    3)
        claude "step 4: verify" && set_checkpoint "done"
        ;;
esac

---

Resource Management

Rate Limiting

import time
import asyncio
 
class RateLimiter:
    def __init__(self, requests_per_minute: int):
        self.interval = 60 / requests_per_minute
        self.last_request = 0
 
    async def wait(self):
        now = time.time()
        wait_time = self.interval - (now - self.last_request)
        if wait_time > 0:
            await asyncio.sleep(wait_time)
        self.last_request = time.time()
 
# Usage
limiter = RateLimiter(requests_per_minute=10)
 
async def call_agent(prompt):
    await limiter.wait()
    # Make agent call

Cost Tracking

#!/bin/bash
# Track token usage across agents
 
LOG_FILE="agent_usage.log"
 
log_usage() {
    echo "$(date -Iseconds) | $1 | $2 tokens" >> "$LOG_FILE"
}
 
# After each agent call, log estimated usage
claude "analyze code"
log_usage "analyze" "estimated_tokens"

---

Best Practices

Do

• Define clear boundaries - Each agent should have a specific role
• Use checkpoints - Save progress for long-running workflows
• Validate outputs - Check agent output before passing to next stage
• Log everything - Track what each agent does for debugging
• Handle failures gracefully - Retry, skip, or escalate as appropriate

Don’t

• Chain too many agents - Context degrades, errors compound
• Share mutable state - Use files or explicit handoffs
• Ignore rate limits - Will get throttled or blocked
• Run without supervision - Review outputs, especially for production changes
• Trust blindly - Agents can make mistakes, always verify

---

Example: Full Refactoring Workflow

#!/bin/bash
# refactor-workflow.sh
 
set -e  # Exit on error
 
PROJECT_DIR="${1:-.}"
REPORTS_DIR="$PROJECT_DIR/.refactor-reports"
mkdir -p "$REPORTS_DIR"
 
echo "=== Phase 1: Analysis ==="
claude "analyze $PROJECT_DIR for refactoring opportunities, focus on:
- Code duplication
- Complex functions
- Missing tests
Output as structured markdown" > "$REPORTS_DIR/analysis.md"
 
echo "=== Phase 2: Prioritization ==="
claude "read $REPORTS_DIR/analysis.md and prioritize changes:
- High: Security, bugs
- Medium: Performance, maintainability
- Low: Style, minor improvements
Output as numbered list" > "$REPORTS_DIR/priorities.md"
 
echo "=== Phase 3: Planning ==="
claude "create step-by-step refactoring plan based on:
- $REPORTS_DIR/analysis.md
- $REPORTS_DIR/priorities.md
Include rollback steps for each change" > "$REPORTS_DIR/plan.md"
 
echo "=== Review Phase ==="
echo "Reports generated in $REPORTS_DIR"
echo "Review before proceeding to implementation"
cat "$REPORTS_DIR/plan.md"

---

Related

• AI CLI Tools - Installation
• Git - Cheat Sheet - For version control during agent workflows