Scaling Agents

Horizontal Scaling

Because agents are stateless, scaling is as simple as adding more instances behind a load balancer. There is no data rebalancing, no leader election, and no partition reassignment required. Each agent independently handles requests by reading from and writing to shared S3 storage and the metadata store.

Load Balancing

Agents should be deployed behind a layer 4 (TCP) or layer 7 (HTTP/gRPC) load balancer. Any agent can handle any request, so simple round-robin load balancing works well. For gRPC clients, use client-side load balancing or an envoy proxy.

# Example: Kubernetes HPA for auto-scaling agents
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: streamhouse-agents
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: streamhouse
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

Sizing Guidelines

Use these guidelines as a starting point for agent sizing.

• Each agent can handle ~100MB/s throughput with default settings
• Memory: 2-4GB per agent (primarily for write buffers and segment cache)
• CPU: 2-4 cores per agent (compression and request handling)
• Start with 3 agents for high availability and scale based on throughput
• Monitor P99 latency and scale up when it exceeds your SLO