AgenticSwarmBenchAgenticSwarmBench

Configuration

YAML config files, environment variables, CLI arguments, and the 4-source hierarchy.

Configuration Hierarchy

AgenticSwarmBench merges configuration from four sources. Highest priority wins:

1
CLI arguments

--endpoint, --model, --context-tokens

2
Environment variables

ASB_ENDPOINT, ASB_MODEL

3
YAML config file

asb --config bench.yml speed ...

4
Defaults

Sensible defaults for everything

Environment Variables

VariableDescription
ASB_ENDPOINTOpenAI-compatible endpoint URL
ASB_MODELModel name
ASB_API_KEYAPI key for the endpoint
ASB_CONTEXT_TOKENSDefault context size in tokens
ASB_CONTEXT_PROFILEDefault context profile
ASB_MODEL_CONTEXT_LENGTHModel's max context window - skips larger scenarios
ASB_DEFEAT_CACHEDefeat prefix caching (true/false)

YAML Config File

Create a YAML file and pass it with --config:

bench.yml
endpoint: http://my-gpu-server:8000
model: my-model
suite: standard
defeat_cache: true
asb --config bench.yml speed

Common Patterns

CI/CD Pipeline

Use environment variables so credentials stay out of config files:

.env
ASB_ENDPOINT=http://inference-server:8000
ASB_MODEL=my-model
ASB_API_KEY=sk-...
asb speed --suite quick --format json -o results.json

Team Config

Commit a YAML config to your repo so everyone runs the same benchmark:

asb.yml
endpoint: http://shared-gpu-server:8000
model: deepseek-r1
suite: full
defeat_cache: true

Override Anything

CLI args always win, so you can override any config:

asb --config bench.yml speed \
  --model different-model \
  --suite quick