Configuration Reference

Complete reference for all ModelTrack configuration options: environment variables, JSON config files, and runtime settings.

Proxy environment variables

Cache settings

The proxy includes an in-memory LRU response cache. Identical requests (same model, messages, system prompt, and temperature) return cached responses, saving both cost and latency.

The cache key is a SHA-256 hash of the model, messages, system prompt, and temperature. Streaming requests are not cached.

API environment variables

Budget configuration

Budgets are defined in data/budgets.json. The proxy reads this file on startup and watches for changes.

{
  "budgets": [
    {
      "team": "ml-research",
      "app": "",
      "monthly_limit": 500.00,
      "action": "block"
    },
    {
      "team": "product",
      "app": "chatbot",
      "monthly_limit": 200.00,
      "action": "warn"
    },
    {
      "team": "product",
      "app": "search",
      "monthly_limit": 100.00,
      "action": "block"
    }
  ]
}

Routing configuration

Routing rules are defined in data/routing.json. They allow the proxy to automatically downgrade to cheaper models when teams approach their budget limits.

{
  "rules": [
    {
      "name": "budget-downgrade",
      "trigger": "budget_percent_above",
      "threshold": 70,
      "from_models": ["claude-sonnet-4-6"],
      "to_model": "claude-haiku-4-5",
      "provider": "anthropic",
      "action": "downgrade",
      "enabled": true
    },
    {
      "name": "openai-budget-downgrade",
      "trigger": "budget_percent_above",
      "threshold": 80,
      "from_models": ["gpt-4o"],
      "to_model": "gpt-4o-mini",
      "provider": "openai",
      "action": "downgrade",
      "enabled": true
    }
  ]
}

Collector namespace map

The collector uses data/namespace_map.json to map Kubernetes namespaces to ModelTrack teams for infrastructure cost attribution.

{
  "ml-research": "ml-research",
  "product-chatbot": "product",
  "default": "platform"
}

Keys are Kubernetes namespace names, values are ModelTrack team names. This allows the collector to attribute infrastructure costs (GPU, compute) to the correct team in the dashboard.

Proxy route paths

The proxy routes requests based on URL path: