Start measuring

The proxy

For agents you cannot edit. The proxy is a small server you run in your own infrastructure. Your agent points its provider base_url at it, and the proxy makes the same routing decision the SDK would, calls your provider with your key, and reports only derived metrics (hashes, counts, token totals) to Ergoda. If you can edit the agent's code, the Python SDK or the TypeScript SDK is simpler and adds no hop.

There is no published container image. The proxy is part of the ergoda package, so you install it with pip, or build an image from pip in three lines (below).

Run it

Open an account at app.ergoda.com first; ergoda login then connects this machine to it.

pip install 'ergoda[server]'
ergoda login               # approve it in the browser; writes ERGODA_URL, ERGODA_KEY, ERGODA_SITE to ergoda.env
ergoda init --frontier anthropic/claude-opus-5 --step resolve_ticket
source ergoda.env          # ERGODA_URL, ERGODA_KEY, ERGODA_SITE, ERGODA_POLICY
export ANTHROPIC_API_KEY=sk-ant-...        # your provider key; it never leaves your network
export ERGODA_PROXY_TOKEN=$(openssl rand -hex 24)   # what your agent presents to the proxy

ergoda dataplane serve --policy "$ERGODA_POLICY" --site "$ERGODA_SITE" \
  --control-plane "$ERGODA_URL" --policy-from-control-plane --port 8080

--site must be the policy's own site_id, which init wrote. The proxy refuses to start without a control plane to report to, unless you pass --no-control-plane (a local trial; nothing is measured). Behind `--host 0.0.0.0 it refuses to start without ERGODA_PROXY_TOKEN`. --policy-from-control-plane makes a route approved in the dashboard reach the proxy without a restart; routes.yaml stays in force until the first fetch succeeds. The proxy writes ergoda-audit.jsonl and ergoda-dataplane.sqlite (shadow quota, breakers, queued metrics) into the directory it runs in; name other places with --audit-log and --state-db.

Point your agent at it

Change one setting, the base URL, and present the proxy token where the provider's key went. The provider key stays in the proxy's environment.

import anthropic

client = anthropic.Anthropic(base_url="http://localhost:8080", api_key=PROXY_TOKEN)
reply = client.messages.create(model="claude-opus-5", max_tokens=1024, messages=messages)
import openai

client = openai.OpenAI(base_url="http://localhost:8080/v1", api_key=PROXY_TOKEN)
reply = client.chat.completions.create(model="gpt-5.6-sol", messages=messages)

The proxy speaks POST /v1/messages (Anthropic, with its event stream) and POST /v1/chat/completions (OpenAI, with stream: true). A worked request:

curl -s http://localhost:8080/v1/messages \
  -H "x-api-key: $ERGODA_PROXY_TOKEN" -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{"model": "claude-opus-5", "max_tokens": 64,
       "messages": [{"role": "user", "content": "Say hello."}]}' -i

The response is your provider's own, with headers saying what happened: X-Ergoda-Variant (the model that answered), X-Ergoda-Escalated (no, or the reason the step went to your frontier model) and X-Ergoda-Route. A wrong token gets 401. A request with no approved route goes to the model it named, exactly as before.

Check a sample request (an OpenAI-shaped JSON body) against your policy before pointing traffic at it:

ergoda dataplane check request.json --policy routes.yaml

It says whether a route matches and which model it serves. The catalogue of tools is part of the match, so the same messages with and without tools are two different steps.

Tracing

If your agent traces with OpenTelemetry, the proxy can add its decision to the same trace: one ergoda.step span per request, a child of the span your HTTP client sent the request from. It carries the attributes the Python SDK writes: the model asked for and the model that answered, ergoda.tier (cheap or frontier), whether and why the request escalated, the route and policy, a shadow the request was drawn for, the environment, the cost when known, the event id (the same id as X-Ergoda-Event), and error.type with an ERROR status when the provider call failed. It has no gate attributes, because the proxy runs no checks on an answer before serving it. No prompt, response, caller id or token count is on it.

Two things have to be true. The proxy's process needs an OpenTelemetry tracer, for example from opentelemetry-instrument:

pip install opentelemetry-distro opentelemetry-exporter-otlp
OTEL_SERVICE_NAME=ergoda-proxy opentelemetry-instrument \
  ergoda dataplane serve --policy "$ERGODA_POLICY" --site "$ERGODA_SITE" ...

And the request has to carry your trace context: the W3C traceparent header, which your HTTP client instrumentation sends. A request without one gets no span; the proxy never starts a trace of its own. Tracing comes after 0.1.5.

In a container

FROM python:3.12-slim
RUN pip install --no-cache-dir 'ergoda[server]'
EXPOSE 8080
CMD ["ergoda", "dataplane", "serve", "--policy", "/data/routes.yaml", "--host", "0.0.0.0", "--control-plane", "https://api.ergoda.com", "--policy-from-control-plane", "--audit-log", "/data/ergoda-audit.jsonl", "--state-db", "/data/ergoda-dataplane.sqlite"]
docker build -t ergoda-proxy .
docker run -p 8080:8080 -v "$PWD/data:/data" \
  -e ERGODA_SITE -e ERGODA_KEY -e ERGODA_PROXY_TOKEN -e ANTHROPIC_API_KEY \
  ergoda-proxy

Put routes.yaml in data/; the command above writes the audit log and state database beside it, so the directory must be writable by the container's user.

What to know

  • The proxy sees your prompts and responses, because it forwards them. It sends Ergoda derived metrics only, never a prompt or a response.
  • A request with no approved route goes to the model it named. A cheaper model's answer that a check refuses is redone by the frontier model the route names. If the control plane is unreachable, the proxy keeps serving with the last policy it had. Under --policy-from-control-plane that lasts 24 hours (ERGODA_POLICY_MAX_AGE_S); then it forwards every request to the model the caller named until the control plane answers.
  • Outside production, start it with --environment staging (or development, eval; ERGODA_ENVIRONMENT works too). It marks everything it reports with that name, and only production traffic counts as evidence (environments). Any other word stops it from starting. This comes after 0.1.5: the 0.1.5 proxy rejects --environment, ignores ERGODA_ENVIRONMENT and reports every row as production.
  • The proxy reads which of your customers a request is for from the request's user (or safety_identifier, or Anthropic's metadata.user_id), or from an X-Ergoda-Caller header, and sends only a digest of it. The dashboard's Customers page and ergoda customers show cost per successful task for each.
  • Each response carries X-Ergoda-Trajectory, the digest of the task the request belongs to (absent when the request has no user turn). When the task is over, report how it ended to the proxy with the same bearer token: POST /v1/task-outcomes with {"trajectory": "<digest>", "outcome": "success"} (or failure, unknown; optional source and cohort labels). An outcome outside the three, or a key that is not the digest, is refused with 422. The cost per successful task report reads it.
  • The proxy is source-available under the same licence as the package.
  • Terms such as route, escalation and frontier model are defined in concepts.