Concepts

Here is every term the quickstart and the dashboard use, in the order you will meet them. For the commands, see the quickstart.

How a step is decided

your code calls create(...)
        |
   is there an approved, enabled route for this step?
        |-- no  --> your frontier model, exactly as before
        |-- yes --> the cheaper model proposes
                        |
                  free checks, in your process
                  (format, tools, loops, the assurance level)
                        |-- pass   --> the cheaper model's answer is used
                        |-- refuse --> your frontier model redoes the step

Every uncertain path ends at the model you were already paying for.

The words

• Step: one named place where your agent calls a model, such as resolve_ticket. You name it with step= on wrap.
• Frontier model: the model your code calls today. It is the default for every step, always.
• Cheaper model: the model a route may serve instead, and only for the steps a person approved.
• Route: the rule for one step: which cheaper model serves it, which frontier model takes over when a check refuses, and who approved it. routes.yaml holds your routes; ergoda init writes a file with none.
• Policy: all of a site's routes and the models they may use. It is the routes.yaml you ship, or the version published on the control plane, and a published policy replaces the file.
• Control plane: the hosted service at api.ergoda.com. It receives what your agent reports (hashes, counts, token totals and costs, never prompts or responses), accumulates it, publishes the policy and serves the dashboard.
• Site and site key: a site is your organization's ID. A site key is the secret an agent or a person presents, and starts with hk_. An agent key sends events and fetches policy; a read key reads reports; a full key also changes routes. An account is opened on the web, at app.ergoda.com/onboarding; ergoda login then writes an agent key to ergoda.env, and ergoda login --read also keeps a read key for the CLI itself.
• Trace: a recorded model call, request and response, one JSON line each. record writes one to your own disk; Langfuse, LangSmith and OpenTelemetry exports are read as they are.
• Chain: one run of your agent, from its first model call to its last (also called a trajectory).
• Agreement: the share of replayed steps where the cheaper model did what the frontier model did, reported with an interval. It is measured on your traces and says nothing about outcomes the replay cannot see.
• Parity: the cheaper model gets your tasks done as well as your frontier model does. It is the goal. Agreement and the escalation rate are how it is estimated, per step.
• Approval: the name on a route. No route to a cheaper model exists without one: ergoda onboard --approved-by "Your Name" writes it.
• Escalation: a step that was meant for the cheaper model but went to the frontier model, because a check refused the proposal or a limit was reached. It costs one extra call and is never an error.
• Recoverable tool: a tool whose effect can be taken back, such as a lookup or a draft. You declare these. By default there are none, and with none declared a route serves nothing cheaper under the default assurance level.
• Irreversible action: a tool call that cannot be undone, such as a refund or a sent email. Under the default level the frontier model makes every one of these. ergoda onboard calls such a tool withheld: it stays on the frontier. A tool it cannot classify from its name and description is undecided, and also stays on the frontier until you decide.
• Governed request: one request your agent sends through Ergoda (the SDK or the proxy), or one request in your own traces that observe mode reads. It counts once, whichever model answered it, however many calls the checks made for it and however many times the same trace is read. It is the billing unit, from the first request.
• Refund: ask within your first 60 days of paid use, counted from your first charge, and we refund what you were charged in those 60 days. No reason is needed, and it is once per organisation (terms §5). The dashboard's invoice preview shows the last day to ask.
• Observe mode: ergoda observe run reads the traces you already log and measures from them, without being in the request path at all. Where a trace names the end user (Langfuse's userId, an OpenTelemetry span's enduser.id or user.id, LiteLLM's end_user, a request's user), it sends a digest of that id, as the SDKs do.
• Caller: which of your customers a request was for. The SDKs and the proxy read it from the request's own end-user field, or you pass it; only a digest salted with your site id reaches Ergoda. It is what the Customers page groups cost per successful task by.
• Attestation: a signed record of which steps ran on which model, and at what escalation rate, over a window. If most of the window went unchecked, it says so and makes no parity claim.

Assurance levels

ERGODA_ASSURANCE sets how much a cheaper model may do without the frontier model signing it. A level can only tighten a route.

• decided (default): the frontier model makes every irreversible decision and the cheaper model carries out the rest. Once a route's own traffic shows the cheaper model making one kind of decision the way the frontier would, that kind can move to it; see autopilot.
• maximum: the frontier model redoes every irreversible action, whatever the evidence says.
• high: the frontier model reviews every irreversible action before it runs.
• standard: the route exactly as approved, with no extra review.

Autopilot

On an approved, enabled route, autopilot may loosen the route's checks when your own traffic shows it can. The limit is a 2% added failure rate at 95% confidence. It never adds a route and never changes which models a route uses. Any move it makes is recorded under the name ergoda:default, because no person accepted that limit. It undoes its own moves when the evidence turns against them, and on a route that sees no traffic for 90 days. To switch it off for your site, use the dashboard's Switch autopilot off page or send {"by": "your name", "enabled": false} to POST /v1/sites/SITE/autopilot. From then on every move toward the cheaper model waits for a named person's approval. Switching off does not undo moves already made: they stay until you revert them or an escalation takes them back, and a route still serving traffic keeps them. {"by": "your name", "default": true} puts the site back on the default.

Cost ceiling (both SDKs and the proxy)

The Python SDK, the TypeScript SDK and the proxy count what a route spends against what your frontier model alone would have cost on the same steps. When a route costs more, it goes back to the frontier model. It may route again once its bill has recovered; a route pinned twice on clear evidence that it costs money stays on the frontier for the rest of the process. The TypeScript SDK enforces it from @ergoda/sdk 0.1.1 and takes your model prices in its prices option (see TypeScript SDK); the proxy enforces it from ergoda 0.1.1. Version 0.1.0 of either package does not.

Environments

Every event says which deployment sent it: production (the default), staging, development or eval. Set it with ERGODA_ENVIRONMENT, with the environment option in either SDK, or with --environment on the proxy and observe mode. Any other word stops the client or the proxy from starting, so a typo such as prod cannot file production traffic under another name.

The setting comes after 0.1.5. ergoda 0.1.5 and @ergoda/sdk 0.1.5 and earlier do not read ERGODA_ENVIRONMENT, and their rows are stored as production whatever it says. Python's environment= and the proxy's --environment fail on them; a JavaScript caller's environment option is ignored like the variable. Until you upgrade, staging or CI traffic sent with your key counts as production evidence.

What it changes: only production traffic is evidence. Routes are loosened, graduated, taken back and expired on production rows alone, a model change counts only when production sees it, and the dashboard's figures and the attestation read production only. A staging stack or a CI eval that shares your site key cannot license a route, keep one alive or revoke one.

What it does not change: routing and billing. The same policy serves every environment, with the same checks and the same cost ceiling, and every environment is metered. The dashboard says how many requests came from outside production. To see one environment's own traffic, ask for it by name: GET /v1/sites/SITE/steps?environment=staging. Those rows carry "evidence": false and are never added to production's.

Turning a step off

On the dashboard, use Send back to frontier beside the step on the Decisions page. It needs nobody's approval, because this direction never does.

In routes.yaml, set the route's enabled: false.

From a script, one request does the same for every process on the site. It needs a full key (the dashboard's API keys page; the agent key ergoda init issued is refused with key is scoped to ingest), and step_cluster_id is the id shown beside the step on the Decisions page, which is also the route's step_cluster_id in the policy:

curl -X POST "$ERGODA_URL/v1/sites/$ERGODA_SITE/policy/revoke" \
  -H "Authorization: Bearer $ERGODA_FULL_KEY" -H "Content-Type: application/json" \
  -d '{"step_cluster_id": "route-3f9a1c2e", "by": "Your Name", "reason": "why"}'

The step then serves your frontier model. It answers 404 when no policy has been published for the site yet; a route that lives only in a local routes.yaml is turned off in that file.

What the record keeps

Four things a review usually asks about later, each kept without you setting anything up:

• Who moved what, and when. Every route names the person who approved it and when, and every published version of the policy is kept with who published it (GET /v1/sites/SITE/policy/history, and the audit log in Security and data). A change autopilot made is recorded under ergoda:default, not a person's name. Sending a step back to the frontier is one button on the Decisions page.
• What not routing would have cost. One trajectory in twenty that reaches a routed step is served by your frontier model alone (the cost holdout), under every assurance level except standard. Comparing whole held-out trajectories with routed ones, in what your provider billed (or, where it returned no bill, its token counts at list price), gives the saving; a cheaper model that takes more steps to finish shows up as a smaller saving, or none (Billing).
• When the model behind a name changes. Every event records the model your provider said answered. If that changes under a route's cheaper or frontier model name (a new snapshot behind an alias, a gateway moving a slug), anything autopilot loosened on that route before the change goes back to what you approved, and the alert says the model changed. This only sees providers that name the model in their response.
• Staging is not evidence. Only production traffic licenses, keeps or takes back a route (Environments).

Latency

What Ergoda adds to a step, measured, with the conditions each number was measured under:

• The decision, in your process. Finding the route and running the free checks make no network call. In the Python SDK, with a model that answers instantly, a warm step on a 200-turn conversation took about 1 ms at both p50 and p95, and the first step of a process 6 ms; importing the SDK took 144 ms and loading the policy 47 ms, once per process (Apple Silicon, load average 3 to 4, 2026-09-18; the same cold path read 230, 96 and 12 ms at load average 4 to 5). These are our machine's numbers, not a promise about yours; experiments/path_overhead_bench.py in the source re-measures them.
• The proxy. On the same machine, a 200-turn request through the proxy on loopback took 3 ms at p50 and 5 ms at p95 with an instant upstream.
• The control plane is not in the call. The policy is cached and refreshed behind your calls, and events are sent from a background thread. One exception: with no routes.yaml shipped, the first step of a process waits for the first policy fetch, which gives up after a 5-second timeout if the control plane does not answer. Ship the file to avoid it (Python SDK; the TypeScript SDK has no local file, so its first step always waits for that fetch). In serverless code, request() ends an invocation with one delivery attempt of at most 1.5 s; an invocation with evidence still being gathered behind it (a graduation audit's frontier call, a shadow comparison) first waits up to 30 s for that to finish.
• An escalated step makes two model calls, one after the other. A step the cheaper model answers and the checks accept makes one call, to the cheaper model. A step the checks refuse makes the cheaper model's call and then your frontier model's, so it takes about the cheaper call's time longer than calling the frontier directly. Levels that have the frontier review or redo irreversible actions add a frontier call to each one. On recorded τ-bench runs (10 to 25 tasks per arm, one machine, one provider, before later changes that shortened a refused step's slowest cases), routed steps averaged 1.5 to 2 model calls where the frontier alone makes one, and wall time per step was 7% to 70% above calling the frontier directly; the cheaper model alone was about 25% faster per step. Those task counts are small enough that the range is a shape, not a figure to plan a p95 budget on.

How often the second call happens is the escalation rate, which the dashboard's Evidence page shows per step. The method and the full tables are in experiments/RESULTS-latency.md and experiments/RESULTS-overhead.md in the source.

If Ergoda is unreachable, or goes away

The routing decision and the checks run in your process; the control plane receives counts and hashes and publishes the policy. What happens without it:

• Your calls keep running. An unknown step, an unreachable control plane or a policy with nothing to say serves your frontier model. An outage never sends a step to a cheaper model it was not already routed to.
• The last policy keeps routing for 24 hours of outage, then stops. An SDK that fetches its policy, and the proxy under --policy-from-control-plane, serve the last good one (the routes.yaml you shipped, or the last one fetched) until the control plane has failed to answer for 24 hours, and then serve your frontier model until it answers again. That bound is how a revert reaches a process that lost its connection. ERGODA_POLICY_MAX_AGE_S=0 (policyMaxAgeMs: 0 in TypeScript) serves the last good policy for as long as the process runs.
• Routing with no control plane at all. The Python SDK and the proxy route from a local policy file and never expire it when they are not told to fetch one: leave ERGODA_URL unset in the SDK, and run the proxy from --policy without --policy-from-control-plane (with --no-control-plane if nothing is to be measured). Then nothing is sent, and nothing moves the routes but you. The published policy is the policy member of GET /v1/sites/SITE/policy; save it as routes.json. The TypeScript SDK has no local policy file: a new process with no control plane serves your frontier model.
• Leaving. Remove the wrap line and your call is exactly the original.
• The source. The SDKs, the proxy and the analyser are published under FSL-1.1-ALv2: you may use, change and run them for anything except offering a competing product, and each version becomes Apache 2.0 two years after it is released. The control plane (the dashboard, the accumulated evidence, autopilot, the attestation) is not published, and running your own copy of it is not offered.