Start measuring

Quickstart

One command prices your agent from a trace export, with no account and no model call. Nothing moves to a cheaper model until a person on your team signs a route, or grants an agent a full-scope site key to publish routes on evidence; until then Ergoda measures the bill you already have.

Requires Python 3.11 or newer. On TypeScript, open the account the same way and follow the TypeScript SDK page from there.

1. Price an export you already have

pip install ergoda
ergoda onboard your-export.jsonl

The export is a JSONL file from Langfuse, LangSmith or OpenTelemetry, or the traces.jsonl that record writes (step 7). The format is detected. The file is read on your machine and nothing is sent anywhere. onboard prints:

  • The bill: model spend in the export at list prices, and a month of it when the export spans 24 hours or more. A shorter export gets no monthly figure, because one day cannot be scaled to a month.
  • The two models: the frontier (the model with the most spend) and a cheaper candidate from the same vendor. Each line says it was inferred; --escalate-to and --serve set them.
  • The movable share: how much of the spend is on steps where the candidate, priced on your logged token counts, saves more than Ergoda's fee. It is an upper bound. Whether the cheaper model can do each step is what the next command measures.
  • Ergoda's fee per step: the fee as a share of that step's model spend, and the saving net of the fee. A step is flagged when the fee is 10% or more of its spend, more than half the most it could save, or more than it could save at all. A small, cheap call pays the same fee per request as a large one; billing has the rules.
  • The next command, ergoda analyse, and what it will cost before you run it. It replays a sample of your calls on the cheaper model with your own API key, so it is the one step that spends money; --dry-run shows the plan for free.

The rest of the output checks that the export can be replayed, what outcome signals it carries, and which tools a wrong call to could be taken back.

No export yet? ergoda onboard --sample runs the same report on twenty synthetic support conversations that ship with the package. Its calls carry no prices, so it shows no dollar figure.

The steps below connect a project so Ergoda measures live traffic and can route it. They need an account.

Using a coding agent? Paste this:

Set up Ergoda in this project by following https://ergoda.com/agents.md. When it asks, I will approve the login in my browser.

The agent does every step below except two that are yours: opening the account, and approving its login in the browser.

2. Open an account

Go to https://app.ergoda.com/onboarding, enter your work email address and the six-digit code Ergoda emails you, and name your organization. It takes about a minute, and it is the only place an account is made. When you come back to the dashboard later, sign in with the email address you signed up with, then enter the six-digit code Ergoda emails you.

3. Connect this project

ergoda login

ergoda login prints a link and a code, and opens the link in your browser. Check that the page shows the same code, choose the organization if you belong to more than one, and press Approve. The terminal then writes ergoda.env (mode 0600) with ERGODA_URL, ERGODA_KEY and ERGODA_SITE, and adds it to .gitignore if the project is a git repository and does not ignore it yet. By default ERGODA_KEY is an ingest key: it sends events and fetches policy, and cannot read reports or change routes. A coding agent following agents.md to operate Ergoda runs ergoda login --scope full instead, and the approval page asks you to grant a full-scope site key, which can publish and revert this site's routes; the record names the key and you as the member who granted it. On a machine with no browser, run ergoda login --no-browser and open the link on any device where you are signed in.

That agent key is the only key the login issues. On your own machine, ergoda login --read also keeps a read key for the CLI in ~/.config/ergoda/credentials.json (or $ERGODA_CONFIG_DIR), which ergoda status uses to count your requests; the approval page shows that it was asked for. A read key reads every report of the organization, so never pass --read on an agent's host. ergoda logout deletes that file; keys stay valid until you revoke them under Keys on the dashboard. The control plane is the hosted service at https://api.ergoda.com; it is not part of the pip package, and running your own is not offered.

4. Write a starting policy

ergoda init --frontier anthropic/claude-opus-5 --step resolve_ticket
source ergoda.env

init writes routes.yaml, which routes nothing, and adds ERGODA_POLICY and ERGODA_DEFAULT_MODEL to ergoda.env, so that one file now sets all five variables. Load it in the shell or service that runs your agent, not only in the terminal where you ran init. A step is one named place where your agent calls a model. Write --frontier as provider/model; a bare id such as haiku is read as an OpenAI model and init warns when that looks wrong.

5. Wrap the call your agent already makes

import anthropic
from ergoda.sdk import Client

client = anthropic.Anthropic()
ergoda = Client.from_env()
create = ergoda.wrap({"anthropic": client.messages.create}, step="resolve_ticket")

# unchanged from here: same arguments, same response object
reply = create(model="claude-opus-5", max_tokens=1024, messages=messages)

On OpenAI, run init with --frontier openai/gpt-5.6-sol and wrap {"openai": client.chat.completions.create}, then call it with model="gpt-5.6-sol". Build the client once at module scope; in a Lambda or Cloud Run handler, wrap the handler body in with ergoda.request():.

6. Run your agent, then check

Run your agent the way you normally do, with ergoda.env loaded. Then:

ergoda status

status checks your policy, the environment, the key and your traces, and prints the one command to run next. After ergoda login it uses the CLI's read key to count the requests that reached Ergoda: a count above zero means the setup works, and the dashboard shows the same requests. Without a login, with only an agent key in the environment, it reports "key accepted" and cannot count them. While you develop, each new kind of decision a step makes prints one line to your terminal saying what it did and why.

7. Record your own traces

from ergoda import record, trajectory

create = ergoda.wrap({"anthropic": record(client.messages.create)}, step="resolve_ticket")
with trajectory(ticket_id):  # one agent run
    reply = create(model="claude-opus-5", max_tokens=1024, messages=messages)

record appends each request and response to traces.jsonl on your own disk and sends nothing anywhere. wrap hands its step= to record, so every line carries the step's name and the report lists the step by that name. Already on Langfuse, LangSmith or OpenTelemetry? Export from there instead, with a name field on each line.

8. Find what can move

ergoda onboard traces.jsonl --step resolve_ticket \
  --serve anthropic/claude-haiku-4-5 --escalate-to anthropic/claude-opus-5
ergoda analyse traces.jsonl --model anthropic/claude-haiku-4-5 --html report.html

onboard prints the bill from step 1, checks the export, lists which tools can be taken back, and prints the command that would write the route; it writes nothing. analyse replays your traces on the cheaper model with your own API key and reports agreement per step, with an interval. It needs that model's key (ANTHROPIC_API_KEY here) and stops, naming the variable, if it is not set. A replay the provider refuses, or one whose reply is not readable, is reported as a failure, never as a disagreement. A trace shorter than 24 hours gets no monthly cost or saving, because one day cannot be scaled to a month; record a day or more.

Before a model change. ergoda qualify traces.jsonl reads the same export offline and says, per step, whether the change could be measured: the volume and time span, which outcome signals exist (retries, escalations to a human, corrections and tool errors, which are proxies and not task success), whether calls link into trajectories, and how long the minimum evidence takes at your traffic. It writes a draft pre-registration with the margin, the outcome, the look date and who decides left for you to fill in, and the draft is refused until you have.

Where a route is approved. With the hosted service, in the dashboard: Approve and sign publishes the route with your name on it, and every process picks it up at its next policy fetch, replacing the routes.yaml you shipped. The two do not merge. onboard --approved-by "Your Name" is the same decision for a site with no control plane, or before one has published a policy: it adds the route to routes.yaml in place, keeping the site and policy id init wrote, and the SDK serves it from that file until a policy is published. Once the site has a published policy that does not serve the same route, `onboard --approved-by` refuses to write it and says to approve on the dashboard (--even-if-published writes it anyway, for processes that never fetch a policy). ergoda status names the policy in force and fails a check for any approval in the file the published policy does not serve. The SDK says the same on stderr, once, when a published policy first replaces the file.

What if

  • Ergoda is down or there is no policy: with no policy in force, every call goes to the model your code asked for. A routes.yaml you ship stays in force and keeps routing what it approves, whether or not Ergoda answers. Client.from_env() does not raise over missing configuration; it prints one line on stderr naming what is missing. model() and step() need ERGODA_DEFAULT_MODEL (or default_model=); without it a step with no route has no model to return, and the SDK says so on stderr the first time.
  • Your script exits: queued events get one delivery attempt of at most 1.5 s. If any are left, the SDK prints one line on stderr. Call ergoda.close() before exiting to wait for them with retries.
  • Latency: the decision runs in your process, in about a millisecond. A step the cheap model gets wrong is redone by your frontier model, so that step takes two calls.
  • You want out: set the route's enabled: false in routes.yaml, or remove the wrap line. Your call is then exactly the original.

Every term above is defined in concepts.