Cost-Per-Task vs. Dollars-Per-Million-Tokens

Your team wants to know which model to run its data engineering agent on. Anthropic's pricing page and its peers answer in dollars per million tokens. Your finance lead asks a different question: what does one finished dbt change cost?

Cost per task answers the finance question. It is total agent spend divided by the number of tasks that passed a completion rule. The spend includes every retry, every failed run and every run that returned nothing. A price per token feeds into that number, but tokens per task and the pass rate decide most of it.

The method has four steps. Write the completion rule, log every attempt, divide spend by completed tasks, and count empty runs in their own column. Add reviewer time only when you record it, because an estimated minute makes the number look precise when it is a guess.

A million tokens is also a weak unit across models. The same prompt can tokenize to a different count on a newer model. Our post on why a token is not a portable unit of work quotes Anthropic's migration guide at "roughly 1x to 1.35x" for one model change.

Cost per Task Divides All Agent Spend by Completed Tasks

Cost per task is the number that stays comparable when you swap the model, the agent or the prompt. A price per token changes meaning with each tokenizer. A pass rate hides what each attempt cost. Cost per task puts both in one figure.

Two runs can show why. A model at a low token price can retry three times and still fail. A model at a higher token price can pass on the first attempt. The per-token price favors the first model, and cost per task favors the second.

Write the Completion Rule Before You Count a Dollar

A task counts as completed only when it passes a rule you wrote before the run. Without a written rule, a generous reviewer marks more tasks as done, and the cost per task drops for no reason.

For dbt work, a practical rule has three parts:

• The change compiles, which dbt compile or dbt build confirms.
• The model and its tests pass with dbt build --select <model>.
• A person reviewed the diff and approved it.

ADE-Bench uses a strict version of the second part. Each task runs in Docker against a dbt project and a database, and a task passes only when all of its dbt tests pass. Use the same binary rule, so a half-working change counts as a failure.

Keep the rule fixed for the whole comparison. If you tighten it halfway, the early and late runs measure different things.

The Formula Counts Every Attempt and Only Completed Tasks

The numerator sums every attempt. The denominator counts only completed tasks. The two counts differ on purpose, because the tasks that passed carry the cost of each failed or empty run.

cost_per_completed_task =
    ( sum of model spend over all attempts, including failed and empty runs
    + reviewer_minutes x cost_per_minute )      <- only if you log the minutes
    / number of tasks that passed the completion rule

empty_run_rate = attempts that wrote no output / all attempts

Report empty_run_rate beside the result. Two setups can share a cost per task and still differ on how often they return nothing. An empty run needs a different response than a wrong one.

To estimate cost per task before you run anything, our token post gives a planning formula:

cost_per_task = (price_per_token x tokens_per_request x (1 + reasoning_overhead)
                 x (1 - cache_hit_rate x cache_discount)
                 x (1 - batch_share x 0.5)) / task_success_rate

The estimate is useful for a budget. The measured formula above is what you compare setups with, because it uses your own tasks and your own failures.

SWE-Bench Prices Opus 4.6 at 1.5X Haiku 4.5 per Resolved Task

The SWE-bench leaderboard publishes an average cost per task next to the resolve rate. Its default view runs each model through the same bash-only agent on SWE-bench Verified, a human-filtered set of 500 tasks. So the model is the only variable, and you can divide cost by resolve rate yourself.

SWE-bench Verified average cost per task, and the same cost divided by resolve rate.

Opus 4.6 lists at $5 per million input tokens, five times Haiku 4.5's $1. On this leaderboard, Opus 4.6 costs 1.67x more per task and about 1.47x more per resolved task. The price ratio overstates the task cost ratio by a factor of three.

Opus 4.5 resolves 10.2 more points than Haiku 4.5 and costs almost twice as much per resolved task. Whether those 10.2 points are worth it depends on what an unresolved task costs you. A person who finishes the task by hand is part of that cost.

These figures have limits. SWE-bench tasks are general software issues, with no dbt project and no warehouse. The harness also moves the score. Anthropic reports 73.3% for Haiku 4.5 on its own scaffold, against 66.6% on the leaderboard's bash-only agent.

Empty Runs Need Their Own Column Next to Pass Rate

A pass rate folds every failure into one number, and a failure that returns nothing behaves differently from a wrong answer. Our DataAgentBench A/B shows the gap. One agent ran 270 trials on each model, with the same prompts, tools and validators.

Cost per passing trial and empty runs from the same 540 trials.

On cost per passing trial, DeepSeek v4 pro wins by about 2.5x. The empty-run column changes the decision for some teams. DeepSeek wrote nothing on 29% of trials and Sonnet on 12%.

An empty run is easy to detect with a presence check, and it still costs you something. If your pipeline retries the task, the retry spend goes into the numerator. If a person picks it up, the reviewer term applies. At a 9-minute median, a retry also delays the result more than a Sonnet retry at 4 minutes.

Our write-up on cost per successful trial has the full failure breakdown. The two models failed a similar number of trials, and the failures took opposite shapes.

Skills Raised ADE-Bench Cost per Attempt by 21% and per Solved Task By 5%

In January 2026 we ran the 43 ADE-Bench tasks on Snowflake with Claude Sonnet 4.5, with and without our dbt skills. The pass rate on ADE-Bench and the cost moved together.

The skills run cost $3.01 more in total and solved three more tasks. So each additional solved task cost about $1.00 in model spend. That marginal figure is the one to compare against a person's time to finish a task by hand.

The 43 tasks across five dbt projects are small demo projects, so the dollar figures will be larger on a production project. The ratio between the two setups is the part to test on your own work.

A plugin experiment on Claude Sonnet 4.6 gave a smaller example of the same arithmetic. On the ADE-Bench task intercom003, the plugin turned a fail into a pass for about $0.13 extra.

Add Reviewer Time Only When You Log It

Reviewer time is the easiest term in the formula to invent. Put it in the formula only when you record it per task. An estimated "ten minutes per review" turns a measured number into a guess.

One low-effort way is a required field in the pull request template, such as review_minutes. A reviewer fills it in when they approve. After two weeks, you have a per-task figure from your own team.

Keep the reviewer term in a separate column as well as in the total. A setup that cuts model spend and doubles review time looks cheaper on the model column alone. What each agent costs a data team lists the other costs that sit above the license price.

Log Each Attempt in One Table to Get Cost per Completed Task

The measurement needs one row per attempt. Claude Code resets its /usage session totals when /clear starts a new session. So one session per task gives you a per-task cost estimate. For billing-grade numbers, the costs doc points to the Usage page in the Claude Console and to OpenTelemetry export.

-- agent_attempts: one row per attempt
-- (task_id, model, attempt_no, usd, wrote_output, build_passed, review_approved)
with per_task as (
    select
        task_id,
        model,
        sum(usd) as usd_all_attempts,
        count(*) as attempts,
        sum(case when not wrote_output then 1 else 0 end) as empty_attempts,
        max(case when build_passed and review_approved then 1 else 0 end) as completed
    from agent_attempts
    group by task_id, model
)
select
    model,
    count(*) as tasks,
    sum(completed) as completed_tasks,
    round(sum(usd_all_attempts), 2) as usd_total,
    round(sum(usd_all_attempts) / nullif(sum(completed), 0), 2) as usd_per_completed_task,
    round(avg(attempts), 2) as attempts_per_task,
    round(sum(empty_attempts) * 1.0 / sum(attempts), 3) as empty_run_rate
from per_task
group by model
order by usd_per_completed_task;

Run the same task list through each setup. An AI data engineering agent such as Altimate Code validates SQL in compiled code before it answers. The query above shows whether that check lowers attempts per task on your project.

Altimate has published no cost per task for Opus, Haiku or GPT models on dbt work. The SWE-bench and DataAgentBench rows above are the closest figures we can source. Where the tokens in one task go splits one Claude Sonnet 4.6 trial into cache reads, cache writes and the rest.

Rerun Your Model Choice on a Fixed Set of Your Own Tasks

Pick a fixed list of tasks from your own backlog, and write the completion rule first. Run each model or agent on the full list, and log every attempt in the table above. Compare cost per completed task and the empty-run rate together.

Repeat the run when a vendor ships a new model or changes a price. A new tokenizer or a new thinking default can move cost per task while the price per token stays flat.

Altimate Code swaps models without code changes. Install Altimate Code with npm install -g altimate-code, then point it at your own LLM key.