The bill after a week of merged pull requests

Money mattersEN2026-10-30

A coding session can be inexpensive and still produce an expensive pull request. The first patch may need several corrections, consume a reviewer’s afternoon, or never merge. For a team evaluating coding agents, token prices are useful inputs, but they are not the final unit of comparison.

A more practical measure is cost per merged pull request: the resources spent across a defined batch of work, divided by the pull requests from that batch that actually merge. Keep API spend and human effort visible separately before combining them.

§ Two workflows, the same kind of work

Consider two workflows tested on comparable small maintenance tasks. Both use the same model, such as Claude Sonnet 5.5, so the experiment focuses on workflow rather than model selection.

Short-context workflow: Give the agent the issue and a narrow file selection. Ask for a patch immediately, then supply more context when tests or review reveal omissions.
Context-first workflow: Let the agent inspect relevant code and tests, establish a short plan, then implement and check the change before handing it to a reviewer.

The first workflow may spend fewer tokens initially. The second may spend more before producing a patch. Neither observation establishes which is cheaper per merged result.

§ A side-by-side accounting example

The following figures are hypothetical bookkeeping inputs, not measured results or Vana API prices. Each workflow receives 40 comparable tasks, with one intended pull request per task and the same observation window.

Short-context workflow: $48 in total API charges, 30 merged pull requests, and 300 minutes of human review and correction across the entire batch.
Context-first workflow: $72 in total API charges, 36 merged pull requests, and 180 minutes of human review and correction across the entire batch.

On API spend alone, the short-context workflow costs $1.60 per merged pull request. The context-first workflow costs $2.00. That makes the shorter workflow cheaper on this narrow measure, even after including charges for unsuccessful attempts.

Now apply an illustrative labor rate of $60 per hour. The short-context workflow incurs $300 in human effort, bringing its combined cost to $348, or $11.60 per merge. The context-first workflow incurs $180 in human effort, bringing its combined cost to $252, or $7.00 per merge.

The ranking changes because the measurement changes. These figures do not prove that longer context saves money. They show why a lower API bill and a lower delivery cost can point in different directions.

§ Keep the denominator honest

Count failed attempts: Include charges and review time for abandoned branches, retries, and rejected patches associated with the batch.
Match task difficulty: Compare similar repositories, change sizes, test requirements, and acceptance criteria. Random assignment helps reduce selection bias.
Fix the window: Record unresolved work separately. A pull request still awaiting review is not necessarily a failure, but excluding its costs makes early results look better.
Check quality after merge: Track reverts, follow-up fixes, and escaped defects. A merge is a useful checkpoint, not proof of a durable result.

§ Separate access costs from workflow costs

Vana API provides one endpoint at https://api.vanaapi.com that supports OpenAI-compatible requests, Anthropic Messages, and Gemini native protocols. It works with Claude Code, Codex, Gemini CLI, and OpenAI SDKs, making it practical to keep access consistent while comparing coding workflows.

Billing is per real token with no monthly fee. For the experiment, attribute actual charges to each task rather than estimating cost from the final response alone. Repeated context, tool-result messages, and correction rounds can all affect usage. Record the model, settings, and workflow version alongside each result.

Create a Vana API account when you are ready to run a small, instrumented comparison. Start with API cost per merge and review minutes per merge as separate columns. Combine them only with an explicit labor assumption, and keep post-merge quality beside both. The useful winner is the workflow that delivers acceptable changes at a sustainable total cost.