Cost per Merged Pull Request: A Practical Guide

A low token price does not tell you what it costs to deliver a working change. For that, use a metric tied to an outcome: cost per merged pull request. This walkthrough builds a practical measurement process, from defining the denominator to comparing workflows. Start with API spend, then add engineering time if you need a fuller economic picture.

§ Step 1: Define the metric and its boundaries

Choose one repository or team and a fixed reporting period. Write down which pull requests qualify before collecting results. For example, include product changes and bug fixes, but exclude automated dependency updates unless those are part of the workflow you are evaluating.

API cost per merged pull request = attributable AI API spend ÷ qualifying merged pull requests.

Keep unsuccessful attempts in the numerator. Excluding abandoned work would make an expensive workflow look artificially efficient. If nothing merges during the period, report spend and zero merges separately rather than calculating a ratio.

§ Step 2: Tag usage at the task level

Assign each coding task an identifier before the first model request. A pull request number may not exist yet, so use a temporary task ID and connect it to the pull request later.

Record the following fields in your usage ledger:

• Repository, task ID, and pull request number when available.
• Request timestamp, client, and requested model.
• Input, output, and any separately billed token categories.
• Actual request charge, currency, and billing source.
• Task outcome: merged, open, abandoned, or superseded.

With Vana API, a shared endpoint can provide a consistent routing point across supported coding clients. Its homepage lists Claude Code, Codex, Gemini CLI, and OpenAI-style integrations. Keep task attribution in your own ledger rather than assuming every client forwards custom metadata. ([vanaapi.com](https://vanaapi.com/))

§ Step 3: Capture the entire attempt

Attribute every request made for a task, not just the request that produced the final patch. Include planning, implementation, debugging, review, and revisions. Record retries when they incur charges; do not assume every failed request is billed.

Use billed charges as the monetary source of truth. If you estimate costs from token counts, preserve the applicable rates and distinguish estimates from settled charges. Reconcile the ledger with your billing records before comparing results.

§ Step 4: Allocate shared and unfinished work

Some sessions support several pull requests. Choose an allocation rule, such as splitting the charge equally among linked tasks, and apply it consistently. Keep unassigned usage visible as overhead instead of silently dropping it.

For a simple weekly report, divide all in-scope weekly spend by qualifying merges that week. Label this a period metric: it mixes work started earlier with work that may merge later. For a task-cohort view, group tasks by start date and follow their costs and outcomes through a stated cutoff.

§ Step 5: Calculate a transparent example

Suppose a weekly ledger contains $120 attributable to merged tasks, $30 attributable to abandoned tasks, and $10 of shared overhead. With 20 qualifying merges, API cost per merged pull request is ($120 + $30 + $10) ÷ 20 = $8.

These are illustrative figures, not provider prices. Report the three spending categories alongside the result so readers can distinguish implementation cost from unsuccessful attempts and overhead.

§ Step 6: Compare workflows without rewarding shortcuts

Compare similar task categories rather than treating a documentation edit and a cross-service refactor as interchangeable. Track review time, completion time, reverts, and post-merge defects alongside API cost. Avoid interpreting a cheaper merge as a better outcome when quality or scope changes.

If useful, calculate a separate fully loaded metric: API charges plus engineering hours multiplied by an explicitly chosen hourly cost, divided by qualifying merges. Keep it separate from the API-only measure.

§ Step 7: Run one controlled improvement

Change one variable at a time: context size, retry limits, review instructions, or model selection. Set a fixed evaluation window and compare cost, completion, and quality for similar tasks.

Create a Vana API account to run a small, tagged pilot. Start with one repository, reconcile the charges, and expand only after you can explain both the spend and the merged outcomes.