Cheaper Models for Background Jobs: A Walkthrough

Background jobs rarely need the most capable model for every request. Tagging support tickets, extracting fields, and drafting internal summaries often have narrow requirements that a cheaper model can satisfy. The goal is not simply to reduce the price per token. It is to lower the cost of each usable result without creating extra retries, review work, or silent errors.

This walkthrough builds a practical process for moving one background task to a cheaper model, measuring the trade-off, and expanding only when the results justify it.

§ Step 1: Choose a bounded task

Start with a job whose output you can check. Extracting an invoice date is easier to evaluate than deciding whether an entire customer relationship needs attention. Prefer tasks with explicit inputs, a small output, and a clear definition of success.

• Good starting points: classification into fixed categories, structured extraction, and short internal summaries.
• Keep separate: actions that move money, change permissions, or publish content without review.

Write down what the worker may do and what remains outside its authority. A cheaper model should not gain broader permissions just because it runs unattended.

§ Step 2: Establish the current baseline

Collect a representative evaluation set from real jobs, including long inputs, incomplete records, and unusual cases. Remove sensitive data where necessary. Record the current model’s results, token usage, elapsed time, retries, and human corrections.

Define acceptance before testing alternatives. For classification, that might mean matching reviewed labels. For extraction, check required fields and their values. For summaries, use a short review rubric covering factual accuracy and missing information.

Compare cost per accepted result, not just cost per request. Include failed attempts and review effort where you can measure them.

§ Step 3: Test one cheaper candidate

Choose one candidate, such as Claude Haiku 4.5 or Gemini 2.5 Flash, and check its current token rates against your existing choice. Treat the candidate as an experiment, not an automatic replacement. Run the same evaluation set with the same instructions first, so model choice is the main variable.

Vana API provides one OpenAI-compatible endpoint, which lets you keep a common integration while testing supported models. If you already use an OpenAI SDK, keep the worker’s surrounding queue, storage, and validation logic unchanged. Create a Vana API account to run a small evaluation before moving production traffic.

§ Step 4: Tighten the worker contract

Once you have a comparison, reduce avoidable work. Send only the fields needed for the task rather than an entire document history. Specify the required output fields, allowed categories, and how to represent missing information.

• Ask for the result rather than an explanation when an explanation is unnecessary.
• Set an output limit that leaves enough room for a complete response.
• Tell the model not to invent values absent from the input.
• Validate the response in application code before accepting it.

Rerun the evaluation after changing instructions or trimming inputs. A shorter prompt is useful only if it preserves the information needed for a correct answer.

§ Step 5: Add bounded retries and escalation

Separate temporary request failures from unsuitable answers. Retry temporary failures with backoff and a firm attempt limit. Do not repeatedly resend an unchanged request when the output fails the same validation rule.

For invalid or ambiguous results, choose a defined next step: one corrective attempt, escalation to your existing higher-capability model, or human review. Use observable triggers such as missing required fields or conflicting extracted values rather than relying solely on the model’s stated confidence.

§ Step 6: Run a limited production pilot

Begin with a small share of eligible jobs. Preserve the original inputs and record the model identifier, prompt version, validation outcome, token usage, and escalation path. Make jobs safe to retry by preventing duplicate downstream writes.

Keep execution timing separate from pricing. Running a request in the background does not itself imply a discount. Vana API bills per real token with no monthly fee, so your comparison should use actual usage and the applicable model rates.

§ Step 7: Expand only when savings hold

Compare the pilot with your baseline. Check accepted-result cost, correction rates, completion time, and the share of jobs escalated. Review failures as well as averages.

If savings disappear after retries and review, narrow the eligible task or keep the original model. If quality remains acceptable and total cost falls, increase traffic gradually and retain an easy rollback. The useful outcome is a dependable routing policy, not a cheaper model attached to every job.