A support queue with fewer model calls

A support assistant starts with a reasonable idea: send each incoming message to a language model and ask for the next action. Then the queue fills with password-reset requests, order-status questions, and repeated policy queries. Most of these need a lookup or a fixed response, not generated prose.

The useful question is not whether a model can handle the task. It is whether interpretation or generation adds enough value to justify another dependency. Walk through one support workflow before deciding where the model belongs.

§ Step 1: Write down the result you actually need

Choose one request, such as “Where is order 1842?” Define the required result: authenticate the customer, retrieve the order, and display its current shipping status. None of those operations requires a language model.

A model might help recognize an order-status question phrased unexpectedly. That is a smaller responsibility than answering the entire request. Separate understanding the message from retrieving facts and executing actions.

§ Step 2: Mark the parts with exact answers

List the operations whose correct output follows from explicit rules or authoritative records:

• Checking whether an order number has a valid format.
• Looking up shipment events in the order system.
• Calculating a refund from recorded charges and policy rules.
• Selecting an approved response for a known status.

Keep these operations in ordinary code. Asking a model to calculate a total or reconstruct account state introduces uncertainty without adding useful flexibility. If a database already contains the answer, query it.

§ Step 3: Try the smallest adequate interface

Before building a conversational flow, test a status page, a search box, or a short form. A customer who already has an order number may finish faster with one field than with several chat turns.

For routing, begin with explicit controls and a few well-defined rules. Offer choices such as shipping, billing, and account access, then allow free text when those choices do not fit. Do not build an increasingly elaborate rules engine to avoid all model use; stop when ambiguous language becomes the actual problem.

§ Step 4: Give the model a bounded job

Suppose customers describe several problems in one paragraph. A model can help extract candidate topics or draft a concise summary for a support agent. Its output should remain a proposal that the application validates, not an authoritative account record.

Define permitted categories, required fields, and an “unclear” outcome. Validate returned identifiers against the source system. If the output is missing fields or cannot be validated, ask the customer a focused question or send the request to a person.

§ Step 5: Compare against a non-model baseline

Build a small evaluation set from representative requests, removing personal information where possible. Compare the existing form or rules-based route with the model-assisted version.

Measure correct routing, unnecessary handoffs, completion time, latency, and cost per resolved request. Review mistakes individually. A fluent summary is not an improvement if it omits the detail an agent needs. Keep a model only where the measured benefit matters to the workflow.

§ Step 6: Make model access replaceable

Once a bounded task earns its place, isolate the model call behind a small application interface. Vana API provides one OpenAI-compatible endpoint and also supports the Anthropic Messages and Gemini native protocols, allowing evaluation through a compatible client without redesigning the surrounding workflow.

Its per-real-token billing and lack of a monthly fee make usage costs worth tracking at the task level. Create a Vana API account when you are ready to evaluate that bounded step. Include retries, timeouts, and validation failures in your measurements, not just successful calls.

§ Step 7: Keep a path that works without generation

If the model is slow or unavailable, order lookup should still work. Preserve fixed responses, direct navigation, and human escalation. Cache stable approved text rather than regenerating it for every customer.

The final design may contain fewer model calls than the prototype. That is a useful outcome. The model handles language where flexibility helps; application code handles facts, rules, and state. Each part has a job you can explain, test, and maintain.