A small classifier at the edge of a queue

Classification is one of the most useful jobs for a language model. It means reading a piece of text and assigning it to a known category: billing, technical support, feedback, spam, urgent, or routine. The output may be only one label, even though the model has understood a longer message.

Because the output is small, classification is a good place to start with a smaller model. The goal is not to choose the most capable model available. The goal is to find the smallest model that makes the right decision often enough for your application.

§ Start with clear labels

A model can only classify reliably when the categories are clear. Write a short definition for each label and include examples of messages that belong there. If two labels overlap, the model will face the same uncertainty as a human reader.

Suppose you are sorting support messages. “Refund requested,” “account access,” and “bug report” are useful categories because they describe different next steps. “Important,” “complicated,” and “customer issue” are harder to apply consistently.

Tell the model to return a fixed value, such as a JSON object containing a category field. Keep explanations out of the production output unless you need them for review. A predictable response is easier to validate and pass to the next part of your system.

§ Build a small test set

Before choosing a model, collect real examples. Thirty to one hundred messages can reveal much more than a few successful prompts. Include ordinary cases, short messages, spelling mistakes, mixed requests, and examples that sit near the boundary between two labels.

Mark the correct answer for every example. Then run the same prompt and test set against each candidate model. Record the number of correct labels, the kinds of mistakes, the response time, and the number of tokens used.

A single accuracy score is useful, but it does not tell the whole story. Misclassifying a general product question may be harmless. Sending a security report to the wrong queue may be serious. Give extra attention to the errors that create real work or risk.

§ Choose a model by consequence

A small model such as Claude Haiku 4.5 may be enough for straightforward categories with short inputs and clear examples. It can handle many routine sorting tasks while keeping the cost of each decision low.

For cases where a mistake is expensive, use a larger fallback such as GPT-6.1 Sol. One practical pattern is to send every message to the smaller model first, then escalate messages that are ambiguous, contain multiple requests, or receive a low confidence signal.

Confidence needs care. A number produced by the model is not automatically a reliable probability. Test it against your labeled examples. If low confidence does not consistently predict mistakes, use simpler rules, such as escalating messages that mention two categories or exceed a chosen length.

§ Keep the prompt and output stable

Classification systems become easier to maintain when the prompt changes slowly. Define the labels, provide a few representative examples, state the required output format, and explain what to do when no label fits. Use a separate “unknown” or “needs review” category when forcing a decision would be worse than asking for help.

Review a sample of decisions after launch. New wording, new products, and new customer problems can change the distribution of messages. Add difficult examples to your test set and rerun the evaluation before changing the model or prompt.

§ Try the smallest useful path

Vana API gives you one OpenAI-compatible endpoint for testing models with familiar SDKs. It also supports the Anthropic Messages and Gemini native protocols, and bills by real token use without a monthly fee. You can compare a small classifier with a fallback while keeping the application code straightforward.

Create a Vana API account, run your labeled examples, and choose the least expensive model that meets your accuracy requirement. That decision is more dependable than choosing by reputation alone, because it is based on the messages your system actually receives.