Start with the task
Choose for the work you need done, not for a leaderboard label. Write one representative task, its inputs, and a short acceptance rubric before comparing models. A model that is excellent at drafting may be a poor fit for strict extraction, long context, tool use, or code review. The models documentation (opens in a new tab) describes model selection; the providers documentation (opens in a new tab) explains that access depends on the provider configuration available to you.
Weigh the trade-offs
- Capability: Can the model follow the required format, reason through the task, and handle the input size?
- Privacy: Where will the input go, and what retention or contractual controls apply? A hosted model is not automatically private merely because the application is local. Confirm the provider's policy and remove unnecessary sensitive data.
- Budget: Consider both price and retries. A cheaper model that needs three attempts may cost more than a reliable one.
- Local deployment: A local model can reduce network exposure and recurring API charges, but hardware, speed, maintenance, and model quality still matter. Local does not mean every input is safe or every task is suitable.
Make a small comparison
Synthetic example: For 20 fictional support messages, compare Model A and Model B with the same messages and rubric: “classify correctly, return valid JSON, do not invent a refund.” Record pass/fail, latency, token or price data when actually available, and any retry. If Model A passes 18 and Model B passes 19, do not call B the winner until you check whether the extra pass justifies its cost and privacy constraints.
Recheck the provider's current model names, limits, and availability before committing to a choice. Verified 2026-09-06.