The Model Landscape, Without the Leaderboards

New model names arrive every few weeks. The leaderboards reshuffle. A launch post calls one of them the best model in the world, and three weeks later a different one wears the crown. From the outside this looks like a field you have to track full-time just to stay current.

You don't. Underneath the churn the shape is stable, and has been for a while. Models sort into three tiers by what they cost and what they can do, and the tiers stay put even as the names inside them rotate.

The frontier / reasoning tier holds the most capable, most expensive, slowest models. They do multi-step reasoning, hold long and messy context, and handle the genuinely hard, ambiguous problems where the path isn't obvious. Novel code architecture, deep analysis, judgment calls. You pay the most per token and you wait the longest for an answer.

The workhorse tier sits in the middle. Strong general capability at a fraction of frontier cost. This is where most real production work belongs: summarizing, classifying, drafting, writing standard code, pulling structured fields out of messy text. Balanced on cost and speed, and the right default once you actually know what the task is.

The cheap / fast tier is the small, quick, low-cost models. They shine on high-volume, simple, latency-sensitive jobs. Routing a request to the right handler, a yes/no classification, autocomplete, a cheap pre-filter that runs before a more expensive step. Ask them to reason across many steps or recall broad world knowledge and they fall over.

Treat the names as disposable and the tiers as durable. When a new model drops, the only real question is which tier it lands in and whether it changes your answer for a specific task. Most of the time it doesn't.

There's a second confusion worth clearing early. "The model" and "the product you reach it through" are not the same thing. The model is the trained weights. The chat app, the API, the IDE assistant, the agent harness each wrap those weights in their own system prompt, their own tools, their own hidden context handling. A prompt that behaves one way in a chat app can behave differently through the raw API, because the app quietly injects instructions you never see. When someone says "the model got worse this week," it's often the product that changed its system prompt, not the weights. If you want to judge a model honestly, reach it through the API, where you control the surface. The chat app is not a clean room.

The selection move follows from all of this. Define what the task actually needs, then reach for the cheapest tier that clears the bar, not the most capable one you can afford.

The instinct most newcomers carry is to start at the top and only come down if cost hurts. Flip it. Start where you think the bar lives, prove the tier below it fails, and only then settle. Cost and latency both improve as you move down, so every tier you can drop lowers the bill and speeds up the response at the same time.

The selection axes give you the language for "what does the task need." Context window is how much text the model can hold at once. Long-document work needs a big one, short calls don't. Modality is whether it handles text only, or vision, or audio, and paying for vision on a text task is pure waste. Latency matters for anything a human waits on, less for an overnight batch job. Cost per token scales with volume, so at high volume your tier choice is most of your bill. Reasoning depth is the real divider. Does the task need genuine multi-step thinking, or is it pattern-matching dressed up to look hard?

Over-buying capability. This is the most common and most expensive mistake, so name it first. Teams default to the frontier tier because it's the safe-feeling choice, then pay 10-30x more per call for reasoning the task never uses. A classification job that a cheap model nails for a fraction of a cent gets run on a frontier model at frontier prices, thousands of times a day. The bill is invisible in testing and brutal at volume. The fix is the procedure above: prove the cheap tier fails before you pay for the expensive one.

Trusting the leaderboard. A benchmark number is not your task's number, for two structural reasons. The first is contamination. Public test sets leak into the web-scraped data models train on, so a model can score high by having effectively seen the answers rather than by reasoning, and you usually can't tell which. The second is overfitting, also called Goodhart's law. Once a benchmark becomes the number labs get judged on, they optimize for it, and it stops measuring the underlying capability it once stood in for. When a measure becomes a target, it stops being a good measure. The most-cited public leaderboards are also the most gamed. The model that tops them can still lose on your actual codebase, your prompt style, your constraints.

Confusing the product with the model. Benchmarking through a chat app and concluding "this model can't do X" when the app's hidden system prompt is the thing getting in the way. Test through the API where you control the surface, or you're measuring the wrapper, not the weights.

Letting "bigger window is better" pick for you. A larger context window costs more and degrades on long inputs (attention spreads thin and facts in the middle get missed). Match the window to the task. A big window you don't need is a slower, pricier call that can also answer worse.

Treating the choice as permanent. You picked a tier six months ago and never re-checked. Names rotate, prices drop, a new workhorse model clears a bar that used to need the frontier. Your eval is the constant. Re-run it when models ship. Don't assume the new shiny one is better for you, and don't assume your old pick is still right.

Each failure mode has the same shape underneath. The generic advice optimizes for sounding safe (use the best model, track every release, bigger is better) and the operator move optimizes for the task in front of you. That same skeptical question, what specific pain does this remove and do I actually have it, applies just as well to the tooling that surrounds models, which the Toolkit takes up on its own.