NVIDIA’s NeMo Switchyard made me stop thinking about “the best AI model” as a permanent choice.
Its router can decide which model should handle a particular request. That is much closer to the way real work behaves: some tasks need speed, some need depth, some need specific tools, and some do not justify the most expensive model available.
So I’m building my own small model router around the work I actually do.
The Question Is Bigger Than Model Quality
A model name is only one part of a useful route.
The work may depend on a repository, a browser session, a spreadsheet, a connected knowledge base, or a long conversation whose context would be expensive to rebuild. It may be routine but urgent, difficult but low-risk, or simple to generate and costly to get wrong.
I want the router to consider the task, context, tools, stakes, cost, and the price of switching. I also want it to be allowed to say don’t switch when continuity is worth more than a marginal gain in capability.
That is the idea that interested me in NVIDIA’s model router. Routing turns model selection from a permanent architecture decision into a policy that can be observed, tested, and changed.
I wanted to see what that would look like at a personal scale.
Astra Adds a New Route, Not a New Default
GPT-6 Astra entering the API made the project more useful.
Astra is designed for difficult end-to-end work across code, browsers, and professional applications. Its strengths in long-horizon execution, very large context, computer use, and tool coordination make it a natural candidate for a frontier route.
So I added one.
The working router now has separate roles for lightweight work, ordinary substantial work, deep reasoning, and the hardest long-running workflows. Astra occupies that last category.
It did not replace the other routes.
Sending every difficult task to the newest model would defeat the purpose of having a router. A smaller model may be faster, cheaper, or entirely sufficient. A strong general-purpose model may still produce better final prose. A specialized product surface may matter more than the base model underneath it.
The useful question is not whether Astra is the winner. It is when Astra earns the route.
That is also why my earlier Astra analysis focused on the shape of its capabilities rather than treating one leaderboard score as a universal answer.
The Router Is Still Being Built
This is a working implementation, not a finished product or a victory lap.
I have a local router with explicit model tiers, an experimental classifier, frozen test prompts, and a small measurement harness. I am still testing where deterministic rules misunderstand intent, when escalation is justified, how consistently a classifier behaves, and what each route costs to reach a usable result.
The phrase usable result matters.
A cheaper model that needs repeated correction may cost more in practice than one expensive pass. A more capable model may still be the wrong choice if moving the work loses important context or requires rebuilding access to the right tools. Token price, elapsed time, retries, and human repair all belong in the comparison.
I expect the routing policy to change as the evidence improves. That is a feature of the project.
The Router Becomes Documentation
The part I find most interesting is that the router itself becomes a small piece of documentation and governance.
It should be able to explain:
- what it chose
- why it chose it
- which model, reasoning level, tools, and product surface make up the route
- what evidence supports the decision
- when escalation or fallback is allowed
- which version of the policy was active
- when the policy changed
Without that record, dynamic model selection becomes another invisible system behavior. With it, the router can be inspected, challenged, and improved.
I’m not trying to pick one winning model.
I’m trying to build a better way to decide which model the work deserves.