A routing policy beats a model popularity contest
GitHub's HydraFusion project puts attention on orchestration rather than a single winner. The practical question is which task deserves a more expensive or slower model, and what happens when confidence is weak. Routing should follow an observable policy, not a vague preference for the newest benchmark leader.
Define task classes
Separate low-risk transformations, retrieval-heavy questions, code changes, and security-sensitive work. For each class, set a quality threshold, latency budget, cost ceiling, and fallback. Evaluate the whole system result, including review time and rollback cost, rather than model output in isolation.
Keep fallback visible
A good routing system logs selected model, inputs allowed by policy, completion outcome, and escalation. This lets teams discover when a supposed quality gain simply moved work to human reviewers. It also prevents a temporary provider outage from becoming a silent product regression.
FAQ
Should every team build a router? No. Start only when task variation is large enough to justify the maintenance burden. What is the first metric? Accepted output per dollar, with review time included.