Smart routing

Every prompt gets classified into a task category in under five milliseconds, then routed to the model that scores best for that category under your constraints. Deterministic, transparent and override-able.

The routing pipeline

Classify the prompt into a TaskType using lexical signals and a tiny prompt classifier. Result: one of coding, reasoning, general, embedding, pilot.
Pull constraints from the request: model='auto' lets the router pick; an explicit model pins it. Constraints can include local-only, a cost ceiling, a latency budget and a preferred provider.
Look up the capability map for the target category. Filter to models that meet the constraints.
Pick the highest scorer with valid hardware availability. RAM-aware loading prevents Soriku from picking a model that won't fit alongside what's already loaded.
Execute the prompt on the chosen model and record the decision for auditability in data/routing_log.jsonl.

Determinism

Given the same capability map, the same constraints and the same prompt, the router always picks the same model. No randomness in the decision step. If you can't reproduce a routing decision, you can't audit it.

Constraints you can set

Constraint	Type	Effect
`prefer_local`	bool	Only consider locally hosted models
`max_cost_per_1k`	float	Reject models whose cost exceeds this cap
`max_latency_ms`	int	Reject models whose recent p95 latency exceeds this
`provider_allowlist`	list	Restrict to specific providers (`ollama`, `anthropic`, `openai`, etc.)
`pinned_model`	string	Bypass routing entirely and use this exact model

Task types

The classifier maps a prompt to one of five task types. The mapping is rule-based and fast.

Task type	Trigger	Routes to
`coding`	Code-generation, debugging, refactoring intents	Models with high `code_generation` or `code_review` scores
`reasoning`	Multi-step analysis, math, planning, formal reasoning	Models with high `reasoning` scores
`general`	Conversational, summary, translation, creative writing	Models with high category-matching general scores
`embedding`	Requests to `/api/v1/embeddings`	Embedding-capable model, defaults to `nomic-embed-text`
`pilot`	Tool-using prompts (file ops, shell, web fetch) detected via verb regex	Pilot-mode worker, see Pilot mode

Overriding the router

Three levels of override, from most general to most specific.

Global pin: set a model in config/models.yaml to always handle a category.
Per-call pin: pass model='qwen2.5-coder:7b' in the API call. The router skips classification.
Persona preference: an agent persona with preferred_model set always routes through that model.

Even with an override, the routing decision is logged with routed_by: pin so audits show the override was deliberate.

Next: agent personas →