Performance
Local AI shouldn't feel slow. When it does, the cause is usually one of three things: GPU not engaged, model too big, or the wrong model loaded.
1. GPU engaged?
On Apple Silicon, check that Ollama is using Metal: ollama serve logs using Metal. If you see falling back to CPU, you have an Ollama version that doesn't support your machine, or a driver issue. Update Ollama.
2. Right model size?
A 70B model on 16 GB RAM will swap to disk and crawl. Stick to 7B for routine work, reach for bigger only when you really need it. See benchmarks/results/capability_map.json to find smaller models that score well enough on your task.
3. Wrong model loaded?
First request after a routing change pays the model-load cost (5-10 seconds typically). Subsequent requests are fast. If you're swapping models on every request, increase keep_alive_seconds in the hardware config.