Memory management
Symptoms of memory pressure, what causes them, and how Soriku handles model swapping when there isn't enough RAM to keep everything loaded.
Symptoms
- Sluggish responses on a previously fast machine
- macOS Activity Monitor shows memory pressure in the red
- Ollama logs
out of memoryor silently terminates models - Benchmark runs abort halfway through with
RAM critical
How routing handles it
Soriku tracks which models are loaded and how much RAM each one occupies. When a new request would push the loaded set over the budget, the router unloads the least-recently-used model first, then loads the new one. You see this as a latency spike on the first request after a model swap.
Tune the budget
# config/models.yaml
hardware:
ram_budget_gb: 12 # how much RAM Soriku may use for models
keep_alive_seconds: 300 # how long to keep a model loaded after last use
Defaults on 16 GB Apple Silicon: 12 GB budget, 5 minute keep-alive. Lower the budget if other apps complain, raise the keep-alive if you're swapping back and forth between the same two models.
When things really go wrong
If Soriku crashes mid-benchmark and the capability map shows partial data, run
python -m benchmarks --reset-stale to clear inconsistent entries before re-running the benchmark.