Per-stage model routing

Every conversation stage — ask, compact, rollup — resolves its own chat backend. A stage with no override inherits the top-level llm: block; give it a BackendConfig to pin it somewhere else. That is the whole model: one default, six independently pinnable call sites.

llm:
  provider: anthropic
  model: claude-opus-5
  api_key_ref: env:ANTHROPIC_API_KEY

conversations:
  compact:
    # extractive → cheap cloud model; abstractive left unset, so it inherits llm:
    extractive_backend:
      provider: anthropic
      model: claude-haiku-4-5
      api_key_env: ANTHROPIC_API_KEY
  ask:
    # answer stage → a local OpenAI-compatible runtime
    backend:
      provider: openai
      model: Qwen3.5-4B-MLX-4bit
      endpoint: http://localhost:8000/v1
      api_key_ref: env:OMLX_API_KEY

The six call sites

Stage	Field	Inherits when unset
`ask` answer	`ask.backend`	`llm:`
`ask` query rewriter	`ask.rewriter_backend`	`ask.backend`, then `llm:`
`compact` extractive	`compact.extractive_backend`	`llm:`
`compact` abstractive	`compact.abstractive_backend`	`llm:`
`rollup` extractive	`rollup.extractive_backend`	`llm:`
`rollup` abstractive	`rollup.abstractive_backend`	`llm:`

The rewriter falls through to the answer stage before llm:, so pinning ask.backend moves the rewriter with it. It keeps its own shorter rewriter_timeout_secs either way — rewriter output is small and falling back to the raw question on timeout is harmless, so it should not burn the full answer-stage budget.

BackendConfig schema

Field	Required	Notes
`provider`	yes	`ollama`, `anthropic`, `openai`, `openrouter`, `gemini`
`model`	yes	provider model id
`endpoint`	no	overrides the provider default
`api_key_ref`	no	secret ref, e.g. `env:OMLX_API_KEY` or `keychain:mur/anthropic` — checked before `api_key_env`
`api_key_env`	no	name of the env var holding the key — the key itself never lives in config
`timeout_secs`	no	default 120

Local runtimes

Any OpenAI-compatible server — omlx, LM Studio, vLLM, llama.cpp — uses provider: openai with endpoint pointed at it. Only Ollama's native API needs provider: ollama.

If the top-level llm: block itself points at a local runtime, set provider: openai there too and every unpinned stage follows it automatically.

Verifying before you have data

compact and ask short-circuit before touching a model when there is nothing to summarize, so a fresh install cannot exercise its own configuration. Use the doctor instead:

mur chat doctor

conversations backends
  ask.generate         openai  Qwen3.5-4B-MLX-4bit  http://localhost:8000/v1  [pinned]
  ask.rewriter         openai  Qwen3.5-4B-MLX-4bit  http://localhost:8000/v1  [pinned]
  compact.extractive   openai  Qwen3.5-4B-MLX-4bit  http://localhost:8000/v1  [pinned]
  compact.abstractive  ollama  qwen3:4b             http://localhost:11434    [follows smart]
  rollup.extractive    …
  rollup.abstractive   …
  · no conversations stage routes through Ollama (skipping reachability probe)
  ✓ openai model Qwen3.5-4B-MLX-4bit listed at http://localhost:8000/v1

Every stage is listed with the provider, model and endpoint it will actually dial. Each distinct endpoint is probed once, against only the models routed to it, and an endpoint no stage uses is not probed at all. Cloud providers are checked for a resolvable key; nothing prints a key value.

Upgrading from an older config

Before per-stage backends, each stage stored a bare model name plus an ollama_endpoint, and resolution assumed Ollama. MUR converts those fields the first time it loads your config and writes the result back once:

a stage still on its shipped defaults becomes an inherit (no override)
a stage you had customized — a different model, or a non-default endpoint such as a remote Ollama box — is pinned to an explicit provider: ollama backend that reproduces exactly what it did before

Nothing is silently re-routed, and the conversion is idempotent. A config that fails to parse is left untouched rather than partially rewritten.

Cost guidance

With a Haiku-extractive / Sonnet-ask split, typical daily use lands around a few dollars per month. Verify a cloud backend end to end with the ignored live test:

ANTHROPIC_API_KEY=... cargo test -p mur-core live_anthropic_haiku_responds -- --ignored