Connect Vane to Ollama: chat, embeddings and Docker networking

Know what stays local

Ollama can serve chat and embedding models on your machine. Vane can call that server without sending prompts to a hosted model provider, but live web searches and page retrieval still contact the internet. Local inference is not fully offline web research.

Reference: Vane 1.12.2 and its native Ollama provider, checked 2026-10-22. This recipe assumes Ollama is installed on the host and Vane uses the standard Docker setup.

1. Prepare both models

Use a modest chat model that fits your machine, and a dedicated embedding model:

ollama pull qwen2.5:7b
ollama pull nomic-embed-text
ollama list
curl -f http://localhost:11434/api/tags

qwen2.5:7b is a starting point, not a compatibility or speed guarantee. Select a smaller model if the hardware checker indicates memory pressure. Local embedding downloads add disk and RAM use.

2. Use an address Docker can reach

For Vane in Docker and Ollama on your host, use http://host.docker.internal:11434. The Compose examples include the host-gateway mapping for Linux. Docker Desktop supplies the host alias on supported desktop platforms.

On Linux, host Ollama may listen only on loopback. Configure the Ollama service’s OLLAMA_HOST to an interface reachable from Docker, restart that service, and restrict incoming access to the needed host/container network. 0.0.0.0:11434 listens on every interface; do not expose it to an untrusted network. This is an Ollama server setting, not a Vane setting.

If both services run in the same Compose network, use the Ollama service name, for example http://ollama:11434. Inside Vane, localhost refers to Vane’s own container.

3. Add the native Ollama provider

In Vane setup, add Ollama and enter the reachable base URL without /v1. Choose the chat model and an embedding model in their respective selectors. Alternatively initialize the provider with:

environment:
  OLLAMA_BASE_URL: http://host.docker.internal:11434

Recreate the container after Compose changes. Existing saved providers can remain; inspect settings rather than assuming an environment edit overwrites them. See configuration precedence.

4. Check from inside Vane

docker compose exec vane node -e "fetch('http://host.docker.internal:11434/api/tags').then(async r=>{console.log(r.status);console.log(await r.text())}).catch(console.error)"

A 200 response and the expected model names establish basic reachability. Then test a short prompt and a small file/query workflow in Vane to exercise chat and embeddings separately.

Model 404, connection refused, embedding failures

Model 404: compare the configured ID to ollama list. Tags matter: use the exact downloaded identifier. A reachable server is not proof that a model exists. A wrong API path can also cause a 404.

Connection refused or timeout: inspect the host service listener, firewall and base URL. Check host requests and in-container requests separately. A working host curl localhost does not prove Docker can reach the host.

Embedding error: confirm the selected model implements embeddings. A chat-only model is not interchangeable with an embedding model. If you change embedding models after indexing documents, plan to rebuild that index using the project’s supported controls because vector dimensions may differ.

Out of memory or very slow responses: try a smaller quantized model, reduce context, close other workloads and check logs. CPU inference may fit memory but still be impractical for interactive research.

Final check

Restart Vane, confirm settings persist, then run a question with a verifiable citation. Keep the Ollama endpoint private. If a request unexpectedly uses a cloud provider, inspect the selected chat and embedding providers before claiming local processing.