Hi everyone, I’m running into some challenges while trying to intergrate Weaviate with an Ollama Docker (with CUDA support) Instance. They are both ran in an docker container on the same server. However, I keep encoutering errors when Weaviate tries to connect to Ollama. I’m using the v4 Python client, and the main issue seems to be related to how Weaviate is trying to communicate with Ollama.
( SInce im a new user I can only use to links so I will be replacing them as following: http://192.168.x.x with URL)
Setup Details
1.Weaviate Docker Setup
- Image:
cr.weaviate.io/semitechnologies/weaviate:1.28.2 - Environment Variables:
environment:
ENABLE_MODULES: text2vec-ollama,generative-ollama
OLLAMA_URL:URL:11434
WEAVIATE_HOST: 0.0.0.0
WEAVIATE_PORT: 8080
WEAVIATE_GRPC: 50051 - Ports:
ports:
– 8080:8080
– 50051:50051
Ollama Docker Setup
- Image: ollama/ollama
- CUDA Support: Configured with nvidia-container-toolkit
- Ports: 11434:11434
Schema Configuration
(since Weaviate is running in a docker container, I have tried host.docker.internal:11434 and localhost:11434 which don’t give a other result then with the current setup)
{
“class”: “Article”,
“vectorizer”: “text2vec-ollama”,
“vectorIndexConfig”: {
“distance”: “cosine”
},
“modelConfig”: {
“text2vec-ollama”: {
“apiEndpoint”: “URL:11434”,
“model”: “nomic-embed-text”
},
“generative-ollama”: {
“apiEndpoint”: “URL:11434”,
“model”: “llama3.2”
}
},
“properties”: [
{“name”: “title”, “dataType”: [“text”]},
{“name”: “content”, “dataType”: [“text”]},
{“name”: “authors”, “dataType”: [“text”]},
{“name”: “publicationDate”, “dataType”: [“date”]},
{“name”: “category”, “dataType”: [“text”]},
{“name”: “tags”, “dataType”: [“text”]},
{“name”: “url”, “dataType”: [“text”]},
{“name”: “scrapeDate”, “dataType”: [“date”]},
{“name”: “createdDate”, “dataType”: [“date”]},
{“name”: “articleId”, “dataType”: [“text”], “index”: true, “primaryKey”: true}
]
}Python Client Initialization
import weaviate
import os
from dotenv import load_dotenv
load_dotenv()
client = weaviate.connect_to_local(
host=os.getenv(“WEAVIATE_HOST”, “127.0.0.1”),
port=int(os.getenv(“WEAVIATE_PORT”, 8080)),
grpc_port=int(os.getenv(“WEAVIATE_GRPC”, 50051))
)
Issue
When running my script to insert data into Weaviate (ran on an other computer then the docker containers), I encouter the following error:
weaviate.exceptions.UnexpectedStatusCodeError: Object was not added! Unexpected status code: 500, with response body: {'error': [{'message': 'update vector: send POST request: Post "http://localhost:11434/api/embed": dial tcp [::1]:11434: connect: connection refused'}]}.
Questions
- Why is Weaviate defaulting to localhost:11434 even thought the schema specifies URL:11434, host.docker.internal:11434 for apiEndpoint?
- Is there any additional configuration needed to make Weaviate v4 properly use the OLLAMA_URL?
- Am I missing any critical step in the setup to make these two services communicate?
- How does Weaviate internally handle the communication with external modules like Ollama? Is there a way to debug these request in more detail?
Appreciation
Thanks in advance for any help or pointers! I’m happy to provide additional logs or configuration details if needed. Let me know how I can further debug this.
Kind regards!
