Local hosted Weaviate and ollama - hybrid search fails

Support
1

Hybrid search is failing with an exception connection refused

Server Setup Information

  • Weaviate Server Version: 1.28.2
  • Deployment Method: Local on Docker
  • Multi Node? Number of Running Nodes: 1
  • Client Language and Version: 4.17.0
  • Multitenancy?: No

I am following the instructions on local setup and trying out RAG using ollama. I can insert the records into weaviate see the vecotrs when I fetch them but when I try hybrid search it fails with
Query call with protocol GRPC search failed with message remote client vectorize: send POST "http://localhost:11434/api/embed": dial tcp [::1]:11434: connect: connection refused

services:
  weaviate:
    command:
    - --host
    - 0.0.0.0
    - --port
    - '8080'
    - --scheme
    - http
    image: cr.weaviate.io/semitechnologies/weaviate:1.28.2
    ports:
    - 8080:8080
    - 50051:50051
    volumes:
    - weaviate_data:/var/lib/weaviate
    restart: always
    environment:
      OLLAMA_URL: http://ollama:11434
      OLLAMA_MODEL: llama3.2:latest
      OLLAMA_EMBED_MODEL: mxbai-embed-large
      QUERY_DEFAULTS_LIMIT: 25
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
      PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
      DEFAULT_VECTORIZER_MODULE: 'text2vec-ollama'
      ENABLE_MODULES: 'text2vec-ollama,generative-ollama, mxbai-embed-large'
      CLUSTER_HOSTNAME: 'node1'
    networks:
      - ollama-weviate
  ollama:
    image: ollama/ollama
    ports:
      - 11434:11434
    volumes:
      - ollama_data:/root/.ollama
      #- ./entrypoint.sh:/entrypoint.sh
    container_name: ollama
    pull_policy: always
    tty: true
    restart: always
    networks:
      - ollama-weviate    
    #entrypoint: ["/bin/bash", "/entrypoint.sh"]      
volumes:
  weaviate_data:
  ollama_data:
networks:
  ollama-weviate:
    name: ollama-weviate
    external: false


from weaviate.classes.config import Configure

client.collections.create(
    "DemoCollection",
    vector_config=[
        Configure.Vectors.text2vec_ollama(
            name="title_vector",
            source_properties=["title"],
            api_endpoint="http://ollama:11434",  # If using Docker, use this to contact your local Ollama instance
            model="mxbai-embed-large",  # The model to use, e.g. "nomic-embed-text"
        )
    ],
    # Additional parameters not shown
)

collection = client.collections.use("DemoCollection")

response = collection.query.near_text(
    query="A holiday film",  # The model provider integration will automatically vectorize the query
    limit=2
)

for obj in response.objects:
    print(obj.properties["title"])


collection = client.collections.use("DemoCollection")

response = collection.query.hybrid(
    query="A holiday film",  # The model provider integration will automatically vectorize the query
    limit=2
)

for obj in response.objects:
    print(obj.properties["title"])

I tried changing the models I still get the same error. It looks like hybrid search is not able to create embeddings for the seach query but how is insert able to create the embeddings and search not able to?

If I provide the vector retrieved from near_text call or generate embeddings from langchain_community OllamaEmbeddings` it works

This works

collection = client.collections.use("DemoCollection")

query_string = "A holiday film"

from langchain_community.embeddings import OllamaEmbeddings

ollama_embeddings = OllamaEmbeddings(
   base_url = "http://host.docker.internal:11434",
   model="mxbai-embed-large"
)

query_vector = ollama_embeddings.embed_query(query_string)

response = collection.query.hybrid(
    query="A holiday film",  # The model provider integration will automatically vectorize the query
    limit=2,
    vector = query_vector
)

for obj in response.objects:
    print(obj.properties["title"])

My goal is to have ollama and weaviate running through Docker and have all the types of search working on it locally where I do not have to generate these embeddings. I need to try few scenarios before I think about moving to cloud

2

hi @RishabhKothaari !! Welcome to our community :hugs: !!

Any reason for using such an old version? :grimacing:

This has worked for me on latest 1.33.4 version:

from weaviate.classes.config import Configure
from weaviate.classes.query import MetadataQuery

client.collections.delete("DemoCollection")

collection = client.collections.create(
    "DemoCollection",
    vector_config=[
        Configure.Vectors.text2vec_ollama(
            name="title_vector",
            source_properties=["title"],
            api_endpoint="http://host.docker.internal:11434",  # If using Docker, use this to contact your local Ollama instance
            model="mxbai-embed-large",  # The model to use, e.g. "nomic-embed-text"
        )
    ],
    # Additional parameters not shown
)

collection.data.insert({"title": "Vacationing on some place"})
collection.data.insert({"title": "Cute little dog"})

now query:

response = collection.query.hybrid(
    query="A holiday film",  # The model provider integration will automatically vectorize the query
    limit=2,
    return_metadata=MetadataQuery(score=True)
)

for obj in response.objects:
    print(obj.properties["title"], obj.metadata.score)

This was the output I got:

Vacationing on some place 0.699999988079071
Cute little dog 0.0

Ps: I am running ollama locally on my laptop, hence the api_endpoint="http://host.docker.internal:11434"

Let me know if this helps!

Thanks!

1 Like