Ollama Embeddings call fails with wrong URL path: /api/embeddings

Support
,
1

Description

I’ve been struggling with getting Ollama and Weaviate (local hosted) to communicate and MAY have found an issue. Can someone confirm this?? Also, I run the client.py from a different computer that resolves weaviate/ollama api’s just fine.

ERROR: (note the timeout for http://host.docker.internal:11434/api/embeddings)

Added document 86 of 87
Added document 87 of 87
Finished adding documents to batch.  Please wait...
{'message': 'Failed to send 34 objects in a batch of 39. Please inspect client.batch.failed_objects or collection.batch.failed_objects for the failed objects.'}
{'message': 'Failed to send 48 objects in a batch of 48. Please inspect client.batch.failed_objects or collection.batch.failed_objects for the failed objects.'}
Batch operation completed.
Failed to send some objects. Details:
[ErrorObject(message='send POST request: Post "http://host.docker.internal:11434/api/embeddings": context deadline exceeded (Client.Timeout exceeded while awaiting headers)', object_=_BatchObject(collection=

Here’s me proving that /api/embed works but /api/embeddings does not FROM the Weaviate container itself. (FYI, Ollama is running in a container on the same server and I am doing DNS trick)

curl http://host.docker.internal:11434/api/embed -d '{
>  "model": "all-minilm",
>  "input": "Why is the sky blue?"
> }'
{"model":"all-minilm","embeddings":[[0.010062016,-0.0017554946,0.050059456,0.046898536,0.05492503,0.008586371,0.105435975,-0.02585886,0.12960042,0.03196424,-0.04446901,-0.008987563,-0.00048934284,-0.063705765,-0.016032169,0.04664046,-0.022036841,-0.15813273,-0.07283887,-0.061328467,-0.0658716,0.05415806,-0.062132724,0.038911566,-0.045823917,0.05493227,-0.035256814,0.012593464,0.042497203,-0.00791991,-0.019023538,0.060972027,0.0369004,0.013471901,-0.025816392,-0.04346177,0.072602205,-0.04852212,0.004289973,-0.02943973,-0.029135313,-0.032912277,-0.018327639,0.015538654,-0.011697433,0.015307811,-0.0093740765,0.02596869,0.095239274,-0.015505323,-0.024563508,0.00905882,-0.07661391,0.01597137,0.0495439,0.11599131,0.0009903827,-0.020357985,0.092338845,0.008486339,-0.057076585,0.068846054,-0.07661979,0.06932093,0.092274524,-0.05546903,-0.05361791,0.008456234,-0.06313479,-0.06637476,-0.025184836,0.018881764,0.0614203,-0.028226346,0.03623095,0.0011074109,0.060684238,-0.06754768,-0.008165614,-0.012759827,0.030947892,-0.06379601,-0.07449585,0.11916402,0.012604237,0.06534184,0.014816104,0.051461816,-0.085212834,0.010246205,-0.007790781,-0.035579104,-0.115383774,-0.030693283,-0.08326488,0.013688024,0.0566172,-0.040913362,0.04265116,0.022126986,0.04688714,-0.051353518,0.030182738,0.007212158,-0.0041854717,-0.031170908,0.07790212,0.034200206,0.06134735,0.0074944603,-0.036250636,-0.08459046,0.021791892,-0.01941079,-0.039836466,0.054774895,-0.033720702,0.018103762,-0.105538726,-0.050379105,-0.011522215,0.03781235,0.022203606,0.08049023,0.007826529,-0.016847076,-0.05940992,-7.2264985e-33,0.13530928,-0.01122062,0.09228662,0.035975594,0.039652325,-0.05498584,-0.035040457,-0.0037004948,-0.019585393,-0.034948528,-0.0056920326,-0.014604142,-0.024266437,-0.04836174,0.04776316,-0.017078744,-0.06096635,0.0059679267,-0.08307662,0.084326036,-0.1046801,0.041656446,-0.036648408,-0.00807082,-0.028182266,-0.0432105,0.03599302,0.07497103,0.056440875,0.011846439,0.098480076,0.10484118,-0.021833764,0.046026118,-0.026338102,-0.050923478,-0.014703361,-0.006385692,-0.085865304,0.028598264,-0.053600207,0.05653976,-0.05975049,0.012445692,0.06619967,-0.013451253,0.038313355,-0.088933945,-0.057463154,0.031985212,-0.034472626,0.02362452,0.014457598,-0.041595947,0.06799603,0.031194357,0.06971566,-0.03505056,-0.0033061015,0.049345832,-0.013366442,-0.0034659673,0.0507598,0.07866189,0.03761729,-0.011602827,0.038128607,0.042007096,-0.012825245,-0.07895265,0.009025603,0.013359126,0.024172096,0.009767894,-0.010923641,-0.08161844,0.027013624,-0.029674161,-0.0043373145,0.013044075,-0.03524456,-0.019715898,0.055353805,-0.061263584,-0.055034723,0.012540969,-0.019183055,-0.012589377,-0.015817907,-0.0694245,-0.0448946,-0.048921186,0.048258252,-0.104466744,-0.10785244,3.5869857e-33,-0.0004916445,-0.08634525,-0.087862216,0.00721585,-0.007399472,-0.016603751,0.04528791,0.06751051,-0.04286227,0.08635874,0.045557436,0.06797927,0.00993781,-0.0030342652,0.05850576,-0.035561092,0.036206532,0.066166386,-0.03778131,-0.062247746,-0.04456004,0.077223405,0.04342943,-0.021262772,-0.021634147,0.062292643,-0.039129134,0.028107705,-0.013056282,0.05112274,-0.036825743,0.054581434,-0.0664262,0.02286992,0.004798484,0.090448886,0.005099579,-0.08310181,-0.055130087,0.07316554,-0.110448785,-0.020279517,0.1125694,-0.053279936,-0.057589278,-0.02394129,0.05659431,0.12725082,0.035939164,-0.043952655,0.017011488,-0.024847863,0.07269543,0.04313807,0.08050782,-0.019497115,-0.034367483,0.09668477,0.051890664,0.010758555,0.040229112,0.0022059581,-0.0075641256,0.0016752295,0.014182178,0.020362137,-0.0231149,0.021473816,-0.009235045,-0.050511464,-0.016234735,-0.08993932,-0.0606745,0.080994345,0.0024622004,0.0415745,0.04373566,-0.025141884,-0.09528701,0.08870627,-0.098396346,-0.0048426352,0.03536255,0.014155159,-0.06458343,-0.07597527,0.0124133555,-0.050230194,-0.055773806,-0.05700181,-0.018454289,-0.0021796268,-0.0022036426,0.035248876,-0.054714452,-1.4308528e-8,-0.007931117,0.026678042,0.0022828944,0.009996726,-0.021681784,-0.021548975,0.11138375,0.004643359,0.0378477,0.0039836173,-0.06688522,-0.028300066,-0.044328276,0.07121455,0.018713769,-0.049081404,-0.103970364,-0.043620154,0.010162136,0.041800696,-0.013576608,-0.033832546,-0.025055477,-0.01362027,0.0034353742,0.033081193,-0.0218651,0.021916177,0.07145605,0.020513555,0.02442516,0.035869926,-0.00096216256,-0.06136724,-0.08539853,0.0074072084,-0.038768604,0.07987996,-0.02557682,-0.06045561,0.060549967,0.08237212,-0.056718823,0.0048675025,0.045130767,0.023777386,0.043506835,0.091047205,-0.051409125,-0.011231628,-0.06888141,0.0072409785,0.07282593,-0.04335108,0.025920803,-0.114089325,-0.009541741,0.022197897,0.026735239,0.003706583,0.015949031,0.0035650954,-0.020685595,0.033551708]],"total_duration":480127694,"load_duration":316268926,"prompt_eval_count":/ 

curl http://host.docker.internal:11434/api/embeddings -d '{
>  "model": "all-minilm",
>  "input": "Why is the sky blue?"
> }'
{"embedding":[]}

CODE SNIPPETS:

import weaviate
from weaviate.connect import ConnectionParams
from weaviate.classes.init import AdditionalConfig, Timeout
from weaviate.classes.config import Configure
import weaviate.classes.config as weaviate_config

def connect_to_weaviate(
    weaviate_host="mothership",
    weaviate_port=8080,
    grpc_port=50051,
    http_secure=False,
    grpc_secure=False,
    init_timeout=300,
    query_timeout=300,
    insert_timeout=600
):
    """
    Connects to a Weaviate instance with custom configuration.

    Returns:
        weaviate.Client: A connected Weaviate client instance.
    """
    try:
        client = weaviate.WeaviateClient(
            connection_params=ConnectionParams.from_params(
                http_host=weaviate_host,
                http_port=weaviate_port,
                http_secure=False,
                grpc_host=weaviate_host,
                grpc_port=grpc_port,
                grpc_secure=False,
            ),
            additional_config=AdditionalConfig(
                timeout=Timeout(init=init_timeout, query=query_timeout, insert=insert_timeout),  # Values in seconds
            ),
            skip_init_checks=True
        )
        client.connect()
        # client = weaviate.connect_to_custom(
        #     http_host=weaviate_host,
        #     http_port=weaviate_port,
        #     http_secure=http_secure,
        #     grpc_host=weaviate_host,
        #     grpc_port=grpc_port,
        #     grpc_secure=grpc_secure,
        #     skip_init_checks=True,
        #     additional_config=AdditionalConfig(
        #         timeout=Timeout(init=init_timeout, query=query_timeout, insert=insert_timeout)
        #     )
        # )
    except Exception as e:
        print(f"Could not connect to Weaviate host: {e}")

    meta_info = client.get_meta()
    print("Meta Data: ", meta_info)

    return client

def create_collection(client, collection_name, properties):
    """
    Creates a new collection in Weaviate.

    Args:
        client (weaviate.Client): The Weaviate client instance.
        collection_name (str): The name of the collection to create.
        properties (list): A list of property configurations for the collection.

    Returns:
        The created collection object.
    """
    print(f"Creating collection '{collection_name}'...")

    collection = client.collections.create(
        name=collection_name,
        properties=properties,
        vectorizer_config=Configure.Vectorizer.text2vec_ollama(
            api_endpoint="http://host.docker.internal:11434",
            model="snowflake-arctic-embed"
        ),
        generative_config=Configure.Generative.ollama(
            api_endpoint="http://host.docker.internal:11434",  
            model="llama3.1"
        )
    )

    return collection

Part of a function that ingests the documents:

    try:
        print("Starting batch operation...")
        with collection.batch.dynamic() as batch:
            for i, doc in enumerate(documents):
                try:
                    # Check if document has required attributes
                    if not all(hasattr(doc, attr) for attr in ["title", "content", "headings", "file_path", "file_name"]):
                        print(f"Skipping document {i+1} due to missing attributes.")
                        continue
               
                    doc_object = {
                        "title": doc.title,
                        "content": doc.content,
                        "headings": doc.headings,
                        "file_path": doc.file_path,
                        "file_name": doc.file_name,
                    }

                    # # Generate embedding
                    # print(f"Generating embedding for document {i+1}...")
                    # embeddings_response = ollama.embeddings(model=EMBEDDING_MODEL, prompt=doc.content)
                    # print(f"Generated embedding for document {i+1}")
                    # if "embedding" not in embeddings_response:
                    #     raise ValueError(f"Failed to get embedding for document {i+1}")
                    # doc_object["embedding"] = embeddings_response["embedding"]

                    batch.add_object(
                        properties=doc_object,
                    )
                    print(f"Added document {i+1} of {len(documents)}")

Server Setup Information

  • Weaviate Server Version: 1.27.1
  • Deployment Method: docker
  • Multi Node? Number of Running Nodes: 1
  • Client Language and Version: Python 3.10.12
  • Multitenancy?: No

Any additional Information

weaviate-client==4.9.2
ollama==0.3.3

2

Link to Ollama docs showing /api/embed: ollama/docs/api.md at main · ollama/ollama · GitHub

Ah it has been superseded
" Note: this endpoint has been superseded by /api/embed"

3

hi @coolrazor !!

Welcome to our community :hugs:

Not sure I understood :thinking:

This is a working example using Weaviate (Docker) and Ollama (Natively):

import weaviate
from weaviate import classes as wvc
client = weaviate.connect_to_local()
print(f"Client: {weaviate.__version__}, Server: client.get_meta().get('version')")

client.collections.delete("Test")
collection = client.collections.create(
        name="Test",
        vectorizer_config=wvc.config.Configure.Vectorizer.text2vec_ollama(
            api_endpoint="http://host.docker.internal:11434",
            model="snowflake-arctic-embed"
        ),
        generative_config=wvc.config.Configure.Generative.ollama(
            api_endpoint="http://host.docker.internal:11434",  
            model="llama3.1"
        )
    )
collection.data.insert({"text": "Why is the sky blue?"})
print(len(collection.query.fetch_objects(include_vector=True).objects[0].vector.get("default")))
print(collection.generate.fetch_objects(single_prompt="answer: {text}").objects[0].generated)
client.close()

Let me know how I can help on this.

Thanks!

4

One interesting thing I noted is that indeed the /embeddings will not work, yet it seems that Ollama module in Weaviate is using it :thinking:

Yet, it is working :confused:

5

super weird but when I take the example you have above and edit it to use client = weaviate.connect_to_custom it still fails.

The script is being ran from my local PC. Weaviate and Ollama are running on a server called “mothership” and connectivity works otherwise.

EDITED SCRIPT:

import weaviate
from weaviate import classes as wvc
import os
from weaviate.classes.init import AdditionalConfig, Timeout
from weaviate.classes.config import Property, DataType
from dotenv import load_dotenv
load_dotenv()

WEAVIATE_HOST = os.getenv("WEAVIATE_HOST")
WEAVIATE_PORT = os.getenv("WEAVIATE_PORT")
EMBEDDING_MODEL = os.getenv("EMBEDDING_MODEL")
OLLAMA_HOST = os.getenv("OLLAMA_HOST")

client = weaviate.connect_to_custom(
    http_host=WEAVIATE_HOST,
    http_port=WEAVIATE_PORT,
    http_secure=False,
    grpc_host=WEAVIATE_HOST,
    grpc_port=50051,
    grpc_secure=False,
    skip_init_checks=True,
    additional_config=AdditionalConfig(
        timeout=Timeout(init=10, query=20, insert=120)  # Values in seconds
    )
)

print(f"Client: {weaviate.__version__}, Server: client.get_meta().get('version')")

client.collections.delete("Test")
collection = client.collections.create(
        name="Test",
        vectorizer_config=wvc.config.Configure.Vectorizer.text2vec_ollama(
            api_endpoint="http://mothership:11434",
            model="snowflake-arctic-embed"
        ),
        generative_config=wvc.config.Configure.Generative.ollama(
            api_endpoint="http://mothership:11434",  
            model="llama3.1"
        )
    )
collection.data.insert({"text": "Why is the sky blue?"})
print(len(collection.query.fetch_objects(include_vector=True).objects[0].vector.get("default")))
print(collection.generate.fetch_objects(single_prompt="answer: {text}").objects[0].generated)
client.close()

ERROR:

python test.py
Client: 4.9.3, Server: client.get_meta().get('version')
Traceback (most recent call last):
  File "/home/razor/repos/assistant/test.py", line 41, in <module>
    collection.data.insert({"text": "Why is the sky blue?"})
  File "/home/razor/repos/assistant/.venvrag/lib/python3.10/site-packages/weaviate/syncify.py", line 23, in sync_method
    return _EventLoopSingleton.get_instance().run_until_complete(
  File "/home/razor/repos/assistant/.venvrag/lib/python3.10/site-packages/weaviate/event_loop.py", line 42, in run_until_complete
    return fut.result()
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 458, in result
    return self.__get_result()
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/home/razor/repos/assistant/.venvrag/lib/python3.10/site-packages/weaviate/collections/data/data.py", line 340, in insert
    return await self._insert(weaviate_obj)
  File "/home/razor/repos/assistant/.venvrag/lib/python3.10/site-packages/weaviate/collections/data/data.py", line 84, in _insert
    await self._connection.post(
  File "/home/razor/repos/assistant/.venvrag/lib/python3.10/site-packages/weaviate/connect/v4.py", line 525, in post
    return await self.__send(
  File "/home/razor/repos/assistant/.venvrag/lib/python3.10/site-packages/weaviate/connect/v4.py", line 480, in __send
    raise e
  File "/home/razor/repos/assistant/.venvrag/lib/python3.10/site-packages/weaviate/connect/v4.py", line 471, in __send
    raise UnexpectedStatusCodeError(error_msg, response=res)
weaviate.exceptions.UnexpectedStatusCodeError: Object was not added! Unexpected status code: 500, with response body: {'error': [{'message': 'update vector: send POST request: Post "http://mothership:11434/api/embeddings": dial tcp 127.0.1.1:11434: connect: connection refused'}]}.
/home/razor/repos/assistant/.venvrag/lib/python3.10/site-packages/weaviate/warnings.py:329: ResourceWarning: Con004: The connection to Weaviate was not closed properly. This can lead to memory leaks.
            Please make sure to close the connection using `client.close()`.
/usr/lib/python3.10/asyncio/selector_events.py:710: ResourceWarning: unclosed transport <_SelectorSocketTransport fd=6 read=idle write=<idle, bufsize=0>>
6

Oh actually I forgot to not use the server hostname since it appears the call is made from within the Weaviate container, not the script (if I’m not mistaken).

I just edit these lines to use my DNS trick:
api_endpoint="http://host.docker.internal:11434",

but now the error I get is different. Although still not working. I tried setting the timeouts to 600 and it still does it.

ERROR:

python test.py
Client: 4.9.3, Server: 1.27.1
1024
Traceback (most recent call last):
  File "/home/razor/repos/assistant/.venvrag/lib/python3.10/site-packages/weaviate/collections/grpc/query.py", line 804, in __call
    res = await _Retry(4).with_exponential_backoff(
  File "/home/razor/repos/assistant/.venvrag/lib/python3.10/site-packages/weaviate/collections/grpc/retry.py", line 31, in with_exponential_backoff
    raise e
  File "/home/razor/repos/assistant/.venvrag/lib/python3.10/site-packages/weaviate/collections/grpc/retry.py", line 28, in with_exponential_backoff
    return await f(*args, **kwargs)
  File "/home/razor/repos/assistant/.venvrag/lib/python3.10/site-packages/grpc/aio/_call.py", line 327, in __await__
    raise _create_rpc_error(
grpc.aio._call.AioRpcError: <AioRpcError of RPC that terminated with:
        status = StatusCode.UNKNOWN
        details = "explorer: list class: extend: extend generate: client not found, empty provider"
        debug_error_string = "UNKNOWN:Error received from peer  {grpc_message:"explorer: list class: extend: extend generate: client not found, empty provider", grpc_status:2, created_time:"2024-11-06T06:39:04.574035292-08:00"}"
>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/razor/repos/assistant/test.py", line 43, in <module>
    print(collection.generate.fetch_objects(single_prompt="answer: {text}").objects[0].generated)
  File "/home/razor/repos/assistant/.venvrag/lib/python3.10/site-packages/weaviate/syncify.py", line 23, in sync_method
    return _EventLoopSingleton.get_instance().run_until_complete(
  File "/home/razor/repos/assistant/.venvrag/lib/python3.10/site-packages/weaviate/event_loop.py", line 42, in run_until_complete
    return fut.result()
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 458, in result
    return self.__get_result()
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/home/razor/repos/assistant/.venvrag/lib/python3.10/site-packages/weaviate/collections/queries/fetch_objects/generate.py", line 75, in fetch_objects
    res = await self._query.get(
  File "/home/razor/repos/assistant/.venvrag/lib/python3.10/site-packages/weaviate/collections/grpc/query.py", line 814, in __call
    raise WeaviateQueryError(str(e), "GRPC search")  # pyright: ignore
weaviate.exceptions.WeaviateQueryError: Query call with protocol GRPC search failed with message <AioRpcError of RPC that terminated with:
        status = StatusCode.UNKNOWN
        details = "explorer: list class: extend: extend generate: client not found, empty provider"
        debug_error_string = "UNKNOWN:Error received from peer  {grpc_message:"explorer: list class: extend: extend generate: client not found, empty provider", grpc_status:2, created_time:"2024-11-06T06:39:04.574035292-08:00"}"
>.
/home/razor/repos/assistant/.venvrag/lib/python3.10/site-packages/weaviate/warnings.py:329: ResourceWarning: Con004: The connection to Weaviate was not closed properly. This can lead to memory leaks.
            Please make sure to close the connection using `client.close()`.
/usr/lib/python3.10/asyncio/selector_events.py:710: ResourceWarning: unclosed transport <_SelectorSocketTransport fd=6 read=idle write=<idle, bufsize=0>>