Use ollama embeddings hosted on a server using weaviate

Support
1

Description

I have setup ollama on a GCP server and exposed it as url. It is accesible through ollama client with AUTH token. Here is sample code:

from ollama import Client
try:
      MODEL_NAME = "nomic-embed-text:latest"
      client = Client(
        host='https://ollama-inferece-url',
        headers={'Authorization': AUTH_TOKEN}
      )
      text = "Hello, this is a test sentence."
      response = client.embeddings(
        model=MODEL_NAME,
        prompt=text
      )

      # Extract embeddings from response
      embedding = response['embedding']
      print(embedding)
      print(f"Single embedding shape: {len(embedding)}")

except Exception as e:
        print(f"Error generating: {e}")

I want to integrate this ollama embeddings with weaviate. Can you provide some sample code or reference for this?

Server Setup Information

  • Weaviate Server Version: 1.28.0
  • Deployment Method:docker
  • Multi Node? Number of Running Nodes: 1
  • Client Language and Version: Python 3.8
  • Multitenancy?: No
2

hi @P_K !!

Welcome back :slight_smile:

Check out this recipe:

It will use ollama to create a RAG locally.

Let me know if that helps!

3

Thanks DudaNogueira.
I have tried this but this works with local ollama embeddings. With the Ollama embedding model hosted on a server, I need to pass in the URL as well as Auth Token for generating embeddings. I unable to find a way to pass in Auth Token to “text2vec-ollama” vectorizer.

Here is sample curl to access the embedding model from local:

curl https://ollama-inference-url/api/embeddings -H "Authorization: AUTH-TOKEN" -H "Content-Type: application/json" -d '{"model": "nomic-embed-text:latest", "prompt": "Why is sky blue?"}'
4

Oh, I see. I believe the ollama module was built with the assumption that Ollama will not require an api token :thinking:

I have quickly checked that module code and have not found a way to provide an API. I believe this is an interesting feature request.

I will check on this over the week and make sure we open a feature request if necessary.

Thanks!