Use DeepInfra as provider for Embedding Model

Support
1

How to use a OpenAI API Compatible embeddings endpoint?

Currently there seems to be no way to add a custom OpenAI API Compatible endpoint. I want to use DeepInfra as they host Qwen3-embedding. How to do this?

2

Okay, this is terrible implementation detail. The base url is DeepInfra https://api.deepinfra.com/v1/openai/embeddings. With https://webhook.site/ i found out that weaviate just appends /v1/embeddings, like, for real this is an PR oversight I guess, cuz it’s unusable now. Either use the full url, or just add /embeddings, but adding the v1 is plain weird

3

hi @alber70g !

the idea of text2vec-openai is to be closely tied with openai and openai only. Of course, we can always push it further :winking_face_with_tongue:

I am afraid there isn’t a module that will allow you to take full control of the api endpoint :thinking:

text2vec-ollama for example will append /api/embed while text2vec-azure will append resource_id, etc.

I see some options from here:

1 - Open a GH Feature request to either allow pass “X-OpenAI-Basepath” or create DeepInfra’s own module

2 - Put something in between, like LiteLLM, and handle this path issue.

3 - hardcode it on Weaviate code :grimacing:

Let me know if this helps!

THanks!

4

I did something similar to 2

I have a caddy proxy setup that’ll proxy from embeddings.mydomain.com/v1/embeddings to https://api.deepinfra.com/v1/openai/embeddings

This seems to work.

Note: it would be nice to have this behaviour of appending /v1/embeddings to the baseURL written down somewhere in the docs of configuring the tex2vec-openai so it’s at least visible. Now it was a black box that I only found out when using the webhook.site test

5

Tip: Whenever you want to learn something directly from the codebase, you can ask here :wink:

6

Thank you! This is a great resource

1 Like