Near_text() with my own vectorizor

Support
1

Description

I am currently setting up a small instance of weaviate to make a proof of concept for myself. I am running my own embedding model for the inserted objects. Running near_vector() works flawlessly, near_text() on the other hand spits out

Query call with protocol GRPC search failed with message panic occurred: ValidateParam was called without any known params present.

I have done some scouring and found that it is possibly related to the fact that I do not have a vectorizor module running on my instance. I know I could recreate my collection with the vectorizer configured but is there a way to specify usage of my own vectorizor?

Server Setup Information

Weaviate docker image

2

Hi @jksnxd,

Unfortunately, there isn’t a simple way to just point Weaviate to your own model.

You have a couple of options:

HuggingFace path

You can deploy your model to HuggingFace, and then you can use the HuggingFace vectorizer to access the model.

Please note, that this comes with a usage fee from HuggingFace.

Customer Vectorizer Model

You can also implement a vectorizer module to work with your own model, and then create a new collection with that vectorizer.

Please note that Weaviate modules are written in GoLang.
For people familiar with GoLang, this usually takes half a day to a day.

Here is a community guide on “How to build a custom module”.

Manual Vectorization

Continue vectorizing the data yourself, and instead of using near_text, you could vectorize your queries, and use near_vector for search.

Note, you can hybrid search (to run a keyword and vector queries together), but you need to provide the vector embedding for each query.
Also, you can use filters, with near_vector and hybrid.

3

If your local vectorizer has an openAI compatible API, you can use the openai-module and change the baseurl (the part before the /v1)

4

Thanks for the direction @sebawita and @Dirk

I have it running perfectly by hosting it via ollama and using the ollama module that was add in 1.25!

5

Yea, the Ollama integration is :fire:
I am glad that helped.

Out of curiosity:
Did you add your own model to Ollama?
Or does Ollama provide the model you need?

If the first one is the case, that would make for a cool blog post :thinking:

6

Yes, the Ollama integration is pretty great.

I was going to do a custom module for Weaviate to have llama.cpp integration but decided I wanted to try to make my own vectorizer work with ollama…and it did! I downloaded a version of my model that was nice and packaged up in the .gguf. I then created a Modelfile the model from said file and the downloaded .gguf.

It now works exactly the same as if I were to use one pulled from Ollama’s repo/library!

I am going to whip up a quick blog post about what I did for the fun of it!

1 Like