Need help to use my own vectorizer and generative model

Support
,
1

I would like to know how to use a locally ran/downloaded vectorizer model to create embeddings and vectors,

i could not find the appropriate doc for that.

2

These are the doc you’re looking for (pick based on your model of choice):

It basically runs two containers, one with Weaviate, and another one with the models

1 Like
3

Also, the code for the transformers inference model container is here:

So if you have a private model, you can use that same endpoint to call your own model, keeping it compatible with the text2vec-transformers module/integration.

Let us know if this helps :slight_smile:

1 Like
4

thanks alot, i did study these docs earlier but i wanted to actually write codes and not use docker container for now.

I came up with this solution for now (tho it does not feels weaviatty way)

here is my model’s code

model = SentenceTransformer('all-MiniLM-L6-v2', device='cuda')

research_docs = client.collections.create(
                name='ResearchDocs',
                vectorizer_config=wvc.config.Configure.Vectorizer.none(),
            )

            results = search_web(query="Deep fake detection", search_engine="both", max_results=2)
            print("Search results:", len(results['results']))

            docs = collapse_results(results)
            print("Docs:", len(docs))
            emb = (model.encode(docs)).tolist()

            wvc_dataObjects = list()
            for i, (d, e) in enumerate(zip(docs, emb)):
                wvc_dataObjects.append(
                    wvc.data.DataObject(
                        properties={
                            "title": results['results'][i]['title'],
                            "content": results['results'][i]['content'],
                        },
                        vector=e
                    ),
                )

            research_docs.data.insert_many(wvc_dataObjects)

I will try out the docker way tho

thanks and sorry for the late reply

5

Is there any way to make both of them run on a single container?

6

Hi!

If you don’t want to use a vectorizer on docker, you will have to do what we call “bring your own vectors”. And that’s what you are doing :slight_smile:

More on that here: Bring your own vectors | Weaviate - Vector Database

The downside of this approach is that you are in charge of vectorizing and re-vectorizing your content when one of its “vectorizable” properties (skip:false) changes.

Also, you will not be able to use nearText, as Weaviate doesn’t know how to vectorize the query.

So you will need to vectorize the query yourself, and use, instead of nearText, nearVector with your vector query. This also applies to hybrid (replacing query parameter by vector) and so on.

Let me know if this helps :slight_smile: