Using local transformers instead of APIs based

Support
1

Hi everyone!

A sample of our docker-compose file is like below:

services:
  t2v-transformers:
    image: cr.weaviate.io/semitechnologies/transformers-inference:sentence-transformers-multi-qa-MiniLM-L6-cos-v1
    environment:
      ENABLE_CUDA: '0'
  ner-transformers:
    image: cr.weaviate.io/semitechnologies/ner-transformers:dslim-bert-base-NER
    environment:
      ENABLE_CUDA: '0'
  sum-transformers:
    image: cr.weaviate.io/semitechnologies/sum-transformers:facebook-bart-large-cnn-1.0.0
    environment:
      ENABLE_CUDA: '0'
  qna-transformers:
    image: cr.weaviate.io/semitechnologies/qna-transformers:deepset-bert-large-uncased-whole-word-masking-squad2
    environment:
        ENABLE_CUDA: '0'


ENABLE_MODULES: 'text2vec-transformers,qna-transformers,ner-transformers,sum-transformers,text-spellcheck,reranker-transformers'

WCS example from QuickStart doc is like:

try:
    questions = client.collections.create(
        name="Question",
        vectorizer_config=wvc.config.Configure.Vectorizer.text2vec_openai()
        generative_config=wvc.config.Configure.Generative.openai()
    )

My question is how can we use the local transformers (in our case: sentence-transformers-multi-qa-MiniLM-L6-cos-v1) instead of API based transformers.

Do we need to use/configure e.g. wvc.config.Configure.Vectorizer.text2vec() OR Weaviate will automatically choose/interpret the text2vec module from docker file ?

Pls guide and help.

Thank you.

2

hi @curious !

Welcome to our community :hugs:

You will need to specify text2vec_transformers as your vectorizer.

For example:

collection = client.collections.create(
    name="Collection",
    vectorizer_config=wvc.config.Configure.Vectorizer.text2vec_transformers()
)

It is also a nice tip to use a code editor that can autocomplete, for example:

image
image1690Ă—702 120 KB

this happens whenever I let the mouse hover text2vec_transformers

Also notice, if you do not specify the inference_url while defining the vectorizer, it will use the default value defined in your variable environment in your docker compose, for example.

Let me know if this helps!

2 Likes
3

Hi @DudaNogueira,

Thank you for the welcome and your response. It worked well and I’m able to do near_text query on my data.

There is one more question. I’ve put another docker file with larger models for t2v, qna, ner transformers etc.:

services:
  t2v-transformers:
    image: cr.weaviate.io/semitechnologies/transformers-inference:sentence-transformers-multi-qa-mpnet-base-cos-v1
    environment:
      ENABLE_CUDA: '0'
  qna-transformers:
    image: cr.weaviate.io/semitechnologies/qna-transformers:deepset-bert-large-uncased-whole-word-masking-squad2
    environment:
      ENABLE_CUDA: '0'
  ner-transformers:
    image: cr.weaviate.io/semitechnologies/ner-transformers:dbmdz-bert-large-cased-finetuned-conll03-english
    environment:
      ENABLE_CUDA: '0'
  sum-transformers:
    image: cr.weaviate.io/semitechnologies/sum-transformers:facebook-bart-large-cnn-1.0.0
    environment:
      ENABLE_CUDA: '0'
  text-spellcheck:
    image: cr.weaviate.io/semitechnologies/text-spellcheck-model:pyspellchecker-en
  reranker-transformers:
    image: cr.weaviate.io/semitechnologies/reranker-transformers:cross-encoder-ms-marco-MiniLM-L-6-v2
    environment:
      ENABLE_CUDA: '0'


ENABLE_MODULES: 'text2vec-transformers,qna-transformers,ner-transformers,sum-transformers,text-spellcheck,ref2vec-centroid,reranker-transformers'

Do I need to confgure the collection for each of them as well e.g. w.r.t my docker-compose file:

vectorizer_config = wvc.config.Configure.Vectorizer.text2vec_transformers(), 
generative_config = wvc.config.Configure.Generative.qna-transformers(), 
rerank_config = wvc.config.Configure.Reranker.reranker-transformers(),

and how to config collection with other modules like: ner-transformers,sum-transformers,text-spellcheck,ref2vec-centroid

Thank you.

4

Hi @curious !

For each of those modules you will need to configure their specifics details.

Some of them will allow you to setup a global, default value as the environment variable, defined for example under services > weaviate > environment section of you docker compose.

You can also define this values on a per collection base, and some of them may even allow you to define the baseurl, for example, on the client initialization.

Please, refer to our modules documentation:

And let me know if you have any doubts on that.

THanks!

5

Thank you @DudaNogueira for your reply and sharing the link. I will go through it.

One more query

If I’ve configured the collection with say text2vec_transformers and populated data in it. Later, if I also want to use generative (Q&A) module with this populated collection, is it possible to do so ?

6

It is, however, some properties are not mutable:

On this case, even it doesn’t kind of “doesn’t make sense” as the generative has no impact on the vectors, you cannot change that configuration.

So whenever you want to change the generative module of a collection, a full reindex is needed :grimacing:

3 Likes
7

Thank you for your reply and guidance. :+1: