# Benchmarking two vectorizers - best pattern?

**URL:** <https://forum.weaviate.io/t/benchmarking-two-vectorizers-best-pattern/1217>\
**Category:** General\
**Created:** [January 12, 2024, 12:18pm UTC](https://forum.weaviate.io/t/benchmarking-two-vectorizers-best-pattern/1217 "2024-01-12T12:18:20Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![rjalex](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/rjalex/32/807_2.png) [@rjalex](https://forum.weaviate.io/u/rjalex)\
**Post date:** [January 12, 2024, 12:18pm UTC](https://forum.weaviate.io/t/benchmarking-two-vectorizers-best-pattern/1217/1 "2024-01-12T12:18:20Z")

</div>

I am in the need of comparing the precision and recall of two different vectorizers on a test corpus of 150000 titles and kickers (in Italian) as my test.

One of them is the OpenAI “text-embedding-ada-002”. So far I have declared a schema for this (with vectorizer = none but guess I might restart Weaviate with this module and let the vectorization happen in the DB and not in my application).

The second vectorizer is SentenceTransformer(“nickprock/sentence-bert-base-italian-uncased”) and this I am adding via my application (I still don’t understand if I can configure this external model as a module).

Until now I filled up the WpAdmin schema with data and the OpeanAI vectorization, made measurements, then wiped Weaviate clean, restarted and repeated with the other vectorizer.

By reading in this forum maybe a better idea would be to declare two distinct schemas?

This is my current definition of the schema I am custom vectorizing:

```auto

def weaviate_create_wp_schema(weclient: weaviate.client.Client, classname: str) -> None:
    """define the data schema (class) as a JSON object to be stored in Weaviate"""
    schema = {
        "classes": [
            {
                "class": classname, # first letter capitalized regardless of how it is here
                "description": "Contains the WP article text data along with embeddings",
                "vectorizer": "none", # we will provide the embedding, not let Weaviate do it
                "properties": [
                    {
                        "name": "slug",
                        "dataType": ["text"],
                        "vectorize": False, # This field will NOT be vectorized
                    },
                    {
                        "name": "kicker",
                        "dataType": ["text"],
                    },
                ],
            }
        ]
    }
    if not check_class_exists(weclient, classname):
        weclient.schema.create(schema)

```

Thanks for any suggestion or example.

---

<div class="post-metadata">

**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\
**Post date:** [January 15, 2024, 6:42pm UTC](https://forum.weaviate.io/t/benchmarking-two-vectorizers-best-pattern/1217/2 "2024-01-15T18:42:02Z")

</div>

Hi!

Yes! For now you will need to define two schemas.

Hopefully, AFAIK, in 1.24 ([current roadmap](https://weaviate.io/developers/weaviate/roadmap)) you will be able to have multiple vectors per object. So this will open up the possibility easily comparing different models using the very same object.

regarding the usage of your custom model as a vectorizer module, one thing you could do is to adapt this container to use your own model:

> **[GitHub - weaviate/t2v-transformers-models: This is the repo for the container...](https://github.com/weaviate/t2v-transformers-models)**
>
> This is the repo for the container that holds the models for the text2vec-transformers module - GitHub - weaviate/t2v-transformers-models: This is the repo for the container that holds the models f...

Let me know if this helps 🙂

---

<div class="post-metadata">

**Author:** ![rjalex](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/rjalex/32/807_2.png) [@rjalex](https://forum.weaviate.io/u/rjalex)\
**Post date:** [January 16, 2024, 12:33pm UTC](https://forum.weaviate.io/t/benchmarking-two-vectorizers-best-pattern/1217/3 "2024-01-16T12:33:29Z")

</div>

Thanks will try as soon as I can.