# Comparing self managed embeddings

**URL:** <https://forum.weaviate.io/t/comparing-self-managed-embeddings/2044>\
**Category:** General\
**Tags:** developer-experience\
**Created:** [April 16, 2024, 1:43pm UTC](https://forum.weaviate.io/t/comparing-self-managed-embeddings/2044 "2024-04-16T13:43:51Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![rjalex](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/rjalex/32/807_2.png) [@rjalex](https://forum.weaviate.io/u/rjalex)\
**Post date:** [April 16, 2024, 1:43pm UTC](https://forum.weaviate.io/t/comparing-self-managed-embeddings/2044/1 "2024-04-16T13:43:51Z")

</div>

Hi fellow Weaviaters 🙂 I wish to test around 50.000 text objects with unique IDs, each with around 10 different embedding models.

So I will be looping in my text list and manually build a vector\_MODELNAME for each of the models I want to evaluate.

How would you efficiently store this in weaviate for this benchmarking case?

I can think of having one collection for each vectorizer.

Even though I am aware of named vectors, as I am not able to use any of the modules such as text2vec-transformers, I am not quite sure how to define/use named vectors with application provided vectors/embeddings.

I will then build a number of text queries each with a list of IDs of expected matching phrases as my ground truths.

Finally I will go though my list of queries, vectorrize them with each vectorizer and with each retrieve the phrases via the corresponding phrases and then match them to the ground truth.

I would love if you could suggest the best approach for this task.

Thank you

---

<div class="post-metadata">

**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\
**Post date:** [April 16, 2024, 6:41pm UTC](https://forum.weaviate.io/t/comparing-self-managed-embeddings/2044/2 "2024-04-16T18:41:35Z")

</div>

Ciao @rjalex !!

That’s look like a really exciting project.

While named vectors would be interesting to use for that, I believe that having one collection per model also will work.

Regarding named vectors, you mean bringing your own vectors to a named vector collection? that’s described here:

> **[Create objects | Weaviate - Vector Database](https://weaviate.io/developers/weaviate/manage-data/create#create-an-object-with-named-vectors)**
>
> The examples on this page demonstrate how to create individual objects in Weaviate.

Let me know how more I can help you on that 🙂