Swaping Vectorizers?

As vector generators get better and change, does that mean that the selected vector generator I’m using for classes needs to change? Or rather, if I want to change the vectorizer for a class, what impacts will that have? Does that mean I need to update all the vectors for the existing classes to use the new vectorizer?

I’m assuming that vectors from one generator don’t necessarily work with vectors from another.

Still learning…

Hi @bob !! Welcome to our community :hugs:

You are right. If you want to change the vectorizer of your classes, you will need to re-vectorize all your objects from that class with this new model and reindex the data, as it will require a brand new vector index.

The best way to accomplish this is migrating your data from one class to a new one (or from one server to a new one). That will trigger the new vectorization while creating your new index.

We have a nice documentation here on migrating your data around

And your assumption is correct: you can’t compare distances between vectors from different models. They will usually have different dimension size and their representations are different from each other.

Let me know if that helps :slight_smile:

Also, check our free, weekly workshops. We have both introductory and intermediate meetings with awesome content going one.

Thanks!

1 Like