How much memory capacity is needed to use multi2vec-bind vectorizer?

General
1

I’m trying to start containers with
weaviate, multi2vec-bind, reranker-transformers.

However, multi2vec-bind container continues to die with following logs.

- INFO:     Started server process [7]
- INFO:     Waiting for application startup.
- INFO:     CUDA_CORE set to cuda:0
- Killed

I’m using 16GB RAM and 12GB VRAM GPU.
Approximately, How much resources do I need?

thank you.

2

hi @Min !!

Welcome to our community :hugs:

That is a hungry model, I tell you that :joy:

While I can run it nice on a macbook pro m2 with 32G, it is still slow, when compared to running on a CUDA enabled device.

I have run this on linux on top of a i7 (don’t recall all the specs now) with 32G, but I had to restart the laptop and then run it first thing.

And sometimes it wouldn’t load the model and crash due to the lack of horse power.

If running on windows or mac, make sure to allow docker to have access to moe resources.

1 Like