System Requirement Hard Limits?

Support
1

I am trying to upload 4 million 384d vectors.

1 million requires ~3Gb of Ram as per documentation.

I have 4Gb of Ram (30Gb ssd) on a low end server. The documentation in one spot makes it sound like a hard cap for vector size. But another spot implies that data is offloaded to the hard drive and is slower, but what is the cap, hd space?

What are the hard limits?

Also qdrant allows memmaps to have a 70% reduction in ram for harddrive tradeoff. I see weaviate has memmaps, is this possible in weaviate as well? They can have 4million (384) in 3Gb of Ram (reportedly).

2

Hi! Sorry for the delay here :grimacing:
Missed this one :frowning: Did you have any findings here?

AFAIK, there are no hard limits. Well, some vector size limits, but not sure anybody needs that ammount of vectors… yet :sweat_smile:

I will poke internally, as I am not expert in this area yet.

Thanks!