Infra Configuration for Docker Setup of Weaviate

We are planning to develop a Global Search functionality for our application that has 1000 million of records.
I want to understand the approach to identify ideal Infra configuration for the setup.

hi @saurbhhsharrma !

Welcome to our community :hugs:

Have you seen this doc?

You can use it as guide on how to properly plan your resources allocation.

For instance, it will depend on how many dimensions each of your vectors have.

More dimensions, more memory.

Also, you can leverage features like Product Quantization in order to compress and reduce the memory usage, as well as, if that’s the case, using multi tenancy, so you can offload unused tenants:

Let me know if this helps :slight_smile: