Documentation - Maximum index size, disk paging

Support
4

Hi Greg!

Thanks for those questions and registering your journey :slight_smile:

So yes, by default, Weaviate will keep data in memory. This is because vectorCacheMaxObjects is set to trilions.

So if you reduce this setting to a number less then your total ammount of objects, you will have some objects in DISK and in RAM.

For this ammount of data, Product Quantization and playing with vectorCacheMaxObjects will help, along of course with having enough disk and memory and finding the balance between speed and memory.

Let me know if this helps. We would love to understand better your usecase.

Feel free to ping us in our slack channels :slight_smile:

Thanks!