High Memory Usage After Upgrading Weaviate to Version 1.25

General
, ,
1

Hello Team,

I hope this message finds you well.

We are encountering a memory issue after recently upgrading our Weaviate version from 1.18 to 1.25. Previously, with version 1.18, our system was running on 32GB of memory without any issues. However, after upgrading to 128GB of memory with the new version, we are seeing a memory usage of 70%. While we have added new records, the memory consumption seems unusually high and unexpected.

Could you please advise on how to resolve this issue? Let me know if you need any additional details or further adjustments.

  • Weaviate Server Version:
  • Deployment Method: Kubernetes
  • Number of Running Nodes: One node
  • Weaviate Version: 1.25.0
2

Hello @sanjeev1678,

How are you doing? I hope you’re having a great Monday!

Great job on upgrading to one of latest Weaviate version, as the version you were using was quite outdated. Between versions 1.18 and 1.25, there have been significant updates.

Could you please share more with us:

  1. Your schema structure.
  2. The data volume, including the number of objects and the size of the vectors.
  3. Any error logs or warnings you’ve seen.
  4. The memory metrics you are observing during operations.
3

Hello @Mohamed_Shahin

Thank you for your response.

I am using the same data schema and volume that I was using with the older 1.18 version. However, I’m noticing that the RAM usage in version 1.25 is significantly higher, more than double what it was before.

There is no error in the logs everything working fine.

Could you please assist me with this issue.

4

hi @sanjeev1678 !!

Have you changed some variable in the server?

There is a configuration that you can set on a per collection basis that can help you on this kind of issues.

Check here for the what can affect the memory usage:

Let me know if thath helps!

THanks!

5

Hello @DudaNogueira

Thanks for your response.

Since this schema is already made and data has been trained and live on the server, can I still tweak it using configuration settings?

If so, could you please guide me on which configuration settings I should adjust?

6

hi!

The environment variables can be changed at any time.

However, the configurations that are bound to the collection may or may not be mutable.

We host a list of mutable collection properties here:

If you want to change a configuration that is not mutable, you will need to reindex or data, meaning copying it over to a new collection in the same cluster or on a new cluster.

Here we have a migration guide for that:

Let me know if that helps!