# High resource usage in Flat Index **URL:** \ **Category:** Support\ **Tags:** developer-experience\ **Created:** [December 6, 2024, 12:22pm UTC](https://forum.weaviate.io/t/high-resource-usage-in-flat-index/9115 "2024-12-06T12:22:27Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Dharanish](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dharanish/32/659_2.png) [@Dharanish](https://forum.weaviate.io/u/Dharanish)\ **Post date:** [December 6, 2024, 12:22pm UTC](https://forum.weaviate.io/t/high-resource-usage-in-flat-index/9115/1 "2024-12-06T12:22:27Z")
Team, I have created 10,000 tenants and imported 100 data for each tenant. My configured GOMAXPROCS and GOMEMLIMIT values as per the setup needs. ![image](https://canada1.discourse-cdn.com/flex027/uploads/weaviate/optimized/2X/7/74a7d1f7f53bd98e113e6895f247c10ff5ec2f0e_2_690x28.png) The above is the resource consumption for the data, and it seems very high for the flat index in terms of both CPU and memory. Weaviate version: 1.23.11 Python client: 4.5.7 Vector Dimension: 384 Replica : 2 No. of properties: 4 Total number of records : 1000000. ---
**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\ **Post date:** [December 6, 2024, 6:51pm UTC](https://forum.weaviate.io/t/high-resource-usage-in-flat-index/9115/2 "2024-12-06T18:51:51Z")
hi @Dharanish ! I would know what would be the usage on this scenario 🤔 I am curious: Have you benchmarked this with other databases? Thanks! ---
**Author:** ![Dharanish](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dharanish/32/659_2.png) [@Dharanish](https://forum.weaviate.io/u/Dharanish)\ **Post date:** [December 11, 2024, 1:03pm UTC](https://forum.weaviate.io/t/high-resource-usage-in-flat-index/9115/3 "2024-12-11T13:03:25Z")
Sorry for the delayed response No, we are primarily using Weaviate for production; there is a new usecase and we don’t evaluate with other DBs yet. ---
**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\ **Post date:** [December 11, 2024, 8:30pm UTC](https://forum.weaviate.io/t/high-resource-usage-in-flat-index/9115/4 "2024-12-11T20:30:48Z")
Oh, ok. So what are the allocated values for memory and cpu, and what are the values of GOMEMLIMIT and GOMAXPROCS? Not sure you have read thru this nice blog article about GOMEMLIMIT: > **[GOMEMLIMIT is a game changer for high-memory applications | Weaviate](https://weaviate.io/blog/gomemlimit-a-game-changer-for-high-memory-applications)** > > Go 1.19 introduced GOMEMLIMIT, which completely changes how you can manage memory limits in Go. Learn how it helps Weaviate be more reliable. Because you are using a flat index, your heap will not be an issue here, (you are storing on disk, not on memory), but setting those variables can force garbage collection kick in quicker and free up memory. Let me know if that helps! THanks! ---
**Author:** ![Dharanish](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dharanish/32/659_2.png) [@Dharanish](https://forum.weaviate.io/u/Dharanish)\ **Post date:** [December 18, 2024, 9:38am UTC](https://forum.weaviate.io/t/high-resource-usage-in-flat-index/9115/5 "2024-12-18T09:38:59Z")
@DudaNogueira , I’ve read that article before and set values for those variables based on our resource needs. I’ve found the catch that this resource consumption is due to PROMETHEUS\_MONITORING\_ENABLED. After disabled, we get ![image](https://canada1.discourse-cdn.com/flex027/uploads/weaviate/original/2X/1/1937da9cc1a8b4abceffb8b294647a58300f6aeb.png) It seems prometheus metrics collection plays a major role in resource consumption. We found this using go profiling tool But don’t know why the Prometheus client used in Weaviate consumes that much amount of resources. ---
**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\ **Post date:** [December 18, 2024, 1:17pm UTC](https://forum.weaviate.io/t/high-resource-usage-in-flat-index/9115/6 "2024-12-18T13:17:37Z")
Nice catch!! Do you see the same behavior in latest versions? A lot has happened since 1.23… 😬 ---
**Author:** ![Dharanish](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dharanish/32/659_2.png) [@Dharanish](https://forum.weaviate.io/u/Dharanish)\ **Post date:** [January 23, 2025, 5:22am UTC](https://forum.weaviate.io/t/high-resource-usage-in-flat-index/9115/7 "2025-01-23T05:22:07Z")
Hi , I saw the same behaviour in 1.27 too. It seems LSM specific metrics grow exponentially with respect to the number of collections and tenants.