# Increase number of shards and update HNSW vector index parameters **URL:** \ **Category:** Support\ **Tags:** python\ **Created:** [May 7, 2024, 9:46pm UTC](https://forum.weaviate.io/t/increase-number-of-shards-and-update-hnsw-vector-index-parameters/2231 "2024-05-07T21:46:06Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![cpwalker](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/cpwalker/32/1635_2.png) [@cpwalker](https://forum.weaviate.io/u/cpwalker)\ **Post date:** [May 7, 2024, 9:46pm UTC](https://forum.weaviate.io/t/increase-number-of-shards-and-update-hnsw-vector-index-parameters/2231/1 "2024-05-07T21:46:06Z")
Hi there, I have questions related to two topics, both motivated by needing to significantly increase query throughput. ### Description #### 1. Increase number of shards (Based on the advice given [here](https://weaviate.io/developers/weaviate/concepts/resources#:~:text=To%20use%20multiple%20CPUs%20efficiently,make%20use%20of%20multiple%20threads.)) I did not set the number of shards when creating my collection, so I have just 1 shard currently. The [documentation](https://weaviate.io/developers/weaviate/config-refs/schema#shardingconfig) says the default number of shards though is 128. Is 128 shards the recommended number to start with? Can I update the number of shards on a collection that already exists? #### 2. Update HNSW vector index parameters I am using a [HNSW index](https://weaviate.io/developers/weaviate/config-refs/schema/vector-index#hnsw-index-parameters). I would like to decrease `dynamicEfMax` and increase `efConstruction`. My understanding is that these changes would make queries faster but imports slower, a trade-off I can make. How do I update `dynamicEfMax` and `efConstruction` for an existing collection? ### Server Setup Information - Weaviate Server Version: 1.24 - Deployment Method: Docker - Multi Node? Number of Running Nodes: 1 node - Client Language and Version: Python v4 ---
**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\ **Post date:** [May 8, 2024, 12:12am UTC](https://forum.weaviate.io/t/increase-number-of-shards-and-update-hnsw-vector-index-parameters/2231/2 "2024-05-08T00:12:48Z")
hi @cpwalker ! the 128 value is for the `virtualPerPhysical` shards. `efConstruction` is not a mutable configuration, as per the [docs on mutability](https://weaviate.io/developers/weaviate/config-refs/schema#mutability). So for that one you will need to reindex your collections. `dynamicEfMax`, on the other hand, is mutable. If you want to increase query throughput , your best path is leveraging replication and using a multi node deployment. You can read more on that here: > **[Replication Architecture | Weaviate - Vector Database](https://weaviate.io/developers/weaviate/concepts/replication-architecture)** > > Weaviate can automatically replicate data across nodes in the background in a cluster with multiple server nodes. This enables a variety of use cases. For example, if a node goes down, another node can shoulder the load without loss of availability... Let me know if this helps. Thanks! ---
**Author:** ![cpwalker](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/cpwalker/32/1635_2.png) [@cpwalker](https://forum.weaviate.io/u/cpwalker)\ **Post date:** [May 8, 2024, 12:31am UTC](https://forum.weaviate.io/t/increase-number-of-shards-and-update-hnsw-vector-index-parameters/2231/3 "2024-05-08T00:31:16Z")
Thanks @DudaNogueira . Do you think it’s worth experimenting with `desiredCount` in the shard config? If so, what values would you try? I need to stay on 1 node for now. If it helps, here is additional context: I have ~8m vectors, each with 768 dimensions. The node has 24 virtual CPUs and 120GB RAM. ---
**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\ **Post date:** [May 8, 2024, 7:35pm UTC](https://forum.weaviate.io/t/increase-number-of-shards-and-update-hnsw-vector-index-parameters/2231/4 "2024-05-08T19:35:21Z")
If you can only use 1 node, I don’t believe having more shards on that same node will be do any good for performance. Much the opposite, as Weaviate will need to check multiple shards for a query in the same node. 🤔 If, by any chance, you are running a multitenant environment, then it could help. In fact, each tenant is a “shard” on it’s own. The upside of using tenants is that you can enable and disable. On that topic, multi tenant, 1.25 will bring some really nice cool features, like tenant TTL: it can be auto deactivated after some time of inactivity and activated when a new event come it’s way and it is in a COLD/DEACTIVATED state. In order to increase QPS, multi node is something to consider for you use case. Other than that, it is about closely monitoring resource consumption, and making sure it has enough room to operate. ---
**Author:** ![cpwalker](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/cpwalker/32/1635_2.png) [@cpwalker](https://forum.weaviate.io/u/cpwalker)\ **Post date:** [May 8, 2024, 9:44pm UTC](https://forum.weaviate.io/t/increase-number-of-shards-and-update-hnsw-vector-index-parameters/2231/5 "2024-05-08T21:44:45Z")
Thank you this is very helpful. ---
**Author:** ![bsandi](https://avatars.discourse-cdn.com/v4/letter/b/73ab20/32.png) [@bsandi](https://forum.weaviate.io/u/bsandi)\ **Post date:** [August 28, 2024, 6:15pm UTC](https://forum.weaviate.io/t/increase-number-of-shards-and-update-hnsw-vector-index-parameters/2231/6 "2024-08-28T18:15:34Z")
Hey @DudaNogueira, I am interested in the tenant TTL, and I wasn’t able to find any documentation around it. Was that implemented? I only see tenant states and that doesn’t work for my use case. It would be great if you can provide any updates ---
**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\ **Post date:** [August 28, 2024, 8:40pm UTC](https://forum.weaviate.io/t/increase-number-of-shards-and-update-hnsw-vector-index-parameters/2231/7 "2024-08-28T20:40:36Z")
hi @bsandi !! Welcome to our community. 🤗 This was under development and it was internal. I believe it was already delivered to some of our paying customers. Let me know if you want to connect to our sales engineers so we can understand your use case better. Also they have more context on this feature. Thanks!!