Dynamic shard creation using tenants

If my K3S-based Weaviate cluster starts with five pods—each hosting one tenant—and I scale the deployment to six pods and later I add another tenant (6th one), will Weaviate automatically assign that new tenant to the newly created pod?

Hi @Saketh

Weaviate will assign the new tenant to the node with the least resource usage. For example, if your cluster starts with five pods (each hosting one tenant) and you scale to six pods before adding a sixth tenant, Weaviate will place the new tenant on the newly added pod—assuming it has the lowest resource usage at that time.

Out of curiosity, could you clarify why you’re trying to add a tenant to a pod? I think there might be a structural misunderstanding here, so I’d like to better understand your use case and what you’re aiming to achieve.

Best regards,

Mohamed Shahin
Weaviate Support Engineer
(Ireland, UTC±00:00/+01:00)

We were thinking of adding more pods for faster query processing time as we horizontally scale our database and add more tenants dynamically.

I would say a 3–5 node cluster should be sufficient unless you’re expecting the database to grow significantly over time, in which case 7–9 nodes would be more appropriate.

Horizontal scaling can also be applied on multi-tenant collections, and it’s perfectly fine to add nodes on the go as you scale. Starting from version 1.32, shard movement is supported:

:link: Weaviate 1.32 Release | Weaviate

This feature will help a lot with scaling, so you’re well covered.

Alright thank you @Mohamed_Shahin !

1 Like