Docker-compose with multi-node setup on production environment

[Sungwon_Jung]

Hello!

It might vary depending on the case, but if I’m looking to manage 5 billion 32-dimensional embeddings, is it okay to operate Weaviate using docker-compose in a multi-node setup for a production environment?

Thank you.

[DudaNogueira]

Hi @Sungwon_Jung ! Welcome to our Community

That should work. 5 bilion is a lot of vectors, so you will need some beefy hardware

We also have a helm chart in case you want to use K8s:

GitHub - weaviate/weaviate-helm: Helm charts to deploy Weaviate to k8s

I have answered a similar question here, but with isolated containers:

Can only Docker Compose be used for multi node deployment?

Hi @codehelen ! Welcome to our community This must be some networking issue along the way, as your docker is correct (see below) A starting point is making sure that weaviate-node2-IP can communicate with weaviate-node1-IP on all specified ports. A good example on how to run with ad-hoc containers (at the same docker host): First, create a docker attachable network docker network create weaviate --atachable note that weaviate, above, can be whatever network name you want. Mak…

Let me know if this helps

Thanks!

[Sungwon_Jung]

Hi, @DudaNogueira

K8s cannot be used, so it seems that we need to operate with docker-compose using a multi-node setup.
I will test further whether docker-compose with multi-node can be used in a production environment.

Thank you.