I want to deploy weaviate on three servers via docker-compose to achieve high availability of the cluster. How should it be configured?

Support
, ,
1

Description

The docker three-node cluster deployment method we adopted before was deployed on one server. However, after the deployment was completed, we felt that there were some performance issues when using it. Now we want to cluster and deploy on three servers as three nodes.

Server Setup Information

  • Weaviate Server Version: 1.30.0
  • Deployment Method: docker
  • Multi Node? Number of Running Nodes: 3
  • Client Language and Version: python / weaviate sdk 4.13.2
  • Multitenancy?: yes

Any additional Information

2

Hi @Knifebro,

Welcome to our community! It’s lovely to have you with us :hugs:

I haven’t personally come across this exact setup before, but perhaps someone else in the community might be able to come in and share their experience with us both.

That said, it is possible for sure with some tweaks in configuration and networking. Specifically, on each server you’d need to:

  • Adjust the docker-compose.yml so that each node has a unique CLUSTER_HOSTNAME, CLUSTER_GOSSIP_BIND_PORT, and CLUSTER_DATA_BIND_PORT.
  • Ensure that both the CLUSTER_JOIN and RAFT_JOIN environment variables reference all nodes in the cluster, using the correct internal hostnames or IPs.

That would be a good place to start - multi-node-configuration:

Best regards,

Mohamed Shahin
Weaviate Support Engineer
(Ireland, UTC±00:00/+01:00)

3

In my configuration file, for the two fields “CLUSTER GOSSIP BIND PORT” and “CLUSTER DATA BIND PORT”, which port should be configured? I previously configured the 7100 and 7101, but every time I started them, it would report a communication issue with the node, saying that the 8300 was not working?

4

This is my current error message

报错
报错1503×837 156 KB

5

Can the docker cluster deployment method support cross-host container communication through the overlay network?