Weaviate replication setup on VM's

Support
1

Hi Team,

We are trying to install weaviate on VM’s with storage volumes attached (persistent disk) instead of k8s installation and we are looking for multi-DC replication.

Please suggest

  1. How to setup replication on VM based installation ?
  2. Is there any way we can have active - active replication setup across multiple k8s clusters spread across multi data center setup ?

Regards,
Adithya

2

hi @adithya.ch ! Welcome to our community :hugs:

AFAIK, as long as each node can communicate with each other, they will replicate, considering that the collection is set up accordingly (replication factor).

Of course, the communication between each node will have more latency as they are across DC, but that is expected.

Let me know if this helps :slight_smile:

3

Thank you for the quick reply !!

Can you share the documentation to enable replication ? What are the parameters to consider in config file.

Regards,
Adithya

4

Thank you for the quick reply @DudaNogueira !!

Can you share the documentation to enable replication ? What are the parameters to consider in config file.

Regards,
Adithya

5

@DudaNogueira

As of now we installed 3 standalone weaviate instances running on port 6443 using “How to run Weaviate using a binary?” document.

Can you share the details on the parameters to be used in weaviate.conf.json to enable replication between three standalone weaviate instances host1, host2, host3 ?

6

Hi! You will enable that on a per collection base.

Check here for more on that:

7

Thank you @DudaNogueira

We tried replication setup and it works after running the export commands

founding node

export PERSISTENCE_DATA_PATH=/tmp/wv
export CLUSTER_HOSTNAME='node0'
export CLUSTER_GOSSIP_BIND_PORT=27060
export CLUSTER_DATA_BIND_PORT=27061


second node

export PERSISTENCE_DATA_PATH=/tmp/wv
export CLUSTER_HOSTNAME='node1'
export CLUSTER_GOSSIP_BIND_PORT=27060
export CLUSTER_DATA_BIND_PORT=27061
export CLUSTER_JOIN=node0:27060

will add similarly for other nodes of the cluster.

But the problem here is, i want to add all these parameters into a config file not with the export commands. I don’t see any of the documents covering the actual parameter names to add into a conf file.

example what’s the key name for “CLUSTER_DATA_BIND_PORT” when adding to a yaml file ? Where i can get that information.

Rehards,
Adithya

8

There was a config file in the past, but it is now marked as deprecated, so not a path to follow, sorry :frowning:

Regarding the environment variables, here you can get more info on those:

Let me know if this helps :slight_smile: