[Question] Is it possible to partition Weaviate across geographically independent nodes (not just in some Kubernetes cluster)?

Version of the software doesn’t matter. Just wondering if possible? Can’t find specific docs.

An example would be if I want to run a massive Weaviate database and have it stored (partitioned) across N number of peer-to-peer connected nodes in a decentralized system.

Thanks.

hi @zorbowl ! Welcome to our community!

As long as each node can communicate with each other, it should work.

However, the “inter node” communication will not be encrypted. :thinking:

There is a new feature, that implements basic auth between the nodes

Hi, awesome!

Could you point me to some specific resources/links for this?

I’m building a p2p network using Weaviate where I want a vector embedding to “come into the network” and then Weaviate “does the magic to find which node has the data and returns it” (I can probably handle the encryption/auth issue).

Thanks DudaNogueira.

I don’t recall we have a doc specifically for that. :thinking:

Regarding the exact node, you can just write to node (or the node your client has more affinity), and Weaviate will take care of finding the correct node of your cluster (considering you are using sharding and replication on multiple nodes).

That is also true for querying. You can just query any node and Weaviate will reach out to other nodes of the cluster to complete the query.

1 Like