Disable replication/raft for embedded Weaviate

Support
,
1

Description

Can replication/raft be disabled for embedded Weaviate?
I see warning/error log apparently related to raft as follows.

weaviate_version = "1.26.4"

client = weaviate.WeaviateClient(
    embedded_options=EmbeddedOptions(
        additional_env_vars={
            "LOG_LEVEL": "warning",
        },
        version=weaviate_version,
    )
)

output log:

{“action”:“raft”,“build_git_commit”:“584532a6b”,“build_go_version”:“go1.23.0”,“build_image_tag”:“localhost”,“build_wv_version”:“1.26.4”,“last-leader-addr”:“”,“last-leader-id”:“”,“level”:“warning”,“msg”:“raft heartbeat timeout reached, starting election”,“time”:“2024-09-08T21:57:38+08:00”}

client.close()

output log:

{“build_git_commit”:“584532a6b”,“build_go_version”:“go1.23.0”,“build_image_tag”:“localhost”,“build_wv_version”:“1.26.4”,“error”:“cannot find peer”,“level”:“error”,“msg”:“transferring leadership”,“time”:“2024-09-08T21:57:40+08:00”}

I’m seeing this behavior in my dev machine MacOS Sonoma 14.6.1 before using actual data in Linux (CentOS).

Server Setup Information

  • Weaviate Server Version: 1.26.4
  • Deployment Method: embedded
  • Multi Node? Number of Running Nodes: 1
  • Client Language and Version: Python 3.11 weaviate-client==4.7.1
  • Multitenancy?: No
2

Good morning @Minyus,

Welcome to the Weaviate community! It’s awesome to have you here with us.

I understand that you’re looking to disable some warning messages related to RAFT. In release 1.25, we introduced the RAFT mechanism,

RAFT is a robust consensus algorithm that improves the fault tolerance of multi-node clusters. With larger Weaviate clusters, coordination between nodes becomes more important, and RAFT helps ensure reliable performance even under heavy loads.

Since you’re currently running version 1.26.4 on a single node, have you considered upgrading to a 3-node cluster? This setup is highly recommended.

Alternatively, if you remain on one node for now and want to reduce the log output on a single node, you could consider setting the LOG_LEVEL to one of the following:

  • panic: Panic entries only.
  • fatal: Fatal entries only.
  • error: Error entries only.
3

hi @Minyus !

You can get rid of that message by setting RAFT_EXPECT_JOIN to 1.

Bear in mind that this is not necessary, as it will have this error log, but it should not affect Weaviate operations, but reduce the error logs on this scenario.

Also, I noticed you are using the python v3 client syntax. Please, consider using the new python v4 client, as it will deliver a lot of improvements.

Here is an example:

import weaviate
client = weaviate.connect_to_embedded(
    environment_variables={
            "LOG_LEVEL": "warning",
            "RAFT_BOOTSTRAP_EXPECT": "1"
        }
)

this was the output I got:

{“level”:“warning”,“msg”:“Multiple vector spaces are present, GraphQL Explore and REST API list objects endpoint module include params has been disabled as a result.”,“time”:“2024-09-09T11:39:12-03:00”}
{“action”:“raft”,“last-leader-addr”:“”,“last-leader-id”:“”,“level”:“warning”,“msg”:“raft heartbeat timeout reached, starting election”,“time”:“2024-09-09T11:39:14-03:00”}

Let me know if this helps!

THanks!