memberlist: Got ping for unexpected node node1 from=59.70.88.222:60544"

Support
1

Description

help!

I’m deploying weaviate on three machine with docker.
I got the error on both master node and client node.
the error is ：

{“level”:“warning”,“msg”:" memberlist: Got ping for unexpected node ‘node2’ from=59.70.88.221:7100",“time”:“2025-03-09T14:51:16Z”}
{“level”:“warning”,“msg”:" memberlist: Got ping for unexpected node node2 from=59.70.88.221:34518",“time”:“2025-03-09T14:51:16Z”}
{“level”:“error”,“msg”:" memberlist: Failed fallback TCP ping: EOF",“time”:“2025-03-09T14:51:16Z”}
{“level”:“info”,“msg”:" memberlist: Suspect node2 has failed, no acks received",“time”:“2025-03-09T14:51:17Z”}
{“level”:“warning”,“msg”:" memberlist: Got ping for unexpected node ‘node2’ from=59.70.88.221:7100",“time”:“2025-03-09T14:51:17Z”}
{“level”:“warning”,“msg”:" memberlist: Got ping for unexpected node node2 from=59.70.88.221:32924",“time”:“2025-03-09T14:51:17Z”}
{“level”:“error”,“msg”:" memberlist: Failed fallback TCP ping: EOF",“time”:“2025-03-09T14:51:17Z”}
{“level”:“info”,“msg”:" memberlist: Suspect node2 has failed, no acks received",“time”:“2025-03-09T14:51:19Z”}
{“level”:“warning”,“msg”:" memberlist: Got ping for unexpected node ‘node2’ from=59.70.88.221:7100",“time”:“2025-03-09T14:51:20Z”}
{“level”:“warning”,“msg”:" memberlist: Got ping for unexpected node node2 from=59.70.88.221:32932",“time”:“2025-03-09T14:51:20Z”}
{“level”:“error”,“msg”:" memberlist: Failed fallback TCP ping: EOF",“time”:“2025-03-09T14:51:20Z”}
{“level”:“info”,“msg”:" memberlist: Marking node2 as failed, suspect timeout reached (0 peer confirmations)“,“time”:“2025-03-09T14:51:21Z”}
{“level”:“info”,“msg”:” memberlist: Suspect node2 has failed, no acks received",“time”:“2025-03-09T14:51:23Z”}

Server Setup Information

  • Weaviate Server Version: 1.26.1
  • Deployment Method: docker
  • Multi Node? Number of Running Nodes: 3
  • Client Language and Version:python
  • Multitenancy?:

Any additional Information

2

hi!

It seems that they are not able to communicate with each other.

The recommended way for multi node cluster is using our helm charts

By the way, why 1.26.1?

We suggest always running the latest versions or at least, in your case. 1.26.latest (as I write, 1.26.17)

Let me know if that helps!

Thanks!