Nodes/shards objects imbalance

Description

We have a problem with replica synchronization (deletions not fixed when nodes fail/restart) in a cluster of 3 nodes and 3 replicas. Running batch/cursor read process with ConsistencyLevel=ALL does not help. Tried even full reading with connect (route) to different (every) nodes.

Some cluster info, private fields are masked (ip-addresses, schema/property names):

Found some issues:

Are any advices avalaible?

Can you tell about ‘SEARCH_FORCE_EXPAND_TO_ALL_REPLICAS’ flag:
Add flag to expand search to all available replicas
How to use it? Not found any use in v1.25.12 sources (or I’m wrong?).

Server Setup Information

  • Weaviate Server Version: 1.25.12
  • Deployment Method: k8s
  • Multi Node? Number of Running Nodes: Yes, 3
  • Client Language and Version: Python3, Python Client v3
  • Multitenancy?: No

hi @wvuser !!

Really sorry, missed this thread :grimacing:

Do you still see this error?

a lot changed specifically for this area om 1.26

Thanks!