I am looking for some help with the following problem.
I have a multi-node setup of weaviate deployed with two replicas.
Initially when creating a the class ExampleClass I left its replicationConfig factor set to 1 and upload approx 140k objects to it.
Afterwards I decided to increase the replication factor of the class from 1 to 2 by updating it schema config.
It is my understanding that weaviate operates with eventual consistency and that by increasing this replication factor the class will eventually all be copied over to the other Node.
When I portforward to the original node-0 and run
(
db.client.query.get(
"ExampleClass",
["product_id", "title_en"],
)
.with_near_text({"concepts": ["fruit"]})
.with_consistency_level(ConsistencyLevel.ALL)
.with_limit(3)
).do()
The response is:
{'data': {'Get': {'ExampleClass': [{'product_id': '211',
'title_en': 'Fruit Punch'},
{'product_id': '210', 'title_en': 'Naranjilla Fruit Pulp'},
{'product_id': '29', 'title_en': 'Mango Fruit Pulp'}]}}}
It works as expected.
However if I portforward to the second replica on node-1 and try the same command the response has no data returned.
{'data': {'Get': {'ExampleClass': []}}}
I can also see both nodes are aware of each other and have objects populated and the vector indexing is completed.
When I execute
client.cluster.get_nodes_status()
See the following
[{'batchStats': {'queueLength': 0, 'ratePerSecond': 96},
'gitHash': 'f381d44',
'name': 'rweaviate-dev-0',
'
{'class': 'ExampleClass',
'name': 'QHtt1NlFVFlW',
'objectCount': 147303,
'vectorIndexingStatus': 'READY',
'vectorQueueLength': 0}
'stats': {'objectCount': 149214, 'shardCount': 5},
'status': 'HEALTHY',
'version': '1.22.5'},
},
{'batchStats': {'queueLength': 0, 'ratePerSecond': 2112},
'gitHash': 'f381d44',
'name': 'weaviate-dev-1',
{'class': 'ExampleClass',
'name': 'QHtt1NlFVFlW',
'objectCount': 147287,
'vectorIndexingStatus': 'READY',
'vectorQueueLength': 0}],
'stats': {'objectCount': 225288, 'shardCount': 6},
'status': 'HEALTHY',
'version': '1.22.5'}]
Iām wondering why when I portforward to weaviate-dev-1 it returns empty lists as its response in all cases but when I portforward to weaviated-dev-0 it works as expected?