Increasing replication factor for class

I am looking for some help with the following problem.

I have a multi-node setup of weaviate deployed with two replicas.

Initially when creating a the class ExampleClass I left its replicationConfig factor set to 1 and upload approx 140k objects to it.

Afterwards I decided to increase the replication factor of the class from 1 to 2 by updating it schema config.

It is my understanding that weaviate operates with eventual consistency and that by increasing this replication factor the class will eventually all be copied over to the other Node.

When I portforward to the original node-0 and run

(
    db.client.query.get(
        "ExampleClass",
        ["product_id", "title_en"],
    )
    .with_near_text({"concepts": ["fruit"]})
    .with_consistency_level(ConsistencyLevel.ALL)
    .with_limit(3)
).do()


The response is:

{'data': {'Get': {'ExampleClass': [{'product_id': '211',
     'title_en': 'Fruit Punch'},
    {'product_id': '210', 'title_en': 'Naranjilla Fruit Pulp'},
    {'product_id': '29', 'title_en': 'Mango Fruit Pulp'}]}}}

It works as expected.

However if I portforward to the second replica on node-1 and try the same command the response has no data returned.

{'data': {'Get': {'ExampleClass': []}}}

I can also see both nodes are aware of each other and have objects populated and the vector indexing is completed.
When I execute

client.cluster.get_nodes_status()

See the following

[{'batchStats': {'queueLength': 0, 'ratePerSecond': 96},
  'gitHash': 'f381d44',
  'name': 'rweaviate-dev-0',
  '
   {'class': 'ExampleClass',
    'name': 'QHtt1NlFVFlW',
    'objectCount': 147303,
    'vectorIndexingStatus': 'READY',
    'vectorQueueLength': 0}
  'stats': {'objectCount': 149214, 'shardCount': 5},
  'status': 'HEALTHY',
  'version': '1.22.5'},
},
 {'batchStats': {'queueLength': 0, 'ratePerSecond': 2112},
  'gitHash': 'f381d44',
  'name': 'weaviate-dev-1',
   {'class': 'ExampleClass',
    'name': 'QHtt1NlFVFlW',
    'objectCount': 147287,
    'vectorIndexingStatus': 'READY',
    'vectorQueueLength': 0}],
  'stats': {'objectCount': 225288, 'shardCount': 6},
  'status': 'HEALTHY',
  'version': '1.22.5'}]

Iā€™m wondering why when I portforward to weaviate-dev-1 it returns empty lists as its response in all cases but when I portforward to weaviated-dev-0 it works as expected?

Hi!

I believe it will sync IF you specify the consistency level:

Note that changing the replication factor after adding data is still in experimental mode

Let me know if this helps.