Hi, my team is attempting to migrate Weaviate Cloud data from version 1.25 to a 1.28 cluster. The core code of the migration script is as follows. We are using the Weaviate client v4.
When executing the migration, the following errors occur. We have tried using dynamic batch or fixed-sized batch settings with size=100 and concurrent_request=1, but there are still a large number of errors. After multiple errors, the cluster node will directly disconnect and restart.
with collection_tgt.batch.dynamic() as batch:
for q in collection_src.iterator(include_vector=True):
batch.add_object(
properties=q.properties,
vector=q.vector[“default”],
uuid=q.uuid
)
Welcome to the community! It’s great to have you here.
I’ve received your ticket in Support and will follow up with a detailed reply there since you are a cloud customer and our ticketing system differs from the community forum. You could add the following snippet to your code to help identify the reason behind any failures:
failed_objects = client.batch.failed_objects
if failed_objects:
print(f"Number of failed objects: {len(failed_objects)}“)
for i, failed_obj in enumerate(failed_objects, 1):
print(f"Failed object {i}: {failed_obj}”)
else:
print(f"All objects successfully inserted into ‘{collection_name}’.")
This code will check for any objects that failed to be inserted and print out the reasons for the failures, helping you pinpoint what might be going wrong.
