Restoring same backup multiple times

General
1

I am encountering an issue when trying to restore from a backup that has already been read by the weaviate instance once before.

This issue can be created via the following steps;

  1. Save a backup with idlatest which contains one class ExampleClass with gcs backend. Backup completes successfully.

  2. Delete ExampleClass within the weaviate instance.

  3. Restore backup with idlatest

  4. Verify restore was sucesfull.

  5. Delete ExampleClass Again within the weaviate instance.

  6. After attempting to restore the backup again I encounter the following error

Backup restore failed: {‘backend’: ‘gcs’, ‘classes’: [‘ExampleClass’], ‘id’: ‘latest’, ‘path’: ‘gs://weaviate/latest’, ‘status’: ‘FAILED’, ‘error’: ‘restore class ExampleClass: write files: move files to destination: move /var/lib/weaviate/.backup.tmp/ExampleClass/exampleclass_QHtt1NlFVFlW.hnsw.commitlog.d /var/lib/weaviate/ExampleClass_QHtt1NlFVFlW.hnsw.commitlog.d: rename /var/lib/weaviate/.backup.tmp/ExampleClass/exampleclass_QHtt1NlFVFlW.hnsw.commitlog.d /var/lib/weaviate/exampleclass_QHtt1NlFVFlW.hnsw.commitlog.d: file exists’}

It seems like it fails because it is trying to use the same log name each time.

Would appreciate any insights people can offer. In particular has anyone been able to preform multiple restores in the same weaviate instance using the same backup id?

2

Hi Landon!

That’s interesting. I will try to reproduce this on my end and will get back to you soon.

Some questions:
Is this a multinode cluster?

Looks like you are using GCP. Have you also noticed this issue while using S3?

Thanks for reporting!

3

I was not able to reproduce this.

As WCS uses gcs, i tried reproducing it there with one of our managed clusters.

I created a python notebook here: