I am getting this error, when I am running weaviate backup(.create) method.
The block list may not contain more than 50,000 blocks.\nERROR CODE: BlockListTooLong\n--------------------------------------------------------------------------------\n\ufeff<?xml version="1.0" encoding="utf-8"?>BlockListTooLongThe block list may not contain more than 50,000 blocks.
I’ve been experimenting with the chunk_size and compressionLevel parameters in the backup configuration to address this issue.
Please find my code.
Given the size of the dataset, I suspect that the error is related to the number of data blocks generated during the backup process. To reduce the total number of blocks, I increase the chunk size, but still giving the same error.
Please let me know how I can fix this issue
“backup class industryintel descriptor: cannot create new backup, backup ‘weaviate-backup_20240902600000’ is not yet released, this means its contents have not yet been fully copied to its destination, try again later”
I was expecting these to be resolved in newer versions of Weaviate.
I understand, but will the issue with “backup not released yet” be resolved in the latest release?
A recurring issue we’re facing is that whenever a previous backup fails for any reason, this “backup release not yet” error appears for all subsequent backups.
This is a significant problem because once this issue occurs, entire backup pipeline stops functioning. It’s strange, and we should have some form of exception-handling mechanism to address and resolve this issue.
The error message we receive is: “cannot create new backup, backup ‘weaviate-backup_20240902600000’ is not yet released.”