Weaviate Backup - 50,000 blocks ERROR CODE: BlockListTooLong

Support
, , ,
1

Hello Team

I am getting this error, when I am running weaviate backup(.create) method.

The block list may not contain more than 50,000 blocks.\nERROR CODE: BlockListTooLong\n--------------------------------------------------------------------------------\n\ufeff<?xml version="1.0" encoding="utf-8"?>BlockListTooLongThe block list may not contain more than 50,000 blocks.

This is a big dataset, and I believe this has something to do with the data block size and chunking issue.
Please let me know how I can fix this issue

  • Weaviate Server Version:
  • Deployment Method: Kubernetes
  • Number of Running Nodes: One node
  • Weaviate Version: 1.25.0
2

hi @sanjeev1678 !

What is the backup module you are using?

I did a search, and this error points to Azure: is it backup-azure?

Let me know if it is Azure indeed.

Thanks!

3

Hello @DudaNogueira

Thanks for responding. Yes this backup-azure.

Please let me know if anything else needed.

4

Hi @sanjeev1678 ,

Following back this topic, I think Add environment overrides for azure blocksize and concurrency by donomii · Pull Request #6468 · weaviate/weaviate · GitHub should have fix this issue.

It’s should be included in the latest release. Default block sized also changed to int64(40 * 1024 * 1024)

1 Like