422 error no backup backend s3

Support
1

Hello,
I’m currently facing an issue with Weaviate while attempting to perform a backup to an S3 bucket. I receive the following error code when attempting to create a backup to an s3 bucket: “Unexpected status code: 422, with response body: {‘error’: [{‘message’: 'no backup backend “s3”: backup: s3 not found, did you enable the right module?”.
This is despite including ‘ENABLE_MODULES=backup-s3’ & ‘BACKUP_S3_BUCKET=bucket_name’ in my .env and docker-compose.yml config.

Here is my code:

Screenshot 2023-07-03 at 09.06.15
Screenshot 2023-07-03 at 09.06.151116×1020 117 KB

Any help is appreciated.
Thanks

2

Are you sure that you have enabled this module?

If you have this module enabled then it should be in the /meta response. Can you verify this?

meta_info = client.get_meta()
print(meta_info)
3

It doesn’t appear in meta_info, I am loading ‘ENABLE_MODULES’ from .env. Can the module be passed as an additional header?

4

Ok, so you don’t have it enabled. You need to update your ENABLE_MODULES and add backup-s3 module env setting in your environment, where you have your Weaviate deployed, this will resolve your problem then.

5

.env:
ENABLE_MODULES=backup-s3
docker yaml:
environment:
ENABLE_MODULES: backup-s3
However I still receive 422 error. I am using weaviate-client==3.21.0

6

Can you post the full docker-compose.yaml (or alternatively Helm values.yaml)? There must be a mistake in there somewhere. It seems ENABLE_MODULES is not correctly propagated to the server.

7

the issue has been resolved.

8

hey, what was your solution?

9

Hey, did you manage to resolve it? I am having the same issue.


weaviate_client = weaviate.connect_to_weaviate_cloud(
    cluster_url=WEAVIATE_CREDENTIALS["cluster_url"],
    auth_credentials=Auth.api_key(WEAVIATE_CREDENTIALS["api_key"]),
    headers={"X-OpenAI-Api-key": openai_api_key},
)

meta_info = weaviate_client.get_meta()

print(meta_info)

shows that there is a ‘backup-gcs’, which I did not provide, with bucket_name ‘weaviate-wcs-prod-cust-europe-west3-workloads-backups’

I have ENABLE_MODULES=backup-s3 in my .env as well as BACKUP_S3_BUCKET, both are loaded and present before the connection is made to weaviate cloud.

But there is no backup-s3 in meta_info!

Is this something that needs to be enabled when the cluster is made on weaviate? As I dont see s3 in modules

Screenshot 2025-02-26 at 00.31.48
Screenshot 2025-02-26 at 00.31.48857×379 33.9 KB

10

Tagging you specifically Zac in hopes that you will remember the solution!

11

hi @Sneezles !!

Welcome to our community :hugs:

For clusters hosted in our cloud, we already configure and keep all the backups in place. You don’t need to worry about that.

You can always call an “adhoc” backup, on top of the ones we do daily and keep securely stored.

For that, you can figure out the backup module your cluster have configured with

client.cluster.get_meta()

and call client.backup.create() passing that backend and id of your new backup.

For more information on backups commands, please refer to:

Thanks!

12

Thx for the warm velcome! :slight_smile:

I still have some questions though, that I did not find in the documentation:

  • You said the backups happend daily, how long are backups saved? How to access a backup from 30 days ago?
  • When doing an adhoc backup, will it expire at some point? Are there extra costs associated with making multiple backups?

Reason I am asking this is because we update our weaviate collection automatically when customers upload their documents. It can happen that they fail, something goes awry, so the idea is to make a backup before something goes wrong, so its easy to restore later. It could be as much as 10 extra backups per day. Should something else be used?

13

Hi!

You can find the different backup policies at:

There are no extra costs, but there may be a limit on number of adhoc backups. All backups will follow the the retention as per the backup policy.

If you want to make sure the ingestion is working as expected, you can, if possible, define deterministic ids and use the batch ingestion. This way, if you run the batch import again, and pass a predefined uuid, Weaviate will update or insert that object.

Additionally, you can also find here a recipe for ingestion with a retry logic:

Let me know if that helps!

THanks!

1 Like