Migration from self-hosted to Weaviate Cloud using backup/restore

Support
, , ,
1

Description

I’m currently running a self-hosted Weaviate instance (v1.22.5 on Kubernetes) and am planning to migrate to Weaviate Cloud.

From the documentation, it seems like there are two ways to move data:

  1. Manual migration
  2. Backup/restore using modules like backup-s3

I am trying to use the second option — backup/restore — for a smoother transition. However, I’m unclear if this is possible on Weaviate Cloud, especially with the serverless tier. From the docs, it’s mentioned that Weaviate Cloud comes with pre-configured modules, and I’m not sure if backup-s3 is available for me to restore data.

Is it possible to migrate data from a self-hosted instance to Weaviate Cloud using backup/restore? Is backup-s3 supported in the serverless tier, or in any other tier of Weaviate Cloud?

This has been a blocker for our migration, and I couldn’t find specific guidance in the documentation or community forums. Any help would be greatly appreciated!

Server Setup Information

  • Weaviate Server Version: 1.22.5
  • Deployment Method: k8s
  • Client Language and Version: Python, weaviate-client 3.25.3
  • Multitenancy: Yes
2

Hi @akhilsharma !!

Welcome to our community :hugs:

We do not provide restoring from your own backup for serverless tiers :frowning:

Also, the best approach here is migrating your data (read from source and write on target) instead of restoring from backup, due to the difference in versions.

Or, you will need to upgrade to 1.22.latest → 1.23.latest → … all the way to our latest 1.30.1

So indeed, the best approach here is to do a migration as documented at:

Let me know if you need help on that!

Thanks!

3

Hi @DudaNogueira,

Thanks so much for the warm welcome and the quick response!

Just to confirm, would it be possible to use the backup/restore method (specifically with backup-s3) if we were to go with the enterprise tier instead?

Thanks again :slight_smile:

4

Hi!

Yes, you can BYOC (bring your own cloud).

Also, at enterprise you will have access to our team that will help on any blockers we may face.

Let me know if that helps!

Thanks!

5

Thanks @DudaNogueira — that definitely helps! Appreciate the support! :raising_hands:

1 Like
6

@akhilsharma Just to add to what my colleague @Duda Nogueira (Weaviate) mentioned — I’ve put together migration scripts that can help you move all your collections and data from one endpoint to another, including local clusters.

You can find it here as notebook so you can take it off into a script file to run on the terminal:

The script has two main steps in the first part, which are exactly what you need. Just add in your endpoints, ensure the connection method works (One local/ Another is cloud), and run the script.

You will know what to do.

Best regards,

Mohamed Shahin
Weaviate Support Engineer
(Ireland, GMT/UTC timezone)