Adding backup module – don't want to lose my data

Support
1

This is a known issue that I have seen discussed 3 years ago relating to docker and a lack of knowledge about Docker. I am currently trying to solve this issue. I am trying to see if I add the environmental variable BACKUP_FILESYSTEM_PATH which is listed in the documentation backups will it allow me to run the commands to create a backup or do I have to restart the container. I am too scared as of now to stop and rerun the container even though the linked website shows it’s an issue with volumes defined as part of docker-compose and not defined externally. I am afraid of losing my data if I stop the container but I know it will definitively delete your volume if you use docker-compose down or something docker-compose related to stop the container. I would just copy over the volume by using docker volume ls to find the volume name you have, and then use docker volume inspect <VOLUME_NAME> to find where it is.

2

Hi David, in order to add the backup module, you need to update your docker-compose.yml and restart it.

To stay on the safe side:

Does your Docker image have a volume attached?

If yes

If yes, then it is safe to restart it anytime.
Then you can update your docker-compose.yml file to include the backup module, and restart it.

If not

Otherwise, you need to copy the data from the container to your local host.

After the data is copied from the container to the local host, you can make any changes that are needed and restart the docker-compose. You will also have to mount the copied files to the Weaviate container.

This can be done as described here.
For example if local path of the copied data is ./copied-data-from-weaviate/ and PERSISTENCE_DATA_PATH=/var/lib/weaviate, it should look like this:

volumes:
  - ./copied-data-from-weaviate:/var/lib/weaviate

How to create a manual backup / local copy:

To copy the data on a running Docker instance, you can call:

docker cp <container_id>:insert/path/to/data/ /local/path/to/data

Where insert/path/to/data/ is the value of the PERSISTENCE_DATA_PATH ENV Var

Example

Here is an example that I used on my instance:

docker cp 052f9edf2b8d8e22d24149b9d39326e41f6fcf793f5f9f4e5c8bb731f7fd5ace:/var/lib/weaviate ~/docker-backup/
  1. 052... – is the ID of my weaviate-1 docker container. Which I found in the Docker Desktop app.
  • FYI, if you are in the same folder as your docker-compose.yml, you can call docker compose ps, it will return only the containers from your current compose stack. Then pick the one with weaviate in the name.
  1. /var/lib/weaviate – is the default location for storing the data.
  2. ~/docker-backup/ – is the folder where I wanted the backup to be copied to.

After running docker cp ... you should check the destination folder. There should be multiple files, their names should reflect the names of your collections.

Let me know if that helps.

2 Likes