Can weaviate run in ECS?

We’re running weaviate in k8s on AWS using EKS (AWS managed kubernetes). I appreciate that the weaviate helm chart has lots of configuration for EKS, such as s3 backups and what not.

For various reasons, we are also interested in running on AWS Elastic Container Service (ECS). I’d like to understand what are the pros and cons of doing so. For example, without the helm chart, can weaviate easily be deployed in a horizontally-scaled configuration in ECS? Is it just a matter of following the docker-compose guides, or do you not recommend running weaviate in container runtimes other than kubernetes?

Thanks

Hi @Adam_Hughes !

I believe it will just be a matter of following the docker guides, as described here:

Each new node should then point to a "founder"node.

Let me know if this helps!

Thanks!

This is helpful thanks.

Do you know if this field

PERSISTENCE_DATA_PATH: '/var/lib/weaviate'

Can point to S3 or it has to be a disk in the cluster?

Hi!

Most certainly not due to latency on read and write operations and other issues that will arise, because S3 is not designed to this.

Considering AWS, you should use EBS or EFS.

Let me know if this helps :slight_smile:

Er sorry I am mixing apples and oranges. I meant for database backups.

Your weaviate helm chart has turnkey support for backing up DB to S3 in a bucket called weaviate-bucket. Does the docker-compose version offer a similar way to backup the DB?

Sure thing.

You can add backup-s3 module to ENABLED_MODULES environment variable and setup the bucket configurations as specified here:

With all set you can start a backup routine as described here:

Note that you can have multiple backup modules enabled and configured, and when creating the backup, you can specify which module you want to use to backup or restore from.

Let me know if that helps :slight_smile:

Thanks!

1 Like

That’s great thank you. Sorry, I should have understood. Thanks for answering all of my questions!

1 Like