Issue with Weaviate Container on Start-up

Support
, ,
1

We deployed Weaviate in a Docker container on ECS with 4 vCPUs and 8 GB of RAM and are using EFS for storage.
We’ve noticed inconsistent behavior during startup, where Weaviate shows random problems. When we tried to launch 4 instances, only 1 started successfully, and the others failed to deploy because the service didn’t start at all.
Even after logging into the other 3 containers, we are unable to get a 200 response from localhost:8080/v1, while it works for only one container at a time.

The Weaviate logs don’t provide any useful information regarding the delay, and we’re stuck because of this issue.

Is there any way to troubleshoot this?

Server Setup Information

  • Weaviate Server Version: 1.26.4
  • Deployment Method: Docker
  • Multi Node? Number of Running Nodes: Single
  • Multitenancy?: No

@DudaNogueira @Mohamed_Shahin @Danny_Williams @Dirk

2

hi @saurbhhsharrma !!

Do you have any logs?

Are those containers running as single node or as a cluster?

3

It is being used on a Single node.

Below are the logs:

image
image1060×437 16.3 KB

4

can you copy and paste the entire logs? from start to quit?

This looks only a partial log :thinking:

6

Hello @saurbhhsharrma,

I believe, the main problem here is that you are using Docker containers to deploy Weaviate instead of the weaviate-helm charts, which is the standard deployment process for Kubernetes. When starting as a docker container, unless configured in the environment variables, each docker instance won’t be aware of the rest of the docker instances. That’s why it is recommended to deploy using weaviate-helm, because it takes care of configuring the environment variables properly, so that a cluster gets succesfuly created.
If you are still interested into using Docker compose for the deployment, here is an example of a docker compose template for deploying multinode clusters.

I hope this helps!

Regards,
José Luis

1 Like