How to run containerized Weaviate as Google Cloud Run

Support
1

Hey everyone,

we are planning to use Weaviate as our vector db for our internal RAG system. However, as we are building our infrastructure in the Google Cloud, we are struggling with deploying Weaviate as a Cloud Run Service. The problem is that when deploying it as Cloud Run only one port can be exposed per application. All attempts to make it run failed: We have tested to deploy two containers one for REST and one for GRPC while sharing the same volume and then access them via the http_endpoint and grpc_endpoint. This however didn’t work due to concurrency problems. Is there any way to make these things work without adding the complexity of a proxy service? Are we missing out some obvious solutions?

We are aware that deploying Weaviate inside Kubernetes (GKE) would be an option, but so far we want to stick to Cloud Run. We are also aware of the existing Weaviate Cloud version, but we want to manage the database ourselves.

Thanks in advance!

2

I managed to get a first workaround by adding a Envoy proxy. However, I now have the same host and port for HTTP and GRPC and the pydantic validation of the connect_to_custom fails because of that. Is there any way to disable this validation? Because apart from that it seems to work.

3

Hi @cluel01 !!

Welcome to our community :hugs:

I never seen this being tried, to be honest. So not sure exactly how it plays out.

But before the client pydantic issue, can you make sure the grpc port is serving?

You can test that with grpcurl:

# lets test our grpc connection
❯ wget https://raw.githubusercontent.com/grpc/grpc/master/src/proto/grpc/health/v1/health.proto
❯ grpcurl -d '{"service": "Weaviate"}' -proto health.proto grpc.weaviate.mydomain.com:50051 grpc.health.v1.Health/Check
{
  "status": "SERVING"
}

Let me know if you can share how you did it as I would love to try that too!

Thanks!