# Unable to connect to weaviate running behind reverse proxy

**URL:** https://forum.weaviate.io/t/unable-to-connect-to-weaviate-running-behind-reverse-proxy/3252
**Category:** Support
**Tags:** bug
**Created:** [August 1, 2024, 4:31pm UTC](https://forum.weaviate.io/t/unable-to-connect-to-weaviate-running-behind-reverse-proxy/3252 "2024-08-01T16:31:13Z")
**Posts on this page:** 15
**Page:** 1

<div class="post-metadata">

### Author: ![gdrajhasekarun](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/gdrajhasekarun/32/2079_2.png) [@gdrajhasekarun](https://forum.weaviate.io/u/gdrajhasekarun)
#### Post date: [August 1, 2024, 4:31pm UTC](https://forum.weaviate.io/t/unable-to-connect-to-weaviate-running-behind-reverse-proxy/3252/1 "2024-08-01T16:31:13Z")

</div>

### Description

I have weaviate installed on my local docker and proxied 8080 to an url and 50051 to another one. using the following code, I’m connecting to them

```auto
client = weaviate.connect_to_custom(
            http_host='host-a',
            grpc_host='host-b',
            http_port=443,
            http_secure=True,
            grpc_port=443,
            grpc_secure=True,
            skip_init_checks=True,
            additional_config=AdditionalConfig(
                connection=ConnectionConfig(
                                session_pool_connections=30,
                                session_pool_maxsize=200,
                                session_pool_max_retries=3,
                            ),
                    # timeout=(60, 180),
                timeout=Timeout(init=60, query=60, insert=180) # Values in seconds
            )
        )

```

getting the following errors when i do self.vectorstore.add\_documents(  
documents=document,  
)

2024-Aug-01 11:19 AM - langchain\_weaviate.vectorstores - ERROR - Failed to add object: None  
Reason: WeaviateBatchError(‘Query call with protocol GRPC batch failed with message \<AioRpcError of RPC that terminated with:\n\tstatus = StatusCode.PERMISSION\_DENIED\n\tdetails = “Received http2 header with status: 403”\n\tdebug\_error\_string = “UNKNOWN:Error received from peer {created\_time:“2024-08-01T11:19:42.004222-05:00”, grpc\_status:7, grpc\_message:“Received http2 header with status: 403”}”\n\>.’)

---

<div class="post-metadata">

### Author: ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)
#### Post date: [August 2, 2024, 1:28pm UTC](https://forum.weaviate.io/t/unable-to-connect-to-weaviate-running-behind-reverse-proxy/3252/2 "2024-08-02T13:28:20Z")

</div>

hi @gdrajhasekarun !!

Welcome to our community 🤗

This indicates an error on exposing your Weaviate.

Can you share more info on how you have exposed?

Also, check this forum thread on this topic:

> [@Weaviate with Traefik and gRPC](https://forum.weaviate.io/t/weaviate-with-traefik-and-grpc/1238/3):
>
> Hi @qnlbnsl ! Sorry for the delay here. Looks like I was finally able to tame this Here is what I got: NOTE: [Check this updated gist on how to correctly expose Weaviate under SSL/TLS using Traefik and running everything with a docker compose](https://gist.github.com/dudanogueira/0dd925cbf63397e256e84e991245aab3/edit) --- version: '3.4' services: weaviate: command: - --host - 0.0.0.0 - --port - '8080' - --scheme - http image: semitechnologies/weaviate:1.23.5 #ports: # - 8081:8080 # unsafe http # - 50052:50051 # unsafe grpc…

---

<div class="post-metadata">

### Author: ![gdrajhasekarun](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/gdrajhasekarun/32/2079_2.png) [@gdrajhasekarun](https://forum.weaviate.io/u/gdrajhasekarun)
#### Post date: [August 2, 2024, 2:35pm UTC](https://forum.weaviate.io/t/unable-to-connect-to-weaviate-running-behind-reverse-proxy/3252/3 "2024-08-02T14:35:42Z")

</div>

This is my docker compose file.

```yaml
version: '3.4'
networks:
  frontend:
    external: true
  backend:
    external: true
    
services:
  weaviate:
    image: cr.weaviate.io/semitechnologies/weaviate:1.25.9
    # ports:
    # - 8080:8080
    # - 50051:50051
    volumes:
      - weaviate_data:/var/lib/weaviate
    restart: on-failure:0
    environment:

      QUERY_DEFAULTS_LIMIT: 25
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
      PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
      DEFAULT_VECTORIZER_MODULE: 'none'
      ENABLE_MODULES: 'text2vec-cohere,text2vec-huggingface,text2vec-palm,text2vec-openai,generative-openai,generative-cohere,generative-palm,ref2vec-centroid,reranker-cohere,qna-openai'
      CLUSTER_HOSTNAME: 'node1'
    networks:
      - frontend
      - backend
    labels:
      - 'traefik.enable=true'
      - 'traefik.docker.network=frontend'
      - 'traefik.http.routers.weavite.entrypoints=websecure'
      - 'traefik.http.routers.weavite.tls=true'
      - 'traefik.http.routers.weavite.tls.certresolver=production'
      - 'traefik.http.routers.weavite.service=weavite_1'
      - 'traefik.http.services.weavite_1.loadbalancer.server.port=8080'
      - 'traefik.http.routers.weavite.rule=Host(`weavite.<domain>`)'

      - 'traefik.http.routers.grc_weavite.entrypoints=websecure'
      - 'traefik.http.routers.grc_weavite.tls=true'
      - 'traefik.http.routers.grc_weavite.tls.certresolver=production'
      - 'traefik.http.routers.grc_weavite.service=grc_weavite_1'
      - 'traefik.http.services.grc_weavite_1.loadbalancer.server.port=50051'
      - 'traefik.http.routers.grc_weavite.rule=Host(`grc-weavite.<domain>`)'
volumes:
  weaviate_data:

```

getting the following error  
2024-Aug-01 11:19 AM - langchain\_weaviate.vectorstores - ERROR - Failed to add object: None  
Reason: WeaviateBatchError(‘Query call with protocol GRPC batch failed with message \<AioRpcError of RPC that terminated with:\n\tstatus = StatusCode.PERMISSION\_DENIED\n\tdetails = “Received http2 header with status: 403”\n\tdebug\_error\_string = “UNKNOWN:Error received from peer {created\_time:“2024-08-01T11:19:42.004222-05:00”, grpc\_status:7, grpc\_message:“Received http2 header with status: 403”}”\n\>.’)

---

<div class="post-metadata">

### Author: ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)
#### Post date: [August 2, 2024, 2:44pm UTC](https://forum.weaviate.io/t/unable-to-connect-to-weaviate-running-behind-reverse-proxy/3252/4 "2024-08-02T14:44:08Z")

</div>

Check this gist:

> **[Build software better, together](https://github.com/login/oauth/authorize?client_id=7e0a3cd836d3e544dbd9&redirect_uri=https%3A%2F%2Fgist.github.com%2Fauth%2Fgithub%2Fcallback%3Freturn_to%3Dhttps%253A%252F%252Fgist.github.com%252Fdudanogueira%252F0dd925cbf63397e256e84e991245aab3%252Fedit&response_type=code&state=a27ce22a2dd81843316694f3c1d6958bae08900e89cf73ef76de7fdc10882aff)**
>
> GitHub is where people build software. More than 100 million people use GitHub to discover, fork, and contribute to over 420 million projects.

Those are the traefik labels I was able to make correctly exposing:

```auto
    labels:
      - "traefik.enable=true"
      # http
      - "traefik.http.services.weaviate_http_service.loadbalancer.server.port=8080"
      - "traefik.http.routers.weaviate_http_router.rule=Host(`weaviate.yourdomain.com`)"
      - "traefik.http.routers.weaviate_http_router.entrypoints=websecure"
      - "traefik.http.routers.weaviate_http_router.service=weaviate_http_service"
      - "traefik.http.routers.weaviate_http_router.tls.certresolver=myresolver"
      # # grpc
      - "traefik.http.services.weaviate_grpc_service.loadbalancer.server.scheme=h2c"
      - "traefik.http.services.weaviate_grpc_service.loadbalancer.server.port=50051"
      - "traefik.http.routers.weaviate_grpc_router.rule=Host(`grpc.weaviate.yourdomain.com`)"
      - "traefik.http.routers.weaviate_grpc_router.entrypoints=grpc"
      - "traefik.http.routers.weaviate_grpc_router.service=weaviate_grpc_service"
      - "traefik.http.routers.weaviate_grpc_router.tls.certresolver=myresolver"

```

---

<div class="post-metadata">

### Author: ![gdrajhasekarun](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/gdrajhasekarun/32/2079_2.png) [@gdrajhasekarun](https://forum.weaviate.io/u/gdrajhasekarun)
#### Post date: [August 2, 2024, 7:14pm UTC](https://forum.weaviate.io/t/unable-to-connect-to-weaviate-running-behind-reverse-proxy/3252/5 "2024-08-02T19:14:33Z")

</div>

Thank you.

Still my issues is not resolved. Is it possible to get me the Traefik config for grpc.

---

<div class="post-metadata">

### Author: ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)
#### Post date: [August 2, 2024, 8:27pm UTC](https://forum.weaviate.io/t/unable-to-connect-to-weaviate-running-behind-reverse-proxy/3252/6 "2024-08-02T20:27:39Z")

</div>

Sure. It is on that thread I pasted above.

here the gist:

> **[Build software better, together](https://github.com/login/oauth/authorize?client_id=7e0a3cd836d3e544dbd9&redirect_uri=https%3A%2F%2Fgist.github.com%2Fauth%2Fgithub%2Fcallback%3Freturn_to%3Dhttps%253A%252F%252Fgist.github.com%252Fdudanogueira%252F0dd925cbf63397e256e84e991245aab3%252Fedit&response_type=code&state=a27ce22a2dd81843316694f3c1d6958bae08900e89cf73ef76de7fdc10882aff)**
>
> GitHub is where people build software. More than 100 million people use GitHub to discover, fork, and contribute to over 420 million projects.

on that same thread there are some ways to test the grpc endpoint.

Note that this should work with any grpc expose method in a reverse proxy.

Thanks!

---

<div class="post-metadata">

### Author: ![gdrajhasekarun](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/gdrajhasekarun/32/2079_2.png) [@gdrajhasekarun](https://forum.weaviate.io/u/gdrajhasekarun)
#### Post date: [August 11, 2024, 1:44pm UTC](https://forum.weaviate.io/t/unable-to-connect-to-weaviate-running-behind-reverse-proxy/3252/7 "2024-08-11T13:44:40Z")

</div>

I followed the the steps mentioned in the thread.

1. Opened GRPC in Cloudflare (No gateway config)
2. Followed the steps for Traefik and Weaviate.

I was not able to connect to Grpc using the command

grpcurl -d ‘{“service”: “Weaviate”}’ -proto health.proto [grpc.weaviate.mydomain.com:50051](http://grpc.weaviate.mydomain.com:50051) grpc.health.v1.Health/Check

---

<div class="post-metadata">

### Author: ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)
#### Post date: [August 12, 2024, 9:38pm UTC](https://forum.weaviate.io/t/unable-to-connect-to-weaviate-running-behind-reverse-proxy/3252/8 "2024-08-12T21:38:03Z")

</div>

Oh, that indicates your GRPC service may not be properly exposed.

What is the error message?

---

<div class="post-metadata">

### Author: ![gdrajhasekarun](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/gdrajhasekarun/32/2079_2.png) [@gdrajhasekarun](https://forum.weaviate.io/u/gdrajhasekarun)
#### Post date: [August 12, 2024, 9:59pm UTC](https://forum.weaviate.io/t/unable-to-connect-to-weaviate-running-behind-reverse-proxy/3252/9 "2024-08-12T21:59:16Z")

</div>

Failed to dial target host “:50051”: context deadline exceeded

---

<div class="post-metadata">

### Author: ![gdrajhasekarun](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/gdrajhasekarun/32/2079_2.png) [@gdrajhasekarun](https://forum.weaviate.io/u/gdrajhasekarun)
#### Post date: [August 12, 2024, 10:29pm UTC](https://forum.weaviate.io/t/unable-to-connect-to-weaviate-running-behind-reverse-proxy/3252/10 "2024-08-12T22:29:12Z")

</div>

Even I did the following. Exposed weaviate with 8080 and 50051 as part of docker instances. now localhost:8080 is responding me with the following response  
**Command:** curl [http://localhost:8080/v1/nodes](http://localhost:8080/v1/nodes)  
**Response:**  
{“nodes”:[{“batchStats”:{“queueLength”:0,“ratePerSecond”:0},“gitHash”:“6aeae65”,“name”:“node1”,“shards”:null,“stats”:{“objectCount”:0,“shardCount”:1},“status”:“HEALTHY”,“version”:“1.23.5”}]}

With GRPC, I did the following.  
**Command:** grpcurl -d ‘{“service”: “Weaviate”}’ -proto health.proto localhost:50051 grpc.health.v1.Health/Check  
**Response** Failed to dial target host “localhost:50051”: tls: first record does not look like a TLS handshake

---

<div class="post-metadata">

### Author: ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)
#### Post date: [August 12, 2024, 10:42pm UTC](https://forum.weaviate.io/t/unable-to-connect-to-weaviate-running-behind-reverse-proxy/3252/11 "2024-08-12T22:42:19Z")

</div>

Oh, wait. If you don’t have SSL, you need to pass the -plaintext, like so

```shell
grpcurl --plaintext -d '{"service": "Weaviate"}' -proto health.proto localhost:50051 grpc.health.v1.Health/Check

```

Thanks!

---

<div class="post-metadata">

### Author: ![gdrajhasekarun](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/gdrajhasekarun/32/2079_2.png) [@gdrajhasekarun](https://forum.weaviate.io/u/gdrajhasekarun)
#### Post date: [August 12, 2024, 10:52pm UTC](https://forum.weaviate.io/t/unable-to-connect-to-weaviate-running-behind-reverse-proxy/3252/12 "2024-08-12T22:52:37Z")

</div>

> [@DudaNogueira](#):
>
> `grpcurl --plaintext -d '{"service": "Weaviate"}' -proto health.proto localhost:50051 grpc.health.v1.Health/Check`

I was able to make it working in local host using the command you gave me. Let me try with the traefik and cloud flare level.

Cloudflare - Opened GRPC at domain level. I was using Zero trust tunnel.

---

<div class="post-metadata">

### Author: ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)
#### Post date: [August 12, 2024, 11:13pm UTC](https://forum.weaviate.io/t/unable-to-connect-to-weaviate-running-behind-reverse-proxy/3252/13 "2024-08-12T23:13:16Z")

</div>

Oh! Is it working now?

---

<div class="post-metadata">

### Author: ![gdrajhasekarun](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/gdrajhasekarun/32/2079_2.png) [@gdrajhasekarun](https://forum.weaviate.io/u/gdrajhasekarun)
#### Post date: [August 12, 2024, 11:14pm UTC](https://forum.weaviate.io/t/unable-to-connect-to-weaviate-running-behind-reverse-proxy/3252/14 "2024-08-12T23:14:18Z")

</div>

Yes, at localhost level. But not at traefik and Cloudflare level. I’m researching on that.

---

<div class="post-metadata">

### Author: ![gdrajhasekarun](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/gdrajhasekarun/32/2079_2.png) [@gdrajhasekarun](https://forum.weaviate.io/u/gdrajhasekarun)
#### Post date: [August 13, 2024, 3:13pm UTC](https://forum.weaviate.io/t/unable-to-connect-to-weaviate-running-behind-reverse-proxy/3252/15 "2024-08-13T15:13:17Z")

</div>

Cloudflare needs all GRPC services to expose as http2. So I’m routing the hostname “weaviate.grpc.” in Traefik with port as 443. Traefik connect using the url to the docker url “[http://172.19.0.14:50051](http://172.19.0.14:50051)”. Do you see any issues here.

I’m mostly a developer with little knowledge on networking.