# Unable to Backup Weaviate Using \`backup-filesystem\` | Failed Connection

**URL:** <https://forum.weaviate.io/t/unable-to-backup-weaviate-using-backup-filesystem-failed-connection/1037>\
**Category:** Support\
**Created:** [November 29, 2023, 6:51am UTC](https://forum.weaviate.io/t/unable-to-backup-weaviate-using-backup-filesystem-failed-connection/1037 "2023-11-29T06:51:05Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![jimkelly](https://avatars.discourse-cdn.com/v4/letter/j/85e7bf/32.png) [@jimkelly](https://forum.weaviate.io/u/jimkelly)\
**Post date:** [November 29, 2023, 6:51am UTC](https://forum.weaviate.io/t/unable-to-backup-weaviate-using-backup-filesystem-failed-connection/1037/1 "2023-11-29T06:51:05Z")

</div>

Hello!

I’m having some trouble in attempting to create a back-up in Weaviate. I get the error:

```shell
UnexpectedStatusCodeException: Backup creation! Unexpected status code: 422, with response body: {'error': [{'message': 'node {"node1" "192.168.112.2:7947"}: unexpected status code 503 (Failed Connection)'}]}.

```

Whenever I try via the following Python code:

```python
import weaviate

client = weaviate.Client(weaviate_client_addr)

result = client.backup.create(
  backup_id='test-backup-1',
  backend='filesystem',
  include_classes=['Records'],
  wait_for_completion=True,
)
print(result)

```

For context I’m running it using Docker with the following `docker-compose.yml`:

```yaml
version: "3.4"

services:
  weaviate:
    image: semitechnologies/weaviate:1.22.5
    ports:
      - 80:8080
    restart: on-failure:0
    environment:
      QUERY_DEFAULTS_LIMIT: 25
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: "true"
      PERSISTENCE_DATA_PATH: "/var/lib/weaviate"
      CLUSTER_HOSTNAME: "node1"
      DEFAULT_VECTORIZER_MODULE: "text2vec-openai"
      ENABLE_MODULES: "backup-filesystem,text2vec-openai,generative-openai"
      TRANSFORMERS_INFERENCE_API: http://t2v-transformers:8080
      NO_PROXY: t2v-transformers,weaviate,localhost,127.0.0.1,node1
      BACKUP_FILESYSTEM_PATH: /opt/weaviate/backup
    volumes:
      - /opt/weaviate/data:/var/lib/weaviate

```

I can see that the `backup-filesystem` module is enabled as when querying the client for the `meta_info` with the following Python script:

```python
import weaviate

client = weaviate.Client(weaviate_client_addr)
meta_info = client.get_meta()
print(meta_info)

```

I receive the following:

```shell
{'hostname': 'http://[::]:8080', 'modules': {'backup-filesystem': {'backupsPath': '/opt/weaviate/backup'}, 'generative-openai': {'documentationHref': 'https://platform.openai.com/docs/api-reference/completions', 'name': 'Generative Search - OpenAI'}, 'text2vec-openai': {'documentationHref': 'https://platform.openai.com/docs/guides/embeddings/what-are-embeddings', 'name': 'OpenAI Module'}}, 'version': '1.22.5'}

```

Additionally, from checking the status of my nodes with the following Python block:

```python
import weaviate

client = weaviate.Client(weaviate_client_addr)
nodes_status = client.cluster.get_nodes_status()
print(nodes_status)

```

I can see that the status is live and running:

```shell
[{'batchStats': {'queueLength': 0, 'ratePerSecond': 0}, 'gitHash': 'ed066e7', 'name': 'node1', 'shards': [{'class': 'Records', 'name': 'TrqAXeHtXtFA', 'objectCount': 1299618, 'vectorIndexingStatus': 'READY', 'vectorQueueLength': 0}], 'stats': {'objectCount': 1592164, 'shardCount': 3}, 'status': 'HEALTHY', 'version': '1.22.5'}]

```

Any advice or clarification in how to resolve this issue would be very much so appreciated. Please let me know if you require any additional details.

---

<div class="post-metadata">

**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\
**Post date:** [November 29, 2023, 2:29pm UTC](https://forum.weaviate.io/t/unable-to-backup-weaviate-using-backup-filesystem-failed-connection/1037/2 "2023-11-29T14:29:29Z")

</div>

Hi! Welcome to our community 🤗

Do you see anything in Weaviate log?

This seems like the client never even connected to Weaviate 🤔

---

<div class="post-metadata">

**Author:** ![kapaski](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/kapaski/32/708_2.png) [@kapaski](https://forum.weaviate.io/u/kapaski)\
**Post date:** [November 30, 2023, 12:03am UTC](https://forum.weaviate.io/t/unable-to-backup-weaviate-using-backup-filesystem-failed-connection/1037/3 "2023-11-30T00:03:58Z")

</div>

Some more info to add, is that our network is behind a corporate proxy, and we have the CLUSTER\_NAME set to “node1”

so far we have managed to set the weaviate container a fixed IP, as shown (192.168.16.2) and added extra hosts entry like “node1: 192.168.16.2” to container’s /etc/hosts file.

however, given curl node1:7947 from inside the container works, callling the backup endpoint still returns error

{“error”:[{“message”:“node {"node1" "192.168.16.2:7947"}: unexpected status code 503 (Failed Connection)”}]}

While it happens, we only captured two lines of log at ‘trace’ level

```auto
{"action":"restapi_request","level":"debug","method":"POST","msg":"received HTTP request","time":"2023-11-29T22:10:35Z","url":{"Scheme":"","Opaque":"","User":null,"Host":"","Path":"/v1/backups/filesystem","RawPath":"","OmitHost":false,"ForceQuery":false,"RawQuery":"","Fragment":"","RawFragment":""}}
{"action":"try_backup","backend":"filesystem","backup_id":"my-very-first-backup","level":"error","msg":"node {\"node1\" \"192.168.16.2:7947\"}: unexpected status code 503 (Failed Connection)","time":"2023-11-29T22:10:47Z","took":12091909569}

```

In the running container, we did set a `no_proxy` env var as  
no\_proxy: 192.168.16.2

to avoid this request go through the corporate proxy, but we are not sure if the weaviate go app is actually taking that, given the curl command above does show it’s using no\_proxy

---

<div class="post-metadata">

**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\
**Post date:** [November 30, 2023, 11:54am UTC](https://forum.weaviate.io/t/unable-to-backup-weaviate-using-backup-filesystem-failed-connection/1037/4 "2023-11-30T11:54:28Z")

</div>

Hi!

Can you try starting the backup using curl?

> **[Backups | Weaviate - vector database](https://weaviate.io/developers/weaviate/configuration/backups#create-backup)**
>
> \- References Backups

something like:

```auto
curl \
    -X POST \
    -H "Content-Type: application/json" \
    -d '{
         "id": "my-very-first-backup",
         "include": ["Article", "Publication"]
        }' \
    http://localhost:8080/v1/backups/filesystem

```

Let me know if that works.

---

<div class="post-metadata">

**Author:** ![kapaski](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/kapaski/32/708_2.png) [@kapaski](https://forum.weaviate.io/u/kapaski)\
**Post date:** [November 30, 2023, 9:13pm UTC](https://forum.weaviate.io/t/unable-to-backup-weaviate-using-backup-filesystem-failed-connection/1037/5 "2023-11-30T21:13:29Z")

</div>

Sure!  
That curl command (using our existing class name though) gave exactly the same error as the previous post shown.

The curl was sent from insde a running weaviate container, where

CLUSTER\_NAME is node1  
no\_proxy env var has set for node1’s container IP and the hostname node1  
/etc/hosts has entry to resolve the hostname node1 to container IP

I feel this could be related to how the GO app makes the node status call before backup and it wasn’t using proxy

Plus we want to know if it can disable the cluster mode with a single container deployment, that way it knows there is only one node and skip the steps like node checking and assume itself is in a cluster

---

<div class="post-metadata">

**Author:** ![mail4dy](https://avatars.discourse-cdn.com/v4/letter/m/8797f3/32.png) [@mail4dy](https://forum.weaviate.io/u/mail4dy)\
**Post date:** [April 4, 2024, 10:13pm UTC](https://forum.weaviate.io/t/unable-to-backup-weaviate-using-backup-filesystem-failed-connection/1037/6 "2024-04-04T22:13:42Z")

</div>

Were you able to backup? I am having some problem. Server cannot connect.
