How to run Weaviate using a binary?

Resources
1

This post describes how to run Weaviate using a binary file:

  1. Download Weaviate binary. Go to releases page to download a suitable Weaviate binary.

  2. Create a weaviate.conf.json file:

{
  "authentication": {
    "anonymous_access": {
      "enabled": true
    }
  },
  "authorization": {
    "admin_list": {
      "enabled": false
    }
  },
  "query_defaults": {
    "limit": 100
  },
  "persistence": {
    "dataPath": "./data"
  }
}
  1. Run a binary:
./weaviate --host 0.0.0.0 --port 8080 --scheme http

Alternatively one can provide a configuration using yaml format.

Create a weaviate.conf.yaml file with this contents:

authentication:
  anonymous_access:
    enabled: true
authorization:
  admin_list:
    enabled: false
query_defaults:
  limit: 100
persistence:
  dataPath: ./data

Run binary:

./weaviate --config-file=weaviate.conf.yaml --host 0.0.0.0 --port 8080 --scheme http
3 Likes
2

Hi and thank you very much for the information!

Two questions:

  1. Is there any plan to document this in the official documentation?
  2. Is there any info on the schema of the configuration file?

Thanks!

1 Like
3

Thank you @antas-marcin for documenting this! Surprised it’s not one of the install options at How to install Weaviate | Weaviate - Vector Database, since it’s the easiest way to boot up the Weaviate server on *nix without any Docker overhead.

What is the full format of the weaviate.conf.json file? How would one enable a module for instance? Is there any official documentation?

UPDATE: Nevermind, running the binary outputs on the first line

{“action”:“config_load”,“config_file_path”:“./weaviate.conf.json”,“level”:“info”,“msg”:“Usage of the weaviate.conf.json file is deprecated and will be removed in the future. Please use environment variables.”

1 Like
4

Hello Dan!

What is the full format of the weaviate.conf.json file ?

Yes, this is the exact file format, you can use those settings to further tune Weaviate, but the preferred way is to pass them as environment variables.

How would one enable a module for instance? Is there any official documentation?

There are two ways that you can do this, either add an enable_modules entry in the config file with coma separated names of the modules that you want to enable, or you can pass this information as an environment variable (preferred way) during runtime, example:

ENABLE_MODULES="text2vec-openai,text2vec-aws" ./weaviate --config-file=weaviate.conf.yaml --host 0.0.0.0 --port 8080 --scheme http

This way you can pass also other environment variables which will be picked up by Weaviate during runtime.

5

In the .go code I see a flat list of CamelCase settings, but it’s unclear how they map to the JSON file (or to the YAML file) format. What is the hierarchy?

For example, the DisableTelemetry setting:

If I create weaviate.config.json with the contents {"DisableTelemetry": true} and run ./weaviate-1.25.2 --config-file=weaviate.conf.json, Weaviate still attempts (and fails to) send telemetry, then exits:

{"action":"startup","level":"error","msg":"telemetry failed to start: push: failed to send request: Post \"https://telemetry.weaviate.io/weaviate-telemetry\": dial tcp: lookup telemetry.weaviate.io on [::1]:53: read udp [::1]:49687-\u003e[::1]:53: read: connection refused","time":"2024-06-05T19:29:45-04:00"}
{"address":"10.0.0.68:8300","level":"info","msg":"current Leader","time":"2024-06-05T19:29:45-04:00"}
{"level":"info","msg":"starting migration from old schema","time":"2024-06-05T19:29:45-04:00"}
{"level":"info","msg":"legacy schema is empty, nothing to migrate","time":"2024-06-05T19:29:45-04:00"}
{"level":"info","msg":"migration from the old schema has been successfully completed","time":"2024-06-05T19:29:45-04:00"}
{"action":"restapi_management","level":"info","msg":"the required flags `--tls-certificate` and `--tls-key` were not specified","time":"2024-06-05T19:29:45-04:00"}

From your example in the first post, I tried snake_casing, but {"disable_telemetry": true} in the JSON resulted in the same behavior of the setting being ignored.

I also created w.yaml with the contents

disable_telemetry: true
DISABLE_TELEMETRY: true

When I run ./weaviate-1.25.2 --config-file=w.yaml, Weaviate still attempts (and fails) to send telemetry.

  1. I think clear documentation of the Weaviate binary config file is critical. I find it awkward to pass many environment variables in a standalone deployment, and the vast majority of (database) sever software uses config files. But it’s not clear in the docs what the format of this file is. (I can’t find any information about the config file for the Weaviate binary among the “installation instructions” or by searching the developer docs).

  2. Weaviate should report errors on unrecognized options in its config file. Obviously neither of the disable_telemetry and DISABLE_TELEMETRY options were recognized.

  3. Passing a nonexistent config file should result in a clear error, but apparently Weaviate just ignores the error.

  4. It seems the JSON format is being deprecated (see my post above)? But the CLI help suggests the JSON is still the default config file:

    image
    image883Ă—96 12 KB

    Please officially document the YAML config file format.

6

It’s advised to pass all other settings using environment variables. so if you want to pass DISABLE_TELEMETRY setting to your binary, then the best way to do this is to pass this setting as an environment variable, example:

  1. create start.sh file
  2. fill in all the desired settings and save file, example:
DISABLE_TELEMETRY="true" \
ENABLE_MODULES="text2vec-openai,text2vec-aws" \
./weaviate --host 0.0.0.0 --port 8080 --scheme http
  1. start Weaviate using this script
sh start.sh