Connection_params for docker configuration

Support
1

I had weaviate in the same docker-compose.yml file as my flask files and I was able to connect to it via this:

connection_params=weaviate.connect.ConnectionParams.from_params(
            http_host="weaviate",
            http_port="8060",
            http_secure=False,
            grpc_host="weaviate",
            grpc_port="50051",
            grpc_secure=False,
        ),

I just moved the weaviate instance to its own docker but still on the same machine. Now I can’t connect to it. I have tried ‘localhost’, ‘172.29.0.2’, ‘127.0.0.1’, ‘0.0.0.0’, ‘host.docker.internal’, and ‘myserver.com’

What am I missing? Do I need to set up a reverse proxy now in nginx?

  • Weaviate Server Version: 1.23.9
  • Deployment Method: Docker
  • Multi Node? 1
  • Client Language and Version: Python 4.4.4
2

Hi! Welcome to our community :hugs:

Can you share your docker compose?

Unless you have changed the default port in the command of you Weaviate service, you should use 8080 for the connection port.

Let me know if that helps.

3

It’s not the port that is the problem, it’s the host, what host do I use to connect from one docker to the other on the same machine?

weaviate:
command:
- --host
- 0.0.0.0
- --port
- ‘8060’
- --scheme
- http
image: semitechnologies/weaviate:1.23.9
ports:
- 8060:8060
- 50051:50051
volumes:
- weaviate_data:/var/lib/weav
restart: on-failure:0
environment:
QUERY_DEFAULTS_LIMIT: 25
AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: ‘true’
PERSISTENCE_DATA_PATH: ‘/var/lib/weav’
DEFAULT_VECTORIZER_MODULE: ‘none’
ASYNC_INDEXING: ‘true’
CLUSTER_HOSTNAME: ‘node1’

volumes:
weaviate_data

4

I already use like this and worked

weaviate:
    container_name: con-project-weaviate
    command:
    - --host
    - 0.0.0.0
    - --port
    - '8080'
    - --scheme
    - http
    image: semitechnologies/weaviate:1.23.10
    ports:
    - 8080:8080
    - 50051:50051
    volumes:
    - weaviate_data:/var/lib/weaviate
    restart: on-failure:0
    env_file:
      - ./python-api/.env
    environment:
      #OPENAI_APIKEY: $OPENAI_API_KEY
      QUERY_DEFAULTS_LIMIT: 25
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
      PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
      DEFAULT_VECTORIZER_MODULE: 'text2vec-openai'
      ENABLE_MODULES: 'text2vec-openai'
      CLUSTER_HOSTNAME: 'node1'
volumes:
  weaviate_data:
5

What did you use for host name in the following?

connection_params=weaviate.connect.ConnectionParams.from_params(
http_host=“weaviate”,
http_port=“8060”,
http_secure=False,
grpc_host=“weaviate”,
grpc_port=“50051”,
grpc_secure=False,
),

6

Hi @Dan_Gordon!

If your client is running in the same network where Weaviate is within docker, and the service name is called weaviate, this should be what you have to use.

Otherwise, if your client is running outside of docker, you should use the IP or the hostname where the ports are binded.

Let me know if this helps.

Thanks!

7

You must use network gateway ip from there,
@Dan_Gordon
-docker network ls
find your network for that project, then
-docker inspect network (networkaddress)
when the data network shown at the top of u can find network gateway IP
when you use that ip form the weaviatehost its works fine
it seems like that (http://172.12.0.1:8080)

8

Hi, I too am trying to connect to a Weaviate docker instance.

connection_params = weaviate.connect.ConnectionParams(
        scheme="http",
        host="localhost",
        port=8080,
        grpc_host="weaviate",
        grpc_port="50051",
        grpc_secure=False,
    )
client = weaviate.connect.Connection(connection_params)

but I get this error:

    connection_params = weaviate.connect.ConnectionParams(
  File "/Users/anand/miniconda3/envs/cerebro/lib/python3.9/site-packages/pydantic/main.py", line 164, in __init__
    __pydantic_self__.__pydantic_validator__.validate_python(data, self_instance=__pydantic_self__)
pydantic_core._pydantic_core.ValidationError: 2 validation errors for ConnectionParams
http
  Field required [type=missing, input_value={'scheme': 'http', 'host'...', 'grpc_secure': False}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.5/v/missing
grpc
  Field required [type=missing, input_value={'scheme': 'http', 'host'...', 'grpc_secure': False}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.5/v/missing
9

hi @divisionby0 !

Can you try the syntax as described here?

For instance:

import weaviate
from weaviate.connect import ConnectionParams
from weaviate.classes.init import AdditionalConfig, Timeout
import os

client = weaviate.WeaviateClient(
    connection_params=ConnectionParams.from_params(
        http_host="localhost",
        http_port="8080",
        http_secure=False,
        grpc_host="localhost",
        grpc_port="50051",
        grpc_secure=False,
    ),
    auth_client_secret=weaviate.auth.AuthApiKey("secr3tk3y"),
    additional_headers={
        "X-OpenAI-Api-Key": os.getenv("OPENAI_APIKEY")
    },
    additional_config=AdditionalConfig(
        timeout=Timeout(init=2, query=45, insert=120),  # Values in seconds
    ),
)

client.connect()  # When directly instantiating, you need to connect manually

Let me know if this works.

Thanks!

10

Short answer - YES, the explicit connection worked…

I tried many things to get inter-container Docker networking to function. I had even found the same web page as above, but stopped at v4-client-helper-functions just above the “explicit” connection section you highlighted. The weaviate.connect_to_custom() looks almost identical to the explicit function, but of course it fails…???

Is there an explanation as to why we need to use an explicit connection in the case of Docker? And why connect_to_custom() will not work?

I’m glad it’s working, but this was hard to figure out…at least for me…

11

Hi @Gene_Mc !

This is because it will all depend on how you have deployed Weaviate in docker.

For example, it can be exposed on a different port, or a different hostname. It can be secured or insecured.

So you need to “pair” your connection parameters with how you have deployed Weaviate.

On the case of connecting a client that is running on a container, to Weaviate that is also running on a container, usually setting the host to weaviate (or whatever service name you set in your docker compose), http_port to 8080, grpc_port to 50051, all unsercure (grpc_secure=False, http_secure=False) should work.

Of course, considering that both services are in the same docker compose or at the same network and has connectivity between them.

Let me know if this works.

If you have any issues, let us know :slight_smile:

12

I think my question made no sense before…

Option1, this did not work when I tried it initially:

client = weaviate.connect_to_custom(
    http_host="weaviate",
    http_port="8080",
    http_secure=False,
    grpc_host="weaviate",
    grpc_port="50051",
    grpc_secure=False,
)

Option 2, but this did work:

client = weaviate.WeaviateClient(
    connection_params=ConnectionParams.from_params(
        http_host="weaviate",
        http_port="8080",
        http_secure=False,
        grpc_host="weaviate",
        grpc_port="50051",
        grpc_secure=False,
    ),
)

client.connect()

Of course had I tried a bunch of things in between, and when I went back and tried the first one, it worked too!

So let me ask a different question… under which circumstances would I use option 2 instead of option 1?

13

Hi!

They are basically the same code. connect_to_custom will instantiate a WeaviateClient, connect and return the (hopefully) connected client.

Check here:

I have tried both of those code you used from the host, using a clean docker compose, and replacing weaviate by localhost and they worked.

So if both your containers has connectivity (same network), it should also work :thinking: