Connection pool

Support
1

There are two parameters in the ConnectionConfig:
session_pool_connections
session_pool_maxsize

I have the following questions:

  1. session_pool_connections = 20, does it mean the application can have maximum 20 connections to the weaviate at the same time?
  2. What does it mean to set session_pool_maxsize = 100?
  3. If the application has 5 threads, and session_pool_connections = 20, what does it mean to each thread?
conn_config = ConnectionConfig(session_pool_connections=20,session_pool_maxsize=20)

config_ = Config(connection_config=conn_config)

client = weaviate.Client(ip,additional_config=config)

With the setup, how do we get a connection from the pool?

Any help is appreciated.

Thank you,
Robert

2

Hi!

Not sure where you got those informations.

You also seem to be using the python v3 client, that is now considered deprecated:

Do you have any hard requirement to use that version?

Thanks!

4

@DudaNogueira sorry for the confusion. I just copied the code from the other post in the forum.

We still have the same thing in v4 but it is in weaviate.config.AdditionalConfig, right?

I am trying to understand how to use connection pool in weaviate.

5

@DudaNogueira I have an other question.

If there are 5 threads in the application, and each thread uses weaviate.connect_to_custom to create a weaviate instance.

Do they share the same connection? Or each one is an individual connection to the weaviate?

6

Hi!

AFAIK, they will open 5 different GRPC connections.

If you are dealing with ingestion, you can also increase the number of concurrent_requests in batch.

Let me know if this helps!

thanks!

7

@DudaNogueira I want to know the meaning of these two parameters:
session_pool_connections
session_pool_maxsize

8

Those parameters are passed here to httpx:

And in httpx page we have this:

  • max_keepalive_connections, number of allowable keep-alive connections, or None to always allow.
  • max_connections, maximum number of allowable connections, or None for no limits.

Let me know if this helps!

Thanks!