Working in multi thread environment with connection pool

General
1

HI,

We are new to weaviate, we are implementing a multi threaded service and we want to integrate weaviate client in it.
I tried to find information regarding connection pooling and connection management in the documentation but didnt find clear summary of this topic.

can you please answer the following questions:

  1. what is the recommended approach when working with weaviate, should I open and close connection after each request?
  2. is there a build in connection pooling mechanism in the framework?
  3. does weaviate client is thread safe, can I use the same client (and connection pool, if exists) across threads?

Thanks,
Eilon

2

Good morning @eilon_shai,

It’s lovely to have you here and welcome to your community! :partying_face:

That’s great questions so I will use the Weaviate Python client — similarly applies to JS/TS.

We advise you working with singleton pattern. When using a singleton client, requests use connections from a pool, managed by the underlying gRPC or HTTP libraries. Each request will borrow a connection, complete the operation, and return it to the pool.

  • Closing the client object (via client.close()) invalidates the entire connection pool, affecting all active requests. So you should avoid closing the client after each request when using the singleton pattern.
  • Creating a new client for each request is inefficient since each request will have its own connection pool, leading to resource overhead and slower performance.

In Weaviate’s context, using the singleton pattern for the client ensures that multiple requests can efficiently share the same underlying connection pool, preventing unnecessary overhead and improving the overall performance of the application. It’s recommended to create a single client object and share it throughout your application using the singleton pattern.

The Python client does concurrent requests in the background, which can significantly speed things up if your server is beefy enough. With the Go client, you would need to build the concurrency yourself.

In summary, for optimal performance with multiple concurrent processes, it’s best to use the singleton pattern, managing client connections based on the application’s lifecycle rather than on a request-by-request basis.

Here are some examples from my own scripts/snippets in my Github of Singleton pattern which you can use as initial or reference:

Best regards,

Mohamed Shahin
Weaviate Support Engineer
(Ireland, GMT/UTC timezone)

3

Thank you Mohamed, I will go over the solution
very appreciate the detailed answer

1 Like
4

Hey,

couple additional points:

  • does weaviate client is thread safe, can I use the same client (and connection pool, if exists) across threads?

Yes, but NOT for batching (eg client.batch or collection.batch) - it also would not speed up things.

We also offer an async api that we’d recommend over multi-threading: Async API | Weaviate

connection pool

You can set (some of) the configs of the underlying httpx client using the following:

with weaviate.connect_to_local(
  ... , 
  additional_config=wvc.init.AdditionalConfig(connection=wvc.init.ConnectionConfig())) as client:
5

Thank you Dirk, can you please provide a complete example of usage with the async API

6

There are a couple of examples here: weaviate-python-client/integration/test_collection_async.py at main · weaviate/weaviate-python-client · GitHub

7

:+1: Thank you, will follow