How to use a weaviate client within FastAPI

Support
1

Following the FastAPI docs, they suggest opening a client session to databases, at least postgres, as follows:

@contextmanager
def get_db():
    db = SessionLocal()
    try:
        yield db
    finally:
        db.close()

This pattern opens a short term connection to the DB and then closes it after a request is parsed. However, I don’t see a similar close() method on the weaviate client.

Any recommendations on how to establish a client connection to weaviate in a restful service? For example, imagine I have a restful app that serves 50 concurrent requests. Should they each open a new connection to weaviate? Are there examples of this? Or is there some existing machinery to create a connection pool etc…

Using the verba codebase for inspiration, looks like they create a single client

client = weaviate.Client(
    additional_headers={"X-OpenAI-Api-Key": openai.api_key},
    embedded_options=EmbeddedOptions(
        persistence_data_path="./.verba/local/share/",
        binary_path="./.verba/cache/weaviate-embedded",
    ),
)

And then call the .query() method repeatedly. I suppose I should take a similar approach?

Any recommendations or further reading on best practices for clients making concurrent requests would be appreciated!

2

Hi! Welcome to our community :hugs: !!

Weaviate client is, for now, a wrapper around Weaviate server REST and GRAPHQL queries/endpoints.

So the connection is closed when the query is returned. unless you do it intentionally, directly in requests, bypassing the client.

There are some work in place to enable the client to connect using GRPC, but it has not been released yet.

Let me know if this helps :slight_smile:

3

Ok thank you.

I guess at the end of the day, I’m asking can I use a single client inside an async handler method in some parallel fashion, or should I create a new client object on every request?

Here is the simple way - just create a new client each time. But if establishing the client has a lot of overhead, this is not performant

from fastapi import FastAPI
import weaviate

app = FastAPI()

@app.get("/")
async def read_root():
    client = weaviate.Client("http://localhost:8080")
    result = client.some_synchronous_method()
    return {"result": result}

On the other hand we can get fancy doing something with parallelization of the client in a threadpool, but I have not explored.

from fastapi import FastAPI
import weaviate
from concurrent.futures import ThreadPoolExecutor

app = FastAPI()

client = weaviate.Client("http://localhost:8080")

executor = ThreadPoolExecutor()

@app.get("/")
async def read_root():
    result = await app.loop.run_in_executor(executor, client.some_synchronous_method)
    return {"result": result}

What is recommended?

4

I believe that when you instantiate the client its just a preparation for the actually .do(), when it finally execute requests.post()

So there should not have much of an overhead

So either way would work, as what matters is the actual query. :thinking:

I’ll ask about this internally!

1 Like
6

I am also thinking of using FastAPI. So given what’s said above about its lightweight and stateless manner (the bit about rest requests calls), is it right to use the fastapi’s “dependency injection (with cleanup)” and just have it spin a weaviate client up per request, and call it .close(…) once the fastapi request route is done.?

7

hi @00.lope.naughts !!

Glad you asked! In Python v4 client we are using GRPC, that will require connection close.

So it is a good practice to always close the connection.

So your approach is correct: instantiate the client, use it, and close after finishing your FastAPI request.

:slight_smile: