How to handle concurrent requests in re-rerank feature in Weaviate?

Currently, re-rank does not support concurrent requests. Will we develop concurrent requests of rerank in the future? Or how to reduce the time consumption of re-rank for multiple collections?Thanks

Hi @ChengCheng !! Welcome to our community :slight_smile:

Not sure I follow. AFAIK, the rerank API will require all documents to be sent at the same time, for example, cohere api:

Can you elaborate on that?


Hi , @DudaNogueira Thanks for your reply. We are using the reranker-transformers. There are multiple classes (collections) in Weaviate. ‘rerank-transformer’ cannot be processed in parallel.
e.g. query 9 collections in parallel, weaviate returns chunks from 9 collections in parallel (you get chunks from 9 collection at the same time), but the weaviate re-ranker re-rank the collections one by one (sequentially)
How can we speed up retrieval when we want to call multiple classes with rerank feature in parallel?