# How to handle concurrent requests in re-rerank feature in Weaviate?

**URL:** <https://forum.weaviate.io/t/how-to-handle-concurrent-requests-in-re-rerank-feature-in-weaviate/1275>\
**Category:** Support\
**Tags:** developer-experience\
**Created:** [January 22, 2024, 5:32am UTC](https://forum.weaviate.io/t/how-to-handle-concurrent-requests-in-re-rerank-feature-in-weaviate/1275 "2024-01-22T05:32:36Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![ChengCheng](https://avatars.discourse-cdn.com/v4/letter/c/45deac/32.png) [@ChengCheng](https://forum.weaviate.io/u/ChengCheng)\
**Post date:** [January 22, 2024, 5:32am UTC](https://forum.weaviate.io/t/how-to-handle-concurrent-requests-in-re-rerank-feature-in-weaviate/1275/1 "2024-01-22T05:32:36Z")

</div>

Currently, re-rank does not support concurrent requests. Will we develop concurrent requests of rerank in the future? Or how to reduce the time consumption of re-rank for multiple collections？Thanks

---

<div class="post-metadata">

**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\
**Post date:** [January 22, 2024, 9:57pm UTC](https://forum.weaviate.io/t/how-to-handle-concurrent-requests-in-re-rerank-feature-in-weaviate/1275/2 "2024-01-22T21:57:40Z")

</div>

Hi @ChengCheng !! Welcome to our community 🙂

Not sure I follow. AFAIK, the rerank API will require all documents to be sent at the same time, for example, cohere api:

> **[Rerank](https://cohere.com/rerank)**
>
> Rerank is a powerful semantic boost for search quality. Improve search precision with Rerank's semantic boost via ElasticSearch or OpenSearch integration. Optimize eCommerce and knowledge base search without overhaul or replacement.

Can you elaborate on that?

Thanks!

---

<div class="post-metadata">

**Author:** ![ChengCheng](https://avatars.discourse-cdn.com/v4/letter/c/45deac/32.png) [@ChengCheng](https://forum.weaviate.io/u/ChengCheng)\
**Post date:** [January 23, 2024, 9:20am UTC](https://forum.weaviate.io/t/how-to-handle-concurrent-requests-in-re-rerank-feature-in-weaviate/1275/3 "2024-01-23T09:20:13Z")

</div>

Hi , @DudaNogueira Thanks for your reply. We are using the reranker-transformers. There are multiple classes (collections) in Weaviate. ‘rerank-transformer’ cannot be processed in parallel.  
_e.g. query 9 collections in parallel, weaviate returns chunks from 9 collections in parallel (you get chunks from 9 collection at the same time), but the weaviate re-ranker re-rank the collections one by one (sequentially)_  
How can we speed up retrieval when we want to call multiple classes with rerank feature in parallel?

---

<div class="post-metadata">

**Author:** ![Benjamin\_Lush](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/benjamin_lush/32/1433_2.png) [@Benjamin\_Lush](https://forum.weaviate.io/u/Benjamin_Lush)\
**Post date:** [May 24, 2024, 10:41pm UTC](https://forum.weaviate.io/t/how-to-handle-concurrent-requests-in-re-rerank-feature-in-weaviate/1275/4 "2024-05-24T22:41:20Z")

</div>

Just a shot in the dark here… perhaps you can run multiple instances of Weaviate and each can have it’s own reranker-transformers instance? I’m having the same problem and am interested in the solution. I’ll let you know how my experiment turns out.

---

<div class="post-metadata">

**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\
**Post date:** [May 29, 2024, 7:29pm UTC](https://forum.weaviate.io/t/how-to-handle-concurrent-requests-in-re-rerank-feature-in-weaviate/1275/5 "2024-05-29T19:29:12Z")

</div>

Hi!

Welcome to our community @Benjamin_Lush ! 🤗

I believe you will need to run multiple instances of the inference/reranker of the same models , and run that behind a load balancer. 🤔

With that you could spread the load?