Reranker- change batch size?

I have a custom reranker integrated with weaviate, it is awesome :slight_smile: I was wondering if it is possible to change the number of documents sent to the reranker at once? It looks like it sends 32 documents at a time even if I set my search to a limit > 32. I can handle many more so it would improve search speed if this was configurable.

Hi @mjsteele!

Are using the transformers-reranker?

it seems it have a 32 documents limit hard coded

Thatโ€™s an interesting feature that would bring more flexibility.
Could you please open a feature request at our Github here?

For now, a workaround would be forking this module, and building as a new, custom one, setting the limits you want. I know, not ideal :frowning_with_open_mouth:

Let me know if this helps!

1 Like

I have added a feature request, thanks!

1 Like