What endpoints are required for a custom reranker?


I have a custom reranker model that I am self hosting. I’d like to enable this in weaviate by replacing parts of the reranker-transformers module. I see the example in the docs of required endpoints for customizing text2vec-transformers, but I can’t seem to find the same for reranker-transformers. Can someone please help me identify what I need to implement/what the incoming requests will look like for rerankers?


I ended up just pulling the semitechnologies/reranker-transformers container and searching around inside. I found the following input class:

class CrossEncoderInput(BaseModel):
query: str
property: Optional[str]
documents: Optional[list[str]]

which looks like what I need! Also- it looks like the same endpoints as the text2vec-transformers except instead of /vectors we have /rerank:
GET /.well-known/live
GET /.well-known/ready
GET /meta
POST /rerank

bonus for those of you who care- I’m running both text2vec-transformers custom model and the custom reranker-transformers at the same url, so all I had to do was add /rerank and update the weaviate config!

Not an answer to your question, but I’m very interested in this. Probably more people might want this. Let me know when you’ve figured it out :slight_smile:

1 Like

I can confirm that what I put in my edit works!


hi @mjsteele ! Thanks for sharing :hugs:

Glad you were able to figure this out!

For instance, here are some links for future reference:

So considering you want to change reranker-transformers inferece api, this is where you can get the endpoints the original container module provides:

A nice place to also understand those API calls is here:


1 Like

That’s awesome! Are you able to share it as a repo? If not, no worries, just curious. thanks!

I copied what @DudaNogueira posted in app.py, the only change is the model instantiation and predict method. Very easy to implement! :slightly_smiling_face:

1 Like

Thanks, that’s awesome