What endpoints are required for a custom reranker?

Support
1

Hi,

I have a custom reranker model that I am self hosting. I’d like to enable this in weaviate by replacing parts of the reranker-transformers module. I see the example in the docs of required endpoints for customizing text2vec-transformers, but I can’t seem to find the same for reranker-transformers. Can someone please help me identify what I need to implement/what the incoming requests will look like for rerankers?

Thanks!!

EDIT:
I ended up just pulling the semitechnologies/reranker-transformers container and searching around inside. I found the following input class:

class CrossEncoderInput(BaseModel):
query: str
property: Optional[str]
documents: Optional[list[str]]

which looks like what I need! Also- it looks like the same endpoints as the text2vec-transformers except instead of /vectors we have /rerank:
GET /.well-known/live
GET /.well-known/ready
GET /meta
POST /rerank

bonus for those of you who care- I’m running both text2vec-transformers custom model and the custom reranker-transformers at the same url, so all I had to do was add /rerank and update the weaviate config!

2

Not an answer to your question, but I’m very interested in this. Probably more people might want this. Let me know when you’ve figured it out :slight_smile:

1 Like
3

I can confirm that what I put in my edit works!

2 Likes
4

hi @mjsteele ! Thanks for sharing :hugs:

Glad you were able to figure this out!

For instance, here are some links for future reference:

So considering you want to change reranker-transformers inferece api, this is where you can get the endpoints the original container module provides:

A nice place to also understand those API calls is here:

Thanks!

1 Like
5

That’s awesome! Are you able to share it as a repo? If not, no worries, just curious. thanks!

6

I copied what @DudaNogueira posted in app.py, the only change is the model instantiation and predict method. Very easy to implement! :slightly_smiling_face:

1 Like
7

Thanks, that’s awesome