according to section below,
This module uses a third-party API and may incur costs
but also
This module is enabled by default on the WCS.
but when I initialize the cloud client, I don’t pass in Cohere creds
client = weaviate.connect_to_wcs(
What’s the way to use Cohere’s english v3 reranker with weaviate?
is there also a way to get visibility into:
to observe how much of a difference it’s making?
Thanks in advance.
hi!
In order to get cohere rerank working, you need to first enable it in your ENABLE_MODULES (already enabled in WCS).
With that module enabled, you will need to initiate your client specifying the cohere API KEY:
import weaviate
import os
# Set these environment variables
URL = os.getenv("YOUR_WCS_URL")
APIKEY = os.getenv("YOUR_WCS_API_KEY")
COHERE_APIKEY = os.environ["COHERE_APIKEY"]
# Connect to a WCS instance
client = weaviate.connect_to_wcs(
cluster_url=URL,
auth_credentials=weaviate.auth.AuthApiKey(APIKEY),
headers={
"X-Cohere-Api-Key": COHERE_APIKEY # Replace with your inference API key
}
)
Now you can create a collection and specify that you want to use that module, like so:
import weaviate.classes as wvc
collection = client.collections.create(
name="MyCollection",
vectorizer_config=wvc.config.Configure.Vectorizer.text2vec_openai(),
reranker_config=wvc.config.Configure.Reranker.cohere(),
)
After applying the reranker, you can access the rerank_score, like so:
query = collection.query.bm25(
query="Animals",
rerank=wvc.query.Rerank(
prop="question",
query="big"
),
return_metadata=wvc.query.MetadataQuery(certainty=True, score=True)
)
for i in query.objects:
print(i.metadata.rerank_score, i.properties.get("question"))
0.01665704 The gavial looks very much like a crocodile except for this bodily feature
0.01395571 Weighing around a ton, the eland is the largest species of this animal in Africa
0.005099818 Heaviest of all poisonous snakes is this North American rattlesnake
0.0005442133 It's the only living mammal in the order Proboseidea
Let me know if this helps 
Thanks Duda!
If I already have a collection in prod, how would I add it?
@DudaNogueira wanted to bump,
If I already have a collection in prod, how would I add it?
Hi @tjh !
Sorry for the delay here 
Unfortunately, this is not a mutable configuration
We even have a GH issue for this (please, leave your thumbs up there! also, while in GH, please, leave us a 
too)
So, in order to enable it for your collection, you will need to create a new one, with all the modules you want, and then migrate your data 
It’s fairly easy process, and we have a guide for that here:
Let me know if this helps
Thanks again Duda,
Upvoted & starred 
one quick followup,
the reranker model eg rerank-english-v2.0
need this be linked to our embedder or embed-english-v3.0 or is that completely separate?
They are separated.
You can use any model for the embeddings/vectorizer, and a totally different generative model or rerank model.
For example, you can rerank using cohere, search your data vectorized using openai, and generate using other model
But be aware, for now, in order to change any of those modules in a collection, a new collection is needed, as those are no mutable:
What is prop and query in this case?
If I want to retrieve 10 documents using vector/hybrid/keyword search (using limit = 10), how do I specify I want to rerank these documents in cross-encoder (reranker), and retrieve the top 5?
hi! prop is the property where the query will act as reranker.
Described here:
If you want to post limit your rerank to 5, you need to do that after the process, something like:
query = collection.query.bm25(
query="Animals",
rerank=wvc.query.Rerank(
prop="question",
query="big"
),
return_metadata=wvc.query.MetadataQuery(certainty=True, score=True)
)
top_5_objects = query.objects[0:5]
Let me know if this helps!
Thanks!
Yes, that helps clear it up. So the reranking takes place after the entire BM25 run is completed in your example, right?
That’s right!
It will perform the search (bm25, vector, hybrid), and then all those results are passed to the reranker.
If you want to somehow “bias” those results instead of doing a reranking, you can also experiment with move_to and move_away:
This will give you some “post” retrieval tweaking.
From my understanding, you can still do post-retrieval tweaking using the vector parameter, even with reranking present. This applies to post-retrieval, pre-rerank stage, yes?
I was not able to find an option to use a generator with this chain, however, since we can’t specify the number of most relevant documents to pick after re-ranking. Is there a way to do that?
Eg. Search → Pick 25 → Rerank → Pick Top 10 → Generate based on top 10?
Currently, I have to move out of the weaviate framework and query my language model independently with the reranked context even though I am using the language model as a generator in my weaviate pipeline.