TLDR; The explained vecotr similarly score included as supplemental information when preforming a hybrid search does not match by a significant amount the same distance metric returned by a near_vector search for the same two object.
For my current setup I have Class A and Class B which both use the same vectorizer and both which uses the cosine distance metric enabling the certainty metric to be returned.
I have noticed the following inconsistency:
When preforming hybrid search for an object by pulling a vector V1 from class A and then searching within class B using the near search operator.
Suppose that one of the results returned for this search is for Object Z1 from class B. It has score S_z
Now if I preform a hybrid search on Class B and provide the same vector used above and ensure the Object Z1 is returned by using a where filter.
By enabling the Explained metrics with response I can see the explained (Unnormalized) Vector score returned for Z1 is different than the one obtained from the near vector search above.
I’ve tried to reconcile this a number of ways including:
- Possibly it using the [0,2] cosine distance. I tried comparing this as well and it didn’t match
- Maybe it returning the ranked fusion score for the vector instead. I tried reversing the calculating calculation based on the score = 1/ rank + 60. This was not it either.
I was under the impression that the vector similarity used in the hybrid search was based on the classes underlying vectorizer but perhaps it is using some other fast dense vectorizer?
Any clarification on how the Vector Similarity component in the hybrid search is be computed would help. Or why it so vastly different that the same metric returned by vector search for the same two objects.