Issue with Distance Value in Weaviate Hybrid Search and Applying Similarity Score Threshold

Support
1

I am using hybrid search in Weaviate and setting alpha=1 as I want to perform purely semantic search in my current scenario. I also need to filter results based on the semantic similarity score.

Here’s my query code:

response = self.collection.query.hybrid(
                query=query_text.strip(),
                query_properties=[field_name],
                alpha=alpha,
                target_vector=[f"{field_name}_embeddings"],
                return_metadata=MetadataQuery(score=True,distance=True,explain_score=True),
                filters=search_filter,
                limit=top_k
            )

            # Process results efficiently
            field_guids = set()
            for obj in response.objects:
                print(f"Object: {obj.metadata}")
                distance = obj.metadata.distance
                print(f"Distance: {distance}")
                score = 1 - distance
                if score < score_threshold:
                    continue  # Skip low-score results

Issues I am facing:

  1. The distance value in obj.metadata is always None.
  2. Since distance is None, I am unable to calculate the semantic similarity score (1 - distance).
  3. What is the correct way to filter search results based on semantic similarity in hybrid search?

Am I missing something in the query setup, or is there a better approach to applying a score threshold in hybrid search? Any suggestions would be greatly appreciated!

2

hi @Rohini_vaidya !!

When you run a hybrid query, Weaviate will fuse the distance (Vector/Similarity search) and the score (Bm25). Because of that, you will not have a distance.

Even if you run a hybrid with alpha 1.

If you want to filter he results based on distance, you need to do a near_text query.

Let me know if this clarifies it for you.

Thanks!

3

Thank you @DudaNogueira

I understood the point.

1 Like