Hybrid score returning NONE scores randomly for the same query with the same chunks

Support
1

Hi, currently facing the following problem:

when extracting the hybrid score in the following manner:


.with_hybrid(
                query=query,
                alpha=0.5,
                vector=embedded_query,
                fusion_type=HybridFusion.RELATIVE_SCORE
            ) \
            .with_additional(['score', 'explainScore', 'id']) 

def get_score_from_object(obj) -> float | None:
    if obj is not None \
            and isinstance(obj, dict) \
            and "_additional" in obj \
            and obj["_additional"] is not None \
            and isinstance(obj["_additional"], dict) \
            and "score" in obj["_additional"]:
        return obj["_additional"]["score"]
    else:
        return None

Score values returned for the same query with the same chunks 3 times delivered following results:

SCORE 0.8715031
SCORE 0.62721163
SCORE 0.6142747
SCORE 0.5678266
SCORE 0.5
SCORE 0.4993174
SCORE 0.47722983

Then:

SCORE 0.018778471
SCORE None
SCORE None
SCORE None
SCORE None
SCORE None
SCORE None

Then:

All scores were None

And then by the 4rth time quering, received the same scores as the first time.

Am I extracting it wrong? Why is it also happening randomly? This problem didn’t seem to occur with the regular vector score…

This is what the response from weaviate looks like with and without None:

SCORE None

{‘_additional’: {‘explainScore’: ‘\nHybrid (Result Set keyword) Document a40a4895-ea75-45c0-b09c-dfde5ff94f11: original score NaN, normalized score: NaN’, ‘id’: ‘a40a4895-ea75-45c0-b09c-dfde5ff94f11’, ‘score’: None}, ‘document’: [{‘_additional’: {‘id’: ‘2260563d-8b45-45b6-93fa-217d651387ac’}

SCORE 0.026299512

{‘_additional’: {‘explainScore’: ‘\nHybrid (Result Set vector) Document 6ebf4e68-f63c-4bdd-b6d1-7ec8cc79f71f: original score 0.7842568, normalized score: 0.026299512’, ‘id’: ‘6ebf4e68-f63c-4bdd-b6d1-7ec8cc79f71f’, ‘score’: ‘0.026299512’}, ‘document’: [{‘_additional’: {‘id’: ‘f03cf04d-7af5-4ca2-a0d6-886feac9ed67’}

SCORE None

{‘_additional’: {‘explainScore’: ‘\nHybrid (Result Set vector) Document 70e58854-ff9d-4721-87da-e446035b3f50: original score 0.81138486, normalized score: 0.17339171 - \nHybrid (Result Set keyword) Document 70e58854-ff9d-4721-87da-e446035b3f50: original score NaN, normalized score: NaN’, ‘id’: ‘70e58854-ff9d-4721-87da-e446035b3f50’, ‘score’: None},
‘document’: [{‘_additional’: {‘id’: ‘7006ad84-b872-4684-b9b7-bcb892623693’}

Thanks for your help!

2

Hi!

What is the version you are using?

Can you reproduce this on latest version?

3

Hi, we are on weaviate server version 1.24.3!

4

We now reproduced with 1.24.8…

5

Hi! Can you share a reproducible code example?

I was not able to reproduce this here :frowning:

Thanks!

6

@DudaNogueiraIt is not reproducible as it happens very randomly. I am experiencing the same issue.

7

hi! Welcome to our community.

What are the version you are using?

Can you reproduce this with latest?