Strange hehavior when combine filters with and operator to filter by reference

General
1

I have a filter like this

        text_chunk_filter = (
                Filter.by_ref("has_index_in_file_data").by_property("file_data_uuid").equal(file_data_uuid)
                & Filter.by_ref("has_index_in_file_data").by_property("index").greater_or_equal(start_index)
                & Filter.by_ref("has_index_in_file_data").by_property("index").less_or_equal(end_index)
        )

this will cause a warning:

{"level":"warning","msg":"Number of found nested reference results exceeds configured QUERY_MAXIMUM_RESULTS. This may result in search performance degradation or even out of memory errors.","nested_reference_results":11922,"query_maximum_results":10000,"time":"2024-03-11T12:09:04Z"}

I have already kinda know what the warning is and trying to solve it but I find a strange thing.
if I use this filter the then the warning will not be triggered

        text_chunk_filter = (
                Filter.by_ref("has_index_in_file_data").by_property("file_data_uuid").equal(file_data_uuid)
        )

As far As I know ther former filter should be more strict than the later one but how could the later one won’t trigger the warning?

2

Hi @shadowlin !

that text_chunk_filter is the only part it changes on that query? the returned properties and references are the same for both, right?

Also, how is that query being made? hybrid?

Thanks!

3

yes.The filter is the only thing changed. the query is a vector only query.
And the “file_data_uuid” filter alone could reduce the returned result to blow 100 objects.
I always find the filter1 & filter2 dosen’t work as I thought in weaviate, is there something I missed?

4

Hi!

Do you think you could reproduce this on a smaller scale and with anonymous data?

I tried some options here, but always got the correct results with my dataset.

Here is a nice variation:

from weaviate import classes as wvc

include_weather = True
include_plane = True

base_filter = wvc.query.Filter.by_property("category").equal("ANIMALS")
weather_filter = wvc.query.Filter.by_property("question").like("weather")
plane_filter = wvc.query.Filter.by_property("question").like("plane")

used_filters = [base_filter]
if include_plane:
    used_filters.append(plane_filter)
if include_weather:
    used_filters.append(weather_filter)

query = questions.query.fetch_objects(
    filters=wvc.query.Filter.any_of(used_filters) # can also be all_of
)
for object in query.objects:
    print(object.properties)
5

I will try to come up with a demo to show the case later.

1 Like