Conditional filter with python client v4

Support
1

I used to query a collection with …query.with_where(filter) with python client v3,
where filter was a formal conditional filter like
{
“operator”: “And”,
“operands”: [
{“path”: [“color”], “operator”: “Equal”, “valueText”: “grau”},
{“path”: [“price”], “operator”: “LessThanEqual”, “valueNumber”: 300},
],
}

looks like that this is not possible anymore with python client v4 (or deprecated at least), is it?

that is unfortunate, as I am building the filter like above and translating it to the new Filter methods or build a string from it to call directly via http is cumbersome.

2

hi @jhc !!

Welcome to our community :hugs:

This is certainly possible.

here is how:

import weaviate
client = weaviate.connect_to_local()
from weaviate.classes.query import Filter

client.collections.delete("Test")
collection = client.collections.create(name="Test")
collection.data.insert({"color": "grau", "price": 300})
collection.data.insert({"color": "grau", "price": 400})
collection.data.insert({"color": "grau", "price": 100})
collection.data.insert({"color": "bope", "price": 400})
collection.data.insert({"color": "bope", "price": 200})

# now we filter

query = collection.query.fetch_objects(
    filters=(
        Filter.by_property("color").equal("grau") & 
        Filter.by_property("price").less_or_equal(300)
    )
)
for object in query.objects:
    print(object.properties)

For more information on filters, check this doc:

Let me know if this helps!

Thanks!

3

Thanks,

but sorry I haven’t made myself clear enaugh. I’m aware that I can build filters as you outlined above, e.g. constructing a filter with subsequently making use of Filter.by_property.

but I have a hugh complex filter as a python structure as I explained, that I could previously (in v3) put into query.with_where(formal_filter).
But with_where does not exist anymore, and there is no substitute.

Or am I missing something?

4

Hi @jhc !!

No worries. I think I got you covered.

Is this what you are looking for?
Given this dataset:

import weaviate
client = weaviate.connect_to_local()
from weaviate.classes.query import Filter

client.collections.delete("Test")
collection = client.collections.create(name="Test")
collection.data.insert({"color": "grau", "price": 300, "location": "Brazil"})
collection.data.insert({"color": "grau", "price": 400, "location": "Netherlands"})
collection.data.insert({"color": "grau", "price": 100, "location": "Netherlands"})
collection.data.insert({"color": "bope", "price": 400, "location": "Egypt"})
collection.data.insert({"color": "bope", "price": 200, "location": "Egypt"})

you can have optional filters added to a base filter, for example:

filter_by_location = True
location = "Brazil"

# now we filter
#base filter
filters=(
    Filter.by_property("color").equal("grau") & 
    Filter.by_property("price").less_or_equal(300)
)

if filter_by_location:
    filters = filters & Filter.by_property("location").equal(location)

query = collection.query.fetch_objects(
    filters=filters
)

for object in query.objects:
    print(object.properties)

Note that you can also do more complex logic, using nested filters.

Like in this example:

response = jeopardy.query.fetch_objects(
    filters=Filter.by_property("answer").like("*bird*") &
            (Filter.by_property("points").greater_than(700) | Filter.by_property("points").less_than(300)),
    limit=3
)

Let me know if this helps!

Thanks!

5

Let me try once more.

In python client v3, I could query weaviate with a filter that consists of python data structures, e.g.

filter_in_python_data_structures = {‘operator’: ‘And’, ‘operands’: [{‘path’: [‘price’], ‘operator’: ‘LessThanEqual’, ‘valueNumber’: 300}, {‘path’: [‘color’], ‘operator’: ‘Equal’, ‘valueText’: ‘schwarz’}, {‘path’: [‘available_sizes’], ‘operator’: ‘Like’, ‘valueText’: ‘44’}, {‘path’: [‘categories’], ‘operator’: ‘Like’, ‘valueText’: ‘Schuhe’}, {‘path’: [‘material’], ‘operator’: ‘Like’, ‘valueText’: ‘Leder’}, {‘path’: [‘brand’], ‘operator’: ‘Equal’, ‘valueText’: ‘Nike’}, {‘path’: [‘certifications’], ‘operator’: ‘Like’, ‘valueText’: ‘Öko-Tex Standard 100’}, {‘path’: [‘condition’], ‘operator’: ‘Equal’, ‘valueText’: ‘neu’}] … }

This filter is long and complex. It is build by another module.

In python client v3, I could query weaviate with directly using this filter:

client.query.get(collection, return_properties).with_where(filter_in_python_data_structures)

But, in python client v4, there is no “with_where” method anymore. I found the methods you indicated, but they would not accept something like filter_in_python_data_structures.
Instead, that is my understanding, I need to build a filter by myself with Filter.by_property etc.

But, I don’t like to do that, as would need to write first a wrapper to transform filter_in_python_data_structures to something that is accepted by collection.query.fetch_objects( filters=…) .
Before doing that, I would rather translate filter_in_python_data_structures to a valid graphQL string and call weaviate without making use of the python client.

My hope is, that I have overlooked something, and that there is a way to still query as it was possible in python client v3. Or, if not, maybe, you are planning to extend the functionality of python client v4 in this way.

6

Oh, I see.

Indeed you will need to generate a v4 python filter as it will not support a v3 filter.

This new client uses GRPC, while the v3 uses Graphql with REST.

So all the underlying apis the python v4 client communicate are different. Because of that, I don’t believe that there will be a support for v3 filters syntax in v4.

Let me know if this helps.

Thanks!

7

ok, thanks.

I have to stick with v3 filters. I’m check what comes closest to this v3 filter. I’m getting from you that it will be the REST interface. Maybe taking over the implementation of v3 of how to translate a v3 filter to what is required by REST.

8

Hi @jhc !

I had a similar thing where I came from one filter definition, and I had to move into v4 Filters.

I think something along this line could help translate filters from v3 to v4:

from weaviate.classes.query import Filter
from weaviate.collections.classes.filters import _Filters, _FilterByProperty


V3_TO_V4_OPERATORS = {
    "Equal": "equal",
    "NotEqual": "not_equal",
    "GreaterThan": "greater_than",
    "LessThan": "less_than",
    "LessThanEqual": "less_or_equal",
    "GreaterThanEqual": "greater_or_equal",
    # TODO: Add other operators here
}

filters = []
filter_: _Filters
filter_by_property: _FilterByProperty
for v3_filter in filters:
    filter_by_property = Filter.by_property(v3_filter["path"])
    operator = V3_TO_V4_OPERATORS[v3_filter["type"]]
    filter_ = getattr(filter_by_property, operator)(v3_filter["valueText"]) # TODO: You will need to extend this value* to more types.
    filters.append(filter_)

filters = Filter.all_of(filters)

Hope it helps!

2 Likes
9

What can probably help you on this situation is to build the graphql query in v3 client and execute it in pyv4.

in v3 instead of calling the .do() you can call .build() and get the graphql string.

In pyv4 you can call it like so:

client = weaviate.connect_to_local()    
client.graphql_raw_query(query.build())

Let me know if this helps.

10

hi @Guillermo_Ripa !!

Nice! Thanks for sharing.

Once thing to consider is that all_of will:

Combine all filters in the input list with an AND operator.

So if you have some OR logic, you will need to adapt and use any_of instead.

2 Likes