Error with date filter not working

General
1

I am encountering issues with a date filter. An example schema is below.

For some objects the Offline Date Value is ‘9999-12-31T23:59:59Z’ corresponding with having no offline date.

However when I try and filter products which have an offline date greater than the current date I get no results.

I provide an example below. Any help would be appreciated.

For example

db.client.query.get("Product", ["product_id", "offline_date","upload_date"]).with_where(
    {"path": ["offline_date"], "operator": "GreaterThan", "valueDate":"2023-12-21T23:31:51.66399Z" }
).with_limit(3).do()

Returns the result:

{'data': {'Get': {'Product': []}}}

{
    "class": "Product",
    "description": "Product Info encoded with PaLM embeddings",
   
 
    "properties": [
        {
            "dataType": ["text"],
            "description": "The product id ,
            "indexFilterable": true,
            "indexSearchable": true,
            "moduleConfig": {
                "text2vec-palm": {"skip": false, "vectorizePropertyName": false}
            },
            "name": "product_id",
            "tokenization": "whitespace"
        },
        {
            "dataType": ["date"],
            "description": "The date the product goes offline",
            "moduleConfig": {
                "text2vec-palm": {"skip": true, "vectorizePropertyName": false}
            },
            "name": "offline_date"
        }
    ],
}
1 Like
2

Hi @Landon_Edwards !

Thanks for reporting!

I was able to reproduce this. Smells like a bug :beetle: to me :thinking:

Here some code for reproducing:

import weaviate
client = weaviate.Client("http://localhost:8080")
client.schema.delete_class("Product")
schema = {
    "class": "Product",
    "description": "Product Info encoded with PaLM embeddings",
    "properties": [
        {
            "dataType": ["text"],
            "description": "The product id" ,
            "indexFilterable": True,
            "indexSearchable": True,
            "moduleConfig": {
                "text2vec-palm": {"skip": False, "vectorizePropertyName": False}
            },
            "name": "product_id",
            "tokenization": "whitespace"
        },
        {
            "dataType": ["date"],
            "description": "The date the product goes offline",
            "moduleConfig": {
                "text2vec-palm": {"skip": True, "vectorizePropertyName": False}
            },
            "name": "offline_date"
        }
    ]
}
client.schema.create_class(schema)

# this data is never fecthed
client.data_object.create({
    "product_id": "123",
    "offline_date": "9999-12-31T23:59:59Z"
    },
    "Product"
)

client.data_object.create({
    "product_id": "321",
    "offline_date": "2000-12-31T23:59:59Z"
    },
    "Product"
)

# show all 2 objects
client.query.get("Product", "product_id offline_date").do()

# should only show 9999 object. but shows none
client.query.get("Product", ["product_id", "offline_date"]).with_where(
    {"path": ["offline_date"], "operator": "GreaterThan", "valueDate":"2023-12-21T23:31:51.66399Z" }
).with_limit(3).do()

# this will only return one result, dated 2023, but not the one dated 9999
client.query.get("Product", ["product_id", "offline_date"]).with_where(
    {"path": ["offline_date"], "operator": "GreaterThan", "valueDate":"1999-12-21T23:31:51.66399Z" }
).with_limit(3).do()

I believe that we can go ahead and create a new issue.

Thanks!

3

Using weaviate-client 4.6.0 and weaviate 1.25.0 here, I’m using 3000-01-01T00:00:00+00:00 for my records and they are all less than 2024-04-12T15:31:07+00:00.

Using 2100-01-01T00:00:00+00:00 however, they are not less than 2024-04-12T15:37:48+00:00. It seems like an overflow problem.