# Not Equal Filter with Word Tokenization with non-alphanumeric characters

**URL:** <https://forum.weaviate.io/t/not-equal-filter-with-word-tokenization-with-non-alphanumeric-characters/5289>\
**Category:** Support\
**Created:** [October 14, 2024, 10:42pm UTC](https://forum.weaviate.io/t/not-equal-filter-with-word-tokenization-with-non-alphanumeric-characters/5289 "2024-10-14T22:42:30Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![dhanshew72](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dhanshew72/32/1809_2.png) [@dhanshew72](https://forum.weaviate.io/u/dhanshew72)\
**Post date:** [October 14, 2024, 10:42pm UTC](https://forum.weaviate.io/t/not-equal-filter-with-word-tokenization-with-non-alphanumeric-characters/5289/1 "2024-10-14T22:42:30Z")

</div>

### Description

Is it possible to use the not equal filter on properties with word tokenization even if they don’t have alphanumeric characters? Unfortunately I have data like `test.com/2` with word tokenization when it wasn’t intended.

For example, I’d want to query `test.com/2` using a not equal filter like below:

```auto
Filter.by_property("my_property").not_equal("test.com/2")

```

Does any workaround exist like query against “test com 2” or update tokenization or search with a different tokenization? My main issue is adding a new field or replacing with the proper tokenization is a lengthy process in a production system.

### Server Setup Information

- Weaviate Server Version: 1.24.0
- Deployment Method: Docker
- Multi Node? Number of Running Nodes: No.
- Client Language and Version: 4.6.5
- Multitenancy?: Yes

---

<div class="post-metadata">

**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\
**Post date:** [October 15, 2024, 9:30pm UTC](https://forum.weaviate.io/t/not-equal-filter-with-word-tokenization-with-non-alphanumeric-characters/5289/2 "2024-10-15T21:30:17Z")

</div>

hi @dhanshew72 !

because you had tokenization set to word, the property value `test.com/2` will be tokenized as `test` `com` `2`

This will proves our point:

```python
client.collections.delete("Test")
collection = client.collections.create(
    name="Test",
    vectorizer_config=wvc.config.Configure.Vectorizer.none(),
)

collection.data.insert_many([
    {"text": "test.com/2"},
    {"text": "test.com/3"},
    {"text": "test.com/4"},

])

```

now we query:

```python
results = collection.query.fetch_objects(
    filters=(
        wvc.query.Filter.by_property("text").equal("test") & 
        wvc.query.Filter.by_property("text").equal("com") & 
        wvc.query.Filter.by_property("text").equal("2")
    )
)
for i in results.objects:
    print("###")
    print(i.properties)

```

results:

> {‘text’: ‘[test.com/2](http://test.com/2)’}

As you want to exclude that filtered objects, not equal on a word tokenization will not help you.

So you can try adding a new property, with the `field` tokenization, and then filling in the content of that property so you can filter it out.

Let me know if this helps 🙂

---

<div class="post-metadata">

**Author:** ![dhanshew72](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dhanshew72/32/1809_2.png) [@dhanshew72](https://forum.weaviate.io/u/dhanshew72)\
**Post date:** [October 16, 2024, 4:09pm UTC](https://forum.weaviate.io/t/not-equal-filter-with-word-tokenization-with-non-alphanumeric-characters/5289/3 "2024-10-16T16:09:53Z")

</div>

Interesting, I’ll make note of that. Thank you.