# Performance Issue when Extracting Documents with Field Filter in Weaviate

**URL:** <https://forum.weaviate.io/t/performance-issue-when-extracting-documents-with-field-filter-in-weaviate/256>\
**Category:** Support\
**Created:** [June 21, 2023, 10:26am UTC](https://forum.weaviate.io/t/performance-issue-when-extracting-documents-with-field-filter-in-weaviate/256 "2023-06-21T10:26:06Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Iammsd07](https://avatars.discourse-cdn.com/v4/letter/i/5fc32e/32.png) [@Iammsd07](https://forum.weaviate.io/u/Iammsd07)\
**Post date:** [June 21, 2023, 10:26am UTC](https://forum.weaviate.io/t/performance-issue-when-extracting-documents-with-field-filter-in-weaviate/256/1 "2023-06-21T10:26:06Z")

</div>

Hello Weaviate Forum Team,

I hope this message finds you well. I am currently facing a performance issue while attempting to extract documents from a “Class” in my Weaviate local instance. Specifically, I would like to retrieve all documents where the field “X” equals a specific value.

Following the official documentation, I have implemented a “with\_where” filter in my code as shown below:

```auto
where_filter = {
    "path": ["X"],
    "operator": "Equal",
    "valueText": "value",
}

result = (
    client.query
    .get("Class", ["content"])
    .with_where(where_filter)
    .do()
)

```

When there are no results found, the response time is quite negligible, with a duration of only a few nanoseconds. However, when there are 70 matches out of a dataset consisting of approximately 6.6 billion documents, the response time increases significantly to around 14 seconds.

Additionally, I have observed that when the number of matches reaches approximately 100,000, the response time further extends to approximately an hour. These extended response times are causing concerns for my application.

I am currently using Weaviate version 1.17.5 and the Weaviate Python client library version 3.15.4.

I would appreciate your assistance in understanding whether this performance behavior is expected or if there might be any improvements or optimizations that I can implement to enhance the query execution time.

Thank you for your attention and support. I look forward to your guidance.

Best regards,  
Harshit

---

<div class="post-metadata">

**Author:** ![jphwang](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/jphwang/32/38_2.png) [@jphwang](https://forum.weaviate.io/u/jphwang)\
**Post date:** [June 21, 2023, 12:09pm UTC](https://forum.weaviate.io/t/performance-issue-when-extracting-documents-with-field-filter-in-weaviate/256/2 "2023-06-21T12:09:59Z")

</div>

Hi @Iammsd07

I suspect you’ll benefit greatly from upgrading the Weaviate version to the latest (and Python too).

The reason is that filtering performance for very strict filters was greatly improved with `1.18` by introduction of Roaring bitmaps.

> **[Weaviate 1.18 release | Weaviate - vector database](https://weaviate.io/blog/weaviate-1-18-release#faster-filtering-with-roaring-bitmaps)**
>
> Weaviate 1.18 introduces Faster Filtering through Bitmap Indexing, HNSW-PQ, Cursor API, and more! Learn all about it.

> **[Filtered Vector Search | Weaviate - vector database](https://weaviate.io/developers/weaviate/concepts/prefiltering#pre-filtering-with-roaring-bitmaps)**
>
> \- References: GraphQL API

Please try that and see if that helps, especially for the strict filters

For the filters with high number of matches - do you need to fetch all 100k documents with a Get request at once?

If not, you could for instance use [`Aggregate`](https://weaviate.io/developers/weaviate/search/aggregate) to see how many objects match, or alternative you could use [pagination](https://weaviate.io/developers/weaviate/search/basics#paginate-with-limit-and-offset) to show subsets of results.

Cheers  
JP

---

<div class="post-metadata">

**Author:** ![Iammsd07](https://avatars.discourse-cdn.com/v4/letter/i/5fc32e/32.png) [@Iammsd07](https://forum.weaviate.io/u/Iammsd07)\
**Post date:** [June 21, 2023, 2:30pm UTC](https://forum.weaviate.io/t/performance-issue-when-extracting-documents-with-field-filter-in-weaviate/256/3 "2023-06-21T14:30:57Z")

</div>

Thanks @jphwang

Upgrading the weaviate version worked, response time for 70 matches was reduced from 14 seconds to 1.3 seconds.

---

<div class="post-metadata">

**Author:** ![jphwang](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/jphwang/32/38_2.png) [@jphwang](https://forum.weaviate.io/u/jphwang)\
**Post date:** [June 21, 2023, 5:43pm UTC](https://forum.weaviate.io/t/performance-issue-when-extracting-documents-with-field-filter-in-weaviate/256/4 "2023-06-21T17:43:28Z")

</div>

Cool! Glad to hear it 🙂