# Keyword, vector and hybrid searching cause less rows to be retrieved

**URL:** <https://forum.weaviate.io/t/keyword-vector-and-hybrid-searching-cause-less-rows-to-be-retrieved/1412>\
**Category:** Support\
**Created:** [February 7, 2024, 12:05pm UTC](https://forum.weaviate.io/t/keyword-vector-and-hybrid-searching-cause-less-rows-to-be-retrieved/1412 "2024-02-07T12:05:37Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![moaazzaki](https://avatars.discourse-cdn.com/v4/letter/m/6f9a4e/32.png) [@moaazzaki](https://forum.weaviate.io/u/moaazzaki)\
**Post date:** [February 7, 2024, 12:05pm UTC](https://forum.weaviate.io/t/keyword-vector-and-hybrid-searching-cause-less-rows-to-be-retrieved/1412/1 "2024-02-07T12:05:37Z")

</div>

### Description

I have a coolection with multi-tenancy enabled, for a given tenant, if I use the following graphql query:

```auto
{
    Get {
        SomeCollection(
            tenant: "some-id"
            limit: 2000
        ){
            someField
        }
    }
}

```

I get 2000 rows returned (I know that this tenant has 10k+ rows). Now when I try this query with bm25 for example:

```auto
{
    Get {
        SomeCollection(
            tenant: "some-id"
            limit: 2000
            bm25:{
                 query: "Some query"
                 properties: ["fieldWithText"]
            }
        ){
            someField
        }
    }
}

```

I get ~1.3k instead, the same applies no matter what limit I choose above 1.3k. Any idea what could the issue be here ?

### Server Setup Information

I tried the above on two different setups:

#### Setup 1

- Weaviate Version: 1.23.2
- Deployment Method: docker
- Multi Node? Number of Running Nodes: 1
- Used Client Language and Version: REST API

#### Setup 2

- Weaviate Version: 1.23.7
- Deployment Method: k8s
- Multi Node? Number of Running Nodes: 3
- Used Client Language and Version: REST API

---

<div class="post-metadata">

**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\
**Post date:** [February 9, 2024, 11:48pm UTC](https://forum.weaviate.io/t/keyword-vector-and-hybrid-searching-cause-less-rows-to-be-retrieved/1412/2 "2024-02-09T23:48:11Z")

</div>

hi @moaazzaki !!

the [BM25](https://weaviate.io/developers/weaviate/api/graphql/search-operators#bm25) search will get you a ranked list of objects that has those query.

Consider the code below:

```auto
import weaviate
from weaviate import classes as wvc
client = weaviate.connect_to_local()

collection = client.collections.create(
    name="MyCollection",
    vectorizer_config=None
)

collection.data.insert({"text": "SOme text here"})
collection.data.insert({"text": "SOme other text here"})
collection.data.insert({"text": "Other stuff"})

collection.query.bm25(
      query="stuff",
      query_properties=["text"],
      return_metadata=wvc.query.MetadataQuery(score=True),
      limit=5,
  )

query = collection.query.bm25(
      query="other stuff",
      query_properties=["text"],
      return_metadata=wvc.query.MetadataQuery(score=True),
      limit=5,
  )
print(len(query.objects))

```

it will yeld only 2 objects.

Let me know if that helps 🙂

---

<div class="post-metadata">

**Author:** ![moaazzaki](https://avatars.discourse-cdn.com/v4/letter/m/6f9a4e/32.png) [@moaazzaki](https://forum.weaviate.io/u/moaazzaki)\
**Post date:** [February 12, 2024, 8:13am UTC](https://forum.weaviate.io/t/keyword-vector-and-hybrid-searching-cause-less-rows-to-be-retrieved/1412/3 "2024-02-12T08:13:35Z")

</div>

Hi @DudaNogueira, thanks for your response!

The problem is that this happens as well for vector & hybrid search, and same data can sometimes work and sometimes not on the same query, so for a tenant with 10k data, I get 6k maximum with hybrid search, and if I removed the collection and insert it again, it can be a different number than 6k (e.g. it was 1.3k before), so I feel like there is a bit of inconsistency here.

---

<div class="post-metadata">

**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\
**Post date:** [February 12, 2024, 10:29pm UTC](https://forum.weaviate.io/t/keyword-vector-and-hybrid-searching-cause-less-rows-to-be-retrieved/1412/4 "2024-02-12T22:29:24Z")

</div>

Hi!

I believe this is the case of a higher efConstruction. From here:

> [@Simple keyword search not working](https://forum.weaviate.io/t/simple-keyword-search-not-working/276/5):
>
> HNSW is an approximate nearest neighbour algorithm. This means there is always a chance a vector will not be found in a search even though it would be found via brute force (using where filter). The solution to this problem is to re-index your data with higher efConstruction parameter (for example efConstruction=512) that way your newly built graph will have higher number of node candidates that will be considered during vector search.

> HNSW is an approximate nearest neighbour algorithm. This means there is always a chance a vector will not be found in a search even though it would be found via brute force (using where filter).
> 
> The solution to this problem is to re-index your data with higher `efConstruction` parameter (for example `efConstruction=512`) that way your newly built graph will have higher number of node candidates that will be considered during vector search.

---

<div class="post-metadata">

**Author:** ![moaazzaki](https://avatars.discourse-cdn.com/v4/letter/m/6f9a4e/32.png) [@moaazzaki](https://forum.weaviate.io/u/moaazzaki)\
**Post date:** [February 13, 2024, 9:42am UTC](https://forum.weaviate.io/t/keyword-vector-and-hybrid-searching-cause-less-rows-to-be-retrieved/1412/5 "2024-02-13T09:42:34Z")

</div>

> [@DudaNogueira](#):
>
> chance a vector will not be found in a search even though it would be found via brute force (using where filter).
> 
> The solution to this problem is to re-index your data with higher `efConstruction` parameter (for example `efConstruction=512`) that way your newly built graph will have higher number of node candidates that will be considered during vector search.

Hi!

Does this also apply on flat index setup ?