# Cannot get total size of result set for pagination for hybrid, vector or bm25 search **URL:** \ **Category:** Support\ **Created:** [April 11, 2025, 9:30pm UTC](https://forum.weaviate.io/t/cannot-get-total-size-of-result-set-for-pagination-for-hybrid-vector-or-bm25-search/20628 "2025-04-11T21:30:57Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![fastdatascience](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/fastdatascience/32/7110_2.png) [@fastdatascience](https://forum.weaviate.io/u/fastdatascience)\ **Post date:** [April 11, 2025, 9:30pm UTC](https://forum.weaviate.io/t/cannot-get-total-size-of-result-set-for-pagination-for-hybrid-vector-or-bm25-search/20628/1 "2025-04-11T21:30:57Z")
### Description I have an API connected to a Weaviate index. The API is running HuggingFace Transformers to convert my search terms to queries - I’m not using Weaviate vectorisation. My API has three search modes: vector, keyword aka bm25, and hybrid. So I am converting the user’s search term into a vector with the transformer model, and then sending it to Weaviate (only vector and hybrid search take the vector) For each of these three modes, I am retrieving only 25 results to display to the user. The remaining results, the user can see if they go to page 2, page 3, etc. The problem is that we have no way to know how many pages there are in the result (except for bm25). The only way I have found is to use aggregate queries to get the size of the total result set. I tried using the aggregate queries with filters, but it never comes to the same size of results as what I get when I run the query with a high limit, like 100,000. **Question 1: if I want to know the size of a bm25 result set, should I use an aggregate with Filter.by\_property like I am doing? It feels like I am reverse engineering/trying to re-implement my search in an entirely different way** **Question 2: if I want to know the size of a vector or hybrid result set, how can I get the same number that I would get if I request all results? I cannot see any way to get this number other than requesting the entire result set** For example, hybrid and vector query for my keyword “anxiety” returns 423 results when I set `distance=0.5`. I cannot find any way to get the number 423 apart from running this query with offset=0 and limit=10000. The function `aggregate.near_vector` with the same vector and filter fields and `distance=0.5` yields a count of 260. I could not get the aggregate hybrid search to work at all. Perhaps it works when Weaviate is deployed together with its own vectorisation instead of me doing the vectorisation on my end, but my deployment is just bare Weaviate without a transformer model so I have not verified this. In short, there is no way that I have found to discover the size of a vector or hybrid result set, other than running the query and requesting the entire result set from Weaviate. Whatever I could find in the documentation which looked like it should do this, did not fulfil this purpose. I could run the aggregation function for Filter (trying to reproduce the same BM25 query, but not sure if this is really doing the same thing under the hood, but I couldn’t see an alternative) and I could run the vector aggregation. For vector aggregation, no matter what parameters I input, I got numbers which were lower than the size of the actual corresponding result set for that search mode. For the hybrid search it was even worse as I could not get the hybrid aggregation to work at all. I got the error “vector index: missing target vector”, although I am supplying that argument. ### TRYING BM25 - TRY FIRST NORMAL WEAVIATE QUERY, TAKE NUMBER OF OBJECTS, AND SEE IF FILTER AGGREGATION CAN GET THE SAME RESULTS - Number of results from BM25 query: 85 (`collection.query.bm25(query=query[0], limit=10000, query_properties=["name", "all_text"])`) - Result of Filter aggregation (should be the same as number of results from original BM25 query): 85 (`collection.aggregate.over_all(filters=Filter.any_of([Filter.by_property("all_text").contains_any(query), Filter.by_property("name").contains_any(query)]) )`) This appears to be the correct number. However I am a bit worried that it won’t always be the case, since I am assembling this number in an entirely different way - how can I tell that the filter in the aggregation will behave the same as bm25? In fact I would expect them to be different since BM25 works on n-grams and is tolerant of spelling mistakes and the filter might not be. I have not verified this either way. ### TRYING VECTOR - TRY FIRST NORMAL WEAVIATE QUERY, TAKE NUMBER OF OBJECTS, AND SEE IF VECTOR AGGREGATION CAN GET THE SAME RESULTS - Number of results from vector query: 423 - Result of vector aggregation (should be the same as number of results from original vector query): 260 ### TRYING HYBRID - TRY FIRST NORMAL WEAVIATE QUERY, TAKE NUMBER OF OBJECTS, AND SEE IF HYBRID AGGREGATION CAN GET THE SAME RESULTS - Number of results from hybrid query: 423 - [I cannot get hybrid aggregation function to work with any parameters - it just give errors (see stack trace below)] ### Server Setup Information - Weaviate Server Version: `cr.weaviate.io/semitechnologies/weaviate:1.29.0` - Deployment Method: docker - Multi Node? Number of Running Nodes: 1 - Client Language and Version: python 4.11.0 - Multitenancy?: no ### Any additional Information I have included my complete program, minus passwords, below. I have included the stack trace of my hybrid aggregation as well. Outputs are included in the script below as comments after `#`. ```auto from weaviate.classes.query import Filter from weaviate.connect import ConnectionParams COLLECTION_NAME = "COLLECTION_NAME" from weaviate.classes.init import Auth import weaviate client = weaviate.WeaviateClient( connection_params=ConnectionParams.from_params( http_host="HOST", http_port=443, http_secure=True, grpc_host="grpc.HOST", grpc_port=50051, grpc_secure=True, ), auth_client_secret=Auth.api_key(APIKEY), skip_init_checks=False ) client.connect() print(client.is_ready()) harmony_index = client.collections.get(COLLECTION_NAME) harmony_count = harmony_index.aggregate.over_all().total_count # This is the search term and the vector of the search term query = ["anxiety"] query_vector = [[3.35463315e-01, 1.51534006e-01, -5.64120829e-01, 6.77912474e-01, 6.65759966e-02, -9.33390558e-02, 6.84192479e-01, 3.64687294e-01, -2.82174069e-02, -1.72709897e-01, -1.24643125e-01, -7.15998327e-03, 2.47695729e-01, -3.59601736e-01, 2.22267076e-01, 3.26149940e-01, 5.84262311e-02, -5.45905173e-01, -1.03112139e-01, 2.28038147e-01, -1.02381267e-01, -4.71212059e-01, 3.69310945e-01, -3.54012638e-01, -3.22700113e-01, 2.76131988e-01, 3.73639278e-02, -1.86167836e-01, -1.00431032e-01, -8.66742358e-02, 3.16369802e-01, -6.70351207e-01, 2.54377663e-01, -4.57104146e-02, 4.37935501e-01, 1.20846249e-01, -1.73799232e-01, 1.06407702e-01, 3.93517256e-01, 6.50031045e-02, 1.12434894e-01, 1.95664406e-01, -2.26766005e-01, 1.35269193e-02, 1.97353542e-01, 6.61122575e-02, 6.88953474e-02, 1.33001581e-02, 2.70984143e-01, 1.45878449e-01, -5.78154763e-03, -3.71801071e-02, 1.28119335e-01, -4.98251289e-01, -1.40645728e-01, 2.06611276e-01, -3.05665553e-01, 4.08114076e-01, -2.22795025e-01, 1.79221079e-01, -1.72730044e-01, 1.81372121e-01, -4.06667352e-01, 4.18124676e-01, 1.18704803e-01, -2.25808918e-01, -4.81796235e-01, -2.71317065e-01, 2.07416728e-01, -2.88918447e-02, 4.86879677e-01, 2.99828529e-01, 1.59070536e-01, 2.79036492e-01, 7.94327259e-01, -2.98845440e-01, 1.18322439e-01, 9.32147503e-02, -7.91112855e-02, 2.15424255e-01, 5.55794299e-01, 5.85238636e-01, 3.98371607e-01, -3.43910068e-01, 1.40659079e-01, -2.13161573e-01, 1.81872055e-01, 1.79136679e-01, -1.64859459e-01, -1.60634145e-02, 3.05600613e-01, -6.29799068e-02, -8.68289918e-03, -3.23678628e-02, 5.36237299e-01, 2.47043207e-01, -3.54895979e-01, -2.95386344e-01, -6.76021099e-01, 2.05918527e+00, 8.84405151e-02, 1.51585415e-01, -1.44557595e-01, 5.76418638e-01, -2.93688744e-01, 3.82334799e-01, -1.09732658e-01, -5.13578117e-01, -3.35707664e-01, -3.60577740e-02, -4.99324650e-01, -3.37365627e-01, 7.23077580e-02, -2.92712152e-01, 1.69516385e-01, -5.71813107e-01, 7.82266632e-02, 1.92815512e-02, 4.70578641e-01, 3.97872686e-01, -1.14258938e-01, -3.79356623e-01, 3.37463945e-01, -2.72577286e-01, 1.92604437e-01, -4.72456068e-01, -1.11911498e-01, -1.22308590e-01, -4.67121303e-02, 1.54250726e-01, -1.78190425e-01, 4.95246530e-01, -1.21425010e-01, -4.81173359e-02, -1.71974421e-01, -1.93188950e-01, 4.11586851e-01, 2.13519856e-01, -6.98918551e-02, -1.64113548e-02, -1.66204765e-01, 5.80064058e-02, 4.60767269e-01, -2.06856892e-01, -3.07465702e-01, 9.16019306e-02, -1.42542109e-01, 1.41571788e-02, 2.34147549e-01, -6.63373843e-02, -1.45412341e-01, 5.88088147e-02, 5.98725118e-02, 5.80832899e-01, 1.10482417e-01, -4.00147140e-02, -5.17827451e-01, 2.83491373e-01, -6.09525084e-01, 2.85991211e-03, -2.26733282e-01, -1.80871442e-01, -1.34952947e-01, 2.15342566e-01, -3.50693278e-02, -3.21026266e-01, -5.27675807e-01, -1.06256664e-01, 2.25965619e-01, -3.39970998e-02, 6.01748489e-02, -1.82754114e-01, -1.67253256e-01, 2.42962301e-01, -2.47286516e-03, -2.72219539e-01, -2.88920581e-01, -2.69849330e-01, -3.10567617e-01, -1.86400875e-01, 4.76120114e-01, -3.54339242e-01, -9.87442508e-02, 1.93104431e-01, 4.08082902e-02, -5.26979230e-02, -8.46949741e-02, -6.33822232e-02, -9.45824087e-02, -5.73025882e-01, -3.20404060e-02, -2.63877124e-01, 5.47255278e-02, 3.21177870e-01, -3.75654370e-01, 2.93490261e-01, -1.98497340e-01, 7.39022717e-02, -3.57463270e-01, 4.37498003e-01, 2.27000758e-01, -4.13726419e-01, 2.68085927e-01, 7.86525756e-02, 2.10468993e-01, -3.84509176e-01, -1.36251524e-01, -1.09019764e-01, 6.01922512e-01, -3.22339207e-01, 3.79671961e-01, 4.81130034e-01, 4.77516413e-01, -1.24746181e-01, -3.12690362e-02, -1.51122734e-01, -2.85211772e-01, 2.57927299e-01, -4.35654335e-02, 3.41354579e-01, 3.38322520e-01, 2.46698424e-01, -4.61305946e-01, -2.62644142e-01, 1.66483477e-01, -4.70261246e-01, -1.16948433e-01, -4.56474632e-01, -2.25364730e-01, 4.70466167e-01, -1.05019033e-01, 8.42545554e-02, 1.75342754e-01, 2.95024484e-01, 2.46478513e-01, 1.48589671e-01, 1.11533143e-01, 1.23442240e-01, -4.58077103e-01, -2.12503076e-01, -6.49597943e-01, 7.34895840e-02, 1.60925761e-01, 4.05711792e-02, -3.46383661e-01, -3.52607250e-01, 1.33997217e-01, -8.34354758e-02, 1.37121633e-01, 1.15843736e-01, -3.00232116e-02, -5.18483341e-01, 4.86333460e-01, -1.37704790e-01, -4.72671628e-01, 4.74346548e-01, 8.64914358e-02, 9.49817002e-02, 1.52383149e-01, 5.01141727e-01, -1.95603967e-01, -3.83717567e-01, -2.10875809e-01, 5.01872338e-02, -2.80028824e-02, 7.68501699e-01, 1.22991778e-01, 1.95511863e-01, 4.09941465e-01, 6.97474420e-01, 3.94268751e-01, 6.15394674e-02, 9.33347344e-02, 3.45813543e-01, 2.50950933e-01, 3.77290249e-01, -5.23763359e-01, 4.43115622e-01, 7.18046948e-02, -4.36774850e-01, -2.27288529e-01, -4.41563755e-01, -2.82906681e-01, 4.00768183e-02, -1.43475056e-01, -4.46148068e-01, -4.15665954e-01, 1.14555739e-01, 8.07124153e-02, 4.43762839e-02, -2.42462948e-01, 1.43958583e-01, 9.72241834e-02, 3.11084121e-01, 1.92619618e-02, -3.49334240e-01, 2.54832387e-01, 2.42490962e-01, 2.62845784e-01, 4.95936066e-01, 2.90113181e-01, -5.81976712e-01, 1.18311614e-01, 1.88573822e-01, -1.30133718e-01, -2.37258554e-01, 2.65680328e-02, -3.49867195e-01, -4.33916420e-01, -1.50669709e-01, -8.72264281e-02, -3.71248759e-02, -2.51314729e-01, -8.26448575e-02, -4.18445468e-02, -1.39517665e-01, -1.89415123e-02, 4.96251583e-01, -1.63653284e-01, -9.27618742e-01, 2.94972267e-02, -2.33644783e-01, -1.28878215e-02, 4.84860688e-01, 2.28712127e-01, -7.29070827e-02, -3.19305658e-01, -5.18278658e-01, -5.38706221e-02, -4.65666622e-01, 2.82632947e-01, -4.47521992e-02, -2.38389149e-02, 1.20173834e-01, -4.23557848e-01, 2.83777714e-01, 1.47676304e-01, 1.23942643e-02, 3.76267523e-01, -1.78621426e-01, 8.82371902e-01, 2.29514942e-01, 5.78418225e-02, -8.75021040e-04, 1.40977785e-01, -2.94940919e-03, -3.16139102e-01, -4.55647141e-01, -7.91564584e-02, 4.89958078e-01, -1.12918414e-01, -5.48998594e-01, -7.82390416e-04, -2.33876958e-01, -3.26787204e-01, 4.62798566e-01, -1.41035780e-01, 1.59310680e-02, -6.85367510e-02, -3.57112795e-01, 1.09244324e-01, 1.86645865e-01, 3.71286362e-01, 1.18220029e-02, -1.85265467e-01, 1.43055975e-01, 1.80511877e-01, -3.01655918e-01, 4.11505476e-02, -2.68879205e-01, -9.79132354e-02, -4.60742623e-01, 2.85828471e-01, -5.15420735e-01, 2.14212790e-01, 1.70870617e-01, -2.43845642e-01, 4.26682234e-01, -4.69405860e-01, -3.87645394e-01, 1.72637045e-01, 2.99873471e-01, -5.24893463e-01, 3.09333056e-01]][0] print( "TRYING BM25 - TRY FIRST NORMAL WEAVIATE QUERY, TAKE NUMBER OF OBJECTS, AND SEE IF BM25 AGGREGATION CAN GET THE SAME RESULTS") harmony_query_response_anxiety_bm25_no_limit = harmony_index.query.bm25( query=query[0], limit=10000, query_properties=["name", "all_text"] ) print("Number of results from BM25 query:", len(harmony_query_response_anxiety_bm25_no_limit.objects)) # Outputs: Number of results from BM25 query: 706 aggregation_filter = harmony_index.aggregate.over_all( filters=Filter.any_of( [Filter.by_property("all_text").contains_any(query), Filter.by_property("name").contains_any(query)]) ) print("Result of Filter aggregation (should be the same as number of results from original BM25 query):", aggregation_filter.total_count) # Outputs: Result of Filter aggregation: 85 print( "TRYING VECTOR - TRY FIRST NORMAL WEAVIATE QUERY, TAKE NUMBER OF OBJECTS, AND SEE IF VECTOR AGGREGATION CAN GET THE SAME RESULTS") harmony_query_response_vector = harmony_index.query.near_vector( near_vector=query_vector, target_vector=["all_text", "name"], limit=10000, distance=0.5 ) print("Number of results from vector query:", len(harmony_query_response_vector.objects)) # Outputs: Number of results from vector query: 423 aggregation_vector = harmony_index.aggregate.near_vector(near_vector=query_vector, distance=0.5, target_vector=["all_text", "name"] ) print("Result of vector aggregation (should be the same as number of results from original vector query):", aggregation_vector.total_count) # Outputs: Result of vector aggregation: 260 print( "TRYING HYBRID - TRY FIRST NORMAL WEAVIATE QUERY, TAKE NUMBER OF OBJECTS, AND SEE IF HYBRID AGGREGATION CAN GET THE SAME RESULTS") harmony_query_response_hybrid = harmony_index.query.hybrid( vector=query_vector, target_vector=["all_text", "name"], query=query[0], limit=10000, max_vector_distance=0.5 ) print("Number of results from hybrid query:", len(harmony_query_response_hybrid.objects)) # Outputs: Number of results from hybrid query: 423 aggregation_vector = harmony_index.aggregate.hybrid(query=query[0], vector=query_vector, max_vector_distance=0.5, target_vector=["all_text", "name"] ) print("Result of vector aggregation (should be the same as number of results from original vector query):", aggregation_vector.total_count) ''' --------------------------------------------------------------------------- AioRpcError Traceback (most recent call last) File ~/anaconda3/lib/python3.12/site-packages/weaviate/collections/grpc/aggregate.py:284, in _AggregateGRPC.__call(self, request) 283 assert self._connection.grpc_stub is not None --> 284 res = await _Retry(4).with_exponential_backoff( 285 0, 286 f"Searching in collection {request.collection}", 287 self._connection.grpc_stub.Aggregate, 288 request, 289 metadata=self._connection.grpc_headers(), 290 timeout=self._connection.timeout_config.query, 291 ) 292 return cast(aggregate_pb2.AggregateReply, res) File ~/anaconda3/lib/python3.12/site-packages/weaviate/collections/grpc/retry.py:31, in _Retry.with_exponential_backoff(self, count, error, f, *args, **kwargs) 30 if e.code() != StatusCode.UNAVAILABLE: ---> 31 raise e 32 logger.info( 33 f"{error} received exception: {e}. Retrying with exponential backoff in {2**count} seconds" 34 ) File ~/anaconda3/lib/python3.12/site-packages/weaviate/collections/grpc/retry.py:28, in _Retry.with_exponential_backoff(self, count, error, f, *args, **kwargs) 27 try: ---> 28 return await f(*args, **kwargs) 29 except AioRpcError as e: File ~/anaconda3/lib/python3.12/site-packages/grpc/aio/_call.py:327, in _UnaryResponseMixin. __await__ (self) 326 else: --> 327 raise _create_rpc_error( 328 self._cython_call._initial_metadata, 329 self._cython_call._status, 330 ) 331 else: AioRpcError: During handling of the above exception, another exception occurred: WeaviateQueryError Traceback (most recent call last) Cell In[37], line 1 ----> 1 aggregation_vector = harmony_index.aggregate.hybrid(query=query[0], vector=query_vector, 2 max_vector_distance=0.5, 3 target_vector=["all_text", "name"] 4 ) 6 print ("Result of vector aggregation (should be the same as number of results from original vector query):", aggregation_vector.total_count) File ~/anaconda3/lib/python3.12/site-packages/weaviate/syncify.py:23, in convert..sync_method(self, __new_name, *args, **kwargs) 20 @wraps(method) # type: ignore 21 def sync_method(self, *args, __new_name=new_name, **kwargs): 22 async_func = getattr(cls, __new_name) ---> 23 return _EventLoopSingleton.get_instance().run_until_complete( 24 async_func, self, *args, **kwargs 25 ) File ~/anaconda3/lib/python3.12/site-packages/weaviate/event_loop.py:42, in _EventLoop.run_until_complete(self, f, *args, **kwargs) 40 raise WeaviateClosedClientError() 41 fut = asyncio.run_coroutine_threadsafe(f(*args, **kwargs), self.loop) ---> 42 return fut.result() File ~/anaconda3/lib/python3.12/concurrent/futures/_base.py:456, in Future.result(self, timeout) 454 raise CancelledError() 455 elif self._state == FINISHED: --> 456 return self.__get_result() 457 else: 458 raise TimeoutError() File ~/anaconda3/lib/python3.12/concurrent/futures/_base.py:401, in Future.__get_result(self) 399 if self._exception: 400 try: --> 401 raise self._exception 402 finally: 403 # Break a reference cycle with the exception in self._exception 404 self = None File ~/anaconda3/lib/python3.12/site-packages/weaviate/collections/aggregations/hybrid.py:100, in _HybridAsync.hybrid(self, query, alpha, vector, query_properties, object_limit, filters, group_by, target_vector, max_vector_distance, total_count, return_metrics) 93 return ( 94 self._to_aggregate_result(res, return_metrics) 95 if group_by is None 96 else self._to_group_by_result(res, return_metrics) 97 ) 98 else: 99 # use grpc --> 100 reply = await self._grpc.hybrid( 101 query=query, 102 alpha=alpha, 103 vector=vector, 104 properties=query_properties, 105 object_limit=object_limit, 106 target_vector=target_vector, 107 distance=max_vector_distance, 108 aggregations=( 109 [metric.to_grpc() for metric in return_metrics] 110 if return_metrics is not None 111 else [] 112 ), 113 filters=_FilterToGRPC.convert(filters) if filters is not None else None, 114 group_by=group_by._to_grpc() if group_by is not None else None, 115 limit=group_by.limit if group_by is not None else None, 116 objects_count=total_count, 117 ) 118 return self._to_result(reply) File ~/anaconda3/lib/python3.12/site-packages/weaviate/collections/grpc/aggregate.py:296, in _AggregateGRPC.__call(self, request) 294 if e.code().name == PERMISSION_DENIED: 295 raise InsufficientPermissionsError(e) --> 296 raise WeaviateQueryError(str(e), "GRPC search") # pyright: ignore 297 except WeaviateRetryError as e: 298 raise WeaviateQueryError(str(e), "GRPC search") WeaviateQueryError: Query call with protocol GRPC search failed with message **Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\ **Post date:** [April 15, 2025, 2:36pm UTC](https://forum.weaviate.io/t/cannot-get-total-size-of-result-set-for-pagination-for-hybrid-vector-or-bm25-search/20628/2 "2025-04-15T14:36:55Z") hi @fastdatascience !! Welcome to our community 🤗 This is because when you do a keyword search, the results are defined: They do or do not have the keyword you are looking for. A vector search (and hybrid), on the other hand, doesn’t. If you do a query, **all objects will have a similarity to your query**. The first one will your best match, the last one, the more distant one, the worse. But it still will be on that list. Ps: this when not using filters. By default, you get that limit as defined in `QUERY_DEFAULTS_LIMIT`. If you intentionally set the limit on the client it can go beyond that. Now your limit will be defined at `QUERY_MAXIMUM_RESULTS` [See environment variables here](https://weaviate.io/developers/weaviate/config-refs/env-vars) So, answering your questions, you can aggregate your bm25 filters and get the total. But not for vector or hybrid. Let me know if this helps or if you have any other interesting questions 🙂 We hold a weekly Office Hours on wednesdays: > **[Online Workshops & Events | Weaviate](https://weaviate.io/community/events)** > > \-Join us at conferences, meetups, webinars or workshops Join us! its super fun! ---
**Author:** ![fastdatascience](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/fastdatascience/32/7110_2.png) [@fastdatascience](https://forum.weaviate.io/u/fastdatascience)\ **Post date:** [April 15, 2025, 3:01pm UTC](https://forum.weaviate.io/t/cannot-get-total-size-of-result-set-for-pagination-for-hybrid-vector-or-bm25-search/20628/3 "2025-04-15T15:01:10Z")
Thank you. This makes sense that the vector search result space would be the entire document collection in the absence of filters, however in reality my queries have filters and it would be nice to have a single Weaviate call that gives me the first 10 results, plus the size of the entire result set. At the moment I am having to do two calls to Weaviate for each query type, once to get the current page of results and once to get the total. This seems very error prone, in particular for the bm25 where I have to first pass my query in the `query` field and the filters in the `filters` field when I am getting the actual results, but then I have to pass everything in the `filters` field to get the aggregation. This seems both a situation that will result in hard to spot bugs, and it’s also inefficient. Would it be possible to add a boolean parameter to all three types of Weaviate query called something like `return_results_count`? That would eliminate this problem. you can see in my code below, I branch for all three query types, and then for each query type I had to make two Weaviate calls, once to query and once to aggregate, taking care to reproduce the same query even though the parameters I have to pass are different. It took a lot of trial and error to get this working and it seems very fragile. If Weaviate returned the results count in all queries (at least optionally), that would remove this problem. ```auto if query_type == QueryType.vector: harmony_query_response = harmony_index.query.near_vector( near_vector=query_vector, target_vector=["all_text", "name"], filters=filters, limit=num_results, offset=offset, distance=max_vector_distance, return_metadata=MetadataQuery(distance=True, score=True), return_properties=return_properties, # don't link from study to variable - too many links, it crashes the query. return_references=return_references, ) aggregation_vector = harmony_index.aggregate.near_vector(near_vector=query_vector, distance=max_vector_distance, target_vector=["all_text", "name"], filters=filters ) # Result of vector aggregation (should be the same as number of results from original vector query: num_results_total = aggregation_vector.total_count elif query_type == QueryType.keyword: harmony_query_response = harmony_index.query.bm25( query=query[0], query_properties=query_properties, filters=filters, limit=num_results, offset=offset, return_metadata=MetadataQuery(distance=True, score=True), return_properties=return_properties, # don't link from study to variable - too many links, it crashes the query. return_references=return_references, ) aggregation_filter = harmony_index.aggregate.over_all( filters=Filter.all_of(filters_list + [Filter.any_of( [Filter.by_property("all_text").contains_any(query), Filter.by_property("name").contains_any(query), Filter.by_property("keywords").contains_any(query)])]) ) # Result of Filter aggregation (should be the same as number of results from original BM25 query num_results_total = aggregation_filter.total_count elif query_type == QueryType.hybrid: harmony_query_response = harmony_index.query.hybrid( vector=query_vector, target_vector=["all_text", "name"], query_properties=query_properties, query=query[0], filters=filters, limit=num_results, offset=offset, max_vector_distance=max_vector_distance, return_metadata=MetadataQuery(distance=True, score=True), return_properties=return_properties, # don't link from study to variable - too many links, it crashes the query. return_references=return_references, alpha=alpha ) # same as for vector search aggregation_vector = harmony_index.aggregate.near_vector(near_vector=query_vector, distance=max_vector_distance, target_vector=["all_text", "name"], filters=filters ) # Result of vector aggregation (should be the same as number of results from original vector query: num_results_total = aggregation_vector.total_count ```