Iterator() yields no results when iterating Weaviate objects

Support
1

Description

Hi, I’m new here. I’m currently using Weaviate for a RAG system, and my dataset is on the scale of several million records. Because of this, I’d like to skip documents that have already been added to the database before running my program.

I vectorized the documents locally and inserted the vectors directly into the database, rather than providing an embedding module.

I came across the iterator() function and thought it might help with this. However, when I try something like for obj in judgements.iterator():, it doesn’t behave as expected — the loop doesn’t run at all, as if no data is being retrieved.

judgements = client.collections.get("Judgement")
count = judgements.aggregate.over_all(total_count=True).total_count
print("Collection count:", count) #  amount of data

for obj in judgements.iterator():
    # doesn't run anything

Interestingly, when I use judgements.query.near_vector, I am still able to retrieve relevant objects.

response = judgements.query.near_vector(
    near_vector=embeddings.embed_query("some query"),
)
for obj in response.objects:
    # it does print some objects out here
    print(obj)

Could someone help me understand what’s going on?

Thanks a lot!

Server Setup Information

  • Weaviate Server Version: 1.33.0-dev
  • Weaviate Client Version : 4.11.0
  • Deployment Method: Local, run in binary file
2

hi @Jacky_Yee !!

Welcome to our community :hugs:

That’s strange. the iterator will run thru all objects right away.

do you see any outstanding logs on Weaviate while the iterator is running?

3

Hi, @DudaNogueira , thanks for your quick reply!
Sorry if this sounds like a silly question, but what kind of logs would actually be considered abnormal?
When I use iterator(), I see logs like {"action":"sparse_object_search_completed", ...} and {"action":"resolve_references_completed", ..., "len":0}.

Also, our school server had an unexpected shutdown recently—could that have affected anything?

The complete logs is here:

{"action":"restapi_request","build_git_commit":"","build_go_version":"go1.24.0","build_image_tag":"","build_wv_version":"1.33.0-dev","level":"debug","method":"GET","msg":"received HTTP request","time":"2025-08-20T01:37:25+08:00","url":{"Scheme":"","Opaque":"","User":null,"Host":"","Path":"/v1/.well-known/openid-configuration","RawPath":"","OmitHost":false,"ForceQuery":false,"RawQuery":"","Fragment":"","RawFragment":""}}
{"action":"restapi_request","build_git_commit":"","build_go_version":"go1.24.0","build_image_tag":"","build_wv_version":"1.33.0-dev","level":"debug","method":"GET","msg":"received HTTP request","time":"2025-08-20T01:37:25+08:00","url":{"Scheme":"","Opaque":"","User":null,"Host":"","Path":"/v1/meta","RawPath":"","OmitHost":false,"ForceQuery":false,"RawQuery":"","Fragment":"","RawFragment":""}}
{"action":"error_group_wait_initiated","build_git_commit":"","build_go_version":"go1.24.0","build_image_tag":"","build_wv_version":"1.33.0-dev","jobs_count":1,"level":"debug","limit":64,"msg":"Waiting for 1 jobs to finish with limit 64","time":"2025-08-20T01:37:26+08:00"}
{"action":"sparse_object_search_completed","build_git_commit":"","build_go_version":"go1.24.0","build_image_tag":"","build_wv_version":"1.33.0-dev","level":"debug","msg":"sparse object search query completed in 129.554µs","params":{"Filters":null,"ClassName":"Judgement","Pagination":{"Offset":0,"Limit":100,"Autocut":0},"Cursor":null,"Sort":null,"Properties":[{"name":"content","isPrimitive":true,"isObject":false,"includeTypeName":false,"refs":null,"objs":null},{"name":"judgement_id","isPrimitive":true,"isObject":false,"includeTypeName":false,"refs":null,"objs":null},{"name":"date","isPrimitive":true,"isObject":false,"includeTypeName":false,"refs":null,"objs":null}],"NearVector":null,"NearObject":null,"KeywordRanking":null,"HybridSearch":null,"GroupBy":null,"TargetVector":"","TargetVectorCombination":null,"Group":null,"ModuleParams":null,"AdditionalProperties":{"classification":false,"refMeta":false,"vector":false,"vectors":null,"certainty":false,"id":true,"creationTimeUnix":false,"lastUpdateTimeUnix":false,"moduleParams":null,"distance":false,"score":false,"explainScore":false,"isConsistent":false,"group":false,"noProps":false},"ReplicationProperties":null,"Tenant":"","IsRefOrigin":false,"Alias":""},"time":"2025-08-20T01:37:26+08:00","took":129554}
{"action":"resolve_references_completed","build_git_commit":"","build_go_version":"go1.24.0","build_image_tag":"","build_wv_version":"1.33.0-dev","len":0,"level":"debug","msg":"resolve references completed in 2.665µs","props":[{"name":"content","isPrimitive":true,"isObject":false,"includeTypeName":false,"refs":null,"objs":null},{"name":"judgement_id","isPrimitive":true,"isObject":false,"includeTypeName":false,"refs":null,"objs":null},{"name":"date","isPrimitive":true,"isObject":false,"includeTypeName":false,"refs":null,"objs":null}],"time":"2025-08-20T01:37:26+08:00","took":2665}
{"action":"search_completed","build_git_commit":"","build_go_version":"go1.24.0","build_image_tag":"","build_wv_version":"1.33.0-dev","level":"debug","msg":"search query completed in 172.955µs","params":{"Filters":null,"ClassName":"Judgement","Pagination":{"Offset":0,"Limit":100,"Autocut":0},"Cursor":null,"Sort":null,"Properties":[{"name":"content","isPrimitive":true,"isObject":false,"includeTypeName":false,"refs":null,"objs":null},{"name":"judgement_id","isPrimitive":true,"isObject":false,"includeTypeName":false,"refs":null,"objs":null},{"name":"date","isPrimitive":true,"isObject":false,"includeTypeName":false,"refs":null,"objs":null}],"NearVector":null,"NearObject":null,"KeywordRanking":null,"HybridSearch":null,"GroupBy":null,"TargetVector":"","TargetVectorCombination":null,"Group":null,"ModuleParams":null,"AdditionalProperties":{"classification":false,"refMeta":false,"vector":false,"vectors":null,"certainty":false,"id":true,"creationTimeUnix":false,"lastUpdateTimeUnix":false,"moduleParams":null,"distance":false,"score":false,"explainScore":false,"isConsistent":false,"group":false,"noProps":false},"ReplicationProperties":null,"Tenant":"","IsRefOrigin":false,"Alias":""},"time":"2025-08-20T01:37:26+08:00","took":172955}
4

I just ran another test and re-uploaded the original data (only a small portion of it). This time, iterator() was able to properly return objects. I’m not sure if the issue was caused by an inconsistent state due to the program being terminated earlier.

I’ll keep updating this thread with any further findings.

1 Like