I am getting hnsw_vector_cache_prefill frequently

Hey @2020ashish, as per thread Metadata properties - #3 by 2020ashish,

My answer based on your last message in this thread, since you have two files and are batching each separately, I would recommend going with a cross-reference approach. It will work well in your case, especially when you need to query and filter metadata. I wouldn’t go for a boolean property (is_new) unless there’s a compelling reason for it.

With two collections and one cross-reference, performance should not be an issue unless your queries become unusually complex. Based on my experience with similar use cases, a cross-reference is a solution for this scenario. This approach is clean, and maintainable for batching workflow.

Does that help?