How does adding more fields to a document affect performance/memory usage?

General
1

Hi! My company is researching the possibility of using Weaviate for our system, and we have a requirement to do a lot of metadata filtering on our vector searches. We originally has a cross-reference type schema but we know this will make things a lot slower.

We were wondering if things would be better if we denormalized and instead just had each metadata field on each vector object, even if there are repeats.

We don’t necessarily care if it increases our disk requirements, but we aren’t sure how adding more document fields would affect search performance and RAM usage. Can anyone shine any light on this?

2

Hi @Christopher_Mailloux - it will impact the inverted index size, but not impact the vector index, as it will still be one vector per object as you know.

While I’m not an expert in this area, my understanding is that the memory usage is driven very much by the vector index. So adding additional fields on the object shouldn’t really impact the memory requirements much.

It shouldn’t significantly impact performance either, as filtering is very efficient.