Size of file changing during insert and after sometime

Support
1

Hi Team,

the size of the file during the inserts are going on is around 860MB, and after that it changed to 293 MB.

Can you please let us know the reason for this behavior?

63.9M embeddinganalysisvectorbatchdbtest_UDpOQ5Ngovpl.hnsw.commitlog.d
4.0K embeddinganalysisvectorbatchdbtest_UDpOQ5Ngovpl.indexcount
4.0K embeddinganalysisvectorbatchdbtest_UDpOQ5Ngovpl.proplengths
4.0K embeddinganalysisvectorbatchdbtest_UDpOQ5Ngovpl.proplengths.bak
4.0K embeddinganalysisvectorbatchdbtest_UDpOQ5Ngovpl.version
860.6M embeddinganalysisvectorbatchdbtest_UDpOQ5Ngovpl_lsm
63.9M embeddinganalysisvectorbatchdbtest_UDpOQ5Ngovpl.hnsw.commitlog.d
4.0K embeddinganalysisvectorbatchdbtest_UDpOQ5Ngovpl.indexcount
4.0K embeddinganalysisvectorbatchdbtest_UDpOQ5Ngovpl.proplengths
4.0K embeddinganalysisvectorbatchdbtest_UDpOQ5Ngovpl.proplengths.bak
4.0K embeddinganalysisvectorbatchdbtest_UDpOQ5Ngovpl.version
293.0M embeddinganalysisvectorbatchdbtest_UDpOQ5Ngovpl_lsm

2

Hi @kranthi_kumar !

This is probably because of Segment Compaction. from here:

  1. There is an async background process called Compaction that optimizes existing segments. It can merge two small segments into a single larger segment and remove redundant data as part of the process.

Let me know if that helps :slight_smile: