Compaction Errors

Support
1

Hi all,

I am seeing a continuous stream of compaction errors in the log stream. Is this normal? Does it have any affect on the performance?

ERRO[0679] compaction failed                             action=lsm_compaction class=Document error="replace compacted segments: precompute segment meta: init bloom filter for secondary index at 0: a secondary bloom filter already exists with path /root/vectorstore_weaviate/weaviate_data/data/weaviate/document_kxlWAyuQqVRN_lsm/objects/segment-1689676389149982374.secondary.0.bloom.tmp" index=document path=/root/vectorstore_weaviate/weaviate_data/data/weaviate/document_kxlWAyuQqVRN_lsm/objects shard=kxlWAyuQqVRN
ERRO[0685] compaction failed                             action=lsm_compaction class=Document error="replace compacted segments: precompute segment meta: init bloom filter for secondary index at 0: a secondary bloom filter already exists with path /root/vectorstore_weaviate/weaviate_data/data/weaviate/document_kxlWAyuQqVRN_lsm/objects/segment-1689676389149982374.secondary.0.bloom.tmp" index=document path=/root/vectorstore_weaviate/weaviate_data/data/weaviate/document_kxlWAyuQqVRN_lsm/objects shard=kxlWAyuQqVRN
ERRO[0689] compaction failed                             action=lsm_compaction class=Document error="replace compacted segments: precompute segment meta: init bloom filter for secondary index at 0: a secondary bloom filter already exists with path /root/vectorstore_weaviate/weaviate_data/data/weaviate/document_kxlWAyuQqVRN_lsm/objects/segment-1689676389149982374.secondary.0.bloom.tmp" index=document path=/root/vectorstore_weaviate/weaviate_data/data/weaviate/document_kxlWAyuQqVRN_lsm/objects shard=kxlWAyuQqVRN
ERRO[0694] compaction failed                             action=lsm_compaction class=Document error="replace compacted segments: precompute segment meta: init bloom filter for secondary index at 0: a secondary bloom filter already exists with path /root/vectorstore_weaviate/weaviate_data/data/weaviate/document_kxlWAyuQqVRN_lsm/objects/segment-1689676389149982374.secondary.0.bloom.tmp" index=document path=/root/vectorstore_weaviate/weaviate_data/data/weaviate/document_kxlWAyuQqVRN_lsm/objects shard=kxlWAyuQqVRN
ERRO[0699] compaction failed                             action=lsm_compaction class=Document error="replace compacted segments: precompute segment meta: init bloom filter for secondary index at 0: a secondary bloom filter already exists with path /root/vectorstore_weaviate/weaviate_data/data/weaviate/document_kxlWAyuQqVRN_lsm/objects/segment-1689676389149982374.secondary.0.bloom.tmp" index=document path=/root/vectorstore_weaviate/weaviate_data/data/weaviate/document_kxlWAyuQqVRN_lsm/objects shard=kxlWAyuQqVRN
ERRO[0703] compaction failed                             action=lsm_compaction class=Document error="replace compacted segments: precompute segment meta: init bloom filter for secondary index at 0: a secondary bloom filter already exists with path /root/vectorstore_weaviate/weaviate_data/data/weaviate/document_kxlWAyuQqVRN_lsm/objects/segment-1689676389149982374.secondary.0.bloom.tmp" index=document path=/root/vectorstore_weaviate/weaviate_data/data/weaviate/document_kxlWAyuQqVRN_lsm/objects shard=kxlWAyuQqVRN
ERRO[0708] compaction failed                             action=lsm_compaction class=Document error="replace compacted segments: precompute segment meta: init bloom filter for secondary index at 0: a secondary bloom filter already exists with path /root/vectorstore_weaviate/weaviate_data/data/weaviate/document_kxlWAyuQqVRN_lsm/objects/segment-1689676389149982374.secondary.0.bloom.tmp" index=document path=/root/vectorstore_weaviate/weaviate_data/data/weaviate/document_kxlWAyuQqVRN_lsm/objects shard=kxlWAyuQqVRN
ERRO[0712] compaction failed                             action=lsm_compaction class=Document error="replace compacted segments: precompute segment meta: init bloom filter for secondary index at 0: a secondary bloom filter already exists with path /root/vectorstore_weaviate/weaviate_data/data/weaviate/document_kxlWAyuQqVRN_lsm/objects/segment-1689676389149982374.secondary.0.bloom.tmp" index=document path=/root/vectorstore_weaviate/weaviate_data/data/weaviate/document_kxlWAyuQqVRN_lsm/objects shard=kxlWAyuQqVRN
1 Like
2

Hi! I have asked about this with our team.

Is there any compaction success messages or all of them fails?

3

I have a similar compaction error issue. All of them fail, never a success. For days on end.

Docker image: semitechnologies/weaviate:1.21.1

Schema for Matches:
[
{“name”: “client_id”, “dataType”: [“int”]},
{“name”: “category_id”, “dataType”: [“int”]},
{“name”: “ctn_id”, “dataType”: [“int”]},
{“name”: “is_description_only”, “dataType”: [“boolean”]},
{“name”: “is_material_ctn”, “dataType”: [“boolean”]},
{“name”: “created_at”, “dataType”: [“date”]},
]

Error message:
{“action”:“lsm_compaction”,“class”:“Matches”,“error”:“replace compacted segments: precompute segment meta: a bloom filter already exists with path /var/lib/weaviate/matches_V0cXxiU61Q6I_lsm/property_is_description_only/segment-1697059248081165705.bloom.tmp”,“index”:“matches”,“level”:“error”,“msg”:“compaction failed”,“path”:“/var/lib/weaviate/matches_V0cXxiU61Q6I_lsm/property_is_description_only”,“shard”:“V0cXxiU61Q6I”,“time”:“2023-10-18T08:42:43Z”}

{“action”:“lsm_compaction”,“class”:“Matches”,“error”:“replace compacted segments: precompute segment meta: a bloom filter already exists with path /var/lib/weaviate/matches_V0cXxiU61Q6I_lsm/property_is_description_only/segment-1697059248081165705.bloom.tmp”,“index”:“matches”,“level”:“error”,“msg”:“compaction failed”,“path”:“/var/lib/weaviate/matches_V0cXxiU61Q6I_lsm/property_is_description_only”,“shard”:“V0cXxiU61Q6I”,“time”:“2023-10-18T08:42:46Z”}

{“action”:“lsm_compaction”,“class”:“Matches”,“error”:“replace compacted segments: precompute segment meta: a bloom filter already exists with path /var/lib/weaviate/matches_V0cXxiU61Q6I_lsm/property_is_description_only/segment-1697059248081165705.bloom.tmp”,“index”:“matches”,“level”:“error”,“msg”:“compaction failed”,“path”:“/var/lib/weaviate/matches_V0cXxiU61Q6I_lsm/property_is_description_only”,“shard”:“V0cXxiU61Q6I”,“time”:“2023-10-18T08:42:49Z”}

{“action”:“lsm_compaction”,“class”:“Matches”,“error”:“replace compacted segments: precompute segment meta: a bloom filter already exists with path /var/lib/weaviate/matches_V0cXxiU61Q6I_lsm/property_is_description_only/segment-1697059248081165705.bloom.tmp”,“index”:“matches”,“level”:“error”,“msg”:“compaction failed”,“path”:“/var/lib/weaviate/matches_V0cXxiU61Q6I_lsm/property_is_description_only”,“shard”:“V0cXxiU61Q6I”,“time”:“2023-10-18T08:42:52Z”}

{“action”:“lsm_compaction”,“class”:“Matches”,“error”:“replace compacted segments: precompute segment meta: a bloom filter already exists with path /var/lib/weaviate/matches_V0cXxiU61Q6I_lsm/property_is_description_only/segment-1697059248081165705.bloom.tmp”,“index”:“matches”,“level”:“error”,“msg”:“compaction failed”,“path”:“/var/lib/weaviate/matches_V0cXxiU61Q6I_lsm/property_is_description_only”,“shard”:“V0cXxiU61Q6I”,“time”:“2023-10-18T08:42:55Z”}