I am currently working on integrating Weaviate to power semantic search and automated diagnostics for our mobile client builds documented at this link, where we need to parse and categorize game mod configurations, client crash dumps, and device-level logs. The goal is to allow our support tools to instantly match user error reports with known configuration conflicts and community patches.
The main issue we are running into is severe query latency spikes when performing hybrid searches over collections that experience high-frequency batch writes. Alongside this, hybrid queries filtered by dynamic attributes like client build numbers and memory limits appear to degrade recall accuracy, often returning mismatched vectors whenever concurrent telemetry ingestion occurs.
Has anyone implemented a similar setup for client diagnostics with Weaviate? I would appreciate guidance on whether we should separate rapid log streams into a staging collection before vectorizing, or if specific HNSW index parameters can better handle this type of sustained write-heavy workload.