Hello! I was curious to try out how Product Quantization work. To embed data I use gtr-t5-large model, which creates 768-dimensional vectors. My database stores around 2k of vectors.
My python code to turn PQ on is following:
client.schema.update_config(
"Document",
{
"vectorIndexConfig": {
"pq": {
"enabled": True,
"trainingLimit": 100000,
"segments": 96
}
}
},
)
where Document is my class name.
I’ve got the following confirmation in docker logs:
semantic_search_analysis-weaviate-1 | {"action":"lsm_compaction","class":"Document","index":"document","level":"warning","msg":"compaction halted due to shard READONLY status","path":"data/document/BPZRlHodv3mc/lsm","shard":"BPZRlHodv3mc","time":"2024-01-31T16:27:24Z"}
semantic_search_analysis-weaviate-1 | {"action":"compress","level":"info","msg":"switching to compressed vectors","time":"2024-01-31T16:27:24Z"}
semantic_search_analysis-weaviate-1 | {"action":"compress","level":"info","msg":"vector compression complete","time":"2024-01-31T16:27:26Z"}
After turning PQ on I can see slightly difference in returning distances, but that’s all. Docker memory usage didn’t change and vector size is the same as it was before vector compression (768).
To find out vector size I apply the code:
top_responses = (
client.query.get("Document", properties=["content", "source"])
.with_near_text({"concepts": ["How to use gpu?"]})
.with_additional(["distance", "vector"])
.with_limit(5)
.do()
)
vector = top_responses["data"]["Get"]["Document"][0]["_additional"]["vector"]
print(len(vector))
My questions are:
- Shouldn’t I see any difference in vectors shape? I supposed that original vectors should be replaced with their compressed ones, but I might miss something.
- What changes I am supposed to observe with Product Quantization? Where I should look for it?
I would appreciate your support!