I am using weaviate version [1.24.4].
I have a schema with properties mention below :
“properties”: [
{
“name”: “text”,
“dataType”: [“text”],
“description”: “Text chunks”,
“tokenization”: “word”,
},
{
“name”: “prop 1”,
“dataType”: [“number”],
“description”: “Prop 1”,
“moduleConfig”: { # Module-specific settings
“text2vec-transformers”: {
“skip”: True,
“vectorizePropertyName”: False,
}
},
},
{
“name”: “prop 2”,
“dataType”: [“number”],
“description”: “Prop 2”,
“moduleConfig”: { # Module-specific settings
“text2vec-transformers”: {
“skip”: True,
“vectorizePropertyName”: False,
}
},
},
{
“name”: “prop 3”,
“dataType”: [“number”],
“description”: “Prop 3”,
“moduleConfig”: { # Module-specific settings
“text2vec-transformers”: {
“skip”: True,
“vectorizePropertyName”: False,
}
},
},
{
“name”: “prop 4”,
“dataType”: [“number”],
“description”: “Prop 4”,
“moduleConfig”: { # Module-specific settings
“text2vec-transformers”: {
“skip”: True,
“vectorizePropertyName”: False,
}
},
},
{
“name”: “prop 5”,
“dataType”: [“number”],
“description”: “Prop 5”,
“moduleConfig”: { # Module-specific settings
“text2vec-transformers”: {
“skip”: True,
“vectorizePropertyName”: False,
}
},
},
]
}
I need to filter on Prop 5 and do a vector search on property “text”.
My total number of records are around 30 millions. I ma using default settings and machine infra is 128 CPU and 256 GB Memory.
My query is taking around 2-3 seconds.
Can you please help me with optimisation. Optimisation related to Schema and Weaviate Setup so that I can achieve the performance within few hundred mili seconds.