Description
I have created a sanbox in Weaviate Cloud to test functionality before putting the vectorstore in production. I am testing openai embeddings:
“text-embedding-3-small” and “text-embedding-3-large”.
I tried to change dimensions of “text-embedding-3-large” to 256, to see how sould retrieval go, but I got error below. “distance between entrypoint and query node: 256 vs 1536”
Can I change dimensions in Sandbox? Or do I have to setup something additional when I am doing hybrid or smilarity search?
This is my setup:
client = weaviate.connect_to_weaviate_cloud(
cluster_url=weaviate_url,
auth_credentials=Auth.api_key(weaviate_api_key),
headers={"X-OpenAI-Api-Key": openai_api_key},
# headers={"X-Google-Studio-Api-Key": google_studio_api_key},
)
vectorizer_config = Configure.Vectorizer.text2vec_openai(
model=model,
model_version="3",
dimensions=dimensions,
)
client.collections.create(
name=collection_name,
vectorizer_config=vectorizer_config,
vector_index_config=Configure.VectorIndex.hnsw(
distance_metric=VectorDistances.COSINE
),
properties=properties,
)
I have changed now everything to model: str = “text-embedding-3-small”, and dimensions: int = 1536, and now retrieval works fine…
Second somehow related question: Does, Google’s “text-embedding-004” embedding works in sandbox? I had earlier some trouble setting it up…
Third question, are this integrations necessary (since this integrations don’t support Google’s embeddings):
# integrations = [
# Integrations.openai(
# api_key=openai_api_key,
# requests_per_minute_embeddings=3000,
# tokens_per_minute_embeddings=1000000,
# ),
# ]
# client.integrations.configure(integrations)
Server Setup Information
- Weaviate Server Version:
- Deployment Method: sanbox in Weaviate Cloud
- Multi Node? Number of Running Nodes: no
- Client Language and Version: Python v4
- Multitenancy?: No
Any additional Information
"StatusCode.UNKNOWN\n\tdetails = “explorer: get class: vector search: object vector search at index …: vector search: knn search: distance between entrypoint and query node: 256 vs 1536: vector lengths don't match”\n\tdebug_error_string = "UNKNOWN:Error received from peer "