below is the code I am using to to add objects to a class.
splitter = RecursiveCharacterTextSplitter(
chunk_size=3000,
chunk_overlap=0
)
splitted_docs_list = splitter.split_documents(encrypted_docs)
for doc_content in splitted_docs_list:
doc_object = {
"content": doc_content.page_content,
"source": doc_content.metadata['source']
}
client.data_object.create(data_object=doc_object, class_name="Data")
Sometimes OpenAI returns an error
WeaviateBaseError("Creating object! Unexpected status code: 500, with response body: {'error': [{'message': 'update vector: unmarshal response body: json: cannot unmarshal number into Go struct field openAIApiError.error.code of type string'}]}.")
What might be causing this error ? Any way to view the error returned by openAI ?
Hi @ananthan-123
That’s strange.
Will it happens with a smaller chunk_size?
What version are you running?
Have you triede with the python client v4? (Altough, this may not be related to the client)
And finally, can you create a complete script that reproduces this with a content you could share?
Thanks!
I tried to run it today and I am not getting the error. I’ll update if it gives error again.
1 Like
This smells like openai returning some unmapped error, so Weaviate couldn’t use it as response to your query.
My suggestion is to use pyv4 (it really improves performance, specially for ingestion) and use batch, making sure to properly handle errors, like so:
Did you have any luck fixing this? Been trying for the last two weeks
For me it happens in any size
hi @manuharari !
Your issue is a different one, check the answer here:
Hi @manuharari !!
I believe this issue is related to this PR:
Note that this PR was backported only to 1.28.15:
I noticed you are using a really old version, 1.26.0
So I suggest you to upgrade path:
1.26.0 → 1.26.latest → restart → 1.27.latest → restart → 1.28.latest
in between each restart, wait all the data get loaded and the cluster stabilized.
Or, of your data is not too big, you can copy your data over to a new cluster running the latest version or the target version.
Let me know …