Error in Batch Addition

[Indrajit_CS]

Hey Everyone!

Read the discussion above. Looks like I am hitting the same issue. Everything was working in our prod till yesterday morning !

Now I am getting this error for any file/url uploads. It’s a single file upload but still hanging.

Here is the lambda function error . We are running Weaviate on docker container.

[ERROR] ConnectionError: Batch was not added to weaviate.
Traceback (most recent call last):
  File "/var/task/lambda_function.py", line 61, in lambda_handler
    vectorstore.add_documents(docs[i:i+500])
  File "/opt/python/langchain/schema/vectorstore.py", line 122, in add_documents
    return self.add_texts(texts, metadatas, **kwargs)
  File "/opt/python/langchain/vectorstores/weaviate.py", line 145, in add_texts
    with self._client.batch as batch:
  File "/opt/python/weaviate/batch/crud_batch.py", line 1646, in __exit__
    self.flush()
  File "/opt/python/weaviate/batch/crud_batch.py", line 1252, in flush
    self._send_batch_requests(force_wait=True)
  File "/opt/python/weaviate/batch/crud_batch.py", line 1151, in _send_batch_requests
    response_objects, nr_objects = done_future.result()
  File "/var/lang/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/var/lang/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/var/runtime/awslambdaric/bootstrap.py", line 186, in handle_event_request
    response = request_handler(event, lambda_context)
  File "/var/task/lambda_function.py", line 61, in lambda_handler
    vectorstore.add_documents(docs[i:i+500])
  File "/opt/python/langchain/schema/vectorstore.py", line 122, in add_documents
    return self.add_texts(texts, metadatas, **kwargs)
  File "/opt/python/langchain/vectorstores/weaviate.py", line 145, in add_texts
    with self._client.batch as batch:
  File "/opt/python/weaviate/batch/crud_batch.py", line 1646, in __exit__
    self.flush()
  File "/opt/python/weaviate/batch/crud_batch.py", line 1252, in flush
    self._send_batch_requests(force_wait=True)
  File "/opt/python/weaviate/batch/crud_batch.py", line 1151, in _send_batch_requests
    response_objects, nr_objects = done_future.result()
  File "/var/lang/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/var/lang/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/var/runtime/awslambdaric/bootstrap.py", line 186, in handle_event_request
    response = request_handler(event, lambda_context)
  File "/var/task/lambda_function.py", line 61, in lambda_handler
    vectorstore.add_documents(docs[i:i+500])
  File "/opt/python/langchain/schema/vectorstore.py", line 122, in add_documents
    return self.add_texts(texts, metadatas, **kwargs)
  File "/opt/python/langchain/vectorstores/weaviate.py", line 145, in add_texts
    with self._client.batch as batch:
  File "/opt/python/weaviate/batch/crud_batch.py", line 1646, in __exit__
    self.flush()
  File "/opt/python/weaviate/batch/crud_batch.py", line 1252, in flush
    self._send_batch_requests(force_wait=True)
  File "/opt/python/weaviate/batch/crud_batch.py", line 1151, in _send_batch_requests
    response_objects, nr_objects = done_future.result()
  File "/var/lang/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/var/lang/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/var/runtime/awslambdaric/bootstrap.py", line 186, in handle_event_request
    response = request_handler(event, lambda_context)
  File "/var/task/lambda_function.py", line 61, in lambda_handler
    vectorstore.add_documents(docs[i:i+500])
  File "/opt/python/langchain/schema/vectorstore.py", line 122, in add_documents
    return self.add_texts(texts, metadatas, **kwargs)
  File "/opt/python/langchain/vectorstores/weaviate.py", line 145, in add_texts
    with self._client.batch as batch:
  File "/opt/python/weaviate/batch/crud_batch.py", line 1646, in __exit__
    self.flush()
  File "/opt/python/weaviate/batch/crud_batch.py", line 1252, in flush
    self._send_batch_requests(force_wait=True)
  File "/opt/python/weaviate/batch/crud_batch.py", line 1151, in _send_batch_requests
    response_objects, nr_objects = done_future.result()
  File "/var/lang/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/var/lang/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/var/runtime/awslambdaric/bootstrap.py", line 186, in handle_event_request
    response = request_handler(event, lambda_context)
  File "/var/task/lambda_function.py", line 61, in lambda_handler
    vectorstore.add_documents(docs[i:i+500])
  File "/opt/python/langchain/schema/vectorstore.py", line 122, in add_documents
    return self.add_texts(texts, metadatas, **kwargs)
  File "/opt/python/langchain/vectorstores/weaviate.py", line 145, in add_texts
    with self._client.batch as batch:
  File "/opt/python/weaviate/batch/crud_batch.py", line 1646, in __exit__
    self.flush()
  File "/opt/python/weaviate/batch/crud_batch.py", line 1252, in flush
    self._send_batch_requests(force_wait=True)
  File "/opt/python/weaviate/batch/crud_batch.py", line 1151, in _send_batch_requests
    response_objects, nr_objects = done_future.result()
  File "/var/lang/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/var/lang/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/var/runtime/awslambdaric/bootstrap.py", line 186, in handle_event_request
    response = request_handler(event, lambda_context)
  File "/var/task/lambda_function.py", line 61, in lambda_handler
    vectorstore.add_documents(docs[i:i+500])
  File "/opt/python/langchain/schema/vectorstore.py", line 122, in add_documents
    return self.add_texts(texts, metadatas, **kwargs)
  File "/opt/python/langchain/vectorstores/weaviate.py", line 145, in add_texts
    with self._client.batch as batch:
  File "/opt/python/weaviate/batch/crud_batch.py", line 1646, in __exit__
    self.flush()
  File "/opt/python/weaviate/batch/crud_batch.py", line 1252, in flush
    self._send_batch_requests(force_wait=True)
  File "/opt/python/weaviate/batch/crud_batch.py", line 1151, in _send_batch_requests
    response_objects, nr_objects = done_future.result()
  File "/var/lang/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/var/lang/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/var/lang/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/opt/python/weaviate/batch/crud_batch.py", line 1099, in _flush_in_thread
    response = self._create_data(
  File "/opt/python/weaviate/batch/crud_batch.py", line 742, in _create_data
    raise RequestsConnectionError("Batch was not added to weaviate.") from conn_err

@DudaNogueira Can you please help on this. This is urgent as our production is down!

Further findings:

When I tested locally I was hitting the same error as the discussion in below thread.

Read timeout because batch size = 1 but it's not

I’m importing some data into a schema, and am getting the following error: ReadTimeout: The 'objects' creation was cancelled because it took longer than the configured timeout of 60s. Try reducing the batch size (currently 1) to a lower value. Aim to on average complete batch request within less than 10s I have set my batch size to 200 currently using with client.batch as batch and batch.batch_size= 200, but am getting this error that says my batch size is 1. I add my objects using: client.bat…