Error in Batch Addition

Hey Everyone!

Read the discussion above. Looks like I am hitting the same issue. Everything was working in our prod till yesterday morning !

Now I am getting this error for any file/url uploads. It’s a single file upload but still hanging.

Here is the lambda function error . We are running Weaviate on docker container.

[ERROR] ConnectionError: Batch was not added to weaviate.
Traceback (most recent call last):
  File "/var/task/lambda_function.py", line 61, in lambda_handler
    vectorstore.add_documents(docs[i:i+500])
  File "/opt/python/langchain/schema/vectorstore.py", line 122, in add_documents
    return self.add_texts(texts, metadatas, **kwargs)
  File "/opt/python/langchain/vectorstores/weaviate.py", line 145, in add_texts
    with self._client.batch as batch:
  File "/opt/python/weaviate/batch/crud_batch.py", line 1646, in __exit__
    self.flush()
  File "/opt/python/weaviate/batch/crud_batch.py", line 1252, in flush
    self._send_batch_requests(force_wait=True)
  File "/opt/python/weaviate/batch/crud_batch.py", line 1151, in _send_batch_requests
    response_objects, nr_objects = done_future.result()
  File "/var/lang/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/var/lang/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/var/runtime/awslambdaric/bootstrap.py", line 186, in handle_event_request
    response = request_handler(event, lambda_context)
  File "/var/task/lambda_function.py", line 61, in lambda_handler
    vectorstore.add_documents(docs[i:i+500])
  File "/opt/python/langchain/schema/vectorstore.py", line 122, in add_documents
    return self.add_texts(texts, metadatas, **kwargs)
  File "/opt/python/langchain/vectorstores/weaviate.py", line 145, in add_texts
    with self._client.batch as batch:
  File "/opt/python/weaviate/batch/crud_batch.py", line 1646, in __exit__
    self.flush()
  File "/opt/python/weaviate/batch/crud_batch.py", line 1252, in flush
    self._send_batch_requests(force_wait=True)
  File "/opt/python/weaviate/batch/crud_batch.py", line 1151, in _send_batch_requests
    response_objects, nr_objects = done_future.result()
  File "/var/lang/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/var/lang/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/var/runtime/awslambdaric/bootstrap.py", line 186, in handle_event_request
    response = request_handler(event, lambda_context)
  File "/var/task/lambda_function.py", line 61, in lambda_handler
    vectorstore.add_documents(docs[i:i+500])
  File "/opt/python/langchain/schema/vectorstore.py", line 122, in add_documents
    return self.add_texts(texts, metadatas, **kwargs)
  File "/opt/python/langchain/vectorstores/weaviate.py", line 145, in add_texts
    with self._client.batch as batch:
  File "/opt/python/weaviate/batch/crud_batch.py", line 1646, in __exit__
    self.flush()
  File "/opt/python/weaviate/batch/crud_batch.py", line 1252, in flush
    self._send_batch_requests(force_wait=True)
  File "/opt/python/weaviate/batch/crud_batch.py", line 1151, in _send_batch_requests
    response_objects, nr_objects = done_future.result()
  File "/var/lang/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/var/lang/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/var/runtime/awslambdaric/bootstrap.py", line 186, in handle_event_request
    response = request_handler(event, lambda_context)
  File "/var/task/lambda_function.py", line 61, in lambda_handler
    vectorstore.add_documents(docs[i:i+500])
  File "/opt/python/langchain/schema/vectorstore.py", line 122, in add_documents
    return self.add_texts(texts, metadatas, **kwargs)
  File "/opt/python/langchain/vectorstores/weaviate.py", line 145, in add_texts
    with self._client.batch as batch:
  File "/opt/python/weaviate/batch/crud_batch.py", line 1646, in __exit__
    self.flush()
  File "/opt/python/weaviate/batch/crud_batch.py", line 1252, in flush
    self._send_batch_requests(force_wait=True)
  File "/opt/python/weaviate/batch/crud_batch.py", line 1151, in _send_batch_requests
    response_objects, nr_objects = done_future.result()
  File "/var/lang/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/var/lang/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/var/runtime/awslambdaric/bootstrap.py", line 186, in handle_event_request
    response = request_handler(event, lambda_context)
  File "/var/task/lambda_function.py", line 61, in lambda_handler
    vectorstore.add_documents(docs[i:i+500])
  File "/opt/python/langchain/schema/vectorstore.py", line 122, in add_documents
    return self.add_texts(texts, metadatas, **kwargs)
  File "/opt/python/langchain/vectorstores/weaviate.py", line 145, in add_texts
    with self._client.batch as batch:
  File "/opt/python/weaviate/batch/crud_batch.py", line 1646, in __exit__
    self.flush()
  File "/opt/python/weaviate/batch/crud_batch.py", line 1252, in flush
    self._send_batch_requests(force_wait=True)
  File "/opt/python/weaviate/batch/crud_batch.py", line 1151, in _send_batch_requests
    response_objects, nr_objects = done_future.result()
  File "/var/lang/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/var/lang/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/var/runtime/awslambdaric/bootstrap.py", line 186, in handle_event_request
    response = request_handler(event, lambda_context)
  File "/var/task/lambda_function.py", line 61, in lambda_handler
    vectorstore.add_documents(docs[i:i+500])
  File "/opt/python/langchain/schema/vectorstore.py", line 122, in add_documents
    return self.add_texts(texts, metadatas, **kwargs)
  File "/opt/python/langchain/vectorstores/weaviate.py", line 145, in add_texts
    with self._client.batch as batch:
  File "/opt/python/weaviate/batch/crud_batch.py", line 1646, in __exit__
    self.flush()
  File "/opt/python/weaviate/batch/crud_batch.py", line 1252, in flush
    self._send_batch_requests(force_wait=True)
  File "/opt/python/weaviate/batch/crud_batch.py", line 1151, in _send_batch_requests
    response_objects, nr_objects = done_future.result()
  File "/var/lang/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/var/lang/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/var/lang/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/opt/python/weaviate/batch/crud_batch.py", line 1099, in _flush_in_thread
    response = self._create_data(
  File "/opt/python/weaviate/batch/crud_batch.py", line 742, in _create_data
    raise RequestsConnectionError("Batch was not added to weaviate.") from conn_err

@DudaNogueira Can you please help on this. This is urgent as our production is down!

Further findings:

When I tested locally I was hitting the same error as the discussion in below thread.