WeaviateBatchError: Query call with protocol GRPC batch failed with message Socket closed

Support
1

Description

I am using following code to insert data in a collection. This code was working till yesterday. Today it is giving following error message.
I have tried updating the weaviate python client to latest version, but the error remains same. Can you please help me resolve this issue?

#add data to crosswalk collection
base_collection = client.collections.get("Competency_model_fw_1")
collection = client.collections.get("Competency_mapping_crosswalk_fw_test")
objects = list()
for idx, row in df.iterrows():
    properties = {
        "tx_comp_name": row["target_title"],
        "tx_comp_desc": row["target_description"]
    }
    base_object = base_collection.query.fetch_objects(
                                filters=wvc.query.Filter.by_property("fw_comp_code").equal(row["target_taxonomy_code_id"]) &
                                    wvc.query.Filter.by_property("tx_comp_name").equal(row["target_title"]),
                                limit=1
                            )
    ref_uuid = base_object.objects[0].uuid

    object = wvc.data.DataObject(properties=properties,
                                 references={"has_code": ref_uuid})
    objects.append(object)

collection.data.insert_many(objects)
print('done')

I am getting following error:
{
“name”: “WeaviateBatchError”,
“message”: “Query call with protocol GRPC batch failed with message Socket closed.”,
“stack”: "---------------------------------------------------------------------------
_InactiveRpcError Traceback (most recent call last)
File ~/code/RAG/crosswalk/venv-crosswalk/lib/python3.8/site-packages/weaviate/collections/batch/grpc_batch_objects.py:137, in _BatchGRPC.__send_batch(self, batch, timeout)
136 res: batch_pb2.BatchObjectsReply
→ 137 res, _ = self._connection.grpc_stub.BatchObjects.with_call(
138 batch_pb2.BatchObjectsRequest(
139 objects=batch,
140 consistency_level=self._consistency_level,
141 ),
142 metadata=metadata,
143 timeout=timeout,
144 )
146 objects: Dict[int, str] = {}

File ~/code/RAG/crosswalk/venv-crosswalk/lib/python3.8/site-packages/grpc/_channel.py:1193, in _UnaryUnaryMultiCallable.with_call(self, request, timeout, metadata, credentials, wait_for_ready, compression)
1187 (
1188 state,
1189 call,
1190 ) = self._blocking(
1191 request, timeout, metadata, credentials, wait_for_ready, compression
1192 )
→ 1193 return _end_unary_response_blocking(state, call, True, None)

File ~/code/RAG/crosswalk/venv-crosswalk/lib/python3.8/site-packages/grpc/_channel.py:1005, in _end_unary_response_blocking(state, call, with_call, deadline)
1004 else:
→ 1005 raise _InactiveRpcError(state)

_InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
\tstatus = StatusCode.UNAVAILABLE
\tdetails = "Socket closed"
\tdebug_error_string = "UNKNOWN:Error received from peer {created_time:"2024-05-20T10:11:07.889871503+05:30", grpc_status:14, grpc_message:"Socket closed"}"

During handling of the above exception, another exception occurred:

WeaviateBatchError Traceback (most recent call last)
Cell In[6], line 28
21 objects.append(object)
23 # collection.data.insert(
24 # properties=properties, # A dictionary with the properties of the object
25 # # uuid=obj_uuid, # A UUID for the object
26 # references={"has_code": ref_uuid}, # e.g. {"hasCategory": "583876f3-e293-5b5b-9839-03f455f14575"}
27 # )
—> 28 collection.data.insert_many(objects)
29 print(‘done’)

File ~/code/RAG/crosswalk/venv-crosswalk/lib/python3.8/site-packages/weaviate/collections/data.py:413, in _DataCollection.insert_many(self, objects)
393 def insert_many(
394 self,
395 objects: Sequence[Union[Properties, DataObject[Properties, Optional[ReferenceInputs]]]],
396 ) → BatchObjectReturn:
397 """Insert multiple objects into the collection.
398
399 Arguments:
(…)
411 If every object in the batch fails to be inserted. The exception message contains details about the failure.
412 """
→ 413 return self._batch_grpc.objects(
414 [
415 (
416 _BatchObject(
417 collection=self.name,
418 vector=obj.vector,
419 uuid=str(obj.uuid if obj.uuid is not None else uuid_package.uuid4()),
420 properties=cast(dict, obj.properties),
421 tenant=self._tenant,
422 references=obj.references,
423 )
424 if isinstance(obj, DataObject)
425 else _BatchObject(
426 collection=self.name,
427 vector=None,
428 uuid=str(uuid_package.uuid4()),
429 properties=cast(dict, obj),
430 tenant=self._tenant,
431 references=None,
432 )
433 )
434 for obj in objects
435 ],
436 timeout=self._connection.timeout_config.insert,
437 )

File ~/code/RAG/crosswalk/venv-crosswalk/lib/python3.8/site-packages/weaviate/collections/batch/grpc_batch_objects.py:97, in _BatchGRPC.objects(self, objects, timeout)
94 weaviate_objs = self.__grpc_objects(objects)
96 start = time.time()
—> 97 errors = self.__send_batch(weaviate_objs, timeout=timeout)
98 elapsed_time = time.time() - start
100 if len(errors) == len(weaviate_objs):
101 # Escape sequence (backslash) not allowed in expression portion of f-string prior to Python 3.12: pylance

File ~/code/RAG/crosswalk/venv-crosswalk/lib/python3.8/site-packages/weaviate/collections/batch/grpc_batch_objects.py:151, in _BatchGRPC.__send_batch(self, batch, timeout)
149 return objects
150 except grpc.RpcError as e:
→ 151 raise WeaviateBatchError(e.details())

WeaviateBatchError: Query call with protocol GRPC batch failed with message Socket closed."
}

Server Setup Information

  • Weaviate Server Version: version: ‘3.4’
  • Deployment Method: docker
  • Client Language and Version: python weaviate-client-4.5.4
1 Like
2

hi!

Do you see any outstanding logs on server?

This error message indicates that the server closed the connection from it’s side.

So getting some logs when this is happening in the server is interesting information to investigate it.

Thanks!