# Inconsistent errors for weaviate batchInsert

**URL:** <https://forum.weaviate.io/t/inconsistent-errors-for-weaviate-batchinsert/3824>\
**Category:** General\
**Created:** [August 28, 2024, 12:15pm UTC](https://forum.weaviate.io/t/inconsistent-errors-for-weaviate-batchinsert/3824 "2024-08-28T12:15:31Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![JK\_Rider](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/jk_rider/32/1671_2.png) [@JK\_Rider](https://forum.weaviate.io/u/JK_Rider)\
**Post date:** [August 28, 2024, 12:15pm UTC](https://forum.weaviate.io/t/inconsistent-errors-for-weaviate-batchinsert/3824/1 "2024-08-28T12:15:31Z")

</div>

I’m currently trying to do a batch insert of several thousand to million vectors for Weaviate cloud, and I am using the following code.

I am occasionally getting the following error {‘message’: ‘Failed to send 53 objects in a batch of 53. Please inspect client.batch.failed\_objects or collection.batch.failed\_objects for the failed objects.’}  
however, both of these are empty.

Is there any way I can improve my error handling to debug the problem? Thank you,

```
     with self.collection.batch.fixed_size(batch_size=1000) as batch:
        for i, data_row in enumerate(property_rows):
            batch.add_object(
                properties=data_row,
                vector={
                    "title_vector": title_vectors[i],
                    "body_vector": body_vectors[i],
                    "keywords_vector": keyword_vectors[i],
                },
                uuid=generate_uuid5(data_row['ext_id'])
            )
        log.info(f"Num Errors: {batch.number_errors}")
        log.error(f"Collection Failed objects outside of loop: {self.collection.batch.failed_objects}")
        log.error(f"Client Failed objects outside of loop: {self.client.batch.failed_objects}")

```

---

<div class="post-metadata">

**Author:** ![Shahin](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/shahin/32/7944_2.png) [@Shahin](https://forum.weaviate.io/u/Shahin)\
**Post date:** [August 28, 2024, 3:12pm UTC](https://forum.weaviate.io/t/inconsistent-errors-for-weaviate-batchinsert/3824/2 "2024-08-28T15:12:38Z")

</div>

Hello @JK_Rider,

I hope you having a lovely week!

Let me share my own code on how I handle error catching and messaging (I added your code bits in the loop instead of mine). Please try it out and see if it helps:

```auto
try: 
    with items.batch.dynamic() as batch:
            for i, data_row in enumerate(property_rows):
            batch.add_object(
                properties=data_row,
                vector={
                    "title_vector": title_vectors[i],
                    "body_vector": body_vectors[i],
                    "keywords_vector": keyword_vectors[i],
                },
                uuid=generate_uuid5(data_row['ext_id'])
            )

    failed_objs_a = items.batch.failed_objects # Get failed objects
    if failed_objs_a:
        print(f"Number of failed objects in the first batch: {len(failed_objs_a)}")
        for i, failed_obj in enumerate(failed_objs_a, 1):
            print(f"Failed object {i}:")
            print(f"Error message: {failed_obj.message}")
    else:
        print("All objects were successfully added.")
except Exception as e:
    print(f"Error during batch import: {e}")
    print(f"Exception details: {str(e)}")
finally:
     client.close()

```

Additionally, I’m a fan of allowing dynamic behavior in batching, but fixed size works well - I recommend using client.batch.dynamic().

---

<div class="post-metadata">

**Author:** ![JK\_Rider](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/jk_rider/32/1671_2.png) [@JK\_Rider](https://forum.weaviate.io/u/JK_Rider)\
**Post date:** [August 28, 2024, 3:15pm UTC](https://forum.weaviate.io/t/inconsistent-errors-for-weaviate-batchinsert/3824/3 "2024-08-28T15:15:57Z")

</div>

Awesome, thanks for the fast response. I’ll give it a try and see if it works.

---

<div class="post-metadata">

**Author:** ![JK\_Rider](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/jk_rider/32/1671_2.png) [@JK\_Rider](https://forum.weaviate.io/u/JK_Rider)\
**Post date:** [August 28, 2024, 6:27pm UTC](https://forum.weaviate.io/t/inconsistent-errors-for-weaviate-batchinsert/3824/4 "2024-08-28T18:27:29Z")

</div>

@Shahin  
Hi Mohamed, Thanks for the assistance I was able to get the following error message.

Query call with protocol GRPC batch failed with message \<AioRpcError of RPC that terminated with:\n\tstatus = StatusCode.UNAVAILABLE\n\tdetails = “sendmsg: Broken pipe (32)”\n\tdebug\_error\_string = “UNKNOWN:Error received from peer {created\_time:“2024-08-28T18:23:59.686140478+00:00”, grpc\_status:14, grpc\_message:“sendmsg: Broken pipe (32)”}”\n\>.

Do you happen to have any insight on this error.

If Additional Insights helps: This is running in AWS Cloud, this error does not happen on my local desktop.

---

<div class="post-metadata">

**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\
**Post date:** [August 28, 2024, 8:31pm UTC](https://forum.weaviate.io/t/inconsistent-errors-for-weaviate-batchinsert/3824/5 "2024-08-28T20:31:13Z")

</div>

hi @JK_Rider !!

Can you paste the full stacktrace?

This can help us identify where this error happens in the code. This error message indicates a connection being blocked or timing out.

---

<div class="post-metadata">

**Author:** ![JK\_Rider](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/jk_rider/32/1671_2.png) [@JK\_Rider](https://forum.weaviate.io/u/JK_Rider)\
**Post date:** [August 28, 2024, 8:57pm UTC](https://forum.weaviate.io/t/inconsistent-errors-for-weaviate-batchinsert/3824/6 "2024-08-28T20:57:09Z")

</div>

Hi @DudaNogueira  
The error was printed out from this part of the code @Shahin , provided. if failed\_objs\_a:  
print(f"Number of failed objects in the first batch: {len(failed\_objs\_a)}“)  
for i, failed\_obj in enumerate(failed\_objs\_a, 1):  
print(f"Failed object {i}:”)  
**print(f"Error message: {failed\_obj.message}")**

If further information is needed, I’m using the following code to connect to Weaviate Cloud.

client = weaviate.connect\_to\_wcs(  
cluster\_url=self.url,  
auth\_credentials=weaviate.auth.AuthApiKey(self.api\_key),  
additional\_config=AdditionalConfig(  
timeout=Timeout(init=60, query=120, insert=720) # Values in seconds  
)  
the error will occasionally also be

Error message: WeaviateBatchError(‘Query call with protocol GRPC batch failed with message \<AioRpcError of RPC that terminated with:\n\tstatus = StatusCode.UNAVAILABLE\n\tdetails = “recvmsg:Connection reset by peer”\n\tdebug\_error\_string = “UNKNOWN:Error received from peer {created\_time:“2024-08-28T23:03:28.550055837+00:00”, grpc\_status:14, grpc\_message:“recvmsg:Connection reset by peer”}”\n\>.’)

---

<div class="post-metadata">

**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\
**Post date:** [August 29, 2024, 6:53pm UTC](https://forum.weaviate.io/t/inconsistent-errors-for-weaviate-batchinsert/3824/7 "2024-08-29T18:53:10Z")

</div>

Hi!

Can you try setting the batch size to a smaller number and then experiment by increasing it?

for example:

```python
with self.collection.batch.fixed_size(batch_size=50) as batch:
...

```