How to catch errors with dynamic batch import

Support
1

Description

I’m trying to batch import data into a collection like so:

with filings_collection.batch.dynamic() as batch:
    for fc in tqdm(filngs_chunked):
        batch.add_object(
            properties=fc,
        )

I’m having trouble catching errors. I find that there is a number_errors attribute, however this doesn’t tell me which object exactly has failed, or what the error is

Server Setup Information

  • Weaviate Server Version: 1.25.5
  • Deployment Method: embed
  • Multi Node? Number of Running Nodes: No, 1 node
  • Client Language and Version: Python 3.12.4, weaviate version 4.6.5
  • Multitenancy?: No

Any additional Information

2

Hello @md1639,

Welcome to our community! it’s great to have you here :hugs:

You can consider capture more information about errors and failed objects by implementing error handling and logging during the batch process by using client.batch.failed_objects & client.batch.failed_references.

Error Handling →

You will find a good example in the documentation:

failed_objs_b = client.batch.failed_objects # Get failed objects
failed_refs_b = client.batch.failed_references # Get failed references

Additionally, Weaviate Pythin Client, the following exception handling can help raises various error conditions:

  • weaviate.exceptions.WeaviateConnectionError for failed connections.
  • weaviate.exceptions.WeaviateQueryError for failed queries.
  • weaviate.exceptions.WeaviateBatchError for failed batch operations.
  • weaviate.exceptions.WeaviateClosedClientError for operations on a closed client.

You can review this module which defines the exceptions that can be raised by the client library.

Exception handling →