How to insert to two schemas in a transaction

Support
1

Consider the following psuedo schemas for a Document and its corresponding chunks.

Document
   - title
   - link

DocChunk
   - name
   - document_id

The idea being that a document is chunked and indexed, and we also store a vector for the document itself. The DocChunks therefore have a reference back to document_id, basically a foreign key.

My question: is it possible to do this transactionally? The steps are:

  1. Insert a vector do Document table; store the ID.
  2. Chunk the document.
  3. Insert chunks into DocChunk, with the ID back to Document.

What would happen tho if the chunk insert failed? Is relational DB, could just wrap this all in a transactional block. Is anything like this possible in Weaviate? If not, do you have any examples of patterns around this?

PS - I see this is similar to Verba but didn’t see how Verba addressed this: https://github.com/weaviate/Verba/blob/main/goldenverba/ingestion/init_schema.py

1 Like
2

Hi @Adam_Hughes,
Weaviate doesn’t provide support for transactions at the moment.

If an insert fails, then Weaviate will return an error.
Which you can use to react to the situation in your code.

I hope this helps.

1 Like
3

Is it in the roadmap (cascading deletes would also be useful) or is this trending too close to a traditional database and would be difficult to support in weaviate’s high-availability design?

4

Hey Adam,
It is not something we have planned at the moment.

Feel free to create a github issue on our repo and share it here, so that others might upvote it :wink: