Received RST_STREAM with code 2 (Internal server error)

Support
1

Description

We are testing a migration script that creates a backup collection, and attempts migrates all the tenants and data from Collection to Collection_Backup (both collections belong to the same cluster). The iterator always ends up erroring our with RST_STREAM with code 2 (Internal server error) after completing ~10 to ~100 iterations

Here is the pseudo code (typescript) :

for await (const item of collection_src.iterator({ includeVector: true })) {
    
    itemsToInsert.push(objectToInsert);

    if (itemsToInsert.length >= 500) {
      await flush();
      itemsToInsert = [];
    }
}

To further isolate, we also tested with this pseudo script, that only iterates and does nothing else (still fails, though after completing ~ 10x iterations)

for await (const item of collection_src.iterator({ includeVector: true })) {
    console.log(`${iteration++}`)
    continue;
}

Server Setup Information

  • Weaviate Server Version: 1.30.4
  • Deployment Method: hosted on weaviate.io
  • Multi Node? Number of Running Nodes: 1 cluster
  • Client Language and Version: “weaviate-client”: “3.6.0”
  • Multitenancy?: Y

Any additional Information

2

hi @Hugo !

Do you have any server error logs?

Also, do you have any reading on the server usage? How big is the dataset?

Depending on the size, doubling it will take more than double the already used resources, at least initially while the ingestion take place.

But you will probably see something on logs.

Thanks!