Weaviate crashing because of following error, can someone help on this

Support
1 
panic: runtime error: slice bounds out of range [:1066] with capacity 922



goroutine 132 [running]:

github.com/weaviate/weaviate/usecases/byteops.(*ReadWriter).CopyBytesFromBuffer(...)

	/go/src/github.com/weaviate/weaviate/usecases/byteops/byteops.go:77

github.com/weaviate/weaviate/adapters/repos/db/lsmkv/segmentindex.(*DiskTree).readNode(0x4002243c00?, {0xffff6ceaf320, 0x39a, 0x39a})

	/go/src/github.com/weaviate/weaviate/adapters/repos/db/lsmkv/segmentindex/disk_tree.go:106 +0x448

github.com/weaviate/weaviate/adapters/repos/db/lsmkv/segmentindex.(*DiskTree).AllKeys(0x4002eba930)

	/go/src/github.com/weaviate/weaviate/adapters/repos/db/lsmkv/segmentindex/disk_tree.go:180 +0x8c

github.com/weaviate/weaviate/adapters/repos/db/lsmkv.(*segment).computeAndStoreBloomFilter(0x40031203c0, {0x4003200770, 0x70})

	/go/src/github.com/weaviate/weaviate/adapters/repos/db/lsmkv/segment_bloom_filters.go:88 +0x38

github.com/weaviate/weaviate/adapters/repos/db/lsmkv.(*segment).initBloomFilter(0x40031203c0)

	/go/src/github.com/weaviate/weaviate/adapters/repos/db/lsmkv/segment_bloom_filters.go:75 +0x90

github.com/weaviate/weaviate/adapters/repos/db/lsmkv.(*segment).initBloomFilters(0x40031203c0, 0xffff6ceae000?)

	/go/src/github.com/weaviate/weaviate/adapters/repos/db/lsmkv/segment_bloom_filters.go:38 +0x28

github.com/weaviate/weaviate/adapters/repos/db/lsmkv.newSegment({0x4003200310, 0x6d}, {0x181bc18?, 0x4002b638f0}, 0x0, 0x0?, 0x1, 0x1, 0x1)

	/go/src/github.com/weaviate/weaviate/adapters/repos/db/lsmkv/segment.go:150 +0x738

github.com/weaviate/weaviate/adapters/repos/db/lsmkv.newSegmentGroup({0x181bc18, 0x4002b638f0}, 0x0, {0x17fca40, 0x40031d4840}, {{0x40030facd0, 0x4e}, {0x13f7698, 0xa}, 0x0, ...})

	/go/src/github.com/weaviate/weaviate/adapters/repos/db/lsmkv/segment_group.go:140 +0x654

github.com/weaviate/weaviate/adapters/repos/db/lsmkv.NewBucket({0x1805c08, 0x2410b20}, {0x40030facd0, 0x4e}, {0x40030e9280, 0x36}, {0x181bc18?, 0x4002b638f0?}, 0x0, {0x17fca40, ...}, ...)

	/go/src/github.com/weaviate/weaviate/adapters/repos/db/lsmkv/bucket.go:151 +0x2b4

github.com/weaviate/weaviate/adapters/repos/db/lsmkv.(*Store).CreateOrLoadBucket(0x40031d47e0, {0x1805c08, 0x2410b20}, {0x40031bf3b0, 0x13}, {0x4002f54960, 0x4, 0x6})

	/go/src/github.com/weaviate/weaviate/adapters/repos/db/lsmkv/store.go:122 +0xec

github.com/weaviate/weaviate/adapters/repos/db.(*Shard).createPropertyValueIndex(0x40031201e0, {0x1805c08, 0x2410b20}, 0x40026e97a0)

	/go/src/github.com/weaviate/weaviate/adapters/repos/db/shard.go:660 +0x4b8

github.com/weaviate/weaviate/adapters/repos/db.(*Shard).createPropertyIndex.func1()

	/go/src/github.com/weaviate/weaviate/adapters/repos/db/shard.go:609 +0x48

golang.org/x/sync/errgroup.(*Group).Go.func1()

	/go/pkg/mod/golang.org/x/sync@v0.3.0/errgroup/errgroup.go:75 +0x58

created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 16

	/go/pkg/mod/golang.org/x/sync@v0.3.0/errgroup/errgroup.go:72 +0x98
2

Hi @Kavali_Kranthi_Kumar !

Welcome to our community.

Does this happend when you perform any action or on start up?

Also, what is the version you are using?

3

To me it happens when trying to fetch all objects of my class Chunk using offset 10000, or trying to use a limit higher than 10000:

Code:

chunks_query_response_1 = (
client.query
.get(“Chunk”,“post_id”)
.with_limit(10000)
.do()
)

chunks_query_response_2 = (
client.query
.get(“Chunk”,“post_id”)
# .with_limit(10000)
.with_offset(10000)
.do()
)

Schema:

class_Document= {
“class”: “Document”,
“description”: “Represents a document with metadata and associated text chunks.”,
“properties”: [
{
“name”: “tags”,
“dataType”: [“string”],
“description”: “Tags from WordPress”
},
{
“name”: “categories”,
“dataType”: [“string”],
“description”: “Categories from WordPress”
},
{
“name”: “postUpdatedAt”,
“dataType”: [“date”],
“description”: “WordPress post last updated timestamp”
},
{
“name”: “title”,
“dataType”: [“string”],
“description”: “Title of the post”
},
{
“name”: “author”,
“dataType”: [“string”],
“description”: “author of the post”
},
{
“name”: “permalink”,
“dataType”: [“string”],
“description”: “Link of the post”
},
{
“name”: “post_id”,
“dataType”: [“int”],
“description”: “WordPress ID for the post”
},
{
“name”: “post_type”,
“dataType”: [“string”],
“description”: “Type of the post, e.g., report, blog, etc.”
}
]
}

class_Chunk= {
“class”: “Chunk”,
“description”: “Represents a chunk of text from a document.”,
“vectorizer”: “text2vec-openai”,
“moduleConfig”: {
“text2vec-openai”: {
“model”: “text-embedding-3-large”,
“type”: “text”,
“dimensions”:1024
}
},
“properties”: [
{
“name”: “text”,
“dataType”: [“text”],
“description”: “Plain text of the chunk”,
“moduleConfig”: {
“text2vec-openai”: {
“skip”: False
}
}
},
{
“name”: “post_id”,
“dataType”: [“int”],
“description”: “WordPress ID for the post”,
“moduleConfig”: {
“text2vec-openai”: {
“skip”: True
}
}
}
]
}

4

HI @J_Marcos_del_Ser ! Welcome to our community :hugs:

Does it also happens with the new python client v4?

the v3 client only uses REST, while v4 uses GRPC and that brings a huge improvement in connectivity.

This may be reaching a connection limit

Let me know if this helps!

5

I am also getting the similar error with Azure File share and as per my analysis and community this issue happens with Azure file share.
( here and here,)

1 Like
6

Hi!

Please, can you create a new forum thread ?

Also, provide the request infos, specially server version.

Thanks!