I am running in docker with the database using a ZFS dataset through a bind mount into the container that gets automatically snapshotted/backed up.
What are generally accepted good parameters for a weaviate DB?
I am thinking particularly of the recordsize, but also other dataset options that come to mind?
As to recordsize, more basically what page size is weaviate using? Couldn’t find that through searching. And then, is it generally recommended to use a storage page size of a multiple of the weaviate page size due to the typically read-heavy workload or just the page size itself?
We don’t believe we explicitly use a pagesize anywhere - A few places may have 4096 bytes hardcoded, but that they are not performance relevant. But currently we use mmap for most IO access (eg the kernel handles reading files and caching the data) and as far as I can see you cannot set an explizit page size there.
There is an open PR that (optionally) converts all memory access to pread (eg we explicitly read data from files) and there the pageSize setting is “whatever is configured for the system” - but that PR is not close to being merged
