# How to choose the correct \`PERSISTENCE\_HNSW\_MAX\_LOG\_SIZE\`

**URL:** <https://forum.weaviate.io/t/how-to-choose-the-correct-persistence-hnsw-max-log-size/21041>\
**Category:** Support\
**Tags:** developer-experience, technical\
**Created:** [May 14, 2025, 5:47am UTC](https://forum.weaviate.io/t/how-to-choose-the-correct-persistence-hnsw-max-log-size/21041 "2025-05-14T05:47:05Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![d\_khlebokazov](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/d_khlebokazov/32/3656_2.png) [@d\_khlebokazov](https://forum.weaviate.io/u/d_khlebokazov)\
**Post date:** [May 14, 2025, 5:47am UTC](https://forum.weaviate.io/t/how-to-choose-the-correct-persistence-hnsw-max-log-size/21041/1 "2025-05-14T05:47:05Z")

</div>

We want to better configure our weaviate cluster to achieve best performance. Here is such configuration in weaviate environments as `PERSISTENCE_HNSW_MAX_LOG_SIZE` and we want to know how to configure it depending on our vector size (512).

From the official documentation we get these explanation:

* * *

### Database parameters for HNSW[​](https://weaviate.io/developers/weaviate/config-refs/schema/vector-index#database-parameters-for-hnsw)

Note that some database-level parameters are available to configure HNSW indexing behavior.

- `PERSISTENCE_HNSW_MAX_LOG_SIZE` is a database-level parameter that sets the maximum size of the HNSW write-ahead-log. The default value is `500MiB`.

Increase this value to improve efficiency of the compaction process, but be aware that this will increase the memory usage of the database. Conversely, decreasing this value will reduce memory usage but may slow down the compaction process.

Preferably, the `PERSISTENCE_HNSW_MAX_LOG_SIZE` should set to a value close to the **size of the HNSW graph.**

* * *

So here are some questions:

- What actually **size of the HNSW graph** means?
- And how to estimate it if we have about 47mln vectors of size 512?

---

<div class="post-metadata">

**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\
**Post date:** [May 16, 2025, 4:50pm UTC](https://forum.weaviate.io/t/how-to-choose-the-correct-persistence-hnsw-max-log-size/21041/2 "2025-05-16T16:50:06Z")

</div>

Hi @d_khlebokazov !!

This [setting](https://weaviate.io/developers/weaviate/config-refs/schema/vector-index#database-parameters-for-hnsw), was introduced with this PR (where there is more infos).

> <https://github.com/weaviate/weaviate/pull/4884>
>
> \### What's being changed:
> \* The condensing requires memory, so it defaults to a… fairly small value
> \* However, on massive machines, where the memory is available, we are currently not making use of it
> \* As a result we leave a lot of redundancy in the HNSW logs "uncondensed" which leads to longer startup times
> \* This PR introduces an env var \`PERSISTENCE\_HNSW\_MAX\_LOG\_SIZE\` to override the value
> \* The default value of 500MiB is unchanged
> \* The variable accepts:
> \* unset -\> use deafult
> \* raw integers -\> interpreted as bytes
> \* SI units, e.g. \`400KiB\`
> \* IEC units, e.g. \`400KB\`
> 
> \### Review checklist
> 
> \- \[\] Documentation has been updated, if necessary. Link to changed documentation:
> \- \[\] Chaos pipeline run or not necessary. Link to pipeline:
> \- \[x\] All new code is covered by tests where it is reasonable.
> \- \[\] Performance tests have been run or not necessary.

It helps prevent massive, uncompacted HNSW logs which can affect startup time. It was introduced to prevent slow startup times caused by many small commit logs reaching the default 512MB limit without being combined.

This setting becomes especially interesting if you not only ingest but also delete a lot of data.

This is about the disk size, and not memory 🤔 So I believe the size they mention is the disk usage of a node.

Let me know if this helps!