# Configuring PQ compression in a collection

**URL:** <https://forum.weaviate.io/t/configuring-pq-compression-in-a-collection/1574>\
**Category:** Support\
**Created:** [February 26, 2024, 11:38am UTC](https://forum.weaviate.io/t/configuring-pq-compression-in-a-collection/1574 "2024-02-26T11:38:03Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jegadeesh](https://avatars.discourse-cdn.com/v4/letter/j/0ea827/32.png) [@Jegadeesh](https://forum.weaviate.io/u/Jegadeesh)\
**Post date:** [February 26, 2024, 11:38am UTC](https://forum.weaviate.io/t/configuring-pq-compression-in-a-collection/1574/1 "2024-02-26T11:38:03Z")

</div>

Hi, I am using Python client v4 to interact with weaviate. I have configured PQ compression using quantizer

```auto
collection.config.update(
    vector_index_config=wvc.config.Reconfigure.VectorIndex.hnsw(
        quantizer=wvc.config.Reconfigure.VectorIndex.Quantizer.pq()
    ))

```

In the configuration of the collection, I could find the quantizer to be configured with PQ.

`vector_index_config=_VectorIndexConfigHNSW(quantizer=_PQConfig(bit_compression=False, segments=0, centroids=256, training_limit=100000, encoder=_PQEncoderConfig(type_=<PQEncoderType.KMEANS: 'kmeans'>, distribution=<PQEncoderDistribution.LOG_NORMAL: 'log-normal'>))`

Is PQ configured correctly and what is the attribute ‘bit\_compression’ corresponds to?

---

<div class="post-metadata">

**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\
**Post date:** [February 26, 2024, 12:14pm UTC](https://forum.weaviate.io/t/configuring-pq-compression-in-a-collection/1574/2 "2024-02-26T12:14:32Z")

</div>

Hi @Jegadeesh !

I also couldn’t find anything in docs about bit\_compression, so I asked internally.

Also, not sure why the segments is 0, but I believe that, according to [here](https://weaviate.io/developers/weaviate/configuration/pq-compression#pq-parameters) after 1.23 Weaviate will define this parameter according to the dimensions of your vectors. So after populating the collection, Weaviate should define this parameter to better suit your case.

I’ll get back to you when I get more information.

Thanks!

---

<div class="post-metadata">

**Author:** ![Jegadeesh](https://avatars.discourse-cdn.com/v4/letter/j/0ea827/32.png) [@Jegadeesh](https://forum.weaviate.io/u/Jegadeesh)\
**Post date:** [February 26, 2024, 3:37pm UTC](https://forum.weaviate.io/t/configuring-pq-compression-in-a-collection/1574/3 "2024-02-26T15:37:06Z")

</div>

Hi @DudaNogueira !

I am trying to configure PQ compression in WCS (Sandbox version).  
Could you tell me whether PQ compression is possible in the Sandbox version of Weaviate?

---

<div class="post-metadata">

**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\
**Post date:** [February 27, 2024, 12:32am UTC](https://forum.weaviate.io/t/configuring-pq-compression-in-a-collection/1574/4 "2024-02-27T00:32:42Z")

</div>

Hi @Jegadeesh !

You can use PQ in sandbox. Sandbox is in fact the very same version you have in a paid cluster and the very same you get from docker or binary. It only has limited resources (memory, cpu, storage), so don’t expect too much from it 😬

Regarding bit\_compression, its no longer used and will be removed in future versions.

Let me know if this helps!

Thanks!

---

<div class="post-metadata">

**Author:** ![Jegadeesh](https://avatars.discourse-cdn.com/v4/letter/j/0ea827/32.png) [@Jegadeesh](https://forum.weaviate.io/u/Jegadeesh)\
**Post date:** [February 27, 2024, 5:41am UTC](https://forum.weaviate.io/t/configuring-pq-compression-in-a-collection/1574/5 "2024-02-27T05:41:18Z")

</div>

Is there any way we could configure the parameter ‘segments’ manually using Python v4 client?

---

<div class="post-metadata">

**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\
**Post date:** [February 27, 2024, 1:26pm UTC](https://forum.weaviate.io/t/configuring-pq-compression-in-a-collection/1574/6 "2024-02-27T13:26:26Z")

</div>

hi @Jegadeesh!

Sure thing. Here is how:

```python
collection.config.update(
    vector_index_config=wvc.config.Reconfigure.VectorIndex.hnsw(
        quantizer=wvc.config.Reconfigure.VectorIndex.Quantizer.pq(segments=1234)
    ))

```

Note, your segments configuration needs to be an integer divisor or the total dimensions of your data.

---

<div class="post-metadata">

**Author:** ![Jegadeesh](https://avatars.discourse-cdn.com/v4/letter/j/0ea827/32.png) [@Jegadeesh](https://forum.weaviate.io/u/Jegadeesh)\
**Post date:** [February 29, 2024, 6:45am UTC](https://forum.weaviate.io/t/configuring-pq-compression-in-a-collection/1574/7 "2024-02-29T06:45:07Z")

</div>

If we are using manual method to enable PQ, does Weaviate automatically define and update the ‘segments’ parameter based on our vector dimension if it’s not set during PQ configuration?

Or do we need to set it manually while configuring PQ in WCS? Also, what if the segments is 0 and is it essential to set ‘segments’ to do PQ compression?

Thanks in advance!

---

<div class="post-metadata">

**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\
**Post date:** [February 29, 2024, 2:49pm UTC](https://forum.weaviate.io/t/configuring-pq-compression-in-a-collection/1574/8 "2024-02-29T14:49:12Z")

</div>

If you specify a segment value, Weaviate will use it.

If you do not specify the segments, the AutoPQ will kick in and come up with the best segment configuration based on your data:

> **[PQ vector compression | Weaviate - Vector Database](https://weaviate.io/developers/weaviate/configuration/pq-compression#configure-autopq)**
>
> Starting in v1.23, AutoPQ simplifies configuring PQ on new collections.

Check here for more info on how to manually use PQ:

> **[PQ vector compression | Weaviate - Vector Database](https://weaviate.io/developers/weaviate/configuration/pq-compression#manually-configure-pq)**
>
> Starting in v1.23, AutoPQ simplifies configuring PQ on new collections.
