# How can I config the gse tokenization

**URL:** <https://forum.weaviate.io/t/how-can-i-config-the-gse-tokenization/7600>\
**Category:** Support\
**Tags:** python, technical\
**Created:** [November 16, 2024, 11:40am UTC](https://forum.weaviate.io/t/how-can-i-config-the-gse-tokenization/7600 "2024-11-16T11:40:26Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Charlie](https://avatars.discourse-cdn.com/v4/letter/c/d6d6ee/32.png) [@Charlie](https://forum.weaviate.io/u/Charlie)\
**Post date:** [November 16, 2024, 11:40am UTC](https://forum.weaviate.io/t/how-can-i-config-the-gse-tokenization/7600/1 "2024-11-16T11:40:26Z")

</div>

### Description

I’m trying to use weaviate to handle chinese documents, I know weaviate support gse as the tokenization, but how can I config the gse. I need to load chinese dict to the gse for particular terms. (The official gse support LoadDict method.)

### The schema looks like as below:

```auto
{
                    "class": self.index_name,
                    "description": "Chunks of Documentations",
                    "vectorizer": "none",
                    "properties": [
                        {
                            "name": "text",
                            "dataType": ["text"],
                            "description": "Content of the document",
                            "tokenization": "gse",
                            "indexSearchable": True,
                        },
                   ]
}

```

Thanks.

---

<div class="post-metadata">

**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\
**Post date:** [November 18, 2024, 1:11pm UTC](https://forum.weaviate.io/t/how-can-i-config-the-gse-tokenization/7600/2 "2024-11-18T13:11:36Z")

</div>

hi @Charlie !!

Welcome to our community 🤗

In order to enable GSE tokenization, you will need to enable it in your server.

For that, you need to set the environment variable `ENABLE_TOKENIZER_GSE` to **true** as documented here:

> **[Collection definition | Weaviate](https://weaviate.io/developers/weaviate/config-refs/schema#gse-and-trigram-tokenization-methods)**
>
> Schema Configuration in Weaviate

Let me know if that helps!

Thanks!

---

<div class="post-metadata">

**Author:** ![Charlie](https://avatars.discourse-cdn.com/v4/letter/c/d6d6ee/32.png) [@Charlie](https://forum.weaviate.io/u/Charlie)\
**Post date:** [November 20, 2024, 10:05am UTC](https://forum.weaviate.io/t/how-can-i-config-the-gse-tokenization/7600/3 "2024-11-20T10:05:23Z")

</div>

Thanks, I already enabled the gse. But I don’t know how to customize the gse option, as I need to load chinese dict like below:

```auto
var seg1 gse.Segmenter
seg1.DictSep = ","
err := seg1.LoadDict("./testdata/test_cn.txt")

```

---

<div class="post-metadata">

**Author:** ![zc-zhangchen](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/zc-zhangchen/32/7746_2.png) [@zc-zhangchen](https://forum.weaviate.io/u/zc-zhangchen)\
**Post date:** [November 17, 2025, 6:49am UTC](https://forum.weaviate.io/t/how-can-i-config-the-gse-tokenization/7600/4 "2025-11-17T06:49:43Z")

</div>

Do you know how GSE loads a custom dictionary?

---

<div class="post-metadata">

**Author:** ![Shahin](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/shahin/32/7944_2.png) [@Shahin](https://forum.weaviate.io/u/Shahin)\
**Post date:** [November 17, 2025, 8:06am UTC](https://forum.weaviate.io/t/how-can-i-config-the-gse-tokenization/7600/5 "2025-11-17T08:06:49Z")

</div>

Hey @zc-zhangchen,

We don’t have this yet.

> **[Collection definition | Weaviate Documentation](https://docs.weaviate.io/weaviate/config-refs/collections#language-specific-tokenization)**
>
> Reference for Weaviate collection parameters.

Mohamed Shahin  
Weaviate Support Engineer  
(Ireland, UTC±00:00/+01:00)