Dear friends, I need to embed millions of Italian language strings using the well tested intfloat/multilingual-e5-large model. If anyone is interested I have uploaded a small repo that will show you how this is done and also test the setup and performance of your new multilingual Weaviate service. …

Ciao amico mio!!! Long time no see :slight_smile: Indeed, I don’t see anything that could prefix this in the inference container code. Maybe it could be added as an ENV VAR, something like: VECTORIZER_TEXT_PREPEND, somewhere around here I will raise this internally with our team. Thanks for exp…

@rjalex with our latest v1.31 (which should be released this week) I have added one small change to transformers module. Now when we do a query we are sending a taskType: query and when we send passage request we add taskType: passage. We can use this information in our transformers inference conta…

Hi Duda yes I was absent for a while due to other job priorities. As always thanks a lot.

Hi Marcin, yes that would be cool. intfloat/multilingual-e5-large · Hugging Face and BAAI/bge-m3 · Hugging Face are as you probably know the best multilingual embedders out there, and as you can see from their HG pages both Milvus and Vespa do support them, so I guess that an optimal support also f…

FYI: our transformers inference container already supports bge-m3 we expose ONNX images.

Very cool to know but I already have 40 million vectors with e5-large :slight_smile: useful for another project though. Thanks

@rjalex I have released v1.12.0 version of the transformers-inference container project and you can use a pre-built intfloat-multilingual-e5-large docker image together with our text2vec-tranformers module to produce the embeddings using this model

Thank you very much. This fixes one issue with this embedder :slight_smile: The other VERY RELEVANT aspect is that this model needs the vector output to be normalized before being stored or used for similarity. This is a short doc explaining why, hope it’s useful. Normalizing the embeddings vector …

I thought that this model already responds with normalized embeddings. I will fix that @rjalex in the next version. Thanks!

@rjalex I just noticed that we already do set normalize_embeddings=True so just pull the latest version and all will be working as expected.

Multilingual embedder for Weaviate

Support

antas-marcin July 6, 2025, 10:02am 8

@rjalex sorry that it took me that long, here’s a PR that adds support for intfloat/multilingual-e5-large model and prepending of query passage prefixes to vectorized inputs

Topic		Replies	Views
Normalizing a vector when using weaviate with a text2vec-transformers Support	2	423	June 27, 2025
Text2vec-openai Batch API Support integration , wcs , python	1	611	July 8, 2024
Use DeepInfra as provider for Embedding Model Support	5	229	October 3, 2025
Weaviate Text Embedding Variations Support	1	881	February 19, 2024
Multi-Lingual Cosine Similarity Search Support	5	711	March 9, 2024

Multilingual embedder for Weaviate

Related topics