Can I use Weaviate with Japanese Language?
Hi!
It will depend if your LLM support it.
Here is a snippet to demonstrate that, using OPEN AI.
Ps: This is a recipe that I am writing for GitHub - weaviate/recipes: This repository shares quick notebooks on how to use various features in Weaviate. it will be published soon
generateTask = "Quelle est la nourriture traditionnelle de ce pays ? Answer in Spanish"
source_file = "brazil-wikipedia-article-text.pdf"
#source_file = "netherlands-wikipedia-article-text.pdf"
result = (
client.query
.get("WikipediaLangChain", "text")
.with_generate(grouped_task = generateTask)
.with_where({
"operator": "Equal",
"path": ["source"],
"valueText": source_file
})
.with_near_text({
"concepts": ["tradicional Food"]
})
.with_limit(5).do()
)
The generated text, for instance is something like:
{
"_additional": {
"generate": {
"error": null,
"groupedResult": "La comida tradicional de Brasil incluye farofa (harina de mandioca), papas fritas, yuca frita, pl\u00e1tanos fritos, carne frita y queso frito, que se consumen con frecuencia ..."
}
...
So the text is indexed and vectorized in English, we did a question in French, and requested the answer in Spanish!
Isn’t that cool?
Let me know if this helps
I have already stored some japanese data in Weaviate DB. How do I query so as get some matches related to question I asked.
Above query in the response does not clear to me. I want to ask question in japanese and is expected that Weaviate will give top 5 similarities for that question.
I am also getting error message as
“message”: “explorer: get class: vectorize params: vectorize params: vectorize params: vectorize keywords: vectorizing corpus ‘[\u6d6a\u901f\u5927\u5b66\u30d7\u30ed\u30b8\u30a7\u30af\u30c8\u306e\u30c7\u30a3\u30b9\u30af\u4f7f\u7528\u91cf\u304c\u5236\u9650\u3092\u8d85\u3048\u3066\u3044\u307e\u3059]’: all words in corpus were either stopwords or not present in the contextionary, cannot build vector”
when I try to query weaviate. I am using contexionary module. is there any vectorizer module specific to Japanese language?