Verba: Failing to embed using Weaviate Service

Description

I’m a little bit lost actuality.

When tried the Weaviate Embedder i’m having this error:

ℹ FileStatus.ERROR | xpto.pdf | Import for Por dentro
Melissa.pdf failed: Import for xpto.pdf failed: Batch
vectorization failed: Vectorization failed for some batches: 404, message='Not
Found', url=URL('http://weaviate:8080/v1/embeddings/embed') | 0

My EMBEDDING_SERVICE_URL and EMBEDDING_SERVICE_KEY are set to my weaviate local server. But… Weaviate does not has the endpoint: /v1/embeddings/embed

The README says something about to HuggingFace. But is this the EMBEDDING_SERVICE ?

How to setup this?

I tried with another local Weaviate instance deployed with Locally Hosted Transformers Text Embeddings. But I do not know how to configure.

In order to follow with our tests… we arre using the OpenAI Embedder. But I’d like avoid cost with dev and tests.

Server Setup Information

  • Weaviate Server Version: 1.24.1
  • Deployment Method: docker
  • Multi Node? Number of Running Nodes: 1
  • Client Language and Version: Verba
  • Multitenancy?: No

Any additional Information

ℹ FileStatus.ERROR | xpto.pdf | Import for Por dentro
Melissa.pdf failed: Import for xpto.pdf failed: Batch
vectorization failed: Vectorization failed for some batches:
weaviate/v1/embeddings/embed | 0

Hi @wtavares !!

I believe this is a service that we will make available… in the future :thinking:

Check here for being notified when it goes GA:

Thanks!

Exactly Duda! Is a service comming soon.

By the way… I’m using OpenAI to this at this momment. But I’d like to avoid costs in DEV.

Could you explain how to use HuggingFace instead? The README does not explain how to setup a docker with this and also how to configure Verba to use.

Thanks in advance.
W

Hummm. Have you tried ollama?

I believe it will be easier than HF.

I’m doing exactly this now!

Any tips about Generation in PT-Br? Or Ollama will do this for me?

First contact with Ollama…

It will depend on the model you have chosen.

I have did a quick research, and I don’t think that ollama current available embed models support PT-BR :frowning: