Local Hugginface vector embedding

Support
1

I’m trying to use (on Windows) the docker image of weaviate 1.26.0 to get vector embedding local to my computer, but when vectorization should occur I get the error ‘failed with status: 429 error: Rate limit reached. Please log in or use a HF access token’.

This is the docker-compose.yml:
version: ‘3.4’

services:
weaviate:
image: semitechnologies/weaviate:1.26.0
ports:
- “9090:8080”
environment:
QUERY_DEFAULTS_LIMIT: 100
AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: ‘true’
ENABLE_MODULES: ‘text2vec-huggingface’ # Enable Hugging Face module
HUGGINGFACE_INFERENCE_MODEL: ‘sentence-transformers/all-MiniLM-L6-v2’
DEFAULT_VECTORIZER_MODULE: text2vec-huggingface
HUGGINGFACE_INFERENCE_API: ‘local’
PERSISTENCE_DATA_PATH: “/var/lib/weaviate” # Inside the container

It seems that it is not possibile to use local vectorization, but only remote (which implies sending text to HuggingFace). Is it correct? What is the meaning of HUGGINGFACE_INFERENCE_API: ‘local’? Thanks!

2

hi @fabriziof64 !!

Welcome to our community :hugs:

This message usually comes from the vectorizer service.

Changing HUGGINGFACE_INFERENCE_API will set the url to where Weaviate will look for Hugging face inference service.

If you want to run transformers locally, check this option:

Also, you can run models using ollama:

Let me know if this helps!

Thanks!