Use of GPU when verba installed on WSL

Description

the use of GPU is highly recommended but not so clear. I am using Verba on my WSL on Windows but as it is using only CPU, it is kind of slow. How can I plug my GPU to help?

Server Setup Information

  • Weaviate Server Version: latest
  • Deployment Method: binary on wsl/windows 11
  • Multi Node? Number of Running Nodes: 1
  • Client Language and Version: english

Any additional Information

hi! Welcome to our community!

The GPU has advantages mainly on the vectorization and generation parts. While using Verba, those are usually handled by OpenAI or Coehre, which already leverages GPU and beefy hardwares.

Unless you are running models locally, GPU will not help you there on running Verba itself.

On that case, you can make sure to run your models on a GPU enabled hardware, and point Verba to those services.

Let me know if this clarifies it for you.

Thanks!