Would like support for using the nvidia mig driver for gpu

Support
1

How to reproduce this bug?

in values yaml
muti2vec-clip:
…
enable_cuda: true
…
#added this line to make it work
cuda_visible_devices: 0
…
nvidia.com/mig-1g.5gb: 1

Also had to modiify file transformersInferenceDeployment.yaml to allow the line cuda_visible_devices
-helm chart 16.4.0

What is the expected behavior?

Was expecting it to work without adding cuda_visible_devices line, the same as it worked when using nvidia.com/gpu: 1

What is the actual behavior?

If cuda_visible_device is not there it fails to start with error cuda_visible_device must be there to use nvidia mig driver.

Supporting information

No response

Server Version

1.20.5

2

Hi @djjeffr ! Welcome to our community!

Thanks you for sharing this.

Just to make it clear: now with those changes, it will work as expected, right?

I believe it’s worth investigating if this is possible to be defined in the docker image itself:

And the changes can be done in the helm repo:

So we could open the PR or issue request there so our team can check it out.

What do you think?

Thanks!

3

Sound good to raise a PR, would be nice if in image/helm chart so I don’t need to modify the file after every new helm chart.

See Fails to load if nvidia mig driver used for gpu · Issue #4323 · weaviate/weaviate · GitHub

1 Like