Batch inference of images with img2vec-neural

Hi - is there a plan to to add support to img2vec-neural for batches of images to vectorise in a single request, to achieve optimal CPU/GPU performance?

Hi @jbendotnet !

You mean Weaviate sending the objects in batch, or only the inference model container allowing multiple images?

if only the inference model container

This would be a good feature request to open at:


Hi @DudaNogueira, I had assumed changes would be required in both codebases [Weaviate, weaviate/i2v-pytorch-models], if it’s possible to just add it in one place, that’s even simpler. I’ll create a feature request in the morning.