Importing 500,000 images with multi2vec-clip

Hi there,

I’m in the process of importing 500,000 objects from a json file. The JSON file has a URL for each image. From my understanding, would i need to download EACH image into a folder, and THEN run the batch importer on that folder of files, or is there a way to pass in an image URL and create base64s from that to import into weaviate?

Thanks all -
Beni

In addition, does anybody know of any services where you can offload the work of giving a list of image URLs, and get a returned list of base64s? If not, I think this would be a super useful tool for Weaviate.

hi @Yanir_Hahitti !! Welcome to our community :hugs:

Your understanding is right: you will need the actual image in order to import them. Weaviate will then store this image, along with it’s vector if you have configured the a vectorizer.

I am not aware of a service or Weaviate module that could do that :grimacing:

Let me know if this helps!

Thanks!

Thats Duda! Yea definitely helps me out. Have a new issue now, but I think i should make a new thread for it, so will do that.

1 Like