# Finetuning to multi2vec-clip with Fashionpedia

**URL:** <https://forum.weaviate.io/t/finetuning-to-multi2vec-clip-with-fashionpedia/2897>\
**Category:** General\
**Created:** [July 2, 2024, 3:43am UTC](https://forum.weaviate.io/t/finetuning-to-multi2vec-clip-with-fashionpedia/2897 "2024-07-02T03:43:01Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jjen\_95](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/jjen_95/32/1861_2.png) [@Jjen\_95](https://forum.weaviate.io/u/Jjen_95)\
**Post date:** [July 2, 2024, 3:43am UTC](https://forum.weaviate.io/t/finetuning-to-multi2vec-clip-with-fashionpedia/2897/1 "2024-07-02T03:43:01Z")

</div>

Hello, I would like to know if multi2vec-clip was trained with Fashionpedia, if not I would like to know if you can finetuning the model so you can better distinguish the clothes, I could run this [Building Multimodal AI in TypeScript | Weaviate - Vector Database](https://weaviate.io/blog/multimodal-search-in-typescript) and it works to detect several garments, but I understand that if you do finetuning makes it more “powerful”.  
I also have other questions  
what GPU do i need for finetuning? a rtx 3060m is enough?  
does finetuning for a multimodal model is the same as for a text-only model like llama3-8B?  
well, if someone has knowledge it would help me a lot with some advice or some prompt to send to the chatgpt to make me an explanation and a guide to get started  
Thanks and greetings 😃

---

<div class="post-metadata">

**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\
**Post date:** [July 3, 2024, 6:54pm UTC](https://forum.weaviate.io/t/finetuning-to-multi2vec-clip-with-fashionpedia/2897/2 "2024-07-03T18:54:32Z")

</div>

hi @Jjen_95 !!

Welcome to our community 🤗

You can definitely train your own clip model and replace it with your own to be used on this container image:

> **[GitHub - weaviate/multi2vec-clip-inference: Weaviate module inference code...](https://github.com/weaviate/multi2vec-clip-inference)**
>
> Weaviate module inference code for the multi2vec-clip module - weaviate/multi2vec-clip-inference

According to that repo docs, this is the model used by default:

> **[sentence-transformers/clip-ViT-B-32 · Hugging Face](https://huggingface.co/sentence-transformers/clip-ViT-B-32)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Let me know if this helps!

Thanks!