Hello,
I have been trying to implement RAG with langchain, and use weaviate vector database as a place to store my embedded data.
I am also using openai integration to create the embeddings.
I am splitting my documents which are in the format of PDFs and MD files and splitting them into chunks, then I am trying to convert them into Embeddings using openai, and store them in my local instance of weaviate.
But I am having problem storing them as embeddings into the database. How do I create the schema and store them in the database?
can anyone help? I am using python v4 client
Hey @Hamza_Khan,
Welcome to our community — it’s lovely to have you here!
That’s a great use case, and we’re definitely here to help.
I recommend checking out our Academy:
it has some great tutorials that are super easy to follow and will guide you step-by-step on how to implement with Weaviate.
Best regards,
Mohamed Shahin
Weaviate Support Engineer
(Ireland, UTC±00:00/+01:00)
Hey, so instead of creating my own schema, i am using the one that is automatically created by langchain. The problem that I was facing with this was the fact that, i have different scripts, and in order to instantiate the db in those files, i had to declare db as global, and then import it as a python module, but that is a bad practice as it, reinitializes ingestion, and basically its not good on the long term, however I found out that while running WeaviateVectorStore.from_documents, you can give the class a name using index_name=” ”
In order to instantiate this in other files/script, you can run:
db = WeaviateVectorStore(client = client, index_name = " ", text_key=“text”, embedding = embeddings) #in order to instantiate the db, we need to write it like this, otherwise it will not work.
the issue i was facing earlier when instantiating was the fact that the positional argument: text_key = “ “, was wrong, and i had no ability to see what it is in the first place because langchain automatically creates a schema with some properties, I just tried this and it worked out