# RAG for text-2-SQL

**URL:** <https://forum.weaviate.io/t/rag-for-text-2-sql/1629>\
**Category:** General\
**Created:** [March 5, 2024, 2:15pm UTC](https://forum.weaviate.io/t/rag-for-text-2-sql/1629 "2024-03-05T14:15:28Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![sky-2002](https://avatars.discourse-cdn.com/v4/letter/s/49beb7/32.png) [@sky-2002](https://forum.weaviate.io/u/sky-2002)\
**Post date:** [March 5, 2024, 2:15pm UTC](https://forum.weaviate.io/t/rag-for-text-2-sql/1629/1 "2024-03-05T14:15:28Z")

</div>

Hey weaviate community. I was came across some questions in different github repos about RAG on text-2-SQL(or text-2-Code in general). I have the following questions:

1. Is RAG used in text-2-SQL? Is yes, how?
2. What is the context that we want to fetch in those use cases?
3. How do we evaluate an RAG system in text-2-SQL use case?

**My idea for point-3** : Ask an LLM to generate a sample data from the retrieved context, run the ground-truth SQL and generated SQL on that data and compare result.

Would like to know in-depth views on this. Also any resources, suggestions or materials are welcome.

**Note:** Posted in slack as well. Posted here on suggestion from @Marion_Nehring on weaviate slack.

---

<div class="post-metadata">

**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\
**Post date:** [March 6, 2024, 9:19pm UTC](https://forum.weaviate.io/t/rag-for-text-2-sql/1629/2 "2024-03-06T21:19:41Z")

</div>

Hi!

I have seen something along those lines here:

[![](https://canada1.discourse-cdn.com/flex027/uploads/weaviate/original/2X/8/8efe7dd9825d8fceb21563131539e41699a305eb.jpeg "Natural Language to SQL | LangChain, SQL Database & OpenAI LLMs") ](https://www.youtube.com/watch?v=w-eTS8YlbZ4)

So you have a natural language question that is translated to a sql instruction, it then retrieves the data, and answer with it.

Another thing to look for is this:

> **[Weaviate Gorilla Part 1 GraphQL APIs | Weaviate - Vector Database](https://weaviate.io/blog/weaviate-gorilla-part-1)**
>
> Fine-tuning LlaMA 7B to use the Weaviate GraphQL APIs

That instead of DBs, it can learn how to consume APIs. This opens up a lot of cool things.

Anyway, just my 2 cents 🙂

---

<div class="post-metadata">

**Author:** ![SomebodySysop](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/somebodysysop/32/70_2.png) [@SomebodySysop](https://forum.weaviate.io/u/SomebodySysop)\
**Post date:** [March 7, 2024, 8:02am UTC](https://forum.weaviate.io/t/rag-for-text-2-sql/1629/3 "2024-03-07T08:02:30Z")

</div>

> [@sky-2002](#):
>
> **My idea for point-3** : Ask an LLM to generate a sample data from the retrieved context, run the ground-truth SQL and generated SQL on that data and compare result.

The questions you ask are not really answerable given all the unknowns. But, with reference to your idea:

All of this is technically possible: User submits data, a cosine similarity search is ran on your vector store, context documents returned, the original prompt with documents is submitted to the LLM.

Here is where this is all going to come down to your prompts. What exactly are you instructing the LLM to do? You will need to give it precise instructions on not only how to generate the sample data, but also how to submit it as SQL to the 2nd part of the system.

I don’t know if you intend to use function/action calls or simply pass the initial results as another prompt to be processed separately. In any event, the LLM that has to create the SQL will need to know:

1. Your existing database schema in detail.
2. How to submit data – how it should be formatted?
3. What to do and what NOT to do.
4. What to do when it doesn’t know what to do.

Here are some notes I have made on the general subject:

- **SQL**
  - OpenAI Text to SQL
    - Prompt Example - Natural Language to SQL
    - [https://platform.openai.com/examples/default-sql-translate](https://platform.openai.com/examples/default-sql-translate)

  - OpenAI Functions calling the Poor Developers Vector Database
    - [OpenAI Functions Calling the Poor Developer's Vector Database](https://blazorhelpwebsite.com/ViewBlogPost/6067)

  - Private Chat with CSV Data (@DevGirl explanation):
    - [Private Chat with CSV data - #7 by DevGirl - API - OpenAI Developer Forum](https://community.openai.com/t/private-chat-with-csv-data/620115/7?u=somebodysysop)

  - My Prompt Example:
    - [How to apply user based access control check while generating SQL queries using OpenAI API using langchain? - #4 by SomebodySysop - API - OpenAI Developer Forum](https://community.openai.com/t/how-to-apply-user-based-access-control-check-while-generating-sql-queries-using-openai-api-using-langchain/621409/4?u=somebodysysop)