Generative search / RAG guide preview - feedback thread

Resources
1

Hi Weaviate friends,

We know that generative search, or RAG, is an important topic to many of you. So we have published a draft / preview of a guide for your feedback.

:arrow_down: You can find it here :arrow_down:

Please use this thread here to tell us what you think, as it will help us to help you. As a template, please let us know:

  • What do you think overall (5: great, 1:not so much)?
  • Is there anything missing that you want to know about?
  • We only have Python examples here (yet - we wanted to get your feedback quickly :wink:). We’ll add the others soon, but which language examples do you want to see most urgently?
  • Any other comments?
1 Like
2

Looks awesome. Curious what settings are available in the LLM generation step? For example, temperature, system message, etc.

3

Hey @ab19 :slight_smile:

Thank you for writing with your feedback, and that is a great question!

Model properties are exposed through the Weaviate module configuration. So for example:

For generative-cohere:

        "generative-cohere": {
          "model": "command-xlarge-nightly",  // Optional - Defaults to `command-xlarge-nightly`. Can also use`command-xlarge-beta` and `command-xlarge`
          "temperatureProperty": <temperature>,  // Optional
          "maxTokensProperty": <maxTokens>,  // Optional
          "kProperty": <k>, // Optional
          "stopSequencesProperty": <stopSequences>, // Optional
          "returnLikelihoodsProperty": <returnLikelihoods>, // Optional

For generative-openai:

        "generative-openai": {
          "model": "gpt-3.5-turbo",  // Optional - Defaults to `gpt-3.5-turbo`
          "temperatureProperty": <temperature>,  // Optional, applicable to both OpenAI and Azure OpenAI
          "maxTokensProperty": <max_tokens>,  // Optional, applicable to both OpenAI and Azure OpenAI
          "frequencyPenaltyProperty": <frequency_penalty>,  // Optional, applicable to both OpenAI and Azure OpenAI
          "presencePenaltyProperty": <presence_penalty>,  // Optional, applicable to both OpenAI and Azure OpenAI
          "topPProperty": <top_p>,  // Optional, applicable to both OpenAI and Azure OpenAI
        },

System messages are not customisable at the moment. I would ask you to open an issue on GitHub if this is important to you. That’s because we do look at what the community votes on to prioritise feature development.

I will add this info in some capacity to the guide shortly.

4

Love it! I already implemented CLI prototype for my domain data following your guide. I think “Create Class Definition” and “Import Data” section has the same piece of code. Also, where can I find all possible generative functions that are supported? I think you have used
grouped_task and single_prompt. What other options are available?

5

Hi @KMohan, at the moment these are the only two options. I know the team is definitely talking about extending these options.

If you would like to see, please file a request - it would help us to prioritize features if our users like yourself create these requests and upvote them :slight_smile:

6

I may not be the right person because, I don’t know what other options are out there. I’m an app developer with no ML experience. I’m relying ML experts to educate us what is possible so I can go and build consumer facing apps using the technology.

7

Yeah that’s fair.

We’ve definitely got more content coming up, and extending our RAG functionalities are in discussions too. Thanks @KMohan :slight_smile: