Generative search / RAG guide preview - feedback thread

Resources
1

Hi Weaviate friends,

We know that generative search, or RAG, is an important topic to many of you. So we have published a draft / preview of a guide for your feedback.

:arrow_down: You can find it here :arrow_down:

Please use this thread here to tell us what you think, as it will help us to help you. As a template, please let us know:

  • What do you think overall (5: great, 1:not so much)?
  • Is there anything missing that you want to know about?
  • We only have Python examples here (yet - we wanted to get your feedback quickly :wink:). We’ll add the others soon, but which language examples do you want to see most urgently?
  • Any other comments?
1 Like
2

Looks awesome. Curious what settings are available in the LLM generation step? For example, temperature, system message, etc.

3

Hey @ab19 :slight_smile:

Thank you for writing with your feedback, and that is a great question!

Model properties are exposed through the Weaviate module configuration. So for example:

For generative-cohere:

        "generative-cohere": {
          "model": "command-xlarge-nightly",  // Optional - Defaults to `command-xlarge-nightly`. Can also use`command-xlarge-beta` and `command-xlarge`
          "temperatureProperty": <temperature>,  // Optional
          "maxTokensProperty": <maxTokens>,  // Optional
          "kProperty": <k>, // Optional
          "stopSequencesProperty": <stopSequences>, // Optional
          "returnLikelihoodsProperty": <returnLikelihoods>, // Optional

For generative-openai:

        "generative-openai": {
          "model": "gpt-3.5-turbo",  // Optional - Defaults to `gpt-3.5-turbo`
          "temperatureProperty": <temperature>,  // Optional, applicable to both OpenAI and Azure OpenAI
          "maxTokensProperty": <max_tokens>,  // Optional, applicable to both OpenAI and Azure OpenAI
          "frequencyPenaltyProperty": <frequency_penalty>,  // Optional, applicable to both OpenAI and Azure OpenAI
          "presencePenaltyProperty": <presence_penalty>,  // Optional, applicable to both OpenAI and Azure OpenAI
          "topPProperty": <top_p>,  // Optional, applicable to both OpenAI and Azure OpenAI
        },

System messages are not customisable at the moment. I would ask you to open an issue on GitHub if this is important to you. That’s because we do look at what the community votes on to prioritise feature development.

I will add this info in some capacity to the guide shortly.