Hi Weaviate friends,
We know that generative search, or RAG, is an important topic to many of you. So we have published a draft / preview of a guide for your feedback.
You can find it here
Please use this thread here to tell us what you think, as it will help us to help you. As a template, please let us know:
- What do you think overall (5: great, 1:not so much)?
- Is there anything missing that you want to know about?
- We only have Python examples here (yet - we wanted to get your feedback quickly ). We’ll add the others soon, but which language examples do you want to see most urgently?
- Any other comments?
Looks awesome. Curious what settings are available in the LLM generation step? For example, temperature, system message, etc.
Thank you for writing with your feedback, and that is a great question!
Model properties are exposed through the Weaviate module configuration. So for example:
"model": "command-xlarge-nightly", // Optional - Defaults to `command-xlarge-nightly`. Can also use`command-xlarge-beta` and `command-xlarge`
"temperatureProperty": <temperature>, // Optional
"maxTokensProperty": <maxTokens>, // Optional
"kProperty": <k>, // Optional
"stopSequencesProperty": <stopSequences>, // Optional
"returnLikelihoodsProperty": <returnLikelihoods>, // Optional
"model": "gpt-3.5-turbo", // Optional - Defaults to `gpt-3.5-turbo`
"temperatureProperty": <temperature>, // Optional, applicable to both OpenAI and Azure OpenAI
"maxTokensProperty": <max_tokens>, // Optional, applicable to both OpenAI and Azure OpenAI
"frequencyPenaltyProperty": <frequency_penalty>, // Optional, applicable to both OpenAI and Azure OpenAI
"presencePenaltyProperty": <presence_penalty>, // Optional, applicable to both OpenAI and Azure OpenAI
"topPProperty": <top_p>, // Optional, applicable to both OpenAI and Azure OpenAI
System messages are not customisable at the moment. I would ask you to open an issue on GitHub if this is important to you. That’s because we do look at what the community votes on to prioritise feature development.
I will add this info in some capacity to the guide shortly.