Size of the generated response

Support
,
1

Description

I created a collection with the following parameters:

...
self.__client.collections.create(
        self.__collection_name,
        vectorizer_config=[Configure.NamedVectors.text2vec_aws(
             name=self.__collection_name,
             region='us-east-1',
             service='bedrock',
             model='cohere.embed-multilingual-v3'
        )],
        generative_config=Configure.Generative.aws(
             region='us-east-1',
             service='bedrock',
             model='anthropic.claude-3-5-sonnet-20240620-v1:0'
        )
)
...

And the RAG is like this:

response = collection.generate.near_text(
              query=prompt,
              limit=3,
              grouped_properties=['description'],
              grouped_task=f'Gere uma descrição longa para um produto na forma de um texto comercial que reforçe suas características e usando o contexto fornecido.\n\n{prompt}'
          )

generated = response.generated
print(generated)

The generated string, which is stored in the generated variable, is incomplete. More words were generated, but they were not returned, leaving the generated description meaningless.

I saw that Claude Sonnet 3.5 on Amazon Bedrock has a length parameter, which limits the size of the response.

How can I inform this parameter through Weaviate?

Big hug, everyone.

Server Setup Information

  • Weaviate Server Version: 1.27.0
  • Deployment Method: Local
  • Multi Node? No
  • Client Language and Version: Python v4
  • Multitenancy? No
2

Hi!! @taigofranca !!

Awesome question!

I see that the generative for anthropic has this parameter exposed on client, but not for the generate.aws.

I have checked with our team, and indeed it is missing.

So we could either add it to generative.bedrock or create a new generative.bedrock_anthropic

Issue created: Missing parameters when using anthropic on amazon bedrock · Issue #1520 · weaviate/weaviate-python-client · GitHub

Thanks for pointing it out!