Quickstart tutorial generative search exception on max_token using Python

Support
,
1

Hi, I am new to Weaviate and followed the QuickStart steps exactly but ran into the following exception. Any help is appreciated!

Description

Implementing the QuickStart on WCD with Python gives the following exception when doing a generative search:

weaviate.exceptions.WeaviateQueryError: Query call with protocol GRPC search failed with message <AioRpcError of RPC that terminated with:
status = StatusCode.UNKNOWN
details = “connection to: OpenAI API failed with status: 400 request-id: req_4419cf1a05ed4792171000454b82a998 error: max_tokens is too large: 4097. This model supports at most 4096 completion tokens, whereas you provided 4097.”
debug_error_string = “UNKNOWN:Error received from peer {created_time:“2024-10-15T19:41:41.686614+02:00”, grpc_status:2, grpc_message:“connection to: OpenAI API failed with status: 400 request-id: req_4419cf1a05ed4792171000454b82a998 error: max_tokens is too large: 4097. This model supports at most 4096 completion tokens, whereas you provided 4097.”}”

Server Setup Information

WCS Sandbox standard

2

hi @Chris_Blom !!

Welcome to our community :hugs:

This is a known issue that was already tackled here:

It seems there was a change in OpenAI Api :thinking:

this should be a workaround, but it’s also falling into the same issue:

client.collections.delete("Test")
collection = client.collections.create(
    name="Test",
    vectorizer_config=wvc.config.Configure.Vectorizer.text2vec_openai(),
    generative_config=wvc.config.Configure.Generative.openai(model="gpt-3.5-turbo", max_tokens=4096),
)

collection.data.insert({
        "text": "something about dogs",
    }
)
collection.data.insert({
        "text": "something about cats",
    }
)

results = collection.generate.near_text(
    query="animals",
    grouped_task="Create a content with {text}",
    single_prompt="Translate {text} to portuguese"
)

I’ll get back here with more information as soon as I get it :slight_smile:

3

ps, for now, if you only want to play around you can use the gpt-4 model, like so:

client.collections.delete("Test")
collection = client.collections.create(
    name="Test",
    vectorizer_config=wvc.config.Configure.Vectorizer.text2vec_openai(),
    generative_config=wvc.config.Configure.Generative.openai(model="gpt-4"),
)

collection.data.insert({
        "text": "something about dogs",
    }
)
collection.data.insert({
        "text": "something about cats",
    }
)

results = collection.generate.near_text(
    query="animals",
    grouped_task="Create a content with {text}",
    single_prompt="Translate {text} to portuguese"
)
4

Hi Duda, thanks for the quick reply and I’ll change to gpt-4. regards, Chris

5

Great! A new patch is being released to address this change in apis.

it should be released probably today.

Thanks!

1 Like