Weaviate request timing out when using gpt4

Support
1

So I am using weaviate for my LLM application and I get the following error:

weaviate Query was not successful! Unexpected status code: "502," with response body: None.

Here is my model config file:

MODULE_CONFIG = {
        "text2vec-openai": {
            "vectorizeClassName": False,
            "model": "ada",
            "modelVersion": "002",
            "type": "text",
        },
        "generative-openai": {"model": "gpt-4"}, 
    }

I am using the sandbox version of weaviate cloud services.

To dig further I hosted weaviate using docker-compose and then started getting the following response:

{"data":{"Get":{"WebsitesData":[{"_additional":{"generate":{"error":"send POST request: Post \"https://api.openai.com/v1/chat/completions\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)","groupedResult":null}}

So for some reason, my queries are getting timed out while using gpt-4. When I use gpt-3.5-turbo everything works fine.

I am using the Python client and here is the query I am running:

self._weaviate_client.query.get(WEAVIATE.CLASS_NAME, ["url", "body_txt"])
            .with_near_text({"concepts": [concept]})
            .with_generate(single_prompt=prompt_query_vectordb)
            .with_limit(num_of_returned_objects)
            .do()

Can I not use gpt-4 with weaviate or am I missing something? Thanks for help.

2

hi @shark_test ! Welcome to our community :hugs:

If you api key gives you access to gpt-4, you should be able to use it.

This is probably a connection issue between Weaviater Server and the OpenAI node/pod your api key may be attached to.

I tested a recipe with gpt-4, and worked fine.

Is it consistent? Have you tried a different api-key?

let me know if that helps :slight_smile:

3

It is now working without changing the API key. Pretty strange but thanks for the help.

1 Like