client.GraphQL().Get() tries to use llama3 instead of the model I want

Support
1

Hi all,

I’m trying to build a simple RAG application in Golang. Everything is working fine, except when I try to get data for the final inference: client.GraphQL().Get() tries to use llama3 instead of the model I want.
I surely missed something as llama3 is the default model, but I could not find where I forgot to indicate the right model to be used.

Description

Local Weaviate built from sources
Local Ollama server

  • I created a class with this code:
    classObj := &models.Class{
    Class: className,
    Vectorizer: “text2vec-ollama”,
    ModuleConfig: map[string]interface{}{
    “text2vec-ollama”: map[string]interface{}{ // Configure the Ollama embedding integration
    “apiEndpoint”: weaviateOllamaURL, // Allow Weaviate from within a Docker container to contact your Ollama instance
    “model”: ollamaEmbeddingModel, // The model to use
    },
    “generative-ollama”: map[string]interface{}{ // Configure the Ollama generative integration
    “apiEndpoint”: weaviateOllamaURL, // Allow Weaviate from within a Docker container to contact your Ollama instance
    “model”: ollamaModel, // The model to use
    },
    },
    }
    err = client.Schema().ClassCreator().WithClass(classObj).Do(ctx)

  • I added data with this code:
    _, err := client.Data().Creator().
    WithClassName(className).
    WithProperties(map[string]interface{}{
    “Title”: braveResponse.Web.Results[r].Title,
    “URL”: braveResponse.Web.Results[r].URL,
    “Content”: resp.Response,
    }).
    Do(ctx)

  • Finally I get the data with this code:
    gs := graphql.NewGenerativeSearch().GroupedResult(prompt)
    response, err := client.GraphQL().Get().
    WithClassName(className).
    WithFields(
    graphql.Field{Name: “title”},
    graphql.Field{Name: “content”},
    ).
    WithGenerativeSearch(gs).
    WithNearText((&graphql.NearTextArgumentBuilder{}).
    WithConcepts(string{concepts})).
    WithLimit(200).
    Do(ctx)

  • Here I get an error as Weaviate tries to use llama3. As workaround I had to do this:
    ollama cp qwen2.5:7b “llama3”, but this is dirty and not handy.

Server Setup Information

  • Weaviate Server Version: latest (git cloned yesterday)
    build_wv_version=1.28.0-dev

  • Deployment Method: local, build from sources, and started with:
    ./tools/dev/run_dev_server.sh local-ollama
    I even edited the script:
    local-ollama)
    AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED=true
    DEFAULT_VECTORIZER_MODULE=text2vec-ollama
    ENABLE_MODULES=“text2vec-ollama,generative-ollama”
    OLLAMA_MODEL=“qwen2.5:7b”
    go_run ./cmd/weaviate-server
    –scheme http
    –host “127.0.0.1”
    –port 8080
    –read-timeout=600s
    –write-timeout=600s
    ;;

  • Multi Node? No

  • Client Language and Version:
    github.com/weaviate/weaviate-go-client/v4@v4.16.0
    go1.23.3

  • Multitenancy?: No

Kind regards,

JC

2

hi @JCP !!

Welcome to our community :hugs:

When you first create your collection, and specify the generative configuration, it will set some defaults for you.

This is probably your scenario. You can specify the model to use for generation while creating the collection.

Since 1.27.1 you can also reconfigure that option, setting different generative_config after the collection was created.

PS: This feature “dynamic RAG” - or the ability to specify different generative configurations - will eventually be available at query time.

Let me know if that helps!

Thanks!

3

Hi Duda,

Thank you for your answer!

I managed to make my code work. It seems I made some error while creating the collection, and it did keep the default llama3. It now uses the right model after re-creation.

Thanks and kind regards!

JC

1 Like
4

Oh! Glad to hear that, @JCP !

Thanks for sharing!

If you need any other help in your Weaviate journey, we are here to help!

5

Thanks, sure I will!

FYI, I just posted this about the experience I had so far with your excellent product: My first steps with Weaviate (and why I hate Python ;) )

Keep up the good work!

1 Like