Error during nearText call: panic occurred: ValidateParam was called without any known params present

[Robert_Oschler]

I am using the Weaviate JS/TS beta client with my Weaviate Cloud Services cluster. Here is my schema for my collection:

    const schema = {
      name: useCollectionName,
      properties: [
        {
          // This field contains the text we want to be able to
          //  search on, using a vector similarity search.
          name: 'chunkText',
          dataType: 'text',
          description: 'The searchable text',
          tokenization: 'lowercase',
          // If TRUE, then the property name ("chunkText")
          //  is made part of the vectorized text.  If FALSE,
          //  then only the value of the "chunkText" field
          //  is vectorized.
          vectorizePropertyName: false,
        },
        {
          // This field contains the URL to the document
          //  the text chunk was taken from.
          name: 'attributionUrl',
          // We use the "object" data type to keep this
          //  field OUT of the vectorized text.
          dataType: 'text',
          description: 'The URL of the source article',
          tokenization: 'whitespace',
          // We declare a moduleConfig field that tells
          //  Weaviate to skip this field when creating
          //  the vectorized text field when we are using
          //  OpenAI for word embeddings (vectorization).
          //
          //  We also don't want the property name in the
          //   vectorized text either.
          moduleConfig: {
            text2vec_openai: {
              skip: true,
              vectorizePropertyName: false
            }
          },
        }
      ],
    }

Which I added successfully via the client.

Then I added 8705 objects via insertMany taking about 8 seconds to complete. However, when I try to do a nearText search, I get the following error:

“panic occurred: ValidateParam was called without any known params present”

I found this thread, where apparently the poster had the same error because he wasn’t using a vectorizer:

WeaviateQueryError: Query call with protocol GRPC search failed with message panic occurred: ValidateParam was called without any known params present

Hi @Debojit_Nath, I am glad you were able to find the issue. I think we should add the example to check for vectors to a troubleshooting guide. Good luck with your project. I can’t wait to see what you will build.

Also, I checked my OpenAI stats where there should have been a flurry of recent requests due to the insertion, but I don’t see any activity at all, not even a call for the vectorizing of the input query.

I believe I am initializing the client object properly with my OpenAI key:

  const client = await weaviate.connectToWCS(
    WEAVIATE_CLUSTER_ENDPOINT_URL_1,
    {
      authCredentials: new weaviate.ApiKey(WEAVIATE_API_KEY_ADMIN_READ_WRITE),
      headers: {
        'X-OpenAI-Api-Key': OPEN_AI_API_KEY_WEAVIATE_1,
      }
    }
  )

Note, I know the environment variables have valid values because I traced through the code to inspect them.

So why am I getting this error and why does it seem that no embeddings/vectorizing operations are taking place?

[DudaNogueira]

Hi! What is the version you are using?

Can you share the code you used?

[Robert_Oschler]

Here it is:

    // -------------------- BEGIN: DEFINE A DATA COLLECTION ------------

    /*
      Configures a class object with:

        Name: Question
        Vectorizer: module text2vec-openai
        Generative module: generative-openai
     */

    /*
    const schema = {
      name: 'Question',
      vectorizer: weaviate.configure.vectorizer.text2VecOpenAI(),
      generative: weaviate.configure.generative.openAI(),
    }
     */

    // Our collection has a single 'text' field for the
    //  article text chunk.  Only 'text' fields are vectorized.
    const articleSchema = {
      name: useCollectionName,
      properties: [
        {
          // This field contains the text we want to be able to
          //  search on, using a vector similarity search.
          name: 'chunkText',
          dataType: 'text',
          description: 'The searchable text',
          tokenization: 'lowercase',
          // If TRUE, then the property name ("chunkText")
          //  is made part of the vectorized text.  If FALSE,
          //  then only the value of the "chunkText" field
          //  is vectorized.
          vectorizePropertyName: false,
        },
        {
          // This field contains the URL to the document
          //  the text chunk was taken from.
          name: 'attributionUrl',
          // We use the "object" data type to keep this
          //  field OUT of the vectorized text.
          dataType: 'text',
          description: 'The URL of the source article',
          tokenization: 'whitespace',
          // We declare a moduleConfig field that tells
          //  Weaviate to skip this field when creating
          //  the vectorized text field when we are using
          //  OpenAI for word embeddings (vectorization).
          //
          //  We also don't want the property name in the
          //   vectorized text either.
          moduleConfig: {
            text2vec_openai: {
              skip: true,
              vectorizePropertyName: false
            }
          },
        }
      ],
    }

    /**
     * Add a new schema to our cluster.
     *
     * @param {Object} schema - The new schema to add.
     *
     * @return {Promise<void>}
     */
    async function addSchema(schema) {
      const errPrefix = `(addSchema) `;

      if (typeof schema !== 'object')
      	throw new Error(`${errPrefix}The schema parameter is not an object.`);

      const newCollection =
        await client.collections.create(schema);
      console.log('Article schema created.  Collection name: ', newCollection['name']);
    }

    const bIsExistingCollection =
      await client.collections.exists(useCollectionName);

    if (bIsExistingCollection || bDoNotAddSchema) {
      console.warn(CONSOLE_CATEGORY, `The following schema already exists.  Not creating schema: ${useCollectionName}`);
    } else {
      console.info(CONSOLE_CATEGORY, `Creating schema named: ${useCollectionName}`);

      await addSchema(articleSchema);
    }

    // -------------------- END  : DEFINE A DATA COLLECTION ------------

    // -------------------- BEGIN: BATCH IMPORT OF OBJECTS (Add Objects) ------------

    async function getJsonDataFromMediumObjectsFile() {
      const useJsonObjectsFilename =
        bIsTesting
      ? FILE_NAME_ARTICLE_JSON_OBJECTS_1_TEST
          : FILE_NAME_ARTICLE_JSON_OBJECTS_1_FULL_SET;

      return loadJsonDataArrayFromFile(DIR_OUTPUT_JSON_OBJECTS, useJsonObjectsFilename);
    }

    /**
     * Get the JSON data from our data directory.
     *
     * @return {Promise<void>}
     */
    async function importArticles() {
      const errPrefix = `(importArticles) `;

      // Get a reference to the collection.
      articleCollection =
        await client.collections.get(useCollectionName);

      // Get the objects to be bulk inserted.
      const aryJsonObjs = await getJsonDataFromMediumObjectsFile();

      if (!Array.isArray(aryJsonObjs))
        throw new Error(`${errPrefix}The aryJsonObjs variable value is not an array.`);
      if (aryJsonObjs.length < 1)
        throw new Error(`${errPrefix}The aryJsonObjs array is empty`);

      console.info(CONSOLE_CATEGORY, `Number of JSON objects loaded and ready for import: ${aryJsonObjs.length}`);

      // Bulk insert the objects.
      const result =
        await articleCollection.data.insertMany(aryJsonObjs)

      console.info('Bulk insertion result: ', result);
    }

    await importArticles();

    // -------------------- END  : BATCH IMPORT OF OBJECTS (Add Objects) ------------

  } else {

    // -------------------- BEGIN: JUST GET DATA FOR EXAMINATION ------------

    // Get the collection data we imported earlier.

    articleCollection =
      await client.collections.get(useCollectionName);

    console.info(`${errPrefix}myCollection object:`);
    console.dir(articleCollection, {depth: null, colors: true});

    console.info(`${errPrefix} Not creating collection or importing data. Continuing.`)

    // -------------------- END  : JUST GET DATA FOR EXAMINATION ------------
  }

  // -------------------- BEGIN: SEMANTIC SEARCH ------------

  /**
   * Do a similarity search using the question in our collection
   *  and use our collection
   *
   * @return {Promise<*>}
   */
  async function nearTextQuery() {
    const errPrefix = `(nearTextQuery) `;

    // Make sure we have a valid collection reference.
    validateCollectionReferenceOrDie(errPrefix);

    // Do a similarity search.
    const result =
      await articleCollection.query.nearText(
        aryTestQueries,{
      returnProperties:
        [
          'datePublished',
          'srcAttributionUrl',
          'chunkText',
        ],
      // Limit to 10 responses.
      limit: 10
    });

    // Show the result.
    console.info (JSON.stringify(result.objects, null, 2));

    return result;
  }

  await nearTextQuery();

  // -------------------- END  : SEMANTIC SEARCH ------------

The error occurs during the articleCollection.query.nearText() call.

[malgamves]

@Robert_Oschler

      vectorizer: weaviate.configure.vectorizer.text2VecOpenAI(),

hey @Robert_Oschler you need to add a vectoriser to your to your collection to make nearText queries. Weaviate needs a Vectoriser defined to create vector embeddings for your data and you vectorise your queries when you make them.

So adding the line i highlighted to your schema definition should work.

[Robert_Oschler]

Which is hilarious considering that if you look at the code I posted I have those very statements commented out.

Looks like when I went to create my full schema I forgot to include those statements. Thanks!

UPDATE: I created a brand new collection with the updated schema:

    const articleSchema = {
      name: useCollectionName,
      properties: [
        {
          // Use OpenAI for creating embeddings (vectorizing).
          vectorizer: weaviate.configure.vectorizer.text2VecOpenAI(),
          // Use OpenAI for generation completion text.
          generative: weaviate.configure.generative.openAI(),
          // This field contains the text we want to be able to
          //  search on, using a vector similarity search.
          name: 'chunkText',
          dataType: 'text',
          description: 'The searchable text',
          tokenization: 'lowercase',
          // If TRUE, then the property name ("chunkText")
          //  is made part of the vectorized text.  If FALSE,
          //  then only the value of the "chunkText" field
          //  is vectorized.
          vectorizePropertyName: false,
        },
        {
          // This field contains the URL to the document
          //  the text chunk was taken from.
          name: 'attributionUrl',
          // We use the "object" data type to keep this
          //  field OUT of the vectorized text.
          dataType: 'text',
          description: 'The URL of the source article',
          tokenization: 'whitespace',
          // We declare a moduleConfig field that tells
          //  Weaviate to skip this field when creating
          //  the vectorized text field when we are using
          //  OpenAI for word embeddings (vectorization).
          //
          //  We also don't want the property name in the
          //   vectorized text either.
          moduleConfig: {
            text2vec_openai: {
              skip: true,
              vectorizePropertyName: false
            }
          },
        }
      ],
    }

It took a few minutes to insert the same objects and created 8531 items.

I then did a near text search and the call did not return for several minutes, and then caused a Node.js process disconnect. My collection has 8531 records. But even if that is a lot, I thought this would be an HNSW search with vectors, and be nearly instant? Instead, it looks like the query took so long the underlying API request timed out the connection, but that may be to the fact I’m still getting the panic error.

I then reran the program, this time without adding the schema or the records, and I am still getting the error:

"/weaviate.v1.Weaviate/Search UNKNOWN: panic occurred: ValidateParam was called without any known params present

Is it possible that my cluster is in a corrupted state overall? Or do I still have something wrong with my schema? Note, still don’t see any activity on my OpenAI developer console for today, at all.

[DudaNogueira]

@malgamves is this a client error message?

Panic message seems… “too serious” for only a parameter validation, WDYT?

Maybe there is room for improvement, showing what was the parameter that was not validated.

Thanks!

[Robert_Oschler]

Agreed. Unfortunately, I am completely dead in the water now and don’t know how to fix this.

[DudaNogueira]

@Robert_Oschler Adding the vectorizer didn’t work?

[malgamves]

Hello @Robert_Oschler sorry about that. It looks like your

 const articleSchema = {
      name: useCollectionName,
      properties: [
        {
          // This field contains the text we want to be able to
          //  search on, using a vector similarity search.
          name: 'chunkText',
          dataType: 'text',
          description: 'The searchable text',
          tokenization: 'lowercase',
          // If TRUE, then the property name ("chunkText")
          //  is made part of the vectorized text.  If FALSE,
          //  then only the value of the "chunkText" field
          //  is vectorized.
          vectorizePropertyName: false,
        },
        {
          // This field contains the URL to the document
          //  the text chunk was taken from.
          name: 'attributionUrl',
          // We use the "object" data type to keep this
          //  field OUT of the vectorized text.
          dataType: 'text',
          description: 'The URL of the source article',
          tokenization: 'whitespace',
          // We declare a moduleConfig field that tells
          //  Weaviate to skip this field when creating
          //  the vectorized text field when we are using
          //  OpenAI for word embeddings (vectorization).
          //
          //  We also don't want the property name in the
          //   vectorized text either.
          moduleConfig: {
            text2vec_openai: {
              skip: true,
              vectorizePropertyName: false
            }
          },
        }
      ],
       // Use OpenAI for creating embeddings (vectorizing).
          vectorizer: weaviate.configure.vectorizer.text2VecOpenAI(),
          // Use OpenAI for generation completion text.
          generative: weaviate.configure.generative.openAI(),
    }

We have a code example to define a collection with a vectoriser.

Generally, you vectorise at the collection level and not the property level, if you want to vectorise at the property level, i.e potentially have different configurations for each property, you should have a look at named vectors.

Hope this helps, let me know if anything is confusing or missing in our docs.

[malgamves]

yes, we’re working on making the error messages more descriptive

[Robert_Oschler]

I’ll give that a try. I did not want to do per-property vectorization. That is the schema that was given to me here:

Is nearText() completely a Weaviate cloud operation, no outbound LLM call?

Hi! The downside of using object data type, for now, is that they are not indexable nor filterable: from docs: As of 1.22, object and object[] datatype properties are not indexed and not vectorized. Future plans include the ability to index nested properties, for example to allow for filtering on nested properties and vectorization options. for the url, you can also use a text, however, as you don’t want it added as part of the vectorization, you mark it to skip, like so: const schema …

Collection level vectorization is fine.

[Robert_Oschler]

Ok, I can report that the new schema works. The nearText() does not fail any more and returns results instead of a panic error. Checked my OpenAI billing console and I do see activity for embeddings. I never realized just how inexpensive that endpoint was! I had to mouse over the cost bar to see it was less than a penny because it wouldn’t even form a bar on the chart that I could see (next to my text completion and image generation activity).

To be safe I left out the “generative” declaration from my schema and only added the “vectorizer” declaration. Is it possible to add the “generative” declaration to my existing collection and if so how? I don’t know how to update an existing collection’s schema. Or do I have to create a new schema and collection to do that?

[DudaNogueira]

Hi! Unfortunately, for now, you cannot add the generative part of the configuration after the collection was created.

you will need to create a new collection with it enabled, then move your data around using a migrate guide like this one:

Migrate data | Weaviate - Vector Database

[Growth_Catalyst]

@malgamves

code example

knowledgebase = client.collections.create(
            name=name,
            description="knowledge base",
            vectorizer_config=wvc.config.Configure.Vectorizer.text2vec_openai(),
            generative_config=wvc.config.Configure.Generative.openai(),
            skip_argument_validation=True,
            properties=[
                wvc.config.Property(name="title", description="title", data_type=wvc.config.DataType.TEXT,
                                    tokenization=Tokenization.LOWERCASE, skip_vectorization=True),
                wvc.config.Property(name="url", description="url", data_type=wvc.config.DataType.TEXT,
                                    tokenization=Tokenization.LOWERCASE,
                                    skip_vectorization=True),
                wvc.config.Property(name="data", description="data", data_type=wvc.config.DataType.TEXT,
                                    tokenization=Tokenization.LOWERCASE, vectorize_property_name=False,
                                    index_searchable=True, index_filterable=True),
            ],
        )

@DudaNogueira @Robert_Oschler

i want near text search to work only based on the property data . i am still getting the same error Query call with protocol GRPC search failed with message panic occurred: ValidateParam was called without any known params present. . can you please help in understanding what am i missing here ?

[DudaNogueira]

hi @Growth_Catalyst !! Welcome to our community

Can you share your query?

Considering you set to skip vectorization for title and url, the resulting vector of your objects will only be based on data property.

So you are all set to do a near text search only based on the data property.

Thanks!

[Growth_Catalyst]

@DudaNogueira thankss!!

this is my query

query = ["why should i do hernia surgery ?"]

response = knowledgebase.query.near_text(
    query=query, limit=3)

how can we do near text search only based on data property ?.. it’s not a named vector right ?
since we’re vectorizing only data property, near text search would work only based on data property right ?

[DudaNogueira]

Hi! I was only able to reproduce this error when a collection has no vectorizer.

Can you check that?

here is a reproducible example that have worked for me:

import weaviate
from weaviate import classes as wvc

client = weaviate.connect_to_local()

name="TestCollection"
client.collections.delete(name)
knowledgebase = client.collections.create(
    name=name,
    description="knowledge base",
    vectorizer_config=wvc.config.Configure.Vectorizer.text2vec_openai(),
    generative_config=wvc.config.Configure.Generative.openai(),
    skip_argument_validation=True,
    properties=[
        wvc.config.Property(name="title", description="title", data_type=wvc.config.DataType.TEXT,
                            tokenization=wvc.config.Tokenization.LOWERCASE, skip_vectorization=True),
        wvc.config.Property(name="url", description="url", data_type=wvc.config.DataType.TEXT,
                            tokenization=wvc.config.Tokenization.LOWERCASE,
                            skip_vectorization=True),
        wvc.config.Property(name="data", description="data", data_type=wvc.config.DataType.TEXT,
                            tokenization=wvc.config.Tokenization.LOWERCASE, vectorize_property_name=False,
                            index_searchable=True, index_filterable=True),
    ],
)
knowledgebase.data.insert({
    "title": "this is a title",
    "url": "http://weaviate.io",
    "data": "this is some data"
})

query = ["why should i do hernia surgery ?"]

response = knowledgebase.query.near_text(
    query=query, limit=3)
print(response.objects)

you can check the vectorizer of your collection with:

knowledgebase.config.get().vectorizer

Let me know if this helps

[Growth_Catalyst]

@DudaNogueira

knowledgebase.config.get().vectorizer

thanks @DudaNogueira
just realised that

knowledgebase = client.collections.get(“knowledgebase”)

the above statement creates a collection if one doesn’t exisit… i was expecting it to return None… it is because of this collection with the specified params was not created

thanks a lot for the help

[DudaNogueira]

Oh… in fact, it doesn’t create a collection.

however, it will silently return a weaviate.collections.collection.Collection object

This can be indeed confusing But the reasoning is to avoid unnecessary hits on the server.

So, as a good practice, you can check the existence of that collection:

collection_exists = client.collections.exists("NonExistant")
print(collection_exists)
>>> False

[Chase_Norton]

I am getting this same error, but only with the v3 Typescript client. If I follow the getting started guide with v2 TS, everything works. If I follow it using the v3, I get

You are using legacy vectorization. The vectorizer and vectorIndexConfig fields will be removed from the API in the future. Please use the vectorizers field instead to created specifically named vectorizers for your collection.

during collection creation and

WeaviateQueryError: Query call with protocol gRPC failed with message: /weaviate.v1.Weaviate/Search UNKNOWN: panic occurred: ValidateParam was called without any known params present

during nearText.

I am simply following: Quickstart Tutorial | Weaviate - Vector Database and have confirmed every step and all api keys are present. Again, if I swap out the code to v2 - everything works. Even, if I just do the collection creation and inserts with v2 and then swap over to v3 query, that works. Something is failing to vectorize in the v3 collection creation is my guess, but again, I’m just following the getting started guide.

Any help would be awesome