Collection does not exist exception in Python V4

General
,
1

Hi,

I am trying to insert data and I would expect Weaviate to throw an exception if the collection I am trying to insert into does not exist. In my system configuration file I set AUTOSCHEMA_ENABLED: 'false' like suggested in the documentation, however, I get no exception or any other error message if I am inserting data using a collection name that does not exist and was therefore not predefined.

Am I missing something or is there really no way of Weaviate warning me here?

2

Hi! I was not able to reproduce this.

For instance:

import weaviate
from weaviate import classes as wvc
client = weaviate.connect_to_local()
client.collections.delete("Test")

client.collections.exists("Test")
# False

test_col = client.collections.get("Test")
# note here: you  get the object instantiated, this doesn't mean the collection exists.

# try insert some data on non existant collection
test_col.data.insert({"text": "this"})

I get:

UnexpectedStatusCodeError: Object was not added! Unexpected status code: 422, with response body: {ā€˜errorā€™: [{ā€˜messageā€™: ā€˜invalid object: class ā€œTestā€ not found in schemaā€™}]}.

AUTOSCHEMA_ENABLED will mean that if there is a collection, without a previously created property, and you try to insert, Weaviate will raise an error. Like so:

col = client.collections.create("New")
col.data.insert({"non_existant": "property"})

Would return this error:

UnexpectedStatusCodeError: Object was not added! Unexpected status code: 422, with response body: {ā€˜errorā€™: [{ā€˜messageā€™: ā€œinvalid object: no such prop with name ā€˜non_existantā€™ found in class ā€˜Newā€™ in the schema. Check your schema files for which properties in this class are availableā€}]}.

Let me know if this helps :slight_smile:

3

Hi, thanks for your answer. I managed to reproduce this with your code, however, my issue occurred during bulk insert. The following code results in no error or exception:

test_col = client.collections.get("Test")

with client.batch.fixed_size(batch_size=100) as batch:
    for i in range(100):
        properties_dict = {
            'col1': "test1",
            'col2': "test2"
        }

        batch.add_object(
            collection="Test",
            properties=properties_dict,
            vector=[0, 1, 2]
        )
4

hi @ggapac !!

Sorry for the delay here.

But can you see the objects stored?

here a code with that:

import weaviate
import os
from weaviate import classes as wvc
client = weaviate.connect_to_local()
client.collections.delete("Test")

client.collections.create(
    "Test",
    vectorizer_config=wvc.config.Configure.Vectorizer.none()
)

test_col = client.collections.get("Test")

with client.batch.fixed_size(batch_size=100) as batch:
    for i in range(100):
        properties_dict = {
            'col1': "test1",
            'col2': "test2"
        }

        batch.add_object(
            collection="Test",
            properties={"text": "1"},
            vector=[0, 1, 2]
        )
print(client.batch.failed_objects)
# no fail

print(test_col.query.fetch_objects().objects[0:5])

Let me know if this helps!

Thanks!

5

Hi, this is not exactly what I am after. Let me explain my use case better.

I have different functions that take care of data insertion, record updates etc. The collection name is passed to these functions as a parameter. Rather than checking if a collection exists every single time I want to interact with the database, I would expect there is an exception that you can catch in this scenario. Let me illustrate my current and preferred insert function workflows in the following pseudocode:

current_insert_function(client, collection_name):
    if collection named collection_name exists:
         insert into collection_name

preferred_insert_function(client, collection_name):
    try:
         insert into collection_name
    catch if collection named collection_name does not exist:
        create collection
        repeat the process

I was just hoping that Weaviate has a way of throwing an exception if you are trying to do something with a non-existent collection, but with the bulk insert code I posted above, this is unfortunately not the case.

6

Oh, I see!!

You are right, this should raise some error.

Otherwise a lot of users can get confused, as it never raised no errors whatsoever.

I have opened an issue here that is somewhat related (on improving batch error handling):

I will link to this thread so we address that when the time comes.

Thanks!