Unclear error

can anyway tell me what this error is about?

{'error': [{'message': 'ref batch: prop isAuthorOf is present, but not a ref, got: []interface {}'}]}

I got it while using client.batch. The program didn’t raise an explicit error or anything, it just printed this line in the log. No idea what to do or if there’s anything potentially wrong

Hi! Welcome to our community!!

Can you share the schema and code that you are using? Also a snippet of data set that we can use to reproduce it?

Thanks!

The field is from the class User with the following schema:

{'class': 'User',
 'invertedIndexConfig': {'bm25': {'b': 0.75, 'k1': 1.2},
  'cleanupIntervalSeconds': 60,
  'indexNullState': True,
  'stopwords': {'additions': None, 'preset': 'en', 'removals': None}},
 'moduleConfig': {'text2vec-openai': {'model': 'ada',
   'modelVersion': '002',
   'type': 'text',
   'vectorizeClassName': False}},
 'properties': [{'dataType': ['int'],
   'description': 'ID of the postgres record of the place',
   'indexFilterable': True,
   'indexSearchable': False,
   'moduleConfig': {'text2vec-openai': {'skip': True,
     'vectorizePropertyName': False}},
   'name': 'postgresId'},
  {'dataType': ['text'],
   'description': 'Name of the user',
   'indexFilterable': True,
   'indexSearchable': True,
   'moduleConfig': {'text2vec-openai': {'skip': False,
     'vectorizePropertyName': False}},
   'name': 'name',
   'tokenization': 'word'},
  {'dataType': ['text'],
   'description': 'Username of the user',
   'indexFilterable': True,
   'indexSearchable': True,
   'moduleConfig': {'text2vec-openai': {'skip': False,
     'vectorizePropertyName': False}},
   'name': 'username',
   'tokenization': 'word'},
  {'dataType': ['text'],
   'description': 'Email of the place',
   'indexFilterable': True,
   'indexSearchable': True,
   'moduleConfig': {'text2vec-openai': {'skip': True,
     'vectorizePropertyName': False}},
   'name': 'email',
   'tokenization': 'word'},
  {'dataType': ['text'],
   'description': 'Role of the place',
   'indexFilterable': True,
   'indexSearchable': True,
   'moduleConfig': {'text2vec-openai': {'skip': True,
     'vectorizePropertyName': False}},
   'name': 'role',
   'tokenization': 'word'},
  {'dataType': ['Document'],
   'description': 'is author of documents',
   'indexFilterable': True,
   'indexSearchable': False,
   'moduleConfig': {'text2vec-openai': {'skip': False,
     'vectorizePropertyName': False}},
   'name': 'isAuthorOf'},
  {'dataType': ['Knowledge'],
   'description': 'is mentioned in',
   'indexFilterable': True,
   'indexSearchable': False,
   'moduleConfig': {'text2vec-openai': {'skip': False,
     'vectorizePropertyName': False}},
   'name': 'isMentionedIn'}],
 'replicationConfig': {'factor': 1},
 'shardingConfig': {'virtualPerPhysical': 128,
  'desiredCount': 1,
  'actualCount': 1,
  'desiredVirtualCount': 128,
  'actualVirtualCount': 128,
  'key': '_id',
  'strategy': 'hash',
  'function': 'murmur3'},
 'vectorIndexConfig': {'skip': False,
  'cleanupIntervalSeconds': 300,
  'maxConnections': 64,
  'efConstruction': 128,
  'ef': -1,
  'dynamicEfMin': 100,
  'dynamicEfMax': 500,
  'dynamicEfFactor': 8,
  'vectorCacheMaxObjects': 1000000000000,
  'flatSearchCutoff': 40000,
  'distance': 'cosine',
  'pq': {'enabled': False,
   'bitCompression': False,
   'segments': 0,
   'centroids': 256,
   'encoder': {'type': 'kmeans', 'distribution': 'log-normal'}}},
 'vectorIndexType': 'hnsw',
 'vectorizer': 'text2vec-openai'}

The code I was using is this:

def create_document_for_learning(client, document_text, author_uuid):
  with client.batch as batch:
    document_uuid_props = {'content': document_text, 'author_uuid': author_uuid}
    document_uuid = generate_uuid5(document_uuid_props, 'Document')

    document_does_not_exist = not client.data_object.exists(uuid=document_uuid, class_name='Document')
    if document_does_not_exist:
      batch.add_data_object(
        data_object={'content': document_text},
        class_name="Document",
        uuid=document_uuid,
      )

    # document - 'belongsToAuthor' - author reference
    batch.add_reference(
      from_object_uuid=document_uuid,
      from_object_class_name='Document',
      from_property_name='belongsToAuthor',
      to_object_uuid=author_uuid,
      to_object_class_name='User',
    )
    # author - 'isAuthorOf' - document reference
    batch.add_reference(
      from_object_uuid=author_uuid,
      from_object_class_name='User',
      from_property_name='isAuthorOf',
      to_object_uuid=document_uuid,
      to_object_class_name='Document',
    )
  return document_uuid

Before that I also remove some references, using the functions below:

def delete_existing_doc_knowledge_labels(env, learning_doc_uuid):
  client = initiate_weaviate_client(env=env)
  removable_uuids, unremovable_uuids = __get_removable_and_unremovable_object_uuids(client, learning_doc_uuid)
  __remove_references('Label', unremovable_uuids['Label'], 'belongsToKnowledge', 'Knowledge', removable_uuids['Knowledge'], client)
  __remove_references('Place', unremovable_uuids['Place'], 'isMentionedIn', 'Knowledge', removable_uuids['Knowledge'], client)
  __remove_references('User', unremovable_uuids['User'], 'isMentionedIn', 'Knowledge', removable_uuids['Knowledge'], client)
  __remove_references('User', unremovable_uuids['User'], 'isAuthorOf', 'Document', removable_uuids['Document'], client)
  __remove_records_by_uuids(removable_uuids, client)
  return removable_uuids

def __remove_references(from_class, from_uuids, reference_property, to_class, removable_to_uuids, client):
  for from_uuid in from_uuids:
    from_entity = client.data_object.get_by_id(from_uuid, class_name=from_class)
    to_uuids_to_keep = []

    for reference in from_entity['properties'].get(reference_property, []):
      to_uuid = reference['beacon'].split('/')[-1]
      if to_uuid in removable_to_uuids: continue
      to_uuids_to_keep.append(to_uuid)

    client.data_object.reference.update(
      from_uuid = from_uuid,
      from_property_name = reference_property,
      to_uuids = to_uuids_to_keep,
      from_class_name=from_class,
      to_class_names=to_class,
    )

I suspect that the removal step, where I essentially overwrite the object’s references with an empty array (if to_uuids_to_keep is empty), may be the issue? not sure

Hi! Sorry for the delay here :grimacing: lost track of this thread. :frowning:

Were you able to solve it by now?

Also, what is the version you are running?

Thanks!

Nope it’s still happening. It usually happens after I set the reference field of an object to None, or an empty array (to delete all references from that object). Then when adding new references using batch, the error is printed out.

Weaviate version 1.19.12
Weaviate python client version: 3.18.0

Have you tried removing the references using the reference.delete method? That might be the issue here.

client.data_object.reference.delete(
    from_class_name="JeopardyQuestion",
    from_uuid=sf_id,
    from_property_name="hasCategory",
    to_class_name="JeopardyCategory",
    to_uuid=museums_id,
)

This error still happens.

How to reproduce.

  1. Create reference between source uuid and target uuid
  2. Delete the reference using : weaviate.WeaviateClient.collections.get(collection_name).data.reference_delete(source_uuid, reference_name, target_uuid)
  3. try to create a new reference on the source_uuid

This error will happen.
weaviate-client on python: 4.4.2
weaviate image: 1.23.9

Hi @GameSetAndMatch !

Sorry for the delay here and also for the request.

But… could you create a python notebook with the steps to reproduce this?

This really helps as sometimes a small piece of code before this outcome is what helps to finding the issue.

Thanks!