Some deleted collections are being restored after restarting Docker Weaviate

Support
1

Description

I’m running Weaviate in Docker for development purposes. I wanted to clean up some collections left over from testing and noticed that they will always be back after restarting the Weaviate Docker container.

I’m deleting the collections via:

client = weaviate.connect_to_custom(
    http_host="localhost",
    http_port=8084,
    http_secure=False,
    grpc_host="localhost",
    grpc_port=50051,
    grpc_secure=False
)

client.collections.delete("knowledge_foobar")

client.close()

Docker compose file:

services:
  weaviate:
    command:
    - --host
    - 0.0.0.0
    - --port
    - '8080'
    - --scheme
    - http
    image: cr.weaviate.io/semitechnologies/weaviate:1.32.5
    ports:
    - 8084:8080
    - 50051:50051
    volumes:
    - "C:/Users/<USER>/Weaviate/weaviate_data:/var/lib/weaviate"
    restart: on-failure:0
    environment:
      QUERY_DEFAULTS_LIMIT: 25
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
      PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
      DEFAULT_VECTORIZER_MODULE: 'none'
      ENABLE_MODULES: ''
      CLUSTER_HOSTNAME: 'node1'

The reason I’m mounting the data from a folder in Windows might be related. I was having problems with data just disappearing after restarting the Docker container. I did not have that problem when mounting the Windows folder.

Server Setup Information

  • Weaviate Server Version: 1.32.5
  • Deployment Method: Docker Desktop on Windows / WSL
  • Multi Node? Number of Running Nodes: 1
  • Client Language and Version: Python, 4.16.9
  • Multitenancy?: Yes

Any additional Information

Before deleting:

grafik
grafik362×103 2.41 KB

After deleting:

grafik

After restarting:

grafik
grafik363×105 2.47 KB

”Change date” shows the folder has been recreated.

This does NOT happen with all collections. I tried to create a some new collections, but they can be deleted normally.

E.g. I can delete the new collection “knowledge_foo” which has been created the same way as the other collections.

import weaviate
from weaviate.classes.config import Configure, Property, DataType

client = weaviate.connect_to_custom(
    http_host="localhost",
    http_port=8084,
    http_secure=False,
    grpc_host="localhost",
    grpc_port=50051,
    grpc_secure=False
)

collection = client.collections.create(
    name="knowledge_foo",
    multi_tenancy_config=Configure.multi_tenancy(
        enabled=True,
        auto_tenant_creation=True,
        auto_tenant_activation=True,
    ),
    properties=[
        Property(name="text", data_type=DataType.TEXT),
        Property(name="foo_id", data_type=DataType.INT),
        Property(name="bar_id", data_type=DataType.INT),
    ],
)

collection.config.add_vector(
    vector_config=Configure.Vectors.self_provided(name="vector_foo")
)

tenant_collection = collection.with_tenant("tenant_1")

tenant_collection.data.insert(
    properties={
        "text": "Text1",
        "foo_id": 1,
        "bar_id": 2,
    },
    vector={
        "vector_foo": [1, 2, 3],
    },
)
2

Hi @torbenw, thanks for reaching out.

Could you please provide more details about the collection names (casesenstive) you’re trying to delete? Specifically:

  • How do the collection names appear in the deletion request? e.g. foobar

  • In what form do they reappear afterward when you call v1/schema or trying to get the schema from client ? e.g. FooBar

I’m asking to help validate whether this issue is related to case sensitivity. We’ve recently discovered a bug involving case sensitivity, so this information would be helpful.

Thanks again

3

Huh… you’re right!

I’m not using the correct class names for the collections that will be restored after restarting.

client.collections.delete("knowledge_foobar")

Deletes the collection, but it will be restored after restarting Weaviate.

client.collections.delete("Knowledge_FOOBAR")

Deletes the collection and it will NOT be restored after restarting Weaviate.

4

Good, thank you so much for trying it out. we will release a fix next week to handle this case @torbenw :folded_hands: