Monitoring Progress When Adding to Weaviate Collection

Support
1

Description

As a New User to Weaviate I would like to be able to monitor progress when I am uploading vector chunks to Weaviate. Did the collection get created? How many chunks are currently loaded successfuly. Did any fail to load successfully etc.

So I am looking for something like a quick CLI or UI to see this sort of thing. After installing weaviate I did not see an Admin UI and searched for a CLI as well but saw nothing in the documentation about this. So I would like to know how people are quickly viewing these kinds of things. I have tried logging at info level and there is not much. At debug it’s quite verbose but not that clear what’s happening. At trace I image alot of stuff the filter through. I also want to be able in generall to quickly view all collections in weaviate etc.

Server Setup Information

  • Weaviate Server Version: Latest
  • Deployment Method: docker compose running in debug mode
  • Multi Node? Number of Running Nodes: Single Node
  • Client Language and Version: Latest

Any additional Information

2

hi @Draco ! Welcome to our community :hug:!

Weaviate will expose metrics that can be monitored with tools like grafana/datadog/prometheus.

We have here a blog post on that:

Let me know if this helps.

Thanks!

3

That’s great. I would like to be able to use cloud based montiroing to Weaviate. I am also looking for a way to inspect the collections that have loaded or are loading during chunking time.

4

There is an endpoint that can give you some information on that:

http://localhost:8080/v1/nodes?output=verbose

Note: just found out the output=verbose is not documented on that api specs. :grimacing:

5

Ok that’s good to be aware of. I will try it!

1 Like