# Embeddings taking up more space than expected?

**URL:** <https://forum.weaviate.io/t/embeddings-taking-up-more-space-than-expected/235>\
**Category:** Support\
**Created:** [June 17, 2023, 11:58pm UTC](https://forum.weaviate.io/t/embeddings-taking-up-more-space-than-expected/235 "2023-06-17T23:58:40Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![frischoko](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/frischoko/32/204_2.png) [@frischoko](https://forum.weaviate.io/u/frischoko)\
**Post date:** [June 17, 2023, 11:58pm UTC](https://forum.weaviate.io/t/embeddings-taking-up-more-space-than-expected/235/1 "2023-06-17T23:58:40Z")

</div>

In my local Weaviate instance, I have about, 4500 objects. For each object, I generate the embeddings myself, as 512-dimensional floating point vectors. I have made sure to set `vectorizer: none` on the objects. So let’s say the embeddings are float 64.  
Each embedding is 512 units long, so 5000 embeddings should take up maybe like 20 megabytes, but disk usage (`du -h`) shows

```auto
1.1M .weaviate_data/image_3pnsXZxaxx5G.hnsw.commitlog.d
1.9M .weaviate_data/image_3pnsXZxaxx5G_lsm/property_name
35M .weaviate_data/image_3pnsXZxaxx5G_lsm/objects
526M .weaviate_data/image_3pnsXZxaxx5G_lsm/property_embedding
1.4M .weaviate_data/image_3pnsXZxaxx5G_lsm/property_path_searchable
556K .weaviate_data/image_3pnsXZxaxx5G_lsm/property__id
2.2M .weaviate_data/image_3pnsXZxaxx5G_lsm/property_path
1.1M .weaviate_data/image_3pnsXZxaxx5G_lsm/property_name_searchable
567M .weaviate_data/image_3pnsXZxaxx5G_lsm
568M .weaviate_data/

```

I’d just like to know where the additional memory usage is coming from? The only idea I have is that Weaviate is allocating more memory than necessary to store the vector. Thanks in advanced.

---

<div class="post-metadata">

**Author:** ![jphwang](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/jphwang/32/38_2.png) [@jphwang](https://forum.weaviate.io/u/jphwang)\
**Post date:** [June 20, 2023, 12:55pm UTC](https://forum.weaviate.io/t/embeddings-taking-up-more-space-than-expected/235/2 "2023-06-20T12:55:55Z")

</div>

Hi @frischoko 👋. That’s a good question. I _think_ it’s due to the way our data store works (it uses a [LSM-tree](https://en.wikipedia.org/wiki/Log-structured_merge-tree)).

But I’ve asked around internally to confirm. I’ll get back to you when I hear back. 🙂

---

<div class="post-metadata">

**Author:** ![antas-marcin](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/antas-marcin/32/20_2.png) [@antas-marcin](https://forum.weaviate.io/u/antas-marcin)\
**Post date:** [June 23, 2023, 7:12am UTC](https://forum.weaviate.io/t/embeddings-taking-up-more-space-than-expected/235/3 "2023-06-23T07:12:19Z")

</div>

Can you share with us also how are you sending the requests to create objects?  
Are you there using `embeddings` property to send vector or `vector`?

---

<div class="post-metadata">

**Author:** ![frischoko](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/frischoko/32/204_2.png) [@frischoko](https://forum.weaviate.io/u/frischoko)\
**Post date:** [June 29, 2023, 7:17pm UTC](https://forum.weaviate.io/t/embeddings-taking-up-more-space-than-expected/235/4 "2023-06-29T19:17:34Z")

</div>

I’m creating objects like this

```go
object := models.Object{
	Class: "Image",
	ID: strfmt.UUID(uuid),
	Properties: map[string]interface{}{
		"name": image.Name,
		"path": image.Path,
		"embedding": vector,
	},
	Vector: vector,
}

```

and inserting them with the batcher

```go
_, err = client.Batch().
	ObjectsBatcher().
	WithObjects(objects...). // Slice of object
	Do(context.Background())

```

---

<div class="post-metadata">

**Author:** ![antas-marcin](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/antas-marcin/32/20_2.png) [@antas-marcin](https://forum.weaviate.io/u/antas-marcin)\
**Post date:** [July 4, 2023, 5:43pm UTC](https://forum.weaviate.io/t/embeddings-taking-up-more-space-than-expected/235/6 "2023-07-04T17:43:24Z")

</div>

@frischoko the reason that you see that so much space is being taken by Weaviate is bc you are also storing the vector as a property, for which Weaviate builds an inverted index.

Weaviate builds inverted index for properties to enable users to perform keyword search, in your case you should not push a vector as a property (unless you want to perform keyword search on `embedding` field, which I think you don’t) you should only store your vector using `Vector` field.

```auto
object := models.Object{
	Class: "Image",
	ID: strfmt.UUID(uuid),
	Properties: map[string]interface{}{
		"name": image.Name,
		"path": image.Path,
// "embedding": vector, <- goes into inverted index for keyword search
	},
	Vector: vector, // <- goes into vector index for vector search
}

```