# Avoid inserting dupes

**URL:** <https://forum.weaviate.io/t/avoid-inserting-dupes/1439>\
**Category:** Support\
**Created:** [February 9, 2024, 3:29pm UTC](https://forum.weaviate.io/t/avoid-inserting-dupes/1439 "2024-02-09T15:29:20Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![rjalex](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/rjalex/32/807_2.png) [@rjalex](https://forum.weaviate.io/u/rjalex)\
**Post date:** [February 9, 2024, 3:29pm UTC](https://forum.weaviate.io/t/avoid-inserting-dupes/1439/1 "2024-02-09T15:29:20Z")

</div>

I am inserting elements with three columns:

- kicker (this is unique)
- slug (this is unique)
- vector (the computed embedding vector of the kicker)  
as follows (where data is a list of json elements):

```auto

batch_size = int(os.getenv("BATCH_SIZE")) 
client.batch.configure(batch_size=batch_size) # Configure batch

# Batch import all objects
with client.batch as batch:
  for item in data:
      properties = {
         textcolname: item[textcolname],
         "slug": item["slug"],
      }
      client.batch.add_data_object(
         class_name=schema_name,
         data_object=properties,
         vector=item[vectcolname] # my vector embeddings go here
      )

```

is there a way to avoid inserting an item if the kicker (which is unique) is already inserted? (in this case also the vector and the slug would be there already)

---

<div class="post-metadata">

**Author:** ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)\
**Post date:** [February 9, 2024, 11:10pm UTC](https://forum.weaviate.io/t/avoid-inserting-dupes/1439/2 "2024-02-09T23:10:41Z")

</div>

Hi @rjalex !

You want to have deterministic ids, by providing your own uuid based on a property or properties.

Check here:

> **[Create objects | Weaviate - Vector Database](https://weaviate.io/developers/weaviate/manage-data/create#generate-deterministic-ids)**
>
> The examples on this page demonstrate how to create individual objects in Weaviate.

Also, we highly recommend using python v4 as it will significantly improves performance:

> **[Batch import | Weaviate - Vector Database](https://weaviate.io/developers/weaviate/manage-data/import#basic-import)**
>
> Batch imports are an efficient way to add multiple data objects and cross-references.

Let me know if that helps 🙂

---

<div class="post-metadata">

**Author:** ![rjalex](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/rjalex/32/807_2.png) [@rjalex](https://forum.weaviate.io/u/rjalex)\
**Post date:** [February 10, 2024, 11:34am UTC](https://forum.weaviate.io/t/avoid-inserting-dupes/1439/3 "2024-02-10T11:34:24Z")

</div>

Very clear. Muito obrigado.