# Fast start code sample and/or article on using a weaviate production cloud based cluster?

**URL:** https://forum.weaviate.io/t/fast-start-code-sample-and-or-article-on-using-a-weaviate-production-cloud-based-cluster/2017
**Category:** Support
**Tags:** wcs
**Created:** [April 14, 2024, 10:30pm UTC](https://forum.weaviate.io/t/fast-start-code-sample-and-or-article-on-using-a-weaviate-production-cloud-based-cluster/2017 "2024-04-14T22:30:48Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![Robert\_Oschler](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/robert_oschler/32/1308_2.png) [@Robert\_Oschler](https://forum.weaviate.io/u/Robert_Oschler)
#### Post date: [April 14, 2024, 10:30pm UTC](https://forum.weaviate.io/t/fast-start-code-sample-and-or-article-on-using-a-weaviate-production-cloud-based-cluster/2017/1 "2024-04-14T22:30:48Z")

</div>

**USE CASE** : I have a collection of web pages as HTML files that will be the basis of my LLM based chat-bot project. I will handle the chunking of the documents myself.My experience in the past has been limited to using the _OpenAI_ API to create the embedding vectors from text chunks and write them to a file,. Then I used that same service to create the embedding vector of the user input before executing a query. To execute a query I would do a linear search through the monolithic target embeddings file using the cosine similarity function to rank and select the top matches.

I now am switching to a **cloud based weaviate instance**. I have created my **production** cluster (_not_ sandbox), and have done the quickstart tutorial using my cluster URL and API key (admin, read/write). It works fine.

I am hoping to find a code sample that will show me how to use my cloud based weaviate cluster to do what I was doing before, using the Open AI embeddings API and my cosine similarity function. Except now of course, I want to use the preconfigured **vectorizing** modules that are part of my weaviate cluster and to use the weaviate cloud based **similarity search** to select the top matches.

Is there a good sample that will show me quickly how to:

- Set up and configure my cluster (e.g. - create a schema, if that is necessary, etc.)
- Attach metadata to each chunk that I store in my weaviate cluster. I need to attach the source URL of the HTML file the chunk belonged to, including a few other fields.
- Vectorize a chunk and store it in my cluster using the JS/TS client
- Vectorize a user query and then run a similarity search, including showing me how to use the most common similarity search parameters (e.g. - maximum number of matches, minimum similarity score, etc.)
- Any performance or optimization tips that would help me

A good sample that shows me how to do these operations would really help get me up to speed quickly and save me a lot of time over perusing through many pages in the API reference.

Also, please list the name of any **Medium** authors that write good articles on using a weaviate cloud cluster.

---

<div class="post-metadata">

### Author: ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)
#### Post date: [April 15, 2024, 10:44am UTC](https://forum.weaviate.io/t/fast-start-code-sample-and-or-article-on-using-a-weaviate-production-cloud-based-cluster/2017/2 "2024-04-15T10:44:36Z")

</div>

hi @Robert_Oschler !! Welcome to our community 🤗

Have you seen our quickstart?

> **[Quickstart Tutorial | Weaviate - Vector Database](https://weaviate.io/developers/weaviate/quickstart)**
>
> Overview

Apart from the chunking, it will give you a pretty straightforward path to get it starting.

Also, we have some recipes in this repo:

> **[GitHub - weaviate/recipes: This repository shares end-to-end notebooks on how...](https://github.com/weaviate/recipes)**
>
> This repository shares end-to-end notebooks on how to use various features and integrations with Weaviate at the core! - weaviate/recipes

That can give you working examples on different features.

Notice that most of those steps (vectorize the chunk, vectorize user query, searching, etc) Weaviate will take care for your. For that you need to pass your OpenAI key when initializing the client.

Let me know if you need further assistance. We are here to help!

Thanks!

---

<div class="post-metadata">

### Author: ![Robert\_Oschler](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/robert_oschler/32/1308_2.png) [@Robert\_Oschler](https://forum.weaviate.io/u/Robert_Oschler)
#### Post date: [April 15, 2024, 4:34pm UTC](https://forum.weaviate.io/t/fast-start-code-sample-and-or-article-on-using-a-weaviate-production-cloud-based-cluster/2017/3 "2024-04-15T16:34:08Z")

</div>

Thanks. That sample looks pretty good so I will go through it. Regarding chunking, any tips on a good size for a typical chunk in the “Goldilocks zone” (not too large, not to small)?

That’s **great** that Weaviate handles the vectorizing. I was using OpenAI for my embeddings work in my linear file trials so I already have an API key, and that gives me confidence because I know their embedding endpoint works well. Hopefully they are not having performance issues as I have been having lately with _ChatGPTPlus_ (network errors, slow generation)?

Regarding LLM usage. I want to do that myself because I have custom code for orchestrating the prompt “massaging” I do, using the selection results from the RAG content selection operation. In other words, I will be using Weaviate for the pre-filtering operation where I take a user input (query) and do a search to get the N-best matches that I pass to the LLM for the completion text operation, and for that operation I will be using the **Gemini 1.5 Pro** API for my LLM work. Is there anything I need to know about doing that? I briefly scanned the quickstart you just mentioned, and it appears to assume that Weaviate will be handling the LLM interaction, which I don’t want. That is why I’m asking this question.

---

<div class="post-metadata">

### Author: ![DudaNogueira](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/dudanogueira/32/7846_2.png) [@DudaNogueira](https://forum.weaviate.io/u/DudaNogueira)
#### Post date: [April 17, 2024, 6:34pm UTC](https://forum.weaviate.io/t/fast-start-code-sample-and-or-article-on-using-a-weaviate-production-cloud-based-cluster/2017/4 "2024-04-17T18:34:40Z")

</div>

Hi!

For chunking, unfortunately, there is no one size fits all.

We have a nice documentation here that may give you some background:

> **[Chunking long texts | Weaviate - Vector Database](https://weaviate.io/developers/academy/py/standalone/chunking)**
>
> Unit overview

One thing you could do, to make the chunking, and the fine control over the generation step, is to leverage a tool like langchain.

I happened to update today a recipe that can be used with that:

> **[recipes/integrations/langchain/loading-data at main · weaviate/recipes](https://github.com/weaviate/recipes/tree/main/integrations/langchain/loading-data)**
>
> This repository shares end-to-end notebooks on how to use various features and integrations with Weaviate at the core! - weaviate/recipes

notice that you can use different LLMs that langchain support for generating the answers, and this will give you the ability to have more control over the documents you want to use for that generation.

Let me know if that helps.

---

<div class="post-metadata">

### Author: ![Robert\_Oschler](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.weaviate.io/robert_oschler/32/1308_2.png) [@Robert\_Oschler](https://forum.weaviate.io/u/Robert_Oschler)
#### Post date: [April 18, 2024, 12:39am UTC](https://forum.weaviate.io/t/fast-start-code-sample-and-or-article-on-using-a-weaviate-production-cloud-based-cluster/2017/5 "2024-04-18T00:39:29Z")

</div>

Replying to help others who are looking for a good starting point for chunking parameters. From the Weaviate chunking tutorial:

_For search with fixed-size chunks, if you don’t have any other factors, try a size of around 100-200 words, and a 20% overlap._

> **[Chunking techniques - 1 | Weaviate - Vector Database](https://weaviate.io/developers/academy/py/standalone/chunking/how_1)**
>
> \<!-- import imageUrl from '../../tmpimages/academyplaceholder.jpg';
