Im following the in Quickstart but getting unmarshal response body error invalid number literal

Support
1

Below is the error i get when i run the code mentioned in the quickStart page of weaviate website, im a beginner when it comes to these thing please help me figure out the issue

$ python ingest.py # running the python file
importing question: 1
importing question: 2
importing question: 3
importing question: 4
importing question: 5
importing question: 6
importing question: 7
importing question: 8
importing question: 9
importing question: 10
{'error': [{'message': 'update vector: unmarshal response body: json: invalid number literal, trying to unmarshal "\\"insufficient_quota\\"" into Number'}]}
{'error': [{'message': 'update vector: unmarshal response body: json: invalid number literal, trying to unmarshal "\\"insufficient_quota\\"" into Number'}]}
{'error': [{'message': 'update vector: unmarshal response body: json: invalid number literal, trying to unmarshal "\\"insufficient_quota\\"" into Number'}]}
{'error': [{'message': 'update vector: unmarshal response body: json: invalid number literal, trying to unmarshal "\\"insufficient_quota\\"" into Number'}]}
{'error': [{'message': 'update vector: unmarshal response body: json: invalid number literal, trying to unmarshal "\\"insufficient_quota\\"" into Number'}]}
{'error': [{'message': 'update vector: unmarshal response body: json: invalid number literal, trying to unmarshal "\\"insufficient_quota\\"" into Number'}]}
{'error': [{'message': 'update vector: unmarshal response body: json: invalid number literal, trying to unmarshal "\\"insufficient_quota\\"" into Number'}]}
{'error': [{'message': 'update vector: unmarshal response body: json: invalid number literal, trying to unmarshal "\\"insufficient_quota\\"" into Number'}]}
{'error': [{'message': 'update vector: unmarshal response body: json: invalid number literal, trying to unmarshal "\\"insufficient_quota\\"" into Number'}]}
{'error': [{'message': 'update vector: unmarshal response body: json: invalid number literal, trying to unmarshal "\\"insufficient_quota\\"" into Number'}]}

this is my code which im following from quickstart page of weaviate website, i have modified the API keys also the json link mentioned in quickstart, that link does not work so i have updated it, please let me know if im missing something
‘https://raw.githubusercontent.com/weaviate-tutorials/quickstart/main/data/jeopardy_tiny.json’ this link doesnt work it gives a connection timed out error, i have replaced it with “https://githubraw.com/weaviate/weaviate-examples/main/jeopardy_small_dataset/jeopardy_tiny.json”

import weaviate
import json
import requests
import json

client = weaviate.Client(
    url = "https://some-endpoint.weaviate.network",  
    auth_client_secret=weaviate.AuthApiKey(api_key=""),  instance API key
    additional_headers = {
        "X-OpenAI-Api-Key": ""  
    }
)

# ===== define collection =====
class_obj = {
    "class": "Question",
    "vectorizer": "text2vec-openai",
    "moduleConfig": {
        "text2vec-openai": {},
        "generative-openai": {} 
    }
}

client.schema.create_class(class_obj)

# ===== import data =====
resp = requests.get()
data = json.loads(resp.text) 

client.batch.configure(batch_size=100)
with client.batch as batch:
    for i, d in enumerate(data):
        print(f"importing question: {i+1}")
        properties = {
            "answer": d["Answer"],
            "question": d["Question"],
            "category": d["Category"],
        }
        batch.add_data_object(
            data_object=properties,
            class_name="Question"
        )
2

Hi @Throw_Away ! Welcome to our community :slight_smile:

This error is because your API KEY have a quota limitation.

You will need to check with OPENAI and get an API KEY with a bigger quota.

Let me know if that helps :slight_smile:

3

Hi Weaviate,

I am having the same issue. If you say that this is an issue with my openai api key quotas, then sure, although I dont understand how … it would be good to know the details.

The openai account I am using is a few months old and openai usage section informs me that I have used up all of my trial credits - most of them having been expired by openai w/o me ever using them.

Given this, I am wondering if there is a way to set this up a free service like bard, something on huggingface or using a llm running locally?

Thanks for your attention.

4

To resolve the issue in my case, i set up a azure openAI resource with standard tier and updated the code

from unstructured.partition.pdf import partition_pdf
from pathlib import Path
import weaviate
import os

client = weaviate.Client(
    url="https://<WEAVIATE_CLIENT>.weaviate.network",
    auth_client_secret=weaviate.AuthApiKey(
        api_key="" #auth key
    ),
    additional_headers={
        "X-Azure-Api-Key": "",
        # "X-OpenAI-Api-Key": "" # if using openAI key
    },
)

client.schema.delete_all()

schema = {
    "class": "Document",
    "vectorizer": "text2vec-openai",
    "properties": [
        {
            "name": "source",
            "dataType": ["text"],
        },
        {
            "name": "abstract",
            "dataType": ["text"],
            "moduleConfig": {
                "text2vec-openai": {"skip": False, "vectorizePropertyName": False}
            },
        },
    ],
    "moduleConfig": {
        "text2vec-openai": {
            "baseURL": "https://<RESOURCE_NAME>.openai.azure.com/",
            "resourceName": "<RESOURCE_NAME>",
            "deploymentId": "<DEPLOYMENT_NAME>",
        },
        "generative-openai": {
            "model": "gpt-3.5-turbo",
            "baseURL": "https://<RESOURCE_NAME>.openai.azure.com/",
            "resourceName": "<RESOURCE_NAME>",
            "deploymentId": "<DEPLOYMENT_NAME>",
        },
    },
}

client.schema.create_class(schema)

data_folder = "data" #folder path which contains all documents

data_objects = []

for path in Path(data_folder).iterdir():
    print(path)
    if path.suffix != ".pdf":
        continue

    print(f"Processing {path.name}...")

    elements = partition_pdf(filename=path)

    abstract_extractor = AbstractExtractor()
    abstract_extractor.consume_elements(elements)

    data_object = {"source": path.name, "abstract": abstract_extractor.abstract()}

    data_objects.append(data_object)


client.batch.configure(batch_size=100)  # Configure batch
with client.batch as batch:
    for data_object in data_objects:
        batch.add_data_object(data_object, "Document")

when i ran this script i was able to upload document to weaviateDB, the issue was because i didn’t have enough tokens to create embedding.

1 Like