SSL Verification failure

Hello, I am a beginner trying out Weaviate. However, when I ran a simple code, I encountered an error that I think is related to the SSL Certificate. I would like to know how to fix this issue, and whether Weaviate has a method to disable SSL verification. (I have updated the certifi library in Python.)

Thank You

This is my code:

This is result with error:

status_weaviate : True
Hello
E0822 10:59:44.842000000 5040 src/core/tsi/ssl_transport_security.cc:1654] Handshake failed with fatal error SSL_ERROR_SSL: error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED.
Traceback (most recent call last):
File “C:\software_dev_flook\KM_AI\KM-AI-V0-main\myenv\Lib\site-packages\weaviate\collections\grpc\query.py”, line 762, in __call
res = await self._connection.grpc_stub.Search(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\software_dev_flook\KM_AI\KM-AI-V0-main\myenv\Lib\site-packages\grpc\aio_call.py”, line 318, in await
raise _create_rpc_error(
grpc.aio._call.AioRpcError: <AioRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = “failed to connect to all addresses; last error: UNKNOWN: ipv4:34.98.85.103:443: Ssl handshake failed: SSL_ERROR_SSL: error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED”
debug_error_string = “UNKNOWN:Error received from peer {created_time:“2024-08-22T03:59:45.0595101+00:00”, grpc_status:14, grpc_message:“failed to connect to all addresses; last error: UNKNOWN: ipv4:34.98.85.103:443: Ssl handshake failed: SSL_ERROR_SSL: error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED”}”

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “C:\software_dev_flook\KM_AI\KM-AI-V0-main\test_weaviate.py”, line 36, in
response = jeopardy.query.hybrid(
^^^^^^^^^^^^^^^^^^^^^^
File “C:\software_dev_flook\KM_AI\KM-AI-V0-main\myenv\Lib\site-packages\weaviate\syncify.py”, line 23, in sync_method
return _EventLoopSingleton.get_instance().run_until_complete(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\software_dev_flook\KM_AI\KM-AI-V0-main\myenv\Lib\site-packages\weaviate\event_loop.py”, line 40, in run_until_complete
return fut.result()
^^^^^^^^^^^^
File “C:\Users\KHORPPUN\AppData\Local\Programs\Python\Python312\Lib\concurrent\futures_base.py”, line 456, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File “C:\Users\KHORPPUN\AppData\Local\Programs\Python\Python312\Lib\concurrent\futures_base.py”, line 401, in __get_result
raise self._exception
File “C:\software_dev_flook\KM_AI\KM-AI-V0-main\myenv\Lib\site-packages\weaviate\collections\queries\hybrid\query.py”, line 107, in
hybrid
res = await self._query.hybrid(
^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\software_dev_flook\KM_AI\KM-AI-V0-main\myenv\Lib\site-packages\weaviate\collections\grpc\query.py”, line 769, in __call
raise WeaviateQueryError(str(e), “GRPC search”) # pyright: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
weaviate.exceptions.WeaviateQueryError: Query call with protocol GRPC search failed with message <AioRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = “failed to connect to all addresses; last error: UNKNOWN: ipv4:34.98.85.103:443: Ssl handshake failed: SSL_ERROR_SSL: error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED”
debug_error_string = “UNKNOWN:Error received from peer {created_time:“2024-08-22T03:59:45.0595101+00:00”, grpc_status:14, grpc_message:“failed to connect to all addresses; last error: UNKNOWN: ipv4:34.98.85.103:443: Ssl handshake failed: SSL_ERROR_SSL: error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED”}”

Hi @Khorppun_Sontipanya

Thanks for the report. Let me see what we can do to help you.

One question up front – since you mentioned the connect_to_wcs method – is this a paid cluster or a free sandbox? If it’s a paid cluster you’re also eligible to contact support@weaviate.io – which may lead to faster responses. But also if it’s a free cluster, we’re happy to help you on a best effort basis right here.

Off the top of my head I can see potential causes:

  1. There could be some sort of a global SSL issue with Weaviate Cloud. I have already pinged the correct folks to investigate if this is the cause.
  2. It could be that something is off with your configuration and the error is an unfortunate error message obscuring the true error.

We’ll get back to you shortly!
-Etienne

Hi @Khorppun_Sontipanya

We have investigated Weaviate Cloud, but we do not see any SSL issues. We would like to help you resolve this as soon as possible. Could you please share the endpoint of the Weaviate Cluster you are using you are experiencing the issue with and send it to support@weaviate.io? Thank you.

This is my endpoint :
https://s9qiyurarzso9rrjklwfda.c0.us-central1.gcp.weaviate.cloud

Now, I try to use free sandbox.

This issue just recently occurred, I never had this issue before. it had never happened before. It only started happening the other day. Could it be related to a Windows update? Because after the Windows update, this problem started. However, some of my friends also have this problem, while others don’t when running the same code as I do.

Thank you for your support.

hi @Khorppun_Sontipanya !

What is the client version you are using?

you can get that information with:

import weaviate
print(weaviate.__version__)

Thanks!

hi @Khorppun_Sontipanya !!

Can you check if you have this library in your python environment?

pip freeze | grep certifi

if you don’t, please, install it:

pip install -U certifi

I checked the version of Weaviate; it is currently version 4.7.1.

And I also have ‘certifi’ in the Python environment, and I’ve already upgraded ‘certifi’.

image

Thank you very much for the effort to help.

hi @Khorppun_Sontipanya !!

Are you behind a corporate network by any chance?

We have identified some situations where, under this condition, it can affect the client connection.

We are working on a PR to with a workaround here:

To check if this is your situation, you access your cluster url with chrome, and do the following:

click the button at the left side of the url. click at Connection is secure:

and finally at Certificate is valid

this is the information you should see (of course, some details may change due to the AZ your cluster is at):

If you see anything other than Issued By “Google Trust Services”, you probably have a networking configuration that is terminating your SSL in between the client and server.

In this scenario, the CA CERT provided by certifi will not trust that issuer.

As a workaround you can replace certifi’s cacert.pm with the one you have.

you can find where this file is located issuing the following command:

python -m certifi

Let me know if this helps!

Thanks