Question on setting up a local weaviate server from source with a local contextionary

General
1

I am trying to run a local weaviate server from source with a local contextionary in order to trace and study the code. Here is the docker compose file with just the contextionary service.

version: '3.4'
services:
  contextionary:
    image: semitechnologies/contextionary:en0.16.0-v1.2.1
    ports:
      - "9999:9999"
    environment:
      OCCURRENCE_WEIGHT_LINEAR_FACTOR: 0.75
      EXTENSIONS_STORAGE_MODE: weaviate
      EXTENSIONS_STORAGE_ORIGIN: http://weaviate:8080

It looks contextionary depending on a running weaviate instance. Is there a way to run contextionary without weaviate as its extensions storage? And what is the EXTENSIONS_STORAGE used for?

2

Hi!

On that case, as you are running Weaviate from source (I suppose you are not running it inside a container, right?), the contextionary will not be able to reach Weaviate.

EXTENSIONS_STORAGE_ORIGIN will define where is the storage for your contextionary.

Let me know if this helps.

3

The weaviate is not running in a container. Is EXTENSIONS_STORAGE must be configured for contextionary? Can I run contextionary without extensions storage. I would like to have a very simple setup so that I can track the weaviate code easily.

4

On that case, using

EXTENSIONS_STORAGE_ORIGIN: http://weaviate:8080

will not work, as the hostname weaviate will be unreachable from the contextionary container.

You will need to define that value as the host ip of your docker that should be accessible from that container.

This can change depending on where you are running your docker stack.

Check here for more on that:

I have not used the contextionary that much (this is a fairly old module, and because of the new vectors embeddings for sentences, it’s not being used very often), but I believe it is necessary to store the context extensions inside Weaviate.

I believe your best option here is to build you own Weaviate container from the source code you want, so you can still use the docker image and doesn’t need to handle your inference models connecting to Weaviate (or vice versa) running directly from binary.

Let me know if that helps!

5

Thanks. I may use another vector embedder such as the sentence transformer which does not require storing data inside Weaviate.

1 Like