Check whether deterministic proxy to drop stale Weaviate context (Cuts token burn by ~50%). Stress-testing it this week

Support
,
1

Hey everyone,

I’ve been researching why enterprise RAG pipelines fail when vector databases scale. The silent killer is ‘Context Rot’, Weaviate retrieves a document that is semantically perfect (high cosine similarity) but factually outdated (e.g., superseded compliance rules or deprecated API specs).

To solve this, I built an open-source localhost proxy (KU-Gateway) that sits directly between Weaviate and the LLM.

It intercepts the retrieved payloads from Weaviate and runs a deterministic mathematical scoring function for temporal decay. If a chunk has decayed past the threshold, the proxy physically drops it before it hits the LLM context window.

I have $174 left in server credits expiring soon, so I’m hosting a 14-day “Zero to Revenue” Hackathon to stress-test the API. I want to see if the Weaviate community can build an agent that actually utilizes time-gated context to make a profit.

Repo for the math: [Link to GitHub] [KU-Gateway]
Grab a rate-limited key for the stress test: [Link to Landing Page]

Would love feedback on the decay function math from the engineers here.

2

Quick update on the decay function math for those asking:

The core logic handling the Weaviate payload interception uses a half-life decay model rather than a hard cutoff.

Freshness_Score = e^(-λt)

Where t is the delta between the current timestamp and the document’s last_updated metadata field, and λ is dynamically adjusted based on the volatility of the source (e.g., API docs have a higher λ than historical compliance records).

If the Freshness_Score drops below the minimum acceptable threshold (currently 0.5 for high-risk trading agents), the proxy intercepts the Weaviate response, excises that specific chunk, and forwards the cleaned JSON to the LLM.

The Uvicorn traces handling the rejection are live in the repo now if anyone wants to review the overhead latency (it’s currently adding <15ms).