← Work · 01 / Retrieval · advisory · Oyelabs

Cite or Decline

Answers built only from the user's own retrieved rows, or nothing — never the nearest passage passed off as the right one.

[ N ]
records embedded
0.60
similarity floor before an answer counts
0
cross-user rows
Python · FastAPI · PostgreSQL · pgvector · BAAI/bge-small-en-v1.5 · AWS Bedrock · Postgres RLSOyelabs2026Private
The 0.60 cut, one screenshot A candidate list with similarity scores, everything below the line greyed and dropped, the surviving chunks flowing into the answer with citations. One frame, no annotation.
Nearest is not relevant; the floor is where that distinction gets enforced.
01

The problem

A finance question has one right context and many plausible ones. "How much do I spend on coffee" retrieves the coffee runs, the café that also sells lunch, a refund, and a subscription with "cafe" in its descriptor. A vector index returns all five, ranked by distance. Distance is not correctness.

The failure is not that the model reasons badly. The failure is that it reasons well over the wrong rows and returns an answer that is confident, specific, and wrong — and in a finance app a wrong number that looks right is worse than no number at all. Baseline: before this, answers were grounded in the user's own data ~[ N ]% of the time — confirm or cut.

02

What changed

BeforeAfter
GroundingModel answers from general knowledgeAnswers only from the user's retrieved rows, or refuses
RetrievalTop-k nearest by cosineKeyword-first, vector fallback, below-0.60 dropped
Cross-item questionsOne passage at a timeBounded walk across the user's entity graph
ContinuityEach session starts coldDurable facts carried across sessions
IsolationApplication-layer filterPostgres RLS on every AI table

Dropping every chunk below a calibrated 0.60 cosine is what made the answers trustworthy. Everything else — the keyword short-circuit, the graph walk, the memory — feeds context; the floor is what decides whether that context is allowed to become an answer.

03

How it works

A question is embedded once to a 384-dim vector. A confident keyword match fills the slots and the vector scan is skipped; otherwise pgvector cosine runs over an HNSW index and drops anything under 0.60. The candidate pool is over-fetched and duplicate spends are collapsed, so the user gets top-k distinct rows even after RLS strips everyone else's. When chunks are thin, the knowledge graph is walked from entities named in the message — merchant, category — following weighted edges a fixed number of hops. Durable facts the user has stated before are pulled from memory into the same context. The model answers from that assembled set with citations, or it returns that it can't — and nothing outside the set ever reaches the prompt.

04

Architecture

  1. Embed. Financial text becomes a 384-dim bge vector, upserted by (source_type, source_id) so re-indexing a row updates in place.
  2. Retrieve. Keyword match short-circuits; else vector cosine over HNSW, gated by the 0.60 floor, duplicate spends collapsed.
  3. Walk. Seed entities from the message traverse knowledge_nodes → knowledge_edges to a capped hop count, each edge rendered as a statement.
  4. Remember. Confidence-gated facts from prior sessions load from user_memory_facts.
  5. Ground. The model cites the assembled set or refuses; nothing outside it is in scope.
  6. Isolate. A forced RLS policy user_id = get_current_user_id() fences every embedding, node, edge, and fact.
Architecture diagram The 0.60 floor as a horizontal line with candidates above and below it; the ones below visibly stop. The graph walk is a short branch off to the side, not the main spine.
Retrieval is the spine; the graph and memory are what the spine reaches for when it comes up short.
05

The decision

A vector index always returns something. Nearest is not the same as relevant.

The easy path is to take the top-k nearest chunks every time and hand them over. It always produces context, the demo is never empty, coverage is a clean 100%, and for most queries the nearest chunk is the right one — so it looks like it works.

It fails on the queries that matter. When a user asks something their data doesn't answer, the nearest chunk is still returned, and the model builds a fluent, wrong answer on top of it. So every chunk below a calibrated 0.60 cosine is dropped, and when nothing clears the floor the system returns nothing. The cost is coverage: on [ N ]% of questions it refuses rather than guess, measured over [ N ] questions — confirm.

The floor also moves the failure to where it's cheap. A miss now surfaces as "no context" at retrieval time, not as a confident wrong number a user has already read and acted on.

06

What's next

[ Confirm before publishing ] Full-text search alongside vectors, via tsvector, so exact terms — merchant codes, tickers — aren't lost to embeddings. A cross-encoder rerank before generation, to order what clears the floor. Graph population from Plaid webhooks, so the knowledge graph stays current without a backfill. Faithfulness scored in CI, not by eye.

What the data revealed — fill after the first eval run. Left blank rather than invented.

Not measured: end-answer correctness — only retrieval recall and whether context was grounded. No latency baseline captured. The 0.60 floor is calibrated on [ N ] sample queries, not a labelled set.