Answers built only from the user's own retrieved rows, or nothing — never the nearest passage passed off as the right one.
A finance question has one right context and many plausible ones. "How much do I spend on coffee" retrieves the coffee runs, the café that also sells lunch, a refund, and a subscription with "cafe" in its descriptor. A vector index returns all five, ranked by distance. Distance is not correctness.
The failure is not that the model reasons badly. The failure is that it reasons well over the wrong rows and returns an answer that is confident, specific, and wrong — and in a finance app a wrong number that looks right is worse than no number at all. Baseline: before this, answers were grounded in the user's own data ~[ N ]% of the time — confirm or cut.
| Before | After | |
|---|---|---|
| Grounding | Model answers from general knowledge | Answers only from the user's retrieved rows, or refuses |
| Retrieval | Top-k nearest by cosine | Keyword-first, vector fallback, below-0.60 dropped |
| Cross-item questions | One passage at a time | Bounded walk across the user's entity graph |
| Continuity | Each session starts cold | Durable facts carried across sessions |
| Isolation | Application-layer filter | Postgres RLS on every AI table |
Dropping every chunk below a calibrated 0.60 cosine is what made the answers trustworthy. Everything else — the keyword short-circuit, the graph walk, the memory — feeds context; the floor is what decides whether that context is allowed to become an answer.
A question is embedded once to a 384-dim vector. A confident keyword match fills the slots and the vector scan is skipped; otherwise pgvector cosine runs over an HNSW index and drops anything under 0.60. The candidate pool is over-fetched and duplicate spends are collapsed, so the user gets top-k distinct rows even after RLS strips everyone else's. When chunks are thin, the knowledge graph is walked from entities named in the message — merchant, category — following weighted edges a fixed number of hops. Durable facts the user has stated before are pulled from memory into the same context. The model answers from that assembled set with citations, or it returns that it can't — and nothing outside the set ever reaches the prompt.
user_id = get_current_user_id() fences every embedding, node, edge, and fact.A vector index always returns something. Nearest is not the same as relevant.
The easy path is to take the top-k nearest chunks every time and hand them over. It always produces context, the demo is never empty, coverage is a clean 100%, and for most queries the nearest chunk is the right one — so it looks like it works.
It fails on the queries that matter. When a user asks something their data doesn't answer, the nearest chunk is still returned, and the model builds a fluent, wrong answer on top of it. So every chunk below a calibrated 0.60 cosine is dropped, and when nothing clears the floor the system returns nothing. The cost is coverage: on [ N ]% of questions it refuses rather than guess, measured over [ N ] questions — confirm.
The floor also moves the failure to where it's cheap. A miss now surfaces as "no context" at retrieval time, not as a confident wrong number a user has already read and acted on.
[ Confirm before publishing ] Full-text search alongside vectors, via tsvector, so exact terms — merchant codes, tickers — aren't lost to embeddings. A cross-encoder rerank before generation, to order what clears the floor. Graph population from Plaid webhooks, so the knowledge graph stays current without a backfill. Faithfulness scored in CI, not by eye.
What the data revealed — fill after the first eval run. Left blank rather than invented.
Not measured: end-answer correctness — only retrieval recall and whether context was grounded. No latency baseline captured. The 0.60 floor is calibrated on [ N ] sample queries, not a labelled set.