"Why does my RAG keep pulling the wrong chunk?" Because most RAG is one dumb step: embed the quer..."Why does my RAG keep pulling the wrong chunk?" Because most RAG is one dumb step: embed the quer...
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Back to feedPost
Yash's avatar
Yash Kumar
pro
• 2d
"Why does my RAG keep pulling the wrong chunk?" Because most RAG is one dumb step: embed the query, cosine-search a vector DB, stuff the top 5 into the prompt. That's the demo. It works right up until someone searches for an exact term - a product code, a name, an acronym — and embeddings, which are built to capture meaning, smooth right over the exact token you needed. Here's the two-stage pipeline I actually run in production: Stage 1 - hybrid retrieval (cast a wide net)
Dense: pgvector similarity. Great at "these mean the same thing."
Sparse: Postgres full-text search. Great at "these are literally the same word."
Fuse the two ranked lists with Reciprocal Rank Fusion (k=60). No score-normalization headaches- it merges by rank, so anything both retrievers like floats to the top. Each side pulls ~50; you're left with ~80 candidates. Stage 2 - rerank (precision pass)
Run those ~80 through a cross-encoder that reads the query and each document together (not as two separate vectors) and scores real relevance.
Keep the top 6. Those go to the LLM. The part people skip: retrieval and ranking are different jobs. Retrieval should be greedy and cheap - grab everything that might be relevant. Ranking should be expensive and picky — it only looks at a small pool, so you can afford a heavier model there. Dense finds meaning. Sparse finds exact terms. Rerank fixes the order. Skip any one and you get the confident, wrong answers RAG is famous for. What's your retrieval stack - dense-only, hybrid, or full rerank? #RAG #AI #backend #systemdesign #vectordatabases
ragagentfullstackdeveloperAI Engineer
Post image
Back to feed
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started