Multi-Agent RAG System for Legal

Multi-Agent RAG System for Legal Document Review

The objective of this project was to engineer an intelligent assistant tailored for legal teams to automate the initial screening of complex contracts, such as PDFs and DOCX files. Legal documents present severe challenges for standard AI architectures due to their extreme length, intricate formatting, and dense domain-specific terminology. Standard Retrieval-Augmented Generation (RAG) pipelines routinely fail in this context because of low context precision—retrieving irrelevant clauses while missing critical cross-references—and faithfulness issues, leading to unacceptable AI hallucinations. The challenge was to overcome these limitations to instantly identify hidden legal risks and non-compliant clauses without requiring manual line-by-line reading, all while maintaining professional-grade accuracy and strict auditability.

To solve this, I developed a production-ready, multi-agent system powered by LangGraph to handle complex workflow routing. The architecture begins with a dual-access interface featuring a Streamlit UI for rapid interactive testing and a FastAPI REST backend for seamless integration with existing legal tech stacks. At the orchestration layer, a dedicated Task Classifier node evaluates incoming user queries and intelligently routes the execution flow to specialized, isolated agents. These include a Risk Agent for scanning unfavorable liability and termination clauses, a Summary Agent for synthesizing high-level contract executive briefs, and a Q&A Agent for answering precise, contextual questions about specific clauses.

The system's accuracy is driven by an advanced Hybrid RAG pipeline explicitly designed to maximize context precision. The pre-retrieval stage utilizes Query Translation and Query Expansion (HyDE) to bridge the gap between colloquial user questions and formal legal phrasing. This feeds into a dual-search engine combining sparse keyword search via BM25 with dense vector search using ChromaDB and OpenAI Embeddings. The post-retrieval phase merges these results using Reciprocal Rank Fusion (RRF) and a FlashRank Reranker. Furthermore, to ensure absolute trust in the system's outputs, I built a dedicated Explainability Layer where every generated claim is permanently coupled with direct source document citations, an explicit confidence metric, and a transparent reasoning trace.

To guarantee reliability before deployment, the system was rigorously evaluated using the RAGAS framework to systematically measure and optimize Faithfulness, Answer Relevancy, and Context Precision. The final result was a robust MVP that successfully slashed initial document review time by 85 percent. By completely eliminating AI hallucinations through multi-tier reranking and comprehensive validation, the platform provides legal professionals with a highly reliable, transparent, and fully auditable tool that safely scales their contract auditing capabilities.