French Legal AI Assistant & Agentic RAG System

Overview

I designed, built, and deployed a specialized Legal AI Assistant for French lawyers using agentic RAG, legal data pipelines, vector search, reranking, open-source LLMs, and citation-grounded answer generation. The system allowed lawyers to ask legal questions and receive answers grounded in French law articles, legal references, and relevant judicial cases.

Problem / Challenge

Legal data is very different from normal document data. A generic RAG pipeline using fixed-size chunks often breaks legal meaning, misses important context, or retrieves incomplete references. The main challenges were: 🔹 Legal documents had different structures and lengths 🔹 Articles and laws could not be randomly split into fixed-size chunks 🔹 Each answer needed traceable legal references 🔹 Retrieval had to understand legal scope, not just semantic similarity 🔹 The system needed to reduce hallucinations for legal users 🔹 Deployment had to respect privacy and regulatory requirements

My Expertise

I worked as the Lead AI Engineer / Agentic RAG Developer responsible for the complete system design and implementation. My responsibilities included: 🔹 Legal data pipeline architecture 🔹 Document parsing and preprocessing 🔹 Custom legal chunking strategy 🔹 Vector database design 🔹 Agentic RAG workflow development 🔹 Retrieval optimization and reranking 🔹 Open-source LLM deployment 🔹 Backend API development with FastAPI 🔹 Secure Azure cloud deployment 🔹 Multi-tenant system support

French Legal Data Engineering Pipeline

I built an automated ETL pipeline to process thousands of French legal documents, articles, and judicial cases. The pipeline handled: 🔹 Raw legal document ingestion 🔹 Text cleaning and normalization 🔹 Legal article extraction 🔹 Section-aware document structuring 🔹 Custom chunk generation 🔹 Metadata extraction for article number, article title, section, source, and reference 🔹 Embedding generation 🔹 Vector database ingestion 🔹 Repeatable updates for future legal data expansion The chunking strategy was designed so legal articles were not cut in the middle or separated from their meaning.

Agentic RAG Workflow

Instead of using a simple one-step vector search, I built a LangGraph-based agentic RAG workflow. The workflow included: 🔹 User query understanding 🔹 Legal intent detection 🔹 Legal domain and scope identification 🔹 Generation of 2–5 targeted legal search queries 🔹 Retrieval of relevant chunks for each query 🔹 Deduplication of repeated results 🔹 Reranking of retrieved legal evidence 🔹 Source-grounded answer generation This improved tested retrieval accuracy from around 50% to 95%+.

Retrieval, Citations & Case Law

The retrieval system was designed to make answers transparent and verifiable. I implemented: 🔹 Vector search for semantic legal retrieval 🔹 Reranking to improve relevance 🔹 Metadata-based source traceability 🔹 Citation-backed answer generation 🔹 Article-level legal references 🔹 Typesense-based retrieval for French judicial cases 🔹 Supporting case law returned with legal answers This allowed lawyers to verify the exact legal source behind each generated response.

Open-Source LLM & Cloud Deployment

I evaluated and deployed open-source LLM infrastructure for private legal AI usage. The deployment included: 🔹 Qwen2.5:14B for French legal reasoning 🔹 Ollama and vLLM for model serving 🔹 Embedding and reranker models on a private Azure GPU VM 🔹 NVIDIA T4 16GB GPU optimization 🔹 Python/FastAPI backend APIs 🔹 Secure Azure deployment in the France region 🔹 Multi-tenant isolated access 🔹 GitHub CI/CD and Linux server management The system was designed for privacy, reliability, and regulatory compliance.

Technologies Used

🔹 Python 🔹 FastAPI 🔹 LangChain 🔹 LangGraph 🔹 LangSmith 🔹 Ollama 🔹 vLLM 🔹 Qwen2.5:14B 🔹 ChromaDB 🔹 Typesense 🔹 Vector Databases 🔹 Reranking Models 🔹 Embedding Models 🔹 Azure Cloud 🔹 Linux 🔹 GitHub CI/CD

Impact

🔹 Built a production-ready legal AI assistant for lawyers 🔹 Improved retrieval accuracy from ~50% to 95%+ in tested scenarios 🔹 Reduced hallucinations through citation-grounded generation 🔹 Enabled lawyers to verify answers using article and case references 🔹 Created a scalable legal data pipeline for thousands of documents 🔹 Deployed private open-source LLM infrastructure for legal compliance 🔹 Delivered a strong foundation for future legal AI workflows