Data extraction pipeline.

Implemented a parallelized data extraction pipeline with diverse sources (web crawling, tabular data,

PDFs, OCR, zipped files), leveraging Redis and Celery in Python to efficiently generate embeddings and

store data in the Pincone vector database.