Moroccan Financial Data: BVC Data Pipeline

Moroccan Financial Data is a standalone data-engineering system that collects, preserves, validates, normalizes, and stores Casablanca Stock Exchange market data. It exposes consistent, query-ready records through a read-only API.

BVC market pages are designed for human browsing, not dependable machine consumption. Pagination, inconsistent headers, localized numbers, missing values, timestamp ambiguity, and repeated runs can silently corrupt downstream data. The pipeline needed to preserve source evidence, explain parsing decisions, and process the same input safely more than once.

I designed and built the complete pipeline from source collection to database persistence and API access. My work covered source analysis, HTTP collection, raw-payload preservation, validation gates, deterministic normalization, PostgreSQL modeling, migrations, failure handling, automated testing, and containerized execution.

BVC Source → HTTP Collector → Raw Payload Storage → Diagnostics & Validation → Deterministic Normalization → PostgreSQL → Read-only API

• Multi-page BVC collection with independent raw payloads and complete provenance

• Diagnostics for table detection, header mapping, pagination, row validity, and timestamp policy

• Locale-aware parsing and deterministic normalization of instruments, latest prices, and historical price bars

• Transactional, idempotent PostgreSQL writes that prevent duplicate data during reprocessing

• Traceable parse and processing errors without duplicate failure records

• Read-only FastAPI endpoints with exact financial-value serialization

• Docker Compose infrastructure, database migrations, retries, fixtures, and reproducible local execution

Raw source payloads are preserved before transformation, validation runs before persistence, and every normalized record remains traceable to its origin. Transactional writes, deterministic rules, and idempotent processing make retries and reprocessing safe.

The final system passed 161 standard tests and 73 PostgreSQL-backed integration tests. Coverage includes collection, pagination, parsing, validation, normalization, idempotency, failure handling, transactional persistence, and API serialization.

The result is a complete and reproducible BVC market-data pipeline that converts raw exchange pages into consistent, query-ready financial data. Invalid inputs are isolated and recorded, every processed record remains auditable, and repeated processing is safe by design.

Python 3.11 • FastAPI • Pydantic Settings • HTTPX • Beautiful Soup • PostgreSQL • SQLAlchemy 2 • Psycopg • Alembic • pytest • Docker Compose

The public repository presents the project architecture, capabilities, and verified outcomes. The proprietary implementation is maintained privately.

https://github.com/Nour-Eddin-01/moroccan-financial-data-showcase