Database Normalization and Consolidation
Hernán Pineda Barragán
Database Normalization and Consolidation
Context:
An area of the company generated a report based on data from 3 different databases (Excel and SQL). Each database had a different structure. With fields that match each other and other unique ones. The matching fields did not have the same writing structure, on the other hand they presented duplicate records between databases, making it necessary to spend a lot of time debugging the database.
Objective:
Implement a semi-automatic process through which the 3 databases can be unified, without losing information from any of them, standardizing data and identifying duplicate records.
Solution:
A database unification process was implemented which had the following sub-processes:
Validation of structure and content of source databases (Identifies Mandatory fields, records received, repeated records)
Normalizing matching fields based on standardization dictionaries.
Consolidation of bases in a unified structure, adding additional fields from each base as requested by the area.
Identification of repeated records between bases
Generation of a base consolidation report including total records received per base, duplicate records, consolidated records and internal business metrics.
Like this project
Posted Feb 4, 2025
Before: Critical daily data in 3 non-normalized database formats, Analysis impossible Solution: data normalization and unification process. More time to analyz
