Database Normalization and Consolidation

Context:

An area of the company generated a report based on data from 3 different databases (Excel and SQL). Each database had a different structure. With fields that match each other and other unique ones. The matching fields did not have the same writing structure, on the other hand they presented duplicate records between databases, making it necessary to spend a lot of time debugging the database.

Objective:

Implement a semi-automatic process through which the 3 databases can be unified, without losing information from any of them, standardizing data and identifying duplicate records.

Solution:

A database unification process was implemented which had the following sub-processes:

Validation of structure and content of source databases (Identifies Mandatory fields, records received, repeated records)

Normalizing matching fields based on standardization dictionaries.

Consolidation of bases in a unified structure, adding additional fields from each base as requested by the area.

Identification of repeated records between bases

Generation of a base consolidation report including total records received per base, duplicate records, consolidated records and internal business metrics.

Like this project

Posted Feb 4, 2025

Before: Critical daily data in 3 non-normalized database formats, Analysis impossible Solution: data normalization and unification process. More time to analyz

Likes

Views

Clients