Data Engineering Solutions for Scalable Data Pipelines