SeLoger

Data engineer · 2019–2021

Batch processing from 8 hours to 45 minutes

Batch processing over 50M+ real estate listings took 8 hours.

My part

I architected the serverless data lake on AWS, developed the Spark jobs over it, built the REST APIs on top, and implemented the machine-learning pipeline for property valuation.

What the team delivered

Recommendations for 10M+ monthly users

p99 latency under 100ms, at 500K requests a day

Valuation accuracy up 35%, a core product feature

€100K a year saved on AWS

Stack: Spark · AWS EMR · S3 · Lambda · CloudFormation · Spring Boot · PostgreSQL · Redis