The client was a USA restaurant with over 2800 restaurant locations. They were looking to improve their order forecasting system and have a contractor develop a custom solution that meets current and future business needs. The solution worked with 2 billion daily transaction records and achieved 94% accuracy saving the company $18million annually.

Role: Lead data engineer

Project Timeline: 18 months

Technologies Used: Python, Airflow, SQL, Redshift, AWS, DynamoDB, Scikit-learn, Tensorflow, Docker, AWS Sagemaker, DevOps, Tableau, Spark, Git

I worked closely with the client's product owner to create the project requirements.

Work with data scientists to determine what the machine learning forecasting model's input data sources are

Work with DBAs and other stakeholders to get access to the data sources

Perform rigorous code reviews with the data scientists to prepare their model's code for deployment

Optimize SQL queries and python code to achieve multi-billion row scalability in ETL processes

Deploy the solution so that it runs on a daily schedule

Send email and text message alerts if the performance of the models falls below a threshold

Automatically retrain and redeploy the model every week

Scale of the data is 2 billion rows daily

Each of the 2800 restaurant locations needs it's own model for each menu items, which means over 84,000 models needed to be deployed

Accuracy needs to be at least 90%

Automatic retraining and redeployment using MLOps

Track model performance and send alerts if accuracy drops

Imbalanced dataset - some menu items are infrequently sold

I lead a team of 5 developers to create a custom forecasting solution for the restaurant chain. The forecasting pipelines ingest "raw" transaction records, aggregate them into 15 minute increments, and then use that as input to train a model. The model is trained to forecast orders up to 6 weeks out and at a 15 minute granularity. The forecasts are then loaded into json format into DynamoDB and used by the restaurant operators through a web application that gives them suggested order notifications. When the forecast anticipates high demand in the upcoming 6-week window, the operators can place the order in advance to ensure there is enough inventory on-hand. The solution is entirely self-sufficient and continues to operate without any additional development effort.

Airflow

SQL

AWS Lambda

Opsgenie

Flyway

AWS Redshift

Python

Docker

Apache Spark

AWS Glue

DynamoDB

Scikit-learn

Tensorflow

AWS Athena

.. and many more skills that were needed to complete this challenging (but fun) project!