Restaurant Supply Chain Order Forecasting using AI





Background

The client was a USA restaurant with over 2800 restaurant locations. They were looking to improve their order forecasting system and have a contractor develop a custom solution that meets current and future business needs. The solution worked with 2 billion daily transaction records and achieved 94% accuracy saving the company $18million annually.

• Role: Lead data engineer
• Project Timeline: 18 months
• Technologies Used: Python, Airflow, SQL, Redshift, AWS, DynamoDB, Scikit-learn, Tensorflow, Docker, AWS Sagemaker, DevOps, Tableau, Spark, Git

Project Requirements

I worked closely with the client's product owner to create the project requirements.

• Work with data scientists to determine what the machine learning forecasting model's input data sources are
• Work with DBAs and other stakeholders to get access to the data sources
• Perform rigorous code reviews with the data scientists to prepare their model's code for deployment
• Optimize SQL queries and python code to achieve multi-billion row scalability in ETL processes
• Deploy the solution so that it runs on a daily schedule
• Send email and text message alerts if the performance of the models falls below a threshold
• Automatically retrain and redeploy the model every week

Challenges

• Scale of the data is 2 billion rows daily
• Each of the 2800 restaurant locations needs it's own model for each menu items, which means over 84,000 models needed to be deployed
• Accuracy needs to be at least 90%
• Automatic retraining and redeployment using MLOps
• Track model performance and send alerts if accuracy drops
• Imbalanced dataset - some menu items are infrequently sold

Solution

Technologies Used

I lead a team of 5 developers to create a custom forecasting solution for the restaurant chain. The forecasting pipelines ingest "raw" transaction records, aggregate them into 15 minute increments, and then use that as input to train a model. The model is trained to forecast orders up to 6 weeks out and at a 15 minute granularity. The forecasts are then loaded into json format into DynamoDB and used by the restaurant operators through a web application that gives them suggested order notifications. When the forecast anticipates high demand in the upcoming 6-week window, the operators can place the order in advance to ensure there is enough inventory on-hand. The solution is entirely self-sufficient and continues to operate without any additional development effort.

Skills

1. Airflow
2. SQL
3. AWS Lambda
4. Opsgenie
5. Flyway
6. AWS Redshift
7. Python
8. Docker
9. Apache Spark
10. AWS Glue
11. DynamoDB
12. Scikit-learn
13. Tensorflow
14. AWS Athena

.. and many more skills that were needed to complete this challenging (but fun) project!



Need a reliable freelancer with reasonable prices, high quality work, and 100% satisfaction guarantee? Contact me today!

Find me on Contra: https://contra.com/ashabbi00_ursnb1o3?utm_campaign=social_sharing&utm_medium=independent_share&utm_source=copy_link

Find me on Linkedin: https://www.linkedin.com/in/ahsinshabbir