Realtime Crawler

Part of Amazon’s Contextual Advertising team. With the growing concern of user data privacy, third party cookie are being banned by Apple and Google. Hence, showing user personalised ads on third party sites has become difficult more than ever. To support this industry shift, contextual advertising comes in, where user is shown ads based on web page content rather than user’s tracked data .

1. Built real time Data pipeline using apache Flink , which process the data ~500K Records per second

2. Worked and enhanced data lake used to store contextual Data.