Simulated and displayed the status of train lines from the Chicago Transit Authority dataset using a streaming event pipeline based on Apache Kafka. This primarily involved six steps which included the following: creating the Kafka producers, creating the Rest Proxy producer, configuring Kafka Connect for the connection between PostgreSQL database and Kafka, configuring the Faust Stream Processor, configuring the KSQL table, and finally creating the Kafka consumers. Python was the primary language used.
Tip: Try to keep this description brief and under 150 words by highlighting the best parts of this project— recruiters don’t spend much time reading detailed descriptions! 💪