Kafka Server Deployment on Ubuntu WSL

Zain

Zain Ul abideen

Running Apache Kafka on Windows 10 using WSL

A Step-by-Step Implementation and Explanation

Author: Zain ul Abideen

Executive Summary

This document provides a professional overview of setting up and using Apache Kafka on Windows 10 using Windows Subsystem for Linux (WSL). It explains the environment setup, Kafka and Zookeeper configuration, creating producers and consumers, and understanding how Kafka works with a simple diagram.

1. Environment Setup

1 Install Ubuntu on Windows via WSL from Microsoft Store.
2 Install Windows Terminal for easier command-line usage.
3 Update and upgrade Ubuntu: sudo apt update && sudo apt upgrade
4 Install Java (required for Kafka): sudo apt install openjdk-17-jdk

2. Apache Kafka & Zookeeper Setup

2 Extract the archive: tar -xzf kafka_2.13-3.8.0.tgz
3 Navigate to the Kafka directory and configure server.properties
4 Start Zookeeper: bin/zookeeper-server-start.sh config/zookeeper.properties
5 Start Kafka: bin/kafka-server-start.sh config/server.properties

3. Producer & Consumer Example

1 Create a Kafka producer: kafka-console-producer.sh --broker-list localhost:9092 --topic test
2 Create a Kafka consumer: kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning
3 Messages sent from the producer appear in the consumer window.

4. How Kafka Works

Apache Kafka is a distributed streaming platform. It uses Zookeeper to manage brokers and topics. Producers send messages to Kafka brokers, which store them in topics. Consumers subscribe to topics and read the messages. Producer Broker

5. Outcome & Benefits

Consumer By completing this setup, Apache Kafka and Zookeeper run smoothly on Windows using WSL. This allows developers to simulate real-world data streaming pipelines, test producer-consumer workflows, and prepare for production-grade deployments on Linux servers.

Conclusion

This project demonstrates how to set up Apache Kafka on Windows using WSL, configure brokers, and establish communication between producer and consumer applications. The setup showcases your ability to work with distributed systems and message streaming technologies.
Like this project

Posted Aug 14, 2024

Successfully installed and configured Kafka server on Ubuntu WSL, streamlining data processing workflows for real-time analytics.Did relevant work as required