Real-Time Messaging Application Development

Real-Time Messaging Application This is an event-driven, real-time messaging application designed with microservices architecture. The project is implemented in Go and uses Apache Kafka for message streaming, PostgreSQL for data persistence, and WebSocket for real-time messaging.

Project Overview The application consists of the following microservices:

WebSocket Service: A service that consumes messages from Kafka and sends them to connected WebSocket clients in real-time.

Persistence Service: A service that consumes messages from Kafka and persists them in a PostgreSQL database.

Authentication Service: A service that handles stateless authentication using JWT (JSON Web Tokens).

Ingestion Service (Optional): A service that receives messages via HTTP and pushes them to Kafka.

Technologies Used Go (Golang): The primary language for implementing the services.

PostgreSQL: A relational database for storing persisted messages.

Apache Kafka: Message broker used for message streaming between services.

WebSocket: Real-time communication protocol used for delivering messages to clients.

JWT (JSON Web Tokens): Used for stateless authentication.

Docker: For containerizing the services.

Kubernetes: For orchestrating and deploying the services in a distributed environment.

Prerequisites Before running the application, ensure the following software is installed:

Docker: Installation Guide

Kubernetes: Installation Guide

Go: Installation Guide

PostgreSQL: Ensure PostgreSQL is set up and running locally or through Docker.

Project Structure plaintext Copy Edit messaging-app/ ├── services/ │ ├── websocket/ # WebSocket service │ ├── persistence/ # Persistence service │ ├── authentication/ # Authentication service │ └── ingestion/ # Ingestion service (Optional) ├── k8s/ # Kubernetes deployment manifests │ ├── deployments/ │ └── manifests/ ├── Dockerfile # Dockerfile for building services └── docker-compose.yml # Docker Compose configuration for local setup Setup Step 1: Build and Run the Services Locally To build and run each service locally, follow these steps:

Clone this repository:

bash Copy Edit git clone https://github.com/yourusername/messaging-app.git cd messaging-app Build each service Docker image (in each service folder):

bash Copy Edit docker build -t websocket-service ./services/websocket docker build -t persistence-service ./services/persistence docker build -t authentication-service ./services/authentication docker build -t ingestion-service ./services/ingestion Run the services with Docker Compose:

bash Copy Edit docker-compose up Step 2: Set Up Kafka and PostgreSQL If you're running Kafka and PostgreSQL locally, you can use Docker to start the required services. Here’s a basic docker-compose.yml configuration to run them:

docker-compose.yml:

yaml Copy Edit version: '3.8' services: kafka: image: wurstmeister/kafka environment: KAFKA_ADVERTISED_LISTENERS: INSIDE://kafka:9093 KAFKA_LISTENER_SECURITY_PROTOCOL: PLAINTEXT KAFKA_LISTENERS: INSIDE://0.0.0.0:9093 KAFKA_LISTENER_NAME_INSIDE: INSIDE KAFKA_LISTENER_PORT: 9093 KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 ports: - "9093:9093" networks: - messaging

zookeeper: image: wurstmeister/zookeeper ports: - "2181:2181" networks: - messaging

postgres: image: postgres:alpine environment: POSTGRES_DB: messaging_db POSTGRES_USER: user POSTGRES_PASSWORD: password ports: - "5432:5432" networks: - messaging

networks: messaging: driver: bridge Run the following command to start Kafka and PostgreSQL:

bash Copy Edit docker-compose up Step 3: Running the Application Once you have the services running, you can test the following functionality:

WebSocket Service: Access the WebSocket service at ws://localhost:8080/ws and connect a WebSocket client.

Authentication Service: Access the JWT authentication service at http://localhost:8081/login to obtain a JWT token.

Ingestion Service: Post a message to http://localhost:8082/ingest to send a message to Kafka.

Step 4: Deploying with Kubernetes To deploy the services using Kubernetes, follow these steps:

Apply the Kubernetes manifests:

bash Copy Edit kubectl apply -f k8s/deployments/ Ensure your Kubernetes cluster is configured and running.

Expose your services using Kubernetes Ingress or LoadBalancer as appropriate.

Step 5: Testing the Services The project includes unit tests for each service. To run them, use the go test command in each service folder:

bash Copy Edit cd services/websocket go test

cd services/persistence go test

cd services/authentication go test

cd services/ingestion go test Architecture The system is based on microservices with Kafka as the message broker. The services communicate asynchronously via Kafka, and the WebSocket service broadcasts messages to clients in real-time.

Data Flow Ingestion Service (Optional): Receives messages via HTTP POST and sends them to the messages Kafka topic.

WebSocket Service: Listens to the messages Kafka topic and pushes messages to connected WebSocket clients.

Persistence Service: Consumes messages from Kafka and stores them in the PostgreSQL database for persistence.

Authentication Service: Issues JWT tokens for stateless user authentication.

Future Enhancements Add a UI frontend to display the real-time messaging interface.

Implement message acknowledgment and retry mechanisms.

Introduce message encryption for secure communication.

Implement rate limiting and authentication for WebSocket connections.

License This project is licensed under the MIT License - see the LICENSE file for details.

Notes Kafka and PostgreSQL can be run on Docker for local development, but you can also set them up using managed services if deploying to the cloud.

The ingestion service is optional and can be omitted if you're primarily using WebSocket for message ingestion.

Make sure your Kafka brokers are correctly configured in each service's code.