High-Concurrency Booking API Development
Kareem Sayed
⚡ Fast Booking API
A high-concurrency booking API built with Java 21, Spring Boot, MongoDB, Redis, and Apache Kafka.
This project demonstrates how a seemingly correct booking implementation can fail under concurrent traffic, causing overselling, and how atomic database operations, idempotency, failure compensation, and reliable event publishing can improve the design.
The project intentionally includes both a BEFORE and an AFTER implementation so the concurrency problem and its solution can be demonstrated and measured.
🎯 Problem
A booking system must guarantee that confirmed bookings never exceed the available capacity.
For example:
The fundamental business invariant is:
A traditional read-check-write implementation can violate this invariant when multiple requests execute concurrently.
🏗 Architecture
❌ BEFORE — Non-Atomic Booking
The initial implementation follows a traditional read-check-write flow:
Example:
This implementation contains a race condition.
Multiple requests can read the same
booked value before another request persists its update.Example:
Under concurrent traffic, the application can therefore accept more bookings than the configured capacity.
✅ AFTER — Atomic Capacity Reservation
The improved implementation performs the capacity check and increment as a single atomic MongoDB operation.
Conceptually:
MongoDB performs the condition check and increment atomically on the slot document.
Concurrent requests can no longer independently read the same capacity value and then update it later.
This protects the core invariant:
🔐 Idempotency
Booking requests contain an idempotency key:
The AFTER implementation uses two levels of idempotency protection.
Redis Fast Path
Redis stores:
A repeated request can therefore return an existing booking without repeating the booking operation.
Durable MongoDB Check
If the Redis entry does not exist, MongoDB is checked using the idempotency key.
This prevents correctness from depending entirely on the Redis cache.
Conceptually:
The idempotency key is also protected by a unique database constraint so concurrent requests using the same key cannot create multiple durable bookings.
🛡 Failure Compensation
Atomic capacity reservation happens before the booking is persisted.
That creates another failure scenario:
Without compensation, the slot counter would remain incremented even though no booking was created.
Example:
One unit of capacity would effectively be lost.
The AFTER implementation therefore performs a compensating operation when booking persistence fails:
The release operation performs:
with protection against decrementing below zero.
Once the booking has been successfully persisted, the capacity is considered consumed.
Therefore, failures in downstream operations such as Redis or event publishing must not release the capacity of an already confirmed booking.
📨 Outbox-Based Event Publishing
Confirmed bookings generate events that must eventually be published to Kafka.
Publishing directly from the request flow creates a failure scenario:
To reduce this coupling, the application uses an Outbox Pattern.
Instead of publishing directly to Kafka from the booking request, an outbox event is stored in MongoDB.
A separate background publisher processes the events:
Outbox Event Lifecycle
A newly created event starts as:
After Kafka confirms successful publishing:
If publishing fails, the event remains available for retry and its retry counter is incremented.
After the configured maximum number of attempts:
This allows Kafka failures to be tracked instead of silently losing events.
⚠️ Transactional Boundary
This portfolio implementation uses a standalone MongoDB instance.
The booking document and outbox event are therefore not written within the same MongoDB transaction.
A failure could theoretically occur between:
For that reason, this project should be described as demonstrating an:
rather than claiming a fully transactional outbox implementation.
A production extension could run MongoDB with transaction support and persist:
within the same database transaction.
🔄 Final AFTER Flow
The resulting booking flow is:
🧪 Load Testing
Load tests were executed using Apache JMeter 5.6.3.
Two different types of tests were performed:
Concurrency correctness testing
Performance/load testing
These tests intentionally measure different characteristics of the system.
🚨 Concurrency Correctness Test
The most important test verifies whether the application maintains capacity correctly under concurrent traffic.
Configuration:
❌ BEFORE Result
The non-atomic implementation produced:
Result:
Overselling occurred.
Four bookings were accepted beyond the configured capacity during this test run.
✅ AFTER Result
The atomic implementation produced:
Result:
No overselling occurred.
The 90 rejected requests represent expected business rejections because the slot had reached capacity; they are not interpreted as infrastructure failures.
📊 Performance Benchmark
For performance testing, slot capacity was increased to:
This prevented normal capacity rejection from distorting the latency and throughput measurements.
The following numbers represent the observed results from the executed JMeter test runs.
100 Concurrent Users
Metric BEFORE AFTER Requests 100 100 Average 15 ms 15 ms Median 14 ms 16 ms P90 20 ms 19 ms P95 30 ms 19 ms P99 45 ms 20 ms Max 46 ms 20 ms Throughput 98.6 req/s 100.2 req/s Error Rate 0% 0%
At this load level, average latency was equal while the AFTER implementation showed lower tail latency in this particular test run.
200 Concurrent Users
Metric BEFORE AFTER Requests 200 200 Average 24 ms 30 ms Median 14 ms 18 ms P90 60 ms 81 ms P95 74 ms 107 ms P99 86 ms 139 ms Max 90 ms 174 ms Throughput 200.0 req/s 194.6 req/s Error Rate 0% 0%
At 200 concurrent users, the BEFORE implementation showed lower latency in this particular run.
However, the concurrency correctness test demonstrates that the BEFORE implementation cannot reliably enforce slot capacity.
500 Concurrent Users
Metric BEFORE AFTER Requests 500 500 Average 882 ms 1080 ms Median 964 ms 1179 ms P90 1139 ms 1331 ms P95 1164 ms 1382 ms P99 1201 ms 1446 ms Max 1252 ms 1499 ms Throughput 245.0 req/s 218.7 req/s Error Rate 0% 0%
A significant latency increase was observed between the 200-user and 500-user tests.
This indicates a saturation or queueing point somewhere in the tested environment.
The JMeter results alone are not sufficient to attribute the bottleneck to MongoDB, Redis, Kafka, the application thread pool, the MongoDB connection pool, JVM resources, or the local load-generation environment.
Additional profiling would be required to identify the bottleneck.
📈 BEFORE vs AFTER
The primary goal of the AFTER implementation is correctness under concurrency, not simply producing a lower average response time.
Observed correctness test:
BEFORE AFTER Concurrent Requests 100 100 Capacity 10 10 Confirmed Bookings 14 ❌ 10 ✅ Rejected Requests 86 90 Overselling YES ❌ NO ✅ Capacity Enforcement Read/Check/Write Atomic Idempotency Database check Redis + MongoDB Failure Compensation No Yes Event Publishing Request flow Outbox-based
The BEFORE implementation was faster in some benchmark runs, but it violated the fundamental business invariant.
The AFTER implementation prioritizes:
while exposing the performance cost of contention for further analysis.
🛠 Technology Stack
Java 21
Spring Boot
Spring Data MongoDB
MongoDB 8
Redis 7
Apache Kafka 3.9
Docker / Docker Compose
Apache JMeter 5.6.3
Maven
🚀 Running the Infrastructure
Start the infrastructure with:
Services:
Verify:
Then start the Spring Boot application.
📂 Project Structure
📌 Engineering Lessons
This project demonstrates practical backend engineering concepts including:
Race conditions in read-modify-write workflows
Concurrency-safe capacity enforcement
Atomic MongoDB operations
Idempotent API design
Redis-backed idempotency fast paths
Durable duplicate detection
Compensating actions for partial failures
Failure boundaries in distributed workflows
Outbox-based event publishing
Kafka publishing retries
Event failure tracking
Event-driven architecture
Load testing with JMeter
Average vs percentile latency
P95/P99 latency analysis
Throughput analysis
Saturation and queueing behavior
Separating correctness testing from performance testing
🔭 Potential Production Improvements
The project intentionally stops short of introducing unnecessary production infrastructure.
Potential extensions include:
Use MongoDB transactions for atomic booking + outbox persistence
Run MongoDB as a replica set
Add safe coordination when multiple Outbox Publisher instances are running
Add dead-letter handling for permanently failed events
Add exponential backoff for event retries
Profile the latency increase between 200 and 500 concurrent users
Analyze MongoDB connection-pool behavior
Analyze Spring Boot/Tomcat thread-pool saturation
Monitor JVM CPU, heap and garbage collection
Add Spring Boot Actuator and Micrometer
Export metrics to Prometheus
Build Grafana dashboards
Run repeatable JMeter tests in non-GUI mode
Run the load generator separately from the application infrastructure
💡 Key Takeaway
A booking API is not correct simply because it returns successful responses quickly.
Under concurrency, correctness requires protecting business invariants.
This project demonstrates the evolution from:
to:
The most important measured result was:
The optimization therefore focuses not only on speed, but on building a booking workflow that remains correct when requests arrive concurrently.
Like this project
Posted Oct 2, 2026
Developed a high-concurrency booking API with improved reliability and concurrency handling.
Likes
0
Views
0
