forge-stream v0.1 — Zero-Dependency CSV Streaming & Ledger Reconciliation Engine in C Developed a zero-dependency CSV streaming library and ledger reconciliation engine written in pure C. The project replaces standard, brittle utility tokenizers with a stateless, callback-driven runtime engine engineered to decouple raw file I/O operations from upper-layer business logic. Objectives

Isolate Processing Side-Effects: Build a stateless parsing loop that processes massive data inputs without maintaining rigid internal states.

Ensure Edge-Case Correctness: Native mitigation against corrupted database formats, including layout shifting, duplicate entries, and platform newline drift.

Decouple Infrastructure: Use native C function pointers to build a reusable parser that can map to alternative analytical data structures without requiring core logic modifications.

Technical Highlights

Chunk-Buffered File Streaming: Uses a static 64KB block buffer framework via fread() to bypass high-frequency disk access and maintain a low memory footprint.

Reentrant Token Extraction: Implements a custom double-pointer cursor navigation engine to guarantee field index alignment across complex data structures.

Inline Space Normalization: Processes raw string padding on the fly via single-pass trimming loops, preserving data type boundaries with zero external memory allocations.

Cross-Platform Line Ending Safety: Automatically sanitizes both Unix LF (\n) and Windows CRLF (\r\n) format frameworks natively.

Technologies Used

Language: C11 (Strict Linux GCC optimization compiler parameters)

Build Automation: GNU Make Toolchain

Version Control: Git Ecosystem (Explicitly tagged semantic production releases)

Engineering Challenges & Solutions

1. The Delimiter Collapse Bug

The Problem: Standard string tokenizers like strtok() automatically collapse consecutive delimiters into a single break. If an inbound ledger data row contains an empty field (,,), the column boundaries shift to the left, resulting in silent data corruption downstream.

The Solution: Developed a custom traversal loop using reentrant pointer mutations. The function scans indices sequentially, detecting and returning distinct empty pointer positions explicitly to ensure absolute column alignment.

2. Floating-Point Numeric Noise

The Problem: Performing raw equality comparisons (amount == bank_settlement) on floating-point data types introduces silent evaluation failure due to standard native IEEE-754 rounding noise.

The Solution: Integrated safe epsilon evaluation logic using the absolute mathematical delta computation (fabs(delta) >= EPSILON). This guarantees data matching accuracy down to standard fractional sub-cent variances.

3. State Ingestion Duplication

The Problem: Duplicate incoming transactional keys can bypass basic calculation accumulators, leading to double-counting errors in memory arrays.

The Solution: Engineered a defensive pre-ingestion screening loop within the broker callback structure that detects duplicate transaction keys and isolates them prior to processing memory allocations.

Project Outcomes

Decoupled Parser Core: Successfully modularized stream_csv to function as a generic data stream provider, processing data blocks at approximately 3.6M lines/second on local development hardware.

Automated Regression Harness: Deployed a 7-case validation matrix (run_tests.sh) checking empty cells, whitespace fields, malformed numbers, and platform newline variances with a 100% pass rate.

Production Release Base: Shipped and locked v0.1 as an open-source asset repository with complete structural developer instructions.

Repository Target Link

GitHub: https://github.com/naresh-cn2/forge-stream