Receipt Processing Pipeline Development

A receipt looks simple. Until a machine has to read it. 🧾 A warung or minimarket receipt is a narrow strip of faded thermal paper with columns that do not line up. People read it without thinking. A model has to guess.

Which number is the price? One real line from an Indonesian receipt reads: 024895 0,02 X Rp.45.518 1.001. The quantity is 0,02, the unit price is 45.518, and the price actually charged is 1.001, the rightmost amount. Take the wrong number and the expense is wrong by a factor of forty. That one rule, "the price of a line is the rightmost amount," is in Notarik's prompt because it went wrong in practice.

Four ways it goes wrong. Reading a receipt fails in a few repeatable ways: the unit price is taken as the line price, "25.000" is read as 25 because the dot looks like a decimal point (in rupiah it is a thousands separator), the items do not add up to the subtotal, and the date is one that cannot be, in the future or far in the past. Each of these is something code can check.

The first try. My first version read receipts with one prompt that returned JSON. It worked on the receipts I tried. It had no tests and no evals, and the same model that read the receipt was the one judging its own math. That is not a measurement.

The goal. Photo in, checked expense out. Three requirements: the right number (totals and lines add up, or it says so), never silent (when it is unsure, you see where), and you decide (nothing is saved until you confirm).

The pipeline. Snap became six steps: preprocess, extract, validate, repair, review, save. Two steps use AI (extract and repair), one is you (review), and the rest is plain code. Preprocess fixes orientation and size and turns PDFs into images. Extract reads the image into a fixed schema. Validate checks the result. Repair re-reads the image when validation finds a problem. Review is where you confirm. Save writes the expense only after that. The flow is an explicit Python graph, not a framework black box.

Read into a schema. The model fills a form: merchant, date, total, subtotal, category, items, and a confidence for each field. If its output does not fit the schema, that counts as a failure and goes to repair, instead of being guessed through.

Validation is just code. Four checks: items add up to the subtotal, the total is not below the subtotal, the date is not in the future or too old, and a price is not secretly a unit price. The validator is not an LLM, so a line of text printed on a receipt cannot talk it into passing. Text on the receipt is treated as data, never as instructions.

Repair, twice at most. When a check fails, the same image is read again with the specific problem listed, and only the failing fields change. Then the checks run again. After two tries, the draft goes to review with the issue still attached, instead of looping forever or pretending it is fine.

Snap it, then review. In the app you open the camera, pick from the gallery, or drop in a PDF. Progress streams while the receipt is read. Then you review the result, edit any field, and confirm. Cancel throws the run away. Nothing is saved before you confirm. ✅

It remembers your corrections. When you fix a merchant's category, Notarik stores the corrected mapping and uses it as an example next time, and can skip the model when a merchant is a clear match. I have not measured how much this helps. 🧠

Where it stands. Notarik is in a closed test with friends and family, about 10 to 20 users. I have not measured accuracy. There is no baseline number yet: the scorer and the graph are in place and tested with scripted model output, but that does not measure a real vision model. The next step is to label about 40 receipts by hand, keep the images out of git, and score the model against them. 🧪 Try it: https://notarik-app.nashir.tech/