When Dictionaries Disagree, Software Needs Editorial Logic Building a pronunciation and stress dictionary taught me that a database can be technically clean and still be linguistically wrong.

Russian word stress is not always predictable from spelling, and the pronunciation of some words varies in real usage. Older and newer norms can coexist, dictionaries may differ in what they recommend or accept, and some sources are descriptive while others are intended for professional speech. For a broadcaster, voice actor, or anyone recording prepared speech, simply showing every attested variant does not always solve the problem. At some point, you need a consistent choice: a pronunciation you can use now, return to months later, and keep consistent throughout a project without reopening the same linguistic debate every time.

A dictionary is not just a list of answers I started building a native Android pronunciation and stress-reference app for Russian with what looked like a fairly straightforward goal: search for a word, see the stress, save difficult entries, and train on them later. The software side was manageable. The harder problem appeared when I began replacing the temporary development data with a source-aware dictionary of my own. At first, the natural assumption was that this would mostly be a data-cleaning task. Collect the entries, normalize spelling, merge duplicates, attach forms, build search indexes, and move on. That assumption lasted until different reputable dictionaries began giving different answers for what looked like the same word. Once that happens, just keep every source is not enough. A reference app has to decide what the user should actually use. For this kind of tool, simply exposing every variant found in every source would defeat part of its purpose: the user still needs a consistent pronunciation to learn and apply. The database needed editorial logic, not just parsing.

The newest source is not automatically the answer The working corpus uses several modern official dictionaries alongside an older broadcast-oriented stress source that I use as an editorial anchor. That immediately creates a difficult question: what should happen when a historical recommendation and a modern dictionary disagree? The tempting rule would be simple: always trust the newest source. That is easy to implement and often wrong. A modern dictionary may preserve the older stress as one of several accepted variants. Another modern source may omit it because of scope rather than rejection. A parser may also lose a variant that is clearly present in the original entry. Conversely, there are cases where modern official sources genuinely converge on a newer norm and the older recommendation is no longer supported. So I ended up with a source-aware decision process. If the older editorial norm can be verified in the original source for the same lexeme and meaning, and at least one current official dictionary still supports it, that stress can remain the canonical answer. If current sources consistently reject it and converge on a different norm, the modern norm wins. If the evidence is incomplete or contradictory, the entry stays in review instead of being guessed.

The difficult cases are structural, not cosmetic A large part of the work turned out to be recognizing cases that should never have been merged in the first place. Homographs, different meanings, grammatical forms and professional or specialist usages can look like competing variants when they are actually separate lexical records. One useful example was a word whose two stress patterns correspond to different meanings. Treating them as two interchangeable pronunciations would have produced a clean-looking record and a bad dictionary entry. The correct fix was structural: split the meanings, preserve the distinction, and let each record have its own answer. This became a general rule. Do not flatten semantic differences just because the written form is the same. Do not infer stress from a related word. Do not assume that two sources refer to the same lexeme merely because a normalized string matches. A database can be perfectly deduplicated and still destroy the linguistic distinctions that matter most.

OCR is a locator, not evidence Historical sources added another layer of difficulty. OCR is incredibly useful when you need to search hundreds of pages, but it is not authoritative enough to decide a pronunciation norm. In my workflow, OCR is treated as a locator. It can tell me where a candidate entry probably appears, but the decision has to be checked against the physical scan. That sounds excessive until you see what a single recognition error can do. In one case, an OCR hit appeared to confirm one word, but the scanned page actually contained a different word with a similar shape. If I had trusted the extracted text, the database would have gained a completely fabricated piece of evidence. The same problem appears in more subtle forms. A normalized parser can quietly drop a spelling or stress variant that exists in the raw source. One entry I reviewed looked as if a modern dictionary supported only one form; going back to the original text showed that an additional variant was explicitly present and had simply disappeared during normalization. At that point the parser was no longer the dictionary. It was only one representation of it.

A useful app needs a canonical layer and an evidence layer This led to an architectural separation that became important for the whole project. The application needs a clean runtime answer, but the editorial process needs to remember why that answer was chosen. For an ordinary training entry, I want one canonical stress. That makes search, exercises and error statistics understandable. But arriving at that answer may involve several sources, rejected alternatives, semantic splits, source-specific labels and manual verification. Throwing all of that away after choosing the final form would make future corrections almost impossible. So the editorial data and the user-facing result have different responsibilities. The evidence layer can preserve conflicting variants, provenance, review states and source-specific information. The runtime layer can stay simple. The simplification happens after the decision, not before it.

The review queue became the real product work Once this logic existed, the project stopped being import several dictionaries into SQLite. Entries began moving through explicit review states. Easy cases could be finalized automatically or semi-automatically; uncertain cases were isolated for manual inspection instead of contaminating the main dataset. That changed the role of automation. Scripts became very good at finding candidates, comparing sources, detecting collisions and reducing tens of thousands of records to a much smaller set that actually required human judgement. But the final difficult cases were difficult for a reason. They involved meaning, source scope, historical change, typography, OCR noise or an ambiguity that could not be resolved safely from string matching alone. This was also a useful lesson for AI-assisted work. AI can process huge amounts of structured material quickly and can help formulate comparison rules, but it is very easy for it to turn a plausible pattern into an invented linguistic rule. In this project, probably is not enough. If the evidence is not there, the correct state is review.

Why this became more interesting than the Android app The Android application itself now has the things you would expect from a modern offline reference tool: fast search, grouped forms, detailed word cards, bookmarks, training, statistics and separate handling for areas such as proper names. Those features matter, but they are no longer the part of the project I find most interesting. The difficult part was designing a system that can say different things with different levels of confidence. Sometimes two forms are genuinely equal. Sometimes one is preferred and another merely accepted. Sometimes the apparent variant is actually another meaning. Sometimes a historical norm remains valid even though a newer source gives additional options. Sometimes a modern consensus really has moved on. And sometimes the sources simply do not give enough evidence to make a safe editorial choice. Software tends to encourage us to flatten this kind of mess into a neat schema. Linguistic data punished that instinct very quickly. The better solution was to make the schema reflect the uncertainty and the distinctions rather than hide them.

What I took from the project I began this as an Android dictionary. It turned into a much larger exercise in data provenance, editorial methodology and decision-system design. The most important lesson was that conflicting authoritative sources are not a data-cleaning bug. They are part of the domain. If software is going to present one answer to a user, somebody still has to define the rules by which that answer is chosen, preserve the evidence behind it, and know when the system should refuse to choose automatically. That is the part of technical product work I keep finding most interesting. The code can store the answer. The difficult work is deciding what the answer is allowed to mean.