I Built a 32,000-Word Uzbek Dictionary and Never Checked If It Was Actually What I Claimed It Was

Описание и применение

Автор создал словарь из 32 000 узбекских слов, но не проверил, что он содержит только существительные. После аудита список исправлен и использован для построения быстрого спелл‑чекера с точностью 87,5 % на тестовом наборе.

Задача

The author claimed a 32,000-word Uzbek list contained only nouns, but it actually included verbs, leading to inaccurate downstream usage.

Результат

An audited Uzbek word list enabled a fast, accurate spell-checker, demonstrating the value of verifying data claims before building tools.

Рабочий процесс

• Collect community word lists
• Filter to nouns only (initially)
• Commit to repo without verification
• Audit list: count suffixes, identify verbs, remove duplicates
• Build fuzzy-matching spell-checker using string similarity
• Generate synthetic test set of 200 words with random typos
• Evaluate top‑1 and top‑5 accuracy
• Analyze shortcomings (synthetic typos, mislabeling, no frequency weighting)
• Publish code and results on GitHub
• Invite community feedback

Доказательства

12.1% of a list I documented as “nouns only” are verb infinitives, not nouns.

Источник

87.5% of the time, the single top suggestion is the exact word the person meant to type. 99% of the time, it’s somewhere in the top 5.

Источник