Loading page...Загружаем страницу...正在加载页面...
AtlasRepo

My 300 unit tests passed while my lead finder returned "Refund Policy" as a person

Описание и применение

Тесты OpenLeads проходили успешно, но реальные поиски возвращали нерелевантные данные. Добавлен nightly benchmark, который выявил и исправил ошибки качества, улучшив покрытие лидов и уменьшив количество мусорных имен.

Задача

Unit tests passed but real searches returned irrelevant or missing results, revealing quality bugs not caught by mocked tests.

Результат

Nightly live benchmarks uncovered and fixed quality bugs that unit tests missed, improving lead coverage, reducing junk names, and clarifying email confidence.

Рабочий процесс

  • 1. Add "openleads bench" command to run 12 real searches against live sources.
  • 2. Score each search on return presence, lead coverage, junk name rate, email evidence, and duration.
  • 3. Commit the JSON scores to BENCHMARK.md via a GitHub Actions workflow nightly.
  • 4. Update the project website to display the latest benchmark JSON.
  • 5. Use benchmark results to refine name filtering, SMTP verification handling, and API fallback logic.

Доказательства

All 300 tests were green, yet real searches returned junk names, missing leads, or timed out.

Источник

I added a command, openleads bench, that runs 12 real searches against the live sources and scores the results.

Источник

The first run was humbling. The latest one: 12 of 12 searches return results, 82% of requested leads are filled, 0% junk names, and 57% of emails are found or built from a pattern the company really uses.

Источник

Your unit tests check that the code does what you wrote. They can't tell you whether what you wrote is any good against today's data.

Источник

  • Дата записи: 2026-10-13T18:41:30.736Z