When a paper flags your training data, the warning lives in a PDF

Описание и применение

Публикация исследовательской статьи выявила, что популярные наборы данных Kaggle не имеют подтверждённой подлинности, но предупреждение остаётся только в PDF. Создано машинно‑читаемое реестр, позволяющее отслеживать такие предупреждения и предотвращать их использование в моделях.

Задача

Clinical prediction models may continue to be used even after their training data is flagged as unverifiable, because the warning is embedded in a PDF and not reflected in the dataset metadata.

Результат

The registry provides a transparent, citation‑based record of datasets flagged for authenticity or provenance issues, preventing their unverified use in downstream models.

Рабочий процесс

• 1. Identify datasets cited in peer‑reviewed literature.
• 2. Review literature for authenticity or provenance warnings.
• 3. Create a JSONL row with dataset id, status, flags, source, and downstream impact.
• 4. Host the registry on GitHub with CC0 license and snapshot semantics.
• 5. Encourage contributions via pull requests with citations.
• 6. Use the registry to filter datasets before model training or deployment.

Доказательства

The authenticity of both datasets could not be verified ...

Источник

125 clinical prediction model studies in peer-reviewed publications