Gemma Hallucinated 29 npm/PyPI Packages. I Built a Tool to Catch Them.

Описание и применение

Модель Gemma генерирует 29 несуществующих пакетов npm/PyPI, что приводит к потенциальной угрозе безопасности. Создан инструмент ghostdeps, который проверяет названия пакетов, предлагаемых ИИ, и классифицирует их как живые, призрачные, мертвые и т.д., предотвращая установку недействительных зависимостей.

Задача

AI language models can hallucinate non‑existent package names, risking accidental installation of malicious or non‑existent code.

Результат

ghostdeps identified 29 out of 118 (24.6%) hallucinated packages in 535 Gemma prompts, with nearly half of niche tasks producing ghosts. Re‑runs showed high repeatability of hallucinations, indicating a systemic issue.

Рабочий процесс

• 1. Prompt the local AI model for a package recommendation. 2. Parse the model’s answer to extract install commands. 3. Query npm and PyPI registries for each extracted package name. 4. Classify each name as alive, GHOST, POSSESSED, dead, tombstone, EXORCISED, or unknown. 5. Report the classification results to the user.
• 6. Optionally rerun the prompt with different seeds or higher temperature to test repeatability of hallucinations.
• 7. Use ghostdeps check on a package.json, requirements file, or pasted chatbot answer to pre‑validate dependencies before installation.

Доказательства

The npm registry has never heard of that package. The GitHub repository returns a 404.

Источник

ghostdeps is a tool for detecting AI-hallucinated dependencies before they make it into your project.

Источник

535 questions, all local, on a laptop with a 4 GB GPU, about 70 minutes of model time.

Источник

29 of 118 (24.6%)

Источник