Self-Calibration via Prompt History Analysis

Описание и применение

Автор проанализировал 1427 своих промптов за шесть недель, чтобы объективно оценить прогресс. Данные показали значительный рост сложности запросов и навыков экспериментального дизайна, что противоречило субъективному ощущению некомпетентности. Это демонстрирует разрыв между самовосприятием и реальными навыками.

Задача

Developers often suffer from a disconnect between their subjective self-assessment (feeling like they are wasting time or lacking skills) and their actual demonstrated capability, leading to unnecessary anxiety and imposter syndrome.

Результат

The analysis revealed that while the developer felt stagnant, their prompts had evolved from simple commands to complex, context-rich strategies involving verification and experimental design. This objective evidence disproved their subjective feelings of incompetence.

Рабочий процесс

• Locate local conversation history files (e.g., JSONL files in ~/.claude/projects/ for Claude Code).
• Write a script to extract and parse prompts from a specific time period.
• Categorize prompts by vocabulary and intent (e.g., adversarial/verify, meta/governance, delegation/automation).
• Measure metrics such as average prompt length and frequency of verification keywords over time.
• Compare the quantitative data of prompt evolution against subjective self-assessment.
• Use the discrepancy to recalibrate self-perception and identify actual skill growth.

Доказательства

I pulled every prompt I typed myself between June 15 and July 23 — 1,427 of them... cut them into six time windows, and measured them window by window.

Источник

In under six weeks, 'make the AI push back, verify this, find the root cause' grew fivefold...

Источник

If you use Claude Code or a similar tool, your conversation history is sitting in jsonl files on your machine... Spend half an hour writing a script to pull your own prompts...

Источник