Self-Calibration via Prompt History Analysis

说明与用途

作者分析了六周内的1427个提示词，以客观评估技能进步。数据显示提示词的复杂性和实验设计能力显著增加，这与他们主观上的无能感相矛盾。这突显了自我认知与实际判断力之间的差距。

问题

Developers often suffer from a disconnect between their subjective self-assessment (feeling like they are wasting time or lacking skills) and their actual demonstrated capability, leading to unnecessary anxiety and imposter syndrome.

结果

The analysis revealed that while the developer felt stagnant, their prompts had evolved from simple commands to complex, context-rich strategies involving verification and experimental design. This objective evidence disproved their subjective feelings of incompetence.

工作流

• Locate local conversation history files (e.g., JSONL files in ~/.claude/projects/ for Claude Code).
• Write a script to extract and parse prompts from a specific time period.
• Categorize prompts by vocabulary and intent (e.g., adversarial/verify, meta/governance, delegation/automation).
• Measure metrics such as average prompt length and frequency of verification keywords over time.
• Compare the quantitative data of prompt evolution against subjective self-assessment.
• Use the discrepancy to recalibrate self-perception and identify actual skill growth.

证据

I pulled every prompt I typed myself between June 15 and July 23 — 1,427 of them... cut them into six time windows, and measured them window by window.

来源

In under six weeks, 'make the AI push back, verify this, find the root cause' grew fivefold...

来源

If you use Claude Code or a similar tool, your conversation history is sitting in jsonl files on your machine... Spend half an hour writing a script to pull your own prompts...

来源