Stop Guessing Where Your LLM Prompt Tokens Go: prompt-flamegraph

Описание и применение

Инструмент prompt‑flamegraph визуализирует распределение токенов в LLM‑промптах, выявляет избыточный контекст, дублирование и ненужные инструменты, позволяя снизить расходы на API.

Задача

Developers building LLM‑powered applications pay per token but lack visibility into which parts of a prompt consume the budget, leading to wasted spend on oversized RAG context, long chat history, or unnecessary tools.

Результат

Developers gain clear insight into token consumption, identify and eliminate waste, and reduce API costs by up to 30% or more.

Рабочий процесс

• Install prompt‑flamegraph via pip
• Prepare a structured prompt JSON with system_prompt, tools, rag_context, chat_history
• Run prompt‑flamegraph CLI or profile_prompt() to generate an interactive HTML flamegraph
• Review the report for total tokens, estimated cost, and waste findings
• Optionally use detect_waste() in code to programmatically identify waste
• Use prompt‑optimizer to automatically clean up the prompt

Доказательства

prompt‑flamegraph takes a structured prompt — system prompt, tools, RAG context, chat history — and renders it as an interactive HTML flamegraph.

Источник

Waste detection: duplicate text, oversized RAG, long history, too many tools.

Источник

It removes duplicate RAG chunks, trims long history, and suggests fewer tools.

Источник