Stop Guessing Where Your LLM Prompt Tokens Go: prompt-flamegraph

Description and use

prompt‑flamegraph visualizes token usage in LLM prompts, detects waste such as duplicate content and excessive RAG or tools, and helps reduce API costs.

Problem

Developers building LLM‑powered applications pay per token but lack visibility into which parts of a prompt consume the budget, leading to wasted spend on oversized RAG context, long chat history, or unnecessary tools.

Outcome

Developers gain clear insight into token consumption, identify and eliminate waste, and reduce API costs by up to 30% or more.

Workflow

• Install prompt‑flamegraph via pip
• Prepare a structured prompt JSON with system_prompt, tools, rag_context, chat_history
• Run prompt‑flamegraph CLI or profile_prompt() to generate an interactive HTML flamegraph
• Review the report for total tokens, estimated cost, and waste findings
• Optionally use detect_waste() in code to programmatically identify waste
• Use prompt‑optimizer to automatically clean up the prompt

Evidence

prompt‑flamegraph takes a structured prompt — system prompt, tools, RAG context, chat history — and renders it as an interactive HTML flamegraph.

Source

Waste detection: duplicate text, oversized RAG, long history, too many tools.

Source

It removes duplicate RAG chunks, trims long history, and suggests fewer tools.

Source