Claude's Context Window in Practice: Caching, Chunking, and Cost

说明与用途

为Claude提供大文档的选择：计数令牌、缓存完整文本或分块检索相关部分，配合最小Python代码和成本估算。

问题

Choosing how to provide Claude with large documents—whether to embed the whole text, cache it, or retrieve relevant chunks—while balancing cost and performance.

结果

By counting tokens first, users can decide whether to cache the entire document or retrieve relevant chunks. Caching reduces repeated token costs for stable content, while chunking and retrieval keep costs low for large or frequently changing corpora.

工作流

• Count tokens of the full document with Anthropic SDK to assess fit within context window.
• If token count is small relative to context window, send entire document and enable caching via cache_control in the system message.
• If token count is large or document changes frequently, split the text into overlapping paragraph chunks using the provided chunk function.
• Embed each chunk with a chosen embedding model and compute similarity scores to the user’s question.
• Select top-k relevant chunks and assemble them into a prompt for Claude.
• Send the prompt to Claude and receive the answer.
• Monitor usage fields such as cache_read_input_tokens to verify caching effectiveness.
• Adjust chunk size, overlap, or k as needed based on performance and cost.

证据

The SDK can count tokens for a request without running it

来源

If the number is a small fraction of the model's context window, long context is on the table

来源

Three habits make caching work: Put stable content first and changing content last

来源

When the corpus is large, changes often, or has to respect per-user permissions, retrieval is the better fit

来源