How I built a vision-token trimmer that prices images for one provider — and a reader caught it

Description and use

Ankita’s trimmer uses a tile‑based formula to estimate image tokens, accurate only for OpenAI. When routed through an OpenAI‑compatible proxy serving Claude, the estimate under‑counts tokens, leading to request rejections and unexpected history trimming.

Problem

The trimmer estimates image token cost using a formula that is only accurate for OpenAI-compatible vision endpoints, causing incorrect trimming when the provider is different (e.g., Claude behind an OpenAI-compatible proxy).

Outcome

The trimmer’s tile‑based estimator works for OpenAI but under‑estimates token cost for providers like Claude, causing over‑trimming or request rejection when used behind an OpenAI‑compatible proxy.

Workflow

• 1. Load image URL and read header to get dimensions.
• 2. Fit image to 2048 pixels on the long side.
• 3. Scale short side to 768 pixels.
• 4. Compute tiles: ceil(width/512) * ceil(height/512).
• 5. Return 85 + 170 * tiles as estimated tokens.
• 6. Trim conversation history to fit within the provider’s token budget using this estimate.

Evidence

The trimmer was weighing a screenshot's base64 string as if it were text, so a 1.6 MB PNG looked like 1.6 MB of context and got deleted before the model ever saw it.

Source

The estimator lives in src/core/history.mjs and takes exactly one argument: the image URL.

Source

Take bickov's example, a 2560×1440 screenshot. Fit to 2048 on the long edge gives 2048×1152; scale the short side to 768 gives 1365×768; that is 3 × 2 tiles, so 85 + 170 × 6 = 1,105 tokens.

Source

Claude does not bill in tiles. It bills roughly by pixels — width × height / 750 — and reads up to a 2576-pixel long edge. The same screenshot comes out at 4,784 tokens on bickov's numbers.

Source