Write a Receipt for Every Remote Inference

Описание и применение

В статье описывается рабочий процесс измерения производительности локального и удалённого инференса, принятия решения о маршруте и записи «квитанции» с хешем запроса, чтобы защитить приватные данные и анализировать эффективность.

Задача

Teams often choose local or remote inference once and never re‑measure, leading to wasted time and privacy risk.

Результат

The workflow enables objective routing decisions, protects private data via hashed prompts, and provides actionable logs for latency and privacy analysis.

Рабочий процесс

• Measure local TPS with bench_local.py
• Measure remote TPS with bench_remote.py
• Measure round‑trip time with rtt.py
• Export metrics to .envrc
• Implement choose_route to compare local vs remote against budget
• Generate a receipt with prompt hash, bytes sent, tokens out, latency, and reason
• Append receipts as JSONL for later analysis

Доказательства

Measure once, encode a small decision function, and keep a receipt for every request that went remote.

Источник

A log line records that something happened. A receipt records why a decision was made.

Источник

Notice what the function does not do. It does not ask whether you like cloud services. It does not care about brand.

Источник