Your Keep-Alive Is Lying to You: Six Connection Myths I Measured on a Free Model Server

Описание и применение

На бесплатных API моделей задержка после периода простоя увеличивается в 4‑5 раз. Необходимо измерять реальные показатели и корректировать таймауты и повторные запросы.

Задача

Developers rely on keep‑alive and retry advice without measuring real latency on free model endpoints, leading to mis‑tuned timeout and retry logic.

Результат

Free tier endpoints exhibit significantly higher latency after idle periods; keep‑alive does not mitigate this. Developers should measure latency and adjust retry/timeout logic accordingly.

Рабочий процесс

• 1. Write a probe script that warms the connection and records latency.
• 2. Run the probe with a specified idle gap (e.g., 300 s) for multiple rounds.
• 3. Record and analyze latency for warm and idle requests.
• 4. Repeat the test using a persistent connection (keep‑alive) to compare performance.
• 5. Adjust client retry and timeout settings based on observed latency patterns.

Доказательства

Notice something? The warm baseline is ~400 ms. The idle requests are 4-5x slower. Every single time.

Источник

Free tiers have SLAs. They have capacity guarantees. Free tiers don't.

Источник