No one measured AI API latency and uptime independently across regions — so I built it

说明与用途

开发者构建了一个独立的追踪器，测量45个AI API在四个地区的延迟和可用性，发现了显著的地区差异、可用性窗口、延迟差距以及模型停用通知周期。

问题

Developers lacked independent, region‑specific measurements of AI API latency and uptime, making it hard to choose providers and plan for model retirements.

结果

The independent tracker uncovered that regional differences can cause up to a 3.5× higher failure rate, that uptime figures vary with the observation window, and that latency can differ 14× across providers. It also revealed median notice periods for model retirements, ranging from 0 to 182 days.

工作流

• Set up a small prober that opens a real connection to each provider’s official API endpoint every 5 minutes.
• Measure DNS → TCP → TLS → TTFB and, where possible, inference time‑to‑first‑token.
• Run probes from four regions: Germany, US Central, Tokyo, São Paulo, directly to providers (no gateway).
• Collect results in SQLite and regenerate a static site every 30 minutes.
• Publish data at llmlatency.dev, including deprecation calendar, uptime, and methodology pages.
• Use the data to compare providers by region, uptime window, latency, and model retirement notice.
• Share findings with the community and invite further measurements.

证据

A small prober opens a real connection to each provider's official API endpoint every 5 minutes and times every phase of the handshake: DNS → TCP → TLS → time-to-first-byte.

来源

Over the same 7 days, on identical requests to identical endpoints, probes from Tokyo failed 0.315% of the time vs 0.089% from Germany — a 3.5× gap.

来源

Right now the fastest p50 TTFB in the fleet is ~99 ms and the slowest is ~1424 ms — a 14× spread between endpoints whose availability is statistically indistinguishable.

来源