No one measured AI API latency and uptime independently across regions — so I built it

Description and use

A developer built an independent tracker measuring latency and uptime of 45 AI APIs from four regions, revealing regional performance gaps, uptime windows, latency spread, and model retirement notice periods.

Problem

Developers lacked independent, region‑specific measurements of AI API latency and uptime, making it hard to choose providers and plan for model retirements.

Outcome

The independent tracker uncovered that regional differences can cause up to a 3.5× higher failure rate, that uptime figures vary with the observation window, and that latency can differ 14× across providers. It also revealed median notice periods for model retirements, ranging from 0 to 182 days.

Workflow

• Set up a small prober that opens a real connection to each provider’s official API endpoint every 5 minutes.
• Measure DNS → TCP → TLS → TTFB and, where possible, inference time‑to‑first‑token.
• Run probes from four regions: Germany, US Central, Tokyo, São Paulo, directly to providers (no gateway).
• Collect results in SQLite and regenerate a static site every 30 minutes.
• Publish data at llmlatency.dev, including deprecation calendar, uptime, and methodology pages.
• Use the data to compare providers by region, uptime window, latency, and model retirement notice.
• Share findings with the community and invite further measurements.

Evidence

A small prober opens a real connection to each provider's official API endpoint every 5 minutes and times every phase of the handshake: DNS → TCP → TLS → time-to-first-byte.

Source

Over the same 7 days, on identical requests to identical endpoints, probes from Tokyo failed 0.315% of the time vs 0.089% from Germany — a 3.5× gap.

Source

Right now the fastest p50 TTFB in the fleet is ~99 ms and the slowest is ~1424 ms — a 14× spread between endpoints whose availability is statistically indistinguishable.

Source