Is Your Free Model Server Cold or Just Slow? A Probe That Tells the Difference

Description and use

The probe measures how a free model server sleeps and wakes, helping distinguish cold starts from genuine latency issues.

Problem

Free model servers often appear slow, but the cause may be a cold start (idle sleep) rather than network or model issues.

Outcome

The probe provides clear latency metrics that reveal whether a free model server is experiencing a cold start, allowing developers to take targeted actions.

Workflow

• 1. Install Node 18+. 2. Save the provided script as coldstart-probe.mjs. 3. Run node coldstart-probe.mjs find <url> [max-idle-min] to determine the idle threshold. 4. Run node coldstart-probe.mjs burst <url> <idle-min> <requests> to simulate a cold start and observe latency. 5. Analyze the output latency values to differentiate cold starts from other delays.

Evidence

The probe is included. Run it before you trust your first request.

Source