A Free AI Server Is a Job Candidate. Interview It Like One.

说明与用途

通过五个小任务的测试脚本，测量免费 AI 代码服务器的响应时间、HTTP 状态、输出大小和令牌使用，揭示冷启动延迟、静默截断和单线程并发问题。

问题

Free AI coding servers often fail under real-world conditions such as cold starts, silent truncation, and concurrency limits, leading to unreliable performance and token exhaustion.

结果

The test revealed that the free server suffers from long cold starts, silent truncation of large inputs, and single‑threaded concurrency, leading to unreliable performance and potential token exhaustion for real workloads.

工作流

• Create a directory of five markdown task files (01‑refactor.md, 02‑debug.md, 03‑migrate.md, 04‑explain.md, 05‑feature.md).
• Write a Bash script (interview.sh) that loops over the task files, POSTs each to the server endpoint, captures HTTP status, total time, and word count, and writes results to a timestamped results directory.
• Run the script once to establish baseline performance.
• Run the script again after ten idle minutes to test cold start behavior.
• Run two instances of the script concurrently to test concurrency limits.
• Collect and analyze the output files to identify 200‑status responses with empty bodies, timeouts, or 429 errors.
• Optionally, use a token counter to estimate monthly token consumption based on the tasks’ token usage.
• Decide whether the free tier meets your team’s reliability and cost requirements.

证据

Run the test once. Then run it again after ten idle minutes. Then run two copies at the same time. That's where free servers confess.

来源

A passing run looks boring: --- 01-refactor --- status=200 time=8.41s words=214 --- 02-debug --- status=200 time=12.02s words=98

来源