A Small LLM Service That Costs Nothing to Run: A Case Study

说明与用途

一个小型LLM服务，免费运行，自动在API状态变化后生成简短的事件说明，使用cron任务和免费层LLM。

问题

Automating the generation of a concise incident note after a hobby API goes down, without incurring costs for LLM usage.

结果

The bot runs continuously for free, generating concise incident notes whenever the API state changes, with minimal latency and no secret data exposure.

工作流

• Set up a cron job to run every 5 minutes that checks the health endpoint and records state changes in a JSON file.
• When a state change is detected, append an event to events.jsonl with the transition details and timestamp.
• Run a script that reads the latest event, constructs a prompt with a fixed shape, and sends it to the MonkeyCode free-tier LLM API.
• Parse the LLM response and append the formatted incident note to incidents.md.
• Ensure the note is not duplicated by checking against existing entries before summarization.
• Optionally, monitor token usage to stay within the free tier allowance.

证据

The bot needs to know when the endpoint changed state. A cron job runs every five minutes; each run fetches the endpoint and compares the result with the last known state, stored in a small JSON file.

来源

When an event is appended, the next step turns it into a note. The prompt asks for a fixed shape, so the output stays parseable.

来源