A Small LLM Service That Costs Nothing to Run: A Case Study

Description and use

A small LLM service that runs for free automatically generates concise incident notes after an API state change, using a cron job and a free-tier LLM.

Problem

Automating the generation of a concise incident note after a hobby API goes down, without incurring costs for LLM usage.

Outcome

The bot runs continuously for free, generating concise incident notes whenever the API state changes, with minimal latency and no secret data exposure.

Workflow

• Set up a cron job to run every 5 minutes that checks the health endpoint and records state changes in a JSON file.
• When a state change is detected, append an event to events.jsonl with the transition details and timestamp.
• Run a script that reads the latest event, constructs a prompt with a fixed shape, and sends it to the MonkeyCode free-tier LLM API.
• Parse the LLM response and append the formatted incident note to incidents.md.
• Ensure the note is not duplicated by checking against existing entries before summarization.
• Optionally, monitor token usage to stay within the free tier allowance.

Evidence

The bot needs to know when the endpoint changed state. A cron job runs every five minutes; each run fetches the endpoint and compares the result with the last known state, stored in a small JSON file.

Source

When an event is appended, the next step turns it into a note. The prompt asks for a fixed shape, so the output stays parseable.

Source