Express Node.js Structured Logs — Poll Error Search for 30-Day Cost Attribution

Описание и применение

В статье описывается, как в Express‑Node.js сервисе логировать каждый шаг AI‑агента, исключая большие полезные нагрузки, и как опрашивать логи для отправки дублирующих Slack‑уведомлений об ошибках, сохраняя при этом контроль над расходами на хранение.

Задача

How to log structured events in an Express Node.js service to enable cost attribution and error polling without inflating storage costs.

Результат

Structured per‑step logging enables accurate cost attribution and efficient error polling while controlling storage costs by excluding large payloads from the main event stream.

Рабочий процесс

• 1. Define a compact event schema with timestamp, schema_version, event_name, level, tenant_id, workflow_id, run_id, step_id, model, input_tokens, output_tokens, elapsed_ms, outcome, and optional error_class/error_message.
• 2. Emit the event after each agent step completes in the Express service (e.g., via stdout, queue, or HTTP exporter).
• 3. Store events in a log backend, retaining three logical classes: attribution (long retention), diagnostic (short, access‑controlled), and alert‑delivery ledger (short, retry‑aware).
• 4. Implement a poller that queries the log backend using a stable cursor (timestamp, event_id) with a small overlap, filters for error events, and checks a delivery ledger to avoid duplicates.
• 5. Send Slack notifications for new error events, recording the event fingerprint in the ledger to prevent re‑delivery.

Доказательства

have the Express Node.js service send structured logs for every completed AI-agent step, then poll the log search for new error events and trigger deduplicated Slack notifications.

Источник

Attach stable tenant, run, model, token, latency, and outcome fields.

Источник

The storage bill is made from event volume, bytes per event, retention time, replication, indexes, and query work.

Источник