Express Node.js Structured Logs — Poll Error Search for 30-Day Cost Attribution

说明与用途

文章说明如何在 Express‑Node.js 服务中记录每个 AI 代理步骤，排除大负载，并通过轮询日志发送去重的 Slack 错误通知，同时控制存储成本。

问题

How to log structured events in an Express Node.js service to enable cost attribution and error polling without inflating storage costs.

结果

Structured per‑step logging enables accurate cost attribution and efficient error polling while controlling storage costs by excluding large payloads from the main event stream.

工作流

• 1. Define a compact event schema with timestamp, schema_version, event_name, level, tenant_id, workflow_id, run_id, step_id, model, input_tokens, output_tokens, elapsed_ms, outcome, and optional error_class/error_message.
• 2. Emit the event after each agent step completes in the Express service (e.g., via stdout, queue, or HTTP exporter).
• 3. Store events in a log backend, retaining three logical classes: attribution (long retention), diagnostic (short, access‑controlled), and alert‑delivery ledger (short, retry‑aware).
• 4. Implement a poller that queries the log backend using a stable cursor (timestamp, event_id) with a small overlap, filters for error events, and checks a delivery ledger to avoid duplicates.
• 5. Send Slack notifications for new error events, recording the event fingerprint in the ledger to prevent re‑delivery.

证据

have the Express Node.js service send structured logs for every completed AI-agent step, then poll the log search for new error events and trigger deduplicated Slack notifications.

来源

Attach stable tenant, run, model, token, latency, and outcome fields.

来源

The storage bill is made from event volume, bytes per event, retention time, replication, indexes, and query work.

来源