Autonomous FinOps Sentinel: Orchestrating TimescaleDB Anomaly Detection with Google Gemini AI in Kestra

Description and use

The Autonomous FinOps Sentinel leverages TimescaleDB, Google Gemini, and Kestra to detect spending anomalies in real time, diagnose root causes with AI, and send contextual Slack alerts, reducing investigation time and eliminating alert fatigue.

Problem

End‑of‑month cloud billing surprises caused by uncoordinated resource provisioning, alert fatigue from static thresholds, and lack of contextual insight into spending spikes.

Outcome

The sentinel detects spending spikes in sub‑millisecond time, provides AI‑generated remediation steps, and sends severity‑based Slack alerts, cutting on‑call investigation time and eliminating alert fatigue.

Workflow

• Trigger hourly schedule in Kestra
• Query TimescaleDB continuous aggregates to compute rolling mean, variance, and z‑score for each spend category
• If z‑score exceeds threshold, build anomaly JSON payload
• Send payload to Google Gemini 1.5 Flash for structured root‑cause analysis
• Receive JSON with severity, summary, remediation_action, urgency_score
• If severity is CRITICAL or HIGH, record incident in Kestra KV store and send Slack Block Kit alert; otherwise log nominal status

Evidence

It creates a real‑time, automated financial surveillance system that monitors cloud spending timeseries in TimescaleDB Continuous Aggregates (CAGGs), detects statistical spending spikes, invokes Google Gemini 1.5 Flash for contextual root‑cause diagnosis, and orchestrates remediation alerts directly into Slack.

Source

Uncoordinated Provisioning: A developer spins up on‑demand GPU clusters (p4d.24xlarge) or an unthrottled LLM embedding pipeline, forgetting to terminate it on Friday night.

Source

The workflow triggers at the top of every hour (io.kestra.plugin.core.trigger.Schedule). It calculates rolling standard deviations over recent hourly buckets from Timescale continuous aggregates:

Source

json { "severity": "CRITICAL", "summary": "Cloud Infrastructure spend surged by 470% (Z-Score: 4.2) due to uncoordinated on‑demand GPU instance provisioning.", "remediation_action": "Audit active AWS us-east EC2 clusters and terminate orphaned p4d.24xlarge on‑demand instances immediately.", "urgency_score": 9 }

Source