Kestra 2.0 Engine Rewrite and Deployment Flexibility

Описание и применение

Kestra 2.0 разделяет систему на контрольный и рабочий слои, позволяя запускать воркеры в изолированных сетях без доступа к базе данных, что повышает гибкость и безопасность.

Задача

Organizations struggled with Kestra’s heavy architecture requiring Kafka and Elasticsearch clusters, limiting deployment flexibility and causing security teams to avoid using it.

Результат

Kestra 2.0 enables secure, scalable orchestration in isolated or multi-cloud environments without requiring heavy infrastructure, improving adoption and reducing operational friction.

Рабочий процесс

• 1. Deploy control plane (scheduler, executor, webserver, indexer, worker controller) in desired location.
• 2. Configure queue (e.g., JDBC) and repository (e.g., PostgreSQL) for control plane.
• 3. Deploy workers in target environment (cloud, on-prem, isolated network).
• 4. Workers open persistent gRPC stream to worker controller; authenticate via TLS, client cert or JWT.
• 5. Submit jobs from control plane; workers execute and return results over same stream.
• 6. Monitor via UI or logs; adjust quotas, trigger conditions, or loop constructs as needed.

Доказательства

In 1.x, every worker needed a connection to the central database. That single design decision dictated where you were allowed to run tasks.

Источник

2.0 splits Kestra into a control plane and a data plane. The executor, the scheduler, the webserver, the indexer and a new worker controller live in the control plane, and none of them run user code.

Источник