The Task Failed, but Where Did the Logs Go? Troubleshooting an Apache DolphinScheduler Failed Task

说明与用途

在 Apache DolphinScheduler 中，任务被标记为失败，但所有运行时元数据为空。元数据和主日志显示失败发生在任务提交阶段，原因是租户配置无效。警报投递失败是独立问题。

问题

A scheduled task in Apache DolphinScheduler is marked as failed in the UI and alerts are triggered, yet all runtime metadata (host, start_time, log_path, execute_path) is NULL, making it unclear why the task failed and where to find logs.

结果

By inspecting metadata and Master logs, the root cause was identified as an invalid tenant configuration causing the Master to mark the task as failed before Worker assignment. Alert routing failure was a separate issue.

工作流

• 1. Retrieve alert record from t_ds_alert to get process_instance_id and taskCode.
• 2. Query t_ds_task_instance with these identifiers to inspect task lifecycle fields.
• 3. Observe that state=6 (FAILURE) but start_time, host, log_path, execute_path, app_link are NULL.
• 4. Conclude task never reached Worker; stop searching for Worker logs.
• 5. Verify Master IP addresses and connectivity from Worker nodes to rule out network issues.
• 6. Search Master logs using workflow instance ID, task instance ID, task code, and task name to locate failure cause.
• 7. Find "Tenant does not exists" error indicating invalid tenant configuration.
• 8. Separate alert delivery failure (no bind plugin instance) from task failure.
• 9. Resolve tenant configuration issue to prevent future failures.

证据

The task instance was created and its state was already 6 (FAILURE). However, except for submit_time, all other runtime-related fields were empty.

来源

"Tenant does not exists" in Master logs

来源