Explainable Causal Reinforcement Learning for smart agriculture microgrid orchestration with inverse simulation verification

Description and use

An explainable causal RL system for smart agriculture microgrid orchestration was built, enabling traceable decisions and inverse simulation verification.

Problem

A solar‑plus‑battery microgrid for a small farming cooperative drained its storage mid‑afternoon despite good irradiance forecasts, because the reinforcement learning (RL) agent made decisions that were statistically optimal but causally wrong and could not be explained by operators.

Outcome

The explainable causal RL pipeline produced a microgrid controller whose decisions could be traced to specific causal factors, enabling operators to audit actions and maintain confidence even when the environment changed.

Workflow

• 1. Build a simplified SCM of the microgrid (e.g., Irradiance → PV_Generation → Battery_SoC → Dispatch_Action → Grid_Import).
• 2. Wrap the gym environment with a CausalMicrogridEnv that records state, action, reward history and uses dowhy for causal effect estimation.
• 3. Train an RL agent (e.g., PPO) on the wrapped environment, allowing it to learn interventions rather than observations.
• 4. Implement a CausalAttentionPolicy where each attention head corresponds to an edge in the SCM, making the policy intrinsically explainable.
• 5. After training, query the SCM to identify the causal driver of each action (e.g., battery discharge at 2 PM caused by irradiance drop).
• 6. Perform inverse simulation verification: run a counterfactual simulator with the same interventions to confirm that the agent’s decisions lead to expected outcomes.
• 7. Deploy the agent to the field, monitor explanations, and adjust the SCM or policy if new causal relationships emerge.

Evidence

The system had a reinforcement learning (RL) agent that had been trained to schedule irrigation pumps, cold storage compressors, and ventilation fans.

Source

The agent had learned correlations between sensor readings and rewards, not the actual cause-and-effect structure of the microgrid.

Source

Combining them with RL produced something genuinely useful: an agent whose decisions could be traced, audited, and verified against a counterfactual simulator.

Source