Explainable Causal Reinforcement Learning for bio‑inspired soft robotics maintenance with embodied agent feedback loops

Description and use

An explainable causal reinforcement learning system was built for soft‑robotic maintenance, leveraging embedded strain sensors and a structural causal model to predict degradation and execute targeted maintenance actions.

Problem

A soft robotic arm kept developing micro‑tears near its bending joints while a standard deep reinforcement learning agent continued to execute the same policy and accumulate reward, indicating the agent had no insight into the underlying degradation.

Outcome

The causal RL pipeline enabled the soft robotic arm to detect early signs of degradation through the rate of change of strain sensor readings, predict latent degradation states, and execute maintenance actions that targeted causal causes, thereby extending the robot’s operational life and providing explainable decision traces.

Workflow

• 1. Embed distributed strain sensors in the silicone tentacle to capture sensor readings (S_t).
• 2. Collect action (A_t) and sensor data over time to train a variational autoencoder that estimates latent degradation state (D_t).
• 3. Use a structural equation model to learn the transition D_t, A_t → D_{t+1}.
• 4. Train the causal world model to predict future sensor readings, ensuring the latent representation captures physical degradation.
• 5. Integrate the causal model into a reinforcement learning agent that selects maintenance actions (M_t) based on predicted degradation causes.
• 6. Deploy the embodied agent to continuously monitor and intervene, providing explainable maintenance decisions.

Evidence

The arm—a pneumatically actuated silicone tentacle modeled loosely on an octopus—was supposed to learn how to grip fragile objects. Instead, it kept developing micro‑tears near its bending joints, and my standard deep reinforcement learning agent had absolutely no idea why.

Source

One interesting finding from my experimentation with embedded strain sensors was that the rate of change of sensor readings, not their absolute values, was the strongest early predictor of failure.

Source

I implemented a lightweight causal world model using a variational autoencoder for latent degradation estimation, combined with a structural equation model for the dynamics.

Source