pg-listen/notify driven sse telemetry for ciede2000 perceptual lock calibration: inside shadow's mini-max direct multi-modal coherence engine

Description and use

Shadow’s multi‑modal pipelines suffered colour drift. A perceptual lock layer using PostgreSQL LISTEN/NOTIFY and SSE was added to keep delta‑E below 1 in real time, eliminating inconsistencies.

Problem

Cross‑modality frames in Shadow drifted by 2–4 delta‑E, causing visible colour inconsistencies across text‑to‑image, image‑to‑video, and audio‑reactive pipelines.

Outcome

The perceptual lock layer reduced cross‑modality delta‑E drift to below 1, eliminating colour inconsistencies while keeping latency low and avoiding heavy operational dependencies.

Workflow

• 1. Render worker computes CIEDE2000 metric against reference and inserts row into perceptual_metrics table.
• 2. Trigger notify_perceptual_lock builds JSON payload and pg_notify publishes to perceptual_lock_v1 channel.
• 3. Separate Node process subscribes via pg-listen, receives event, broadcasts via SSE hub.
• 4. SSE consumer enqueues calibration job to adjust latent space for next frame.
• 5. Calibration loop runs inside generation, back‑propagating perceptual loss to maintain lock.

Evidence

We hit a wall eight months into building Shadow. Our text-to-image, image-to-video, and audio-reactive pipelines were producing frames that, individually, looked correct. Put them on the same timeline and the colours drifted.

Source

We landed on PostgreSQL LISTEN/NOTIFY. It is not glamorous. It does not have Kafka's replay or partitioning. What it does have is transactional consistency with the frame metadata we were already storing in the same database.

Source