pg-listen/notify driven sse telemetry for ciede2000 perceptual lock calibration: inside shadow's mini-max direct multi-modal coherence engine

Описание и применение

В проекте Shadow цветовые отклонения между разными потоками генерации приводили к заметным несоответствиям. Внедрён слой «перцептуального замка», использующий PostgreSQL LISTEN/NOTIFY и SSE для низкой задержки, чтобы в реальном времени минимизировать максимальный delta‑E между модальностями.

Задача

Cross‑modality frames in Shadow drifted by 2–4 delta‑E, causing visible colour inconsistencies across text‑to‑image, image‑to‑video, and audio‑reactive pipelines.

Результат

The perceptual lock layer reduced cross‑modality delta‑E drift to below 1, eliminating colour inconsistencies while keeping latency low and avoiding heavy operational dependencies.

Рабочий процесс

• 1. Render worker computes CIEDE2000 metric against reference and inserts row into perceptual_metrics table.
• 2. Trigger notify_perceptual_lock builds JSON payload and pg_notify publishes to perceptual_lock_v1 channel.
• 3. Separate Node process subscribes via pg-listen, receives event, broadcasts via SSE hub.
• 4. SSE consumer enqueues calibration job to adjust latent space for next frame.
• 5. Calibration loop runs inside generation, back‑propagating perceptual loss to maintain lock.

Доказательства

We hit a wall eight months into building Shadow. Our text-to-image, image-to-video, and audio-reactive pipelines were producing frames that, individually, looked correct. Put them on the same timeline and the colours drifted.

Источник

We landed on PostgreSQL LISTEN/NOTIFY. It is not glamorous. It does not have Kafka's replay or partitioning. What it does have is transactional consistency with the frame metadata we were already storing in the same database.

Источник