How I QA a Video Editor With One Test That Uses the App Like a Person

说明与用途

单个 Appium 脚本模拟用户操作，使用 SSIM 对比截图，并用 ffprobe 验证导出文件，确保视频编辑器预览与最终输出一致。

问题

Unit tests miss visual glitches and export mismatches in a video editor.

结果

The single end‑to‑end test reliably detects preview glitches, export mismatches, and ensures exported videos meet strict SDR and format requirements, preventing user‑reported bugs.

工作流

• Run appium/run.sh to lease a simulator, pin UI elements, build the app, populate Photos library with ffmpeg‑generated clips, start Appium server, and launch pytest journey.
• Use Appium to perform real taps, drags, and sheet interactions, waiting for state changes via the hidden uitest‑state element.
• Take screenshots at key moments, ensuring two identical captures, exact frame alignment, and masking of animated tiles, then compare with approved baselines using SSIM thresholds.
• Pause editor at 1, 5.5, 8, and 11.5 s, capture canvas frames, export the project, extract frames from the .mp4, and compare each pair.
• Pull exported file from app container and validate codec, resolution, duration, audio track, and absence of black frames using ffprobe.

证据

Unit tests can't see a preview that blinks or an export that drifts from what you edited.

来源

It opens the app on a fresh simulator, picks clips in the Photos picker, trims, splits, adds a transition, generates captions, styles them, adds a text overlay, plays the project, exports it and opens the share screen.

来源

Under test, the editor mounts a hidden accessibility element called uitest-state.

来源

At key moments the journey takes a screenshot and compares it with an approved baseline using SSIM. The threshold is 0.985.

来源

During the journey, the test pauses the editor at 1, 5.5, 8 and 11.5 seconds and saves what the canvas shows. After the export, it pulls the frames at the same instants out of the .mp4 and compares each pair.

来源

The export is pulled out of the app's container and checked with ffprobe: H.264, SDR (bt709), 8‑bit 4:2:0, 1080×1920 at 30 fps, duration within one frame of the timeline, an audio track is present, no flat black frame anywhere in the video.

来源