Let the AI click. Don’t let it grade.

说明与用途

文章讨论了将 AI 集成到 e2e 测试中的方法，提出混合策略：使用 agent.act 进行操作，使用 expect 进行确定性检查，只有在语义验证时才使用 agent.assert。这样可以降低模型成本、提高可靠性并更容易检测回归。

问题

Determining which test steps should invoke an AI model versus which should remain deterministic code to balance cost, reliability, and maintainability in end‑to‑end testing.

结果

Adopting a hybrid AI‑driven testing strategy reduced model calls from every step to only interaction steps, lowered costs, improved determinism, and made regressions easier to detect.

工作流

• 1. Write tests using agent.act for actions and expect for deterministic checks.
• 2. Commit the .e2e/cache/ folder to version control and enable --strict-cache in CI.
• 3. Run tests; agent.act steps replay from cache unless stale, while expect assertions validate UI state.
• 4. Use agent.assert only for outcomes that cannot be expressed as locators, such as chatbot responses.

证据

"agent.act" hands a goal in plain English to a model that drives the app.

来源

"agent.assert" always run live. A judgment is never cached.

来源

The recording is a 1.4 KB JSON file. It holds two actions…

来源

The product broke. My dead button: exit 1, ASSERTION_FAILED…

来源