Preventing AI Test Hallucinations by Indexing data-testid and ARIA Attributes

Description and use

TokenCap indexes data-testid and ARIA attributes so AI agents can generate accurate selectors for tests, eliminating common selector errors.

Problem

AI coding agents often fail to generate correct DOM element selectors when writing integration and end‑to‑end tests, leading to test failures.

Outcome

By indexing explicit testing attributes, TokenCap enables AI agents to produce accurate, accessibility‑friendly selectors, eliminating common test hallucinations.

Workflow

• 1. TokenCap parses JSX AST nodes to extract testing attributes (data-testid, data-test, data-cy, role, aria-label, etc.).
• 2. Extracted attributes are stored in src/frontend/testIds.js and included in component snapshots.
• 3. When tokencap make runs, component snapshots expose testing selectors with line numbers.
• 4. AI coding agents consume these snapshots and generate tests using getByTestId or getByRole selectors.
• 5. Tests are executed, avoiding failures caused by incorrect CSS selectors.

Evidence

Agents routinely fail at one specific step: guessing DOM element selectors.

Source

In TokenCap, we solve this by parsing JSX AST nodes and extracting explicit testing attributes in src/frontend/testIds.js.

Source

When tokencap make runs, frontend components include their test locator contract: ... getByTestId("billing-submit-button") ... getByRole("combobox", { name: "Billing Tier" }) ...

Source