Stop tapping coordinates: automating Android phones by their elements

Описание и применение

Переключение от координатных нажатий к селекторам элементов устраняет ошибки, возникающие из‑за различий в экранах Android‑устройств, делая автоматизацию более надёжной.

Задача

Automating multiple Android phones with coordinate taps fails when screen size, font, or layout changes, causing taps to hit wrong areas.

Результат

Switching from coordinate taps to element-based selectors eliminates failures caused by varying screen layouts, making automation more robust across devices.

Рабочий процесс

• Identify element on screen using Inspector
• Choose the most stable selector (resource-id > content-desc > text)
• Use XPath to locate element at runtime
• Tap element within its bounds
• If element not visible, scroll or wait for it to appear
• If element still missing, fallback to image search or OCR
• Repeat until element is tapped or failure reported

Доказательства

If you have ever automated more than one Android phone, you have met this bug: the flow taps (540, 1620), which was the Sign in button on your test phone — and on the next phone it is the space between two buttons

Источник

Android can describe what is on screen as a hierarchy of views. Each node has a class (android.widget.Button) and attributes: resource-id, text, content-desc, class, package, bounds, index, and flags like clickable and enabled

Источник

That is the same model Appium Inspector shows you, and it means a selector language you probably already know works on it: XPath.

Источник

A selector that finds the element today can stop finding it after the next app update. A rough order, from sturdiest to most fragile: 1) resource-id 2) content-desc 3) text 4) a child's or neighbour's text 5) position

Источник

In WarpX Phone the Inspector does this ranking for you: you click an element on the phone's screen, and it lists the selectors that name only that element, scored by how likely each is to keep working

Источник