Common Element Locating Strategies in Appium for Mobile Automation

Appium is a mature mobile automation testing framework that extends the WebDriver protocol, allowing users familiar with Selenium to quickly adopt it for mobile apps.

The author shares practical knowledge gathered while building a UI monitoring tool for a news feed, focusing on various element locating methods.

Basic locating strategies covered include:

Id locating: driver.findElement(By.id("com.android.calculator2:id/formula"))

Name locating: driver.findElement(By.name("新闻"))

Class name locating: driver.findElement(By.className("android.widget.Button"))

XPath locating: driver.findElement(By.xpath("//android.widget.LinearLayout[@resource-id='com.xxxx.browser.browser:id/ag9']")) (XPath can use text, content‑desc, id, index, etc.)

Android UIAutomator locating: driver.findElementsByAndroidUIAutomator("new UiSelector().className("+classname+").index("+index+")") (any element attribute can be used)

Relative coordinate locating: int width = appdriver.manage().window().getsize().width int height = appdriver.manage().window().getsize().height; Touchaction touchaction = new touchaction(appdriver); touchaction.tap(x1,y1).perform().release();

When an element lacks a unique identifier, two solutions are suggested:

Iterate over elements found by a common attribute (e.g., class name) and select the desired one by index or matching content: WebElement webElement = (WebElement) driver.findElements(By.className("android.view.View")).get(6);

Extract element content using OCR when neither unique attributes nor iteration work; the article demonstrates using Tesseract‑OCR, a Google‑supported open‑source project for Chinese text recognition.

Since Appium 1.9.0, image‑based element locating is possible by leveraging the OpenCV library to match template images against screenshots, handling variations such as rotation or scaling.

An example of capturing a screenshot for image‑based locating is provided:

File scrFile = ((TakesScreenshot) appDriver).getScreenshotAs(OutputType.FILE);

The article concludes with a brief reminder that image‑based locating is useful when visual elements have no reliable textual identifiers.