Comparative Analysis of Open-Source UI Automation Testing Tools

Abstract: As the scope of automated testing expands within the bank, the reliance on commercial tools such as QTP/UFT creates license competition; therefore, adopting open‑source or free UI automation tools is an inevitable trend. This paper compares several mainstream open‑source UI automation tools for reference.

Keywords: open‑source, automated testing, Selenium, UIAutomation

Background and Significance Automated testing, evolved from manual testing, is essential for continuous iteration and rapid delivery in commercial banking. It reduces manual effort, expands coverage, and lowers production risk for full‑regression and data‑intensive tests.

The bank previously relied on QuickTest Professional (QTP/UFT), which is costly, has limited licenses, a bulky client architecture, and poor support for certain C/S applications, making a shift to open‑source tools inevitable.

Tool Introductions

2.1 Selenium Selenium, developed by ThoughtWorks, is a popular open‑source web‑testing suite comprising Selenium IDE (recording), Selenium Grid (parallel execution), and Selenium RC (script execution). Selenium Grid consists of a Hub that distributes test cases to Remote Controls (RCs). The tool supports locating elements by selenium.type("id=abc","百度") , selenium.type("name=search","百度") , selenium.click("link=最近更改") , and XPath such as selenium.type("xpath=//input[@name='user.email']","[email protected]") .

2.2 UIAutomation Microsoft’s UIAutomation is a .NET component for automating C/S applications. It represents UI elements as AutomationElement objects, organized in a UI Tree with the desktop as the root. The architecture includes UIAutomationProvider.dll, UIAutomationTypes.dll (server side), UIAutomationClient.dll, UIAutomationTypes.dll (client side), UIAutomationCore.dll (core), and UIAutomationClientSideProvides.dll.

2.3 Watir Watir is a Ruby‑based open‑source web automation tool supporting browsers like Firefox, Opera, and IE. Watir scripts use the .rb extension and start with require 'watir' . Example snippets: ie = Watir::IE.new or ie = Watir::IE.start("http://abc.com") ; ie.goto("http://abc.com") ; ie.checkbox(:name,"check").set .

2.4 Sahi Developed by Tyto Software, Sahi is a proxy‑based open‑source web automation tool that injects JavaScript to interact with page elements. It handles HTTPS, works independently of the site, and excels at dynamic ID handling and implicit waits. Its workflow includes recording (via proxy), script refinement (parameterization and external data), and playback with reporting.

2.5 Katalon Studio Katalon Studio is a powerful automation solution built on Selenium and Appium, supporting Web, Mobile, and API testing. It integrates with CI/CD pipelines and tools like JIRA, Jenkins, and Git, and offers analytics dashboards, custom keywords, and extensibility via external JARs.

2.6 Robot Framework Robot Framework, originated by Nokia‑Siemens and written in Python, is a keyword‑driven, cross‑platform automation framework. It supports Python, Java, Jython, IronPython, and PyPy, and provides a rich ecosystem of built‑in and third‑party libraries. Its architecture separates data, test framework, libraries/tools, and the system under test.

Tool Comparison

3.1 Browser and OS Support All tools are cross‑platform. Selenium, Sahi, Watir‑WebDriver, Katalon Studio, and Robot Framework run on Windows, macOS, and Linux with various browsers (Chrome, Firefox, IE, Safari where supported). UIAutomation focuses on client‑side applications and has limited web support.

3.2 Learning Curve Selenium offers IDE recording but requires programming for script maintenance; UIAutomation needs tools like Spy++ and has a steep entry barrier; Watir demands Ruby knowledge; Sahi provides a built‑in recorder with moderate difficulty; Katalon Studio supports both non‑programmers (Object Spy) and programmers; Robot Framework is easier for those familiar with keyword‑driven testing but still requires scripting.

3.3 Supported Languages Selenium supports Java, PHP, Python, Ruby, etc.; UIAutomation primarily uses C#; Watir uses Ruby; Sahi supports its own script, Java, and Ruby; Katalon Studio uses Java/Groovy; Robot Framework supports Python and Java.

Summary For web‑based recording, Katalon Studio offers the most user‑friendly visual scripting, while Selenium provides the widest language flexibility. UIAutomation excels in client‑side application recording with accurate element identification.

Conclusion The bank’s Automated Testing Platform (ATP) has already wrapped QTP, Selenium, and UIAutomation to provide a unified description language, reducing entry barriers and meeting project needs. Future work will continue researching open‑source tools, reducing reliance on commercial products, and encapsulating new tools to further improve efficiency.