2017 Direct Banking App Evaluation Report and Scoring Criteria

In 2016, direct banking apps proliferated across China, expanding from a few to over 60 providers, yet no third‑party evaluation existed. Sina Finance’s financial evaluation team partnered with Baidu MTC to launch the "2017 Direct Banking" assessment, applying a media‑plus‑technology approach to produce an objective, professional report that drives industry progress.

This year the evaluation scope was broadened to include joint‑stock banks, city commercial banks, and other institutions, covering a total of 60 banks. Sina Finance used Baidu MTC’s testing results as the sole technical and user‑experience reference for selecting the top ten direct banking apps at the 2017 China Banking Development Forum.

The assessment examined five major dimensions—technical performance, user‑experience convenience, functional investment features, operational security, and development potential—through more than 58 specific indicators (see the attached scoring standards). Technical evaluation followed Baidu MTC’s industry‑standard criteria, focusing on compatibility issues, performance metrics, UI problems, efficiency, plugin security, and vulnerability detection.

Each workday a new report on a direct‑banking app was released; by the time of writing, 39 app reports had been published and are accessible at http://finance.sina.com.cn/zt_d/2017zx .

Baidu Mobile Test Center (MTC) is a leading one‑stop mobile application testing platform that has supported over 140 Baidu mobile product lines, serving more than 150,000 developers and completing over 20 million test runs, ensuring app availability across major app stores.

For detailed test reports or inquiries about compatibility, functionality, performance, or security issues of apps on mainstream mobile devices, please contact Baidu MTC at [email protected] .

Below is the translated scoring rubric used for the 2017 evaluation:

Evaluation Category

Sub‑Category

Specific Indicator

Technical (20 points)

Compatibility

Mainstream device compatibility rate

Compatibility issues

Performance

Startup duration

CPU usage

Memory usage

Data traffic consumption

Battery consumption

Code Security

Application security level

Plugin status

Vulnerability overview

UI Clarity

Screen issues (flicker, black/white screens)

Convenience (25 points)

App Download

Fuzzy name search

Acquisition channels (PC website, WeChat, mobile banking)

UI Experience

Main interface layout

Self‑service settings

Intelligent adjustments

Registration & Account Opening

Process clarity

Operation smoothness

Technical innovation

Support for multiple bank cards (list, limits)

Login & Transaction

Remember username

Activation amount after opening

Funds Management

Card replacement capability

Automatic transfer to savings products

Change Management

Support for changing registered phone

Clarity of phone change process

Account Closure

Electronic account cancellation

Clear cancellation prompts

Customer Service

Phone support

Online support

Feedback mechanism

Guidance

New user guide

FAQ

Functionality (28 points)

Wealth Management

Bank wealth products (count, proportion, highest yield)

Baby products (type, yield)

Funds (count, type)

Insurance

Precious metals

Other

Deposits & Loans

Deposits

Loans

Life Services

Credit card repayment

Bill payment & top‑up

Other

Security (17 points)

Account Opening & Login

Opening information

Login password

New password types

CAPTCHA or anti‑bot measures

Security prompts

Forgot Password

Information entry

Dynamic password via mobile

Fund Transfers

Transfer password (inbound)

Transfer password (outbound)

Transfer change alerts

Information Disclosure

Prominent major announcement location

Complete wealth product contracts

Clear product maturity dates

Product yield calculation start date

Growth Potential (10 points)

App update frequency

Strategic importance

Additional Points (10 points)

Innovative cases

Negative public opinion

The evaluation also included supplementary scoring for innovation cases and negative public sentiment, providing a comprehensive view of each app’s strengths and weaknesses.