How to Preprocess Evaluation Data: Expert Survey, Min‑MSE & Max‑Min Methods Explained

1 Data Preprocessing Methods

In comprehensive evaluation indices, some indicators are crucial while others are negligible, and they may differ in type, unit, or magnitude, causing incompatibility and complicating overall assessment. To reflect reality accurately, eliminate these disparities through preprocessing, which includes indicator selection, standardization, dimensionless conversion, and quantifying qualitative data.

2 Selection of Evaluation Indicators

Based on the evaluation purpose, gather relevant indicator information for the specific object and content, then apply appropriate selection methods to choose primary indicators and discard secondary ones, simplifying the indicator system.

Common selection methods include the Delphi expert survey, minimum mean‑square‑error method , and maximum‑minimum deviation method .

2.1 Expert Survey Method

The expert survey comprises the Delphi method and brainstorming. The Delphi method, introduced by the RAND Corporation in 1964, is an intuitive forecasting technique suitable for long‑term predictions when objective data are scarce.

It relies on experts’ knowledge and experience to judge, assess, and predict issues.

2.1.1 Steps

Procedure:

Appoint a moderator and form a dedicated team.

Draft a clear, specific questionnaire with a reasonable number of questions and necessary background material.

Select experts who are representative, knowledgeable, reputable, and possess strong judgment and insight; typically 10–50 experts.

Conduct multiple rounds of consultation (usually three): First round: pose questions and collect completed questionnaires. Second round: provide summarized opinions and request revised responses. Third round: finalize conclusions by aggregating reconsidered opinions.

Analyze results, often using the median to form conclusions, and compile a report.

2.1.2 Characteristics

Iterative communication via written correspondence.

Multi‑directional input from experts across various fields.

Anonymous feedback to encourage independent views.

Repeated controlled feedback to converge opinions.

Statistical aggregation of expert judgments into a collective conclusion.

2.1.3 Applicable Conditions

The method is effective when:

Data are lacking, insufficient, or costly to obtain.

Evaluating new technologies without existing data.

Non‑technical factors (environmental, public opinion, political) dominate decision‑making.

Additionally, when the volume of raw information is massive and processing costs are high, expert surveys become a cost‑effective alternative.

2.2 Minimum Mean‑Square‑Error Method

For each evaluation object with multiple indicators, the method assumes that if an indicator’s values are similar across objects, its impact on the overall evaluation is minimal, allowing the indicator to be removed.

2.2.1 Steps

Calculate the mean and mean‑square deviation for each indicator.

Identify the smallest mean‑square deviation.

If the smallest deviation is below a predefined threshold, delete the corresponding indicator. Repeat for all indicators to obtain the final indicator system.

2.3 Maximum‑Minimum Deviation Method

Similar to the previous method, this approach evaluates the range of indicator values across objects.

2.3.1 Steps

Compute the maximum deviation for each indicator.

Compute the minimum deviation.

If the minimum deviation meets a set criterion, remove the associated indicator; continue until the final set is derived.

3 Summary

This article introduced three key data selection techniques for evaluation: the expert survey method, the minimum mean‑square‑error method, and the maximum‑minimum deviation method.

References

ThomsonRen https://github.com/ThomsonRen/mathmodels

刘建明，王泰玄等.宣传舆论学大辞典..经济日报出版社.

https://baike.sogou.com/v7811976.htm

司守奎，孙玺菁 Python数学实验与建模