Practical Approaches to Deploying Machine Learning Models: Real‑time SOA, PMML, Rserve, and Spark

The author, a BI manager at Ctrip Hotels, discusses common frustrations when complex models (GBDT, XGBoost) outperform simpler ones but are hard to deploy, leading engineers to request a switch to logistic regression for speed.

He then outlines three main deployment scenarios and corresponding solutions:

Real‑time, small‑volume predictions: Use SOA to call Rserve or a Python‑httpserve service. For up to ~1,000 requests per call, 95% of responses can be returned within 100 ms; larger volumes may require batching or multithreading.

Real‑time, large‑volume predictions: Convert the trained model to PMML, wrap it in a Java class, and call it directly from Java, eliminating external runtime dependencies and achieving fast, stable performance.

Offline (D+1) predictions: Run the model via an Rscript or Python script scheduled with a tool such as cron; a simple shell wrapper can invoke the script and load results into Hive.

All three approaches share a common SOA layer for data preprocessing; most transformations should be performed there to avoid performance penalties.

PMML conversion and usage are demonstrated with links to the jpmml‑evaluator project and a GitHub example showing how to load and evaluate a PMML model in Java.

Python model deployment uses PMML conversion for scikit‑learn and XGBoost models, noting that missing‑value handling can affect predictions. A single record prediction takes about 1 ms.

R model deployment can be done via PMML or Rserve. The Rserve approach involves deploying R and Rserve on a server, writing a Java SOA client, and invoking a prediction function. Example resources for Rserve setup are provided.

The Rserve prediction function is shown below:

Pred.R <- function(x1,x2,x3){
  data <- cbind(x1,x2,x3)
  # feature engineering
  score <- predict(modelname, data, type = 'prob')
  return(list(score))
}

Spark model deployment trains models (often XGBoost) in Scala, then loads the model class in Java for fast inference. Options include deploying the model on a Spark cluster or exporting it to PMML for use elsewhere.

Simple shell‑based offline deployment is illustrated with an R script and a shell wrapper:

# 数据导出
data_filename = xxx
file_date = xxx
result = xxx
updatedt = xxx
cd path
hive -e "USE tmp_xxxdb;SELECT * FROM db.table1;" > ${data_filname};
# R脚本预测
Rscript path/predict.R  $file_date
if [ $? -ne 0 ]
then 
  echo "Running RScript Failure"
fi
# R预测的结果导入Hive表
list1="use tmp_htlbidb;
load data local inpath 'path/$result'
overwrite into table table2 partition(dt='${updatedt}');"
hive -e "$list1"

Scheduling is performed with a simple crontab entry:

>crontab -e
-------------------------
### 每天5点进行预测模型；
0 5 * * * sh predict.sh

Finally, the article discusses data flow considerations: distinguishing offline vs. real‑time data, using Redis with dual‑batch keys for fallback, ensuring logging and monitoring (e.g., via Elasticsearch), and implementing fallback mechanisms for API timeouts.

The author invites readers to discuss best practices and provides links to related technical articles.