distributed machine learning

Didi Tech

Jan 25, 2024 · Artificial Intelligence

Ray-native XGBoost Training Platform: Architecture, Performance, and Technical Challenges

Didi’s new Ray‑native XGBoost training platform replaces the fault‑prone Spark solution with a fully Pythonic, fault‑tolerant architecture that leverages Ray’s autoscaling and gang‑scheduling, delivering 2–6× speedups, reduced failure rates, efficient sparse‑vector handling, scalable hyper‑parameter search, and improved resource utilization for large‑scale machine‑learning workloads.

Hyperparameter OptimizationRayXGBoost

0 likes · 20 min read

Ray-native XGBoost Training Platform: Architecture, Performance, and Technical Challenges

DataFunTalk

May 28, 2021 · Artificial Intelligence

JD's Open‑Source Federated Learning Solution 9N‑FL: Architecture, Features, Timeline, and Business Impact

This article introduces JD's open‑source federated learning platform 9N‑FL, explaining the data‑island problem, the fundamentals and classifications of federated learning, its four key features, the system’s layered architecture, development timeline, real‑world advertising use case results, and future enhancements.

9N-FLFederated LearningJD

0 likes · 15 min read

JD's Open‑Source Federated Learning Solution 9N‑FL: Architecture, Features, Timeline, and Business Impact

58 Tech

Nov 20, 2020 · Artificial Intelligence

Evolution and Practice of the 58.com AI Algorithm Platform (WPAI)

The article details the development, architecture, and optimization of 58.com’s AI algorithm platform (WPAI), covering its background, overall design, large‑scale distributed machine learning, deep‑learning platform features, inference performance enhancements, GPU resource scheduling improvements, and future directions.

AI PlatformDeep LearningGPU scheduling

0 likes · 15 min read

Evolution and Practice of the 58.com AI Algorithm Platform (WPAI)

DataFunTalk

Oct 14, 2020 · Artificial Intelligence

Angel Machine Learning Platform: Architecture, Deep Learning Extensions, and Applications in Tencent Advertising Recommendation System

This article introduces Tencent's self‑built Angel distributed machine‑learning platform, describes its architecture and deep‑learning extensions (Parameter Server and AllReduce), explains how it powers the advertising recommendation pipeline with models such as DSSM, VLAD and YOLO, and presents extensive training‑level optimizations that yield multi‑fold performance improvements.

AngelDeep LearningPerformance Optimization

0 likes · 15 min read

Angel Machine Learning Platform: Architecture, Deep Learning Extensions, and Applications in Tencent Advertising Recommendation System

DataFunTalk

Jul 26, 2020 · Artificial Intelligence

Federated Learning: Fundamentals, Applications, Challenges, and Implementation Methods

This article explains federated learning as a privacy‑preserving distributed machine learning paradigm, discusses why it has become popular, describes its three core components, demonstrates its advantages over traditional models, outlines real‑world use cases in medicine and finance, and analyzes current technical and commercial challenges together with implementation techniques such as horizontal/vertical federation and homomorphic encryption.

Artificial IntelligenceFederated Learningbig data

0 likes · 32 min read

Federated Learning: Fundamentals, Applications, Challenges, and Implementation Methods

JD Tech Talk

Apr 3, 2020 · Artificial Intelligence

Federated Learning: Application Prospects, Deployment Challenges, and Implementation Methods

This article examines federated learning’s wide‑range application prospects in healthcare, mobile internet, and finance, analyzes the technical and regulatory challenges of deploying such systems, and explains the concrete implementation steps for horizontal and vertical federated learning architectures.

AIFederated LearningFinance

0 likes · 11 min read

Federated Learning: Application Prospects, Deployment Challenges, and Implementation Methods

JD Tech Talk

Mar 27, 2020 · Artificial Intelligence

Understanding Federated Learning: Origins, Applications, and Privacy Protection Techniques

This article explains the rapid rise of federated learning, its technical foundations combining machine learning, distributed computing, and privacy protection, practical use cases, intuitive privacy examples, and empirical evidence that it can improve model performance without compromising data security.

Artificial IntelligenceFederated Learningdata security

0 likes · 15 min read

Understanding Federated Learning: Origins, Applications, and Privacy Protection Techniques

High Availability Architecture

Aug 2, 2017 · Artificial Intelligence

A Comparative Study of Distributed Machine Learning Platforms: Design Methods and Evaluation

This article surveys design approaches for distributed machine learning platforms, classifies them into basic dataflow, parameter‑server, and advanced dataflow models, examines examples such as Spark, PMLS, TensorFlow and MXNet, and presents performance evaluations and future research directions.

SparkTensorFlowdistributed machine learning

0 likes · 10 min read

A Comparative Study of Distributed Machine Learning Platforms: Design Methods and Evaluation

Architects Research Society

Oct 19, 2015 · Artificial Intelligence

Efficient Distributed Machine Learning on Azure: Overcoming Communication Bottlenecks

The article discusses Microsoft’s research on scalable distributed machine‑learning on Azure, highlighting the challenges of communication overhead, the use of Vowpal Wabbit and Statistical Query Model techniques, and proposing algorithms that reduce iteration counts to achieve faster, cost‑effective predictive analytics for large‑scale data.

Azure MLbig datacommunication bottleneck

0 likes · 12 min read

Efficient Distributed Machine Learning on Azure: Overcoming Communication Bottlenecks

Art of Distributed System Architecture Design

Aug 21, 2015 · Artificial Intelligence

Facebook’s Distributed Recommendation System: Architecture, Algorithms, and Performance

The article explains how Facebook built a large‑scale distributed recommendation system using Apache Giraph, collaborative filtering with matrix factorization, SGD and ALS algorithms, a novel work‑to‑work communication scheme, and performance optimizations that achieve ten‑fold speedups on billions of ratings.

ALSApache GiraphFacebook

0 likes · 9 min read