IBM Donates SystemML to Apache Incubator, Joining the Open‑Source Machine Learning Wave

Tech giants are racing to open‑source their machine‑learning systems to become dominant platforms, and Apache SystemML now carries a clear enterprise focus.

IBM announced on Monday that its machine‑learning system, called SystemML, has been accepted as an Apache Incubator project.

The Apache Incubator serves as the entry point for projects to join the Apache Software Foundation, ensuring code donations meet legal guidelines and community principles.

IBM plans to donate SystemML as an open‑source project in June.

This milestone reflects a growing trend of open‑sourcing machine‑learning tools, exemplified by Google’s TensorFlow and Facebook’s contributions to the Torch project.

Enterprises will soon have a variety of open‑source machine‑learning codebases to choose from; TensorFlow and Torch target neural‑network training, while SystemML aims to broaden the ecosystem for business‑oriented use.

Companies go open source to attract more data, which in turn makes their platforms more powerful; IBM is focusing on the enterprise angle, and other players such as Microsoft may follow, though not necessarily via the Apache route.

Historical precedents like the open‑source success of MapReduce and Hadoop show how open analytics can pay off.

IBM’s SystemML, now Apache SystemML, is used to create industry‑specific machine‑learning algorithms for enterprise data analysis, offering a single codebase that can serve multiple industries and platforms, and potentially opening doors to IBM’s broader analytics portfolio.

The Apache SystemML project has already incorporated over 320 patches, more than 90 contributions to Apache Spark, and involvement from 15 additional organizations.

SystemML provides declarative large‑scale machine learning (ML) that aims at flexible specification of ML algorithms and automatic generation of hybrid runtime plans ranging from single‑node, in‑memory computations to distributed computations on Apache Hadoop and Apache Spark. ML algorithms are expressed in an R or Python syntax, including linear algebra primitives, statistical functions, and ML‑specific constructs. This high‑level language significantly increases data‑scientist productivity by offering full flexibility in expressing custom analytics and data‑independence from underlying input formats and physical data representations. Automatic optimization based on data characteristics such as disk distribution, sparsity, and processing environment (e.g., number of nodes, CPU, memory per node) ensures both efficiency and scalability.

For more information, visit the Apache SystemML project page: http://systemml.incubator.apache.org/ .