Tag

PyODPS

1 views collected around this technical thread.

DaTaobao Tech

Sep 13, 2024 · Big Data

Extending PyODPS with PAI‑Designer for Dynamic Offline Data Processing

By integrating PAI‑Designer with PyODPS, users can build visual offline workflows that overcome ODPS’s lack of network access, dynamic configuration, and image‑processing limits, using reusable Python components, OSS role‑ARNs, remote configuration fetching, and custom Docker images to read/write MaxCompute and OSS data.

Data ProcessingDockerMaxCompute

0 likes · 19 min read

Extending PyODPS with PAI‑Designer for Dynamic Offline Data Processing

DaTaobao Tech

Sep 11, 2024 · Big Data

Practical Guide to Using PyODPS for Flexible Data Processing

The article walks through a first‑time user’s experience with PyODPS, showing how its Python‑based DataFrame API offers more flexible JSON field statistics, multi‑condition filtering, and custom aggregations than traditional ODPS SQL, while noting a steep learning curve and syntax quirks.

Data ProcessingMaxComputePyODPS

0 likes · 11 min read

Practical Guide to Using PyODPS for Flexible Data Processing