BestHub
search
Sign in
Tag

DStream

1 views collected around this technical thread.

Beike Product & Technology
Beike Product & Technology
Sep 20, 2019 · Big Data

Understanding DStream Construction and Execution in Spark Streaming

This article explains how Spark Streaming's DStream abstraction is built from InputDStream through successive transform operators, details the internal ForEachDStream implementation, describes the job generation and scheduling workflow, and outlines how Beike's real‑time platform leverages these mechanisms for large‑scale streaming tasks.

DStreamScalaSpark Streaming
0 likes · 10 min read
Understanding DStream Construction and Execution in Spark Streaming
Qunar Tech Salon
Qunar Tech Salon
Jun 20, 2018 · Big Data

How Spark Streaming Submits Tasks: Internal Mechanics and Code Walkthrough

This article explains the internal workflow of Spark Streaming task submission, detailing how StreamingContext, DStream, receivers, and output operators are transformed into RDD jobs, and includes annotated Scala code examples that illustrate each step of the process.

Big DataDStreamScala
0 likes · 13 min read
How Spark Streaming Submits Tasks: Internal Mechanics and Code Walkthrough