Tag

DataStream

1 views collected around this technical thread.

Tencent Cloud Developer
Tencent Cloud Developer
Jul 2, 2024 · Big Data

Apache Flink Deployment with Pulsar Connector: Setup, Demos, and Best Practices

This guide shows how to deploy Apache Flink 1.17 in Docker, configure off‑heap memory, connect it to Pulsar via the 4.1.0‑1.17 connector, run example jobs that copy topics and perform windowed word‑count, and provides Maven dependencies, custom serialization tips, batching settings, and version‑specific best‑practice notes.

Apache FlinkDataStreamDocker Deployment
0 likes · 20 min read
Apache Flink Deployment with Pulsar Connector: Setup, Demos, and Best Practices
vivo Internet Technology
vivo Internet Technology
May 24, 2023 · Big Data

Kafka Real-time Data Archiving to Hive: Flink SQL and DataStream Implementation Solutions

The article explains how to archive Kafka real‑time data to Hive using either Flink SQL, which quickly creates partitioned ORC tables but requires timezone handling, or Flink DataStream for more complex pipelines, and offers best‑practice guidance on data quality, system complexity, security, and performance.

DataStreamFlinkFlink SQL
0 likes · 15 min read
Kafka Real-time Data Archiving to Hive: Flink SQL and DataStream Implementation Solutions
DataFunTalk
DataFunTalk
Sep 8, 2022 · Databases

Elasticsearch as a Time Series Engine: Practices, Challenges, and Alibaba Cloud TimeStream Solutions

This article explains why Elasticsearch is being adapted as a time‑series engine, outlines its unique characteristics and challenges such as high query complexity and storage cost, and introduces Alibaba Cloud’s TimeStream solution with optimizations like index settings, compression, down‑sampling, and Prometheus integration.

DataStreamElasticsearchbig data
0 likes · 13 min read
Elasticsearch as a Time Series Engine: Practices, Challenges, and Alibaba Cloud TimeStream Solutions
Big Data Technology Architecture
Big Data Technology Architecture
Nov 2, 2021 · Big Data

Comprehensive Guide to FlinkSQL and Table API: Background, Dependencies, Planners, and Usage

This article provides a detailed introduction to FlinkSQL, covering its background, the Table API, required dependencies, differences between old and Blink planners, various API usage patterns, connector configurations for CSV, Kafka, Elasticsearch, MySQL, and how to convert between DataStream and Table in Flink's unified batch‑stream processing model.

ConnectorDataStreamFlinkSQL
0 likes · 23 min read
Comprehensive Guide to FlinkSQL and Table API: Background, Dependencies, Planners, and Usage
DataFunTalk
DataFunTalk
Jun 29, 2021 · Big Data

In-depth Analysis of Flink SQL 1.13 Features and Improvements

This article provides a comprehensive overview of Apache Flink SQL 1.13, detailing new Window TVF support, cumulate windows, performance optimizations, time‑zone handling, enhanced Hive compatibility, SQL client upgrades, DataStream‑Table conversion improvements, and outlines the roadmap for the upcoming 1.14 release.

DataStreamFlinkHive Integration
0 likes · 15 min read
In-depth Analysis of Flink SQL 1.13 Features and Improvements
HomeTech
HomeTech
Mar 11, 2020 · Big Data

Streaming SQL with Apache Flink: Theory, Platform Optimizations, and Real‑Time Use Cases

This article introduces Apache Flink's Streaming SQL, explains its theoretical foundations such as the table‑stream relationship and watermark semantics, describes the platform's practical enhancements—including source/sink wrappers, built‑in functions, and native Retract Stream support—and showcases several real‑time computation examples.

Apache FlinkDataStreamRetract Stream
0 likes · 31 min read
Streaming SQL with Apache Flink: Theory, Platform Optimizations, and Real‑Time Use Cases
Qunar Tech Salon
Qunar Tech Salon
Feb 20, 2019 · Big Data

Building Real-Time User Behavior Engineering with Apache Flink: Architecture, Features, and Implementation

This article introduces the design and implementation of a real‑time user behavior engineering platform at Qunar using Apache Flink, covering Flink's core characteristics, distributed runtime, DataStream programming model, fault‑tolerance, back‑pressure handling, event‑time processing, windowing, watermarks, and practical code examples for filtering, splitting, joining, and state management.

CheckpointDataStreamEventTime
0 likes · 18 min read
Building Real-Time User Behavior Engineering with Apache Flink: Architecture, Features, and Implementation