BestHub
search
Sign in
Tag

Flink SQL

1 views collected around this technical thread.

vivo Internet Technology
vivo Internet Technology
May 24, 2023 · Big Data

Kafka Real-time Data Archiving to Hive: Flink SQL and DataStream Implementation Solutions

The article explains how to archive Kafka real‑time data to Hive using either Flink SQL, which quickly creates partitioned ORC tables but requires timezone handling, or Flink DataStream for more complex pipelines, and offers best‑practice guidance on data quality, system complexity, security, and performance.

Big DataDataStreamFlink
0 likes · 15 min read
Kafka Real-time Data Archiving to Hive: Flink SQL and DataStream Implementation Solutions
Tencent Cloud Developer
Tencent Cloud Developer
May 21, 2021 · Big Data

Tencent Cloud Oceanus: Flink SQL Optimization and Extension Practices

Tencent Cloud Oceanus, a computing service powering internal apps like WeChat and external partners such as Bilibili, scales to over 30,000 cores handling 5 PB daily and 500,000 jobs, and tackles Flink SQL’s syntax, function and operational limits with table‑valued functions, incremental and enhanced tumble windows, and caching‑based retraction optimization that cuts downstream data volume up to 30× and improves join performance by about 20 %.

Big DataFlink SQLOceanus
0 likes · 19 min read
Tencent Cloud Oceanus: Flink SQL Optimization and Extension Practices