batch processing

Collection size

120 articles

Page 3 of 6

Big Data Technology Architecture

Aug 10, 2021 · Big Data

Building a Real‑Time Data Warehouse with Apache Flink and Apache Iceberg: Architecture, Challenges, and Best Practices

This article presents Tencent's practical experience of constructing a real‑time data warehouse by integrating Apache Flink with Apache Iceberg, covering background pain points of traditional Lambda architectures, Iceberg's table format and capabilities, Flink‑Iceberg sink design, small‑file handling, and future roadmap for a unified streaming‑batch data lake.

Apache FlinkApache IcebergBatch Processing

0 likes · 20 min read

Building a Real‑Time Data Warehouse with Apache Flink and Apache Iceberg: Architecture, Challenges, and Best Practices

IT Architects Alliance

Apr 23, 2023 · Backend Development

Common Interface Performance Optimization Strategies

This article outlines a comprehensive set of backend interface optimization techniques—including batch processing, asynchronous execution, caching, pre‑processing, pooling, parallelization, indexing, transaction sizing, code restructuring, deep pagination, SQL tuning, and lock granularity—to reduce latency and improve overall system efficiency.

API optimizationAsynchronous ExecutionBatch Processing

0 likes · 8 min read

Common Interface Performance Optimization Strategies

IT Architects Alliance

Apr 17, 2023 · Backend Development

Spring Batch Tutorial: Introduction, Architecture, Core Interfaces, and Practical Implementation

This article provides a comprehensive overview of Spring Batch, covering its purpose, typical business scenarios, core architecture and interfaces, and detailed step‑by‑step code examples for configuring jobs, steps, flows, parallel execution, decision making, nested jobs, data reading and writing, item processing, and job scheduling within a Spring Boot application.

Batch ProcessingJavaJob Scheduling

0 likes · 14 min read

Spring Batch Tutorial: Introduction, Architecture, Core Interfaces, and Practical Implementation

IT Architects Alliance

Jan 3, 2023 · Fundamentals

Performance Optimization Techniques: Indexing, Caching, Compression, Prefetching, Throttling, and Batch Processing

The article explores a wide range of performance‑optimization strategies—including indexing, caching, compression, prefetching, peak‑shaving, and batch processing—explaining their trade‑offs, practical applications, and how they relate to hardware latency and system design in modern computing environments.

Batch ProcessingIndexingPerformance

0 likes · 34 min read

Performance Optimization Techniques: Indexing, Caching, Compression, Prefetching, Throttling, and Batch Processing

IT Architects Alliance

May 14, 2022 · Backend Development

Request Collapsing Techniques: Hystrix Collapser, Custom BatchCollapser, and ConcurrentHashMultiset

This article compares three request‑collapsing techniques—Hystrix Collapser, a custom BatchCollapser, and Guava’s ConcurrentHashMultiset—detailing their designs, implementations, configurations, and suitable scenarios for reducing downstream load and improving system throughput, including code examples, timer‑based batching, and thread‑safe container usage.

Batch ProcessingGuavaHystrix

0 likes · 14 min read

Request Collapsing Techniques: Hystrix Collapser, Custom BatchCollapser, and ConcurrentHashMultiset

Java Architect Essentials

Dec 5, 2023 · Backend Development

Comprehensive Interface Performance Optimization Strategies

This article presents a collection of practical backend interface optimization techniques—including batch processing, asynchronous execution, caching, pre‑processing, pooling, parallelism, indexing, transaction management, code restructuring, pagination, SQL tuning, and lock granularity—to reduce latency and improve overall system efficiency.

API optimizationBatch ProcessingPerformance

0 likes · 10 min read

Comprehensive Interface Performance Optimization Strategies

Java Architect Essentials

Nov 12, 2023 · Backend Development

Request Merging in Java: Concept, Pros & Cons, and Implementation with ScheduledExecutorService

This article explains the concept of request merging for high‑concurrency web services, outlines its advantages and drawbacks, and provides a complete Java implementation using ScheduledExecutorService, a memory queue, and a generic BatchCollapser utility with usage examples.

Batch ProcessingJavaScheduledExecutorService

0 likes · 8 min read

Request Merging in Java: Concept, Pros & Cons, and Implementation with ScheduledExecutorService

Java Architect Essentials

Oct 3, 2023 · Backend Development

Implementing Request Merging with ScheduledExecutorService in Java

This article explains the concept, advantages, and drawbacks of request merging, and provides a complete Java implementation using ScheduledExecutorService, a memory queue, and generic batch handling interfaces, along with usage examples and sample code.

Batch ProcessingConcurrencyJava

0 likes · 8 min read

Implementing Request Merging with ScheduledExecutorService in Java

Architecture Digest

Oct 24, 2023 · Backend Development

Request Merging and Batch Processing in Spring Boot to Reduce Database Connections

This article explains how to merge multiple user requests on the server side, batch them into a single SQL query using a blocking queue and CompletableFuture, and handle high‑concurrency scenarios with scheduled tasks and timeout‑aware queues to save database connection resources.

Batch ProcessingConcurrencyJava

0 likes · 13 min read

Request Merging and Batch Processing in Spring Boot to Reduce Database Connections

Architecture Digest

Apr 26, 2023 · Backend Development

Common API Performance Optimization Strategies

This article outlines a comprehensive set of backend API performance optimization techniques—including batch processing, asynchronous execution, caching, pre‑processing, pooling, parallelization, indexing, transaction handling, SQL tuning, and lock granularity—to reduce latency and improve system efficiency.

API performanceAsynchronous ExecutionBatch Processing

0 likes · 9 min read

Common API Performance Optimization Strategies

Architecture Digest

Nov 7, 2022 · Backend Development

Comprehensive Guide to Software Performance Optimization: Indexing, Compression, Caching, Prefetching, Throttling, and Batch Processing

This article presents a thorough, multi‑part exploration of software performance optimization techniques—including indexing, compression, caching, prefetching, peak‑shaving, and batch processing—explaining their principles, trade‑offs, practical applications, and how they relate to hardware constraints and system design.

Batch ProcessingIndexingPerformance

0 likes · 36 min read

Comprehensive Guide to Software Performance Optimization: Indexing, Compression, Caching, Prefetching, Throttling, and Batch Processing

Architecture Digest

Jan 21, 2022 · Big Data

Building a Real-Time Data Warehouse with Flink: Architecture, Core Concepts, and Practical Implementation

This article explains how to build a unified stream‑batch real‑time data warehouse using FlinkSQL, covering prerequisite knowledge, five core concepts, two implementation approaches, a comparison of traditional versus real‑time architectures, and a comprehensive hands‑on example, illustrated with diagrams.

Batch ProcessingFlinkdata architecture

0 likes · 6 min read

Building a Real-Time Data Warehouse with Flink: Architecture, Core Concepts, and Practical Implementation

Architecture Digest

Jan 20, 2022 · Backend Development

Implementing Scheduled Device Upgrade with Spring Batch and Quartz in Spring Boot

This article explains how to handle a PC‑triggered device upgrade record by using Quartz for timed execution and Spring Batch for bulk processing, detailing Maven dependencies, YAML configuration, service and batch classes, custom reader/writer logic, a processor that calls an upgrade‑dispatch API, and the overall challenges encountered.

Batch ProcessingDatabaseJava

0 likes · 13 min read

Implementing Scheduled Device Upgrade with Spring Batch and Quartz in Spring Boot

Architecture Digest

Jul 17, 2021 · Backend Development

Introduction to Spring Batch and Its Core Concepts

Spring Batch is a lightweight, comprehensive Java batch processing framework that provides reusable features such as job/step architecture, ItemReader/Writer/Processor, chunk processing, transaction management, and restart capabilities, with detailed explanations of core concepts, configuration examples, and best practices for building robust enterprise batch jobs.

Batch ProcessingChunkJava

0 likes · 19 min read

Introduction to Spring Batch and Its Core Concepts

Architecture Digest

Mar 11, 2020 · Big Data

Apache Flink: Unified Stream and Batch Processing Architecture and Core Concepts

This article provides a comprehensive overview of Apache Flink, explaining how it unifies stream and batch processing on a single runtime, detailing its key features, APIs, libraries, architectural components, fault‑tolerance mechanisms, scheduling, iterative processing, and back‑pressure monitoring.

Apache FlinkBatch Processingbackpressure

0 likes · 20 min read

Apache Flink: Unified Stream and Batch Processing Architecture and Core Concepts

Architecture Digest

Sep 3, 2017 · Big Data

An Overview of Big Data Processing Frameworks: Batch, Stream, and Hybrid Systems

This article introduces the evolution of big‑data processing from Google’s MapReduce concept to modern open‑source frameworks, defines big data and its 3V characteristics, outlines typical processing pipelines, and compares batch, stream, and hybrid systems such as Hadoop, Storm, Samza, Spark, and Flink.

Batch ProcessingBig DataData Processing

0 likes · 20 min read

An Overview of Big Data Processing Frameworks: Batch, Stream, and Hybrid Systems

Code Ape Tech Column

Sep 16, 2023 · Backend Development

Batch Request Merging in Java to Reduce Database Connections

This article explains how to merge multiple user‑detail requests on the server side using a blocking queue, scheduled thread pool and CompletableFuture in Spring Boot, thereby converting many individual SQL calls into a single batch query, saving database connections and improving high‑concurrency performance.

Batch ProcessingCompletableFutureConcurrency

0 likes · 13 min read

Batch Request Merging in Java to Reduce Database Connections

Code Ape Tech Column

Dec 29, 2022 · Backend Development

Batch Request Merging in Spring Boot to Reduce Database Connection Overhead

The article explains how to merge multiple user queries into a single batch request using a queue, scheduled thread pool and CompletableFuture in Spring Boot, thereby reducing database connections and improving resource utilization while handling high‑concurrency scenarios.

Batch ProcessingCompletableFutureConcurrency

0 likes · 13 min read

Batch Request Merging in Spring Boot to Reduce Database Connection Overhead

Code Ape Tech Column

Jul 16, 2022 · Backend Development

Spring Batch Architecture Overview and Core Concepts

This article introduces Spring Batch as a lightweight, comprehensive batch‑processing framework for enterprise applications, explains its overall architecture, and details core concepts such as Job, JobInstance, JobParameters, JobExecution, Step, StepExecution, ExecutionContext, JobRepository, JobLauncher, ItemReader, ItemWriter, ItemProcessor, chunk processing, skip/failure handling, best‑practice guidelines, and common troubleshooting tips.

Batch ProcessingChunkItemReader

0 likes · 20 min read

Spring Batch Architecture Overview and Core Concepts

FunTester

Oct 12, 2019 · Backend Development

Java Script for Batch Uploading Thousands of Images with File Path Retrieval and Request Execution

This article demonstrates a Java solution for automatically collecting absolute file paths, reading image URLs from a text file, and sequentially uploading over 12,000 images to a server, including code for path extraction, request handling, and error logging.

Batch ProcessingFile UploadJava

0 likes · 7 min read

Java Script for Batch Uploading Thousands of Images with File Path Retrieval and Request Execution