From KV Cache to Harness: How DeepSeek Is Shifting Costs to the System Layer

DeepSeek’s V4‑Preview (released 2026‑04‑24) offers two model variants: V4‑Pro with 1.6 T total parameters (49 B activation) and V4‑Flash with 284 B total parameters (13 B activation), both supporting a 1 M token context window. The announcement emphasizes that long‑context workloads dramatically reduce compute and memory costs compared with earlier generations.

The V2 paper (arXiv:2405.04434) already quantified the efficiency gains: training cost dropped 42.5 % relative to the 67 B baseline, KV‑Cache memory usage fell 93.3 %, and maximum generation throughput increased 5.76×. These figures illustrate the first wave of cost migration from raw GPU compute to smarter algorithmic tricks.

DeepSeek’s engineering roadmap (V2 → V4) consistently targets three layers of cost reduction:

Cache: MLA compresses KV‑Cache with low‑rank latent representations; Engram moves static knowledge to conditional memory look‑ups.

Memory & Storage: longer contexts are off‑loaded to disk caches (best‑effort hit rate) and memory‑side look‑ups, reducing reliance on expensive HBM.

Hardware & Scheduling: MoE sparsifies activation, while custom kernels (TileKernels) and compiler optimizations handle the remaining compute.

These optimizations mean that the most expensive resource is no longer matrix multiplication on H100‑class GPUs but the engineering of the surrounding system. KV‑Cache, once a minor inference tweak, becomes a runtime cost centre for long‑task agents that repeatedly ingest codebases, tool specifications, logs, and test results.

DeepSeek’s hiring signal for a "Harness" team reflects a strategic shift: the model is treated as an engine, while the surrounding execution environment—file I/O, command execution, test feedback, context management, observability, verification, and governance—forms a distinct engineering layer. The Model + Harness = Agent equation captures this view.

Agent Harness surveys (e.g., "Agent Harness Engineering: A Survey" PDF) describe the layer in two parts: an execution/structure tier (environment, tool interfaces, context, lifecycle) and a control tier (observability, verification, governance). This separation explains why cheap models do not automatically yield cheap agents; the system layer can re‑introduce cost through repeated context handling, tool misuse, or inadequate validation.

From a hardware‑ecosystem perspective, DeepSeek’s TileKernels project (GitHub: https://github.com/deepseek-ai/TileKernels) implements MoE routing, Engram gating, and other kernels in TileLang, aiming to bridge algorithmic advances and hardware back‑ends. Successful kernel adoption would allow the model to dictate hardware‑specific data layouts, communication patterns, and scheduling policies, similar to the OpenAI‑AMD partnership (AMD‑OpenAI 6 GW deal).

Business‑model implications follow the technical shift: if DeepSeek can define a workload that hardware vendors, cloud providers, and toolchains optimize for, it may capture value beyond API margins—controlling standards, influencing hardware roadmaps, and extracting ecosystem fees. The article lists five observable signals for this transition:

Sustained low price and high cache‑hit rates for long‑context workloads.

Public hardware‑adapter optimizations targeting DeepSeek’s MoE, KV‑Cache, and Engram.

Open‑source engineering artifacts (TileKernels, kernel benchmarks, reproducibility scripts).

Mature Harness products that manage repository context, tool calls, command execution, testing feedback, permissions, and rollback.

Transparent commercial agreements (hardware procurement, joint road‑maps, equity incentives).

Until such signals materialize, DeepSeek remains a low‑cost model supplier; once they appear, the company could evolve into an AI‑infrastructure player that defines the next‑generation workload stack.