Llama 4 Open‑Source Release Marred by Performance Failures and Alleged Training‑Data Cheating

Meta released Llama 4, but the launch turned into a scandal when insiders reported that the model’s training data had been mixed with benchmark test sets to artificially boost scores, effectively cheating on standard evaluations.

Multiple internal employees, including a whistleblower who refused to be named in the technical report, disclosed that senior leadership imposed a hard deadline for delivery at the end of April, leading to resignations within the organization.

Early open‑source testing showed Llama 4’s code‑generation abilities were far below expectations, with the Maverick variant producing irregular, non‑physical animations and performing worse than GPT‑4o. Comparative tests by community members demonstrated that Llama 4’s programming performance was comparable only to much smaller models such as Qwen‑32B, and lagged behind state‑of‑the‑art models like Gemini Flash, Grok 3, DeepSeek V3, and Sonnet 3.5/7.

Further analysis highlighted that the model’s official performance charts label the Maverick version as “optimized for conversationality,” suggesting a deliberate bias toward benchmark scores rather than genuine capability. Researchers also observed significant discrepancies between the publicly downloadable Maverick model and the version hosted on LM Arena.

Additional internal reports indicated that the training process repeatedly failed to achieve SOTA benchmarks, prompting Meta’s leadership to mix various benchmark datasets into the later training stages to artificially improve results.

Community reactions were overwhelmingly negative, with users describing Llama 4 as a disappointing programming model, too large for practical deployment, and lacking meaningful improvements over previous versions.