Top 10 AI Papers This Week: SkillOpt, Agent Distillation, and Sleeping LLMs

This article highlights ten AI research papers that advance agent training paradigms, inference efficiency, long‑context reliability, and scientific methodology.

1. SkillOpt: Treating SKILL.md as Trainable Parameters

Microsoft Research proposes a counter‑intuitive idea: view a concise natural‑language skill document (SKILL.md) as a frozen agent’s "trainable state" and optimise the document itself via rollout, reflection, and bounded editing.

Skill document as parameter: an optimiser model suggests add/delete/modify commands for the document; each edit must pass a held‑out validation set. The paper introduces a "text learning rate" to control rewrite aggressiveness, and expresses batch and momentum in textual form rather than gradients.

Validation gate replaces intuition: edits are accepted only after validation, turning prompt‑tuning from a feel‑based activity into a measurable optimisation loop.

52 wins, 0 losses: on six benchmarks and seven target models, SkillOpt beats Trace2Skill, TextGrad, GEPA, EvoSkill, human‑written skills and one‑shot skills in all 52 pairwise comparisons. Compared with a no‑skill baseline, GPT‑5.5 direct dialogue gains +23.5 points, Codex loop +24.8, Claude Code environment +19.1.

Significance: if the skill document is the true optimisation target, the bottleneck shifts from base‑model capability to how effectively we train the frozen agent’s natural‑language state, offering a cheap, model‑agnostic lever that most teams have not yet adopted.

2. Compiling the Agent Pipeline into Model Weights

The paper demonstrates that a full agentic workflow can be distilled into the weights of a small model, reducing inference cost by roughly two orders of magnitude while keeping task quality near the frontier.

Learning the process, not just the answer: the distilled model internalises multi‑step LLM calls, tool usage, intermediate scratchpad, and decision points, thereby learning the orchestration logic itself.

Scheduler dissolves into the model: classic agent frameworks invoke a planner loop on every request; embedding this loop in the weights eliminates the per‑call scheduling overhead.

Near‑frontier quality at hundred‑fold cost reduction: on evaluation tasks, the distilled small model matches the original workflow’s quality while inference cost drops ~100×, mainly because many model calls are compressed into a single forward pass.

Significance: production agents often pay thousands of dollars daily for the same orchestration logic; compiling that logic once into a cheap model fundamentally changes the economic model, especially for high‑frequency narrow scenarios.

3. AutoScientists: Decentralised AI Scientist Teams

Harvard’s AutoScientists hand over long‑cycle scientific computation to a decentralised AI agent team, removing the central planner entirely.

No central planner: agents interpret a shared experimental state, form teams around promising hypotheses, and regroup when progress stalls; coordination emerges from shared state rather than a top‑level controller, enabling parallel search.

Evaluate before spending: proposals are critiqued and scored before allocating compute, reducing wasted attempts and preventing agents from repeatedly hitting the same dead‑ends.

Hard results on real scientific tasks: on BioML‑Bench (24 biomedical ML tasks), AutoScientists achieve a 74.4% average leaderboard percentile, a +8.33% improvement over the previous strongest AI agent.

Significance: many multi‑agent systems funnel decisions to a single planner, creating a bottleneck; decentralised self‑organisation plus explicit failure sharing offers an alternative long‑term scientific search blueprint, validated on demanding benchmarks.

4. Giving Language Models a "Sleep" Mechanism

Attention scales poorly with context length; this paper introduces a sleep‑like consolidation where recent context is folded into fast weights, then the KV cache is cleared.

Consolidate then clear cache: recent context is compressed into the SSM block’s fast weights; only after this is the KV cache discarded, allowing the agent to retain learned information without carrying the full attention bill forward.

Compute moves to sleep phase, latency stays low: extra computation occurs offline during consolidation, while awake‑time prediction latency remains low; the trade‑off is explicit and controllable.

More complex tasks benefit most: longer sleep periods yield larger performance gains, especially on tasks requiring intricate reasoning over long histories; the mechanism helps where vanilla attention is most strained.

Significance: long‑context agents are the first systems to feel the quadratic cost of context; a biologically inspired consolidation step offers a principled alternative for "effectively infinite" context windows and can be cleanly inserted into existing state‑space architectures.

5. Modifying the Interface, Not the Model

Life‑Harness shows that many failures of frozen LLM agents stem from mismatched interfaces rather than reasoning ability, and can be fixed at runtime without retraining.

Failures become reusable interventions: recurring errors are turned into four runtime fixes—action grounding, environment contract, trajectory adjustment, procedural skill—each acting as a harness‑level patch that the agent can reuse later.

Model frozen, environment unchanged: only the interface between model and environment is adjusted, making the method plug‑and‑play for any backbone and avoiding fine‑tuning cost and risk.

Stable overall gains: across seven deterministic agent benchmarks and eighteen model backbones, Life‑Harness improves 116 of 126 model‑environment pairs, with an average relative uplift of 88.5%; the effect remains stable across model scales.

Significance: provides further evidence for the "code‑as‑harness" thesis: a large proportion of agent failures are interface issues that can be repaired at runtime, shifting the lever from the model to the execution layer.

6. Frontier Efficiency: Choose Context Strategies by Reuse Rate

The paper models context‑strategy selection as a deployment‑aware optimisation problem that jointly considers task performance, token cost, and reuse rate.

Reuse‑aware cost model: a parameterised log‑utility metric captures diminishing returns of additional context while amortising preprocessing cost; adjusting the reuse parameter enables fair comparison of strategies under different deployment modes.

Clear operational boundary: analysis reveals a sharp conversion boundary between retrieval‑based and preprocessing‑based methods; which side is superior flips with the number of reuses, so a universal default is rarely optimal.

Real token savings: on 5,000 HotpotQA instances, the deployment‑aware optimisation reduces effective tokens by ~25% at comparable performance; after amortisation, memory compression cuts token cost by over 50% compared with full‑context inference.

Significance: most teams set a context strategy once and pay per request; treating context management as an explicit cost‑performance optimisation turns guesswork into a measurable decision, yielding double‑digit percentage savings in common workloads.

7. Predicting Scientific Progress with AI – How Reliable Is It?

The CUSP benchmark introduces 4,760 real scientific events across multiple disciplines, each aligned to a verifiable knowledge cutoff, and asks models to perform feasibility assessment, mechanistic reasoning, generative design, and time prediction.

Recognition ≠ foresight: models can identify plausible directions but cannot reliably predict which will materialise, systematically mis‑estimating occurrence times.

Domain variance, time hardest: performance varies widely across fields; AI‑related progress time is more predictable than biology, chemistry, or physics, and time prediction is the weakest task overall.

Not just training cutoff: whether an event falls before or after the model’s training cutoff has little impact; extra pre‑cutoff knowledge helps but does not close the gap, and for high‑citation breakthroughs the gap widens.

Significance: models exhibit systematic over‑confidence and answer bias, making uncertainty estimates untrustworthy; CUSP offers a controlled way to measure where models are useful (direction identification) and where they fail (result prediction).

8. Your Agent Also Ages

AgingBench is a longitudinal reliability benchmark for agent lifecycles, arguing that long‑lived agents are still evaluated as if they were freshly initialised.

Compression aging: summarisation during writing discards details needed later.

Interference aging: accumulated similar memories crowd out target facts.

Revision aging: changed or derived states are not correctly updated.

Maintenance aging: routine lifecycle events cause wear.

The benchmark encodes cross‑session structure with a time‑dependent DAG, producing an "aging curve" over the entire runtime rather than a single‑point score, and highlights where repairs should focus.

9. Harness Complexity Is Not Always Better

The paper splits LLM‑agent harnesses into task decomposition and guided execution, and studies how increasing harness granularity affects performance.

Task decomposition + guided execution: breaking a task into sub‑goals and reshaping local action distributions during execution.

Findings: finer‑grained harnesses can improve the execution process but may lower overall task success due to over‑decomposition, over‑pruning, or hallucinated execution. Surprisingly, a "partial harness" that specifies only the initial steps and leaves the remainder to the agent can outperform a fully structured workflow.

10. Epicure: Embedding Food in Multilingual Space

Epicure trains multilingual food embeddings from 4.14 M recipes (11 sources, 7 languages), normalising ingredient strings to 1,790 canonical entries via an LLM‑enhanced pipeline.

Three skip‑gram variants: Metapath2Vec walks over (i) recipe co‑occurrence only, (ii) FlavorDB chemical structure only, or (iii) a mix of both.

Result: a compact, downloadable "food emergence geometry map" that positions embeddings along a spectrum from chemical‑structure to culinary‑context, demonstrating that representation learning generalises far beyond text to everyday domains.

If I had to pick one paper to read tonight, SkillOpt would be my top choice because it turns SKILL.md from an "experience craft" into a measurable optimisation, offering an immediately applicable methodological advance for existing agents.