Is the Daily Emergence of Large Language Models Beneficial?

Opening Remarks

Every day a new large language model (LLM) appears; is this phenomenon good?

First Phenomenon

Since the open‑source releases of LLaMA and ChatGLM and the accumulation of diverse “Self‑Instruct” datasets, the two key ingredients for LLMs—base models and instruction data—have become abundant, leading to an exponential increase in new models.

Open‑sourcing LLMs certainly enriches the ecosystem, but the sheer volume raises two contrasting views.

Meaningful Aspects

Anyone can experiment with a model, gain hands‑on experience, and test low‑requirement vertical applications.

Less Meaningful Aspects

Re‑packaging a modest base model (e.g., LLaMA‑7B or ChatGLM‑6B) with publicly scraped instruction data, merely renaming it, adds little value unless it offers distinct advantages.

To make open‑source efforts more worthwhile, the author suggests:

Scale up base models (e.g., LLaMA‑30B or 65B) and combine them with the most comprehensive instruction data while reducing inference resource demands.

Enhance Chinese capability by further pre‑training LLaMA‑type models on high‑quality Chinese data, then fine‑tune with full instruction sets.

Adapt open‑source models to specific domains by integrating domain‑specific data, creating specialized open‑source LLMs.

Explore novel technical improvements beyond the current LLaMA + instruction pipeline to inspire the LLM community.

Second Phenomenon

A pressing need exists for a comprehensive, authoritative Chinese LLM evaluation suite; without it, many new models claim superiority without a common benchmark, drowning truly strong models in noise.

Building such benchmarks involves challenges: selecting evaluation dimensions, designing metrics, sourcing data that is not part of pre‑training corpora, and deciding whether to disclose test examples.

Ideally, two test sets should be provided: one assessing base‑model capabilities and another measuring performance after instruction fine‑tuning, ensuring both foundational strength and downstream usefulness are recognized.

Overall, the vibrant but chaotic proliferation of LLMs is a natural phase for rapid technological catch‑up, provided that robust evaluation standards are established.

Conclusion

Thank you for reading.