最近,我对科研这件事的理解,正在发生一次很大的变化。
一个月前,我还是一个坚定的 “review 派 researcher”。
我一直觉得,科研是一件非常严谨的事情。既然严谨,那么 AI 写出来的每一行代码,最好都要经过人工 review。至少,你得保证代码逻辑是自己过了一遍脑子的,是掌控在自己手里的。只有这样心里才踏实。
但在我亲手 build 了一个 20w+ 代码量的项目之后(详见我的上一篇博客),这个想法彻底冲击了。
我愈发强烈感觉,Agent 对软件工程的冲击,已经开始传导到科研界。
对于 AI 方向的大部分 idea 来说,只要你合理地利用好 agent,代码实现和实验推进的周期,完全可以从几个月压缩到一个星期。
以前,我们写代码,很多时候是为了把 idea 跑起来。代码只是 idea 的附属品。它能跑就行,至于结构是不是清晰、模块是不是解耦、后续好不好扩展、能不能复用,往往都不是第一优先级。于是最后写出来的东西,常常是一个很典型的 toy code:短期内能工作,长期看很脆弱。你想往里面再融一个新算法,或者加一个新模块,改起来会非常痛苦。
但在 agent 时代,我认为们需要用一种 “做产品” 的思维,去写 idea 的代码。
在过去,很多研究代码的目标,是 “能用”。而现在,一个更有价值的目标应该是:能持续地产生实验。
也就是说,你搭建的东西,最好不是一次性的。它应该支持你很方便地启动实验、管理实验、查看实验、归档实验、复现实验。你最好有统一的配置管理,有成体系的数据流转,有清晰的目录结构,有能审计的日志和结果记录。甚至再往前走一步,你可能还需要图形化界面,需要数据库,需要任务调度,需要性能优化。
这些东西,在过去常常会被很多 researcher 下意识地归类为:“这不是重点”“这是工程活”“这是基建,不高级”。
但问题在于,当你的研究开始变复杂,当实验数量开始变多,当你真的要做大量消融、大表对比、版本回溯、失败案例分析时,你就会发现:这些所谓“不高级”的东西,恰恰决定了你的科研效率上限。
一个好的实验平台,是后续所有实验的基石。
平台一旦搭好,后面的很多事情都会变得异常顺滑。你会发现,原本很麻烦的消融实验,变成了改几个配置就能跑;原本很痛苦的大表实验,变成了批量调度和统一汇总;原本做完就散落在各个文件夹里的实验结果,也开始变得可归档、可审计、可流转。
更重要的是,一旦你的数据归档做得足够好,实验过程中产生的这些中间结果、日志、偏好、失败轨迹,甚至都可以进一步沉淀成训练数据,继续反哺后续系统。到了这个时候,实验就不再只是实验本身了,它开始变成一个真正意义上的数据飞轮。
这件事以前不是没人意识到,而是大多数 researcher 即使意识到了,也很难真正去做。因为在过去,如果你没有比较深的软件工程背景,要一个人把这些基建都补起来,其实是非常困难的。你不只是要想明白系统怎么设计,还要自己把一大堆繁琐但必要的实现细节一寸一寸啃下来。这件事既慢,又累,而且短期内看起来并不 “出活”。所以很多人最终都会畏难而放弃,告诉自己:算了,先把 idea 验证了再说。于是大家一边嫌 toy code 难维护,一边继续制造新的 toy code。
这可能是我最近感受最深的一件事。今天作为 researcher,我们真正需要亲自牢牢把握住的,不仅是核心算法的每一行实现细节,而且还应该有更上层的东西:系统应该怎么拆,模块边界在哪里,数据流应该怎么走,实验平台应该遵循什么设计哲学,哪些抽象是值得长期维护的,哪些地方未来一定会成为瓶颈。换句话说,researcher 的重心,正在从“亲手写完所有代码”,逐渐转向“设计好系统,并驾驭 agent 完成实现”。
在这种范式下,一个 idea 的落地周期会被大幅压缩。原来你可能要花一个月,才能写出一套勉强能跑的 toy code;现在,你完全有可能在一个星期内,围绕这个 idea 搭出一个完整的实验平台雏形。到那个时候,被压缩的不只是写代码的时间,而是整个科研迭代的周期。一个想法从提出,到实现,到跑通,到扩展,到做系统性验证,这条链路会整体变短。科研的节奏会变得更快,试错成本会变得更低,AI 的进步速度也会被进一步推高。
某种意义上,这甚至会反过来加速 AI for AI 本身。
By the way, 我觉得 AI for AI 是一个很有意思的研究方向。
Recently, my understanding of research has been going through a pretty major shift.
One month ago, I was still firmly a “review-first” researcher.
I always believed that research was something extremely rigorous. And if it is rigorous, then every line of code written by AI should ideally be reviewed by a human. At the very least, you need to make sure the logic has gone through your own mind, and that the code is still under your control. Only then do you feel safe.
But after personally building a 200k+ codebase with Agents, which I wrote about in my previous blog post, that belief was shaken pretty hard.
I increasingly feel that the impact of Agents on software engineering has already started propagating into research.
For most ideas in AI, if you use Agents properly, the time required to implement the code and push the experiments forward can be compressed from several months into one week.
In the past, when we wrote code, the purpose was often just to get an idea running. Code was only an accessory to the idea. As long as it ran, that was enough. Whether the structure was clean, whether the modules were decoupled, whether it was easy to extend later, whether it could be reused, none of these were usually the first priority. So what we ended up with was often very typical toy code: it works in the short term, but is fragile in the long term. If you later want to merge in a new algorithm or add a new module, the whole thing becomes painful to modify.
But in the Agent era, I think we need to write code for ideas with a product-building mindset.
In the past, the goal of much research code was simply to be “usable.” Now, I think a more valuable goal should be: can it continuously generate experiments?
In other words, what you build should ideally not be disposable. It should make it easy to launch experiments, manage experiments, inspect experiments, archive experiments, and reproduce experiments. You should probably have unified configuration management, a systematic data flow, a clear directory structure, and auditable logs and result records. If you go one step further, you may even need a graphical interface, a database, task scheduling, and performance optimization.
In the past, many researchers would instinctively categorize these things as: “not the point,” “just engineering work,” or “infrastructure, not something intellectually interesting.”
But the problem is that once your research becomes more complex, once the number of experiments starts growing, once you really need to run large numbers of ablations, big comparison tables, version rollbacks, and failure case analyses, you will realize that these supposedly “uninteresting” things are exactly what determine the upper bound of your research efficiency.
A good experiment platform is the foundation for all experiments that follow.
Once the platform is in place, many things afterward become unusually smooth. Ablations that used to be annoying become a matter of changing a few configuration values. Big table experiments that used to be painful become batch scheduling and unified aggregation. Results that used to scatter across random folders after each run become archivable, auditable, and transferable.
More importantly, once your data archiving is good enough, the intermediate outputs, logs, preferences, and failure trajectories produced during experiments can themselves be accumulated as training data, and then fed back into future systems. At that point, an experiment is no longer just an experiment. It starts becoming a real data flywheel.
It is not that nobody realized this before. The issue is that even when most researchers did realize it, it was still very hard to actually do. In the past, if you did not have a fairly deep software engineering background, filling in all this infrastructure by yourself was extremely difficult. You did not only need to think clearly about the system design; you also had to grind through a huge number of tedious but necessary implementation details inch by inch. It was slow, exhausting, and in the short term it did not look like “real progress.” So many people eventually backed away and told themselves: forget it, let’s validate the idea first. And then everyone kept complaining that toy code was hard to maintain, while continuing to produce more toy code.
This may be the thing I have felt most strongly recently. Today, as researchers, what we truly need to keep firmly in our own hands is not only every implementation detail of the core algorithm, but also something at a higher level: how the system should be decomposed, where the module boundaries should be, how the data should flow, what design philosophy the experiment platform should follow, which abstractions are worth maintaining long term, and which parts will definitely become bottlenecks in the future. In other words, the center of gravity for researchers is shifting from “writing all the code by hand” toward “designing the system well and driving Agents to complete the implementation.”
Under this paradigm, the landing cycle of an idea will be dramatically shortened. In the past, it might have taken you a month to write a barely runnable piece of toy code. Now, it is entirely possible to build the prototype of a full experiment platform around an idea within a week. At that point, what gets compressed is not only the time spent writing code, but the entire research iteration cycle. The chain from proposing an idea, to implementing it, to getting it running, to extending it, to doing systematic validation will become shorter as a whole. The rhythm of research will become faster, the cost of trial and error will become lower, and the speed of AI progress will be pushed even higher.
In a sense, this may even accelerate AI for AI itself.
By the way, I think AI for AI is a very interesting research direction.