I have been talking about these questions on X for a while. Between my own changing outlook and the conversations I have had with the community, it feels like time to gather some of those thoughts into a blog post. So, here goes.
I spent a long time working on embodied AI, then moved into world models and video generation. Now I am working on language models. It has been a considerable change of direction, and I understand why someone might hesitate to make it. Even when you feel that something needs to change, leaving familiar ground takes a particular kind of restlessness.
For me, that restlessness comes from a persistent ambition: I want to discover architectures that bring about more capable intelligence, and I want to help make that happen. Following that ambition has meant crossing the boundaries between fields.
This essay will inevitably leave out many of the thoughts that brought me here. I will try to follow the thread as honestly as I can.
A Question I Carried Through Beijing
Wudaokou. Zhongguancun. Darongcheng. Blue Frog, Huanshi, Luckin Coffee. Beneath the scattered shadows of autumn leaves in Beijing, and on a bench in Ditan Park in the clear light of a winter afternoon, I returned to the same question again and again: how had our field come to feel this way?
From 2024 onward, I became increasingly uneasy with how readily techniques from large language models were being carried over into embodied AI, sometimes with little reconsideration of the problems specific to robotics. Borrowing across fields is valuable. What troubled me was how often borrowing seemed to become the research agenda itself.
I would hear experienced roboticists describe robotics as essentially engineering, with little room left for fundamental research. At first, that frustrated me. I felt that a field with so much history, and such an extraordinary promise, ought to offer more space for original ideas. Over time, though, I began to understand the observation behind those words, even if I would still phrase it differently.
Where Architecture Meets Intelligence
Computer vision helped bring us ideas such as AlexNet and residual learning, ideas that reshaped far more than vision. I have often wondered how much of that history was contingent. Perhaps some of those architectural ideas could have emerged first in language modeling, under slightly different circumstances.
That thought led me to a broader question. If language is a medium through which knowledge is organized, while vision is one of the modalities through which we encounter the world, might language modeling offer a more direct setting for studying the architectures underlying general intelligence?
This is increasingly how I have come to understand the pull of language models. My interest is in the possibility that architectural progress there can change how knowledge is represented, combined, and used, with consequences that reach across modalities.
I still care deeply about visual understanding and embodied intelligence. Those are the fields that shaped me. But I began to feel that the question I most wanted to pursue was moving upstream: toward the architectures that shape the capabilities on which so much downstream work depends.
Learning to Apply My Criticism to My Own Work
I am proud of the original ideas I explored in vision. Some felt like contributions I was particularly well placed to make. Yet beyond the satisfaction of making them, the positive feedback was often limited. That gap was difficult to live with.
For a while, I explained it mainly through the incentives around me. I watched relatively modest adaptations attract considerable attention when they came from prominent groups or researchers with strong reputations. It was easy to feel that visibility and influence mattered more than the substance of an idea.
Eventually, I began to ask whether there was a more structural explanation. If much of a modality’s progress increasingly depends on capabilities developed in more general models, then some architectural changes within that modality may have less lasting importance than their presentation suggests. I found myself wondering how much of the work I was seeing, including my own, would find its way into systems people actually use.
Research is especially difficult to judge when neither theory nor practical deployment offers a clear way to test its claims. In that space, intuition, reputation, and a convincing story can carry substantial weight. Even work that uses generative models to produce attention maps, for example, left me asking what explanatory insight the maps actually established, and how that insight could be tested. I wanted clearer grounds for deciding which ideas mattered.
The uncomfortable part was recognizing that the same standard applied to me. Some of the work I valued most was also difficult to translate into practical systems. I could not reasonably ask others to demonstrate lasting value while treating originality alone as sufficient evidence for my own contributions.
That realization was sobering. It also helped me let go of some of my resentment. I began to think more carefully about where I could do work that would meet the standards I cared about.
As foundation models improve, the scope for separate contributions can shift. I can imagine visual generation becoming increasingly focused on rendering, and embodied systems increasingly focused on the harness around a capable model: the interfaces and engineering that let it act in the world. Further progress may absorb parts of those layers as well.
Rendering and harnesses are substantial technical challenges. What concerns me is the possibility of research agendas continuing to revolve around distinctions that no longer make a meaningful difference in practice. If improving models force us to reconsider those distinctions, I would welcome it. A field should be able to outgrow its familiar boundaries, and researchers, myself included, should be able to change direction with it.
Toward a Wider Sky
When I began working on Next Transformer, I felt an unexpected sense of relief. Outside the corner of the field where I had felt so hemmed in, the sky was clear.
I found myself drawn toward questions whose answers I could try to build, train, and evaluate in working systems. That is the kind of feedback I had been looking for: a closer connection between an architectural idea, its engineering consequences, and the capabilities it makes possible.
I want to pursue my ambition in a setting where that connection is central. I want to work on ideas that can become part of real systems, and use those systems to learn something about the nature of intelligence. I had spent too much energy worrying about whose account of progress would prevail. It was time to put more of that energy into building.
That is why I am making this change, and why it feels worth leaving familiar ground behind.
I hope these thoughts will carry me toward a wider sky.
